2:41 a.m.
Hi Dave,
I found the problem that the crash utility fails during the initialization.
This problem happened on i386 linux-2.6.22 like the following:
$ crash vmlinux vmcore
crash 4.0-4.4
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
crash: invalid size request: 0 type: "__per_cpu_offset"
$
The cause is that the array number of the symbol "__per_cpu_offset"
cannot be taken. In linux-2.6.21, i386's "__per_cpu_offset" is defined
in include/asm-generic/percpu.h like the following:
extern unsigned long __per_cpu_offset[NR_CPUS];
But in linux-2.6.22, it is defined in include/asm-i386/percpu.h like
the following, and the array number is not described in the debugging
information.
extern unsigned long __per_cpu_offset[];
Here is the patch for solving the problem. If the array number is not
taken, the crash utility assumes that it is the defined value NR_CPUS.
Or, should get_array_length() be fixed to get the array number from
init/main.c ?
Thanks
Ken'ichi Ohmichi
--- crash-4.0-4.4.org/kernel.c 2007-07-21 04:19:23.000000000 +0900
+++ crash-4.0-4.4/kernel.c 2007-07-24 19:51:55.000000000 +0900
@@ -170,7 +170,7 @@ kernel_init()
else
i = get_array_length("__per_cpu_offset", NULL, 0);
get_symbol_data("__per_cpu_offset",
- sizeof(long)*(i <= NR_CPUS ? i : NR_CPUS),
+ sizeof(long)*((i && (i <= NR_CPUS)) ? i : NR_CPUS),
&kt->__per_cpu_offset[0]);
kt->flags |= PER_CPU_OFF;
}
_
9:28 a.m.
Ken'ichi Ohmichi wrote:
Hi Dave,
I found the problem that the crash utility fails during the initialization.
This problem happened on i386 linux-2.6.22 like the following:
$ crash vmlinux vmcore
crash 4.0-4.4
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
crash: invalid size request: 0 type: "__per_cpu_offset"
$
The cause is that the array number of the symbol "__per_cpu_offset"
cannot be taken. In linux-2.6.21, i386's "__per_cpu_offset" is defined
in include/asm-generic/percpu.h like the following:
extern unsigned long __per_cpu_offset[NR_CPUS];
But in linux-2.6.22, it is defined in include/asm-i386/percpu.h like
the following, and the array number is not described in the debugging
information.
extern unsigned long __per_cpu_offset[];
Here is the patch for solving the problem. If the array number is not
taken, the crash utility assumes that it is the defined value NR_CPUS.
Or, should get_array_length() be fixed to get the array number from
init/main.c ?
Ken'ichi -- thanks for tracking that down.
I don't see how get_array_length() can be "fixed" in this case -- if the
vmlinux file doesn't have the info, it doesn't have it. I'm not sure
what you mean by getting it from init/main.c?
Anyway, using NR_CPUS is probably safe, although it runs the remote
risk of reading into a non-existent section of memory or uncopied
section of dumpfile.
How about using Cliff Wickman's new get_cpus_possible() function
from his LKCD_KERNTYPES patch? Since get_cpus_possible() returns 0
on failure, your fix below should be left in place, but it might be
worth also trying get_cpus_possible() if get_array_length() returns 0?
Dave
Thanks
Ken'ichi Ohmichi
--- crash-4.0-4.4.org/kernel.c 2007-07-21 04:19:23.000000000 +0900
+++ crash-4.0-4.4/kernel.c 2007-07-24 19:51:55.000000000 +0900
@@ -170,7 +170,7 @@ kernel_init()
else
i = get_array_length("__per_cpu_offset", NULL, 0);
get_symbol_data("__per_cpu_offset",
- sizeof(long)*(i <= NR_CPUS ? i : NR_CPUS),
+ sizeof(long)*((i && (i <= NR_CPUS)) ? i : NR_CPUS),
&kt->__per_cpu_offset[0]);
kt->flags |= PER_CPU_OFF;
}
_
--
Crash-utility mailing list
Crash-utility(a)redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
4:02 a.m.
Hi Dave,
2007/07/25 10:28:04 -0400, Dave Anderson <anderson(a)redhat.com> wrote:
> Here is the patch for solving the problem. If the array number is
not
> taken, the crash utility assumes that it is the defined value NR_CPUS.
> Or, should get_array_length() be fixed to get the array number from
> init/main.c ?
>
Ken'ichi -- thanks for tracking that down.
I don't see how get_array_length() can be "fixed" in this case -- if the
vmlinux file doesn't have the info, it doesn't have it. I'm not sure
what you mean by getting it from init/main.c?
Sorry for my insufficient explanation.
On linux-2.6.22, the external declaration of i386's __per_cpu_offset[]
is in include/asm-i386/percpu.h, and the definition of __per_cpu_offset[]
is in init/main.c:
unsigned long __per_cpu_offset[NR_CPUS] __read_mostly;
There are their debugging information in the dwarf section, and the
definition's one includes the array length.
I explain by following output of the dwarfdump command.
* The debugging information of "extern unsigned long __per_cpu_offset[];"
There is not the attribute DW_AT_upper_bound and the crash utility cannot
get the array number.
<1><32685> DW_TAG_variable
DW_AT_name __per_cpu_offset
DW_AT_decl_file 83 include/asm/percpu.h
DW_AT_decl_line 48
DW_AT_type <32674>
DW_AT_external yes(1)
DW_AT_declaration yes(1)
<1><32674> DW_TAG_array_type
DW_AT_sibling <32685>
DW_AT_type <83>
<2><32683> DW_TAG_subrange_type
* The debugging information of "unsigned long __per_cpu_offset[NR_CPUS]
__read_mostly;"
There is the attribute DW_AT_upper_bound and the crash utility can do it.
<1><58437> DW_TAG_variable
DW_AT_name __per_cpu_offset
DW_AT_decl_file 1 init/main.c
DW_AT_decl_line 363
DW_AT_type <33>
DW_AT_external yes(1)
DW_AT_location DW_OP_addr 0xc03fd400
<1>< 33> DW_TAG_array_type
DW_AT_sibling <49>
DW_AT_type <56>
<2>< 42> DW_TAG_subrange_type
DW_AT_type <49>
DW_AT_upper_bound 31 <--- Here!
I thought get_array_length() should retry to get the array length from
the next if it could not get it. But I prefer your following idea because
it is simple and does not make other effects.
How about using Cliff Wickman's new get_cpus_possible() function
from his LKCD_KERNTYPES patch? Since get_cpus_possible() returns 0
on failure, your fix below should be left in place, but it might be
worth also trying get_cpus_possible() if get_array_length() returns 0?
That sounds good, but I cannot find Cliff Wickman's patch (including
get_cpus_possible() function). Can I see his patch ?
Thanks
Ken'ichi Ohmichi
8:15 a.m.
Ken'ichi Ohmichi wrote:
Hi Dave,
2007/07/25 10:28:04 -0400, Dave Anderson <anderson(a)redhat.com> wrote:
>>Here is the patch for solving the problem. If the array number is not
>>taken, the crash utility assumes that it is the defined value NR_CPUS.
>>Or, should get_array_length() be fixed to get the array number from
>>init/main.c ?
>>
>
>Ken'ichi -- thanks for tracking that down.
>
>I don't see how get_array_length() can be "fixed" in this case -- if
the
>vmlinux file doesn't have the info, it doesn't have it. I'm not sure
>what you mean by getting it from init/main.c?
Sorry for my insufficient explanation.
On linux-2.6.22, the external declaration of i386's __per_cpu_offset[]
is in include/asm-i386/percpu.h, and the definition of __per_cpu_offset[]
is in init/main.c:
unsigned long __per_cpu_offset[NR_CPUS] __read_mostly;
There are their debugging information in the dwarf section, and the
definition's one includes the array length.
I explain by following output of the dwarfdump command.
* The debugging information of "extern unsigned long __per_cpu_offset[];"
There is not the attribute DW_AT_upper_bound and the crash utility cannot
get the array number.
<1><32685> DW_TAG_variable
DW_AT_name __per_cpu_offset
DW_AT_decl_file 83 include/asm/percpu.h
DW_AT_decl_line 48
DW_AT_type <32674>
DW_AT_external yes(1)
DW_AT_declaration yes(1)
<1><32674> DW_TAG_array_type
DW_AT_sibling <32685>
DW_AT_type <83>
<2><32683> DW_TAG_subrange_type
* The debugging information of "unsigned long __per_cpu_offset[NR_CPUS]
__read_mostly;"
There is the attribute DW_AT_upper_bound and the crash utility can do it.
<1><58437> DW_TAG_variable
DW_AT_name __per_cpu_offset
DW_AT_decl_file 1 init/main.c
DW_AT_decl_line 363
DW_AT_type <33>
DW_AT_external yes(1)
DW_AT_location DW_OP_addr 0xc03fd400
<1>< 33> DW_TAG_array_type
DW_AT_sibling <49>
DW_AT_type <56>
<2>< 42> DW_TAG_subrange_type
DW_AT_type <49>
DW_AT_upper_bound 31 <--- Here!
I thought get_array_length() should retry to get the array length from
the next if it could not get it. But I prefer your following idea because
it is simple and does not make other effects.
I see. But as it stands now, the crash utility has no dwarfdump-like
functions to pull out such data, but rather relies on gdb to do the
dirty work. There's probably something in the gdb innards that could
access it, but I'm not aware of it.
But it's not that important...
>How about using Cliff Wickman's new get_cpus_possible() function
>from his LKCD_KERNTYPES patch? Since get_cpus_possible() returns 0
>on failure, your fix below should be left in place, but it might be
>worth also trying get_cpus_possible() if get_array_length() returns 0?
That sounds good, but I cannot find Cliff Wickman's patch (including
get_cpus_possible() function). Can I see his patch ?
It's already in crash 4.0-4.4,in kernel.c. But as it turns out,
I don't think it's necessary either.
Given that:
1. this is an x86-only issue, i.e., where it's declared as
as an extern without an array index,
2. the NR_CPUS is relatively small,
3. the "__per_cpu_offset" array is (empirically) located
long before "_end",
4. the data types that follow it are going to be populated with
non-zero data,
then there's virtually no chance it will bump into a non-existent page.
I also thought of simply determining the difference between the
address of "__per_cpu_offset" and the address of the symbol that
follows it, and dividing that by the size of a long.
Anyway, I've decided that I like your original patch. If it ever
causes a problem in the future, it will be simple enough to fix.
Thanks,
Dave
10:29 a.m.
| From: Dave Anderson <anderson(a)redhat.com>
| Ken'ichi Ohmichi wrote:
| > On linux-2.6.22, the external declaration of i386's __per_cpu_offset[]
| > is in include/asm-i386/percpu.h, and the definition of __per_cpu_offset[]
| > is in init/main.c:
| >
| > unsigned long __per_cpu_offset[NR_CPUS] __read_mostly;
| >
| >
| > There are their debugging information in the dwarf section, and the
| > definition's one includes the array length.
| > I explain by following output of the dwarfdump command.
| >
| > * The debugging information of "extern unsigned long
__per_cpu_offset[];"
| > There is not the attribute DW_AT_upper_bound and the crash utility cannot
| > get the array number.
| > <1><32685> DW_TAG_variable
| > DW_AT_name __per_cpu_offset
| > DW_AT_decl_file 83 include/asm/percpu.h
| > DW_AT_decl_line 48
| > DW_AT_type <32674>
| > DW_AT_external yes(1)
| > DW_AT_declaration yes(1)
| > <1><32674> DW_TAG_array_type
| > DW_AT_sibling <32685>
| > DW_AT_type <83>
| > <2><32683> DW_TAG_subrange_type
| >
| > * The debugging information of "unsigned long __per_cpu_offset[NR_CPUS]
__read_mostly;"
| > There is the attribute DW_AT_upper_bound and the crash utility can do it.
| > <1><58437> DW_TAG_variable
| > [a1][a1][a1][a1] DW_AT_name __per_cpu_offset
| > DW_AT_decl_file 1 init/main.c
| > DW_AT_decl_line 363
| > DW_AT_type <33>
| > DW_AT_external yes(1)
| > DW_AT_location DW_OP_addr 0xc03fd400
| > <1>< 33> DW_TAG_array_type
| > DW_AT_sibling <49>
| > DW_AT_type <56>
| > <2>< 42> DW_TAG_subrange_type
| > DW_AT_type <49>
| > DW_AT_upper_bound 31 <--- Here!
| >
| >
| > I thought get_array_length() should retry to get the array length from
| > the next if it could not get it. But I prefer your following idea because
| > it is simple and does not make other effects.
|
| I see. But as it stands now, the crash utility has no dwarfdump-like
| functions to pull out such data, but rather relies on gdb to do the
| dirty work. There's probably something in the gdb innards that could
| access it, but I'm not aware of it.
|
| But it's not that important...
This makes me quite suspicious. I assume that you are correct that
the problem is in gdb. It still feels like there is a serious problem
here: that the most complete definition is not being made available.
You have an effective work-around for __per_cpu_offset, but I wonder
how many other ways this same problem will come up.
6362
days inactive
6365
days old
devel@lists.crash-utility.osci.io
4 comments
3 participants
participants (3)
-
D. Hugh Redelmeier -
Dave Anderson -
Ken'ichi Ohmichi