8:39 p.m.
I'm having a problem with the Fedora 7 kernel 2.6.21-1.3228.fc7 for
x86_64. So I am trying to use crash-4.0-4.2 to analyze the problem.
(The version of crash in F7 won't work with this kernel.)
Since the kernel bug appears to be in irq handline, learning more
about IRQ state is important.
When I use the irq command I get this error message:
irq: cannot determine size of irq_desc_t
Is this a bug or a known limitation of crash?
8:51 a.m.
D. Hugh Redelmeier wrote:
I'm having a problem with the Fedora 7 kernel 2.6.21-1.3228.fc7
for
x86_64. So I am trying to use crash-4.0-4.2 to analyze the problem.
(The version of crash in F7 won't work with this kernel.)
Since the kernel bug appears to be in irq handline, learning more
about IRQ state is important.
When I use the irq command I get this error message:
irq: cannot determine size of irq_desc_t
Is this a bug or a known limitation of crash?
Could be shifting sands syndrome, which happens all the time.
Perhaps "irq_desc_t" has been removed from that kernel?
I've got a slightly older 2.6.21-1.3125.fc7 source tree
hanging around that shows this:
/*
* Migration helpers for obsolete names, they will go away:
*/
#define hw_interrupt_type irq_chip
typedef struct irq_chip hw_irq_controller;
#define no_irq_type no_irq_chip
typedef struct irq_desc irq_desc_t;
Interesting, the "go away" comment -- perhaps it's no longer
there in -1.3228?
Anyway presuming that the irq_desc structure is still in place,
you could try rebuilding crash by changing all instances of
the string arguments below that use "irq_desc_t" to "irq_desc":
Text string: "irq_desc_t"
File Line
0 kernel.c 281 MEMBER_OFFSET_INIT(irq_desc_t_status, "irq_desc_t",
"status");
1 kernel.c 282 if (MEMBER_EXISTS("irq_desc_t", "handler"))
2 kernel.c 283 MEMBER_OFFSET_INIT(irq_desc_t_handler, "irq_desc_t",
"handler");
3 kernel.c 285 MEMBER_OFFSET_INIT(irq_desc_t_chip, "irq_desc_t",
"chip");
4 kernel.c 286 MEMBER_OFFSET_INIT(irq_desc_t_action, "irq_desc_t",
"action");
5 kernel.c 287 MEMBER_OFFSET_INIT(irq_desc_t_depth, "irq_desc_t",
"depth");
6 kernel.c 349 STRUCT_SIZE_INIT(irq_desc_t, "irq_desc_t");
7 ppc.c 111 STRUCT_SIZE_INIT(irq_desc_t, "irq_desc_t");
8 ppc64.c 197 STRUCT_SIZE_INIT(irq_desc_t, "irq_desc_t");
For example, the 0'th line above would be:
MEMBER_OFFSET_INIT(irq_desc_t_status, "irq_desc", "status");
You can test whether the structure sizes and structure member
offsets have been successfully determined by entering
"help -o", which gives the full dump of all member offsets,
structure sizes and array sizes, or in this case, add a string
qualifier argument to pare it down:
crash> help -o irq_desc
offset_table:
irq_desc_t_status: 40
irq_desc_t_handler: -1
irq_desc_t_chip: 8
irq_desc_t_action: 32
irq_desc_t_depth: 44
size_table:
irq_desc_t: 256
array_table:
irq_desc: 239
crash>
In the kernel above, the old "handler" member has been
renamed to "chip" in later kernels. In your kernel, I would
presume that you've got -1's in all but the irq_desc array_table
entry.
Dave
1:18 p.m.
| From: Dave Anderson <anderson(a)redhat.com>
| D. Hugh Redelmeier wrote:
| > I'm having a problem with the Fedora 7 kernel 2.6.21-1.3228.fc7 for
| > x86_64. So I am trying to use crash-4.0-4.2 to analyze the problem.
| > (The version of crash in F7 won't work with this kernel.)
| > When I use the irq command I get this error message:
| > irq: cannot determine size of irq_desc_t
|
| Could be shifting sands syndrome, which happens all the time.
|
| Perhaps "irq_desc_t" has been removed from that kernel?
| I've got a slightly older 2.6.21-1.3125.fc7 source tree
| hanging around that shows this:
|
| /*
| * Migration helpers for obsolete names, they will go away:
| */
| #define hw_interrupt_type irq_chip
| typedef struct irq_chip hw_irq_controller;
| #define no_irq_type no_irq_chip
| typedef struct irq_desc irq_desc_t;
|
| Interesting, the "go away" comment -- perhaps it's no longer
| there in -1.3228?
I'm playing with linux-2.6.22-rc7 now. It has the same problem with
the crash irq command.
Those lines are still in
linux-2.6.22-rc7/include/linux/irq.h
so I don't know why this problem has come up.
In any case, I have a patch that seems to fix the problem and that
should be backward compatible. It is based on the suggestions you
outlined. I'm starting with the crash-4.0-4.3.
===================================================================
RCS file: RCS/kernel.c,v
retrieving revision 1.1
diff -u -r1.1 kernel.c
--- kernel.c 2007/07/07 17:23:06 1.1
+++ kernel.c 2007/07/07 17:56:59
@@ -49,7 +49,7 @@
static void BUG_bytes_init(void);
static int BUG_x86(void);
static int BUG_x86_64(void);
-
+static /*const*/ char *irq_desc_type_name;
/*
@@ -278,13 +278,19 @@
STRUCT_SIZE_INIT(hlist_head, "hlist_head");
STRUCT_SIZE_INIT(hlist_node, "hlist_node");
- MEMBER_OFFSET_INIT(irq_desc_t_status, "irq_desc_t", "status");
- if (MEMBER_EXISTS("irq_desc_t", "handler"))
- MEMBER_OFFSET_INIT(irq_desc_t_handler, "irq_desc_t", "handler");
+ /* see linux-* /include/linux/irq.h to find how IRQ information is represented by the
kernel */
+ if (STRUCT_EXISTS("irq_desc_t"))
+ irq_desc_type_name = "irq_desc_t";
+ else
+ irq_desc_type_name = "irq_desc";
+
+ MEMBER_OFFSET_INIT(irq_desc_t_status, irq_desc_type_name, "status");
+ if (MEMBER_EXISTS(irq_desc_type_name, "handler"))
+ MEMBER_OFFSET_INIT(irq_desc_t_handler, irq_desc_type_name, "handler");
else
- MEMBER_OFFSET_INIT(irq_desc_t_chip, "irq_desc_t", "chip");
- MEMBER_OFFSET_INIT(irq_desc_t_action, "irq_desc_t", "action");
- MEMBER_OFFSET_INIT(irq_desc_t_depth, "irq_desc_t", "depth");
+ MEMBER_OFFSET_INIT(irq_desc_t_chip, irq_desc_type_name, "chip");
+ MEMBER_OFFSET_INIT(irq_desc_t_action, irq_desc_type_name, "action");
+ MEMBER_OFFSET_INIT(irq_desc_t_depth, irq_desc_type_name, "depth");
if (STRUCT_EXISTS("hw_interrupt_type")) {
MEMBER_OFFSET_INIT(hw_interrupt_type_typename,
"hw_interrupt_type", "typename");
@@ -346,7 +352,7 @@
MEMBER_OFFSET_INIT(irqaction_dev_id, "irqaction", "dev_id");
MEMBER_OFFSET_INIT(irqaction_next, "irqaction", "next");
- STRUCT_SIZE_INIT(irq_desc_t, "irq_desc_t");
+ STRUCT_SIZE_INIT(irq_desc_t, irq_desc_type_name);
STRUCT_SIZE_INIT(irq_cpustat_t, "irq_cpustat_t");
MEMBER_OFFSET_INIT(irq_cpustat_t___softirq_active,
================ end of patch ================
1:04 p.m.
Now that the irq command appears to work, I notice that it is too
verbose.
On my machine (AMD Athlon x2), it unconditionally prints 2304 structs,
each taking 27 lines. Of those, only 23 appear to have been used in
the day that the machine had been up.
Is there a simple way to print only the used entries or perhaps only
print an entry if it is different from the one before it?
Does that kind of logic exist for printing other tables? If not,
would it be useful?
I wonder why the kernel allocates so many irq_desc entries. According
to "nm -f sysv vmlinux", irq_desc is 294912 bytes -- more than the
whole RAM on the first UNIX machine I used.
I was looking at x86_64.c: x86_64_dump_irq. Its last line says:
error(FATAL, "ia64_dump_irq: irq_desc[] does not exist?\n");
It should say:
error(FATAL, "x86_64_dump_irq: irq_desc[] does not exist?\n");
9:41 a.m.
D. Hugh Redelmeier wrote:
Now that the irq command appears to work, I notice that it is too
verbose.
On my machine (AMD Athlon x2), it unconditionally prints 2304 structs,
each taking 27 lines. Of those, only 23 appear to have been used in
the day that the machine had been up.
Is there a simple way to print only the used entries or perhaps only
print an entry if it is different from the one before it?
Does that kind of logic exist for printing other tables? If not,
would it be useful?
Wow -- until fairly recently the NR_IRQS definition has been a
relatively small number, and on most architectures it still is.
I don't think there are any other commands that print such
verbose output for a table containing that many structures
where most of them are unused.
I suppose if it's obvious that an IRQ index is not in use, it's output
could be shortened, or maybe a new option could be added that only
prints out "used" entries. I await your patch suggestion...
I wonder why the kernel allocates so many irq_desc entries. According
to "nm -f sysv vmlinux", irq_desc is 294912 bytes -- more than the
whole RAM on the first UNIX machine I used.
Amazing isn't it?
I was looking at x86_64.c: x86_64_dump_irq. Its last line says:
error(FATAL, "ia64_dump_irq: irq_desc[] does not exist?\n");
It should say:
error(FATAL, "x86_64_dump_irq: irq_desc[] does not exist?\n");
Thanks -- I'll fix that...
Dave
4:14 p.m.
| From: D. Hugh Redelmeier <hugh(a)mimosa.com>
| On my machine (AMD Athlon x2), it unconditionally prints 2304 structs,
| each taking 27 lines. Of those, only 23 appear to have been used in
| the day that the machine had been up.
| I wonder why the kernel allocates so many irq_desc entries. According
| to "nm -f sysv vmlinux", irq_desc is 294912 bytes -- more than the
| whole RAM on the first UNIX machine I used.
include/asm-x86_64/irq.h defines NR_IRQS:
#define NR_IRQS (NR_VECTORS + (32 *NR_CPUS))
NR_VECTORS is 256.
So NR_CPUS must be defined to be 64 on my machine. I wish I had that
many CPUs.
NR_CPUS seems to be defined here:
include/linux/threads.h:17:#define NR_CPUS CONFIG_NR_CPUS
CONFIG_NR_CPUS is defined in .config:
CONFIG_NR_CPUS=64
There is a comment in thread.h that suggests that NR_CPUS is the
number of bits in a bitmask on the platform (32 or 64).
Perhaps the array of irqs is addressed in a way that you know which
ones are useless if you know how many CPUs there actually are (at
runtime). Or perhaps if you know how many there were at most since
processors might come and go.
4:30 p.m.
D. Hugh Redelmeier wrote:
| From: D. Hugh Redelmeier <hugh(a)mimosa.com>
| On my machine (AMD Athlon x2), it unconditionally prints 2304 structs,
| each taking 27 lines. Of those, only 23 appear to have been used in
| the day that the machine had been up.
| I wonder why the kernel allocates so many irq_desc entries. According
| to "nm -f sysv vmlinux", irq_desc is 294912 bytes -- more than the
| whole RAM on the first UNIX machine I used.
include/asm-x86_64/irq.h defines NR_IRQS:
#define NR_IRQS (NR_VECTORS + (32 *NR_CPUS))
NR_VECTORS is 256.
So NR_CPUS must be defined to be 64 on my machine. I wish I had that
many CPUs.
NR_CPUS seems to be defined here:
include/linux/threads.h:17:#define NR_CPUS CONFIG_NR_CPUS
CONFIG_NR_CPUS is defined in .config:
CONFIG_NR_CPUS=64
There is a comment in thread.h that suggests that NR_CPUS is the
number of bits in a bitmask on the platform (32 or 64).
Perhaps the array of irqs is addressed in a way that you know which
ones are useless if you know how many CPUs there actually are (at
runtime). Or perhaps if you know how many there were at most since
processors might come and go.
Do your unused IRQs show "no_irq_chip" as the handler?
3:04 a.m.
| From: Dave Anderson <anderson(a)redhat.com>
| Do your unused IRQs show "no_irq_chip" as the handler?
They do seem to.
You can look at a sample here:
https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=158724
Part of this:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=242836
9:19 a.m.
D. Hugh Redelmeier wrote:
| From: Dave Anderson <anderson(a)redhat.com>
| D. Hugh Redelmeier wrote:
| > I'm having a problem with the Fedora 7 kernel 2.6.21-1.3228.fc7 for
| > x86_64. So I am trying to use crash-4.0-4.2 to analyze the problem.
| > (The version of crash in F7 won't work with this kernel.)
| > When I use the irq command I get this error message:
| > irq: cannot determine size of irq_desc_t
|
| Could be shifting sands syndrome, which happens all the time.
|
| Perhaps "irq_desc_t" has been removed from that kernel?
| I've got a slightly older 2.6.21-1.3125.fc7 source tree
| hanging around that shows this:
|
| /*
| * Migration helpers for obsolete names, they will go away:
| */
| #define hw_interrupt_type irq_chip
| typedef struct irq_chip hw_irq_controller;
| #define no_irq_type no_irq_chip
| typedef struct irq_desc irq_desc_t;
|
| Interesting, the "go away" comment -- perhaps it's no longer
| there in -1.3228?
I'm playing with linux-2.6.22-rc7 now. It has the same problem with
the crash irq command.
Those lines are still in
linux-2.6.22-rc7/include/linux/irq.h
so I don't know why this problem has come up.
In any case, I have a patch that seems to fix the problem and that
should be backward compatible. It is based on the suggestions you
outlined. I'm starting with the crash-4.0-4.3.
===================================================================
RCS file: RCS/kernel.c,v
retrieving revision 1.1
diff -u -r1.1 kernel.c
--- kernel.c 2007/07/07 17:23:06 1.1
+++ kernel.c 2007/07/07 17:56:59
@@ -49,7 +49,7 @@
static void BUG_bytes_init(void);
static int BUG_x86(void);
static int BUG_x86_64(void);
-
+static /*const*/ char *irq_desc_type_name;
/*
@@ -278,13 +278,19 @@
STRUCT_SIZE_INIT(hlist_head, "hlist_head");
STRUCT_SIZE_INIT(hlist_node, "hlist_node");
- MEMBER_OFFSET_INIT(irq_desc_t_status, "irq_desc_t", "status");
- if (MEMBER_EXISTS("irq_desc_t", "handler"))
- MEMBER_OFFSET_INIT(irq_desc_t_handler, "irq_desc_t", "handler");
+ /* see linux-* /include/linux/irq.h to find how IRQ information is represented by the
kernel */
+ if (STRUCT_EXISTS("irq_desc_t"))
+ irq_desc_type_name = "irq_desc_t";
+ else
+ irq_desc_type_name = "irq_desc";
+
+ MEMBER_OFFSET_INIT(irq_desc_t_status, irq_desc_type_name, "status");
+ if (MEMBER_EXISTS(irq_desc_type_name, "handler"))
+ MEMBER_OFFSET_INIT(irq_desc_t_handler, irq_desc_type_name, "handler");
else
- MEMBER_OFFSET_INIT(irq_desc_t_chip, "irq_desc_t", "chip");
- MEMBER_OFFSET_INIT(irq_desc_t_action, "irq_desc_t", "action");
- MEMBER_OFFSET_INIT(irq_desc_t_depth, "irq_desc_t", "depth");
+ MEMBER_OFFSET_INIT(irq_desc_t_chip, irq_desc_type_name, "chip");
+ MEMBER_OFFSET_INIT(irq_desc_t_action, irq_desc_type_name, "action");
+ MEMBER_OFFSET_INIT(irq_desc_t_depth, irq_desc_type_name, "depth");
if (STRUCT_EXISTS("hw_interrupt_type")) {
MEMBER_OFFSET_INIT(hw_interrupt_type_typename,
"hw_interrupt_type", "typename");
@@ -346,7 +352,7 @@
MEMBER_OFFSET_INIT(irqaction_dev_id, "irqaction", "dev_id");
MEMBER_OFFSET_INIT(irqaction_next, "irqaction", "next");
- STRUCT_SIZE_INIT(irq_desc_t, "irq_desc_t");
+ STRUCT_SIZE_INIT(irq_desc_t, irq_desc_type_name);
STRUCT_SIZE_INIT(irq_cpustat_t, "irq_cpustat_t");
MEMBER_OFFSET_INIT(irq_cpustat_t___softirq_active,
================ end of patch ================
I still don't understand why it's not being picked up in the
debuginfo data. There was a problem with gcc 3.4.*, and then
subsequently a regression in gcc 4.0.0, both of which were related
to data structure information not being contained in the vmlinux
debuginfo data.
Anyway, thanks -- the patch looks good. Even with the mystery
it's worth updating it in preparation for its eventual advertised
removal from the kernel. In the Linux 2.2 days when the crash
code was originally written, and through 2.4, the kernel defined
it as:
typedef struct {
...
} irq_desc_t;
In 2.6 a "struct irq_desc" struct was introduced.
Queued for the next release.
Thanks,
Dave
6347
days inactive
6356
days old
devel@lists.crash-utility.osci.io
9 comments
2 participants
participants (2)
-
D. Hugh Redelmeier -
Dave Anderson