Applied patch and tested against 2 cores known to show the failure. Once the patch was applied, the issue was resolved. Tested-by: John Pittman <jpittman(a)redhat.com&gt;

On Mon, Mar 22, 2021 at 4:31 AM Lianbo Jiang <lijiang(a)redhat.com&gt; wrote: > > Currently, some command such as 'sys' may cause subsequent 'set scope' > commands to fail because it may not find the correct symtab associated > with PC and SECTION in the find_pc_sect_symtab(), eventually, this will > cause the following failure: > > crash> mod -S 3.10.0-957.el7.x86_64 > crash> mod -s dm_service_time > crash> set scope st_create > set: gdb cannot find text block for address: st_create > crash> mod -d dm_service_time > crash> mod -sr dm_service_time > crash> set scope st_create > scope: ffffffffc044d270 (st_create) > crash> sys > KERNEL: 3.10.0-957.el7.x86_64/vmlinux > DUMPFILE: crash/vmcore [PARTIAL DUMP] > CPUS: 48 > DATE: Sat Aug 3 11:41:00 EDT 2019 > UPTIME: 1 days, 10:28:48 > LOAD AVERAGE: 2.95, 1.02, 0.40 > TASKS: 1060 > NODENAME: iop053063.lss.emc.com > RELEASE: 3.10.0-957.el7.x86_64 > VERSION: #1 SMP Thu Oct 4 20:48:51 UTC 2018 > MACHINE: x86_64 (2999 Mhz) > MEMORY: 127.9 GB > PANIC: "SysRq : Trigger a crash" > crash> set scope st_create > set: gdb cannot find text block for address: st_create > > To find the correct symtab, let's check whether there is an address > mapping to 'block' in the symtab searching loop and the PC is in the > range. If the symtab associated with PC is found, and then use it. > > Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com&gt; > --- > gdb-7.6.patch | 38 +++++++++++++++++++++++++++++++++++++- > 1 file changed, 37 insertions(+), 1 deletion(-) > > diff --git a/gdb-7.6.patch b/gdb-7.6.patch > index f64b55fe547a..90629889f8c4 100644 > --- a/gdb-7.6.patch > +++ b/gdb-7.6.patch > @@ -2500,4 +2500,40 @@ diff -up gdb-7.6/opcodes/configure.orig gdb-7.6/opcodes/configure > +struct target_desc *tdesc_aarch64; > #include "features/aarch64.c" > #include "features/aarch64-without-fpu.c" > - > + > +--- gdb-7.6/gdb/symtab.c.orig > ++++ gdb-7.6/gdb/symtab.c > +@@ -2065,7 +2065,7 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) > + struct symtab *s = NULL; > + struct symtab *best_s = NULL; > + struct objfile *objfile; > +- CORE_ADDR distance = 0; > ++ CORE_ADDR distance = 0, start, end; > + struct minimal_symbol *msymbol; > + > + /* If we know that this is not a text address, return failure. This is > +@@ -2102,10 +2102,20 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) > + bv = BLOCKVECTOR (s); > + b = BLOCKVECTOR_BLOCK (bv, GLOBAL_BLOCK); > + > +- if (BLOCK_START (b) <= pc > +- && BLOCK_END (b) > pc > +- && (distance == 0 > +- || BLOCK_END (b) - BLOCK_START (b) < distance)) > ++ start = BLOCK_START (b); > ++ end = BLOCK_END (b); > ++ > ++ /* > ++ * If we have an addrmap mapping code addresses to blocks, and pc > ++ * is in the range [start, end), let's use it. > ++ */ > ++ if ((pc >= start && pc < end) && BLOCKVECTOR_MAP (bv)) { > ++ if (addrmap_find (BLOCKVECTOR_MAP (bv), pc)) > ++ return s; > ++ } > ++ > ++ if ((pc >= start && pc < end) && ((distance == 0) > ++ || (end - start < distance))) > + { > + /* For an objfile that has its functions reordered, > + find_pc_psymtab will find the proper partial symbol table > -- > 2.17.1 > > -- > Crash-utility mailing list > Crash-utility(a)redhat.com > https://listman.redhat.com/mailman/listinfo/crash-utility >

Hi Lianbo, -----Original Message----- > Currently, some command such as 'sys' may cause subsequent 'set scope' > commands to fail because it may not find the correct symtab associated > with PC and SECTION in the find_pc_sect_symtab(), eventually, this will > cause the following failure: I could also reproduce the issue and the patch tested ok. BTW, do you know what part of the sys command causes this issue?

It looks like the "set scope st_create" command itself should do the same thing to gdb, I cannot see what changes the behavior from the commit message.

Thanks, Kazu > > crash> mod -S 3.10.0-957.el7.x86_64 > crash> mod -s dm_service_time > crash> set scope st_create > set: gdb cannot find text block for address: st_create > crash> mod -d dm_service_time > crash> mod -sr dm_service_time > crash> set scope st_create > scope: ffffffffc044d270 (st_create) > crash> sys > KERNEL: 3.10.0-957.el7.x86_64/vmlinux > DUMPFILE: crash/vmcore [PARTIAL DUMP] > CPUS: 48 > DATE: Sat Aug 3 11:41:00 EDT 2019 > UPTIME: 1 days, 10:28:48 > LOAD AVERAGE: 2.95, 1.02, 0.40 > TASKS: 1060 > NODENAME: iop053063.lss.emc.com > RELEASE: 3.10.0-957.el7.x86_64 > VERSION: #1 SMP Thu Oct 4 20:48:51 UTC 2018 > MACHINE: x86_64 (2999 Mhz) > MEMORY: 127.9 GB > PANIC: "SysRq : Trigger a crash" > crash> set scope st_create > set: gdb cannot find text block for address: st_create > > To find the correct symtab, let's check whether there is an address > mapping to 'block' in the symtab searching loop and the PC is in the > range. If the symtab associated with PC is found, and then use it. > > Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com&gt; > --- > gdb-7.6.patch | 38 +++++++++++++++++++++++++++++++++++++- > 1 file changed, 37 insertions(+), 1 deletion(-) > > diff --git a/gdb-7.6.patch b/gdb-7.6.patch > index f64b55fe547a..90629889f8c4 100644 > --- a/gdb-7.6.patch > +++ b/gdb-7.6.patch > @@ -2500,4 +2500,40 @@ diff -up gdb-7.6/opcodes/configure.orig gdb-7.6/opcodes/configure > +struct target_desc *tdesc_aarch64; > #include "features/aarch64.c" > #include "features/aarch64-without-fpu.c" > - > + > +--- gdb-7.6/gdb/symtab.c.orig > ++++ gdb-7.6/gdb/symtab.c > +@@ -2065,7 +2065,7 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) > + struct symtab *s = NULL; > + struct symtab *best_s = NULL; > + struct objfile *objfile; > +- CORE_ADDR distance = 0; > ++ CORE_ADDR distance = 0, start, end; > + struct minimal_symbol *msymbol; > + > + /* If we know that this is not a text address, return failure. This is > +@@ -2102,10 +2102,20 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) > + bv = BLOCKVECTOR (s); > + b = BLOCKVECTOR_BLOCK (bv, GLOBAL_BLOCK); > + > +- if (BLOCK_START (b) <= pc > +- && BLOCK_END (b) > pc > +- && (distance == 0 > +- || BLOCK_END (b) - BLOCK_START (b) < distance)) > ++ start = BLOCK_START (b); > ++ end = BLOCK_END (b); > ++ > ++ /* > ++ * If we have an addrmap mapping code addresses to blocks, and pc > ++ * is in the range [start, end), let's use it. > ++ */ > ++ if ((pc >= start && pc < end) && BLOCKVECTOR_MAP (bv)) { > ++ if (addrmap_find (BLOCKVECTOR_MAP (bv), pc)) > ++ return s; > ++ } > ++ > ++ if ((pc >= start && pc < end) && ((distance == 0) > ++ || (end - start < distance))) > + { > + /* For an objfile that has its functions reordered, > + find_pc_psymtab will find the proper partial symbol table > -- > 2.17.1

> -----Original Message----- > 在 2021年04月01日 17:24, HAGIO KAZUHITO(萩尾　一仁) 写道: >> Hi Lianbo, >> >> -----Original Message----- >>> Currently, some command such as 'sys' may cause subsequent 'set scope' >>> commands to fail because it may not find the correct symtab associated >>> with PC and SECTION in the find_pc_sect_symtab(), eventually, this will >>> cause the following failure: >> >> I could also reproduce the issue and the patch tested ok. >> >> BTW, do you know what part of the sys command causes this issue? > > Good question, only the sys command can not cause this failure. QE tried to > reproduce it many times, and eventually found the following command series > can easily reproduce this issue. ok, so is the trigger of this issue unclear?

I would like to know it if you know. And nitpicking at a few below.

> >> It looks like the "set scope st_create" command itself should do the same >> thing to gdb, I cannot see what changes the behavior from the commit message. >> > The "set scope st_create" command will get the address of block containing the > specified PC value via calling the block_for_pc() as below: > > gdb_set_crash_scope()-> > gdb_command_funnel()-> > gdb_set_crash_block()-> > block_for_pc()-> > block_for_pc_sect()-> > blockvector_for_pc_sect()-> > find_pc_sect_symtab()----------- The problem is here. > > But actually, it doesn't get the expected address of block from the find_pc_sect_symtab(), > so we have to correct it, check whether there is an address mapping to the 'block' in > the symtab searching loop and the PC value is in the range. If the symtab associated with > the PC value is found, and then use it. > > Thanks. > Lianbo > >> Thanks, >> Kazu >> >>> >>> crash> mod -S 3.10.0-957.el7.x86_64 >>> crash> mod -s dm_service_time >>> crash> set scope st_create >>> set: gdb cannot find text block for address: st_create >>> crash> mod -d dm_service_time >>> crash> mod -sr dm_service_time >>> crash> set scope st_create >>> scope: ffffffffc044d270 (st_create) >>> crash> sys >>> KERNEL: 3.10.0-957.el7.x86_64/vmlinux >>> DUMPFILE: crash/vmcore [PARTIAL DUMP] >>> CPUS: 48 >>> DATE: Sat Aug 3 11:41:00 EDT 2019 >>> UPTIME: 1 days, 10:28:48 >>> LOAD AVERAGE: 2.95, 1.02, 0.40 >>> TASKS: 1060 >>> NODENAME: iop053063.lss.emc.com >>> RELEASE: 3.10.0-957.el7.x86_64 >>> VERSION: #1 SMP Thu Oct 4 20:48:51 UTC 2018 >>> MACHINE: x86_64 (2999 Mhz) >>> MEMORY: 127.9 GB >>> PANIC: "SysRq : Trigger a crash" >>> crash> set scope st_create >>> set: gdb cannot find text block for address: st_create >>> >>> To find the correct symtab, let's check whether there is an address >>> mapping to 'block' in the symtab searching loop and the PC is in the >>> range. If the symtab associated with PC is found, and then use it. >>> >>> Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com&gt; >>> --- >>> gdb-7.6.patch | 38 +++++++++++++++++++++++++++++++++++++- >>> 1 file changed, 37 insertions(+), 1 deletion(-) >>> >>> diff --git a/gdb-7.6.patch b/gdb-7.6.patch >>> index f64b55fe547a..90629889f8c4 100644 >>> --- a/gdb-7.6.patch >>> +++ b/gdb-7.6.patch >>> @@ -2500,4 +2500,40 @@ diff -up gdb-7.6/opcodes/configure.orig gdb-7.6/opcodes/configure >>> +struct target_desc *tdesc_aarch64; >>> #include "features/aarch64.c" >>> #include "features/aarch64-without-fpu.c" >>> - >>> + It should not edit the previous patch :-) >>> +--- gdb-7.6/gdb/symtab.c.orig >>> ++++ gdb-7.6/gdb/symtab.c >>> +@@ -2065,7 +2065,7 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) patching file gdb-7.6/gdb/symtab.c Hunk #1 succeeded at 2080 (offset 15 lines). Hunk #2 succeeded at 2117 (offset 15 lines). Offsetting is needed, it would be better to have no messages like this to check future patches easily.

But you don't need to repost.

Thanks, Kazu >>> + struct symtab *s = NULL; >>> + struct symtab *best_s = NULL; >>> + struct objfile *objfile; >>> +- CORE_ADDR distance = 0; >>> ++ CORE_ADDR distance = 0, start, end; >>> + struct minimal_symbol *msymbol; >>> + >>> + /* If we know that this is not a text address, return failure. This is >>> +@@ -2102,10 +2102,20 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) >>> + bv = BLOCKVECTOR (s); >>> + b = BLOCKVECTOR_BLOCK (bv, GLOBAL_BLOCK); >>> + >>> +- if (BLOCK_START (b) <= pc >>> +- && BLOCK_END (b) > pc >>> +- && (distance == 0 >>> +- || BLOCK_END (b) - BLOCK_START (b) < distance)) >>> ++ start = BLOCK_START (b); >>> ++ end = BLOCK_END (b); >>> ++ >>> ++ /* >>> ++ * If we have an addrmap mapping code addresses to blocks, and pc >>> ++ * is in the range [start, end), let's use it. >>> ++ */ >>> ++ if ((pc >= start && pc < end) && BLOCKVECTOR_MAP (bv)) { >>> ++ if (addrmap_find (BLOCKVECTOR_MAP (bv), pc)) >>> ++ return s; >>> ++ } >>> ++ >>> ++ if ((pc >= start && pc < end) && ((distance == 0) >>> ++ || (end - start < distance))) >>> + { >>> + /* For an objfile that has its functions reordered, >>> + find_pc_psymtab will find the proper partial symbol table >>> -- >>> 2.17.1 >>

-----Original Message----- > 在 2021年04月06日 09:53, HAGIO KAZUHITO(萩尾　一仁) 写道: >>> -----Original Message----- >>> 在 2021年04月01日 17:24, HAGIO KAZUHITO(萩尾　一仁) 写道: >>>> Hi Lianbo, >>>> >>>> -----Original Message----- >>>>> Currently, some command such as 'sys' may cause subsequent 'set scope' >>>>> commands to fail because it may not find the correct symtab associated >>>>> with PC and SECTION in the find_pc_sect_symtab(), eventually, this will >>>>> cause the following failure: >>>> >>>> I could also reproduce the issue and the patch tested ok. >>>> >>>> BTW, do you know what part of the sys command causes this issue? >>> >>> Good question, only the sys command can not cause this failure. QE tried to >>> reproduce it many times, and eventually found the following command series >>> can easily reproduce this issue. >> >> ok, so is the trigger of this issue unclear? > > I'm not very sure what triggered it due to the complicated steps, and until now, I haven't > seen that a single command(or two commands) can cause this error. > > But I know what it happened. The problem is that the best match loop always assumes that a > symtab with a best match can be found. But, in fact, it didn't find out the best match in > the following case. > > ... > crash> sys > blockvector_for_pc_sect() line number: 174 symtab:0x8647670 pc:0xffffffffc044d270 module file > name:drivers/md/dm-service-time.c > blockvector_for_pc_sect() line number: 179 bl:0x856ec90 pc:0xffffffffc044d270 > blockvector_for_pc_sect() line number: 186 bl:0x856ec90 b:0x86351b0 pc:0xffffffffc044d270 > blockvector_for_pc_sect() line number: 174 symtab:0x8647670 pc:0xffffffffc044d270 module file > name:drivers/md/dm-service-time.c > blockvector_for_pc_sect() line number: 179 bl:0x856ec90 pc:0xffffffffc044d270 > blockvector_for_pc_sect() line number: 186 bl:0x856ec90 b:0x86351b0 pc:0xffffffffc044d270 > ... > blockvector_for_pc_sect() line number: 174 symtab:0xeb014f0 pc:0xffffffffc044d270 module file > name:net/ipv6/netfilter/ip6table_filter.c > blockvector_for_pc_sect() line number: 179 bl:0xe9c3b00 pc:0xffffffffc044d270 > blockvector_for_pc_sect() line number: 183 bl:0xe9c3b00 pc:0xffffffffc044d270 > ... > > crash> set scope st_create > blockvector_for_pc_sect() line number: 174 symtab:0xeb014f0 pc:0xffffffffc044d270 module file > name:net/ipv6/netfilter/ip6table_filter.c > blockvector_for_pc_sect() line number: 179 bl:0xe9c3b00 pc:0xffffffffc044d270 > blockvector_for_pc_sect() line number: 183 bl:0xe9c3b00 pc:0xffffffffc044d270 > set: gdb cannot find text block for address: st_create > > To fix it, a simple way is to check the address mapping(to the 'block') in the best match loop. Thanks for the explanation. I could not find the key condition that causes the issue, but the patch works well and no ill effect was detected in my test.

Shortened the procedure to reproduce and applied:

https://github.com/crash-utility/crash/commit/6974853 Thanks, Kazu > >> I would like to know it if you know. >> >> And nitpicking at a few below. >> > No worry about this. > >>> >>>> It looks like the "set scope st_create" command itself should do the same >>>> thing to gdb, I cannot see what changes the behavior from the commit message. >>>> >>> The "set scope st_create" command will get the address of block containing the >>> specified PC value via calling the block_for_pc() as below: >>> >>> gdb_set_crash_scope()-> >>> gdb_command_funnel()-> >>> gdb_set_crash_block()-> >>> block_for_pc()-> >>> block_for_pc_sect()-> >>> blockvector_for_pc_sect()-> >>> find_pc_sect_symtab()----------- The problem is here. >>> >>> But actually, it doesn't get the expected address of block from the find_pc_sect_symtab(), >>> so we have to correct it, check whether there is an address mapping to the 'block' in >>> the symtab searching loop and the PC value is in the range. If the symtab associated with >>> the PC value is found, and then use it. >>> >>> Thanks. >>> Lianbo >>> >>>> Thanks, >>>> Kazu >>>> >>>>> >>>>> crash> mod -S 3.10.0-957.el7.x86_64 >>>>> crash> mod -s dm_service_time >>>>> crash> set scope st_create >>>>> set: gdb cannot find text block for address: st_create >>>>> crash> mod -d dm_service_time >>>>> crash> mod -sr dm_service_time >>>>> crash> set scope st_create >>>>> scope: ffffffffc044d270 (st_create) >>>>> crash> sys >>>>> KERNEL: 3.10.0-957.el7.x86_64/vmlinux >>>>> DUMPFILE: crash/vmcore [PARTIAL DUMP] >>>>> CPUS: 48 >>>>> DATE: Sat Aug 3 11:41:00 EDT 2019 >>>>> UPTIME: 1 days, 10:28:48 >>>>> LOAD AVERAGE: 2.95, 1.02, 0.40 >>>>> TASKS: 1060 >>>>> NODENAME: iop053063.lss.emc.com >>>>> RELEASE: 3.10.0-957.el7.x86_64 >>>>> VERSION: #1 SMP Thu Oct 4 20:48:51 UTC 2018 >>>>> MACHINE: x86_64 (2999 Mhz) >>>>> MEMORY: 127.9 GB >>>>> PANIC: "SysRq : Trigger a crash" >>>>> crash> set scope st_create >>>>> set: gdb cannot find text block for address: st_create >>>>> >>>>> To find the correct symtab, let's check whether there is an address >>>>> mapping to 'block' in the symtab searching loop and the PC is in the >>>>> range. If the symtab associated with PC is found, and then use it. >>>>> >>>>> Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com&gt; >>>>> --- >>>>> gdb-7.6.patch | 38 +++++++++++++++++++++++++++++++++++++- >>>>> 1 file changed, 37 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/gdb-7.6.patch b/gdb-7.6.patch >>>>> index f64b55fe547a..90629889f8c4 100644 >>>>> --- a/gdb-7.6.patch >>>>> +++ b/gdb-7.6.patch >>>>> @@ -2500,4 +2500,40 @@ diff -up gdb-7.6/opcodes/configure.orig gdb-7.6/opcodes/configure >>>>> +struct target_desc *tdesc_aarch64; >>>>> #include "features/aarch64.c" >>>>> #include "features/aarch64-without-fpu.c" >>>>> - >>>>> + >> >> It should not edit the previous patch :-) >> >>>>> +--- gdb-7.6/gdb/symtab.c.orig >>>>> ++++ gdb-7.6/gdb/symtab.c >>>>> +@@ -2065,7 +2065,7 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) >> >> patching file gdb-7.6/gdb/symtab.c >> Hunk #1 succeeded at 2080 (offset 15 lines). >> Hunk #2 succeeded at 2117 (offset 15 lines). >> >> Offsetting is needed, it would be better to have no messages like this >> to check future patches easily. >> > Thank you for pointing out this issue. My intention is to have a new blank line between my patch and > the above patches in order to make it clear. Feel free to revert this change. > >> But you don't need to repost. >> > OK, Thanks. > > Lianbo > >> Thanks, >> Kazu >> >>>>> + struct symtab *s = NULL; >>>>> + struct symtab *best_s = NULL; >>>>> + struct objfile *objfile; >>>>> +- CORE_ADDR distance = 0; >>>>> ++ CORE_ADDR distance = 0, start, end; >>>>> + struct minimal_symbol *msymbol; >>>>> + >>>>> + /* If we know that this is not a text address, return failure. This is >>>>> +@@ -2102,10 +2102,20 @@ find_pc_sect_symtab (CORE_ADDR pc, struct obj_section *section) >>>>> + bv = BLOCKVECTOR (s); >>>>> + b = BLOCKVECTOR_BLOCK (bv, GLOBAL_BLOCK); >>>>> + >>>>> +- if (BLOCK_START (b) <= pc >>>>> +- && BLOCK_END (b) > pc >>>>> +- && (distance == 0 >>>>> +- || BLOCK_END (b) - BLOCK_START (b) < distance)) >>>>> ++ start = BLOCK_START (b); >>>>> ++ end = BLOCK_END (b); >>>>> ++ >>>>> ++ /* >>>>> ++ * If we have an addrmap mapping code addresses to blocks, and pc >>>>> ++ * is in the range [start, end), let's use it. >>>>> ++ */ >>>>> ++ if ((pc >= start && pc < end) && BLOCKVECTOR_MAP (bv)) { >>>>> ++ if (addrmap_find (BLOCKVECTOR_MAP (bv), pc)) >>>>> ++ return s; >>>>> ++ } >>>>> ++ >>>>> ++ if ((pc >= start && pc < end) && ((distance == 0) >>>>> ++ || (end - start < distance))) >>>>> + { >>>>> + /* For an objfile that has its functions reordered, >>>>> + find_pc_psymtab will find the proper partial symbol table >>>>> -- >>>>> 2.17.1 >>>> >>