December 2024 - Crash-utility - Crash Utility List Archives

[PATCH] arm64: add pac mask to better support gdb stack unwind

by Guanyou Chen

Hi Lianbo & Tao Currently, gdb passthroughs of 'bt', 'frame', 'up', 'down', 'info, locals' don't work on arm64 machine enabled pauth. This is due to gdb not knowing the lr register real values to unwind the stack frames. ---------------------------- gdb passthrough (eg. "bt") | | crash -------------------------> | | | gdb_interface | | | | | | ---------------------- | get_kernel_pac_mask | | | | crash_target<-------------------------+--| gdb | | --------------------------+->| | | arm64: CONFIG_ARM64_KERNELPACMASK| | | | other: ~0UL | | | | | ---------------------- | ---------------------------- With the patch: crash> gdb bt #0 __switch_to (prev=prev@entry=0xffffff8001af92c0, next=next@entry=0xffffff889da7a580) at /proc/self/cwd/common/arch/arm64/kernel/process.c:569 #1 0xffffffd3602132c0 in context_switch (rq=0xffffff8a7295a080, prev=0xffffff8001af92c0, next=0xffffff889da7a580, rf=<optimized out>) at /proc/self/cwd/common/kernel/sched/core.c:5515 #2 __schedule (sched_mode=<optimized out>, sched_mode@entry=2147859424) at /proc/self/cwd/common/kernel/sched/core.c:6843 #3 0xffffffd3602136d8 in schedule () at /proc/self/cwd/common/kernel/sched/core.c:6917 ... Without the patch: crash> gdb bt #0 __switch_to (prev=0xffffff8001af92c0, next=0xffffff889da7a580) at /proc/self/cwd/common/arch/arm64/kernel/process.c:569 #1 0x9fc5c5d3602132c0 in ?? () Backtrace stopped: previous frame identical to this frame (corrupt stack?) Signed-off-by: Guanyou.Chen <chenguanyou(a)xiaomi.com> --- gdb-10.2.patch | 24 ++++++++++++++++++++++++ gdb_interface.c | 11 +++++++++++ 2 files changed, 35 insertions(+) diff --git a/gdb-10.2.patch b/gdb-10.2.patch index c867660..4c13a6b 100644 --- a/gdb-10.2.patch +++ b/gdb-10.2.patch @@ -16216,3 +16216,27 @@ exit 0 printf_filtered (_("Backtrace stopped: %s\n"), frame_stop_reason_string (trailing)); } +--- gdb-10.2/gdb/frame.c.orig ++++ gdb-10.2/gdb/frame.c +@@ -944,6 +944,10 @@ frame_find_by_id (struct frame_id id) + return NULL; + } + ++#ifdef CRASH_MERGE ++extern "C" unsigned long crash_get_kernel_pac_mask(void); ++#endif ++ + static CORE_ADDR + frame_unwind_pc (frame_info_ptr this_frame) + { +@@ -974,6 +978,10 @@ frame_unwind_pc (struct frame_info_ptr *this_frame) + try + { + pc = gdbarch_unwind_pc (prev_gdbarch, this_frame); ++#ifdef CRASH_MERGE ++ CORE_ADDR mask = crash_get_kernel_pac_mask(); ++ pc |= mask; ++#endif + pc_p = true; + } + catch (const gdb_exception_error &ex) diff --git a/gdb_interface.c b/gdb_interface.c index 315711e..765dafe 100644 --- a/gdb_interface.c +++ b/gdb_interface.c @@ -1083,3 +1083,14 @@ int crash_get_current_task_reg (int regno, const char *regname, return machdep->get_current_task_reg(regno, regname, regsize, value); } +/* arm64 kernel lr pac mask */ +unsigned long crash_get_kernel_pac_mask(void); +unsigned long crash_get_kernel_pac_mask(void) +{ +#ifdef ARM64 + struct machine_specific *ms = machdep->machspec; + return ms->CONFIG_ARM64_KERNELPACMASK; +#else + return ~0UL; +#endif /* !ARM64 */ +} -- 2.34.1 Thanks, Guanyou

18 hours, 25 minutes

2
1
0 / 0

Re: [PATCH] Fix incorrect 'bt -v' output suggesting overflow

by lijiang

On Tue, Dec 17, 2024 at 12:14 PM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Tue, 17 Dec 2024 17:12:23 +1300 > From: Tao Liu <ltao(a)redhat.com> > Subject: [Crash-utility] Re: [PATCH] Fix incorrect 'bt -v' output > suggesting overflow > To: Lucas Oakley <soakley(a)redhat.com> > Cc: devel(a)lists.crash-utility.osci.io > Message-ID: > < > CAO7dBbUQ6HWH03iE6Knd9wW5den0hwAsGORXsHzVVJXxx1-O7g(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi Lucas, > > Thanks for the fix, LGTM, ack. > > Applied(with slight changes): https://github.com/crash-utility/crash/commit/69aef36ff4bc374b510db6037d3... > Thanks, > Tao Liu > > On Sun, Dec 15, 2024 at 12:03 PM Lucas Oakley <soakley(a)redhat.com> wrote: > > > > Change check_stack_overflow() to check if the thread_info's cpu > > member is smaller than possible existing CPUs, rather than the > > kernel table's cpu number (kt->cpus). The kernel table's cpu number > > is changed on some architectures to reflect the highest numbered > > online cpu + 1. This can cause a false positive in > > check_stack_overflow() if the cpu member of a parked task's > > thread_info structure, assigned to an offlined cpu, is larger than > > the kt->cpus but lower than the number of existing logical cpus. > > An example of this is RHEL 7 on s390x or RHEL 8 on ppc64le when > > the highest numbered CPU is offlined. > > > > Signed-off-by: Lucas Oakley <soakley(a)redhat.com> > > --- > > task.c | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/task.c b/task.c > > index 33de7da..93dab0e 100644 > > --- a/task.c > > +++ b/task.c > > @@ -11253,12 +11253,12 @@ check_stack_overflow(void) > > cpu = 0; > > break; > > } > > - if (cpu >= kt->cpus) { > > + if (cpu >= get_cpus_present()) { > > if (!overflow) > > print_task_header(fp, tc, 0); > > fprintf(fp, > > " possible stack overflow: > thread_info.cpu: %d >= %d\n", > > - cpu, kt->cpus); > > + cpu, get_cpus_present()); > > overflow++; total++; > > } > > } > > -- > > 2.47.1 >

1 week, 3 days

1
0
0 / 0

Re: [PATCH] Fix incorrect 'bt -v' output suggesting overflow

by lijiang

Hi, Lucas Thank you for the fix. On Tue, Dec 17, 2024 at 12:14 PM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Sat, 14 Dec 2024 18:01:14 -0500 > From: Lucas Oakley <soakley(a)redhat.com> > Subject: [Crash-utility] [PATCH] Fix incorrect 'bt -v' output > suggesting overflow > To: devel(a)lists.crash-utility.osci.io > Message-ID: <20241214230114.2854910-1-soakley(a)redhat.com> > Content-Type: text/plain; charset="US-ASCII"; x-default=true > > Change check_stack_overflow() to check if the thread_info's cpu > member is smaller than possible existing CPUs, rather than the > kernel table's cpu number (kt->cpus). The kernel table's cpu number > is changed on some architectures to reflect the highest numbered > online cpu + 1. This can cause a false positive in > check_stack_overflow() if the cpu member of a parked task's > thread_info structure, assigned to an offlined cpu, is larger than > the kt->cpus but lower than the number of existing logical cpus. > An example of this is RHEL 7 on s390x or RHEL 8 on ppc64le when > the highest numbered CPU is offlined. > > Signed-off-by: Lucas Oakley <soakley(a)redhat.com> > --- > task.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/task.c b/task.c > index 33de7da..93dab0e 100644 > --- a/task.c > +++ b/task.c > @@ -11253,12 +11253,12 @@ check_stack_overflow(void) > cpu = 0; > break; > } > - if (cpu >= kt->cpus) { > + if (cpu >= get_cpus_present()) { > if (!overflow) > print_task_header(fp, tc, 0); > fprintf(fp, > " possible stack overflow: > thread_info.cpu: %d >= %d\n", > - cpu, kt->cpus); > + cpu, get_cpus_present()); > overflow++; total++; > } > } > To avoid calling get_cpus_present() twice, I would tend to modify it as below: diff --git a/task.c b/task.c index 33de7da2a692..49f771e275c1 100644 --- a/task.c +++ b/task.c @@ -11238,6 +11238,8 @@ check_stack_overflow(void) } if (VALID_MEMBER(thread_info_cpu)) { + int cpus = get_cpus_present(); + switch (cpu_size) { case 1: @@ -11253,12 +11255,12 @@ check_stack_overflow(void) cpu = 0; break; } - if (cpu >= kt->cpus) { + if (cpu >= cpus) { if (!overflow) print_task_header(fp, tc, 0); fprintf(fp, " possible stack overflow: thread_info.cpu: %d >= %d\n", - cpu, kt->cpus); + cpu, cpus); overflow++; total++; } } What do you think? Lianbo -- > 2.47.1 >

1 week, 3 days

2
2
0 / 0

Re: [PATCH] x86_64: Mark #VC stack unavailable when CONFIG_AMD_MEM_ENCRYPT is not set

by lijiang

On Thu, Dec 12, 2024 at 10:44 AM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Thu, 12 Dec 2024 12:53:08 +1300 > From: Tao Liu <ltao(a)redhat.com> > Subject: [Crash-utility] Re: [PATCH] x86_64: Mark #VC stack > unavailable when CONFIG_AMD_MEM_ENCRYPT is not set > To: Hou Wenlong <houwenlong.hwl(a)antgroup.com> > Cc: devel(a)lists.crash-utility.osci.io > Message-ID: > <CAO7dBbWi06FCj36_UPMXN3cs_=xQZd-oM= > hAZ7eZrVpTkVWLkA(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi Hou, > > On Wed, Dec 11, 2024 at 2:36 PM Hou Wenlong <houwenlong.hwl(a)antgroup.com> > wrote: > > > > When 'CONFIG_AMD_MEM_ENCRYPT' is not set, the IDT handler for #VC is not > > registered, but the address of the #VC IST stack is still set in > > tss_setup_ist(). Therefore, the name of the associated exception stack > > is "UNKNOWN" instead of "VC". Although the base of the exception stack > > is not zero and is available, it is not accessible, which will cause the > > backtrace to fail when attempting to access the #VC stack. To fix this, > > remove the name check. > > > > Signe-off-by: Hou Wenlong <houwenlong.hwl(a)antgroup.com> > > --- > > x86_64.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/x86_64.c b/x86_64.c > > index e7f8fe25b31f..ee12ba095e6e 100644 > > --- a/x86_64.c > > +++ b/x86_64.c > > @@ -1540,7 +1540,7 @@ x86_64_ist_init(void) > > > > ms->stkinfo.available[c][i] = TRUE; > > /* VC stack can be unmapped if SEV-ES is > disabled or not supported. */ > > - if > (STREQ(ms->stkinfo.exception_stacks[i], "VC") && > > + if (ms->stkinfo.ebase[c][i] && > > !accessible(ms->stkinfo.ebase[c][i])) > > I agree the ms->stkinfo.ebase[c][i] should always be checked for > "accessible", not just for "VC" stacks, so removing the "VC" name > checking is reasonable. The code LGTM, so ack. > Applied(with slight changes): https://github.com/crash-utility/crash/commit/8a11aa07d7b0f0e0fc165365376... > > Thanks, > Tao Liu > > > ms->stkinfo.available[c][i] = > FALSE; > > } > > > > base-commit: f13853cef53f5c5463a51021edbc81977e2b1405 > > -- > > 2.31.1 >

1 week, 5 days

1
0
0 / 0

[PATCH] Fix incorrect 'bt -v' output suggesting overflow

by Lucas Oakley

Change check_stack_overflow() to check if the thread_info's cpu member is smaller than possible existing CPUs, rather than the kernel table's cpu number (kt->cpus). The kernel table's cpu number is changed on some architectures to reflect the highest numbered online cpu + 1. This can cause a false positive in check_stack_overflow() if the cpu member of a parked task's thread_info structure, assigned to an offlined cpu, is larger than the kt->cpus but lower than the number of existing logical cpus. An example of this is RHEL 7 on s390x or RHEL 8 on ppc64le when the highest numbered CPU is offlined. Signed-off-by: Lucas Oakley <soakley(a)redhat.com> --- task.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/task.c b/task.c index 33de7da..93dab0e 100644 --- a/task.c +++ b/task.c @@ -11253,12 +11253,12 @@ check_stack_overflow(void) cpu = 0; break; } - if (cpu >= kt->cpus) { + if (cpu >= get_cpus_present()) { if (!overflow) print_task_header(fp, tc, 0); fprintf(fp, " possible stack overflow: thread_info.cpu: %d >= %d\n", - cpu, kt->cpus); + cpu, get_cpus_present()); overflow++; total++; } } -- 2.47.1

1 week, 6 days

2
1
0 / 0

crash: seek error: kernel virtual address: ffffffff8292c360 type: "page_offset_base"

by xiangyu.chen＠aol.com

Hi Crash-utility folks, Recently, I am using crash to debug a live system. When I execute the crash, it cannot enter the crash cmd shell, it was broken by crash seek error. My kernel turned on the CONFIG_RANDOMIZE_BASE, and I added the kaslr=auto parameter to crash. If I disable the CONFIG_RANDOMIZE_BASE, the issue has gone. Could you please help to check if any config/parameter I was missing? Thanks. The following is my system information: crash version: 8.0.6 kernel version: 6.1.118 Host system: qemux86-64 Here is the full log with -d: root@intel-x86-64:/boot# crash -S /boot/vmlinux-6.1.118-yocto-standard /proc/kcore -d10 --kaslr=auto crash 8.0.6 Copyright (C) 2002-2025 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011, 2020-2024 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. Copyright (C) 2015, 2021 VMware, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. get_live_memory_source: /proc/kcore proc_kcore_data: flags: 500 (KCORE_LOCAL|KCORE_ELF64) segments: 10 elf_header: 56471f1eb9c0 header_size: 12324 notes64: 56471f1eba00 load64: 56471f1eba38 notes32: 0 load32: 0 vmcoreinfo: 0 size_vmcoreinfo: 0 Elf64_Phdr: p_type: 4 (PT_NOTE) p_flags: 0 p_offset: 2a8 p_vaddr: 0 p_paddr: 0 p_filesz: 11644 p_memsz: 0 p_align: 0 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 7fffaa404000 p_vaddr: ffffffffaa400000 p_paddr: b8600000 p_filesz: 33751040 p_memsz: 33751040 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 286600004000 p_vaddr: ffffa86600000000 p_paddr: ffffffffffffffff p_filesz: 35184372088831 p_memsz: 35184372088831 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 7fffc0004000 p_vaddr: ffffffffc0000000 p_paddr: ffffffffffffffff p_filesz: 1056964608 p_memsz: 1056964608 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 126ac0005000 p_vaddr: ffff926ac0001000 p_paddr: 1000 p_filesz: 647168 p_memsz: 647168 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 4d8880004000 p_vaddr: ffffcd8880000000 p_paddr: ffffffffffffffff p_filesz: 12288 p_memsz: 12288 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 126ac0104000 p_vaddr: ffff926ac0100000 p_paddr: 100000 p_filesz: 12902400 p_memsz: 12902400 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 4d8880008000 p_vaddr: ffffcd8880004000 p_paddr: ffffffffffffffff p_filesz: 204800 p_memsz: 204800 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 126ac0d53000 p_vaddr: ffff926ac0d4f000 p_paddr: d4f000 p_filesz: 3207118848 p_memsz: 3207118848 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 126bc0004000 p_vaddr: ffff926bc0000000 p_paddr: 100000000 p_filesz: 4294967296 p_memsz: 4294967296 p_align: 4096 Elf64_Phdr: p_type: 1 (PT_LOAD) p_flags: 7 p_offset: 4d8884004000 p_vaddr: ffffcd8884000000 p_paddr: ffffffffffffffff p_filesz: 67108864 p_memsz: 67108864 p_align: 4096 Elf64_Nhdr: n_namesz: 5 ("CORE") n_descsz: 336 n_type: 1 (NT_PRSTATUS) Elf64_Nhdr: n_namesz: 5 ("CORE") n_descsz: 136 n_type: 3 (NT_PRPSINFO) Elf64_Nhdr: n_namesz: 5 ("CORE") n_descsz: 7808 n_type: 4 (NT_TASKSTRUCT) Elf64_Nhdr: n_namesz: 11 ("VMCOREINFO") n_descsz: 3279 n_type: 0 (unknown) OSRELEASE=6.1.118-yocto-standard BUILD-ID=1400000003000000474e5500a343709706409b9c PAGESIZE=4096 SYMBOL(init_uts_ns)=ffffffffac037a80 OFFSET(uts_namespace.name)=0 SYMBOL(node_online_map)=ffffffffac06d3e8 SYMBOL(swapper_pg_dir)=ffffffffabe0a000 SYMBOL(_stext)=ffffffffaa400000 SYMBOL(vmap_area_list)=ffffffffabf12fb0 SYMBOL(mem_section)=ffff926cbfff5000 LENGTH(mem_section)=2048 SIZE(mem_section)=16 OFFSET(mem_section.section_mem_map)=0 NUMBER(SECTION_SIZE_BITS)=27 NUMBER(MAX_PHYSMEM_BITS)=46 SIZE(page)=64 SIZE(pglist_data)=15616 SIZE(zone)=1600 SIZE(free_area)=104 SIZE(list_head)=16 SIZE(nodemask_t)=8 OFFSET(page.flags)=0 OFFSET(page._refcount)=52 OFFSET(page.mapping)=24 OFFSET(page.lru)=8 OFFSET(page._mapcount)=48 OFFSET(page.private)=40 OFFSET(page.compound_dtor)=16 OFFSET(page.compound_order)=17 OFFSET(page.compound_head)=8 OFFSET(pglist_data.node_zones)=0 OFFSET(pglist_data.nr_zones)=14624 OFFSET(pglist_data.node_start_pfn)=14632 OFFSET(pglist_data.node_spanned_pages)=14648 OFFSET(pglist_data.node_id)=14656 OFFSET(zone.free_area)=192 OFFSET(zone.vm_stat)=1472 OFFSET(zone.spanned_pages)=136 OFFSET(free_area.free_list)=0 OFFSET(list_head.next)=0 OFFSET(list_head.prev)=8 OFFSET(vmap_area.va_start)=0 OFFSET(vmap_area.list)=40 LENGTH(zone.free_area)=11 SYMBOL(prb)=ffffffffabe647d8 SYMBOL(printk_rb_static)=ffffffffabe647e0 SYMBOL(clear_seq)=ffffffffac307f20 SIZE(printk_ringbuffer)=88 OFFSET(printk_ringbuffer.desc_ring)=0 OFFSET(printk_ringbuffer.text_data_ring)=48 OFFSET(printk_ringbuffer.fail)=80 SIZE(prb_desc_ring)=48 OFFSET(prb_desc_ring.count_bits)=0 OFFSET(prb_desc_ring.descs)=8 OFFSET(prb_desc_ring.infos)=16 OFFSET(prb_desc_ring.head_id)=24 OFFSET(prb_desc_ring.tail_id)=32 SIZE(prb_desc)=24 OFFSET(prb_desc.state_var)=0 OFFSET(prb_desc.text_blk_lpos)=8 SIZE(prb_data_blk_lpos)=16 OFFSET(prb_data_blk_lpos.begin)=0 OFFSET(prb_data_blk_lpos.next)=8 SIZE(printk_info)=88 OFFSET(printk_info.seq)=0 OFFSET(printk_info.ts_nsec)=8 OFFSET(printk_info.text_len)=16 OFFSET(printk_info.caller_id)=20 OFFSET(printk_info.dev_info)=24 SIZE(dev_printk_info)=64 OFFSET(dev_printk_info.subsystem)=0 LENGTH(printk_info_subsystem)=16 OFFSET(dev_printk_info.device)=16 LENGTH(printk_info_device)=48 SIZE(prb_data_ring)=32 OFFSET(prb_data_ring.size_bits)=0 OFFSET(prb_data_ring.data)=8 OFFSET(prb_data_ring.head_lpos)=16 OFFSET(prb_data_ring.tail_lpos)=24 SIZE(atomic_long_t)=8 OFFSET(atomic_long_t.counter)=0 SIZE(latched_seq)=24 OFFSET(latched_seq.val)=8 LENGTH(free_area.free_list)=6 NUMBER(NR_FREE_PAGES)=0 NUMBER(PG_lru)=4 NUMBER(PG_private)=13 NUMBER(PG_swapcache)=10 NUMBER(PG_swapbacked)=19 NUMBER(PG_slab)=9 NUMBER(PG_hwpoison)=23 NUMBER(PG_head_mask)=65536 NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129 NUMBER(HUGETLB_PAGE_DTOR)=2 NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257 SYMBOL(kallsyms_names)=ffffffffaba0f7d8 SYMBOL(kallsyms_num_syms)=ffffffffaba0f7d0 SYMBOL(kallsyms_token_table)=ffffffffabbf5400 SYMBOL(kallsyms_token_index)=ffffffffabbf5788 SYMBOL(kallsyms_offsets)=ffffffffab9984f8 SYMBOL(kallsyms_relative_base)=ffffffffaba0f7c8 NUMBER(phys_base)=2384461824 SYMBOL(init_top_pgt)=ffffffffabe0a000 NUMBER(pgtable_l5_enabled)=0 SYMBOL(node_data)=ffffffffac06c3a0 LENGTH(node_data)=64 KERNELOFFSET=29400000 NUMBER(KERNEL_IMAGE_SIZE)=1073741824 NUMBER(sme_mask)=0 readmem: read_proc_kcore() -> /proc/kcore VMCOREINFO: NUMBER(phys_base): 2384461824 -> 8e200000 gdb /boot/vmlinux-6.1.118-yocto-standard GNU gdb (GDB) 10.2 Copyright (C) 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-wrs-linux". Type "show configuration" for configuration details. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... GETBUF(344 -> 0) GETBUF(1500 -> 1) FREEBUF(1) FREEBUF(0) <readmem: ffffffff8292c360, KVADDR, "page_offset_base", 8, (FOE|Q), 56471ef799c8> <read_proc_kcore: addr: ffffffff8292c360 paddr: 90b2c360 cnt: 8> crash: seek error: kernel virtual address: ffffffff8292c360 type: "page_offset_base"

2 weeks, 3 days

3
6
0 / 0

Re: [PATCH] x86_64: Mark #VC stack unavailable when CONFIG_AMD_MEM_ENCRYPT is not set

by lijiang

On Wed, Dec 11, 2024 at 10:22 AM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Tue, 10 Dec 2024 20:18:43 +0800 > From: "Hou Wenlong" <houwenlong.hwl(a)antgroup.com> > Subject: [Crash-utility] [PATCH] x86_64: Mark #VC stack unavailable > when CONFIG_AMD_MEM_ENCRYPT is not set > To: devel(a)lists.crash-utility.osci.io > Cc: Hou Wenlong <houwenlong.hwl(a)antgroup.com> > Message-ID: <f1fca9918841de9a48f428d498f50964f1dab543.1733831073.git.h > ouwenlong.hwl(a)antgroup.com> > > When 'CONFIG_AMD_MEM_ENCRYPT' is not set, the IDT handler for #VC is not > registered, but the address of the #VC IST stack is still set in > tss_setup_ist(). Therefore, the name of the associated exception stack > is "UNKNOWN" instead of "VC". Although the base of the exception stack > is not zero and is available, it is not accessible, which will cause the > backtrace to fail when attempting to access the #VC stack. To fix this, > remove the name check. > > Thank you for the fix, Hou. > Signe-off-by: Hou Wenlong <houwenlong.hwl(a)antgroup.com> > --- > x86_64.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/x86_64.c b/x86_64.c > index e7f8fe25b31f..ee12ba095e6e 100644 > --- a/x86_64.c > +++ b/x86_64.c > @@ -1540,7 +1540,7 @@ x86_64_ist_init(void) > > ms->stkinfo.available[c][i] = TRUE; > /* VC stack can be unmapped if SEV-ES is > disabled or not supported. */ > - if (STREQ(ms->stkinfo.exception_stacks[i], > "VC") && > + if (ms->stkinfo.ebase[c][i] && > Can you help double check if the above condition is redundant in your case? Seems the check '!accessible(ms->stkinfo.ebase[c][i])' should be enough. Thanks Lianbo !accessible(ms->stkinfo.ebase[c][i])) > ms->stkinfo.available[c][i] = > FALSE; > } > > base-commit: f13853cef53f5c5463a51021edbc81977e2b1405 > -- > 2.31.1 >

2 weeks, 3 days

4
4
0 / 0

Re: [PATCH] gdb bt: multiple stacks support (x86_64)

by lijiang

On Mon, Nov 11, 2024 at 2:54 PM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Mon, 11 Nov 2024 16:24:25 +1300 > From: Tao Liu <ltao(a)redhat.com> > Subject: [Crash-utility] Re: [PATCH] gdb bt: multiple stacks support > (x86_64) > To: Alexey Makhalov <alexey.makhalov(a)broadcom.com> > Cc: devel(a)lists.crash-utility.osci.io > Message-ID: > <CAO7dBbWO84Uhw=i6yFOJT-gbQ= > CQ6QTzj-g+tQG9+B4TK-Kw-Q(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi Alexey, > > Thanks a lot for your improvement! This is a feature which I planned > to support for the next crash release, and you have provided a good > start for it. > > I have tried your patch over the weekend, and currently I'm working on > an improvement, and will send it upstream later. > Do you mean you will post an improved patchset based on the current patch to support this feature on X86 64/aarch64/ppc64 arches? Should we ignore the current patch from Alex(no need to review this one)? Thanks Lianbo > > On Sat, Nov 9, 2024 at 2:12 PM Alexey Makhalov > <alexey.makhalov(a)broadcom.com> wrote: > > > > gdb target analyzes only one task at a time and it backtraces > > only straight C stack until end of the stack. If stacks were concatenated > > during exceptions or interrupts, gdb bt will show only the topmost one. > > > > Introduce multiple stacks support in gdb target, which can be observed > > as a different threads from gdb perspective. > > 'gdb info threads' - to see list of in-kenrel stacks to given task. > > 'gdb thread <Id>' - to switch. > > 'gdb bt' - to show it. > > I like the idea, this is really nice! > > > > > Implmentation is machine specific. In x86_64, I use cmd_bt() to add > > additional gdb threads (gdb_add_substack(stack_id) call). Once added, > > gdb will may call machdep->get_current_task_reg() with corresonding > > stack_id (sid: new argument). > > Note: crash 'bt' command must be called for addition threads to appear. > > > I guess this is not very user-friendly, I'm trying to improve that. > > > No threads/stacks support for arm64 and ppc64, x86_64 only. > > > > Example of #GP fault in the kernel caught by SCTP task.. > > > > crash> bt > > PID: 94228 TASK: ffff96a6766a8000 CPU: 31 COMMAND: "SCTP" > > #0 [ffffbb67437e7220] panic at ffffffff99b4f60b > > #1 [ffffbb67437e72c0] die_addr at ffffffff99033650 > > #2 [ffffbb67437e72f0] exc_general_protection at ffffffff99b9194b > > #3 [ffffbb67437e7390] asm_exc_general_protection at ffffffff99c00b47 > > [exception RIP: crypto_aead_encrypt+9] > > RIP: ffffffff995ce269 RSP: ffffbb67437e7440 RFLAGS: 00010246 > > RAX: 0fdd59d2b3d89ecb RBX: 0000000000000000 RCX: 0000000000000c90 > > RDX: ffff96a368508110 RSI: 0000000000000000 RDI: ffff96a348352060 > > RBP: ffffbb67437e7650 R8: 0000000000000001 R9: ffff96a3685080c8 > > R10: ffff96a348351c78 R11: 00000000d5a09e53 R12: 0000000000000008 > > R13: ffff96a348352010 R14: ffff96a348352000 R15: 0000000000000001 > > ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 > > #4 [ffffbb67437e7440] echainiv_encrypt at ffffffffc0ae82c2 [echainiv] > > #5 [ffffbb67437e7658] crypto_aead_encrypt at ffffffff995ce27c > > #6 [ffffbb67437e7668] esp_output_tail at ffffffffc0add3fc [esp4] > > #7 [ffffbb67437e76f8] esp_output at ffffffffc0addedf [esp4] > > #8 [ffffbb67437e7760] xfrm_output_resume at ffffffff99a9186a > > #9 [ffffbb67437e77e0] xfrm_output at ffffffff99a91fba > > #10 [ffffbb67437e7810] __xfrm4_output at ffffffff99a7b0e6 > > #11 [ffffbb67437e7820] xfrm4_output at ffffffff99a7b172 > > #12 [ffffbb67437e7890] ip_local_out at ffffffff99a000ef > > #13 [ffffbb67437e78b8] __ip_queue_xmit at ffffffff99a0028e > > #14 [ffffbb67437e7918] sctp_v4_xmit at ffffffffc0afe0f8 [sctp] > > #15 [ffffbb67437e79f0] sctp_packet_singleton at ffffffffc0b0bc47 [sctp] > > #16 [ffffbb67437e7a60] sctp_outq_flush at ffffffffc0b0c636 [sctp] > > #17 [ffffbb67437e7b08] sctp_outq_uncork at ffffffffc0b0d85c [sctp] > > #18 [ffffbb67437e7b18] sctp_do_sm at ffffffffc0afbaa6 [sctp] > > #19 [ffffbb67437e7d08] __sctp_connect at ffffffffc0b17893 [sctp] > > #20 [ffffbb67437e7d78] __sctp_setsockopt_connectx at ffffffffc0b17a6d > [sctp] > > #21 [ffffbb67437e7da8] sctp_getsockopt at ffffffffc0b1c892 [sctp] > > #22 [ffffbb67437e7eb8] sock_common_getsockopt at ffffffff9993c6e7 > > #23 [ffffbb67437e7ec8] __sys_getsockopt at ffffffff9993afac > > #24 [ffffbb67437e7f18] __x64_sys_getsockopt at ffffffff9993b0bf > > #25 [ffffbb67437e7f28] x64_sys_call at ffffffff99004ca5 > > #26 [ffffbb67437e7f38] do_syscall_64 at ffffffff99b90e34 > > #27 [ffffbb67437e7f50] entry_SYSCALL_64_after_hwframe at > ffffffff99c00126 > > RIP: 00007f12c63028ea RSP: 00007f10e41d9b28 RFLAGS: 00000206 > > RAX: ffffffffffffffda RBX: 0000000000000050 RCX: 00007f12c63028ea > > RDX: 000000000000006f RSI: 0000000000000084 RDI: 0000000000000050 > > RBP: 00007f10a00009b0 R8: 00007f10e41d9b3c R9: 00007f10ac000a5c > > R10: 00007f10e41d9b40 R11: 0000000000000206 R12: 00007f10e41db120 > > R13: 0000000000000050 R14: 0000000000000010 R15: 000000000289e070 > > ORIG_RAX: 0000000000000037 CS: 0033 SS: 002b > > > > crash> gdb bt > > #0 0xffffffff998eaadf in __inb (port=100) at > ./arch/x86/include/asm/shared/io.h:22 > > #1 i8042_read_status () at drivers/input/serio/i8042-acpipnpio.h:54 > > #2 i8042_panic_blink (state=<optimized out>) at > drivers/input/serio/i8042.c:1137 > > #3 0xffffffff99b4f60b in panic (fmt=fmt@entry=0xffffffff9a42c4cb > "Fatal exception") at kernel/panic.c:460 > > #4 0xffffffff99b49b84 in oops_end (flags=<optimized out>, flags@entry=582, > regs=<optimized out>, regs@entry=0xffffbb67437e7398, signr=<optimized > out>) at arch/x86/kernel/dumpstack.c:382 > > #5 0xffffffff99033650 in die_addr (str=str@entry=0xffffbb67437e7304 > "general protection fault, probably for non-canonical address > 0xfdd59d2b3d89edb", regs=regs@entry=0xffffbb67437e7398, err=err@entry=0, > gp_addr=<optimized out>) at arch/x86/kernel/dumpstack.c:462 > > #6 0xffffffff99b9194b in __exc_general_protection (error_code=0, > regs=0xffffbb67437e7398) at arch/x86/kernel/traps.c:784 > > #7 exc_general_protection (regs=0xffffbb67437e7398, error_code=0) at > arch/x86/kernel/traps.c:729 > > #8 0xffffffff99c00b47 in asm_exc_general_protection () at > ./arch/x86/include/asm/idtentry.h:564 > > > > crash> gdb info threads > > Id Target Id Frame > > * 1 94228 SCTP (stack 0) 0xffffffff998eaadf in __inb (port=100) at > ./arch/x86/include/asm/shared/io.h:22 > > 2 94228 SCTP (stack 1) crypto_aead_encrypt (req=req@entry=0xffff96a348352060) > at crypto/aead.c:86 > > The stack(0,1) doesn't make sense for users, I prefer to set marks for > each threads, as the follows: > > crash> info threads > Detaching from process 18613 > Id Target Id Frame > * 1 18336 insmod crash_setup_regs (oldregs=<optimized > out>, newregs=<optimized out>) at ./arch/x86/include/asm/kexec.h:134 > 2 18336 insmod "eframe" 0xffffffffa008300e in ?? () > > So the user will know what kind of stack the thread is... > > Anyway, please wait a while and let's work this out together. > > Thanks, > Tao Liu > > > > > crash> gdb thread 2 > > [Switching to thread 2 (94228 SCTP (stack 1))] > > #0 crypto_aead_encrypt (req=req@entry=0xffff96a348352060) at > crypto/aead.c:86 > > 86 crypto/aead.c: No such file or directory. > > > > crash> gdb bt > > #0 crypto_aead_encrypt (req=req@entry=0xffff96a348352060) at > crypto/aead.c:86 > > #1 0xffffffffc0ae82c2 in echainiv_encrypt (req=0xffff96a348352010) at > crypto/echainiv.c:82 > > #2 0xffffffff995ce27c in crypto_aead_encrypt (req=0xffff96a348352060) > at crypto/aead.c:94 > > #3 0xffffffffc0add3fc in esp_output_tail () > > #4 0xffffffffc0addedf in esp_output () > > #5 0xffffffff99a9186a in xfrm_output_one (err=0, > skb=0xffff96a3c852b300) at net/xfrm/xfrm_output.c:553 > > #6 xfrm_output_resume (sk=sk@entry=0xffff96a348368000, skb=skb@entry=0xffff96a3c852b300, > err=<optimized out>, err@entry=1) at net/xfrm/xfrm_output.c:588 > > #7 0xffffffff99a91fba in xfrm_output2 (skb=0xffff96a3c852b300, > sk=0xffff96a348368000, net=0xffff96a365582580) at net/xfrm/xfrm_output.c:615 > > #8 xfrm_output (sk=0xffff96a348368000, skb=0xffff96a3c852b300) at > net/xfrm/xfrm_output.c:765 > > #9 0xffffffff99a7b0e6 in __xfrm4_output (net=<optimized out>, > sk=<optimized out>, skb=<optimized out>) at net/ipv4/xfrm4_output.c:28 > > #10 0xffffffff99a7b172 in NF_HOOK_COND (pf=2 '\002', hook=4, > okfn=0xffffffff99a7b0c0 <__xfrm4_output>, cond=<optimized out>, > out=0xffff96a496ff2000, in=0x0, skb=0xffff96a3c852b300, > sk=0xffff96a348368000, net=0xffff96a365582580) at > ./include/linux/netfilter.h:291 > > #11 xfrm4_output (net=0xffff96a365582580, sk=0xffff96a348368000, > skb=0xffff96a3c852b300) at net/ipv4/xfrm4_output.c:33 > > #12 0xffffffff99a000ef in dst_output (skb=0xffff96a368508110, sk=0x0, > net=0xffff96a348352060) at ./include/net/dst.h:444 > > #13 ip_local_out (net=0xffff96a348352060, sk=0x0, > skb=0xffff96a368508110) at net/ipv4/ip_output.c:126 > > #14 0xffffffff99a0028e in __ip_queue_xmit (sk=sk@entry=0xffff96a348368000, > skb=skb@entry=0xffff96a3c852b300, fl=fl@entry=0xffff96a348351830, > tos=tos@entry=186 '\272') at net/ipv4/ip_output.c:532 > > #15 0xffffffffc0afe0f8 in sctp_v4_xmit (skb=0xffff96a3c852b300, > t=0xffff96a348351800) at net/sctp/protocol.c:1071 > > #16 0xffffffffc0b1f553 in sctp_packet_transmit (packet=packet@entry=0xffffbb67437e79f8, > gfp=gfp@entry=3264) at net/sctp/output.c:653 > > #17 0xffffffffc0b0bc47 in sctp_packet_singleton (transport=<optimized > out>, chunk=chunk@entry=0xffff96a34c96f500, gfp=3264) at > net/sctp/outqueue.c:783 > > #18 0xffffffffc0b0c636 in sctp_outq_flush_ctrl (ctx=0xffffbb67437e7aa0) > at net/sctp/outqueue.c:914 > > #19 sctp_outq_flush (q=0xffff96a3483585b8, rtx_timeout=rtx_timeout@entry=0, > gfp=<optimized out>) at net/sctp/outqueue.c:1212 > > #20 0xffffffffc0b0d85c in sctp_outq_uncork (q=q@entry=0xffff96a3483585b8, > gfp=gfp@entry=3264) at net/sctp/outqueue.c:764 > > #21 0xffffffffc0afbaa6 in sctp_cmd_interpreter (state=<optimized out>, > status=<optimized out>, gfp=<optimized out>, commands=0xffffbb67437e7b68, > event_arg=<optimized out>, asoc=0xffff96a348358000, ep=<optimized out>, > subtype=..., event_type=<optimized out>) at net/sctp/sm_sideeffect.c:1819 > > #22 sctp_side_effects (gfp=<optimized out>, > commands=0xffffbb67437e7b68, status=<optimized out>, event_arg=<optimized > out>, asoc=<synthetic pointer>, ep=<optimized out>, state=<optimized out>, > subtype=..., event_type=<optimized out>) at net/sctp/sm_sideeffect.c:1199 > > #23 sctp_do_sm (net=<optimized out>, event_type=event_type@entry=SCTP_EVENT_T_PRIMITIVE, > subtype=..., subtype@entry=..., state=<optimized out>, ep=<optimized > out>, asoc=<optimized out>, event_arg=<optimized out>, gfp=<optimized out>) > at net/sctp/sm_sideeffect.c:1170 > > #24 0xffffffffc0b1e2f0 in sctp_primitive_ASSOCIATE (net=<optimized > out>, asoc=asoc@entry=0xffff96a348358000, arg=arg@entry=0x0) at > net/sctp/primitive.c:73 > > #25 0xffffffffc0b17893 in __sctp_connect (sk=sk@entry=0xffff96a348368000, > kaddrs=kaddrs@entry=0xffff96a342085030, addrs_size=addrs_size@entry=16, > flags=2050, assoc_id=assoc_id@entry=0xffffbb67437e7df4) at > ./include/net/net_namespace.h:369 > > #26 0xffffffffc0b17a6d in __sctp_setsockopt_connectx (sk=sk@entry=0xffff96a348368000, > kaddrs=kaddrs@entry=0xffff96a342085030, addrs_size=16, > assoc_id=assoc_id@entry=0xffffbb67437e7df4) at net/sctp/socket.c:1334 > > #27 0xffffffffc0b1c892 in sctp_getsockopt_connectx3 > (optlen=0x7f10e41d9b3c, optval=0x7f10e41d9b40 <error: Cannot access memory > at address 0x7f10e41d9b40>, len=16, sk=0xffff96a348368000) at > net/sctp/socket.c:1419 > > #28 sctp_getsockopt (sk=0xffff96a348368000, level=<optimized out>, > optname=<optimized out>, optval=0x7f10e41d9b40 <error: Cannot access memory > at address 0x7f10e41d9b40>, optlen=<optimized out>) at > net/sctp/socket.c:8124 > > #29 0xffffffff9993c6e7 in sock_common_getsockopt (sock=<optimized out>, > level=0, optname=1750106384, optval=0xc90 <error: Cannot access memory at > address 0xc90>, optlen=0x1) at net/core/sock.c:3652 > > #30 0xffffffff9993afac in __sys_getsockopt (fd=<optimized out>, > level=132, optname=111, optval=0x7f10e41d9b40 <error: Cannot access memory > at address 0x7f10e41d9b40>, optlen=<optimized out>) at net/socket.c:2327 > > #31 0xffffffff9993b0bf in __do_sys_getsockopt (optlen=<optimized out>, > optval=<optimized out>, optname=<optimized out>, level=<optimized out>, > fd=<optimized out>) at net/socket.c:2342 > > #32 __se_sys_getsockopt (optlen=<optimized out>, optval=<optimized > out>, optname=<optimized out>, level=<optimized out>, fd=<optimized out>) > at net/socket.c:2339 > > #33 __x64_sys_getsockopt (regs=<optimized out>) at net/socket.c:2339 > > #34 0xffffffff99004ca5 in x64_sys_call (regs=regs@entry=0xffffbb67437e7f58, > nr=<optimized out>) at ./arch/x86/include/generated/asm/syscalls_64.h:56 > > #35 0xffffffff99b90e34 in do_syscall_x64 (nr=<optimized out>, > regs=0xffffbb67437e7f58) at arch/x86/entry/common.c:51 > > #36 do_syscall_64 (regs=0xffffbb67437e7f58, nr=<optimized out>) at > arch/x86/entry/common.c:81 > > #37 0xffffffff99c00126 in entry_SYSCALL_64 () at > arch/x86/entry/entry_64.S:121 > > > > Now we can use GDB to see the root cause. > > > > Signed-off-by: Alexey Makhalov <alexey.makhalov(a)broadcom.com> > > --- > > arm64.c | 2 +- > > crash_target.c | 25 ++++++++++++++++++---- > > defs.h | 3 ++- > > gdb_interface.c | 6 +++--- > > ppc64.c | 2 +- > > x86_64.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++-- > > 6 files changed, 81 insertions(+), 12 deletions(-) > > > > diff --git a/arm64.c b/arm64.c > > index 608b19d..62f91d8 100644 > > --- a/arm64.c > > +++ b/arm64.c > > @@ -204,7 +204,7 @@ out: > > > > static int > > arm64_get_current_task_reg(int regno, const char *name, > > - int size, void *value) > > + int size, void *value, int unused) > > { > > struct bt_info bt_info, bt_setup; > > struct task_context *tc; > > diff --git a/crash_target.c b/crash_target.c > > index 1080976..8b17ef8 100644 > > --- a/crash_target.c > > +++ b/crash_target.c > > @@ -27,8 +27,9 @@ void crash_target_init (void); > > > > extern "C" int gdb_readmem_callback(unsigned long, void *, int, int); > > extern "C" int crash_get_current_task_reg (int regno, const char > *regname, > > - int regsize, void *val); > > + int regsize, void *val, int sid); > > extern "C" int gdb_change_thread_context (void); > > +extern "C" int gdb_add_substack (int sid); > > extern "C" void crash_get_current_task_info(unsigned long *pid, char > **comm); > > > > /* The crash target. */ > > @@ -64,9 +65,10 @@ public: > > unsigned long pid; > > char *comm; > > crash_get_current_task_info(&pid, &comm); > > - return string_printf ("%ld %s", pid, comm); > > + if (thread_count(this) == 1) > > + return string_printf ("%ld %s", pid, comm); > > + return string_printf ("%ld %s (stack %ld)", pid, comm, ptid.tid()); > > } > > - > > }; > > > > static void supply_registers(struct regcache *regcache, int regno) > > @@ -79,7 +81,7 @@ static void supply_registers(struct regcache > *regcache, int regno) > > if (regsize > sizeof (regval)) > > error (_("fatal error: buffer size is not enough to fit register > value")); > > > > - if (crash_get_current_task_reg (regno, regname, regsize, (void > *)&regval)) > > + if (crash_get_current_task_reg (regno, regname, regsize, (void > *)&regval, inferior_thread()->ptid.tid())) > > regcache->raw_supply (regno, regval); > > else > > regcache->raw_supply (regno, NULL); > > @@ -144,7 +146,22 @@ crash_target_init (void) > > extern "C" int > > gdb_change_thread_context (void) > > { > > + for (thread_info *tp : current_inferior()->threads_safe()) > > + if (tp->ptid.tid_p()) > > + delete_thread (tp); > > target_fetch_registers(get_current_regcache(), -1); > > reinit_frame_cache(); > > return TRUE; > > } > > + > > +/* Add a thread for each additional stack. Use stack ID as a thread ID > */ > > +extern "C" int > > +gdb_add_substack (int sid) > > +{ > > + ptid_t ptid = ptid_t(CRASH_INFERIOR_PID, 0, sid); > > + > > + thread_info *tp = find_thread_ptid (current_inferior(), ptid); > > + if (tp == nullptr) > > + add_thread_silent (current_inferior()->process_target(), ptid); > > + return TRUE; > > +} > > diff --git a/defs.h b/defs.h > > index b93a7a6..bb2bc20 100644 > > --- a/defs.h > > +++ b/defs.h > > @@ -1081,7 +1081,7 @@ struct machdep_table { > > void (*get_irq_affinity)(int); > > void (*show_interrupts)(int, ulong *); > > int (*is_page_ptr)(ulong, physaddr_t *); > > - int (*get_current_task_reg)(int, const char *, int, void *); > > + int (*get_current_task_reg)(int, const char *, int, void *, int); > > int (*is_cpu_prstatus_valid)(int cpu); > > }; > > > > @@ -8301,5 +8301,6 @@ enum ppc64_regnum { > > > > /* crash_target.c */ > > extern int gdb_change_thread_context (void); > > +extern int gdb_add_substack (int sid); > > > > #endif /* !GDB_COMMON */ > > diff --git a/gdb_interface.c b/gdb_interface.c > > index 315711e..c138c94 100644 > > --- a/gdb_interface.c > > +++ b/gdb_interface.c > > @@ -1074,12 +1074,12 @@ unsigned long crash_get_kaslr_offset(void) > > > > /* Callbacks for crash_target */ > > int crash_get_current_task_reg (int regno, const char *regname, > > - int regsize, void *value); > > + int regsize, void *value, int sid); > > int crash_get_current_task_reg (int regno, const char *regname, > > - int regsize, void *value) > > + int regsize, void *value, int sid) > > { > > if (!machdep->get_current_task_reg) > > return FALSE; > > - return machdep->get_current_task_reg(regno, regname, regsize, > value); > > + return machdep->get_current_task_reg(regno, regname, regsize, > value, sid); > > } > > > > diff --git a/ppc64.c b/ppc64.c > > index 782107b..1cf06e3 100644 > > --- a/ppc64.c > > +++ b/ppc64.c > > @@ -2512,7 +2512,7 @@ ppc64_print_eframe(char *efrm_str, struct > ppc64_pt_regs *regs, > > > > static int > > ppc64_get_current_task_reg(int regno, const char *name, int size, > > - void *value) > > + void *value, int unused) > > { > > struct bt_info bt_info, bt_setup; > > struct task_context *tc; > > diff --git a/x86_64.c b/x86_64.c > > index e7f8fe2..2e7cde4 100644 > > --- a/x86_64.c > > +++ b/x86_64.c > > @@ -126,7 +126,7 @@ static int x86_64_get_framesize(struct bt_info *, > ulong, ulong, char *); > > static void x86_64_framesize_debug(struct bt_info *); > > static void x86_64_get_active_set(void); > > static int x86_64_get_kvaddr_ranges(struct vaddr_range *); > > -static int x86_64_get_current_task_reg(int, const char *, int, void *); > > +static int x86_64_get_current_task_reg(int, const char *, int, void *, > int); > > static int x86_64_verify_paddr(uint64_t); > > static void GART_init(void); > > static void x86_64_exception_stacks_init(void); > > @@ -143,6 +143,14 @@ struct machine_specific x86_64_machine_specific = { > 0 }; > > static const char *exception_functions_orig[]; > > static const char *exception_functions_5_8[]; > > > > +/* > > + * Additional stacks entry registers for gdb target. > > + * See 'gdb info threads' > > + */ > > +#define MAX_STACKS_NUM 5 > > +ulong stack_idx; > > +ulong stacks_regs[MAX_STACKS_NUM][SS_REGNUM + 1]; > > + > > /* Use this hardwired version -- sometimes the > > * debuginfo doesn't pick this up even though > > * it exists in the kernel; it shouldn't change. > > @@ -3551,6 +3559,7 @@ x86_64_low_budget_back_trace_cmd(struct bt_info > *bt_in) > > irq_eframe = 0; > > last_process_stack_eframe = 0; > > bt->call_target = NULL; > > + stack_idx = 0; > > rsp = bt->stkptr; > > ms = machdep->machspec; > > > > @@ -4159,6 +4168,7 @@ x86_64_dwarf_back_trace_cmd(struct bt_info *bt_in) > > last_process_stack_eframe = 0; > > bt->call_target = NULL; > > bt->bptr = 0; > > + stack_idx = 0; > > rsp = bt->stkptr; > > if (!rsp) { > > error(INFO, "cannot determine starting stack pointer\n"); > > @@ -4799,6 +4809,36 @@ x86_64_exception_frame(ulong flags, ulong kvaddr, > char *local, > > } else if (machdep->flags & ORC) > > bt->bptr = rbp; > > > > + > > + /* > > + * Preserve registers set for each additional in-kernel stack > > + * up to MAX_STACKS_NUM. > > + */ > > + if (!(cs & 3) && verified && stack_idx < MAX_STACKS_NUM) { > > + stacks_regs[stack_idx][RAX_REGNUM] = rax; > > + stacks_regs[stack_idx][RBX_REGNUM] = rbx; > > + stacks_regs[stack_idx][RCX_REGNUM] = rcx; > > + stacks_regs[stack_idx][RDX_REGNUM] = rdx; > > + stacks_regs[stack_idx][RSI_REGNUM] = rsi; > > + stacks_regs[stack_idx][RDI_REGNUM] = rdi; > > + stacks_regs[stack_idx][RBP_REGNUM] = rbp; > > + stacks_regs[stack_idx][RSP_REGNUM] = rsp; > > + stacks_regs[stack_idx][R8_REGNUM] = r8; > > + stacks_regs[stack_idx][R9_REGNUM] = r9; > > + stacks_regs[stack_idx][R10_REGNUM] = r10; > > + stacks_regs[stack_idx][R11_REGNUM] = r11; > > + stacks_regs[stack_idx][R12_REGNUM] = r12; > > + stacks_regs[stack_idx][R13_REGNUM] = r13; > > + stacks_regs[stack_idx][R14_REGNUM] = r14; > > + stacks_regs[stack_idx][R15_REGNUM] = r15; > > + stacks_regs[stack_idx][RIP_REGNUM] = rip; > > + stacks_regs[stack_idx][EFLAGS_REGNUM] = rflags; > > + stacks_regs[stack_idx][CS_REGNUM] = cs; > > + stacks_regs[stack_idx][SS_REGNUM] = ss; > > + /* Skip stack 0 (main stack), start with index 1 */ > > + gdb_add_substack (stack_idx + 1); > > + stack_idx++; > > + } > > if (kvaddr) > > FREEBUF(pt_regs_buf); > > > > @@ -9236,7 +9276,7 @@ x86_64_get_kvaddr_ranges(struct vaddr_range *vrp) > > > > static int > > x86_64_get_current_task_reg(int regno, const char *name, > > - int size, void *value) > > + int size, void *value, int sid) > > { > > struct bt_info bt_info, bt_setup; > > struct task_context *tc; > > @@ -9256,6 +9296,17 @@ x86_64_get_current_task_reg(int regno, const char > *name, > > if (!tc) > > return FALSE; > > > > + /* Non zero stack ID, use saved regs */ > > + if (sid && sid <= MAX_STACKS_NUM) { > > + switch (regno) { > > + case RAX_REGNUM ... SS_REGNUM: > > + memcpy(value, &stacks_regs[sid - 1][regno], > size > 8 ? 8 : size); > > + return TRUE; > > + default: > > + return FALSE; > > + } > > + } > > + > > /* > > * Task is active, grab CPU's registers > > */ > > -- > > 2.43.5 >

2 weeks, 3 days

3
4
0 / 0

[PATCH v2] "kmem -i" extremely slow on dumps from large memory systems

by Aureau, Georges (Kernel Tools ERT)

The "kmem -i" command is extremely slow (appears to hang) on dumps from large memory systems. For example, on 120GB crash dump from a high-end server with 32TB of RAM (ie. 8Giga Pages), the "kmem -i" command is taking over 50 minutes to execute on a DL380 Gen10. To report basic general memory usage figures, we should only be reading global counters, without having to walk the full flat/sparse mem map page table. Hence, dump_kmeminfo() should first be reading globals, and then only call dump_mem_map() if important information (ie. slabs or total ram) is missing. Signed-off-by: Georges Aureau <georges.aureau(a)hpe.com> -- diff --git a/memory.c b/memory.c index 8c01ed0..eb4574c 100644 --- a/memory.c +++ b/memory.c @@ -8546,18 +8546,23 @@ dump_kmeminfo(void) ulong get_buffers; ulong get_slabs; char buf[BUFSIZE]; + ulong flags; - - BZERO(&meminfo, sizeof(struct meminfo)); - meminfo.flags = GET_ALL; - dump_mem_map(&meminfo); - get_totalram = meminfo.get_totalram; - shared_pages = meminfo.get_shared; - get_buffers = meminfo.get_buffers; - get_slabs = meminfo.get_slabs; + /* + * By default, we will no longer call dump_mem_map() as this is too + * slow for large memory systems. If we have to call it (eg. missing + * important information such as slabs or total ram), we will also + * collect shared pages. Otherwise, we won't print shared pages. + * Shared page + */ + flags = GET_SHARED_PAGES; + shared_pages = 0; + get_totalram = 0; + get_buffers = 0; + get_slabs = 0; /* - * If vm_stat array exists, override page search info. + * If vm_stat array does not exists, then set mem map flag. */ if (vm_stat_init()) { if (dump_vm_stat("NR_SLAB", &nr_slab, 0)) @@ -8572,9 +8577,10 @@ dump_kmeminfo(void) if (dump_vm_stat("NR_SLAB_UNRECLAIMABLE_B", &nr_slab, 0)) get_slabs += nr_slab; } + } else { + flags |= GET_SLAB_PAGES; } - fprintf(fp, "%s", kmeminfo_hdr); /* * Get total RAM based upon how the various versions of si_meminfo() * have done it, latest to earliest: @@ -8586,9 +8592,32 @@ dump_kmeminfo(void) symbol_exists("_totalram_pages")) { totalram_pages = vt->totalram_pages ? vt->totalram_pages : get_totalram; - } else - totalram_pages = get_totalram; + } else { + flags |= GET_TOTALRAM_PAGES; + } + /* + * If we want more than just shared pages (missing slabs or total ram) + * then call dump_mem_map() and also collect buffers. + */ + if (flags != GET_SHARED_PAGES) { + BZERO(&meminfo, sizeof(struct meminfo)); + flags |= GET_BUFFERS_PAGES; + meminfo.flags = flags; + dump_mem_map(&meminfo); + /* Update the missing information */ + if (flags & GET_SLAB_PAGES) { + get_slabs = meminfo.get_slabs; + } + if (flags & GET_TOTALRAM_PAGES) { + get_totalram = meminfo.get_totalram; + totalram_pages = get_totalram; + } + shared_pages = meminfo.get_shared; + get_buffers = meminfo.get_buffers; + } + + fprintf(fp, "%s", kmeminfo_hdr); fprintf(fp, "%13s %7ld %11s ----\n", "TOTAL MEM", totalram_pages, pages_to_size(totalram_pages, buf)); @@ -8613,9 +8642,11 @@ dump_kmeminfo(void) * differently than the kernel -- it just tallies the non-reserved * pages that have a count of greater than 1. */ - pct = (shared_pages * 100)/totalram_pages; - fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", - "SHARED", shared_pages, pages_to_size(shared_pages, buf), pct); + if (shared_pages) { + pct = (shared_pages * 100)/totalram_pages; + fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", + "SHARED", shared_pages, pages_to_size(shared_pages, buf), pct); + } subtract_buffer_pages = 0; if (symbol_exists("buffermem_pages")) { @@ -8634,7 +8665,7 @@ dump_kmeminfo(void) fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", "BUFFERS", buffer_pages, pages_to_size(buffer_pages, buf), pct); - if (CRASHDEBUG(1)) + if (CRASHDEBUG(1)&&(flags&GET_BUFFERS_PAGES)) error(NOTE, "pages with buffers: %ld\n", get_buffers); /*

2 weeks, 4 days

1
0
0 / 0

[PATCH] "kmem -i" extremely slow on dumps from large memory systems

by Aureau, Georges (Kernel Tools ERT)

The "kmem -i" command is extremely slow (appears to hang) on dumps from large memory systems. For example, on 120GB crash dump from a high-end server with 32TB of RAM (ie. 8Giga Pages), the "kmem -i" command is taking over 50 minutes to execute on a DL380 Gen10. To report basic general memory usage figures, we should only be reading global counters, without having to walk the full flat/sparse mem map page table. Hence, dump_kmeminfo() should first be reading globals, and then only call dump_mem_map() if important information (ie. slabs or total ram) is missing. Signed-off-by: Georges Aureau <georges.aureau(a)hpe.com<mailto:georges.aureau@hpe.com>> -- diff --git a/memory.c b/memory.c index 8c01ed0..8e2b2e9 100644 --- a/memory.c +++ b/memory.c @@ -8546,18 +8546,19 @@ dump_kmeminfo(void) ulong get_buffers; ulong get_slabs; char buf[BUFSIZE]; + ulong flags; - - BZERO(&meminfo, sizeof(struct meminfo)); - meminfo.flags = GET_ALL; - dump_mem_map(&meminfo); - get_totalram = meminfo.get_totalram; - shared_pages = meminfo.get_shared; - get_buffers = meminfo.get_buffers; - get_slabs = meminfo.get_slabs; + /* + * By default, we will no longer call dump_mem_map() as this is too + * slow for large memory systems. If we have to call it (eg. missing + * important information such as slabs or total ram), we will also + * collect shared pages. Otherwise, we won't print shared pages. + */ + flags = GET_SHARED_PAGES; + shared_pages = 0; /* - * If vm_stat array exists, override page search info. + * If vm_stat array does not exists, then set mem map flag. */ if (vm_stat_init()) { if (dump_vm_stat("NR_SLAB", &nr_slab, 0)) @@ -8571,10 +8572,11 @@ dump_kmeminfo(void) get_slabs = nr_slab; if (dump_vm_stat("NR_SLAB_UNRECLAIMABLE_B", &nr_slab, 0)) get_slabs += nr_slab; + } else { + flags |= GET_SLAB_PAGES; } } - fprintf(fp, "%s", kmeminfo_hdr); /* * Get total RAM based upon how the various versions of si_meminfo() * have done it, latest to earliest: @@ -8586,9 +8588,32 @@ dump_kmeminfo(void) symbol_exists("_totalram_pages")) { totalram_pages = vt->totalram_pages ? vt->totalram_pages : get_totalram; - } else - totalram_pages = get_totalram; + } else { + flags |= GET_TOTALRAM_PAGES; + } + /* + * If we want more than just shared pages (missing slabs or total ram) + * then call dump_mem_map() and also collect buffers. + */ + if (flags != GET_SHARED_PAGES) { + BZERO(&meminfo, sizeof(struct meminfo)); + flags |= GET_BUFFERS_PAGES; + meminfo.flags = flags; + dump_mem_map(&meminfo); + /* Update the missing information */ + if (flags & GET_SLAB_PAGES) { + get_slabs = meminfo.get_slabs; + } + if (flags & GET_TOTALRAM_PAGES) { + get_totalram = meminfo.get_totalram; + totalram_pages = get_totalram; + } + shared_pages = meminfo.get_shared; + get_buffers = meminfo.get_buffers; + } + + fprintf(fp, "%s", kmeminfo_hdr); fprintf(fp, "%13s %7ld %11s ----\n", "TOTAL MEM", totalram_pages, pages_to_size(totalram_pages, buf)); @@ -8613,9 +8638,11 @@ dump_kmeminfo(void) * differently than the kernel -- it just tallies the non-reserved * pages that have a count of greater than 1. */ - pct = (shared_pages * 100)/totalram_pages; - fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", - "SHARED", shared_pages, pages_to_size(shared_pages, buf), pct); + if (shared_pages) { + pct = (shared_pages * 100)/totalram_pages; + fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", + "SHARED", shared_pages, pages_to_size(shared_pages, buf), pct); + } subtract_buffer_pages = 0; if (symbol_exists("buffermem_pages")) { @@ -8634,7 +8661,7 @@ dump_kmeminfo(void) fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", "BUFFERS", buffer_pages, pages_to_size(buffer_pages, buf), pct); - if (CRASHDEBUG(1)) + if (CRASHDEBUG(1)&&(flags&GET_BUFFERS_PAGES)) error(NOTE, "pages with buffers: %ld\n", get_buffers); /*

2 weeks, 4 days

2
2
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Crash-utility December 2024