On 10/9/24 16:38, Aditya Gupta wrote: > Few vmcores don't have vmcoreinfo elf note, such as those created using > virsh-dump. > > On architectures such as PowerPC64, vmcoreinfo is mandatory to fetch the > first_vmalloc_address, for vmcores of upstream linux, since crash-utility commit: > > commit 5b24e363a898 ("get vmalloc start address from vmcoreinfo") > > Try reading from the 'vmcoreinfo_data' symbol instead, if the vmcoreinfo > crash tries to read in case of diskdump/netdump is empty/missing. > > The approach to read 'vmcoreinfo_data' was used for a live kernel, which can be > reused in the case of missing vmcoreinfo note also, as the > 'vmcoreinfo_data' symbol is available with vmcore too > > Hence rename 'vmcoreinfo_read_string' in kernel.c to > 'vmcoreinfo_read_from_memory', and use it in netdump.c and diskdump.c > too. > > Reported-by: Anushree Mathur <anushree.mathur(a)linux.ibm.com&gt; > Tested-by: Anushree Mathur <anushree.mathur(a)linux.ibm.com&gt; > Signed-off-by: Aditya Gupta <adityag(a)linux.ibm.com&gt; > --- > defs.h | 1 + > diskdump.c | 18 ++++++++++++++++++ > kernel.c | 9 ++++----- > netdump.c | 19 +++++++++++++++++++ > 4 files changed, 42 insertions(+), 5 deletions(-) > > diff --git a/defs.h b/defs.h > index 2231cb68b804..910264e12314 100644 > --- a/defs.h > +++ b/defs.h > @@ -6166,6 +6166,7 @@ void dump_kernel_table(int); > void dump_bt_info(struct bt_info *, char *where); > void dump_log(int); > void parse_kernel_version(char *); > +char *vmcoreinfo_read_from_memory(const char *); > > #define LOG_LEVEL(v) ((v) & 0x07) > #define SHOW_LOG_LEVEL (0x1) > diff --git a/diskdump.c b/diskdump.c > index ce3cbb7b12dd..30d0c87f84c1 100644 > --- a/diskdump.c > +++ b/diskdump.c > @@ -1041,6 +1041,13 @@ pfn_to_pos(ulong pfn) > return desc_pos; > } > > +/** > + * Check if vmcoreinfo in vmcore is missing/empty > + */ > +static bool is_vmcoreinfo_empty(void) > +{ > + return (dd->sub_header_kdump->size_vmcoreinfo == 0); > +} > > /* > * Determine whether a file is a diskdump creation, and if TRUE, > @@ -1088,6 +1095,17 @@ is_diskdump(char *file) > > pc->read_vmcoreinfo = vmcoreinfo_read_string; > > + /* > + * vmcoreinfo can be empty in case of dump collected via virsh-dump > + * > + * check if vmcoreinfo is not available in vmcore, and try to read > + * thev vmcore from memory, using "vmcoreinfo_data" symbol > + */ > + if (is_vmcoreinfo_empty()) { > + error(WARNING, "vmcoreinfo is empty, will read from symbols\n"); > + pc->read_vmcoreinfo = vmcoreinfo_read_from_memory; > + } > + > if ((pc->flags2 & GET_LOG) && KDUMP_CMPRS_VALID()) { > pc->dfd = dd->dfd; > pc->readmem = read_diskdump; > diff --git a/kernel.c b/kernel.c > index adb19ad8725d..7d26a5c5a0a1 100644 > --- a/kernel.c > +++ b/kernel.c > @@ -99,7 +99,6 @@ static ulong dump_audit_skb_queue(ulong); > static ulong __dump_audit(char *); > static void dump_audit(void); > static void dump_printk_safe_seq_buf(int); > -static char *vmcoreinfo_read_string(const char *); > static void check_vmcoreinfo(void); > static int is_pvops_xen(void); > static int get_linux_banner_from_vmlinux(char *, size_t); > @@ -11852,8 +11851,8 @@ dump_printk_safe_seq_buf(int msg_flags) > * Returns a string (that has to be freed by the caller) that contains the > * value for key or NULL if the key has not been found. > */ > -static char * > -vmcoreinfo_read_string(const char *key) > +char * > +vmcoreinfo_read_from_memory(const char *key) > { > char *buf, *value_string, *p1, *p2; > size_t value_length; > @@ -11918,10 +11917,10 @@ check_vmcoreinfo(void) > switch (get_symbol_type("vmcoreinfo_data", NULL, NULL)) > { > case TYPE_CODE_PTR: > - pc->read_vmcoreinfo = vmcoreinfo_read_string; > + pc->read_vmcoreinfo = vmcoreinfo_read_from_memory; > break; > case TYPE_CODE_ARRAY: > - pc->read_vmcoreinfo = vmcoreinfo_read_string; > + pc->read_vmcoreinfo = vmcoreinfo_read_from_memory; > break; > } > } > diff --git a/netdump.c b/netdump.c > index b4e2a5cb2037..c69c7a1e80db 100644 > --- a/netdump.c > +++ b/netdump.c > @@ -111,6 +111,14 @@ map_cpus_to_prstatus(void) > FREEBUF(nt_ptr); > } > > +/** > + * Check if vmcoreinfo in vmcore is missing/empty > + */ > +static bool is_vmcoreinfo_empty(void) > +{ > + return (nd->size_vmcoreinfo == 0); > +} > + > /* > * Determine whether a file is a netdump/diskdump/kdump creation, > * and if TRUE, initialize the vmcore_data structure. > @@ -464,6 +472,17 @@ is_netdump(char *file, ulong source_query) > > pc->read_vmcoreinfo = vmcoreinfo_read_string; > > + /* > + * vmcoreinfo can be empty in case of dump collected via virsh-dump > + * > + * check if vmcoreinfo is not available in vmcore, and try to read > + * thev vmcore from memory, using "vmcoreinfo_data" symbol > + */ > + if (is_vmcoreinfo_empty()) { > + error(WARNING, "vmcoreinfo is empty, will read from symbols\n"); > + pc->read_vmcoreinfo = vmcoreinfo_read_from_memory; > + } > + > if ((source_query == KDUMP_LOCAL) && > (pc->flags2 & GET_OSRELEASE)) > kdump_get_osrelease(); > > > I have tested the patch and it is working perfectly fine for me.Here is my analysis: > *Using crash tool without applying patch*: > > > > # crash /usr/lib/debug/lib/modules/6.11.2-300.fc41.ppc64le/vmlinux ./sample.dump > > > > crash 8.0.5-4.fc41 > > Copyright (C) 2002-2024 Red Hat, Inc. > > Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation > > Copyright (C) 1999-2006 Hewlett-Packard Co > > Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited > > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. > > Copyright (C) 2005, 2011, 2020-2024 NEC Corporation > > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. > > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. > > Copyright (C) 2015, 2021 VMware, Inc. > > This program is free software, covered by the GNU General Public License, > > and you are welcome to change it and/or distribute copies of it under > > certain conditions. Enter "help copying" to see the conditions. > > This program has absolutely no warranty. Enter "help warranty" for details. > > > GNU gdb (GDB) 10.2 > > Copyright (C) 2021 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. > > Type "show copying" and "show warranty" for details. > > This GDB was configured as "powerpc64le-unknown-linux-gnu". > > Type "show configuration" for configuration details. > > Find the GDB manual and other documentation resources online at: > > <http://www.gnu.org/software/gdb/documentation/>;. > > > > For help, type "help". > > Type "apropos word" to search for commands related to "word"... > > > > crash: invalid kernel virtual address: 41000e41df41de49 type: "first vmlist addr" > > > > Errors like the one above typically occur when the kernel and memory source > > do not match. These are the files being used: > > > > KERNEL: /usr/lib/debug/lib/modules/6.11.2-300.fc41.ppc64le/vmlinux > > DUMPFILE: ./sample.dump > > > > > > > *Using crash tool after applying patch*: > > > > # /root/crash/crash /usr/lib/debug/lib/modules/6.11.2-300.fc41.ppc64le/vmlinux ./sample.dump > > > > crash 8.0.5++ > > Copyright (C) 2002-2024 Red Hat, Inc. > > Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation > > Copyright (C) 1999-2006 Hewlett-Packard Co > > Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited > > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. > > Copyright (C) 2005, 2011, 2020-2024 NEC Corporation > > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. > > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. > > Copyright (C) 2015, 2021 VMware, Inc. > > This program is free software, covered by the GNU General Public License, > > and you are welcome to change it and/or distribute copies of it under > > certain conditions. Enter "help copying" to see the conditions. > > This program has absolutely no warranty. Enter "help warranty" for details. > > > WARNING: vmcoreinfo is empty, will read from symbols > > GNU gdb (GDB) 10.2 > > Copyright (C) 2021 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. > > Type "show copying" and "show warranty" for details. > > This GDB was configured as "powerpc64le-unknown-linux-gnu". > > Type "show configuration" for configuration details. > > Find the GDB manual and other documentation resources online at: > > <http://www.gnu.org/software/gdb/documentation/>;. > > > > For help, type "help". > > Type "apropos word" to search for commands related to "word"... > > > > KERNEL: /usr/lib/debug/lib/modules/6.11.2-300.fc41.ppc64le/vmlinux > > DUMPFILE: ./sample.dump [PARTIAL DUMP] > > CPUS: 16 > > DATE: Mon Oct 7 01:18:14 CDT 2024 > > UPTIME: 01:04:55 > > LOAD AVERAGE: 31.05, 10.52, 3.81 > > TASKS: 434 > > NODENAME: sample.aus.stglabs.ibm.com > > RELEASE: 6.11.2-300.fc41.ppc64le > > VERSION: #1 SMP Fri Oct 4 16:30:24 UTC 2024 > > MACHINE: ppc64le (2750 Mhz) > > MEMORY: 260 GB > > PANIC: "" > > PID: 0 > > COMMAND: "swapper/0" > > TASK: c000000003ca5380 (1 of 16) [THREAD_INFO: c000000003ca5380] > > CPU: 0 > > STATE: TASK_RUNNING > > WARNING: panic task not found > > > > crash> bt > > PID: 0 TASK: c000000003ca5380 CPU: 0 COMMAND: "swapper/0" > > #0 [c000000003ce3aa0] init_task at c000000003ca5380 (unreliable) > > #1 [c000000003ce3c50] __switch_to at c000000000020500 > > #2 [c000000003ce3cc0] __schedule at c0000000015999b0 > > #3 [c000000003ce3e00] schedule_idle at c00000000159af4c > > #4 [c000000003ce3e30] do_idle at c00000000020c2ec > > #5 [c000000003ce3ea0] cpu_startup_entry at c00000000020c700 > > #6 [c000000003ce3ed0] rest_init at c0000000000113a4 > > #7 [c000000003ce3f10] console_on_rootfs at c00000000300f204 > > #8 [c000000003ce3fe0] start_here_common at c00000000000e998 > > crash> > > > > I have tried the same for different kernel versions also and this patch worked. > > Thanks, > > Anushree Mathur >

The patch LGTM, so ack. Thanks, Tao Liu On Thu, Oct 10, 2024 at 12:08 AM Aditya Gupta <adityag(a)linux.ibm.com&gt; wrote:

Hi, Aditya Thank you for the fix. On Wed, Oct 9, 2024 at 7:08 PM Aditya Gupta <adityag(a)linux.ibm.com&gt; wrote: I tried to generate a core dump file with "virsh dump" and got an error message: # ./crash vmlinux ../coredump crash 8.0.6 Copyright (C) 2002-2025 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011, 2020-2024 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. Copyright (C) 2015, 2021 VMware, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. WARNING: --vmcoreinfo is empty, will read from symbols crash: gdb_interface: gdb not initialized? Could you please double check? BTW: the kernel version is 6.12, I tested it based on aarch64 machine. Thanks Lianbo On architectures such as PowerPC64, vmcoreinfo is mandatory to fetch the