2:09 a.m.
Hi, Lianbo
> In addition, I just got a dump file generated by the "virsh
dump" command, the dumpfile doesn't have the vmcoreinfo, and get the same errors
> with or without this patch, can you also double check? Maybe it is a similar issue.
>
Ok, let me check further.
and can you tell me what kernel versin and linux distribution you are used?
> [root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep
vabits_actual=48
I used ubuntu-18.04-arm64 to reproduce your issue,kernel version
is:4.15.18, in this version,arm64 still not support
vabits_actual(5383cc6efed1 arm64: mm: Introduce vabits_actual),so we
can't use "--machdep vabits_actual=48", after remove "--machdep
vabits_actual=48",crash-utility can work fine.
qianli zhao <zhaoqianligood(a)gmail.com> 于2022年6月30日周四 13:41写道：
>
> Hi, Lianbo
>
> >
> > On Wed, Jun 29, 2022 at 2:40 PM qianli zhao <zhaoqianligood(a)gmail.com>
wrote:
> >>
> >> Hi,Lianbo
> >> Thank you for your feedback.
> >>
> >> My kernel version is 5.10.59, and this issue is happened when the
> >>
> >> ramdumps without vmcoreinfo.
> >
> >
> > Thank you for the explanation.
> >
> > It is important to point this(*without vmcoreinfo*) out in the patch log. That
can help understand this issue. Can you add it to the patch log?
> sure, will update in next patchset
>
> In addition, I just got a dump file generated by the "virsh
dump" command, the dumpfile doesn't have the vmcoreinfo, and get the same errors
> with or without this patch, can you also double check? Maybe it is a similar issue.
>
Ok, let me check further.
and can you tell me what kernel versin and linux distribution you are used?
> [root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep
vabits_actual=48
> >
>
> lijiang <lijiang(a)redhat.com> 于2022年6月29日周三 17:31写道：
> >
> > On Wed, Jun 29, 2022 at 2:40 PM qianli zhao <zhaoqianligood(a)gmail.com>
wrote:
> >>
> >> Hi,Lianbo
> >> Thank you for your feedback.
> >>
> >> My kernel version is 5.10.59, and this issue is happened when the
> >>
> >> ramdumps without vmcoreinfo.
> >
> >
> > Thank you for the explanation.
> >
> > It is important to point this(*without vmcoreinfo*) out in the patch log. That
can help understand this issue. Can you add it to the patch log?
> >
> > In addition, I just got a dump file generated by the "virsh dump"
command, the dumpfile doesn't have the vmcoreinfo, and get the same errors
> > with or without this patch, can you also double check? Maybe it is a similar
issue.
> >
> > [root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep
vabits_actual=48
> >
> > crash 8.0.1++
> > Copyright (C) 2002-2022 Red Hat, Inc.
> > Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
> > Copyright (C) 1999-2006 Hewlett-Packard Co
> > Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
> > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> > Copyright (C) 2005, 2011, 2020-2022 NEC Corporation
> > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > Copyright (C) 2015, 2021 VMware, Inc.
> > This program is free software, covered by the GNU General Public License,
> > and you are welcome to change it and/or distribute copies of it under
> > certain conditions. Enter "help copying" to see the conditions.
> > This program has absolutely no warranty. Enter "help warranty" for
details.
> >
> > NOTE: setting vabits_actual to: 48
> >
> > WARNING: kimage_voffset cannot be determined from the dumpfile.
> > Try using the command line option: --machdep kimage_voffset=<addr>
> > GNU gdb (GDB) 10.2
> > Copyright (C) 2021 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law.
> > Type "show copying" and "show warranty" for details.
> > This GDB was configured as "aarch64-unknown-linux-gnu".
> > Type "show configuration" for configuration details.
> > Find the GDB manual and other documentation resources online at:
> > <http://www.gnu.org/software/gdb/documentation/>;.
> >
> > For help, type "help".
> > Type "apropos word" to search for commands related to
"word"...
> >
> > crash: read error: kernel virtual address: ffff8000119cee00 type:
"possible"
> > WARNING: cannot read cpu_possible_map
> > crash: read error: kernel virtual address: ffff8000119cf208 type:
"present"
> > WARNING: cannot read cpu_present_map
> > crash: read error: kernel virtual address: ffff8000119cec00 type:
"online"
> > WARNING: cannot read cpu_online_map
> > crash: read error: kernel virtual address: ffff8000119cf408 type:
"active"
> > WARNING: cannot read cpu_active_map
> > crash: read error: kernel virtual address: ffff8000120c97a8 type:
"shadow_timekeeper xtime_sec"
> > crash: read error: kernel virtual address: ffff8000119e1668 type:
"init_uts_ns"
> > crash: vmlinux and dump.core do not match!
> >
> > Usage:
> >
> > crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form)
> > crash [OPTION]... [NAMELIST] (live system form)
> >
> > Enter "crash -h" for details.
> >
> > Thanks.
> > Lianbo
> >
> >> I guess your scene may have vmcore,the following logic helps to
> >> correctly locate MODULES/VMALLOC ranges with right kernel version.
> >>
> >> 279 ms->kimage_end = (sp ? sp->value : 0);
> >> 280
> >> 281 if (ms->struct_page_size && (r =
> >> arm64_get_va_range(ms))) {------------->//If there is a vmcore, it
> >> will go to the following codes
> >> 282 /* We can get all the
> >> MODULES/VMALLOC/VMEMMAP ranges now.*/
> >> 283 ms->modules_vaddr =
r->modules_vaddr;
> >> 284 ms->modules_end =
> >> r->modules_end - 1;
> >> 285 ms->vmalloc_start_addr =
> >> r->vmalloc_start_addr;
> >> 286 ms->vmalloc_end =
> >> r->vmalloc_end - 1;
> >> 287 ms->vmemmap_vaddr =
r->vmemmap_vaddr;
> >> 288 ms->vmemmap_end =
> >> r->vmemmap_end - 1;
> >> 289 } else if (ms->VA_BITS_ACTUAL)
> >> {------------>//unable to locate the version and other information
> >> through vmcore
> >> }
> >>
> >> lijiang <lijiang(a)redhat.com> 于2022年6月29日周三 14:21写道：
> >> >
> >> > Hi, Qianli
> >> > On Tue, Jun 28, 2022 at 9:55 AM lijiang <lijiang(a)redhat.com>
wrote:
> >> >>
> >> >> Hi, Kazu and Qianli
> >> >>
> >> >> On Tue, Jun 28, 2022 at 9:17 AM HAGIO KAZUHITO(萩尾 一仁)
<k-hagio-ab(a)nec.com> wrote:
> >> >>>
> >> >>> Hi Qianli,
> >> >>>
> >> >>> thanks for the patch and explanation. I was off.
> >> >>>
> >> >>> On 2022/06/27 11:24, qianli zhao wrote:
> >> >>> > Hi,Kazu
> >> >>> >
> >> >>> > Would you like to help review this patch?
> >> >>>
> >> >>> Sure, I think I can review it this week.
> >> >>>
> >> >>> Lianbo, can you possibly reproduce and test this?
> >> >>
> >> >>
> >> >> OK, I will have a look and give feedback later.
> >> >
> >> >
> >> > Could you please point out the kernel version? I tried it with the
latest kernel and did not reproduce this issue when disabling the kaslr feature(#
CONFIG_RANDOMIZE_BASE is not set or nokaslr)
> >> >
> >> > crash > help -m
> >> > ..
> >> > vmalloc_start_addr: ffff800008000000
> >> > vmalloc_end: fffffbffefffffff
> >> > modules_vaddr: ffff800000000000
> >> > modules_end: ffff800007ffffff
> >> > vmemmap_vaddr: fffffc0000000000
> >> > vmemmap_end: fffffdffffffffff
> >> > ...
> >> >
> >> > And the following information is dumped from the kernel(commit:
941e3e791269)
> >> > ...
> >> > [ 0.000000] Virtual kernel memory layout:
> >> > [ 0.000000] modules : 0xffff800000000000 - 0xffff800008000000
( 128 MB)
> >> > [ 0.000000] vmalloc : 0xffff800008000000 - 0xfffffbfff0000000
(126975 GB)
> >> > ...
> >> > [ 0.000000] vmemmap : 0xfffffc0000000000 - 0xfffffe0000000000
( 2048 GB maximum)
> >> > [ 0.000000] 0xfffffc000000bc00 - 0xfffffc023df40000
( 9183 MB actual)
> >> >
> >> > I'm wondering if that can be only reproduced on the old kernel,
right? Or did I miss anything else?
> >> >
> >> > Thanks.
> >> > Lianbo
> >> >
> >> >>
> >> >>>
> >> >>> Kazu
> >> >>>
> >> >>> >
> >> >>> > qianli zhao <zhaoqianligood(a)gmail.com> 于2022年6月24日周五
10:56写道：
> >> >>> >
> >> >>> >>
> >> >>> >> Hi,all
> >> >>> >>
> >> >>> >> Here's some explanation for this patch
> >> >>> >>
> >> >>> >> Without patch:
> >> >>> >> Consider the following scenario
> >> >>> >> ->arm64_init(PRE_GDB)
> >> >>> >> case PRE_GDB:
> >> >>> >> ...
> >> >>> >> 292 } else if
(ms->VA_BITS_ACTUAL) {
> >> >>> >> 293
ms->modules_vaddr =
> >> >>> >> (st->_stext_vmlinux & TEXT_OFFSET_MASK) -
> >> >>> >>
ARM64_MODULES_VSIZE;-->//ms->modules_vaddr=0xfffffffff8000000
> >> >>> >> 294
ms->modules_end =
> >> >>> >> ms->modules_vaddr + ARM64_MODULES_VSIZE
> >> >>> >> -1;--->//ms->modules_end=0xffffffffffffffff
> >> >>> >> 295
ms->vmalloc_start_addr =
> >> >>> >> ms->modules_end +
1;--->//ms->vmalloc_start_addr=0
> >> >>> >> 296 } else {
> >> >>> >> ....
> >> >>> >> }
> >> >>> >>
arm64_calc_kimage_voffset();
> >> >>> >> .....
> >> >>> >>
> >> >>> >> Since arm64_calc_kimage_voffset() depends on
vmalloc_start_addr,
> >> >>> >> kimage_voffset cannot be calculated correctly.
> >> >>> >>
> >> >>> >> st->_stext_vmlinux can be initialized in
numeric_forward(),just set
> >> >>> >> st->_stext_vmlinux to UNINITIALIZED.
> >> >>> >>
> >> >>> >> ============
> >> >>> >> log as below:
> >> >>> >>
> >> >>> >> $ ~/crash/crash/crash vmlinux DDRCS0.bin@0x80000000
--machdep vabits_actual=48
> >> >>> >>
> >> >>> >> crash 8.0.1++
> >> >>> >> Copyright (C) 2002-2022 Red Hat, Inc.
> >> >>> >> Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
> >> >>> >> Copyright (C) 1999-2006 Hewlett-Packard Co
> >> >>> >> Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
> >> >>> >> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> >> >>> >> Copyright (C) 2005, 2011, 2020-2022 NEC Corporation
> >> >>> >> Copyright (C) 1999, 2002, 2007 Silicon Graphics,
Inc.
> >> >>> >> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical
Linux, Inc.
> >> >>> >> Copyright (C) 2015, 2021 VMware, Inc.
> >> >>> >> This program is free software, covered by the GNU
General Public License,
> >> >>> >> and you are welcome to change it and/or distribute
copies of it under
> >> >>> >> certain conditions. Enter "help copying" to
see the conditions.
> >> >>> >> This program has absolutely no warranty. Enter
"help warranty" for details.
> >> >>> >>
> >> >>> >> NOTE: setting vabits_actual to: 48
> >> >>> >>
> >> >>> >> WARNING: kimage_voffset cannot be determined from the
dumpfile.
> >> >>> >> Try using the command line option: --machdep
kimage_voffset=<addr>
> >> >>> >> GNU gdb (GDB) 10.2
> >> >>> >> Copyright (C) 2021 Free Software Foundation, Inc.
> >> >>> >> License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
> >> >>> >> This is free software: you are free to change and
redistribute it.
> >> >>> >> There is NO WARRANTY, to the extent permitted by law.
> >> >>> >> Type "show copying" and "show
warranty" for details.
> >> >>> >> This GDB was configured as
"--host=x86_64-pc-linux-gnu
> >> >>> >> --target=aarch64-elf-linux".
> >> >>> >> Type "show configuration" for configuration
details.
> >> >>> >> Find the GDB manual and other documentation resources
online at:
> >> >>> >>
<http://www.gnu.org/software/gdb/documentation/>;.
> >> >>> >>
> >> >>> >> For help, type "help".
> >> >>> >> Type "apropos word" to search for commands
related to "word"...
> >> >>> >>
> >> >>> >> crash: read error: kernel virtual address:
ffff80001083d4a0 type:
> >> >>> >> "kernel_config_data"
> >> >>> >> WARNING: cannot read kernel_config_data
> >> >>> >> crash: read error: kernel virtual address:
ffff80001170e798 type: "possible"
> >> >>> >> WARNING: cannot read cpu_possible_map
> >> >>> >> crash: read error: kernel virtual address:
ffff80001170e7a8 type: "present"
> >> >>> >> WARNING: cannot read cpu_present_map
> >> >>> >> crash: read error: kernel virtual address:
ffff80001170e788 type: "online"
> >> >>> >> WARNING: cannot read cpu_online_map
> >> >>> >> crash: read error: kernel virtual address:
ffff80001170e7c0 type: "active"
> >> >>> >> WARNING: cannot read cpu_active_map
> >> >>> >> crash: read error: kernel virtual address:
ffff8000122e00f0 type:
> >> >>> >> "shadow_timekeeper xtime_sec"
> >> >>> >> crash: read error: kernel virtual address:
ffff80001171dc04 type: "init_uts_ns"
> >> >>> >> crash: vmlinux and /var/tmp/ramdump_elf_m2ivkg do not
match!
> >> >>> >>
> >> >>> >> Usage:
> >> >>> >>
> >> >>> >> crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS]
(dumpfile form)
> >> >>> >> crash [OPTION]... [NAMELIST]
(live system form)
> >> >>> >>
> >> >>> >> Enter "crash -h" for details.
> >> >>> >>
> >> >>> >> Qianli Zhao <zhaoqianligood(a)gmail.com>
于2022年6月24日周五 00:14写道：
> >> >>> >>>
> >> >>> >>> From: Qianli Zhao <qianli.zhao(a)horizon.ai>
> >> >>> >>>
> >> >>> >>> Setting st->_stext_vmlinux to UNINITIALIZED to
search for "_stext" from the vmlinux
> >> >>> >>> Without the patch, if we do not enable kaslr, will
get the wrong
> >> >>> >>> MODULES/VMALLOC ranges, cause parsing dump
failure
> >> >>> >>>
> >> >>> >>> Signed-off-by: Qianli Zhao
<qianli.zhao(a)horizon.ai>
> >> >>> >>> ---
> >> >>> >>> arm64.c | 3 +++
> >> >>> >>> 1 file changed, 3 insertions(+)
> >> >>> >>>
> >> >>> >>> diff --git a/arm64.c b/arm64.c
> >> >>> >>> index 0f615cf..4458a66 100644
> >> >>> >>> --- a/arm64.c
> >> >>> >>> +++ b/arm64.c
> >> >>> >>> @@ -149,6 +149,9 @@ arm64_init(int when)
> >> >>> >>>
> >> >>> >>> ms = machdep->machspec;
> >> >>> >>>
> >> >>> >>> + if (ms->VA_BITS_ACTUAL)
> >> >>> >>> + st->_stext_vmlinux =
UNINITIALIZED;
> >> >>> >>> +
> >> >>> >>> if (!ms->kimage_voffset
&& STREQ(pc->live_memsrc, "/dev/crash"))
> >> >>> >>> ioctl(pc->mfd,
DEV_CRASH_ARCH_DATA, &ms->kimage_voffset);
> >> >>> >>>
> >> >>> >>> --
> >> >>> >>> 2.17.1
> >> >>> >>>
> >>