3:09 p.m.
Hello
I am getting the following error when I use crash on arm64 platform (NXP LS1043A, with 4
A53 cores).
crash: read error: kernel virtual address: ffff00007b616100 type: "memory section
root table"
Additional information is that when I boot the kernel with nokaslr in bootargs, the crash
seems to work well.
Have any one else seen similar issue?
The kernel version is top of mainline with the below patch applied:
- arm64/crash_core: Export TCR_EL1.T1SZ in vmcoreinfo
(http://lists.infradead.org/pipermail/kexec/2019-November/023962.html)
And the crash-utils is from here: https://github.com/crash-utility/crash.git
commit 5e975dd8c817ea6aea35e1e15b83c378aee9c136
Author: Dave Anderson <anderson(a)redhat.com>
Date: Tue Dec 24 08:43:52 2019 -0500
When determining the ARM64 kernel's "vabits_actual" value by reading
the new TCR_EL1.T1SZ vmcoreinfo entry, display its value during
session initialization only when invoking crash with "-d1" or larger
-d debug value.
(anderson(a)redhat.com)
Logs of crash -d1 are below:
nxa19049@lsv03080:~/data/ups/crash$ ./crash -d1 ../linux/vmlinux ../linux/vmcore
crash 7.2.7++
Copyright (C) 2002-2019 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
vmcore_data:
flags: c0 (KDUMP_LOCAL|KDUMP_ELF64)
ndfd: 3
ofp: 7fa9998cf620
header_size: 8192
num_pt_load_segments: 4
pt_load_segment[0]:
file_offset: 2000
phys_start: 82080000
phys_end: 839bc000
zero_fill: 0
pt_load_segment[1]:
file_offset: 193e000
phys_start: 80000000
phys_end: d8000000
zero_fill: 0
pt_load_segment[2]:
file_offset: 5993e000
phys_start: fb000000
phys_end: fb800000
zero_fill: 0
pt_load_segment[3]:
file_offset: 5a13e000
phys_start: fbc00000
phys_end: fbe00000
zero_fill: 0
elf_header: 2724410
elf32: 0
notes32: 0
load32: 0
elf64: 2724410
notes64: 2724450
load64: 2724488
sect0_64: 0
nt_prstatus: 2725410
nt_prpsinfo: 0
nt_taskstruct: 0
task_struct: 0
arch_data1: (unused)
arch_data2: (unused)
switch_stack: 0
page_size: 4096
xen_kdump_data: (unused)
num_prstatus_notes: 4
num_qemu_notes: 0
vmcoreinfo: 2725a98
size_vmcoreinfo: 1939
nt_prstatus_percpu:
0000000002725410 00000000027255ac 0000000002725748 00000000027258e4
nt_qemu_percpu:
backup_src_start: 0
backup_src_size: 0
backup_offset: 0
Elf64_Ehdr:
e_ident: \177ELF
e_ident[EI_CLASS]: 2 (ELFCLASS64)
e_ident[EI_DATA]: 1 (ELFDATA2LSB)
e_ident[EI_VERSION]: 1 (EV_CURRENT)
e_ident[EI_OSABI]: 0 (ELFOSABI_SYSV)
e_ident[EI_ABIVERSION]: 0
e_type: 4 (ET_CORE)
e_machine: 183 (EM_AARCH64)
e_version: 1 (EV_CURRENT)
e_entry: 0
e_phoff: 40
e_shoff: 0
e_flags: 0
e_ehsize: 40
e_phentsize: 38
e_phnum: 5
e_shentsize: 0
e_shnum: 0
e_shstrndx: 0
Elf64_Phdr:
p_type: 4 (PT_NOTE)
p_offset: 4096 (1000)
p_vaddr: 0
p_paddr: 0
p_filesz: 3612 (e1c)
p_memsz: 3612 (e1c)
p_flags: 0 ()
p_align: 0
Elf64_Phdr:
p_type: 1 (PT_LOAD)
p_offset: 8192 (2000)
p_vaddr: ffffd3ebfa480000
p_paddr: 82080000
p_filesz: 26460160 (193c000)
p_memsz: 26460160 (193c000)
p_flags: 7 (PF_X|PF_W|PF_R)
p_align: 0
Elf64_Phdr:
p_type: 1 (PT_LOAD)
p_offset: 26468352 (193e000)
p_vaddr: ffff000000000000
p_paddr: 80000000
p_filesz: 1476395008 (58000000)
p_memsz: 1476395008 (58000000)
p_flags: 7 (PF_X|PF_W|PF_R)
p_align: 0
Elf64_Phdr:
p_type: 1 (PT_LOAD)
p_offset: 1502863360 (5993e000)
p_vaddr: ffff00007b000000
p_paddr: fb000000
p_filesz: 8388608 (800000)
p_memsz: 8388608 (800000)
p_flags: 7 (PF_X|PF_W|PF_R)
p_align: 0
Elf64_Phdr:
p_type: 1 (PT_LOAD)
p_offset: 1511251968 (5a13e000)
p_vaddr: ffff00007bc00000
p_paddr: fbc00000
p_filesz: 2097152 (200000)
p_memsz: 2097152 (200000)
p_flags: 7 (PF_X|PF_W|PF_R)
p_align: 0
Elf64_Nhdr:
n_namesz: 5 ("CORE")
n_descsz: 392
n_type: 1 (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 0 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
X0: 00000000000000e0 X1: ffff00007b5c4d80 X2:
ffff
2c147fdf5000
X3: 000000000007b688 X4: 0000000000000015 X5:
00ff
ffffffffffff
X6: 0000000035f70457 X7: 0000000000000000 X8:
ffff
00007b5c3e3c
X9: 00000000000041b3 X10: ffff00007b5c3e1c X11:
0000
0000000325d3
X12: 00000000000325d3 X13: 071c71c71c71c71c X14:
0000
000000000000
X15: 0000000000000000 X16: 0000000000000000 X17:
0000
000000000000
X18: 0000000000000000 X19: 0000001e69146f40 X20:
ffff
000053175800
X21: 0000000000000001 X22: ffff00007b5c3b00 X23:
0000
000000000001
X24: 0000001e633ee5f0 X25: ffff00007b5c3b00 X26:
ffff
000053175800
X27: 0000000000000001 X28: 0000000082c16344 X29:
ffff
d3ebfbb73e50
LR: ffffd3ebfadbbab0 SP: ffffd3ebfbb73e50 PC:
ffff
d3ebfadbbab8
PSTATE: 60000005 FPVALID: 00000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
00000000000000e0 ffff00007b5c4d80
ffff2c147fdf5000 000000000007b688
0000000000000015 00ffffffffffffff
0000000035f70457 0000000000000000
ffff00007b5c3e3c 00000000000041b3
ffff00007b5c3e1c 00000000000325d3
00000000000325d3 071c71c71c71c71c
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000001e69146f40
ffff000053175800 0000000000000001
ffff00007b5c3b00 0000000000000001
0000001e633ee5f0 ffff00007b5c3b00
ffff000053175800 0000000000000001
0000000082c16344 ffffd3ebfbb73e50
ffffd3ebfadbbab0 ffffd3ebfbb73e50
ffffd3ebfadbbab8 0000000060000005
0000000000000000
Elf64_Nhdr:
n_namesz: 5 ("CORE")
n_descsz: 392
n_type: 1 (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 0 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
X0: 00000000000000e0 X1: ffff00007b5dad80 X2:
ffff
2c147fe0b000
X3: 000000000007b688 X4: 0000000000000015 X5:
00ff
ffffffffffff
X6: 0000000035f70457 X7: 0000000000000000 X8:
ffff
00007b5d9e3c
X9: 0000000000000db6 X10: ffff00007b5d9e1c X11:
0000
000000085aa2
X12: 0000000000085aa2 X13: 071c71c71c71c71c X14:
0000
000000000000
X15: 0000000000000000 X16: 0000000000000000 X17:
0000
000000000000
X18: 0000000000000000 X19: 0000001e69147148 X20:
ffff
000053176000
X21: 0000000000000001 X22: ffff00007b5d9b00 X23: 0000
000000000001
X24: 0000001e599e8230 X25: ffff00007b5d9b00 X26:
ffff
000053176000
X27: 0000000000000001 X28: 0000000000000000 X29:
ffff
800010083ec0
LR: ffffd3ebfadbbab0 SP: ffff800010083ec0 PC:
ffff
d3ebfadbbab8
PSTATE: 60000005 FPVALID: 00000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
00000000000000e0 ffff00007b5dad80
ffff2c147fe0b000 000000000007b688
0000000000000015 00ffffffffffffff
0000000035f70457 0000000000000000
ffff00007b5d9e3c 0000000000000db6
ffff00007b5d9e1c 0000000000085aa2
0000000000085aa2 071c71c71c71c71c
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000001e69147148
ffff000053176000 0000000000000001
ffff00007b5d9b00 0000000000000001
0000001e599e8230 ffff00007b5d9b00
ffff000053176000 0000000000000001
0000000000000000 ffff800010083ec0
ffffd3ebfadbbab0 ffff800010083ec0
ffffd3ebfadbbab8 0000000060000005
0000000000000000
Elf64_Nhdr:
n_namesz: 5 ("CORE")
n_descsz: 392
n_type: 1 (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 0 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
X0: 00000000000000e0 X1: ffff00007b5f0d80 X2:
ffff
2c147fe21000
X3: 000000000007b688 X4: 0000000000000015 X5:
00ff
ffffffffffff
X6: 0000000035f70457 X7: 0000000000000000 X8:
ffff
00007b5efe3c
X9: ffff80001008beb0 X10: ffff00007b5efe1c X11:
0000
00000000466e
X12: 000000000000466e X13: 071c71c71c71c71c X14:
0000
000000000000
X15: 0000000000000000 X16: 0000000000000000 X17:
0000
000000000000
X18: 0000000000000000 X19: 0000001e69147148 X20:
ffff
000053176800
X21: 0000000000000001 X22: ffff00007b5efb00 X23:
0000
000000000001
X24: 0000001e68f584b8 X25: ffff00007b5efb00 X26:
ffff
000053176800
X27: 0000000000000001 X28: 0000000000000000 X29:
ffff
80001008bec0
LR: ffffd3ebfadbbab0 SP: ffff80001008bec0 PC:
ffff
d3ebfadbbab8
PSTATE: 60000005 FPVALID: 00000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
00000000000000e0 ffff00007b5f0d80
ffff2c147fe21000 000000000007b688
0000000000000015 00ffffffffffffff
0000000035f70457 0000000000000000
ffff00007b5efe3c ffff80001008beb0
ffff00007b5efe1c 000000000000466e
000000000000466e 071c71c71c71c71c
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000001e69147148
ffff000053176800 0000000000000001
ffff00007b5efb00 0000000000000001
0000001e68f584b8 ffff00007b5efb00
ffff000053176800 0000000000000001
0000000000000000 ffff80001008bec0
ffffd3ebfadbbab0 ffff80001008bec0
ffffd3ebfadbbab8 0000000060000005
0000000000000000
Elf64_Nhdr:
n_namesz: 5 ("CORE")
n_descsz: 392
n_type: 1 (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 884 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
X0: ffff00004de23400 X1: 0000000000000000 X2:
ffff
00004f58c600
X3: 0000000000000000 X4: ffffd3ebfbb954b8 X5:
ffff
00004f58c600
X6: 0000000000000001 X7: ffff8000102fbbe0 X8:
0000
000000000146
X9: 00000000ffffffd0 X10: ffffd3ebfbb7a120 X11:
0000
000005f5e0ff
X12: ffffd3ebfbb90000 X13: ffff8000102fb8f5 X14:
ffff
8000902fb8e7
X15: ffffd3ebfbb798c8 X16: 0000000000000000 X17:
0000
000000000000
X18: ffffffffffffffff X19: ffffd3ebfbb79000 X20:
ffff
d3ebfbb95000
X21: 0000000000000000 X22: ffffd3ebfbd7a000 X23:
ffff
8000102fbaf8
X24: 0000000000000004 X25: 0000000000000000 X26:
ffff
d3ebfbce4000
X27: 0000000000000000 X28: ffff00004f58c600 X29:
ffff
8000102fbab0
LR: ffffd3ebfa58ded8 SP: ffff8000102fbab0 PC:
ffff
d3ebfa58df5c
PSTATE: 60000085 FPVALID: 00000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000374 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
0000000000000000 0000000000000000
ffff00004de23400 0000000000000000
ffff00004f58c600 0000000000000000
ffffd3ebfbb954b8 ffff00004f58c600
0000000000000001 ffff8000102fbbe0
0000000000000146 00000000ffffffd0
ffffd3ebfbb7a120 0000000005f5e0ff
ffffd3ebfbb90000 ffff8000102fb8f5
ffff8000902fb8e7 ffffd3ebfbb798c8
0000000000000000 0000000000000000
ffffffffffffffff ffffd3ebfbb79000
ffffd3ebfbb95000 0000000000000000
ffffd3ebfbd7a000 ffff8000102fbaf8
0000000000000004 0000000000000000
ffffd3ebfbce4000 0000000000000000
ffff00004f58c600 ffff8000102fbab0
ffffd3ebfa58ded8 ffff8000102fbab0
ffffd3ebfa58df5c 0000000060000085
0000000000000000
Elf64_Nhdr:
n_namesz: 11 ("VMCOREINFO")
n_descsz: 1939
n_type: 0 (unused)
OSRELEASE=5.5.0-rc4-00068-gf8631e1
PAGESIZE=4096
SYMBOL(init_uts_ns)=ffffd3ebfbb81bf8
SYMBOL(node_online_map)=ffffd3ebfbb7a3e8
SYMBOL(swapper_pg_dir)=ffffd3ebfb67a000
SYMBOL(_stext)=ffffd3ebfa481000
SYMBOL(vmap_area_list)=ffffd3ebfbb9b258
SYMBOL(mem_section)=ffff00007b616100
LENGTH(mem_section)=1024
SIZE(mem_section)=16
OFFSET(mem_section.section_mem_map)=0
SIZE(page)=64
SIZE(pglist_data)=7936
SIZE(zone)=1664
SIZE(free_area)=104
SIZE(list_head)=16
SIZE(nodemask_t)=8
OFFSET(page.flags)=0
OFFSET(page._refcount)=52
OFFSET(page.mapping)=24
OFFSET(page.lru)=8
OFFSET(page._mapcount)=48
OFFSET(page.private)=40
OFFSET(page.compound_dtor)=16
OFFSET(page.compound_order)=17
OFFSET(page.compound_head)=8
OFFSET(pglist_data.node_zones)=0
OFFSET(pglist_data.nr_zones)=7200
OFFSET(pglist_data.node_start_pfn)=7208
OFFSET(pglist_data.node_spanned_pages)=7224
OFFSET(pglist_data.node_id)=7232
OFFSET(zone.free_area)=192
OFFSET(zone.vm_stat)=1472
OFFSET(zone.spanned_pages)=112
OFFSET(free_area.free_list)=0
OFFSET(list_head.next)=0
OFFSET(list_head.prev)=8
OFFSET(vmap_area.va_start)=0
OFFSET(vmap_area.list)=40
LENGTH(zone.free_area)=11
SYMBOL(log_buf)=ffffd3ebfbb90b40
SYMBOL(log_buf_len)=ffffd3ebfbb90b38
SYMBOL(log_first_idx)=ffffd3ebfbd5566c
SYMBOL(clear_idx)=ffffd3ebfbd55678
SYMBOL(log_next_idx)=ffffd3ebfbd55668
SIZE(printk_log)=16
OFFSET(printk_log.ts_nsec)=0
OFFSET(printk_log.len)=8
OFFSET(printk_log.text_len)=10
OFFSET(printk_log.dict_len)=12
LENGTH(free_area.free_list)=6
NUMBER(NR_FREE_PAGES)=0
NUMBER(PG_lru)=4
NUMBER(PG_private)=13
NUMBER(PG_swapcache)=10
NUMBER(PG_swapbacked)=19
NUMBER(PG_slab)=9
NUMBER(PG_hwpoison)=22
NUMBER(PG_head_mask)=65536
NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129
NUMBER(HUGETLB_PAGE_DTOR)=2
NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257
NUMBER(VA_BITS)=48
NUMBER(kimage_voffset)=0xffffd3eb78400000
NUMBER(PHYS_OFFSET)=0xffffafb780000000
NUMBER(tcr_el1_t1sz)=0x10
KERNELOFFSET=53ebea400000
CRASHTIME=1517155176
readmem: read_kdump()
KASLR:
_stext from vmlinux: ffff800010081000
_stext from vmcoreinfo: ffffd3ebfa481000
relocate: 53ebea400000 (87998116MB)
crash: pv_ops exists: ARCH_PVOPS
vmcoreinfo : vabits_actual: 48
using ffffafb780000000 as phys_offset
kimage_voffset: ffffd3eb78400000
phys_offset: ffffafb780000000
gdb ../linux/vmlinux
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-unknown-linux-gnu
--target=aarch64-elf-linux"...
WARNING: kernel relocated [87998116MB]: patching 135244 gdb minimal_symbol values
crash: CONFIG_NR_CPUS: 256
crash: CONFIG_HZ: 250
crash: # CONFIG_DEBUG_INFO_REDUCED is not set
cpu_possible_mask: cpus: 0 1 2 3
cpu_present_mask: cpus: 0 1 2 3
cpu_online_mask: cpus: 0 1 2 3
cpu_active_mask: cpus: 0 1 2 3
xtime timespec.tv_sec: 5a6df368: Sun Jan 28 21:29:36 2018
utsname:
sysname: Linux
nodename: localhost
release: 5.5.0-rc4-00068-gf8631e1
version: #18 SMP PREEMPT Fri Jan 3 19:16:28 IST 2020
machine: aarch64
domainname: (none)
base kernel version: 5.5.0
verify_namelist:
dumpfile /proc/version:
Linux version 5.5.0-rc4-00068-gf8631e1 (mailto:nxa19049@lsv03080.swis.in-blr01.nxp.com)
(gcc version 7.3.0 (GCC)) #18 SMP PREEMPT Fri Jan 3 19:16:28 IST 2020
../linux/vmlinux:
Linux version 5.5.0-rc4-00068-gf8631e1 (mailto:nxa19049@lsv03080.swis.in-blr01.nxp.com)
(gcc version 7.3.0 (GCC)) #18 SMP PREEMPT Fri Jan 3 19:16:28 IST 2020
crash: get_cpus_present: present: 4
hypervisor: (undetermined)
ikconfig: 2550 valid configs.
SECTION_SIZE_BITS: 30
CONFIG_ARM64_VA_BITS: 48
VA_BITS_ACTUAL: 48
(calculated) VA_BITS: 48
PAGE_OFFSET: ffff000000000000
VA_START: ffff800000000000
modules: ffff800008000000 - ffff80000fffffff
vmalloc: ffff800010000000 - fffffdffbffeffff
kernel image: ffffd3ebfa480000 - ffffd3ebfbdbc000
vmemmap: fffffe0000000000 - ffffffffffffffff
irq_stack_ptr:
type: 1, TYPE_CODE_PTR
target_typecode: 8, TYPE_CODE_INT
target_length: 8
length: 8
crash: test msg 2crash: test msg 2 p=7b5b9050 cpunum=0crash: read error: kernel virtual
address: ffff00007b5b9050 type: "IRQ stack pointer"
crash: test msg 2 p=7b5cf050 cpunum=1crash: read error: kernel virtual address:
ffff00007b5cf050 type: "IRQ stack pointer"
crash: test msg 2 p=7b5e5050 cpunum=2crash: read error: kernel virtual address:
ffff00007b5e5050 type: "IRQ stack pointer"
crash: test msg 2 p=7b5fb050 cpunum=3crash: read error: kernel virtual address:
ffff00007b5fb050 type: "IRQ stack pointer"
crash: builtin stackframe.sp offset differs from kernel version
crash: builtin stackframe.pc offset differs from kernel version
kernel NR_CPUS: 256
PAGESIZE=4096
mem_section_size = 8192
NR_SECTION_ROOTS = 1024
NR_MEM_SECTIONS = 262144
SECTIONS_PER_ROOT = 256
SECTION_ROOT_MASK = 0xff
PAGES_PER_SECTION = 262144
crash: read error: kernel virtual address: ffff00007b616100 type: "memory section
root table"
Regards
Poonam