6:12 a.m.
Hello
I've been trying to open crashdump produced by 2.6.27.21 SMP kernel
with 2G memory split without luck.
I tried to generate the dump by copying from /proc/vmcore and with
makedumpfile 1.3.3. Both without success.
The crash works fine with uniprocessor kernel as well as SMP kernel
using standard memory split. Architecture is 32bit x86 and the crash
was generated inside vmware server 2.0.
Is this supported configuration? Have anybody tried crash dump with 2G
split in recent 2.6.x kernels?
I'll attach some debug output from both non-working smp dump and
working UP dump.
The kernel with debug symbols uses different flavor in name (smp-syms)
than the image without those (smp), so crash always warns that kernel
and dump do not match.
------- output from non-working SMP dump (dumped directly from /proc/vmcore)
# crash -d7 ....
crash 4.0-8.9
....
vmcore_data:
flags: a0 (KDUMP_LOCAL|KDUMP_ELF32)
ndfd: 3
ofp: f7eba4c0
header_size: 540
num_pt_load_segments: 4
pt_load_segment[0]:
file_offset: 21c
phys_start: 0
phys_end: a0000
zero_fill: 0
pt_load_segment[1]:
file_offset: a021c
phys_start: 100000
phys_end: 1000000
zero_fill: 0
pt_load_segment[2]:
file_offset: fa021c
phys_start: 2800000
phys_end: fef0000
zero_fill: 0
pt_load_segment[3]:
file_offset: e69021c
phys_start: ff00000
phys_end: 10000000
zero_fill: 0
elf_header: 9a266f8
elf32: 9a266f8
notes32: 9a2672c
load32: 9a2674c
elf64: 0
notes64: 0
load64: 0
nt_prstatus: 9a267cc
nt_prpsinfo: 0
nt_taskstruct: 0
task_struct: 0
page_size: 0
switch_stack: 0
xen_kdump_data: (unused)
num_prstatus_notes: 2
vmcoreinfo: 0
size_vmcoreinfo: 0
nt_prstatus_percpu:
09a267cc 09a26870
.... skip to end
<readmem: 8053f700, KVADDR, "kernel_config_data", 32768, (ROE), a5f17c0>
WARNING: could not find MAGIC_START!
<readmem: 807348c0, KVADDR, "cpu_possible_map", 4, (ROE), 8430680>
cpu_possible_map: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30 31
<readmem: 806c478c, KVADDR, "cpu_present_map", 4, (ROE), 8430680>
cpu_present_map: 3 7 8 9 10 14 18 19 21 22 31
<readmem: 806c42a0, KVADDR, "cpu_online_map", 4, (ROE), 8430680>
cpu_online_map: 8 9 14 18 19 21 22 31
<readmem: 807506e0, KVADDR, "xtime", 8, (FOE), 8408d38>
<readmem: 80679e84, KVADDR, "init_uts_ns", 390, (ROE), 840931c>
crash: cannot determine base kernel version
<readmem: 8053a000, KVADDR, "accessible check", 4, (ROE|Q), ff96df04>
<readmem: 8053a000, KVADDR, "readstring characters", 1499, (ROE|Q),
ff96cf00>
linux_banner:
&y �&y �&y �(z �]z �jz ��z �a� �f� ��� � � ��� �9� �� �0� �6� �<� �Њ
�B� �H� �N� �
crash: s3/boot/vmlinux-2.6.27.21+smp-syms and
kerneldump-20090526-140502 do not match
------- output from working UP crash (this was generated with makedumpfile)
compressed kdump: header->utsname.machine:
diskdump_data:
filename: (null)
flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
dfd: 3
ofp: 0
machine_type: 3 (EM_386)
header: 94c96f8
signature: "KDUMP "
header_version: 2
utsname:
sysname:
nodename:
release:
version:
machine:
domainname:
timestamp:
tv_sec: 0
tv_usec: 0
status: 0 ()
block_size: 4096
sub_hdr_size: 1
bitmap_blocks: 4
max_mapnr: 65536
total_ram_blocks: 0
device_blocks: 0
written_blocks: 0
current_cpu: 0
nr_cpus: 1
tasks[nr_cpus]: 0
sub_header: 0 (n/a)
sub_header_kdump: 94ca700
phys_base: 0
dump_level: 1 (0x1) (DUMP_EXCLUDE_ZERO)
data_offset: 6000
block_size: 4096
block_shift: 12
bitmap: 94cb708
bitmap_len: 16384
dumpable_bitmap: 94cf710
byte: 0
bit: 0
compressed_page: 94e3768
curbufptr: 0
page_cache_hdr[0]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 94d3760
pg_hit_count: 0
page_cache_hdr[1]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 94d4760
pg_hit_count: 0
page_cache_hdr[2]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 94d5760
.... skipped rest of page_cache stuff
page_cache_buf: 94d3760
evict_index: 0
evictions: 0
accesses: 0
cached_reads: 0
valid_pages: 94d3718
... skipped some stuff
<readmem: 804ed6c0, KVADDR, "kernel_config_data", 32768, (ROE), a009250>
WARNING: could not find MAGIC_START!
<readmem: 8066cce8, KVADDR, "cpu_possible_map", 4, (ROE), 8430680>
cpu_possible_map: 0
<readmem: 8066ccec, KVADDR, "cpu_present_map", 4, (ROE), 8430680>
cpu_present_map: 0
<readmem: 8066cce4, KVADDR, "cpu_online_map", 4, (ROE), 8430680>
cpu_online_map: 0
<readmem: 806e09d0, KVADDR, "xtime", 8, (FOE), 8408d38>
<readmem: 80626004, KVADDR, "init_uts_ns", 390, (ROE), 840931c>
base kernel version: 2.6.27
<readmem: 804e8000, KVADDR, "accessible check", 4, (ROE|Q), ff87ce14>
<readmem: 804e8000, KVADDR, "readstring characters", 1499, (ROE|Q),
ff87be10>
WARNING: cannot find matching kernel version in
u/boot/vmlinux-2.6.27.21+up-syms file:
verify_namelist:
/proc/version:
Linux version 2.6.27.21+up (root@x) (gcc version 4.1.2 20061115
(prerelease) (Debian 4.1.1-21)) #1 Fri May 15 13:56:19 UTC 2009
utsname version: #1 Fri May 15 13:56:19 UTC 2009
u/boot/vmlinux-2.6.27.21+up-syms:
<readmem: 802577e0, KVADDR, "x86_omit_frame_pointer", 8, (ROE), ff87dfd8>
<readmem: 806e93ec, KVADDR, "high_memory", 4, (FOE), 840b504>
<readmem: 806e93e0, KVADDR, "mem_map", 4, (FOE), 840b50c>
<readmem: 806e940c, KVADDR, "vmlist", 4, (FOE), ff87d974>
<readmem: 8f8028a4, KVADDR, "first vmlist addr", 4, (ROE), ff87d970>
<readmem: 8066cd5c, KVADDR, "totalram_pages", 4, (FOE), 840b514>
.... skipped lot of output
KERNEL: u/boot/vmlinux-2.6.27.21+up-syms
DUMPFILE: kerneldump-20090527-075410 [PARTIAL DUMP]
CPUS: 1
DATE: Wed May 27 10:53:54 2009
<readmem: 8062b8f8, KVADDR, "jiffies_64", 8, (FOE), ff87d9c8>
crash: convert_time: 283566 (453ae)
UPTIME: 00:04:43
<readmem: 806e05e0, KVADDR, "avenrun array", 12, (FOE), ff87d9c0>
LOAD AVERAGE: 0.14, 0.14, 0.06
TASKS: 141
NODENAME: tp3
RELEASE: 2.6.27.21+up
VERSION: #1 Fri May 15 13:56:19 UTC 2009
MACHINE: i686 <readmem: 806bff50, KVADDR, "cpu_khz", 4, (FOE),
ff87d9e0>
(2658 Mhz)
MEMORY: 255.5 MB
PANIC: NOTE: GETBUF request > MAX_CACHE_SIZE: 65536
"[ 136.195328] SysRq : Trigger a crashdump"
PID: 3208
COMMAND: "bash"
TASK: 8f1be0c0 [THREAD_INFO: 88d44000]
CPU: 0
<readmem: 80630790, KVADDR, "task_state_array", 4, (ROE), ff87d998>
<readmem: 805b2fd4, KVADDR, "readstring characters", 44, (ROE|Q),
ff87c370>
...
------- some output from working SMP standard memory split (but PAE
enabled) crash
mcore_data:
flags: a0 (KDUMP_LOCAL|KDUMP_ELF32)
ndfd: 3
ofp: f7f074c0
header_size: 540
num_pt_load_segments: 4
pt_load_segment[0]:
file_offset: 21c
phys_start: 0
phys_end: a0000
zero_fill: 0
pt_load_segment[1]:
file_offset: a021c
phys_start: 100000
phys_end: 1000000
zero_fill: 0
....
elf_header: 99eb6f8
elf32: 99eb6f8
notes32: 99eb72c
load32: 99eb74c
...
<readmem: c053e700, KVADDR, "kernel_config_data", 32768, (ROE), a5b67f8>
crash: CONFIG_NR_CPUS: 32
crash: CONFIG_HZ: 100
<readmem: c07318c0, KVADDR, "cpu_possible_map", 4, (ROE), 842fea0>
cpu_possible_map: 0 1
<readmem: c06c378c, KVADDR, "cpu_present_map", 4, (ROE), 842fea0>
cpu_present_map: 0 1
<readmem: c06c32a0, KVADDR, "cpu_online_map", 4, (ROE), 842fea0>
cpu_online_map: 0 1
...
PANIC: NOTE: GETBUF request > MAX_CACHE_SIZE: 65536
"[ 31.044652] SysRq : Trigger a crashdump"
PID: 2745
....
---------- end of snips
Thanks in advance,
Tero