8:45 a.m.
----- Original Message -----
Hi David & Dave,
Executing "crash" on a physical machine (not VirtualBox):
# crash
crash 7.1.3
Copyright (C) 2002-2014 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
crash: /boot/xen-4.5.gz: original filename unknown
Use "-f /boot/xen-4.5.gz" on command line to prevent this message.
WARNING: machine type mismatch:
crash utility: X86_64
/var/tmp/xen-4.5.gz_VIOmfp: X86
crash: /boot/symtypes-3.12.49-6-default.gz: original filename unknown
Use "-f /boot/symtypes-3.12.49-6-default.gz" on command line to prevent
this message.
crash: /boot/symvers-3.12.49-6-default.gz: original filename unknown
Use "-f /boot/symvers-3.12.49-6-default.gz" on command line to prevent
this message.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
crash: this kernel may be configured with CONFIG_STRICT_DEVMEM, which
renders /dev/mem unusable as a live memory source.
crash: trying /proc/kcore as an alternative to /dev/mem
KERNEL: /boot/vmlinux-3.12.49-6-xen.gz
DEBUGINFO: /usr/lib/debug/boot/vmlinux-3.12.49-6-xen.debug
DUMPFILE: /proc/kcore
CPUS: 128
DATE: Thu Nov 19 09:37:49 2015
UPTIME: 00:34:57
LOAD AVERAGE: 1.77, 1.21, 1.02
TASKS: 1328
NODENAME: dl980-5
RELEASE: 3.12.49-6-xen
VERSION: #1 SMP Mon Oct 26 16:05:37 UTC 2015 (11560c3)
MACHINE: x86_64 (1995 Mhz)
MEMORY: 125.9 GB
PID: 39777
COMMAND: "crash"
TASK: ffff881eacaaa100 [THREAD_INFO: ffff881e8ff46000]
CPU: 3
STATE: TASK_RUNNING (ACTIVE)
crash>
I can see the crash will use "/proc/kcore" instead of "/dev/mem". So
I
try the same thing on VirtualBox:
# crash /boot/vmlinux-3.12.49-6-xen.gz /proc/kcore
crash 7.1.3
Copyright (C) 2002-2014 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
KERNEL: /boot/vmlinux-3.12.49-6-xen.gz
DEBUGINFO: /usr/lib/debug/boot/vmlinux-3.12.49-6-xen.debug
DUMPFILE: /proc/kcore
CPUS: 1
DATE: Thu Nov 19 01:53:01 2015
UPTIME: 05:42:13
LOAD AVERAGE: 0.19, 0.06, 0.06
TASKS: 239
NODENAME: linux-6ev3
RELEASE: 3.12.49-6-xen
VERSION: #1 SMP Mon Oct 26 16:05:37 UTC 2015 (11560c3)
MACHINE: x86_64 (2594 Mhz)
MEMORY: 855.2 MB
PID: 3106
COMMAND: "crash"
TASK: ffff88002ec5c040 [THREAD_INFO: ffff88000b3e2000]
CPU: 0
STATE: TASK_RUNNING (ACTIVE)
crash>
It seems OK now.
So my questions are:
(1) Is it OK to use "/proc/kcore" instead of "/dev/mem" as a
workaround?
Is there any side-effect?
Yes /proc/kcore is fine, and it's not really a "workaround". Since your
kernel
was configured with CONFIG_STRICT_DEVMEM, /dev/mem is unusable because it does not
allow reads above 1MB physical. And so as you saw above on the physical machine,
crash then tries to use /proc/kcore as an alternative, and it worked OK.
I don't really understand the behavior you see on VirtualBox. You are running
the same vmlinux-3.12.49-6-xen.debug kernel, but the /dev/mem restriction is
not recognized, and from your earlier posts, it appears to have continued reading
from /dev/mem, but was getting back bogus data.
Note that on some Red Hat kernel versions, /proc/kcore is also restricted, and that
is why we ship the "/dev/crash" driver for reading live memory.
(2) Execute "crash -d8" on physical machine will cause
crash utility core
dump.
I can reproduce this, so I'll look into it. It's related to the /dev/mem
"test"
to determine whether the kernel was configured with CONFIG_STRICT_DEVMEM, where
it tries to read from pfn 257 (just above the CONFIG_STRICT_DEVMEM limit), but
gets into an infinite loop when used in conjunction with -d.
Anyway, just continue to use /proc/kcore and you should be fine.
Thanks,
Dave
Use gdb to debug it:
# gdb /usr/bin/crash core-crash-11-0-0-40072-1447945769
GNU gdb (GDB; SUSE Linux Enterprise 12) 7.9.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.opensuse.org/>;.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>;.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/crash...Reading symbols from
/usr/lib/debug/usr/bin/crash.debug...done.
done.
[New LWP 40072]
Core was generated by `crash -d8'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f5119001fd0 in get_cie_encoding (cie=0x7f5119004cd8) at
../../../libgcc/unwind-dw2-fde.c:272
272 ../../../libgcc/unwind-dw2-fde.c: No such file or directory.
(gdb)
(gdb) bt
#0 0x00007f5119001fd0 in get_cie_encoding (cie=0x7f5119004cd8) at
../../../libgcc/unwind-dw2-fde.c:272
#1 0x00007f5119002699 in get_fde_encoding (f=0x7f5119006050) at
../../../libgcc/unwind-dw2-fde.c:319
#2 _Unwind_IteratePhdrCallback (info=info@entry=0x7fff463c10e0,
size=size@entry=64, ptr=ptr@entry=0x7fff463c1160)
at ../../../libgcc/unwind-dw2-fde-dip.c:408
#3 0x00007f51196a3f3c in __GI___dl_iterate_phdr
(callback=callback@entry=0x7f5119002270 <_Unwind_IteratePhdrCallback>,
data=data@entry=0x7fff463c1160) at dl-iteratephdr.c:76
#4 0x00007f51190035c3 in _Unwind_Find_FDE (pc=0x7f5119001aa7
<_Unwind_Backtrace+55>, bases=bases@entry=0x7fff463c1498)
at ../../../libgcc/unwind-dw2-fde-dip.c:459
#5 0x00007f5118ffff86 in uw_frame_state_for
(context=context@entry=0x7fff463c13f0, fs=fs@entry=0x7fff463c1240)
at ../../../libgcc/unwind-dw2.c:1241
#6 0x00007f51190011d0 in uw_init_context_1
(context=context@entry=0x7fff463c13f0,
outer_cfa=outer_cfa@entry=0x7fff463c16a0,
outer_ra=0x7f511967bf46 <__GI___backtrace+86>) at
../../../libgcc/unwind-dw2.c:1562
#7 0x00007f5119001aa8 in _Unwind_Backtrace (trace=0x7f511967bdd0
<backtrace_helper>, trace_argument=0x7fff463c16a0)
at ../../../libgcc/unwind.inc:283
#8 0x00007f511967bf46 in __GI___backtrace
(array=array@entry=0x7fff463c1710, size=size@entry=4) at
../sysdeps/x86_64/backtrace.c:109
#9 0x000000000047add7 in __error (type=type@entry=1,
fmt=fmt@entry=0x853d38 "read(/dev/mem, %lx, %ld): %ld (%lx)\n") at
tools.c:52
#10 0x0000000000490c91 in read_dev_mem (fd=4, bufptr=0x7fff463c1f28,
cnt=8, addr=0, paddr=1052672) at memory.c:2298
#11 0x0000000000486398 in readmem (addr=1052672,
memtype=memtype@entry=4, buffer=buffer@entry=0x7fff463c1f28,
size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2198
#12 0x000000000048704a in devmem_is_restricted () at memory.c:2414
#13 readmem (addr=1052672, memtype=memtype@entry=4,
buffer=buffer@entry=0x7fff463c1fb8, size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2209
#14 0x000000000048704a in devmem_is_restricted () at memory.c:2414
#15 readmem (addr=1052672, memtype=memtype@entry=4,
buffer=buffer@entry=0x7fff463c2048, size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2209
#16 0x000000000048704a in devmem_is_restricted () at memory.c:2414
#17 readmem (addr=1052672, memtype=memtype@entry=4,
buffer=buffer@entry=0x7fff463c20d8, size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2209
#18 0x000000000048704a in devmem_is_restricted () at memory.c:2414
#19 readmem (addr=1052672, memtype=memtype@entry=4,
buffer=buffer@entry=0x7fff463c2168, size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2209
#20 0x000000000048704a in devmem_is_restricted () at memory.c:2414
#21 readmem (addr=1052672, memtype=memtype@entry=4,
buffer=buffer@entry=0x7fff463c21f8, size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2209
#22 0x000000000048704a in devmem_is_restricted () at memory.c:2414
#23 readmem (addr=1052672, memtype=memtype@entry=4,
buffer=buffer@entry=0x7fff463c2288, size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2209
#24 0x000000000048704a in devmem_is_restricted () at memory.c:2414
......
Below is always:
readmem (addr=1052672, memtype=memtype@entry=4,
buffer=buffer@entry=0x7fff463c2288, size=size@entry=8,
type=type@entry=0x8563e1 "devmem_is_allowed - pfn 257",
error_handle=error_handle@entry=6) at memory.c:2209
0x000000000048704a in devmem_is_restricted () at memory.c:2414
Seems a dead-loop, but not sure.
Thanks!
Best Regards
Nan Xiao
On Thu, Nov 19, 2015 at 1:51 AM, David Mair <dmair(a)suse.com> wrote:
> On 11/18/2015 08:00 AM, Dave Anderson wrote:
>>
>> <snip>
>>
>> Can anybody on this mailing list confirm that crash runs OK on Xen dom0
>> kernels of this vintage?
>>
>
> It does. I currently do the crash maintenance at SUSE and put together
> the 7.1.3 case being asked about. It and the kernel being discussed are
> part of a currently un-released service pack gold master candidate as I
> write. Myself and one of the Xen team here were able to use crash on a
> dom0 on bare-metal without the problems being reported by the OP.
> VirtualBox being the cause looks like a reasonable consideration.
>
> --
> David.
>
> --
> Crash-utility mailing list
> Crash-utility(a)redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
--
Crash-utility mailing list
Crash-utility(a)redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility