crashdc: problem when running crash in kexec environment for SLES11
- First Post
- Replies
- Stats
-
Go to
- ----- 2025 -----
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello,
As I indicated previously, I'm having a problem with running crash from
within the kexec environment of SLES11.
Actually, crash itself runs fine, but its gdb portion seems to be having
problems. Here is a capture of what happens :
Running /usr/bin/run-crashdc-sles11.sh
crashexe : /usr/bin/crash
crashoutput : /root/var/crash/2009-09-29-17:45/crash-data-200909291746.txt
namelist : /root/boot/vmlinux-2.6.27.23-0.1-default
vmcorefile : /root/var/crash/2009-09-29-17:45/vmcore
debuginfofile : /root/usr/lib/debug/boot/vmlinux-2.6.27.23-0.1-default.debug
This portion above is debug info from crashdc.
crash 4.0-7.6
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
NOTE: stdin: not a tty
cannot determine relocation value: not a live system
gdb /root/usr/lib/debug/boot/vmlinux-2.6.27.23-0.1-default.debug
dlopen failed on 'libthread_db.so.1' - libthread_db.so.1: cannot open shared
object file: No such file or directory
GDB will not be able to debug pthreads.
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".../usr/bin/crashdc: line 160:
682 Killed $crashexe -d $crashdebug $namelist $debuginfofile $vmcorefile
< $crashcmd
File /root/var/crash/2009-09-29-17:45/crash-data-200909291746.txt has been generated
generated a crash-data file to /root/var/crash/2009-09-29-17:45
..done
Restarting system.
My feeling is that the "dlopen failed on 'libthread_db.so.1'" might be
causing this.
Bernhard Walle might have an idea. Here is what I have in
/etc/sysconfig/kdump :
KDUMP_REQUIRED_PROGRAMS="/bin/basename /usr/bin/crash
/usr/bin/crashdc /usr/bin/run-crashdc-sles11.sh /etc/rc.status /bin/gzip /bin/logger
/usr/bin/gdb /lib/libpthread.so.0 /lib/libthread_db.so.1"
This line allow for inclusion of the listed file into the initramfs file
that gets loaded at kexec time. Maybe crash/gdb is looking for it where
it's not, but right now I'm a bit stucked.
As a side not to Bernhard, is there a way to have kdump stop its
execution while in the kexec kernel ? Just like when it hits an error
and drop to a shell ?
TIA,
- --
Louis Bouchard, Linux Support Engineer
Team lead, EMEA Linux Competency Center,
Linux Ambassador, HP
HP Services 1 Ave du Canada
HP France Z.A. de Courtaboeuf
louis.bouchard(a)hp.com 91 947 Les Ulis
http://www.hp.com/go/linux France
http://www.hp.com/fr
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkrDc2gACgkQDvqokHrhnCyU6QCdHSlzw6q2u0qoifczMp3AGMob
woAAoMbhGJaDgiQ6GhqB+PNTCvsyELB6
=l3xC
-----END PGP SIGNATURE-----
Louis Bouchard schrieb:
Bernhard Walle might have an idea. Here is what I have in
/etc/sysconfig/kdump :
> KDUMP_REQUIRED_PROGRAMS="/bin/basename /usr/bin/crash /usr/bin/crashdc
/usr/bin/run-crashdc-sles11.sh /etc/rc.status /bin/gzip /bin/logger /usr/bin/gdb
/lib/libpthread.so.0 /lib/libthread_db.so.1"
This line allow for inclusion of the listed file into the initramfs file
that gets loaded at kexec time. Maybe crash/gdb is looking for it where
it's not, but right now I'm a bit stucked.
As a side not to Bernhard, is there a way to have kdump stop its
execution while in the kexec kernel ? Just like when it hits an error
and drop to a shell ?
KDUMP_CONTINUE_ON_ERROR=false and KDUMP_IMMEDIATE_REBOOT=false? Or am I
understanding the question wrong?
Regards,
Bernhard
Hi again,
Louis Bouchard schrieb:
> crash 4.0-7.6
> Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
details.
> NOTE: stdin: not a tty
> cannot determine relocation value: not a live system
> gdb /root/usr/lib/debug/boot/vmlinux-2.6.27.23-0.1-default.debug
> dlopen failed on 'libthread_db.so.1' - libthread_db.so.1: cannot open shared
object file: No such file or directory
> GDB will not be able to debug pthreads.
> GNU gdb 6.1
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for
details.
> This GDB was configured as "i686-pc-linux-gnu".../usr/bin/crashdc: line
160: 682 Killed $crashexe -d $crashdebug $namelist $debuginfofile
$vmcorefile < $crashcmd
> File /root/var/crash/2009-09-29-17:45/crash-data-200909291746.txt has been generated
> generated a crash-data file to /root/var/crash/2009-09-29-17:45
> ..done
> Restarting system.
My feeling is that the "dlopen failed on 'libthread_db.so.1'" might be
causing this.
I don't think so. I just tried to reproduce the problem here with
openSUSE 11.1 (I have no SLES). In my case, I also got the "killed", but
it was too less memory. I changed the reservation to be 256 MiB for the
crashkernel, and now it works.
I did in the shell I got in the kexec environment:
% ln -s root/boot/vmlinux-2.6.27.7-9-pae .
% ln -s root/usr/lib/debug/boot/vmlinux-2.6.27.7-9-pae.debug .
% echo bt > cmdlist
% root/usr/bin/crash vmlinux-2.6.27.7-9-pae /proc/vmcore -i cmdlist
With this, I got a backtrace. I also get the libthread_db warning, but
it was harmless.
BTW: Did you try to set LD_LIBRARY_PATH? Maybe it's necessary so that
dlopen() finds the lib. But I didn't investigate ...
HTH.
Regards,
Bernhard
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello everyone,
Bernhard Walle a écrit :
I don't think so. I just tried to reproduce the problem here with
openSUSE 11.1 (I have no SLES). In my case, I also got the "killed", but
it was too less memory. I changed the reservation to be 256 MiB for the
crashkernel, and now it works.
For info, I'm doing the dev on an i686 VMware guest for now.
Bernhard found the SLES11 problem : memory was too tight to allow
crash/gdb to run.
I increased the mem reservation with the following :
crashkernel=256M-:128M@16M
and got crash to run my commands. I have a smaller problem now. I'm
issueing the following commands to crash :
!echo "### Crash Context Information" >>
/var/crash/2009-09-25-11:31/crash-data-200909251134.txt
!echo "crash> sys" >>
/var/crash/2009-09-25-11:31/crash-data-200909251134.txt
sys >> /var/crash/2009-09-25-11:31/crash-data-200909251134.txt
!echo -n `date`;echo "### Preserved Kernel Ring Buffer"
!echo "### Preserved Kernel Ring Buffer" >>
/var/crash/2009-09-25-11:31/crash-data-200909251134.txt
!echo "crash> log" >>
/var/crash/2009-09-25-11:31/crash-data-200909251134.txt
log >> /var/crash/2009-09-25-11:31/crash-data-200909251134.txt
I did include /bin/echo in the KDUMP_REQUIRED_PROGRAMS variable, but
crash still returns :
crash: cannot open pipe: echo "### Crash Context
Information" >> /root/var/crash/2009-10-01-15:51/crash-data-200910011552.txt
crash: cannot open pipe: echo "crash> sys" >>
/root/var/crash/2009-10-01-15:51/crash-data-200910011552.txt
crash: cannot open pipe: echo -n `date`;echo "### Preserved Kernel Ring
Buffer"
crash: cannot open pipe: echo "### Preserved Kernel Ring Buffer" >>
/root/var/crash/2009-10-01-15:51/crash-data-200910011552.txt
So it looks like the ! to shell out the "echo "... command doesn't work,
most probably because of the limited commands in the kexec environment.
Now I could try to chroot crash execution, but if I could find the
single command to add to KDUMP_REQUIRED_PROGRAMS to work around this, I
would prefer.
Can I beg for your deep knowledge another time ? ;-)
Kind regards and thanks for your precious help,
- --
Louis Bouchard, Linux Support Engineer
Team lead, EMEA Linux Competency Center,
Linux Ambassador, HP
HP Services 1 Ave du Canada
HP France Z.A. de Courtaboeuf
louis.bouchard(a)hp.com 91 947 Les Ulis
http://www.hp.com/go/linux France
http://www.hp.com/fr
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkrEt3kACgkQDvqokHrhnCzkGACfSTCD2GEzn+o06QM4p6w6Fnl2
a8AAniPNimmltKUzlqAFxwMNwIjzt4gs
=5vIn
-----END PGP SIGNATURE-----
5692
days inactive
5693
days old
devel@lists.crash-utility.osci.io
3 comments
2 participants
tags (0)
participants (2)
-
Bernhard Walle -
Louis Bouchard