----- Original Message ----- OK. Note that you don't have to use the /boot/System.map-2.6.32-25-generic file on the command line, presuming that the live kernel is actually running that same /home/amer/ubuntu-lucid/debian/build/build-generic/vmlinux kernel. Yes -- don't do that. Running "dd /dev/crash > image.dd" just dumps a bunch of bytes into the image.dd file. Dumpfiles by definition need some kind of bookkeeping header in order for the crash utility to know where the contents of a particular page are located in the dumpfile. Crash supports over a dozen different dumpfile formats. If you want to take a snapshot of your live system without crashing it, you can try the "snap" extension module: # cd <path-to>/crash-5.1.2 # make extensions ... # That will build the "snap.so" extension module (along with a few others found in the crash-5.1.2/extensions subdirectory). The run live -- note that normally you don't have to enter anything on the command line if the debuginfo vmlinux file of the running kernel is in a "known" location. I don't know how Ubuntu sets things up, so it's possible that you at least need to enter the path to your vmlinux file (and the System.map if that vmlinux file is different from the running kernel). Anyway, get a live session going, load the "snap.so" extension module, and dump kernel memory into an ELF-style dumpfile. For example: # crash crash 5.1.2 Copyright (C) 2002-2011 Red Hat, Inc. Copyright (C) 2004, 2005, 2006 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.0 Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"... KERNEL: /usr/lib/debug/lib/modules/2.6.18-128.el5/vmlinux DUMPFILE: /dev/crash CPUS: 8 DATE: Thu Feb 24 09:21:24 2011 UPTIME: 37 days, 00:57:58 LOAD AVERAGE: 0.29, 0.38, 0.18 TASKS: 262 NODENAME: crash.usersys.redhat.com RELEASE: 2.6.18-128.el5 VERSION: #1 SMP Wed Dec 17 11:41:38 EST 2008 MACHINE: x86_64 (1995 Mhz) MEMORY: 1 GB PID: 27743 COMMAND: "crash" TASK: ffff81002750a820 [THREAD_INFO: ffff81003db8c000] CPU: 2 STATE: TASK_RUNNING (ACTIVE) crash> extend snap.so ./extensions/snap.so: shared object loaded crash> snap live.dump live.dump: [100%] -rw-r--r-- 1 root root 1071686392 Feb 24 09:18 live.dump crash> q # Then use the "live.dump" dumpfile: # crash /usr/lib/debug/lib/modules/2.6.18-128.el5/vmlinux live.dump crash 5.1.2 Copyright (C) 2002-2011 Red Hat, Inc. Copyright (C) 2004, 2005, 2006 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.0 Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"... KERNEL: /usr/lib/debug/lib/modules/2.6.18-128.el5/vmlinux DUMPFILE: live.dump CPUS: 8 DATE: Thu Feb 24 09:18:13 2011 UPTIME: 37 days, 00:54:47 LOAD AVERAGE: 0.31, 0.11, 0.06 TASKS: 257 NODENAME: crash.usersys.redhat.com RELEASE: 2.6.18-128.el5 VERSION: #1 SMP Wed Dec 17 11:41:38 EST 2008 MACHINE: x86_64 (1995 Mhz) MEMORY: 1 GB PANIC: "" PID: 27603 COMMAND: "crash" TASK: ffff81001a57f100 [THREAD_INFO: ffff81001ad22000] CPU: 2 STATE: TASK_RUNNING (PANIC) crash> You may see some command errors because the system is not frozen when the dump is taken, so vital things may be changing while the memory is being copied. For example, "bt" on the active tasks may show nonsensical data, given that they were running when the memory was being copied. Hope this helps, Dave

----- Original Message ----- Ok, so your Ubuntu system does not put the vmlinux, or you have not put the vmlinux file in one of the several "known" locations. First, you're using a version of the crash utility (4.1.0) that is well over two years old. Using "/dev/crash" on the command line has never been supported until crash-5.1.1, which was only released 12/23/10. 5.1.1 - Fix for the potential to miss tasks when walking the pid_hash table ... [ cut ] ... - Fix to allow "/dev/crash" to be entered on the command line for live sessions. Because it is used automatically if it exists, it is never necessary to enter it on the command line. However, if it is used, without the patch, the session fails during initializaion with the error message "crash: /dev/crash: No such file or directory" if the crash.ko driver is a module (RHEL4/RHEL5), or "crash: /dev/crash: not a supported file format" if the driver is built into the kernel (RHEL6). (anderson(a)redhat.com) Well, that refers to a patched derivative of the crash utility. I can assure you that using the output of a bunch of bytes with no header has *never* been supported as a dumpfile type from the versions posted upstream at http://people.redhat.com/anderson. Note that in that blog, there is this: Now let's test the newly obtained memory dump to see if it works. I'm going to use the RH Crash Utility with the volatile patch which you can find here: # ./crash -f /boot/System.map-2.6.27-14-generic /usr/src/linux-source-2.6.27/vmlinux crash.dd --volatile crash 4.0-8.9 Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Red Hat, Inc. ... I can also assure you that the upstream crash utility has never had a "--volatile" command line argument, so perhaps that "volatile patch" has something to do with it? Dave