[ANNOUNCE] crash version 6.0.4 is available
by Dave Anderson
Download from: http://people.redhat.com/anderson
Changelog:
- Fix to allow the recently-added "mod -g" and "mod -r" options to be
used together. Without the patch, if both options were used, the
command would fail with a "mod: invalid option" error complaining
about one or the other option letter.
(bob.montgomery(a)hp.com)
- Additional update for 3.1.x and later kernels configured with
CONFIG_SLAB, which have replaced the kmem_cache.nodelists[] array
with a pointer to an outside array. Without the patch, depending
upon a system's cpu configuration and actual cpu count, the crash
session may display "crash: unable to initialize kmem slab cache
subsystem" during invocation, or if it does succeed, "kmem -s" may
generate a segmentation violation.
(bob.montgomery(a)hp.com, anderson(a)redhat.com)
- Document the "crash [-h|--help] all" option in the crash.8 man
page and in the "crash [-h|--help]" output.
(anderson(a)redhat.com)
- Fix the S390/S390X-specific "s390dbf" command's "hex_ascii" debug
data printing routine to prevent the display of non-ASCII characters.
(holzheu(a)linux.vnet.ibm.com)
- Fix for ARM stack unwinding on 3.2 and later kernels due to commit:
http://git.kernel.org/linus/de66a979012dbc66b1ec0125795a3f79ee667b8a
(rabin(a)rab.in)
- Implemented a new "search -x <count>" option that displays the
memory contents before and after any found search target. The
before and after memory context will consist of "count" memory items
of the same size as the searched-for value. This option is not
applicable with the -c option.
(zhangyanfei(a)cn.fujitsu.com)
- Fix for the x86_64 Xen hypervisor "bt" command. Without the patch,
the contents of the RDX register in exception frames incorrectly
shows the contents of the RCX register.
(s.ikarashi(a)jp.fujitsu.com)
- Implementation of a platform-based vmalloc address translation scheme
for the 32-bit PPC architecture, introducing suppport for the PPC44X
platform while maintaining the current default platform. Related to
that, the PTE translation function used by "vtop" properly handles
platforms that use 64-bit PTEs, and the "mach" command displays the
kernel's "powerpc_base_platform" name string.
(suzuki(a)in.ibm.com)
- Fix for the usage of native gdb commands where the command output is
redirected to a pipe and then redirected to a file. Without the
patch, a command construct such as:
crash> gdb-command | shell-command > output-file
would cause the embedded gdb module to fail with the error message
"gdb: gdb request failed: gdb-command | shell-command".
(anderson(a)redhat.com)
- Fix to prevent a crash session that is run over a network connection
that is killed/removed from going into 100% cpu-time loop. The fix
that went into crash-5.0.2 to handle the change in behavior of the
built-in readline() library call does not suffice in cases where
readline() never gets a chance to be called. Accordingly, the crash
session is now initialized with a PR_SET_PDEATHSIG prctl setting,
which will cleanly kill itself upon its parent's death.
(anderson(a)redhat.com)
- Fix for the support of PPC64 compressed kdumps, a regression that
was introduced in crash-6.0.3 when support for 32-bit PPC compressed
kdumps was implemented. Without the patch, the crash session
fails to initialize, showing this warning message:
WARNING: machine type mismatch:
crash utility: PPC64
vmcore: PPC
followed by the error: "crash: vmcore: not a supported file format".
(anderson(a)redhat.com)
- Fix for the x86_64 "bt" command to prevent the possible skipping
of the stack frame just above an exception frame that indicates
"[exception RIP: unknown or invalid address]". This highly-unlikely
event could occur if the kernel jumps to a bogus text location and
attempts to execute it, or if the exception occurs in vmalloc space
that was allocated with module_alloc() by a systemtap kprobe-handler,
and therefore has no symbolic reference.
(anderson(a)redhat.com)
12 years, 4 months
Using crash to set a breakpoint
by jeff.reagen＠emc.com
Is it possible to set a gdb breakpoint when using crash to debug a live kernel? I would like to do this in RHEL 6. I've tried using
Crash> gdb break 0xffff.... Only to have crash return "crash: prohibited gdb command: break. Is there any way to "make" this work?
Thanks,
Jeff
12 years, 4 months
HEADS-UP: [PATCH v2 0/4] Compressed KDUMP core analysis support for PPC32 breaks PPC64 compressed kdump support
by Dave Anderson
Unfortunately crash-6.0.3 does not work with ppc64 compressed kdumps
because of the recently-added support for the 32-bit ppc architecture:
- Implemented support for the analysis of 32-bit PPC compressed kdump
vmcores.
(suzuki(a)in.ibm.com)
It fails like so:
# crash vmlinux vmcore
crash 6.0.3
Copyright (C) 2002-2012 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
WARNING: machine type mismatch:
crash utility: PPC64
vmcore: PPC
crash: vmcore: not a supported file format
Usage:
crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
crash [OPTION]... [NAMELIST] (live system form)
Enter "crash -h" for details.
#
It does not affect ELF core dumps.
I'll release crash-6.0.4 a bit sooner that I expected. Sorry for any
inconvenience this may have caused.
Dave
12 years, 4 months
crash endlessly looping on stdout error
by Guy Streeter
We have a recurring problem in our crash analysis system, where remote users
get disconnected and crash starts endlessly looping trying to write to stdout.
An strace of a recent instance is looping on:
write(1, " JIFFIES\n", 10) = -1 EIO (Input/output error)
but that isn't always the output string.
this is a problem in out shared environment because the orphaned crash tasks
eat up the CPUs, and we don't have the privilege to kill each others tasks.
thanks,
--Guy
12 years, 4 months
[extension module] extension module for translating KVM guest's physical address to host's address
by qiaonuohan
Hi Dave and manager of qemu-discuss,
I have implemented an extension module for translating KVM guest's
physical address to host's address. The related command is "qemu-vtop".
The attachment is the code I implemented. Please copy the file to the
crash-<version>/extensions subdirectory and from the top-level
crash-<version> directory, enter:
$ make extensions
(please refer to http://people.redhat.com/anderson/extensions.html)
To add the module's command(s) to a running crash session, enter:
crash> extend qemu-vtop.so
"mod -s kvm" is needed to load kvm debug-info to offer some data to
qemu-vtop command.
The command uses information of struct "kvm_memslots" to get the
relationship between gpa and hva. Struct "kvm_memslots" is holded by a
file, called "anon_inode:kvm-vm", opened by qemu-kvm. Then, "do_vtop" is
called to get hpa. About the detailed implementation, please refer to
attachment.
The functionality is limited, but I think it is useful when debugging a
qemu-kvm virtual machine. So I post it.
--
--
Regards
Qiao Nuohan
12 years, 4 months
[PATCH v4 0/4] vmalloc translation support for PPC
by Suzuki K. Poulose
The following series implements :
* An infrastructure for platform based vmalloc translation for PPC32
* vmalloc translation support for PPC44x
Changes since V3:
* Use platform specific 'page flags'
* Avoid calling dump_mem_map() by checking if there is a page associated
with the physical memory. (Suggested by: Dave)
Changes since V2:
* Rebased to crash-6.0.3
* Maintains a list of probe functions, rather than platform
definitions.
Each platform can define their own probe_function which would get the name of the
ppc platform (read from kernel) and the probe can check if it is one of its variant.
The probe function can then update the 'platform' defintions for the virtual address
translation.
If none of the defined platforms match, we fall back to using the default PPC32
definitions.
---
Suzuki K. Poulose (4):
Avoid dump_mem_map when there is no page
[ppc] virtual address translation bits for PPC44x
[ppc] Support for platform based Virtual address translation
[ppc] Non-linear address translation routine
defs.h | 101 +++++++++++++++++---
memory.c | 4 -
ppc.c | 318 ++++++++++++++++++++++++++++++++++++++------------------------
3 files changed, 280 insertions(+), 143 deletions(-)
--
Suzuki K. Poulose
12 years, 4 months
Re: [Crash-utility] "char-mapper"
by Dave Anderson
----- Original Message -----
> Hi Dave,
>
> Just to throw another wrench into the mix, you might find customized character
> classifications useful. First, you'd specify an interesting
> category:
>
> redirection "|!>"
>
> then you generate a table and some macros and then:
>
> p1 = BRK_REDIRECTION_CHARS(p2);
> if (*p1 != NUL) {
> fp = handle_redirection(p1);
> *p1 = NUL;
> }
>
> To play with it, install gperf and autogen, git-clone the autogen sources,
> go into autogen/add-on/char-mapper and type "make".
>
> If you use it, I recommend checking the generated source into git/svn.
> The git repo is on both sourceforge and savannah.gnu.org
>
> Anyway, thanks for the patch, though I think there may still be a problem.
> Anything after the first pipe character can be arbitrary shell. viz:
> crash-command ! cat | sed s/// > file
> which is why I recommended the char classifier stuff.
> The BRK macro stops the scan on the first match of a character from
> the REDIRECTION class of characters.
Right - a combined ! and | would also be a problem. But that's easy
enough to fix.
Thanks again,
Dave
12 years, 4 months
Another parsing bug -- likely confusion caused by open parenthesis
by Bruce Korb
crash> p/x *(*(struct trace_page*)0xFFFF8804125CBA60).page | sed 's/^\$[0-9]* =/0xFFFF8804125CBA60 =/' >> trace-data.txt\n
gdb: gdb request failed: p/x *(*(struct trace_page*)0xFFFF8804125CBA60).page | sed 's/^\$[0-9]* =/0xFFFF8804125CBA60 =/'
crash> p/x *(*(struct trace_page*)0xFFFF8804125CBA60).page
$1467 = {
flags = 0x200000000000000,
_count = {
counter = 0x1
},
{
_mapcount = {
counter = 0xffffffff
},
{
inuse = 0xffff,
objects = 0xffff
}
},
{
{
private = 0x0,
mapping = 0x0
},
ptl = {
raw_lock = {
slock = 0x0
}
},
slab = 0x0,
first_page = 0x0
},
{
index = 0x0,
freelist = 0x0,
reserve = 0x0,
frag_count = {
counter = 0x0
}
},
lru = {
next = 0xdead000000100100,
prev = 0xdead000000200200
}
}
==============
P.S. speaking of dead horses: were the fprintf(fp...) placed *before*
the command pipe redirection (like when the command had just been read
in and not yet scanned), *THEN* the output would go to the right place.
Here, the command scanning just goes off into the weeds and misses the
pipe operator entirely.
12 years, 4 months
[PATCH] x86_64: print wrong value for RDX register.
by Seiichi Ikarashi
Hi Dave,
crash has a funny bug that bt command wrongly shows RCX value
as RDX value for exception stack of x86_64 xen hypervisor.
Here is an example.
RDX is 0xcfc but bt shows 0x4 (== RCX) for it.
ffff828c802f0fa0: 0000000000000246 00000000ffffffff F...............
R8 RAX
ffff828c802f0fb0: 0000000000000004 0000000000000cfc ................
RCX ^^^^ RDX ^^^^
ffff828c802f0fc0: 0000000000000000 00000000800f0000 ................
RSI RDI
crash> bt
PCPU: 0 VCPU: ffff8300695be080
#0 [ffff828c802f0f40] kexec_crash at ffff828c8010d227
#1 [ffff828c802f0f50] handle_ist_exception at ffff828c801926f7
[exception RIP: pci_conf_read+130]
RIP: ffff828c801537e2 RSP: ffff828c802f7de0 RFLAGS: 00000006
RAX: 00000000ffffffff RBX: 0000000000000004 RCX: 0000000000000004
RDX: 0000000000000004 RSI: 0000000000000000 RDI: 00000000800f0000
^^^^
RBP: 0000000000000cfc R8: 0000000000000246 R9: 0000000000000000
R10: 0000000000000000 R11: 000000000000ffff R12: 0000000000000004
R13: 0000000000000000 R14: 0000000000000000 R15: ffff8300695be080
ORIG_RAX: 0000000200000000 CS: e008 SS: 0000
--- <exception stack> ---
#2 [ffff828c802f7de0] guest_io_read at ffff828c80148345
#3 [ffff828c802f7e20] do_general_protection at ffff828c8014a00c
#4 [ffff828c802f7eb0] reprogram_timer at ffff828c8012c55e
#5 [ffff828c802f7f20] handle_exception_saved at ffff828c80192495
crash>
Signed-off-by: Seiichi Ikarashi <s.ikarashi(a)jp.fujitsu.com>
--- a/x86_64.c 2012-02-04 04:22:33.000000000 +0900
+++ b/x86_64.c 2012-02-21 17:04:09.032121837 +0900
@@ -6562,7 +6562,7 @@ x86_64_print_eframe_regs_hyper(struct bt
fprintf(fp, " RAX: %016lx RBX: %016lx RCX: %016lx\n",
up[10], up[5], up[11]);
fprintf(fp, " RDX: %016lx RSI: %016lx RDI: %016lx\n",
- up[11], up[13], up[14]);
+ up[12], up[13], up[14]);
fprintf(fp, " RBP: %016lx R8: %016lx R9: %016lx\n",
up[4], up[9], up[8]);
fprintf(fp, " R10: %016lx R11: %016lx R12: %016lx\n",
12 years, 4 months
[PATCH v3 0/3] vmalloc translation support for PPC
by Suzuki K. Poulose
The following series implements :
* An infrastructure for platform based vmalloc translation for PPC32
* vmalloc translation support for PPC44x
Changes since V2:
* Rebased to crash-6.0.3
* Maintains a list of probe functions, rather than platform
definitions.
Each platform can define their own probe_function which would get the name of the
ppc platform (read from kernel) and the probe can check if the platform is one of its
variant. The probe function can then update the 'platform' defintions for the virtual
address translation.
If none of the defined platforms match, falls back to using the default PPC32
definitions.
---
Suzuki K. Poulose (3):
[ppc] virtual address translation bits for PPC44x
[ppc] Support for platform based Virtual address translation
[ppc] Non-linear address translation routine
defs.h | 19 ++++
ppc.c | 282 +++++++++++++++++++++++++++++++++++++---------------------------
2 files changed, 178 insertions(+), 123 deletions(-)
--
Suzuki Poulose
12 years, 4 months
