[PATCH] crash: Add LZO Compression Support
by HATAYAMA Daisuke
Hello Dave,
This is LZO patch for crash utility. makedumpfile will support LZO
compression in version 1.4.4, and it will be released this or next
month.
LZO patch for makedumpfile is here:
[PATCH makedumpfile 0/2] LZO Compression Support
http://lists.infradead.org/pipermail/kexec/2012-February/006075.html
Thanks.
HATAYAMA, Daisuke
12 years, 1 month
- 1
- 0
[PATCH makedumpfile v2 0/4] LZO Compression Support
by HATAYAMA Daisuke
The following series implements LZO compression support to
makedumpfile. LZO is as good as in size but by far better in speed
than ZLIB, readucing down time during generation of crash dump and
refiltering.
The RFC discussion was made here:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
How to build:
1. Get lzo libraries: lzo, lzo-devel and lzo-minilzo from either of
the following:
1) Original author's website:
http://www.oberhumer.com/opensource/lzo/
2) yum framework on fedora. Older releases don't have the packages.
2. Apply the patch set to makedumpfile v1.4.2.
3. Do make as follows:
$ make USELZO=on
Note: In default, no LZO compression support is included.
How to use:
Introduce new -l option. If a user specify this, makedumpfile
generates dumpfile compressed by pages with lzo compression.
Example)
$ makedumpfile -l vmcore dumpfile
Performance evaluation:
- Kumagai-san's evaluation simulating working servers:
http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
- My evaluation focusing on the worst cases:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
LZO Support for crash:
I'll post LZO support patch for crash after makedumpfile merges
these patches.
Changelog:
v1 => v2:
- Add build condition for LZO support. Enable LZO support if
specifying USELZO=on to make command.
- Avoid LONG_MAX/ULONG_MAX redefinitions.
---
HATAYAMA Daisuke (4):
Add build condition for LZO support
Add help and manual messages about LZO compression support
Avoid LONG_MAX/ULONG_MAX redefinitions
Add LZO Support
Makefile | 5 ++++
common.h | 4 +++
diskdump_mod.h | 3 ++-
makedumpfile.8 | 6 +++--
makedumpfile.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++------
makedumpfile.h | 4 +++
print_info.c | 16 +++++++------
7 files changed, 86 insertions(+), 19 deletions(-)
--
HATAYAMA Daisuke
12 years, 1 month
- 3
- 14
Fwd: s390x fixes
by Dave Anderson
----- Forwarded Message -----
From: "Michael Holzheu" <holzheu(a)linux.vnet.ibm.com>
To: "Dave Anderson" <anderson(a)redhat.com>
Cc: crash-utility-owner(a)redhat.com
Sent: Wednesday, May 2, 2012 8:49:56 AM
Subject: Re: s390x fixes
Hi Dave,
On Mon, 30 Apr 2012 16:53:46 -0400 (EDT)
Dave Anderson <anderson(a)redhat.com> wrote:
> I've got a couple simple bug fixes for s390x that I want to
> run by you, plus a third one that I don't have a fix for.
>
> First the easy ones:
>
> (1) "bt -t" and "bt -T" fail on the active task on a live system:
>
> crash> bt -t
> PID: 34875 TASK: 14342540 CPU: 1 COMMAND: "crash"
> bt: invalid/stale stack pointer for this task: 0
> crash> bt -T
> PID: 34875 TASK: 14342540 CPU: 1 COMMAND: "crash"
> bt: invalid/stale stack pointer for this task: 0
> crash>
>
> That can be fixed by adding a !LIVE() check to s390x_get_stack_frame()
> so that it will use (bt->task + OFFSET(task_struct_thread_ksp):
>
> /* get the stack pointer */
> if(esp){
> - if(s390x_has_cpu(bt)){
> + if (!LIVE() && s390x_has_cpu(bt)) {
> ksp = ULONG(lowcore +
> MEMBER_OFFSET("_lowcore", "gpregs_save_area") + (15 *
> S390X_WORD_SIZE)); } else {
> readmem(bt->task +
> OFFSET(task_struct_thread_ksp), KVADDR, &ksp, sizeof(void *),
> "thread_struct ksp", FAULT_ON_ERROR);
> }
> *esp = ksp;
> } else {
>
Looks good, thanks!
> (2) "vm -p" can show bogus data when a page is not mapped, like this
> example:
>
> crash> vm -p 1
> PID: 1 TASK: 17b91120 CPU: 1 COMMAND: "init"
> MM PGD RSS TOTAL_VM
> 14f48400 14f4c000 344k 3116k
> VMA START END FLAGS FILE
> 14b88c80 2aab283b000 2aab2862000
> 8001875 /sbin/init VIRTUAL PHYSICAL
> 2aab283b000 SWAP: (unknown swap location) OFFSET: 0
> 2aab283c000 SWAP: (unknown swap location) OFFSET: 0
> 2aab283d000 SWAP: (unknown swap location) OFFSET: 0
> 2aab283e000 SWAP: (unknown swap location) OFFSET: 0
> 2aab283f000 SWAP: (unknown swap location) OFFSET: 0
> 2aab2840000 SWAP: (unknown swap location) OFFSET: 0
> 2aab2841000 SWAP: (unknown swap location) OFFSET: 0
> ...
>
> And that's because when a "machdep->uvtop()" operation is done on a
> user page that is not resident, the machine-dependent function should
> return FALSE -- but it should return the PTE value in the paddr
> pointer field so that it can be translated by vm_area_page_dump().
> The s390x_uvtop() does not return the PTE, so the failed output can
> vary, because it's using an uninitialized "paddr" stack variable.
> But this is another easy fix, in this case to s390x_vtop():
>
> /* lookup virtual address in page tables */
> int s390x_vtop(ulong table, ulong vaddr, physaddr_t *phys_addr, int
> verbose) {
> ulong entry, paddr;
> int level, len;
>
> + *phys_addr = 0;
Looks also good. But probably I should implement a better fix that
returns the pte for s390x swap entries.
>
> (3) Even with the (2) applied, however, "vm -p" can fail to translate
> user addresses in another situation. If you try this, you'll
> see a number of failures like this:
>
> crash> foreach user vm -p | grep PID
> PID: 1 TASK: 17b91120 CPU: 1 COMMAND: "init"
> PID: 599 TASK: 14fbc140 CPU: 1 COMMAND: "udevd"
> PID: 955 TASK: 14343620 CPU: 0 COMMAND: "udevd"
> PID: 961 TASK: 13f19220 CPU: 1 COMMAND: "udevd"
> PID: 1246 TASK: 14cc0ab0 CPU: 0 COMMAND: "auditd"
> PID: 1247 TASK: 14f88240 CPU: 0 COMMAND: "auditd"
> PID: 1271 TASK: 140a3320 CPU: 0 COMMAND: "rsyslogd"
> vm: read error: kernel virtual address: 0 type: "entry"
> PID: 1272 TASK: 14b11520 CPU: 0 COMMAND: "rs:main
> Q:Reg" vm: read error: kernel virtual address: 0 type: "entry"
> PID: 1273 TASK: 16a32440 CPU: 1 COMMAND: "rsyslogd"
> vm: read error: kernel virtual address: 0 type: "entry"
> PID: 1274 TASK: 14c3cbb0 CPU: 0 COMMAND: "rsyslogd"
> vm: read error: kernel virtual address: 0 type: "entry"
> ...
>
> And if I take a particular case:
>
> crash> vm -p
> PID: 5088 TASK: 14399420 CPU: 1 COMMAND: "mingetty"
> MM PGD RSS TOTAL_VM
> 14e49c00 147f8000 116k 2180k
> ... [ cut ] ...
> VMA START END FLAGS FILE
> 14c49bc0 8dee1000 8df02000 100073
> VIRTUAL PHYSICAL
> 8dee1000 ef03000
> 8dee2000 (not mapped)
> 8dee3000 (not mapped)
> 8dee4000 (not mapped)
> 8dee5000 (not mapped)
> 8dee6000 (not mapped)
> 8dee7000 (not mapped)
> 8dee8000 (not mapped)
> 8dee9000 (not mapped)
> 8deea000 (not mapped)
> 8deeb000 (not mapped)
> 8deec000 (not mapped)
> 8deed000 (not mapped)
> 8deee000 (not mapped)
> 8deef000 (not mapped)
> 8def0000 (not mapped)
> 8def1000 (not mapped)
> 8def2000 (not mapped)
> 8def3000 (not mapped)
> 8def4000 (not mapped)
> 8def5000 (not mapped)
> 8def6000 (not mapped)
> 8def7000 (not mapped)
> 8def8000 (not mapped)
> 8def9000 (not mapped)
> 8defa000 (not mapped)
> 8defb000 (not mapped)
> 8defc000 (not mapped)
> 8defd000 (not mapped)
> 8defe000 (not mapped)
> 8deff000 (not mapped)
> vm: read error: kernel virtual address: 0 type: "entry"
> crash>
>
> So in this example, the page that's failing is 8df00000, which is
> located in the VMA's range from 8dee1000 to 8df02000. But the
> machdep->uvtop() operation fails unexpectedly:
>
> crash> vtop -u 8df00000 -u
> VIRTUAL PHYSICAL
> vtop: read error: kernel virtual address: 0 type: "entry"
> crash>
>
> And that "entry" readmem() is in s390x.c code that I don't wish
> to screw around with...
See reply to your other note.
Michael
12 years, 2 months
- 2
- 3
Fwd: s390x fixes
by Dave Anderson
Mistakenly cc'd to "crash-utility-owner(a)redhat.com" instead of this
list...
----- Forwarded Message -----
From: "Dave Anderson" <anderson(a)redhat.com>
To: "Michael Holzheu" <holzheu(a)linux.vnet.ibm.com>
Cc: crash-utility-owner(a)redhat.com
Sent: Monday, April 30, 2012 4:53:46 PM
Subject: s390x fixes
Hi Michael,
I've got a couple simple bug fixes for s390x that I want to
run by you, plus a third one that I don't have a fix for.
First the easy ones:
(1) "bt -t" and "bt -T" fail on the active task on a live system:
crash> bt -t
PID: 34875 TASK: 14342540 CPU: 1 COMMAND: "crash"
bt: invalid/stale stack pointer for this task: 0
crash> bt -T
PID: 34875 TASK: 14342540 CPU: 1 COMMAND: "crash"
bt: invalid/stale stack pointer for this task: 0
crash>
That can be fixed by adding a !LIVE() check to s390x_get_stack_frame()
so that it will use (bt->task + OFFSET(task_struct_thread_ksp):
/* get the stack pointer */
if(esp){
- if(s390x_has_cpu(bt)){
+ if (!LIVE() && s390x_has_cpu(bt)) {
ksp = ULONG(lowcore + MEMBER_OFFSET("_lowcore",
"gpregs_save_area") + (15 * S390X_WORD_SIZE));
} else {
readmem(bt->task + OFFSET(task_struct_thread_ksp),
KVADDR, &ksp, sizeof(void *),
"thread_struct ksp", FAULT_ON_ERROR);
}
*esp = ksp;
} else {
(2) "vm -p" can show bogus data when a page is not mapped, like this example:
crash> vm -p 1
PID: 1 TASK: 17b91120 CPU: 1 COMMAND: "init"
MM PGD RSS TOTAL_VM
14f48400 14f4c000 344k 3116k
VMA START END FLAGS FILE
14b88c80 2aab283b000 2aab2862000 8001875 /sbin/init
VIRTUAL PHYSICAL
2aab283b000 SWAP: (unknown swap location) OFFSET: 0
2aab283c000 SWAP: (unknown swap location) OFFSET: 0
2aab283d000 SWAP: (unknown swap location) OFFSET: 0
2aab283e000 SWAP: (unknown swap location) OFFSET: 0
2aab283f000 SWAP: (unknown swap location) OFFSET: 0
2aab2840000 SWAP: (unknown swap location) OFFSET: 0
2aab2841000 SWAP: (unknown swap location) OFFSET: 0
...
And that's because when a "machdep->uvtop()" operation is done on a user
page that is not resident, the machine-dependent function should return
FALSE -- but it should return the PTE value in the paddr pointer field
so that it can be translated by vm_area_page_dump(). The s390x_uvtop()
does not return the PTE, so the failed output can vary, because it's using
an uninitialized "paddr" stack variable. But this is another easy fix,
in this case to s390x_vtop():
/* lookup virtual address in page tables */
int s390x_vtop(ulong table, ulong vaddr, physaddr_t *phys_addr, int verbose)
{
ulong entry, paddr;
int level, len;
+ *phys_addr = 0;
(3) Even with the (2) applied, however, "vm -p" can fail to translate
user addresses in another situation. If you try this, you'll
see a number of failures like this:
crash> foreach user vm -p | grep PID
PID: 1 TASK: 17b91120 CPU: 1 COMMAND: "init"
PID: 599 TASK: 14fbc140 CPU: 1 COMMAND: "udevd"
PID: 955 TASK: 14343620 CPU: 0 COMMAND: "udevd"
PID: 961 TASK: 13f19220 CPU: 1 COMMAND: "udevd"
PID: 1246 TASK: 14cc0ab0 CPU: 0 COMMAND: "auditd"
PID: 1247 TASK: 14f88240 CPU: 0 COMMAND: "auditd"
PID: 1271 TASK: 140a3320 CPU: 0 COMMAND: "rsyslogd"
vm: read error: kernel virtual address: 0 type: "entry"
PID: 1272 TASK: 14b11520 CPU: 0 COMMAND: "rs:main Q:Reg"
vm: read error: kernel virtual address: 0 type: "entry"
PID: 1273 TASK: 16a32440 CPU: 1 COMMAND: "rsyslogd"
vm: read error: kernel virtual address: 0 type: "entry"
PID: 1274 TASK: 14c3cbb0 CPU: 0 COMMAND: "rsyslogd"
vm: read error: kernel virtual address: 0 type: "entry"
...
And if I take a particular case:
crash> vm -p
PID: 5088 TASK: 14399420 CPU: 1 COMMAND: "mingetty"
MM PGD RSS TOTAL_VM
14e49c00 147f8000 116k 2180k
... [ cut ] ...
VMA START END FLAGS FILE
14c49bc0 8dee1000 8df02000 100073
VIRTUAL PHYSICAL
8dee1000 ef03000
8dee2000 (not mapped)
8dee3000 (not mapped)
8dee4000 (not mapped)
8dee5000 (not mapped)
8dee6000 (not mapped)
8dee7000 (not mapped)
8dee8000 (not mapped)
8dee9000 (not mapped)
8deea000 (not mapped)
8deeb000 (not mapped)
8deec000 (not mapped)
8deed000 (not mapped)
8deee000 (not mapped)
8deef000 (not mapped)
8def0000 (not mapped)
8def1000 (not mapped)
8def2000 (not mapped)
8def3000 (not mapped)
8def4000 (not mapped)
8def5000 (not mapped)
8def6000 (not mapped)
8def7000 (not mapped)
8def8000 (not mapped)
8def9000 (not mapped)
8defa000 (not mapped)
8defb000 (not mapped)
8defc000 (not mapped)
8defd000 (not mapped)
8defe000 (not mapped)
8deff000 (not mapped)
vm: read error: kernel virtual address: 0 type: "entry"
crash>
So in this example, the page that's failing is 8df00000, which is
located in the VMA's range from 8dee1000 to 8df02000. But the
machdep->uvtop() operation fails unexpectedly:
crash> vtop -u 8df00000 -u
VIRTUAL PHYSICAL
vtop: read error: kernel virtual address: 0 type: "entry"
crash>
And that "entry" readmem() is in s390x.c code that I don't wish
to screw around with...
Hoping you can help,
Dave
12 years, 2 months
- 2
- 3