Re: [Crash-utility] Kernel Crash Analysis on Android
by Shankar, AmarX
Hi Dave,
Thanks for your info regarding kexec tool.
I am unable to download kexec from below link.
http://www.kernel.org/pub/linux/kernel/people/horms/kexec-tools/kexec-too...
It says HTTP 404 Page Not Found.
Could you please guide me on this?
Thanks & Regards,
Amar Shankar
> On Wed, Mar 21, 2012 at 06:00:00PM +0000, Shankar, AmarX wrote:
>
> > I want to do kernel crash Analysis on Android Merrifield Target.
> >
> > Could someone please help me how to do it?
>
> Merrifield is pretty much similar than Medfield, e.g it has x86 core. So I
> guess you can follow the instructions how to setup kdump on x86 (see
> Documentation/kdump/kdump.txt) unless you already have that configured.
>
> crash should support this directly presuming you have vmlinux/vmcore files to
> feed it. You can configure crash to support x86 on x86_64 host by running:
>
> % make target=X86
> & make
>
> (or something along those lines).
Right -- just the first make command will suffice, i.e., when running
on an x86_64 host:
$ wget http://people.redhat.com/anderson/crash-6.0.4.tar.gz
$ tar xzf crash-6.0.4.tar.gz
...
$ cd crash-6.0.4
$ make target=X86
...
$ ./crash <path-to>/vmlinux <path-to>/vmcore
Dave
From: Shankar, AmarX
Sent: Wednesday, March 21, 2012 11:30 PM
To: 'crash-utility(a)redhat.com'
Subject: Kernel Crash Analysis on Android
Hi,
I want to do kernel crash Analysis on Android Merrifield Target.
Could someone please help me how to do it?
Thanks & Regards,
Amar Shankar
7 months, 2 weeks
[PATCH] kmem, snap: iomem/ioport display and vmcore snapshot support
by HATAYAMA Daisuke
Some days ago I was in a situation that I had to convert vmcore in
kvmdump format into ELF since some extension module we have locally
can be used only on relatively old crash utility, around version 4,
but such old crash utility cannot handle kvmdump format.
To do the conversion in handy, I used snap command with some modifications
so that it tries to use iomem information in vmcore instead of host's
/proc/iomem. This patch is its cleaned-up version.
In this development, I naturally got down to also making an interface
for an access to resource objects, and so together with the snap
command's patch, I also extended kmem command for iomem/ioport
support. Actually:
kmem -r displays /proc/iomem
crash> kmem -r
00000000-0000ffff : reserved
00010000-0009dbff : System RAM
0009dc00-0009ffff : reserved
000c0000-000c7fff : Video ROM
...
and kmem -R displays /proc/ioport
crash> kmem -R
0000-001f : dma1
0020-0021 : pic1
0040-0043 : timer0
0050-0053 : timer1
...
Looking into old version of kernel source code back, resource structure
has been unchanged since linux-2.4.0. I borrowed the way of walking on
resouce tree in this patch from the lastest v3.3-rc series, but I
guess the logic is also applicable to old kernels. I expect Dave's
regression testsuite.
Also, there would be another command more sutable for iomem/ioport.
If necessay, I'll repost the patch.
---
HATAYAMA Daisuke (4):
Add vmcore snapshot support
Add kmem -r and -R options
Add dump iomem/ioport functions; a helper for resource objects
Add a helper function for iterating resource objects
defs.h | 9 ++++
extensions/snap.c | 54 ++++++++++++++++++++++-
help.c | 2 +
memory.c | 122 +++++++++++++++++++++++++++++++++++++++++++++++++++--
4 files changed, 180 insertions(+), 7 deletions(-)
--
Thanks.
HATAYAMA Daisuke
7 months, 2 weeks
Re: [Crash-utility] question about phys_base
by Dave Anderson
----- Original Message -----
> >
> > OK, so then I don't understand what you mean by "may be the same"?
> >
> > You didn't answer my original question, but if I understand you correctly,
> > it would be impossible for the qemu host to create a PT_LOAD segment that
> > describes an x86_64 guest's __START_KERNEL_map region, because the host
> > doesn't know that what kind of kernel the guest is running.
>
> Yes. Even if the guest is linux, it is still impossible to do it. Because
> the guest maybe in the second kernel.
>
> qemu-dump walks all guest's page table and collect virtual address and
> physical address mapping. If the page is not used by guest, the virtual is set
> to 0. I create PT_LOAD according to such mapping. So if the guest is linux,
> there may be a PT_LOAD segment that describes __START_KERNEL_map region.
> But the information stored in PT_LOAD maybe for the second kernel. If crash
> uses it, crash will see the second kernel, not the first kernel.
Just to be clear -- what do you mean by the "second" kernel? Do you
mean that a guest kernel crashed guest, and did a kdump operation,
and that second kdump kernel failed somehow, and now you're trying
to do a "virsh dump" on the kdump kernel?
Dave
7 months, 2 weeks
question about phys_base
by Wen Congyang
Hi, Dave
I am implementing a new dump command in the qemu. The vmcore's
format is elf(like kdump). And I try to provide phys_base in
the PT_LOAD. But if the os uses the first vcpu do kdump, the
value of phys_base is wrong.
I find a function x86_64_virt_phys_base() in crash's code.
Is it OK to call this function first? If the function
successes, we do not calculate phys_base according to PT_LOAD.
Thanks
Wen Congyang
7 months, 2 weeks
[PATCH] runq: search current task's runqueue explicitly
by HATAYAMA Daisuke
Currently, runq sub-command doesn't consider CFS runqueue's current
task removed from CFS runqueue. Due to this, the remaining CFS
runqueus that follow the current task's is not displayed. This patch
fixes this by making runq sub-command search current task's runqueue
explicitly.
Note that CFS runqueue exists for each task group, and so does CFS
runqueue's current task, and the above search needs to be done
recursively.
Test
====
On vmcore I made 7 task groups:
root group --- A --- AA --- AAA
+ +- AAB
|
+- AB --- ABA
+- ABB
and then I ran three CPU bound tasks, which is exactly the same as
int main(void) { for (;;) continue; return 0; }
for each task group, including root group; so total 24 tasks. For
readability, I annotated each task name with its belonging group name.
For example, loop.ABA belongs to task group ABA.
Look at CPU0 collumn below. [before] lacks 8 tasks and [after]
successfully shows all tasks on the runqueue, which is identical to
the result of [sched debug] that is expected to ouput correct result.
I'll send this vmcore later.
[before]
crash> runq | cat
CPU 0 RUNQUEUE: ffff88000a215f80
CURRENT: PID: 28263 TASK: ffff880037aaa040 COMMAND: "loop.ABA"
RT PRIO_ARRAY: ffff88000a216098
[no tasks queued]
CFS RB_ROOT: ffff88000a216010
[120] PID: 28262 TASK: ffff880037cc40c0 COMMAND: "loop.ABA"
<cut>
[after]
crash_fix> runq
CPU 0 RUNQUEUE: ffff88000a215f80
CURRENT: PID: 28263 TASK: ffff880037aaa040 COMMAND: "loop.ABA"
RT PRIO_ARRAY: ffff88000a216098
[no tasks queued]
CFS RB_ROOT: ffff88000a216010
[120] PID: 28262 TASK: ffff880037cc40c0 COMMAND: "loop.ABA"
[120] PID: 28271 TASK: ffff8800787a8b40 COMMAND: "loop.ABB"
[120] PID: 28272 TASK: ffff880037afd580 COMMAND: "loop.ABB"
[120] PID: 28245 TASK: ffff8800785e8b00 COMMAND: "loop.AB"
[120] PID: 28246 TASK: ffff880078628ac0 COMMAND: "loop.AB"
[120] PID: 28241 TASK: ffff880078616b40 COMMAND: "loop.AA"
[120] PID: 28239 TASK: ffff8800785774c0 COMMAND: "loop.AA"
[120] PID: 28240 TASK: ffff880078617580 COMMAND: "loop.AA"
[120] PID: 28232 TASK: ffff880079b5d4c0 COMMAND: "loop.A"
<cut>
[sched debug]
crash> runq -d
CPU 0
[120] PID: 28232 TASK: ffff880079b5d4c0 COMMAND: "loop.A"
[120] PID: 28239 TASK: ffff8800785774c0 COMMAND: "loop.AA"
[120] PID: 28240 TASK: ffff880078617580 COMMAND: "loop.AA"
[120] PID: 28241 TASK: ffff880078616b40 COMMAND: "loop.AA"
[120] PID: 28245 TASK: ffff8800785e8b00 COMMAND: "loop.AB"
[120] PID: 28246 TASK: ffff880078628ac0 COMMAND: "loop.AB"
[120] PID: 28262 TASK: ffff880037cc40c0 COMMAND: "loop.ABA"
[120] PID: 28263 TASK: ffff880037aaa040 COMMAND: "loop.ABA"
[120] PID: 28271 TASK: ffff8800787a8b40 COMMAND: "loop.ABB"
[120] PID: 28272 TASK: ffff880037afd580 COMMAND: "loop.ABB"
<cut>
Diff stat
=========
defs.h | 1 +
task.c | 37 +++++++++++++++++--------------------
2 files changed, 18 insertions(+), 20 deletions(-)
Thanks.
HATAYAMA, Daisuke
7 months, 2 weeks
[RFC] makedumpfile, crash: LZO compression support
by HATAYAMA Daisuke
Hello,
This is a RFC patch set that adds LZO compression support to
makedumpfile and crash utility. LZO is as good as in size but by far
better in speed than ZLIB, leading to reducing down time during
generation of crash dump and refiltering.
How to build:
1. Get LZO library, which is provided as lzo-devel package on recent
linux distributions, and is also available on author's website:
http://www.oberhumer.com/opensource/lzo/.
2. Apply the patch set to makedumpfile v1.4.0 and crash v6.0.0.
3. Build both using make. But for crash, do the following now:
$ make CFLAGS="-llzo2"
How to use:
I've newly used -l option for lzo compression in this patch. So for
example, do as follows:
$ makedumpfile -l vmcore dumpfile
$ crash vmlinux dumpfile
Request of configure-like feature for crash utility:
I would like configure-like feature on crash utility for users to
select wheather to add LZO feature actually or not in build-time,
that is: ./configure --enable-lzo or ./configure --disable-lzo.
The reason is that support staff often downloads and installs the
latest version of crash utility on machines where lzo library is not
provided.
Looking at the source code, it looks to me that crash does some kind
of configuration processing in a local manner, around configure.c,
and I guess it's difficult to use autoconf tools directly.
Or is there another better way?
Performance Comparison:
Sample Data
Ideally, I must have measured the performance for many enough
vmcores generated from machines that was actually running, but now
I don't have enough sample vmcores, I couldn't do so. So this
comparison doesn't answer question on I/O time improvement. This
is TODO for now.
Instead, I choosed worst and best cases regarding compression
ratio and speed only. Specifically, the former is /dev/urandom and
the latter is /dev/zero.
I get the sample data of 10MB, 100MB and 1GB by doing like this:
$ dd bs=4096 count=$((1024*1024*1024/4096)) if=/dev/urandom of=urandom.1GB
How to measure
Then I performed compression for each block, 4096 bytes, and
measured total compression time and output size. See attached
mycompress.c.
Result
See attached file result.txt.
Discussion
For both kinds of data, lzo's compression was considerably quicker
than zlib's. Compression ratio is about 37% for urandom data, and
about 8.5% for zero data. Actual situation of physical memory
would be in between the two cases, and so I guess average
compression time ratio is between 37% and 8.5%.
Although beyond the topic of this patch set, we can estimate worst
compression time on more data size since compression is performed
block size wise and the compression time increases
linearly. Estimated worst time on 2TB memory is about 15 hours for
lzo and about 40 hours for zlib. In this case, compressed data
size is larger than the original, so they are really not used,
compression time is fully meaningless. I think compression must be
done in parallel, and I'll post such patch later.
Diffstat
* makedumpfile
diskdump_mod.h | 3 +-
makedumpfile.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++------
makedumpfile.h | 12 +++++++
3 files changed, 101 insertions(+), 12 deletions(-)
* crash
defs.h | 1 +
diskdump.c | 20 +++++++++++++++++++-
diskdump.h | 3 ++-
3 files changed, 22 insertions(+), 2 deletions(-)
TODO
* evaluation including I/O time using actual vmcores
Thanks.
HATAYAMA, Daisuke
7 months, 2 weeks
Re: [Crash-utility] [RFI] Support Fujitsu's sadump dump format
by tachibana＠mxm.nes.nec.co.jp
Hi Hatayama-san,
On 2011/06/29 12:12:18 +0900, HATAYAMA Daisuke <d.hatayama(a)jp.fujitsu.com> wrote:
> From: Dave Anderson <anderson(a)redhat.com>
> Subject: Re: [Crash-utility] [RFI] Support Fujitsu's sadump dump format
> Date: Tue, 28 Jun 2011 08:57:42 -0400 (EDT)
>
> >
> >
> > ----- Original Message -----
> >> Fujitsu has stand-alone dump mechanism based on firmware level
> >> functionality, which we call SADUMP, in short.
> >>
> >> We've maintained utility tools internally but now we're thinking that
> >> the best is crash utility and makedumpfile supports the sadump format
> >> for the viewpoint of both portability and maintainability.
> >>
> >> We'll be of course responsible for its maintainance in a continuous
> >> manner. The sadump dump format is very similar to diskdump format and
> >> so kdump (compressed) format, so we estimate patch set would be a
> >> relatively small size.
> >>
> >> Could you tell me whether crash utility and makedumpfile can support
> >> the sadump format? If OK, we'll start to make patchset.
I think it's not bad to support sadump by makedumpfile. However I have
several questions.
- Do you want to use makedumpfile to make an existing file that sadump has
dumped small?
- It isn't possible to support the same form as kdump-compressed format
now, is it?
- When the information that makedumpfile reads from a note of /proc/vmcore
(or a header of kdump-compressed format) is added by an extension of
makedumpfile, do you need to modify sadump?
Thanks
tachibana
> >
> > Sure, yes, the crash utility can always support another dumpfile format.
> >
>
> Thanks. It helps a lot.
>
> > It's unclear to me how similar SADUMP is to diskdump/compressed-kdump.
> > Does your internal version patch diskdump.c, or do you maintain your
> > own "sadump.c"? I ask because if your patchset is at all intrusive,
> > I'd prefer it be kept in its own file, primarily for maintainability,
> > but also because SADUMP is essentially a black-box to anybody outside
> > Fujitsu.
>
> What I meant when I used ``similar'' is both literally and
> logically. The format consists of diskdump header-like header, two
> kinds of bitmaps used for the same purpose as those in diskump format,
> and memory data. They can be handled in common with the existing data
> structure, diskdump_data, non-intrusively, so I hope they are placed
> in diskdump.c.
>
> On the other hand, there's a code to be placed at such specific
> area. sadump is triggered depending on kdump's progress and so
> register values to be contained in vmcore varies according to the
> progress: If crash_notes has been initialized when sadump is
> triggered, sadump packs the register values in crash_notes; if not
> yet, packs registers gathered by firmware. This is sadump specific
> processing, so I think putting it in specific sadump.c file is a
> natural and reasonable choise.
>
> Anyway, I have not made any patch set for this. I'll post a patch set
> when I complete.
>
> Again, thanks a lot for the positive answer.
>
> Thanks.
> HATAYAMA, Daisuke
>
>
> _______________________________________________
> kexec mailing list
> kexec(a)lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
7 months, 2 weeks
Re: [Crash-utility] crash: read error on type: "memory section root table"
by HAGIO KAZUHITO(萩尾 一仁)
-----Original Message-----
> Hello,
>
> Sorry to cross post on both ML, I'm not sure which one would be the most suitable.
>
> Issue on analysis with crash-7.3.1 on a Centos 8 machine:
> crash: read error: kernel virtual address: ffff8f4fff7fc000 type: "memory section root table"
>
> Crash machine has a Rocky Linux 8.5 based kernel with following config options:
> - CONFIG_RANDOMIZE_BASE=y
> - CONFIG_RANDOMIZE_MEMORY=y
> - CONFIG_SPARSEMEM_MANUAL=y
> - CONFIG_SPARSEMEM=y
> - CONFIG_SPARSEMEM_EXTREME=y
> - CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
> - CONFIG_KEXEC_CORE=y
> - CONFIG_KEXEC=y
> - CONFIG_KEXEC_FILE=y
>
> Kexec-tools package is from Centos Stream repo: kexec-tools-2.0.20-68.el8.2.5ale.x86_64
>
> /proc/vmcore is packaged with :
> /sbin/makedumpfile -D -d 0 -c --message-level 15 /proc/vmcore /tmpd/crashdump-${linux_ver}-${date_time}
>
> At kernel panic, I get:
> Dumping memory to crash partition
> This may take a while, please wait...
> makedumpfile: version 1.7.0 (released on 8 Nov 2021)
> command line: /sbin/makedumpfile -D -d 0 -c --message-level 15 /proc/vmcore /tmpd/crashdump--20220329-1538
>
> sadump: does not have partition header
> sadump: read dump device as unknown format
> sadump: unknown format
> phys_start phys_end virt_start virt_end
> LOAD[ 0] 8000000 9a2c000 ffffffff8a400000 ffffffff8be2c000
> LOAD[ 1] 100000 3b000000 ffff8f4fc0100000 ffff8f4ffb000000
> LOAD[ 2] 3d800000 3e341000 ffff8f4ffd800000 ffff8f4ffe341000
> LOAD[ 3] 3ed7b000 3eee2000 ffff8f4ffed7b000 ffff8f4ffeee2000
> LOAD[ 4] 3f63a000 3f800000 ffff8f4fff63a000 ffff8f4fff800000
> Linux kdump
> VMCOREINFO :
> OSRELEASE=4.18.0-348.12.2.el8_5-ale
> PAGESIZE=4096
> page_size : 4096
> SYMBOL(init_uts_ns)=ffffffff8b653600
> SYMBOL(node_online_map)=ffffffff8b7630a8
> SYMBOL(swapper_pg_dir)=ffffffff8b64c000
> SYMBOL(_stext)=ffffffff8a400000
> SYMBOL(vmap_area_list)=ffffffff8b6a47a0
> SYMBOL(mem_map)=ffffffff8bd25828
> SYMBOL(contig_page_data)=ffffffff8b726600
> SYMBOL(mem_section)=ffff8f4fff7fc000
hm, probably I've never seen a system that has both mem_map and mem_section, but
it looks like makedumpfile works fine.. i.e. recognizes it as SPARSEMEM_EXTREME
correctly.
> LENGTH(mem_section)=2048
> SIZE(mem_section)=16
> OFFSET(mem_section.section_mem_map)=0
> SIZE(page)=64
> SIZE(pglist_data)=5696
> SIZE(zone)=1216
> SIZE(free_area)=72
> SIZE(list_head)=16
> SIZE(nodemask_t)=8
> OFFSET(page.flags)=0
> OFFSET(page._refcount)=52
> OFFSET(page.mapping)=24
> OFFSET(page.lru)=8
> OFFSET(page._mapcount)=48
> OFFSET(page.private)=40
> OFFSET(page.compound_dtor)=16
> OFFSET(page.compound_order)=17
> OFFSET(page.compound_head)=8
> OFFSET(pglist_data.node_zones)=0
> OFFSET(pglist_data.nr_zones)=4944
> OFFSET(pglist_data.node_start_pfn)=4952
> OFFSET(pglist_data.node_spanned_pages)=4968
> OFFSET(pglist_data.node_id)=4976
> OFFSET(zone.free_area)=192
> OFFSET(zone.vm_stat)=1104
> OFFSET(zone.spanned_pages)=96
> OFFSET(free_area.free_list)=0
> OFFSET(list_head.next)=0
> OFFSET(list_head.prev)=8
> OFFSET(vmap_area.va_start)=0
> OFFSET(vmap_area.list)=40
> LENGTH(zone.free_area)=11
> SYMBOL(log_buf)=ffffffff8b67d3c0
> SYMBOL(log_buf_len)=ffffffff8b67d3bc
> SYMBOL(log_first_idx)=ffffffff8bd1a3d8
> SYMBOL(clear_idx)=ffffffff8bd1a3a4
> SYMBOL(log_next_idx)=ffffffff8bd1a3c8
> SIZE(printk_log)=16
> OFFSET(printk_log.ts_nsec)=0
> OFFSET(printk_log.len)=8
> OFFSET(printk_log.text_len)=10
> OFFSET(printk_log.dict_len)=12
> LENGTH(free_area.free_list)=4
> NUMBER(NR_FREE_PAGES)=0
> NUMBER(PG_lru)=5
> NUMBER(PG_private)=12
> NUMBER(PG_swapcache)=9
> NUMBER(PG_swapbacked)=18
> NUMBER(PG_slab)=8
> NUMBER(PG_head_mask)=32768
> NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129
> NUMBER(HUGETLB_PAGE_DTOR)=2
> NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257
> SYMBOL(alcatel_dump_info)=ffffffff8b647000
> NUMBER(phys_base)=-37748736
> SYMBOL(init_top_pgt)=ffffffff8b64c000
> NUMBER(pgtable_l5_enabled)=0
> KERNELOFFSET=9400000
> NUMBER(KERNEL_IMAGE_SIZE)=1073741824
> NUMBER(sme_mask)=0
> CRASHTIME=1648561077
>
> phys_base : fffffffffdc00000 (vmcoreinfo)
>
> max_mapnr : 3f800
> There is enough free memory to be done in one cycle.
>
> Buffer size for the cyclic mode: 65024
> page_offset : ffff8f4fc0000000 (pt_load)
> num of NODEs : 1
> Memory type : SPARSEMEM_EX
>
> mem_map pfn_start pfn_end
> mem_map[ 0] ffff8f4ffa000000 0 8000
> mem_map[ 1] ffff8f4ffa200000 8000 10000
> mem_map[ 2] ffff8f4ffa400000 10000 18000
> mem_map[ 3] ffff8f4ffa600000 18000 20000
> mem_map[ 4] ffff8f4ffa800000 20000 28000
> mem_map[ 5] ffff8f4ffaa00000 28000 30000
> mem_map[ 6] ffff8f4ffac00000 30000 38000
> mem_map[ 7] ffff8f4ffae00000 38000 3f800
> mmap() is available on the kernel.
> Copying data : [100.0 %] | eta: 0s
> Writing erase info...
> offset_eraseinfo: ca157f3, size_eraseinfo: 0
>
> The dumpfile is saved to /tmpd/crashdump--20220329-1538.
>
> makedumpfile Completed.
> Rebooting the system...
>
> And latest logs from a 'crash -d 7' command are:
> <.>
> kernel NR_CPUS: 2
> <readmem: ffffffff8bd25820, KVADDR, "high_memory", 8, (FOE), 55e05ecb3608>
> <read_diskdump: addr: ffffffff8bd25820 paddr: 9925820 cnt: 8>
> PAGESIZE=4096
> mem_section_size = 16384
> NR_SECTION_ROOTS = 2048
> NR_MEM_SECTIONS = 524288
> SECTIONS_PER_ROOT = 256
> SECTION_ROOT_MASK = 0xff
> PAGES_PER_SECTION = 32768
> <readmem: ffffffff8bd26db0, KVADDR, "mem_section", 8, (FOE), 7ffdbf96a440>
> <read_diskdump: addr: ffffffff8bd26db0 paddr: 9926db0 cnt: 8>
> <readmem: ffff8f4fff7fc000, KVADDR, "memory section root table", 16384, (FOE), 55e06391b840>
> <read_diskdump: addr: ffff8f4fff7fc000 paddr: 3f7fc000 cnt: 4096>
> crash: read error: kernel virtual address: ffff8f4fff7fc000 type: "memory section root table"
>
> The address (ffff8f4fff7fc000) seems to be inside the LOAD[4] range and is recorded as 'mem_section' with
> VMCOREINFO.
Yes, this says it's sane, and its paddr also looks sane..
So I'm not sure why read_diskdump() returns READ_ERROR, could you debug it?
I'm suspecting the read() below in cache_page() returns something, e.g.
--- a/diskdump.c
+++ b/diskdump.c
@@ -1189,10 +1189,13 @@ cache_page(physaddr_t paddr)
return PAGE_INCOMPLETE;
}
} else {
+ ssize_t r;
if (lseek(dd->dfd, pd.offset, SEEK_SET) == failed)
return SEEK_ERROR;
- if (read(dd->dfd, dd->compressed_page, pd.size) != pd.size)
+ if ((r = read(dd->dfd, dd->compressed_page, pd.size)) != pd.size) {
+ error(INFO, "errno=%d r=%ld pd.size=%u\n", errno, r, pd.size);
return READ_ERROR;
+ }
}
if (pd.flags & DUMP_DH_COMPRESSED_ZLIB) {
although another path may be returning it.
Thanks,
Kazu
1 year, 11 months
Re: [Crash-utility] [PATCH v2 4/4] make: replace make by $(MAKE)
by lijiang
Hi, Sven and Kazu
The patchset really saves compilation time for crash build and the
following three patches look good to me(need a minor adjustment as Kazu
mentioned).
Thank you for the work, Sven.
[PATCH v2 1/4] make: set --no-print-directory once
[PATCH v2 3/4] make: use -C instead of (cd x; make)
[PATCH v2 4/4] make: replace make by $(MAKE)
Given that the gdb patch has specific rules, Kazu(or me) can help to pack
them into one patch with Sven's signature, if you have no objections.
Or could you please update the patchset? Sven.
Thanks.
Lianbo
On Thu, Dec 23, 2021 at 5:14 PM <crash-utility-request(a)redhat.com> wrote:
> Date: Thu, 23 Dec 2021 00:33:16 +0000
> From: HAGIO KAZUHITO(?????) <k-hagio-ab(a)nec.com>
> To: Sven Schnelle <svens(a)linux.ibm.com>
> Cc: "Discussion list for crash utility usage, maintenance and
> development" <crash-utility(a)redhat.com>
> Subject: Re: [Crash-utility] [PATCH v2 4/4] make: replace make by
> $(MAKE)
> Message-ID:
> <
> OS3PR01MB677601D042591DF6451F477DDD7E9(a)OS3PR01MB6776.jpnprd01.prod.outlook.com
> >
>
> Content-Type: text/plain; charset="iso-2022-jp"
>
> Hi Sven,
>
> thank you for the update.
>
> -----Original Message-----
> > diff --git a/gdb-10.2.patch b/gdb-10.2.patch
> > index 1280d0688e83..afdbe99eae0d 100644
> > --- a/gdb-10.2.patch
> > +++ b/gdb-10.2.patch
> > @@ -73,7 +73,7 @@
> > # Removing the old gdb first works better if it is running, at least
> on SunOS.
> > gdb$(EXEEXT): gdb.o $(LIBGDB_OBS) $(CDEPS) $(TDEPLIBS)
> > $(SILENCE) rm -f gdb$(EXEEXT)
> > -+ @make -C ../.. GDB_FLAGS=-DGDB_10_2 library
> > ++ @$(MAKE) -C ../.. GDB_FLAGS=-DGDB_10_2 library
> > $(ECHO_CXXLD) $(CC_LD) $(INTERNAL_LDFLAGS) $(WIN32LDAPP) \
> > - -o gdb$(EXEEXT) gdb.o $(LIBGDB_OBS) \
> > - $(TDEPLIBS) $(TUI_LIBRARY) $(CLIBS) $(LOADLIBES)
>
> When detecting any change of the gdb patch, it tries to re-apply the new
> one
> using "patch -N --fuzz=0" in order to update the gdb. Please refer to
> Makefile::rebuild and the head of the gdb-10.2.patch.
>
> So I think that, to make a change to the gdb sources,
>
> 1. we have to "add" patches to the end of the gdb patch, and
> 2. if there are multiple patches for a gdb file, the "patch -N" doesn't
> work,
> so we have to revert the gdb file to the original one.
>
> (I will add this custom to crash guidelines later..)
>
> In your patches, there are multiple patches for gdb-10.2/gdb/Makefile.in
> so I would suggest that:
>
> 1. pack the 1/4, 3/4 and 4/4 patches into a patch, do the three things
> together.
> 2. add a hunk for the gdb-10.2/gdb/Makefile.in to the end of the
> gdb-10.2.patch.
> 3. add the following change to the head the gdb-10.2.patch.
>
> --- a/gdb-10.2.patch
> +++ b/gdb-10.2.patch
> @@ -8,6 +8,11 @@
> # shell script that can restore any gdb file to its original state prior
> # to all subsequent patch applications.
>
> +tar xvzmf gdb-10.2.tar.gz \
> + gdb-10.2/gdb/Makefile.in
> +
> +exit 0
> +
>
> (yeah, we forgot the "exit 0" here when moving to gdb-10.2..)
>
> Thanks,
> Kazu
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 23 Dec 2021 14:42:33 +0800
> From: Lianbo Jiang <lijiang(a)redhat.com>
> To: crash-utility(a)redhat.com
> Subject: [Crash-utility] [PATCH] Handle blk_mq_ctx member changes for
> kernels v5.16-rc1~75^2~44
> Message-ID: <20211223064233.16028-1-lijiang(a)redhat.com>
>
> Kernel commit <9a14d6ce4135> ("block: remove debugfs blk_mq_ctx
> dispatched/merged/completed attributes") removed the member
> rq_dispatched and rq_completed from struct blk_mq_ctx. Without
> this patch, crash will fail with the following error:
>
> crash> dev -d
> MAJOR GENDISK NAME REQUEST_QUEUE TOTAL ASYNC SYNC
>
> dev: invalid structure member offset: blk_mq_ctx_rq_dispatched
> FILE: dev.c LINE: 4229 FUNCTION: get_one_mctx_diskio()
>
> Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com>
> ---
> dev.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/dev.c b/dev.c
> index effe789f38d8..dd21511e5dfc 100644
> --- a/dev.c
> +++ b/dev.c
> @@ -4246,6 +4246,10 @@ get_mq_diskio(unsigned long q, unsigned long
> *mq_count)
> unsigned long mctx_addr;
> struct diskio tmp;
>
> + if (!MEMBER_EXISTS("blk_mq_ctx", "rq_dispatched") &&
> + !MEMBER_EXISTS("blk_mq_ctx", "rq_completed"))
> + return;
> +
> memset(&tmp, 0x00, sizeof(struct diskio));
>
> readmem(q + OFFSET(request_queue_queue_ctx), KVADDR, &queue_ctx,
> --
> 2.20.1
>
>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 23 Dec 2021 09:13:19 +0000
> From: "d.hatayama(a)fujitsu.com" <d.hatayama(a)fujitsu.com>
> To: "shogo.matsumoto(a)fujitsu.com" <shogo.matsumoto(a)fujitsu.com>
> Cc: "'crash-utility(a)redhat.com'" <crash-utility(a)redhat.com>
> Subject: Re: [Crash-utility] [PATCH] log: output logs of printk safe
> buffers
> Message-ID:
> <
> TYAPR01MB6507BFD2235A5C6806C3D0E5957E9(a)TYAPR01MB6507.jpnprd01.prod.outlook.com
> >
>
> Content-Type: text/plain; charset="iso-2022-jp"
>
> Hi,
>
> >
> > ________________________________________
> > From: crash-utility-bounces(a)redhat.com <crash-utility-bounces(a)redhat.com>
> on behalf of shogo.matsumoto(a)fujitsu.com <shogo.matsumoto(a)fujitsu.com>
> > Sent: Thursday, December 16, 2021 16:39
> > To: 'crash-utility(a)redhat.com'
> > Subject: [Crash-utility] [PATCH] log: output logs of printk safe buffers
> >
> > We sometimes overlook logs written to printk safe buffers
> > (safe_print_seq/nmi_print_seq) which have not been flushed yet.
> >
> > This patch will output unflushed logs of the safe buffers
> > at the bottom of log command output as follows:
> >
> > [nmi_print_seq] CPU: 0 BUFFER: ffff888063c18ac0 LEN: 28
> > nmi print seq test message
> > [safe_print_seq] CPU: 1 BUFFER: ffff888063d19ae0 LEN: 30
> > safe print seq test message
>
> Could you share how to test this patch?
> such as how to create a memory dump where some messages are left
> in each log buffers without being flushed.
> I guess it would be helpful for reviewers.
>
> >
> > Note that the safe buffer (struct printk_safe_seq_buf) was introduced
> > in kernel-4.11 and removed in kernel-5.15.
>
> Describing the exact commit hashs in the kernel git repo are helpful.
>
> >
> > Signed-off-by: Shogo Matsumoto <shogo.matsumoto(a)fujitsu.com>
> > ---
> > defs.h | 3 +++
> > kernel.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 61 insertions(+)
> >
> > diff --git a/defs.h b/defs.h
> > index 7e2a16e..3ee51e0 100644
> > --- a/defs.h
> > +++ b/defs.h
> > @@ -2146,6 +2146,8 @@ struct offset_table { /* stash
> of commonly-used offsets */
> > long wait_queue_entry_private;
> > long wait_queue_head_head;
> > long wait_queue_entry_entry;
> > + long printk_safe_seq_buf_len;
> > + long printk_safe_seq_buf_buffer;
> > };
> >
> > struct size_table { /* stash of commonly-used sizes */
> > @@ -2310,6 +2312,7 @@ struct size_table { /* stash of
> commonly-used sizes */
> > long prb_desc;
> > long wait_queue_entry;
> > long task_struct_state;
> > + long printk_safe_seq_buf_buffer;
> > };
>
> Could you add support for the new members to help -o?
>
> help -o dumps contents of offset_table, size_table and array_table:
>
> crash> help help
>
> NAME
> help -get help
> ...snip...
> -n - dumpfile contents/statistics
> -o - offset_table and size_table
> -p - program_context
> -r - dump registers from dumpfile header
> -s - symbol table data
> -t - task_table
> -T - task_table plus context_array
> -v - vm_table
> -V - vm_table (verbose)
> -x - text cache
> -z - help options
> crash> help -o | tail
> prio_array_queue: 0
> height_to_maxindex: 0
> height_to_maxnodes: 0
> pid_hash: 0
> kmem_cache_node: 1024
> kmem_cache_cpu_slab: 0
> rt_prio_array_queue: 0
> task_struct_rlim: 0
> signal_struct_rlim: 0
> vm_numa_stat: 0
>
>
> >
> > struct array_table {
> > diff --git a/kernel.c b/kernel.c
> > index f4598ea..cc97176 100644
> > --- a/kernel.c
> > +++ b/kernel.c
> > @@ -93,6 +93,7 @@ static void source_tree_init(void);
> > static ulong dump_audit_skb_queue(ulong);
> > static ulong __dump_audit(char *);
> > static void dump_audit(void);
> > +static void dump_printk_safe_seq_buf(void);
> > static char *vmcoreinfo_read_string(const char *);
> > static void check_vmcoreinfo(void);
> > static int is_pvops_xen(void);
> > @@ -5048,6 +5049,7 @@ cmd_log(void)
> > }
> >
> > dump_log(msg_flags);
> > + dump_printk_safe_seq_buf();
> > }
> >
> >
> > @@ -11534,6 +11536,62 @@ dump_audit(void)
> > error(INFO, "kernel audit log is empty\n");
> > }
> >
> > +static void
> > +__dump_printk_safe_seq_buf(char *buf_name)
> > +{
> > + int cpu, buffer_size;
> > + char *buffer;
> > +
> > + if (!symbol_exists(buf_name)) {
> > + return;
> > + }
> > +
> > + buffer_size = SIZE(printk_safe_seq_buf_buffer);
> > + buffer = GETBUF(buffer_size);
> > + for (cpu = 0; cpu < kt->cpus; cpu++) {
> > + ulong len_addr, buffer_addr;
> > + int len;
> > +
> > + len_addr = symbol_value(buf_name) +
> kt->__per_cpu_offset[cpu] + OFFSET(printk_safe_seq_buf_len);
> > + buffer_addr = symbol_value(buf_name) +
> kt->__per_cpu_offset[cpu] + OFFSET(printk_safe_seq_buf_buffer);
> > + readmem(len_addr, KVADDR, &len, STRUCT_SIZE("atomic_t"),
> "printk_safe_seq_buf len", FAULT_ON_ERROR);
> > + readmem(buffer_addr, KVADDR, buffer, buffer_size,
> "printk_safe_seq_buf buffer", FAULT_ON_ERROR);
> > +
> > + if (len > 0) {
> > + int i, n;
> > + char *p;
> > + fprintf(fp, "[%s] CPU: %d BUFFER: %lx LEN:
> %d\n", buf_name, cpu, buffer_addr, len);
> > + n = (len <= buffer_size) ? len : buffer_size;
> > + for (i = 0, p = buffer; i < n; i++, p++) {
> > + if (*p == 0x1) { //SOH
> > + i++; p++;
> > + continue;
> > + } else {
> > + fputc(ascii(*p) ? *p : '.', fp);
> > + }
> > + }
> > + fputc('\n', fp);
> > + }
> > + }
> > + FREEBUF(buffer);
> > +}
> > +
> > +static void
> > +dump_printk_safe_seq_buf(void)
> > +{
> > + if (!STRUCT_EXISTS("printk_safe_seq_buf"))
> > + return;
> > +
> > + if (INVALID_SIZE(printk_safe_seq_buf_buffer)) {
> > + MEMBER_OFFSET_INIT(printk_safe_seq_buf_len,
> "printk_safe_seq_buf", "len");
> > + MEMBER_OFFSET_INIT(printk_safe_seq_buf_buffer,
> "printk_safe_seq_buf", "buffer");
> > + MEMBER_SIZE_INIT(printk_safe_seq_buf_buffer,
> "printk_safe_seq_buf", "buffer");
> > + }
> > +
> > + __dump_printk_safe_seq_buf("nmi_print_seq");
> > + __dump_printk_safe_seq_buf("safe_print_seq");
> > +}
> > +
> > /*
> > * Reads a string value from the VMCOREINFO data stored in (live)
> memory.
> > *
>
>
>
>
>
> ------------------------------
>
> --
> Crash-utility mailing list
> Crash-utility(a)redhat.com
> https://listman.redhat.com/mailman/listinfo/crash-utility
>
> End of Crash-utility Digest, Vol 195, Issue 24
> **********************************************
>
>
1 year, 11 months
