January 2025 - Crash-utility - Crash Utility List Archives

Re: [PATCH] arm64: add pac mask to better support gdb stack unwind

by lijiang

Thank you for the patch, Guanyou. On Tue, Dec 31, 2024 at 7:22 PM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Thu, 26 Dec 2024 00:08:50 +0800 > From: Guanyou Chen <chenguanyou9338(a)gmail.com> > Subject: [Crash-utility] [PATCH] arm64: add pac mask to better support > gdb stack unwind > To: Lianbo <lijiang(a)redhat.com>, Tao Liu <ltao(a)redhat.com>, > devel(a)lists.crash-utility.osci.io > Message-ID: > < > CAHS3RMXJG6OxB_zmgnr60KriPOxo9tnb_63K+rjJHfk1t7aT0A(a)mail.gmail.com> > Content-Type: multipart/mixed; boundary="0000000000009f35d6062a1a727c" > > --0000000000009f35d6062a1a727c > Content-Type: multipart/alternative; > boundary="0000000000009f35d5062a1a727a" > > --0000000000009f35d5062a1a727a > Content-Type: text/plain; charset="UTF-8" > > Hi Lianbo & Tao > > Currently, gdb passthroughs of 'bt', 'frame', 'up', 'down', > 'info, locals' don't work on arm64 machine enabled pauth. > This is due to gdb not knowing the lr register real values > to unwind the stack frames. > > ---------------------------- > gdb passthrough (eg. "bt") | | > crash -------------------------> | | > | gdb_interface | > | | > | | > | ---------------------- | > get_kernel_pac_mask | | | | > crash_target<-------------------------+--| gdb | | > --------------------------+->| | | > arm64: CONFIG_ARM64_KERNELPACMASK| | | | > other: ~0UL | | | | > | ---------------------- | > ---------------------------- > > With the patch: > crash> gdb bt > #0 __switch_to (prev=prev@entry=0xffffff8001af92c0, > next=next@entry=0xffffff889da7a580) > at /proc/self/cwd/common/arch/arm64/kernel/process.c:569 > #1 0xffffffd3602132c0 in context_switch (rq=0xffffff8a7295a080, > prev=0xffffff8001af92c0, next=0xffffff889da7a580, rf=<optimized out>) at > /proc/self/cwd/common/kernel/sched/core.c:5515 > #2 __schedule (sched_mode=<optimized out>, sched_mode@entry > =2147859424) > at /proc/self/cwd/common/kernel/sched/core.c:6843 > #3 0xffffffd3602136d8 in schedule () at > /proc/self/cwd/common/kernel/sched/core.c:6917 > ... > > Without the patch: > crash> gdb bt > #0 __switch_to (prev=0xffffff8001af92c0, next=0xffffff889da7a580) at > /proc/self/cwd/common/arch/arm64/kernel/process.c:569 > #1 0x9fc5c5d3602132c0 in ?? () > Backtrace stopped: previous frame identical to this frame (corrupt > stack?) > > Signed-off-by: Guanyou.Chen <chenguanyou(a)xiaomi.com> > --- > gdb-10.2.patch | 24 ++++++++++++++++++++++++ > gdb_interface.c | 11 +++++++++++ > 2 files changed, 35 insertions(+) > > diff --git a/gdb-10.2.patch b/gdb-10.2.patch > index c867660..4c13a6b 100644 > --- a/gdb-10.2.patch > +++ b/gdb-10.2.patch > @@ -16216,3 +16216,27 @@ exit 0 > printf_filtered (_("Backtrace stopped: %s\n"), > frame_stop_reason_string (trailing)); > } > +--- gdb-10.2/gdb/frame.c.orig > ++++ gdb-10.2/gdb/frame.c > +@@ -944,6 +944,10 @@ frame_find_by_id (struct frame_id id) > + return NULL; > + } > + > ++#ifdef CRASH_MERGE > ++extern "C" unsigned long crash_get_kernel_pac_mask(void); > ++#endif > ++ > + static CORE_ADDR > + frame_unwind_pc (frame_info_ptr this_frame) > + { > +@@ -974,6 +978,10 @@ frame_unwind_pc (struct frame_info_ptr *this_frame) > + try > + { > + pc = gdbarch_unwind_pc (prev_gdbarch, this_frame); > ++#ifdef CRASH_MERGE > ++ CORE_ADDR mask = crash_get_kernel_pac_mask(); > ++ pc |= mask; > Can you help double check if a sanity check needs to be added? I saw the value 'pc|mask' is always checked in crash code as below: ... LR = regs->regs[30]; if (is_kernel_text (LR | ms->CONFIG_ARM64_KERNELPACMASK)) LR |= ms->CONFIG_ARM64_KERNELPACMASK; ... if yes, the value pc can be passed as an argument in crash_get_kernel_pac_mask(), and then deal with this one. > ++#endif > + pc_p = true; > + } > + catch (const gdb_exception_error &ex) > diff --git a/gdb_interface.c b/gdb_interface.c > index 315711e..765dafe 100644 > --- a/gdb_interface.c > +++ b/gdb_interface.c > @@ -1083,3 +1083,14 @@ int crash_get_current_task_reg (int regno, const > char *regname, > return machdep->get_current_task_reg(regno, regname, regsize, value); > } > > +/* arm64 kernel lr pac mask */ > +unsigned long crash_get_kernel_pac_mask(void); > +unsigned long crash_get_kernel_pac_mask(void) > +{ > +#ifdef ARM64 > + struct machine_specific *ms = machdep->machspec; > + return ms->CONFIG_ARM64_KERNELPACMASK; > +#else > + return ~0UL; > The "~0UL" is 0xFFFFFFFFFFFFFFFF, is this expected? Or do you want it to overflow? ++ pc |= mask; It probably has the same result, but the "return 0UL" should be more readable. Please see kernel code: vmcoreinfo_append_str("NUMBER(KERNELPACMASK)=0x%llx\n", system_supports_address_auth() ? ptrauth_kernel_pac_mask() : 0); BTW: I cannot apply your patch with git command, and ran into some issues, probably it is a coding issue. Thanks Lianbo > +#endif /* !ARM64 */ > +} > -- > 2.34.1 > > Thanks, > Guanyou >

2 months, 3 weeks

2
5
0 / 0

Re: [PATCH v3] "kmem -i" extremely slow on dumps from large memory systems

by lijiang

On Thu, Jan 9, 2025 at 6:41 AM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Thu, 9 Jan 2025 11:39:18 +1300 > From: Tao Liu <ltao(a)redhat.com> > Subject: [Crash-utility] Re: [PATCH v3] "kmem -i" extremely slow on > dumps from large memory systems > To: "Aureau, Georges (Kernel Tools ERT)" <georges.aureau(a)hpe.com> > Cc: "devel(a)lists.crash-utility.osci.io" > <devel(a)lists.crash-utility.osci.io> > Message-ID: > < > CAO7dBbULiC-3S+1a5O8NzT7PqaAos_ocFjQew6wWR8MSv0LbLw(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi Aureau, > > Thanks for the improvements. LGTM, so ack. > Applied(with minor changes): https://github.com/crash-utility/crash/commit/b39f8b558f9c71f91ae5f623a06... Thanks Lianbo > > Thanks, > Tao Liu > > On Thu, Jan 9, 2025 at 3:38 AM Aureau, Georges (Kernel Tools ERT) > <georges.aureau(a)hpe.com> wrote: > > > > The "kmem -i" command is extremely slow (appears to hang) on dumps from > large memory systems. > > > > For example, on 120GB crash dump from a high-end server with 32TB of RAM > (ie. 8Giga Pages), > > the "kmem -i" command is taking over 50 minutes to execute on a DL380 > Gen10. To report basic > > general memory usage figures, we should only be reading global counters, > without having to walk > > the full flat/sparse mem map page table. Hence, dump_kmeminfo() should > first be reading globals, > > and then only call dump_mem_map() if important information (ie. slabs or > total ram) is missing. > > > > Signed-off-by: Georges Aureau <georges.aureau(a)hpe.com> > > -- > > help.c | 4 +++- > > memory.c | 72 > +++++++++++++++++++++++++++++++++++++++++++++++----------------- > > 2 files changed, 56 insertions(+), 20 deletions(-) > > > > diff --git a/help.c b/help.c > > index 4c30047..8bd74ac 100644 > > --- a/help.c > > +++ b/help.c > > @@ -6825,7 +6825,9 @@ char *help_kmem[] = { > > " -F same as -f, but also dumps all pages linked to that > header.", > > " -c walks through the page_hash_table and verifies > page_cache_size.", > > " -C same as -c, but also dumps all pages in the > page_hash_table.", > > -" -i displays general memory usage information", > > +" -i displays general memory usage information.", > > +" Note: SHARED is no longer printed (too slow on large > memory systems)", > > +" unless specifying \"-i=shared\".", > > " -v displays the mapped virtual memory regions allocated by > vmalloc().", > > " -V displays the kernel vm_stat table if it exists, or in more > recent", > > " kernels, the vm_zone_stat, vm_node_stat and vm_numa_stat > tables,", > > diff --git a/memory.c b/memory.c > > index 8c01ed0..ea00ef1 100644 > > --- a/memory.c > > +++ b/memory.c > > @@ -237,7 +237,7 @@ static void dump_vmlist(struct meminfo *); > > static void dump_vmap_area(struct meminfo *); > > static int get_vmap_area_list_from_nodes(ulong **); > > static int dump_page_lists(struct meminfo *); > > -static void dump_kmeminfo(void); > > +static void dump_kmeminfo(struct meminfo *); > > static int page_to_phys(ulong, physaddr_t *); > > static void display_memory(ulonglong, long, ulong, int, void *); > > static char *show_opt_string(struct searchinfo *); > > @@ -5107,7 +5107,7 @@ cmd_kmem(void) > > BZERO(&value[0], sizeof(ulonglong)*MAXARGS); > > pc->curcmd_flags &= ~HEADER_PRINTED; > > > > - while ((c = getopt(argcnt, args, > "gI:sS::rFfm:pvczCinl:L:PVoh")) != EOF) { > > + while ((c = getopt(argcnt, args, > "gI:sS::rFfm:pvczCi::nl:L:PVoh")) != EOF) { > > switch(c) > > { > > case 'V': > > @@ -5124,6 +5124,9 @@ cmd_kmem(void) > > > > case 'i': > > iflag = 1; > > + if (optarg && strcmp(optarg, "=shared") == 0) { > > + meminfo.flags = GET_SHARED_PAGES; > > + } > > break; > > > > case 'h': > > @@ -5410,7 +5413,7 @@ cmd_kmem(void) > > } > > > > if (iflag == 1) > > - dump_kmeminfo(); > > + dump_kmeminfo(&meminfo); > > > > if (pflag == 1) > > dump_mem_map(&meminfo); > > @@ -8515,7 +8518,7 @@ bailout: > > char *kmeminfo_hdr = " PAGES TOTAL > PERCENTAGE\n"; > > > > static void > > -dump_kmeminfo(void) > > +dump_kmeminfo(struct meminfo *mi) > > { > > int i, len; > > ulong totalram_pages; > > @@ -8546,18 +8549,23 @@ dump_kmeminfo(void) > > ulong get_buffers; > > ulong get_slabs; > > char buf[BUFSIZE]; > > + ulong flags; > > > > - > > - BZERO(&meminfo, sizeof(struct meminfo)); > > - meminfo.flags = GET_ALL; > > - dump_mem_map(&meminfo); > > - get_totalram = meminfo.get_totalram; > > - shared_pages = meminfo.get_shared; > > - get_buffers = meminfo.get_buffers; > > - get_slabs = meminfo.get_slabs; > > + /* > > + * By default, we will no longer call dump_mem_map() as this is > too > > + * slow for large memory systems. If we have to call it (eg. > missing > > + * important information such as slabs or total ram), we will > also > > + * collect shared pages. Otherwise, we won't print shared pages > unless > > + * the caller explicitly requested shared pages ("kmem > -i=shared"). > > + */ > > + flags = mi->flags; > > + shared_pages = 0; > > + get_totalram = 0; > > + get_buffers = 0; > > + get_slabs = 0; > > > > /* > > - * If vm_stat array exists, override page search info. > > + * If vm_stat array does not exists, then set mem map flag. > > */ > > if (vm_stat_init()) { > > if (dump_vm_stat("NR_SLAB", &nr_slab, 0)) > > @@ -8572,9 +8580,10 @@ dump_kmeminfo(void) > > if (dump_vm_stat("NR_SLAB_UNRECLAIMABLE_B", > &nr_slab, 0)) > > get_slabs += nr_slab; > > } > > + } else { > > + flags |= GET_SLAB_PAGES; > > } > > > > - fprintf(fp, "%s", kmeminfo_hdr); > > /* > > * Get total RAM based upon how the various versions of > si_meminfo() > > * have done it, latest to earliest: > > @@ -8586,9 +8595,32 @@ dump_kmeminfo(void) > > symbol_exists("_totalram_pages")) { > > totalram_pages = vt->totalram_pages ? > > vt->totalram_pages : get_totalram; > > - } else > > - totalram_pages = get_totalram; > > + } else { > > + flags |= GET_TOTALRAM_PAGES; > > + totalram_pages = 0; > > + } > > > > + /* > > + * If the caller wants shared pages or if we are missing > important data > > + * (ie. slab or totalram) then go through the slow > dump_mem_map() path. > > + */ > > + if (flags) { > > + BZERO(&meminfo, sizeof(struct meminfo)); > > + meminfo.flags = GET_ALL; > > + dump_mem_map(&meminfo); > > + /* Update the missing information */ > > + if (flags & GET_SLAB_PAGES) { > > + get_slabs = meminfo.get_slabs; > > + } > > + if (flags & GET_TOTALRAM_PAGES) { > > + get_totalram = meminfo.get_totalram; > > + totalram_pages = get_totalram; > > + } > > + shared_pages = meminfo.get_shared; > > + get_buffers = meminfo.get_buffers; > > + } > > + > > + fprintf(fp, "%s", kmeminfo_hdr); > > fprintf(fp, "%13s %7ld %11s ----\n", "TOTAL MEM", > > totalram_pages, pages_to_size(totalram_pages, buf)); > > > > @@ -8613,9 +8645,11 @@ dump_kmeminfo(void) > > * differently than the kernel -- it just tallies the > non-reserved > > * pages that have a count of greater than 1. > > */ > > - pct = (shared_pages * 100)/totalram_pages; > > - fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", > > - "SHARED", shared_pages, pages_to_size(shared_pages, > buf), pct); > > + if (flags & GET_SHARED_PAGES) { > > + pct = (shared_pages * 100)/totalram_pages; > > + fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", > > + "SHARED", shared_pages, > pages_to_size(shared_pages, buf), pct); > > + } > > > > subtract_buffer_pages = 0; > > if (symbol_exists("buffermem_pages")) { >

2 months, 4 weeks

1
0
0 / 0

Re: [PATCH] kmem: fix the determination of slab page due to invalid page_type

by lijiang

On Tue, Jan 14, 2025 at 11:52 AM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Thu, 9 Jan 2025 12:01:29 +1300 > From: Tao Liu <ltao(a)redhat.com> > Subject: [Crash-utility] Re: [PATCH] kmem: fix the determination of > slab page due to invalid page_type > To: qiwu.chen(a)transsion.com > Cc: devel(a)lists.crash-utility.osci.io > Message-ID: > < > CAO7dBbW-ab1TxUZKWtDqPitR7vj_s4UMy9BzsyxyCr26zY1ORQ(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi qiwu, > > Thanks for the fix, LGTM, so ack. > Applied: https://github.com/crash-utility/crash/commit/a713368a3474a5f0a322d0c503e... Thanks Lianbo > > Thanks, > Tao Liu > > > On Wed, Jan 8, 2025 at 10:57 PM <qiwu.chen(a)transsion.com> wrote: > > > > There is fail when excute "kmem -i" on ARM64+kernel-5.10 vmcore: > > > > crash> kmem -i > > kmem: invalid kernel virtual address: ffffffff0be00030 type: "page_type" > > > > The slab page is determined by the PG_slab bit of page flag on this > vmcore, > > when traverse the pages of sparse memory, page_slab() fails to readmem > > page_type for a invalid page. Fix this by enhance the determination of > slab page. > > > > Signed-off-by: qiwu.chen <qiwu.chen(a)transsion.com> > > --- > > memory.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/memory.c b/memory.c > > index 8c01ed0..e4a058a 100644 > > --- a/memory.c > > +++ b/memory.c > > @@ -374,6 +374,8 @@ static bool page_slab(ulong page, ulong flags) > > if (vt->flags & SLAB_PAGEFLAGS) { > > if ((flags >> vt->PG_slab) & 1) > > return TRUE; > > + else > > + return FALSE; > > } > > > > if (VALID_MEMBER(page_page_type)) { > > -- > > 2.25.1 >

2 months, 4 weeks

1
0
0 / 0

Re: [PATCH] Fix "net -a" option on Linux 6.13 and later kernels

by lijiang

On Wed, Jan 8, 2025 at 7:32 AM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Wed, 8 Jan 2025 12:30:11 +1300 > From: Tao Liu <ltao(a)redhat.com> > Subject: [Crash-utility] Re: [PATCH] Fix "net -a" option on Linux 6.13 > and later kernels > To: HAGIO KAZUHITO(萩尾一仁) <k-hagio-ab(a)nec.com> > Cc: "devel(a)lists.crash-utility.osci.io" > <devel(a)lists.crash-utility.osci.io> > Message-ID: > <CAO7dBbVYFpdtgMOvtnxp4MJJB_p4GyG= > 2_je7yMt33H40JmZZA(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi Kazu, > > Thanks for the fix, the patch LGTM, so ack. > > Applied: https://github.com/crash-utility/crash/commit/8eb5279fdd2e447e593fc04c86a... Thanks Lianbo > Thanks, > Tao Liu > > On Tue, Jan 7, 2025 at 3:46 PM HAGIO KAZUHITO(萩尾一仁) <k-hagio-ab(a)nec.com> > wrote: > > > > From: Kazuhito Hagio <k-hagio-ab(a)nec.com> > > > > Kernel commit e4e3fd0a99d5 ("Merge branch > 'improve-neigh_flush_dev-performance'") > > and its patch set, which are contained in Linux 6.13 and later kernels, > > removed neigh_hash_table.hash_buckets and neighbour.next members, and > > introduced neigh_hash_table.hash_heads and neighbour.hash members > > instead with the hlist implementation. > > > > Without the patch, the "net -a" option fails with the following error: > > > > crash> net -a > > > > net: invalid structure member offset: neigh_table_hash_buckets > > FILE: net.c LINE: 701 FUNCTION: dump_arp() > > ... > > > > Signed-off-by: Kazuhito Hagio <k-hagio-ab(a)nec.com> > > --- > > defs.h | 2 ++ > > net.c | 26 +++++++++++++++++++++----- > > symbols.c | 2 ++ > > 3 files changed, 25 insertions(+), 5 deletions(-) > > > > diff --git a/defs.h b/defs.h > > index d8d378ef94b1..8378cbab2f73 100644 > > --- a/defs.h > > +++ b/defs.h > > @@ -2269,6 +2269,8 @@ struct offset_table { /* stash > of commonly-used offsets */ > > long task_struct_thread_context_x26; > > long task_struct_thread_context_x27; > > long task_struct_thread_context_x28; > > + long neigh_table_hash_heads; > > + long neighbour_hash; > > }; > > > > struct size_table { /* stash of commonly-used sizes */ > > diff --git a/net.c b/net.c > > index 31d3f2bf7c2f..4e14f19fc097 100644 > > --- a/net.c > > +++ b/net.c > > @@ -169,6 +169,11 @@ net_init(void) > > "neigh_hash_table", "hash_shift"); > > MEMBER_OFFSET_INIT(neigh_table_hash_buckets, > > "neigh_hash_table", "hash_buckets"); > > + /* Linux 6.13 and later */ > > + if (INVALID_MEMBER(neigh_table_hash_buckets)) { > > + > MEMBER_OFFSET_INIT(neigh_table_hash_heads, "neigh_hash_table", > "hash_heads"); > > + MEMBER_OFFSET_INIT(neighbour_hash, > "neighbour", "hash"); > > + } > > } else { > > MEMBER_OFFSET_INIT(neigh_table_hash_buckets, > > "neigh_table", "hash_buckets"); > > @@ -698,9 +703,13 @@ dump_arp(void) > > "neigh_table key_len", FAULT_ON_ERROR); > > > > if (VALID_MEMBER(neigh_table_nht_ptr)) { > > - readmem(nht + OFFSET(neigh_table_hash_buckets), > > - KVADDR, &hash, sizeof(hash), > > - "neigh_hash_table hash_buckets ptr", > FAULT_ON_ERROR); > > + /* Linux 6.13 and later */ > > + if (VALID_MEMBER(neigh_table_hash_heads)) > > + readmem(nht + OFFSET(neigh_table_hash_heads), > KVADDR, &hash, > > + sizeof(hash), "neigh_hash_table > hash_heads ptr", FAULT_ON_ERROR); > > + else > > + readmem(nht + OFFSET(neigh_table_hash_buckets), > KVADDR, &hash, > > + sizeof(hash), "neigh_hash_table > hash_buckets ptr", FAULT_ON_ERROR); > > > > readmem(hash, KVADDR, hash_buckets, hash_bytes, > > "neigh_hash_table hash_buckets", FAULT_ON_ERROR); > > @@ -831,8 +840,15 @@ print_neighbour_q(ulong addr, int key_len) > > > > arp_state_to_flags(state); > > > > - readmem(addr + OFFSET(neighbour_next), KVADDR, > > - &addr, sizeof(addr), "neighbour next", > FAULT_ON_ERROR); > > + /* Linux 6.13 and later kernels use hlist. */ > > + if (VALID_MEMBER(neighbour_hash)) { > > + readmem(addr + OFFSET(neighbour_hash), KVADDR, > &addr, > > + sizeof(addr), "neighbour hash", > FAULT_ON_ERROR); > > + if (addr) > > + addr -= OFFSET(neighbour_hash); > > + } else > > + readmem(addr + OFFSET(neighbour_next), KVADDR, > &addr, > > + sizeof(addr), "neighbour next", > FAULT_ON_ERROR); > > } > > > > FREEBUF(ha_buf); > > diff --git a/symbols.c b/symbols.c > > index a3cd0f3f6e1f..0b63e11fa567 100644 > > --- a/symbols.c > > +++ b/symbols.c > > @@ -11028,6 +11028,7 @@ dump_offset_table(char *spec, ulong makestruct) > > > > fprintf(fp, " neighbour_next: %ld\n", > > OFFSET(neighbour_next)); > > + fprintf(fp, " neighbour_hash: %ld\n", > OFFSET(neighbour_hash)); > > fprintf(fp, " neighbour_primary_key: %ld\n", > > OFFSET(neighbour_primary_key)); > > fprintf(fp, " neighbour_ha: %ld\n", > > @@ -11038,6 +11039,7 @@ dump_offset_table(char *spec, ulong makestruct) > > OFFSET(neighbour_nud_state)); > > fprintf(fp, " neigh_table_hash_buckets: %ld\n", > > OFFSET(neigh_table_hash_buckets)); > > + fprintf(fp, " neigh_table_hash_heads: %ld\n", > OFFSET(neigh_table_hash_heads)); > > fprintf(fp, " neigh_table_hash_mask: %ld\n", > > OFFSET(neigh_table_hash_mask)); > > fprintf(fp, " neigh_table_hash_shift: %ld\n", > > -- > > 2.31.1 >

2 months, 4 weeks

1
0
0 / 0

Re: [PATCH] Add compilation requirements note in README

by lijiang

On Wed, Jan 8, 2025 at 7:32 AM <devel-request(a)lists.crash-utility.osci.io> wrote: > Date: Mon, 6 Jan 2025 10:57:25 +1300 > From: Tao Liu <ltao(a)redhat.com> > Subject: [Crash-utility] Re: [PATCH] Add compilation requirements note > in README. > To: wonderzyp(a)gmail.com > Cc: devel(a)lists.crash-utility.osci.io, yeping.zheng(a)nio.com > Message-ID: > < > CAO7dBbVVZf2XBHLGsvEPp5CDc-d0bS0QcG-DkiGWROcd064XpA(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi yeping, > > On Mon, Jan 6, 2025 at 5:18 AM <wonderzyp(a)gmail.com> wrote: > > > > From: "yeping.zheng" <wonderzyp(a)gmail.com> > > > > Hi Lianbo, > > > > > This is for Fedora, but I'm not sure if they have the same package > name on other distributions. Can you help to investigate it? Maybe we can > list dependencies based on known distributions. > > > > I have tried to build crash in Ubuntu, ArchLinux and openSUSE docker > > images, and successful after installing the software mentioned in the > patch. > > > > Is there any platforms need to be added? > > > We cannot enumerate all platforms, at least this is a reference. > Thanks, > > yeping. > > > > Signed-off-by: yeping.zheng <yeping.zheng(a)nio.com> > > --- > > README | 6 ++++++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/README b/README > > index 02fc56c..3a4d936 100644 > > --- a/README > > +++ b/README > > @@ -72,6 +72,12 @@ > > After the kernel is re-compiled, the uncompressed "vmlinux" kernel > > that is created in the top-level kernel build directory must be > saved. > > > > + o Requirements for building: > > + Fedora: make gcc gcc-c++ ncurses-devel zlib-devel lzo-devel > snappy-devel bison wget patch texinfo libzstd-devel > > + Ubuntu/Debian: make gcc g++ libncurses-dev zlib1g-dev liblzo2-dev > libsnappy-dev bison wget patch texinfo libzstd-dev > > + Arch: make gcc ncurses zlib lzo snappy bison wget patch texinfo > zstd > > + openSUSE: make gcc gcc-c++ ncurses-devel zlib-devel lzo-devel > snappy-devel bison wget patch texinfo libzstd-devel > > + > > The requirements are for making a full "$make lzo snappy zstd" > support. E.g. If we don't need lzo, then lzo-devel is not a must for > fedora. Though I think this is not a big issue for this doc. > > Anyway, ack for this patch. > Applied: https://github.com/crash-utility/crash/commit/7357d822124c917014346115a4e... Thanks Lianbo > > Thanks, > Tao Liu > > > > To build the crash utility: > > > > $ tar -xf crash-8.0.6.tar.gz > > -- > > 2.47.1 >

3 months

1
0
0 / 0

[PATCH] Fix misleading CPU count in display_sys_stats()

by soakley＠redhat.com

From: Lucas Oakley <soakley(a)redhat.com> This simplication fixes the total CPU count being reported incorrectly in ppc64le and s390x systems when some number of CPUs have been offlined, as the kt->cpus value is adjusted. This adds the word "OFFLINE" to the 'sys' output for s390x and ppc64le, like exists for x86_64 and aarch64 when examining systems with offlined CPUs. Without patch: KERNEL: /debug/4.18.0-477.10.1.el8_8.s390x/vmlinux DUMPFILE: /proc/kcore CPUS: 1 With patch: KERNEL: /debug/4.18.0-477.10.1.el8_8.s390x/vmlinux DUMPFILE: /proc/kcore CPUS: 2 [OFFLINE: 1] Signed-off-by: Lucas Oakley <soakley(a)redhat.com> --- kernel.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/kernel.c b/kernel.c index 8c2e0ca..3e190f1 100644 --- a/kernel.c +++ b/kernel.c @@ -5816,15 +5816,13 @@ display_sys_stats(void) pc->kvmdump_mapfile); } - if (machine_type("PPC64")) - fprintf(fp, " CPUS: %d\n", get_cpus_to_display()); - else { - fprintf(fp, " CPUS: %d", kt->cpus); - if (kt->cpus - get_cpus_to_display()) - fprintf(fp, " [OFFLINE: %d]", - kt->cpus - get_cpus_to_display()); - fprintf(fp, "\n"); - } + int number_cpus_to_display = get_cpus_to_display(); + int number_cpus_present = get_cpus_present(); + fprintf(fp, " CPUS: %d", number_cpus_present); + if (number_cpus_present != number_cpus_to_display) + fprintf(fp, " [OFFLINE: %d]", + number_cpus_present - number_cpus_to_display); + fprintf(fp, "\n"); if (ACTIVE()) get_xtime(&kt->date); -- 2.47.1

3 months

2
1
0 / 0

[PATCH] kmem: fix the determination of slab page due to invalid page_type

by qiwu.chen＠transsion.com

There is fail when excute "kmem -i" on ARM64+kernel-5.10 vmcore: crash> kmem -i kmem: invalid kernel virtual address: ffffffff0be00030 type: "page_type" The slab page is determined by the PG_slab bit of page flag on this vmcore, when traverse the pages of sparse memory, page_slab() fails to readmem page_type for a invalid page. Fix this by enhance the determination of slab page. Signed-off-by: qiwu.chen <qiwu.chen(a)transsion.com> --- memory.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/memory.c b/memory.c index 8c01ed0..e4a058a 100644 --- a/memory.c +++ b/memory.c @@ -374,6 +374,8 @@ static bool page_slab(ulong page, ulong flags) if (vt->flags & SLAB_PAGEFLAGS) { if ((flags >> vt->PG_slab) & 1) return TRUE; + else + return FALSE; } if (VALID_MEMBER(page_page_type)) { -- 2.25.1

3 months

2
1
0 / 0

[PATCH v3] "kmem -i" extremely slow on dumps from large memory systems

by Aureau, Georges (Kernel Tools ERT)

The "kmem -i" command is extremely slow (appears to hang) on dumps from large memory systems. For example, on 120GB crash dump from a high-end server with 32TB of RAM (ie. 8Giga Pages), the "kmem -i" command is taking over 50 minutes to execute on a DL380 Gen10. To report basic general memory usage figures, we should only be reading global counters, without having to walk the full flat/sparse mem map page table. Hence, dump_kmeminfo() should first be reading globals, and then only call dump_mem_map() if important information (ie. slabs or total ram) is missing. Signed-off-by: Georges Aureau <georges.aureau(a)hpe.com> -- help.c | 4 +++- memory.c | 72 +++++++++++++++++++++++++++++++++++++++++++++++----------------- 2 files changed, 56 insertions(+), 20 deletions(-) diff --git a/help.c b/help.c index 4c30047..8bd74ac 100644 --- a/help.c +++ b/help.c @@ -6825,7 +6825,9 @@ char *help_kmem[] = { " -F same as -f, but also dumps all pages linked to that header.", " -c walks through the page_hash_table and verifies page_cache_size.", " -C same as -c, but also dumps all pages in the page_hash_table.", -" -i displays general memory usage information", +" -i displays general memory usage information.", +" Note: SHARED is no longer printed (too slow on large memory systems)", +" unless specifying \"-i=shared\".", " -v displays the mapped virtual memory regions allocated by vmalloc().", " -V displays the kernel vm_stat table if it exists, or in more recent", " kernels, the vm_zone_stat, vm_node_stat and vm_numa_stat tables,", diff --git a/memory.c b/memory.c index 8c01ed0..ea00ef1 100644 --- a/memory.c +++ b/memory.c @@ -237,7 +237,7 @@ static void dump_vmlist(struct meminfo *); static void dump_vmap_area(struct meminfo *); static int get_vmap_area_list_from_nodes(ulong **); static int dump_page_lists(struct meminfo *); -static void dump_kmeminfo(void); +static void dump_kmeminfo(struct meminfo *); static int page_to_phys(ulong, physaddr_t *); static void display_memory(ulonglong, long, ulong, int, void *); static char *show_opt_string(struct searchinfo *); @@ -5107,7 +5107,7 @@ cmd_kmem(void) BZERO(&value[0], sizeof(ulonglong)*MAXARGS); pc->curcmd_flags &= ~HEADER_PRINTED; - while ((c = getopt(argcnt, args, "gI:sS::rFfm:pvczCinl:L:PVoh")) != EOF) { + while ((c = getopt(argcnt, args, "gI:sS::rFfm:pvczCi::nl:L:PVoh")) != EOF) { switch(c) { case 'V': @@ -5124,6 +5124,9 @@ cmd_kmem(void) case 'i': iflag = 1; + if (optarg && strcmp(optarg, "=shared") == 0) { + meminfo.flags = GET_SHARED_PAGES; + } break; case 'h': @@ -5410,7 +5413,7 @@ cmd_kmem(void) } if (iflag == 1) - dump_kmeminfo(); + dump_kmeminfo(&meminfo); if (pflag == 1) dump_mem_map(&meminfo); @@ -8515,7 +8518,7 @@ bailout: char *kmeminfo_hdr = " PAGES TOTAL PERCENTAGE\n"; static void -dump_kmeminfo(void) +dump_kmeminfo(struct meminfo *mi) { int i, len; ulong totalram_pages; @@ -8546,18 +8549,23 @@ dump_kmeminfo(void) ulong get_buffers; ulong get_slabs; char buf[BUFSIZE]; + ulong flags; - - BZERO(&meminfo, sizeof(struct meminfo)); - meminfo.flags = GET_ALL; - dump_mem_map(&meminfo); - get_totalram = meminfo.get_totalram; - shared_pages = meminfo.get_shared; - get_buffers = meminfo.get_buffers; - get_slabs = meminfo.get_slabs; + /* + * By default, we will no longer call dump_mem_map() as this is too + * slow for large memory systems. If we have to call it (eg. missing + * important information such as slabs or total ram), we will also + * collect shared pages. Otherwise, we won't print shared pages unless + * the caller explicitly requested shared pages ("kmem -i=shared"). + */ + flags = mi->flags; + shared_pages = 0; + get_totalram = 0; + get_buffers = 0; + get_slabs = 0; /* - * If vm_stat array exists, override page search info. + * If vm_stat array does not exists, then set mem map flag. */ if (vm_stat_init()) { if (dump_vm_stat("NR_SLAB", &nr_slab, 0)) @@ -8572,9 +8580,10 @@ dump_kmeminfo(void) if (dump_vm_stat("NR_SLAB_UNRECLAIMABLE_B", &nr_slab, 0)) get_slabs += nr_slab; } + } else { + flags |= GET_SLAB_PAGES; } - fprintf(fp, "%s", kmeminfo_hdr); /* * Get total RAM based upon how the various versions of si_meminfo() * have done it, latest to earliest: @@ -8586,9 +8595,32 @@ dump_kmeminfo(void) symbol_exists("_totalram_pages")) { totalram_pages = vt->totalram_pages ? vt->totalram_pages : get_totalram; - } else - totalram_pages = get_totalram; + } else { + flags |= GET_TOTALRAM_PAGES; + totalram_pages = 0; + } + /* + * If the caller wants shared pages or if we are missing important data + * (ie. slab or totalram) then go through the slow dump_mem_map() path. + */ + if (flags) { + BZERO(&meminfo, sizeof(struct meminfo)); + meminfo.flags = GET_ALL; + dump_mem_map(&meminfo); + /* Update the missing information */ + if (flags & GET_SLAB_PAGES) { + get_slabs = meminfo.get_slabs; + } + if (flags & GET_TOTALRAM_PAGES) { + get_totalram = meminfo.get_totalram; + totalram_pages = get_totalram; + } + shared_pages = meminfo.get_shared; + get_buffers = meminfo.get_buffers; + } + + fprintf(fp, "%s", kmeminfo_hdr); fprintf(fp, "%13s %7ld %11s ----\n", "TOTAL MEM", totalram_pages, pages_to_size(totalram_pages, buf)); @@ -8613,9 +8645,11 @@ dump_kmeminfo(void) * differently than the kernel -- it just tallies the non-reserved * pages that have a count of greater than 1. */ - pct = (shared_pages * 100)/totalram_pages; - fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", - "SHARED", shared_pages, pages_to_size(shared_pages, buf), pct); + if (flags & GET_SHARED_PAGES) { + pct = (shared_pages * 100)/totalram_pages; + fprintf(fp, "%13s %7ld %11s %3ld%% of TOTAL MEM\n", + "SHARED", shared_pages, pages_to_size(shared_pages, buf), pct); + } subtract_buffer_pages = 0; if (symbol_exists("buffermem_pages")) {

3 months

2
1
0 / 0

[PATCH] Fix "net -a" option on Linux 6.13 and later kernels

by HAGIO KAZUHITO(萩尾　一仁)

From: Kazuhito Hagio <k-hagio-ab(a)nec.com> Kernel commit e4e3fd0a99d5 ("Merge branch 'improve-neigh_flush_dev-performance'") and its patch set, which are contained in Linux 6.13 and later kernels, removed neigh_hash_table.hash_buckets and neighbour.next members, and introduced neigh_hash_table.hash_heads and neighbour.hash members instead with the hlist implementation. Without the patch, the "net -a" option fails with the following error: crash> net -a net: invalid structure member offset: neigh_table_hash_buckets FILE: net.c LINE: 701 FUNCTION: dump_arp() ... Signed-off-by: Kazuhito Hagio <k-hagio-ab(a)nec.com> --- defs.h | 2 ++ net.c | 26 +++++++++++++++++++++----- symbols.c | 2 ++ 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/defs.h b/defs.h index d8d378ef94b1..8378cbab2f73 100644 --- a/defs.h +++ b/defs.h @@ -2269,6 +2269,8 @@ struct offset_table { /* stash of commonly-used offsets */ long task_struct_thread_context_x26; long task_struct_thread_context_x27; long task_struct_thread_context_x28; + long neigh_table_hash_heads; + long neighbour_hash; }; struct size_table { /* stash of commonly-used sizes */ diff --git a/net.c b/net.c index 31d3f2bf7c2f..4e14f19fc097 100644 --- a/net.c +++ b/net.c @@ -169,6 +169,11 @@ net_init(void) "neigh_hash_table", "hash_shift"); MEMBER_OFFSET_INIT(neigh_table_hash_buckets, "neigh_hash_table", "hash_buckets"); + /* Linux 6.13 and later */ + if (INVALID_MEMBER(neigh_table_hash_buckets)) { + MEMBER_OFFSET_INIT(neigh_table_hash_heads, "neigh_hash_table", "hash_heads"); + MEMBER_OFFSET_INIT(neighbour_hash, "neighbour", "hash"); + } } else { MEMBER_OFFSET_INIT(neigh_table_hash_buckets, "neigh_table", "hash_buckets"); @@ -698,9 +703,13 @@ dump_arp(void) "neigh_table key_len", FAULT_ON_ERROR); if (VALID_MEMBER(neigh_table_nht_ptr)) { - readmem(nht + OFFSET(neigh_table_hash_buckets), - KVADDR, &hash, sizeof(hash), - "neigh_hash_table hash_buckets ptr", FAULT_ON_ERROR); + /* Linux 6.13 and later */ + if (VALID_MEMBER(neigh_table_hash_heads)) + readmem(nht + OFFSET(neigh_table_hash_heads), KVADDR, &hash, + sizeof(hash), "neigh_hash_table hash_heads ptr", FAULT_ON_ERROR); + else + readmem(nht + OFFSET(neigh_table_hash_buckets), KVADDR, &hash, + sizeof(hash), "neigh_hash_table hash_buckets ptr", FAULT_ON_ERROR); readmem(hash, KVADDR, hash_buckets, hash_bytes, "neigh_hash_table hash_buckets", FAULT_ON_ERROR); @@ -831,8 +840,15 @@ print_neighbour_q(ulong addr, int key_len) arp_state_to_flags(state); - readmem(addr + OFFSET(neighbour_next), KVADDR, - &addr, sizeof(addr), "neighbour next", FAULT_ON_ERROR); + /* Linux 6.13 and later kernels use hlist. */ + if (VALID_MEMBER(neighbour_hash)) { + readmem(addr + OFFSET(neighbour_hash), KVADDR, &addr, + sizeof(addr), "neighbour hash", FAULT_ON_ERROR); + if (addr) + addr -= OFFSET(neighbour_hash); + } else + readmem(addr + OFFSET(neighbour_next), KVADDR, &addr, + sizeof(addr), "neighbour next", FAULT_ON_ERROR); } FREEBUF(ha_buf); diff --git a/symbols.c b/symbols.c index a3cd0f3f6e1f..0b63e11fa567 100644 --- a/symbols.c +++ b/symbols.c @@ -11028,6 +11028,7 @@ dump_offset_table(char *spec, ulong makestruct) fprintf(fp, " neighbour_next: %ld\n", OFFSET(neighbour_next)); + fprintf(fp, " neighbour_hash: %ld\n", OFFSET(neighbour_hash)); fprintf(fp, " neighbour_primary_key: %ld\n", OFFSET(neighbour_primary_key)); fprintf(fp, " neighbour_ha: %ld\n", @@ -11038,6 +11039,7 @@ dump_offset_table(char *spec, ulong makestruct) OFFSET(neighbour_nud_state)); fprintf(fp, " neigh_table_hash_buckets: %ld\n", OFFSET(neigh_table_hash_buckets)); + fprintf(fp, " neigh_table_hash_heads: %ld\n", OFFSET(neigh_table_hash_heads)); fprintf(fp, " neigh_table_hash_mask: %ld\n", OFFSET(neigh_table_hash_mask)); fprintf(fp, " neigh_table_hash_shift: %ld\n", -- 2.31.1

3 months

2
1
0 / 0

[PATCH] Add compilation requirements note in README.

by wonderzyp＠gmail.com

From: "yeping.zheng" <wonderzyp(a)gmail.com> Hi Lianbo, > This is for Fedora, but I'm not sure if they have the same package name on other distributions. Can you help to investigate it? Maybe we can list dependencies based on known distributions. I have tried to build crash in Ubuntu, ArchLinux and openSUSE docker images, and successful after installing the software mentioned in the patch. Is there any platforms need to be added? Thanks, yeping. Signed-off-by: yeping.zheng <yeping.zheng(a)nio.com> --- README | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README b/README index 02fc56c..3a4d936 100644 --- a/README +++ b/README @@ -72,6 +72,12 @@ After the kernel is re-compiled, the uncompressed "vmlinux" kernel that is created in the top-level kernel build directory must be saved. + o Requirements for building: + Fedora: make gcc gcc-c++ ncurses-devel zlib-devel lzo-devel snappy-devel bison wget patch texinfo libzstd-devel + Ubuntu/Debian: make gcc g++ libncurses-dev zlib1g-dev liblzo2-dev libsnappy-dev bison wget patch texinfo libzstd-dev + Arch: make gcc ncurses zlib lzo snappy bison wget patch texinfo zstd + openSUSE: make gcc gcc-c++ ncurses-devel zlib-devel lzo-devel snappy-devel bison wget patch texinfo libzstd-devel + To build the crash utility: $ tar -xf crash-8.0.6.tar.gz -- 2.47.1

3 months, 1 week

2
1
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Crash-utility January 2025