Hi Tao, Thanks for your reply,  1) Try to drop the "-S" of crash cmdline to see if it works. I have tried without -S,  it was broken by another issue, the crash reported an error : _note_9: st_name: 1692 st_value: ffffffffc04d747c st_shndx: 22 st_info: ? __this_module: st_name: 1700 st_value: ffffffffc04d8380 st_shndx: 29 st_info: ? cleanup_module: st_name: 1714 st_value: ffffffffc04d6c48 st_shndx: 7 st_info: T   FREEBUF(1) FREEBUF(0) <readmem: ffffffffab2df068, KVADDR, "nr_threads", 4, (FOE), 55f06caa495c> <read_proc_kcore: addr: ffffffffab2df068 paddr: 11cedf068 cnt: 4> crash: per_cpu_symbol_search(per_cpu__runqueues): NULL crash: cannot determine idle task addresses from init_tasks[] or runqueues[] crash: cannot resolve "init_task_union" 2) If not, then please try to disable the kernel kaslr at bootup by appending "nokaslr" to kernel cmdline, and invoke crash as your original way. After append the "nokaslr" to kernel command line, the crash without -S still report "crash: cannot resolve "init_task_union" But with "-S" parameter, crash can enter the CLI as normal. So, regarding the behavior, when without -S, the crash report "crash: cannot resolve "init_task_union"", this should  not related to kaslr, right?  If so, is that means missing kernel configuration/patch or gdb needs a special version? The current kernel is 6.1 stable, gdb version is 10.2. Thanks, Br, Xiangyu 在 2024年12月12日 星期四 上午07:46:05 [GMT+8]， Tao Liu&lt;ltao(a)redhat.com&gt; 寫道： Hi xiangyu, On Fri, Dec 6, 2024 at 9:25 PM <xiangyu.chen(a)aol.com&gt; wrote: 1) Try to drop the "-S" of crash cmdline to see if it works. 2) If not, then please try to disable the kernel kaslr at bootup by appending "nokaslr" to kernel cmdline, and invoke crash as your original way. Hope this helps. Thanks, Tao Liu

Hi Tao, Thanks for your info. Appending detail of my local setup : System is using linux-yocto, BSP is x86-64 on qemu. (OE-core: https://git.openembedded.org/openembedded-core/log/?h=mickledore) (Kernel: linux-yocto 6.1 https://git.yoctoproject.org/linux-yocto/log/?h=v6.1/standard/x86) Br, Xiangyu 在 2024年12月12日 星期四 下午12:07:49 [GMT+8]， Tao Liu&lt;ltao(a)redhat.com&gt; 寫道： Hi Xiangyu, On Thu, Dec 12, 2024 at 4:15 PM Xiangyu Chen <xiangyu.chen(a)aol.com&gt; wrote: Not sure either, I hardly use the -S parameter. I will create a test env and do some debugging on my own. Will share findings later... Thanks, Tao Liu

Hi Tao, Thanks for your great help, after integrating the patch, the crash tool is working in my environment. Currently, with -S and --kaslr=auto, the crash can enter the CLI normally, and symbols display correct. Thanks! Regaring without -S reporting "crash: cannot resolve "init_task_union", I am still invesgating, seems related the symtab in my own kernel image, once have some new finding, I will update the result in this thread. Thanks! Br, Xiangyu 在 2024年12月13日 星期五 上午11:20:55 [GMT+8]， Tao Liu&lt;ltao(a)redhat.com&gt; 寫道： Hi Xiangyu, On Thu, Dec 12, 2024 at 4:15 PM Xiangyu Chen <xiangyu.chen(a)aol.com&gt; wrote: Let me try to answer your questions. Without '-S', crash will try to find the symbols within vmlinux, but somehow it failed to find the symbols as specified in tasks.c:get_idle_threads(). So it fails down to reach "init_task_union", which is a very old symbol and no longer exists in kernels nowadays. I cannot tell why the symbols are missing in your kernel, maybe due to your gcc or kernel config or else. Currently the "-S" option is problematic and it doesn't deal with kaslr issue. So if you use "-S" with kaslr enabled, it will fail with no doubt. So when you disabled kaslr and use "-S" at the same time, this can work as expected. Please check the attachments, in which I drafted a patch to fix the kaslr issue for "-S". This should work when you use "-S" with kaslr enabled by your kernel. Please see if it can work for you. Thanks, Tao Liu > > So, regarding the behavior, when without -S, the crash report "crash: cannot resolve "init_task_union"", this should > not related to kaslr, right?  If so, is that means missing kernel configuration/patch or gdb needs a special version? > The current kernel is 6.1 stable, gdb version is 10.2. > > > > Thanks, > > Br, > Xiangyu > > > 在 2024年12月12日 星期四 上午07:46:05 [GMT+8]， Tao Liu&lt;ltao(a)redhat.com&gt; 寫道： > > Hi xiangyu, > > On Fri, Dec 6, 2024 at 9:25 PM <xiangyu.chen(a)aol.com&gt; wrote: > > > > Hi Crash-utility folks, > > > > Recently, I am using crash to debug a live system. > > When I execute the crash, it cannot enter the crash cmd shell, it was broken by crash seek error. > > My kernel turned on the CONFIG_RANDOMIZE_BASE, and I added the kaslr=auto parameter to crash. > > If I disable the CONFIG_RANDOMIZE_BASE, the issue has gone. > > Could you please help to check if any config/parameter I was missing? Thanks. > > > > The following is my system information: > > crash version: 8.0.6 > > kernel version: 6.1.118 > > Host system: qemux86-64 > > > > Here is the full log with -d: > > root@intel-x86-64:/boot# crash -S /boot/vmlinux-6.1.118-yocto-standard /proc/kcore -d10 --kaslr=auto > > > 1) Try to drop the "-S" of crash cmdline to see if it works. > 2) If not, then please try to disable the kernel kaslr at bootup by > appending "nokaslr" to kernel cmdline, and invoke crash as your > original way. > > Hope this helps. > > Thanks, > Tao Liu > > > crash 8.0.6 > > Copyright (C) 2002-2025  Red Hat, Inc. > > Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation > > Copyright (C) 1999-2006  Hewlett-Packard Co > > Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited > > Copyright (C) 2006, 2007  VA Linux Systems Japan K.K. > > Copyright (C) 2005, 2011, 2020-2024  NEC Corporation > > Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc. > > Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc. > > Copyright (C) 2015, 2021  VMware, Inc. > > This program is free software, covered by the GNU General Public License, > > and you are welcome to change it and/or distribute copies of it under > > certain conditions.  Enter "help copying" to see the conditions. > > This program has absolutely no warranty.  Enter "help warranty" for details. > > > > get_live_memory_source: /proc/kcore > > proc_kcore_data: > >            flags: 500 (KCORE_LOCAL|KCORE_ELF64) > >        segments: 10 > >      elf_header: 56471f1eb9c0 > >      header_size: 12324 > >          notes64: 56471f1eba00 > >          load64: 56471f1eba38 > >          notes32: 0 > >          load32: 0 > >      vmcoreinfo: 0 > >  size_vmcoreinfo: 0 > > > >  Elf64_Phdr: > >        p_type: 4 (PT_NOTE) > >        p_flags: 0 > >      p_offset: 2a8 > >        p_vaddr: 0 > >        p_paddr: 0 > >      p_filesz: 11644 > >        p_memsz: 0 > >        p_align: 0 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 7fffaa404000 > >        p_vaddr: ffffffffaa400000 > >        p_paddr: b8600000 > >      p_filesz: 33751040 > >        p_memsz: 33751040 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 286600004000 > >        p_vaddr: ffffa86600000000 > >        p_paddr: ffffffffffffffff > >      p_filesz: 35184372088831 > >        p_memsz: 35184372088831 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 7fffc0004000 > >        p_vaddr: ffffffffc0000000 > >        p_paddr: ffffffffffffffff > >      p_filesz: 1056964608 > >        p_memsz: 1056964608 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 126ac0005000 > >        p_vaddr: ffff926ac0001000 > >        p_paddr: 1000 > >      p_filesz: 647168 > >        p_memsz: 647168 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 4d8880004000 > >        p_vaddr: ffffcd8880000000 > >        p_paddr: ffffffffffffffff > >      p_filesz: 12288 > >        p_memsz: 12288 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 126ac0104000 > >        p_vaddr: ffff926ac0100000 > >        p_paddr: 100000 > >      p_filesz: 12902400 > >        p_memsz: 12902400 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 4d8880008000 > >        p_vaddr: ffffcd8880004000 > >        p_paddr: ffffffffffffffff > >      p_filesz: 204800 > >        p_memsz: 204800 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 126ac0d53000 > >        p_vaddr: ffff926ac0d4f000 > >        p_paddr: d4f000 > >      p_filesz: 3207118848 > >        p_memsz: 3207118848 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 126bc0004000 > >        p_vaddr: ffff926bc0000000 > >        p_paddr: 100000000 > >      p_filesz: 4294967296 > >        p_memsz: 4294967296 > >        p_align: 4096 > > > >  Elf64_Phdr: > >        p_type: 1 (PT_LOAD) > >        p_flags: 7 > >      p_offset: 4d8884004000 > >        p_vaddr: ffffcd8884000000 > >        p_paddr: ffffffffffffffff > >      p_filesz: 67108864 > >        p_memsz: 67108864 > >        p_align: 4096 > > > >  Elf64_Nhdr: > >      n_namesz: 5 ("CORE") > >      n_descsz: 336 > >        n_type: 1 (NT_PRSTATUS) > > > >  Elf64_Nhdr: > >      n_namesz: 5 ("CORE") > >      n_descsz: 136 > >        n_type: 3 (NT_PRPSINFO) > > > >  Elf64_Nhdr: > >      n_namesz: 5 ("CORE") > >      n_descsz: 7808 > >        n_type: 4 (NT_TASKSTRUCT) > > > >  Elf64_Nhdr: > >      n_namesz: 11 ("VMCOREINFO") > >      n_descsz: 3279 > >        n_type: 0 (unknown) > > > >      OSRELEASE=6.1.118-yocto-standard > >      BUILD-ID=1400000003000000474e5500a343709706409b9c > >      PAGESIZE=4096 > >      SYMBOL(init_uts_ns)=ffffffffac037a80 > >      OFFSET(uts_namespace.name)=0 > >      SYMBOL(node_online_map)=ffffffffac06d3e8 > >      SYMBOL(swapper_pg_dir)=ffffffffabe0a000 > >      SYMBOL(_stext)=ffffffffaa400000 > >      SYMBOL(vmap_area_list)=ffffffffabf12fb0 > >      SYMBOL(mem_section)=ffff926cbfff5000 > >      LENGTH(mem_section)=2048 > >      SIZE(mem_section)=16 > >      OFFSET(mem_section.section_mem_map)=0 > >      NUMBER(SECTION_SIZE_BITS)=27 > >      NUMBER(MAX_PHYSMEM_BITS)=46 > >      SIZE(page)=64 > >      SIZE(pglist_data)=15616 > >      SIZE(zone)=1600 > >      SIZE(free_area)=104 > >      SIZE(list_head)=16 > >      SIZE(nodemask_t)=8 > >      OFFSET(page.flags)=0 > >      OFFSET(page._refcount)=52 > >      OFFSET(page.mapping)=24 > >      OFFSET(page.lru)=8 > >      OFFSET(page._mapcount)=48 > >      OFFSET(page.private)=40 > >      OFFSET(page.compound_dtor)=16 > >      OFFSET(page.compound_order)=17 > >      OFFSET(page.compound_head)=8 > >      OFFSET(pglist_data.node_zones)=0 > >      OFFSET(pglist_data.nr_zones)=14624 > >      OFFSET(pglist_data.node_start_pfn)=14632 > >      OFFSET(pglist_data.node_spanned_pages)=14648 > >      OFFSET(pglist_data.node_id)=14656 > >      OFFSET(zone.free_area)=192 > >      OFFSET(zone.vm_stat)=1472 > >      OFFSET(zone.spanned_pages)=136 > >      OFFSET(free_area.free_list)=0 > >      OFFSET(list_head.next)=0 > >      OFFSET(list_head.prev)=8 > >      OFFSET(vmap_area.va_start)=0 > >      OFFSET(vmap_area.list)=40 > >      LENGTH(zone.free_area)=11 > >      SYMBOL(prb)=ffffffffabe647d8 > >      SYMBOL(printk_rb_static)=ffffffffabe647e0 > >      SYMBOL(clear_seq)=ffffffffac307f20 > >      SIZE(printk_ringbuffer)=88 > >      OFFSET(printk_ringbuffer.desc_ring)=0 > >      OFFSET(printk_ringbuffer.text_data_ring)=48 > >      OFFSET(printk_ringbuffer.fail)=80 > >      SIZE(prb_desc_ring)=48 > >      OFFSET(prb_desc_ring.count_bits)=0 > >      OFFSET(prb_desc_ring.descs)=8 > >      OFFSET(prb_desc_ring.infos)=16 > >      OFFSET(prb_desc_ring.head_id)=24 > >      OFFSET(prb_desc_ring.tail_id)=32 > >      SIZE(prb_desc)=24 > >      OFFSET(prb_desc.state_var)=0 > >      OFFSET(prb_desc.text_blk_lpos)=8 > >      SIZE(prb_data_blk_lpos)=16 > >      OFFSET(prb_data_blk_lpos.begin)=0 > >      OFFSET(prb_data_blk_lpos.next)=8 > >      SIZE(printk_info)=88 > >      OFFSET(printk_info.seq)=0 > >      OFFSET(printk_info.ts_nsec)=8 > >      OFFSET(printk_info.text_len)=16 > >      OFFSET(printk_info.caller_id)=20 > >      OFFSET(printk_info.dev_info)=24 > >      SIZE(dev_printk_info)=64 > >      OFFSET(dev_printk_info.subsystem)=0 > >      LENGTH(printk_info_subsystem)=16 > >      OFFSET(dev_printk_info.device)=16 > >      LENGTH(printk_info_device)=48 > >      SIZE(prb_data_ring)=32 > >      OFFSET(prb_data_ring.size_bits)=0 > >      OFFSET(prb_data_ring.data)=8 > >      OFFSET(prb_data_ring.head_lpos)=16 > >      OFFSET(prb_data_ring.tail_lpos)=24 > >      SIZE(atomic_long_t)=8 > >      OFFSET(atomic_long_t.counter)=0 > >      SIZE(latched_seq)=24 > >      OFFSET(latched_seq.val)=8 > >      LENGTH(free_area.free_list)=6 > >      NUMBER(NR_FREE_PAGES)=0 > >      NUMBER(PG_lru)=4 > >      NUMBER(PG_private)=13 > >      NUMBER(PG_swapcache)=10 > >      NUMBER(PG_swapbacked)=19 > >      NUMBER(PG_slab)=9 > >      NUMBER(PG_hwpoison)=23 > >      NUMBER(PG_head_mask)=65536 > >      NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129 > >      NUMBER(HUGETLB_PAGE_DTOR)=2 > >      NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257 > >      SYMBOL(kallsyms_names)=ffffffffaba0f7d8 > >      SYMBOL(kallsyms_num_syms)=ffffffffaba0f7d0 > >      SYMBOL(kallsyms_token_table)=ffffffffabbf5400 > >      SYMBOL(kallsyms_token_index)=ffffffffabbf5788 > >      SYMBOL(kallsyms_offsets)=ffffffffab9984f8 > >      SYMBOL(kallsyms_relative_base)=ffffffffaba0f7c8 > >      NUMBER(phys_base)=2384461824 > >      SYMBOL(init_top_pgt)=ffffffffabe0a000 > >      NUMBER(pgtable_l5_enabled)=0 > >      SYMBOL(node_data)=ffffffffac06c3a0 > >      LENGTH(node_data)=64 > >      KERNELOFFSET=29400000 > >      NUMBER(KERNEL_IMAGE_SIZE)=1073741824 > >      NUMBER(sme_mask)=0 > > > > readmem: read_proc_kcore() -> /proc/kcore > > VMCOREINFO: NUMBER(phys_base): 2384461824 -> 8e200000 > > gdb /boot/vmlinux-6.1.118-yocto-standard > > GNU gdb (GDB) 10.2 > > Copyright (C) 2021 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. > > Type "show copying" and "show warranty" for details. > > This GDB was configured as "x86_64-wrs-linux". > > Type "show configuration" for configuration details. > > Find the GDB manual and other documentation resources online at: > >    <http://www.gnu.org/software/gdb/documentation/>;. > > > > For help, type "help". > > Type "apropos word" to search for commands related to "word"... > > GETBUF(344 -> 0) > >  GETBUF(1500 -> 1) > > > >  FREEBUF(1) > > FREEBUF(0) > > <readmem: ffffffff8292c360, KVADDR, "page_offset_base", 8, (FOE|Q), 56471ef799c8> > > <read_proc_kcore: addr: ffffffff8292c360 paddr: 90b2c360 cnt: 8> > > crash: seek error: kernel virtual address: ffffffff8292c360  type: "page_offset_base" > > -- > > Crash-utility mailing list -- devel(a)lists.crash-utility.osci.io > > To unsubscribe send an email to devel-leave(a)lists.crash-utility.osci.io > > https://${domain_name}/admin/lists/devel.lists.crash-utility.osci.io/ > > Contribution Guidelines: https://github.com/crash-utility/crash/wiki >