[Crash-utility] Re: [PATCH v5 0/5] Improve stack unwind on ppc64

Sure. Thanks a lot for your reviews, I sent a V6 with a minor change in patch #5. Thanks, Aditya Gupta > > Thanks > > Lianbo > > > > > > > 'unavailable' frame being shown. Though it's funny that bug helped in a way to > > > > uncover these warnings, this would have hit in earlier versions also. > > > Indeed, it can be triggered conditionally. > > > > > > I have no other issues, thank you for the great job, Aditya. > > Thanks, > > Aditya Gupta > > > > > > > > Thanks > > > > > > Lianbo > > > > > > > Thanks, > > > > Aditya Gupta > > > > > > > > > crash> gdb frame > > > > > #0  <unavailable> in ?? () > > > > > crash> set gdb on > > > > > gdb: on > > > > > gdb> bt > > > > > #0  <unavailable> in ?? () > > > > > #1  0xc0000000000f570c in plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:111 > > > > > #2  0xc000000001004dd8 in cede_processor () at > > > > > ./arch/powerpc/include/asm/plpar_wrappers.h:37 > > > > > #3  check_and_cede_processor () at drivers/cpuidle/cpuidle-pseries.c:83 > > > > > #4  0xc000000001005000 in shared_cede_loop (dev=<optimized out>, > > > > > drv=<optimized out>, index=<optimized out>) at > > > > > drivers/cpuidle/cpuidle-pseries.c:256 > > > > > #5  0xc000000001004498 in cpuidle_enter_state > > > > > (dev=dev@entry=0xc0000001ff5910c0, drv=drv@entry=0xc000000002b8f558 > > > > > <pseries_idle_driver>, index=index@entry=1) at drivers/cpuidle/cpuidle.c:267 > > > > > #6  0xc000000000c0eb4c in cpuidle_enter (drv=0xc000000002b8f558 > > > > > <pseries_idle_driver>, dev=0xc0000001ff5910c0, index=<optimized out>) at > > > > > drivers/cpuidle/cpuidle.c:388 > > > > > #7  0xc0000000001ce2bc in call_cpuidle (drv=<optimized out>, > > > > > drv@entry=0xc000000002b8f558 <pseries_idle_driver>, dev=<optimized out>, > > > > > dev@entry=0xc0000001ff5910c0, next_state=<optimized out>) at > > > > > kernel/sched/idle.c:134 > > > > > #8  0xc0000000001d5d68 in cpuidle_idle_call () at kernel/sched/idle.c:215 > > > > > #9  0xc0000000001d5f58 in do_idle () at kernel/sched/idle.c:282 > > > > > #10 0xc0000000001d6298 in cpu_startup_entry (state=<optimized out>) at > > > > > kernel/sched/idle.c:380 > > > > > #11 0xc000000000011030 in rest_init () at init/main.c:730 > > > > > #12 0xc000000002004c80 in arch_call_rest_init () at init/main.c:827 > > > > > #13 0xc0000000020051ec in start_kernel () at init/main.c:1072 > > > > > #14 0xc00000000000e788 in start_here_common () at > > > > > arch/powerpc/kernel/head_64.S:1039 > > > > > gdb> info threads > > > > >   Id   Target Id         Frame > > > > > * 1    CPU 0             plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > >   2    CPU 1             plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > >   3    CPU 2             plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > >   4    CPU 3             plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > >   5    CPU 4             plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > >   6    CPU 5             0xc00000000028b5e8 in crash_setup_regs > > > > > (oldregs=<optimized out>, newregs=0xc00000005d3e7958) at > > > > > ./arch/powerpc/include/asm/kexec.h:69 > > > > >   7    CPU 6             plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > >   8    CPU 7             plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > > gdb: page excluded: kernel virtual address: c000000002004c80 type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c7c type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c78 type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c80 type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c7c type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c78 type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051ec type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e8 type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e4 type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051ec type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e8 type: > > > > > "gdb_readmem_callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e4 type: > > > > > "gdb_readmem_callback" > > > > > > > > > > gdb> thread 1 > > > > > [Switching to thread 1 (CPU 0)] > > > > > #0  plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > > 114             li      r4,0 > > > > > gdb> bt > > > > > #0  plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > > #1  0xc000000001004dd8 in cede_processor () at > > > > > ./arch/powerpc/include/asm/plpar_wrappers.h:37 > > > > > #2  check_and_cede_processor () at drivers/cpuidle/cpuidle-pseries.c:83 > > > > > #3  0xc000000001005000 in shared_cede_loop (dev=<optimized out>, > > > > > drv=<optimized out>, index=<optimized out>) at > > > > > drivers/cpuidle/cpuidle-pseries.c:256 > > > > > #4  0xc000000001004498 in cpuidle_enter_state > > > > > (dev=dev@entry=0xc0000001ff5910c0, drv=drv@entry=0xc000000002b8f558 > > > > > <pseries_idle_driver>, index=index@entry=1) at drivers/cpuidle/cpuidle.c:267 > > > > > #5  0xc000000000c0eb4c in cpuidle_enter (drv=0xc000000002b8f558 > > > > > <pseries_idle_driver>, dev=0xc0000001ff5910c0, index=<optimized out>) at > > > > > drivers/cpuidle/cpuidle.c:388 > > > > > #6  0xc0000000001ce2bc in call_cpuidle (drv=<optimized out>, > > > > > drv@entry=0xc000000002b8f558 <pseries_idle_driver>, dev=<optimized out>, > > > > > dev@entry=0xc0000001ff5910c0, next_state=<optimized out>) at > > > > > kernel/sched/idle.c:134 > > > > > #7  0xc0000000001d5d68 in cpuidle_idle_call () at kernel/sched/idle.c:215 > > > > > #8  0xc0000000001d5f58 in do_idle () at kernel/sched/idle.c:282 > > > > > #9  0xc0000000001d6298 in cpu_startup_entry (state=<optimized out>) at > > > > > kernel/sched/idle.c:380 > > > > > #10 0xc000000000011030 in rest_init () at init/main.c:730 > > > > > gdb: page excluded: kernel virtual address: c000000002004c80 type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c7c type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c78 type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c80 type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c7c type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c000000002004c78 type: > > > > > "gdb_readmem callback" > > > > > #11 0xc000000002004c80 in arch_call_rest_init () at init/main.c:827 > > > > > gdb: page excluded: kernel virtual address: c0000000020051ec type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e8 type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e4 type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051ec type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e8 type: > > > > > "gdb_readmem callback" > > > > > gdb: page excluded: kernel virtual address: c0000000020051e4 type: > > > > > "gdb_readmem callback" > > > > > #12 0xc0000000020051ec in start_kernel () at init/main.c:1072 > > > > > #13 0xc00000000000e788 in start_here_common () at > > > > > arch/powerpc/kernel/head_64.S:1039 > > > > > gdb> thread 3 > > > > > [Switching to thread 3 (CPU 2)] > > > > > #0  plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > > 114             li      r4,0 > > > > > gdb> bt > > > > > #0  plpar_hcall_norets_notrace () at > > > > > arch/powerpc/platforms/pseries/hvCall.S:114 > > > > > #1  0xc000000001004dd8 in cede_processor () at > > > > > ./arch/powerpc/include/asm/plpar_wrappers.h:37 > > > > > #2  check_and_cede_processor () at drivers/cpuidle/cpuidle-pseries.c:83 > > > > > #3  0xc000000001005000 in shared_cede_loop (dev=<optimized out>, > > > > > drv=<optimized out>, index=<optimized out>) at > > > > > drivers/cpuidle/cpuidle-pseries.c:256 > > > > > #4  0xc000000001004498 in cpuidle_enter_state > > > > > (dev=dev@entry=0xc0000001ff7910c0, drv=drv@entry=0xc000000002b8f558 > > > > > <pseries_idle_driver>, index=index@entry=1) at drivers/cpuidle/cpuidle.c:267 > > > > > #5  0xc000000000c0eb4c in cpuidle_enter (drv=0xc000000002b8f558 > > > > > <pseries_idle_driver>, dev=0xc0000001ff7910c0, index=<optimized out>) at > > > > > drivers/cpuidle/cpuidle.c:388 > > > > > #6  0xc0000000001ce2bc in call_cpuidle (drv=<optimized out>, > > > > > drv@entry=0xc000000002b8f558 <pseries_idle_driver>, dev=<optimized out>, > > > > > dev@entry=0xc0000001ff7910c0, next_state=<optimized out>) at > > > > > kernel/sched/idle.c:134 > > > > > #7  0xc0000000001d5d68 in cpuidle_idle_call () at kernel/sched/idle.c:215 > > > > > #8  0xc0000000001d5f58 in do_idle () at kernel/sched/idle.c:282 > > > > > #9  0xc0000000001d6298 in cpu_startup_entry (state=<optimized out>) at > > > > > kernel/sched/idle.c:380 > > > > > #10 0xc00000000005f048 in start_secondary (unused=<optimized out>) at > > > > > arch/powerpc/kernel/smp.c:1680 > > > > > #11 0xc00000000000e058 in start_secondary_prolog () at > > > > > arch/powerpc/kernel/head_64.S:885 > > > > > gdb> > > > > > > > > > > > > > > > Could you please check it again? Or am I missing anything? I did the test > > > > > based on upstream kernel 6.7.0-rc7(commit:8735c7c84d1b). > > > > > > > > > > BTW: I did not see the similar issues after applying the v4 patch set. > > > > > > > > > > > > > > > Thanks > > > > > > > > > > Lianbo > > > > > > > > > > > Known Issues: > > > > > > ============= > > > > > > > > > > > > 1. In gdb mode, 'bt' might fail to show backtrace in few vmcores collected > > > > > > from older kernels. This is a known issue due to register mismatch, and > > > > > > its fix has been merged upstream: > > > > > > > > > > > > This can also cause some 'invalid kernel virtual address' errors during gdb > > > > > > unwinding the stack registers > > > > > > > > > > > > Commit: https://github.com/torvalds/linux/commit/b684c09f09e7a6af3794d4233ef78581... > > > > > > > > > > > > Fixing GDB passthroughs on other architectures > > > > > > ============================================== > > > > > > > > > > > > Much of the work for making gdb passthroughs like 'gdb bt', 'gdb > > > > > > thread', 'gdb info locals' etc. has been done by the patches introducing > > > > > > 'machdep->get_cpu_reg' and this series fixing some issues in that. > > > > > > > > > > > > Other architectures should be able to fix these gdb functionalities by > > > > > > simply implementing 'machdep->get_cpu_reg (cpu, regno, ...)'. > > > > > > > > > > > > The reasoning behind that has been explained with a diagram in commit > > > > > > description of patch #1 > > > > > > > > > > > > I will assist with my findings/observations fixing it on ppc64 whenever needed. > > > > > > > > > > > > Changelog: > > > > > > ========== > > > > > > > > > > > > V5: > > > > > > + changes in patch #1: made ppc64_get_cpu_reg static, and remove unreachable > > > > > > code > > > > > > + changes in patch #3: fixed typo 'ppc64_renum' instead of 'ppc64_regnum', > > > > > > remove unneeded if condition > > > > > > + changes in patch #5: implement refresh regcache on per thread, instead of all > > > > > > threads at once > > > > > > > > > > > > V4: > > > > > > + fix segmentation fault in live debugging (change in patch #1) > > > > > > + mention live debugging not supported in cover letter and patch #1 > > > > > > + fixed some checkpatch warnings (change in patch #5) > > > > > > > > > > > > V3: > > > > > > + default gdb thread will be the crashing thread, instead of being > > > > > > thread '0' > > > > > > + synchronise crash cpu and gdb thread context > > > > > > + fix bug in gdb_interface, that replaced gdb's output stream, losing > > > > > > output in some cases, such as info threads and extra output in info > > > > > > variables > > > > > > + fix 'info threads' > > > > > > > > > > > > RFC V2: > > > > > > - removed patch implementing 'frame', 'up', 'down' in crash > > > > > > - updated the cover letter by removing the mention of those commands other > > > > > > than the respective gdb passthrough > > > > > > > > > > > > Aditya Gupta (5): > > > > > > ppc64: correct gdb passthroughs by implementing machdep->get_cpu_reg > > > > > > remove 'frame' from prohibited commands list > > > > > > synchronise cpu context changes between crash/gdb > > > > > > fix gdb_interface: restore gdb's output streams at end of > > > > > > gdb_interface > > > > > > fix 'info threads' command > > > > > > > > > > > > crash_target.c | 44 ++++++++++++++++ > > > > > > defs.h | 130 +++++++++++++++++++++++++++++++++++++++++++++++- > > > > > > gdb-10.2.patch | 110 +++++++++++++++++++++++++++++++++++++++- > > > > > > gdb_interface.c | 2 +- > > > > > > kernel.c | 47 +++++++++++++++-- > > > > > > ppc64.c | 95 +++++++++++++++++++++++++++++++++-- > > > > > > task.c | 14 ++++++ > > > > > > tools.c | 2 +- > > > > > > 8 files changed, 434 insertions(+), 10 deletions(-) > > > > > > >