Re: [Crash-utility] [PATCH 1/2] Revert "Fix segfault in arm64_is_kernel_exception_frame() when corrupt stack pointer address is given"
noon
On Wed, Jun 7, 2023 at 8:00 PM <crash-utility-request(a)redhat.com> wrote:
Date: Wed, 7 Jun 2023 18:37:33 +0900
From: HATAYAMA Daisuke <d.hatayama(a)fujitsu.com>
To: crash-utility(a)redhat.com
Cc: d.hatayama(a)fujitsu.com
Subject: [Crash-utility] [PATCH 1/2] Revert "Fix segfault in
arm64_is_kernel_exception_frame() when corrupt stack pointer
address
is given"
Message-ID: <20230607093734.247-1-d.hatayama(a)fujitsu.com>
Content-Type: text/plain; charset="US-ASCII"; x-default=true
This reverts commit 9868ebc8e648e5791764a51567a23efae7170d9b.
The commit 9868ebc8e648e5791764a51567a23efae7170d9b causes the issue
that bt command fails to show backtraces for the tasks that is running
in the user mode at the moment of the kernel panic as follows:
crash> bt 1734
PID: 1734 TASK: ffff000000392200 CPU: 4 COMMAND: "insmod"
bt: invalid stack pointer is given
Thank you for pointing out this issue, HATAYAMA.
Anyway, I did not reproduce the above issue. Seems it can not always be
reproduced.
# ./crash /home/vmlinux /var/crash/127.0.0.1-2023-06-09-05\:20\:38/vmcore -s
WARNING: cpu 2: invalid NT_PRSTATUS note (n_type != NT_PRSTATUS)
WARNING: cpu 1: cannot find NT_PRSTATUS note
WARNING: cpu 2: cannot find NT_PRSTATUS note
crash> ps insmod
PID PPID CPU TASK ST %MEM VSZ RSS COMM
1684 1683 0 ffff06738f1cdd00 ZO 0.0 0 0 insmod
crash> bt 1684
PID: 1684 TASK: ffff06738f1cdd00 CPU: 0 COMMAND: "insmod"
(no stack)
crash>
Thanks.
Lianbo
The root cause is that while the commit added a sanity check into
STACK_OFFSET_TYPE() to validate if a given candidate address of any
interrupt or exception frame is contained within the range of the
corresponding kernel stack, the premise that the STACK_OFFSET_TYPE()
should not return out-of-the-buffer address, is wrong.
Reexamining the relevant surrounding part of the backtracing code, it
looks to me now that the STACK_OFFSET_TYPE() is originally expected to
return an out-of-the-buffer address, like the address of the top of
the corresponding kernel stack, e.g. at here:
static int
arm64_in_kdump_text(struct bt_info *bt, struct arm64_stackframe *frame)
{
...
if (bt->flags & BT_USER_SPACE)
start = (ulong
*)&bt->stackbuf[(ulong)(STACK_OFFSET_TYPE(bt->stacktop))];
else {
Note that the above bt 1734 aborts here.
Hence, the current implementation policy around STACK_OFFSET_TYPE()
looks that the caller side is responsible for understanding the fact
in advance and for avoiding making buffer overrun carefully.
To fix this issue, revert the commit.
Signed-off-by: HATAYAMA Daisuke <d.hatayama(a)fujitsu.com>
---
defs.h | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/defs.h b/defs.h
index bfda0c4..3e7d6cf 100644
--- a/defs.h
+++ b/defs.h
@@ -976,10 +976,7 @@ struct bt_info {
#define STACK_OFFSET_TYPE(OFF) \
(((ulong)(OFF) > STACKSIZE()) ? \
- (((ulong)(OFF) < (ulong)(bt->stackbase) || (ulong)(OFF) >=
(ulong)(bt->stackbase) + STACKSIZE()) ? \
- error(FATAL, "invalid stack pointer is given\n") : \
- (ulong)((ulong)(OFF) - (ulong)(bt->stackbase))) : \
- (ulong)(OFF))
+ (ulong)((ulong)(OFF) - (ulong)(bt->stackbase)) : (ulong)(OFF))
#define GET_STACK_ULONG(OFF) \
*((ulong *)((char *)(&bt->stackbuf[(ulong)(STACK_OFFSET_TYPE(OFF))])))
--
2.25.1
1:24 p.m.
New subject: [PATCH 1/2] Revert "Fix segfault in arm64_is_kernel_exception_frame() when corrupt stack pointer address is given"
Thank you for pointing out this issue, HATAYAMA.
Anyway, I did not reproduce the above issue. Seems it can not always be reproduced.
# ./crash /home/vmlinux /var/crash/127.0.0.1-2023-06-09-05\:20\:38/vmcore -s
WARNING: cpu 2: invalid NT_PRSTATUS note (n_type != NT_PRSTATUS)
WARNING: cpu 1: cannot find NT_PRSTATUS note
WARNING: cpu 2: cannot find NT_PRSTATUS note
crash> ps insmod
PID PPID CPU TASK ST %MEM VSZ RSS COMM
1684 1683 0 ffff06738f1cdd00 ZO 0.0 0 0 insmod
crash> bt 1684
PID: 1684 TASK: ffff06738f1cdd00 CPU: 0 COMMAND: "insmod"
(no stack)
crash>
The problematic case is the active tasks running in user mode at the
moment of kernel panic. In most cases, it's enough to prepare some
programs that running in infinite loop just like:
# while : ; do continue ; done &
[3] 3295
Just in case, note that this issue is different from the one of
corrupt mapping of NT_PRSTATUS notes. You don't need to use the
reproduction steps I shared. It's enough to prepare the above busy
loop process in advance, make the kernel panic and then use bt command
for the busy loop process.
Thanks.
HATAYAMA, Daisuke
11:02 a.m.
New subject: [PATCH 1/2] Revert "Fix segfault in arm64_is_kernel_exception_frame() when corrupt stack pointer address is given"
On Mon, Jun 12, 2023 at 7:24 PM Daisuke Hatayama (Fujitsu) <
d.hatayama(a)fujitsu.com> wrote:
> Thank you for pointing out this issue, HATAYAMA.
>
> Anyway, I did not reproduce the above issue. Seems it can not always be
reproduced.
>
> # ./crash /home/vmlinux
/var/crash/127.0.0.1-2023-06-09-05\:20\:38/vmcore -s
> WARNING: cpu 2: invalid NT_PRSTATUS note (n_type != NT_PRSTATUS)
> WARNING: cpu 1: cannot find NT_PRSTATUS note
> WARNING: cpu 2: cannot find NT_PRSTATUS note
> crash> ps insmod
> PID PPID CPU TASK ST %MEM VSZ RSS COMM
> 1684 1683 0 ffff06738f1cdd00 ZO 0.0 0 0
insmod
> crash> bt 1684
> PID: 1684 TASK: ffff06738f1cdd00 CPU: 0 COMMAND: "insmod"
> (no stack)
> crash>
The problematic case is the active tasks running in user mode at the
moment of kernel panic. In most cases, it's enough to prepare some
programs that running in infinite loop just like:
# while : ; do continue ; done &
[3] 3295
Just in case, note that this issue is different from the one of
corrupt mapping of NT_PRSTATUS notes. You don't need to use the
Thank you for the explanation, HATAYAMA. It's true, they are different
issues, that is why it can not always be reproduced.
crash> ps insmod
PID PPID CPU TASK ST %MEM VSZ RSS COMM
1696 1695 2 ffff2e420cf5a900 RU 0.0 7168 3840 insmod
crash> bt 1696
PID: 1696 TASK: ffff2e420cf5a900 CPU: 2 COMMAND: "insmod"
#0 [ffff800013eefae0] __switch_to at ffffc029d3cc9d24
#1 [ffff800013eefb10] __schedule at ffffc029d475c1fc
#2 [ffff800013eefba0] preempt_schedule_common at ffffc029d475cd7c
#3 [ffff800013eefbb0] _cond_resched at ffffc029d475cdc8
#4 [ffff800013eefbc0] down_read at ffffc029d475fdbc
#5 [ffff800013eefbe0] blocking_notifier_call_chain at ffffc029d3d66024
#6 [ffff800013eefc10] do_init_module at ffffc029d3e1040c
#7 [ffff800013eefc40] load_module at ffffc029d3e12948
#8 [ffff800013eefda0] __se_sys_finit_module at ffffc029d3e12ebc
#9 [ffff800013eefe60] __arm64_sys_finit_module at ffffc029d3e12f7c
#10 [ffff800013eefe80] do_el0_svc at ffffc029d3cd9300
#11 [ffff800013eefeb0] el0_sync_handler at ffffc029d3cc9374
#12 [ffff800013eefff0] el0_sync at ffffc029d3cc2b7c
PC: 0000ffff9b7637e4 LR: 0000aaaabe6b3e48 SP: 0000ffffc6f33810
X29: 0000ffffc6f33810 X28: 0000000000000000 X27: 0000000000000000
X26: 0000000000000002 X25: 0000000000000000 X24: 0000ffffc6f338e8
X23: 0000aaaad7da1840 X22: 0000000000000000 X21: 0000000000000000
X20: 0000aaaabe6bd520 X19: 0000aaaad7da1860 X18: 0000000000000000
X17: 0000ffff9b7637c0 X16: 0000aaaabe6dfd98 X15: 0000000000000070
X14: 0000000000000002 X13: 000000000000270f X12: 0000000000000000
X11: 0000000000000000 X10: 0000000000000000 X9: 0000aaaad7da1960
X8: 0000000000000111 X7: 0000000000000001 X6: 0000000000000001
X5: 0000000000000db1 X4: 0000000000000000 X3: 0000000000000003
X2: 0000000000000000 X1: 0000aaaabe6bd520 X0: 0000000000000003
ORIG_X0: 0000000000000003 SYSCALLNO: 111 PSTATE: 40001000
reproduction steps I shared. It's enough to prepare the above busy
loop process in advance, make the kernel panic and then use bt
command
for the busy loop process.
You are right. I have reproduced the current problem with an infinite loop
process.
crash> ps t
PID PPID CPU TASK ST %MEM VSZ RSS COMM
8419 1896 2 ffff08aa9360ff00 RU 0.0 2432 1216 t
crash> bt 8419
PID: 8419 TASK: ffff08aa9360ff00 CPU: 2 COMMAND: "t"
bt: invalid stack pointer is given
I have no other issues about it.
Thanks.
Lianbo
Thanks.
HATAYAMA, Daisuke
464
days inactive
468
days old
devel@lists.crash-utility.osci.io
2 comments
2 participants
participants (2)
-
Daisuke Hatayama (Fujitsu) -
lijiang