4:01 a.m.
Subject: [BUG/RFC] arm64: stack init functions fail in POST_GDB when percpu data is in
vmalloc area.
Hi,
While running crash on a large arm64 NUMA system, crash fails to
initialize the arm64 stack subsystems during arm64_init(POST_GDB) and
aborts startup with SEEK_ERROR on every percpu IRQ stack pointer.
Environment
-----------
- architecture : aarch64 (64K pages, CONFIG_ARM64_64K_PAGES=y)
- kernel : 6.x, large NUMA, many CPUs
- crash : current master (arm64.c)
Background on percpu allocation on this machine
-----------------------------------------------
On this arm64 machine the kernel's first percpu chunk does NOT live in
the linear map, it lives in the vmalloc area. The arm64 percpu setup
path in drivers/base/arch_numa.c:setup_per_cpu_areas() first tries
pcpu_embed_first_chunk() and, when that fails, falls back to
pcpu_page_first_chunk()
Observed symptom / reproduction log
-----------------------------------
With crash started on the vmcore, the percpu IRQ stack pointers are all in the
ffff800100xxxxxx range, which is the vmalloc area on this kernel.
Because vt->vmalloc_start is still 0 at POST_GDB, arm64_kvtop() takes
the early "VTOP()" shortcut and returns a bogus physical address far
beyond max_mapnr, so read_diskdump() reports SEEK_ERROR:
...
<readmem: ffff800100000050, KVADDR, "IRQ stack pointer", 8, (ROE),
1bfe2ef0>
<read_diskdump: addr: ffff800100000050 paddr: 800100000050 cnt: 8>
read_diskdump: SEEK_ERROR: paddr/pfn: 800100000050/800100000 max_mapnr: 60c000000
crash: seek error: kernel virtual address: ffff800100000050 type: "IRQ stack
pointer"
<readmem: ffff800100021050, KVADDR, "IRQ stack pointer", 8, (ROE),
1bfe2ef8>
<read_diskdump: addr: ffff800100021050 paddr: 800100021050 cnt: 8>
read_diskdump: SEEK_ERROR: paddr/pfn: 800100021050/800100021 max_mapnr: 60c000000
crash: seek error: kernel virtual address: ffff800100021050 type: "IRQ stack
pointer"
<readmem: ffff800100042050, KVADDR, "IRQ stack pointer", 8, (ROE),
1bfe2f00>
<read_diskdump: addr: ffff800100042050 paddr: 800100042050 cnt: 8>
read_diskdump: SEEK_ERROR: paddr/pfn: 800100042050/800100042 max_mapnr: 60c000000
crash: seek error: kernel virtual address: ffff800100042050 type: "IRQ stack
pointer"
<readmem: ffff800100063050, KVADDR, "IRQ stack pointer", 8, (ROE),
1bfe2ef08>
<read_diskdump: addr: ffff800100063050 paddr: 800100063050 cnt: 8>
read_diskdump: SEEK_ERROR: paddr/pfn: 800100063050/800100063 max_mapnr: 60c000000
crash: seek error: kernel virtual address: ffff800100063050 type: "IRQ stack
pointer"
... (one SEEK_ERROR per CPU)
Note the wrong physical addresses (800100xxxxxx) come from a direct
linear-map style translation (addr - PAGE_OFFSET) which is only valid
for the linear map, not for vmalloc-backed percpu memory.
Root cause
----------
In arm64_init() the three stack initializers are invoked in the
POST_GDB phase:
case POST_GDB:
...
if (!machdep->machspec->CONFIG_ARM64_KERNELPACMASK)
arm64_recalc_KERNELPACMASK();
arm64_irq_stack_init(); /* <-- readmem on percpu var */
arm64_overflow_stack_init(); /* <-- readmem on percpu var */
arm64_stackframe_init();
break;
These helpers end up calling readmem() on percpu addresses that, on
the machine described above, live in the vmalloc range. The
translation goes through arm64_kvtop() (arm64.c:1963), which for any
vmalloc address requires:
1. vt->vmalloc_start != 0 -> to reach the IS_VMALLOC_ADDR() branch
2. vt->kernel_pgd[0] -> to walk the kernel page tables
Both are populated by vm_init(), which runs in the POST_VM phase --
i.e. *after* POST_GDB. As a result, in arm64_kvtop() we hit:
if (!vt->vmalloc_start) {
*paddr = VTOP(kvaddr); /* wrong for vmalloc addresses */
return TRUE;
}
and the returned physical address is bogus, so the subsequent
readmem() fails with SEEK_ERROR (as shown in the log above: paddr
800100xxxxxx is the linear-map translation of a vmalloc address, and
it falls well beyond max_mapnr 0x60c000000).
Proposed fix
------------
Move the three stack initializers from POST_GDB to POST_VM, so that
vt->vmalloc_start and vt->kernel_pgd are guaranteed to be ready
before any readmem() that may touch vmalloc-backed percpu data.
diff --git a/arm64.c b/arm64.c
index d177092..654b972 100644
--- a/arm64.c
+++ b/arm64.c
@@ -787,7 +787,9 @@ arm64_init(int when)
*/
if(!machdep->machspec->CONFIG_ARM64_KERNELPACMASK)
arm64_recalc_KERNELPACMASK();
+ break;
+ case POST_VM:
arm64_irq_stack_init();
arm64_overflow_stack_init();
arm64_stackframe_init();
Testing
-------
Verified on:
- the failing large-NUMA arm64 system whose percpu first chunk was
created by pcpu_page_first_chunk() (i.e. percpu lives in vmalloc):
with the patch, the percpu IRQ stack pointers are translated
through the kernel page tables and crash starts cleanly. "bt" on
the panic task produces correct irq/overflow/standby stacks;
- a regular arm64 QEMU VM whose percpu first chunk was created by
pcpu_embed_first_chunk() (i.e. percpu lives in the linear map):
no regression, stack initialization still happens before any
"bt" command.
I'd like to get feedback on whether POST_VM is the preferred phase,
or whether the maintainers would rather guard the readmem() calls
inside the three helpers with a fallback (e.g. defer percpu
translation until later). I can send a formal patch with
Signed-off-by once the approach is agreed.
Thanks