9:14 p.m.
This series fix the bug caused by arm64 kernel memory layout changes
v8 -> v9:
reorder the series
drop phys_offset_nominal
This series can work on v5.13 kernel (RHEL-9) and for compatibility, on v4.18 (RHEL-8)
Cc: HAGIO KAZUHITO <k-hagio-ab(a)nec.com>
Cc: Lianbo Jiang <lijiang(a)redhat.com>
Cc: Bhupesh Sharma <bhupesh.sharma(a)linaro.org>
To: crash-utility(a)redhat.com
Pingfan Liu (4):
crash-utility/arm64: rename ARM64_PAGE_OFFSET_ACTUAL as
ARM64_FLIP_PAGE_OFFSET_ACTUAL
crash-utility/arm64: assign page_offset with kernel configure value
crash-utility/arm64: use dedicated bits to record the mem layout
changes
crash-utility/arm64: implement switchable PTOV()/VTOP()
arm64.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++--------
defs.h | 15 ++++++++------
2 files changed, 62 insertions(+), 14 deletions(-)
--
2.29.2
9:14 p.m.
New subject: [PATCH v9 1/4] crash-utility/arm64: rename ARM64_PAGE_OFFSET_ACTUAL as ARM64_FLIP_PAGE_OFFSET_ACTUAL
Renaming, so it is better to reflect the flip layout of kernel VA, which
is introduced by kernel commit 14c127c957c1 ("arm64: mm: Flip kernel VA space")
Signed-off-by: Pingfan Liu <piliu(a)redhat.com>
Cc: HAGIO KAZUHITO <k-hagio-ab(a)nec.com>
Cc: Lianbo Jiang <lijiang(a)redhat.com>
Cc: Bhupesh Sharma <bhupesh.sharma(a)linaro.org>
To: crash-utility(a)redhat.com
---
arm64.c | 10 ++++++----
defs.h | 3 ++-
2 files changed, 8 insertions(+), 5 deletions(-)
diff --git a/arm64.c b/arm64.c
index 8934961..9fe1a4a 100644
--- a/arm64.c
+++ b/arm64.c
@@ -217,10 +217,12 @@ arm64_init(int when)
arm64_calc_VA_BITS();
arm64_calc_KERNELPACMASK();
ms = machdep->machspec;
+
+ /* vabits_actual introduced after mm flip, so it should be flipped layout */
if (ms->VA_BITS_ACTUAL) {
- ms->page_offset = ARM64_PAGE_OFFSET_ACTUAL;
- machdep->identity_map_base = ARM64_PAGE_OFFSET_ACTUAL;
- machdep->kvbase = ARM64_PAGE_OFFSET_ACTUAL;
+ ms->page_offset = ARM64_FLIP_PAGE_OFFSET_ACTUAL;
+ machdep->identity_map_base = ARM64_FLIP_PAGE_OFFSET_ACTUAL;
+ machdep->kvbase = ARM64_FLIP_PAGE_OFFSET_ACTUAL;
ms->userspace_top = ARM64_USERSPACE_TOP_ACTUAL;
} else {
ms->page_offset = ARM64_PAGE_OFFSET;
@@ -401,7 +403,7 @@ arm64_init(int when)
fprintf(fp, "CONFIG_ARM64_VA_BITS: %ld\n", ms->CONFIG_ARM64_VA_BITS);
fprintf(fp, " VA_BITS_ACTUAL: %ld\n", ms->VA_BITS_ACTUAL);
fprintf(fp, "(calculated) VA_BITS: %ld\n", ms->VA_BITS);
- fprintf(fp, " PAGE_OFFSET: %lx\n", ARM64_PAGE_OFFSET_ACTUAL);
+ fprintf(fp, " PAGE_OFFSET: %lx\n", ARM64_FLIP_PAGE_OFFSET_ACTUAL);
fprintf(fp, " VA_START: %lx\n", ms->VA_START);
fprintf(fp, " modules: %lx - %lx\n", ms->modules_vaddr,
ms->modules_end);
fprintf(fp, " vmalloc: %lx - %lx\n", ms->vmalloc_start_addr,
ms->vmalloc_end);
diff --git a/defs.h b/defs.h
index 6bb00e2..af0dc8f 100644
--- a/defs.h
+++ b/defs.h
@@ -3232,7 +3232,8 @@ typedef signed int s32;
#define ARM64_PAGE_OFFSET ((0xffffffffffffffffUL) \
<< (machdep->machspec->VA_BITS - 1))
-#define ARM64_PAGE_OFFSET_ACTUAL ((0xffffffffffffffffUL) \
+/* kernels >= v5.4 the kernel VA space is flipped */
+#define ARM64_FLIP_PAGE_OFFSET_ACTUAL ((0xffffffffffffffffUL) \
- ((1UL) << machdep->machspec->VA_BITS_ACTUAL) + 1)
#define ARM64_USERSPACE_TOP ((1UL) << machdep->machspec->VA_BITS)
--
2.29.2
9:14 p.m.
New subject: [PATCH v9 2/4] crash-utility/arm64: assign page_offset with kernel configure value
On RHEL9, crash hits a bug when executing "crash /proc/kcore" : seek
error: kernel virtual address: ffff6a0f3fff0000 type: "pmd page"
The kernel virtual address does not vary with va_actual bits, instead,
is determined by configuration value. But crash does not observe this
fact.
Since vabits_actual related kernel commit is introduced after arm64 mm
layout flip commit, so changes are safe under the condition if
(ms->VA_BITS_ACTUAL), and keep else branch untouched.
Signed-off-by: Pingfan Liu <piliu(a)redhat.com>
Cc: HAGIO KAZUHITO <k-hagio-ab(a)nec.com>
Cc: Lianbo Jiang <lijiang(a)redhat.com>
Cc: Bhupesh Sharma <bhupesh.sharma(a)linaro.org>
To: crash-utility(a)redhat.com
---
arm64.c | 7 ++++---
defs.h | 1 +
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/arm64.c b/arm64.c
index 9fe1a4a..149db36 100644
--- a/arm64.c
+++ b/arm64.c
@@ -220,9 +220,10 @@ arm64_init(int when)
/* vabits_actual introduced after mm flip, so it should be flipped layout */
if (ms->VA_BITS_ACTUAL) {
- ms->page_offset = ARM64_FLIP_PAGE_OFFSET_ACTUAL;
- machdep->identity_map_base = ARM64_FLIP_PAGE_OFFSET_ACTUAL;
- machdep->kvbase = ARM64_FLIP_PAGE_OFFSET_ACTUAL;
+ ms->page_offset = ARM64_FLIP_PAGE_OFFSET;
+ /* useless on arm64 */
+ machdep->identity_map_base = ARM64_FLIP_PAGE_OFFSET;
+ machdep->kvbase = ARM64_FLIP_PAGE_OFFSET;
ms->userspace_top = ARM64_USERSPACE_TOP_ACTUAL;
} else {
ms->page_offset = ARM64_PAGE_OFFSET;
diff --git a/defs.h b/defs.h
index af0dc8f..60dcd39 100644
--- a/defs.h
+++ b/defs.h
@@ -3233,6 +3233,7 @@ typedef signed int s32;
#define ARM64_PAGE_OFFSET ((0xffffffffffffffffUL) \
<< (machdep->machspec->VA_BITS - 1))
/* kernels >= v5.4 the kernel VA space is flipped */
+#define ARM64_FLIP_PAGE_OFFSET (-(1UL) <<
machdep->machspec->CONFIG_ARM64_VA_BITS)
#define ARM64_FLIP_PAGE_OFFSET_ACTUAL ((0xffffffffffffffffUL) \
- ((1UL) << machdep->machspec->VA_BITS_ACTUAL) + 1)
--
2.29.2
9:14 p.m.
New subject: [PATCH v9 3/4] crash-utility/arm64: use dedicated bits to record the mem layout changes
arm64 memory layout experiences big changes due to the following kernel
commits in date decending order:
5. 7bc1a0f9e176 arm64: mm: use single quantity to represent the PA to VA translation
...
4. b6d00d47e81a arm64: mm: Introduce 52-bit Kernel VAs
...
3. 5383cc6efed1 arm64: mm: Introduce vabits_actual
...
2. 14c127c957c1 arm64: mm: Flip kernel VA space
...
1. f80fb3a3d508 arm64: add support for kernel ASLR
For 1, crash-utils has already used NEW_VMEMMAP to trace it.
For 2, crash-utils lacks a flag to tag it and handle it differently.
For 3, two important kernel variables vabits_actual and phyvirt_offset are introduced
For 4, since it comes immediately after 3, crash-utility does not need
to distinguish it.
For 5, kerne variable phyvirt_offset is removed
These changes have effects on PTOV()/VTOP() formula. So introducing
two bits HAS_PHYSVIRT_OFFSET and FLIPPED_VM as hint to apply different
formula.
Signed-off-by: Pingfan Liu <piliu(a)redhat.com>
Cc: HAGIO KAZUHITO <k-hagio-ab(a)nec.com>
Cc: Lianbo Jiang <lijiang(a)redhat.com>
Cc: Bhupesh Sharma <bhupesh.sharma(a)linaro.org>
To: crash-utility(a)redhat.com
---
arm64.c | 10 ++++++++++
defs.h | 2 ++
2 files changed, 12 insertions(+)
diff --git a/arm64.c b/arm64.c
index 149db36..b04369f 100644
--- a/arm64.c
+++ b/arm64.c
@@ -563,6 +563,10 @@ arm64_dump_machdep_table(ulong arg)
fprintf(fp, "%sMACHDEP_BT_TEXT", others++ ? "|" : "");
if (machdep->flags & NEW_VMEMMAP)
fprintf(fp, "%sNEW_VMEMMAP", others++ ? "|" : "");
+ if (machdep->flags & FLIPPED_VM)
+ fprintf(fp, "%sFLIPPED_VM", others++ ? "|" : "");
+ if (machdep->flags & HAS_PHYSVIRT_OFFSET)
+ fprintf(fp, "%sHAS_PHYSVIRT_OFFSET", others++ ? "|" :
"");
fprintf(fp, ")\n");
fprintf(fp, " kvbase: %lx\n", machdep->kvbase);
@@ -997,6 +1001,7 @@ arm64_calc_physvirt_offset(void)
if (READMEM(pc->mfd, &physvirt_offset, sizeof(physvirt_offset),
sp->value, sp->value -
machdep->machspec->kimage_voffset) > 0) {
+ machdep->flags |= HAS_PHYSVIRT_OFFSET;
ms->physvirt_offset = physvirt_offset;
}
}
@@ -3963,6 +3968,11 @@ arm64_calc_VA_BITS(void)
error(FATAL, "cannot determine VA_BITS_ACTUAL\n");
}
+ /*
+ * The mm flip commit is introduced before 52-bits VA, which is before the
+ * commit to export NUMBER(TCR_EL1_T1SZ)
+ */
+ machdep->flags |= FLIPPED_VM;
return;
}
diff --git a/defs.h b/defs.h
index 60dcd39..4fe6562 100644
--- a/defs.h
+++ b/defs.h
@@ -3213,6 +3213,8 @@ typedef signed int s32;
#define NEW_VMEMMAP (0x80)
#define VM_L4_4K (0x100)
#define UNW_4_14 (0x200)
+#define FLIPPED_VM (0x400)
+#define HAS_PHYSVIRT_OFFSET (0x800)
/*
* Get kimage_voffset from /dev/crash
--
2.29.2
9:14 p.m.
New subject: [PATCH v9 4/4] crash-utility/arm64: implement switchable PTOV()/VTOP()
Crash encounters a bug like the following:
...
SECTION_SIZE_BITS: 30
CONFIG_ARM64_VA_BITS: 52
VA_BITS_ACTUAL: 48
(calculated) VA_BITS: 48
PAGE_OFFSET: ffff000000000000
VA_START: ffff800000000000
modules: ffff800008000000 - ffff80000fffffff
vmalloc: ffff800010000000 - ffffffdfdffeffff
kernel image: ffff800010000000 - ffff800012750000
vmemmap: ffffffdfffe00000 - ffffffffffffffff
<readmem: ffff800011c53bc8, KVADDR, "nr_irqs", 4, (FOE), b47bdc>
<read_kdump: addr: ffff800011c53bc8 paddr: eb453bc8 cnt: 4>
read_netdump: addr: ffff800011c53bc8 paddr: eb453bc8 cnt: 4 offset: 1c73bc8
irq_stack_ptr:
type: 1, TYPE_CODE_PTR
target_typecode: 8, TYPE_CODE_INT
target_length: 8
length: 8
GNU_GET_DATATYPE[thread_union]: returned via gdb_error_hook
<readmem: ffff000b779c0050, KVADDR, "IRQ stack pointer", 8, (ROE),
3a37bea0>
<read_kdump: addr: ffff000b779c0050 paddr: fff1000bf79c0050 cnt: 8>
read_netdump: READ_ERROR: offset not found for paddr: fff1000bf79c0050
crash: read error: kernel virtual address: ffff000b779c0050 type: "IRQ stack
pointer"
<readmem: ffff000b77a60050, KVADDR, "IRQ stack pointer", 8, (ROE),
3a37bea8>
<read_kdump: addr: ffff000b77a60050 paddr: fff1000bf7a60050 cnt: 8>
read_netdump: READ_ERROR: offset not found for paddr: fff1000bf7a60050
...
Apparently, for a normal system, the 'paddr: fff1000bf79c0050' is
unreasonable.
This bug connects with kernel commit 7bc1a0f9e176 ("arm64: mm: use
single quantity to represent the PA to VA translation"), which removes
physvirt_offset kernel variable and change the PTOV()/VTOP() formulas.
Implement switchable PTOV()/VTOP() to cope with different kernel
version.
Signed-off-by: Pingfan Liu <piliu(a)redhat.com>
Cc: HAGIO KAZUHITO <k-hagio-ab(a)nec.com>
Cc: Lianbo Jiang <lijiang(a)redhat.com>
Cc: Bhupesh Sharma <bhupesh.sharma(a)linaro.org>
To: crash-utility(a)redhat.com
---
arm64.c | 40 ++++++++++++++++++++++++++++++++++++----
defs.h | 9 ++++-----
2 files changed, 40 insertions(+), 9 deletions(-)
diff --git a/arm64.c b/arm64.c
index b04369f..75fe5d6 100644
--- a/arm64.c
+++ b/arm64.c
@@ -994,8 +994,6 @@ arm64_calc_physvirt_offset(void)
ulong physvirt_offset;
struct syment *sp;
- ms->physvirt_offset = ms->phys_offset - ms->page_offset;
-
if ((sp = kernel_symbol_search("physvirt_offset")) &&
machdep->machspec->kimage_voffset) {
if (READMEM(pc->mfd, &physvirt_offset, sizeof(physvirt_offset),
@@ -1003,8 +1001,13 @@ arm64_calc_physvirt_offset(void)
machdep->machspec->kimage_voffset) > 0) {
machdep->flags |= HAS_PHYSVIRT_OFFSET;
ms->physvirt_offset = physvirt_offset;
+ return;
}
}
+
+ /* Useless if no symbol 'physvirt_offset', just keep semantics */
+ ms->physvirt_offset = ms->phys_offset - ms->page_offset;
+
}
static void
@@ -1051,6 +1054,7 @@ arm64_calc_phys_offset(void)
if (READMEM(pc->mfd, &phys_offset, sizeof(phys_offset),
vaddr, paddr) > 0) {
ms->phys_offset = phys_offset;
+
return;
}
}
@@ -1178,6 +1182,24 @@ arm64_init_kernel_pgd(void)
vt->kernel_pgd[i] = value;
}
+ulong arm64_PTOV(ulong paddr)
+{
+ ulong v;
+ struct machine_specific *ms = machdep->machspec;
+
+ /*
+ * Either older kernel before kernel has 'physvirt_offset' or newer kernel
which
+ * removes 'physvirt_offset' has the same formula:
+ * #define __phys_to_virt(x) ((unsigned long)((x) - PHYS_OFFSET) | PAGE_OFFSET)
+ */
+ if (!(machdep->flags & HAS_PHYSVIRT_OFFSET))
+ v = (paddr - ms->phys_offset) | PAGE_OFFSET;
+ else
+ v = paddr - ms->physvirt_offset;
+
+ return v;
+}
+
ulong
arm64_VTOP(ulong addr)
{
@@ -1188,8 +1210,18 @@ arm64_VTOP(ulong addr)
return addr - machdep->machspec->kimage_voffset;
}
- if (addr >= machdep->machspec->page_offset)
- return addr + machdep->machspec->physvirt_offset;
+ if (addr >= machdep->machspec->page_offset) {
+ if (machdep->flags & HAS_PHYSVIRT_OFFSET) {
+ return addr + machdep->machspec->physvirt_offset;
+ } else {
+ /*
+ * Either older kernel before kernel has 'physvirt_offset' or newer kernel
which
+ * removes 'physvirt_offset' has the same formula:
+ * #define __lm_to_phys(addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET)
+ */
+ return (addr & ~ PAGE_OFFSET) + machdep->machspec->phys_offset;
+ }
+ }
else if (machdep->machspec->kimage_voffset)
return addr - machdep->machspec->kimage_voffset;
else /* no randomness */
diff --git a/defs.h b/defs.h
index 4fe6562..5faa936 100644
--- a/defs.h
+++ b/defs.h
@@ -3091,11 +3091,6 @@ typedef u64 pte_t;
#define _64BIT_
#define MACHINE_TYPE "ARM64"
-#define PTOV(X) \
- ((unsigned long)(X) - (machdep->machspec->physvirt_offset))
-
-#define VTOP(X) arm64_VTOP((ulong)(X))
-
#define USERSPACE_TOP (machdep->machspec->userspace_top)
#define PAGE_OFFSET (machdep->machspec->page_offset)
#define VMALLOC_START (machdep->machspec->vmalloc_start_addr)
@@ -3105,6 +3100,9 @@ typedef u64 pte_t;
#define MODULES_VADDR (machdep->machspec->modules_vaddr)
#define MODULES_END (machdep->machspec->modules_end)
+#define PTOV(X) arm64_PTOV((ulong)(X))
+#define VTOP(X) arm64_VTOP((ulong)(X))
+
#define IS_VMALLOC_ADDR(X) arm64_IS_VMALLOC_ADDR((ulong)(X))
#define PAGEBASE(X) (((ulong)(X)) & (ulong)machdep->pagemask)
@@ -5909,6 +5907,7 @@ void unwind_backtrace(struct bt_info *);
void arm64_init(int);
void arm64_dump_machdep_table(ulong);
ulong arm64_VTOP(ulong);
+ulong arm64_PTOV(ulong);
int arm64_IS_VMALLOC_ADDR(ulong);
ulong arm64_swp_type(ulong);
ulong arm64_swp_offset(ulong);
--
2.29.2
1:40 a.m.
Thanks for the update, Pingfan.
The v9 change looks good to me. But I ran into some conflicts when
applying these patches locally to test, so
I would suggest submitting patches based on the latest crash code
every time, which can also help Kazu save time
although he will handle such issues.
For the v9:
Acked-by: Lianbo Jiang <lijiang(a)redhat.com>
On Fri, Jul 2, 2021 at 10:14 AM Pingfan Liu <piliu(a)redhat.com> wrote:
This series fix the bug caused by arm64 kernel memory layout changes
v8 -> v9:
reorder the series
drop phys_offset_nominal
This series can work on v5.13 kernel (RHEL-9) and for compatibility, on v4.18 (RHEL-8)
Cc: HAGIO KAZUHITO <k-hagio-ab(a)nec.com>
Cc: Lianbo Jiang <lijiang(a)redhat.com>
Cc: Bhupesh Sharma <bhupesh.sharma(a)linaro.org>
To: crash-utility(a)redhat.com
Pingfan Liu (4):
crash-utility/arm64: rename ARM64_PAGE_OFFSET_ACTUAL as
ARM64_FLIP_PAGE_OFFSET_ACTUAL
crash-utility/arm64: assign page_offset with kernel configure value
crash-utility/arm64: use dedicated bits to record the mem layout
changes
crash-utility/arm64: implement switchable PTOV()/VTOP()
arm64.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++--------
defs.h | 15 ++++++++------
2 files changed, 62 insertions(+), 14 deletions(-)
--
2.29.2
8:38 p.m.
On Fri, Jul 2, 2021 at 2:40 PM lijiang <lijiang(a)redhat.com> wrote:
Thanks for the update, Pingfan.
The v9 change looks good to me. But I ran into some conflicts when
applying these patches locally to test, so
I would suggest submitting patches based on the latest crash code
every time, which can also help Kazu save time
although he will handle such issues.
For the v9:
Acked-by: Lianbo Jiang <lijiang(a)redhat.com>
Thanks for your review and help.
This series is based on commit eaf14f8 Fix 'waitq' command for Linux
4.13 and later kernels
To Kazu, should I post a rebase version? (I have tried to rebase it
against HEAD. It can rebase automatically)
Thanks,
Pingfan
9:04 p.m.
-----Original Message-----
On Fri, Jul 2, 2021 at 2:40 PM lijiang <lijiang(a)redhat.com>
wrote:
>
> Thanks for the update, Pingfan.
>
> The v9 change looks good to me. But I ran into some conflicts when
> applying these patches locally to test, so
> I would suggest submitting patches based on the latest crash code
> every time, which can also help Kazu save time
> although he will handle such issues.
>
> For the v9:
> Acked-by: Lianbo Jiang <lijiang(a)redhat.com>
>
Thanks for your review and help.
This series is based on commit eaf14f8 Fix 'waitq' command for Linux
4.13 and later kernels
To Kazu, should I post a rebase version? (I have tried to rebase it
against HEAD. It can rebase automatically)
There's no need to repost. The v9 patchset was cleanly applied on
the current head and built with 'make target=ARM64'.
Thanks,
Kazu
3:24 a.m.
-----Original Message-----
This series fix the bug caused by arm64 kernel memory layout changes
v8 -> v9:
reorder the series
drop phys_offset_nominal
Thanks again for the update, applied with some slight adjustments:
https://github.com/crash-utility/crash/compare/d6b4f36...f53b73e
After removing the phys_offset_nominal, it looks like only the 1/2 and
2/2 patches might need to resolve the issues, but syncing with kernel
code will be also good for easier understanding.
Thanks,
Kazu
2:31 a.m.
Hi Kazu and Lianbo,
Thank both of you for making this fine and merged.
Regards,
Pingfan
On Mon, Jul 5, 2021 at 4:24 PM HAGIO KAZUHITO(萩尾 一仁) <k-hagio-ab(a)nec.com> wrote:
-----Original Message-----
> This series fix the bug caused by arm64 kernel memory layout changes
>
> v8 -> v9:
> reorder the series
> drop phys_offset_nominal
Thanks again for the update, applied with some slight adjustments:
https://github.com/crash-utility/crash/compare/d6b4f36...f53b73e
After removing the phys_offset_nominal, it looks like only the 1/2 and
2/2 patches might need to resolve the issues, but syncing with kernel
code will be also good for easier understanding.
Thanks,
Kazu
1227
days inactive
1231
days old
devel@lists.crash-utility.osci.io
9 comments
3 participants
participants (3)
-
HAGIO KAZUHITO(萩尾 一仁) -
lijiang -
Pingfan Liu