3:44 p.m.
Hi all,
This patchset was developed to improve detection of kaslr_offset and phys base.
First patch fixes VMware facility where CPU #0 has zeroed IDTR.
Second one introduces alternative method of finding kaslr_offset and phys base.
Alexey Makhalov (2):
vmware_vmss: get proper ITDR
kaslr: get offset by walking page tree
kaslr_helper.c | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
vmware_vmss.c | 21 +++++++++++
2 files changed, 136 insertions(+)
--
2.11.0
3:44 p.m.
New subject: [PATCH 1/2] vmware_vmss: get proper ITDR
To make kaslr_offset detection code happy.
calc_kaslr_offset() requires valid CR3 and IDTR values in order
to find kernel start virtual and physical addresses.
One of CPUs might have zeroed ITDR register caused by triggering
triple fault. If CPU #0 has zeroed IDTR, walk through other CPUs
to get valid and not zeroed value.
In case of only one CPU, we should use another approach such as
page tree walking to get kasr offset.
Signed-off-by: Alexey Makhalov <amakhalov(a)vmware.com>
---
vmware_vmss.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/vmware_vmss.c b/vmware_vmss.c
index b168f29..8aa0de0 100644
--- a/vmware_vmss.c
+++ b/vmware_vmss.c
@@ -881,6 +881,27 @@ vmware_vmss_get_cr3_idtr(ulong *cr3, ulong *idtr)
*cr3 = vmss.regs64[0]->cr[3];
*idtr = vmss.regs64[0]->idtr;
+ /*
+ * debug.guest is generated on guest triple fault. One of the possible
+ * ways to triple fault is: zeroing IDTR and int3.
+ * This is used in linux to reboot machine using triple fault:
+ * "reboot=t" cmdline.
+ * In that case CPU which triggered triple fault will have zeroed
+ * IDTR. IDTR and CR3 are used to calculate kaslr offset and phys
+ * base.
+ * If IDTR on CPU0 is zeroed and we have several CPUs, return IDTR from
+ * another CPU.
+ */
+ if (vmss.num_vcpus > 1 && *idtr == 0) {
+ int i;
+ for (i = 0; i < vmss.num_vcpus; i++) {
+ if (vmss.vcpu_regs[i] == REGS_PRESENT_ALL) {
+ *idtr = vmss.regs64[i]->idtr;
+ break;
+ }
+ }
+ }
+
return TRUE;
}
--
2.11.0
12:58 a.m.
New subject: [PATCH 1/2] vmware_vmss: get proper ITDR
diff --git a/vmware_vmss.c b/vmware_vmss.c
index b168f29..8aa0de0 100644
--- a/vmware_vmss.c
+++ b/vmware_vmss.c
@@ -881,6 +881,27 @@ vmware_vmss_get_cr3_idtr(ulong *cr3, ulong *idtr)
*cr3 = vmss.regs64[0]->cr[3];
*idtr = vmss.regs64[0]->idtr;
+ /*
+ * debug.guest is generated on guest triple fault. One of the possible
+ * ways to triple fault is: zeroing IDTR and int3.
+ * This is used in linux to reboot machine using triple fault:
+ * "reboot=t" cmdline.
+ * In that case CPU which triggered triple fault will have zeroed
+ * IDTR. IDTR and CR3 are used to calculate kaslr offset and phys
+ * base.
+ * If IDTR on CPU0 is zeroed and we have several CPUs, return IDTR from
+ * another CPU.
+ */
+ if (vmss.num_vcpus > 1 && *idtr == 0) {
+ int i;
+ for (i = 0; i < vmss.num_vcpus; i++) {
+ if (vmss.vcpu_regs[i] == REGS_PRESENT_ALL) {
+ *idtr = vmss.regs64[i]->idtr;
+ break;
+ }
+ }
+ }
+
return TRUE;
}
It looks that this is useful regardless of crash dump mechanism such as VMware, virsh
dump, sadump and so on.
Should be in kaslr_helper.c.
Thanks.
HATAYAMA, Daisuke
7:58 p.m.
New subject: [PATCH 1/2] vmware_vmss: get proper ITDR
Hi Daisuke,
It looks that this is useful regardless of crash dump mechanism such as VMware, virsh
dump, sadump and so on.
Should be in kaslr_helper.c.
Agree. It can be added later after introduction of pc->get_register(cpu, regno) API, so
high-level code such as kaslr_helper.c can use it.
As of now, backup method (page tree walking) implemented in this patch set will handle
zeroed IDTR.
Thanks,
—Alexey
3:44 p.m.
New subject: [PATCH 2/2] kaslr: get offset by walking page tree
This method requires only valid CR3. It walks through
page tree starting from __START_KERNEL_map to get real
_stext and its physical address.
It is used as backup method to get kaslr offset, if
IDTR is not valid (zeroed). It might happen when kernel
invalidates IDT, for example triggering triple fault on
reboot (reboot=t cmdline).
This method does not support PTI (Page Table Isolation)
case where CR3 points to the isolated page tree. So, use
it only when CR3 points to "full" kernel.
Signed-off-by: Alexey Makhalov <amakhalov(a)vmware.com>
---
kaslr_helper.c | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 115 insertions(+)
diff --git a/kaslr_helper.c b/kaslr_helper.c
index bb19e54..f11cb55 100644
--- a/kaslr_helper.c
+++ b/kaslr_helper.c
@@ -322,6 +322,104 @@ quit:
}
/*
+ * Find virtual (VA) and physical (PA) addresses of kernel start
+ *
+ * va:
+ * Actual address of the kernel start (_stext) placed
+ * randomly by kaslr feature. To be more accurate,
+ * VA = _stext(from vmlinux) + kaslr_offset
+ *
+ * pa:
+ * Physical address where the kerenel is placed.
+ *
+ * In nokaslr case, VA = _stext (from vmlinux)
+ * In kaslr case, virtual address of the kernel placement goes
+ * in this range: ffffffff80000000..ffffffff9fffffff, or
+ * __START_KERNEL_map..+512MB
+ *
+ * https://www.kernel.org/doc/Documentation/x86/x86_64/mm.txt
+ *
+ * Randomized VA will be the first valid page starting from
+ * ffffffff80000000 (__START_KERNEL_map). Page tree entry of
+ * this page will contain the PA of the kernel start.
+ *
+ * NOTES:
+ * 1. This method does not support PTI (Page Table Isolation)
+ * case where CR3 points to the isolated page tree.
+ * 2. 4-level paging support only, as caller (calc_kaslr_offset)
+ * does not support 5-level paging.
+ */
+static int
+find_kernel_start(ulong *va, ulong *pa)
+{
+ int i, pgd_idx, pud_idx, pmd_idx, pte_idx;
+ uint64_t pgd_pte, pud_pte, pmd_pte, pte;
+
+ pgd_idx = pgd_index(__START_KERNEL_map);
+ pud_idx = pud_index(__START_KERNEL_map);
+ pmd_idx = pmd_index(__START_KERNEL_map);
+ pte_idx = pte_index(__START_KERNEL_map);
+
+ for (; pgd_idx < PTRS_PER_PGD; pgd_idx++) {
+ pgd_pte = ULONG(machdep->pgd + pgd_idx * sizeof(uint64_t));
+ if (pgd_pte & _PAGE_PRESENT)
+ break;
+ pud_idx = pmd_idx = pte_idx = 0;
+ }
+ if (pgd_idx == PTRS_PER_PGD)
+ return FALSE;
+
+ FILL_PUD(pgd_pte & PHYSICAL_PAGE_MASK, PHYSADDR, PAGESIZE());
+ for (; pud_idx < PTRS_PER_PUD; pud_idx++) {
+ pud_pte = ULONG(machdep->pud + pud_idx * sizeof(uint64_t));
+ if (pud_pte & _PAGE_PRESENT)
+ break;
+ pmd_idx = pte_idx = 0;
+ }
+ if (pud_idx == PTRS_PER_PUD)
+ return FALSE;
+ if (pud_pte & _PAGE_PSE) {
+ /* 1GB page */
+ *va = (~__VIRTUAL_MASK) | ((ulong)pgd_idx << __PGDIR_SHIFT) |
+ ((ulong)pud_idx << PUD_SHIFT);
+ *pa = pud_pte & PHYSICAL_PAGE_MASK;
+ return TRUE;
+ }
+
+ FILL_PMD(pud_pte & PHYSICAL_PAGE_MASK, PHYSADDR, PAGESIZE());
+ for (; pmd_idx < PTRS_PER_PMD; pmd_idx++) {
+ pmd_pte = ULONG(machdep->pmd + pmd_idx * sizeof(uint64_t));
+ if (pmd_pte & _PAGE_PRESENT)
+ break;
+ pte_idx = 0;
+ }
+ if (pmd_idx == PTRS_PER_PMD)
+ return FALSE;
+ if (pmd_pte & _PAGE_PSE) {
+ /* 2MB page */
+ *va = (~__VIRTUAL_MASK) | ((ulong)pgd_idx << __PGDIR_SHIFT) |
+ ((ulong)pud_idx << PUD_SHIFT) | (pmd_idx << PMD_SHIFT);
+ *pa = pmd_pte & PHYSICAL_PAGE_MASK;
+ return TRUE;
+ }
+
+ FILL_PTBL(pmd_pte & PHYSICAL_PAGE_MASK, PHYSADDR, PAGESIZE());
+ for (; pte_idx < PTRS_PER_PTE; pte_idx++) {
+ pte = ULONG(machdep->ptbl + pte_idx * sizeof(uint64_t));
+ if (pte & _PAGE_PRESENT)
+ break;
+ }
+ if (pte_idx == PTRS_PER_PTE)
+ return FALSE;
+
+ *va = (~__VIRTUAL_MASK) | ((ulong)pgd_idx << __PGDIR_SHIFT) |
+ ((ulong)pud_idx << PUD_SHIFT) | (pmd_idx << PMD_SHIFT) |
+ (pte_idx << PAGE_SHIFT);
+ *pa = pmd_pte & PHYSICAL_PAGE_MASK;
+ return TRUE;
+}
+
+/*
* Calculate kaslr_offset and phys_base
*
* kaslr_offset:
@@ -445,6 +543,22 @@ retry:
goto quit;
}
+ if (idtr == 0 && st->_stext_vmlinux && (st->_stext_vmlinux !=
UNINITIALIZED)) {
+ ulong va, pa;
+ ret = find_kernel_start(&va, &pa);
+ if (ret == FALSE)
+ goto quit;
+ if (CRASHDEBUG(1)) {
+ fprintf(fp, "calc_kaslr_offset: _stext(vmlinux): %lx\n",
st->_stext_vmlinux);
+ fprintf(fp, "calc_kaslr_offset: kernel start VA: %lx\n", va);
+ fprintf(fp, "calc_kaslr_offset: kernel start PA: %lx\n", pa);
+ }
+ kaslr_offset = va - st->_stext_vmlinux;
+ phys_base = pa - (va - __START_KERNEL_map);
+
+ goto found;
+ }
+
/* Convert virtual address of IDT table to physical address */
if (!kvtop(NULL, idtr, &idtr_paddr, verbose)) {
if (SADUMP_DUMPFILE())
@@ -505,6 +619,7 @@ retry:
fprintf(fp, "kaslr_helper: asssuming the kdump 1st kernel.\n");
}
+found:
if (CRASHDEBUG(1)) {
fprintf(fp, "calc_kaslr_offset: kaslr_offset=%lx\n",
kaslr_offset);
--
2.11.0
1343
days inactive
1355
days old
devel@lists.crash-utility.osci.io
4 comments
2 participants
participants (2)
-
Alexey Makhalov -
d.hatayama＠fujitsu.com