Hello,
I have been using the crash tool to teach paging. It is an excellent
tool for simplifying page table walks for students. However, I have
encountered a persistent issue regarding stale data when inspecting
mappings that are populated mid-session.
When using the vtop command on a mapping that is not yet populated,
and then running it again after a memory operation has occurred, vtop
continues to return NULL or stale entries. This happens because the
FILL_PUD (and similar) macros check if the current PUD address matches
the last_pud_read address. If they match, the tool skips the readmem()
call, even if the underlying physical memory has changed.
In the debugging session below, I demonstrate that rd -p showed the
populated PUD entry, but vtop still reported 0. I was able to resolve
this by manually forcing a re-read in GDB by resetting the cache
variable:
(gdb) set machdep->last_pud_read=1
992 #define IS_LAST_PUD_READ(pud) ((ulong)(pud) == machdep->last_pud_read)
...
1001 #define FILL_PUD(PUD, TYPE, SIZE)
\
1002 if (!IS_LAST_PUD_READ(PUD)) {
\
1003 readmem((ulonglong)((ulong)(PUD)), TYPE,
machdep->pud, \
1004 SIZE, "pud page", FAULT_ON_ERROR);
\
1005 machdep->last_pud_read = (ulong)(PUD);
\
1006 }
Steps to Reproduce:
1) Run vtop on an unpopulated user address.
2) Trigger a page fault/memory access in the target process to
populate the entry.
3) Run vtop again; it will still show "(not mapped)" despite the
physical memory being updated.
Is this caching behavior intended for performance, or should a way to
invalidate this cache for live sessions be implemented?
crash> vtop -c 5084 0x41414000
[Detaching after fork from child process 5085]
VIRTUAL PHYSICAL
41414000 (not mapped)
PGD: 6e80000 => 7fc8067
PUD: 7fc8008 => 0
VMA START END FLAGS FILE
ffff88801ec382b8 41414000 41415000 8100073
crash> rd -p 7fc8008
[Detaching after fork from child process 5086]
7fc8008: 0000000000000000 ........
crash> vtop -c 5084 0x41414000
[Detaching after fork from child process 5087]
VIRTUAL PHYSICAL
41414000 (not mapped)
PGD: 6e80000 => 7fc8067
PUD: 7fc8008 => 0
VMA START END FLAGS FILE
ffff88801ec382b8 41414000 41415000 8100073
crash> rd -p 7fc8008
[Detaching after fork from child process 5088]
7fc8008: 000000000e576067 g`W.....
crash>
Thread 1 "crash" received signal SIGINT, Interrupt.
0x00007ffff629d141 in pselect () from /lib64/libc.so.6
=> 0x00007ffff629d141 <pselect+193>: 48 3d 00 f0 ff ff cmp
$0xfffffffffffff000,%rax
(gdb) en 2
(gdb) c
Continuing.
vtop -c 5084 0x41414000
[Detaching after fork from child process 5089]
VIRTUAL PHYSICAL
Thread 1 "crash" hit Breakpoint 2, x86_64_pud_offset
(pgd_pte=<optimized out>, vaddr=1094795264, verbose=0, IS_XEN=0) at
x86_64.c:1970
1970 FILL_PUD(pud_paddr, PHYSADDR, PAGESIZE());
=> 0x00005555557f5da5 <x86_64_pud_offset+85>: 48 8b 96 40 01 00 00 mov
0x140(%rsi),%rdx
0x00005555557f5dac <x86_64_pud_offset+92>: 48 39 9e 20 01 00 00 cmp
%rbx,0x120(%rsi)
0x00005555557f5db3 <x86_64_pud_offset+99>: 74 32 je
0x5555557f5de7 <x86_64_pud_offset+151>
0x00005555557f5db5 <x86_64_pud_offset+101>: 8b 4e 18 mov 0x18(%rsi),%ecx
0x00005555557f5db8 <x86_64_pud_offset+104>: 41 b9 01 00 00 00 mov
$0x1,%r9d
0x00005555557f5dbe <x86_64_pud_offset+110>: be 04 00 00 00 mov $0x4,%esi
0x00005555557f5dc3 <x86_64_pud_offset+115>: 48 89 df mov %rbx,%rdi
0x00005555557f5dc6 <x86_64_pud_offset+118>: 4c 8d 05 6c a8 58 00
lea 0x58a86c(%rip),%r8 # 0x555555d80639
0x00005555557f5dcd <x86_64_pud_offset+125>: e8 8e e2 f5 ff callq
0x555555754060 <readmem>
0x00005555557f5dd2 <x86_64_pud_offset+130>: 48 8b 35 87 8e a9 00
mov 0xa98e87(%rip),%rsi # 0x55555628ec60 <machdep>
0x00005555557f5dd9 <x86_64_pud_offset+137>: 48 89 9e 20 01 00 00
mov %rbx,0x120(%rsi)
0x00005555557f5de0 <x86_64_pud_offset+144>: 48 8b 96 40 01 00 00
mov 0x140(%rsi),%rdx
(gdb) set machdep->last_pud_read=1 <- This forces the PUD to be re-read
(gdb) dis
(gdb) c
Continuing.
41414000 f67e000
PGD: 6e80000 => 7fc8067
PUD: 7fc8008 => e576067
PMD: e576050 => aa1c067
PTE: aa1c0a0 => 800000000f67e867
PAGE: f67e000
PTE PHYSICAL FLAGS
800000000f67e867 f67e000 (PRESENT|RW|USER|ACCESSED|DIRTY|NX)
VMA START END FLAGS FILE
ffff88801ec382b8 41414000 41415000 8100073
PAGE PHYSICAL MAPPING INDEX CNT FLAGS
ffffea00003d9f80 f67e000 ffff8880142d40c1 41414 1 fffffc0040028
uptodate,lru,swapbacked
crash>
Best regards,
--
Anderson Nascimento
Allele Security Intelligence
https://www.allelesecurity.com