10:39 a.m.
Somewhere between the RHEL5 (2.6.18-based) and RHEL6 timeframe,
the ppc64 architecture has started using a virtual memmap scheme
for the arrays of page structures used to describe/handle
each physical page of memory.
In RHEL5, the page structures in the memmap array were unity-mapped
(i.e., the physical address is or'd with c000000000000000), as
"kmem -n" shows below in the sparsemem data breakdown under MEM_MAP:
crash> kmem -n
... [ snip ] ...
NR SECTION CODED_MEM_MAP MEM_MAP PFN
0 c000000000750000 c000000000760000 c000000000760000 0
1 c000000000750008 c000000000760000 c000000000763800 256
2 c000000000750010 c000000000760000 c000000000767000 512
3 c000000000750018 c000000000760000 c00000000076a800 768
4 c000000000750020 c000000000760000 c00000000076e000 1024
5 c000000000750028 c000000000760000 c000000000771800 1280
6 c000000000750030 c000000000760000 c000000000775000 1536
7 c000000000750038 c000000000760000 c000000000778800 1792
8 c000000000750040 c000000000760000 c00000000077c000 2048
9 c000000000750048 c000000000760000 c00000000077f800 2304
10 c000000000750050 c000000000760000 c000000000783000 2560
11 c000000000750058 c000000000760000 c000000000786800 2816
12 c000000000750060 c000000000760000 c00000000078a000 3072
...
also shown via the memmap page structure listing displayed by
"kmem -p":
crash> kmem -p
PAGE PHYSICAL MAPPING INDEX CNT FLAGS
c000000000760000 0 0 0 1 400
c000000000760038 10000 0 0 1 400
c000000000760070 20000 0 0 1 400
c0000000007600a8 30000 0 0 1 400
c0000000007600e0 40000 0 0 1 400
c000000000760118 50000 0 0 1 400
c000000000760150 60000 0 0 1 400
c000000000760188 70000 0 0 1 400
c0000000007601c0 80000 0 0 1 400
c0000000007601f8 90000 0 0 1 400
...
In RHEL6 (2.6.31-38.el6) the memmap page array is apparently
virtually memmap'd -- using a virtual range of memory starting
at a heretofore-unseen virtual address range starting at
f000000000000000:
crash> kmem -n
... [ snip ] ...
NR SECTION CODED_MEM_MAP MEM_MAP PFN
0 c000000002160000 f000000000000000 f000000000000000 0
1 c000000002160020 f000000000000000 f000000000006800 256
2 c000000002160040 f000000000000000 f00000000000d000 512
3 c000000002160060 f000000000000000 f000000000013800 768
4 c000000002160080 f000000000000000 f00000000001a000 1024
5 c0000000021600a0 f000000000000000 f000000000020800 1280
6 c0000000021600c0 f000000000000000 f000000000027000 1536
7 c0000000021600e0 f000000000000000 f00000000002d800 1792
8 c000000002160100 f000000000000000 f000000000034000 2048
9 c000000002160120 f000000000000000 f00000000003a800 2304
10 c000000002160140 f000000000000000 f000000000041000 2560
... [ snip ] ...
crash> kmem -p
PAGE PHYSICAL MAPPING INDEX CNT FLAGS
f000000000000000 0 0 0 0 0
f000000000000068 10000 0 0 0 0
f0000000000000d0 20000 0 0 0 0
f000000000000138 30000 0 0 0 0
f0000000000001a0 40000 0 0 0 0
f000000000000208 50000 0 -4611686016392006416 0 0
f000000000000270 60000 0 0 0 0
f0000000000002d8 70000 0 0 0 0
f000000000000340 80000 0 0 0 0
f0000000000003a8 90000 0 -4611686016730798344 0 0
f000000000000410 a0000 0 0 0 0
f000000000000478 b0000 0 0 0 0
f0000000000004e0 c0000 0 0 0 c0000000651534e0
f000000000000548 d0000 0 0 0 0
...
But as can be seen in the "kmem -p" output, and when using other
commands that actually read the data in the page structure, the
data read is either bogus or the readmem() of the address just fails
the virtual address translation and indicates that the page is not mapped.
Because the page structures' virtual address is not unity-mapped,
the page address gets translated via page table walk-through in the
same manner as vmalloc()'d addresses. In the ppc64 architecture,
the vmalloc range starts at d000000000000000:
crash> mach
...
KERNEL VIRTUAL BASE: c000000000000000
KERNEL VMALLOC BASE: d000000000000000
...
Since the ppc64 virtual-to-physical address translations of
these f000000000000000-based addresses returns either a
bogus physical address or fails entirely, this in turn causes
bizarre errors in crash commands that actually read the contents
of page structures -- such as "kmem -s", where slub data is
stored in the page structure.
So my speculation (guess?) is that the ppc64.c ppc64_vtop()
function needs updating to properly translate these addresses.
Since the ppc64 stuff in the crash utility was written by, and
has been maintained by IBM (and since I am ppc64-challenged),
can you guys take a look at what needs to be done?
Thanks,
Dave