Re: [Crash-utility] crash seek error, failed to read vmcore file

----- "Pavan Naregundi" <pavan(a)linux.vnet.ibm.com&gt; wrote: I don't have access to the original "problem" ppc64 machine, but here I'm logged into another ppc64, where the memory advertised in /proc/device-tree is as expected. It has these file memory@xxx/reg files, showing a set of contiguous memory chunks. The first one is 128MB, followed by a series of 16MB chunks: memory@0: 0000000000000000 0000000008000000 memory@8000000: 0000000008000000 0000000001000000 memory@9000000: 0000000009000000 0000000001000000 memory@a000000: 000000000a000000 0000000001000000 memory@b000000: 000000000b000000 0000000001000000 memory@c000000: 000000000c000000 0000000001000000 memory@d000000: 000000000d000000 0000000001000000 memory@e000000: 000000000e000000 0000000001000000 memory@f000000: 000000000f000000 0000000001000000 memory@10000000: 0000000010000000 0000000001000000 memory@11000000: 0000000011000000 0000000001000000 memory@12000000: 0000000012000000 0000000001000000 memory@13000000: 0000000013000000 0000000001000000 memory@14000000: 0000000014000000 0000000001000000 memory@15000000: 0000000015000000 0000000001000000 memory@16000000: 0000000016000000 0000000001000000 memory@17000000: 0000000017000000 0000000001000000 memory@18000000: 0000000018000000 0000000001000000 memory@19000000: 0000000019000000 0000000001000000 memory@1a000000: 000000001a000000 0000000001000000 memory@1b000000: 000000001b000000 0000000001000000 memory@1c000000: 000000001c000000 0000000001000000 memory@1d000000: 000000001d000000 0000000001000000 memory@1e000000: 000000001e000000 0000000001000000 memory@1f000000: 000000001f000000 0000000001000000 memory@20000000: 0000000020000000 0000000001000000 memory@21000000: 0000000021000000 0000000001000000 memory@22000000: 0000000022000000 0000000001000000 memory@23000000: 0000000023000000 0000000001000000 memory@24000000: 0000000024000000 0000000001000000 memory@25000000: 0000000025000000 0000000001000000 memory@26000000: 0000000026000000 0000000001000000 memory@27000000: 0000000027000000 0000000001000000 memory@28000000: 0000000028000000 0000000001000000 memory@29000000: 0000000029000000 0000000001000000 memory@2a000000: 000000002a000000 0000000001000000 memory@2b000000: 000000002b000000 0000000001000000 memory@2c000000: 000000002c000000 0000000001000000 memory@2d000000: 000000002d000000 0000000001000000 memory@2e000000: 000000002e000000 0000000001000000 memory@2f000000: 000000002f000000 0000000001000000 memory@30000000: 0000000030000000 0000000001000000 memory@31000000: 0000000031000000 0000000001000000 memory@32000000: 0000000032000000 0000000001000000 memory@33000000: 0000000033000000 0000000001000000 memory@34000000: 0000000034000000 0000000001000000 memory@35000000: 0000000035000000 0000000001000000 memory@36000000: 0000000036000000 0000000001000000 memory@37000000: 0000000037000000 0000000001000000 memory@38000000: 0000000038000000 0000000001000000 memory@39000000: 0000000039000000 0000000001000000 memory@3a000000: 000000003a000000 0000000001000000 memory@3b000000: 000000003b000000 0000000001000000 memory@3c000000: 000000003c000000 0000000001000000 memory@3d000000: 000000003d000000 0000000001000000 memory@3e000000: 000000003e000000 0000000001000000 memory@3f000000: 000000003f000000 0000000001000000 memory@40000000: 0000000040000000 0000000001000000 memory@41000000: 0000000041000000 0000000001000000 memory@42000000: 0000000042000000 0000000001000000 memory@43000000: 0000000043000000 0000000001000000 memory@44000000: 0000000044000000 0000000001000000 memory@45000000: 0000000045000000 0000000001000000 memory@46000000: 0000000046000000 0000000001000000 memory@47000000: 0000000047000000 0000000001000000 memory@48000000: 0000000048000000 0000000001000000 memory@49000000: 0000000049000000 0000000001000000 memory@4a000000: 000000004a000000 0000000001000000 memory@4b000000: 000000004b000000 0000000001000000 memory@4c000000: 000000004c000000 0000000001000000 memory@4d000000: 000000004d000000 0000000001000000 memory@4e000000: 000000004e000000 0000000001000000 memory@4f000000: 000000004f000000 0000000001000000 memory@50000000: 0000000050000000 0000000001000000 memory@51000000: 0000000051000000 0000000001000000 memory@52000000: 0000000052000000 0000000001000000 memory@53000000: 0000000053000000 0000000001000000 memory@54000000: 0000000054000000 0000000001000000 memory@55000000: 0000000055000000 0000000001000000 memory@56000000: 0000000056000000 0000000001000000 memory@57000000: 0000000057000000 0000000001000000 memory@58000000: 0000000058000000 0000000001000000 memory@59000000: 0000000059000000 0000000001000000 memory@5a000000: 000000005a000000 0000000001000000 memory@5b000000: 000000005b000000 0000000001000000 memory@5c000000: 000000005c000000 0000000001000000 memory@5d000000: 000000005d000000 0000000001000000 memory@5e000000: 000000005e000000 0000000001000000 memory@5f000000: 000000005f000000 0000000001000000 memory@60000000: 0000000060000000 0000000001000000 memory@61000000: 0000000061000000 0000000001000000 memory@62000000: 0000000062000000 0000000001000000 memory@63000000: 0000000063000000 0000000001000000 memory@64000000: 0000000064000000 0000000001000000 memory@65000000: 0000000065000000 0000000001000000 memory@66000000: 0000000066000000 0000000001000000 memory@67000000: 0000000067000000 0000000001000000 memory@68000000: 0000000068000000 0000000001000000 memory@69000000: 0000000069000000 0000000001000000 memory@6a000000: 000000006a000000 0000000001000000 memory@6b000000: 000000006b000000 0000000001000000 memory@6c000000: 000000006c000000 0000000001000000 memory@6d000000: 000000006d000000 0000000001000000 memory@6e000000: 000000006e000000 0000000001000000 memory@6f000000: 000000006f000000 0000000001000000 memory@70000000: 0000000070000000 0000000001000000 memory@71000000: 0000000071000000 0000000001000000 memory@72000000: 0000000072000000 0000000001000000 memory@73000000: 0000000073000000 0000000001000000 memory@74000000: 0000000074000000 0000000001000000 memory@75000000: 0000000075000000 0000000001000000 memory@76000000: 0000000076000000 0000000001000000 memory@77000000: 0000000077000000 0000000001000000 memory@78000000: 0000000078000000 0000000001000000 memory@79000000: 0000000079000000 0000000001000000 memory@7a000000: 000000007a000000 0000000001000000 memory@7b000000: 000000007b000000 0000000001000000 So the end of physical memory is at 7b000000 + 1000000, which I can verify by running crash on the live system: crash> eval 7b000000 + 0x1000000 hexadecimal: 7c000000 (1984MB) decimal: 2080374784 octal: 17400000000 binary: 0000000000000000000000000000000001111100000000000000000000000000 crash> eval 7c000000 / 64k hexadecimal: 7c00 (31KB) decimal: 31744 octal: 76000 binary: 0000000000000000000000000000000000000000000000000111110000000000 crash> p num_physpages num_physpages = $9 = 31744 crash> and which matches what's in the kernel memory zone data: crash> kmem -n NODE SIZE PGLIST_DATA BOOTMEM_DATA NODE_ZONES 0 31744 c00000007bfdf280 c000000000a53c50 c00000007bfdf280 c00000007bfe1900 c00000007bfe3f80 MEM_MAP START_PADDR START_MAPNR f000000000000000 0 0 ZONE NAME SIZE MEM_MAP START_PADDR START_MAPNR 0 DMA 31744 f000000000000000 0 0 1 Normal 0 0 0 0 2 Movable 0 0 0 0 ... So everything looks fine. But if your system has just a single /proc/device-tree/memory@0 directory whose size doesn't match up with what the live kernel is using, then that's the kernel bug. BTW, you can get minimal data from your truncated vmcore using the --minimal switch that IBM contributed a while back: # crash --minimal vmcore vmlinux It at least offers the log, dis, rd, sym and eval commands, which may or may not help. It's actually come in quite handy a few times. Anyway, if you guys come up with a kernel fix, can you post it here as well? Thanks, Dave