Re: [Crash-utility] earlier patch introducing the --kaslr option

Hi Andy, I've got a ELF kdump vmcore that was created in-house from a kernel configured with CONFIG_RANDOMIZE_BASE. I thought I might be able to analyze it by applying your earlier patch that introduced the --kaslr option. The kernel does not have the offset registered in the vmcoreinfo, and so I'm trying to determine the offset, but with no luck. Earlier, Kees had mentioned this: >> FWIW, the offset reported during a panic to dmesg is: >> (unsigned long)&_text - __START_KERNEL Where does it get reported during a panic exactly? Here's the oops trace, gotten by running "strings" on the vmcore:

$ strings vmcore ... [ cut ] ... SysRq : Trigger a crash "BUG: unable to handle kernel NULL pointer dereference at (null) "IP: [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b PGD 3a067 PUD 2e067 PMD 0 Oops: 0002 [#1] PREEMPT SMP 0" Modules linked in: CPU: 0 PID: 1720 Comm: bash Not tainted 3.14.0-rc1+ #1130" task: ffff88001d028000 ti: ffff88001c986000 task.ti: ffff88001c986000 RIP: 0010:[<ffffffff992bf6cf>] [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b RSP: 0018:ffff88001c987e90 EFLAGS: 000100920" RAX: 000000000000000f RBX: ffffffff9975ed50 RCX: 0000000000000000 RDX: ffff88001d028000 RSI: ffff88001cc0e338 RDI: 0000000000000063 RBP: ffff88001c987e90 R08: 0000000000000002 R09: 0000000000000000 R10: ffffffff994e9630 R11: 0000000000000000 R12: 0000000000000007 R13: 0000000000000246 R14: 0000000000000063 R15: 0000000000000000 FS: 00007f0ec2181740(0000) GS:ffff88001cc00000(0000) knlGS:00000000000000000" CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 000000001d36f000 CR4: 00000000000006f0 Stack: ffff88001c987ec8 ffffffff992bfc88 0000000000000002 00007f0ec21870000" 0000000000000002 ffff88001c987f58 0000000000000000 ffff88001c987ee80" ffffffff992c000c ffff88001c92acc0 00007f0ec2187000 ffff88001c987f080" Call Trace: [<ffffffff992bfc88>] __handle_sysrq+0x9b/0x133 [<ffffffff992c000c>] write_sysrq_trigger+0x2d/0x3e [<ffffffff991681cb>] proc_reg_write+0x45/0x65 [<ffffffff9911897c>] vfs_write+0xbf/0x17c [<ffffffff9911918f>] SyS_write+0x44/0x7a [<ffffffff9949ad7d>] system_call_fastpath+0x1a/0x1f0" Code: 4f 00 00 55 b8 01 00 00 00 48 89 e5 75 07 0f b6 05 b3 20 4f 00 83 e0 01 5d c3 55 c7 05 03 18 61 00 01 00 00 00 48 89 e5 0f ae f8 <c6> 04 25 00 00 00 00 01 5d c3 55 31 c0 c7 05 ba dc 46 00 07 00 0" "RIP [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b RSP <ffff88001c987e90> CR2: 0000000000000000 ttySffffffff99000000 T _text UUUU UUUU VMCOREINFO OSRELEASE=3.14.0-rc1+ PAGESIZE=4096 SYMBOL(init_uts_ns)=ffffffff99713250 SYMBOL(node_online_map)=ffffffff997b0c68 SYMBOL(swapper_pg_dir)=ffffffff9970e000 SYMBOL(_stext)=ffffffff990001c8 SYMBOL(vmap_area_list)=ffffffff99745c20 SYMBOL(mem_map)=ffffffff9a1253a8 SYMBOL(contig_page_data)=ffffffff99790000 SYMBOL(mem_section)=ffffffff9a126000 LENGTH(mem_section)=2048 SIZE(mem_section)=16 OFFSET(mem_section.section_mem_map)=0 SIZE(page)=64 SIZE(pglist_data)=53248 SIZE(zone)=12288 SIZE(free_area)=88 SIZE(list_head)=16 SIZE(nodemask_t)=8 OFFSET(page.flags)=0 OFFSET(page._count)=28 OFFSET(page.mapping)=8 OFFSET(page.lru)=32 OFFSET(page._mapcount)=24 OFFSET(page.private)=48 OFFSET(pglist_data.node_zones)=0 OFFSET(pglist_data.nr_zones)=49240 OFFSET(pglist_data.node_start_pfn)=49304 OFFSET(pglist_data.node_spanned_pages)=49320 OFFSET(pglist_data.node_id)=49328 OFFSET(zone.free_area)=256 OFFSET(zone.vm_stat)=4280 OFFSET(zone.spanned_pages)=8232 OFFSET(free_area.free_list)=0 OFFSET(list_head.next)=0 OFFSET(list_head.prev)=8 OFFSET(vmap_area.va_start)=0 OFFSET(vmap_area.list)=48 LENGTH(zone.free_area)=11 SYMBOL(log_buf)=ffffffff9972d290 SYMBOL(log_buf_len)=ffffffff9972d288 SYMBOL(log_first_idx)=ffffffff9a11eb48 SYMBOL(log_next_idx)=ffffffff9a11eb38 SIZE(printk_log)=16 OFFSET(printk_log.ts_nsec)=0 OFFSET(printk_log.len)=8 OFFSET(printk_log.text_len)=10 OFFSET(printk_log.dict_len)=12 LENGTH(free_area.free_list)=5 NUMBER(NR_FREE_PAGES)=0 NUMBER(PG_lru)=5 NUMBER(PG_private)=11 NUMBER(PG_swapcache)=16 NUMBER(PG_slab)=7 NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128 SYMBOL(phys_base)=ffffffff99713010 SYMBOL(init_level4_pgt)=ffffffff9970e000 CRASHTIME=1391826079 OSRELEASE=3.14.0-rc1+ PAGESIZE=4096 SYMBOL(init_uts_ns)=ffffffff99713250 SYMBOL(node_online_map)=ffffffff997b0c68 SYMBOL(swapper_pg_dir)=ffffffff9970e000 SYMBOL(_stext)=ffffffff990001c8 SYMBOL(vmap_area_list)=ffffffff99745c20 SYMBOL(mem_map)=ffffffff9a1253a8 SYMBOL(contig_page_data)=ffffffff99790000 SYMBOL(mem_section)=ffffffff9a126000 LENGTH(mem_section)=2048 SIZE(mem_section)=16 OFFSET(mem_section.section_mem_map)=0 SIZE(page)=64 SIZE(pglist_data)=53248 SIZE(zone)=12288 SIZE(free_area)=88 SIZE(list_head)=16 SIZE(nodemask_t)=8 OFFSET(page.flags)=0 OFFSET(page._count)=28 OFFSET(page.mapping)=8 OFFSET(page.lru)=32 OFFSET(page._mapcount)=24 OFFSET(page.private)=48 OFFSET(pglist_data.node_zones)=0 OFFSET(pglist_data.nr_zones)=49240 OFFSET(pglist_data.node_start_pfn)=49304 OFFSET(pglist_data.node_spanned_pages)=49320 OFFSET(pglist_data.node_id)=49328 OFFSET(zone.free_area)=256 OFFSET(zone.vm_stat)=4280 OFFSET(zone.spanned_pages)=8232 OFFSET(free_area.free_list)=0 OFFSET(list_head.next)=0 OFFSET(list_head.prev)=8 OFFSET(vmap_area.va_start)=0 OFFSET(vmap_area.list)=48 LENGTH(zone.free_area)=11 SYMBOL(log_buf)=ffffffff9972d290 SYMBOL(log_buf_len)=ffffffff9972d288 SYMBOL(log_first_idx)=ffffffff9a11eb48 SYMBOL(log_next_idx)=ffffffff9a11eb38 SIZE(printk_log)=16 OFFSET(printk_log.ts_nsec)=0 OFFSET(printk_log.len)=8 OFFSET(printk_log.text_len)=10 OFFSET(printk_log.dict_len)=12 LENGTH(free_area.free_list)=5 NUMBER(NR_FREE_PAGES)=0 NUMBER(PG_lru)=5 NUMBER(PG_private)=11 NUMBER(PG_swapcache)=16 NUMBER(PG_slab)=7 NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128 SYMBOL(phys_base)=ffffffff99713010 SYMBOL(init_level4_pgt)=ffffffff9970e000 CRASHTIME=1391826079 ... Anyway, the /proc/kallsyms file of the crashing system was saved, and it shows this: ffffffff99000000 T _text and if I subtract __START_KERNEL (ffffffff80000000) from that, I get what I presume is the kaslr offset of 0x19000000. The vmcore core header would seeminlgy confirm that: $ readelf -a vmcore ... [ cut ] ... Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align NOTE 0x0000000000001000 0x0000000000000000 0x0000000000000000 0x00000000000007f8 0x00000000000007f8 0 LOAD 0x0000000000002000 0xffffffff99000000 0x0000000019000000 <=== 0x0000000001183000 0x0000000001183000 RWE 0 LOAD 0x0000000001185000 0xffff880000001000 0x0000000000001000 0x000000000009f000 0x000000000009f000 RWE 0 LOAD 0x0000000001224000 0xffff880000100000 0x0000000000100000 0x0000000010f00000 0x0000000010f00000 RWE 0 LOAD 0x0000000012124000 0xffff880019000000 0x0000000019000000 0x0000000005194000 0x0000000005194000 RWE 0 LOAD 0x00000000172b8000 0xffff88001e1c1000 0x000000001e1c1000 0x00000000017c0000 0x00000000017c0000 RWE 0 LOAD 0x0000000018a78000 0xffff88001f9e5000 0x000000001f9e5000 0x00000000005fb000 0x00000000005fb000 RWE 0 But if I try that value with your patch applied, it fails in the same manner as if I don't use the --kaslr option at all: $ crash --kaslr 0x19000000 vmlinux vmcore crash 7.0.5rc12 Copyright (C) 2002-2014 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"... WARNING: could not find MAGIC_START! WARNING: cannot read linux_banner string crash: vmlinux and vmcore do not match! Usage: crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form) crash [OPTION]... [NAMELIST] (live system form) Enter "crash -h" for details. $ Any ideas? I can give you the vmlinux/vmcore/kallsyms triplet if you'd like. Thanks, Dave