Hi Dave,
 Because I didn't subscribe the list, so please CC me when reply:) My
colleague help to forward your reply to me, so I decide to reply
directly in order to keep this mail in the thread other than new a thread.
On 11/23/2017 02:04 PM, Fei, Jie/è´¹ æ° wrote:
>
>
>
> -------- Forwarded Message --------
> Subject: Re: [Crash-utility] crash: page excluded: kernel virtual
> address: ffffffff81e3da50 type: "page_offset_base"
> Date: Wed, 22 Nov 2017 09:35:40 -0500
> From: Dave Anderson <anderson(a)redhat.com>
> Reply-To: Discussion list for crash utility usage, maintenance and
> development <crash-utility(a)redhat.com>
> To: Discussion list for crash utility usage, maintenance and
> development <crash-utility(a)redhat.com>
>
>
>
> ----- Original Message -----
>> Hi,
>>
>> I am using the latest crash tool & kernel 4.14 compiled from source,
>> and
>> I got the following error message. As I searched, this is fixed in
>> crash
>> 7.2.0, but I still have it here. So, is anyone has a clue?
>
> It's always going to be a crap-shoot with very recent upstream kernels,
> but you haven't given enough information to determine what the issue is.
>
Sorry, I am to new dump/crash area.
> If the vmcore was created by "virsh dump", and the kernel has KASLR
> enabled,
> then it's just not supported at this time. Otherwise, perhaps the
> output of
> "crash -d8" may yield some clues.
>
I am using kdump to create the vmcore. Actually, I did solve this issue
by add "nokaslr" to kernel parameter when I used the built-in crash of
Fedora. But when turn to the compiled version of my own, that solution
don't work. here is the output of `crash -d8`:
$ sudo ./crash -d8 /var/crash/127.0.0.1-2017-11-22-18\:21\:42/vmcore
../linux/vmlinux
crash 7.2.0++
Copyright (C) 2002-2017 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for
details.
compressed kdump: header->utsname.machine: x86_64
compressed kdump: memory bitmap offset: 2000
diskdump_data:
filename: /var/crash/127.0.0.1-2017-11-22-18:21:42/vmcore
flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
dfd: 3
ofp: 0
machine_type: 62 (EM_X86_64)
header: 1013930
signature: "KDUMP "
header_version: 6
utsname:
sysname: Linux
nodename: IAAS1
release: 4.14.0
version: #1 SMP Wed Nov 15 10:32:46 CST 2017
machine: x86_64
domainname: (none)
timestamp:
tv_sec: 5a154fac
tv_usec: 0
status: 2 (DUMP_DH_COMPRESSED_LZO)
block_size: 4096
sub_hdr_size: 1
bitmap_blocks: 80
max_mapnr: 1310208
total_ram_blocks: 0
device_blocks: 0
written_blocks: 0
current_cpu: 0
nr_cpus: 4
tasks[nr_cpus]: 0
0
0
0
sub_header: 0 (n/a)
sub_header_kdump: 1014940
phys_base: 0
dump_level: 31 (0x1f)
(DUMP_EXCLUDE_ZERO|DUMP_EXCLUDE_CACHE|DUMP_EXCLUDE_CACHE_PRI|DUMP_EXCLUDE_USER_DATA|DUMP_EXCLUDE_FREE)
split: 0
start_pfn: (unused)
end_pfn: (unused)
offset_vmcoreinfo: 5648 (0x1610)
size_vmcoreinfo: 1883 (0x75b)
OSRELEASE=4.14.0
PAGESIZE=4096
SYMBOL(init_uts_ns)=ffffffff81e10280
SYMBOL(node_online_map)=ffffffff82030e80
SYMBOL(swapper_pg_dir)=ffffffff81e09000
SYMBOL(_stext)=ffffffff81000000
SYMBOL(vmap_area_list)=ffffffff81efc470
SYMBOL(mem_section)=ffffffff82401dc0
LENGTH(mem_section)=2048
SIZE(mem_section)=16
OFFSET(mem_section.section_mem_map)=0
SIZE(page)=64
SIZE(pglist_data)=172864
SIZE(zone)=1664
SIZE(free_area)=104
SIZE(list_head)=16
SIZE(nodemask_t)=128
OFFSET(page.flags)=0
OFFSET(page._refcount)=28
OFFSET(page.mapping)=8
OFFSET(page.lru)=32
OFFSET(page._mapcount)=24
OFFSET(page.private)=48
OFFSET(page.compound_dtor)=40
OFFSET(page.compound_order)=44
OFFSET(page.compound_head)=32
OFFSET(pglist_data.node_zones)=0
OFFSET(pglist_data.nr_zones)=172192
OFFSET(pglist_data.node_start_pfn)=172200
OFFSET(pglist_data.node_spanned_pages)=172216
OFFSET(pglist_data.node_id)=172224
OFFSET(zone.free_area)=192
OFFSET(zone.vm_stat)=1472
OFFSET(zone.spanned_pages)=112
OFFSET(free_area.free_list)=0
OFFSET(list_head.next)=0
OFFSET(list_head.prev)=8
OFFSET(vmap_area.va_start)=0
OFFSET(vmap_area.list)=48
LENGTH(zone.free_area)=11
SYMBOL(log_buf)=ffffffff81e58480
SYMBOL(log_buf_len)=ffffffff81e5847c
SYMBOL(log_first_idx)=ffffffff823340f8
SYMBOL(clear_idx)=ffffffff823340cc
SYMBOL(log_next_idx)=ffffffff823340e8
SIZE(printk_log)=16
OFFSET(printk_log.ts_nsec)=0
OFFSET(printk_log.len)=8
OFFSET(printk_log.text_len)=10
OFFSET(printk_log.dict_len)=12
LENGTH(free_area.free_list)=6
NUMBER(NR_FREE_PAGES)=0
NUMBER(PG_lru)=5
NUMBER(PG_private)=12
NUMBER(PG_swapcache)=9
NUMBER(PG_slab)=8
NUMBER(PG_hwpoison)=22
NUMBER(PG_head_mask)=32768
NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128
NUMBER(HUGETLB_PAGE_DTOR)=2
NUMBER(phys_base)=0
SYMBOL(init_top_pgt)=ffffffff81e09000
SYMBOL(node_data)=ffffffff8202c6c0
LENGTH(node_data)=1024
KERNELOFFSET=0
NUMBER(KERNEL_IMAGE_SIZE)=1073741824
CRASHTIME=1511346092
offset_note: 4200 (0x1068)
size_note: 3332 (0xd04)
notes_buf: 1015950
num_prstatus_notes: 4
notes[0]: 1015950 (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 114 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
ORIG_RAX: ffffffffffffffff fpvalid: 0
R15: ffffffff82445b80 R14: ffffffff822f3422
R13: 0000000000000020 R12: 00000000000026f5
RBP: ffffc90000adbce0 RBX: ffffffff82445b80
R11: ffffffff822f342d R10: 0000000000000000
R9: 000000000000000f R8: 0000000000000000
RAX: 000000799065724b RCX: 0000007990656c1b
RDX: 0000000000000000 RSI: 0000000000000000
RDI: 0000000000000cdd RIP: ffffffff8188600f
RFLAGS: 0000000000000097 RSP: ffffc90000adbce0
FS_BASE: 0000000000000000
GS_BASE: 0000000000000000
CS: 0010 SS: 0018 DS: 0000
ES: 0000 FS: 0000 GS: 0000
notes[1]: 1015ab4 (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 2030 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
ORIG_RAX: ffffffffffffffff fpvalid: 0
R15: ffff880131999500 R14: ffffffff81f76fa0
R13: 0000000000000000 R12: 0000000000000007
RBP: ffffc90001dabdd8 RBX: 0000000000000063
R11: ffffffff822f342d R10: 0000000000000001
R9: 0000000000000007 R8: 00000000000002d9
RAX: 000000000000000f RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff88013fa8e138
RDI: 0000000000000063 RIP: ffffffff81532ad6
RFLAGS: 0000000000010282 RSP: ffffc90001dabdd8
FS_BASE: 00007fe58251fb80
GS_BASE: 0000000000000000
CS: 0010 SS: 0018 DS: 0000
ES: 0000 FS: 0000 GS: 0000
notes[2]: 1015c18 (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 0 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
ORIG_RAX: ffffffffffffffff fpvalid: 0
R15: ffffffff81f65618 R14: 0000000000000020
R13: 0000000000000004 R12: 0000000000000003
RBP: ffffc900006a3e60 RBX: 0000000000000008
R11: 000000000000298b R10: ffffc900006a3e40
R9: 0000000000000018 R8: 00000000000054a2
RAX: 0000000000000020 RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffffffff81f65480
RDI: 0000000000000002 RIP: ffffffff81895ddc
RFLAGS: 0000000000000046 RSP: ffffc900006a3e40
FS_BASE: 0000000000000000
GS_BASE: 0000000000000000
CS: 0010 SS: 0018 DS: 0000
ES: 0000 FS: 0000 GS: 0000
notes[3]: 1015d7c (NT_PRSTATUS)
si.signo: 0 si.code: 0 si.errno: 0
cursig: 0 sigpend: 0 sighold: 0
pid: 0 ppid: 0 pgrp: 0 sid:0
utime: 0.000000 stime: 0.000000
cutime: 0.000000 cstime: 0.000000
ORIG_RAX: ffffffffffffffff fpvalid: 0
R15: ffffffff81f65618 R14: 0000000000000020
R13: 0000000000000004 R12: 0000000000000003
RBP: ffffc900006abe60 RBX: 0000000000000008
R11: 000000000000037e R10: ffffc900006abe40
R9: 0000000000000008 R8: 00000000ffffffff
RAX: 0000000000000020 RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffffffff81f65480
RDI: 0000000000000003 RIP: ffffffff81895ddc
RFLAGS: 0000000000000046 RSP: ffffc900006abe40
FS_BASE: 0000000000000000
GS_BASE: 0000000000000000
CS: 0010 SS: 0018 DS: 0000
ES: 0000 FS: 0000 GS: 0000
snapshot_task: 0
num_qemu_notes: 0
NOTE offsets: 1068 (NT_PRSTATUS)
11cc (NT_PRSTATUS)
1330 (NT_PRSTATUS)
1494 (NT_PRSTATUS)
offset_eraseinfo: 0 (0x0)
size_eraseinfo: 0 (0x0)
start_pfn_64: (unused)
end_pfn_64: (unused)
max_mapnr_64: 1310208 (0x13fe00)
data_offset: 52000
block_size: 4096
block_shift: 12
bitmap: 7f8ff466c010
bitmap_len: 327680
max_mapnr: 1310208 (0x13fe00)
dumpable_bitmap: 7f8ff461b010
byte: 0
bit: 0
compressed_page: 10470a0
curbufptr: 0
page_cache_hdr[0]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1037090
pg_hit_count: 0
page_cache_hdr[1]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1038090
pg_hit_count: 0
page_cache_hdr[2]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1039090
pg_hit_count: 0
page_cache_hdr[3]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 103a090
pg_hit_count: 0
page_cache_hdr[4]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 103b090
pg_hit_count: 0
page_cache_hdr[5]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 103c090
pg_hit_count: 0
page_cache_hdr[6]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 103d090
pg_hit_count: 0
page_cache_hdr[7]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 103e090
pg_hit_count: 0
page_cache_hdr[8]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 103f090
pg_hit_count: 0
page_cache_hdr[9]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1040090
pg_hit_count: 0
page_cache_hdr[10]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1041090
pg_hit_count: 0
page_cache_hdr[11]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1042090
pg_hit_count: 0
page_cache_hdr[12]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1043090
pg_hit_count: 0
page_cache_hdr[13]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1044090
pg_hit_count: 0
page_cache_hdr[14]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1045090
pg_hit_count: 0
page_cache_hdr[15]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 1046090
pg_hit_count: 0
page_cache_buf: 1037090
evict_index: 0
evictions: 0
accesses: 0
cached_reads: 0
valid_pages: 1036680
readmem: read_diskdump()
crash: pv_init_ops exists: ARCH_PVOPS
VMCOREINFO: NUMBER(phys_base): 0 -> 0
gdb ../linux/vmlinux
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<
http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
GETBUF(288 -> 0)
GETBUF(1500 -> 1)
FREEBUF(1)
FREEBUF(0)
<readmem: ffffffff81e3da50, KVADDR, "page_offset_base", 8, (FOE),
d83568>
<read_diskdump: addr: ffffffff81e3da50 paddr: 1e3da50 cnt: 8>
read_diskdump: PAGE_EXCLUDED: paddr/pfn: 1e3da50/1e3d
crash: page excluded: kernel virtual address: ffffffff81e3da50 type:
"page_offset_base"
--
Sincerely,
Cao jin
OK, so you seem to be describing two issues:
(1) With KASLR enabled, the vmcore cannot be read by either
Fedora's 7.2.0-1 or upstream 7.2.0++.
(2) With KASLR disabled, the vmcore can be read by Fedora's
7.2.0-1, but cannot be read by 7.2.0++,
When you run crash on the live system, what happens when:
(1) KASLR is enabled, running Fedora's 7.2.0-1
(2) KASLR is enabled, running upstream 7.2.0++
(2) KASLR is disabled, running Fedora's 7.2.0-1
(3) KASLR is disabled, running upstream 7.2.0++
I don't see anything unusual in the output above, presuming the
the phys_base of 0 is correct. When you run crash on the live
system, what is the output of "help -m | grep phys_base"?
I need to compare the crash -d8 output above with the same
crash -d8 output from the working session using the same
KASLR disabled vmcore, showing at least up until the point that
it does the readmem of "page_offset_base" above?
Also, if you can make the 2 vmcores available, please send me
the details of how to download them in a private email to
anderson(a)redhat.com or anderson(a)prospeed.net. If you have
modified/rebuilt the Fedora kernel, I would also need a copy
of the vmlinux file as well.
We are on Thanksgiving holiday until next Monday, so I won't
be able to spend any time looking at this issue until then.
Dave