9:15 a.m.
- Introduced support for xendumps of fully-virtualized x86 kernels
taken while running on an x86 Xen host (32-bit on 32-bit host).
(anderson(a)redhat.com)
- Introduced support for xendumps of fully-virtualized x86 kernels
taken while running on an x86_64 Xen host (32-bit on 64-bit host).
(anderson(a)redhat.com)
- Introduced support for xendumps of fully-virtualized x86_64 kernels.
(anderson(a)redhat.com)
- Introduced support for xendumps of para-virtualized ia64 kernels.
It should be noted that currently the ia64 Xen kernel does not
lay down a switch_stack for the panic task, so only raw "bt -t"
backtraces can be done on the panic task. (anderson(a)redhat.com)
- Introduced support for "xm save" dumpfiles of para-virtualized ia64
kernels, which use a completely different format than that used for
x86 and x86_64. (anderson(a)redhat.com)
- Additional support for the current kexec/kdump patch for Xen:
1) Merged second round of "xencrash" patches, which allows a crash
session to be alternatively brought up against the xen-syms
binary instead of a vmlinux kernel. (oda(a)valinux.co.jp)
2) Using the xencrash feature above, the pfn_to_mfn_list_list value
of any guest domain that was running when the dom0 or hypervisor
crashed can be determined; that pfn value can in turn be used
as an argument to a new "--p2m_mfn [pfn]" crash command line
option. That will allow a crash session to be run against any
guest domain. Therefore, with a single dom0/hypervisor vmcore,
the following types of crash sessions may be initiated:
$ crash vmlinux-dom0 vmcore
$ crash xen-syms vmcore
$ crash --p2m_mfn [pfn] vmlinux-guest-#1 vmcore
$ crash --p2m_mfn [pfn] vmlinux-guest-#2 vmcore
$ ...
(anderson(a)redhat.com)
3) Fixed "help -n" debug output to properly display the contents
of the new XEN_ELFNOTE_CRASH_INFO and XEN_ELFNOTE_CRASH_REGS
ELF note types. (anderson(a)redhat.com)
- Turn off the LKCD dumpfile-access "spinner" when "crash -s" is
used.
(castor.fu(a)3pardata.com)
- Update to MODULES_IN_CWD code segment so that it will work on 2.6
kernels where modules end with ".ko". This requires that kernel.c
is compiled with -DMODULES_IN_CWD. (castor.fu(a)3pardata.com)
- Support LKCD "map" files in lieu of standard System.map files.
Without this patch, crash would fail with an error message of the
sort: "crash: map.4: not a supported file format". (bwalle(a)suse.de)
- The ia64 PR_UNALIGN_NOPRINT and PR_FPEMU_NOPRINT prctl commands have
been moved earlier in time, in order to prevent "unaligned access"
messages when accessing ELF header contents. (anderson(a)redhat.com)
- The dlopen() call used by the "extensions" facility has been changed
to use the RTLD_GLOBAL flag, so that symbols from an extension object
will be visable to subsequently loaded modules. (asid(a)hp.com)
Download from: http://people.redhat.com/anderson
8:04 p.m.
On Wed, Dec 20, 2006 at 10:15:32AM -0500, Dave Anderson wrote:
- Introduced support for xendumps of para-virtualized ia64 kernels.
It should be noted that currently the ia64 Xen kernel does not
lay down a switch_stack for the panic task, so only raw "bt -t"
backtraces can be done on the panic task. (anderson(a)redhat.com)
Hi Dave.
The current "xm dump-core" on ia64 loses some registers infomation
which is saved on xen register stack.
e.g. r33, ... aren't saved in domU xendump file.
Probably ia64 specific code would be necessarry for it.
This will be addressed as post-3.0.4 effort and the format will be changed.
--
yamahata
7:54 a.m.
Isaku Yamahata wrote:
On Wed, Dec 20, 2006 at 10:15:32AM -0500, Dave Anderson wrote:
> - Introduced support for xendumps of para-virtualized ia64 kernels.
> It should be noted that currently the ia64 Xen kernel does not
> lay down a switch_stack for the panic task, so only raw "bt -t"
> backtraces can be done on the panic task. (anderson(a)redhat.com)
Hi Dave.
The current "xm dump-core" on ia64 loses some registers infomation
which is saved on xen register stack.
e.g. r33, ... aren't saved in domU xendump file.
Probably ia64 specific code would be necessarry for it.
This will be addressed as post-3.0.4 effort and the format will be changed.
--
yamahata
I'm not sure exactly what the ramifications are of an ia64 "xm dump-core"
on a paravirtualized kernel. It would seem to depend upon what, if anything,
was "active" at the time.
My reference to the switch_stack above was for an ia64 kernel that panicked
on its own account; the test dump I used was killed with a write to
/proc/sysrq-trigger:
.crash> bt
PID: 1554 TASK: e000000000988000 CPU: 0 COMMAND: "bash"
bt: xendump: switch_stack possibly not saved -- try "bt -t"
#0 [BSP:e000000000988f00] schedule at a0000001005e0420
crash>
It uses the stale information from the last time it called schedule(),
so the backtrace fails.
Using "bt -t" walks the process stack for kernel return addresses,
and the "reverse" BSP information just above the task_struct shows
the path taken:
crash> bt -t
PID: 1554 TASK: e000000000988000 CPU: 0 COMMAND: "bash"
START: schedule at a0000001005e0420
[e000000000989238] xen_trace_syscall at a000000100065020
[e000000000989288] sys_write at a000000100155d30
[e0000000009892b8] vfs_write at a0000001001551e0
[e000000000989308] write_sysrq_trigger at a0000001001e3250
[e000000000989320] __handle_sysrq at a00000010039bca0
[e000000000989380] sysrq_handle_crashdump at a00000010039c460
[e0000000009893e8] do_wait at a00000010008b080
[e000000000989438] schedule_timeout at a0000001005e22a0
[e000000000989450] ext3_lookup at a0000002001eb7d0
[e000000000989460] cleanup_module at a000000200202a10
[e000000000989470] ext3_find_entry at a0000002001e77c0
[e0000000009894b0] __wait_on_buffer at a00000010015b580
[e0000000009894c0] ll_rw_block at a00000010015c2b0
[e000000000989500] out_of_line_wait_on_bit at a0000001005e2940
[e000000000989520] __wait_on_bit at a0000001005e27d0
[e000000000989540] sync_buffer at a00000010015b7c0
[e000000000989558] io_schedule at a0000001005e2170
[e000000000989588] __delayacct_blkio_start at a0000001000fa4b0
[e000000000989608] io_schedule at a0000001005e21a0
[e000000000989630] __do_IRQ at a0000001000f2450
[e000000000989660] do_softirq at a000000100093100
[e000000000989698] blkif_int at a000000200152070
[e000000000989710] end_that_request_first at a00000010027ae30
[e000000000989748] __end_that_request_first at a00000010027a460
[e000000000989778] bio_endio at a0000001001622b0
[e00000000098fca0] schedule at a0000001005e0420
[e00000000098fd10] vhpt_miss at a000000100000002
[e00000000098fd60] vhpt_miss at a000000100000002
[e00000000098fdc8] dummycon_dummy at a0000001002dd380
[e00000000098fdd0] vhpt_miss at a000000100000003
crash>
Well, it at least shows it going as far as sysrq_handle_crashdump(),
and any further addresses of function calls were never pushed into
the BSP. (?)
Anyway, the problem is that the ia64 shutdown path in a para-virtualized ia64
kernel does not lay down a switch_stack -- as is done by the netdump,
diskdump and kdump facilities. Without a switch_stack register dump,
a backtrace is impossible.
It's a simple thing to do -- at some point during the shutdown
path, presumably xen_panic_event(), the panicking process would
need to make a call to the unw_init_running() function, which lays
down a switch_stack on the kernel stack, and then continues on to
the next function in the shutdown path. For example, the kdump
facility for ia64 does this:
[ system crashes ]
crash_kexec()
machine_kexec()
...
The ia64 version of machine_kexec() does this:
void machine_kexec(struct kimage *image)
{
unw_init_running(ia64_machine_kexec, image);
for(;;);
}
The call to unw_init_running() never returns, but rather
it lays down a switch_stack on the kernel stack, and then
calls the ia64_machine_kexec() function:
extern void *efi_get_pal_addr(void);
static void ia64_machine_kexec(struct unw_frame_info *info, void *arg)
{
struct kimage *image = arg;
relocate_new_kernel_t rnk;
void *pal_addr = efi_get_pal_addr();
unsigned long code_addr = (unsigned
long)page_address(image->control_code_page);
unsigned long vector;
int ii;
if (image->type == KEXEC_TYPE_CRASH) {
crash_save_this_cpu();
current->thread.ksp = (__u64)info->sw - 16;
}
... (continue shutdown path)
The address of the switch stack address is found in the unw_frame_info
structure passed in, and gets stored in the current->thread.ksp of
the panicking task. With that simple procedure, the crash utility
will then have all that it needs to do a backtrace of the panicking task.
Since the para-virtualized ia64 kernel shuts down when panic()
calls atomic_notifier_call_chain(), which in turn goes through the
panic_notifier list -- which leads to the ia64 version of xen_panic_event():
static int
xen_panic_event(struct notifier_block *this, unsigned long event, void *ptr)
{
HYPERVISOR_shutdown(SHUTDOWN_crash);
/* we're never actually going to get here... */
return NOTIFY_DONE;
}
The ia64 would need to "jump through the hoop" of a call to
unw_init_running() before it calls HYPERVISOR_shutdown()
Thanks,
Dave
9:19 p.m.
On Thu, Dec 21, 2006 at 08:54:23AM -0500, Dave Anderson wrote:
Isaku Yamahata wrote:
The current "xm dump-core" on ia64 loses some registers infomation
which is saved on xen register stack.
e.g. r33, ... aren't saved in domU xendump file.
Probably ia64 specific code would be necessarry for it.
This will be addressed as post-3.0.4 effort and the format will be changed.
I'm not sure exactly what the ramifications are of an ia64 "xm dump-core"
on a paravirtualized kernel. It would seem to depend upon what, if anything,
was "active" at the time.
The problems here is no ramification.
"xm dump-core" command on ia64 or domU panic create something
with x86-dump code, however no one has verfied the resulted file
deeply yet.
Anyway in the near feature, xendump format would be revised into elf format
and its implemenation would be rewritten.
The ia64 would need to "jump through the hoop" of a call
to
unw_init_running() before it calls HYPERVISOR_shutdown()
Is the attached patch what you want?
--
yamahata
9:06 a.m.
Isaku Yamahata wrote:
> The ia64 would need to "jump through the hoop" of a
call to
> unw_init_running() before it calls HYPERVISOR_shutdown()
Is the attached patch what you want?
Not quite -- it leaves out the most important part. Your
patch will cause a switch_stack register dump to be laid down,
but the address of the switch_stack needs to be stored in the
task_struct->thread.ksp of the panicking task:
diff -r 114c8cc94466 -r 262eb4ba1212
linux-2.6-xen-sparse/arch/ia64/kernel/setup.c
--- a/linux-2.6-xen-sparse/arch/ia64/kernel/setup.c Thu Dec 21 17:47:31 2006 +0900
+++ b/linux-2.6-xen-sparse/arch/ia64/kernel/setup.c Fri Dec 22 12:03:18 2006 +0900
@@ -78,10 +78,17 @@ EXPORT_SYMBOL(__per_cpu_offset);
#endif
#ifdef CONFIG_XEN
+static void
+xen_panic_hypercall(struct unw_frame_info *info, void *arg)
+{
+ HYPERVISOR_shutdown(SHUTDOWN_crash);
+ /* we're never actually going to get here... */
+}
+
static int
xen_panic_event(struct notifier_block *this, unsigned long event, void *ptr)
{
- HYPERVISOR_shutdown(SHUTDOWN_crash);
+ unw_init_running(xen_panic_hypercall, NULL);
/* we're never actually going to get here... */
return NOTIFY_DONE;
}
So the xen_panic_hypercall() should look like this:
static void
xen_panic_hypercall(struct unw_frame_info *info, void *arg)
{
current->thread.ksp = (__u64)info->sw - 16;
HYPERVISOR_shutdown(SHUTDOWN_crash);
/* we're never actually going to get here... */
}
Doing the store to the current->thread.ksp as above mimics
the behavior of all of the other tasks on the system.
Thanks,
Dave
7:33 p.m.
On Fri, Dec 22, 2006 at 10:06:01AM -0500, Dave Anderson wrote:
Not quite -- it leaves out the most important part. Your
patch will cause a switch_stack register dump to be laid down,
but the address of the switch_stack needs to be stored in the
task_struct->thread.ksp of the panicking task:
Ouch. attached the updated patch.
# HG changeset patch
# User yamahata(a)valinux.co.jp
# Date 1167010270 -32400
# Node ID 1f2ea340120cfc05caabb4b0c5cb2c24d8c6431b
# Parent 80c5b5914b795b28249c8f7251ecd24b53186339
As Dave requested, call unw_init_running() and save ksp before SHUTDOWN_crash.
This makes crash command happy.
PATCHNAME: unw_init_running_before_shutdown_crash
Signed-off-by: Dave Anderson <anderson(a)redhat.com>
Signed-off-by: Isaku Yamahata <yamahata(a)valinux.co.jp>
diff -r 80c5b5914b79 -r 1f2ea340120c linux-2.6-xen-sparse/arch/ia64/kernel/setup.c
--- a/linux-2.6-xen-sparse/arch/ia64/kernel/setup.c Wed Dec 20 14:55:02 2006 -0700
+++ b/linux-2.6-xen-sparse/arch/ia64/kernel/setup.c Mon Dec 25 10:31:10 2006 +0900
@@ -77,10 +77,18 @@ EXPORT_SYMBOL(__per_cpu_offset);
#endif
#ifdef CONFIG_XEN
+static void
+xen_panic_hypercall(struct unw_frame_info *info, void *arg)
+{
+ current->thread.ksp = (__u64)info->sw - 16;
+ HYPERVISOR_shutdown(SHUTDOWN_crash);
+ /* we're never actually going to get here... */
+}
+
static int
xen_panic_event(struct notifier_block *this, unsigned long event, void *ptr)
{
- HYPERVISOR_shutdown(SHUTDOWN_crash);
+ unw_init_running(xen_panic_hypercall, NULL);
/* we're never actually going to get here... */
return NOTIFY_DONE;
}
--
yamahata
9 p.m.
Hi, Dave
The belows are great information for xen-ia64.
We are happy because of starting support it.
- Introduced support for xendumps of para-virtualized ia64 kernels.
It should be noted that currently the ia64 Xen kernel does not
lay down a switch_stack for the panic task, so only raw "bt -t"
backtraces can be done on the panic task. (anderson(a)redhat.com)
- Introduced support for "xm save" dumpfiles of para-virtualized ia64
kernels, which use a completely different format than that used for
x86 and x86_64. (anderson(a)redhat.com)
Can we use bt with "xm save" dumpfiles of para-virtualized ia64?
Is "xm save" dumpfile better for crash-utils than "xendump" file?
Best Regards,
Akio Takebe
7:58 a.m.
Akio Takebe wrote:
Hi, Dave
The belows are great information for xen-ia64.
We are happy because of starting support it.
>- Introduced support for xendumps of para-virtualized ia64 kernels.
> It should be noted that currently the ia64 Xen kernel does not
> lay down a switch_stack for the panic task, so only raw "bt -t"
> backtraces can be done on the panic task. (anderson(a)redhat.com)
>
>- Introduced support for "xm save" dumpfiles of para-virtualized ia64
> kernels, which use a completely different format than that used for
> x86 and x86_64. (anderson(a)redhat.com)
>
Can we use bt with "xm save" dumpfiles of para-virtualized ia64?
Is "xm save" dumpfile better for crash-utils than "xendump" file?
Best Regards,
Akio Takebe
Here's what my test "xm save" dump session looks like:
# crash /usr/dumps/xensave/*
crash 4.0-3.15
Copyright (C) 2002, 2003, 2004, 2005, 2006 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "ia64-unknown-linux-gnu"...
KERNEL: /usr/dumps/xensave/vmlinux-2.6.18-1.2839.el5.xi5xen
DUMPFILE: /usr/dumps/xensave/xen-ia64-guest-xm-save.vmcore
CPUS: 1
DATE: Fri Dec 8 09:16:34 2006
UPTIME: 00:01:43
LOAD AVERAGE: 0.53, 0.30, 0.11
TASKS: 62
NODENAME: dhcp83-121.boston.redhat.com
RELEASE: 2.6.18-1.2839.el5.xi5xen
VERSION: #1 SMP Thu Dec 7 10:14:34 EST 2006
MACHINE: ia64 (900 Mhz)
MEMORY: 960 MB
PID: 1523
COMMAND: "suspend"
TASK: e0000000301e0000 [THREAD_INFO: e0000000301e0fc0]
CPU: 0
STATE: TASK_RUNNING (SUSPEND)
crash> ps
PID PPID CPU TASK ST %MEM VSZ RSS COMM
0 0 0 a000000100734000 RU 0.0 0 0 [swapper]
1 0 0 e0000000005e0000 IN 0.2 4448 1728 init
2 1 0 e000000000608000 IN 0.0 0 0 [migration/0]
3 1 0 e000000000610000 IN 0.0 0 0 [ksoftirqd/0]
4 1 0 e000000000618000 IN 0.0 0 0 [watchdog/0]
5 1 0 e000000004c38000 IN 0.0 0 0 [events/0]
6 1 0 e0000000009e8000 IN 0.0 0 0 [khelper]
7 1 0 e0000000056e0000 IN 0.0 0 0 [kthread]
9 7 0 e0000000056c8000 IN 0.0 0 0 [xenwatch]
10 7 0 e0000000056d0000 IN 0.0 0 0 [xenbus]
15 7 0 e000000005608000 IN 0.0 0 0 [kblockd/0]
16 7 0 e000000005630000 IN 0.0 0 0 [kacpid]
26 7 0 e000000004c80000 IN 0.0 0 0 [cqueue/0]
30 7 0 e000000005680000 IN 0.0 0 0 [khubd]
32 7 0 e000000004c90000 IN 0.0 0 0 [kseriod]
83 7 0 e0000000054e8000 IN 0.0 0 0 [pdflush]
84 7 0 e0000000054f0000 IN 0.0 0 0 [pdflush]
85 7 0 e0000000054f8000 IN 0.0 0 0 [kswapd0]
86 7 0 e000000005518000 IN 0.0 0 0 [aio/0]
211 7 0 e000000000a78000 IN 0.0 0 0 [kpsmoused]
231 7 0 e000000000c60000 IN 0.0 0 0 [kmirrord]
236 7 0 e000000000cb0000 IN 0.0 0 0 [ksnapd]
239 7 0 e000000000cd0000 IN 0.0 0 0 [kjournald]
261 7 0 e00000003b090000 IN 0.0 0 0 [kauditd]
289 1 0 e00000000d948000 IN 0.1 5536 1408 udevd
1024 1 0 e000000036140000 IN 0.1 5376 1216 dhclient
1115 1 0 e0000000337f0000 IN 0.2 14960 1600 auditd
1116 1 0 e000000033818000 IN 0.2 14960 1600 auditd
1117 1115 0 e000000033820000 IN 0.9 73056 8960 python
1130 1 0 e000000033908000 IN 0.1 4528 1472 syslogd
1133 1 0 e000000033b78000 IN 0.1 4352 1088 klogd
1163 1 0 e0000000324b8000 IN 0.1 4736 1472 portmap
1183 1 0 e000000032528000 IN 0.2 5088 2080 rpc.statd
1214 1 0 e000000031568000 IN 0.2 12096 1840 rpc.idmapd
1231 1 0 e00000003a810000 IN 0.2 6112 2272 dbus-daemon
1241 1 0 e00000003a7e8000 IN 0.2 5456 1872 hcid
1245 1 0 e000000031ca8000 IN 0.1 4512 1296 sdpd
1267 1 0 e000000031ff0000 IN 0.0 0 0 [krfcommd]
1303 1 0 e000000031ce8000 IN 0.3 15120 2656 pcscd
1317 1 0 e000000030238000 IN 0.1 5408 1216 hidd
1328 1 0 e0000000304e0000 IN 0.3 15120 2656 pcscd
1332 1 0 e000000030518000 IN 0.3 57680 2832 automount
1333 1 0 e000000030520000 IN 0.3 57680 2832 automount
1334 1 0 e000000030528000 IN 0.3 57680 2832 automount
1337 1 0 e0000000306b0000 IN 0.3 57680 2832 automount
1340 1 0 e0000000306b8000 IN 0.3 57680 2832 automount
1350 1 0 e000000030650000 IN 0.1 4336 1344 acpid
1360 1 0 e000000030988000 IN 0.6 74880 5632 cupsd
1373 1 0 e000000030240000 IN 0.3 12224 2768 sshd
1392 1 0 e000000030e10000 IN 0.4 19280 4304 sendmail
1400 1 0 e000000030ff0000 IN 0.4 16944 3744 sendmail
1419 1 0 e00000002f080000 IN 0.1 4448 1248 gpm
1445 1 0 e00000002f100000 IN 0.2 63200 2352 crond
1455 1 0 e00000002f190000 IN 0.1 3408 1392 anacron
1464 1 0 e00000002f240000 IN 0.1 5952 1344 atd
1476 1 0 e00000002ec78000 IN 2.5 106560 24224 yum-updatesd
1486 1 0 e00000002ec08000 IN 0.6 9888 5920 hald
1487 1486 0 e00000002f1c8000 IN 0.3 6832 2752 hald-runner
1493 1487 0 e00000002ef20000 IN 0.2 4400 2032 hald-addon-acpi
1519 1 0 e00000002d1d8000 IN 0.1 4688 944 smartd
1522 1 0 e00000002d380000 IN 0.1 3456 1376 agetty
1523 7 0 e0000000301e0000 RU 0.0 0 0
[suspend]
crash> bt
PID: 1523 TASK: e0000000301e0000 CPU: 0 COMMAND: "suspend"
#0 [BSP:e0000000301e11a8] schedule at a0000001005e0420
#1 [BSP:e0000000301e1168] read_reply at a0000001003ca070
#2 [BSP:e0000000301e1120] xs_talkv at a0000001003ca520
#3 [BSP:e0000000301e10f0] xs_unwatch at a0000001003ca7a0
#4 [BSP:e0000000301e10d0] xs_suspend at a0000001003cb200
#5 [BSP:e0000000301e10b0] xenbus_suspend at a0000001003cddb0
crash>
Dave
10:55 p.m.
Hi Dave,
"crash xen-syms vmcore" works fine. Thank you !
BTW,
I think xendumps of fully-virtualized x86/x86_64 (also ia64) kernel
have a magic number XC_CORE_MAGIC_HVM(0xF00FEBEE) which is defferent
from para-virtualized. Please check.
I tried a xendump of fully-virtualized ia64 kernel
with change magic number check as follows.
---
xendump.h
+#define XC_CORE_MAGIC_HVM 0xF00FEBEE
xendump.c
- if (xcp->xch_magic != XC_CORE_MAGIC)
+ if (xcp->xch_magic != XC_CORE_MAGIC && xcp->xch_magic !=
XC_CORE_MAGIC_HVM)
---
It works well execpt bt problem same as para-virtualized one.
Thanks.
--
Itsuro ODA <oda(a)valinux.co.jp>
8:19 a.m.
Itsuro ODA wrote:
Hi Dave,
"crash xen-syms vmcore" works fine. Thank you !
BTW,
I think xendumps of fully-virtualized x86/x86_64 (also ia64) kernel
have a magic number XC_CORE_MAGIC_HVM(0xF00FEBEE) which is defferent
from para-virtualized. Please check.
I tried a xendump of fully-virtualized ia64 kernel
with change magic number check as follows.
---
xendump.h
+#define XC_CORE_MAGIC_HVM 0xF00FEBEE
xendump.c
- if (xcp->xch_magic != XC_CORE_MAGIC)
+ if (xcp->xch_magic != XC_CORE_MAGIC && xcp->xch_magic !=
XC_CORE_MAGIC_HVM)
---
It works well execpt bt problem same as para-virtualized one.
Thanks.
--
Itsuro ODA <oda(a)valinux.co.jp>
Hi Oda-san,
Ah, OK -- we (Red Hat) are still using an older version of the
xenctrl library from (3.0.3), and it uses the old XC_CORE_MAGIC.
So I didn't see any difference between para- and fully-virtualized
magic numbers; I would guess that the difference is a change in
the library as opposed to para- vs fully-virtualized xendumps?
Here's my fully-virtualized x86_64 dump header:
crash> help -n
...
xc_core:
header:
xch_magic: f00febed (XC_CORE_MAGIC)
xch_nr_vcpus: 1
xch_nr_pages: 68762 (0x10c9a)
xch_ctxt_offset: 24 (0x18)
xch_index_offset: 5192 (0x1448)
xch_pages_offset: 557056 (0x88000)
p2m_frames: 0
p2m_frame_index_list:
crash>
My FV x86 from an x86 host:
crash> help -n
...
xc_core:
header:
xch_magic: f00febed (XC_CORE_MAGIC)
xch_nr_vcpus: 1
xch_nr_pages: 131372 (0x2012c)
xch_ctxt_offset: 24 (0x18)
xch_index_offset: 2824 (0xb08)
xch_pages_offset: 528384 (0x81000)
p2m_frames: 0
p2m_frame_index_list:
crash>
and my FV x86 from an x86_64 host:
crash> help -n
...
xc_core:
header:
xch_magic: f00febed (XC_CORE_MAGIC)
xch_nr_vcpus: 1
xch_nr_pages: 131372 (0x2012c)
xch_ctxt_offset: 24 (0x18)
xch_index_offset: 5192 (0x1448)
xch_pages_offset: 1056768 (0x102000)
p2m_frames: 0
p2m_frame_index_list:
crash>
Anyway, I'm happy that everything still works with the new
magic number -- and whatever else might have changed with it.
Also, I appreciate your verification that crash works with
ia64 fully-virtualized xendumps (with your magic number fix).
I'll update the crash package to support the new magic value.
Thanks again,
Dave
11:54 p.m.
Hi Dave,
- Introduced support for xendumps of para-virtualized ia64 kernels.
It should be noted that currently the ia64 Xen kernel does not
lay down a switch_stack for the panic task, so only raw "bt -t"
backtraces can be done on the panic task. (anderson(a)redhat.com)
page_index may contain INVALID_MFN (0xffffffffffffffffUL).
I think it is better to check it.
Here is a patch for this.
---
--- crash-4.0-3.15-org/xendump.h 2006-12-21 09:57:29.000000000 +0900
+++ crash-4.0-3.15/xendump.h 2006-12-21 14:37:35.000000000 +0900
@@ -113,3 +113,8 @@
uint32_t hypercall_imm; /* Break imm for Xen hypercalls. */
/* #endif */
} xen_domctl_arch_setup_t;
+
+#ifdef IA64
+#define INVALID_MFN (0xffffffffffffffffUL)
+#endif
+
--- crash-4.0-3.15-org/xendump.c 2006-12-21 09:57:29.000000000 +0900
+++ crash-4.0-3.15/xendump.c 2006-12-21 14:42:48.000000000 +0900
@@ -31,6 +31,7 @@
static void xc_core_p2m_create(void);
static ulong xc_core_pfn_to_page_index(ulong);
+static int check_mfn_is_valid(ulong);
/*
* Determine whether a file is a xendump creation, and if TRUE,
@@ -1362,7 +1363,7 @@
off_t offset;
if (xd->flags & XC_CORE_NO_P2MM)
- return pfn;
+ return check_mfn_is_valid(pfn) ? pfn : PFN_NOT_FOUND;
idx = pfn/PFNS_PER_PAGE;
@@ -1405,6 +1406,34 @@
return mfn_idx;
}
+static int
+check_mfn_is_valid(ulong idx)
+{
+ ulong mfn;
+
+ if (idx >= xd->xc_core.header.xch_nr_pages)
+ return FALSE;
+
+ if (lseek(xd->xfd,
+ (off_t)xd->xc_core.header.xch_index_offset + idx * sizeof(ulong),
+ SEEK_SET) == -1) {
+ error(INFO, "cannot lseek to page index\n");
+ return FALSE;
+ }
+
+ if (read(xd->xfd, &mfn, sizeof(ulong)) != sizeof(ulong)) {
+ error(INFO, "cannot read index page %d\n", idx);
+ return FALSE;
+ }
+
+ if (mfn == INVALID_MFN) {
+ error(INFO, "idx: %lx indicates INVALID_MFN\n", idx);
+ return FALSE;
+ } else {
+ return TRUE;
+ }
+}
+
/*
* Store the panic task's stack hooks from where it was found
* in get_active_set_panic_task().
---
Thanks.
--
Itsuro ODA <oda(a)valinux.co.jp>
9 a.m.
Itsuro ODA wrote:
Hi Dave,
> - Introduced support for xendumps of para-virtualized ia64 kernels.
> It should be noted that currently the ia64 Xen kernel does not
> lay down a switch_stack for the panic task, so only raw "bt -t"
> backtraces can be done on the panic task. (anderson(a)redhat.com)
page_index may contain INVALID_MFN (0xffffffffffffffffUL).
I think it is better to check it.
Here is a patch for this.
---
--- crash-4.0-3.15-org/xendump.h 2006-12-21 09:57:29.000000000 +0900
+++ crash-4.0-3.15/xendump.h 2006-12-21 14:37:35.000000000 +0900
@@ -113,3 +113,8 @@
uint32_t hypercall_imm; /* Break imm for Xen hypercalls. */
/* #endif */
} xen_domctl_arch_setup_t;
+
+#ifdef IA64
+#define INVALID_MFN (0xffffffffffffffffUL)
+#endif
+
--- crash-4.0-3.15-org/xendump.c 2006-12-21 09:57:29.000000000 +0900
+++ crash-4.0-3.15/xendump.c 2006-12-21 14:42:48.000000000 +0900
@@ -31,6 +31,7 @@
static void xc_core_p2m_create(void);
static ulong xc_core_pfn_to_page_index(ulong);
+static int check_mfn_is_valid(ulong);
/*
* Determine whether a file is a xendump creation, and if TRUE,
@@ -1362,7 +1363,7 @@
off_t offset;
if (xd->flags & XC_CORE_NO_P2MM)
- return pfn;
+ return check_mfn_is_valid(pfn) ? pfn : PFN_NOT_FOUND;
idx = pfn/PFNS_PER_PAGE;
@@ -1405,6 +1406,34 @@
return mfn_idx;
}
+static int
+check_mfn_is_valid(ulong idx)
+{
+ ulong mfn;
+
+ if (idx >= xd->xc_core.header.xch_nr_pages)
+ return FALSE;
+
+ if (lseek(xd->xfd,
+ (off_t)xd->xc_core.header.xch_index_offset + idx * sizeof(ulong),
+ SEEK_SET) == -1) {
+ error(INFO, "cannot lseek to page index\n");
+ return FALSE;
+ }
+
+ if (read(xd->xfd, &mfn, sizeof(ulong)) != sizeof(ulong)) {
+ error(INFO, "cannot read index page %d\n", idx);
+ return FALSE;
+ }
+
+ if (mfn == INVALID_MFN) {
+ error(INFO, "idx: %lx indicates INVALID_MFN\n", idx);
+ return FALSE;
+ } else {
+ return TRUE;
+ }
+}
+
/*
* Store the panic task's stack hooks from where it was found
* in get_active_set_panic_task().
---
Thanks.
--
Itsuro ODA <oda(a)valinux.co.jp>
Oda-san,
Good catch. I was never reading "bad addresses", so I didn't
see any issue there. Now, after going back and checking it out,
I see that if you attempt to read an uninstantiated pfn, it returns
the page index, and so it seeks to and reads whatever's in the
xendump from that location. Given the way the FV xendump is
laid out, there has to be *something* there in the dumpfile, which
I would have thought would have been sparse file space and
read as all zeroes? But in some dumps, it looks like there's "junk"
data there? Whatever -- it's harmless.
In any case, I agree with you -- it's much better to return a read
error as xc_core_read() expects. I'll just adjust your patch slightly
to handle all architectures.
Thanks,
Dave
2:32 p.m.
Dave Anderson wrote:
Itsuro ODA wrote:
> Hi Dave,
>
> > - Introduced support for xendumps of para-virtualized ia64 kernels.
> > It should be noted that currently the ia64 Xen kernel does not
> > lay down a switch_stack for the panic task, so only raw "bt -t"
> > backtraces can be done on the panic task. (anderson(a)redhat.com)
>
> page_index may contain INVALID_MFN (0xffffffffffffffffUL).
> I think it is better to check it.
> Here is a patch for this.
>
> ---
> --- crash-4.0-3.15-org/xendump.h 2006-12-21 09:57:29.000000000 +0900
> +++ crash-4.0-3.15/xendump.h 2006-12-21 14:37:35.000000000 +0900
> @@ -113,3 +113,8 @@
> uint32_t hypercall_imm; /* Break imm for Xen hypercalls. */
> /* #endif */
> } xen_domctl_arch_setup_t;
> +
> +#ifdef IA64
> +#define INVALID_MFN (0xffffffffffffffffUL)
> +#endif
> +
> --- crash-4.0-3.15-org/xendump.c 2006-12-21 09:57:29.000000000 +0900
> +++ crash-4.0-3.15/xendump.c 2006-12-21 14:42:48.000000000 +0900
> @@ -31,6 +31,7 @@
>
> static void xc_core_p2m_create(void);
> static ulong xc_core_pfn_to_page_index(ulong);
> +static int check_mfn_is_valid(ulong);
>
> /*
> * Determine whether a file is a xendump creation, and if TRUE,
> @@ -1362,7 +1363,7 @@
> off_t offset;
>
> if (xd->flags & XC_CORE_NO_P2MM)
> - return pfn;
> + return check_mfn_is_valid(pfn) ? pfn : PFN_NOT_FOUND;
>
> idx = pfn/PFNS_PER_PAGE;
>
> @@ -1405,6 +1406,34 @@
> return mfn_idx;
> }
>
> +static int
> +check_mfn_is_valid(ulong idx)
> +{
> + ulong mfn;
> +
> + if (idx >= xd->xc_core.header.xch_nr_pages)
> + return FALSE;
> +
> + if (lseek(xd->xfd,
> + (off_t)xd->xc_core.header.xch_index_offset + idx * sizeof(ulong),
> + SEEK_SET) == -1) {
> + error(INFO, "cannot lseek to page index\n");
> + return FALSE;
> + }
> +
> + if (read(xd->xfd, &mfn, sizeof(ulong)) != sizeof(ulong)) {
> + error(INFO, "cannot read index page %d\n", idx);
> + return FALSE;
> + }
> +
> + if (mfn == INVALID_MFN) {
> + error(INFO, "idx: %lx indicates INVALID_MFN\n", idx);
> + return FALSE;
> + } else {
> + return TRUE;
> + }
> +}
> +
> /*
> * Store the panic task's stack hooks from where it was found
> * in get_active_set_panic_task().
> ---
>
> Thanks.
> --
> Itsuro ODA <oda(a)valinux.co.jp>
Oda-san,
Good catch. I was never reading "bad addresses", so I didn't
see any issue there. Now, after going back and checking it out,
I see that if you attempt to read an uninstantiated pfn, it returns
the page index, and so it seeks to and reads whatever's in the
xendump from that location. Given the way the FV xendump is
laid out, there has to be *something* there in the dumpfile, which
I would have thought would have been sparse file space and
read as all zeroes? But in some dumps, it looks like there's "junk"
data there? Whatever -- it's harmless.
In any case, I agree with you -- it's much better to return a read
error as xc_core_read() expects. I'll just adjust your patch slightly
to handle all architectures.
Thanks,
Dave
Oda-san,
Thanks again for the "overnight-QA" and for introducing support
for xendumps of fully-virtualized ia64 kernels! I've posted a new
crash version with both fixes.
I modified your check_mfn_is_valid() function so that it could
be used by all 3 architectures as well as for xendumps of
32-bit kernels running on a 64-bit host:
/*
* In xendumps containing INVALID_MFN markers in the page index,
* return the validity of the pfn.
*/
static int
xc_core_pfn_valid(ulong pfn)
{
ulong mfn;
off_t offset;
if (pfn >= (ulong)xd->xc_core.header.xch_nr_pages)
return FALSE;
offset = (off_t)xd->xc_core.header.xch_index_offset;
if (xd->flags & XC_CORE_64BIT_HOST)
offset += (off_t)(pfn * sizeof(ulonglong));
else
offset += (off_t)(pfn * sizeof(ulong));
/*
* The lseek and read should never fail, so report
* any errors unconditionally.
*/
if (lseek(xd->xfd, offset, SEEK_SET) == -1) {
error(INFO,
"xendump: cannot lseek to page index for pfn %lx\n",
pfn);
return FALSE;
}
if (read(xd->xfd, &mfn, sizeof(ulong)) != sizeof(ulong)) {
error(INFO,
"xendump: cannot read index page for pfn %lx\n",
pfn);
return FALSE;
}
/*
* If it's an invalid mfn, let the caller decide whether
* to display an error message (unless debugging).
*/
if (mfn == INVALID_MFN) {
if (CRASHDEBUG(1))
error(INFO,
"xendump: pfn %lx contains INVALID_MFN\n",
pfn);
return FALSE;
}
return TRUE;
}
Thanks again,
Dave
9:40 p.m.
Hi Dave,
Thank you for your quick work !!
Thanks again for the "overnight-QA" and for introducing
support
for xendumps of fully-virtualized ia64 kernels! I've posted a new
crash version with both fixes.
You are welcome. It is day time in Japan :-)
Thanks.
--
Itsuro ODA <oda(a)valinux.co.jp>
7015
days inactive
7020
days old
devel@lists.crash-utility.osci.io
13 comments
4 participants
participants (4)
-
Akio Takebe -
Dave Anderson -
Isaku Yamahata -
Itsuro ODA