Re: [Crash-utility] [ANNOUNCE][RFC] gcore extension module: user-mode process core dump
Hello Dave,
I've downloaded vmcore and vmlinux. Thanks a lot.
From: Dave Anderson <anderson(a)redhat.com>
Subject: Re: [ANNOUNCE][RFC] gcore extension module: user-mode process core dump
Date: Mon, 31 Jan 2011 08:51:04 -0500 (EST)
Hello Daisuke,
The test dump can be found here:
http://people.redhat.com/anderson/.gcore_test_dump
One important thing to note -- the dumpfile was taken with
the "snap.so" extension module while running live. It
selects the "crash" process that was doing the live dump
as the panic task. So when you do a backtrace on it, it
looks like this:
crash> bt
PID: 2080 TASK: ffff880079ed2480 CPU: 0 COMMAND: "crash"
#0 [ffff88007a615b08] schedule at ffffffff81480533
#1 [ffff88007a615bf0] rcu_read_unlock at ffffffff811edfd3
#2 [ffff88007a615c00] avc_has_perm_noaudit at ffffffff811eea76
#3 [ffff88007a615c90] avc_has_perm at ffffffff811eeae3
#4 [ffff88007a615d10] inode_has_perm at ffffffff811f2815
#5 [ffff88007a615de8] might_fault at ffffffff810f22ec
#6 [ffff88007a615e80] might_fault at ffffffff810f2335
#7 [ffff88007a615eb0] crash_read at ffffffffa004f103 [crash]
#8 [ffff88007a615f00] vfs_read at ffffffff8112115b
#9 [ffff88007a615f40] sys_read at ffffffff81121278
#10 [ffff88007a615f80] system_call_fastpath at ffffffff81009c72
RIP: 000000333a0d41b0 RSP: 00007fffac23a7f0 RFLAGS: 00000206
RAX: 0000000000000000 RBX: ffffffff81009c72 RCX: 0000000000000000
RDX: 0000000000001000 RSI: 0000000000ca5440 RDI: 0000000000000004
RBP: 0000000000000004 R8: 000000007a615000 R9: 0000000000000006
R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000ca5440
R13: 0000000000001000 R14: 0000000000001000 R15: 000000007a615000
ORIG_RAX: 0000000000000000 CS: 0033 SS: 002b
crash>
Now, when using "snap.so" to create a dumpfile, all of the "active"
backtraces are not legitimate, because they were *running* when their
kernel stacks were being read. So, for example, the "snap.so" code
was running -- doing a read() -- when the "crash" stack was read. But
since it had not panicked, there were no legitimate starting RIP/RSP
values to use for starting points for the backtrace. So frame #'s 0
through #7 above should not be accepted as "real". But I presume that
starting from frame #7 , would be correct.
Ah, there's no method to obtain active registers...
If register values is unavailable for an active task, gcore is now
treating it in the same way as for a sleeping task. This means gcore
chooses RIP and RSP the scheduler saved last time.
Applying this story to here, it seems to me that the old logic of
resotre_frame_pointer() can surely result in non-termination around
the frame #7, since at the point old stack frame is switching to new
one and a list of frame pointers is not connected.
I'll now verify this story by looking at vmcore you gave me.
Thanks,
HATAYAMA Daisuke