2:47 p.m.
Hello,
I have a question about trying to decipher the values of parameters passed
to a function in "crash". I understand "bt -f" and "bt -F"
prints the
stack data, but I am having a hard time deciphering the stack to get
access to the values of parameters passed to a function. I understand the
compiler could have optimized the parameters into registers. If so, is
there a compiler option to turn it off? If not, is my only option to
browse the object file to see what registers are used? Is there any
extensions (experimental or hack) that I can add to crash to display
function parameter values.
In the following crash, I am trying to understand the value of the function
parameters - e, buf, len. Any help or pointers would be very appreciated.
c code:
int
doread(EB *e, uchar *buf, int len)
{
return queueread(e->rq, buf, len);
}
From crash:
crash> bt
PID: 2725 TASK: ffff880353c17500 CPU: 1 COMMAND: "bash"
#0 [ffff88036276d540] machine_kexec at ffffffff8103281b
#1 [ffff88036276d5a0] crash_kexec at ffffffff810ba662
#2 [ffff88036276d670] oops_end at ffffffff81501290
#3 [ffff88036276d6a0] no_context at ffffffff81043bab
#4 [ffff88036276d6f0] __bad_area_nosemaphore at ffffffff81043e35
#5 [ffff88036276d740] bad_area at ffffffff81043f5e
#6 [ffff88036276d770] __do_page_fault at ffffffff81044710
#7 [ffff88036276d890] do_page_fault at ffffffff8150326e
#8 [ffff88036276d8c0] page_fault at ffffffff81500625
[exception RIP: queueread+32]
RIP: ffffffffa03e4b70 RSP: ffff88036276d978 RFLAGS: 00010286
RAX: 00000000000005ae RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000001000 RSI: ffff8803613c0020 RDI: 0000000000000000
RBP: ffff88036276d9a8 R8: 0000000000000d44 R9: 0000000050c91762
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8803613c0020
R13: ffff880341780290 R14: 00000000000237f8 R15: ffff880341780020
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#9 [ffff88036276d9b0] elread at ffffffffa03ecd25 [ethdrv]
#10 [ffff88036276d9c0] elechosrv at ffffffffa03eef4d [ethdrv]
#11 [ffff88036276da00] edwritectl at ffffffffa03dff0e [ethdrv]
#12 [ffff88036276de40] writectl at ffffffffa03f028b [ethdrv]
#13 [ffff88036276de60] proc_file_write at ffffffff811e6e44
#14 [ffff88036276dea0] proc_reg_write at ffffffff811e0abe
#15 [ffff88036276def0] vfs_write at ffffffff8117b068
#16 [ffff88036276df30] sys_write at ffffffff8117ba81
#17 [ffff88036276df80] system_call_fastpath at ffffffff8100b0f2
RIP: 0000003a29ada3c0 RSP: 00007fffe92f1a60 RFLAGS: 00010202
RAX: 0000000000000001 RBX: ffffffff8100b0f2 RCX: 0000000000000065
RDX: 000000000000000a RSI: 00007fab2c281000 RDI: 0000000000000001
RBP: 00007fab2c281000 R8: 000000000000000a R9: 00007fab2c272700
R10: 00000000fffffff7 R11: 0000000000000246 R12: 000000000000000a
R13: 0000003a29d8c780 R14: 000000000000000a R15: 0000000000e75130
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
crash> bt -f
..............
#8 [ffff88036276d8c0] page_fault at ffffffff81500625
[exception RIP: queueread+32]
RIP: ffffffffa03e4b70 RSP: ffff88036276d978 RFLAGS: 00010286
RAX: 00000000000005ae RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000001000 RSI: ffff8803613c0020 RDI: 0000000000000000
RBP: ffff88036276d9a8 R8: 0000000000000d44 R9: 0000000050c91762
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8803613c0020
R13: ffff880341780290 R14: 00000000000237f8 R15: ffff880341780020
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
ffff88036276d8c8: ffff880341780020 00000000000237f8
ffff88036276d8d8: ffff880341780290 ffff8803613c0020
ffff88036276d8e8: ffff88036276d9a8 0000000000000000
ffff88036276d8f8: 0000000000000000 0000000000000000
ffff88036276d908: 0000000050c91762 0000000000000d44
ffff88036276d918: 00000000000005ae 0000000000000000
ffff88036276d928: 0000000000001000 ffff8803613c0020
ffff88036276d938: 0000000000000000 ffffffffffffffff
ffff88036276d948: ffffffffa03e4b70 0000000000000010
ffff88036276d958: 0000000000010286 ffff88036276d978
ffff88036276d968: 0000000000000018 ffffffffa03ed062
ffff88036276d978: 000005ae613c01ab ffff880341780290
ffff88036276d988: 00000000000005ae ffff8803613c0020
ffff88036276d998: ffff880341780290 00000000000237f8
ffff88036276d9a8: ffff88036276d9b8 ffffffffa03ecd25
#9 [ffff88036276d9b0] elread at ffffffffa03ecd25 [ethdrv]
ffff88036276d9b8: ffff88036276d9f8 ffffffffa03eef4d
...................
crash> bt -F
#8 [ffff88036276d8c0] page_fault at ffffffff81500625
[exception RIP: queueread+32]
RIP: ffffffffa03e4b70 RSP: ffff88036276d978 RFLAGS: 00010286
RAX: 00000000000005ae RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000001000 RSI: ffff8803613c0020 RDI: 0000000000000000
RBP: ffff88036276d9a8 R8: 0000000000000d44 R9: 0000000050c91762
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8803613c0020
R13: ffff880341780290 R14: 00000000000237f8 R15: ffff880341780020
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
ffff88036276d8c8: [size-131072] 00000000000237f8
ffff88036276d8d8: [size-131072] [size-8192]
ffff88036276d8e8: ffff88036276d9a8 0000000000000000
ffff88036276d8f8: 0000000000000000 0000000000000000
ffff88036276d908: 0000000050c91762 0000000000000d44
ffff88036276d918: 00000000000005ae 0000000000000000
ffff88036276d928: 0000000000001000 [size-8192]
ffff88036276d938: 0000000000000000 ffffffffffffffff
ffff88036276d948: queueread+32 0000000000000010
ffff88036276d958: 0000000000010286 ffff88036276d978
ffff88036276d968: 0000000000000018 elwrite+98
ffff88036276d978: 000005ae613c01ab [size-131072]
ffff88036276d988: 00000000000005ae [size-8192]
ffff88036276d998: [size-131072] 00000000000237f8
ffff88036276d9a8: ffff88036276d9b8 elread+21
#9 [ffff88036276d9b0] elread at ffffffffa03ecd25 [ethdrv]
ffff88036276d9b8: ffff88036276d9f8 elechosrv+173
........................
Regards,
Ahmed.
4:38 p.m.
----- Original Message -----
Hello,
I have a question about trying to decipher the values of parameters
passed to a function in "crash". I understand "bt -f" and "bt
-F"
prints the stack data, but I am having a hard time deciphering the
stack to get access to the values of parameters passed to a
function. I understand the compiler could have optimized the
parameters into registers. If so, is there a compiler option to turn
it off?
None that I'm aware of...
If not, is my only option to browse the object file to see
what registers are used? Is there any extensions (experimental or
hack) that I can add to crash to display function parameter values.
Every so often there is a crash "project" that people have undertaken
to try to do what you're asking, but it always seems to silently tail
off into oblivion.
If it were easy to do for x86_64 with it's highly-optimized register
usage for parameter passing, it would have been implemented.
Anyway, typically you would take the return addresses shown at the
end of each "bt" frame, and do a "dis -rl <address>" and note
where
the argument registers came from.
In the following crash, I am trying to understand the value of the
function parameters - e, buf, len. Any help or pointers would be
very appreciated.
c code:
int
doread(EB *e, uchar *buf, int len)
{
return queueread(e->rq, buf, len);
}
From crash:
crash> bt
PID: 2725 TASK: ffff880353c17500 CPU: 1 COMMAND: "bash"
#0 [ffff88036276d540] machine_kexec at ffffffff8103281b
#1 [ffff88036276d5a0] crash_kexec at ffffffff810ba662
#2 [ffff88036276d670] oops_end at ffffffff81501290
#3 [ffff88036276d6a0] no_context at ffffffff81043bab
#4 [ffff88036276d6f0] __bad_area_nosemaphore at ffffffff81043e35
#5 [ffff88036276d740] bad_area at ffffffff81043f5e
#6 [ffff88036276d770] __do_page_fault at ffffffff81044710
#7 [ffff88036276d890] do_page_fault at ffffffff8150326e
#8 [ffff88036276d8c0] page_fault at ffffffff81500625
[exception RIP: queueread+32]
RIP: ffffffffa03e4b70 RSP: ffff88036276d978 RFLAGS: 00010286
RAX: 00000000000005ae RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000001000 RSI: ffff8803613c0020 RDI: 0000000000000000
RBP: ffff88036276d9a8 R8: 0000000000000d44 R9: 0000000050c91762
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8803613c0020
R13: ffff880341780290 R14: 00000000000237f8 R15: ffff880341780020
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#9 [ffff88036276d9b0] elread at ffffffffa03ecd25 [ethdrv]
#10 [ffff88036276d9c0] elechosrv at ffffffffa03eef4d [ethdrv]
#11 [ffff88036276da00] edwritectl at ffffffffa03dff0e [ethdrv]
#12 [ffff88036276de40] writectl at ffffffffa03f028b [ethdrv]
#13 [ffff88036276de60] proc_file_write at ffffffff811e6e44
#14 [ffff88036276dea0] proc_reg_write at ffffffff811e0abe
#15 [ffff88036276def0] vfs_write at ffffffff8117b068
#16 [ffff88036276df30] sys_write at ffffffff8117ba81
#17 [ffff88036276df80] system_call_fastpath at ffffffff8100b0f2
RIP: 0000003a29ada3c0 RSP: 00007fffe92f1a60 RFLAGS: 00010202
RAX: 0000000000000001 RBX: ffffffff8100b0f2 RCX: 0000000000000065
RDX: 000000000000000a RSI: 00007fab2c281000 RDI: 0000000000000001
RBP: 00007fab2c281000 R8: 000000000000000a R9: 00007fab2c272700
R10: 00000000fffffff7 R11: 0000000000000246 R12: 000000000000000a
R13: 0000003a29d8c780 R14: 000000000000000a R15: 0000000000e75130
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
The arguments passed to queueread() from elread() are probably
still sitting in exception frame registers, although queueread()
may have moved them around or destroyed the original values.
I would do:
crash> dis -rl ffffffffa03ecd25
and note that the arguments were probably stored in rdi, rsi, and
whatever register is used for the 3rd arg. Then disassemble queueread()
until the exception RIP and note whether the registers are still
intact, moved, or perhaps destroyed, i.e.,
crash> dis -rl queueread+32
< check out what happened to the argument registers >
The "l" of the -rl is not necessary, but I always do it by default
for line number info.
Another thing you might try if your kdump is still in ELF format
(i.e., not subsequently compressed by makedumpfile) is this:
$ gdb vmlinux vmcore
...
(gdb) bt
Sometimes that may yield some parameter information.
Dave
FWIW, here are several threads from the past:
[RFC] [Crash-utility] Patch to use gdb's bt in crash
https://www.redhat.com/archives/crash-utility/2006-August/msg00036.html
[Crash-utility] Re: [RFC] Crash patch for DWARF CFI based unwind support
https://www.redhat.com/archives/crash-utility/2006-November/msg00031.html
Note: this required Jan Beulich's x86-specific unwind kernel patch that
was subsequently pulled from the upstream kernel.
[Crash-utility] enhance bt command
https://www.redhat.com/archives/crash-utility/2008-March/msg00001.html
Crash-utility] backtrace - how do I get local and formal parametes
https://www.redhat.com/archives/crash-utility/2008-March/msg00049.html
[Crash-utility] Determining values of arguments and local variable in x86_64 crash
dumps
https://www.redhat.com/archives/crash-utility/2008-July/msg00000.html
[Crash-utility] crash on x86_64: how to dump arg registers?
https://www.redhat.com/archives/crash-utility/2008-September/msg00004.html
[Crash-utility] [RFC]: Feature to display local arguments and variables
https://www.redhat.com/archives/crash-utility/2009-April/msg00004.html
(see April 2009 archives for follow-ups)
[Crash-utility] [PATCH 0/3] Display local variables & function parameters from
stack frames
https://www.redhat.com/archives/crash-utility/2009-May/msg00012.html
4213
days inactive
4213
days old
devel@lists.crash-utility.osci.io
1 comments
2 participants
participants (2)
-
Ahmed Al-Mehdi -
Dave Anderson