12:02 p.m.
Hello,
I have a kernel panic that prints a backtrace, but no kernel
dump. The lines in the backtrace has the usual format:
[<addr1>] ? func1+num1/num2 [module1]
I understand that num1 is the address offset from the beginning of func1.
What is num2?
I tried to narrow down the location in func1() by doing the following steps:
loaded file1.o file into gdb and issued a "disassemble func1".
The disassembled version of func1, the lines pertaining to function calls
in func1() has the following format:
...............callq num3 <func1+num4> <====
And NOT the following format I am used to:
.................callq <addr2> <func2>
(func2 is a function called from within func1)
My question is related to line marked with <====
- Looking closely as the values of num3 and num4, the instruction seems to
point to a location somewhere in func1 itself, and not the called function-
func2. I must be reading the instruction wrong? How does one interpret the
"calls" instruction.
- I understand I can't get something like addr2 in the line marked with
<==== as the object file is not linked to the kernel. However, is there
any way or tools I can use so the function name shows up in the the line
<====. That would make it easier for me to understand the disassembled
code.
Using gdb on the kernel module (*.ko) did not make a difference in the
disassemble output.
I apologize for my cryptic question and for posting this question here as
this is not related to crash, however, I felt the audience of this mailing
list might be able to help.
Thank you,
Ahmed.
2:35 p.m.
----- Original Message -----
Hello,
I have a kernel panic that prints a backtrace, but no kernel dump.
The lines in the backtrace has the usual format:
[<addr1>] ? func1+num1/num2 [module1]
I understand that num1 is the address offset from the beginning of
func1. What is num2?
The num2 value is the size in bytes of the whole function.
I tried to narrow down the location in func1() by doing the following
steps:
loaded file1.o file into gdb and issued a "disassemble func1".
The disassembled version of func1, the lines pertaining to function
calls in func1() has the following format:
...............callq num3 <func1+num4> <====
And NOT the following format I am used to:
.................callq <addr2> <func2>
(func2 is a function called from within func1)
My question is related to line marked with <====
- Looking closely as the values of num3 and num4, the instruction
seems to point to a location somewhere in func1 itself, and not the
called function- func2. I must be reading the instruction wrong? How
does one interpret the "calls" instruction.
No, you are reading the instruction correctly. (unresolved)
- I understand I can't get something like addr2 in the line
marked
with <==== as the object file is not linked to the kernel. However,
is there any way or tools I can use so the function name shows up in
the the line <====. That would make it easier for me to understand
the disassembled code.
Using gdb on the kernel module (*.ko) did not make a difference in
the disassemble output.
You might try the "list" command to show "what's around" a
particular
address.
Taking this simple example, if you want to know what function is being
called from nfs_cache_destroy+9:
(gdb) disass nfs_cache_destroy
Dump of assembler code for function nfs_cache_destroy:
0x0000000000016750 <+0>: callq 0x16755 <nfs_cache_destroy+5>
0x0000000000016755 <+5>: push %rbp
0x0000000000016756 <+6>: mov %rsp,%rbp
0x0000000000016759 <+9>: callq 0x1675e <nfs_cache_destroy+14>
0x000000000001675e <+14>: pop %rbp
0x000000000001675f <+15>: retq
End of assembler dump.
(gdb)
Pass the address that is performing the callq to the list
command like this:
(gdb) list *0x0000000000016759
0x16759 is in nfs_cache_destroy (fs/nfs/cache_lib.c:164).
159 sunrpc_init_cache_detail(cd);
160 }
161
162 void nfs_cache_destroy(struct cache_detail *cd)
163 {
164 sunrpc_destroy_cache_detail(cd);
165 }
(gdb)
And it's obvious in this case that it's calling sunrpc_destroy_cache_detail().
Dave
4:05 p.m.
Another way to determine what's being called is to dump
the relocation information. Again, in the nfs_cache_destroy()
function below, there are two callq instructions, one at
the very beginning of the function at 0x0000000000016750,
and the other one at 0x0000000000016759:
(gdb) disass nfs_cache_destroy
Dump of assembler code for function nfs_cache_destroy:
0x0000000000016750 <+0>: callq 0x16755 <nfs_cache_destroy+5>
0x0000000000016755 <+5>: push %rbp
0x0000000000016756 <+6>: mov %rsp,%rbp
0x0000000000016759 <+9>: callq 0x1675e <nfs_cache_destroy+14>
0x000000000001675e <+14>: pop %rbp
0x000000000001675f <+15>: retq
End of assembler dump.
(gdb)
The callq instructions consist of a one-byte "e8" opcode
followed by 4 bytes of relative displacement, so in the
example above, those 4 bytes would be located at addresses
0x0000000000016751 and 0x000000000001675a respectively.
(i.e., the callq site plus 1)
So then if you dump the relocation information from the
object file, you can see that __fentry__() is called at the
beginning of the function (generated by the ftrace facility,
and appearing at the beginning of all functions if it's
configured), but more importangly, sunrpc_destroy_cache_detail()
is the target of the second callq:
$ objdump --reloc nfs.ko
... [ cut ] ...
0000000000016741 R_X86_64_PC32 __fentry__-0x0000000000000004
000000000001674a R_X86_64_PC32 sunrpc_init_cache_detail-0x0000000000000004
0000000000016751 R_X86_64_PC32 __fentry__-0x0000000000000004
000000000001675a R_X86_64_PC32 sunrpc_destroy_cache_detail-0x0000000000000004
0000000000016761 R_X86_64_PC32 __fentry__-0x0000000000000004
0000000000016769 R_X86_64_32S .data+0x0000000000000720
...
which is what the list command showed:
(gdb) list *0x0000000000016759
0x16759 is in nfs_cache_destroy (fs/nfs/cache_lib.c:164).
159 sunrpc_init_cache_detail(cd);
160 }
161
162 void nfs_cache_destroy(struct cache_detail *cd)
163 {
164 sunrpc_destroy_cache_detail(cd);
165 }
(gdb)
The ftrace-generated call to __fentry__() will never be shown in
the text listing.
Dave
6:11 a.m.
Fantastic. I liked how you used "objdump -r" to get the function call
name. I can do that on a system that does not have source code
(specifially code related my loaded module) on it to narrow down the
location of the panic. I found the adding the "-d" in addition very
helpful, "objdump -d -r <*.ko>"
Thank you very much for taking time to respond to my questions in such
detail, much appreciated.
--Ahmed.
On Thu, Feb 14, 2013 at 2:05 PM, Dave Anderson <anderson(a)redhat.com> wrote:
Another way to determine what's being called is to dump
the relocation information. Again, in the nfs_cache_destroy()
function below, there are two callq instructions, one at
the very beginning of the function at 0x0000000000016750,
and the other one at 0x0000000000016759:
(gdb) disass nfs_cache_destroy
Dump of assembler code for function nfs_cache_destroy:
0x0000000000016750 <+0>: callq 0x16755 <nfs_cache_destroy+5>
0x0000000000016755 <+5>: push %rbp
0x0000000000016756 <+6>: mov %rsp,%rbp
0x0000000000016759 <+9>: callq 0x1675e <nfs_cache_destroy+14>
0x000000000001675e <+14>: pop %rbp
0x000000000001675f <+15>: retq
End of assembler dump.
(gdb)
The callq instructions consist of a one-byte "e8" opcode
followed by 4 bytes of relative displacement, so in the
example above, those 4 bytes would be located at addresses
0x0000000000016751 and 0x000000000001675a respectively.
(i.e., the callq site plus 1)
So then if you dump the relocation information from the
object file, you can see that __fentry__() is called at the
beginning of the function (generated by the ftrace facility,
and appearing at the beginning of all functions if it's
configured), but more importangly, sunrpc_destroy_cache_detail()
is the target of the second callq:
$ objdump --reloc nfs.ko
... [ cut ] ...
0000000000016741 R_X86_64_PC32 __fentry__-0x0000000000000004
000000000001674a R_X86_64_PC32
sunrpc_init_cache_detail-0x0000000000000004
0000000000016751 R_X86_64_PC32 __fentry__-0x0000000000000004
000000000001675a R_X86_64_PC32
sunrpc_destroy_cache_detail-0x0000000000000004
0000000000016761 R_X86_64_PC32 __fentry__-0x0000000000000004
0000000000016769 R_X86_64_32S .data+0x0000000000000720
...
which is what the list command showed:
(gdb) list *0x0000000000016759
0x16759 is in nfs_cache_destroy (fs/nfs/cache_lib.c:164).
159 sunrpc_init_cache_detail(cd);
160 }
161
162 void nfs_cache_destroy(struct cache_detail *cd)
163 {
164 sunrpc_destroy_cache_detail(cd);
165 }
(gdb)
The ftrace-generated call to __fentry__() will never be shown in
the text listing.
Dave
--
Crash-utility mailing list
Crash-utility(a)redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
4229
days inactive
4232
days old
devel@lists.crash-utility.osci.io
3 comments
2 participants
participants (2)
-
Ahmed Al-Mehdi -
Dave Anderson