2:57 p.m.
So, the key will be to find a difference between 2.6.10 and
2.6.11's symbol contents.
Hi Badari,
Looking at what appears to be Andi's wholesale patch set that
made the VM changes, I think it may be possible to use
"boot_vmalloc_pgt" as a qualifier, because it went away with
the new scheme. Also, "vmalloc_fault" was added, but since
it's a static function, its only caller may have inlined it
in the new kernel.
Also, given the VM changes, I'm still amazed that the virtual
to physical translation of vmalloc and user page addresses
still works. Are you sure that "vtop" on vmalloc and user
space addresses works correctly?
To verify user-space address translation is working, you
should be able to do something like this. Run crash in a
live session, and look at the very beginning of its address
space, and read the first few bytes:
crash> set
PID: 8052
COMMAND: "crash"
TASK: 10018e1f7f0 [THREAD_INFO: 100189e4000]
CPU: 3
STATE: TASK_RUNNING (ACTIVE)
crash> vm
PID: 8052 TASK: 10018e1f7f0 CPU: 3 COMMAND: "crash"
MM PGD RSS TOTAL_VM
1002fc08040 10018286000 63376k 121176k
VMA START END FLAGS FILE
1002b8269f8 400000 788000 1875 /usr/bin/crash
1002bb5d3c8 888000 8ab000 101873 /usr/bin/crash
100288d9408 8ab000 8f8000 100077
100302a0688 9aa000 9c2000 101873 /usr/bin/crash
10028f49b98 9c2000 1b50000 100077
10028f49358 2a95556000 2a95558000 100073
1002edad688 2a95584000 2a95587000 100073
...
crash> rd -u 0x400000
400000: 00010102464c457f .ELF....
crash>
You should see the first bytes of the executable's ELF header,
as verified by the "ELF" string there.
To verify module virtual addresses translation, try disassembling
a module text address, say some ext3 function, and verifying that
it makes sense?
Thanks,
Dave
3:51 p.m.
New subject: [PATCH} crash 4.0-2.8 fixes to make it work onx86_64 for 2.6.14
On Wed, 2005-11-02 at 15:57 -0500, Dave Anderson wrote:
> So, the key will be to find a difference between 2.6.10 and
> 2.6.11's symbol contents.
Hi Badari,
Looking at what appears to be Andi's wholesale patch set that
made the VM changes, I think it may be possible to use
"boot_vmalloc_pgt" as a qualifier, because it went away with
the new scheme. Also, "vmalloc_fault" was added, but since
it's a static function, its only caller may have inlined it
in the new kernel.
Also, given the VM changes, I'm still amazed that the virtual
to physical translation of vmalloc and user page addresses
still works. Are you sure that "vtop" on vmalloc and user
space addresses works correctly?
To verify user-space address translation is working, you
should be able to do something like this. Run crash in a
live session, and look at the very beginning of its address
space, and read the first few bytes:
crash> set
PID: 8052
COMMAND: "crash"
TASK: 10018e1f7f0 [THREAD_INFO: 100189e4000]
CPU: 3
STATE: TASK_RUNNING (ACTIVE)
crash> vm
PID: 8052 TASK: 10018e1f7f0 CPU: 3 COMMAND: "crash"
MM PGD RSS TOTAL_VM
1002fc08040 10018286000 63376k 121176k
VMA START END FLAGS FILE
1002b8269f8 400000 788000 1875 /usr/bin/crash
1002bb5d3c8 888000 8ab000 101873 /usr/bin/crash
100288d9408 8ab000 8f8000 100077
100302a0688 9aa000 9c2000 101873 /usr/bin/crash
10028f49b98 9c2000 1b50000 100077
10028f49358 2a95556000 2a95558000 100073
1002edad688 2a95584000 2a95587000 100073
...
crash> rd -u 0x400000
400000: 00010102464c457f .ELF....
crash>
Yuck. User virtual is screwed up :(crash> set
PID: 5253
COMMAND: "crash"
TASK: 101269b9730 [THREAD_INFO: 10112fbe000]
CPU: 0
STATE: TASK_RUNNING (ACTIVE)
crash> vm
PID: 5253 TASK: 101269b9730 CPU: 0 COMMAND: "crash"
MM PGD RSS TOTAL_VM
1011cc0b6c0 10114fd3000 126372k 121900k
VMA START END FLAGS FILE
101270d7c08 400000 78b000
1875 /root/crash-4.0-2.8.new/crash
10127966aa8 88b000 8ad000
101873 /root/crash-4.0-2.8.new/crash
10037e58268 8ad000 1a67000 100073
10037e58688 2a95556000 2a95558000 100073
10009f7c148 2a95574000 2a95577000 100073
1012731b6c8 2a95577000 2a983b9000 71 /usr/lib/locale/locale-
archive
...
crash> rd -u 0x400000
rd: invalid user virtual address: 400000 type: "64-bit UVADDR"
Ideas on why ?
You should see the first bytes of the executable's ELF header,
as verified by the "ELF" string there.
To verify module virtual addresses translation, try disassembling
a module text address, say some ext3 function, and verifying that
it makes sense?
vmalloc space seems to be fine:
Thanks, crash> mod
MODULE NAME SIZE
OBJECT FILE
ffffffff88012600 dm_mod 70872 (not loaded) [CONFIG_KALLSYMS]
ffffffff8805ef80 ipv6 312832 (not loaded) [CONFIG_KALLSYMS]
ffffffff8806e200 parport 47244 (not loaded) [CONFIG_KALLSYMS]
ffffffff88073800 lp 17232 (not loaded) [CONFIG_KALLSYMS]
ffffffff8807fd00 parport_pc 33896 (not loaded) [CONFIG_KALLSYMS]
ffffffff8808ab00 usbserial 39280 (not loaded) [CONFIG_KALLSYMS]
ffffffff8808e900 hw_random 7968 (not loaded) [CONFIG_KALLSYMS]
ffffffff88098f80 uhci_hcd 38304 (not loaded) [CONFIG_KALLSYMS]
ffffffff880a4600 ehci_hcd 39944 (not loaded) [CONFIG_KALLSYMS]
ffffffff880acd80 i2c_core 29568 (not loaded) [CONFIG_KALLSYMS]
ffffffff880b1700 i2c_i801 11540 (not loaded) [CONFIG_KALLSYMS]
ffffffff880b6000 joydev 13952 (not loaded) [CONFIG_KALLSYMS]
ffffffff880bb000 edd 13984 (not loaded) [CONFIG_KALLSYMS]
crash> dis ip6_dst_lookup
0xffffffff88017c30 <ip6_dst_lookup>: push %rbp
0xffffffff88017c31 <ip6_dst_lookup+1>: mov %rsp,%rbp
0xffffffff88017c34 <ip6_dst_lookup+4>: sub $0x40,%rsp
0xffffffff88017c38 <ip6_dst_lookup+8>: mov %r12,0xffffffffffffffe0(%
rbp)
0xffffffff88017c3c <ip6_dst_lookup+12>: xor %r12d,%r12d
0xffffffff88017c3f <ip6_dst_lookup+15>: test %rdi,%rdi
0xffffffff88017c42 <ip6_dst_lookup+18>: mov %r13,0xffffffffffffffe8(%
rbp)
0xffffffff88017c46 <ip6_dst_lookup+22>: mov %r14,0xfffffffffffffff0(%
rbp)
0xffffffff88017c4a <ip6_dst_lookup+26>: mov %r15,0xfffffffffffffff8(%
rbp)
0xffffffff88017c4e <ip6_dst_lookup+30>: mov %rbx,0xffffffffffffffd8(%
rbp)
0xffffffff88017c52 <ip6_dst_lookup+34>: mov %rdi,%r13
0xffffffff88017c55 <ip6_dst_lookup+37>: mov %rsi,%r14
0xffffffff88017c58 <ip6_dst_lookup+40>: mov %rdx,%r15
0xffffffff88017c5b <ip6_dst_lookup+43>: movq $0x0,(%rsi)
0xffffffff88017c62 <ip6_dst_lookup+50>: je 0xffffffff88017e45
0xffffffff88017c68 <ip6_dst_lookup+56>: mov 0x230(%rdi),%rax
0xffffffff88017c6f <ip6_dst_lookup+63>: lea 0x88(%rdi),%r12
0xffffffff88017c76 <ip6_dst_lookup+70>: mov %r12,%rdi
0xffffffff88017c79 <ip6_dst_lookup+73>: mov %rax,0xffffffffffffffd0(%
rbp)
0xffffffff88017c7d <ip6_dst_lookup+77>: mov 0x4c(%rax),%edx
0xffffffff88017c80 <ip6_dst_lookup+80>: mov %edx,0xffffffffffffffcc(%
rbp)
0xffffffff88017c83 <ip6_dst_lookup+83>: callq 0xffffffff80404620
<_read_lock>
0xffffffff88017c88 <ip6_dst_lookup+88>: mov 0x70(%r13),%rbx
Dave
--
Crash-utility mailing list
Crash-utility(a)redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
4:17 p.m.
New subject: [PATCH} crash 4.0-2.8 fixes to make it workonx86_64 for 2.6.14
Badari Pulavarty wrote:
On Wed, 2005-11-02 at 15:57 -0500, Dave Anderson wrote:
> > So, the key will be to find a difference between 2.6.10 and
> > 2.6.11's symbol contents.
>
> Hi Badari,
>
> Looking at what appears to be Andi's wholesale patch set that
> made the VM changes, I think it may be possible to use
> "boot_vmalloc_pgt" as a qualifier, because it went away with
> the new scheme. Also, "vmalloc_fault" was added, but since
> it's a static function, its only caller may have inlined it
> in the new kernel.
>
> Also, given the VM changes, I'm still amazed that the virtual
> to physical translation of vmalloc and user page addresses
> still works. Are you sure that "vtop" on vmalloc and user
> space addresses works correctly?
>
> To verify user-space address translation is working, you
> should be able to do something like this. Run crash in a
> live session, and look at the very beginning of its address
> space, and read the first few bytes:
>
> crash> set
> PID: 8052
> COMMAND: "crash"
> TASK: 10018e1f7f0 [THREAD_INFO: 100189e4000]
> CPU: 3
> STATE: TASK_RUNNING (ACTIVE)
> crash> vm
> PID: 8052 TASK: 10018e1f7f0 CPU: 3 COMMAND: "crash"
> MM PGD RSS TOTAL_VM
> 1002fc08040 10018286000 63376k 121176k
> VMA START END FLAGS FILE
> 1002b8269f8 400000 788000 1875 /usr/bin/crash
> 1002bb5d3c8 888000 8ab000 101873 /usr/bin/crash
> 100288d9408 8ab000 8f8000 100077
> 100302a0688 9aa000 9c2000 101873 /usr/bin/crash
> 10028f49b98 9c2000 1b50000 100077
> 10028f49358 2a95556000 2a95558000 100073
> 1002edad688 2a95584000 2a95587000 100073
> ...
>
> crash> rd -u 0x400000
> 400000: 00010102464c457f .ELF....
> crash>
Yuck. User virtual is screwed up :(crash> set
PID: 5253
COMMAND: "crash"
TASK: 101269b9730 [THREAD_INFO: 10112fbe000]
CPU: 0
STATE: TASK_RUNNING (ACTIVE)
crash> vm
PID: 5253 TASK: 101269b9730 CPU: 0 COMMAND: "crash"
MM PGD RSS TOTAL_VM
1011cc0b6c0 10114fd3000 126372k 121900k
VMA START END FLAGS FILE
101270d7c08 400000 78b000
1875 /root/crash-4.0-2.8.new/crash
10127966aa8 88b000 8ad000
101873 /root/crash-4.0-2.8.new/crash
10037e58268 8ad000 1a67000 100073
10037e58688 2a95556000 2a95558000 100073
10009f7c148 2a95574000 2a95577000 100073
1012731b6c8 2a95577000 2a983b9000 71 /usr/lib/locale/locale-
archive
...
crash> rd -u 0x400000
rd: invalid user virtual address: 400000 type: "64-bit UVADDR"
Ideas on why ?
Not really -- I would have thought it would failed much later
in the x86_64_uvtop() function when translating the virtual
address to a pte.
But the message above only comes if the call to IS_UVADDR()
call fails in readmem():
/*
* Screen out any error conditions.
*/
switch (memtype)
{
case UVADDR:
if (!CURRENT_CONTEXT()) {
if (PRINT_ERROR_MESSAGE)
error(INFO, "no current user process\n");
goto readmem_error;
}
if (!IS_UVADDR(addr, CURRENT_CONTEXT())) {
if (PRINT_ERROR_MESSAGE)
error(INFO, INVALID_UVADDR, addr, type);
goto readmem_error;
}
break;
but IS_UVADDR() would translate to a simple call to x86_64_is_uvaddr():
static int
x86_64_is_uvaddr(ulong addr, struct task_context *tc)
{
return (addr < USERSPACE_TOP);
}
...so that make little sense to me -- although it should be easy
enough to debug.
>
> You should see the first bytes of the executable's ELF header,
> as verified by the "ELF" string there.
>
> To verify module virtual addresses translation, try disassembling
> a module text address, say some ext3 function, and verifying that
> it makes sense?
>
vmalloc space seems to be fine:
> Thanks, crash> mod
MODULE NAME SIZE OBJECT FILE
ffffffff88012600 dm_mod 70872 (not loaded) [CONFIG_KALLSYMS]
ffffffff8805ef80 ipv6 312832 (not loaded) [CONFIG_KALLSYMS]
ffffffff8806e200 parport 47244 (not loaded) [CONFIG_KALLSYMS]
ffffffff88073800 lp 17232 (not loaded) [CONFIG_KALLSYMS]
ffffffff8807fd00 parport_pc 33896 (not loaded) [CONFIG_KALLSYMS]
ffffffff8808ab00 usbserial 39280 (not loaded) [CONFIG_KALLSYMS]
ffffffff8808e900 hw_random 7968 (not loaded) [CONFIG_KALLSYMS]
ffffffff88098f80 uhci_hcd 38304 (not loaded) [CONFIG_KALLSYMS]
ffffffff880a4600 ehci_hcd 39944 (not loaded) [CONFIG_KALLSYMS]
ffffffff880acd80 i2c_core 29568 (not loaded) [CONFIG_KALLSYMS]
ffffffff880b1700 i2c_i801 11540 (not loaded) [CONFIG_KALLSYMS]
ffffffff880b6000 joydev 13952 (not loaded) [CONFIG_KALLSYMS]
ffffffff880bb000 edd 13984 (not loaded) [CONFIG_KALLSYMS]
crash> dis ip6_dst_lookup
0xffffffff88017c30 <ip6_dst_lookup>: push %rbp
0xffffffff88017c31 <ip6_dst_lookup+1>: mov %rsp,%rbp
0xffffffff88017c34 <ip6_dst_lookup+4>: sub $0x40,%rsp
0xffffffff88017c38 <ip6_dst_lookup+8>: mov %r12,0xffffffffffffffe0(%
rbp)
0xffffffff88017c3c <ip6_dst_lookup+12>: xor %r12d,%r12d
0xffffffff88017c3f <ip6_dst_lookup+15>: test %rdi,%rdi
0xffffffff88017c42 <ip6_dst_lookup+18>: mov %r13,0xffffffffffffffe8(%
rbp)
0xffffffff88017c46 <ip6_dst_lookup+22>: mov %r14,0xfffffffffffffff0(%
rbp)
0xffffffff88017c4a <ip6_dst_lookup+26>: mov %r15,0xfffffffffffffff8(%
rbp)
0xffffffff88017c4e <ip6_dst_lookup+30>: mov %rbx,0xffffffffffffffd8(%
rbp)
0xffffffff88017c52 <ip6_dst_lookup+34>: mov %rdi,%r13
0xffffffff88017c55 <ip6_dst_lookup+37>: mov %rsi,%r14
0xffffffff88017c58 <ip6_dst_lookup+40>: mov %rdx,%r15
0xffffffff88017c5b <ip6_dst_lookup+43>: movq $0x0,(%rsi)
0xffffffff88017c62 <ip6_dst_lookup+50>: je 0xffffffff88017e45
0xffffffff88017c68 <ip6_dst_lookup+56>: mov 0x230(%rdi),%rax
0xffffffff88017c6f <ip6_dst_lookup+63>: lea 0x88(%rdi),%r12
0xffffffff88017c76 <ip6_dst_lookup+70>: mov %r12,%rdi
0xffffffff88017c79 <ip6_dst_lookup+73>: mov %rax,0xffffffffffffffd0(%
rbp)
0xffffffff88017c7d <ip6_dst_lookup+77>: mov 0x4c(%rax),%edx
0xffffffff88017c80 <ip6_dst_lookup+80>: mov %edx,0xffffffffffffffcc(%
rbp)
0xffffffff88017c83 <ip6_dst_lookup+83>: callq 0xffffffff80404620
<_read_lock>
0xffffffff88017c88 <ip6_dst_lookup+88>: mov 0x70(%r13),%rbx
Hmmm, OK if that makes sense to you. My 2.6.9 version of
ip6_dst_lookup() doesn't have a call to _read_lock(), but
maybe it's changed...
But the fact that you've got a module list means that the
linked list of modules -- which is linked by a list_head in
the module data structures themselves -- means that their
vmalloc addresses must be resolving OK.
Dave
6893
days inactive
6893
days old
devel@lists.crash-utility.osci.io
2 comments
2 participants
participants (2)
-
Badari Pulavarty -
Dave Anderson