Per On Tue, Mar 5, 2013 at 6:30 PM, Per Fransson <per.fransson.ml(a)gmail.com&gt; wrote: > Hi, > > On Tue, Mar 5, 2013 at 9:22 AM, Lei Wen <adrian.wenl(a)gmail.com&gt; wrote: >> Per, >> >> On Tue, Mar 5, 2013 at 3:25 PM, Per Fransson <per.fransson.ml(a)gmail.com&gt; wrote: >>> Hi Lei, >>> >>> On Tue, Mar 5, 2013 at 1:22 AM, Lei Wen <adrian.wenl(a)gmail.com&gt; wrote: >>>> Hi Per, >>>> >>>> >>>> On Tue, Mar 5, 2013 at 4:38 AM, Per Fransson <per.fransson.ml(a)gmail.com&gt; >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> On Mon, Mar 4, 2013 at 8:49 PM, Mika Westerberg <mika.westerberg(a)iki.fi&gt; >>>>> wrote: >>>>> > On Mon, Mar 04, 2013 at 10:20:42AM +0800, Lei Wen wrote: >>>>> >> I met "dis" command not correct issue when use the crash, any idea? >>>>> >> For built-in "dis" command in crash: >>>>> >> crash> dis task_rq_lock >>>>> >> 0xc015a2d8 <task_rq_lock>: rscsgt r0, sp, r3, lsl #14 >>>>> >> 0xc015a2dc <task_rq_lock+4>: mrcgt 8, 7, r0, cr2, cr13, {5} >>>>> >> 0xc015a2e0 <task_rq_lock+8>: mcrvc 8, 4, r3, cr13, cr3, {6} >>>>> >> 0xc015a2e4 <task_rq_lock+12>: lslsvc r3, r10, r8 >>>>> >> 0xc015a2e8 <task_rq_lock+16>: bl 0xc049fe34 >>>>> >> <__ip_route_output_key+220> >>>>> > >>>>> > Looks weird. >>>>> > >>>>> > What is the kernel version? Does the 'dis' command work for other >>>>> > functions? >>>>> > >>>>> >>>>> You could do a check on one of the instructions - the 'bl' comes to >>>>> mind. Not sure, but I believe it should amount to: >>>>> >>>>> 0xeb000000 | (((0xc049fe34-0xc015a2f0) >> 2) & 0x00ffffff) >>>>> >>>>> i.e. >>>>> >>>>> 0xeb0d16d1 >>>>> >>>>> Is that what you get with >>>>> >>>>> crash> rd 0xc015a2e8 >>>>> >>>>> ? >>>>> >>>>> If not, try a >>>>> >>>>> crash> search 0xeb0d16d1 >>>>> >>>>> and see if it turns up somewhere else. >>>> >>>> >>>> >>>> >>>> Yes, it is that value. >>>> >>>> crash> rd 0xc015a2e8 >>>> >>>> c015a2e8: eb0d16d1 .... >>>> >>>> >>>> >>>> While in gdb, show the same address's value, it would be: >>>> >>>> (gdb) x 0xc015a2e8 >>>> >>>> 0xc015a2e8 <task_rq_lock+16>: 0xe1a05000 >>>> >>>> >>>> >>>> Why it didn't match with each other? Any idea? >>>> >>>> >>> >>> Nope, no idea. When you're using gdb, do you feed it the coredump as >>> well, or just the vmlinux? if you get the same strange result with >>> gdb+vmlinux+coredump, I think you should try to match some known data, >>> e.g. the 'bl' and see if the contents are offset somehow. Try the gdb >>> search command on 0xeb0d16d1. >> >> Your hypothesis is correct. >> When feed dump image with vmlinux to the gdb, I get exactly same result >> as crash... >> >> How to use the search command in gdb? >> > > Oh, it's 'find' in gdb. To look for 0xeb0d16d1 in the virtual interval > 0xc0000000--0xe0000000 you would: > > (gdb) find /w 0xc0000000, +0x20000000, 0xeb0d16d1 > > or use your favorite hex editor. > > If the dump isn't offset, it could be overwrites. Here is the result: (gdb) find /w 0xc0000000, +0x20000000, 0xeb0d16d1 0xc015a2e8 <task_rq_lock+16> 1 pattern found. Seems its output is exactly as crash's rd command...

Thanks, Lei

Per, On Tue, Mar 5, 2013 at 8:26 PM, Per Fransson <per.fransson.ml(a)gmail.com&gt; wrote: > On Tue, Mar 5, 2013 at 12:58 PM, Lei Wen <adrian.wenl(a)gmail.com&gt; wrote: >> Per >> >> On Tue, Mar 5, 2013 at 6:30 PM, Per Fransson <per.fransson.ml(a)gmail.com&gt; wrote: >>> Hi, >>> >>> On Tue, Mar 5, 2013 at 9:22 AM, Lei Wen <adrian.wenl(a)gmail.com&gt; wrote: >>>> Per, >>>> >>>> On Tue, Mar 5, 2013 at 3:25 PM, Per Fransson <per.fransson.ml(a)gmail.com&gt; wrote: >>>>> Hi Lei, >>>>> >>>>> On Tue, Mar 5, 2013 at 1:22 AM, Lei Wen <adrian.wenl(a)gmail.com&gt; wrote: >>>>>> Hi Per, >>>>>> >>>>>> >>>>>> On Tue, Mar 5, 2013 at 4:38 AM, Per Fransson <per.fransson.ml(a)gmail.com&gt; >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> On Mon, Mar 4, 2013 at 8:49 PM, Mika Westerberg <mika.westerberg(a)iki.fi&gt; >>>>>>> wrote: >>>>>>> > On Mon, Mar 04, 2013 at 10:20:42AM +0800, Lei Wen wrote: >>>>>>> >> I met "dis" command not correct issue when use the crash, any idea? >>>>>>> >> For built-in "dis" command in crash: >>>>>>> >> crash> dis task_rq_lock >>>>>>> >> 0xc015a2d8 <task_rq_lock>: rscsgt r0, sp, r3, lsl #14 >>>>>>> >> 0xc015a2dc <task_rq_lock+4>: mrcgt 8, 7, r0, cr2, cr13, {5} >>>>>>> >> 0xc015a2e0 <task_rq_lock+8>: mcrvc 8, 4, r3, cr13, cr3, {6} >>>>>>> >> 0xc015a2e4 <task_rq_lock+12>: lslsvc r3, r10, r8 >>>>>>> >> 0xc015a2e8 <task_rq_lock+16>: bl 0xc049fe34 >>>>>>> >> <__ip_route_output_key+220> >>>>>>> > >>>>>>> > Looks weird. >>>>>>> > >>>>>>> > What is the kernel version? Does the 'dis' command work for other >>>>>>> > functions? >>>>>>> > >>>>>>> >>>>>>> You could do a check on one of the instructions - the 'bl' comes to >>>>>>> mind. Not sure, but I believe it should amount to: >>>>>>> >>>>>>> 0xeb000000 | (((0xc049fe34-0xc015a2f0) >> 2) & 0x00ffffff) >>>>>>> >>>>>>> i.e. >>>>>>> >>>>>>> 0xeb0d16d1 >>>>>>> >>>>>>> Is that what you get with >>>>>>> >>>>>>> crash> rd 0xc015a2e8 >>>>>>> >>>>>>> ? >>>>>>> >>>>>>> If not, try a >>>>>>> >>>>>>> crash> search 0xeb0d16d1 >>>>>>> >>>>>>> and see if it turns up somewhere else. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Yes, it is that value. >>>>>> >>>>>> crash> rd 0xc015a2e8 >>>>>> >>>>>> c015a2e8: eb0d16d1 .... >>>>>> >>>>>> >>>>>> >>>>>> While in gdb, show the same address's value, it would be: >>>>>> >>>>>> (gdb) x 0xc015a2e8 >>>>>> >>>>>> 0xc015a2e8 <task_rq_lock+16>: 0xe1a05000 >>>>>> >>>>>> >>>>>> >>>>>> Why it didn't match with each other? Any idea? >>>>>> >>>>>> >>>>> >>>>> Nope, no idea. When you're using gdb, do you feed it the coredump as >>>>> well, or just the vmlinux? if you get the same strange result with >>>>> gdb+vmlinux+coredump, I think you should try to match some known data, >>>>> e.g. the 'bl' and see if the contents are offset somehow. Try the gdb >>>>> search command on 0xeb0d16d1. >>>> >>>> Your hypothesis is correct. >>>> When feed dump image with vmlinux to the gdb, I get exactly same result >>>> as crash... >>>> >>>> How to use the search command in gdb? >>>> >>> >>> Oh, it's 'find' in gdb. To look for 0xeb0d16d1 in the virtual interval >>> 0xc0000000--0xe0000000 you would: >>> >>> (gdb) find /w 0xc0000000, +0x20000000, 0xeb0d16d1 >>> >>> or use your favorite hex editor. >>> >>> If the dump isn't offset, it could be overwrites. >> >> Here is the result: >> (gdb) find /w 0xc0000000, +0x20000000, 0xeb0d16d1 >> 0xc015a2e8 <task_rq_lock+16> >> 1 pattern found. >> >> Seems its output is exactly as crash's rd command... >> > > Oh, right. We already knew that. You need to search for the one you > expected to be there, i.e. 0xe1a05000, but that might be a common > pattern. Maybe grab something completely different, e.g. the > linux_banner. Do a strings -a -t x <dump file> | grep "Linux version" > and see if it's at the offset you expected it to be. > Seems with "gdb vmlinux <dump file>" method, original pattern cannot be matched as exactly place. From previous disasembly code start from <task_rq_lock+16>, the following should be: 0xe1a05000 0xe1a08001 0xe59f4058 But with the three word searching in the whole memory, this kind of pattern cannot be found. (gdb) find /w 0xc0000000, +0x20000000, 0xe1a05000,0xe1a08001,0xe59f4058 Pattern not found. Weird, isn't it?

Thanks, Lei

Per, [snip..] > > What do you get when you do: > > $ nm vmlinux | grep linux_banner > > $ readelf -l <dumpfile> > > $ strings -a -t x <dumpfile> | grep "Linux version" Here is the result: nm vmlinux | grep linux_banner c05bc06b R linux_banner readelf -l 2013-01-18 Elf file type is CORE (Core file) Entry point 0x0 There are 2 program headers, starting at offset 52 Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align NOTE 0x000074 0x00000000 0x00000000 0x00000 0x00000 0 LOAD 0x000074 0xc0000000 0x00000000 0x20000000 0x20000000 RWE 0 strings -a -t x 2013-01-18 | grep "Linux version" 5bc0df Linux version 3.4.5+ (1@2) (gcc version 4.6.x-google 20120106 (prerelease) (GCC) ) #1 SMP PREEMPT Tue Jan 15 15:51:59 CST 2013

Thanks, Lei