Re: [Crash-utility] Why module's global symbol cannot be displayed in crash? [ARM]

On Sun, Mar 24, 2013 at 02:15:19PM -0400, Dave Anderson wrote: > > > ----- Original Message ----- > > On Fri, Mar 22, 2013 at 09:51:39AM -0400, Dave Anderson wrote: > > > > > > > > > ----- Original Message ----- > > > > On Thu, Mar 21, 2013 at 03:02:54PM -0400, Dave Anderson wrote: > > > > > If for some reason you can't get them, I can make them > > > > > available to > > > > > you. > > > > > And Lei Wen can also give you a sample dumpfile from his > > > > > environment. > > > > > > > > Got them from Luc. > > > > > > > > > > Are you able to access module symbols on ARM dump (the one > > > > > > that Luc provided)? > > > > > > Or is it failing completely? > > > > > > > > > > I *think* so... > > > > > > > > > > This module text disassembly looks right: > > > > > > > > > > crash> dis usbnet_suspend > > > > > 0xbf000ae8 <usbnet_suspend>: push {r3, r4, r5, lr} > > > > > 0xbf000aec <usbnet_suspend+4>: add r0, r0, #32 > > > > > 0xbf000af0 <usbnet_suspend+8>: mov r5, r1 > > > > > 0xbf000af4 <usbnet_suspend+12>: bl 0xc01b8264 > > > > > <dev_get_drvdata> > > > > > 0xbf000af8 <usbnet_suspend+16>: ldrb r3, [r0, #36] ; 0x24 > > > > > 0xbf000afc <usbnet_suspend+20>: mov r4, r0 > > > > > 0xbf000b00 <usbnet_suspend+24>: add r2, r3, #1 > > > > > 0xbf000b04 <usbnet_suspend+28>: cmp r3, #0 > > > > > 0xbf000b08 <usbnet_suspend+32>: strb r2, [r0, #36] ; 0x24 > > > > > 0xbf000b0c <usbnet_suspend+36>: bne 0xbf000bdc > > > > > <usbnet_suspend+244> > > > > > 0xbf000b10 <usbnet_suspend+40>: mrs r3, CPSR > > > > > 0xbf000b14 <usbnet_suspend+44>: orr r3, r3, #128 ; 0x80 > > > > > 0xbf000b18 <usbnet_suspend+48>: msr CPSR_c, r3 > > > > > 0xbf000b1c <usbnet_suspend+52>: mov r0, #1 > > > > > 0xbf000b20 <usbnet_suspend+56>: bl 0xc0015f40 > > > > > <add_preempt_count> > > > > > 0xbf000b24 <usbnet_suspend+60>: ldr r3, [r4, #200] ; 0xc8 > > > > > 0xbf000b28 <usbnet_suspend+64>: cmp r3, #0 > > > > > 0xbf000b2c <usbnet_suspend+68>: beq 0xbf000b70 > > > > > <usbnet_suspend+136> > > > > > 0xbf000b30 <usbnet_suspend+72>: tst r5, #1024 ; 0x400 > > > > > 0xbf000b34 <usbnet_suspend+76>: beq 0xbf000b70 > > > > > <usbnet_suspend+136> > > > > > 0xbf000b38 <usbnet_suspend+80>: mrs r3, CPSR > > > > > ... > > > > > > > > > > This (r) data looks OK: > > > > > > > > > > crash> p smsc95xx_netdev_ops > > > > > smsc95xx_netdev_ops = $8 = { > > > > > ndo_init = 0, > > > > > ndo_uninit = 0, > > > > > ndo_open = 0xbf000514 <usbnet_open>, > > > > > ndo_stop = 0xbf000bec <usbnet_stop>, > > > > > ndo_start_xmit = 0xbf001a60 <usbnet_start_xmit>, > > > > > ndo_select_queue = 0, > > > > > ndo_change_rx_flags = 0, > > > > > ndo_set_rx_mode = 0, > > > > > ndo_set_multicast_list = 0xbf008abc <smsc95xx_set_multicast>, > > > > > ndo_set_mac_address = 0xc025d854 <eth_mac_addr>, > > > > > ndo_validate_addr = 0xc025d6f8 <eth_validate_addr>, > > > > > ndo_do_ioctl = 0xbf00926c <smsc95xx_ioctl>, > > > > > ndo_set_config = 0, > > > > > ndo_change_mtu = 0xbf000de0 <usbnet_change_mtu>, > > > > > ndo_neigh_setup = 0, > > > > > ndo_tx_timeout = 0xbf000d4c <usbnet_tx_timeout>, > > > > > ndo_get_stats64 = 0, > > > > > ndo_get_stats = 0, > > > > > ndo_vlan_rx_add_vid = 0, > > > > > ndo_vlan_rx_kill_vid = 0, > > > > > ndo_set_vf_mac = 0, > > > > > ndo_set_vf_vlan = 0, > > > > > ndo_set_vf_tx_rate = 0, > > > > > ndo_get_vf_config = 0, > > > > > ndo_set_vf_port = 0, > > > > > ndo_get_vf_port = 0, > > > > > ndo_setup_tc = 0, > > > > > ndo_add_slave = 0, > > > > > ndo_del_slave = 0, > > > > > ndo_fix_features = 0, > > > > > crash> > > > > > > > > I'm able to see the same. > > > > > > > > Setting suitable debug level reveals: > > > > > > > > bf00f040 (bf00f000): scsi_wait_scan syms: 0 gplsyms: 0 ksyms: 1 > > > > bf00a1f8 (bf008000): smsc95xx syms: 0 gplsyms: 0 ksyms: 60 > > > > bf002a40 (bf000000): usbnet syms: 0 gplsyms: 24 ksyms: 65 > > > > > > > > The ksyms comes from KALLSYMS and by default it only includes > > > > text and > > > > inittext symbols. This explains why Lei is not able to see data > > > > etc. symbols > > > > when he runs 'sym -m <module>'. > > > > > > > > So I believe crash on ARM works as it should in this case. > > > > > > I note that the symbols exported by ARM modules prior to mod -[sS] > > > contains a bunch of "$d" and "$a" symbols. The ARM > > > arm_verify_symbol() > > > function rejects symbols of that type, but that is only called if > > > the > > > "mod -[sS]" function is run. > > > > > > In other words, this is the flow during session initialization: > > > > > > module_init() > > > store_module_symbols_v2() -> symbols from KALLSYMS + > > > in-kernel module struct > > > > > > And if "mod -[sS]" is done, it goes like this: > > > > > > cmd_mod() > > > do_module_cmd() > > > load_module_symbols() > > > store_load_module_symbols() -> symbols from module.ko file > > > machdep->verify_symbol() > > > > > > So the "$d" and "$a" are there from the initialization-time onward. > > > > > > But since store_module_symbols_v2() has never called > > > machdep->verify_symbol() > > > I'm a bit hesitant to make it do so for all architectures without > > > knowing the > > > consequences. But it certainly seems legitimate in the > > > "machine_type("ARM")" case. > > > > Indeed. However, I'm a bit concerned because there is this check: > > > > if (STREQ(name, "swapper_pg_dir")) > > machdep->flags |= KSYMS_START; > > > > if (!name || !strlen(name) || !(machdep->flags & > > KSYMS_START)) > > return FALSE; > > > > so if the KSYMS_START is not yet set (is that possible?) we might reject a > > valid symbol from a module. > > KSYMS_START gets set during session initialization when parsing the base kernel > symbols from the vmlinux file in symtab_init()/store_symbols(), and it could never > get reset. OK, thanks for the clarification. Then I guess we can do something like the below patch? I'm not exactly sure if we can use ec->st_info below but it looks like arm_verify_symbol() doesn't use that anyway. diff --git a/symbols.c b/symbols.c index 4fb397c..10085be 100755 --- a/symbols.c +++ b/symbols.c @@ -2004,6 +2004,14 @@ store_module_kallsyms_v2(struct load_module *lm, int start, int curr, if (*nameptr == '\0') continue; + /* + * On ARM we have linker mapping symbols like '$a' and '$d'. + * Make sure that these don't end up into our symbol list. + */ + if (machine_type("ARM") && + !machdep->verify_symbol(nameptr, ec->st_value, ec->st_info)) + continue; + if (CRASHDEBUG(7)) fprintf(fp, "%s: st_name: %ld st_value: %lx st_shndx: %ld st_info: %c\n",