2:37 a.m.
On 2023/11/08 17:16, HAGIO KAZUHITO(萩尾 一仁) wrote:
On 2023/11/08 12:01, HAGIO KAZUHITO(萩尾 一仁) wrote:
> Hi Tao,
>
> thank you for the information.
>
> I'm looking into it, I noticed that the unexpected symbol
"floopy_module_init"
> is in section .init.text. Crash side doesn't have the symbol info, probably
> the address is already freed and reused by the virtio_blk module?
>
> crash> mod -S
> ...
> crash> sym -m floppy | grep MODULE
> ffffffffc0092000 MODULE START: floppy
> ffffffffc00a2f29 MODULE END: floppy
> crash> sym -m virtio_blk | grep MODULE
> ffffffffc00a4000 MODULE START: virtio_blk
> ffffffffc00a86ec MODULE END: virtio_blk
> crash> gdb info symbol floppy_module_init
> init_module in section .init.text of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> virtblk_freeze + 33 in section .text of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/virtio_blk.ko.debug
> crash> gdb info address floppy_module_init
> Symbol "floppy_module_init" is a function at address 0xffffffffc00a4131.
> crash> sym floppy_module_init
> symbol not found: floppy_module_init
> possible alternatives:
> (none found)
>
> So still not sure, but this might be an issue of handling the init
> section/symbols of modules..
Hmm, it looks like several sections are mapped outside of the current
module range. What if they are not specified or they are mapped to
e.g. an unused range..? or any idea?
First, "mod -S -r" can reproduce this without the patch 1/2 or the commands.
(If the root cause is fixed, I think no need to apply the patch 1/2.)
crash-ups> mod -S -r
crash-ups> p virtio_mq_ops
virtio_mq_ops = $1 = {
queue_rq = 0xffffffffc00a45b0 <floppy_module_init+1151>,
map_queue = 0xffffffff813015c0 <blk_mq_map_queue>,
The following trial patch, which does not map the .init.text section
specifically, fixes the issue.
--- a/symbols.c
+++ b/symbols.c
@@ -13283,7 +13283,7 @@ add_symbol_file_kallsyms(struct load_module *lm, struct
gnu_request *req)
shift_string_right(req->buf, strlen(buf));
BCOPY(buf, req->buf, strlen(buf));
retval = TRUE;
- } else {
+ } else if (!STREQ(section_name, ".init.text")) {
sprintf(buf, " -s %s 0x%lx", section_name,
section_vaddr);
while ((len + strlen(buf)) >= buflen) {
RESIZEBUF(req->buf, buflen, buflen * 2);
crash-dev> mod -S -r
crash-dev> p virtio_mq_ops
virtio_mq_ops = $1 = {
queue_rq = 0xffffffffc00a45b0 <virtio_queue_rq>,
map_queue = 0xffffffff813015c0 <blk_mq_map_queue>,
{
In this way, it looks like the .init.text sections of modules are
mapped to 0x0, but many other sections like .modinfo are mapped here
even without the trial patch.
crash-dev> gdb info address floppy_module_init
Symbol "floppy_module_init" is a function at address 0x131.
crash-dev> gdb info symbol floppy_module_init
irq_stack_union + 305 in section .data..percpu of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/vmlinux
local_init + 305 in section .init.text of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-mod.ko.debug
____versions + 49 in section __versions of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-region-hash.ko.debug
____versions + 145 in section __versions of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio.ko.debug
__UNIQUE_ID_vermagic5 + 30 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-log.ko.debug
__UNIQUE_ID_srcversion7 + 1 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-mirror.ko.debug
__UNIQUE_ID_alias7 + 30 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio_pci.ko.debug
____versions + 145 in section __versions of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio_ring.ko.debug
__UNIQUE_ID_alias16 + 36 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/ata/ata_generic.ko.debug
____versions + 17 in section __versions of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/fs/mbcache.ko.debug
__UNIQUE_ID_vermagic5 + 45 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/net/virtio_net.ko.debug
__UNIQUE_ID_vermagic5 + 44 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/cdrom/cdrom.ko.debug
__UNIQUE_ID_vermagic5 + 14 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/scsi/sr_mod.ko.debug
__module_depends + 1 in section .modinfo of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.debug
____versions + 81 in section __versions of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/fs/binfmt_misc.ko.debug
init_module in section .init.text of
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
...
If a module already does not have its init memory range, it might be
a bit better to not specify "-s .init.text <addr>" to add-symbol-file..
Thanks,
Kazu
>
> crash-dev> sym -m floppy | grep MODULE
> ffffffffc0092000 MODULE START: floppy
> ffffffffc00a2f29 MODULE END: floppy
> crash-dev> set debug 1
> debug: 1
> crash-dev> mod -s floppy
> ffffffffc009a000 .note.gnu.build-id
> ffffffffc0092000 .text
> ffffffffc00a4000 .init.text <<< higher than MODULE END
> ffffffffc009971d .text.unlikely
> ffffffffc00998ea .exit.text
> ffffffffc009a040 .rodata
> ffffffffc009a4d0 .rodata.str1.1
> ffffffffc009ad98 .rodata.str1.8
> ffffffffc009be14 .smp_locks
> ffffffffc009be98 .parainstructions
> ffffffffc009beb8 __param
> ffffffffc009bf18 __mcount_loc
> ffffffffc009d000 .data
> ffffffffc00a5000 .init.data <<<
> ffffffffc009db00 .gnu.linkonce.this_module
> ffffffffc009dd40 .bss
> ffffffffc00a6000 .symtab <<<
> ffffffffc00a80b8 .strtab <<<
> add-symbol-file
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
0xffffffffc0092000 -s .note.gnu.build-id 0xffffffffc009a000 -s .init.text
0xffffffffc00a4000 -s .text.unlikely 0xffffffffc009971d -s .exit.text 0xffffffffc00998ea
-s .rodata 0xffffffffc009a040 -s .rodata.str1.1 0xffffffffc009a4d0 -s .rodata.str1.8
0xffffffffc009ad98 -s .smp_locks 0xffffffffc009be14 -s .parainstructions
0xffffffffc009be98 -s __param 0xffffffffc009beb8 -s __mcount_loc 0xffffffffc009bf18 -s
.data 0xffffffffc009d000 -s .init.data 0xffffffffc00a5000 -s .gnu.linkonce.this_module
0xffffffffc009db00 -s .bss 0xffffffffc009dd40 -s .symtab 0xffffffffc00a6000 -s .strtab
0xffffffffc00a80b8
> add symbol table from file
"/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug"
at
> .text_addr = 0xffffffffc0092000
> .note.gnu.build-id_addr = 0xffffffffc009a000
> .init.text_addr = 0xffffffffc00a4000
> .text.unlikely_addr = 0xffffffffc009971d
> .exit.text_addr = 0xffffffffc00998ea
> .rodata_addr = 0xffffffffc009a040
> .rodata.str1.1_addr = 0xffffffffc009a4d0
> .rodata.str1.8_addr = 0xffffffffc009ad98
> .smp_locks_addr = 0xffffffffc009be14
> .parainstructions_addr = 0xffffffffc009be98
> __param_addr = 0xffffffffc009beb8
> __mcount_loc_addr = 0xffffffffc009bf18
> .data_addr = 0xffffffffc009d000
> .init.data_addr = 0xffffffffc00a5000
> .gnu.linkonce.this_module_addr = 0xffffffffc009db00
> .bss_addr = 0xffffffffc009dd40
> .symtab_addr = 0xffffffffc00a6000
> .strtab_addr = 0xffffffffc00a80b8
> warning: section .symtab not found in
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> warning: section .strtab not found in
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> MODULE NAME BASE SIZE OBJECT FILE
> ffffffffc009db00 floppy ffffffffc0092000 69417
/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
>