Hi Dave: Thanks for your prompt feedback and valuable suggestion! on the kdump'd kernel, correct? Yes, you are right, any module was added, the crash tool will not worked well. I'll check if core_size was in the kernel which we used. If there is no this member, I'll backport the support on this memeber in the following. Best Regards, James Liu Jianbo | WIND RIVER | Senior Engineer - Technical Support Tel 86 28 65318098 | Cell 86 13558641588 | Fax 86 28 65319983 Manage your support account: https://support.windriver.com Ask a Technical Question https://ask.windriver.com Submit a Service Request https://windriver.force.com/support ________________________________________ 发件人: crash-utility-bounces(a)redhat.com [crash-utility-bounces(a)redhat.com] 代表 Dave Anderson [anderson(a)redhat.com] 发送时间: 2016年1月12日 1:07 收件人: Discussion list for crash utility usage, maintenance and development 主题: Re: [Crash-utility] failed to do crash analysis after insert custom module in the first kernel. ----- Original Message ----- That's a 3-year-old version of crash, you really should upgrade to the most recent version. I'm guessing that when crash does work OK in #2, there were *no* kernel modules installed on the kdump'd kernel, correct? Crash has a hardwired presumption that the kernel's "module" structure has a "core_size" member. That member has existed since the beginning of Linux kernel version 2.6 if I'm not mistaken. (It used to be "module.size" back in the 2.4 era) The module.core_size member is not inside an #ifdef of any kind, so I can't explain why it is not being found by the embedded gdb module. If you do this, is the core_size member not shown?: $ gdb vmlinux.host.20160108nokgdbserial ... (gdb) ptype struct module type = struct module { ... I only have one 2.6.34-era kernel on-hand, and when I run the command above, it looks like this: $ gdb vmlinux-2.6.34-11.fc14 GNU gdb (GDB) Fedora (7.4.50.20120120-54.fc17) Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /tmp/vmlinux-2.6.34-11.fc14...done. (gdb) ptype struct module type = struct module { enum module_state state; struct list_head list; char name[56]; struct module_kobject mkobj; struct module_attribute *modinfo_attrs; const char *version; const char *srcversion; struct kobject *holders_dir; const struct kernel_symbol *syms; const long unsigned int *crcs; unsigned int num_syms; struct kernel_param *kp; unsigned int num_kp; unsigned int num_gpl_syms; const struct kernel_symbol *gpl_syms; const long unsigned int *gpl_crcs; const struct kernel_symbol *unused_syms; const long unsigned int *unused_crcs; unsigned int num_unused_syms; unsigned int num_unused_gpl_syms; const struct kernel_symbol *unused_gpl_syms; const long unsigned int *unused_gpl_crcs; const struct kernel_symbol *gpl_future_syms; const long unsigned int *gpl_future_crcs; unsigned int num_gpl_future_syms; unsigned int num_exentries; struct exception_table_entry *extable; int (*init)(void); void *module_init; void *module_core; unsigned int init_size; unsigned int core_size; <== NOTE core_size member here unsigned int init_text_size; unsigned int core_text_size; struct mod_arch_specific arch; unsigned int taints; unsigned int num_bugs; struct list_head bug_list; struct bug_entry *bug_table; Elf64_Sym *symtab; Elf64_Sym *core_symtab; unsigned int num_symtab; unsigned int core_num_syms; char *strtab; char *core_strtab; struct module_sect_attrs *sect_attrs; struct module_notes_attrs *notes_attrs; void *percpu; unsigned int percpu_size; char *args; struct tracepoint *tracepoints; unsigned int num_tracepoints; const char **trace_bprintk_fmt_start; unsigned int num_trace_bprintk_fmt; struct ftrace_event_call *trace_events; unsigned int num_trace_events; long unsigned int *ftrace_callsites; unsigned int num_ftrace_callsites; struct list_head modules_which_use_me; struct task_struct *waiter; void (*exit)(void); struct module_ref *refptr; ctor_fn_t *ctors; unsigned int num_ctors; } (gdb) By the way, you should be able to at least come up into a crash session by entering: $ crash --no_modules vmlinux.host.20160108nokgdbserial vmcore-1970-01-01 Doing that will skip kernel module initialization and therefore the function that's killing your crash session. However, I'm guessing that you're trying to debug a kernel module, and so it may not be all that helpful. Dave -- Crash-utility mailing list Crash-utility(a)redhat.com https://www.redhat.com/mailman/listinfo/crash-utility

Hi Dave: /******************************************************/ case KMOD_V2: MEMBER_OFFSET_INIT(module_num_syms, "module", "num_syms"); MEMBER_OFFSET_INIT(module_list, "module", "list"); MEMBER_OFFSET_INIT(module_gpl_syms, "module", "gpl_syms"); MEMBER_OFFSET_INIT(module_num_gpl_syms, "module", "num_gpl_syms"); MEMBER_OFFSET_INIT(module_module_core, "module", "module_core"); MEMBER_OFFSET_INIT(module_core_size, "module", "core_size"); ......... if (PAX()) { /* * PaX shall populate RX offsets into original ones. * This decision can work out PaX with less impact for * existing routines. * * Since PaX module dose not own text size, * these values should be sharpen by section attributes * while loading them from object file. */ MEMBER_OFFSET_INIT(module_module_core, "module", "module_core_rx"); MEMBER_OFFSET_INIT(module_core_size, "module", // looks this version crash have codes to deal with this member, "core_size_rx"); MEMBER_OFFSET_INIT(module_core_text_size, "module", "core_size_rx"); MEMBER_OFFSET_INIT(module_module_init, "module", "module_init_rx"); MEMBER_OFFSET_INIT(module_init_size, "module", "init_size_rx"); MEMBER_OFFSET_INIT(module_init_text_size, "module", "init_size_rx"); } /******************************************************/ Looks this version crash have codes to deal with this member, but on my side, it did not get into branch, still try to get code_size as module_core_size. If I change codes: /******************************************************/ case KMOD_V2: MEMBER_OFFSET_INIT(module_num_syms, "module", "num_syms"); MEMBER_OFFSET_INIT(module_list, "module", "list"); MEMBER_OFFSET_INIT(module_gpl_syms, "module", "gpl_syms"); MEMBER_OFFSET_INIT(module_num_gpl_syms, "module", "num_gpl_syms"); MEMBER_OFFSET_INIT(module_module_core, "module", "module_core_rx"); MEMBER_OFFSET_INIT(module_core_size, "module", "core_size"); /******************************************************/ The vmcore can be analysed well. If I miss some configuration in cgl kernel to coordinate with crash tool? /****************************************************/ #define PAX() ((THIS_KERNEL_VERSION >= LINUX(2,6,27)) && \ VALID_MEMBER(module_module_core_rx)) /****************************************************/ Looks Best Regards, James Liu Jianbo | WIND RIVER | Senior Engineer - Technical Support Tel 86 28 65318098 | Cell 86 13558641588 | Fax 86 28 65319983 Manage your support account: https://support.windriver.com Ask a Technical Question https://ask.windriver.com Submit a Service Request https://windriver.force.com/support ________________________________________ 发件人: Liu, Jianbo (James) 发送时间: 2016年1月12日 10:36 收件人: Discussion list for crash utility usage, maintenance and development 主题: 答复: [Crash-utility] 答复: failed to do crash analysis after insert custom module in the first kernel. Hi Dave; I did not find core_size member, but core_size_rx, core_size_rw instead: /**********************************/ void *module_init_rx; void *module_init_rw; void *module_core_rx; void *module_core_rw; unsigned int init_size_rw; unsigned int core_size_rw; unsigned int init_size_rx; unsigned int core_size_rx; /**********************************/ The problem should be here, thanks a lot!! Best Regards, James Liu Jianbo | WIND RIVER | Senior Engineer - Technical Support Tel 86 28 65318098 | Cell 86 13558641588 | Fax 86 28 65319983 Manage your support account: https://support.windriver.com Ask a Technical Question https://ask.windriver.com Submit a Service Request https://windriver.force.com/support ________________________________________ 发件人: crash-utility-bounces(a)redhat.com [crash-utility-bounces(a)redhat.com] 代表 Liu, Jianbo (James) [James.Liu(a)windriver.com] 发送时间: 2016年1月12日 10:23 收件人: Discussion list for crash utility usage, maintenance and development 主题: [Crash-utility] 答复: failed to do crash analysis after insert custom module in the first kernel. Hi Dave: Thanks for your prompt feedback and valuable suggestion! on the kdump'd kernel, correct? Yes, you are right, any module was added, the crash tool will not worked well. I'll check if core_size was in the kernel which we used. If there is no this member, I'll backport the support on this memeber in the following. Best Regards, James Liu Jianbo | WIND RIVER | Senior Engineer - Technical Support Tel 86 28 65318098 | Cell 86 13558641588 | Fax 86 28 65319983 Manage your support account: https://support.windriver.com Ask a Technical Question https://ask.windriver.com Submit a Service Request https://windriver.force.com/support ________________________________________ 发件人: crash-utility-bounces(a)redhat.com [crash-utility-bounces(a)redhat.com] 代表 Dave Anderson [anderson(a)redhat.com] 发送时间: 2016年1月12日 1:07 收件人: Discussion list for crash utility usage, maintenance and development 主题: Re: [Crash-utility] failed to do crash analysis after insert custom module in the first kernel. ----- Original Message ----- That's a 3-year-old version of crash, you really should upgrade to the most recent version. I'm guessing that when crash does work OK in #2, there were *no* kernel modules installed on the kdump'd kernel, correct? Crash has a hardwired presumption that the kernel's "module" structure has a "core_size" member. That member has existed since the beginning of Linux kernel version 2.6 if I'm not mistaken. (It used to be "module.size" back in the 2.4 era) The module.core_size member is not inside an #ifdef of any kind, so I can't explain why it is not being found by the embedded gdb module. If you do this, is the core_size member not shown?: $ gdb vmlinux.host.20160108nokgdbserial ... (gdb) ptype struct module type = struct module { ... I only have one 2.6.34-era kernel on-hand, and when I run the command above, it looks like this: $ gdb vmlinux-2.6.34-11.fc14 GNU gdb (GDB) Fedora (7.4.50.20120120-54.fc17) Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /tmp/vmlinux-2.6.34-11.fc14...done. (gdb) ptype struct module type = struct module { enum module_state state; struct list_head list; char name[56]; struct module_kobject mkobj; struct module_attribute *modinfo_attrs; const char *version; const char *srcversion; struct kobject *holders_dir; const struct kernel_symbol *syms; const long unsigned int *crcs; unsigned int num_syms; struct kernel_param *kp; unsigned int num_kp; unsigned int num_gpl_syms; const struct kernel_symbol *gpl_syms; const long unsigned int *gpl_crcs; const struct kernel_symbol *unused_syms; const long unsigned int *unused_crcs; unsigned int num_unused_syms; unsigned int num_unused_gpl_syms; const struct kernel_symbol *unused_gpl_syms; const long unsigned int *unused_gpl_crcs; const struct kernel_symbol *gpl_future_syms; const long unsigned int *gpl_future_crcs; unsigned int num_gpl_future_syms; unsigned int num_exentries; struct exception_table_entry *extable; int (*init)(void); void *module_init; void *module_core; unsigned int init_size; unsigned int core_size; <== NOTE core_size member here unsigned int init_text_size; unsigned int core_text_size; struct mod_arch_specific arch; unsigned int taints; unsigned int num_bugs; struct list_head bug_list; struct bug_entry *bug_table; Elf64_Sym *symtab; Elf64_Sym *core_symtab; unsigned int num_symtab; unsigned int core_num_syms; char *strtab; char *core_strtab; struct module_sect_attrs *sect_attrs; struct module_notes_attrs *notes_attrs; void *percpu; unsigned int percpu_size; char *args; struct tracepoint *tracepoints; unsigned int num_tracepoints; const char **trace_bprintk_fmt_start; unsigned int num_trace_bprintk_fmt; struct ftrace_event_call *trace_events; unsigned int num_trace_events; long unsigned int *ftrace_callsites; unsigned int num_ftrace_callsites; struct list_head modules_which_use_me; struct task_struct *waiter; void (*exit)(void); struct module_ref *refptr; ctor_fn_t *ctors; unsigned int num_ctors; } (gdb) By the way, you should be able to at least come up into a crash session by entering: $ crash --no_modules vmlinux.host.20160108nokgdbserial vmcore-1970-01-01 Doing that will skip kernel module initialization and therefore the function that's killing your crash session. However, I'm guessing that you're trying to debug a kernel module, and so it may not be all that helpful. Dave -- Crash-utility mailing list Crash-utility(a)redhat.com https://www.redhat.com/mailman/listinfo/crash-utility -- Crash-utility mailing list Crash-utility(a)redhat.com https://www.redhat.com/mailman/listinfo/crash-utility