Re: [Crash-utility] failed to do crash analysis after insert custom module in the first kernel.

----- Original Message ----- That's a 3-year-old version of crash, you really should upgrade to the most recent version. I'm guessing that when crash does work OK in #2, there were *no* kernel modules installed on the kdump'd kernel, correct? Crash has a hardwired presumption that the kernel's "module" structure has a "core_size" member. That member has existed since the beginning of Linux kernel version 2.6 if I'm not mistaken. (It used to be "module.size" back in the 2.4 era) The module.core_size member is not inside an #ifdef of any kind, so I can't explain why it is not being found by the embedded gdb module. If you do this, is the core_size member not shown?: $ gdb vmlinux.host.20160108nokgdbserial ... (gdb) ptype struct module type = struct module { ... I only have one 2.6.34-era kernel on-hand, and when I run the command above, it looks like this: $ gdb vmlinux-2.6.34-11.fc14 GNU gdb (GDB) Fedora (7.4.50.20120120-54.fc17) Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /tmp/vmlinux-2.6.34-11.fc14...done. (gdb) ptype struct module type = struct module { enum module_state state; struct list_head list; char name[56]; struct module_kobject mkobj; struct module_attribute *modinfo_attrs; const char *version; const char *srcversion; struct kobject *holders_dir; const struct kernel_symbol *syms; const long unsigned int *crcs; unsigned int num_syms; struct kernel_param *kp; unsigned int num_kp; unsigned int num_gpl_syms; const struct kernel_symbol *gpl_syms; const long unsigned int *gpl_crcs; const struct kernel_symbol *unused_syms; const long unsigned int *unused_crcs; unsigned int num_unused_syms; unsigned int num_unused_gpl_syms; const struct kernel_symbol *unused_gpl_syms; const long unsigned int *unused_gpl_crcs; const struct kernel_symbol *gpl_future_syms; const long unsigned int *gpl_future_crcs; unsigned int num_gpl_future_syms; unsigned int num_exentries; struct exception_table_entry *extable; int (*init)(void); void *module_init; void *module_core; unsigned int init_size; unsigned int core_size; <== NOTE core_size member here unsigned int init_text_size; unsigned int core_text_size; struct mod_arch_specific arch; unsigned int taints; unsigned int num_bugs; struct list_head bug_list; struct bug_entry *bug_table; Elf64_Sym *symtab; Elf64_Sym *core_symtab; unsigned int num_symtab; unsigned int core_num_syms; char *strtab; char *core_strtab; struct module_sect_attrs *sect_attrs; struct module_notes_attrs *notes_attrs; void *percpu; unsigned int percpu_size; char *args; struct tracepoint *tracepoints; unsigned int num_tracepoints; const char **trace_bprintk_fmt_start; unsigned int num_trace_bprintk_fmt; struct ftrace_event_call *trace_events; unsigned int num_trace_events; long unsigned int *ftrace_callsites; unsigned int num_ftrace_callsites; struct list_head modules_which_use_me; struct task_struct *waiter; void (*exit)(void); struct module_ref *refptr; ctor_fn_t *ctors; unsigned int num_ctors; } (gdb) By the way, you should be able to at least come up into a crash session by entering: $ crash --no_modules vmlinux.host.20160108nokgdbserial vmcore-1970-01-01 Doing that will skip kernel module initialization and therefore the function that's killing your crash session. However, I'm guessing that you're trying to debug a kernel module, and so it may not be all that helpful. Dave