[PATCH] Fix for "bt pid" command not printing enough stack trace
by Lianbo Jiang
Currently, the "bt pid" command may not print enough stack trace and the
remaining frames will be truncated on x86_64. For example:
Without the patch:
crash> bt 493113
PID: 493113 TASK: ff2e34ecbd3ca2c0 CPU: 27 COMMAND: "sriov_fec_daemo"
#0 [ff77abc4e81cfb08] __schedule at ffffffff81b239cb
#1 [ff77abc4e81cfb70] schedule at ffffffff81b23e2d
#2 [ff77abc4e81cfb88] schedule_timeout at ffffffff81b2c9e8
RIP: 000000000047cdbb RSP: 000000c0000975a8 RFLAGS: 00000216
RAX: ffffffffffffffda RBX: 000000c00004e000 RCX: 000000000047cdbb
RDX: 000000000000000c RSI: 000000c000097798 RDI: 0000000000000009
RBP: 000000c0000975f8 R8: 0000000000000001 R9: 000000c00098d680
R10: 000000000000000c R11: 0000000000000216 R12: 000000c000097688
R13: 0000000000000000 R14: 000000c0006c3520 R15: 00007f5e359946b7
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
With the patch:
crash> bt 493113
PID: 493113 TASK: ff2e34ecbd3ca2c0 CPU: 27 COMMAND: "sriov_fec_daemo"
#0 [ff77abc4e81cfb08] __schedule at ffffffff81b239cb
#1 [ff77abc4e81cfb70] schedule at ffffffff81b23e2d
#2 [ff77abc4e81cfb88] schedule_timeout at ffffffff81b2c9e8
#3 [ff77abc4e81cfc68] vfio_unregister_group_dev at ffffffffc10e76ae [vfio]
#4 [ff77abc4e81cfca8] vfio_pci_core_unregister_device at ffffffffc11bb599 [vfio_pci_core]
#5 [ff77abc4e81cfcc0] vfio_pci_remove at ffffffffc103e045 [vfio_pci]
#6 [ff77abc4e81cfcd0] pci_device_remove at ffffffff815d7513
#7 [ff77abc4e81cfcf0] device_release_driver_internal at ffffffff81708baa
#8 [ff77abc4e81cfd20] unbind_store at ffffffff81705f6f
#9 [ff77abc4e81cfd50] kernfs_fop_write_iter at ffffffff81454bf1
#10 [ff77abc4e81cfd88] new_sync_write at ffffffff813aad8c
#11 [ff77abc4e81cfe20] vfs_write at ffffffff813adb36
#12 [ff77abc4e81cfe58] ksys_write at ffffffff813adeb2
#13 [ff77abc4e81cfe90] do_syscall_64 at ffffffff81b17159
#14 [ff77abc4e81cff50] entry_SYSCALL_64_after_hwframe at ffffffff81c0009b
RIP: 000000000047cdbb RSP: 000000c0000975a8 RFLAGS: 00000216
RAX: ffffffffffffffda RBX: 000000c00004e000 RCX: 000000000047cdbb
RDX: 000000000000000c RSI: 000000c000097798 RDI: 0000000000000009
RBP: 000000c0000975f8 R8: 0000000000000001 R9: 000000c00098d680
R10: 000000000000000c R11: 0000000000000216 R12: 000000c000097688
R13: 0000000000000000 R14: 000000c0006c3520 R15: 00007f5e359946b7
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
Let's add a check function that jump to schedule_timeout(), just like
the schedule_timeout_*() in x86_64_function_called_by().
Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com>
---
x86_64.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/x86_64.c b/x86_64.c
index 42ade4817ad9..16850d98dc2d 100644
--- a/x86_64.c
+++ b/x86_64.c
@@ -4487,7 +4487,8 @@ x86_64_function_called_by(ulong rip)
*/
if (sp) {
if ((STREQ(sp->name, "schedule_timeout_interruptible") ||
- STREQ(sp->name, "schedule_timeout_uninterruptible")))
+ STREQ(sp->name, "schedule_timeout_uninterruptible") ||
+ STREQ(sp->name, "wait_for_completion_interruptible_timeout")))
sp = symbol_search("schedule_timeout");
if (STREQ(sp->name, "__cond_resched"))
--
2.41.0
10 months, 2 weeks
[Crash-utility][RESEND PATCH v2 00/10] add LoongArch64 platform support
by Ming Wang
Changes between v1 and v2:
- Simplified the LoongArch64 gdb's backport patch and appended it to the gdb-10.2.patch.
- Add building the loongarch crash binary on an X86 64 host support.
- Fix offset_table format issues.
- Move private definitions code to loongarch64.c.
This patch set are for Crash-utility tool, it make crash tool support on
loongarch64 architecture and the common commands(bt, p, rd, mod, log, set,
dis, and so on).
The patch sets were tested on a loongArch64 Loongson-3C5000 processor. Can
successfully enter the crash command line and support for common command.
...
KERNEL: vmlinux
DUMPFILE: /proc/kcore
CPUS: 16
DATE: Thu Jul 27 19:51:21 CST 2023
UPTIME: 06:35:11
LOAD AVERAGE: 0.15, 0.03, 0.01
TASKS: 257
NODENAME: localhost.localdomain
RELEASE: 5.10.0-60.102.0.128.oe2203.loongarch64
VERSION: #1 SMP Fri Jul 14 04:17:09 UTC 2023
MACHINE: loongarch64 (2200 Mhz)
MEMORY: 64 GB
PID: 2964
COMMAND: "crash"
TASK: 9000000098805500 [THREAD_INFO: 9000000094d48000]
CPU: 6
STATE: TASK_RUNNING (ACTIVE)
crash>
crash> dis -l start_kernel
/linux-loongarch64/init/main.c: 883
0x9000000001030818 <start_kernel>: 0x0141ee40
/linux-loongarch64/init/main.c: 879
0x900000000103081c <start_kernel+4>: 0x90000000
/linux-loongarch64/init/main.c: 883
0x9000000001030820 <start_kernel+8>: addu16i.d $zero, $t8, 8179(0x1ff3)
/linux-loongarch64/init/main.c: 879
...
About the LoongArch64 Architecture:
https://www.kernel.org/doc/html/latest/loongarch/index.html
Thanks and regards,
Ming
Ming Wang (10):
Add LoongArch64 framework code support
LoongArch64: Make the crash tool successfully enter the crash command
line
LoongArch64: Add 'pte' command support
LoongArch64: Add 'mach' command support
LoongArch64: Add 'bt' command support
LoongArch64: Add 'help -m/M' command support
LoongArch64: Add 'help -r' command support
LoongArch64: Add 'irq' command support
LoongArch64: Add "--kaslr" command line option support
LoongArch64: Add LoongArch64 architecture support information
Makefile | 7 +-
README | 4 +-
configure.c | 39 +-
crash.8 | 2 +-
defs.h | 164 +-
diskdump.c | 24 +-
gdb-10.2.patch | 12818 ++++++++++++++++++++++++++++++++++++++++++
help.c | 13 +-
lkcd_vmdump_v1.h | 2 +-
lkcd_vmdump_v2_v3.h | 5 +-
loongarch64.c | 1366 +++++
main.c | 3 +-
netdump.c | 27 +-
ramdump.c | 2 +
symbols.c | 33 +-
15 files changed, 14484 insertions(+), 25 deletions(-)
create mode 100644 loongarch64.c
base-commit: 55a43bcefa20161c7e56ed0e309e90e941f47efc
--
2.39.2
11 months
Re: [PATCH v3] add "files -n" command for an inode
by lijiang
On Thu, Nov 2, 2023 at 9:35 AM Huang Shijie <shijie(a)os.amperecomputing.com>
wrote:
> In the NUMA machine, it is useful to know the memory distribution of
> an inode page cache:
> How many pages in the node 0?
> How many pages in the node 1?
>
> Add "files -n" command to get the memory distribution information:
> 1.) Add new argument for dump_inode_page_cache_info()
> 2.) make page_to_nid() a global function.
> 3.) Add summary_inode_page() to check each page's node
> information.
> 4.) Use print_inode_summary_info() to print the
> memory distribution information of an inode.
>
> Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
> ---
> v2 --> v3:
> 1.) Always return 1 for summary_inode_page().
> 2.) Add more comment for help_files.
>
>
Thank you for the update, Shijie.
> ---
> defs.h | 1 +
> filesys.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++-----
> help.c | 14 +++++++++++++-
> memory.c | 3 +--
> 4 files changed, 63 insertions(+), 8 deletions(-)
>
> diff --git a/defs.h b/defs.h
> index 788f63a..1fe2d0b 100644
> --- a/defs.h
> +++ b/defs.h
> @@ -5750,6 +5750,7 @@ int dump_inode_page(ulong);
> ulong valid_section_nr(ulong);
> void display_memory_from_file_offset(ulonglong, long, void *);
> void swap_info_init(void);
> +int page_to_nid(ulong);
>
> /*
> * filesys.c
> diff --git a/filesys.c b/filesys.c
> index 1d0ee7f..2c7cc74 100644
> --- a/filesys.c
> +++ b/filesys.c
> @@ -49,7 +49,7 @@ static int match_file_string(char *, char *, char *);
> static ulong get_root_vfsmount(char *);
> static void check_live_arch_mismatch(void);
> static long get_inode_nrpages(ulong);
> -static void dump_inode_page_cache_info(ulong);
> +static void dump_inode_page_cache_info(ulong, void *callback);
>
> #define DENTRY_CACHE (20)
> #define INODE_CACHE (20)
> @@ -2192,8 +2192,31 @@ get_inode_nrpages(ulong i_mapping)
> return nrpages;
> }
>
> +/* Used to collect the numa information for an inode */
> +static ulong *numa_node;
> +
> +static void
> +print_inode_summary_info(void)
> +{
> + int i;
> +
> + fprintf(fp, " NODE PAGES\n");
> + for (i = 0; i < vt->numnodes; i++)
> + fprintf(fp, " %2d %8ld\n", i, numa_node[i]);
> +}
> +
> +static int
> +summary_inode_page(ulong page)
> +{
> + int node = page_to_nid(page);
> +
> + if (0 <= node && node < vt->numnodes)
> + numa_node[node]++;
> + return 1;
> +}
>
A clear error message would be nice when the "files -n" command fails.
What's your opinion on the following changes?
+static int
+summary_inode_page(ulong page)
+{
+ int node;
+
+ if (!is_page_ptr(page, NULL))
+ error(FATAL, "Invalid inode page(0x%lx)\n", page);
+
+ node = page_to_nid(page);
+ if (node < 0 || node >= vt->numnodes)
+ error(FATAL, "Invalid node(%d) for page(0x%lx)\n", node,
page);
+
+ numa_node[node]++;
+
+ return 1;
+}
Without the above changes, it will print a lot of failures of "invalid
page" once the corresponding inode page is invalid, unless that is expected
behavior.
crash> files -n ffff8ea84c130938
INODE NRPAGES
ffff8ea84c130938 62527
files: page_to_nid: invalid page: 0
files: page_to_nid: invalid page: 0
files: page_to_nid: invalid page: 0
files: page_to_nid: invalid page: 10
files: page_to_nid: invalid page: 10
files: page_to_nid: invalid page: 10
files: page_to_nid: invalid page: 20
files: page_to_nid: invalid page: 20
files: page_to_nid: invalid page: 20
files: page_to_nid: invalid page: 30
files: page_to_nid: invalid page: 30
files: page_to_nid: invalid page: 30
files: page_to_nid: invalid page: 40
files: page_to_nid: invalid page: 40
files: page_to_nid: invalid page: 40
files: page_to_nid: invalid page: 50
files: page_to_nid: invalid page: 50
files: page_to_nid: invalid page: 50
files: page_to_nid: invalid page: 60
files: page_to_nid: invalid page: 60
...
And also says that in help.c:
+" -n inode given a hexadecimal inode address, check all the pages",
+" in the page cache, and display a NUMA node distribution",
+" if the inode page is valid, otherwise it will fail.",
Just my suggestions, any thoughts?
Thanks.
Lianbo
+
> static void
> -dump_inode_page_cache_info(ulong inode)
> +dump_inode_page_cache_info(ulong inode, void *callback)
> {
> char *inode_buf;
> ulong i_mapping, nrpages, root_rnode, xarray, count;
> @@ -2236,7 +2259,7 @@ dump_inode_page_cache_info(ulong inode)
> root_rnode = i_mapping + OFFSET(address_space_page_tree);
>
> lp.index = 0;
> - lp.value = (void *)&dump_inode_page;
> + lp.value = callback;
>
> if (root_rnode)
> count = do_radix_tree(root_rnode, RADIX_TREE_DUMP_CB, &lp);
> @@ -2276,7 +2299,7 @@ cmd_files(void)
> ref = NULL;
> refarg = NULL;
>
> - while ((c = getopt(argcnt, args, "d:R:p:c")) != EOF) {
> + while ((c = getopt(argcnt, args, "d:n:R:p:c")) != EOF) {
> switch(c)
> {
> case 'R':
> @@ -2295,11 +2318,31 @@ cmd_files(void)
> display_dentry_info(value);
> return;
>
> + case 'n':
> + if (VALID_MEMBER(address_space_page_tree) &&
> + VALID_MEMBER(inode_i_mapping)) {
> + value = htol(optarg, FAULT_ON_ERROR, NULL);
> +
> + /* Allocate the array for this inode */
> + numa_node = malloc(sizeof(ulong) *
> vt->numnodes);
> + BZERO(numa_node, sizeof(ulong) *
> vt->numnodes);
> +
> + dump_inode_page_cache_info(value, (void
> *)&summary_inode_page);
> +
> + /* Print out the NUMA node information for
> this inode */
> + print_inode_summary_info();
> +
> + free(numa_node);
> + numa_node = NULL;
> + } else
> + option_not_supported('n');
> + return;
> +
> case 'p':
> if (VALID_MEMBER(address_space_page_tree) &&
> VALID_MEMBER(inode_i_mapping)) {
> value = htol(optarg, FAULT_ON_ERROR, NULL);
> - dump_inode_page_cache_info(value);
> + dump_inode_page_cache_info(value, (void
> *)&dump_inode_page);
> } else
> option_not_supported('p');
> return;
> diff --git a/help.c b/help.c
> index cc7ab20..e9e28b7 100644
> --- a/help.c
> +++ b/help.c
> @@ -7850,7 +7850,7 @@ NULL
> char *help_files[] = {
> "files",
> "open files",
> -"[-d dentry] | [-p inode] | [-c] [-R reference] [pid | taskp] ... ",
> +"[-d dentry] | [-p inode] | [-n inode] | [-c] [-R reference] [pid |
> taskp] ... ",
> " This command displays information about open files of a context.",
> " It prints the context's current root directory and current working",
> " directory, and then for each open file descriptor it prints a pointer",
> @@ -7863,6 +7863,8 @@ char *help_files[] = {
> " specific, and only shows the data requested.\n",
> " -d dentry given a hexadecimal dentry address, display its inode,",
> " super block, file type, and full pathname.",
> +" -n inode given a hexadecimal inode address, check all the pages",
> +" in the page cache, and display a NUMA node
> distribution.",
> " -p inode given a hexadecimal inode address, dump all of its
> pages",
> " that are in the page cache.",
> " -c for each open file descriptor, prints a pointer to its",
> @@ -7974,6 +7976,16 @@ char *help_files[] = {
> " ca1ddde0 2eeef000 f59b91ac 3 2 82c
> referenced,uptodate,lru,private",
> " ca36b300 3b598000 f59b91ac 4 2 82c
> referenced,uptodate,lru,private",
> " ca202680 30134000 f59b91ac 5 2 82c
> referenced,uptodate,lru,private",
> +" ",
> +" For the inode at address ffff07ff8c6f97f8, display the NUMA node",
> +" distribution of its pages that are in the page cache:",
> +" %s> files -n ffff07ff8c6f97f8",
> +" INODE NRPAGES",
> +" ffff07ff8c6f97f8 25240",
> +" ",
> +" NODE PAGES",
> +" 0 25240",
> +" 1 0",
> " ",
> NULL
> };
> diff --git a/memory.c b/memory.c
> index 86ccec5..ed1a4fb 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -300,7 +300,6 @@ static int dump_vm_event_state(void);
> static int dump_page_states(void);
> static int generic_read_dumpfile(ulonglong, void *, long, char *, ulong);
> static int generic_write_dumpfile(ulonglong, void *, long, char *, ulong);
> -static int page_to_nid(ulong);
> static int get_kmem_cache_list(ulong **);
> static int get_kmem_cache_root_list(ulong **);
> static int get_kmem_cache_child_list(ulong **, ulong);
> @@ -19846,7 +19845,7 @@ is_kmem_cache_addr_common(ulong vaddr, char *kbuf)
> /*
> * Kernel-config-neutral page-to-node evaluator.
> */
> -static int
> +int
> page_to_nid(ulong page)
> {
> int i;
> --
> 2.40.1
>
>
11 months, 2 weeks
[PATCH 0/3] s390x: uncouple physical and virtual memory spaces
by Alexander Gordeev
Hi all,
Currently physical and virtual addresses are the same on S390X,
but in the future it is going to be uncoupled just like on any
other architecture. This series supports that feature, but it
does not impact the current and older kernel versions.
Patch 2 is basically NOP and only fix semantics.
Patch 3 uses the feature if it is available in kernel.
Thanks!
Alexander Gordeev (3):
Fix identity_map_base value dump on S390
s390x: fix virtual vs physical address confusion
s390x: uncouple physical and virtual memory spaces
defs.h | 20 ++++-
s390.c | 2 +-
s390x.c | 245 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
3 files changed, 255 insertions(+), 12 deletions(-)
--
2.40.1
11 months, 2 weeks
[BUG FIXED]fix bug of CACHED in kmem -i show memory
by 薛国伦
Hi lianbo:
I have a bug fixed need to merge to crash-utility which patch in attachment.
It can obviously see that kmem -i show memory info of system.
But CACHED have wrong output for add up size of cached memory.
The reason of this bug is:
newest kernel version use "struct address_space *swapper_spaces[MAX_SWAPFILES]"
instead of "struct address_space swapper_spaces[MAX_SWAPFILES]" in old kernel.
this change bring into kernel after kernel 4.11.0
So newest version need to readmem twice, first is address of one struct address_space
second is value of struct address_space.
This bug fix patch add twice readmem when read swapper_spaces struct, it can accurately
read nrpage in struct address_space. And i found that definition of struct address_space swapper_spaces
change in kernel 4.11.0, so i make kernel version compatible for this change.
fix before:
crash> kmem -i
PAGES TOTAL PERCENTAGE
TOTAL MEM 2854115 10.9 GB ----
FREE 169699 662.9 MB 5% of TOTAL MEM
USED 2684416 10.2 GB 94% of TOTAL MEM
SHARED 891094 3.4 GB 31% of TOTAL MEM
BUFFERS 329 1.3 MB 0% of TOTAL MEM
CACHED 873327085626 3331478.4 GB 30598875% of TOTAL MEM
SLAB 230128 898.9 MB 8% of TOTAL MEM
fix after:
crash> kmem -i
PAGES TOTAL PERCENTAGE
TOTAL MEM 2854115 10.9 GB ----
FREE 169699 662.9 MB 5% of TOTAL MEM
USED 2684416 10.2 GB 94% of TOTAL MEM
SHARED 891094 3.4 GB 31% of TOTAL MEM
BUFFERS 329 1.3 MB 0% of TOTAL MEM
CACHED 1729018 6.6 GB 60% of TOTAL MEM
SLAB 230128 898.9 MB 8% of TOTAL MEM
Thanks
Best Regards
#/******本邮件及其附件含有小米公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from XIAOMI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!******/#
11 months, 3 weeks
[PATCH] symbols: handle module symbols outside strbuf
by Stephen Brennan
Module symbol names can get overwritten by live patches or ksplice in
odd corner cases, so that the pointer no longer points within the string
buffer. Gracefully fallback to reading the string directly from the
kernel image in these cases, to avoid possible segmentation faults
reading outside the bounds of strbuf.
Signed-off-by: Stephen Brennan <stephen.s.brennan(a)oracle.com>
---
Hi folks - I encountered a segfault on a vmcore which had a module
symbol that had gotten its name overwritten by a ksplice (live patch).
It seems like there's not a guarantee that module symbol names _must_
live within the same symbol buffer, and there is even logic to prevent
reading too much data into strbuf in those cases.
This patch simply ensures that symbol names which start outside of the
strbuf which we copied, are read directly from the kernel image, rather
than indexing past the bounds of strbuf. I encountered this in
store_module_symbols_v2() and have tested it there, but I replicated the
code to the other versions. I will try to test it out on the other
variants as well, but I thought I'd share the patch now.
symbols.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/symbols.c b/symbols.c
index 176c950..e70dd69 100644
--- a/symbols.c
+++ b/symbols.c
@@ -1704,7 +1704,7 @@ store_module_symbols_v1(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && (unsigned long)modsym->name - first < strbuflen)
strcpy(buf1,
&strbuf[(ulong)modsym->name - first]);
else
@@ -2080,7 +2080,7 @@ store_module_symbols_6_4(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(syms, modsym, i) - first < strbuflen)
strcpy(buf1, &strbuf[modsym_name(syms, modsym, i) - first]);
else
read_string(modsym_name(syms, modsym, i), buf1, BUFSIZE-1);
@@ -2148,7 +2148,7 @@ store_module_symbols_6_4(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(gpl_syms, modsym, i) - first < strbuflen)
strcpy(buf1, &strbuf[modsym_name(gpl_syms, modsym, i) - first]);
else
read_string(modsym_name(gpl_syms, modsym, i), buf1, BUFSIZE-1);
@@ -2456,7 +2456,7 @@ store_module_symbols_v2(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(syms, modsym, i) - first < strbuflen)
strcpy(buf1,
&strbuf[modsym_name(syms, modsym, i) - first]);
else
@@ -2529,7 +2529,7 @@ store_module_symbols_v2(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(gpl_syms, modsym, i) - first < strbuflen)
strcpy(buf1,
&strbuf[modsym_name(gpl_syms, modsym, i) - first]);
else
--
2.39.3
11 months, 3 weeks
[PATCH 1/2] Add a new helper function get_value_vmcore
by Huang Shijie
Add get_value_vmcore() to get the symbol value for @name.
Also add macro GET_SYM to simplify the code.
Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
---
defs.h | 1 +
kernel.c | 85 +++++++++++++++++++-------------------------------------
2 files changed, 30 insertions(+), 56 deletions(-)
diff --git a/defs.h b/defs.h
index 1fe2d0b..eec7b3e 100644
--- a/defs.h
+++ b/defs.h
@@ -6055,6 +6055,7 @@ int hide_offline_cpu(int);
int get_highest_cpu_online(void);
int get_highest_cpu_present(void);
int get_cpus_to_display(void);
+bool get_value_vmcore(const char *name, ulong *v);
void get_log_from_vmcoreinfo(char *file);
int in_cpu_map(int, int);
void paravirt_init(void);
diff --git a/kernel.c b/kernel.c
index 6dcf414..caf6149 100644
--- a/kernel.c
+++ b/kernel.c
@@ -104,6 +104,20 @@ static void check_vmcoreinfo(void);
static int is_pvops_xen(void);
static int get_linux_banner_from_vmlinux(char *, size_t);
+/* Return TRUE if we succeed, return FALSE on failure. */
+bool
+get_value_vmcore(const char *name, ulong *v)
+{
+ char *string = pc->read_vmcoreinfo(name);
+
+ if (!string)
+ return FALSE;
+
+ *v = htol(string, RETURN_ON_ERROR, NULL);
+ free(string);
+ return TRUE;
+}
+
/*
* popuplate the global kernel table (kt) with kernel version
* information parsed from UTSNAME/OSRELEASE string
@@ -10984,6 +10998,12 @@ hypervisor_init(void)
fprintf(fp, "hypervisor: %s\n", kt->hypervisor);
}
+#define GET_SYM(s,v) \
+ if (get_value_vmcore((s), &(v))) { \
+ if (CRASHDEBUG(1)) \
+ fprintf(fp, s ": %lx\n", v); \
+ }
+
/*
* Get and display the kernel log buffer using the vmcoreinfo
* data alone without the vmlinux file.
@@ -11024,62 +11044,15 @@ get_log_from_vmcoreinfo(char *file)
} else
error(FATAL, "VMCOREINFO: cannot determine page size\n");
- if ((string = pc->read_vmcoreinfo("SYMBOL(log_buf)"))) {
- vmc->log_buf_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(log_buf): %lx\n",
- vmc->log_buf_SYMBOL);
- free(string);
- }
- if ((string = pc->read_vmcoreinfo("SYMBOL(log_end)"))) {
- vmc->log_end_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(log_end): %lx\n",
- vmc->log_end_SYMBOL);
- free(string);
- }
- if ((string = pc->read_vmcoreinfo("SYMBOL(log_buf_len)"))) {
- vmc->log_buf_len_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(log_buf_len): %lx\n",
- vmc->log_buf_len_SYMBOL);
- free(string);
- }
- if ((string = pc->read_vmcoreinfo("SYMBOL(logged_chars)"))) {
- vmc->logged_chars_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(logged_chars): %lx\n",
- vmc->logged_chars_SYMBOL);
- free(string);
- }
- if ((string = pc->read_vmcoreinfo("SYMBOL(log_first_idx)"))) {
- vmc->log_first_idx_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(log_first_idx): %lx\n",
- vmc->log_first_idx_SYMBOL);
- free(string);
- }
- if ((string = pc->read_vmcoreinfo("SYMBOL(log_next_idx)"))) {
- vmc->log_next_idx_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(log_next_idx): %lx\n",
- vmc->log_next_idx_SYMBOL);
- free(string);
- }
- if ((string = pc->read_vmcoreinfo("SYMBOL(phys_base)"))) {
- vmc->phys_base_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(phys_base): %lx\n",
- vmc->phys_base_SYMBOL);
- free(string);
- }
- if ((string = pc->read_vmcoreinfo("SYMBOL(_stext)"))) {
- vmc->_stext_SYMBOL = htol(string, RETURN_ON_ERROR, NULL);
- if (CRASHDEBUG(1))
- fprintf(fp, "SYMBOL(_stext): %lx\n",
- vmc->_stext_SYMBOL);
- free(string);
- }
+ GET_SYM("SYMBOL(log_buf)", vmc->log_buf_SYMBOL);
+ GET_SYM("SYMBOL(log_end)", vmc->log_end_SYMBOL);
+ GET_SYM("SYMBOL(log_buf_len)", vmc->log_buf_len_SYMBOL);
+ GET_SYM("SYMBOL(logged_chars)", vmc->logged_chars_SYMBOL);
+ GET_SYM("SYMBOL(log_first_idx)", vmc->log_first_idx_SYMBOL);
+ GET_SYM("SYMBOL(log_next_idx)", vmc->log_next_idx_SYMBOL);
+ GET_SYM("SYMBOL(phys_base)", vmc->phys_base_SYMBOL);
+ GET_SYM("SYMBOL(_stext)", vmc->_stext_SYMBOL);
+
if ((string = pc->read_vmcoreinfo("OFFSET(log.ts_nsec)"))) {
vmc->log_ts_nsec_OFFSET = dtol(string, RETURN_ON_ERROR, NULL);
if (CRASHDEBUG(1))
--
2.40.1
12 months
[PATCH v2] symbols: expand all kernel module symtable if not all expanded previously
by Tao Liu
There is an issue that, for kernel modules, "dis -rl" fails to display
module's code line number data after execute "bt" cmd in crash.
Without the patch:
crsah> mod -S
crash> bt
PID: 1500 TASK: ff2bd8b093524000 CPU: 16 COMMAND: "lpfc_worker_0"
#0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
...snip...
#7 [ff2c9f725c39fc00] page_fault at ffffffff8ea0114e
[exception RIP: lpfc_nlp_get+210]
RIP: ffffffffc0f60f82 RSP: ff2c9f725c39fcb0 RFLAGS: 00010046
RAX: 0000000000000046 RBX: ff2bd8d8ac056000 RCX: 0000000000fffffc
RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000046
RBP: ff2bd8d8ac056090 R8: 0000000000000000 R9: 0000000000000000
R10: ff2bd90d1f8701c0 R11: 0000000000000001 R12: ff2bd93320482ae0
R13: ff2bd93051a80524 R14: ff2bd93051a80000 R15: ff2bd9332079fc00
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
...snip...
crash> dis -rl ffffffffc0f60f82
0xffffffffc0f60eb0 <lpfc_nlp_get>: nopl 0x0(%rax,%rax,1) [FTRACE NOP]
0xffffffffc0f60eb5 <lpfc_nlp_get+5>: push %rbp
0xffffffffc0f60eb6 <lpfc_nlp_get+6>: push %rbx
0xffffffffc0f60eb7 <lpfc_nlp_get+7>: test %rdi,%rdi
With the patch:
crash> mod -S
crash> bt
PID: 1500 TASK: ff2bd8b093524000 CPU: 16 COMMAND: "lpfc_worker_0"
#0 [ff2c9f725c39f9e0] machine_kexec at ffffffff8e0686d3
...snip...
#7 [ff2c9f725c39fc00] page_fault at ffffffff8ea0114e
[exception RIP: lpfc_nlp_get+210]
RIP: ffffffffc0f60f82 RSP: ff2c9f725c39fcb0 RFLAGS: 00010046
RAX: 0000000000000046 RBX: ff2bd8d8ac056000 RCX: 0000000000fffffc
RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000046
RBP: ff2bd8d8ac056090 R8: 0000000000000000 R9: 0000000000000000
R10: ff2bd90d1f8701c0 R11: 0000000000000001 R12: ff2bd93320482ae0
R13: ff2bd93051a80524 R14: ff2bd93051a80000 R15: ff2bd9332079fc00
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#8 [ff2c9f725c39fcc0] __lpfc_sli_release_iocbq_s4 at ffffffffc0f2f425 [lpfc]
...snip...
crash> dis -rl ffffffffc0f60f82
/usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6756
0xffffffffc0f60eb0 <lpfc_nlp_get>: nopl 0x0(%rax,%rax,1) [FTRACE NOP]
/usr/src/debug/kernel-4.18.0-425.13.1.el8_7/linux-4.18.0-425.13.1.el8_7.x86_64/drivers/scsi/lpfc/lpfc_hbadisc.c: 6759
0xffffffffc0f60eb5 <lpfc_nlp_get+5>: push %rbp
The root cause is, after kernel module been loaded by command mod, the symtable
is not expanded in gdb side. crash command bt or dis will trigger such an
expansion. However the symtable expansion is different for the 2 commands:
The stack trace of "dis -rl" for symtable expanding:
#0 0x00000000008d8d9f in add_compunit_symtab_to_objfile (cu=cu@entry=0xe6a77a0) at symfile.c:2914
#1 0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector (this=<optimized out>, static_block=static_block@entry=0xfbe4b60, section=1, expandable=expandable@entry=0) at buildsym.c:1072
#2 0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block (this=<optimized out>, static_block=static_block@entry=0xfbe4b60, section=<optimized out>, expandable=expandable@entry=0) at buildsym.c:1106
#3 0x000000000077e8e9 in process_full_comp_unit (pretend_language=<optimized out>, cu=0x8ee4c60) at /usr/include/c++/8/bits/unique_ptr.h:716
#4 process_queue (per_objfile=0xc54c870) at dwarf2/read.c:9220
#5 dw2_do_instantiate_symtab (per_cu=<optimized out>, per_objfile=0xc54c870, skip_partial=<optimized out>) at dwarf2/read.c:2448
#6 0x000000000077ed67 in dw2_instantiate_symtab (per_cu=0xdd0a320, per_objfile=0xc54c870, skip_partial=<optimized out>) at dwarf2/read.c:2472
#7 0x000000000077f75e in dw2_expand_all_symtabs (objfile=<optimized out>) at dwarf2/read.c:3768
#8 0x00000000008f254d in gdb_get_line_number (req=0x7fffffffb1f0) at symtab.c:7112
#9 0x00000000008f22af in gdb_command_funnel_1 (req=0x7fffffffb1f0) at symtab.c:7023
#10 0x00000000008f2003 in gdb_command_funnel (req=0x7fffffffb1f0) at symtab.c:6965
#11 0x00000000005b7f02 in gdb_interface (req=req@entry=0x7fffffffb1f0) at gdb_interface.c:409
#12 0x00000000005f5bd8 in get_line_number (addr=18446744072651935408, buf=buf@entry=0x7fffffffd460 "", reserved=reserved@entry=0) at symbols.c:4440
#13 0x000000000059e574 in cmd_dis () at kernel.c:2143
The stack trace of "bt" for symtable expanding:
#0 0x00000000008d8d9f in add_compunit_symtab_to_objfile (cu=cu@entry=0x1ad15630) at symfile.c:2914
#1 0x00000000006d3293 in buildsym_compunit::end_symtab_with_blockvector (this=<optimized out>, static_block=static_block@entry=0x1db0be30, section=1, expandable=expandable@entry=0) at buildsym.c:1072
#2 0x00000000006d336a in buildsym_compunit::end_symtab_from_static_block (this=<optimized out>, static_block=static_block@entry=0x1db0be30, section=<optimized out>, expandable=expandable@entry=0) at buildsym.c:1106
#3 0x000000000077e8e9 in process_full_comp_unit (pretend_language=<optimized out>, cu=0x7465240) at /usr/include/c++/8/bits/unique_ptr.h:716
#4 process_queue (per_objfile=0xc113810) at dwarf2/read.c:9220
#5 dw2_do_instantiate_symtab (per_cu=<optimized out>, per_objfile=0xc113810, skip_partial=<optimized out>) at dwarf2/read.c:2448
#6 0x000000000077ed67 in dw2_instantiate_symtab (per_cu=0xdd069d0, per_objfile=0xc113810, skip_partial=<optimized out>) at dwarf2/read.c:2472
#7 0x000000000077f8ed in dw2_lookup_symbol (objfile=<optimized out>, block_index=STATIC_BLOCK, name=0x7fffffffc890 "cpumask_t", domain=STRUCT_DOMAIN) at dwarf2/read.c:3669
#8 0x00000000008e6d03 in lookup_symbol_via_quick_fns (objfile=0xdd277a0, block_index=STATIC_BLOCK, name=0x7fffffffc890 "cpumask_t", domain=STRUCT_DOMAIN) at symtab.c:2392
#9 0x00000000008e7153 in lookup_symbol_in_objfile (objfile=0xdd277a0, block_index=STATIC_BLOCK, name=0x7fffffffc890 "cpumask_t", domain=STRUCT_DOMAIN) at symtab.c:2541
#10 0x00000000008e73c6 in lookup_symbol_global_or_static_iterator_cb (objfile=0xdd277a0, cb_data=0x7fffffffc470) at symtab.c:2615
#11 0x00000000008b99c4 in svr4_iterate_over_objfiles_in_search_order (gdbarch=<optimized out>, cb=0x8e7342 <lookup_symbol_global_or_static_iterator_cb(objfile*, void*)>, cb_data=0x7fffffffc470, current_objfile=0x0) at solib-svr4.c:3248
#12 0x00000000008e754e in lookup_global_or_static_symbol (name=0x7fffffffc890 "cpumask_t", block_index=STATIC_BLOCK, objfile=0x0, domain=STRUCT_DOMAIN) at symtab.c:2660
#13 0x00000000008e75da in lookup_static_symbol (name=0x7fffffffc890 "cpumask_t", domain=STRUCT_DOMAIN) at symtab.c:2678
#14 0x00000000008e632c in lookup_symbol_aux (name=0x7fffffffc890 "cpumask_t", match_type=symbol_name_match_type::FULL, block=0x0, domain=STRUCT_DOMAIN, language=language_c, is_a_field_of_this=0x0) at symtab.c:2122
#15 0x00000000008e5a7a in lookup_symbol_in_language (name=0x7fffffffc890 "cpumask_t", block=0x0, domain=STRUCT_DOMAIN, lang=language_c, is_a_field_of_this=0x0) at symtab.c:1889
#16 0x00000000008e5b30 in lookup_symbol (name=0x7fffffffc890 "cpumask_t", block=0x0, domain=STRUCT_DOMAIN, is_a_field_of_this=0x0) at symtab.c:1915
#17 0x00000000008f2a4a in gdb_get_datatype (req=0x7fffffffc730) at symtab.c:7229
#18 0x00000000008f22c0 in gdb_command_funnel_1 (req=0x7fffffffc730) at symtab.c:7027
#19 0x00000000008f2003 in gdb_command_funnel (req=0x7fffffffc730) at symtab.c:6965
#20 0x00000000005b7f02 in gdb_interface (req=req@entry=0x7fffffffc730) at gdb_interface.c:409
#21 0x00000000005f8a9f in datatype_info (name=name@entry=0xa8454d "cpumask_t", member=member@entry=0x0, dm=dm@entry=0xfffffffffffffffc) at symbols.c:5715
#22 0x0000000000599947 in cpu_map_size (type=<optimized out>) at kernel.c:913
#23 0x00000000005a975d in get_cpus_online () at kernel.c:9556
#24 0x0000000000637a8b in diskdump_get_prstatus_percpu (cpu=16) at diskdump.c:2277
#25 0x000000000062f0e4 in get_netdump_regs_x86_64 (bt=0x7fffffffd950, ripp=0x7fffffffd130, rspp=0x7fffffffd138) at netdump.c:3471
#26 0x000000000059fe68 in back_trace (bt=bt@entry=0x7fffffffd950) at kernel.c:3092
#27 0x00000000005ab1cb in cmd_bt () at kernel.c:2859
For the stacktrace of "dis -rl", it calls dw2_expand_all_symtabs() to expand
all symtable of the objfile, or "*.ko.debug" in our case. However for
the stacktrace of "bt", it doesn't expand all, but only a subset of symtable
which is enough to find a symbol by dw2_lookup_symbol(). As a result, the
objfile->compunit_symtabs, which is the head of a single linked list of
struct compunit_symtab, is not NULL but didn't contain all symtables. It
will not be reinitialized in gdb_get_line_number() by "dis -rl" because
!objfile_has_full_symbols(objfile) check will fail, so it cannot display
the proper code line number data.
Since objfile_has_full_symbols(objfile) check cannot ensure all symbols
been expanded, this patch add a new member as a flag for struct objfile
to record if all symbols have been expanded. The flag will be set only ofter
expand_all_symtabs been called.
Signed-off-by: Tao Liu <ltao(a)redhat.com>
---
v1 -> v2: new method for kernel module symtabs expansion.
v1: expand all kernel modules symtabs when loading by mod -s/-S
v2: record if a specific kernel module's symtab been all expanded,
and skip re-expansion if true.
---
gdb-10.2.patch | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/gdb-10.2.patch b/gdb-10.2.patch
index d81030d..2f7d585 100644
--- a/gdb-10.2.patch
+++ b/gdb-10.2.patch
@@ -3187,3 +3187,53 @@ exit 0
result = stringtab + symbol_entry->_n._n_n._n_offset;
}
else
+--- gdb-10.2/gdb/objfiles.h.orig
++++ gdb-10.2/gdb/objfiles.h
+@@ -712,6 +712,8 @@ struct objfile
+ next time. If an objfile does not have the symbols, it will
+ never have them. */
+ bool skip_jit_symbol_lookup = false;
++
++ bool all_symtabs_expanded = false;
+ };
+
+ /* A deleter for objfile. */
+--- gdb-10.2/gdb/symfile.c.orig
++++ gdb-10.2/gdb/symfile.c
+@@ -1133,8 +1133,10 @@ symbol_file_add_with_addrs (bfd *abfd, const char *name,
+ printf_filtered (_("Expanding full symbols from %ps...\n"),
+ styled_string (file_name_style.style (), name));
+
+- if (objfile->sf)
++ if (objfile->sf) {
+ objfile->sf->qf->expand_all_symtabs (objfile);
++ objfile->all_symtabs_expanded = true;
++ }
+ }
+
+ /* Note that we only print a message if we have no symbols and have
+--- gdb-10.2/gdb/symtab.c.orig
++++ gdb-10.2/gdb/symtab.c
+@@ -7097,8 +7097,9 @@ gdb_get_line_number(struct gnu_request *req)
+ */
+ if (req->lm) {
+ objfile = req->lm->loaded_objfile;
+- if (!objfile_has_full_symbols(objfile) && objfile->sf) {
++ if (!objfile->all_symtabs_expanded && objfile->sf) {
+ objfile->sf->qf->expand_all_symtabs(objfile);
++ objfile->all_symtabs_expanded = true;
+ sal = find_pc_line(pc, 0);
+ }
+ }
+@@ -7761,8 +7765,10 @@ iterate_datatypes (struct gnu_request *req)
+ {
+ for (objfile *objfile : current_program_space->objfiles ())
+ {
+- if (objfile->sf)
++ if (objfile->sf) {
+ objfile->sf->qf->expand_all_symtabs(objfile);
++ objfile->all_symtabs_expanded = true;
++ }
+
+ for (compunit_symtab *cust : objfile->compunits ())
+ {
--
2.40.1
12 months
Test mail (please ignore)
by liutgnu
For the new crash-utility mailing list, I and lianbo often receive a reject mail claiming: "mx.google.com rejected your message..."
I suspect it is due to the buggy mailing list server, or the misconfiguration of my own mail client. So I send the test mail from a different client.
Please ignore this email and sorry for the bothering.
Thanks,
Tao Liu
12 months
[PATCH] symbols: handle module symbols outside strbuf
by Stephen Brennan
Module symbol names can get overwritten by live patches or ksplice in
odd corner cases, so that the pointer no longer points within the string
buffer. Gracefully fallback to reading the string directly from the
kernel image in these cases, to avoid possible segmentation faults
reading outside the bounds of strbuf.
Signed-off-by: Stephen Brennan <stephen.s.brennan(a)oracle.com>
---
Hi folks - I encountered a segfault on a vmcore which had a module
symbol that had gotten its name overwritten by a ksplice (live patch).
It seems like there's not a guarantee that module symbol names _must_
live within the same symbol buffer, and there is even logic to prevent
reading too much data into strbuf in those cases.
This patch simply ensures that symbol names which start outside of the
strbuf which we copied, are read directly from the kernel image, rather
than indexing past the bounds of strbuf. I encountered this in
store_module_symbols_v2() and have tested it there, but I replicated the
code to the other versions. I will try to test it out on the other
variants as well, but I thought I'd share the patch now.
symbols.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/symbols.c b/symbols.c
index 176c950..e70dd69 100644
--- a/symbols.c
+++ b/symbols.c
@@ -1704,7 +1704,7 @@ store_module_symbols_v1(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && (unsigned long)modsym->name - first < strbuflen)
strcpy(buf1,
&strbuf[(ulong)modsym->name - first]);
else
@@ -2080,7 +2080,7 @@ store_module_symbols_6_4(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(syms, modsym, i) - first < strbuflen)
strcpy(buf1, &strbuf[modsym_name(syms, modsym, i) - first]);
else
read_string(modsym_name(syms, modsym, i), buf1, BUFSIZE-1);
@@ -2148,7 +2148,7 @@ store_module_symbols_6_4(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(gpl_syms, modsym, i) - first < strbuflen)
strcpy(buf1, &strbuf[modsym_name(gpl_syms, modsym, i) - first]);
else
read_string(modsym_name(gpl_syms, modsym, i), buf1, BUFSIZE-1);
@@ -2456,7 +2456,7 @@ store_module_symbols_v2(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(syms, modsym, i) - first < strbuflen)
strcpy(buf1,
&strbuf[modsym_name(syms, modsym, i) - first]);
else
@@ -2529,7 +2529,7 @@ store_module_symbols_v2(ulong total, int mods_installed)
BZERO(buf1, BUFSIZE);
- if (strbuf)
+ if (strbuf && modsym_name(gpl_syms, modsym, i) - first < strbuflen)
strcpy(buf1,
&strbuf[modsym_name(gpl_syms, modsym, i) - first]);
else
--
2.39.3
12 months
