Re: [Crash-utility] [PATCH v2] arm64: fix the "pud page" crash for live system
by lijiang
On Wed, Mar 23, 2022 at 8:00 PM <crash-utility-request(a)redhat.com> wrote:
> Date: Wed, 23 Mar 2022 18:25:48 +0000
> From: Huang Shijie <shijie(a)os.amperecomputing.com>
> To: k-hagio-ab(a)nec.com
> Cc: patches(a)amperecomputing.com, zwang(a)amperecomputing.com,
> darren(a)os.amperecomputing.com, crash-utility(a)redhat.com,
> lijiang(a)redhat.com, Huang Shijie <shijie(a)os.amperecomputing.com>
> Subject: [Crash-utility] [PATCH v2] arm64: fix the "pud page" crash
> for live system
> Message-ID: <20220323182548.301261-1-shijie(a)os.amperecomputing.com>
> Content-Type: text/plain
>
> 1.) When I tested live system with "crash vmlinux /proc/kcore" in kernel
> v5.7,
> I met the following crash issue:
> ........................................
> crash: seek error: kernel virtual address: ffff75e9fffff000 type:
> "pud page"
> ........................................
>
> 2.) The root cause is the PTOV does not work correctly for some kernel,
> and then arm64_vtop_4level_4k() does not work correctly too.
>
> Why PTOV does not work?
>
> Because the physvirt_offset does not get the correct value.
>
> 3.) This patch uses symbol_value_from_proc_kallsyms() to get the
> virtual address of "physvirt_offset", and then uses
> READMEM(,..,KCORE_USE_VADDR) to get the correct value of
> "physvirt_offset".
>
> And also updates the ms->phys_offset which is initialized with
> a wrong value in kernel version [5.4, 5.10).
>
> Also add more comments for arm64_calc_physvirt_offset().
>
> Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
> ---
> v1 --> v2:
> v1 tried to get the correct value for phys_offset,
> and then get the physvirt_offset correctly.
>
> v2 tried to get the physvirt_offset correctly,
> and them update phys_offset correctly.
>
> Tested this patch with kernel 5.7.
>
Thank you for the update, Shijie.
Applied(with Kazu's fix) and I also modified the patch log:
https://github.com/crash-utility/crash/commit/8827424f2b05587b8aaaeb7aae0...
Thanks.
Lianbo
---
> arm64.c | 28 ++++++++++++++++++++++++++++
> 1 file changed, 28 insertions(+)
>
> diff --git a/arm64.c b/arm64.c
> index b843032..60ae644 100644
> --- a/arm64.c
> +++ b/arm64.c
> @@ -1399,15 +1399,43 @@ arm64_calc_kimage_voffset(void)
> ms->kimage_voffset += (kt->relocate * -1);
> }
>
> +/*
> + * The physvirt_offset only exits in kernel [5.4, 5.10)
> + *
> + * 1) In kernel v5.4, the patch:
> + * "5383cc6efed137 arm64: mm: Introduce vabits_actual"
> + *
> + * introduced the physvirt_offset.
> + *
> + * 2) In kernel v5.10, the patch:
> + * "7bc1a0f9e17658 arm64: mm: use single quantity
> + * to represent the PA to VA translation"
> + * removed the physvirt_offset.
> + */
> static void
> arm64_calc_physvirt_offset(void)
> {
> struct machine_specific *ms = machdep->machspec;
> ulong physvirt_offset;
> struct syment *sp;
> + ulong value;
>
> if ((sp = kernel_symbol_search("physvirt_offset")) &&
> machdep->machspec->kimage_voffset) {
> + if (pc->flags & PROC_KCORE) {
> + value =
> symbol_value_from_proc_kallsyms("physvirt_offset");
> + if ((value != BADVAL) &&
> + (READMEM(pc->mfd, &physvirt_offset,
> sizeof(ulong),
> + value, KCORE_USE_VADDR) > 0)) {
> + machdep->flags |= HAS_PHYSVIRT_OFFSET;
> + ms->physvirt_offset = physvirt_offset;
> +
> + /* Update the ms->phys_offset which is
> wrong */
> + ms->phys_offset = ms->physvirt_offset +
> ms->page_offset;
> + return;
> + }
> + }
> +
> if (READMEM(pc->mfd, &physvirt_offset,
> sizeof(physvirt_offset),
> sp->value, sp->value -
> machdep->machspec->kimage_voffset) > 0) {
> --
> 2.30.2
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> --
> Crash-utility mailing list
> Crash-utility(a)redhat.com
> https://listman.redhat.com/mailman/listinfo/crash-utility
>
>
> ------------------------------
>
> End of Crash-utility Digest, Vol 198, Issue 27
> **********************************************
>
>
2 years, 8 months
Re: [Crash-utility] [PATCH] kernel: fix start-up time degradation caused by strings command
by lijiang
On Wed, Mar 23, 2022 at 4:11 PM <crash-utility-request(a)redhat.com> wrote:
> Date: Wed, 23 Mar 2022 08:09:49 +0000
> From: "d.hatayama(a)fujitsu.com" <d.hatayama(a)fujitsu.com>
> To: "crash-utility(a)redhat.com" <crash-utility(a)redhat.com>
> Subject: [Crash-utility] [PATCH] kernel: fix start-up time degradation
> caused by strings command
> Message-ID:
> <
> TYAPR01MB6507E11837E16B365D7743B695189(a)TYAPR01MB6507.jpnprd01.prod.outlook.com
> >
>
> Content-Type: text/plain; charset="iso-2022-jp"
>
> verify_namelist() uses strings command and scans full part of vmlinux
> file to find linux_banner string. However, vmlinux file is quite large
> these days, reaching over 500MB. As a result, this degradates start-up
> time of crash command 10 or more seconds. (Of course, this depends on
> machines you use for investigation, but I guess typically we cannot
> use such powerful machines to investigate crash dump...)
>
> To resolve this issue, let's use bfd library and read linux_banner
> string in vmlinux file directly.
>
> A simple benchmark shows the following result:
>
> Without the fix:
>
> # cat ./commands.txt
> quit
> # time ./crash -i ./commands.txt \
> /usr/lib/debug/lib/modules/5.16.15-201.fc35.x86_64/vmlinux \
> /var/crash/*/vmcore >/dev/null 2>&1
>
> real 0m20.251s
> user 0m19.022s
> sys 0m1.054s
>
> With the fix:
>
> # time ./crash -i ./commands.txt \
> /usr/lib/debug/lib/modules/5.16.15-201.fc35.x86_64/vmlinux \
> /var/crash/*/vmcore >/dev/null 2>&1
>
> real 0m6.528s
> user 0m6.143s
> sys 0m0.431s
>
This looks pretty good, thank you for the improvement, Hatayama.
Applied:
https://github.com/crash-utility/crash/commit/cd8954023bd474521a9d45e2b09...
BTW: Currently, crash performance issues may become more and more
prominent on multi-core and large memory systems, hope to have more
optimization work in the future.
Thanks.
Lianbo
> Note that this commit keeps the original logic that uses strings
> command for backward compatibility for in case.
>
> Signed-off-by: HATAYAMA Daisuke <d.hatayama(a)fujitsu.com>
> ---
> Makefile | 2 +-
> kernel.c | 41 +++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 42 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 007d030..e520b12 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -364,7 +364,7 @@ task.o: ${GENERIC_HFILES} task.c
> ${CC} -c ${CRASH_CFLAGS} task.c ${WARNING_OPTIONS} ${WARNING_ERROR}
>
> kernel.o: ${GENERIC_HFILES} kernel.c
> - ${CC} -c ${CRASH_CFLAGS} kernel.c ${WARNING_OPTIONS}
> ${WARNING_ERROR}
> + ${CC} -c ${CRASH_CFLAGS} kernel.c -I${BFD_DIRECTORY}
> -I${GDB_INCLUDE_DIRECTORY} ${WARNING_OPTIONS} ${WARNING_ERROR}
>
> printk.o: ${GENERIC_HFILES} printk.c
> ${CC} -c ${CRASH_CFLAGS} printk.c ${WARNING_OPTIONS}
> ${WARNING_ERROR}
> diff --git a/kernel.c b/kernel.c
> index 1c63447..92434a3 100644
> --- a/kernel.c
> +++ b/kernel.c
> @@ -23,6 +23,11 @@
> #include <ctype.h>
> #include <stdbool.h>
> #include "xendump.h"
> +#if defined(GDB_7_6) || defined(GDB_10_2)
> +#define __CONFIG_H__ 1
> +#include "config.h"
> +#endif
> +#include "bfd.h"
>
> static void do_module_cmd(ulong, char *, ulong, char *, char *);
> static void show_module_taint(void);
> @@ -97,6 +102,7 @@ static void dump_printk_safe_seq_buf(int);
> static char *vmcoreinfo_read_string(const char *);
> static void check_vmcoreinfo(void);
> static int is_pvops_xen(void);
> +static int get_linux_banner_from_vmlinux(char *, size_t);
>
>
> /*
> @@ -1324,6 +1330,12 @@ verify_namelist()
> target_smp = strstr(kt->utsname.version, " SMP ") ? TRUE : FALSE;
> namelist_smp = FALSE;
>
> + if (get_linux_banner_from_vmlinux(buffer, sizeof(buffer)) &&
> + strstr(buffer, kt->proc_version)) {
> + found = TRUE;
> + goto found;
> + }
> +
> sprintf(command, "/usr/bin/strings %s", namelist);
> if ((pipe = popen(command, "r")) == NULL) {
> error(INFO, "%s: %s\n", namelist, strerror(errno));
> @@ -1384,6 +1396,7 @@ verify_namelist()
> }
> }
>
> +found:
> if (found) {
> if (CRASHDEBUG(1)) {
> fprintf(fp, "verify_namelist:\n");
> @@ -11770,3 +11783,31 @@ check_vmcoreinfo(void)
> }
> }
> }
> +
> +static
> +int get_linux_banner_from_vmlinux(char *buf, size_t size)
> +{
> + struct bfd_section *sect;
> + long offset;
> +
> + sect = bfd_get_section_by_name(st->bfd, ".rodata");
> + if (!sect)
> + return FALSE;
> +
> + /*
> + * Although symbol_value() returns dynamic symbol value that
> + * is affected by kaslr, which is different from static symbol
> + * value in vmlinux file, but relative offset to linux_banner
> + * object in .rodata section is idential.
> + */
> + offset = symbol_value("linux_banner") - symbol_value(".rodata");
> +
> + if (!bfd_get_section_contents(st->bfd,
> + sect,
> + buf,
> + offset,
> + size))
> + return FALSE;
> +
> + return TRUE;
> +}
> --
> 2.31.1
>
2 years, 8 months
[PATCH v2] arm64: fix the "pud page" crash for live system
by Huang Shijie
1.) When I tested live system with "crash vmlinux /proc/kcore" in kernel v5.7,
I met the following crash issue:
........................................
crash: seek error: kernel virtual address: ffff75e9fffff000 type: "pud page"
........................................
2.) The root cause is the PTOV does not work correctly for some kernel,
and then arm64_vtop_4level_4k() does not work correctly too.
Why PTOV does not work?
Because the physvirt_offset does not get the correct value.
3.) This patch uses symbol_value_from_proc_kallsyms() to get the
virtual address of "physvirt_offset", and then uses
READMEM(,..,KCORE_USE_VADDR) to get the correct value of
"physvirt_offset".
And also updates the ms->phys_offset which is initialized with
a wrong value in kernel version [5.4, 5.10).
Also add more comments for arm64_calc_physvirt_offset().
Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
---
v1 --> v2:
v1 tried to get the correct value for phys_offset,
and then get the physvirt_offset correctly.
v2 tried to get the physvirt_offset correctly,
and them update phys_offset correctly.
Tested this patch with kernel 5.7.
---
arm64.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/arm64.c b/arm64.c
index b843032..60ae644 100644
--- a/arm64.c
+++ b/arm64.c
@@ -1399,15 +1399,43 @@ arm64_calc_kimage_voffset(void)
ms->kimage_voffset += (kt->relocate * -1);
}
+/*
+ * The physvirt_offset only exits in kernel [5.4, 5.10)
+ *
+ * 1) In kernel v5.4, the patch:
+ * "5383cc6efed137 arm64: mm: Introduce vabits_actual"
+ *
+ * introduced the physvirt_offset.
+ *
+ * 2) In kernel v5.10, the patch:
+ * "7bc1a0f9e17658 arm64: mm: use single quantity
+ * to represent the PA to VA translation"
+ * removed the physvirt_offset.
+ */
static void
arm64_calc_physvirt_offset(void)
{
struct machine_specific *ms = machdep->machspec;
ulong physvirt_offset;
struct syment *sp;
+ ulong value;
if ((sp = kernel_symbol_search("physvirt_offset")) &&
machdep->machspec->kimage_voffset) {
+ if (pc->flags & PROC_KCORE) {
+ value = symbol_value_from_proc_kallsyms("physvirt_offset");
+ if ((value != BADVAL) &&
+ (READMEM(pc->mfd, &physvirt_offset, sizeof(ulong),
+ value, KCORE_USE_VADDR) > 0)) {
+ machdep->flags |= HAS_PHYSVIRT_OFFSET;
+ ms->physvirt_offset = physvirt_offset;
+
+ /* Update the ms->phys_offset which is wrong */
+ ms->phys_offset = ms->physvirt_offset + ms->page_offset;
+ return;
+ }
+ }
+
if (READMEM(pc->mfd, &physvirt_offset, sizeof(physvirt_offset),
sp->value, sp->value -
machdep->machspec->kimage_voffset) > 0) {
--
2.30.2
2 years, 8 months
Replace IPv4 functions with IPv6-capable functions
by David Cantrell
Discovered during build scans using rpminspect of this software in
Fedora, CentOS, and RHEL. One thing rpminspect checks for is for any
software using functions from a runtime-configurable list of
"forbidden functions". A long standing goal is to ensure software
built and shipped in these distributions does not make use of
IPv4-only functions unless it is absolutely required.
Here are two patches to address the use of the IPv4-only functions in
this project.
(I am not a member of this mailing list, so for follow-ups make sure
my email address is included.)
Thanks,
--
David Cantrell <dcantrell(a)redhat.com>
Red Hat, Inc. | Boston, MA | EST5EDT
2 years, 8 months
How to interpret seek error on type memory section ?
by Agrain Patrick
Hello,
The system is running a 4.18 (CentOS 8) kernel based on the CentOS 8.0 distribution.
I tested a dummy crash (echo h > /proc/sysrq-trigger) with both kexec-tools 2.0.19 and 2.0.20 packages.
The dump process seems to go to the end without complaining with following command:
/sbin/makedumpfile -D -d 0 -c --message-level 7 /proc/vmcore /tmpd/crashdump-${linux_ver}-${date_time}
I transfer the vmcore on a CentOS 8 machine running crash-7.2.9.
Following message appears during crash reading the core file:
crash: seek error: kernel virtual address: ffffffff8249c441 type: "memory section"
I'm a bit lost, because this mechanism was working on that type of hardware. What changed ? In the meanwhile, we switched from SysVinit to systemD. But I cannot imagine any impact.
Where should I look at ?
Thanks a lot.
Best regards,
Patrick Agrain
2 years, 8 months
[PATCH 1/2] Add a new helper arm64_get_pkd
by Huang Shijie
Use this new helper to get the pkd which
may be used in future.
Also move struct proc_kcore_data to defs.h to
avoid the compiler errors.
Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
---
defs.h | 16 ++++++++++++++++
netdump.c | 5 +++++
netdump.h | 12 ------------
3 files changed, 21 insertions(+), 12 deletions(-)
diff --git a/defs.h b/defs.h
index 81ac049..4f06bdf 100644
--- a/defs.h
+++ b/defs.h
@@ -39,6 +39,7 @@
#include <signal.h>
#include <assert.h>
#include <errno.h>
+#include <elf.h>
#include <dirent.h>
#include <time.h>
#include <zlib.h>
@@ -6647,6 +6648,20 @@ int sparc64_vmalloc_addr(ulong);
/*
* netdump.c
*/
+
+struct proc_kcore_data {
+ uint flags;
+ uint segments;
+ char *elf_header;
+ size_t header_size;
+ Elf64_Phdr *load64;
+ Elf64_Phdr *notes64;
+ Elf32_Phdr *load32;
+ Elf32_Phdr *notes32;
+ void *vmcoreinfo;
+ uint size_vmcoreinfo;
+};
+
int is_netdump(char *, ulong);
uint netdump_page_size(void);
int read_netdump(int, void *, int, ulong, physaddr_t);
@@ -6689,6 +6704,7 @@ int kdump_phys_base(ulong *);
int kdump_set_phys_base(ulong);
int arm_kdump_phys_base(ulong *);
int arm_kdump_phys_end(ulong *);
+struct proc_kcore_data *arm64_get_pkd(void);
int is_proc_kcore(char *, ulong);
int proc_kcore_init(FILE *, int);
int read_proc_kcore(int, void *, int, ulong, physaddr_t);
diff --git a/netdump.c b/netdump.c
index ff273b4..3066c80 100644
--- a/netdump.c
+++ b/netdump.c
@@ -3864,6 +3864,11 @@ get_netdump_regs_arm64(struct bt_info *bt, ulong *eip, ulong *esp)
machdep->get_stack_frame(bt, eip, esp);
}
+struct proc_kcore_data *arm64_get_pkd(void)
+{
+ return pkd;
+}
+
static void
get_netdump_regs_mips(struct bt_info *bt, ulong *eip, ulong *esp)
{
diff --git a/netdump.h b/netdump.h
index ffd60b8..119cb72 100644
--- a/netdump.h
+++ b/netdump.h
@@ -148,15 +148,3 @@ struct vmcore_data {
#define MAX_KCORE_ELF_HEADER_SIZE (32768)
-struct proc_kcore_data {
- uint flags;
- uint segments;
- char *elf_header;
- size_t header_size;
- Elf64_Phdr *load64;
- Elf64_Phdr *notes64;
- Elf32_Phdr *load32;
- Elf32_Phdr *notes32;
- void *vmcoreinfo;
- uint size_vmcoreinfo;
-};
--
2.30.2
2 years, 8 months
[PATCH] arm64: fix the wrong vmemmap_end
by Huang Shijie
Before the kernel v5.7, the VMEMMAP_END does not exit.
In the patch
"e397e1b arm64: update the modules/vmalloc/vmemmap ranges"
it sets -1(0xffffffffffffffffUL) for vmemmap_end for
the kernel versions before v5.7.
But from the arch/arm64/mm/dump.c (before kernel v5.7):
......................................
{ VMEMMAP_START + VMEMMAP_SIZE, "vmemmap end" }
......................................
we can know that the vmemmap_end should always be:
vmemmap_end = vmemmap_vaddr + vmemmap_size;
This patch fixes this issue.
Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
---
arm64.c | 26 +++++++++++---------------
1 file changed, 11 insertions(+), 15 deletions(-)
diff --git a/arm64.c b/arm64.c
index d81c2b9..f75599e 100644
--- a/arm64.c
+++ b/arm64.c
@@ -285,11 +285,7 @@ arm64_init(int when)
ms->vmalloc_start_addr = r->vmalloc_start_addr;
ms->vmalloc_end = r->vmalloc_end - 1;
ms->vmemmap_vaddr = r->vmemmap_vaddr;
- if (THIS_KERNEL_VERSION >= LINUX(5, 7, 0))
- ms->vmemmap_end = r->vmemmap_end - 1;
- else
- ms->vmemmap_end = -1;
-
+ ms->vmemmap_end = r->vmemmap_end - 1;
} else if (ms->VA_BITS_ACTUAL) {
ms->modules_vaddr = (st->_stext_vmlinux & TEXT_OFFSET_MASK) - ARM64_MODULES_VSIZE;
ms->modules_end = ms->modules_vaddr + ARM64_MODULES_VSIZE -1;
@@ -739,15 +735,15 @@ static struct kernel_range *arm64_get_range_v5_4(struct machine_specific *ms)
vmemmap_size = (_PAGE_END(v) - PAGE_OFFSET) >> vmem_shift;
r->vmemmap_vaddr = (-vmemmap_size - SZ_2M);
- if (THIS_KERNEL_VERSION >= LINUX(5, 7, 0)) {
- /*
- * In the v5.7, the patch: "bbd6ec605c arm64/mm: Enable memory hot remove"
- * adds the VMEMMAP_END.
- */
- r->vmemmap_end = r->vmemmap_vaddr + vmemmap_size;
- } else {
- r->vmemmap_end = 0xffffffffffffffffUL;
- }
+ /*
+ * In the v5.7, the patch: "bbd6ec605c arm64/mm: Enable memory hot remove"
+ * adds the VMEMMAP_END.
+ *
+ * But before the VMEMMAP_END was added to kernel, we can also see
+ * the following in arch/arm64/mm/dump.c:
+ * { VMEMMAP_START + VMEMMAP_SIZE, "vmemmap end" },
+ */
+ r->vmemmap_end = r->vmemmap_vaddr + vmemmap_size;
/* Get the VMALLOC_START ~ VMALLOC_END */
PUD_SIZE = arm64_get_pud_size();
@@ -795,7 +791,7 @@ static struct kernel_range *arm64_get_range_v5_0(struct machine_specific *ms)
vmemmap_size = (1UL << (v - machdep->pageshift - 1 + arm64_get_struct_page_max_shift(ms)));
r->vmemmap_vaddr = page_offset - vmemmap_size;
- r->vmemmap_end = 0xffffffffffffffffUL; /* this kernel does not have VMEMMAP_END */
+ r->vmemmap_end = r->vmemmap_vaddr + vmemmap_size; /* See the arch/arm64/mm/dump.c */
/* Get the VMALLOC_START ~ VMALLOC_END */
PUD_SIZE = arm64_get_pud_size();
--
2.30.2
2 years, 8 months
[PATCH v2] arm64: use the vmcore info to get module/vmalloc/vmemmap ranges
by Huang Shijie
The kernel patch has been merged:
"arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP ranges"
https://git.kernel.org/arm64/c/2369f171d5c5
and it will be available in v5.18-rc1.
So we can get module/vmalloc/vmemmap ranges from the vmcore info
since the v5.18, and we have no need to calculate them by hand
any more.
This patch adds a new hook arm64_get_range_v5_18 which could parse
out all the module/vmalloc/vmemmap ranges from the vmcore info.
Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
---
v1-->v2:
1.) Change return types to TRUE/FALSE
for arm64_get_vmcoreinfo_ul().
2.) Change version limit from LINUX(99,0,0) to LINUX(999, 0, 0)
I guess LINUX(99, 0, 0) can be used for 25 years,
so I extended to LINUX(999,0,0).
3.) Tested this patch with the live system mode.
(linux-next kernel + Crash)
---
arm64.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 47 insertions(+), 2 deletions(-)
diff --git a/arm64.c b/arm64.c
index 2dcef46..e3fa316 100644
--- a/arm64.c
+++ b/arm64.c
@@ -560,6 +560,48 @@ static int arm64_get_struct_page_max_shift(struct machine_specific *ms)
return (int)ceil(log2(ms->struct_page_size));
}
+/* Return TRUE if we succeed, return FALSE on failure. */
+static int arm64_get_vmcoreinfo_ul(unsigned long *vaddr, const char* label)
+{
+ char *string = pc->read_vmcoreinfo(label);
+
+ if (!string)
+ return FALSE;
+
+ *vaddr = strtoul(string, NULL, 0);
+ free(string);
+ return TRUE;
+}
+
+/*
+ * The change is caused by the kernel patch since v5.18-rc1:
+ * "arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP ranges"
+ */
+static struct kernel_range *arm64_get_range_v5_18(struct machine_specific *ms)
+{
+ struct kernel_range *r = &tmp_range;
+
+ /* Get the MODULES_VADDR ~ MODULES_END */
+ if (!arm64_get_vmcoreinfo_ul(&r->modules_vaddr, "NUMBER(MODULES_VADDR)"))
+ return NULL;
+ if (!arm64_get_vmcoreinfo_ul(&r->modules_end, "NUMBER(MODULES_END)"))
+ return NULL;
+
+ /* Get the VMEMMAP_START ~ VMEMMAP_END */
+ if (!arm64_get_vmcoreinfo_ul(&r->vmemmap_vaddr, "NUMBER(VMEMMAP_START)"))
+ return NULL;
+ if (!arm64_get_vmcoreinfo_ul(&r->vmemmap_end, "NUMBER(VMEMMAP_END)"))
+ return NULL;
+
+ /* Get the VMALLOC_START ~ VMALLOC_END */
+ if (!arm64_get_vmcoreinfo_ul(&r->vmalloc_start_addr, "NUMBER(VMALLOC_START)"))
+ return NULL;
+ if (!arm64_get_vmcoreinfo_ul(&r->vmalloc_end, "NUMBER(VMALLOC_END)"))
+ return NULL;
+
+ return r;
+}
+
/*
* The change is caused by the kernel patch since v5.17-rc1:
* "b89ddf4cca43 arm64/bpf: Remove 128MB limit for BPF JIT programs"
@@ -765,8 +807,11 @@ static struct kernel_range *arm64_get_range_v5_0(struct machine_specific *ms)
static struct kernel_va_range_handler kernel_va_range_handlers[] = {
{
- LINUX(5,17,0),
- LINUX(99,0,0), /* Just a boundary, Change it later */
+ LINUX(5,18,0),
+ LINUX(999,0,0), /* Just a boundary */
+ get_range: arm64_get_range_v5_18,
+ }, {
+ LINUX(5,17,0), LINUX(5,18,0),
get_range: arm64_get_range_v5_17,
}, {
LINUX(5,11,0), LINUX(5,17,0),
--
2.30.2
2 years, 8 months
Re: [Crash-utility] [PATCH v3] ps: Add support to "ps -l|-m" to properly display process
by lijiang
On Thu, Mar 3, 2022 at 10:43 AM <crash-utility-request(a)redhat.com> wrote:
> Date: Wed, 2 Mar 2022 21:37:45 +0000
> From: Austin Kim <austindh.kim(a)gmail.com>
> To: k-hagio-ab(a)nec.com, lijiang(a)redhat.com, crash-utility(a)redhat.com
> Cc: kernel-team(a)lge.com, mikeseohyungjin(a)gmail.com
> Subject: [Crash-utility] [PATCH v3] ps: Add support to "ps -l|-m" to
> properly display process
> Message-ID: <20220302213745.GA868@raspberrypi>
> Content-Type: text/plain; charset=us-ascii
>
> Sometimes kernel image is generated without CONFIG_SCHEDSTATS or
> CONFIG_SCHED_INFO.
> Where relevant commit id is f6db83479932 ("sched/stat: Simplify the
> sched_info accounting")
>
> - CONFIG_SCHED_INFO: KERNEL_VERSION >= LINUX(4,2,0)
> - CONFIG_SCHEDSTATS: KERNEL_VERSION < LINUX(4,2,0)
>
> Running crash-utility with above kernel image,
> "ps -l" option cannot display all processes sorted with most recently-run
> process.
> Also "ps -m" option cannot display all processes with timestamp.
>
> crash> ps -l or crash> ps -m
> ps: last-run timestamps do not exist in this kernel
> Usage: ps [-k|-u|-G] [-s]
> [-p|-c|-t|-[l|m][-C cpu]|-a|-g|-r|-S]
> [pid | task | command] ...
> Enter "help ps" for details.
>
> This is because output of "ps -l|-m" depends on
> task_struct.sched_info.last_arrival.
>
> Without CONFIG_SCHEDSTATS or CONFIG_SCHED_INFO, 'sched_info' field is not
> included
> in task_struct. In this case we make "ps -l|-m" option to access
> 'exec_start'
> field of sched_entity where 'exec_start' is task_struct.se.exec_start.
>
> The 'task_struct.se.exec_start' contains the most recently-executed
> timestamp
> when process is running in the below cases;
>
> - enqueued to runqueue
> - dequeued from runqueue
> - scheduler tick is invoked
> - etc
>
> 'task_struct.se.exec_start' could be one of statistics which indicates the
> most
> recently-run timestamp of process activity.
>
Thank you for the update, Austin.
If the task is migraged, the value of the 'task_struct.se.exec_start ' will
be set to zero and may be updated in the next scheduling with the new time.
In this case, does the patch still work well as expected?
Thanks.
Lianbo
With this patch, "ps -l|-m" option works well without CONFIG_SCHEDSTATS or
> CONFIG_SCHED_INFO.
>
> Signed-off-by: Austin Kim <austindh.kim(a)gmail.com>
> ---
> defs.h | 1 +
> help.c | 5 +++--
> symbols.c | 2 ++
> task.c | 20 ++++++++++++++++----
> 4 files changed, 22 insertions(+), 6 deletions(-)
>
> diff --git a/defs.h b/defs.h
> index bf2c59b..841bd0b 100644
> --- a/defs.h
> +++ b/defs.h
> @@ -2168,6 +2168,7 @@ struct offset_table { /* stash of
> commonly-used offsets */
> long sbitmap_queue_min_shallow_depth;
> long sbq_wait_state_wait_cnt;
> long sbq_wait_state_wait;
> + long sched_entity_exec_start;
> };
>
> struct size_table { /* stash of commonly-used sizes */
> diff --git a/help.c b/help.c
> index 8347668..6ca7c92 100644
> --- a/help.c
> +++ b/help.c
> @@ -1442,7 +1442,8 @@ char *help_ps[] = {
> " and system times.",
> " -l display the task's last-run timestamp value, using either
> the",
> " task_struct's last_run value, the task_struct's timestamp
> value",
> -" or the task_struct's sched_entity last_arrival value,
> whichever",
> +" the task_struct's sched_info last_arrival value",
> +" or the task_struct's sched_entity exec_start value,
> whichever",
> " applies, of selected, or all, tasks; the list is sorted with
> the",
> " most recently-run task (with the largest timestamp) shown
> first,",
> " followed by the task's current state.",
> @@ -1621,7 +1622,7 @@ char *help_ps[] = {
> " > 9497 1 0 ffff880549ec2ab0 RU 0.0 42314692 138664
> oracle",
> " ",
> " Show all tasks sorted by their task_struct's last_run, timestamp, or",
> -" sched_entity last_arrival timestamp value, whichever applies:\n",
> +" sched_info last_arrival or sched_entity exec_start timestamp value,
> whichever applies:\n",
> " %s> ps -l",
> " [20811245123] [IN] PID: 37 TASK: f7153030 CPU: 2 COMMAND:
> \"events/2\"",
> " [20811229959] [IN] PID: 1756 TASK: f2a5a570 CPU: 2 COMMAND:
> \"ntpd\"",
> diff --git a/symbols.c b/symbols.c
> index ba5e274..1c40586 100644
> --- a/symbols.c
> +++ b/symbols.c
> @@ -10290,6 +10290,8 @@ dump_offset_table(char *spec, ulong makestruct)
> OFFSET(sched_entity_my_q));
> fprintf(fp, " sched_entity_on_rq: %ld\n",
> OFFSET(sched_entity_on_rq));
> + fprintf(fp, " sched_entity_exec_start: %ld\n",
> + OFFSET(sched_entity_exec_start));
> fprintf(fp, " cfs_rq_nr_running: %ld\n",
> OFFSET(cfs_rq_nr_running));
> fprintf(fp, " cfs_rq_rb_leftmost: %ld\n",
> diff --git a/task.c b/task.c
> index 864c838..2c12196 100644
> --- a/task.c
> +++ b/task.c
> @@ -334,9 +334,15 @@ task_init(void)
> if (VALID_MEMBER(task_struct_sched_info))
> MEMBER_OFFSET_INIT(sched_info_last_arrival,
> "sched_info", "last_arrival");
> + MEMBER_OFFSET_INIT(task_struct_se, "task_struct", "se");
> + if (VALID_MEMBER(task_struct_se)) {
> + STRUCT_SIZE_INIT(sched_entity, "sched_entity");
> + MEMBER_OFFSET_INIT(sched_entity_exec_start,
> "sched_entity", "exec_start");
> + }
> if (VALID_MEMBER(task_struct_last_run) ||
> VALID_MEMBER(task_struct_timestamp) ||
> - VALID_MEMBER(sched_info_last_arrival)) {
> + VALID_MEMBER(sched_info_last_arrival) ||
> + VALID_MEMBER(sched_entity_exec_start)) {
> char buf[BUFSIZE];
> strcpy(buf, "alias last ps -l");
> alias_init(buf);
> @@ -3559,7 +3565,8 @@ cmd_ps(void)
> case 'm':
> if (INVALID_MEMBER(task_struct_last_run) &&
> INVALID_MEMBER(task_struct_timestamp) &&
> - INVALID_MEMBER(sched_info_last_arrival)) {
> + INVALID_MEMBER(sched_info_last_arrival) &&
> + INVALID_MEMBER(sched_entity_exec_start)) {
> error(INFO,
> "last-run timestamps do not exist in this
> kernel\n");
> argerrs++;
> @@ -3574,7 +3581,8 @@ cmd_ps(void)
> case 'l':
> if (INVALID_MEMBER(task_struct_last_run) &&
> INVALID_MEMBER(task_struct_timestamp) &&
> - INVALID_MEMBER(sched_info_last_arrival)) {
> + INVALID_MEMBER(sched_info_last_arrival) &&
> + INVALID_MEMBER(sched_entity_exec_start)) {
> error(INFO,
> "last-run timestamps do not exist in this
> kernel\n");
> argerrs++;
> @@ -6020,7 +6028,11 @@ task_last_run(ulong task)
> timestamp = tt->last_task_read ?
> ULONGLONG(tt->task_struct +
> OFFSET(task_struct_sched_info) +
> OFFSET(sched_info_last_arrival)) : 0;
> -
> + else if (VALID_MEMBER(sched_entity_exec_start))
> + timestamp = tt->last_task_read ?
> ULONGLONG(tt->task_struct +
> + OFFSET(task_struct_se) +
> + OFFSET(sched_entity_exec_start)) : 0;
> +
> return timestamp;
> }
>
> --
> 2.20.1
>
2 years, 8 months
[PATCH] arm64: use the vmcore info to get module/vmalloc/vmemmap ranges
by Huang Shijie
The kernel patch has been merged:
"arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP ranges"
https://git.kernel.org/arm64/c/2369f171d5c5
and it will be available in v5.18-rc1.
So we can get module/vmalloc/vmemmap ranges from the vmcore info
since the v5.18, and we have no need to calculate them by hand
any more.
This patch adds a new hook arm64_get_range_v5_18 which could parse
out all the module/vmalloc/vmemmap ranges from the vmcore info.
Signed-off-by: Huang Shijie <shijie(a)os.amperecomputing.com>
---
arm64.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 47 insertions(+), 2 deletions(-)
diff --git a/arm64.c b/arm64.c
index 841016c..b91938e 100644
--- a/arm64.c
+++ b/arm64.c
@@ -569,6 +569,48 @@ static int arm64_get_struct_page_max_shift(struct machine_specific *ms)
return 0;
}
+/* Return 0 if we succeed, return 1 on failure. */
+static int arm64_get_vmcoreinfo_ul(unsigned long *vaddr, const char* label)
+{
+ char *string = pc->read_vmcoreinfo(label);
+
+ if (!string)
+ return 1;
+
+ *vaddr = strtoul(string, NULL, 0);
+ free(string);
+ return 0;
+}
+
+/*
+ * The change is caused by the kernel patch since v5.18-rc1:
+ * "arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP ranges"
+ */
+static struct kernel_range *arm64_get_range_v5_18(struct machine_specific *ms)
+{
+ struct kernel_range *r = &tmp_range;
+
+ /* Get the MODULES_VADDR ~ MODULES_END */
+ if (arm64_get_vmcoreinfo_ul(&r->modules_vaddr, "NUMBER(MODULES_VADDR)"))
+ return NULL;
+ if (arm64_get_vmcoreinfo_ul(&r->modules_end, "NUMBER(MODULES_END)"))
+ return NULL;
+
+ /* Get the VMEMMAP_START ~ VMEMMAP_END */
+ if (arm64_get_vmcoreinfo_ul(&r->vmemmap_vaddr, "NUMBER(VMEMMAP_START)"))
+ return NULL;
+ if (arm64_get_vmcoreinfo_ul(&r->vmemmap_end, "NUMBER(VMEMMAP_END)"))
+ return NULL;
+
+ /* Get the VMALLOC_START ~ VMALLOC_END */
+ if (arm64_get_vmcoreinfo_ul(&r->vmalloc_start_addr, "NUMBER(VMALLOC_START)"))
+ return NULL;
+ if (arm64_get_vmcoreinfo_ul(&r->vmalloc_end, "NUMBER(VMALLOC_END)"))
+ return NULL;
+
+ return r;
+}
+
/*
* The change is caused by the kernel patch since v5.17-rc1:
* "b89ddf4cca43 arm64/bpf: Remove 128MB limit for BPF JIT programs"
@@ -774,8 +816,11 @@ static struct kernel_range *arm64_get_range_v5_0(struct machine_specific *ms)
static struct kernel_va_range_handler kernel_va_range_handlers[] = {
{
- LINUX(5,17,0),
- LINUX(99,0,0), /* Just a boundary, Change it later */
+ LINUX(5,18,0),
+ LINUX(99,0,0), /* Just a boundary */
+ get_range: arm64_get_range_v5_18,
+ }, {
+ LINUX(5,17,0), LINUX(5,18,0),
get_range: arm64_get_range_v5_17,
}, {
LINUX(5,11,0), LINUX(5,17,0),
--
2.30.2
2 years, 8 months
