[ANNOUNCE] crash-8.0.3 is available
by HAGIO KAZUHITO(萩尾 一仁)
Download from: https://crash-utility.github.io/
or
https://github.com/crash-utility/crash/releases
The GitHub master branch serves as a development branch that will contain
all patches that are queued for the next release:
$ git clone https://github.com/crash-utility/crash.git
Changelog:
faed05021873 crash-8.0.2 -> crash-8.0.3
47216437e79a Fix "net" command on kernel configured with CONFIG_IPV6=m
2c7310aa7c5d Replace lseek/read into pread for kcore and vmcore reading
538b9ed4564d Fix "fuser" command to properly deal with an invalid argument
4ced10342261 Fix "vm -M" option to properly deal with an invalid argument
97269209aa70 xen: adjust to new scheduler structures
9ee564cd1a46 xen: get stack address via stack_base array if available
4a59c38be2cd xen: fix stacksize
489093c2183f Fix "kmem -n" option to display memory blocks on Linux 6.3-rc1 and later
ade71c3ec1d2 gdb: Fix an assertion failure in dw2_find_pc_sect_compunit_symtab()
5a652ed0c8db Fix for "net -n" option to properly deal with an invalid argument
57dda56af5c7 Fix C99 compatibility issues in embedded copy of GDB
38325fab5337 Enhance "net" command to display IPv6 address of network interface
d0d6cf868577 Fix for "search -u" option failing in maple tree kernel
daa43fa5324f x86_64: Fix "bt" command on kernels with random_kstack_offset=on
59c19818190d Fix for "dis" command to correctly display the offset of disassembly code
e0e6e4a7ee03 Fix for "bt" command unnecessarily printing an exception frame
277da34dd5da Fix for "kmem -i" option to not print invalid values for CACHED
c64a827e0bca Fix for "net -s" option to show IPv6 addresses on Linux 3.13 and later
9253b40a0ecb Fix "kmem -s|-S" not working properly on RHEL8.6 and later
92de7c34b1f9 Fix for "bt" command printing "bogus exception frame" warning
46344aa2f92b Dump maple tree offset variables by "help -o"
49f6c2095d82 Update the help text of "tree" command for maple tree
9efc1f68a44f Introduce maple tree vma iteration to vm_area_dump()
222176a0a6c1 Add do_maple_tree() for maple tree operations
16a696762cbf Add maple tree support to "tree" command
872cad2d63b3 Port the maple tree data structures and functions
ac96e17d1de5 SLAB: Fix for "kmem -s|-S" options on Linux 6.2-rc1 and later
120d6e89fc14 SLAB: Fix for "kmem -s|-S" options on Linux 6.1 and later
a053a1442dff gdb: Fix an assertion failure in the gdb's copy_type()
4cf7c714e3cc Fix build failure due to no EM_RISCV with glibc-2.23 and earlier
41d4b85ea50e Fix for "kmem -i" to display correct SLAB statistics on Linux 5.9 and later
d83df2fb66cd SLUB: Fix for offset change of struct slab members on Linux 6.2-rc1
0d5ad129252a RISCV64: Add the implementation of symbol verify
3f4714967961 RISCV64: Add 'mach' command support
6c281cd355c9 RISCV64: Add 'help -m/M' command support
5cfcdb4ebcb1 RISCV64: Add 'help -r' command support
0d9fcbe3803c RISCV64: Add 'bt' command support
67216c741c4e RISCV64: Add irq command support
b410e14f7865 RISCV64: Add 'dis' command support
e7119d2225e1 RISCV64: Make crash tool enter command line and support some commands
12c31560000a Add RISCV64 framework code support
88a4910d95d4 Fix "mount" command to appropriately display the mount dumps
f182d08bab20 Fix for mm_struct.rss_stat conversion into percpu_counter
df1f0cba729f x86_64: Fix for move of per-cpu variables into struct pcpu_hot
2f1085df4dc9 EPPIC extension support for crash-8.x + gdb-10.x
141e75f3c11c arm64: handle vabits_actual symbol missing case
5f27639196c3 arm64: fix backtraces of KASAN kernel dumpfile truncated
9f1256958d2b ps: Provide an option to display no header line
1e35ad44dc55 Mark start of 8.0.3 development phase with version 8.0.2++
Full changelog: https://crash-utility.github.io/changelog/ChangeLog-8.0.3.txt
or
https://github.com/crash-utility/crash/compare/8.0.2...8.0.3
1 year, 4 months
Re: [Crash-utility] [PATCH v2] Replace lseek/read into pread for kcore and vmcore reading.
by lijiang
On Thu, Apr 20, 2023 at 8:00 PM <crash-utility-request(a)redhat.com> wrote:
> Date: Thu, 20 Apr 2023 18:25:22 +0800
> From: Tao Liu <ltao(a)redhat.com>
> To: crash-utility(a)redhat.com
> Subject: [Crash-utility] [PATCH v2] Replace lseek/read into pread for
> kcore and vmcore reading.
> Message-ID: <20230420102521.33698-1-ltao(a)redhat.com>
> Content-Type: text/plain; charset="US-ASCII"; x-default=true
>
> Previously crash use lseek/read for kcore and vmcore reading, this
> involves 2
> syscalls. And we can replace them with pread, only 1 syscall is needed for
> kcore/vmcore reading, and we can have a better performance. Please note
> there
> are plenty of places in crash using lseek/read, this patch doesn't modify
> all
> of them, just the most commonly used kcore and diskdump vmcore reading.
>
> Signed-off-by: Tao Liu <ltao(a)redhat.com>
> ---
>
> v1 -> v2: add offset check before pread.
>
> ---
> diskdump.c | 31 +++++++++++++++++++++++++------
> netdump.c | 19 ++++++++++++++-----
> 2 files changed, 39 insertions(+), 11 deletions(-)
>
> diff --git a/diskdump.c b/diskdump.c
> index cf5f5d9..6ecd08a 100644
> --- a/diskdump.c
> +++ b/diskdump.c
> @@ -515,16 +515,24 @@ arm_kdump_header_adjust(int header_version)
> static int
> read_pd(int fd, off_t offset, page_desc_t *pd)
> {
> - const off_t failed = (off_t)-1;
> + int ret;
>
> if (FLAT_FORMAT()) {
> if (!read_flattened_format(fd, offset, pd, sizeof(*pd)))
> return READ_ERROR;
> } else {
> - if (lseek(fd, offset, SEEK_SET) == failed)
> + if (offset < 0) {
> + if (CRASHDEBUG(8)) {
> + fprintf(fp, "read_pd: invalid offset:
> %lx\n", offset);
> + }
> return SEEK_ERROR;
> - if (read(fd, pd, sizeof(*pd)) != sizeof(*pd))
> + }
> + if ((ret = pread(fd, pd, sizeof(*pd), offset)) !=
> sizeof(*pd)) {
> + if (ret == -1 && CRASHDEBUG(8)) {
> + fprintf(fp, "read_pd: pread error: %s\n",
> strerror(errno));
> + }
> return READ_ERROR;
> + }
> }
>
> return 0;
> @@ -1125,7 +1133,6 @@ cache_page(physaddr_t paddr)
> off_t seek_offset;
> page_desc_t pd;
> const int block_size = dd->block_size;
> - const off_t failed = (off_t)-1;
> ulong retlen;
> #ifdef ZSTD
> static ZSTD_DCtx *dctx = NULL;
> @@ -1190,10 +1197,22 @@ cache_page(physaddr_t paddr)
> return PAGE_INCOMPLETE;
> }
> } else {
> - if (lseek(dd->dfd, pd.offset, SEEK_SET) == failed)
> + if (pd.offset < 0) {
> + if (CRASHDEBUG(8)) {
> + fprintf(fp,
> + "read_diskdump/cache_page: "
> + "invalid offset: %lx\n",
> pd.offset);
> + }
> return SEEK_ERROR;
> - if (read(dd->dfd, dd->compressed_page, pd.size) != pd.size)
> + }
> + if ((ret = pread(dd->dfd, dd->compressed_page, pd.size,
> pd.offset)) != pd.size) {
> + if (ret == -1 && CRASHDEBUG(8)) {
> + fprintf(fp,
> + "read_diskdump/cache_page: "
> + "pread error: %s\n",
> strerror(errno));
> + }
> return READ_ERROR;
> + }
> }
>
> if (pd.flags & DUMP_DH_COMPRESSED_ZLIB) {
> diff --git a/netdump.c b/netdump.c
> index 01af145..696f5fd 100644
> --- a/netdump.c
> +++ b/netdump.c
> @@ -4336,7 +4336,7 @@ no_nt_prstatus_exists:
> int
> read_proc_kcore(int fd, void *bufptr, int cnt, ulong addr, physaddr_t
> paddr)
> {
> - int i;
> + int i, ret;
> size_t readcnt;
> ulong kvaddr;
> Elf32_Phdr *lp32;
> @@ -4436,11 +4436,20 @@ read_proc_kcore(int fd, void *bufptr, int cnt,
> ulong addr, physaddr_t paddr)
> if (offset == UNINITIALIZED)
> return SEEK_ERROR;
>
> - if (lseek(fd, offset, SEEK_SET) != offset)
> - perror("lseek");
> -
> - if (read(fd, bufptr, readcnt) != readcnt)
> + if (offset < 0) {
> + if (CRASHDEBUG(8)) {
> + fprintf(fp, "read_proc_kcore: "
> + "invalid offset: %lx\n", offset);
> + }
> + return SEEK_ERROR;
> + }
> + if ((ret = pread(fd, bufptr, readcnt, offset)) != readcnt) {
> + if (ret == -1 && CRASHDEBUG(8)) {
> + fprintf(fp, "read_proc_kcore: "
> + "pread error: %s\n", strerror(errno));
> + }
> return READ_ERROR;
> + }
>
> return cnt;
> }
>
Thank you for the update, Tao.
The v2 is expected. So: Ack
Thanks.
Lianbo
--
> 2.33.1
>
1 year, 4 months
[PATCH v2] Replace lseek/read into pread for kcore and vmcore reading.
by Tao Liu
Previously crash use lseek/read for kcore and vmcore reading, this involves 2
syscalls. And we can replace them with pread, only 1 syscall is needed for
kcore/vmcore reading, and we can have a better performance. Please note there
are plenty of places in crash using lseek/read, this patch doesn't modify all
of them, just the most commonly used kcore and diskdump vmcore reading.
Signed-off-by: Tao Liu <ltao(a)redhat.com>
---
v1 -> v2: add offset check before pread.
---
diskdump.c | 31 +++++++++++++++++++++++++------
netdump.c | 19 ++++++++++++++-----
2 files changed, 39 insertions(+), 11 deletions(-)
diff --git a/diskdump.c b/diskdump.c
index cf5f5d9..6ecd08a 100644
--- a/diskdump.c
+++ b/diskdump.c
@@ -515,16 +515,24 @@ arm_kdump_header_adjust(int header_version)
static int
read_pd(int fd, off_t offset, page_desc_t *pd)
{
- const off_t failed = (off_t)-1;
+ int ret;
if (FLAT_FORMAT()) {
if (!read_flattened_format(fd, offset, pd, sizeof(*pd)))
return READ_ERROR;
} else {
- if (lseek(fd, offset, SEEK_SET) == failed)
+ if (offset < 0) {
+ if (CRASHDEBUG(8)) {
+ fprintf(fp, "read_pd: invalid offset: %lx\n", offset);
+ }
return SEEK_ERROR;
- if (read(fd, pd, sizeof(*pd)) != sizeof(*pd))
+ }
+ if ((ret = pread(fd, pd, sizeof(*pd), offset)) != sizeof(*pd)) {
+ if (ret == -1 && CRASHDEBUG(8)) {
+ fprintf(fp, "read_pd: pread error: %s\n", strerror(errno));
+ }
return READ_ERROR;
+ }
}
return 0;
@@ -1125,7 +1133,6 @@ cache_page(physaddr_t paddr)
off_t seek_offset;
page_desc_t pd;
const int block_size = dd->block_size;
- const off_t failed = (off_t)-1;
ulong retlen;
#ifdef ZSTD
static ZSTD_DCtx *dctx = NULL;
@@ -1190,10 +1197,22 @@ cache_page(physaddr_t paddr)
return PAGE_INCOMPLETE;
}
} else {
- if (lseek(dd->dfd, pd.offset, SEEK_SET) == failed)
+ if (pd.offset < 0) {
+ if (CRASHDEBUG(8)) {
+ fprintf(fp,
+ "read_diskdump/cache_page: "
+ "invalid offset: %lx\n", pd.offset);
+ }
return SEEK_ERROR;
- if (read(dd->dfd, dd->compressed_page, pd.size) != pd.size)
+ }
+ if ((ret = pread(dd->dfd, dd->compressed_page, pd.size, pd.offset)) != pd.size) {
+ if (ret == -1 && CRASHDEBUG(8)) {
+ fprintf(fp,
+ "read_diskdump/cache_page: "
+ "pread error: %s\n", strerror(errno));
+ }
return READ_ERROR;
+ }
}
if (pd.flags & DUMP_DH_COMPRESSED_ZLIB) {
diff --git a/netdump.c b/netdump.c
index 01af145..696f5fd 100644
--- a/netdump.c
+++ b/netdump.c
@@ -4336,7 +4336,7 @@ no_nt_prstatus_exists:
int
read_proc_kcore(int fd, void *bufptr, int cnt, ulong addr, physaddr_t paddr)
{
- int i;
+ int i, ret;
size_t readcnt;
ulong kvaddr;
Elf32_Phdr *lp32;
@@ -4436,11 +4436,20 @@ read_proc_kcore(int fd, void *bufptr, int cnt, ulong addr, physaddr_t paddr)
if (offset == UNINITIALIZED)
return SEEK_ERROR;
- if (lseek(fd, offset, SEEK_SET) != offset)
- perror("lseek");
-
- if (read(fd, bufptr, readcnt) != readcnt)
+ if (offset < 0) {
+ if (CRASHDEBUG(8)) {
+ fprintf(fp, "read_proc_kcore: "
+ "invalid offset: %lx\n", offset);
+ }
+ return SEEK_ERROR;
+ }
+ if ((ret = pread(fd, bufptr, readcnt, offset)) != readcnt) {
+ if (ret == -1 && CRASHDEBUG(8)) {
+ fprintf(fp, "read_proc_kcore: "
+ "pread error: %s\n", strerror(errno));
+ }
return READ_ERROR;
+ }
return cnt;
}
--
2.33.1
1 year, 4 months
Re: [Crash-utility] [PATCH 5/5] [RFC] Replace lseek/read into pread for kcore and vmcore reading
by lijiang
Hi, Tao and Kazu
On Tue, Apr 11, 2023 at 8:00 PM <crash-utility-request(a)redhat.com> wrote:
> Date: Tue, 11 Apr 2023 02:01:36 +0000
> From: HAGIO KAZUHITO(?????) <k-hagio-ab(a)nec.com>
> To: Tao Liu <ltao(a)redhat.com>
> Cc: "Discussion list for crash utility usage, maintenance and
> development" <crash-utility(a)redhat.com>
> Subject: Re: [Crash-utility] [PATCH 5/5] [RFC] Replace lseek/read into
> pread for kcore and vmcore reading.
> Message-ID: <bfb9b27d-f5b6-9615-3419-24d97c13aeeb(a)nec.com>
> Content-Type: text/plain; charset="utf-8"
>
> On 2023/04/11 10:23, Tao Liu wrote:
> > Hi Kazu,
> >
> > Sorry for the late reply.
>
> No problem :-)
>
> >
> > On Fri, Apr 7, 2023 at 10:36?AM HAGIO KAZUHITO(?????)
> > <k-hagio-ab(a)nec.com> wrote:
> >>
> >> On 2023/03/25 13:12, Tao Liu wrote:
> >>> Previously crash use lseek/read for kcore and vmcore reading, this
> involves 2
> >>> syscalls. And we can replace them with pread, only 1 syscall is needed
> for
> >>> kcore/vmcore reading, and we can have a better performance. Please
> note there
> >>> are plenty of places in crash using lseek/read, this patch doesn't
> modify all
> >>> of them, just the most commonly used kcore and diskdump vmcore reading.
> >>
> >> This is nice as an optimization, but can we distinguish a seek error and
> >> read error with errno as currently we can do?
> >>
> >
> > Sorry I didn't understand your question...
> >
> > The functions which I modified in this patch didn't use a global errno
> > to distinguish error types, they used a returned value instead.
> > Currently if read error, READ_ERROR will be returned, same as seek
> > error returns SEEK_ERROR. Do you want to distinguish pread error by
> > introducing a new error number as PREAD_ERROR?
>
> No, I would like to know which causes an error in pread(), seek or read.
> In other words, is there any way to know which is wrong, the specified
> offset or file/media?
>
>
Good questions.
In fact, the SEEK_ERROR is simply returned to the top-level call, but it
seems that there is no corresponding processing for the type of errors at
the top-level code. Only saw two cases(please correct me if I missed
anything):
[1] readmem()
...
case SEEK_ERROR:
if (PRINT_ERROR_MESSAGE)
error(INFO, SEEK_ERRMSG,
memtype_string(memtype, 0), addr, type);
goto readmem_error;
case READ_ERROR:
if (PRINT_ERROR_MESSAGE)
error(INFO, READ_ERRMSG,
memtype_string(memtype, 0), addr, type);
if ((pc->flags & DEVMEM) && (kt->flags &
PRE_KERNEL_INIT) &&
!(error_handle & NO_DEVMEM_SWITCH) &&
devmem_is_restricted() &&
switch_to_proc_kcore()) {
error_handle &= ~QUIET;
return(readmem(addr, memtype, bufptr, size,
type, error_handle));
}
goto readmem_error;..
[2] read_diskdump()
...
if ((ret = cache_page(curpaddr)) < 0) {
if (CRASHDEBUG(8))
fprintf(fp, "read_diskdump: "
"%s: cannot cache page: %llx\n",
ret == SEEK_ERROR ?
"SEEK_ERROR" : "READ_ERROR",
(ulonglong)curpaddr);
return ret;
}
...
Given that, only recognizing the type of errors, it can not do anything.
Would it really make sense to distinguish the type of errors(seek/read
errors)?
BTW: Could it be helpful to print an actual error when the pread() fails?
For example:
+ ssize_t prdcnt = pread(fd, bufptr, readcnt, offset);
+ if (prdcnt != readcnt) {
+ if (prdcnt == -1)
+ error(WARNING, "/proc/kcore: %s\n",
strerror(errno));
return READ_ERROR;
+ }
Thanks.
Lianbo
1 year, 4 months
[PATCH 0/5] [RFC] Multi-thread support for search cmd
by Tao Liu
The primary part of the patchset will introduce multithread support for search
cmd to improve its performance. A search operation is mainly made up with 2
steps: 1) readmem data into pagebuf, 2) search specific values within the
pagebuf. A typical workflow of search is as follows:
for addr from low to high:
do
readmem(addr, pagebuf)
search_value(value, pagebuf)
addr += pagesize
done
There are 2 points which we can accelerate: 1) readmem don't have to wait
search_value, when search_value is working, readmem can read the next pagebuf
at the same time. 2) higher addr don't have to wait lower addr, they can be
processed at the same time if we carefully arrange the output order.
For point 1, we introduce zones for pagebuf, e.g. search_value can work on
zone 0 while readmem can prepare the data for zone 1. For point 2, we introduce
multiple search_value in threads, e.g. readmem will prepare 100 pages as a
batch, then we will have 4 threads of search_value, thread 0 handles page 1~25,
thread 2 handles page 26~50 page, thread 3 handles page 51~75, thread 4 handles
page 76~100.
A typical workflow of multithread search implemented in this patchset is as
follows, which removed thread synchronization:
pagebuf[ZONE][BATCH]
zone_index = buf_index = 0
create_thread(4, search_value)
for addr from low to high:
do
if buf_index < BATCH
readmem(addr, pagebuf[zone_index][buf_index++])
addr += pagesize
else
start_thread(pagebuf[zone_index], 0/4 * BATCH, 1/4 * BATCH)
start_thread(pagebuf[zone_index], 1/4 * BATCH, 2/4 * BATCH)
start_thread(pagebuf[zone_index], 2/4 * BATCH, 3/4 * BATCH)
start_thread(pagebuf[zone_index], 3/4 * BATCH, 4/4 * BATCH)
zone_index++
buf_index = 0
fi
done
readmem works in the main process and not multi-threaded, because readmem will
not only read data from vmcore, decompress it, but walk through page tables if
virtual address given. It is hard to reimplement it into thread safe version,
search_value is easier to be made thread-safe. By carefully choose batch size
and thread num, we can maximize the concurrency.
The last part of the patchset, is replacing lseek/read to pread for kcore and
diskdumped vmcore.
Here is the performance test result chart. Please note the vmcore and
kcore are tested seperately on 2 different machines. crash-orig is the
crash compiled with clean upstream code, crash-pread is the code with only
pread patch applied(patch 5), crash-multi is the code with only multithread
patches applied(patch 1~4).
ulong search:
$ time echo "search abcd" | ./crash-orig vmcore vmlinux > /dev/null
$ time echo "search abcd -f 4 -n 4" | ./crash-multi vmcore vmlinux > /dev/null
45G vmcore 64G kcore
real user sys real user sys
crash-orig 16m56.595s 15m57.188s 0m56.698s 1m37.982s 0m51.625s 0m46.266s
crash-pread 16m46.366s 15m55.790s 0m48.894s 1m9.179s 0m36.646s 0m32.368s
crash-multi 16m26.713s 19m8.722s 1m29.263s 1m27.661s 0m57.789s 0m54.604s
string search:
$ time echo "search -c abcddbca" | ./crash-orig vmcore vmlinux > /dev/null
$ time echo "search -c abcddbca -f 4 -n 4" | ./crash-multi vmcore vmlinux > /dev/null
45G vmcore 64G kcore
real user sys real user sys
crash-orig 33m33.481s 32m38.321s 0m52.771s 8m32.034s 7m50.050s 0m41.478s
crash-pread 33m25.623s 32m35.019s 0m47.394s 8m4.347s 7m35.352s 0m28.479s
crash-multi 16m31.016s 38m27.456s 1m11.048s 5m11.725s 7m54.224s 0m44.186s
Discussion:
1) Either multithread and pread patches can improve the performance a
bit, so if both patches applied, the performance can be better.
2) Multi-thread search performs much better in search time consumptive
tasks, such as string search.
Tao Liu (5):
Introduce multi-thread to search_virtual
Introduce multi-thread to search_physical
Introduce multi-thread to string search
Introduce multi-thread options to search cmd
Replace lseek/read into pread for kcore and vmcore reading.
defs.h | 6 +
diskdump.c | 11 +-
help.c | 17 +-
memory.c | 1176 +++++++++++++++++++++++++++++++++++++++++-----------
netdump.c | 5 +-
task.c | 14 +
6 files changed, 969 insertions(+), 260 deletions(-)
--
2.33.1
1 year, 5 months
[PATCH v2] Fix "fuser" command to properly deal with an invalid argument
by Lianbo Jiang
The man page of the "fuser" command suggests that the argument can be a
full pathname or inode address. However, the "fuser" command accepts an
invalid argument and prints a bogus result as below:
crash> fuser x
PID TASK COMM USAGE
100507 ffff9914431f4c80 "packagekitd" fd
100508 ffff991574e59980 "gmain" fd
100509 ffff9914431f3300 "gdbus" fd
102020 ffff991574400000 "sshd" fd
102043 ffff991441d19980 "sshd" fd
The current fuser command has no checking mechanism to determine if an
argument is valid or not. Lets add it to handle such cases.
With the patch:
crash> fuser x
fuser: invalid argument: x
In addition, also add a note that fuser does not expect an argument other
than an inode address and full pathname, if others are specified, the output
can be an unexpected result.
Reported-by: Buland Kumar Singh <bsingh(a)redhat.com>
Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com>
---
filesys.c | 8 +++++++-
help.c | 3 +++
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/filesys.c b/filesys.c
index d64b54a9b822..307da05a774f 100644
--- a/filesys.c
+++ b/filesys.c
@@ -3421,7 +3421,13 @@ cmd_fuser(void)
doing_fds = doing_mmap = 0;
while (args[optind]) {
- spec_string = args[optind];
+ ulong spec_addr;
+ spec_string = args[optind];
+ spec_addr = htol(spec_string, RETURN_ON_ERROR|QUIET, NULL);
+ if ((spec_addr == BADADDR || !IS_KVADDR(spec_addr)) &&
+ spec_string[0] != '/')
+ error(FATAL, "invalid argument: %s\n", args[optind]);
+
if (STRNEQ(spec_string, "0x") && hexadecimal(spec_string, 0))
shift_string_left(spec_string, 2);
len = strlen(spec_string);
diff --git a/help.c b/help.c
index 9a5cd3615589..ded4ee19d955 100644
--- a/help.c
+++ b/help.c
@@ -7990,6 +7990,9 @@ char *help_fuser[] = {
" listed.\n",
" pathname the full pathname of the file.",
" inode the hexadecimal inode address for the file.",
+" Note: fuser does not expect an argument other than an inode address",
+" and a full path, and if others are specified, the output can",
+" be an unexpected one.",
"\nEXAMPLES",
" Display the tasks using file /usr/lib/libkfm.so.2.0.0\n",
" %s> fuser /usr/lib/libkfm.so.2.0.0",
--
2.37.1
1 year, 5 months
[PATCH] Fix "fuser" command to properly deal with an invalid argument
by Lianbo Jiang
The man page of the "fuser" command suggests that the argument can be a
full pathname or inode address. However, the "fuser" command accepts an
invalid argument and prints a bogus result as below:
crash> fuser x
PID TASK COMM USAGE
100507 ffff9914431f4c80 "packagekitd" fd
100508 ffff991574e59980 "gmain" fd
100509 ffff9914431f3300 "gdbus" fd
102020 ffff991574400000 "sshd" fd
102043 ffff991441d19980 "sshd" fd
The current fuser command has no checking mechanism to determine if an
argument is valid or not. Lets add it to handle such cases.
In addition, the fuser can also accept the dentry and file address, for
example:
crash> files -d ffff894d826e7240
DENTRY INODE SUPERBLK TYPE PATH
ffff894d826e7240 ffff894d805b7520 ffff894d80ce5000 REG /root/proc/kcore
crash> fuser ffff894d826e7240
PID TASK COMM USAGE
1237273 ffff89502ddf0000 "crash" fd
1237280 ffff895079cb0000 "gdb worker" fd
1237281 ffff894f9a124000 "gdb worker" fd
1237282 ffff895017a28000 "gdb worker" fd
1237283 ffff894d89c40000 "gdb worker" fd
...
Essentially, the fuser command still calls the vm(vm_area_dump())
and files(open_files_dump()) commands(see foreach()) to achieve
this goal. So the man page needs to be updated, and there is no
functional change.
With the patch:
crash> fuser -n x
fuser: invalid file name: x
crash> fuser -p x
fuser: invalid virtual address: x
crash> fuser -p 0xffff894d826e7240
PID TASK COMM USAGE
1237273 ffff89502ddf0000 "crash" fd
1237280 ffff895079cb0000 "gdb worker" fd
1237281 ffff894f9a124000 "gdb worker" fd
1237282 ffff895017a28000 "gdb worker" fd
1237283 ffff894d89c40000 "gdb worker" fd
...
Reported-by: Buland Kumar Singh <bsingh(a)redhat.com>
Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com>
---
defs.h | 1 +
filesys.c | 49 ++++++++++++++++++++++++++++++-------------------
help.c | 22 ++++++++++++----------
memory.c | 15 +++++++++++++++
4 files changed, 58 insertions(+), 29 deletions(-)
diff --git a/defs.h b/defs.h
index 12ad6aaa0998..d2ba00f1a4b1 100644
--- a/defs.h
+++ b/defs.h
@@ -5736,6 +5736,7 @@ char *fill_dentry_cache(ulong);
void clear_dentry_cache(void);
char *fill_inode_cache(ulong);
void clear_inode_cache(void);
+int check_vm_flags(ulonglong);
int monitor_memory(long *, long *, long *, long *);
int is_readable(char *);
struct list_pair {
diff --git a/filesys.c b/filesys.c
index d64b54a9b822..ccc8092c10b5 100644
--- a/filesys.c
+++ b/filesys.c
@@ -3378,19 +3378,19 @@ clear_inode_cache(void)
/*
- * This command displays the tasks using specified files or sockets.
- * Tasks will be listed that reference the file as the current working
- * directory, root directory, an open file descriptor, or that mmap the
- * file.
- * The argument can be a full pathname without symbolic links, or inode
- * address.
+ * This command displays the tasks associated the specified files, virtual
+ * address or vm_flags. Tasks will be listed that reference the file as the
+ * current working directory, root directory, an open file descriptor, or
+ * that mmap the file.
+ * The argument can be a full pathname without symbolic links, or inode
+ * address, dentry address etc.
*/
void
cmd_fuser(void)
{
- int c;
- char *spec_string, *tmp;
+ int c, pflag = 0, fflag = 0, nflag = 0;
+ char *spec_string = NULL, *tmp;
struct foreach_data foreach_data, *fd;
char task_buf[BUFSIZE];
char buf[BUFSIZE];
@@ -3399,29 +3399,42 @@ cmd_fuser(void)
int doing_fds, doing_mmap, len;
int fuser_header_printed, lockd_header_printed;
- while ((c = getopt(argcnt, args, "")) != EOF) {
+ while ((c = getopt(argcnt, args, "p:f:n:")) != EOF) {
switch(c)
{
+ case 'n':
+ nflag = 1;
+ spec_string = optarg;
+ if (!file_exists(optarg, NULL) && !comm_exists(optarg))
+ error(FATAL, "invalid file name: %s\n", optarg);
+ break;
+ case 'p':
+ pflag = 1;
+ ulonglong vaddr = htoll(optarg, FAULT_ON_ERROR, NULL);
+ if (!IS_KVADDR(vaddr))
+ error(FATAL, "invalid virtual address: %s\n", optarg);
+ spec_string = optarg;
+ break;
+ case 'f':
+ fflag = 1;
+ ulonglong llvalue = htoll(optarg, FAULT_ON_ERROR, NULL);
+ if (!check_vm_flags(llvalue))
+ error(FATAL, "invalid vm flags: %s\n", optarg);
+ spec_string = optarg;
+ break;
default:
argerrs++;
break;
}
}
- if (argerrs)
- cmd_usage(pc->curcmd, SYNOPSIS);
-
- if (!args[optind]) {
+ if (argerrs || (!pflag && !fflag && !nflag))
cmd_usage(pc->curcmd, SYNOPSIS);
- return;
- }
sprintf(fuser_header, " PID %s COMM USAGE\n",
mkstring(buf, VADDR_PRLEN, CENTER, "TASK"));
doing_fds = doing_mmap = 0;
- while (args[optind]) {
- spec_string = args[optind];
if (STRNEQ(spec_string, "0x") && hexadecimal(spec_string, 0))
shift_string_left(spec_string, 2);
len = strlen(spec_string);
@@ -3505,11 +3518,9 @@ cmd_fuser(void)
BZERO(uses, 20);
}
close_tmpfile();
- optind++;
if (!fuser_header_printed && !lockd_header_printed) {
fprintf(fp, "No users of %s found\n", spec_string);
}
- }
}
static void
diff --git a/help.c b/help.c
index 9a5cd3615589..62fca314a58e 100644
--- a/help.c
+++ b/help.c
@@ -7981,18 +7981,20 @@ NULL
char *help_fuser[] = {
"fuser",
"file users",
-"[pathname | inode]",
-" This command displays the tasks using specified files or sockets.",
-" Tasks will be listed that reference the file as the current working",
-" directory, root directory, an open file descriptor, or that mmap the",
-" file. If the file is held open in the kernel by the lockd server on",
-" behalf of a client discretionary file lock, the client hostname is",
-" listed.\n",
-" pathname the full pathname of the file.",
-" inode the hexadecimal inode address for the file.",
+"[-p addr | -n filename | -f vm_flags]",
+" This command displays the tasks associated the specified files, virtual",
+" address or vm_flags. Tasks will be listed that reference the file as the",
+" current working, directory, root directory, an open file descriptor, or",
+" that mmap the file. If the file is held open in the kernel by the lockd",
+" server on behalf of a client discretionary file lock, the client hostname",
+" is listed.\n",
+" -n filename the full pathname of the file.",
+" -p addr the hexadecimal virtual address for the file. It may be a",
+" valid inode address, dentry address, file address, etc.",
+" -f vm_flags the FLAGS (vm_flags) value."
"\nEXAMPLES",
" Display the tasks using file /usr/lib/libkfm.so.2.0.0\n",
-" %s> fuser /usr/lib/libkfm.so.2.0.0",
+" %s> fuser -n /usr/lib/libkfm.so.2.0.0",
" PID TASK COMM USAGE",
" 779 c5e82000 \"kwm\" mmap",
" 808 c5a8e000 \"krootwm\" mmap",
diff --git a/memory.c b/memory.c
index 0568f18eb9b7..b2c12f146d95 100644
--- a/memory.c
+++ b/memory.c
@@ -3727,6 +3727,21 @@ cmd_vm(void)
#define VM_PFN_AT_MMAP 0x40000000ULL /* PFNMAP vma that is fully mapped at mmap time */
#define VM_MERGEABLE 0x80000000ULL /* KSM may merge identical pages */
+int check_vm_flags(ulonglong flags)
+{
+ if ( flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC|VM_MAYSHARE|
+ VM_GROWSDOWN|VM_GROWSUP|VM_SHM|VM_DENYWRITE|
+ VM_EXECUTABLE|VM_LOCKED|VM_IO|VM_SEQ_READ|
+ VM_RAND_READ|VM_DONTCOPY|VM_DONTEXPAND|VM_RESERVED|
+ VM_BIGPAGE|VM_BIGMAP|VM_HUGETLB|VM_NONLINEAR|
+ VM_MAPPED_COPY|VM_INSERTPAGE|VM_ALWAYSDUMP|VM_CAN_NONLINEAR|
+ VM_MIXEDMAP|VM_SAO|VM_PFN_AT_MMAP|VM_MERGEABLE))
+ return TRUE;
+
+ return FALSE;
+}
+
static void
do_vm_flags(ulonglong flags)
{
--
2.37.1
1 year, 5 months
Re: [Crash-utility] [PATCH 0/5] [RFC] Multi-thread support for search cmd
by lijiang
Thank you for the patchset, Tao.
On Sat, Mar 25, 2023 at 8:00 PM <crash-utility-request(a)redhat.com> wrote:
> Date: Sat, 25 Mar 2023 12:12:12 +0800
> From: Tao Liu <ltao(a)redhat.com>
> To: crash-utility(a)redhat.com
> Subject: [Crash-utility] [PATCH 0/5] [RFC] Multi-thread support for
> search cmd
> Message-ID: <20230325041217.8184-1-ltao(a)redhat.com>
> Content-Type: text/plain; charset="US-ASCII"; x-default=true
>
> The primary part of the patchset will introduce multithread support for
> search
> cmd to improve its performance. A search operation is mainly made up with 2
>
To be honest, I'm not sure if it's really worth introducing multi-thread
only for a "search"
command, as the "search" command is not commonly used, and the performance
issue
only occurs when the "search" command reads large memory. Let's see if Kazu
has any
comments about it.
But the [PATCH 5/5] looks good to me, and it should be a separate patch.
Thanks.
Lianbo
> steps: 1) readmem data into pagebuf, 2) search specific values within the
> pagebuf. A typical workflow of search is as follows:
>
> for addr from low to high:
> do
> readmem(addr, pagebuf)
> search_value(value, pagebuf)
> addr += pagesize
> done
>
> There are 2 points which we can accelerate: 1) readmem don't have to wait
> search_value, when search_value is working, readmem can read the next
> pagebuf
> at the same time. 2) higher addr don't have to wait lower addr, they can be
> processed at the same time if we carefully arrange the output order.
>
> For point 1, we introduce zones for pagebuf, e.g. search_value can work on
> zone 0 while readmem can prepare the data for zone 1. For point 2, we
> introduce
> multiple search_value in threads, e.g. readmem will prepare 100 pages as a
> batch, then we will have 4 threads of search_value, thread 0 handles page
> 1~25,
> thread 2 handles page 26~50 page, thread 3 handles page 51~75, thread 4
> handles
> page 76~100.
>
> A typical workflow of multithread search implemented in this patchset is as
> follows, which removed thread synchronization:
>
> pagebuf[ZONE][BATCH]
> zone_index = buf_index = 0
> create_thread(4, search_value)
> for addr from low to high:
> do
> if buf_index < BATCH
> readmem(addr, pagebuf[zone_index][buf_index++])
> addr += pagesize
> else
> start_thread(pagebuf[zone_index], 0/4 * BATCH, 1/4 * BATCH)
> start_thread(pagebuf[zone_index], 1/4 * BATCH, 2/4 * BATCH)
> start_thread(pagebuf[zone_index], 2/4 * BATCH, 3/4 * BATCH)
> start_thread(pagebuf[zone_index], 3/4 * BATCH, 4/4 * BATCH)
> zone_index++
> buf_index = 0
> fi
> done
>
> readmem works in the main process and not multi-threaded, because readmem
> will
> not only read data from vmcore, decompress it, but walk through page
> tables if
> virtual address given. It is hard to reimplement it into thread safe
> version,
> search_value is easier to be made thread-safe. By carefully choose batch
> size
> and thread num, we can maximize the concurrency.
>
> The last part of the patchset, is replacing lseek/read to pread for kcore
> and
> diskdumped vmcore.
>
> Here is the performance test result chart. Please note the vmcore and
> kcore are tested seperately on 2 different machines. crash-orig is the
> crash compiled with clean upstream code, crash-pread is the code with only
> pread patch applied(patch 5), crash-multi is the code with only multithread
> patches applied(patch 1~4).
>
> ulong search:
>
> $ time echo "search abcd" | ./crash-orig vmcore vmlinux > /dev/null
> $ time echo "search abcd -f 4 -n 4" | ./crash-multi vmcore vmlinux >
> /dev/null
>
> 45G vmcore 64G kcore
> real user sys real user
> sys
> crash-orig 16m56.595s 15m57.188s 0m56.698s 1m37.982s
> 0m51.625s 0m46.266s
> crash-pread 16m46.366s 15m55.790s 0m48.894s 1m9.179s
> 0m36.646s 0m32.368s
> crash-multi 16m26.713s 19m8.722s 1m29.263s 1m27.661s
> 0m57.789s 0m54.604s
>
> string search:
>
> $ time echo "search -c abcddbca" | ./crash-orig vmcore vmlinux >
> /dev/null
> $ time echo "search -c abcddbca -f 4 -n 4" | ./crash-multi vmcore
> vmlinux > /dev/null
>
> 45G vmcore 64G kcore
> real user sys real user
> sys
> crash-orig 33m33.481s 32m38.321s 0m52.771s 8m32.034s
> 7m50.050s 0m41.478s
> crash-pread 33m25.623s 32m35.019s 0m47.394s 8m4.347s
> 7m35.352s 0m28.479s
> crash-multi 16m31.016s 38m27.456s 1m11.048s 5m11.725s
> 7m54.224s 0m44.186s
>
> Discussion:
>
> 1) Either multithread and pread patches can improve the performance a
> bit, so if both patches applied, the performance can be better.
>
> 2) Multi-thread search performs much better in search time consumptive
> tasks, such as string search.
>
> Tao Liu (5):
> Introduce multi-thread to search_virtual
> Introduce multi-thread to search_physical
> Introduce multi-thread to string search
> Introduce multi-thread options to search cmd
> Replace lseek/read into pread for kcore and vmcore reading.
>
> defs.h | 6 +
> diskdump.c | 11 +-
> help.c | 17 +-
> memory.c | 1176 +++++++++++++++++++++++++++++++++++++++++-----------
> netdump.c | 5 +-
> task.c | 14 +
> 6 files changed, 969 insertions(+), 260 deletions(-)
>
> --
> 2.33.1
>
1 year, 5 months