[Crash-utility] Re: [PATCH RFC][makedumpfile 00/10] btf/kallsyms based eppic extension for mm page filtering

Thank you for the suggestion. I think it's a good idea, but epppic needs careful consideration.

I'm sorry, but please let me check for a moment.

Thanks, Masa On 2025/08/05 12:16, Tao Liu wrote: > Kindly ping... > > Any comments for this patchset? > > Thanks, > Tao Liu > > > On Tue, Jun 10, 2025 at 9:57 PM Tao Liu <ltao(a)redhat.com&gt; wrote: >> A) This patchset will introduce the following features to makedumpfile: >> >> 1) Enable eppic script for memory pages filtering. >> 2) Enable btf and kallsyms for symbol type and address resolving. >> 3) Port maple tree data structures and functions, primarily used for >> vma iteration. >> >> B) The purpose of the features are: >> >> 1) Currently makedumpfile filters mm pages based on page flags, because flags >> can help to determine one page's usage. But this page-flag-checking method >> lacks of flexibility in certain cases, e.g. if we want to filter those mm >> pages occupied by GPU during vmcore dumping due to: >> >> a) GPU may be taking a large memory and contains sensitive data; >> b) GPU mm pages have no relations to kernel crash and useless for vmcore >> analysis. >> >> But there is no GPU mm page specific flags, and apparently we don't need >> to create one just for kdump use. A programmable filtering tool is more >> suitable for such cases. In addition, different GPU vendors may use >> different ways for mm pages allocating, programmable filtering is better >> than hard coding these GPU specific logics into makedumpfile in this case. >> >> 2) Currently makedumpfile already contains a programmable filtering tool, aka >> eppic script, which allows user to write customized code for data erasing. >> However it has the following drawbacks: >> >> a) cannot do mm page filtering. >> b) need to access to debuginfo of both kernel and modules, which is not >> applicable in the 2nd kernel. >> c) Poor performance, making vmcore dumping time unacceptable (See >> the following performance testing). >> >> makedumpfile need to resolve the dwarf data from debuginfo, to get symbols >> types and addresses. In recent kernel there are dwarf alternatives such >> as btf/kallsyms which can be used for this purpose. And btf/kallsyms info >> are already packed within vmcore, so we can use it directly. >> >> 3) Maple tree data structures are used in recent kernels, such as vma >> iteration. So maple tree poring is needed. >> >> With these, this patchset introduces an upgraded eppic, which is based on >> btf/kallsyms symbol resolving, and is programmable for mm page filtering. >> The following info shows its usage and performance, please note the tests >> are performed in 1st kernel: >> >> $ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore >> /tmp/dwarf.out -x /lib/debug/lib/modules/6.11.8-300.fc41.x86_64/vmlinux >> --eppic eppic_scripts/filter_amdgpu_mm_pages.c >> real 14m6.894s >> user 4m16.900s >> sys 9m44.695s >> >> $ time ./makedumpfile -d 31 -l /var/crash/127.0.0.1-2025-06-10-18\:03\:12/vmcore >> /tmp/btf.out --eppic eppic_scripts/filter_amdgpu_mm_pages.c >> real 0m10.672s >> user 0m9.270s >> sys 0m1.130s >> >> -rw------- 1 root root 367475074 Jun 10 18:06 btf.out >> -rw------- 1 root root 367475074 Jun 10 21:05 dwarf.out >> -rw-rw-rw- 1 root root 387181418 Jun 10 18:03 /var/crash/127.0.0.1-2025-06-10-18:03:12/vmcore >> >> C) Discussion: >> >> 1) GPU types: Currently only tested with amdgpu's mm page filtering, others >> are not tested. >> 2) Code structure: There are some similar code shared by makedumpfile and >> crash, such as maple tree data structure, also I planed to port the >> btf/kallsyms code to crash as well, so there are code duplications for >> crash & makedumpfile. Since I havn't working on crash poring, code change >> on btf/kallsyms is expected. How can we share the code, creating a common >> library or keep the duplication as it is? >> 3) OS: The code can work on rhel-10+/rhel9.5+ on x86_64/arm64/s390/ppc64. >> Others are not tested. >> >> D) Testing: >> >> 1) If you don't want to create your vmcore, you can find a vmcore which I >> created with amdgpu mm pages unfiltered [1], the amdgpu mm pages are >> allocated by program [2]. You can use the vmcore in 1st kernel to filter >> the amdgpu mm pages by the previous performance testing cmdline. To >> verify the pages are filtered in crash: >> >> Unfiltered: >> crash> search -c "!QAZXSW@#EDC" >> ffff96b7fa800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX >> ffff96b87c800000: !QAZXSW@#EDCXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX >> crash> rd ffff96b7fa800000 >> ffff96b7fa800000: 405753585a415121 !QAZXSW@ >> crash> rd ffff96b87c800000 >> ffff96b87c800000: 405753585a415121 !QAZXSW@ >> >> Filtered: >> crash> search -c "!QAZXSW@#EDC" >> crash> rd ffff96b7fa800000 >> rd: page excluded: kernel virtual address: ffff96b7fa800000 type: "64-bit KVADDR" >> crash> rd ffff96b87c800000 >> rd: page excluded: kernel virtual address: ffff96b87c800000 type: "64-bit KVADDR" >> >> 2) You can use eppic_scripts/print_all_vma.c against an ordinary vmcore to >> test only btf/kallsyms functions by output all VMAs if no amdgpu >> vmcores/machine avaliable. >> >> [1]: https://people.redhat.com/~ltao/core/ >> [2]: https://gist.github.com/liutgnu/a8cbce1c666452f1530e1410d1f352df >> >> Tao Liu (10): >> dwarf_info: Support kernel address randomization >> dwarf_info: Fix a infinite recursion bug for search_domain >> Add page filtering function >> Add btf/kallsyms support for symbol type/address resolving >> Export necessary btf/kallsyms functions to eppic extension >> Port the maple tree data structures and functions >> Supporting main() as the entry of eppic script >> Enable page filtering for dwarf eppic >> Enable page filtering for btf/kallsyms eppic >> Introducing 2 eppic scripts to test the dwarf/btf eppic extension >> >> Makefile | 6 +- >> btf.c | 919 +++++++++++++++++++++++++ >> btf.h | 176 +++++ >> dwarf_info.c | 15 +- >> eppic_maple.c | 431 ++++++++++++ >> eppic_maple.h | 8 + >> eppic_scripts/filter_amdgpu_mm_pages.c | 36 + >> eppic_scripts/print_all_vma.c | 29 + >> erase_info.c | 123 +++- >> erase_info.h | 22 + >> extension_btf.c | 218 ++++++ >> extension_eppic.c | 41 +- >> extension_eppic.h | 6 +- >> kallsyms.c | 371 ++++++++++ >> kallsyms.h | 42 ++ >> makedumpfile.c | 21 +- >> makedumpfile.h | 11 + >> 17 files changed, 2448 insertions(+), 27 deletions(-) >> create mode 100644 btf.c >> create mode 100644 btf.h >> create mode 100644 eppic_maple.c >> create mode 100644 eppic_maple.h >> create mode 100644 eppic_scripts/filter_amdgpu_mm_pages.c >> create mode 100644 eppic_scripts/print_all_vma.c >> create mode 100644 extension_btf.c >> create mode 100644 kallsyms.c >> create mode 100644 kallsyms.h >> >> -- >> 2.47.0 >>