On Mar 22, 2021, at 8:12 PM, d.hatayama@fujitsu.com wrote:

Wengang,

Thanks for your patch.

From a UEK5 vmcore, I see that

What arch and kernel version do you use?
What is the last kernel version you didn't see this issue?


It’s Oracle Unbreakable Enterprise Kernel V5 for X86_64, source code is based on 4.14 mainline stable kernel.
That is the latest kernel I tried. 

Thanks,
Wengang
I need to identify the change in the kernel side that caused this issue.


crash> p &__start___trace_bprintk_fmt
$1 = (const char *(*)[]) 0xffffffffa163f1d0
crash> p &__stop___trace_bprintk_fmt
$2 = (const char *(*)[]) 0xffffffffa163f1f0

so (0xffffffffa163f1f0 - 0xffffffffa163f1d0)/8 = 4.

there are two zero addresses at index 2 and 3.
crash> rd __start___trace_bprintk_fmt 4
ffffffffa163f1d0:  ffffffffa11ccdca ffffffffa11ccdca   ................
ffffffffa163f1e0:  0000000000000000 0000000000000000   ................

current implementation will fail (no output for trace show command)
on seeing the zero addresses.

fix: ignore zero addresses in add_print_address.

tested to be good (compared the ftrace log from vmcore and that from live system).

Signed-off-by: Wengang Wang <wen.gang.wang@oracle.com>

Thanks.
HATAYAMA, Daisuke