Andrew Hecox wrote:
On Thu, 2008-02-07 at 11:27 -0500, Dave Anderson wrote:
> Andrew Hecox wrote:
>> On Thu, 2008-02-07 at 10:32 -0500, Dave Anderson wrote:
>>> Andrew Hecox wrote:
>>>> hello,
>>>>
>>>> I'm looking at a customer issue where diskdumpmsg is unable to read
a
>>>> vmcore file. It is not clear if this a problem with the vmcore file or
>>>> diskdumpmsg. I can load the vmcore with crash and in my naive usage of
>>>> it, can see no problems. However, I'm new to the tool so that
doesn't
>>>> give me a lot of confidence.
>>>>
>>>> Does anyone have any suggestions on how or if I can use crash to help
>>>> determine if there's corruption in the vmcore file? Or any other way
of
>>>> approaching the problem?
>>>>
>>>> Thanks much,
>>>>
>>>> Andrew
>>>>
>>> I'm not sure what you expect the crash utility to do -- if it comes
>>> up to a prompt with no error or warning messages, it means that the
>>> ELF header contains what appears to be valid usable information,
>>> and that the minimum kernel memory contents required to set up the
>>> crash utility's notion of the running system are all in place.
That's
>>> not to say that there is no chance that the vmcore contains some
>>> corruption that was not recognized.
>>>
>> Thanks. Any other suggestions on how to determine if a vmcore is
"valid"
>> or is that not even a reasonable question to try and ask? The problem
>> I'm trying to solve is described better below:
>>
>>> With respect to diskdumpmsg, as I understand it, it was fairly recently
>>> changed from a perl script to a C file so that it could be run
>>> earlier in time so as to be able to use the swap partition. Looking
>>> at main() in the diskdumpmsg.c file (version 1.4.1-2), there are numerous
>>> error types and associated error messages. What do you mean when you
>>> say that "diskdumpmsg is unable to read a vmcore file"?
>> Specifically:
>>
>> - user reported a floating point exception from diskdump on startup
>> - the result was reproducible locally but only with their vmcore file
>> - fpe occurred in get_logbuf:
>> log_end %= log_buf_len;
>> - log_buf_len had been set to 0 in read_buffer
>> if (!page_is_dumpable(pfn, dump->device)) {
>> memset(buf, 0, copy_len);
>> } else {
>> - I don't know enough to say if the page really wasn't dumpable.
>> static inline bool page_is_dumpable(unsigned int nr, DumpDevice *device)
>> {
>> return device->dumpable_bitmap[nr>>3] & (1 << (nr &
7));
>> }
>> - I wrote a patch with one way to avoid the FPE (attached) and sent it
>> to SEG.
>>
>> Now I'm trying to determine if the vmcore file should be readable by
>> diskdumpmsg. In other words, is this a problem in diskdumpmsg post-crash
>> or a problem with the vmcore file prior to it getting to diskdumpmsg.
>> Unfortunately, I don't understand the problem domain very well at all,
>> hence the probably naive questions :)
>>
>> Any suggestions are appreciated.
>>
>> -Andrew
> So it appears that the page containing the log_buf_len symbol is not
> readable or contained in the dumpfile. BTW, is this a compressed
> dumpfile or an ELF formatted dumpfile? And what "dump_level" did
> they configure?
>
compressed, level is 19.
> Anyway, back to the log_buf_len symbol read, what happens when you
> enter the "log" command while in a crash session? It attempts to
> read that symbol immediately.
>
I get what appears to be a full and valid dump of the kernel message
buffer.
The crash utility has the same page_is_dumpable() function, which I presume
looks at precisely the same bitmap data from the dumpfile. And that
must be working, given that the "log" command works as expected.
One difference is that diskdumpmsg uses /boot/System.map-<release> for
the symbol values, whereas crash uses the vmlinux file. It might be
of interest to determine whether the value of "log_buf_len" used by
diskdumpmsg is the same symbol value as used by crash.
Dave