At 02/17/2012 12:30 AM, Dave Anderson Wrote:

>> Yes. Even if the guest is linux, it is still impossible to do it. Because >> the guest maybe in the second kernel. >> >> qemu-dump walks all guest's page table and collect virtual address and >> physical address mapping. If the page is not used by guest, the virtual is set >> to 0. I create PT_LOAD according to such mapping. So if the guest linux, >> there may be a PT_LOAD segment that describes __START_KERNEL_map region. >> But the information stored in PT_LOAD maybe for the second kernel. If crash >> uses it, crash will see the second kernel, not the first kernel. > > Just to be clear -- what do you mean by the "second" kernel? Do you > mean that a guest kernel crashed guest, and did a kdump operation, > and that second kdump kernel failed somehow, and now you're trying > to do a "virsh dump" on the kdump kernel? Yes, the second kernel means kdump kernel. If kdump failed, the user can use it to dump the guest's memory.

At 02/17/2012 10:20 PM, Dave Anderson Wrote: > > > ----- Original Message ----- >> At 02/17/2012 12:30 AM, Dave Anderson Wrote: > >>>> Yes. Even if the guest is linux, it is still impossible to do it. Because >>>> the guest maybe in the second kernel. >>>> >>>> qemu-dump walks all guest's page table and collect virtual address and >>>> physical address mapping. If the page is not used by guest, the virtual is set >>>> to 0. I create PT_LOAD according to such mapping. So if the guest linux, >>>> there may be a PT_LOAD segment that describes __START_KERNEL_map region. >>>> But the information stored in PT_LOAD maybe for the second. If crash >>>> uses it, crash will see the second kernel, not the first kernel. >>> >>> Just to be clear -- what do you mean by the "second" kernel? Do you >>> mean that a guest kernel crashed guest, and did a kdump operation, >>> and that second kdump kernel failed somehow, and now you're trying >>> to do a "virsh dump" on the kdump kernel? >> >> Yes, the second kernel means kdump kernel. If kdump failed, the user can >> use it to dump the guest's memory. > > OK, so will your code present two different "types" of ELF headers? I donot understand what do you want to say. Do you mean there is two ELF headers in qemu-generated vmcore?

> > No. What I want to understand is how x86_64_calc_phys_base() will > be able to confidently recognize that an ELF file was qemu-generated, > so that it can then do the right thing.

The guest is in first kernel: # readelf /tmp/vm2.save -l| grep 0xffffffff8 LOAD 0x00000000010226b8 0xffffffff81000000 0x0000000001000000 LOAD 0x00000000010226b8 0xffffffff81000000 0x0000000001000000 LOAD 0x00000000010226b8 0xffffffff81000000 0x0000000001000000 LOAD 0x00000000010226b8 0xffffffff81000000 0x0000000001000000

The guest is in the second kernel(vcpu > 1) ]# readelf /tmp/vm2.save2 -l| grep 0xffffffff8 LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 LOAD 0x0000000004017be0 0xffffffff81000000 0x0000000004000000

The guest is in the second kernel(vcpu = 1) [root@ghost ~]# readelf /tmp/vm2.save3 -l| grep 0xffffffff8 LOAD 0x0000000004001e4c 0xffffffff81000000 0x0000000004000000 I donot find differentiate qemu-genetated ELF headers from dump-generated ELF headers.

At 02/27/2012 10:10 PM, Dave Anderson Wrote:

>> The guest is in the second kernel(vcpu > 1) >> ]# readelf /tmp/vm2.save2 -l| grep 0xffffffff8 >> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >> LOAD 0x0000000004017be0 0xffffffff81000000 0x0000000004000000 > > Again, it's not clear why there are multiple segments with the same > same virtual address, but I'm guessing that the one segment that starts > at 0x0000000004000000 is associated with the second kernel, and the other > ones are for vcpus that ran in the first kernel? > >> The guest is in the second kernel(vcpu = 1) >> [root@ghost ~]# readelf /tmp/vm2.save3 -l| grep 0xffffffff8 >> LOAD 0x0000000004001e4c 0xffffffff81000000 0x0000000004000000 >> >> I donot find differentiate qemu-genetated ELF headers from dump-generated ELF >> headers. > > Kdump-generated vmcores cannot have multiple START_KERNEL_map segments. > But with dumps where (vpcu = 1), there could be confusion since it's not obvious > if START_KERNEL_map region belongs to the first or second kernel. > > That being the case, I don't see how this can be supported cleanly by the crash' > utility unless there is a NOTE, or some other obvious identifier, that absolutely > confirms that the dumpfile was qemu-generated. The note information stored in qemu-generated core: Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align NOTE 0x000000000000edd0 0x0000000000000000 0x0000000000000000 0x0000000000000590 0x0000000000000590 0 I think its format is the same as kdump's vmcore. Does kdump-generated core's virtaddr is always 0? If so, What about to set virt_addr to -1 in qemu-generated core?

At 02/28/2012 02:30 PM, HATAYAMA Daisuke Wrote: > From: Wen Congyang <wency(a)cn.fujitsu.com&gt; > Subject: Re: [Crash-utility] question about phys_base > Date: Tue, 28 Feb 2012 13:10:38 +0800 > >> At 02/27/2012 10:10 PM, Dave Anderson Wrote: > >>>> The guest is in the second kernel(vcpu > 1) >>>> ]# readelf /tmp/vm2.save2 -l| grep 0xffffffff8 >>>> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >>>> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >>>> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >>>> LOAD 0x0000000004017be0 0xffffffff81000000 0x0000000004000000 >>> >>> Again, it's not clear why there are multiple segments with the same >>> same virtual address, but I'm guessing that the one segment that starts >>> at 0x0000000004000000 is associated with the second kernel, and the other >>> ones are for vcpus that ran in the first kernel? >>> >>>> The guest is in the second kernel(vcpu = 1) >>>> [root@ghost ~]# readelf /tmp/vm2.save3 -l| grep 0xffffffff8 >>>> LOAD 0x0000000004001e4c 0xffffffff81000000 0x0000000004000000 >>>> >>>> I donot find differentiate qemu-genetated ELF headers from dump-generated ELF >>>> headers. >>> >>> Kdump-generated vmcores cannot have multiple START_KERNEL_map segments. >>> But with dumps where (vpcu = 1), there could be confusion since it's not obvious >>> if START_KERNEL_map region belongs to the first or second kernel. >>> >>> That being the case, I don't see how this can be supported cleanly by the crash' >>> utility unless there is a NOTE, or some other obvious identifier, that absolutely >>> confirms that the dumpfile was qemu-generated. >> >> The note information stored in qemu-generated core: >> Program Headers: >> Type Offset VirtAddr PhysAddr >> FileSiz MemSiz Flags Align >> NOTE 0x000000000000edd0 0x0000000000000000 0x0000000000000000 >> 0x0000000000000590 0x0000000000000590 0 >> >> I think its format is the same as kdump's vmcore. Does kdump-generated core's >> virtaddr is always 0? If so, What about to set virt_addr to -1 in qemu-generated >> core? >> > > In general, such characteristic should not be used. You should prepare > a solid interface. Even if using them, it should be limited to as > workaround to avoid some issue. > > Why not use qemu's CPU state? Include it as note information with good > name, and we can use it to distinguish which. Like: > > $ readelf -n vmcore > > Notes at offset 0x000001c8 with length 0x00000838: > Owner Data size Description > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > QEMU 0x00000557 Unknown note type: (0x00000000) > > Or QEMUCPUState is better? Good idea. I will try it, and hope gdb can also work.

At 02/28/2012 04:52 PM, HATAYAMA Daisuke Wrote:

>>> In general, such characteristic should not be used. You should prepare >>> a solid interface. Even if using them, it should be limited to as >>> workaround to avoid some issue. >>> >>> Why not use qemu's CPU state? Include it as note information with good >>> name, and we can use it to distinguish which. Like: >>> >>> $ readelf -n vmcore >>> >>> Notes at offset 0x000001c8 with length 0x00000838: >>> Owner Data size Description >>> CORE 0x00000150 NT_PRSTATUS (prstatus structure) >>> CORE 0x00000150 NT_PRSTATUS (prstatus structure) >>> QEMU 0x00000557 Unknown note type: (0x00000000) >>> >>> Or QEMUCPUState is better? >> >> Good idea. I will try it, and hope gdb can also work. >> > > Tools basically ignore unknown notes. Looking into gdb, it appears to > ignore unknown information. > > static bfd_boolean > elfcore_grok_note (bfd *abfd, Elf_Internal_Note *note) > { > const struct elf_backend_data *bed = get_elf_backend_data (abfd); > > switch (note->type) > { > default: > return TRUE; > <cut> > > You might need to add new command to output contents of new note if > it's necessary. My goal is: 1. gdb uses NT_PRSTATUS, and can work well 2. crash uses unknown notes, and can get phys_base from it. Another question: What is QEMUCPUState? I donot find its definition?

What note->type shoule be for "QEMU"? If we choose an unused value, the value may be used in the future.

From: Dave Anderson <anderson(a)redhat.com&gt; Subject: Re: [Crash-utility] question about phys_base Date: Tue, 28 Feb 2012 09:44:09 -0500 (EST) > > > ----- Original Message ----- >> At 02/28/2012 04:52 PM, HATAYAMA Daisuke Wrote: > >>>>> In general, such characteristic should not be used. You should prepare >>>>> a solid interface. Even if using them, it should be limited to as >>>>> workaround to avoid some issue. >>>>> >>>>> Why not use qemu's CPU state? Include it as note information with good >>>>> name, and we can use it to distinguish which. Like: >>>>> >>>>> $ readelf -n vmcore >>>>> >>>>> Notes at offset 0x000001c8 with length 0x00000838: >>>>> Owner Data size Description >>>>> CORE 0x00000150 NT_PRSTATUS (prstatus structure) >>>>> CORE 0x00000150 NT_PRSTATUS (prstatus structure) >>>>> QEMU 0x00000557 Unknown note type: (0x00000000) >>>>> >>>>> Or QEMUCPUState is better? >>>> >>>> Good idea. I will try it, and hope gdb can also work. >>>> >>> >>> Tools basically ignore unknown notes. Looking into gdb, it appears to >>> ignore unknown information. >>> >>> static bfd_boolean >>> elfcore_grok_note (bfd *abfd, Elf_Internal_Note *note) >>> { >>> const struct elf_backend_data *bed = get_elf_backend_data (abfd); >>> >>> switch (note->type) >>> { >>> default: >>> return TRUE; >>> <cut> >>> >>> You might need to add new command to output contents of new note if >>> it's necessary. >> >> My goal is: >> 1. gdb uses NT_PRSTATUS, and can work well >> 2. crash uses unknown notes, and can get phys_base from it. >> >> Another question: >> >> What is QEMUCPUState? I donot find its definition? > > It's just the note's name character string. Either "QEMU", "QEMUCPUState", > or whatever unique character string you prefer would suffice. > >> What note->type shoule be for "QEMU"? If we choose an unused value, the >> value may be used in the future. > > Why not do the same thing as the "VMCOREINFO" note, and leave it > as an "illegal/unknown" type of 0:: > > $ cat /usr/include/elf.h > ... [ cut ] ... > > /* Legal values for note segment descriptor types for core files. */ > > #define NT_PRSTATUS 1 /* Contains copy of prstatus struct */ > #define NT_FPREGSET 2 /* Contains copy of fpregset struct */ > #define NT_PRPSINFO 3 /* Contains copy of prpsinfo struct */ > ... > $ > > $ readelf -a vmcore > ... [ cut ] ... > > Notes at offset 0x00000190 with length 0x00000f1c: > Owner Data size Description > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > CORE 0x00000150 NT_PRSTATUS (prstatus structure) > VMCOREINFO 0x000003e4 Unknown note type: (0x00000000) > $ > It looks OK to me for now. But rigorously, it might be better to give it a name QEMU and then prepare variety of information such as NT_CPUSTATE and others to be needed later.

Thanks. HATAYAMA, Daisuke

At 02/28/2012 04:52 PM, HATAYAMA Daisuke Wrote: > From: Wen Congyang <wency(a)cn.fujitsu.com&gt; > Subject: Re: [Crash-utility] question about phys_base > Date: Tue, 28 Feb 2012 16:36:59 +0800 > >> At 02/28/2012 02:30 PM, HATAYAMA Daisuke Wrote: >>> From: Wen Congyang <wency(a)cn.fujitsu.com&gt; >>> Subject: Re: [Crash-utility] question about phys_base >>> Date: Tue, 28 Feb 2012 13:10:38 +0800 >>> >>>> At 02/27/2012 10:10 PM, Dave Anderson Wrote: >>> >>>>>> The guest is in the second kernel(vcpu > 1) >>>>>> ]# readelf /tmp/vm2.save2 -l| grep 0xffffffff8 >>>>>> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >>>>>> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >>>>>> LOAD 0x0000000001017be0 0xffffffff81000000 0x0000000001000000 >>>>>> LOAD 0x0000000004017be0 0xffffffff81000000 0x0000000004000000 >>>>> >>>>> Again, it's not clear why there are multiple segments with the same >>>>> same virtual address, but I'm guessing that the one segment that starts >>>>> at 0x0000000004000000 is associated with the second kernel, and the other >>>>> ones are for vcpus that ran in the first kernel? >>>>> >>>>>> The guest is in the second kernel(vcpu = 1) >>>>>> [root@ghost ~]# readelf /tmp/vm2.save3 -l| grep 0xffffffff8 >>>>>> LOAD 0x0000000004001e4c 0xffffffff81000000 0x0000000004000000 >>>>>> >>>>>> I donot find differentiate qemu-genetated ELF headers from dump-generated ELF >>>>>> headers. >>>>> >>>>> Kdump-generated vmcores cannot have multiple START_KERNEL_map segments. >>>>> But with dumps where (vpcu = 1), there could be confusion since it's not obvious >>>>> if START_KERNEL_map region belongs to the first or second kernel. >>>>> >>>>> That being the case, I don't see how this can be supported cleanly by the crash' >>>>> utility unless there is a NOTE, or some other obvious identifier, that absolutely >>>>> confirms that the dumpfile was qemu-generated. >>>> >>>> The note information stored in qemu-generated core: >>>> Program Headers: >>>> Type Offset VirtAddr PhysAddr >>>> FileSiz MemSiz Flags Align >>>> NOTE 0x000000000000edd0 0x0000000000000000 0x0000000000000000 >>>> 0x0000000000000590 0x0000000000000590 0 >>>> >>>> I think its format is the same as kdump's vmcore. Does kdump-generated core's >>>> virtaddr is always 0? If so, What about to set virt_addr to -1 in qemu-generated >>>> core? >>>> >>> >>> In general, such characteristic should not be used. You should prepare >>> a solid interface. Even if using them, it should be limited to as >>> workaround to avoid some issue. >>> >>> Why not use qemu's CPU state? Include it as note information with good >>> name, and we can use it to distinguish which. Like: >>> >>> $ readelf -n vmcore >>> >>> Notes at offset 0x000001c8 with length 0x00000838: >>> Owner Data size Description >>> CORE 0x00000150 NT_PRSTATUS (prstatus structure) >>> CORE 0x00000150 NT_PRSTATUS (prstatus structure) >>> QEMU 0x00000557 Unknown note type: (0x00000000) >>> >>> Or QEMUCPUState is better? >> >> Good idea. I will try it, and hope gdb can also work. >> > > Tools basically ignore unknown notes. Looking into gdb, it appears to > ignore unknown information. > > static bfd_boolean > elfcore_grok_note (bfd *abfd, Elf_Internal_Note *note) > { > const struct elf_backend_data *bed = get_elf_backend_data (abfd); > > switch (note->type) > { > default: > return TRUE; > <cut> > > You might need to add new command to output contents of new note if > it's necessary. My goal is: 1. gdb uses NT_PRSTATUS, and can work well 2. crash uses unknown notes, and can get phys_base from it. Another question: What is QEMUCPUState? I donot find its definition? What note->type shoule be for "QEMU"? If we choose an unused value, the value may be used in the future.