Re: [Crash-utility] dom0 analysis for IA64

Hi Dave, This is because IA64's pseudo-physical memory map (domain on xen specific). phys-to-machine mapping is managed as 3-level page table. pgd looks like: ------------------------------------------------------------- crash> doms DID DOMAIN ST T MAXPAGE TOTPAGE VCPU SHARED_I P2M_MFN 32753 f000000007dac080 ?? O 0 0 0 0 ---- 32754 f000000007ff0080 ?? X 0 0 0 0 ---- 32767 f000000007ff4080 ?? I 0 0 1 0 ---- crash> domain f000000007da4080 struct domain { domain_id = 0, shared_info = 0xf000000007d90000, ... arch = { mm = { pgd = 0xf00000007d8b0000 }, ... crash> rd 0xf00000007d8b0000 256 f00000007d8b0000: 000000007c8d8000 0000000000000000 ...|............ f00000007d8b0010: 0000000000000000 0000000000000000 ................ f00000007d8b0020: 0000000000000000 0000000000000000 ................ f00000007d8b0030: 0000000000000000 0000000000000000 ................ f00000007d8b0040: 0000000000000000 0000000000000000 ................ f00000007d8b0050: 0000000000000000 0000000000000000 ................ f00000007d8b0060: 0000000000000000 0000000000000000 ................ f00000007d8b0070: 0000000000000000 0000000000000000 ................ f00000007d8b0080: 000000007f428000 0000000000000000 ..B............. f00000007d8b0090: 0000000000000000 0000000000000000 ................ ... f00000007d8b07c0: 0000000000000000 0000000000000000 ................ f00000007d8b07d0: 0000000000000000 0000000000000000 ................ f00000007d8b07e0: 0000000000000000 0000000000000000 ................ f00000007d8b07f0: 0000000000000000 000000007bed4000 .........@.{.... ------------------------------------------------------------------------- (256 * 2048 = 524288) It is certain that (pseudo-)physical memory "256GB-" and "-4TB" exits. These area are shared by domain-0 and xen hypervisor. These area should be accessed in dom0's analysis session. (I said:) It is wrong. This should be "the memory usage is large if pseudo-physical memory layout is sparse." And it is always sparse actually... Thanks. On Thu, 10 May 2007 11:15:49 -0400 Dave Anderson <anderson(a)redhat.com&gt; wrote: > Itsuro ODA wrote: > > > Hi Dave, > > > > The attached patch enables to analyze dom0 linux from > > whole memory dump on IA64. (for crash-4.0-4.1) > > It is just quick hack. > > (I was asked from IA64 Xen developers and made it.) > > > > Each domain manages own machine memory by domain.arch.mm.pgd > > in IA64. It is 3-level page table. > > I thougnt the mfn of domain.arch.mm.pgd can be regarded as > > p2m_mfn. > > > > I intended to modify as less existent code as possible. > > (maybe xen_kdump_p2m should be prepare for each arch ?) > > > > Would you consider to support dom0 analysis for IA64 ? > > > > I prepared two sample dumps. Please find from the following > > URLs. > > > > 1) http://people.valinux.co.jp/~oda/20070510-sample-dump-1.tar > > contents: > > - vmcore.gz > > This is taken by a hard assist dump. netdump style ELF vmcore. > > So XEN_ELFNOTE_CRASH_INFO does not exist. > > - vmcore.ka.gz > > It is coverted to kdump style and added XEN_ELFNOTE_CRASH_INFO > > manually. > > - vmlinux.debug.gz > > for dom0 analysis > > - xen-syms-2.6.18-8.el5.gz > > for xencrash > > > > To get p2m_mfn, xencrash's doms command is usefull. > > -------------------------------------------------------------------------- > > # crash xen-syms-2.6.18-8.el5 vmcore > > ... > > crash> doms > > DID DOMAIN ST T MAXPAGE TOTPAGE VCPU SHARED_I P2M_MFN > > 32753 f000000007ac8080 RU O 0 0 0 0 ---- > > 32754 f000000007acc080 RU X 0 0 0 0 ---- > > > 32767 f000000007ff8080 RU I 0 0 4 0 ---- > > 0 f000000007aa4080 RU 0 10000 fc28 1 f000000007a88000 1abb7 > > >* 1 f000000007a78080 RU U 10603 10603 3 f000000007a5c000 1a909 > > crash> > > ---------------------------------------------------------------------------- > > > > Then normal crash session with --p2m_mfn option. > > ---------------------------------------------------------------------------- > > # crash --p2m_mfn=1abb7 vmlinux.debug vmcore > > > > I'm still downloading the above, so I haven't been able > to test it yet... > > > ... > > ---------------------------------------------------------------------------- > > > > vmcore.ka has XEN_ELFNOTE_CRASH_INFO. so --p2m_mfn option not need. > > ---------------------------------------------------------------------------- > > # crash vmlinux.debug vmcore.ka > > ... > > ---------------------------------------------------------------------------- > > > > --p2m_mfn option is effective only if a vmcore has XEN_ELFNOTE_CRASH_INFO > > now. > > I think specifying --p2m_mfn option is regarded as the vmcore is > > XEN_CORE_DUMPFILE(). The patch supports this. > > I think it is necessary for dumps which does not have > > XEN_ELFNOTE_CRASH_INFO such as above sample. > > > > 2) http://people.valinux.co.jp/~oda/20070510-sample-dump-2.tar > > contents: > > - vmcore.tiger.iomem_machine.gz > > taken by Xen kdump > > - vmlinux-xen-ia64.bz2 > > - xen-syms-ia64.bz2 > > > > I asked Xen kdump developper (simon@valinux) to add "p2m_mfn" to > > XEN_ELFNOTE_CRASH_INFO. > > So this change of Xen kdump is not open yet. > > If this is OK for crash, it will be commited. > > > > Thanks. > > -- > > Itsuro ODA <oda(a)valinux.co.jp&gt; > > > > > $ ./crash vmlinux-xen-ia64 vmcore.tiger.iomem_machine > ... > crash> help -n > ... > xen_kdump_data: > flags: 5 (KDUMP_P2M_INIT|KDUMP_MFN_LIST) > p2m_mfn: 1f62c > cr3: 0 > last_mfn_read: 1fd09 > page: 6000000000c96bd0 > accesses: 1340 > cache_hits: 1255 (95%) > p2m_frames: 524288 > p2m_mfn_frame_list: 200000000539c010 > 1efba 1f5cc 1fd09 1e185 1d984 1d183 1c982 1c181 1b980 1b17f 1a97e 1a17d 1997c 1917b 1897a 18179 17978 17177 16976 16175 15974 > 15173 14972 14171 13970 1316f 1296e 1216d 1196c 1116b 1096a 10169 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 1efb8 1efb7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 1efb6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > > If you enter "crash -d2", you'll also see all 512k, mostly useless, > entries... > > Dave > > -- Itsuro ODA <oda(a)valinux.co.jp&gt;