Re: [Crash-utility] "crash: cannot allocate any more memory!" for 2.6.27.4

----- "Dheeraj Sangamkar" <dheerajrs(a)gmail.com&gt; wrote: Live system analysis is impossible if /dev/mem won't allow you to read physical memory. Each read attempt above results in an "Operation not permitted" (EPERM) error. They fail because of a pain-in-the-ass CONFIG_STRICT_DEVMEM configuration option that was recently added to upstream kernels. When the crash utility makes a read() from /dev/mem, it ends up in the kernel's drivers/char/mem.c driver, and proceeds like so: read_mem() range_is_allowed() devmem_is_allowed() Here is the call in read_mem() that returns the EPERM: if (!range_is_allowed(p >> PAGE_SHIFT, count)) return -EPERM; Here are the two versions of range_is_allowed() in drivers/char/mem.c, which depend upon CONFIG_STRICT_DEVMEM: #ifdef CONFIG_STRICT_DEVMEM static inline int range_is_allowed(unsigned long pfn, unsigned long size) { u64 from = ((u64)pfn) << PAGE_SHIFT; u64 to = from + size; u64 cursor = from; while (cursor < to) { if (!devmem_is_allowed(pfn)) { printk(KERN_INFO "Program %s tried to access /dev/mem between %Lx->%Lx.\n", current->comm, from, to); return 0; } cursor += PAGE_SIZE; pfn++; } return 1; } #else static inline int range_is_allowed(unsigned long pfn, unsigned long size) { return 1; } #endif And the call to devmem_is_allowed() above ends up here, where only the first 256 pages (1MB) of physical memory is accessible: int devmem_is_allowed(unsigned long pagenr) { if (pagenr <= 256) return 1; if (!page_is_ram(pagenr)) return 1; return 0; } So that makes /dev/mem useless for the crash utility... Bernhard Walle made a valiant attempt to make this a run-time configurable rather than a compile-time config option, http://lkml.org/lkml/2008/11/16/117 But the LKML thread went off into the weeds arguing whether the CONFIG_STRICT_DEVMEM imposition itself should be ripped out, rather than putting lipstick on a pig. And so it got nowhere... This same 256-page restriction has been in place in RHEL4 and RHEL5 kernels as well, because the same guy who managed to push it upstream formerly worked at Red Hat, and convinced the powers-that-be here that /dev/mem was a security hole. As a result, I had to write a whole new physical memory-access device driver (/dev/crash) that is a read-only "misc" driver that the crash utility automatically loads when run on a live system. But it's only available in the Red Hat RHEL4 and RHEL5 distribution, although it can be ported easily enough. So you've got 3 options: 1. Rebuild your kernel with CONFIG_STRICT_DEVMEM turned off. 2. Port the RHEL /dev/crash driver to your kernel. I did suggest that in another crash-utility thread here: https://www.redhat.com/archives/crash-utility/2008-October/msg00057.html 3. Write a kprobe module that forces devmem_is_allowed() to return 1 always: http://www.redhat.com/archives/crash-utility/2008-March/msg00036.html If you've got the power, obviously #1 is the best option. But if you cannot change kernels, probably the kprobe option #3 is easier than porting the RHEL /dev/crash driver. Check Documentation/kprobes.txt file and the samples/kprobes directory in your kernel build tree, and then use the RHEL kprobe example I put in the thread referenced in #3 above. If that won't work (and I don't know why it wouldn't), then you can always port the RHEL /dev/crash driver that's attached to the thread reference in #2 above. Dave