Re: [Crash-utility] crash-5.0: zero-size memory-allocation

----- "ville mattila" <ville.mattila(a)stonesoft.com&gt; wrote: Nice bug report! Here's what's happening: It's related to this patch that went into 4.1.0: - Fix for a potential failure to initialize the kmem slab cache subsystem on 2.6.22 and later CONFIG_SLAB kernels if the dumpfile has pages excluded by the makedumpfile facility. Without the patch, the following error message would be displayed during initialization: "crash: page excluded: kernel virtual address: <address> type: kmem_cache_s buffer", followed by "crash: unable to initialize kmem slab cache subsystem". (anderson(a)redhat.com) The patch was put in place due to this definition of the kmem_cache data structure: struct kmem_cache { /* 1) per-cpu data, touched during every alloc/free */ struct array_cache *array[NR_CPUS]; /* 2) Cache tunables. Protected by cache_chain_mutex */ unsigned int batchcount; unsigned int limit; ... [ snip ] ... * We put nodelists[] at the end of kmem_cache, because we want to size * this array to nr_node_ids slots instead of MAX_NUMNODES * (see kmem_cache_init()) * We still use [MAX_NUMNODES] and not [1] or [0] because cache_cache * is statically defined, so we reserve the max number of nodes. */ struct kmem_list3 *nodelists[MAX_NUMNODES]; /* * Do not add fields after nodelists[] */ }; where for all kernel instances of the kmem_cache data structure *except* for the head "cache_cache" kmem_cache structure, every other kmem_cache structure in the kernel has its nodelists[] array downsized to whatever "nr_node_ids" is initialized to. The actual size of all of the downsized kmem_cache data structures can be found in the head "cache_cache.buffer_size" field. But when the crash utility queries gdb for the size of a kmem_cache structure it gets the "full" size as declared in the vmlinux debuginfo data. And so whenever a kmem_cache structure was read by crash, it was using the "full" size instead of the downsized size. Doing that type of over-sized read could potentially extend into the next page, and there was a reported case where doing that happened to extend into a page that was excluded by makedumpfile. Hence the kmem_cache_downsize() function added to memory.c. Anyway, given that your debug output shows: kmem_cache_downsize: SIZE(kmem_cache_s): 204 cache_cache.buffer_size: 0 kmem_cache_downsize: nr_node_ids: 1 In vm_init() there was an initial STRUCT_SIZE_INIT(kmem_cache_s, ...) that set the size to 204 bytes. But then kmem_cache_downsize() was called to downsize to whatever cache_cache.buffer_size contains: ... buffer_size = UINT(cache_buf + MEMBER_OFFSET("kmem_cache", "buffer_size")); if (buffer_size < SIZE(kmem_cache_s)) { ASSIGN_SIZE(kmem_cache_s) = buffer_size; if (kernel_symbol_exists("nr_node_ids")) { get_symbol_data("nr_node_ids", sizeof(int), &nr_node_ids); vt->kmem_cache_len_nodes = nr_node_ids; } else vt->kmem_cache_len_nodes = 1; if (CRASHDEBUG(1)) { fprintf(fp, "\nkmem_cache_downsize: SIZE(kmem_cache_s): %ld " "cache_cache.buffer_size: %d\n", STRUCT_SIZE("kmem_cache"), buffer_size); fprintf(fp, "kmem_cache_downsize: nr_node_ids: %ld\n", vt->kmem_cache_len_nodes); } } But your kernel shows cache_cache.buffer_size set to zero -- and the ASSIGN_SIZE(kmem_cache_s) above dutifully downsized the data structure size from 204 to zero. Later on, that size was used to allocate a kmem_cache buffer, which failed when a GETBUF() was called with a zero-size. I guess a check could be made above for a zero cache_cache.buffer_size, but why would that ever be? Try this: # crash --no_kmem_cache vmlinux vmcore which will allow you to get past the kmem_cache initialization. Then enter: crash> p cache_cache Does the "buffer_size" member really show zero? BTW, you can work around the problem by commenting out the call to kmem_cache_downsize() in vm_init(). (And if you're using makedumpfile with excluded pages, hope that the problem I described above doesn't occur...) Dave