issue: runq -g
by vinayak menon
Hi Dave,
I was getting the following error upon executing "runq -g".
runq: invalid structure member offset: cfs_rq_throttled
FILE: task.c LINE: 7615 FUNCTION: print_group_header_fair()
[./crash] error trace: 80e98b0 => 80e9815 => 80e577f => 8143bde
TASK_GROUP: dac06c80 CFS_RQ: dac06080 <apps>runq: input string
too large: "000000000808ec50" (16 vs 8)
"cfs_bandwidth" is available in task_group irrespective of whether
CONFIG_CFS_BANDWIDTH is enabled or not (but empty). But "throttled" is
available in cfs_rq only if config is enabled. The following patch
fixes this.
diff --git a/task.c b/task.c
index dc0310a..a119c60 100755
--- a/task.c
+++ b/task.c
@@ -7611,7 +7611,7 @@ print_group_header_fair(int depth, ulong cfs_rq, void *t)
if (tgi->name)
fprintf(fp, " <%s>", tgi->name);
- if (VALID_MEMBER(task_group_cfs_bandwidth)) {
+ if (VALID_MEMBER(cfs_rq_throttled)) {
readmem(cfs_rq + OFFSET(cfs_rq_throttled), KVADDR,
&throttled, sizeof(int), "cfs_rq throttled",
FAULT_ON_ERROR);
Thanks,
Vinayak
10 years, 11 months
transaction_t in jbd module vs. transaction_t in jbd2 module
by Brad Hubbard
Hello,
We recently encountered an issue where a transaction_t struct appeared
to be corrupt but this ended up being a symbol conflict as the customer
was using ext3 and ext4 and had both modules in use. Unloading the jbd
module did not help and the only way to see the correct representation
of the the transaction_t struct in question was to restart crash without
any modules loaded and load only the jbd2 module.
Is this a known issue? Is there a strategy that needs to be followed for
dealing with duplicate symbols?
--
Kindest Regards,
Brad Hubbard
Software Maintenance Engineer
Red Hat Global Support Services
Asia Pacific Region
10 years, 11 months
[ANNOUNCE] crash 7.0.4 is available
by Dave Anderson
Download from: http://people.redhat.com/anderson
Changelog:
- Fix for the "ps" command's display of per-task RSS and %MEM values
in Linux 2.6.34 and later kernels in which SPLIT_RSS_COUNTING is
enabled. Without the patch, the values are only taken from each
task's mm_struct.rss_stat structure, which may contain stale values
because they may not be synchronized with the RSS values stored
in each per-thread task_struct.rss_stat structure; this may lead
to invalid or slightly low RSS values, and worst-case, the %MEM
value may show garbage percentage values.
(vinayakm.list(a)gmail.com)
- Addressed a few (harmless) Coverity Scan complaints in diskdump.c:
1579:dead_error_line – Execution cannot reach this expression ""|""
inside statement "fprintf(fp, "%sDUMP_DH_COMP...".
1574:dead_error_line – Execution cannot reach this expression ""|""
inside statement "fprintf(fp, "%sDUMP_HEADER_...".
1571:dead_error_line – Execution cannot reach this expression ""|""
inside statement "fprintf(fp, "%sDUMP_HEADER_...".
(anderson(a)redhat.com)
- Addressed two warnings when compiling diskdump.c on 32-bit architectures
when the snappy library is built in:
diskdump.c:1046: warning: passing argument 3 of
'snappy_uncompressed_length' from incompatible pointer type
/usr/include/snappy-c.h:120: note: expected ‘size_t *’ but argument
is of type ‘ulong *’
diskdump.c:1056: warning: passing argument 4 of ‘snappy_uncompress’
from incompatible pointer type
/usr/include/snappy-c.h:103: note: expected ‘size_t *’ but argument
is of type ‘ulong *’
(anderson(a)redhat.com)
- Created a simpler interface with the internal do_list() function.
Currently, if a caller wants to gather the contents of a list into
an array, it must do the following:
(1) call hq_open() so that the list contents will be verified and
saved in the hash queue
(2) call do_list() to store the list in the hash queue and return
the number of entries in the list
(3) allocate a buffer to store the array of entries in the list
(4) pass the allocated buffer to retrieve_list() to be populated
from the hash queue
(5) call hq_close()
With this patch, if the passed-in list_data.flags field has a new
LIST_ALLOCATE bit set, then do_list() will perform steps (1), (3),
(4) and (5) above. The caller can access the allocated array via
a new list_data.list_ptr member, and, when done parsing the list,
the allocated buffer should be returned via FREEBUF(). The only
restriction is that the hash queue cannot be currently in use, or
the do_list() call will fail. It should also be noted that there
are circumstances where it still makes sense that steps (1), (3),
(4) and (5) are performed by the do_list() caller.
(anderson(a)redhat.com)
- Modified the internal parent_list() function, used by "ps -p", to
utilize the simpler new do_list() functionality described above.
(anderson(a)redhat.com)
- Modified the internal dump_vmap_area() function, used by the
"kmem -v", "kmem <address>" and "search" commands, to utilize
the simpler new do_list() functionality.
(anderson(a)redhat.com)
- Modified the internal nr_blockdev_pages() function, used by the
"kmem -i", to utilize the simpler new do_list() functionality.
(anderson(a)redhat.com)
- Modified the internal get_mount_list() function, used by the
"mount" and "files -d" commands, to utilize the simpler new
do_list() functionality.
(anderson(a)redhat.com)
- Modified the internal get_kmem_cache_list() function, used by the
"kmem -[sS]", "kmem <address>", "rd -S" and "bf -F" commands,
to utilize the simpler new do_list() functionality.
(anderson(a)redhat.com)
- Modified the internal show_net_devices_v2() and show_net_devices_v3()
functions, used by the "net" command, to utilize the simpler new
do_list() functionality.
(anderson(a)redhat.com)
- The "help -r" option has been extended to dump the X86 and X86_64
registers stored in the NT_PRSTATUS notes in netdump ELF, kdump ELF,
and compressed kdump dumpfiles. Without the patch, the option only
supports ELF dumpfiles created by the "virsh dump --memory-only"
facility.
(anderson(a)redhat.com)
- Modified the "runq -g" display to show the CFS task_group and cfs_rq
addresses; also the current task is also displayed in its CFS or RT
queue with with the notation "[CURRENT]" appended to the task data.
(Anthony.Chen(a)Teradata.com)
- Additional modification to the "runq -g" to display the task_group
and rt_rq addresses for the RT queues, similar in nature to the CFS
queue changes are done by the patch above. In addition, the CFS
rb_root and RT prio_array addresses are no longer shown given that
they can be determined by looking at the cfs_rq and rt_rq structures
whose addresses are now displayed.
(anderson(a)redhat.com)
- Modified the behavior of the "mod -t" option when running against
Linux 2.6.18 and earlier kernels such that the hexadecimal value of
the module->license_gplok member is always displayed. Without the
patch, if a module's license_gplok boolean or bitmask value is 0,
it would only be displayed if the module was unsigned.
(anderson(a)redhat.com, atomlin(a)redhat.com)
- Modified the internal dump_tasks_in_task_group_rt_rq() and
dump_RT_prio_array() functions, used by the "runq" command,
to utilize the simpler new do_list() functionality.
(anderson(a)redhat.com)
- Resurrection of the remote analysis capability for use with the
"xen-crashd" daemon running on a Xen Dom0 host, which communicates
with a paused or shutdown DomU guest kernel. The daemon can be
accessed like so:
$ crash localhost:5001,/dev/xenmem vmlinux
(dslutz(a)verizon.com)
- Prevent the X86_64 "bt" command from using starting RSP and RIP
values taken from the NT_PRSTATUS notes of kdump dumpfiles if the
RSP address is not in the task's kernel stack, or in any of the
relevant per-cpu exception stacks. This can happen when the number
of NT_PRSTATUS notes does not match the number of online cpus.
Without the patch, the command may generate a segmentation violation,
fail with the error message "bt: cannot determine starting stack
pointer", or fail with an error message indicating that the command
"cannot transition" from an exception stack to the previous stack.
(anderson(a)redhat.com)
- Fix for the X86_64 "bt" command for displaying the backtraces of
active tasks running on the non-crashing cpus in kdump dumpfiles
in which the "crash_nmi_callback" function frame does not appear
in the per-cpu NMI exception stacks. That function frame is
normally used as the starting point for the backtraces of those
tasks, but if it does not exist, the "notify_die" frame will be
used instead. Without the patch, the backtraces of the active
non-crashing tasks are incorrect or incomplete.
(anderson(a)redhat.com)
- Increment the X86_64 NR_CPUS maximum value from 5120 to 8192
to be able to account for Linux 3.13-rc1 and later kernels
in which CONFIG_MAXSMP has been configured. If that is true,
the CONFIG_NR_CPUS value is overridden, and the kernel NR_CPUS
value will be set to 8192. This will cause the crash session to
fail with the messages "WARNING: kernel-configured NR_CPUS (8192)
greater than compiled-in NR_CPUS (5120)" and "crash: recompile crash
with larger NR_CPUS".
(anderson(a)redhat.com)
- A crash-7.0.3 fix for the proper determination of the kernel NR_CPUS
configurable for Linux 3.8 and later kernels introduced a regression
in Linux 3.8 and later kernels if:
(1) the kernel is configured with CONFIG_SLAB, and
(2) the sum of the kernel's NR_CPUS and MAX_NUMNODES values exceed
the NR_CPUS value compiled into the crash utility.
Without the patch, the crash session generates a segmentation fault
while it indicates "please wait: gathering kmem slab cache data".
(anderson(a)redhat.com)
- Update for the extensions/trace.c extension module to support the
ftrace data structure changes introduced in Linux 3.10 kernels.
Without the patch, loading the module with the "extend" command fails
with the error message "extend: trace.so: no commands registered:
shared object unloaded".
(d.hatayama(a)jp.fujitsu.com)
- Increment the S390 and S390X NR_CPUS maximum value from 64 to 512.
(holzheu(a)linux.vnet.ibm.com)
- Implemented support for the redesigned per-slab object bookkeeping
that was introduced in Linux 3.13-rc1 for kernels configured with
CONFIG_SLAB. In those kernels, the head page structure associated
with each slab is overloaded to serve as the (now removed) slab data
structure, and the array of kmem_bufctl_t data structures that used
to be appended to each slab data structure for object management has
been replaced by a new freelist stack mechanism. Without the patch,
the crash session would fail during initialization with the error
message "crash: invalid structure member offset: kmem_cache_s_c_num".
It should be noted that this patch has only been tested on 3.13-rc1
kernels, which do not have the modified freelist_idx_t in place as
of yet, so the replacement of integer-sized indexes with byte or
short sized indexes had not been checked in. Furthermore, if this
proposed patch set gets accepted:
[RFC][PATCH 0/3] re-shrink 'struct page' when SLUB is on.
https://lkml.org/lkml/2013/12/11/589
then both CONFIG_SLAB and CONFIG_SLUB support in the crash utility
will be broken yet again.
(anderson(a)redhat.com)
- In order to facilitate the building of the crash binary with either
or both of the optional LZO or SNAPPY compression libraries, two new
Makefile targets have been added:
$ make lzo
$ make snappy
Without the patch, the CFLAGS.extra and LDFLAGS.extra files must be
created or modified as described in these changelog entries:
http://people.redhat.com/anderson/crash.changelog.html#LZO
http://people.redhat.com/anderson/crash.changelog.html#SNAPPY
This patch simply does the work automatically. After having done it
one time, there is no need to use the targets for subsequent builds.
The relevant libraries must pre-exist on the build machine.
(anderson(a)redhat.com)
- Long overdue update of the README file.
(anderson(a)redhat.com)
10 years, 11 months
[PATCH] s390(x): Increase NR_CPUS to 512
by Michael Holzheu
Hello Dave,
For s390(x) in the near future kernels can have more than 64 CPUs. So
change the defines in "defs.h" accordingly.
Signed-off-by: Michael Holzheu <holzheu(a)linux.vnet.ibm.com>
---
defs.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- a/defs.h
+++ b/defs.h
@@ -125,10 +125,10 @@
#define NR_CPUS (2048)
#endif
#ifdef S390
-#define NR_CPUS (64)
+#define NR_CPUS (512)
#endif
#ifdef S390X
-#define NR_CPUS (64)
+#define NR_CPUS (512)
#endif
#ifdef ARM
#define NR_CPUS (4)
10 years, 11 months
getting values of local variable
by Vivek Anand
How can i get the value of a structure variable used in a kernel module.The
structure is private to the module. Using struct command did not yield any
result as the structure type is unknown to crash. Thanks.
--
Vivek Anand T Kallampally
10 years, 11 months
