Re: [Crash-utility] crash: cannot gather a stable task list via pid_hash (500 retries)

Eugene Teo wrote: Hi Eugene, I can't reproduce this one -- here on a freshly-installed x86_64 running 2.6.24.3-12.fc8: # crash crash 4.0-6.1 Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008 Red Hat, Inc. Copyright (C) 2004, 2005, 2006 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb 6.1 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"... KERNEL: /usr/lib/debug/lib/modules/2.6.24.3-12.fc8/vmlinux DUMPFILE: /dev/crash CPUS: 8 DATE: Mon Mar 17 11:40:59 2008 UPTIME: 00:17:01 LOAD AVERAGE: 0.50, 0.18, 0.06 TASKS: 169 NODENAME: hp-dl585g2-01.rhts.boston.redhat.com RELEASE: 2.6.24.3-12.fc8 VERSION: #1 SMP Tue Feb 26 14:21:30 EST 2008 MACHINE: x86_64 (2812 Mhz) MEMORY: 8 GB PID: 2938 COMMAND: "crash" TASK: ffff8102440f48b0 [THREAD_INFO: ffff81027d912000] CPU: 1 STATE: TASK_RUNNING (ACTIVE) crash> From the debug output, there's no useful info re: task ffff81012f0811d0 other than the fact that it's being seen as a duplicate in a pid_hash chain, and that it doesn't appear to be a temporary "shifting-sands" condition because 500 retries are not alleviating the condition. But you can probably tinker with the task refresh function in the crash utility to skip it, and then investigate why it's there twice. The crash function should be refresh_hlist_task_table_v3(): crash> help -t | grep refresh refresh_task_table: refresh_hlist_task_table_v3() crash> This piece here in refresh_hlist_task_table_v3() is what gets retried a maximum of 500 times: if (!is_idle_thread(next) && !hq_enter(next)) { error(INFO, "%sduplicate task in pid_hash: %lx\n", DUMPFILE() ? "\n" : "", next); if (DUMPFILE()) break; hq_close(); retries++; goto retry_pid_hash; } Try modifying it to either "continue", or perhaps just "break". If that's the only irregularity found, you should be able to get a "crash>" prompt; and then you can look at task ffff81012f0811d0, because the first instance found in the pid_hash chain should be listed by "ps" as a task. Dave