{"id":6500,"date":"2011-12-21T12:24:10","date_gmt":"2011-12-21T12:24:10","guid":{"rendered":"http:\/\/www.beautifulwork.org\/?p=6500"},"modified":"2011-12-21T12:24:10","modified_gmt":"2011-12-21T12:24:10","slug":"hung_task_timeout_secs","status":"publish","type":"post","link":"https:\/\/www.trueangle.org\/index.php\/2011\/12\/21\/hung_task_timeout_secs\/","title":{"rendered":"hung_task_timeout_secs   ( modifying system parameters under \/proc\/sys file tree)"},"content":{"rendered":"<pre style=\"font-size:70%\">\nLinux Parameter \n\n$cat \/proc\/sys\/kernel\/hung_task_timeout_secs\n120\n$\n\nParameter Related\n\n\nTEST-MAIL1 ~ #dmesg\n[cut]\n\"echo 0 &gt; \/proc\/sys\/kernel\/hung_task_timeout_secs\" disables this message.\nrm D ffff88107f472c40 0 16705 22512 0x00000000\nffff881014693810 0000000000000086 ffff881000000000 ffff88102013b040\n0000000000012c40 ffff880471855fd8 0000000000012c40 ffff880471854010\nffff880471855fd8 0000000000012c40 ffff881017ff8e40 0000000100000000\nCall Trace:\n[] ? schedule_timeout+0x1ed\/0x2d0\n[] ? dlmlock+0x8a\/0xda0 [ocfs2_dlm]\n[] ? wait_for_common+0x12c\/0x1a0\n[] ? try_to_wake_up+0x280\/0x280\n[] ? __ocfs2_cluster_lock+0x1f0\/0x780 [ocfs2]\n[] ? wait_for_common+0x150\/0x1a0\n[] ? ocfs2_buffer_cached+0x8c\/0x180 [ocfs2]\n[] ? ocfs2_inode_lock_full_nested+0x126\/0x540 [ocfs2]\n[] ? ocfs2_lookup_lock_orphan_dir+0x6e\/0x1b0 [ocfs2]\n[] ? ocfs2_lookup_lock_orphan_dir+0x6e\/0x1b0 [ocfs2]\n[] ? ocfs2_prepare_orphan_dir+0x4a\/0x290 [ocfs2]\n[] ? ocfs2_unlink+0x6e1\/0xbb0 [ocfs2]\n[] ? may_link+0xda\/0x170\n[] ? vfs_unlink+0x9e\/0x100\n[] ? do_unlinkat+0x1a1\/0x1d0\n[] ? vfs_readdir+0xa0\/0xe0\n[] ? fsnotify_find_inode_mark+0x2b\/0x40\n[] ? dnotify_flush+0x54\/0x110\n[] ? filp_close+0x5c\/0x90\n[] ? system_call_fastpath+0x16\/0x1b\n\n\nClassroom\n\nWhile  waiting for  read()  or write()  to\/from  a file  descriptor\nreturn, the process  will be put in a special  kind of sleep, known\nas \"D\"  or \"Disk Sleep\". This  is special, because  the process can\nnot  be killed  or interrupted  while in  such a  state.  A process\nwaiting for  a return from  ioctl() would also  be put to  sleep in\nthis manner.\n\nsource related :http:\/\/stackoverflow.com\/questions\/1475683\/linux-process-states\n\nParameter Code Internals\n\n\n\/*\n * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for\n * a really long time (120 seconds). If that happens, print out\n * a warning.\n *\/\nstatic void check_hung_uninterruptible_tasks(unsigned long timeout)\n{\n\tint max_count = sysctl_hung_task_check_count;\n\tint batch_count = HUNG_TASK_BATCHING;\n\tstruct task_struct *g, *t;\n\n\t\/*\n\t * If the system crashed already then all bets are off,\n\t * do not report extra hung tasks:\n\t *\/\n\tif (test_taint(TAINT_DIE) || did_panic)\n\t\treturn;\n\n\trcu_read_lock();\n\tdo_each_thread(g, t) {\n\t\tif (!max_count--)\n\t\t\tgoto unlock;\n\t\tif (!--batch_count) {\n\t\t\tbatch_count = HUNG_TASK_BATCHING;\n\t\t\trcu_lock_break(g, t);\n\t\t\t\/* Exit if t or g was unhashed during refresh. *\/\n\t\t\tif (t-&gt;state == TASK_DEAD || g-&gt;state == TASK_DEAD)\n\t\t\t\tgoto unlock;\n\t\t}\n\t\t\/* use \"==\" to skip the TASK_KILLABLE tasks waiting on NFS *\/\n\t\tif (t-&gt;state == TASK_UNINTERRUPTIBLE)\n\t\t\tcheck_hung_task(t, timeout);\n\t} while_each_thread(g, t);\n unlock:\n\trcu_read_unlock();\n}\n\n\nRelated From Research Paper\n\nKernel  data collection  tools. Several  monitoring  facilities are\nprovided by  the Linux  kernel, which have  been exploited  in this\nwork. In  particular, we use  KProbes which inserts  breakpoints in\narbitrary binary code locations in charge of triggering user-defined\nhandler  functions. Handlers  can  be used  to collect  information\nabout internal kernel  variables; subsequently, kernel execution is\nrestored. Kdump is a tool  for failure data collection based on the\nexecution of  a secondary kernel,  namely capture kernel,  which is\npreliminarily  loaded  into  a  reserved memory  region.  When  the\nprimary kernel fails, the capture  kernel is executed; then, it can\ncollect failure  data by reading  the main memory  state.  Built-in\nhang  detection mechanisms. Several  hang detection  mechanisms are\navailable in the Linux OS,  which can be enabled by recompiling the\nkernel.  In particular, the  following facilities  can be  used for\nhang  detection: Soft  lockup detection,  i.e., the  kernel detects\nwhether a  \"canary\" task  is not scheduled  within a  timeout; Hard\nlockup detection, i.e.,  if any CPU in the  system does not handles\nlocal    timer    interrupt   for    longer    than   a    timeout;\nSleep-inside-spinlock   checking,  i.e.,  assertions   that  verify\nwhether there are spinlocks  that have been acquired before calling\na  sleeping function  (i.e., a  function during  which  the current\nthread may block and be preempted by the scheduler); Checks on lock\nAPI  usage, that  is: missing  lock initialization,  release  of an\nalready freed lock, release of a  lock by a thread or CPU different\nfrom the lock holder, lock data structure corruption.\n\nsource : http:\/\/tinyurl.com\/7pt5j9a\n\nAssessment and Improvement of Hang Detection in the Linux Operating System\n2009 28th IEEE International Symposium on Reliable Distributed Systems\n\n\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Linux Parameter $cat \/proc\/sys\/kernel\/hung_task_timeout_secs 120 $ Parameter Related TEST-MAIL1 ~ #dmesg [cut] &#8220;echo 0 &gt; \/proc\/sys\/kernel\/hung_task_timeout_secs&#8221; disables this message. rm D ffff88107f472c40 0 16705 22512 0x00000000 ffff881014693810 0000000000000086 ffff881000000000 ffff88102013b040 0000000000012c40 ffff880471855fd8 0000000000012c40 ffff880471854010 ffff880471855fd8 0000000000012c40 ffff881017ff8e40 0000000100000000 Call Trace: [] ? schedule_timeout+0x1ed\/0x2d0 [] ? dlmlock+0x8a\/0xda0 [ocfs2_dlm] [] ? wait_for_common+0x12c\/0x1a0 [] ? try_to_wake_up+0x280\/0x280 [] ? &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.trueangle.org\/index.php\/2011\/12\/21\/hung_task_timeout_secs\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;hung_task_timeout_secs   ( modifying system parameters under \/proc\/sys file tree)&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"page-templates\/full-width.php","format":"standard","meta":[],"categories":[83],"tags":[913,1545],"_links":{"self":[{"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/posts\/6500"}],"collection":[{"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/comments?post=6500"}],"version-history":[{"count":0,"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/posts\/6500\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/media?parent=6500"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/categories?post=6500"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.trueangle.org\/index.php\/wp-json\/wp\/v2\/tags?post=6500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}