[Ace-users] [ace-bugs] trying to fix an ACE hang

Douglas C. Schmidt schmidt at dre.vanderbilt.edu
Thu Nov 29 15:43:56 CST 2007


Hi Greg,

>YES! I think found exactly this issue:
>
>=20
>
>Sun May  5 19:14:34 2002  Douglas C. Schmidt
><schmidt at macarena.cs.wustl.edu>
>
>=20
>
>                * ace/Message_Queue_T.cpp: Modified all the enqueue*()
>methods so that
>
>                  their calls to notify() occur *outside* of the monitor
>lock.
>
>                  This change prevents deadlock from occurring when a
>reactor's
>
>                  notification pipe is full.  Thanks to Sasha Agranov
>
>                  <sagranov at COMGATES.co.il> for reporting this.
>
>=20
>
>Did it creep back in somehow?

I suspect if you look further along in the changelog entries you'll
find that this change was reverted since it broke other things..

Thanks,

Doug

>=20
>
>greg
>
>=20
>
>From: Steve Huston [mailto:shuston at riverace.com]=20
>Sent: Thursday, November 29, 2007 4:15 PM
>To: Greg Popovitch; ace-bugs at cse.wustl.edu
>Subject: RE: [ace-bugs] trying to fix an ACE hang
>
>=20
>
>Hi Greg,
>
>=20
>
>Thanks for the PROBLEM-REPORT-FORM.
>
>=20
>
>Doug poked me to see if I remembered this type of problem. Your
>description looks familiar - I've a feeling we've been down this road
>before, but can't recall the details. Could you please scan through the
>ACE_wrappers/ChangeLogs files for changes in this area to see if we've
>tried this fix before?
>
>=20
>
>Thanks!
>-Steve
>
>--
>Steve Huston, Riverace Corporation
>Want to take ACE training on YOUR schedule?
>See http://www.riverace.com/training.htm
>
>	-----Original Message-----
>	From: ace-bugs-bounces at cse.wustl.edu
>[mailto:ace-bugs-bounces at cse.wustl.edu] On Behalf Of Greg Popovitch
>	Sent: Thursday, November 29, 2007 2:23 PM
>	To: ace-bugs at cse.wustl.edu
>	Subject: [ace-bugs] trying to fix an ACE hang
>
>	Hi,
>
>	=20
>
>	I have a hang on Windows Vista 64 and also on linux 64 (RHEL 4).
>
>
>	=20
>
>	One thread is enqueuing a message on an ACE_Message_Queue for
>another thread and hangs on the notification to the WFMO reactor. The
>ACE_WFMO_Reactor_Notify::notify() hangs because its internal message
>queue is full.
>
>	=20
>
>	The other thread hangs while checking of the message_queue is
>empty because it can't grab its ACE_GUARD. Therefore it can't dequeue
>messages from the WFMO_Reactor.
>
>	=20
>
>	My question:=20
>
>	=20
>
>	In file Message_Queue_T.cpp,
>
>	=20
>
>	Would it be OK to move the "this->notify ()" outside of the
>scope of the ACE_GUARD in:
>
>	=20
>
>	template <ACE_SYNCH_DECL> int
>
>	ACE_Message_Queue<ACE_SYNCH_USE>::enqueue_head
>(ACE_Message_Block *new_item,
>
>	                                                ACE_Time_Value
>*timeout)
>
>	{
>
>	  ACE_TRACE ("ACE_Message_Queue<ACE_SYNCH_USE>::enqueue_head");
>
>	  int queue_count =3D 0;
>
>	  {
>
>	    ACE_GUARD_RETURN (ACE_SYNCH_MUTEX_T, ace_mon, this->lock_,
>-1);
>
>	=20
>
>	    if (this->state_ =3D=3D ACE_Message_Queue_Base::DEACTIVATED)
>
>	      {
>
>	        errno =3D ESHUTDOWN;
>
>	        return -1;
>
>	      }
>
>	=20
>
>	    if (this->wait_not_full_cond (ace_mon, timeout) =3D=3D -1)
>
>	      return -1;
>
>	=20
>
>	    queue_count =3D this->enqueue_head_i (new_item);
>
>	=20
>
>	    if (queue_count =3D=3D -1)
>
>	      return -1;
>
>	=20
>
>	     this->notify ();  //******** move after closing brace below
>??? ******
>
>	  }
>
>	  return queue_count;
>
>	}
>
>	=20
>
>	=20
>
>	   ACE VERSION: 5.6
>
>	=20
>
>	    HOST MACHINE and OPERATING SYSTEM:
>
>	        Vindows Vista
>
>	=20
>
>	    TARGET MACHINE and OPERATING SYSTEM, if different from HOST:
>
>	=20
>
>	    COMPILER NAME AND VERSION (AND PATCHLEVEL):
>
>	        Visual Studio 2005
>
>	=20
>
>	Thanks,
>
>	=20
>
>	greg
>
>
>------_=_NextPart_001_01C832CE.C912EB7B
>Content-Type: text/html;
>	charset="us-ascii"
>Content-Transfer-Encoding: quoted-printable
>
><html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
>xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
>xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
>xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
>xmlns=3D"http://www.w3.org/TR/REC-html40">
>
><head>
><META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
>charset=3Dus-ascii">
><meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)">
><title>Message</title>
><style>
><!--
> /* Font Definitions */
> @font-face
>	{font-family:"Cambria Math";
>	panose-1:2 4 5 3 5 4 6 3 2 4;}
>@font-face
>	{font-family:Calibri;
>	panose-1:2 15 5 2 2 2 4 3 2 4;}
>@font-face
>	{font-family:Tahoma;
>	panose-1:2 11 6 4 3 5 4 4 2 4;}
> /* Style Definitions */
> p.MsoNormal, li.MsoNormal, div.MsoNormal
>	{margin:0in;
>	margin-bottom:.0001pt;
>	font-size:11.0pt;
>	font-family:"Calibri","sans-serif";}
>a:link, span.MsoHyperlink
>	{mso-style-priority:99;
>	color:blue;
>	text-decoration:underline;}
>a:visited, span.MsoHyperlinkFollowed
>	{mso-style-priority:99;
>	color:purple;
>	text-decoration:underline;}
>p
>	{mso-style-priority:99;
>	mso-margin-top-alt:auto;
>	margin-right:0in;
>	mso-margin-bottom-alt:auto;
>	margin-left:0in;
>	font-size:12.0pt;
>	font-family:"Times New Roman","serif";}
>p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
>	{mso-style-priority:34;
>	margin-top:0in;
>	margin-right:0in;
>	margin-bottom:0in;
>	margin-left:.5in;
>	margin-bottom:.0001pt;
>	font-size:11.0pt;
>	font-family:"Calibri","sans-serif";}
>span.EmailStyle18
>	{mso-style-type:personal;
>	font-family:"Calibri","sans-serif";
>	color:windowtext;}
>span.EmailStyle20
>	{mso-style-type:personal-reply;
>	font-family:"Calibri","sans-serif";
>	color:#1F497D;}
>.MsoChpDefault
>	{mso-style-type:export-only;
>	font-size:10.0pt;}
>@page Section1
>	{size:8.5in 11.0in;
>	margin:1.0in 1.0in 1.0in 1.0in;}
>div.Section1
>	{page:Section1;}
>-->
></style>
><!--[if gte mso 9]><xml>
> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
></xml><![endif]--><!--[if gte mso 9]><xml>
> <o:shapelayout v:ext=3D"edit">
>  <o:idmap v:ext=3D"edit" data=3D"1" />
> </o:shapelayout></xml><![endif]-->
></head>
>
><body lang=3DEN-US link=3Dblue vlink=3Dpurple>
>
><div class=3DSection1>
>
><p class=3DMsoNormal><span style=3D'color:#1F497D'>YES! I think found =
>exactly this
>issue:<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'><o:p>&nbsp;</o:p></span></p>
>
><p class=3DMsoNormal><span style=3D'color:#1F497D'>Sun May&nbsp; 5 =
>19:14:34 2002&nbsp;
>Douglas C. Schmidt&nbsp; =
>&lt;schmidt at macarena.cs.wustl.edu&gt;<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'><o:p>&nbsp;</o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
>nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; *
>ace/Message_Queue_T.cpp: Modified all the enqueue*() methods so =
>that<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
>nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; their calls to
>notify() occur *outside* of the monitor lock.<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span style=3D'color:#1F497D'>&nbsp; =
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
>nbsp; &nbsp;&nbsp;This change
>prevents deadlock from occurring when a reactor's<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
>nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; notification
>pipe is full.&nbsp; Thanks to Sasha Agranov<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
>nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;
>&lt;sagranov at COMGATES.co.il&gt; for reporting =
>this.<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'><o:p>&nbsp;</o:p></span></p>
>
><p class=3DMsoNormal><span style=3D'color:#1F497D'>Did it creep back in =
>somehow?<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'><o:p>&nbsp;</o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'>greg<o:p></o:p></span></p>
>
><p class=3DMsoNormal><span =
>style=3D'color:#1F497D'><o:p>&nbsp;</o:p></span></p>
>
><div>
>
><div style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt =
>0in 0in 0in'>
>
><p class=3DMsoNormal><b><span =
>style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span>=
></b><span
>style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Steve =
>Huston
>[mailto:shuston at riverace.com] <br>
><b>Sent:</b> Thursday, November 29, 2007 4:15 PM<br>
><b>To:</b> Greg Popovitch; ace-bugs at cse.wustl.edu<br>
><b>Subject:</b> RE: [ace-bugs] trying to fix an ACE =
>hang<o:p></o:p></span></p>
>
></div>
>
></div>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><div>
>
><p class=3DMsoNormal><span =
>style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";
>color:blue'>Hi Greg,</span><span =
>style=3D'font-size:12.0pt;font-family:"Times New =
>Roman","serif"'><o:p></o:p></span></p>
>
></div>
>
><div>
>
><p class=3DMsoNormal><span style=3D'font-size:12.0pt;font-family:"Times =
>New Roman","serif"'>&nbsp;<o:p></o:p></span></p>
>
></div>
>
><div>
>
><p class=3DMsoNormal><span =
>style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";
>color:blue'>Thanks for the PROBLEM-REPORT-FORM.</span><span =
>style=3D'font-size:
>12.0pt;font-family:"Times New Roman","serif"'><o:p></o:p></span></p>
>
></div>
>
><div>
>
><p class=3DMsoNormal><span style=3D'font-size:12.0pt;font-family:"Times =
>New Roman","serif"'>&nbsp;<o:p></o:p></span></p>
>
></div>
>
><div>
>
><p class=3DMsoNormal><span =
>style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";
>color:blue'>Doug poked me to see if I remembered this type of problem. =
>Your
>description looks familiar - I've a feeling we've been down this road =
>before,
>but can't recall the details. Could you please scan through the
>ACE_wrappers/ChangeLogs files for changes in this area to see if we've =
>tried
>this fix before?</span><span =
>style=3D'font-size:12.0pt;font-family:"Times New =
>Roman","serif"'><o:p></o:p></span></p>
>
></div>
>
><div>
>
><p class=3DMsoNormal><span style=3D'font-size:12.0pt;font-family:"Times =
>New Roman","serif"'>&nbsp;<o:p></o:p></span></p>
>
></div>
>
><div>
>
><p class=3DMsoNormal><span =
>style=3D'font-size:10.0pt;font-family:"Arial","sans-serif";
>color:blue'>Thanks!<br>
>-Steve</span><span style=3D'font-size:12.0pt;font-family:"Times New =
>Roman","serif"'><o:p></o:p></span></p>
>
></div>
>
><p><span style=3D'font-size:10.0pt'>--<br>
>Steve Huston, Riverace Corporation<br>
>Want to take ACE training on YOUR schedule?<br>
>See <a =
>href=3D"http://www.riverace.com/training.htm">http://www.riverace.com/tra=
>ining.htm</a></span><o:p></o:p></p>
>
><blockquote style=3D'border:none;border-left:solid blue =
>1.5pt;padding:0in 0in 0in 4.0pt;
>margin-left:3.75pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'=
>>
>
><p class=3DMsoNormal style=3D'margin-bottom:12.0pt'><span =
>style=3D'font-size:10.0pt;
>font-family:"Tahoma","sans-serif"'>-----Original Message-----<br>
><b>From:</b> ace-bugs-bounces at cse.wustl.edu
>[mailto:ace-bugs-bounces at cse.wustl.edu] <b>On Behalf Of </b>Greg =
>Popovitch<br>
><b>Sent:</b> Thursday, November 29, 2007 2:23 PM<br>
><b>To:</b> ace-bugs at cse.wustl.edu<br>
><b>Subject:</b> [ace-bugs] trying to fix an ACE hang</span><span
>style=3D'font-size:12.0pt;font-family:"Times New =
>Roman","serif"'><o:p></o:p></span></p>
>
><p class=3DMsoNormal>Hi,<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;<o:p></o:p></p>
>
><p class=3DMsoNormal>I have a hang on Windows Vista 64 and also on linux =
>64 (RHEL
>4). <o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>One thread is enqueuing a message on an =
>ACE_Message_Queue
>for another thread and hangs on the notification to the WFMO reactor. =
>The
>&nbsp;ACE_WFMO_Reactor_Notify::notify() hangs because its internal =
>message
>queue is full.<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>The other thread hangs while checking of the =
>message_queue
>is empty because it can&#8217;t grab its ACE_GUARD. Therefore it =
>can&#8217;t dequeue
>messages from the WFMO_Reactor.<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>My question: <o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>In file Message_Queue_T.cpp,<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>Would it be OK to move the &#8220;this-&gt;notify =
>()&#8221; outside of
>the scope of the ACE_GUARD in:<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>template &lt;ACE_SYNCH_DECL&gt; int<o:p></o:p></p>
>
><p =
>class=3DMsoNormal>ACE_Message_Queue&lt;ACE_SYNCH_USE&gt;::enqueue_head =
>(ACE_Message_Block
>*new_item,<o:p></o:p></p>
>
><p =
>class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
>nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&n=
>bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
>sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
>p;&nbsp;
>ACE_Time_Value *timeout)<o:p></o:p></p>
>
><p class=3DMsoNormal>{<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp; ACE_TRACE
>(&quot;ACE_Message_Queue&lt;ACE_SYNCH_USE&gt;::enqueue_head&quot;);<o:p><=
>/o:p></p>
>
><p class=3DMsoNormal>&nbsp; int queue_count =3D 0;<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp; {<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; ACE_GUARD_RETURN =
>(ACE_SYNCH_MUTEX_T,
>ace_mon, this-&gt;lock_, -1);<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; if (this-&gt;state_ =3D=3D
>ACE_Message_Queue_Base::DEACTIVATED)<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; {<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; errno =
>=3D
>ESHUTDOWN;<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return =
>-1;<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp; &nbsp;if (this-&gt;wait_not_full_cond =
>(ace_mon,
>timeout) =3D=3D -1)<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return =
>-1;<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; queue_count =3D =
>this-&gt;enqueue_head_i
>(new_item);<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; if (queue_count =3D=3D =
>-1)<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; return =
>-1;<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp; &nbsp;&nbsp;this-&gt;notify ();&nbsp;
>//******** move after closing brace below ??? ******<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp; }<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp; return queue_count;<o:p></o:p></p>
>
><p class=3DMsoNormal>}<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp; ACE VERSION: 5.6<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; HOST MACHINE and OPERATING =
>SYSTEM:<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Vindows =
>Vista<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; TARGET MACHINE and OPERATING =
>SYSTEM, if
>different from HOST:<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; COMPILER NAME AND VERSION (AND
>PATCHLEVEL):<o:p></o:p></p>
>
><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Visual =
>Studio
>2005<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>Thanks,<o:p></o:p></p>
>
><p class=3DMsoNormal><o:p>&nbsp;</o:p></p>
>
><p class=3DMsoNormal>greg<o:p></o:p></p>
>
></blockquote>
>
></div>
>
></body>
>
></html>
>
>------_=_NextPart_001_01C832CE.C912EB7B--
>


-- 
Dr. Douglas C. Schmidt                       Professor and Associate Chair
Electrical Engineering and Computer Science  TEL: (615) 343-8197
Vanderbilt University                        WEB: www.dre.vanderbilt.edu/~schmidt
Nashville, TN 37203                          NET: d.schmidt at vanderbilt.edu



More information about the Ace-users mailing list