[ace-users] use of ACE_HAS_REACTOR_NOTIFICATION_QUEUE in Select Reactor

Douglas C. Schmidt schmidt at dre.vanderbilt.edu
Tue Aug 7 06:44:12 CDT 2007


Hi,

>     i would like to add that the case is not specific to schedule_time and cancel_timer, it happens also in the case when
>     the 3 threads is
>     doing register_handler for WRITE_EVENT. (Other 2 condition remains the same)...
>    
>     Question : is there any way to make the reactor wake-up from ACE_OS::select after a time interval event if there are no
>     events to handle so that a dead-lock is prevented ???

Sure, you can pass a timeout to the reactor event loop method.  Please
see Chapters 3 and 4 of C++NPv2 <www.cs.wustl.edu/~schmidt/ACE/book2/>
for details.

Thanks,

        Doug

>     regards,
>     Prem
> 
>     On 8/7/07, Premkumar P <kumaran.prem at gmail.com> wrote:
>    
>         Hi,
>         i tried using ACE_HAS_REACTOR_NOTIFICATION_QUEUE but it does not help. :(
>        
>         i will try explaining my use case first,
>        
>         1) i have a SOCK_Acceptor accepting connections - it uses a Select Reactor.
>        
>         2) handlers created for the connections accepted by the above acceptor are assigned a different Reactor instance to
>         do read and write data on the connection  ( I will refere this Reactor as the processing REACTOR, which is
>         different from the reactor that accepts connection)
>        
>         3) the acceptor calls open() on the handler cleated, in the open of the handler, i schedule a timer to keep check
>         of ideal connections. The timer will be canceled before the handler is deleted.
>        
>         4) "the problem is in the above step. "
>        
>         Condition : the processing reactor is waiting on ACE_OS::select indefinitely for an event to occur. The acceptor
>         accepted a new connection and called open(), which tries to schedule a timer, this call gets blocked in
>         schedule_timer method call. At the same time a third thread which has the response for a request read earlier (on a
>         different connection - which uses the same processing REACTOR) found that the connection is no more alive called
>         cancel_timer so that it can delete the handler instance after unregistering it.
>        
>         "and i have a dead lock"
>         i managed to get a stack trace of the the threads (Acceptor, processing REACTOR, and a third thread)
>        
>         Processing REACTOR waiting for events to occur:
>           [1] __pollsys(0xcffbdb80, 0x3, 0x0, 0x0), at 0xd1100fe5
>           [2] _pollsys(0xcffbdb80, 0x3, 0x0, 0x0), at 0xd10f572f
>           [3] _pselect(0xe, 0x83f7b04, 0xd112a868, 0xd112a868, 0x0, 0x0), at 0xd10aee1a
>           [4] _select(0xe, 0x83f7b04, 0x0, 0x0, 0x0, 0x0), at 0xd10af110
>         =>[5] ACE_OS::select(width = 14, rfds = 0x83f7b04, wfds = (nil), efds = (nil), timeout = (nil)), line 46 in
>         "OS_NS_sys_select.inl"
>           [6] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::wait_for_multiple_events(this = 0x83f7ae0,
>         dispatch_set = CLASS, max_wait_time = (nil)), line 1104 in "Select_Reactor_T.cpp"
>           [7] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::handle_events_i(this = 0x83f7ae0, max_wait_time =
>         (nil)), line 1443 in "Select_Reactor_T.cpp"
>           [8] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::handle_events(this = 0x83f7ae0, max_wait_time =
>         (nil)), line 1424 in "Select_Reactor_T.cpp"
>           [9] ACE_Reactor::run_reactor_event_loop(this = 0x83e37d0, eh = (nil)), line 233 in " Reactor.cpp"
>           [10] HTTP::SyncHTTPListener::ReactorLoop::run(this = 0x83e39b0), line 50 in "SyncTCPListener.h"
>           [11] ZThread::ThreadImpl::dispatch(0x83cfd28, 0x83f8560, 0xcffbdf98, 0x0), at 0xd156581f
>           [12] ZThread::__unnamed_AAAAAfdNjGUKh::Launcher::run(0x80449c4), at 0xd1566602
>           [13] _dispatch(0x80449c4), at 0xd156a35d
>           [14] _thr_setup(0xd0ac3000), at 0xd10ffc42
>           [15] _lwp_start(0xcffbdb80, 0x3, 0x0, 0x0, 0x0, 0x3), at 0xd10fff30
> 
>         Acceptor thread accepted a new connection and called open()
>           [1] __lwp_park(0x0, 0x0), at 0xd10fffa9
>           [2] cond_sleep_queue(0xd00bc6a0, 0x8693a54, 0x0), at 0xd10fa6f3
>           [3] cond_wait_queue(0xd00bc6a0, 0x8693a54, 0x0, 0x0), at 0xd10fa805
>           [4] _cond_wait(0xd00bc6a0, 0x8693a54), at 0xd10facfe
>           [5] cond_wait(0xd00bc6a0, 0x8693a54), at 0xd10fad40
>           [6] _pthread_cond_wait(0xd00bc6a0, 0x8693a54), at 0xd10fad79
>         =>[7] ACE_OS::cond_timedwait(cv = 0xd00bc6a0, external_mutex = 0x8693a54, timeout = (nil)), line 415 in " signal.h"
>           [8] ACE_Condition_Thread_Mutex::wait(this = 0xd00bc6a0, mutex = CLASS, abstime = (nil)), line 98 in
>         "Condition_Thread_Mutex.cpp"
>           [9] ACE_Condition_Thread_Mutex::wait(this = 0xd00bc6a0, abstime = (nil)), line 107 in
>         "Condition_Thread_Mutex.cpp"
>           [10] ACE_Token::ACE_Token_Queue_Entry::wait(this = 0xd00bc698, timeout = (nil), lock = CLASS), line 130 in
>         "Token.inl"
>           [11] ACE_Token::shared_acquire(this = 0x8693a40, sleep_hook_func = (nil), arg = (nil), timeout = (nil), op_type =
>         WRITE_TOKEN), line 264 in " Token.cpp"
>           [12] ACE_Token::acquire(this = 0x8693a40, timeout = (nil)), line 341 in "Token.cpp"
>           [13] ACE_Guard<ACE_Reactor_Token_T<ACE_Token> >::acquire(this = 0xd00bc784), line 12 in "Guard_T.inl"
>           [14] ACE_Guard<ACE_Reactor_Token_T<ACE_Token> >::ACE_Guard(this = 0xd00bc784, l = CLASS), line 38 in
>         "Guard_T.inl"
>           [15] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::schedule_timer(this = 0x86933a0, handler = 0x87092d4,
>         arg = (nil), delay_time = CLASS, interval = CLASS), line 727 in "Select_Reactor_T.cpp"
>           [16] ACE_Reactor::schedule_timer(this = 0x86a27a8, event_handler = 0x87092d4, arg = (nil), delta = CLASS,
>         interval = CLASS), line 689 in "Reactor.cpp"
>           [17] SyncHTTPConnHandler::open(this = 0x8709270, stream = 0x8d534d0, clientaddr = CLASS, clienthostname = CLASS,
>         port = CLASS), line 139 in " SyncHTTPConnHandler.cpp"
>           [18] SyncHTTPConnAcceptor::handle_input(this = 0x83f4ce8, handle = 9), line 559 in "SyncHTTPConnAcceptor.cpp"
>           [19] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::notify_handle(this = 0x83ef540, handle = 9, mask =
>         1U, ready_mask = CLASS, event_handler = 0x83f4ce8, ptmf = 0xd00bdc34), line 817 in "Select_Reactor_T.cpp"
>           [20] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::dispatch_io_set(this = 0x83ef540,
>         number_of_active_handles = 1, number_of_handlers_dispatched = 1, mask = 1, dispatch_mask = CLASS, ready_mask =
>         CLASS, callback = 0xd00bdc94), line 1205 in "Select_Reactor_T.cpp"
>           [21] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::dispatch_io_handlers(this = 0x83ef540, dispatch_set =
>         CLASS, number_of_active_handles = 1, number_of_handlers_dispatched = 1), line 1263 in "Select_Reactor_T.cpp"
>           [22] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::dispatch(this = 0x83ef540, active_handle_count = 1,
>         dispatch_set = CLASS), line 1367 in "Select_Reactor_T.cpp"
>           [23] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::handle_events_i(this = 0x83ef540, max_wait_time =
>         (nil)), line 1447 in "Select_Reactor_T.cpp"
>           [24] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::handle_events(this = 0x83ef540, max_wait_time =
>         (nil)), line 1424 in "Select_Reactor_T.cpp"
>           [25] ACE_Reactor::run_reactor_event_loop(this = 0x83e36e0, eh = (nil)), line 233 in " Reactor.cpp"
>           [26] HTTP::SyncHTTPListener::ReactorLoop::run(this = 0x83e3788), line 52 in "SyncHTTPListener.h"
>           [27] ZThread::ThreadImpl::dispatch(0x83cfd28, 0x83f4de8, 0xd00bdf98, 0x0), at 0xd156581f
>           [28] ZThread::__unnamed_AAAAAfdNjGUKh::Launcher::run(0x80449c4), at 0xd1566602
>           [29] _dispatch(0x80449c4), at 0xd156a35d
>           [30] _thr_setup(0xd0ac2c00), at 0xd10ffc42
>           [31] _lwp_start(0x0, 0x0, 0x0, 0x8693a54, 0xd0ac2c00, 0xd1129000), at 0xd10fff30
> 
>         a third thread that processed the request (will initialize the write if connection still alive) - in this case will
>         delete the handler as the connection is not alive
>        
>           [1] __lwp_park(0x0, 0x0), at 0xd10fffa9
>           [2] cond_sleep_queue(0xd07bc968, 0x8693a54, 0x0), at 0xd10fa6f3
>           [3] cond_wait_queue(0xd07bc968, 0x8693a54, 0x0, 0x0), at 0xd10fa805
>           [4] _cond_wait(0xd07bc968, 0x8693a54), at 0xd10facfe
>           [5] cond_wait(0xd07bc968, 0x8693a54), at 0xd10fad40
>           [6] _pthread_cond_wait(0xd07bc968, 0x8693a54), at 0xd10fad79
>         =>[7] ACE_OS::cond_timedwait(cv = 0xd07bc968, external_mutex = 0x8693a54, timeout = (nil)), line 415 in
>         "OS_NS_Thread.inl"
>           [8] ACE_Condition_Thread_Mutex::wait(this = 0xd07bc968, mutex = CLASS, abstime = (nil)), line 98 in
>         "Condition_Thread_Mutex.cpp"
>           [9] ACE_Condition_Thread_Mutex::wait(this = 0xd07bc968, abstime = (nil)), line 107 in
>         "Condition_Thread_Mutex.cpp"
>           [10] ACE_Token::ACE_Token_Queue_Entry::wait(this = 0xd07bc960, timeout = (nil), lock = CLASS), line 130 in "
>         Token.inl"
>           [11] ACE_Token::shared_acquire(this = 0x8693a40, sleep_hook_func = (nil), arg = (nil), timeout = (nil), op_type =
>         WRITE_TOKEN), line 264 in "Token.cpp"
>           [12] ACE_Token::acquire(this = 0x8693a40, timeout = (nil)), line 341 in " Token.cpp"
>           [13] ACE_Guard<ACE_Reactor_Token_T<ACE_Token> >::acquire(this = 0xd07bca38), line 12 in "Guard_T.inl"
>           [14] ACE_Guard<ACE_Reactor_Token_T<ACE_Token> >::ACE_Guard(this = 0xd07bca38, l = CLASS), line 38 in
>         "Guard_T.inl"
>           [15] ACE_Select_Reactor_T<ACE_Reactor_Token_T<ACE_Token> >::cancel_timer(this = 0x86933a0, handler = 0x8b47504,
>         dont_call_handle_close = 1), line 695 in "Select_Reactor_T.cpp"
>           [16] ACE_Reactor::cancel_timer(this = 0x86a27a8, event_handler = 0x8b47504, dont_call_handle_close = 1), line 704
>         in " Reactor.cpp"
>           [17] HTTP::SyncHTTPConnHandler::handleSendError(this = 0x8b474a0), line 921 in "SyncHTTPConnHandler.cpp"
>           [18] HTTP::SyncHTTPConnHandler::sendResponse(this = 0x8b474a0), line 799 in " SyncHTTPConnHandler.cpp"
>           [19] itech::gateway::transport::HTTP::SyncHTTPConnHandler::processed(this = 0x8b474a0, msgctx = 0x8568d48), line
>         950 in "SyncHTTPConnHandler.cpp"
>           -----------------------------------------------------------------------------------------------
>           [26] ZThread::__unnamed_AAAAA9bNjGkZd::GroupedRunnable::run(0x8551100, 0x83b96a8, 0xd07bde68, 0xd1552518), at
>         0xd15530d1
>           [27] ZThread::__unnamed_AAAAA9bNjGkZd::Worker::run(0x83cf178, 0x83d1760, 0xd07bdf78, 0xd15657f0), at 0xd15524c6
>           [28] ZThread::ThreadImpl::dispatch(0x83cfd28, 0x83d1760, 0xd07bdf98, 0x0), at 0xd156581f
>           [29] ZThread::__unnamed_AAAAAfdNjGUKh::Launcher::run(0x8045328), at 0xd1566602
>           [30] _dispatch(0x8045328), at 0xd156a35d
>           [31] _thr_setup(0xd0ac1000), at 0xd10ffc42
>           [32] _lwp_start(0x0, 0x0, 0x0, 0x8693a54, 0xd0ac1000, 0xd1129000), at 0xd10fff30
> 
>         i tried debugging but the Reactor implementation is not that easy for me to understand
>         any help would be greatly helpful, thankx in advance.
>        
>         Reagards,
>         Prem
>        
>         On 8/6/07, Douglas C. Schmidt < schmidt at dre.vanderbilt.edu> wrote:
> 
>             Hi Prem,
>            
>             > that for the quick reply, and the info.  I dont think it is a bug, i
>             > had written my application compiling ACE with the option enabled, when
>             > upgrading to ACE 5.5.10 - i forgot to enable this option, and i am
>             > seeing some strange behavior (that i dont expect!!!). I am recompiling
>             > ACE with this option enabled will try to look deep what is happening.
>            
>             Please see Chapter 4 of C++NPv2 for more explanations of what may be
>             going on.
>            
>             Thanks,
>            
>                     Doug
> 
> _______________________________________________
> ace-users mailing list
> ace-users at mail.cse.wustl.edu
> http://mail.cse.wustl.edu/mailman/listinfo/ace-users



More information about the Ace-users mailing list