[Ace-users] Using ACE_Svc_Handler::msg_queue() for buffering is slow

[azathoth]

> This doesn't explain why in the second case it's still slow:
>
> >If I comment out the code that uses the message block so it always
> >goes to into the queue, send will be called with 8192 bytes.

My mistake, in this case it was actually calling send more than I
thought it was (using strace to verify). Nagle's algorithm does seem
the likely candidate.

I'm left wondering if I've failed to set TCP_NODELAY correctly:

int Param = 1;
this->peer().set_option(0, TCP_NODELAY, &Param, sizeof(Param));

inside the constructor for the ACE_Svc_Handler?

On Nov 5, 9:42 am, azathoth <jebar... at gmail.com> wrote:
>
> Basically, if I use the queue, no matter how much data I send, it's
> always slow. If I don't it sends at a fast rate.
>
> (I did try using TCP_NODELAY and the results are still the same.)
>
> On Nov 4, 6:30 pm, schm... at dre.vanderbilt.edu (Douglas C. Schmidt)
> wrote:
>
> > Hi,
>
> > >If I limit the size of the message block to 10 bytes, and I want to
> > >send 8192 bytes to the server: the send command will be called once
> > >with 10 bytes, then again pulling the remaining data off the message
> > >queue to send 8182 bytes.
>
> > Ah, it sounds like you're running into Nagel's algorithm.  If you
> > disable this by enabling TCP_NODELAY then your performance should be
> > much better.  Please see chapter 3 of C++NPv1
> > <www.cs.wustl.edu/~schmidt/ACE/book1/> for details on this.
>
> > Thanks,
>
> >         Doug
>
> > > (provided the client is ready to receive it
> > >all, which it almost always is).
>
> > >If I comment out the code that uses the message block so it always
> > >goes to into the queue, send will be called with 8192 bytes.
>
> > >If I force the message block to 8192 bytes, send will again be called
> > >with 8192 bytes.
>
> > >The above approaches take the same amount of time, in fact the time
> > >spent inside the handle_output function for a test run sending
> > >10megabytes is less than a second. In the first two approahces, the
> > >entire 10megs will take at least 2 minutes to send. Only in the final
> > >case, the entire 10 megs will take about 2 seconds to send.
>
> > >As far as I can tell profiling the program, next to no time is
> > >actually being spent inside the code, it seems to be spending most of
> > >it's time waiting. I almost suspect that putting thins in the Message
> > >queue is somehow interacting with the reactor in a way to slow down
> > >event dispatching. My next step is probably going to be putting
> > >Message Blocks in a std::queue rather than the ACE one.
>
> > >On Nov 4, 3:38 pm, schm... at dre.vanderbilt.edu (Douglas C. Schmidt)
> > >wrote:
> > >> Hi,
>
> > >> >None, i.e. ACE_NULL_MUTEX
>
> > >> Ok, great.  One more question - what's the size of the data that
> > >> you're sending with each approach, i.e., are you sending an 8k
> > >> buffer at one time with the first approach and 10 byte buffers at a
> > >> time with the other?  I recommend you use strace or ethereal to see
> > >> what's going on wrt the data entering/leaving your machine.
>
> > >> Thanks,
>
> > >>         Doug
>
> > >> On Nov 3, 11:59 pm, schm... at dre.vanderbilt.edu (Douglas C. Schmidt)
>
> > >> >wrote:
> > >> >> Hi,
>
> > >> >> > No, it uses the reactor the same amount. The ACE_Message_Queue is
> > >> >> > used as a fallback if the ACE_Message_Block gets full. If the
> > >> >> > 'buffer gets full it just puts all the data in the queue, when
> > >> >> > handle_output is called it sends all the data it can from both the
> > >> >> > buffer and the queue. handle_output is called the same number of
> > >> >> > times regardless of method used.
>
> > >> >> Ok, so what type of synchronization are you using for the ACE_Message_Queue?
>
> > >> >> Thanks,
>
> > >> >>         Doug
> > >> >> --
> > >> >> Dr. Douglas C. Schmidt                       Professor and Associate Chair
> > >> >> Electrical Engineering and Computer Science  TEL: (615) 343-8197
> > >> >> Vanderbilt University                        WEB:www.dre.vanderbilt.edu/~schmidt
> > >> >> Nashville, TN 37203                          NET: d.schm... at vanderbilt.edu
>
> > >> --
> > >> Dr. Douglas C. Schmidt                       Professor and Associate Chair
> > >> Electrical Engineering and Computer Science  TEL: (615) 343-8197
> > >> Vanderbilt University                        WEB:www.dre.vanderbilt.edu/~schmidt
> > >> Nashville, TN 37203                          NET: d.schm... at vanderbilt.edu
>
> > --
> > Dr. Douglas C. Schmidt                       Professor and Associate Chair
> > Electrical Engineering and Computer Science  TEL: (615) 343-8197
> > Vanderbilt University                        WEB:www.dre.vanderbilt.edu/~schmidt
> > Nashville, TN 37203                          NET: d.schm... at vanderbilt.edu