[ace-bugs] ACE_Svc_Handler handle_input not called in some scenarios randomly when using ACE_SSL_Sock_Stream and ACE_Acceptor

Tue Apr 21 10:04:24 CDT 2020

Hi,

I wrote a simple server and client example and built it with the latest ACE 6.5.8. I could still recreate the issue that was earlier reported.

Please find attached the tar file containing the server and client example applications written.
Linux OS version of our development environment: Red Hat Enterprise Linux Server release 6.4
GCC version: gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-3)

Steps to run the server and client example applications are as follows.

  1.  Server: tls_server_app <server_ip:port> <debug_level> <path_to_SSL_key_file> <path_to_SSL_certificate_file> <size_of_client_msg>
For example: tls_server_app 172.21.41.55:4443 info server.key server.cert 494
The current client example sends a message of 494 bytes. This needs to be changed accordingly for a different sized client message.
  2.  Client: tls_client_app <server_ip> <port> <path_to_SSL_certificate_file>
For example: tls_client_app 172.21.41.55 4443 server.pem
  3.  Once we run the server and the client for about 20-30 minutes (or sometimes even earlier), we see that the following log on the server indicating that there is no handle_input received for the inactivity time duration.
2020-04-21 14:08:54.333208 [19984] TCP Connection will be closed as connection is inactive for long time.

Regards,
Nagesh
From: Johnny Willemsen <jwillemsen at remedy.nl>
Sent: Tuesday, April 14, 2020 2:26 PM
To: Somisetty, Nagesh <NSomisetty at tnsi.com>; ace-bugs at list.isis.vanderbilt.edu
Cc: Zhao, Changchun <czhao at tnsi.com>; Santhappanavara, Sharath <SSanthappanavara at tnsi.com>; Gupta, Ajay <agupta at tnsi.com>
Subject: Re: [ace-bugs] ACE_Svc_Handler handle_input not called in some scenarios randomly when using ACE_SSL_Sock_Stream and ACE_Acceptor

Hi,

Maybe also try a newer RHEL release with newer SSL version, maybe it is
an issue in the underlying OS.

Johnny Willemsen
Remedy IT
http://www.remedy.nl<http://www.remedy.nl>

On 4/14/20 10:52 AM, Johnny Willemsen wrote:
> Hi,
>
> 6.5.0 is the latest minor, 6.5.8 is the latest micro but not much has
> very likely changed in this area.
>
> Please try to extend/add a unit test under ACE_wrappers/tests to
> reproduce this issue.
>
> Regards,
>
> Johnny Willemsen
> Remedy IT
> http://www.remedy.nl<http://www.remedy.nl>
>
> On 4/13/20 2:59 PM, Somisetty, Nagesh wrote:
>> Thanks Johnny.
>>
>>
>>
>> I’ve also tried with the ACE 6.5.0, but it did not help. I was still
>> observing the same issues as reported in the PRF.
>>
>>
>>
>> Regards,
>>
>> Nagesh
>>
>>
>>
>> *From:*Johnny Willemsen <jwillemsen at remedy.nl<mailto:jwillemsen at remedy.nl>>
>> *Sent:* Monday, April 13, 2020 6:00 PM
>> *To:* Somisetty, Nagesh <NSomisetty at tnsi.com<mailto:NSomisetty at tnsi.com>>;
>> ace-bugs at list.isis.vanderbilt.edu<mailto:ace-bugs at list.isis.vanderbilt.edu>
>> *Cc:* Zhao, Changchun <czhao at tnsi.com<mailto:czhao at tnsi.com>>; Santhappanavara, Sharath
>> <SSanthappanavara at tnsi.com<mailto:SSanthappanavara at tnsi.com>>; Gupta, Ajay <agupta at tnsi.com<mailto:agupta at tnsi.com>>
>> *Subject:* Re: [ace-bugs] ACE_Svc_Handler handle_input not called in
>> some scenarios randomly when using ACE_SSL_Sock_Stream and ACE_Acceptor
>>
>>
>>
>> Hi,
>>
>> Thanks for using the PRF form. 6.2.0 is already pretty old, can you trie
>> ACE 6.5.8 which you can download from
>> http://download.dre.vanderbilt.edu<http://download.dre.vanderbilt.edu>, that is the most recent micro release.
>>
>> Regards,
>>
>> Johnny Willemsen
>>
>> Remedy IT
>>
>> http://www.remedy.nl<http://www.remedy.nl>
>>
>> On 4/13/20 11:28 AM, Somisetty, Nagesh wrote:
>>
>> ACE VERSION: 6.2.0
>>
>> HOST MACHINE and OPERATING SYSTEM: RHEL Server 6.8
>> If on Windows based OS's, which version of WINSOCK do you
>> use?:
>>
>> TARGET MACHINE and OPERATING SYSTEM, if different from HOST: RHEL
>> Server 6.8
>> COMPILER NAME AND VERSION (AND PATCHLEVEL): g++ (GCC) 4.4.7 20120313
>> (Red Hat 4.4.7-3)
>>
>> THE $ACE_ROOT/ace/config.h FILE [if you use a link to a platform-
>> specific file, simply state which one]:
>>
>> THE $ACE_ROOT/include/makeinclude/platform_macros.GNU FILE [if you
>> use a link to a platform-specific file, simply state which one
>> (unless this isn't used in this case, e.g., with Microsoft Visual
>> C++)]:
>>
>> CONTENTS OF $ACE_ROOT/bin/MakeProjectCreator/config/default.features
>> (used by MPC when you generate your own makefiles):
>>
>> AREA/CLASS/EXAMPLE AFFECTED:
>> ACE_Svc_Handler handle_input not called (in some cases) when using
>> ACE_SSL_Sock_Stream and ACE_Acceptor
>>
>> DOES THE PROBLEM AFFECT:
>> COMPILATION?
>> LINKING?
>> On Unix systems, did you run make realclean first?
>> EXECUTION?
>> OTHER (please specify)?
>> Described problem affects the EXECUTION of the application
>>
>> SYNOPSIS:
>> ACE_Svc_Handler handle_input not called in some scenarios when using
>> ACE_Acceptor and ACE_SSL_Sock_Stream
>>
>> DESCRIPTION:
>> 1) I have an ACE_Acceptor based server application that works either
>> as a TCP or TLS server.
>> 2) The TCP and TLS services are defined as below,
>> typedef ACE_Acceptor<TCPServer, ACE_SOCK_ACCEPTOR> TCPService;
>> typedef ACE_Acceptor<TLSServer, ACE_SSL_SOCK_ACCEPTOR> TLSService;
>> where TCPServer and TLSServer are derived from ACE_Svc_Handler as below
>> class TCPServer : public ACE_Svc_Handler<ACE_SOCK_STREAM,
>> ACE_NULL_SYNCH>
>> class TLSServer : public ACE_Svc_Handler<ACE_SSL_SOCK_STREAM,
>> ACE_NULL_SYNCH>
>> Both these TCPServer and TLSServer have implemented the handle_input
>> callbacks for handling of incoming events from the socket.
>> 3) Our observation is that while using the TLSServer, we see that
>> the handle_input is not called in a few cases as listed below.
>> a) When the application attempts to do a peer().recv(buff,len) on
>> receiving a handle_input and the application buffer size remaining
>> is smaller than the number of bytes available on the socket to be
>> read, only the number of bytes that can be read into the buffer are
>> actually read. After this the application expects another
>> handle_input for the remaining bytes that are still on the socket
>> that are yet to be read. In this scenario, we never receive the
>> subsequent handle_input event so that we can do the next
>> peer().recv(buf,len) call to read the remaining bytes from the
>> socket. As a workaround for this, we changed the buffer allocation
>> logic to ensure the application doesn't wait for the next
>> handle_input if there are more bytes than the remaining buffer size.
>> We solved this problem with the workaround, but we encountered the
>> issues described in 3b below.
>> b) On running load tests with a TLS client which keeps sending https
>> requests continuously, we found that the server randomly freezes
>> after anywhere between 1 and 30 minutes and when we check it looks
>> like we never received a handle_input event at the TLSServer. There
>> is no logic on the client that causes it to stop sending requests.
>> So, it has to be something to do with the handle_input events not
>> being notified to the application.
>> 4) Please note that the issues 3a and 3b are not observed while
>> using TCPServer and TCP clients. The logic in the handle_input for
>> both TCP and TLS is exactly the same.
>>
>>
>> REPEAT BY:
>> [What you did to get the error; include test program or session
>> transcript if at all possible. ]
>>
>> SAMPLE FIX/WORKAROUND:
>> No fix available. Looking for a solution or an explanation.
>>
>> ------------------------------------------------------------------------
>>
>> This email has been scanned for email related threats and delivered
>> safely by Mimecast.
>> For more information please visit http://www.mimecast.com<http://www.mimecast.com>
>>
>> ------------------------------------------------------------------------
>>
>>
>>
>> _______________________________________________
>>
>> ace-bugs mailing list
>>
>> ace-bugs at list.isis.vanderbilt.edu<mailto:ace-bugs at list.isis.vanderbilt.edu> <mailto:ace-bugs at list.isis.vanderbilt.edu>
>>
>> http://list.isis.vanderbilt.edu/cgi-bin/mailman/listinfo/ace-bugs<http://list.isis.vanderbilt.edu/cgi-bin/mailman/listinfo/ace-bugs>
>>
>>
>>
>> ------------------------------------------------------------------------
>> This email has been scanned for email related threats and delivered
>> safely by Mimecast.
>> For more information please visit http://www.mimecast.com<http://www.mimecast.com>
>> ------------------------------------------------------------------------
> _______________________________________________
> ace-bugs mailing list
> ace-bugs at list.isis.vanderbilt.edu<mailto:ace-bugs at list.isis.vanderbilt.edu>
> http://list.isis.vanderbilt.edu/cgi-bin/mailman/listinfo/ace-bugs<http://list.isis.vanderbilt.edu/cgi-bin/mailman/listinfo/ace-bugs>
---------------------------------------------------------------------------------------
 This email has been scanned for email related threats and delivered safely by Mimecast.
 For more information please visit http://www.mimecast.com
---------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.isis.vanderbilt.edu/pipermail/ace-bugs/attachments/20200421/9cb3f379/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ace_tls_2.5.8_issue.tar
Type: application/x-tar
Size: 30720 bytes
Desc: ace_tls_2.5.8_issue.tar
URL: <http://list.isis.vanderbilt.edu/pipermail/ace-bugs/attachments/20200421/9cb3f379/attachment-0001.tar>