QFJ-285 Proxy support fixes

[soliad]

Fix proxy support with alternative socket hosts
Update http proxy to use CONNECT

[soliad]

Let me know if you prefer a PR on the master branch

[xiaodwy]

Hi @soliad ,

Thanks for your update and sorry for the late reply(I have no chance to do the test during holidays).
I have tested your fix and found some issues.
(In my testing, 9880 is good FIX port, 1111 is not exists and 50000 is an active echo server. 8889 is socks proxy, 3128 is http proxy.)

1. The proxy works good with socks and reconnecting without SSL.
2. The proxy not works with SSL enabled in both side:

SocketConnectHost=172.31.1.55
SocketConnectPort=9880
SocketUseSSL=Y
ProxyType=socks
ProxyVersion=5
ProxyPort=8889

Got an exception on the server side:

javax.net.ssl.SSLHandshakeException: no cipher suites in common
...

3. The proxy with HTTP proxy still not work(without SSL):

SocketConnectHost=172.31.1.55
SocketConnectPort=9880
SocketUseSSL=N
ProxyType=http
ProxyVersion=1.1
ProxyPort=3128

Here're the logs:

[2;37m2017-01-03 15:55:09,660�[55 [DEBUG] [NioProcessor-2] AbstractHttpLogicHandler.java:138 messageReceived:##   response header received:
HTTP/1.1 407 Proxy Authentication Required\r\n
Server: squid/3.4.9\r\n
Mime-Version: 1.0\r\n
Date: Tue, 03 Jan 2017 07:55:09 GMT\r\n
Content-Type: text/html\r\n
Content-Length: 3143\r\n
X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0\r\n
Vary: Accept-Language\r\n
Content-Language: en\r\n
Proxy-Authenticate: Basic realm="proxy"\r\n
X-Cache: MISS from xiaodongs-Mac-mini.local\r\n
Via: 1.1 xiaodongs-Mac-mini.local (squid/3.4.9)\r\n
Connection: keep-alive\r\n
\r\n

�[2;37m2017-01-03 15:55:09,661�[55 [DEBUG] [NioProcessor-2] AbstractHttpLogicHandler.java:362 decodeResponse:##   parseResponse()
�[2;37m2017-01-03 15:55:09,662�[55 [DEBUG] [NioProcessor-2] IoSessionEventQueue.java:77 enqueueEventIfNecessary:## ??? >> Enqueue IoSessionEvent@166cc62d - [ (0x00000001: nio socket, client, 0.0.0.0/0.0.0.0:60519 => /172.31.1.55:3128), - CLOSED event -]
�[2;37m2017-01-03 15:55:09,663�[55 [DEBUG] [NioProcessor-2] IoSessionEventQueue.java:61 discardSessionQueueEvents:## Event queue CLEARED
<20170103-07:55:11, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 2015 ms.)
<20170103-07:55:14, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 5024 ms.)
<20170103-07:55:17, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 8032 ms.)
<20170103-07:55:20, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 11035 ms.)
<20170103-07:55:23, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 14039 ms.)
<20170103-07:55:26, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 17048 ms.)
<20170103-07:55:29, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 20057 ms.)
<20170103-07:55:32, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 23066 ms.)
<20170103-07:55:35, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 26074 ms.)
<20170103-07:55:38, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 29084 ms.)
<20170103-07:55:41, FIX.4.4:BANZAI->EXEC, event> (Pending connection not established after 32088 ms.)

It seems that the HTTP proxy server(squid) closes the TCP connection immediately right after sending a 407 code to require authenticate. I think this is a reasonable behavior(close the connection) of the HTTP proxy, so we need to handle this scenario in the code, could you take a look?
(I have no idea to change the behavior of squid to keep the connection, so I can't do more testing on it)

Thanks,
Dong

[soliad]

Hi @xiaodwy

Unfortunately I dont have a http proxy on hand to test, but I'll take a look at re connection with SSL

Arnaud

[RamyaDhatch]

Hi @soliad ,
Does this work with HTTP (with SSL) ?Did you get a chance to test this ?

[soliad]

It should work, but I didn't manage to test with http proxy, only with socks and SSL

[RamyaDhatch]

Hello, It sends CONNECT http request with the fix.However it looks like SSL handshake is not happening after that. Thank you. Ramya

…

On Thu, Jan 26, 2017 at 6:00 PM Arnaud Lech ***@***.***> wrote: It should work, but I didn't manage to test with http proxy, only with socks and SSL — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#92 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYLuZs3Y8NN4Y5O7-6zBe-BlDHCrKKQiks5rWSWIgaJpZM4LYNna> .

[RamyaDhatch]

It works very well with SOCKS proxy type (SSL enabled). Could you please check Http with SSL when you get a chance. Thank you. Regards, Dhatchayani G On Fri, Jan 27, 2017 at 12:42 PM, Ramya Gurunathan <[email protected]> wrote:

…

Hello, It sends CONNECT http request with the fix.However it looks like SSL handshake is not happening after that. Thank you. Ramya On Thu, Jan 26, 2017 at 6:00 PM Arnaud Lech ***@***.***> wrote: > It should work, but I didn't manage to test with http proxy, only with > socks and SSL > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#92 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AYLuZs3Y8NN4Y5O7-6zBe-BlDHCrKKQiks5rWSWIgaJpZM4LYNna> > . >

[soliad]

I cannot test with an http proxy anytime soon.
Maybe we should just deactivate http proxy and keep only socks

[chrjohn]

@soliad
IMHO although the feature does not seem to work completely (unfortunately cannot test it myself) I'd rather leave it in the code. Earlier we didn't have any proxy support at all. Maybe someone has time to fix it later on.
Thanks for your work. 👍

[soliad]

@chrjohn sorry for not being able to test this combination.
Anyway the fix in this PR will at least fix reconnection and http proxy without ssl
We could add a note in the doc about issues using http proxy + ssl (as it seems to be the known case with issue).
From what I tested and understand of the discussion:
http proxy + no ssl is ok
no proxy + ssl is ok
socks proxy + ssl is ok

[chrjohn]

Hi @soliad , @xiaodwy ,
I have come across this issue in MINA: https://issues.apache.org/jira/browse/DIRMINA-844
Looks like the problem you are facing.
We plan to integrate MINA 2.0.17 as soon as it is released but it will not make it into the QFJ 2.0.0 release, though.
Cheers,
Chris.

[riddhiagr]

Hi,
I am not able to connect through HTTP proxy without ssl. I am providing
ProxyType=http
ProxyVersion=1.1
ProxyHost=X.X.X.X
ProxyPort=nnnn
Is there something else I need to configure to create the connection?

[chrjohn]

To rule out any problems with MINA please try the latest 2.0.1 or 2.1.0 SNAPSHOT. There was a bug regarding proxy connections. Configuration can be found here https://quickfixj.org/usermanual/2.0.0//usage/configuration.html

[riddhiagr]

Hi,
Thanks for the quick reply. I can not use SNAPSHOT versions. Is there any plan to release 2.0.1 anytime soon?

[chrjohn]

No, currently no plan.
But why can't you use SNAPSHOT versions? I am not talking about production. Just test it in the dev environment and if it works wait for the release.
BTW, if you have further questions you should use the mailing list: https://sourceforge.net/projects/quickfixj/lists/quickfixj-users
Maybe it makes sense to post your configuration and the error message you are getting there.

[riddhiagr]

Ok. I will test with the SNAPSHOT version. Will update the result.
Thanks.

[riddhiagr]

Hi,
I tried to contact through mailing list but could not find a way to send my query there, will search further on how to do that.
For now: Could you provide a jira id/link (if exists) which tracks the code changes to resolve the proxy support bugs?

[chrjohn]

Hmm, you just need to enter your mail address here: https://sourceforge.net/projects/quickfixj/lists/quickfixj-users and click on "subscribe".
Then they should send you a mail to confirm the subscription. When that is confirmed you should be able to send mail to [email protected] .

The only code change after this PR #92 has been fixed was #177 to include the new MINA version.

[rishavkumar]

Hi,
Regarding http proxy , i was able to get it working for un-autheticated http proxy. But not able to get it working when authentication is enabled on proxy.

While diagnosing the problem, i found that inside ProtocolFactory.createIoProxyConnector NO_AUTH, DIGEST and BASIC all have been added to the order, while creating the ProxyConnector object which requires ProxyIoSession.
But I see that mina core raise an exception as soon as it is sees the NO_AUTH in the preferred order list of authentication methods.
On removing the NO_AUTH from preferred order list, it starts working with authenticated http proxy as well.
Please let us know in case this is intentional and there is some other way to get it working on authentication enabled http proxy.

[chrjohn]

Could you please supply a stack trace on where exactly the Exception is thrown?
So in your opinion

• NO_AUTH should be set in preferred order list when not using SSL and
• NO_AUTH should not be set when using SSL?

I am not very familiar with the proxy code so please excuse my ignorance.

If you are able to provide a unit test it would probably help on fixing the issue. Or even better create a new pull request with the fix.

Thanks,
Chris.

[marlowa]

I am getting an SSL handshake error when using quickfixj 2.1.0. It reports "SEND TLSv1.2 ALERT: fatal, description = internal_error" during proxy authentication.

I have set SocketUseSSL=Y in my config file and SocketConnectPort to 443. My software is running behind a corporate firewall so I have to use the corporate proxy. This is also configured in the config file with ProxyType=http. ProxyUser and ProxyPassword are also set.

Using curl to get to the remote URL works. It employs the corporate proxy via the http{s}_proxy environment variables and its detailed trace shows the handshake working correctly. However, my program encounters an SSLException, shown by the trace below.

I was getting something like this when usijng quickfixj 1.6.3, which uses an old version of mina (2.0.17) which has a bug reported in this area. With those versions if reported a NEEDS_WRAP error but with quickfixj 2.1.0 it now reports internal error.

Here is the log:

3879 [NioProcessor-2] DEBUG org.apache.mina.proxy.handlers.http.AbstractHttpLogicHandler  -  messageReceived()
DEBUG [NioProcessor-2] (AbstractHttpLogicHandler.java:138) -   response header received:
HTTP/1.0 407 Proxy Authentication Required\r\n
Proxy-Authenticate: NTLM\r\n
Proxy-Connection: keep-alive\r\n
Content-Length: 0\r\n
Set-Cookie: SERVERID=proxy11; path=/\r\n
\r\n

3883 [NioProcessor-2] DEBUG org.apache.mina.proxy.handlers.http.AbstractHttpLogicHandler  -   response header received:
HTTP/1.0 407 Proxy Authentication Required\r\n
Proxy-Authenticate: NTLM\r\n
Proxy-Connection: keep-alive\r\n
Content-Length: 0\r\n
Set-Cookie: SERVERID=proxy11; path=/\r\n
\r\n

DEBUG [NioProcessor-2] (AbstractHttpLogicHandler.java:359) -   parseResponse()
3883 [NioProcessor-2] DEBUG org.apache.mina.proxy.handlers.http.AbstractHttpLogicHandler  -   parseResponse()
NioProcessor-2, called closeOutbound()
NioProcessor-2, closeOutboundInternal()
NioProcessor-2, called closeInbound()
NioProcessor-2, fatal error: 80: Inbound closed before receiving peer's close_notify: possible truncation attack?
javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
NioProcessor-2, SEND TLSv1.2 ALERT:  fatal, description = internal_error
NioProcessor-2, Exception sending alert: java.io.IOException: writer side was already closed.
DEBUG [NioProcessor-2] (SslHandler.java:212) - Unexpected exception from SSLEngine.closeInbound().
javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
	at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666)
	at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634)
	at sun.security.ssl.SSLEngineImpl.closeInbound(SSLEngineImpl.java:1561)
	at org.apache.mina.filter.ssl.SslHandler.destroy(SslHandler.java:210)
	at org.apache.mina.filter.ssl.SslFilter.initiateClosure(SslFilter.java:787)
	at org.apache.mina.filter.ssl.SslFilter.filterClose(SslFilter.java:707)

[chrjohn]

Hi,

sorry, I think I cannot be much of help regarding the proxy stuff since I never used it. Maybe someone else can. I would advise you write a mail to the mailing list: https://sourceforge.net/projects/quickfixj/lists/quickfixj-users

But did you try to play around with the other proxy parameters like ProxyDomain, ProxyWorkstation etc? From what I gathered from googling they seem to be used for NTLM authentication.

Chris.