Apache 2.2.26 and backend server - HTTP 503

Hi there.

I have and apache server in front a wildfly server running for 2 years now. Recently I deployed on wildfly a java web start program (a desktop program that has auto-update on startup) and when e start the program, my apache server start to respond http 503 for everything. Then i need to wait for a while and sometimes i need to restart my backend server.

If I bypass apache and connect directly to wildfly, everything runs ok.

My suspicious is that apache is not handling http 304 from wildfly. Or maybe apache is running out of resources? the files has 3mb em average.

Apache log:

[Sun Feb 28 16:51:52 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:51:52 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:52:04 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:52:04 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:51 2016] [error] (105)No buffer space available: proxy: HTTP: attempt to connect to 127.0.0.1:8080 (localhost) failed
[Sun Feb 28 16:54:51 2016] [error] ap_proxy_connect_backend disabling worker for (localhost)
[Sun Feb 28 16:54:51 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:51 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:51 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:52 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:52 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:52 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:52 2016] [error] proxy: HTTP: disabled connection for (localhost)
[Sun Feb 28 16:54:52 2016] [error] proxy: HTTP: disabled connection for (localhost)

Wildfly responses log:

FIRST REQUEST:
----------------------------REQUEST---------------------------
URI=/xxxpdvweb/lib/liquibase.jar
characterEncoding=null
contentLength=-1
contentType=null
header=Connection=Keep-Alive
header=X-Forwarded-For=189.114.143.103
header=X-Forwarded-Server=_wildcard_.xxx.com.br
header=User-Agent=Apache-HttpClient/4.2.6 (java 1.5)
header=Host=demo.xxx.com.br
header=X-Forwarded-Host=demo.xxx.com.br
locale=[]
method=GET
protocol=HTTP/1.1
queryString=
remoteAddr=/127.0.0.1:21991
remoteHost=localhost
scheme=http
host=demo.xxx.com.br
serverPort=8080

--------------------------RESPONSE--------------------------
contentLength=2183684
contentType=application/octet-stream
header=Connection=keep-alive
header=Last-Modified=Fri, 26 Feb 2016 01:07:10 GMT
header=X-Powered-By=Undertow/1
header=Server=WildFly/8
header=Content-Type=application/octet-stream
header=Content-Length=2183684
header=Date=Sun, 28 Feb 2016 20:36:52 GMT
status=200

SECOND REQUEST:
----------------------------REQUEST---------------------------
URI=/xxxpdvweb/lib/liquibase.jar
characterEncoding=null
contentLength=-1
contentType=null
header=Connection=Keep-Alive
header=X-Forwarded-For=189.114.143.103
header=X-Forwarded-Server=_wildcard_.xxx.com.br
header=User-Agent=Apache-HttpClient/4.2.6 (java 1.5)
header=If-Modified-Since=Fri, 26 Feb 2016 01:07:10 GMT
header=Host=demo.xxx.com.br
header=X-Forwarded-Host=demo.xxx.com.br
locale=[]
method=GET
protocol=HTTP/1.1
queryString=
remoteAddr=localhost/127.0.0.1:21989
remoteHost=localhost
scheme=http
host=demo.xxx.com.br
serverPort=8080
--------------------------RESPONSE--------------------------
contentLength=0
contentType=null
header=Connection=keep-alive
header=X-Powered-By=Undertow/1
header=Server=WildFly/8
header=Content-Length=0
header=Date=Sun, 28 Feb 2016 20:37:01 GMT
status=304



Anybody has a hint?

Thanks

 

Almost giving up and connecting directly to the backend server.

Sometimes the problem seems to be related with the os (CentOS), but Im really in the dark here.

 

ProxyPass / http://127.0.0.1:8080/ retry=0 timeout=5
ProxyPassReverse / http://127.0.0.1:8080/

Code (markup):

Does this work?

 

I tried using only retry=0 and the VPS got stuck for a while.
At night (nobody is connected) I will try again using this combination to see what will happen.

Thanks for the answers

 

Also I read somewhere you could try to disable keepalive (keepalive_timeout 0; ) and decrease size of tcp buffers (listen 80 default rcvbuf=8k sndbuf=32k; ). Try and see if it works...

 

Can you tell me where i can change this? Im not very familiar with apache yet

 

Do you also have nginx? Because those were for nginx

 

No i dont use nginx .

Well, the parameters didnt work, but this yes:

<IfModule prefork.c>
StartServers 2
MinSpareServers 1
MaxSpareServers 5
ServerLimit 40
MaxClients 40
MaxRequestsPerChild 4000
</IfModule>

for some reason, the values were low, so i increased a little and the problem seems to be solved, but:

Connecting directly with the backend server, the response time is (ms):
19089, 20389, 28436, 30534, 30831, 30195, 31534

Using apache to redirect:
21359, 22993, 34582, 37808, 70360 (wow), 70155, ...

Any hints of how to get better results? (That 70360 was to high)

I have +- 250 MB for apache, maybe i should use lower values ?

Thanks!