Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-3299

Bad Gateway The proxy server received an invalid response from an upstream server.

    Details

    • Type: Bug
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: 2.2.4
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      CentOS 5.5
      Java 1.6.0_20
      Artifactory 2.2.4 Bundled Version
      Frontend HTTP: Apache httpd Version 2.2.3 with mod_jk/1.2.28

      Description

      Trying to uppload an new artifact, clicking on upload button the following message appears in browser:
      Bad Gateway
      The proxy server received an invalid response from an upstream server.
      Access to artifactory is thru AJP with frontend apache httpd server
      Trying this on jetty http port directly works

      In consoleout.log the is the following message:
      2010-06-17 14:33:45.368:WARN::handle failed?
      java.io.IOException: FULL
      at org.eclipse.jetty.ajp.Ajp13Parser.fill(Ajp13Parser.java:199)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseNext(Ajp13Parser.java:553)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseAvailable(Ajp13Parser.java:156)
      at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:250)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)
      2010-06-17 14:33:45.478:WARN::handle failed?
      java.io.IOException: FULL
      at org.eclipse.jetty.ajp.Ajp13Parser.fill(Ajp13Parser.java:199)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseNext(Ajp13Parser.java:553)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseAvailable(Ajp13Parser.java:156)
      at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:250)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)
      2010-06-17 14:33:45.589:WARN::handle failed?
      java.io.IOException: FULL
      at org.eclipse.jetty.ajp.Ajp13Parser.fill(Ajp13Parser.java:199)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseNext(Ajp13Parser.java:553)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseAvailable(Ajp13Parser.java:156)
      at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:250)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)
      2010-06-17 14:33:45.730:WARN::handle failed?
      java.io.IOException: FULL
      at org.eclipse.jetty.ajp.Ajp13Parser.fill(Ajp13Parser.java:199)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseNext(Ajp13Parser.java:553)
      at org.eclipse.jetty.ajp.Ajp13Parser.parseAvailable(Ajp13Parser.java:156)
      at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:250)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)

        Attachments

          Activity

          Hide
          yoavl Yoav Landman added a comment -

          Can you attach the relevant part from your current apache config? Did this configuration work for you before?

          Show
          yoavl Yoav Landman added a comment - Can you attach the relevant part from your current apache config? Did this configuration work for you before?
          Hide
          yoavl Yoav Landman added a comment -

          I missed the fact that it's AJP - we have seen this many times before.
          My suggestion in this case would be to use mod_proxy not mod_proxy_ajp with Jetty.

          From Jetty's side it looks like the combination of Jetty7 + AJP is not a happy marriage:
          "It is recommended to NOT use the AJP protocol, and superior performance and clearer semantics will be achieve using HTTP." - http://wiki.eclipse.org/Jetty/Tutorial/Apache#Configuring_mod_proxy_ajp
          Whatever that means the suggestion is switch to mod_proxy.

          Show
          yoavl Yoav Landman added a comment - I missed the fact that it's AJP - we have seen this many times before. My suggestion in this case would be to use mod_proxy not mod_proxy_ajp with Jetty. From Jetty's side it looks like the combination of Jetty7 + AJP is not a happy marriage: "It is recommended to NOT use the AJP protocol, and superior performance and clearer semantics will be achieve using HTTP." - http://wiki.eclipse.org/Jetty/Tutorial/Apache#Configuring_mod_proxy_ajp Whatever that means the suggestion is switch to mod_proxy.
          Hide
          markus.pohle Markus Pohle added a comment -

          Yes, it is AJP, as written before it is mod_jk. And yes it worked before. Till version 2.2.1 everything was fine. Then starting with 2.2.3 and now with 2.2.4 this message appears.

          My mod_jk.conf looks like this:
          ===========================

          LoadModule jk_module modules/mod_jk.so

          JkShmFile "logs/mod_jk.shm"
          JkLogFile "logs/mod_jk.log"
          JkLogLevel info

          JKWorkerProperty worker.list=jkstatus,artifactorylb

          JKWorkerProperty worker.jkstatus.type=status

          JKWorkerProperty worker.artifactorylb.type=lb
          JKWorkerProperty worker.artifactorylb.balance_workers=artifactory

          JKWorkerProperty worker.artifactory.port=8039
          JKWorkerProperty worker.artifactory.host=127.0.0.1
          JKWorkerProperty worker.artifactory.type=ajp13

          JKMount /artifactory* artifactorylb

          My AJP Config in jetty.xml is this:
          =========================

          <!-- AJP13 Connector configuration -->
          <Call name="addConnector">
          <Arg>
          <New class="org.eclipse.jetty.ajp.Ajp13SocketConnector">
          <Set name="port">8039</Set>
          </New>
          </Arg>
          </Call>

          I do not use mod_proxy_ajp and mod_proxy, and normally I do not want to switch. If there is now way to fix it to work with mod_jk and AJP i will try Tomcat as servlet container.

          Show
          markus.pohle Markus Pohle added a comment - Yes, it is AJP, as written before it is mod_jk. And yes it worked before. Till version 2.2.1 everything was fine. Then starting with 2.2.3 and now with 2.2.4 this message appears. My mod_jk.conf looks like this: =========================== LoadModule jk_module modules/mod_jk.so JkShmFile "logs/mod_jk.shm" JkLogFile "logs/mod_jk.log" JkLogLevel info JKWorkerProperty worker.list=jkstatus,artifactorylb JKWorkerProperty worker.jkstatus.type=status JKWorkerProperty worker.artifactorylb.type=lb JKWorkerProperty worker.artifactorylb.balance_workers=artifactory JKWorkerProperty worker.artifactory.port=8039 JKWorkerProperty worker.artifactory.host=127.0.0.1 JKWorkerProperty worker.artifactory.type=ajp13 JKMount /artifactory* artifactorylb My AJP Config in jetty.xml is this: ========================= <!-- AJP13 Connector configuration --> <Call name="addConnector"> <Arg> <New class="org.eclipse.jetty.ajp.Ajp13SocketConnector"> <Set name="port">8039</Set> </New> </Arg> </Call> I do not use mod_proxy_ajp and mod_proxy, and normally I do not want to switch. If there is now way to fix it to work with mod_jk and AJP i will try Tomcat as servlet container.
          Hide
          yoavl Yoav Landman added a comment -

          2.2.3 and above uses Jetty 7 and the problem is probably around AJP support in general, since both mod_jk and mod_proxy_ajp are not recommend by Jetty 7 according to their wiki.
          We will look into this and try to reproduce it, but switching to Tomcat is a good option. FYI, all our and our clients production servers use Tomcat + AJP (mod_proxy_ajp) with no issues.

          Show
          yoavl Yoav Landman added a comment - 2.2.3 and above uses Jetty 7 and the problem is probably around AJP support in general, since both mod_jk and mod_proxy_ajp are not recommend by Jetty 7 according to their wiki. We will look into this and try to reproduce it, but switching to Tomcat is a good option. FYI, all our and our clients production servers use Tomcat + AJP (mod_proxy_ajp) with no issues.

            People

            • Assignee:
              yoavl Yoav Landman
              Reporter:
              markus.pohle Markus Pohle
            • Votes:
              2 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: