2

A curl command in a Bash script (called by cron) asks a web service/API, on another device/server (hosts both the API and the SQL database), to perform an operation on a database. How long this operation takes, depends on the amount of rows on which the operation is performed.

It is important that the API communicates the result back to me, so it is programmed to do so.

I notice that when the operation is performed on a larger subset of rows (say hundreds of thousands, rather than tens of thousands), that takes – ballpark figure – more than a couple of minutes for the operation to complete, I get no feedback but only an error:

“502 – Web server received and invalid response while acting as a gateway or proxy server”

“When the Web server contacted the upstream content server, it received an invalid response from the content server.”

I looked up the options curl gives me and keep-alive is the default setting. I nonetheless specified it, but no luck:

curl -s --keep-alive-time 5 –X GET “https://api.url.com/api/call” –H “accept: application/octet-stream” –H “Authorization: Bearer $password”

Since that is the only command I invoke and, apart from keep-alive, --connect-timeout and –m as per this thread don’t seem to be applicable, I’m fast running out of ideas.

Could anyone point me in the right direction?

EDIT: Checking the logs, as suggested, indicates I need to increase the ARR time-out ([Windows article][2]). Other threads explain the simple steps.

However, I do not see Application Request Routing (ARR) in IIS 1607 (Windows Server 2016).

[2]: http://(https://docs.microsoft.com/en-us/iis/extensions/troubleshooting-application-request-routing/troubleshooting-502-errors-in-arr

Nils Deschrijver
  • 41
  • 1
    HTTP 502 is a server-side error, and it indicates that you have a gateway/proxy server in between. You can check its logs for more information. One possibility I can think of is the response being too large. A web server can be configured not to process messages beyond a specified size. – Haxiel Oct 07 '20 at 08:03
  • The response is always a very short string (think: max. a couple of names of people). – Nils Deschrijver Oct 08 '20 at 10:14
  • You say this is triggered by performing the operation on 100s of thousands but not 10s of thousands... sounds like you're hammering the API gateway so hard, the thing it's talking to can't keep up and is internally dropping connections. – Philip Couling Oct 08 '20 at 11:57
  • 1
    @NilsDeschrijver Addressing the edit - Can you add the version of IIS you're running? I found a few instruction documents, but the process apparently varies by version. Something else I want to add is regarding the API itself. If the response time varies widely, the operation may not be well suited to a synchronous HTTP call. – Haxiel Oct 08 '20 at 13:08
  • @Haxiel Done! Thank you. – Nils Deschrijver Oct 09 '20 at 12:51
  • 1
    @NilsDeschrijver Okay, that must be IIS 10. Can you check this QA and see if it resolves the issue? – Haxiel Oct 09 '20 at 17:10
  • Increased ARR timeout to 20 minutes but still getting 502 error, despite operation taking 2 minutes. – Nils Deschrijver Oct 13 '20 at 15:08

0 Answers0