Problems connecting to SagePay's TEST and SIM servers - FIXED

Since Tuesday afternoon we have found that we can't connect to the TEST or SIM servers at SagePay. Live is fine however, even though the exact same code is used.

It's a strange problem as an immediate connection failure is received when attempting to POST a transaction registration to either the test or sim servers.

After this had happened we contacted SagePay support who did some checks but told us that there are no problems on the server. However, when I did a quick Twitter search I found another developer who is also getting this issue. I confirmed the problem via my local development server in the office and on the live server so it doesn't appear to be a specific ISP routing problem etc.

However over the past few day SagePay have confirmed that a small number of user's are having problems. It's most certainy not affecting the majority though.

This has led support to believe that the problem is due to Coldfusion and the way it is sending the POST request to the SagePay server.

The Solution

Luckily while throwing this problem out to Twitter a very helpful user (@Zefer) suggested that the problem might be due to the server sending back a gzipped response which cfhttp in Coldfusion can't handle (and certain URL functions in PHP it seems). To fix this we just need to send headers to tell the server what we can accept:

<cfhttpparam type="Header" name="Accept-Encoding" value="deflate;q=0">
<cfhttpparam type="Header" name="TE" value="deflate;q=0">

The raw headers sent in the request will look like this:

Accept-Encoding: deflate;q=0
TE: deflate;q=0

Once I made this change the problem went away immediately.

I assume that SagePay must have made a system change on SIM and TEST but not on LIVE (luckily).

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Robert Rawlins's Gravatar Hey, nice work finding that solution J, I've had the exact same issue with gZipped responses coming back from certain providers publishing their RSS feeds and tackle it in much the same way.

I found it a little confusing at the time, I guess they do it to try and save on bandwidth or something? Just seems odd to compress data like that in this day and age.

Rob
# Posted By Robert Rawlins | 9/18/09 1:57 PM
James Allen's Gravatar Hey thanks Rob.

There is a good reason for compression though as it not only saves on the bandwidth of the website but also means users get the pages quicker as they aren't downloading as much data.

I'm going to implement gzip compression on errrrrrr.com to help with the JS load and hopefully improve the speed for our users.

I am surprised CF8 can't handle GZIP compressed responses yet though. Wonder if CF9 addresses this...
# Posted By James Allen | 9/18/09 2:01 PM
Robert Rawlins's Gravatar Just looking at the solution I have here and it looks like this:

<cfhttpparam type="header" name="Accept-Encoding" value="compress,gzip,deflate" />

slightly different to yours but I would imagine is achieving the same thing.

Rob
# Posted By Robert Rawlins | 9/18/09 4:17 PM
James Allen's Gravatar That is a bit weird as it appears that is telling the server cfhttp can accept gzip compressed content which I don't think it can. But it worked for you so maybe that's not what it's doing or it .. 'just works'.. :)
# Posted By James Allen | 9/18/09 5:30 PM
Glyn Jackson's Gravatar Basically it's to do with the HTTP Compression in IIS. CFHTTP couldn't read the compressed header. This is due to IIS's compression scheme being incompatible with CFHTTP

I reported this as an issue to Adobe some time back as it was fixed in MX7 but was back in CF8 :(

http://www.newebia.co.uk/blog/index.cfm/2008/12/5/...
# Posted By Glyn Jackson | 9/18/09 8:18 PM
James Alllen's Gravatar Thanks Glyn, I thought I had heard something about IIS compression and cfhttp. Amazed that it was fixed in CF7 but came back in as an issue in CF8.. Crazy.

I hope it's fixed for CF9 - it's about time!
# Posted By James Alllen | 9/18/09 9:30 PM
© 2014 James Allen | Contact Me
This blog runs on the awesome power of BlogCFC - created by Raymond Camden. This blog is running version 5.9.