Easy Python SOAP with SUDS, with a few Date/Time gotchas.


by
27 Sep
September 27, 2012

suds duckIn the world of web services, SOAP is a bit like the quip about the old Holy Roman Empire. It’s neither Simple nor an Object nor a Protocol (talk amongst yourselves). Actually though, the main thing is that it’s not all that simple. It’s just a teensy bit verbose and over-engineered for most uses. So in order to consume a partner’s SOAP web service with a minimum of overhead and teeth-gnashing, we use a very lightweight SOAP client called “suds”. In addition to its too-precious name, the other benefit of suds is that it is simple. To connect to a web service, you just do the following:

from suds.client import Client
url = 'http://localhost:3000/webservices/WebServiceTestThing?wsdl'
client = Client(url)

And you have a connection to the web service. It’s just that easy™. Let’s say you have a web service that tells you when it’s time for lunch.

lunchtime = client.service.getWhenItsTimeForLunch('Monday')

And you’re set. The nice thing about suds is it can take complex (or simple) response messages and turn them into easy-to-use Python types. It’s much easier to work with a Python dictionary than a long string of XML, and all date/time elements are automagically translated into built-in datetime objects. And… therein lies the problem. The ‘auto’ part is there, but it’s not quite ‘magical’.

The problem first showed itself when we got some unusual results in the times we were receiving from the webservice we were consuming. Things along the lines of:
08:32:44.000177
09:44:12.000552

Always with the three zeros, so it “obviously” wasn’t milliseconds. What could those numbers be? Offset information? The documentation we got for the webservice had timezone information listed as part of the result string we’d be getting, but these didn’t look like any timezone offset format I knew of.

There was much confusion until we took a look at the actual XML the web service was returning. There, the times were listed with milliseconds, and with timezone offset information showing they were UTC (Universal Coordinated Time; thanks, The French). What happened? Well, it turns out there are some “deficiencies” in the suds date module. First, milliseconds were handled incorrectly if the subsecond component wasn’t exactly 6 digits long. The webservice we were calling was MS-SQL based, and so only returns three digits. The suds module padded the result with zeros, but did it on the wrong end. Not a big deal, but a bit confusing.

This turned out to be a known issue, and there’s an open ticket on the suds webpage. In the end, though,  it was fortunate that this problem occurred for us, because in that ticket we found a couple of other “deficiencies” that would have certainly caused us issues down the line, and probably would have been much harder to troubleshoot. The second issue is that the timezone handling capabilities of the Python datetime classes aren’t used; the suds module just converts the value to localtime. No timezone offset information is provided to the caller. This kind of hidden “conversion” is good-intentioned, I suppose, but since we were expecting UTC time information, we didn’t really want it to happen. Still, once we know about it, we can deal with it. Except… the conversion also seems to ignore daylight savings information. So, any solution we implemented quite possibly would have broken once it was time to ‘fall back’. There’s also a small issue that fractional hours are ignored in the offset conversion, which didn’t affect us right now since most ‘half-hour’ timezones are follow-their-own-drummer locales like Caracas and Kabul.

Time Zones

Still, several things here that can cause very subtle errors in date handling. Fortunately for us, the kind person who reported the defect also provided a patch to fix things, including using timezone-aware datetime objects. After a code review of the patch and some testing to confirm it did what it says on the tin, we patched our suds libraries and were back in business.

So, what did we learn? Well, there’s a few lessons here. First, even if you rely on a module or library and it’s worked well for you in the past, there could be hidden problems with it that don’t show up until you try to use it a new way. Additionally, we all sometimes rely on things that aren’t quite ‘released.’ The current version on suds is 0.40, and that’s not likely to change any time soon (The ticket above is going on two years old). Now, in this day and age we really don’t have to wait until ’1.00′ to use something for production tasks. (How long was gmail in beta?) But it’s good to keep in mind that what you’re using might not be completely… “complete.” And finally, when troubleshooting, it’s very helpful to have what the person at the other end of the line has sent you and compare that to what you have received. There’s plenty of rather opaque open seas between the start of the voyage and the destination, and plenty of opportunity for something to be changed just a little bit.

UPDATE: Well, there’s always one more thing. While testing on a server that hadn’t been patched yet, we discovered some more odd behavior. The web service we were consuming sent a date/time object, which it then expected back when calling again. However, it wasn’t treated by the service on a second call as a date/time comparison, but as a token; either it matched or it didn’t. And since the format of the token was ‘helpfully’ changed; the call failed to operate properly.

So perhaps a couple extra lessons can be learned: first, be careful when ‘helpfully’ altering something without the end-user’s awareness. Doing an automatic replace of ‘UR’ to ‘you are’ sounds great and all, but then someone wants to talk about Ancient Sumeria and all heck breaks loose. And secondly, of course… When you patch something… make sure you patch all the servers.

Tags: , , , ,
© Copyright 2017 Findaway. All rights reserved.