Implementing AtomPub

WIN and FAIL: the protocol handler has been developed and tested, but has taken more than twice the effort originally estimated. The failure here is, I think more a failure of estimation than indicative of a technical problem.

Factors involved in the development of the AtomPub handler include: learning the relevant details of the AtomPub protocol; learning how to use the jQuery Ajax API; dealing with idiosyncracies of the eXist implementation of AtomPub; developing a framework for composing functions that return results via asynchronous callbacks.

The failure of estimation comes from not appreciating the range of supporting activities needed to implement the AtomPub handlers. Had the activities been identified when performing the estimates, I feel the estimates would have been closer to the final effort actually used. But there is a tension here: too fine a breakdown when estimating tasks leads to a plan that bears little resemblence to the actual activities for which progress is reported. Maybe finer-grained task breakdowns could be used for the purpose of estimation, but disregarded for the purposes of progress reporting. Or maybe the real problem here is simply doing task estimation without the benefit of group discussion with other team members? Maybe also helpful would be an outline of test cases used to drive the test-led development process.

Some specific technical issues faced down include:

• checking the formal specification for relevant details of the AtomPub protocol (0.5 day)
• learning how to use the jQuery Ajax API (0.5 day)
• dealing with idiosyncracies of the eXist implementation of AtomPub (about a days worth of trial-end error activity added to the main development task)
• developing and testing a framework for composing functions that return results via asynchronous callbacks (about 1 day, not recorded in the progress summary as it was performed in odd fragments of time during a vacation period)
• coding the test cases and handler (about 2 days, which was pretty much the original estimate)
• tracking down an obscure bug caused by erroneous multiple invocations of an asynchronous callback method (about 0.5 day)

So, all-in-all, the task took about 5 days, rather than the two estimated. A large part of this overrun can be attributed to learning details of technologies not used previously.

The AtomPub specification is bit vague in some areas about the URIs used for accessing and updating feeds and feed items. In particular, it gives no clear indication (by design, I think) of how feed URIs are constructed. I further feel that the eXist implementation of AtomPub may diverge in some respects from the spirit if not the letter of the AtomPub specification (and hence other implementations), by virtue of the way it uses a number of different base URIs for accessing different AtomPub-related functions. Also, the eXist implementation I use does not seem to implement service documents that allow discovery of atom feeds - this being at odds with eXist's own documention as well as the intent of the AtomPub spec; this will make it harder to implement a pure browsing interface for loading and saving shuffl workspaces. The AtomPub test cases have been developed to reflect the way that eXist works, and may need to be adjusted when refining the AtomPub handler to work with different server implementations.

Another tricky issue was that when creating an AtomPub "Media Resource" a resource title cannot be specified, so it must be applied by updating the Atom Entry returned by the initial create operation, which adds some complexity to the asynchronous completion logic in the handler.