Showing posts with label atompub. Show all posts
Showing posts with label atompub. Show all posts

Wednesday, 9 September 2009

DELETE on AtomPub media resource

I read somewhere that deleting an AtomPub media resource associated with a feed item (using HTTP DELETE) should also delete the feed item. It turns out that the eXist AtomPub implementation does not support this, claiming (erroneously) that the resource does not exist.
Examining the AtomPub spec provides little support (though on first reading I actually thought this did support what I want to do):
To delete a Member Resource, a client sends a 
DELETE request to its Member URI, as specified 
in [RFC2616].  The deletion of a Media Link
Entry SHOULD result in the deletion of the 
corresponding Media Resource.
-- RFC 5023, section 9.4
This is unfortunate for Shuffl, as I had been hoping to not get too bogged down in feed items vs resources. Now I might need to include a feed item URI along with card data if I'm to be able to delete the card data. Or maybe I'll just delete the feed and recreate it without the missing data?
More generally, I am beginning to wonder if AtomPub was the best choice for persisting Shuffl data. I estimate the complexities of dealing with AtomPub have cost me a week or so in slippage, and that I would be better off with WebDAV. I'll stick with it for now, but I'm thinking about a plug-in framework for Shuffl back-end support so I can support alternative protocols.

Friday, 28 August 2009

AtomPub with jQuery implementation done

WIN and FAIL: the AtomPub protocol handler has been developed and tested, but has taken more than twice the effort originally estimated. The failure here is, I think more a failure of estimation than indicative of a technical problem. For details, see http://shuffl-announce.blogspot.com/2009/08/implementing-atompub.html.

Implementing AtomPub

WIN and FAIL: the protocol handler has been developed and tested, but has taken more than twice the effort originally estimated. The failure here is, I think more a failure of estimation than indicative of a technical problem.
Factors involved in the development of the AtomPub handler include: learning the relevant details of the AtomPub protocol; learning how to use the jQuery Ajax API; dealing with idiosyncracies of the eXist implementation of AtomPub; developing a framework for composing functions that return results via asynchronous callbacks.
The failure of estimation comes from not appreciating the range of supporting activities needed to implement the AtomPub handlers. Had the activities been identified when performing the estimates, I feel the estimates would have been closer to the final effort actually used. But there is a tension here: too fine a breakdown when estimating tasks leads to a plan that bears little resemblence to the actual activities for which progress is reported. Maybe finer-grained task breakdowns could be used for the purpose of estimation, but disregarded for the purposes of progress reporting. Or maybe the real problem here is simply doing task estimation without the benefit of group discussion with other team members? Maybe also helpful would be an outline of test cases used to drive the test-led development process.
Some specific technical issues faced down include:
  • checking the formal specification for relevant details of the AtomPub protocol (0.5 day)
  • learning how to use the jQuery Ajax API (0.5 day)
  • dealing with idiosyncracies of the eXist implementation of AtomPub (about a days worth of trial-end error activity added to the main development task)
  • developing and testing a framework for composing functions that return results via asynchronous callbacks (about 1 day, not recorded in the progress summary as it was performed in odd fragments of time during a vacation period)
  • coding the test cases and handler (about 2 days, which was pretty much the original estimate)
  • tracking down an obscure bug caused by erroneous multiple invocations of an asynchronous callback method (about 0.5 day)
So, all-in-all, the task took about 5 days, rather than the two estimated. A large part of this overrun can be attributed to learning details of technologies not used previously.
The AtomPub specification is bit vague in some areas about the URIs used for accessing and updating feeds and feed items. In particular, it gives no clear indication (by design, I think) of how feed URIs are constructed. I further feel that the eXist implementation of AtomPub may diverge in some respects from the spirit if not the letter of the AtomPub specification (and hence other implementations), by virtue of the way it uses a number of different base URIs for accessing different AtomPub-related functions. Also, the eXist implementation I use does not seem to implement service documents that allow discovery of atom feeds - this being at odds with eXist's own documention as well as the intent of the AtomPub spec; this will make it harder to implement a pure browsing interface for loading and saving shuffl workspaces. The AtomPub test cases have been developed to reflect the way that eXist works, and may need to be adjusted when refining the AtomPub handler to work with different server implementations.
Another tricky issue was that when creating an AtomPub "Media Resource" a resource title cannot be specified, so it must be applied by updating the Atom Entry returned by the initial create operation, which adds some complexity to the asynchronous completion logic in the handler.

Sunday, 16 August 2009

Blogs, research data and preservation

Skimming through an aggregation of JISCRI posts circulated by David Flanders, I noticed this mention of ArchivePress, which seems to be relevant to some of the goals of the research group in which Shuffl is being developed. Our interests are re-use and preservation of research data, most of which does not make its way into archival journals and is lost when the original researcher "moves on". ArchivePress is also about preservation of useful knowledge that doesn't make it into archival journals, and I'm thinking the ideas may be also applicable to data. Shuffl is part of an activity that attempts to make it easier to capture and share highly heterogeneous data from small research teams, but does not of itself address preservation. Can the acquisition of research data benefit from the journal pattern that underpins the operation of blogs? And as such, can data preservation build upon projects like ArchivePress? Factors in favour:
  • Shuffl is already being designed to use Atom (via AtomPub), a format with its roots in representing blogs
  • Research data is typically captured over a period of time
  • The card metaphor used by Shuffl operates at a a level of granularity that is arguably comparable with a blog post
Factors (maybe) against:
  • Shuffl is intended to allow progressive refinement of structure in data, both within and between data held on different cards - it is not clear now these refinements would be captured and navigated in a journal-like framework
  • AchivePress seems to be WordPress-specific - I don't know if this is a problem
I think I need to be more sensitive to developments in the area of "data blogs" - I just tried to Google for that, and didn't immediately see anything very enlightening. Maybe the closest thing I've come across personally is http://timetric.com/, which was discussed at a recent Oxford Geek Nights session. Maybe myExperiment and related work has something to offer, though it appears to be very workflow-oriented? I'm sure there's more.

Thursday, 9 July 2009

Shuffl back-end selected

The initial back-end software for Shuffl has been selected and checked out. I'm planning to use AtomPub for the back-end protocol, and eXist as the back-end database. Rationale for the choice can be seen at http://code.google.com/p/shuffl/wiki/BackendSystemsSurvey. I've installed eXist (piece of cake!) and run a series of tests against the out-of-box installation to demonstrate the identified capabilities are all present. See http://code.google.com/p/shuffl/source/browse/#svn/trunk/spike.