Shuffl sprint 6 plan

The plan for Sprint 6 has been posted at http://code.google.com/p/shuffl/wiki/SprintPlan_6.

The main targets for this sprint will be to work towards improved usability in two areas:

• Visualization of data: after demonstrating the data graphing interface to Chris Holland, I hope to find out what he really wants, and to get some representative sample data.
• Improve usability of the interface for importing data and loading/saving workspaces.

Shuffl sprint 5 progress review

The plan for sprint 5 has been updated with progress and review notes:

• http://code.google.com/p/shuffl/wiki/SprintPlan_5

This was a sprint of highs and lows - there were significant wins, but also more distractions, and the direction of progress seems to be diverging more from the original plan. A user demonstration session was postponed, to take place early in the next sprint. I have also spent a fair amount of time cleaning up existing code in various ways.

On the positive side, a demonstrable drag-and-drop data graphing interface has been completed (as planned), and I worked with Jun Zhao to successfully implement a card containing a FlyUI gene finder widget (not planned).

I have started to look at making the data access modules more robust - currently, an I/O error (such as accessing a non-existent web resource) can leave the application broken.

Encouragingly, plans are now being laid for projects that will continue aspects of Shuffl beyond the current Rapid Innovation project.

Task completion velocity is down significantly from the previous sprint (tasks completed/effort spent: 4.5/7.4 = 0.61), which reflects the various distractions from the intended development plan.

FlyUI GeneFinder widget in Shuffl card

I recently spent a day working with my colleague Jun Zhao to successfully implement a Shuffl card containing a FlyUI (http://code.google.com/p/flyui/) gene finder widget.

This achievement is an important confidence builder, as it further demonstrates the power and flexibility of the Shuffl card plugin model: we were able to embed quite complex FlyUI widget logic unmodified into a Shuffl card. The FlyUI widgets are built using the Yahoo YUI libraries, so this success also demonstrates that Shuffl (based on jQuery) can coexist with YUI code.

The remaining challenge is to find easy ways to deploy such applications. To overcome the Browsers' "same origin" restriction on Ajax calls, the Shuffl+GeneFinder application was deployed via an Apache server configured to redirect and proxy requests to the SPARQL endpoint for FlyBase data hosted at OpenFlydata.org. In this, Shuffl is playing out quite nicely as a flexible user interface toolkit for application developers, but it is not yet clear how to deploy Shuffl as a general purpose, user-configurable data viewing and annotation tool. Maybe we should think about deploying a Shuffl application server? Could Google Apps, or similar, be recruited for this purpose?

Shuffl drag-and-drop data visualization

An initial cut of data visualization in Shuffl is working. A "data table card" card can be dropped on a "data graph card" to display graphs of the data in the table. Thanks to jQuery, the drag-and-drop interface was a breeze to code, and the model pattern discussed previously makes it very easy to link the UI functions to underlying data logic. I'm using a jQuery plugin called 'flot' (http://code.google.com/p/flot/) to do the actual graphing.

Instructions for running the demonstration are at http://code.google.com/p/shuffl/wiki/Shuffl_Demonstration.

I intend to demonstrate this to one of my research collaborators to help determine how this should be carried forward.

I've also updated the project front-page (http://code.google.com/p/shuffl/) to reflect the current state of the project, and added a screenshot and updated links to the demonstration application and Shuffl-based brief introduction.

MVC pattern and mock widgets

I've just discovered that the MVC approach I've adopted (http://shuffl-announce.blogspot.com/2009/09/framework-for-testing-shuffl-card.html) makes it really easy to create mock shuffl cards (in the sense of mock objects: http://en.wikipedia.org/wiki/Mock_object) for testing.

Here's one I used today to test the drag-and-drop drop-target logic in a data visualization card:

// Instantiate mock table card

var tc = jQuery("<div/>");

tc.model('shuffl:labels',carddatagraph_labels1);

tc.model('shuffl:series',carddatagraph_series1);

The jQuery.model plugin I mentioned previously provides all the additional logic needed to ensure the mock card object responds as required to requests for information.

JISCRI reporting tags

I've decided to vary slightly the tags I use for reporting-via-blog. Updated details are noted at http://code.google.com/p/shuffl/wiki/ProjectPlanOutline_200906_200911.

In summary:

• some progress reports are neither WINs nor FAILs, just reports.
• and there was no tag set for ongoing planning, which I think is required for an agile project structure - goals change and new plans are created for each sprint/iteration - none of the existing proposed tag sets really fit this.

Shuffl sprint 5 plan

The plan for Sprint 5 has been posted at http://code.google.com/p/shuffl/wiki/SprintPlan_5.

Responding to research user desiderata, project activities are now diverging quite significantly from the original outline plan (other than user engagement still being a key activity at this stage).

The primary focus for this sprint will be to complete the initial data graphing display for the visualization use case, to demonstrate this to the researcher who requested this, and use that session to gather more detailed requirements for this aspect of Shuffl.

Secondary foci will be: improving aspects of the user interface, especially for workspace loading/saving, and creating some user documentation.

I also have an ongoing discussion with Ross Gardler about long-term sustainability and CLAs (contributor licence agreements) - see http://code.google.com/p/shuffl/issues/detail?id=9 for details.

Shuffl sprint 4 progress review

The plan for sprint 4 has been updated with progress and review notes:

• http://code.google.com/p/shuffl/wiki/SprintPlan_4

In outline, the sprint activity was:

• week 1: complete workspace persistence (initial cut)
• week 2: UI for data access; expand testing framework; card refactoring to MVC pattern
• week 3: tabular data loading and display; started on data graphing

The sprint has been characterized by good progress and technical compromises. Many of the completed tasks are not implemented completely as I would like or had originally intended, so I feel I may be accumulating some "technical debt" that will need to be addressed before the project can be truly usable by unsupported researchers and others. The user interface for saving/restoring workspaces is a case in point. See http://code.google.com/ p/shuffl/wiki/ TechnicalDebt for more details.

There has been less community engagement than planned, though some new use-cases have been noted (FlyKit, e-learning).

For the first time, I have calculated a "velocity"figure (planned effort for completed tasks/total sprint effort): 11.0/11.4 = 0.96. This figure was a surprise to me: I was expecting it to be lower on account of the unplanned time spent on admin, etc. Total sprint effort here does not include the small amount of unplanned additional time spent on other projects. Prediction of progress remains challenging with so many competing demands on limited time.

Shuffl data visualization use-case updated

I now have permission to publicly post my full notes from a meeting with Chris Holland, a researcher for whom I am targetting an application of Shuffl (and one of the supporters of our bid). Notes of our meeting are here:

• http://code.google.com/p/shuffl/wiki/20090902_UseCase_DataVisualization