Agile modelling and development

It has often bothered me a little that the early stages of a project seem to fly in the face of the "normal" agile mantra that all effort should be directed to satisfying a user story. But in early project stages, I find myself spending time on high-level design activities activities, and creating technical infrastructure, that of themselves do not directly contribute to a user story. I find myself wondering if that's because I'm too technically motivated to be a really effective agile developer. Recently, a colleague pointed me to an essay, Agile Model Driven Development (AMDD) [1], from which I take some considerable comfort. What is recognized here is that even for agile development, it can be appropriate to take some time (but not too much!) to perform "requirements envisioning" and "initial architecture envisioning", to "get a good gut feel what the project is all about" and to "identify an architecture that has a good chance of working". As always with agile development, the goal is "to get something that is just barely good enough so that your team can get going". This essay also says "For your architecture a whiteboard sketch overviewing how the system will be built end-to-end is good enough" - it seems to me that the important thing here, to be emphasized, is end-to-end (or: front-to-back). That is, to sketch a system that connects all the way from a front-end user to a back-end storage or service. Without really planning it, this approach is broadly what I've been doing in the early stages of Shuffl. In the light of this article, time I have spent, e.g., evaluating back-ends, even though the first iteration does not call for any persistence, does seem to be appropriate. As I near the end of the first iteration, I think I can say that having a view of what the back end may look like does help me to make decisions about how to structure aspects of the user interface code. An area where I've found my own practice of agile development has fallen down is poor estimation of effort and tasks. Getting the balance of granularity right for estimating is, I think, critical: too coarse a granularity and key elements are overlooked; too fine and the plans made don't reflect the actual development process when code has to be cut. Part of the problem here may be working alone, rather than part of a team, so that valuable discussion and review elements of the process are absent. Recommended reading! [1] Agile Model Driven Development (AMDD), by Scott Ambler. http://www.agilemodeling.com/essays/amdd.htm