Plans, Planning and Speculation - There's Time for Everything

It has been funny to me from the very beginning that agile methods are often claimed to lack the planning activity while the opposite holds. The difference lies in the fact that planning for the unknown becomes speculation. Prolonged speculation period brings very little added value and is thus kept to minimum in agile projects until better knowledge is achieved by active learning. Simon Baker in AGILE IN ACTION reminded about the two great early agilists believing in planning instead of plans.

Life in a Cubicle

Are you working in a sorry ass company that forces people to work in cubicles in order to foster communication and effectivity? It's sad, but turn on your speakers, pump up the volume, and check out this James Blundt remix.

Hope it helps.

Portfolio Through The Grapewine

Many agile methods miss to address the issue of portfolio management. This is not such a surprise as many organizations miss to address the issue all together. Agile thinking pretty much assumes that the only intelligent way is to work on a single project at the time, and that projects should face delivery or cancellation as soon as possible. After all having too many projects in the inventory is one of the main sources of waste. While all this is very much true, the life on planet earth is sometimes somewhat different. The impression "I heard it through the grapewine" is made famous by Marvin Gaye. The impression means information traveling through gossip and rumor. In some organizations this seems to be the primary channel for portfolio management.

Team is defined as group of people with shared objectives and goals (term "team" is misused a lot!). Teams become superior over individuals when they share these goals and thus have a common direction. Agile methods take advantage of this phenomenon by planning for short enough increments and gaining commitment by having estimates given by the people doing the actual work. This is called time-boxing.

When a team in large organization is piloting agile methods it is usually working as a shadow system parallel to institutional system in the organization. In this case the above mentioned "I heard it through the grapewine" -effect can be very harmful. Each team member will have several inputs about the future of the current project, as well as information about the future projects. Along the current increment short notice invitations to future project meetings will arrive and so on. This makes agile planning unmotivating and even barking mad. Of course the same applies to any method of planning under these conditions. However time-boxing would be perfect tool against "I heard it through the grapewine" -effect, allowing organizations to harness the power of small teams with shared goals and direction. DeMarco calls these teams Jelled (DeMarco and Lister, 1999). If new announcements and meeting invitations are only made public in the same natural rhythm as the team already operates, their negative effect to Jell would be minimized. Who is responsible for this rhythm? Top management, middle management, project management, line management, team leadership? I think this needs to be considered independently for each case, but it is a joint responsibility and everyone should understand the reasoning.

"Increment duration depends on how long you can keep the change away". Allowing rumors and gossip make this a very short period.

The Uneducated Industry

In many cases professional embedded developer has never attended a single programming course at any level. Further she may have never opened a book on even the most fundamental programming practices. She does not read professional magazines. Embedded programmers are often electronics graduates. Even more scary is that the same applies to managers of embedded developers. They have never attended a single programming course at any level, NOR any managing course at any level. Embedded development department managers have just become managers because they have earlier been successful in programming (and designing the HW) with those 1kB micros solo virtuoso and ad-hoc.

Imagine that. In the past years we have reached an era where a light switch has more bit crunching power than Commodore 64. In few years a key chain, a pen or your T-shirt may out power the old 64. The complexity of these devices is exploded by the communication capabilities which enables distributed intelligence. These increasingly complex systems need to be developed within ever shorter time-to-market by team(s) of embedded software developers co-designing the system with teams from other disciplines. Members of such teams need to have proper skills for development, but even more challenging is that developers and their managers need to have social skills in order to keep the whole development ecosystem fit.

These skills are now been teached in programming and management departments, books and articles, and then tuned and learned within methodologies like agile development. However getting these next generation people into industry is a slow process.

It is essential to teach the existing people in the industry to learn.

Norm Day at the Office

Back at work, and running a orientation week before we begin a 3 month release plan, which aims at delivering the first possible release candidate for an embedded product family. We chose a meeting cabinet that we found out to be available.






Fist we checked it up. It seemed to be ok.









We could not find a projector in the premises, so we had to settle for Meeting Charts provided by our friends at 3M. Good news; no .PPT's.

Survey Shows: Agile Champions Live Higher

There has been survey results published at Agile 2006 conference. Pete Behrens has written a nice summary. The survey was about agile adaptation and was run by VersionOne. As VersionOne makes their business in agile scope, the data sample may be "a bit" biased. Even while I do not put much emphasis on the result that 75% of companies deploy agile methods I do see value in other finding:

"The internal agile champion is moving up in the organization to VP from team leader two years ago."

Agile methods have so far been largely deployed by a younger champion engineer in an organization that has gotten frustrated with the dysfunctioning methods. This data may show that agile development is moving directly higher from the trenches. I just hope that the values do not get twisted on their way up. The original ones work so nicely....

Agile Methods and Firmware Development

I made my paper "Agile Methods and Firmware Development" available. You can find a link to the paper from the sidebar, under "My Papers". This is my initial review and analysis of different software lifecycle models and their applicability to firmware development as part of Ph.D. studies. Paper is dated a few years back. One might say that the current journey started upon writing this paper.

Abstract— The size and complexity of software continues to grow
at a steady pace. This is also true for software embedded in our
everyday electronics, which we have called simple devices. The
term firmware is used to describe the low-level software in
embedded systems. It may even be hard to divide firmware and
actual hardware. Software development for such a target has
special characteristics such as a culture of hacking, small teams
and multiple hats, co-design issues, one-time designs, correctness
and robustness requirements, lack of tools and unconventional
customers. Software process models have been studied also in
this environment to ease the pain of developing more complex
systems. I introduce four currently used methods to develop
firmware; build-and-fix, waterfall, ROPES and RUP SE.

Agile methods are getting a lot of attention in the software
development community at the moment. I review the agile
methods which are most documented. The suitability of these to
firmware development is evaluated. It is also analyzed whether
firmware development could benefit from agile methods.

It is shown that agile methods are not the new cure-all solution
to firmware development, but they are applicable. Their full use
needs modification and innovative thinking. It is, however, shown
that firmware development can surely benefit from the usage of
agile methods.

embUnit Translators for CruiseControl

Back home and time to get tech oriented again. Our CruiseControl and unit test practices are proceeding at steady pace, so I thought I will make our XSL translators for embUnit unit test framework public in this forum at this point. You can find them here (unittests.xsl and testdetails.xsl). If you are an embedded programmer like me, and do not currently know much about CruiseControl, and/or XML, I'll try to briefly explain how to set it up. If there is enough interest I will make a more detailed intro to it.

You need at least to have following installed:

1. embUnit unit test framework and compiled libraries
2. Java SDK. Remember to set JAVA_HOME environment variable to point to install dir.
3. CruiseControl, of course
4. Make tools. I recommend for Windows users the whole Cygwin installation to cut down the hassle

Now you can run the Cruisecontrol Ant built Java and JUnit sample, but before you can start cruising with your embedded C project with Make and embUnit framework you need to at least do following modifications:

1. Build a simple C project which has makefile for automated build. Greate a CVS project and check your project out to your build server for CruiseControl (use Subversion if you are ahead of us, but you are on your own with this). Look at Driving on CruiseControl articles by Lasse Koskela for help.
2. Modify our config.xml to suite your project (see my earlier post about it) and replace the file in CruiseControl root
3. Replace unittests.xsl and testdetails.xsl files in CruiseControl folder ./webapps/cruisecontrol/xsl.
4. Launch CruiseControl and start cruising...

And here are some screenshots on how it looks.

Here a test fails (purely for demonstration purposes of course). Details of failed test are displayed below.








Details of all tests are also available.









After a brief moment everything is in order again. I used the green bar to signal this.

Battery Charging and Reflection

Yesterday evening we arrived to Lappeenranta and lake Saimaa. I have to say it is a beautiful scenario for battery charging. Even while the city was offering a festival (in finnish), I decided to take a day off from beer drinking and - well - read about change.

My book of choise, Fearless Change, by Ph.D's Linda Rising and Mary Lynn Manns, is of great value for anyone trying to get innovation diffused in organization. The book describes 48 patterns for driving and sustaining change. Many of the patterns we have found out during our journey. Among them a pattern "Time for Reflection" in which doctors recommend:

To learn from the past, take time at regular intervals to evaluate what is working well and what should be done differently.

This is excatly what we need to do again after the vacation.

Small Countries Seem to be Aggressively Agile

Google trends has been used to map agile development interest in different countries. Jonathan Cogley started it with a search for 'How Agile is Your Country'. Dave Nicolette went on with revisited search term set, and got this google trend. By choosing the regions you find (at least I did at the time) Finland in 10th place. So it is no wonder agile conferences in Finland are packed.

Prisoners of Our Habits - Act Four

Prisoners of Our Habits. Act three. Executive manager enters.

Executive manager is reading an email from project manager
"Dear Sir, I overheard this developer, John Smith, saying
his technical solution may cause $1.00 increase
in bill-of-material."

EXEC'MNG: -"Stop the press. This John Smith starts
immediately to investigate alternative solutions,
what ever they are."

Product under development may have hundreds of relationships. Tradeoffs between these relationships are being made in hourly basis. When one of these issues gets taken away from the context, it may seem like a big deal. The big deal in fact is when a bad decision (based on bad information) like one above is made the project environment, a complex adaptive system (CAS), gets an unexpected input. What happens in CAS is totally unpredictable. Typically the system is pushed into chaos and disorder. This is why agile development blocks these inputs from the team for a certain time, and opens the window for these new inputs only frequently enough. Command and control is traditional way of managing projects. Developers have not been encouraged to self-organize traditionally.

And old habits are slow to die.

I know paradigm shift is extremely slow, and so does James Shore (see his Change Diary). I'll be on vacation for couple of weeks and after that it would be perfect time to reflect and adjust the course of this agile journey. I just hope couple of weeks is enough for the old batteries.

Prisoners of Our Habits - Act Three

Prisoners of Our Habits. Act three. A manager enters.

DEV -"We should try to learn new ways of doing thingies 
      around here. We could participate in action research
      program, you know?"

MNG -"Hmm, That sounds like a lot of money. We could buy 
      a lot of chairs in two-day course for that. Wouldn't 
      that be great?"

...

There is a huge difference between being teached and learning. To start with there is a difference of acquiring existing knowledge versus creating new knowledge. Further there is a huge gap between action learning and a static content course which may have zero contextual relation with developer's current situation. I have seen the affect of these courses, and it does not impress me. It is just like many other mental models in this industry; a pretty model without connection to reality. But hey, traditionally developers have not taken part in action learning.

And old habits are slow to die.

Prisoners of Our Habits - Act Two

Prisoners of Our Habits. Act two. Project manager enters.

PM:   -"You need to focus on project's
        paper deliverables for the next two months"

DEV1: -"Which papers you mean? For whom they are,
        and for what purpose?"

PM:   -"I do not know, but reviewers know all the
        buzzwords, so my ass needs to be covered."

DEV1: -"Eh, That does not sound so valuable and motivating?"

PM:   -"It's not supposed to be. I fill in dashboards
        which no one reads. You should have no problem
        creating papers with no value."

...

The above is not a Dilbert strip. It is status quo in many development projects. Dashboards illustrating Gannts, PERTTs etc. are what we traditionally use for project management, and paper documents are what we traditionally use for progress monitoring and project quality assurance. So what the hell is wrong with that you ask? Again, it is a nice model, I really do like it. It has only one flaw; it does not hold in reality.

Then again working prototype, incremental release plan, Sprint backlog, or burndown chart are not used traditionally.

And old habits are slow to die.

Prisoners of Our Habits - Act One

I have been describing continuous integration for an embedded firmware project targeting a low-end microcontoller. Based on discussions I see a need for pointing out something about my view;

I do not mean that you need to apply continuous integration, unit tests running on both the host and target, etc. if you are writing a 200 LOC blinking LED application for 8-pin microcontroller for fun. This is a job for Solo-Virtuoso, and methods should be alike. The project I'm talking about, while targeting the low-end microcontrollers, will develop tens of thousands of lines of C source and will eventually be compiled into ten or so configurations for mass production. The configurations have some special areas where expertise is needed, like power management and wireless RF technology. This type of project cannot fit the Solo-Virtuoso category. Instead it requires a collaborative group of people - a job for an agile team and this team should adapt these practices.

Purely based on my biased participant observation, the trend of firmware projects is towards bigger and more complex so we are going to need these practices. The problem in firmware domain is that former electrical engineers (now programmers, or even worse, managers) are not willing to see this increase in complexity, but remain prisoners of their old habits. They are still in denial. My recent experiences manifest this very thing.

Prisoners of Our Habits. Act one. Programmer enters.

A colleague following our agile journey mentioned that based on his observations of daily Scrum meetings and collaboration we continuously run into problems caused by someone changing an interface and all of a sudden none of the configurations work. His obvious solution for this was; "there is a need for up-front designed interface contracts between modules". Problem solved, eh? We do not have full set of requirements, nor the freezed hardware architecture. For me iron-bound interface contracts seem pretty far fetched objective, but then again, it's just me. Still I would be amazed (but happily so) to find this wizard pulling out a comprehensive up-front architecture for us. Oh, and preferably delivered as a PowerPoint presentation I might add. If you happen to know this guy, ask him to drop me a line. I could use a good laugh.

I see these discussions about interfaces as a solution, not the symptom. It is good to see team members having courage to change things, to refactor when opportunity arises. These conflicts usually get solved easily after the daily Scrum at latest, so to me these are just short architecture meetings. In addition there is a huge difference compared to normal high ceremony architecture meeting: decisions get made and implemented immediately and feedback is available the next day. I have witnessed this working extremely well, and fast, even with programmers with junior skill level. This is just not the way it has been done traditionally.

Old habits die slowly.

Agile Business Conference - Program Draft Available

There is a program draft available for this year's Agile Business Conference in London. I have attended this conference for two years and I can recommend it. It is organized by DSDM Consortium and the title has word 'business' in it, but the agenda is about agile in general. This years highlight for me seems to be 'Agile and Embedded Systems' (Pekka Abrahamsson) . I'm also very interested in presentation by Mike Criffiths - 'Utilising Agile Methods Alongside the PMBOK Guide'. I see very little value in this balancing, but this is something we currently often need to live with and nevertheless we should always remember and reflect where we are coming from. This helps us to understand where we would like to go next (post-agilism).

Again interesting to see keynotes from Kent Beck, Sean Hanly, David Taylor and Polyanna Pixton. I really hope I'll be able to make it! If you are planning to be there, and are interested in agile [product] development, drop me a line and we will catch up.