Isn’t it time to restrict the obedience to the business side deciding about what is being developed? Shouldn’t IT considerations at least be as seriously considered when it comes to how to deliver the right systems and quickly? Agile methods are particularly appropriate for BI systems which are close to the end users. That users often cannot precisely formulate their needs initially, is one of the strongest arguments for an incremental and iterative process. But there are aspects specific to BI favouring prototyping. Encouraging active user involvement throughout the development is certainly of high value for both sides. Delivering the parts of highest priority first and using that output can give very useful input for subsequent phases including the option of stopping the project, if first results made the project obsolete or changed the requirements drastically. Among the strongest critisms BI teams face are being slow and not delivering very useful results. Both issues have been strong motivations for agile approaches since the beginning.

As Wikipedia puts it, agile development is “based on iterative development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams”. It is in particular “a business approach that aligns development with customer needs and company goals.”

We might need to reinterpret some of the principles of the agile manifesto. The principle working software is delivered frequently might sometimes be read results are delivered frequently. It is the close collaboration between the BI team and the users supported by frequent output to the users keeping them up to date that matters here. A principle that seems very hard to swallow and contrary to most IT development culture says that even late changes in requirements are welcomed. Ok, maybe not all changes are appropriate and need to be expressed so late. But this openess and flexibility is key to agile development. It means accepting that it is hard to impossible for users to sign and close BI requirements before having seen the data.

The fact that relatively few organisations apply agile processes probably does not mean that people are all happy with the waterfall process. Many large organizations have difficulty bridging the gap between the traditional waterfall method and an agile one. It would also require getting around reasons predominantely external to BI such as overall budgeting, planning and roadmap management and prioritisation.

Wikipedia describes the agile home ground as projects of low criticality with a small team of senior developers and requirements that change very often. A consultative management culture would help.

Don’t get me wrong. Major infrastructure projects, be it enterprise data warehouse or something operational, are not good initial candidates for process innovation. Where the big picture is important to be kept in mind, where the long term maintenance and flexibility has to be assured and users interact with the deliverable only indirectly, a classical planned approach still has its virtue. But when it comes to small analytical marts and BI interfaces we should take a good dose of self-confidence and go agile even it takes a few serious discussions.