Why Build a Data Warehouse When You Can Lease One?

Back in 2008, I was working for a telco in the Dominican Republic, consulting for their data warehouse team. After a review of their data warehouse and several discussion with stakeholders, I proposed a complete re-architecture of the data warehouse and the reporting platforms that the data warehouse supported. When I had to cost out the project, I included some hardware, primarily for the Ab Initio ETL tool we were going to migrate to, but also for the development and test servers, and more tape backup machines. All in all, the tab for the entire project grew to almost 2 million USD. It was just another reminder to me of how expensive a data warehouse can be. In trying to convince the management board of the project, I conducted interviews with the major stakeholders to try and gauge the value of the data warehouse to the organization. Of course, a data warehouse supporting business intelligence was of enough value to the organization to justify it, but the numbers can be hard to swallow never the less.

All of that reminded me of the Data Warehouse Maturity Model I have seen time and again at conferences I have attended and presented at. It shows that a data warehouse has reached its ultimate maturity when it achieves its potential for the return on the investment of building a data warehouse. That is, the business uses the data warehouse and the information within it to drive the business. I started to think about this while I was putting the business case together for this project. Too often, I have found that the decision makers in an organization are not always aware of the value of the data warehouse, and see it as a big, expensive, slow moving giant that they have to live with rather than really benefit from. It is beyond the scope of this article to discuss all the reasons why this might become the perception of the data warehouse, but it was sure the case here, which is why I needed to include in my business case the feedback from all the users of the data warehouse.

The organization in question was looking for ways to reduce the costs of the data warehouse and it occurred to me that one way to reduce the cost of it was to remove it all together from the business. I don’t mean throw it away (that would be insane), but why not get a little creative? Take all the hardware, the software, and the people that build, operate and maintain it all, and use it to spin off a separate business? This business would be initially funded by the parent organization, contracted by the parent to provide all the data warehouse services it did before but at a more favourable cost. It would operate as an independent business with the sole core competency of doing data warehousing, and be allowed to also sell its services to other organizations as well. By being wholly owned by the parent company and being able to expand its customer base beyond the parent, allows it to become a revenue centre for the parent. Eventually the separation would probably need to be more complete, I suppose, but still, it goes from being a cost centre to a revenue centre. Sweet!

Afterwards, I thought, hey, I’ve been doing software design and development for over 20 years, data warehousing for almost 10, and ran my own small consulting firm. Why don’t I just start up a business that provides a data warehouse as a service? Why, I thought to myself, that’s a great idea and would be a lot of fun! I know a couple of organizations that are moving in the direction of having other “systems as a service” anyway (at least outsourcing other major business support systems), so I could probable lock up a customer or two within a couple of years, and who knows? I started to seriously investigate the possibility. I would need to build/buy/rent a data center and build/rent/lease a bunch of servers and disk, not really a problem, but I would (ideally) need to come up with a data architecture that could support multiple clients, possibly across different industries. That would mean, design it, build it, test it, implement it, and then go sell it. Now that’s a lot of work. And I know that you can buy such industry specific data models. Teradata did, or maybe still does, have one, Oracle has one to, and I am sure there are others. Possible still, but a lot of work, and it would mean I might miss the opportunities I had because of the timing.

In continuing my investigation into setting all this up, I came across a company that already provides Data Warehousing as a Service (DaaS), called Kognitio. They have their own data centres, they have their own data models, and although they are the only ones I could find, but there might be others by now. So, someone had my genius idea before me (no surprise really). I still think that for an organization that is trying become more mature in their data warehousing, this is a good solution to get off the ground fast and jump the “gulf between ‘Inform Exectutives’ and ‘Empower Knowledge Workers’” in the afore-mentioned data warehouse maturity model. If you are looking to implement a new data warehouse, or re-architect one, or try to out-source you existing one, I suggest you give them a call. I think I might, just to see if they need an affiliate in Switzerland, where I live.

What do you thing of Data Warehouse as a Service? Good idea or bad? Easy or difficult to implement? Too big a change to your organization or not? Let us know your thoughts in a comment!