Datawarehouse Automation

Yesterday I attended the DWH Automation conference in Leuven (Belgium), hosted by BI-Community.

Organizers

It was organized by Ronald Damhof and Tom Breur.

Sponsors

Among the sponsors were Centennium, WhereScape, Qosqo, BIReady, Genesee Academy, timeXtender and TripWire Solutions.1

General presentations

The presentations given by [RonaldDamhof] and [TomBreur] were largely the same as last year at the Data Vault Automation conference in Utrecht (The Netherlands). They both focus on Agile BI and the importance of DWH automation in Agile BI. For that matter, the use of Data Vault modeling for the Enterprise Data Warehouse (EDW) seems the only methodology that truly supports Agile BI and is the one that is the most easy to automate due to its patterns of hubs, links and satellites.

Hans Hultgren from Genesee Academy gave a very interesting presentation about the meaning of Data Warehousing. Nowadays there are a lot of different terms in Data Warehousing and some of these have different meanings depending on who you’re asking to define the term. He focused on the importance to talk about the meaning of the term, instead of the term itself. Depending on the meaning, several layers can be defined in a Data Warehouse solution, each of which has a specific purpose that can be (partially) automated or not.

Frederik Naessens from K25 gave a small presentation on how to use the ERWin data modeling tool to generate the various models, such as a Data Vault model for the EDW. It’s a poor man’s solution focused on being able to create awareness of the need of DWH automation tools.

Product presentations

The following companies presented their DWH Automation solution with “SlideWare”2:

  • TripWire Solutions
  • DWhite
  • BIReady

While the other companies gave a live demo of their products:

  • WhereScape
  • Qosqo

TripWire Solutions

Dirk Vermeiren from TripWire Solutions focuses on Oracle and presented their accelerators. They gave a complete overview of the layers they implement in a Data Warehouse solution and which of those layers can be automated. Data Vault modeling is used for the layer that contains the EDW.

My 2 cents: looks promising for a specific market (Oracle).

DWhite

DWhite presented a solution that is not yet commercialized, because it is still in the works, but already used at a particular client. It is focused on Microsoft BI at the moment, but should support “everything” independent of the modeling used.

My 2 cents: I got the impression it is a one-man show and the goal is set pretty high, so I don’t think this will make it.

BIReady

Gertjan Vlug from BIReady, as the last presenter of the day, decided to give some kind of wrap-up of the day and picked in where their product fits in. They are one of the pioneers and their product focuses on using a business model from which the rest can be automated, instead of using source models. BIReady can handle any type of modeling and also uses (but not necessarily) Data Vault.

My 2 cents: I want to see a demo, looks very promising and I really like the fact that it starts with a business model instead of a source model.

WhereScape

Robert gave a stunning Star Wars introduction that made sure that he got everyone’s attention. It was funny but still hit the spot. After that Terry took over and gave a small demo of WhereScape 3D and WhereScape RED. They had been demoing already at their booth, so it was kept short.

WhereScape RED is a stunning product. It really seems to do it all. It also takes care of the ETL itself, scheduling etc.

My 2 cents: WhereScape really knows its business and has a great product.

Qosqo

Jeroen Klep from Qosqo gave a demo of their Quipu product. It is open source and can be adapted to your needs by changing the templates. It is not meant to replace it all, but to be complementary to investments already made. Quipu is still young, but also looks very promising. It can automate design of staging, EDW and data marts. The EDW is based on Data Vault.

My 2 cents: Quipu has to be taken seriously and could become a true competitor for the other players such as WhereScape and BIReady.

Conclusion

While Agile BI is not only “hot”, but also necessary in a changing world, the need for being able to automate large parts of it is inevitable. There are some great players in the market that can help you with that.


  1. Don’t shoot me if I forgot one of the sponsors 

  2. They did not give a live demo, but only presented the solution using slides.