BI Redux

Upto couple of years back larger size of BI system gave a BI team bragging rights. And the threshold to join the club kept getting bigger (you have a size of 6 only? we are approaching 11 (TeraBytes ofcourse)).

Well, it may still be so. But the novelty is giving way to weary realization that there are no easy tools to get the genie back in the bottle.

The DB size may be adding a few TBs every year, multiplying costs through increased hardware and resources to manage it.

Organizations have looked at solutions like BIA (and overlooked the budget increases for hardware and related services) to continue to derive incremental value out of their BI systems. Questions not asked often enough are "is all that disksize containing real meaningful data", "is this data what users are using, or, going to use in future", "are there holes in my dataset - placeholders that are never used but eating up diskspace", "are there orphan fields in my schema" and so on.

It is good to explore and find the best combination of optimizing techniques such as aggregates however it may not be enough anymore. With many organizations reaching a critical DB size after which the incremental cost with increasing size of DB will rise significantly due to the complexity involved in normal maintenance tasks (eg backup, system copy, DB reorg etc) it is about time to look at what more a BI team shall be exploring.

My opinion is, now that the BI teams are confident and comfortable managing their BI system, they need to proactively identify and implement methods that reduce the growth of DB (or in best case, reduce the DB size followed by a minimal growth rate). This would involve analyzing existing data models, remodelling, archiving, discarding (yes, this is what it ultimately will need to come to, archiving offline is what it will mean in real terms) data and so on. Sooner organizations get to this mindset (of managing the DB size) in addition to applying other emerging techniques and solutions in SAP BI, better it will be long term.

It would be a welcome change to see bragging rights (and bonuses) linked to SAP BI systems growth (ie lack thereof)