Performance Issues in Incremental Warehouse Maintenance

Abstract

A well-known challenge in data warehousing is the efficient incremental maintenance of warehouse data in the presence of source data updates. In this paper, we identify several critical data representation and algorithmic choices that must be made when developing the machinery of an incrementally maintained data warehouse. For each decision area, we identify various alternatives and evaluate them through extensive experiments. We show that picking the right alternative can lead to dramatic performance gains, and we propose guidelines for making the right decisions under different scenarios. All of the issues addressed in this paper arose in our development of WHIPS, a prototype data warehousing system supporting incremental maintenance.