Maintenance of Data Cubes and Summary Tables in a Warehouse

Abstract

Data warehouses contain large amounts of information, often collected from a variety of independent sources. Decision-support functions in a warehouse, such as on-line analytical processing involve hundreds of complex aggregate queries over large volumes of data. It is not feasible to compute these queries by scanning the data sets each time. Warehouse applications therefore build a number of summary tables, or materialized aggregate views, to help them increase the system performance. As changes, most notably new transactional data, are collected at the data sources, all summary tables at the warehouse that depend upon this data need to be updated. Usually, source changes are loaded into the warehouse at regular intervals, usually once a day, in a batch window, and the warehouse is made unavailable for querying while it is updated. Since the number of summary tables that need to be maintained is often large, a critical issue for data warehousing is how to maintain the summary tables effciently. In this paper we propose a method of maintaining aggregate views (the summary-delta table methoand use it to solve two problems in maintaining summary tables in a warehouse: (1) how to effciently maintain a summary table while minimizing the batch window needed for maintenance, and (2) how to maintain a large set of summary tables over the same base tables. We show that much of the work required for maintaining one summary table by the summary-delta method can be re-used in maintaining other summary tables, so that a set of summary tables can be maintained effciently. While several papers have addressed the issues relating to choosing and materializing a set of summary tables, this is the first paper to address maintaining summary tables effciently