Define Data Warehousing

Data Warehousing

The data warehousing market consists of tools, technologies, and methodologies that allow for the construction, usage, management, and maintenance of the hardware and software used for a data warehouse, as well as the actual data itself. The term Data Warehouse was coined by Bill Inmon in 1990, which he defined in the following way “A warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management’s decision making process”.

Data warehousing is combining data from multiple and usually varied sources into one comprehensive and easily manipulated database. Common accessing systems of data warehousing include queries, analysis and reporting. Because data warehousing creates one database in the end, the number of sources can be anything you want it to be, provided that the system can handle the volume, of course. The final result, however, is homogeneous data, which can be more easily manipulated.

Data warehousing is commonly used by companies to analyze trends over time. In other words, companies may very well use data warehousing to view day-to- day operations, but its primary function is facilitating strategic planning resulting from long-term data overviews. From such overviews, business models, forecasts, and other reports and projections can be made. Routinely, because the data stored in data warehouses is intended to provide more overview-like reporting, the data is read-only. If you want to update the data stored via data warehousing, you’ll need to build a new query when you’re done.

This is not to say that data warehousing involves data that is never updated. On the contrary, the data stored in data warehouses is updated all the time. It’s the reporting and the analysis that take more of a long-term view.

Data warehousing is not the be-all and end-all for storing all of a company’s data. Rather, data warehousing is used to house the necessary data for specific analysis. More comprehensive data storage requires different capacities that are more static and less easily manipulated than those used for data warehousing. Data warehousing is typically used by larger companies analyzing larger sets of data for enterprise purposes. Smaller companies wishing to analyze just one subject, for example, usually access data marts, which are much more specific and targeted in their storage and reporting. Data warehousing often includes smaller amounts of data grouped into data marts. In this way, a larger company might have at its disposal both data warehousing and data marts, allowing users to choose the source and functionality depending on current needs.

Data Warehousing Project Cycle

After the tools and team personnel selections are made, the data warehouse project can begin. The following are the typical processes involved in the data warehousing project cycle.

Requirement Gathering

Physical Environment Setup

Data Modeling

ETL

OLAP Cube Design

Front End Development

Report Development

Performance Tuning

Query Optimization

Quality Assurance

Rolling out to Production

Production Maintenance

Incremental Enhancements

Each item listed below represents a typical data warehouse phase, and has several sections

Task Description This section describes what typically needs to be accomplished during this particular data warehouse phase.

Time Requirement A rough estimate of the amount of time this particular data warehouse task takes.

Deliverables Typically at the end of each data warehouse task, one or more documents are produced that fully describe the steps and results of that particular task. This is especially important for consultants to communicate their results to the clients.

Possible Pitfalls Things to watch out for. Some of them obvious, some of them not so obvious. However, all of them are real.