Data Warehouse Backlash?

Smaller solutions appear

By Jeffrey Bartlett

For nearly two years now, vendors of all types--server systems, RDBMS and tools--have been stampeding user organizations toward data warehousing. Their main cattle prod has been the electric shock of the buzzwords competitive advantage. Customer service and decision support are the most widely touted applications for achieving this goal. To judge from the recent DB Expo, held in April in San Francisco, the herd is in full stampede.

One of four conference divisions was dedicated to data warehousing and parallel computing, and it included as many sessions as the omnibus database and client/server division. Clearly, the expo organizers and the vendors and consultants who led the conference saw a need to jump on the subject with both feet. But the rush to sell these solutions may be running into its first wall. Customers know they have a problem with data access, but they're not sure building a warehouse is the answer.

The data warehouse generally is understood as an enterprise-wide system; by definition it almost has to be. As a new, large-scale construction, it is expensive, complex to understand and complicated to implement and maintain. To do it right requires outlays of time and money that not every company can afford, no matter how worried they are about competitive advantage. The idea of starting such a project at the top of the IT infrastructure could seem to be just too much.

Enter the Data Mart

Vendors show signs of realizing this and, ever resourceful, are proposing warehouse-style solutions on a smaller scale. The concept of the data mart, a department-size data repository tailored for specific users and their applications, was on everyone's lips. There were a couple of obvious reasons for this. A data mart can be built faster and more cheaply than a warehouse, and it does not require revamping the entire architecture. Instead, it can serve as the proof of a concept that lets a supplier get a foot in the door.

This tactic reveals another vendor strategy. Apparently everyone--from hardware and RDBMS giants to small consulting shops--involved in the current data access frenzy has implementation teams who can take the customer through the whole process, from defining needs to building a pilot to managing a warehouse after it is built. Consulting is a growing source of revenue for vendors in general, and in this arena, where knowledge of fast-changing technology is scarce, they see a major opportunity.

But the data mart itself raises crucial questions, particularly in the area of integration. Following the metaphor, how do you tie all the local data marts into the corporate distribution center? Ultimately, the central data warehouse has to populate the data marts, but marts often spring up in departments independently of the overall corporate structure. Potentially, user organizations and their IS departments are looking at a reprise of the rivalry between the data center and departmental LANs--a situation that many enterprises are still struggling to overcome and are not anxious to see happen again.

Back on the Big End

Despite the interest in data access on a smaller scale, conference sessions on the very large database (VLDB) and parallel computing, in which I was interested, seemed to draw well. There's no doubt that data keeps growing like mushrooms in the forest, and the IS pros who attended DB Expo are seriously engaged in trying to deal with it.

One consultant leading a conference session tried to pinpoint the state of the art at the high end. Nagraj Alur of Data Base Associates of Morgan Hill, CA, posed the rhetorical question, "Are corporations really exploiting parallel processing technologies?" In response he answered, "Yes, they expect to."

This equivocation may not be what anybody wants to hear, but it probably is accurate. Those on the supply side of the equation can run out all the new strategies they want, but from the implementation side we're not there yet.