Early last year, soon after their company was acquired by consumer products giant Nestle S.A., officials at Dreyer's Grand Ice Cream decided they needed a better way to track and improve the ice cream manufacturer's performance. Like most companies, 79-year-old Dreyer's for years had generated monthly reports that were reviewed by a handful of analysts and top managers seeking insight into the company's performance. Those reports, however, often arrived too late to allow operational managers on the company's front lines to do much to head off problems, such as supply chain inefficiencies resulting in stock-outs at the retail level.
"We were getting reports on our historical progress, not on what was going on right now," says Ajay Raikar, director of growth and profitability at $2 billion Dreyer's. "We didn't have information that was close to real time, so we weren't close to the pulse of the business, and we couldn't influence what our results would be for the current month."
So Dreyer's officials decided to change not just how the company generates reports, but also how its managers run the business by putting in place what Raikar calls a "KPI culture." All managers at Dreyer's, including those running warehouses and plants, were given a handful of key performance indicators — between five and 10 — that they would be responsible for tracking and managing on a daily basis. Warehouse managers, for example, might track perfect order, labor productivity, and safety KPIs.
In support of that initiative, Dreyer's deployed at each of its six facilities a real-time business intelligence (BI) appliance from Cognos Corp., CognosNow, that downloads transactional data from Dreyer's key business systems into an in-memory database that is used to generate up-to-the minute reports on the status of the company's operations. Managers can drill down into the numbers to understand the causes of trends or problems.
Now, Raikar says, managers up and down the Dreyer's organization get daily reports on what went right and what went wrong the previous day, and insights into what they can do to correct problems. "We can now influence the process closer to the time we see a deviation," Raikar says. "We get better decision-making, from our company's leadership down to the manager who runs the warehouse."
Manufacturers have access to lots of data that could, in theory, be used to track and tune business processes on the fly. Between ERP, plant floor, supply chain, and CRM systems, most manufacturers are generating enough data to choke a horse.
The problem has been getting the right data into the hands of front-line operational managers quickly enough to help them fix or even head off inventory build-ups or out-of-stock blips before they bloom into serious problems. Traditional BI tools, such as canned reports or data warehouses, typically weren't designed for that purpose. Increasingly, however, manufacturers are turning to a rapidly emerging set of technologies that promise to deliver what some call operational business intelligence — BI that is real-time, easily accessed by front-line operational managers, and tightly integrated with day-to-day business processes. These operational BI tools offer manufacturers a way to react more quickly and effectively to an accelerating rate of business change and its effects on such critical issues as inventories, quality, and order fulfillment, experts say.
"In order to react to changes in the business brought on by things like globalization and intense competition, manufacturers need more visibility into the operational aspects of their organizations, the shop floor, the supply chain, etc.," says John Hagarty, an analyst at AMR Research. "For that reason, the notion of operational BI is resonating in a big way right now."
Most manufacturers are only too familiar with the forces generating the need for improved operational agility. Shorter and shorter product life cycles, particularly in vertical markets such as high-tech electronics, mean that manufacturers can no longer afford to wait until the end of the month, or even the end of the week, to understand whether production yields are meeting expectations. Similarly, as supply networks become more complex and global, manufacturers need to understand quickly how a problem involving one supplier or part affects specific customers and what to do about it. At the same time, to avoid running afoul of government regulations or increasingly common customer service-level agreements (SLAs), manufacturers need to recognize and fix operational problems as soon as they crop up.
Unfortunately, BI systems, as traditionally designed and deployed, haven't been very good at providing real-time insights into operational performance. For one thing, BI technology has not typically operated in an interactive or real-time fashion, but rather in batch mode. In a traditional BI environment, data from multiple transactional systems, such as ERP, is usually downloaded to a data warehouse or data mart on a pre-defined schedule after going through an extract/transform/load (ETL) process. At most manufacturing companies, the process takes place weekly or monthly.
This means that, though rich in historical data and context, most BI environments based on data warehouses are not good at providing users with insight into an event or series of events that have just happened. At the same time, because data warehouses must store lots of historical information, they usually aggregate data, sometimes making it difficult for users to extract operational, detailed information when they need it.
On top of that, most BI systems have not been designed for use by managers and others involved in operational roles. Instead, most are intended for a handful of top managers and business analysts making long-term, strategic decisions, such as whether to make an acquisition or enter a new business. This means that, from a data presentation point of view, most BI systems aren't integrated with the ERP, warehouse management, MES, and other transactional systems that operational managers use all day, every day. It also means that they often don't present data in a quick, easy-to-consume format, such as a dashboard, that most operational managers require.
In-Memory Momentum
Providing real-time BI to operational users is certainly not a new idea. Banks and credit card companies for years have combined rules engines with real-time transaction data to spot and even prevent fraud as it occurs. But those systems tend to be proprietary and expensive.
Most manufacturers also are able to get limited real-time information from daily reports generated by ERP and other transactional systems. Such reports, however, are typically static and don't lend themselves to ad hoc analysis by operational users.
Now, however, vendors are developing a wide range of off-the-shelf tools that can be used to create operational BI systems that are both powerful and flexible. Some come from traditional BI providers and are intended to make data warehouses and other BI infrastructure more real-time and easier for operational managers to use. Others come from vendors of composite applications and business process management tools and are focused on monitoring business processes, alerting operational managers when something goes wrong, and even automating responses that can address a problem or exception.
Enterprise applications giant SAP AG is one vendor attempting to make traditional data warehouses more real-time. The company last year began shipping its NetWeaver BI Accelerator, an add-on appliance that works with the NetWeaver BI data warehouse environment. The BI Accelerator slashes the time it takes NetWeaver BI to respond to queries by loading data from the SAP data warehouse directly into the memory of a hardware appliance, where it can be queried on the fly. Bypassing the need for data experts to build traditional query-optimized data structures, the BI Accelerator lets end users, such as operations managers, form their own queries that can be executed rapidly. According to SAP NetWeaver Marketing Director Lothar Schubert, 50 organizations have deployed the BI Accelerator in the past year.
Seventy percent of large enterprises will deploy such in-memory query-optimization products by 2012, according to a recent report by Gartner.
BI software vendor Information Builders is attacking the same query optimization problem in a different way. The company recently updated its WebFOCUS BI environment to include a feature called Active Reports, which automates the delivery of real-time, interactive reports to users. Not only do the reports tap into more real-time data, but also operational users can slice and dice the data any way they need to.
The Hillman Group, a manufacturer and distributor of fasteners, keys, and other hardware products, has begun using the WebFOCUS technology to deliver real-time reports to supply and logistics network managers. The reports tell them how well the company is doing at filling customer orders. When users launch a request for fill-rate information, the system retrieves data from several transactional systems and assembles it on the fly, says Jim Honerkamp, the company's CIO.
The real-time reports, Honerkamp says, are important to Hillman because the company has service-level agreements in place with many of its customers requiring 99% fill rates. Recognizing quickly when the company may be in danger of missing fill-rate targets can save Hillman big money. In fact, Honerkamp says, last year Hillman recouped $130,000 from one of its largest vendors because, with real-time information, it was able to prove it had satisfied its fill-rate promises when the customer believed Hillman hadn't.
"Operational BI has reduced the time between us identifying a business issue and identifying the solution," Honerkamp says. "In many cases, before we had the tool, it could be days to weeks before we spotted a problem with fill rates. Now, because we can put the tool in the hands of operational end users, the time frame is often minutes or hours."
Accelerating queries, however, doesn't do much to improve the way traditional BI environments extract data from transaction systems and load it into analytical data stores. And in many environments, that is the part of the process that is most batch-oriented and contributes the most latency, experts say.
A host of BI vendors are beginning to attack that piece of the puzzle as well, however. BI software vendor Teradata, for example, has added a real-time data loader to its data warehouse environment. Using publish-and-subscribe integration tools, such as IBM's MQ Series, the real-time data loader connects to transactional systems and downloads data from them in real time. The data is stored in temporary tables in the Teradata data warehouse where it can be used to generate real-time dashboards or analysis.
Freescale Semiconductor, a $6.4 billion semiconductor manufacturer, is using Teradata's real-time data loading capabilities in combination with an in-memory data analysis tool to quickly deliver insights to yield engineers tracking the efficiency of the company's manufacturing processes around the world. Every day, 50 gigabytes of data from semiconductor test and production machines flows continuously into the company's Teradata data warehouse. There, it can be accessed in near real time by engineers using DXP 2.0, an in-memory analytical tool from start-up vendor Spotfire Inc., which was recently acquired by TIBCO Software Inc. The Spotfire tool pulls the near real-time test data from the Teradata database, allowing engineers to analyze, visualize, and correct conditions that may be reducing Freescale's manufacturing yields.
Recently, for example, Freescale engineers were able, within hours, to isolate a particular semiconductor yield problem to one of three chambers in a wafer fabrication tool and fix it. That kind of quick response is critical for Freescale, which, like all technology manufacturers, contends with ever-shrinking product life spans.
"Before, we might never have found the problem. Or, if we did, it might have taken an engineer manually putting together the analysis over weeks, if not months," says Alan Yost, an IT manager in Freescale's computer integrated manufacturing group.
Similar in-memory analytical tools include CognosNow and TM1 from Applix Inc. Applix was recently acquired by Cognos.
Another emerging class of tools, dubbed enterprise information integration (EII), takes a slightly different approach to enabling the collection and analysis of real-time data from multiple sources. EII middleware tools sit between a wide variety of source systems, such as ERP or control systems, and end-user tools, such as analytical software. They combine connectors to source systems with metadata repositories that provide context for understanding data from different sources.
Rather than aggregating and moving large amounts of historical data to a data warehouse as traditional BI environments do, EII systems provide real-time access to data residing in transactional systems, as needed by end users. Since EII systems don't aggregate and replicate the data held in source systems, they typically deliver fresher, more up-to-date data.
One vendor of such EII tools is Composite Software Inc., a San Mateo, CA, startup that provides services-based data virtualization software that integrates with analytical tools from traditional BI vendors, such as Cognos, Business Objects, and Microstrategy. According to Peter Tran, vice president of product marketing at Composite, manufacturers are using the company's EII tools to track processes, such as order fulfillment and multi-party inventory management, in real time.
Other EII tool providers include Information Builders, with its recently acquired iWay product; Attunity Ltd.; and Business Objects.
Fresh Data, as You Need It
But, in order to meet the needs of managers on the operational front lines, operational BI needs to do more than just surface real-time data. It also needs to present that data in the right form and context so that operational managers can easily understand and act on it.
"When operational people need to ask a question, they need the answer immediately, and they need it in a form they can quickly digest," says Doug Lawson, CEO at Incuity Software Inc., a maker of real-time manufacturing intelligence software. "If they don't get that, they'll quickly move on to the next problem."
Incuity, for example, presents manufacturing intelligence information in formats with which engineers and others on the front lines are familiar, such as x-y plots and time series trend charts.
And ERP vendors such as Microsoft and SAP are going further, integrating BI into portals and role-based user interfaces where operational users live. Microsoft, for example, has begun to upgrade its business application suites, such as Dynamics AX and GP, with portal-based user interfaces that are tailored for specific roles in the enterprise and feature contextual BI. Among the 50 roles that Microsoft has so far identified are operational titles, such as warehouse manager and production manager.
At the same time, Microsoft is integrating BI, business monitoring/alerting, and visualization capabilities directly into its business applications. The company's recently announced PerformancePoint Server 2007, which supports analytics, visualization, and business monitoring and alerting, is now being integrated with Dynamics AX.
"It used to be that ERP was one product and BI was another, and users would have to copy and paste between them. Now it's being integrated so that an order scheduler might be looking at a number of activities from the ERP application alongside KPIs that are relevant on his home page," says Mogens Elsberg, general manager of dynamics ERP program management at Microsoft.
And enterprise applications giant SAP is moving in the same direction. In October, the company announced plans to acquire BI software vendor Business Objects SA for $6.78 billion. The deal will expand SAP's customer base and lay the groundwork for integrating real-time BI directly into the SAP applications using tools such as Business Objects' Data Federator, an EII-style data virtualization tool.
At a press conference announcing the acquisition, SAP CEO Henning Kagermann said the deal will allow SAP to "integrate the transaction space with the analytical space. We can now deliver business processes that are enriched by analytical applications and, in the end, bring more closed-loop process decision support to the market."
Everybody Wants to Get into the Act
But vendors of BI software aren't the only ones beginning to address manufacturers' need for more real-time visibility into operations. Providers of enterprise integration and business process management tools are attacking the same issue by giving manufacturers up-to-the-minute visibility into whether specific business processes are operating at, above, or below expectations.
Enterprise application and BPM vendors such as TIBCO, IDS Scheer, and Software AG's webMethods unit have long provided tools that let manufacturers model, test, deploy, and operate automated business processes. These tools might be used, for example, to create workflows that link ERP-based order management with processes supported by a CRM system.
Now these vendors are adding to those tools business activity monitoring (BAM) engines that let users set up KPIs for those processes and automatically monitor whether they are meeting those benchmarks. WebMethods, for example, offers BAM engines that monitor SAP-related processes, as well as processes that touch external suppliers. Similarly, IDS Scheer recently added BAM capabilities to its Aris BPM environment through a partnership with rules-engine software provider Systar.
In addition to generating alerts when a business process bogs down or a specified event occurs, those BAM/BPM tools can kick off a separate workflow or business process aimed at fixing whatever has gone wrong.
"It's one thing to look in the rearview mirror and see how processes have behaved in the past. It's quite another to have detailed visibility into processes on a minute-by-minute basis," says Matt Green, product line director for process applications at webMethods. "BAM enables you to react to potential problems much faster than you could with a BI tool."
The encroachment of enterprise integration/BPM vendors into the operational BI space has begun to blur the lines between BI and BPM vendors and has stimulated a series of recent acquisitions and technology-sharing deals. TIBCO's acquisition of Spotfire is one example. Like IDS Scheer, BI vendor Teradata has partnered with BAM technology provider Systar in order to add an event-monitoring capability to its BI environment.
And application vendors also are beginning to show interest in the type of event monitoring and alerts that BAM technologies can provide. SAP, for example, recently released a new version of its NetWeaver platform that includes NetWeaver Process Integration, a replacement for NetWeaver's Exchange Infrastructure element. NetWeaver Process Integration includes a new event-processing feature. So, in addition to helping users integrate applications, the Process Integration product will help them monitor business events and resolve alerts, SAP officials say.
While the notion of operational BI is clearly on the rise among manufacturers and technology providers, operational BI should not be thought of as a replacement for traditional BI, experts say. The wide variety of technologies that can be used to enable operational BI will become increasingly important to manufacturers needing to monitor events as they occur and to respond quickly. But traditional BI will still be important for supporting more strategic business decisions.
"We see the two augmenting each other," says Paul Hoy, global manufacturing industry director at Cognos. "They answer fundamentally different types of questions. Operational BI lets you look at how you're doing right now. But it's also important to have insights into how you've been doing over time."