Original Post From Thomas H. Davenport
Like many new information technologies, Big Data can bring about dramatic cost reductions, substantial improvements in the time required to perform a computing task, or new product and service offerings. Which of these benefits are you seeking? The technologies and concepts behind Big Data allow organizations to achieve a variety of objectives, but you are unlikely to realize all the possible benefits. Deciding what your organization wants from Big Data is a critical decision that you need to address. The answer has implications for not only the outcome from Big Data, but also the process—who leads the initiative, where it fits within your organization, and how you manage the project.
If you are looking for cost reduction, you’re probably conscious of the fact that MIPS and terabyte storage for structured data are now most cheaply delivered through Big Data technologies like Hadoop clusters. One company’s cost comparison, for example, estimated that the cost of storing one terabyte for a year was $37,000 for a traditional relational database, $5,000 for a data appliance, and only $2,000 for a Hadoop cluster. Of course, these figures are not directly comparable, in that the more traditional technologies may be somewhat more reliable and easily managed.
If you’re focusing primarily on cost reduction, then the decision to adopt Big Data tools is relatively straightforward. It should be made primarily by the IT organization on largely technical and economic criteria. You may want to involve some of your users and sponsors in debating the data management advantages and disadvantages of this kind of storage, but that’s about it.
The second key benefit of Big Data tools is time reduction. Macy’s Inc.’s merchandise pricing optimization application provides a classic example of reducing the cycle time for complex and large-scale analytical calculations from hours or even days to minutes or seconds. The department store chain has been able to reduce the time to optimize pricing of its 73 million items for sale from over 27 hours to just over 1one hour. Described as “high performance analytics,” or HPA, by the software vendor SAS, HPA makes it possible for Macy’s to re-price items much more frequently to adapt to changing conditions in the retail marketplace. This HPA application doesn’t employ a Hadoop cluster, but it does take advantage of parallel computing and in-memory software architectures. Macy’s also says it achieved 70% hardware cost reductions.
If your company is primarily interested in time reduction, you need to work much more closely with the owner of the relevant business process. A key question is what you are going to do with all the time saved in the process. Good answers include:
We’re going to iterate and tune the model much more frequently to get a better solution;
We’re going to use many more variables and more data to compute a real-time offer for our customers;
We’re going to be able to respond much more rapidly to contingencies in our environment.
Bad answers (at least in strict business terms) include playing more golf, drinking more coffee, or finally having enough time for that three-martini lunch.
To my mind, the best thing an organization can do with Big Data is to develop new products and services. The best organization at this may be LinkedIn Corp. , which has used Big Data and data scientists to develop a broad array of product offerings and features, including People You May Know, Groups You May Like, Jobs You May Be Interested In, Who’s Viewed My Profile, and several others General Electric Co. is primarily focused on Big Data for improving services—among other things, to optimize the service contracts and maintenance intervals for industrial products Google Inc. , of course, uses Big Data to refine its core search and ad-serving algorithms Zynga Inc. uses Big Data to target games and game-related products to customers Netflix Inc. created the well-known Netflix prize for the data science team that could optimize the company’s movie recommendations for customers. The testing firm Kaplan uses Big Data to begin advising customers on effective learning and test-preparation strategies. These companies’ Big Data efforts are directly focused on products, services, and customers.
This has important implications, of course, for the organizational locus of Big Data and the processes and pace of new product development. Obviously you need to be working closely with the product development team, and perhaps marketing as well. These projects should probably be sponsored by a business leader, rather than a technician or data scientist. You may not save a lot of money or time, but you may well add some big numbers to your company’s top line.
I hope you now agree that deciding what you want to accomplish with Big Data is one of the first and most important things you should do with this resource. If you’re not discussing that yet within your organization, get busy!
AltaFlux Corporation is a global HCM cloud consulting partner based in Troy, Michigan. We empower organizations by streamlining, transforming, and optimizing key human capital management (HCM) processes with industry-leading HCM cloud solutions like SAP SuccessFactors, Benefitfocus, WorkForce Software and Dell Boomi.