The three universal questions companies ask about big data
Surely, you've been hearing about Big Data. It's all the rage right now, but many companies don't understand how Big Data can possibly affect them. The fact is that disruption is having an impact on every organization today, but it's not just the business itself, it's how you do business including the way you make decisions -and data can help you make better decisions.
Andrew McAfee, who studies and lectures on the influence of IT on business at MIT, talked about some of the trends he's seeing in big data at the Alfresco Summit in Boston this week. He explained that most companies don't make decisions based on data. Instead, they follow whatever is the highest paid person's opinion, regardless of whether that makes sense or not.
He says once he convinces an executive team that big data is a real phenomenon that every company needs to be looking at, they all have the same three questions.
Question One: How much data are we talking about?
Most companies believe they've been dealing with copious amounts of information for years, so when people like McAfee walk into their company talking about big data, they are skeptical. McAfee says the best way to understand just how much data we are talking about is to follow a history of how we described what we considered a large amount of data.
"The volume of data [today] is outstripping our ability to describe what's going on," McAfee told the audience in Boston. "Big is not just marketing hype, it really does mean something," he added.
He explained that in 1980, a company launched and called itself TeraData. It chose that name because 10x12 zeros was about as big as data got and that was great…for 28 years. But by 2008, 12 zeros wasn't enough and we moved to the Petabyte era (10x15 zeros). That was good for four years at which point we entered the Exabyte era (10x18 zeros). That was good for just 6 months at which point we entered the Yottabyte era (10x24 zeros).
McAfee explained that there is a group of astrophysicists and astronomers who measure really, really, really big numbers. They last met in 1991. They thought we would never need a number bigger than 10x24 zeros, but we will soon use that up and the group now has to come up with some new ways describing large numbers.
In the next few years, there will be ever larger amounts of data coming from sensors (also known as the Internet of Things), from our smartphones, from our social networks, and from just about every data-enabled device and online service you can imagine, and some you can't. That's why unless you're Google or Facebook or perhaps a financial services company, you hasn't even sniffed the amount of data we're talking about yet.
Question 2: So what? What does this have to do with my company?
After companies understand the scope of the data we're talking about, the next question is why it should matter to them? If they aren't a data-driven organization, what kind of impact could this trend possibly have on them?
For this, McAfee gave a few areas where just about any business could benefit from using data for predicting and forecasting. When you follow the data, you'll find the answers to your business issues.
He used the example of his colleague Erik Brynjolfsson. He and Lynn Wu set out to prove they could predict housing prices as well as or better than the experts at the National Association of Realtors. The NAR had been doing it for years and were considered the gold standard for predicting housing prices using a mix of economic indicators, GDP growth, interest rates and so forth to build a statistical model.
Newsweek made waves this week with an article that claims to unmask Satoshi Nakamoto, the previously anonymous person whose name was the only one listed on the 2008 whitepaper that launched the modern cryptocurrency movement.
IBM has announced a competition to promote the development of apps powered by its Watson cognitive computing platform. But some apps already are in use or in the pipeline.
If you've got a Windows XP machine -- either at home or in the office -- consider yourself lucky. In the past, you'd upgrade to a more recent Windows operating system without a thought. Today, you have many options.