Saturday, 30 November 2013

On Big Data and Travel

I've been meaning to write about "Big Data" for a while, especially when as an Artificial Intelligence graduate I saw the Venn diagram at the start of a recent Amadeus sponsored report included both machine learning and natural language processing. Also in Bain & Company's report their research showed that analytical capability had a strong correlation with top performing companies. So I thought I'd sum up some of the articles out there on the subject at the moment.


Is it just hype?

Gartner's analysis is that Big Data is currently at the peak of inflated expectations and heading towards the trough of disillusionment -  and Wired's article asking Is Big Data in the ‘Trough of Disillusionment’? showed areas that is already more advanced.

Is the case that
as the Japan Times OpEd Deflating the hype on big data claims
"big data isn’t much more than a sexier version of statistics, with a few new tools that allow us to think more broadly about what data can be and how we generate it."

I would suspect so from the travel industry articles I'll mention later on.

other articles on the hype aspect:

Problems

The Dangers of Faith In Data by Scott Kerbun describes 6 reasons we should be wary of trusting data:
"1. The Data Paradox. No matter how much data you have, you will still depend on intuition to decide how to interpret, explain and use the data ...
2. No team or organization is Data-driven. Data is non conscious: it is merely a list of stupid, dead numbers. Data doesn’t not have a brain and therefore can’t drive or lead anything...
3. Data is a flashlight. Data gives you specific information about a singular vector of information...
4. Ban the phrase “The data says.” Data can’t say anything for the same reason it can’t drive anything: data is inert. People, including data experts or growth hackers, can never speak singularly for the data...
5. Cognitive Bias pollutes our view of data. We know our brains are kludges, vulnerable to optical illusions. We also have blind spots in our cognition called cognitive biases...
6. Cui Bono -”who benefits?” Who paid for this data? What was their reason for paying for it? What ambitions do they have? ..."

People

Swati M. writing on Medium seems to hit the nail on the head about the hype and problems in We’re Missing the Human Side of Big Data
"We seem to think that Big Data technologies alone will solve or provide insight into business problems. But let’s get back to reality for now - data technologies are merely tools with capacity - they require people to have impact."

But which people? Over at the Harvard Business Review Matt Ariker, Tim McGuire, and Jesko Perry write about the Five Roles You Need on Your Big Data Team

"1. Data Hygienists
2. Data Explorers
3. Business Solution Architects
4. Data Scientists
5. Campaign Experts"
again they talk about the right people and creating the right culture for a successful big data project. This is echoed by Forbes in their Top 10 Strategic CIO Issues For 2013
     "In the past year or so, much of the talk about Big Data has obscured the fact that the real issue is enabling intelligent and instantaneous analysis to provide optimal insights for business decisions. CIOs need to ensure they’re looking at these high-volume, high-velocity challenges in the right way: as business enablers, not tech projects."
So it's not all about the technology and just getting Hadoop installed and a large dataset thrown at it won't solve all your problems.


Strategies

The ThoughtWork's blog Successful Strategies for Analytics and Big Data by John Spens makes the point that
 "Big Data Analytics must focus on insight and action."
How do we achieve that? Well Agility, Big Data and Analytics by Ken Collier (also at ThoughtWorks) details an agile approach and makes the point
"There is a temptation to think of data science as an isolated precursor to application development. This isolation is akin to doing big design up front and is anathema to agile development. If your software solution includes an advanced analytics capability, then data science is best viewed as a role within a cross-functional agile team rather than an isolated specialty."
In The Era of Big Data is For Real Tripp Babbitt has faith that Big Data can help us find the answers to what is currently "unknown and unknowable" and suggests that 
".. how we go about finding and collecting the right data still seems the most worthwhile path."
ThoughtWork's agile approach, with cross functional teams working closely with the business, could be a good method to utilize that focuses on that. As part of a conversation on twitter Tripp also made the following observation
the hype does seem to me to be about "doing" Big Data rather than strategies to deal with Big Data "happening". 

Travel Applications

Working in the travel industry "how can we use this?" is the most interesting question to me. Travel gets a mention in Information week's 6 sectors that are deriving value from Big Data, which is a promising start. 

Within the industry Amadeus certainly think that there is a payoff from big data for the travel industry and have published a report written by Professor Tom Davenport, noting that take up isn't even across the industry for example hotel chains making more use than airlines. The report (available via the article in the previous link) has various examples within the travel industry and I would expect travel management companies to become increasingly apply Big Data Analytics. In a follow up article On value, hype and zettabytes: A big data Q&A with Professor Tom Davenport  he notes online travel agencies as being leading users but most of the industry are behind the finance and retail sectors.


A couple of links that show more practical analysis of data in the travel industry are How big data influences design at Hotels.com, which shows an example of how data is being used for actionable insights, and again in Analysis: dealing with big data the travel companies interviewed seem to be ignoring the hype and investing in the analysis and making best use of the data that they have.

My view is that looking at data is a good way of guiding innovations and finding gaps in the market, does this need to be "Big Data" though? I think that depends on the market and type of question posed, for example in On the optimal distribution of traffic of network airlines by Xavier Fagedaa and Ricardo Flores-Fillolb they use a relatively modest dataset to build a model, which could be used to aid the choice for regional jets or a low cost brand in certain routes.

Update: A couple of hours after hitting publish I read Mark Van Rijmenam's article How Your Travel Journey Can Be Improved With Big Data. This puts forward a vision of airlines and airports sharing data together, which allows the whole journey to be improved both in terms of the service offered and passenger experience.

Other travel related links:

...... and finally 25 Cartoons To Give Current Big Data Hype A Perspective

R for Product Management

Photo by  Štefan Štefančík  on  Unsplash Since my previous blog post I have made some progress on being able to replace most of what I...