Last week at this time we saw the most stunning presidential election ever in the United States. Contrary to every opinion poll, contrary to every news publication in the nation and internationally, and contrary to the opinions of many in his own political party. Donald Trump won the presidential election by a healthy margin of electoral votes (at last count Trump had 306 electoral votes versus 232 for Clinton) This has left many people wondering how so many analysts could have been so consistent in making such a bad prediction. What did the data really predict?
On the eve of the election, Tom Anderson of OdinText composed. a text analysis of people’s responses to simple policy questions about each candidate. Here are the results of the analysis which was published before any of the state elections results were known.
The Blue represents Clinton’s results and the red represents Trump’s results of the survey. Statistically there were significant differences indicating a higher level of joy for Trump which Mr. Anderson attributed partly to Mr. Trump’s positive campaign slogan, “ Make America Great Again.” Descriptions for both candidates exhibited a lot of anger but the proportion of comments for Clinton were significantly higher. When I saw this early Tuesday morning in Hong Kong which was before the election results were known in the United States, I knew what the results would be.
What I suspect is that most analysts and press reacted emotionally to the rhetoric in the campaign from the candidates and did not listen to what the data from opinion surveys were telling them. The other problem may have been that the surveys were too superficial and did not tease out the real intentions of the people who were surveyed. To do this requires more than just asking who people intended to vote for, but posing questions about how people felt about the differing positions on different issues and correlating the results across several surveys over time to identify common underlying sentiments. Pro Republican and Pro democratic surveys needed to be analyzed together.
It all comes down to data, and the ability to search and analyze data from different sources. This requires a set of tools that can connect and interrogate multi-structured data from local, remote and in the cloud, prepare the data, and analyze the data. Tools like this would allow us to make better predictions and enable our workforce to connect to data in new ways, perform discoveries and explorations, identify new business opportunities, and determine new ways to improve operational processes and decisions.
Ironically we were planning to announce an enhancement to out Hitachi Content platform (HCP), on November 7th that would provide such capability but we postponed it due to the media focus on the election. Today we are announcing Hitachi Content Intelligence that would add content intelligence to our Hitachi Content Platform.
Hitachi Content Intelligence is the most flexible and comprehensive policy-based data exploration, search, and content analytics solution on the market today. It enables our customers to easily explore their muti-structured data, regardless of where it resides, with features and interfaces they are familiar with. IT Administrators and Content Managers benefit from complete control over what data sources to aggregate, designing processes to ensure that maximum insight and value is gleaned from the data, and ensuring those insights are surfaced quickly and are actionable. Best of all, this is all accomplished without compromising the visibility, management, control, sensitivity, and security requirements of the enterprise”