Friday, April 22, 2016

Is Technology Going to Take your Job?

With the explosion of big and not-so-big data there has been an explosion in research on ways in which to use that data. One of the avenues of exploration has been in the area of artificial intelligence, where the capabilities of machines to emulate human behaviour is growing. For example, in the areas of professional services, like accounting and law, new software tools combined with the availability of massive amounts of data are making it possible to increase the numbers of evidence-based decisions as opposed to judgmental decisions. This in turn eliminates or reduces the more routine roles.

There is a long history of machines replacing human endeavour, and it is clear that this trend will if anything accelerate with the growth in the capabilities of technology.


"A 2013 study by researchers at Oxford University posited that as many as 47% of all jobs in the United States are at risk of “computerization.” And many respondents in a recent Pew Research Center canvassing of technology experts predicted that advances in robotics and computing applications will result in a net displacement of jobs over the coming decades – with potentially profound implications for both workers and society as a whole." (Pew Research Website)

Interestingly a majority of those surveyed said there own job would still exist in future.

For more on this trend, you can click on this link.

Saturday, April 16, 2016

The Changing Face of Data

The availability of data from corporate systems, legacy systems, social media, public databases, Internet of Things and numerous other sources has been much discussed. Most and perhaps all of these sources of data have often been grouped under the label of Big Data.

While companies are coming to recognize the growing importance of big data, there are issues around the ability to actually use it. Some of the data is structured (organized in standard formats and understandable on its own). Other data is unstructured, meaning any useful analysis can only come after some restructuring is carried out to make the data understandable by the analytics tools being used.  Some of these tools, often based on the Hadoop framework, can handle unstructured data. Others have difficulty.

The difficulty is compounded by the fact that the data of interest is becoming available in different forms beyond that of simple numeric data. It includes text (for which analytical tools have been available for years), video, audio, graphics and other forms. The latter are very difficult to structure, particularly with tools that can be used across platforms.

The answer comes in different forms. One approach is to structure as much data as possible, using recognized standards such as XML and XBRL. But that generally applies in an effective way only to numeric data or structured non-numeric data.

Besides structuring data, an approach is to build larger data storage areas, where tools can be used across a variety of formats in some consistent way. While this is not a magic wand to fix all the analysis issues, it does allow for a more coordinated approach to data analytics and management. Many companies are going this route.

Check out this link.

Thursday, April 14, 2016

Big Data Analytics Merging with Enterprise Systems

In a recent address in San Jose California, Doug Cutting, creator of Hadoop, the open source framework for big data analytics, reviewed the course of Hadoop and Big Data over the past ten years (yes, its been happening for ten years!). He pointed out how Big Data analytics has now moved into the space previously held by ERP and other Enterprise systems. Companies are using big data analytics increasingly to enable evidence-based decisions.

Over those 10 years, new and more powerful technologies have been introduced to improve Hadoop and enable better analysis. While much analysis started with MapReduce, many organizations are now using Apache Spark - also open source and powerful.

In a new study issued by Oxford Economics, it was pointed out that the use of big data analytics will explode with the availability of new data from the Internet of Things (IoT), a rapidly growing feature of the internet under which all kinds of items are connected to the internet and generating data. This would include appliances, houses, cars, and so on - your imagination is the limit. The study supports the predictions of Mr Cutting that big data analytics using Hadoop will become a central part of enterprise systems over the next ten years.

For more on these topics, check out this article and this report.