Use Big Data Analytics for Marketing Strategy

A report by McKinsey on more than 250 engagements over five years shows that companies that put data at the centre of their marketing and sales decisions improve their marketing return on investment (MROI) by 15 – 20 percent. If this is applied to the estimated $1 trillion in global annual marketing spending, that adds up to $150 – $200 billion of additional value.

This is the primary reason why big data analytics has revolutionized marketing and sales. Analysis of big data can reveal new opportunities for a company. And the companies can tailor their product to customer wishes and beliefs, thus influencing their decision behaviour. In order to make analytics work, it is essential that a company invest on the latest techniques to enable fast analysis on the rapidly expanding pool of big data available to them., such as automated “algorithmic marketing,” an which provides for the processing of vast amounts of data through a “self-learning” process to create better and more relevant interactions with consumers - a kind of combination of data analytics and artificial intelligence. For an interesting take on this point of view, check this link.

How Big Data Analytics can Help with People/Talent Management

Analytics is entering into management at all levels. No longer a geek thing, it has become the fodder of top management. Check out this article on the subject.

How Big Data Analytics Helps Toyota Manage Accounts

For those who wonder how big data analytics is being used to help management, the case of Toyota Financial Services (TFS) serves as an illuminating one. TFS finances the ale of cars to customers and carries a portfolio of about $80 billion worldwide.

During the financial recession, delinquencies rose dramatically. The conventional collection techniques seemed to be consuming a lot of resources with mixed results.

TFS implemented a big data approach under which they collected data pertinent to the customers and applied algorithms to optimize the collection processes.

"their teams collaborated to create a new approach that included multiple technologies to assess individual consumers for their risk. FICO had developed an algorithm that allowed TFS to estimate which customers needed attention and the best way to approach each of them. Other technologies included SAS for statistics and predictive analytics, Oracle software and database software, IBM Pure Data (formerly known as Netezza), Tableau Software integrated into the user interface, Informatica for data integration, VMware for virtualization, and more. The solution relies on multiple technologies from multiple vendors and resides in Toyota Financial Services' many data centers."

The combination of optimization techniques, predictive analytics and prescriptive analytics all combined to yield fairer treatment of their customers, better use of resources and a better collections outcome.

For a more complete rundown, and the source of the above quote, click this link.

Google Analytics vs Spark

In a recent exhaustive, study, Mammoth Data found that Google Cloud Dataflow outperformed Apache Spark in several categories. This is important because both Google and Spark are widely used for big data analytics. Most companies are jumping into the big data world for reasons of competitive necessity.

In its benchmark, Mammoth Data identified five key areas where Google Cloud Dataflow equalled or exceeded Apache Spark:

• Greater performance
• Developer friendly
• Operational simplicity
• Easy integration
• Open-source

For more detail, check out this link.

Data Analytics on iPhones and iPads

iPhones and iPads are not often thought of as useful for data analytics. However, they can be so because some apps are available that simply act as a client under which the data remains on the cloud and the analysis takes place there too. Analyzing data on the cloud is the way of the future, since the idea for downloading is not feasible for much big data and users would have to resort to samples if downloading is used.

Data analytics apps range from simple analysis tools like Google Analytics to WolframAlpha, Statistics Visualizers and Roambi and even some analytics programming languages like Scala and Python. Those who are interested can spend hours experimenting with these tools while learning at the same time. Most are free from the Apple App Store. Some work in conjunction with Siri, thus enabling some StarTrek-like analysis.

For a brief rundown on 10 of these apps, check out this page.

Is Technology Going to Take your Job?

With the explosion of big and not-so-big data there has been an explosion in research on ways in which to use that data. One of the avenues of exploration has been in the area of artificial intelligence, where the capabilities of machines to emulate human behaviour is growing. For example, in the areas of professional services, like accounting and law, new software tools combined with the availability of massive amounts of data are making it possible to increase the numbers of evidence-based decisions as opposed to judgmental decisions. This in turn eliminates or reduces the more routine roles.

There is a long history of machines replacing human endeavour, and it is clear that this trend will if anything accelerate with the growth in the capabilities of technology.

"A 2013 study by researchers at Oxford University posited that as many as 47% of all jobs in the United States are at risk of “computerization.” And many respondents in a recent Pew Research Center canvassing of technology experts predicted that advances in robotics and computing applications will result in a net displacement of jobs over the coming decades – with potentially profound implications for both workers and society as a whole." (Pew Research Website)

Interestingly a majority of those surveyed said there own job would still exist in future.

For more on this trend, you can click on this link.

The Changing Face of Data

The availability of data from corporate systems, legacy systems, social media, public databases, Internet of Things and numerous other sources has been much discussed. Most and perhaps all of these sources of data have often been grouped under the label of Big Data.

While companies are coming to recognize the growing importance of big data, there are issues around the ability to actually use it. Some of the data is structured (organized in standard formats and understandable on its own). Other data is unstructured, meaning any useful analysis can only come after some restructuring is carried out to make the data understandable by the analytics tools being used.  Some of these tools, often based on the Hadoop framework, can handle unstructured data. Others have difficulty.

The difficulty is compounded by the fact that the data of interest is becoming available in different forms beyond that of simple numeric data. It includes text (for which analytical tools have been available for years), video, audio, graphics and other forms. The latter are very difficult to structure, particularly with tools that can be used across platforms.

The answer comes in different forms. One approach is to structure as much data as possible, using recognized standards such as XML and XBRL. But that generally applies in an effective way only to numeric data or structured non-numeric data.

Besides structuring data, an approach is to build larger data storage areas, where tools can be used across a variety of formats in some consistent way. While this is not a magic wand to fix all the analysis issues, it does allow for a more coordinated approach to data analytics and management. Many companies are going this route.

Check out this link.

Big Data Analytics Merging with Enterprise Systems

In a recent address in San Jose California, Doug Cutting, creator of Hadoop, the open source framework for big data analytics, reviewed the course of Hadoop and Big Data over the past ten years (yes, its been happening for ten years!). He pointed out how Big Data analytics has now moved into the space previously held by ERP and other Enterprise systems. Companies are using big data analytics increasingly to enable evidence-based decisions.

Over those 10 years, new and more powerful technologies have been introduced to improve Hadoop and enable better analysis. While much analysis started with MapReduce, many organizations are now using Apache Spark - also open source and powerful.

In a new study issued by Oxford Economics, it was pointed out that the use of big data analytics will explode with the availability of new data from the Internet of Things (IoT), a rapidly growing feature of the internet under which all kinds of items are connected to the internet and generating data. This would include appliances, houses, cars, and so on - your imagination is the limit. The study supports the predictions of Mr Cutting that big data analytics using Hadoop will become a central part of enterprise systems over the next ten years.

For more on these topics, check out this article and this report.  

Big Data Defined

With all the discussion about big data, there is a persistent problem. There is not general agreement on a definition of big data. For some, it means data available on the internet generally; for others, it's data coming from social media, or the internet of things. It sometimes refers to unstructured data and for others includes structured data such as that available from relational databases.

Sometimes big data is defined according to the tools used to analyze it, such as Hadoop or Spark. For others it relates to data from enterprise systems, like ERP and CRM.

Thee are lots of definitions around. Wikipedia, for example, says "big data is a term for data sets that are so large or complex that traditional data processing applications are inadequate." Most people would say this definition is too narrow.

Webopedia defines it as follows: "Big data is a buzzword, or catch-phrase, meaning a massive volume of both structured and unstructured data that is so large it is difficult to process using traditional database and software techniques."

This definition is better as it focusses on structured and unstructured data, which encompasses both data from traditional business systems as well as internet data such as that from social media. It also refers to massive in quantity, which is one of the defining characteristics.

A more analytical approach to defining big data is through the use of the words Volume, Variety and Velocity, perhaps with the word Variability. But sole use of these words does not clearly define big data. Nevertheless, they do provide a framework for a definition. Volume means very large amounts of data. Variety means data coming from very different sources, from business systems to the Internet of Things. Velocity is important because big data is always moving in fast, and there is a trend now to streaming analytics to recognize this. Variability means the data changes in volume, format and source.

Wrap these together and we can approach a definition. Big data is structured and unstructured data coming from a variety of sources, such as business systems, social media and the internet of things, moving at a high velocity and with frequently changing sources, formats and subject matter.

This definition may not be perfect and elegant, but at least it is broad but specific and encompasses the generally understood characteristics of big data.

For some further reading, check out these references: from Techtarget, Wikipedia and Webopedia.

Bank of Russia Implements XBRL for SMEs

Online media portal Russia Today is reporting that the Bank of Russia is planning on simplifying procedures for the issuance of securities by SMEs, including the introduction of XBRL. The intention is to improve the bond market by cutting costs and improving the flow of information to investors. They also plans legislation changes to improve overall handling of debt with an eye towards reducing reliance on banks as a source of funding for SMEs. More regulators are seizing on the opportunity presented by using structured data and the resulting transparency to meet strategic aims like promoting economic growth, transforming capital markets or improving government performance. This is a trend we expect to see more of and to make its way down into the enterprise as well.  (source: XBRL International Newsletter)

Business Goes for Streaming Analytics

In a move that is likely to prove a watershed, business has been moving into Streaming Analytics. A recent report by Forrester Research pinpointed this trend in a study of the adoption of data analytics. They pointed out that three years ago, business was struggling with ways to apply analytics to their existing data stores and some big data.

Forrester defines streaming analytics software as technology that "can filter, aggregate, enrich, and analyze a high throughput of data from multiple disparate live data sources and in any data format to identify simple and complex patterns to visualize business in real-time, detect urgent situations, and automate immediate actions."(source)

A number of prominent software providers and technologies are available, including "Apache Spark Streaming, Apache Storm, Data Torrent, IBM, Informatica, SAP, Software AG, SQLstream, Strim (WebAction), TIBCO, and Vitria," says Forrester.

The advent of streaming analytics could herald a new era in business decision making. In the past, decisions have largely been based on historical information with attempts to extrapolate into the future using whatever current information and intuition is available. Streaming analytics will reduce the uncertainty of this approach and add some real science to the decision making process.

Where Big Data Analytics is Headed

We hear much about big data and big data analytics. And we are told that it is the big next thing. But how much of this is hype. And where is it really going?

Forbes has published a summary of predictions that sheds some light on the whole matter. It highlights how the amount of data will grow exponentially, that analytics tools will be changing to move past SQL into Spark and others. More importantly they talk of how new user friendly tools are being released, such as those from Microsoft and Salesforce, that do not require programming expertise. Even more importantly they predict that machine learning will play a big role in the future of data analytics, with perhaps even tools that operate free of people.  They also stress the importance of prescriptive analytics, which takes descriptive and predictive analytics to the next stage by indicating not only what will happen in the future but why it will happen, paving the way for serious support of decision making.

Overall the development of data analytics is heading for uncharted waters and may take directions we don't see yet.

For this thought-stimulating article in Forbes Magazine, follow this link.

Predictive Analytics can Improve Business Decisions

Predictive Analytics is a means of studying large amounts of data and drawing from it inferences about future behaviour of customers, employees, stakeholders, and others. While other kinds of analysis can indicate problem areas in, say sales, and tell management what isn't working, predictive analytics can indicate what policies are likely to work before they are implemented.

The availability of big data is particularly useful for predictive analytics because of the sheer volume of data and the coverage of behaviour it encompasses.

Companies are therefore using predictive analytics with increasing success in a variety of circumstances, including analyzing individual customer traits to determine how best to serve them and to determine the most effective procurement strategies in advance. Specific information on the elasticity of demand can also be used to determine the best price/production strategies. There is a myriad of possible scenarios where predictive analytics can be used, which accounts for its popularity.

For some specific examples, check out this site.

Data Analytics for the Internet of Things

As the numbers of buildings, cars, appliances and other things get connected to the internet, and the data generated by this connectivity grows in volume, And yet, the variety and sheer scope of the data available almost defies interpretation and analysis.  The task is not only one of analyzing data from different platforms, but the more difficult task of analyzing data coming from widely disparate devices.

Data showing driving conditions in a particular area, for example, can be distorted by a myriad of non relevant events. The same goes for liveability conditions in a particular type of building. The volume of IoT data is so great that is can't even be analyzed on the cloud.

Data analytics is attempting to address this data, but it has been recognized that there needs to be some sort or order brought into the data and the analytical approach taken.

A team of researchers supported by the National Science Foundation in the US is looking into this issue and is charged with developing a framework for conducting data analytics across a variety of IoT devices. The framework will consist of an organization of software that will facilitate communications and research into the data. The research is led by Stacy Patterson, the Clare Boothe Luce Assistant Professor of Computer Science at Rensselaer Polytechnic Institute (RPI). Read more at: http://phys.org/news/2016-03-internet-thingsa-framework-analytics-digital.html#jCp

Using Data Analytics for Developing Emerging Markets

Data Analytics, including that involving big data, is increasingly being used for decision making in a variety of organizations. Some of the data is embedded in new systems and some is gathered from the internet on, for example, social media and through the internet of things.

The essential task of big data analytics is to transform raw and structured data from a variety of disparate sources into actionable knowledge.

This process involves using new tools, often based on Hadoop, such as Google DataQuery and BigQuery. It makes use of contemporary computer capabilities such as high speed communications, massive storage capability and super powerful processors.

A good example of big data analytics in action is that of emerging energy markets, such as that in the Gulf of Mexico's Mexican region.

"Data analytics is being used through most of the lifecycle of offshore activities. During seismic and reservoir characterization studies, data sources with 3D seismic data, well logs and faults, are integrated and analyzed to support decisions related to achieving key targets in flow assurance, field optimization, drilling performance, well categorization and so forth. Benefits range from attaining optimal reservoir exploitation rate to forecasting the decline of new wells.

"For fixed, floating and subsea assets, data analytics starts with collecting data at the asset level, including operating parameters, equipment status, structural stresses and environmental data. For moving assets such as offshore support vessels and dynamic positioning floaters/vessels, data collection can also include location, direction and speed."

In this way big data analytics facilitates decision making in a difficult market. For more on this particular application, check out this link.

LLoyds and Google Team Up

Lloyds Group has teamed up with Google in analyzing big data relating to its insurance customer's non-personal behaviour. In the initial project they analyzed a year's worth of data in under one minute using Google BigQuery. One of the outcomes is that they were able to reduce certain response times from 96 hours to 30 minutes - a remarkable achievement.

The work will continue with a wider array of data and more Google tools for big data analytics, including Data Flow and Big Table.

Big Data Analytics is starting to show real results and while there is still a novelty factor to it, within the year it will be a competitive necessity in many industries.

Big Data Analytics Begins to Mature

The ability to make effective use of big data has been hampered by the lack of big data skills along with the lack of useful tools for analysis. Both of these areas are being addressed by interested organizations.

The lack of big data scientists has been bemoaned since the advent of big data and the realization of its potential. Rutgers is making an important step forward by offering a new program that focuses on big data skills. On March 29, the Center for Innovation Education at Rutgers University will begin its 44-week skills-based technology career certificate program for professionals who want to gain skills in big data disciplines. Initially the course is only open to recent grads in the US. But is marks the beginning of much needed data oriented education that hopefully will spread.

And also there are important innovations going on in the availability of analytical tools. Google features strongly in this field with its announcement that Google Dataproc, its managed Apache Hadoop and Apache Spark service, is now available to the public. Who other than the world's most prominent exploiter of data would step up to the major challenges of offering powerful user oriented tools for big data analysis using the Hadoop system which has been the core of much big data analytical activity.

We can expect a rash of new product announcements as big data gains in importance for business policy. Check out this article for more.

Blockchain Gains Traction in Banking and Finance

An opinion piece in TechCrunch Online has some interesting things to say about the uptake of blockchain technology in the finance sector. The article relates how both the NASDAQ in the US and the Australian Stock Exchange, along with many other financial institutions, are looking to employ the technology as a way to provide shared infrastructure which allows transactions, that can now can take days to finalise, to instead be settled instantly and transparently. Blockchain is the technology underlying Bitcoin. (Source)

Hadoop Changing How Companies use Data

The era of big data has been changing corporate systems. First is the influx of data from systems like CRM and others. The advent of data availability from social media is greatly increasing the flow of big data. In this there is much opportunity.

The issue is that the data is very large in volume and usually too large to place in a data base or to download. Also, much of it is unstructured data that is difficult to analyze.

Hadoop is a technology that enables distributed storage of data and distributed analysis on an efficient basis. It enables the data to be analyzed without downloading it, which is a major advantage. It also is increasingly being tied into various data analytics tools that can deal with vast quantities of unstructured data and perform predictive analysis.

With the opportunities beginning to be realized, a new vigour is being added to decision making. For more, check out this link.

Smartphones - Expect Innovation in Decision Support

A recent study released by Pew Research shows the ownership of smartphones in the US has continued to increase over all other forms of personal electronic device. “Today, 68% of U.S. adults have a smartphone, up from 35% in 2011.” Also, “tablet computer ownership has edged up to 45% among adults”, as compared to 3% five years ago.

At the same time, personal computers have stayed at about the same level of ownership, meaning that relatively speaking, the use of small mobile devices has grown relative to computers.

Indeed, research shows that the use of smartphones and tablets for accessing the internet far exceeds the use of computers for that purpose. At a time when a growing percentage of data available to people now resides in the cloud, this is a significant trend. It means that decisions are increasingly made by people getting their data from the cloud through the use of smartphones.

Smartphones and tablets are very different from computers in important respects. Obviously, they have very small screens compared to computers, which means that websites need to be formatted so they can be read on those smaller screens. The best way to accomplish this is to have the websites prepared using ‘responsive design’, under which the website detects the type of device attempting to access it and presents the data in a format that is suitable for that device. Many organizations are using this technique to present information.

Another major difference between mobile personal devices and laptops is in that the former have less computing power than laptops or other computers. That means that people who must use data for more complex decision making must either use a computer to do the analysis or else, if they are using a smartphone, have access to online tools that will interact with a smartphone. Or else, they need apps that incorporate online analytical capability.

In certain fields, online analytical capability is getting richer. Certainly the power of smartphone apps is growing considerably.

As an example, most companies present their investor relations information in a special section of their website. Some present this information using responsive design.  Some others, far fewer at this point, make available IR apps for smartphones. The apps format key information for the smartphones and in some cases make available online analytical capabilities through those apps.

Investment decisions can be complex decisions and is an area where a great deal of innovation is going to happen over the next few years to reflect the growing use of smartphones in making decisions. Watch for a lot more apps and a lot more analytical capability where investors can select the data that is needed for their decisions and analyze it on their smartphones.