Lack of Skilled Staff Inhibits Adoption of IoT

A recent report by Cisco and Capita states that IT leaders are concerned that they do not have enough skilled resources to carry out potential IoT adoptions. This is an issue in several important and new areas of IT Management. It points to a need to change the approach to IT Education and certification, perhaps to one of emphasizing cooperative training with industry rather than gaining university degrees.

University degrees take a long time and the people acquiring them have little or no experience. Cooperative programs with trade schools and community colleges are faster and produce people with important and useful on-the-job experience. Check out this short summary of the report.

IoT Connections to Exceed Mobile Phones

The Internet of Things has been growing exponentially. A recent report by Ericsson finds that by 2018, the number of IoT connections will exceed that of mobile phones.

This growth is generated by such phenomena as smart houses, transport logistics, medical applications, autonomous cars and remote manufacturing.

Many of these applications are highly sensitive and require a very high degree of reliability and stability. Therefore they are a focus of IT management and a drain on  resources. With a projected annual growth rate of 23%, such IoT devices are becoming ubiquitous.

For a rundown on the Ericsson report, check this link.

Growing Volumes of Data Create Management Issues

The world supply of data is growing rapidly. That's no secret. At present the global volume of data is 4.4 zettabytes. One Zettabyte equals one sextillion bytes or 10 to the 16th power or one trillion gigabytes. It's hard to imagine, but clearly that's a lot of data. Predictions by IDC in a recent study indicate that global data will increase from 4.4 to 44 zettabytes over the next four years. Business will bear the brunt of this growth in data and will need to manage it.

The cloud is crucial to managing the data. But there also needs to be an organized and efficient means of integrating the data with corporate systems, including ERP and other enterprise systems. An additional complication is that much of the ultimate output needs to be channeled to mobile devices, including smartphones. There is a growing realization that the mobile world in which we live is an ongoing reality and the primary means of consuming information for the foreseeable future.

While systems integration has been with us for many years, the integration process involved with the new data is a relatively new field. It must take place in the cloud first and that primarily involves organizing the data so it can be processed. Companies are using "data lakes" for the first stage and then transitioning to cloud based data warehouses and various applications that feed into mobile apps.

For an interesting overview of the processes, here's a good article on the subject.

The Sharing Economy Grows

In May, Pew Research released a report on its surveys about various key aspects of the sharing economy. They covered ride sharing, home sharing and crowdfunding.  The results are interesting. they show that a large proportion of adults have used ride sharing (like Uber) and find it a satisfactory (or better) experience. Most are young urban adults. Fewer use home sharing and there is a common concern expressed among users or potential users about safety. Even fewer have used crowdfunding, but its use is growing.  For the reports, follow this link.

How to Make your Company Data-driven

Optimum use of data in management is rapidly shifting from competitive advantage to competitive necessity. But implementing an effective data use policy can result in spending massive amounts of money with poor results if it is not done well.

Serious planning is a necessity. As with any planning activity, definition of objectives is crucial, including specific definition of strategic and operational business objectives, and identification of data that will meet these objectives.

There are massive amounts of data on the internet, but not all are readily available for consumption. This is a significant consideration. Also, if you are planing to use internet based data, don't even think about developing your own systems to handle it. It should be capable of being analyzed without downloading. There are numerous tools available in the cloud where online cloud-based analysis can be carried out.

Ensure the results of the analysis are placed in the right hands. This would have been planned in the objectives identification phase. But you need to make sure that the results are configured and visualized in ways that make good use of graphs and other visuals or pictorials. Often the recipients are busy and won't take the time to spend a additional time on the analysis unless there is a rapid gratification.

This leads to another point about presenting the results. It must be done with a minimum of disruption to the recipients normal routines.  Again, this approach will lead to better use of the analysis.

For additional thoughts on this, and a source for this posting, click this link.

Job Automation - The big Trend of the 21st Century

People who do physical jobs have been faced with job losses because of automation for several decades now. The trend has been accelerating with the automation of entire factories and the need of retrain millions of workers.

Now the automation trend is spreading into the professions, even though their jobs are often not physical. At risk are accountants, lawyers, architects, to name a few. The automation of these fields does not mean that all will lose their jobs, but it does mean that large parts of their jobs can be automated, leading to a lower demand for them. For example a recent major report for the Canadian Bar Association called "Futures: Transforming the Delivery of Legal Services in Canada that outlined a strategy to deal with transformative change in the delivery of legal services. One of the driving forces for the change was technological innovation.

Last year, CPA Canada released a report setting out Drivers of Change in the accounting profession, with a strong emphasis on technological change. Other writers have written about how mobile devices, social media and data analytics will change the profession.

Behind the rapid spread of automation into the professions is the growing power of artificial intelligence. This enables technology to replace or significantly modify jobs with more and more intellectual content.

The Pew Research Center recently did a major survey into the views of a wide spectrum of professions on this issue. The results are quite revealing.

Mobile Brings Data to Real Time Decision Making

The increasing us of mobile smartphones and tablets is making it possible to generate data in the field and transmit it directly for analysis. In one example recently cited where such an approach was used for helping refugees,  mobile data, including GPS data and the results of interviews were collected and sent directly to a center using ESRI analytics tools.

There, the data could be immediately visualized, analyzed and used for ongoing decisions.

"With ESRI software, we were able to visualize what was going on,'' said Andrew Schroeder, director of research and analysis at Direct Relief, a Santa Barbara, Calif., nonprofit organization that coordinates humanitarian aid and care for people in poverty or emergencies, noting that ArcGIS's mapping capabilities "allowed us to understand some of the dynamics around issues such as who seeks care, where they are from and what their neighborhood conditions are."

For more on this experience with real time data analytics, check out this article.

How Big Data can Enable Targeted Marketing

Traditional marketing has used the broad broom approach, where advertisements are placed on media for general consumption, such as TV, Newspapers, etc. However, these media are attracting a smaller cohort than they did and therefore the ads miss their targets. Also, advertising on broad based media has become very expensive. The best example of this is the massive millions that are spend on superbowl ads.

Many of the people who are missed through the broad based marketing are now using hand held devices, such as smart phones and tablets. This applies to many younger people but is not limited to them. So advertising campaigns need to be segmented so that multiple campaigns can be run simultaneously that are directed to those segments.

This is where big data comes in. Data can be obtained from social media networks and internet usage data that show where and when the target audiences are using their media. Such advertising can then be much more effective in reaching audiences and promoting sales.

This article shows how targeted marketing techniques can be used in election campaigns and what business can learn from politics. Check it out.

Big Data for Cybersecurity

The growing use of big data has definite links to the problem of cybersecurity. Contrary to what many might think, the linkage is not all negative. While the risks in use of big data need to be controlled as do any other IT application, nevertheless big data can be useful in helping to control that risk. For example, predictive analytics can be applied to a combination of historical data and statistical metadata to give enterprises the ability to predict the probability of an intrusive event happening in the future. Also, big data analytics enables the data to be retained in its original form, whether structured or unstructured, thus providing much more flexibility in the analytical process. For a good rundown on this area, click this link.

Your Job is Being Redefined

A recent report by McKinsey & Co examines the role of automation in work life. It shows that 45% of work activities overall could be automated. It goes on to say, however, that only 5% of occupations can be replaced with current technology. So few occupations are likely to be replaced with technology in the near future. However, 60% of all occupations could have 30% or more of their activities automated.

This redefinition of jobs extends across the spectrum, from professionals like accountants and lawyers to travel agents (already largely redefined), bank tellers (heavily redefined) and airline pilots (who often steer as little as 4% of a flight.

Job redefinition is a major challenge for people as well as their employers. It calls for constant retraining and continuing change. Clearly adaptability and learning are more and more crucial to success in this evolving world.

Use Big Data Analytics for Marketing Strategy

A report by McKinsey on more than 250 engagements over five years shows that companies that put data at the centre of their marketing and sales decisions improve their marketing return on investment (MROI) by 15 – 20 percent. If this is applied to the estimated $1 trillion in global annual marketing spending, that adds up to $150 – $200 billion of additional value.

This is the primary reason why big data analytics has revolutionized marketing and sales. Analysis of big data can reveal new opportunities for a company. And the companies can tailor their product to customer wishes and beliefs, thus influencing their decision behaviour. In order to make analytics work, it is essential that a company invest on the latest techniques to enable fast analysis on the rapidly expanding pool of big data available to them., such as automated “algorithmic marketing,” an which provides for the processing of vast amounts of data through a “self-learning” process to create better and more relevant interactions with consumers - a kind of combination of data analytics and artificial intelligence. For an interesting take on this point of view, check this link.

How Big Data Analytics can Help with People/Talent Management

Analytics is entering into management at all levels. No longer a geek thing, it has become the fodder of top management. Check out this article on the subject.

How Big Data Analytics Helps Toyota Manage Accounts

For those who wonder how big data analytics is being used to help management, the case of Toyota Financial Services (TFS) serves as an illuminating one. TFS finances the ale of cars to customers and carries a portfolio of about $80 billion worldwide.

During the financial recession, delinquencies rose dramatically. The conventional collection techniques seemed to be consuming a lot of resources with mixed results.

TFS implemented a big data approach under which they collected data pertinent to the customers and applied algorithms to optimize the collection processes.

"their teams collaborated to create a new approach that included multiple technologies to assess individual consumers for their risk. FICO had developed an algorithm that allowed TFS to estimate which customers needed attention and the best way to approach each of them. Other technologies included SAS for statistics and predictive analytics, Oracle software and database software, IBM Pure Data (formerly known as Netezza), Tableau Software integrated into the user interface, Informatica for data integration, VMware for virtualization, and more. The solution relies on multiple technologies from multiple vendors and resides in Toyota Financial Services' many data centers."

The combination of optimization techniques, predictive analytics and prescriptive analytics all combined to yield fairer treatment of their customers, better use of resources and a better collections outcome.

For a more complete rundown, and the source of the above quote, click this link.

Google Analytics vs Spark

In a recent exhaustive, study, Mammoth Data found that Google Cloud Dataflow outperformed Apache Spark in several categories. This is important because both Google and Spark are widely used for big data analytics. Most companies are jumping into the big data world for reasons of competitive necessity.

In its benchmark, Mammoth Data identified five key areas where Google Cloud Dataflow equalled or exceeded Apache Spark:

• Greater performance
• Developer friendly
• Operational simplicity
• Easy integration
• Open-source

For more detail, check out this link.

Data Analytics on iPhones and iPads

iPhones and iPads are not often thought of as useful for data analytics. However, they can be so because some apps are available that simply act as a client under which the data remains on the cloud and the analysis takes place there too. Analyzing data on the cloud is the way of the future, since the idea for downloading is not feasible for much big data and users would have to resort to samples if downloading is used.

Data analytics apps range from simple analysis tools like Google Analytics to WolframAlpha, Statistics Visualizers and Roambi and even some analytics programming languages like Scala and Python. Those who are interested can spend hours experimenting with these tools while learning at the same time. Most are free from the Apple App Store. Some work in conjunction with Siri, thus enabling some StarTrek-like analysis.

For a brief rundown on 10 of these apps, check out this page.

Is Technology Going to Take your Job?

With the explosion of big and not-so-big data there has been an explosion in research on ways in which to use that data. One of the avenues of exploration has been in the area of artificial intelligence, where the capabilities of machines to emulate human behaviour is growing. For example, in the areas of professional services, like accounting and law, new software tools combined with the availability of massive amounts of data are making it possible to increase the numbers of evidence-based decisions as opposed to judgmental decisions. This in turn eliminates or reduces the more routine roles.

There is a long history of machines replacing human endeavour, and it is clear that this trend will if anything accelerate with the growth in the capabilities of technology.

"A 2013 study by researchers at Oxford University posited that as many as 47% of all jobs in the United States are at risk of “computerization.” And many respondents in a recent Pew Research Center canvassing of technology experts predicted that advances in robotics and computing applications will result in a net displacement of jobs over the coming decades – with potentially profound implications for both workers and society as a whole." (Pew Research Website)

Interestingly a majority of those surveyed said there own job would still exist in future.

For more on this trend, you can click on this link.

The Changing Face of Data

The availability of data from corporate systems, legacy systems, social media, public databases, Internet of Things and numerous other sources has been much discussed. Most and perhaps all of these sources of data have often been grouped under the label of Big Data.

While companies are coming to recognize the growing importance of big data, there are issues around the ability to actually use it. Some of the data is structured (organized in standard formats and understandable on its own). Other data is unstructured, meaning any useful analysis can only come after some restructuring is carried out to make the data understandable by the analytics tools being used.  Some of these tools, often based on the Hadoop framework, can handle unstructured data. Others have difficulty.

The difficulty is compounded by the fact that the data of interest is becoming available in different forms beyond that of simple numeric data. It includes text (for which analytical tools have been available for years), video, audio, graphics and other forms. The latter are very difficult to structure, particularly with tools that can be used across platforms.

The answer comes in different forms. One approach is to structure as much data as possible, using recognized standards such as XML and XBRL. But that generally applies in an effective way only to numeric data or structured non-numeric data.

Besides structuring data, an approach is to build larger data storage areas, where tools can be used across a variety of formats in some consistent way. While this is not a magic wand to fix all the analysis issues, it does allow for a more coordinated approach to data analytics and management. Many companies are going this route.

Check out this link.

Big Data Analytics Merging with Enterprise Systems

In a recent address in San Jose California, Doug Cutting, creator of Hadoop, the open source framework for big data analytics, reviewed the course of Hadoop and Big Data over the past ten years (yes, its been happening for ten years!). He pointed out how Big Data analytics has now moved into the space previously held by ERP and other Enterprise systems. Companies are using big data analytics increasingly to enable evidence-based decisions.

Over those 10 years, new and more powerful technologies have been introduced to improve Hadoop and enable better analysis. While much analysis started with MapReduce, many organizations are now using Apache Spark - also open source and powerful.

In a new study issued by Oxford Economics, it was pointed out that the use of big data analytics will explode with the availability of new data from the Internet of Things (IoT), a rapidly growing feature of the internet under which all kinds of items are connected to the internet and generating data. This would include appliances, houses, cars, and so on - your imagination is the limit. The study supports the predictions of Mr Cutting that big data analytics using Hadoop will become a central part of enterprise systems over the next ten years.

For more on these topics, check out this article and this report.  

Big Data Defined

With all the discussion about big data, there is a persistent problem. There is not general agreement on a definition of big data. For some, it means data available on the internet generally; for others, it's data coming from social media, or the internet of things. It sometimes refers to unstructured data and for others includes structured data such as that available from relational databases.

Sometimes big data is defined according to the tools used to analyze it, such as Hadoop or Spark. For others it relates to data from enterprise systems, like ERP and CRM.

Thee are lots of definitions around. Wikipedia, for example, says "big data is a term for data sets that are so large or complex that traditional data processing applications are inadequate." Most people would say this definition is too narrow.

Webopedia defines it as follows: "Big data is a buzzword, or catch-phrase, meaning a massive volume of both structured and unstructured data that is so large it is difficult to process using traditional database and software techniques."

This definition is better as it focusses on structured and unstructured data, which encompasses both data from traditional business systems as well as internet data such as that from social media. It also refers to massive in quantity, which is one of the defining characteristics.

A more analytical approach to defining big data is through the use of the words Volume, Variety and Velocity, perhaps with the word Variability. But sole use of these words does not clearly define big data. Nevertheless, they do provide a framework for a definition. Volume means very large amounts of data. Variety means data coming from very different sources, from business systems to the Internet of Things. Velocity is important because big data is always moving in fast, and there is a trend now to streaming analytics to recognize this. Variability means the data changes in volume, format and source.

Wrap these together and we can approach a definition. Big data is structured and unstructured data coming from a variety of sources, such as business systems, social media and the internet of things, moving at a high velocity and with frequently changing sources, formats and subject matter.

This definition may not be perfect and elegant, but at least it is broad but specific and encompasses the generally understood characteristics of big data.

For some further reading, check out these references: from Techtarget, Wikipedia and Webopedia.

Bank of Russia Implements XBRL for SMEs

Online media portal Russia Today is reporting that the Bank of Russia is planning on simplifying procedures for the issuance of securities by SMEs, including the introduction of XBRL. The intention is to improve the bond market by cutting costs and improving the flow of information to investors. They also plans legislation changes to improve overall handling of debt with an eye towards reducing reliance on banks as a source of funding for SMEs. More regulators are seizing on the opportunity presented by using structured data and the resulting transparency to meet strategic aims like promoting economic growth, transforming capital markets or improving government performance. This is a trend we expect to see more of and to make its way down into the enterprise as well.  (source: XBRL International Newsletter)