This blog was originally posted on Direct2DellEMC Blog.
Running Out Of Time….
30 seconds remaining in the game up by 1 point my team is at the foul line. We go 1 for 1 on the free throws. Up 2 and the other team has the ball with 30 seconds. The opposing team takes the ball down the court as the clock expires and drains a 3 pointer.
The college basketball playoff season is time remind us the importance of playing to the clock. Did your team have more points than the opposing team when time expired?
What about your data? Has time expired on the business value of you data?
Expiring Data Does Data Expire?
Most organizations are trying to wrap their head around how to make faster decision based on their data. One of the quickest emerging trend is to use that data before their competitors even have a chance to react. This is where the concept of data expiration plays a key role. To prevent data expiration organizations must realize and execute on the business value of the data faster than the competition.
For example a ride sharing company has a small window of time to use their data (competitive advantage). The user (customer) says it would like a ride to a particular location then the application processes the best car (employee/contractor) to send to the user’s location. If the application wasn’t able to process the user’s location and car in real-time the user would find an alternate method for getting from one location to another. The time value of the data has an expiration and the window for the competitive edge is only getting smaller. User demand it!
Technologies Enabling Faster Analytics
The Internet of Thing or IoT enables low powered sensors with IP address to send data back to in turn be processed. It’s more than the SmartWatches from a couple of years ago. Imagine washing machine connected to the internet. While it might be marvelous to manage the washing machine from a smart phone how about the ability for the manufacturer to monitor the machines performance. One day the analytics in the washing machine notices that a $10 part is failing. If the part is replaced quickly it could avoid the whole machine from needing to be replaced and prevent flood the laundry room. Next the owner is sent an email is explaining the part needs to be replaced and coordinated list of available times. All this possible because the washing machine diagnostic files were analyzed before the machine failed.
In Charles Duhigg’s Smarter Faster Better he discusses how humans face information blindness when given too many choices. Organizations face the same problem when trying to analyze their data that is until recently. Machine Learning has given organization the ability to sift though millions and billions of data points to find value in data. Early in my analytics career I worked on a project to help systems administrators parse through proxy log files looking for bad actors. The administrators had an enormous task of shifting through millions to billions of log files. Most of the day to day work was only reporting on events some odd days or weeks after they occurred. Now with Machine Learning tools and software like Splunk, Tensorflow, Caffe, and others those administrators can be alerted in real-time when security threats occur.
Streaming Analytics allows for real-time analysis of data as it streams in. In the past analytical processing relied on batch systems using bound data sets. Batch processing tended to have long cycles to results that were out of date once processed. Now with processing frameworks like Apache Spark, Beam, Flink and others unbound infinite data sets are processed in real-time. Faster processing of the data is the feedback loop applications need to ensure business value is captured before the data expires.
Capture Your Data Before Time Expires…..Time Value of Data…..
While time is expiring on this year’s NCAA Tournament and your team’s chances of a Championship, make sure the time value of your data doesn’t expire. Find out more tips about capturing data value, Data Analytics Architecture, and other cool topics in Big Data by subscribing to the Big Data Beard Podcast.