A Practical Stream of Consciousness Curriculum to Big Data 101.1-Intro or Ongoing Journey into the Heart of Data Darkness_20171209_v01

Within any organization there are groups of individuals who are seldom seen but are some of the hardest working, ofttimes under-appreciated, and yet the most critical resources to the company. There is one such group that I work with on a regular basis. To visit them it is a bit like Leonidas seeking the Ephors of Sparta (see Zach Snyder’s 300). You must ascend the lonely tower in Richardson, Texas to the top and seek their cave. It was during one such pilgrimage I was asked a humbling question, and an even more humbling request: “What EXACTLY is it that you do?” and “How do I learn to do what you do?”

In actuality, their cave is two darkened conference rooms of dual monitor workstations bolted to a series of tables and flat panel TVs mounted to the walls. A heavy incense hangs within the conference-cave consisting of old hot sauce pilfered from local burrito chains, stale coffee, Drakkar Noir, and humanity enclosed in a small space. It is from this that the Ephors and visitors gain their visions.

To the right of the cave’s entrance is the Altar, next to a Tree of Perpetual Christmas decorated with Cat5 cable, decommissioned UCS parts, and surmounted by a stuffed honey badger. The Altar is a folding table with half empty barrels of big box club cheese balls dating back to 4Q2011. (The cheese balls are still edible, though mildly hallucinogenic. We periodically test them on ourselves and new hires.) It was at this altar while placing my end of quarter offerings for wisdom, guidance, last-minute BOM finalizations, SKU requests, and absolution for any number of sins that the question was posed to me: “What exactly is it that you do?”

I froze. For a moment I stood there like a bathroom roach with the light turned on, antennae twitching, fighting the urge to cry, “I take the requirements from the customers to the engineers! I am a people person!” However, as a member of the world’s oldest profession, sales, I quickly recovered. After a long conversation and a lot of white-boarding about “Big Data evangelism”, “differentiators”, “business outcomes and insights”, and “solutions portfolio” there came the request: “How do I learn to do what you do?”

For any of us in our careers this should be a very humbling experience that we should foster and encourage, if we feel what we do has any real value or validity. For me it is especially humbling. I am not a computer scientist or a mathematician; I am a poli-sci major who did stats because he sucked at languages, sometimes even English. My first exposure to data science was doing statistical analysis of coup d’états or the international arms trade. As a kid I would tell my dad, “I don’t care how High Mem Sys works I just want Wing Commander to freaking boot.” I got into this because the whole time I was reading Kissinger and Friedman, and dreaming about becoming the next Robert Gates, the stipend that actually paid my tuition had me working in data centers and computer labs. Who knew that in the 1990s there wasn’t much of a market for poli-sci majors who were not multi-lingual, or that healthcare reform would be difficult? Probably everybody. But hey, Stats is the new Russian or Arabic and I now understand computer memory management better.

The point is, I think like most people of my generation that became interested in statistics, big data, analytics, and machine learning there was not a curriculum, or if there was we did not follow it. So, when the question confronts me “how do I learn to do what you do?”, it is very much like Robin turning from the regimented, ordered example of Batman to look at the Joker and ask, “How do I be like you?”

“Well Mr. Grayson, your first step – plummet into a vat of chemicals while escaping a bad situation, go more than a little crazy, continually surround yourself with people who are better than you that challenge you, and come out with a huge smile on your face on the other side!”

Actually, except for the chemicals that is not a bad synopsis of my career arc, though completely unhelpful.

The fact of the matter is, even now, for most people I encounter in the industry they do not have the opportunity to follow the Batman’s path. MIT or UC Berkley’s Data Science program may be in my son’s future, but probably not mine or many of us who are established in the industry. In my mind that is part of the beauty and mission of the BigDataBeard community. It the end, we are all technological stand-up philosophers taking a communal swan dive into a really deep vat somewhere at Ace Chemicals in the hopes of it being transformative.

This will be part one in what I hope will be a long running series to help us all come out with a shared basis of knowledge and a smile on the other side. Planned future installments will include historic visions of computing in culture, introductory programming resources for python, practical stories of statistical analysis, Hadoop fundamentals, and deep learning and AI.  We’ll also post current reading list/syllabus and references as we prepare for the next blog.  As always we would welcome suggestions and feedback.