Splunk My Ride! (and my Run and Swim) Part 1

GPS Exchange Format, or GPX, is a common import and export format for exercise tracking applications like Strava and Garmin.  As may have been mentioned in previous posts my sons and I are semi-active cyclists and multi-sport age group pseudo-athletes.  We use Strava (and XLS – don’t judge – you use it as an enterprise app somewhere too) as our source of truth for bike rides, runs, and swims.  Like many similar applications Strava uses GPX as its primary import and export format, however I had previously only used it to move data between Strava and Training Peaks.  I had never actually imported GPX into Splunk until now, and Brett “Ski Daddy Data Beard” Roberts is doing something similar using data generated while skiing at Lake Tahoe.  (Stand by for our upcoming podcast on how data and mobile applications are changing UX at Squaw Valley!)

GPX is a light weight XML data format for the transfer of GPS data and fairly pervasive.  Unfortunately, I have to admit I found a lot less conversations about using GPX and Splunk than I would have expected.  Anyone working with GPX and Splunk with some good tips, tricks or sources feel free to leave comments below or reach out to me @queboduck – I would love to mind share.  This is a learning opportunity for me.  One of the opportunities I did do was use this to practice creating a source type by creating a [gpx_import] source type in props.conf.  In the interest of full disclosure some of this came from another blog, but it did not provide the full syntax or steps so I will provide what I have done, and I will happily accept any constructive feedback.  A little about the environment – I am running Splunk 7.0.2 on a reclamated MacBook Pro from mid 2010 running MacOS 10.13.2.  Word of warning – As I have said, this is my first time working with GPX data so you may want to wait for parts 2 or 3 of this blog and see how this comes out.

So opening a terminal on my Splunk host I navigated $SPLUNK_HOME/etc/system/local which in my case is /Applications/Splunk/etc/system/local.  I did not have props.conf present so I just created one using vi with the following entries:

[gpx_import]

BREAK_ONLY_BEFORE = <trkpt

KV_MODE = xml

NO_BINARY_CHECK = true

SHOULD_LINEMERGE = true

pulldown_type = true

FIELDALIAS-rootfields = trkpt.ele as Height trkpt{@lat} as Latitude trkpt{@lon} as Longitude

And by all means feel free to skip the step where you stand there for 5 minutes wondering what you did wrong because you forget to restart Splunk.

The first test import of GPX data was from a Zwift virtual ride saved in Strava.  The data header comes in as an event with the date and time stamp of the import, however all subsequent events are cataloged with the appropriate date stamp with time intervals every second.  Each one second entry has latitude, longitude, elevation, power, heart rate, and cadence.

So far so good…  Next Step:  Making Something Out of the Data! (After importing more data)

Source Material:

Wikipedia GPS Exchange Format

Splunk Answers: How to ingest Strava GPX…

Splunk Blog: Downhill Splunking pt 1