Accessing Sleep & Activity Data from Garmin Wearables | by Adam Brownell | Jun, 2022
A Friendly how-to guide on extracting Garmin Venu 2 data for people in a hurry
About 6 months ago, I wrote on how to convert .fit files from Garmin into CSVs.
Since then, I got a new Venu 2 Garmin watch, and with it there is more data to process!! Specifically around cleaning sleep and activity data from my Garmin wearables
Just as before, I’m hoping this article is a nice stand-in for the missing/outdated documentation I found when going through Garmin’s site.
Spoiler: I’ve written a script to quickly process all Garmin data here. Enjoy! 🌴 🏄
In my previous post, I assumed all the usable data only found in DI_CONNECT/DI-Connect-Fitness-Uploaded-Files .zip
Since then, I have found more data!
After getting the data dump from Garmin (as explained in my previous post), here is the file structure of DI-CONNECT :
I’ll be going through each file + the dated contained within, at least for me. Just like many of the folders included in the data dump, many of the files are empty — partially due to me not using all the features of my watch, and partially because my watch does not have all the features.
All of the code written in the appropriate git repo follows this general structure:
Below we will go each new Garmin json file and what data to expect.
Under the DI-Connect-Wellnessfolder we have two files, one focused on health snapshots and the other focused on aggregate sleep data.
These are the most interesting files for analysis outside of the DI-Connect-Fitness-Uploaded-Files.
…_wellnessActivities.json
This json file contains data around the Health Snapshot Activity on your Garmin watch. Running the activity involves sitting still for 2 minutes so your match can make accurate biometric readings
The reason that you need to be still is because it is well known that movement causes accuracy issues with wrist-based monitors.
Data included:
- Time of Day
- Heart Rate
- Respiratory Rate
- “Stress”
- Oxygen Saturation
- Heart Rate Variability
“Stress” is in quotation marks because it is a proprietary, Garmin-specific metric that is obfuscated on how it is calculated
Also, since this activity needs to be manually triggered, it is possible for it to be empty.
…_sleepData.json
This json file contains data about your sleep, aggregated to the per-session level.
Data included:
- Seconds in each stage of sleep (Awake, Light, Deep, REM)
- Respiratory Rate across sleep session
- “Scores” around quality, duration, and recovery of sleep session
- Number of times your woke during the night
- Garmin-generated insights and feedback based on your sleep session
“Scores” is in quotation marks because it is a proprietary, Garmin-specific metric that is obfuscated on how it is calculated
The feedback and insights columns were unexpected + pretty interesting, because I’m not sure this data is easily available in the Garmin Connect app:
Under the DI-Connect-Userfolder we have 14 files focused on your Garmin profile
While there are a bunch of files, some are not interesting for analysis — 3 are your profile picture of various sizes, one ( ...-social-profile.json) contains the information shown on your Garmin profile, two files ( user_profile.json & user_profile_suica.json) just have your name & email on file, and another ( user_contact.json) has your emergency contact.
A few files may have potential for interesting information, but for me they did not:
user_reminders.jsonwas empty, likely because I do not use remindersuser_settings.jsonwas only 1 line long and marked the HR between each zone. I was using the default (see below)user_biometrics.jsoncould potentially contain biometrics data like height and VO2Max, but everything was Null for me
To quickly check a json file, such as user_settings.json, you can simply do…
file_name = os.getcwd()+'/DI_CONNECT/DI-Connect-User/user_settings.json'
with open(file_name) as file:
j = json.load(file)
json.dumps(j)
…which for me generated:
That leaves us with 3 Files to analyze:
FitnessAgeData.json
This json file contains data about your Fitness Age. Garmin describes Fitness Age as:
Fitness age is an estimate of how fit you are compared to your actual age.
It is calculated by comparing your current VO2 max fitness level to the normal values of people of different ages within your same gender …[as well as your] activity intensity, resting heart rate and body fat percentage or body mass index (BMI)
Data Included:
- Your age, BMI, and Resting Heart Rate
- Total number of vigorous weeks, days, and minutes
- What a healthy BMI, body fat %, and VO2Max would look like for you based on your current weight/height/gender
- Your VO2Max
- Your “Age” in terms of overall health, bmi, and body fat %
“Age” is in quotation marks because it is a proprietary, Garmin-specific metric that is obfuscated on how it is calculated
The readings are not necessarily everyday as I hoped (as you can see a large gap in dates from the above screenshot), but certainly is an interesting dataset to explore over longer timescales.
HydrationLogFile….json
So I do not currently use the Hydration Widget on my watch, so my data is a little boring/sparse. It does make an entry whenever you use an activity since it estimates sweat loss.
However, if you do, here is what you can expect the data to contain:
- Time of Reading
- Hydration source + activity ID
- Amount of hydration, duration of activity
- Estimated amount of sweat loss during activity
UDSFile_….json
The UDS File, which I’m assuming stands for “User Daily Summary”, contains daily aggregated of steps, stairs, stress, and bodyBattery
The Data in this file includes:
- Daily Time & Distance features like
totalDistanceMeters - HeartRate features
- Stress Features like
awakeAverageStressLevel - Body Battery Features like
maxBodyBattery - Additional day-aggregate Respiration and SPO2 Features
These files all relate to discreet physical activity you engage in with your Garmin.
Unfortunately, I do not use my Garmin to set goals or plans, so most of these files are empty for me.
This leaves only one file with data:
…summarizedActivities.json
As the name implies, this tables contains the summaries of the Activities you engage in while wearing your Garmin. Like Health Snapshot, this is not a passive form of data collection as you need to manually tell your watch you are exercising.
Nevertheless, you can generate following data, depending on what activity you engage in:
- Activity Name & Start time
- Intensity Minutes
- Elevation Gains/Loss, and distance travelled, and location
- RespirationRate stats
- Total sets and reps
Like most of the Fitness files, I do not golf or use my watch for Golf, so these files are empty. Maybe in the future I’ll pick up the sport and start collecting some data on the subject!
The final data treasure trove comes in the form of a .zip file, rather than a json as we have seen in the past.
I have written about how to read this zip file previously, but this was a somewhat manual process (or required making a macro). Since then I’ve built a more programmatic way to address this tranformation requirement:
I know I should use os.subprocess but this was the simplest and most interpretable way to port over my previously manual process
As mentioned previously, I combined the processing steps required into a git repo!
- Cloning a Git Repo
Simply open Terminal and type in:
gh repo clone adam1brownell/garmin_data
That will download the code I’ve written.
The navigate into the directory via
cd adam1brownell/garmin_data
2. Accessing Garmin Connect Data
Assuming you have downloaded the Garmin Connect data locally and FitCSVJar file as detailed in my previous post, all you’ll need to do to process your files is run the following command:
python parse_garmin_connect.py [connect/data/folder/location] [FitCSV/file/location]
replacing [connect/data/folder/location] and [FitCSV/file/location] with the appropriate file locations. Tip: On Mac you can drag & drop files into Terminal to autmatically copy over the file location
It should look something like this:
The above command may take a while, as it is processing the fit files.
But once it is done, you should have several csv files of your Garmin data, easily accessible and ready to be analyzed!!
A Friendly how-to guide on extracting Garmin Venu 2 data for people in a hurry
About 6 months ago, I wrote on how to convert .fit files from Garmin into CSVs.
Since then, I got a new Venu 2 Garmin watch, and with it there is more data to process!! Specifically around cleaning sleep and activity data from my Garmin wearables
Just as before, I’m hoping this article is a nice stand-in for the missing/outdated documentation I found when going through Garmin’s site.
Spoiler: I’ve written a script to quickly process all Garmin data here. Enjoy! 🌴 🏄
In my previous post, I assumed all the usable data only found in DI_CONNECT/DI-Connect-Fitness-Uploaded-Files .zip
Since then, I have found more data!
After getting the data dump from Garmin (as explained in my previous post), here is the file structure of DI-CONNECT :
I’ll be going through each file + the dated contained within, at least for me. Just like many of the folders included in the data dump, many of the files are empty — partially due to me not using all the features of my watch, and partially because my watch does not have all the features.
All of the code written in the appropriate git repo follows this general structure:
Below we will go each new Garmin json file and what data to expect.
Under the DI-Connect-Wellnessfolder we have two files, one focused on health snapshots and the other focused on aggregate sleep data.
These are the most interesting files for analysis outside of the DI-Connect-Fitness-Uploaded-Files.
…_wellnessActivities.json
This json file contains data around the Health Snapshot Activity on your Garmin watch. Running the activity involves sitting still for 2 minutes so your match can make accurate biometric readings
The reason that you need to be still is because it is well known that movement causes accuracy issues with wrist-based monitors.
Data included:
- Time of Day
- Heart Rate
- Respiratory Rate
- “Stress”
- Oxygen Saturation
- Heart Rate Variability
“Stress” is in quotation marks because it is a proprietary, Garmin-specific metric that is obfuscated on how it is calculated
Also, since this activity needs to be manually triggered, it is possible for it to be empty.
…_sleepData.json
This json file contains data about your sleep, aggregated to the per-session level.
Data included:
- Seconds in each stage of sleep (Awake, Light, Deep, REM)
- Respiratory Rate across sleep session
- “Scores” around quality, duration, and recovery of sleep session
- Number of times your woke during the night
- Garmin-generated insights and feedback based on your sleep session
“Scores” is in quotation marks because it is a proprietary, Garmin-specific metric that is obfuscated on how it is calculated
The feedback and insights columns were unexpected + pretty interesting, because I’m not sure this data is easily available in the Garmin Connect app:
Under the DI-Connect-Userfolder we have 14 files focused on your Garmin profile
While there are a bunch of files, some are not interesting for analysis — 3 are your profile picture of various sizes, one ( ...-social-profile.json) contains the information shown on your Garmin profile, two files ( user_profile.json & user_profile_suica.json) just have your name & email on file, and another ( user_contact.json) has your emergency contact.
A few files may have potential for interesting information, but for me they did not:
user_reminders.jsonwas empty, likely because I do not use remindersuser_settings.jsonwas only 1 line long and marked the HR between each zone. I was using the default (see below)user_biometrics.jsoncould potentially contain biometrics data like height and VO2Max, but everything was Null for me
To quickly check a json file, such as user_settings.json, you can simply do…
file_name = os.getcwd()+'/DI_CONNECT/DI-Connect-User/user_settings.json'
with open(file_name) as file:
j = json.load(file)
json.dumps(j)
…which for me generated:
That leaves us with 3 Files to analyze:
FitnessAgeData.json
This json file contains data about your Fitness Age. Garmin describes Fitness Age as:
Fitness age is an estimate of how fit you are compared to your actual age.
It is calculated by comparing your current VO2 max fitness level to the normal values of people of different ages within your same gender …[as well as your] activity intensity, resting heart rate and body fat percentage or body mass index (BMI)
Data Included:
- Your age, BMI, and Resting Heart Rate
- Total number of vigorous weeks, days, and minutes
- What a healthy BMI, body fat %, and VO2Max would look like for you based on your current weight/height/gender
- Your VO2Max
- Your “Age” in terms of overall health, bmi, and body fat %
“Age” is in quotation marks because it is a proprietary, Garmin-specific metric that is obfuscated on how it is calculated
The readings are not necessarily everyday as I hoped (as you can see a large gap in dates from the above screenshot), but certainly is an interesting dataset to explore over longer timescales.
HydrationLogFile….json
So I do not currently use the Hydration Widget on my watch, so my data is a little boring/sparse. It does make an entry whenever you use an activity since it estimates sweat loss.
However, if you do, here is what you can expect the data to contain:
- Time of Reading
- Hydration source + activity ID
- Amount of hydration, duration of activity
- Estimated amount of sweat loss during activity
UDSFile_….json
The UDS File, which I’m assuming stands for “User Daily Summary”, contains daily aggregated of steps, stairs, stress, and bodyBattery
The Data in this file includes:
- Daily Time & Distance features like
totalDistanceMeters - HeartRate features
- Stress Features like
awakeAverageStressLevel - Body Battery Features like
maxBodyBattery - Additional day-aggregate Respiration and SPO2 Features
These files all relate to discreet physical activity you engage in with your Garmin.
Unfortunately, I do not use my Garmin to set goals or plans, so most of these files are empty for me.
This leaves only one file with data:
…summarizedActivities.json
As the name implies, this tables contains the summaries of the Activities you engage in while wearing your Garmin. Like Health Snapshot, this is not a passive form of data collection as you need to manually tell your watch you are exercising.
Nevertheless, you can generate following data, depending on what activity you engage in:
- Activity Name & Start time
- Intensity Minutes
- Elevation Gains/Loss, and distance travelled, and location
- RespirationRate stats
- Total sets and reps
Like most of the Fitness files, I do not golf or use my watch for Golf, so these files are empty. Maybe in the future I’ll pick up the sport and start collecting some data on the subject!
The final data treasure trove comes in the form of a .zip file, rather than a json as we have seen in the past.
I have written about how to read this zip file previously, but this was a somewhat manual process (or required making a macro). Since then I’ve built a more programmatic way to address this tranformation requirement:
I know I should use os.subprocess but this was the simplest and most interpretable way to port over my previously manual process
As mentioned previously, I combined the processing steps required into a git repo!
- Cloning a Git Repo
Simply open Terminal and type in:
gh repo clone adam1brownell/garmin_data
That will download the code I’ve written.
The navigate into the directory via
cd adam1brownell/garmin_data
2. Accessing Garmin Connect Data
Assuming you have downloaded the Garmin Connect data locally and FitCSVJar file as detailed in my previous post, all you’ll need to do to process your files is run the following command:
python parse_garmin_connect.py [connect/data/folder/location] [FitCSV/file/location]
replacing [connect/data/folder/location] and [FitCSV/file/location] with the appropriate file locations. Tip: On Mac you can drag & drop files into Terminal to autmatically copy over the file location
It should look something like this:
The above command may take a while, as it is processing the fit files.
But once it is done, you should have several csv files of your Garmin data, easily accessible and ready to be analyzed!!
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.