btroia / basis-data-export

Utility that exports and saves your Basis B1 or Peak device's uploaded sensor data.

Basis Data Export

Utility that exports and saves your Basis Health Tracker (B1/Peak) uploaded sensor data, as described at https://www.quantifiedbob.com/2013/04/liberating-your-data-from-the-basis-b1-band/ You can learn more about Basis at http://www.mybasis.com/

UPDATE JANUARY 1, 2017 -

RIP Basis - Intel has officially shut down Basis, including access to any historical data (users had until December 31, 2016 to export data).

UPDATE AUGUST 3, 2016 -

Intel (who acquired Basis) has notified users that they will be discontinuing all Basis devices and services on December 31, 2016. Make sure to download all of your data before then! See http://www.mybasis.com/safety/ to learn more.

Instructions

In order to use this script, you must already have a Basis account (and a Basis B1 band).

Usage:

This script can be run several ways. You can (and should probably) first edit the BASIS_USERNAME, BASIS_PASSWORD, and BASIS_EXPORT_FORMAT values under "Settings" in the file basisdataexport.php so you don't have to specify those values every time the script is run. Make sure the data/ folder is writable!

///////////////////////////////////////////////////////
// Settings
///////////////////////////////////////////////////////

// Specify your Basis username, password, and default export format. Leaving blank 
// will require inputting these values manually each time the script is run.
define('BASIS_USERNAME', '[YOUR BASIS USERNAME]');
define('BASIS_PASSWORD', '[YOUR BASIS PASSWORD');
define('BASIS_EXPORT_FORMAT', 'json');

// Enable/disable debug mode
define('DEBUG', false);

Method 1 - Interactive Mode

$ php basisdataexport.php
-------------------------
Basis data export script.
-------------------------
Enter Basis username [[email protected]]: [email protected]
Enter Basis password [********]: 
Enter data export date (YYYY-MM-DD) [2016-08-22] : 
Enter export format (json|csv|html) [json] : 

1. Open a terminal window and cd to this script's directory
2. Type php basisdataexport.php
3. Follow the prompts (hit ENTER to use default values)
4. Your data will be saved to /data/basis-data-[YYYY-MM-DD].[format]

Method 2 - Via command-line arguments (useful for crons)

$ php basisdataexport.php [email protected] -pMyBasisPassword -d2016-08-22 -fcsv

Usage php basisdataexport.php -h -u[username] -p[pass] -d[YYYY-MM-DD] -f[json|csv|html]

Options:
  -u  Basis username (if not used, defaults to BASIS_USERNAME)
  -p  Basis password (if not used, defaults to BASIS_PASSWORD)
  -d  Data export date (YYYY-MM-DD) (if not used, defaults to current date)
  -f  Data export format (json|csv|html) (if not used, defaults to json)
  -h  Show this help text

Make sure there are no spaces between any flags and values!

Method 3 - Via web browser

This requires that the scripts are in a location that is executable via a web server.

http://localhost/basis-data-export/basisdataexport.php?u=[basis_username]&p=[basis_password]&d=[YYYY-MM-DD]&f=[format]

Saving Your Data

If the script runs successfully, your data will be saved in the data/ folder. Files are saved in the format basis-data-[YYYY-MM-DD].[format] (i.e., basis-data-2014-04-04.json).

That's it! (for now).

Biometric Data

Basis currently returns the following biometric data points. They each represent an average (i.e., for heart rate, GSR, skin/air temperature) or sum (i.e., steps, calories) over the previous 1-minute period:

• Time - time reading was taken
• Heart Rate - beats per minute
• Steps - number of steps taken
• Calories - number of calories burned
• GSR - Galvanic skin response (i.e., sweat/skin conductivity. Learn more about GSR here - http://en.wikipedia.org/wiki/Skin_conductance)
• Skin Temperature - skin temperature (degrees F)
• Air Temperature - air temperatute (degrees F)

There are some other aggregate metrics included in the reponse such as min/max/average/standard deviation metrics for each set of data.

Sleep Data

Basis currently returns the following sleep data points (it's in a different format than the metrics data because it's a newer version of their API):

• Start Time - local timestamp in MM/DD/YY HH:MM format
• Start Time ISO - user timestamp in ISO format
• Start Time Time Zone - user time zone, i.e., America/New_York
• Start Time Offset - minutes offset from GMT (i.e., -240 for America/New_York)
• End Time - user timestamp in MM/DD/YY HH:MM format
• End Time ISO - user timestamp in ISO format
• End Time Time Zone - user time zone, i.e., America/New_York
• End Time Offset - minutes offset from GMT (i.e., -240 for America/New_York)
• Light Minutes - number of minutes in "light sleep" stage
• Deep Minutes - number of minutes in "deep sleep" stage
• REM Minutes - number of minutes in "REM sleep" stage
• Interruption Minutes - number of minutes interrupted (i.e., woke up, went to bathroom, etc.)
• Unknown Minutes - number of minutes unknown (i.e., device wasn't able to take readings)
• Tosses and Turns - number of tosses and turns that occurred
• Type - always returns 'sleep'
• Actual Seconds - total sleep time recorded, in seconds
• Calories - number of calories burned
• Average Heart Rate - beats per minute
• Minimum Heart Rate - beats per minute (currently this always returns null)
• Maximum Heart Rate - beats per minute (currently this always returns null)
• State - returns 'complete' if event was completed at time of export
• Version - API version used (i.e., 2)
• ID - internal id for given sleep event

Sleep data is often broken into multiple segments - Basis treats each sleep activity as a separate 'event' if there is a 15-minute gap in readings (i.e., sensors couldn't detect anything).

Activity Data

Similar to sleep data, Basis currently returns the following activity data points (walking, biking, running):

• Start Time - local timestamp in MM/DD/YY HH:MM format
• Start Time ISO - user timestamp in ISO format
• Start Time Time Zone - user time zone, i.e., America/New_York
• Start Time Offset - minutes offset from GMT (i.e., -240 for America/New_York)
• End Time - user timestamp in MM/DD/YY HH:MM format
• End Time ISO - user timestamp in ISO format
• End Time Time Zone - user time zone, i.e., America/New_York
• End Time Offset - minutes offset from GMT (i.e., -240 for America/New_York)
• Type - activity type (i.e., walk, run, bike)
• Actual Seconds - total activity time, in seconds
• Steps - number of steps taken
• Calories - number of calories burned
• Minutes - actual number of minutes activity was performed (excludes time not moving?)
• Average Heart Rate - beats per minute (currently this always returns null)
• Minimum Heart Rate - beats per minute (currently this always returns null)
• Maximum Heart Rate - beats per minute (currently this always returns null)
• State - returns 'complete' if event was completed at time of export
• Version - API version used (i.e., 2)
• ID - internal id for given sleep event

Tips

• You can set up a cron to run once per day to automatically grab your previous day's data (assuming you are syncing your device each day)

• If you want to archive data across a date range you can use curl's [ ] syntax to do it easily (thanks to @Edrabbit for the tip!). For example, to get all of May cached in /data:

  curl http://localhost/basis-data-export/basisdataexport.php?d=2013-05-[01-31]

Special Tip for Windows Users and Setting Up SSL/cURL

(Thanks to @joshuagarity for the tip!) If you haven't already, you will need to install an SSL certificate on your system in order for the script to work (because the user authentication only works over SSL, and is required by cURL). Here's how:

• Download a certificate file at http://curl.haxx.se/ca/cacert.pem and save it somewhere on your computer

• Run update php.ini -- add curl.cainfo = "PATH_TO/cacert.pem" where "PATH_TO/cacert.pem" is the location of the certificate file you just downloaded. If you are running XAMPP, you would save the cacert.pem to c:\xampp\cacert.pem. Then, open XAMPP, click on Config, then selected PHP.ini. On the second line of the file paste the following:

  curl.cainfo = "C:\xampp\cacert.pem"

• Lastly, save the file and you should be able to run the script.

basis-data-export's Issues

Unable to use Method 2 - Via command-line arguments

Interactive mode works fine, but when I use method 2 with the same credentials and date, I receive the following error:

Caught exception: ERROR: Unable to login! Check your username and password.
Caught exception: ERROR: Unable to login! Check your username and password.
Caught exception: ERROR: Unable to login! Check your username and password.

I'm using the following command:

php basisdataexport.php [email protected] -p*********** -d2015-03-01 -fjson

No Advanced Sleep Analysis data

Doesn't capture the Advanced Sleep Analysis data... i.e
http://www.mybasis.com/wp-content/uploads/2014/01/Basis_Web_Sleep_Dashboard.png

But otherwise still works.

no data in exported files

Thanks for writing this code!

I have successfully used this script a few months ago. I'm running it again now, and all the exported data files are empty; the html and csv files are blank, and the json file just says:

{"error_description": ["start", "end"], "description": ["start", "end"], "error": "missing_param"}

I double-checked everything, and I am using a valid login, and there is data on the Basis website for the dates I'm trying. Could it be that Basis updated the website and this no longer works? Or can you give me any ideas on how to debug this?

Thanks!

Multiple days exporting data

Hello!
How do you export data from multiple days, e.g. 2015-06-23 to 2015-06-27 ?
Thank you!

Timestamp field is blank

Thanks for this great work!
After getting the certs in place, I was able to run the scripts and download the data successfully. However, my timestamp column is blank. Did anyone else notice this too?

preg_match error

Heey,

Thank you for creating this tool. I am trying to download my data but this errors keeps popping up:
Warning: preg_match() expects parameter 2 to be string, resource given in C:\xampp\htdocs\xampp\BasisExport.class.php on line 88
Caught exception: ERROR: Unable to login! Check your username and password.

I did check my username and password and they are correct. It seems the $result parameter is not a string? Is there anything I can do solve this problem?

Thanks,

Jorrit

Login problems

I've uploaded the files to my webserver since I don't have a localhost running on this machine and am not well-versed in terminal usage. When I go to

http://www.mayodev.com/Basis/basisdataexport.php?date=2014-05-[01-31]

I get the following error:

Caught exception: ERROR: Unable to login! Check your username and password.

I have it configured to ask for username and password each time, but it goes straight here without asking. I also tried it with the username and password hard coded into the php file, but got the same result.

As a note, I did have to add the following line to the class file to avoid an SSL error (see below):
CURLOPT_SSL_VERIFYPEER => false,

No idea if that is part of the problem or not.

My Basis is the Carbon Steel edition
Chrome Browser (latest)

Any help is appreciated!

Caught exception: ERROR: cURL error - SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

broken authentication as of 2014.01.08 pls help

Around 1/8 or so, the basis website seems to require some sort of authentication.

The $basis_userid seems the same when fished out using Chrome's inspect element, but the php script can no longer pull, it gets a 401 access denied.

/opt/local/bin/lynx --dump 'http://127.0.0.1/~raydawg/basis-data-export/basisdataexport.php'
Warning:
file_get_contents(https://app.mybasis.com/api/v1/chart/52a659b775ec5351
23c68041.json?summary=true&interval=60&units=s&start_date=2014-01-12&st
art_offset=0&end_offset=0&heartrate=true&steps=true&calories=true&gsr=t
rue&skin_temp=true&air_temp=true&bodystates=true)
[[1]function.file-get-contents]: failed to open stream: HTTP request
failed! HTTP/1.1 401 Unauthorized in
/Users/raydawg/Sites/basis-data-export/basisdataexport.php on
line 74
...

74 if(!$basisdata = file_get_contents($dataurl)) {
75 if ($debug) {
76 echo 'Error retrieving data: ' . $dataurl;
77 }
78 } else {

I also had to add these since the PHP included in Mac OS X was complaining about the time zone and memory limit, though they're unrelated:

17 ini_set('memory_limit', '-1'); // -1 here implies no limit, you can even set it to a bigger number like '192M' for 192Mb space.
18 date_default_timezone_set ('America/New_York');
19
20
21 $basis_userid = '52xxxxxxxxxxxxxxxxxxxxxx';

(actual ID replaced by x's to preserve privacy)

Sleep tracking data?

Any chance that you'll be able to implement the sleep tracking data to download that information?

Suggest including the following bash script for automation:

#!/bin/sh

#
now=`date +"%Y-%m-%d" -d $1`
end=`date +"%Y-%m-%d" -d $2`
while [ "$now" != "$end" ] ; 
do 
        echo "Downloading: "$now
        php basisdataexport.php -u[username] -p[password] -d$now -f[format]
        now=`date +"%Y-%m-%d" -d "$now + 1 day"`; 
done

Just a suggestion, that way it is easy to export range of dates by doing
./test.sh 2014-10-07 2014-10-09

:)

Parse Error

Hey,

I really love this tool, but was welcomed with this message when I ran 'php basisdataexport.php' running the latest version of OS X 10.9 Mavericks.

Parse error: parse error, expecting `":: (T_PAAMAYIM_NEKUDOTAYIM)"' in /Users/angelceballos/Documents/Development/basis-export-tool/basisdataexport.php on line 93

Change to API user ID?

It appears that the API may have changed. I am now seeing a v2 under networks and rather than a static user ID, it appears that a variable "me" is being used.

I'm not sure as I never saw how this previously looked, but I'm unable to find the original XHR that is referred to in the documentation.

I was able to find an ID string inside:

https://app.mybasis.com/api/v2/users/me/days/2014-02-14/activities

Outstanding

This is fantastic. Since the raw data is pretty much the main thing I'm looking for in a wearable, I might buy the Basis B1 because of this script. My only question is whether you think it will be reasonable for someone like me (who knows very little about programming) to execute it. It looks like I can either copy/paste your code into a Linux client on a virtual machine or put it somewhere my browser can access (which I assume includes somewhere locally) and do it through that route. Does that sound right? If so, I think I can handle that. Is there anything else I should know?
Thanks!!

Blank Data File

Hi,

First off, thanks for doing this! Much appreciated. I hope I can contribute something.

I downloaded this yesterday and had an error with the digital certificate where it would give me an ERROR: cURL error - SSL certificate problem: self signed certificate in certificate chain.

I fixed that by downloading CA certs to check against and adding a few lines to the CURL_SETOPT_ARRAY starting on line 68 in BasisExport.class.php

CURLOPT_SSL_VERIFYPEER => true,
CURLOPT_SSL_VERIFYHOST => 2,
CURLOPT_CAINFO => 'CABundle.crt',

Then, I didn't get the error. However, instead of downloading data, I just get the header row in CSV, HTML, With JSON I get ERROR: could not save data to file and a 0KB data file. Any idea what is going on?

-etavares