onnela-lab / beiwedata Goto Github PK

Sample both Wi-Fi and GPS; identify unique locations based on Wi-Fi signal patterns; associate physical locations from the GPS with these patterns; explore how frequently need to sample the GPS to know the location of the top 90% of most frequent locations.

In the new app, files are now named [username]_[data stream]_[timestamp].csv instead of just [data stream]_[timestamp].csv. This likely breaks a lot of the functions that parsed names based on _. Just a warning. Will fix this weekend.

Right now each plotting function has some sort of axis manipulation that is "best fit" for my situation. At some point, we should make this generic so that it will work in 95% of all situations and a separate function set will help ease the axis manipulation.

Perhaps I can try to merge some of the data generation stuff from cdrhelper to generate more call and text data. This will make plotting and manipulating the data seem more realistic and may reveal issues we haven't found due to sparsity.

Go back and fix this later. Every plotting function should create a new fig and return fig, axes. Remove saving function from plotting functions -- or think of a better way to integrate it so it can be used in headless scripting mode.

For WiFi and Bluetooth "most frequent" plots, add an option to add an "all others" row. That way you can tell periods where there were no devices around. Should be a simple line fix with some y-axis relabeling moving around.

Worst case, make a separate function.

New plots should show number of unique corresponders and cumulative calls/texts throughout the the day.

Perhaps four colors representing quantiles of signal strength or something similar. Try to not let it be too continuous as literature indicates signal strength is highly dependent on lots of variables and subtle changes are probably not indicative of much.

This would be a hierarchical decision tree, i.e., first check the accelerometer (if movement, then done); then check the GPS (if location changed, then done), ...

Also try different panels for text/calls.

Timestamps are often duplicated or in the wrong order. For example, in my data, there are three rows out of order and 1352 rows that have duplicated time stamps.

In [33]: %paste
a_files = list_data_files('accel', './mkdata')
a_df = import_df(a_files, setindex=False)
len(a_df)
print ((a_df.timestamp.diff() < 0).sum(), (a_df.timestamp.diff() == 0).sum())

## -- End pasted text --
(3, 1352)

Make call text plots so that markers are + with ---- representing call duration. For now, keep missed calls on same line as incoming calls, but use a different marker circumscribed +, for example.