krausefx / fxlifesheet

Tracking the key metrics of my life

FxLifeSheet

---

See the outcome on howisFelix.today

See the resulting work on howisFelix.today

---

---

Goals

To answer questions about my life, like

• How does living in different cities affect other factors like fitness, productivity and happiness?
• How does sleep affect my day, my fitness level, and happiness?
• How does the weather, and the different seasons affect my life?
• Are there any trends over the last few years?
• How does computer time, work and hours in meetings affect my personal life?

Key decisions for this project:

• Use a single database, owned and hosted by me, with all the data I've collected over the years
• Be able to easily add and remove questions on the fly, as I learn what's beneficial to track
• Full control of how the data is visualized
• Works well for frequent flyers with mixed time zones
• 100% fully open source, MIT licensed and self-hosted

---

This repo contains

• The Telegram bot for entering data manually
• The data visualization, check out the results on howisFelix.today
• The database info

---

Questions to answer

There are many questions that can be answered using this data, here are some main ones I want to look into over the next few months

Mood/Happines related:

• How does the city I'm in influence my level of happiness?
• How does the weather/climate influence my mood
• How does travel affect my mood?
• How do certain macro nutrition goals affect my mood?
• What amount of hours of sleep have the best influence on mood?

Fitness related

• How do certain macro goals influence my lean body mass and fat levels?
• How does my daily kcal intake affect my social life?
• How does alcohol affect my training and fitness progress?
• Do more daily steps influence
  • my mood
  • the amount of audio books I'm reading
  • feeling connected with family & friends as often calling them
• What are the areas in life that suffer while cutting (kcal deficit):
  • mood
  • stress
  • productivity
  • energy

Travel

• What are the areas in life that suffer when I travel? Investigate:
  • fitness progress
  • productivity
  • stress
  • health
  • sleep
  • feeling connected with friends

Productivity

• How many hours did I spend on my phone and laptop over the months?
• Do I stick to my yearly book goals and how do other factors influence how much I'm reading?
• Am I more productive when I set the goals for the day the night before?

Implementation

Assumptions

• The user is in random time zones at random times and switches often, therefore the bot can't know about your daily schedule. This puts a lot of focus on averages, as it doesn't matter if a value was entered at 11pm that day, or 8am the next one, the numbers will even out, as only daily, weekly and monthly averages are considered when rendering graphs.

This repo contains the Telegram bot that is responsible for collecting the data.

There are 2 ways to input data: by the user telling the bot to ask for all the values, and by a regular interval of the bot asking you (similar to the deprecated mood bot)

Configuration

lifesheet.json

Available values for schedule:

• eightTimesADay
• daily
• weekly
• manual

User initiates data inputs

Mood

Using fourTimesADay, this will replace the mood bot

• How are you feeling today?

Morning

/awake

This will trigger the morning questions, like:

• Sleep duration
• Sleep quality
• Body weight

Evening

/asleep

This will trigger the end-of-day questions like

• Fitness related:
  • Alcohol intake
  • Macro adherence
  • Hunger issues?
  • Fatigu/Lethargy?
  • Feel stressed?
  • Caffeine intake?
  • How healthy do you feel today?
  • Number of steps
• Productivity related
  • Did I solve actual programming/technical problems?
• Social
  • Felt like enough time by myself?
  • Felt like enough in control of my own time and schedule?
  • Felt like enough socializing?
  • Felt like enough going out, bars, restaurants, dancing etc.
• Personal growth
  • Learned new skills or things?
  • Went out of my comfort zone?
  • Number of minutes of Audible
• Other
  • Meditated
  • Note: what was the main thing I did today?
  • Boolean: Did I set goals for the next day?
  • Do you feel excited about what's ahead in the future?

Week

/week

This will trigger questions that take longer to reply, so they're only done weekly

• Fitness related
  • Current macros
    • g of Carbs
    • g of Protein
    • g of Fat
  • Body measurements
  • Overall training adherence
  • Note: Comments on fitness
• Productivity
  • Overall happiness with life progress of the week, do I go into the right direction?
  • Number of open Trello tasks (from howisFelix.today)
  • Number of emails in Inbox less than 5?
  • Average daily hours on computer
  • Average daily iOS screen time (minus MyFitnessPal and Strong app)
• Social
  • Felt like spent enough time with family?
  • Had deep conversations with close friends?
• Other
  • Did I travel, this includes every city more than 1h away, this is relevant for both fitness and productivity
  • Note of all locations I was at (cities)
  • Got out of my comfort zone & experienced/tried new things?
  • Do you feel like having to travel somewhere?
  • Do you feel like you're missing out on things?
  • Played computer games by myself
  • Played computer games with friends or family?

Telegram

Insert for available commands

skip - Skip a question that was asked
report - Generate one page report
track - Track a specific value only
mood - Track your mood
awake - First thing in the morning
asleep - Right before going to sleep
week - Once per week metrics
skip_all - Remove all queued questions

Development

Running locally

npm run dev

Debugging

After using npm run dev, open chrome://inspect to use the Chrome Dev Tools

Setup

Environment variables

.keys file or however you manage your secret env variables:

export TELEGRAM_BOT_TOKEN=""
export TELEGRAM_USER_ID=""
export TELEGRAM_CHAT_ID=""

export DATABASE_URL=""

export LIFESHEET_JSON_URL=""

Scheduler

Not using Heroku any more, in railway.app, just add this as its own service

npm run scheduler

Postgres

Create a new Postgres database, and run the SQL queries defined in db/create_tables.sql

Data Visualization

To analyze the data, check out the visual_playground subfolder https://github.com/KrauseFx/FxLifeSheet/tree/master/visual_playground

Origins

The original implementation abused Google Sheets as a database, and I used Google Data Studio to visualize the data. Both implementations ended up not working pretty quickly.

fxlifesheet's Issues

Basic validation of config file

Migrate to official client library

Looks like the one is quite limited, probably makes sense to use whatever Google recommends/provides https://developers.google.com/sheets/api/quickstart/nodejs

Stop making use of ctx

Right now, in the code base we use ctx and even cache the ctx object. Let's get rid of this

Fitbit data

Once I got my Fitbit, let's look at what numbers they provide

In particular

• average heart rate
• Exact sleep duration
• Sleep quality

Initial deployment to Zeit

This will require me to fix the credential loading

Fix determining the right city

Looks like the API I'm using doesn't work properly, and uses California instead of San Francisco. Had issues with Vienna too, which lists Vienna as a state

Parse all commands as lower case

This is quite annoying because we use bot.hears

Store seconds since 1970 as a column also

Note: What was the main thing I did today

Add an easy way to skip a question

[later] Track my blood levels from the blood tests over the last years

Implement scheduler

Auth check for user who submits data

Find a way to take permanent notes

Monthly reflections

Currently I write reflections on various topics about once per month. Ideally I can somehow attach them

One-offs

I take fitness notes about starting a cut or a bulk. Ideally I can attach them somehow.

---

The notes are written in Markdown (obviously) and are stored in Bear Notes currently, synced via iCloud (blackbox with no API AFAIK)

There is a solution to be found

Add "Did I set goals for the next day?" binary question

Number of swarm check ins per day

Wait for header message to be sent to avoid race condition

Build initial graphs on Google Sheets

Add command to manually track a value

e.g.

/track employer

Should automatically find the question with the "employer" key and ask & save the value

Measure productivity somehow

Not sure if it's good enough to use the RescueTime or Timing App productivity score. The problem is that the score changes as you change its config.

Also, e.g. none of those apps track the difference between using Telegram to chat with friends, or to build bots

Use the 5 minute journal questions

Go through them and see if they make sense. I'd assume they thought carefully about them

I am grateful for…
What would make today great?
Daily affirmation. I am…
3 Amazing things that happened today…
How could I have made today even better?

via https://www.intelligentchange.com/blogs/news/the-five-minute-journal-questions

Go through TODOs and move them into GitHub issues

Store quarter

Migrate notes from mood bot

I have about 10 notes, manually migrating them should be enough

Decide on range of available values

• Should the range be from 0-5 like the mood bot or 0%-100% like the training notes
• Should the ranges always be defined as 0 being negative (feel like not being happy with this part) and 5 or 100 being positive (happy with this part of life)

Import historic Moves data

I have 7 years of Moves data (thanks FB for ruining this for everyone), let's pipe them over to FxLifeSheet, I guess the most useful thing to take is the city/country I stayed in each day, as well as the number of steps. Before doing this, #39 has to be solved, for reverse geocoding to be fixed

Ideas for graphs

https://www.reddit.com/r/dataisbeautiful/comments/7n8ocd/every_single_hour_of_my_2017_recorded_oc/

Average sleep duration

---

Hours spent on phone, hours spent on laptop

---

Similar visual spreadsheet as this tweet

Via https://twitter.com/ali01/status/1078647580731547648

Allow tracking of pictures (or any other files)

This will be relevant for weekly/monthly fitness progress photos.

The simplest solution would be to just store the Telegram attachment ID. This should still allow us to access those pictures as part of reports. However AFAIK there is no simple public URL we could just link, you'd need a full API client (to check)

So the alternative would be to pipe those pictures over to Google Cloud Storage

Add weekly questions

Provide weekly and monthly summaries via Telegram

• Look into how I can pipe graphs from Google Sheets into Telegram, maybe using a public Google Dashboard in combination with a screenshot that gets sent

Unify auth check

Right now we have the following code various times

    if (ctx.update.message.from.username != process.env.TELEGRAM_USER_ID) {
      return;
    }

We might be able to use a middleware or similar

Add boolean flag option

Questions I want answered with those reports

bear://x-callback-url/open-note?id=852D2A8E-75F1-45D0-804D-A3977918FE43-306-000019EF03F7637F

Think about "influencing factors" for various questions

Ask what influenced the mood positively, negatively, and allow answers like

Friends, Family, Relaxing, Travel, Gym, Weather, Traffic

Each of those can be used as positive or as negative influence. This way you can easily search which of those factors contribute to your mood the most often.

On the other hand, isn't that kind of like asking those factors directly, and rating those, and if they're on a bad rating, then they're a negative influence for your mood.

Will need to think more about this

New rows should use the time stamp of message sent, not server timestamp

As sending might be delayed

Setup proper dev environment

Setup dev environment, 2 bots, one for dev, one for production, production bot should have 0 history, maybe even separate sheet?

Find alternative to Google Data Studio

A few things that I don't like about it:

• Only 3 dimensions at pivot tables
• No way to customize the heatmap for pivot tables, in particular:
  • from and to color, they only support one color
  • No way to have the color be calculated based on the row/column, so what ends up happening is that I can't render boolean values with range values or number values
• Generally seems pretty limited actually
• Sometimes it just resets sorting
• No good way to fetch screenshots/reports via an API, so I can't get them via Telegram. Daily/Weekly email reports are good enough though I guess

Do some basic error handling when user sends invalid inputs to the bots

Add config option to just re-use last value

This is useful for e.g. employer, as it doesn't change often, but we want to add it every month anyway

Set up some kind of automated backups of the Google Sheets

Migrate my existing mood data

The data I already have tracked for over a year via https://github.com/krausefx/mood

Import historic DEXA data

I did 3 DEXA scans, let's track all those values in here also

Parse commands

Separate worker from runner

Right now we have a setInterval loop in app.ts, which is just wrong. Let's separate those.

web: Web server
worker: Telegram reminder + receiving telegram messages

Write down health symptoms

Somehow track stuff like headache

Support decimal points for Number inputs

[Maybe] Make use of force_reply

This is a nice benefit for the user to see what they're replying to I guess. However it looks like it doesn't support custom keyboards in combination, which kind of defeats the purpose telegraf/telegraf#82

Show confirmation when value was uploaded successfully, and include last input

Support custom button texts per value