callysto / hackathon

Hackathon Challenges and Resources

Hackathon Challenges and Resources

These are notebooks and other resources to use as challenges for Callysto hackathons or related events. You're also welcome to use them for other non-commercial purposes.

Notebooks in this branch were tested on Callysto Hub.

Callysto Hub basics

Callysto Hub is a free Jupyter notebook environment provided by Cybera and PIMS.

To log in to Callysto Hub you can use a Google or Microsoft account.

The first time you log in your personal environment will be created where all your work will be stored.

Copy notebooks to Callysto Hub

Hackathon notebooks can be automatically copied to your Callysto Hub environment using this git-pull link.

Or to copy them manually:

• Select New -> Terminal in the top right corner.
• In the terminal window type git clone https://github.com/callysto/hackathon -b jupyter
• Close the web page with terminal once the repository has been copied.

Open Notebooks in Callysto Hub

After you have copied the hackathon notebooks, to open them in Callysto Hub:

• Click on the logo in the top left corner, it will redirect you to the home screen.
• Click on hackathon repository folder and select the folder or notebook you are planning to work on.

• To get started select the Prep_materials folder and work through the prepatory notebooks in this order:

1. Markdown basics.ipynb
2. Python and pandas basics solutions.ipynb
3. Matplotlib basics.ipynb
4. Cufflinks basics.ipynb

Some Jupyter Notebook Basics

Run a cell

To run a cell, select it and then either:

• click the Cell menu and select Run Cells
• or hit Shift-Enter
• or click the Run button in the toolbar

Create a new cell

In order to create a new cell:

• click the Insert menu then select Insert Cell Above or Insert Cell Below
• or click the plus button in the top left corner

Delete a cell

In order to delete a cell, select it and either:

• click the Edit menu then select Delete Cells
• or click on scisors button in the top left corner

Undo deleting cells

If you accidentally deleted an incorrect cell, you can always bring it back:

• click the Edit menu then select Undo Delete Cells

Download notebook

If you want to download the notebook in .ipynb format to your local machine:

• Save the notebook first:
  • click the Save and checkpoint button in the top left corner
• Click the File menu then select Download as -> Notebook(.ipynb)

hackathon's Issues

Basics of Python: Fixes and Enhancements

https://github.com/callysto/hackathon/blob/master/PrepMaterials/python-and-pandas-basics.ipynb

• Like the step 2 from the 'Start Here Tutorials' This step 3 links on the website needs link the the updated notebook

• The link notebook should have saved cleared outputs

• The accompanying instructional video needs to be inserted in the notebook. Even further, it should be included in each instructional pieces of the video and the video should start at the particular part in the notebook. Similar to the previous notebook.

• With all notebooks with lots of comments include a "quick python clear comment tip" On a Mac- command "?", not sure 🤷‍♀️. This should be included in this notebook because it requires the user to change the codes.

• Exercise 1- include a input command that provides a prompt for the user
  

• Basics of DataFrames and pandas- Here are instructions for basic instructions for Dataframes and pandas, it would be nice for the user to input the required information in the 'Select colums by name" instead of it provided to them.

• Exercise 2-again suggest to use the 'command and ?' to clear the comment on the line

• Under the section 'Select data from colums by condition" Fix spelling mistake here

• Remove the comment from this part
  

• Exercise 4- 'Calculating number of rows by group' notebook does not match what is in the instructional video

Breaking code and breaking down the code required fixes

https://github.com/callysto/curiosity-hackathon/blob/main/learning-code-soccer.ipynb

This notebook is from this page https://callysto-hackathon.netlify.app/chapter4#1. "Start Here" step 2. It is a very engaging and helpful notebook. It provides explicit instructions and gets the user to troubleshoot. The video flows nicely with the notebook. In saying that, it requires some python fixes by the user.

• The link on the webpage does not go to the current "Fixed" notebook. https://callysto-hackathon.netlify.app/chapter4#1
• Make sure the saved notebook has the outputs cleared, as the current saved version does not.

Deploy Jupyter Notebooks Offline

One of the challenges Callysto faces is supporting students who don't have access to internet.

Our platform is currently accessible provided the student has internet.

Why not install locally? Well, part of the point of having an online platform was to remove the need to install Anaconda, Python3, and the right version of all the dependencies we use in our notebooks.

The online Callysto Hub is ready to launch all of our notebooks https://callysto.ca/learning_modules/

But what if a student has limited to no access to internet?

The challenge is proposing different technologies that:

1. Ease as much as possible the need to install software and dependencies

2. Are deployable offline

3. Are compatible with Windows, MacOS and Linux/Ubuntu operating systems.

A few solutions include:

• Setting up a Docker container (and packaging in a user-friendly manner)
• Packaging everything into a tar file that can be saved on a USB
• Developing an executable script that allows for easy installation (command line tool, or user interface). The executable file installs Python3, Jupyter notebook, clones our repositories, and installs the adequate versions for dependencies used in our notebooks.

Your task is helping us identify technologies, scripts, and open source tools that would allow us to provide our material to students who have limited to no access to internet.

Create a google doc (and share a link in the comments) with ideas, projects that tackle this issue, initiatives, etc.

If you have code or collaborate with someone who does this, please point us to the GitHub repo along with a link to any licenses used. You are welcome to collaborate with us on a solution from scratch. Please indicate if this is something you are interested in.

Explore Callysto notebooks

Click on any of the following links to explore some of the notebooks, user interfaces and data narratives we have developed. You will need a Google or Microsoft account to log in.

Data Science Course - Links to notebooks

Module: What is Data Dcience?

The Olympics: An introduction to manipulating data using Jupyter notebooks

Exploring Gapminder data with Jupyter Notebooks

Module: Introduction to Data Visualization using Jupyter Notebooks

Introductory data visualization

Choosing the right type of visualization for your data

Accessible colour schemes in data visualization

Resources for data visualizations (using Vancouver Open Data)

Speed and power

Higher Dimension Visualization

Module: Interpreting Data Visualization Results

Multiple visualizations

Types of correlations

Exponential growth

Population over time

Final Project Template

Final Project Wiki

Sample Projects: Stats Can Notebooks

Statistics Canada https://www.statcan.gc.ca/

Python api for Statistics Canada New Data Model (NDM) https://github.com/ianepreston/stats_can

Birth, death and marriage (1946 - 2019) (hidden code notebook)

Labour force characteristics by Indigenous group and educational attainment (2007 - 2019) (open code notebook)

Participation in outdoor activities

Develop a user interface using Python to display multiple visualizations

If the dataset you are working on has lots of different values, or there are multiple ways of visualizing the data that result in interesting insights, create a user interface (you can use bokeh or ipywidgets, or other Python libraries) to allow others to easily re-create those visualizations.

Testing repo permissions, comments and contributor workflow.

Getting member outside organization to submit new content to repo.

Provide your name, GitHub ID and affiliation

We will give you credits in a README file [coming soon], where we will add:

1. Your name
2. Affiliation
3. Your contribution

Add the information above in the comment section within this issue. Include a short blurb (3 - 5 sentences) on what you worked on during the sprint.

Fork and create a pull request for your work

Fork the https://github.com/callysto/hackathon repository to create a copy in your profile. From your copy you can add, remove or modify content. Documentation on how to fork a repository.

In your copy of the repository, add a folder with an appropriate title.

For example, if you are working on a topic on climate change, such as CO2 emissions, your folder can be named

climate-change-co2/

Within that directory, create a directory scripts/ for your python scripts, a directory images/ for any pictures or media you want to attach, and a directory data/ for any datasets you want to store locally.

Save your notebooks under climate-change-co2/ (resp the name you gave your directory).

Summary of directory structure:

hackathon/
-|your-topic-here/
-|-| data/
-|-| images/
-|-| scripts/

Create a pull request to main branch in https://github.com/callysto/hackathon once you are ready to submit for review. Documentation on how to create a pull request.

Create issues and issue of issues.

Work on JupyterCon issues.

Develop a Python script that can obtain the data online

Once you have identified and selected a dataset, work to develop a Python script that will download the data into a Jupyter notebook.

You can put into practice web scrapping and API skills, or you can import if the data is readily available in the form of a CSV, TSV, JSON or EXCEL file.

A sample video is provided here.

Important

Ensure you add the following information at the top of your script

#Author:
#Created on:
"""
Summary of script
"""

Comment your code, our Callysto team will work with your code after the JupyterCon2020 sprint.

Issue of Issues

Welcome to JupyterCon2020 Callysto Sprint Session!

Summary

In this issue you can find links to all tasks.

Project management tool https://github.com/callysto/hackathon/projects/1

Team lead @lgfunderburk
Maintainer @byrcyb
Mentors @misterhay @mlamoureux @BryceHaley

If you have any questions about the project, email [email protected].

Sprint Summary

Click the video below to learn more about our sprint:

Templates, resources and sample notebooks

If you are seeking inspiration, look here.

Issue Directory

Grassroots contributor

Pick this option if you'd prefer to work on a notebook from scratch.

1. Select a topic you like from this list.

2. Identify open data sets relevant to the topic you found.

3. Develop a Python script that can download and load the data into a Jupyter notebook.

4. Develop a Jupyter notebook exploring the dataset, and questions you are interested in.

5. Create user interfaces that allow users to easily create multiple visualizations.

Remember to incorporate your work into this repository.

Let us know how you prefer we give you credit.

Data Ambassador

Pick this task if you'd prefer to focus on bridging the gap between members with a technical and a non-technical background. The task can either be focused on language (interpretation of results), or infrastructure (deploying a set of notebooks offline).

The main tasks involved in being a Data Ambassador:

1. Help us identify solutions that allow offline access to Jupyter notebooks.

2. Help us create YouTube videos and content whose focus is on interpreting data. Notebooks provided.

Remember to incorporate your work into this repository.

Let us know how you prefer we give you credit.

Perfectionist

Pick this option if you'd prefer to focus on perfecting specific aspects of an existing notebook that visualizes data. These aspects include, but are not limited to commenting on visualizations, changing wording within notebooks, improving functionality of notebooks.

Here are the main repositories where we keep our work:

Notebooks on sustainability https://github.com/callysto/math-sustainability-case-studies

Lesson plans https://github.com/callysto/lesson-plans

Curriculum notebooks https://github.com/callysto/curriculum-notebooks

Hackathon notebooks https://github.com/callysto/hackathon

Please fork the repository (or repositories), and create a pull request to propose your changes.

Let us know how you prefer we give you credit.

Select one topic you are interested in from the following categories

JupyterCon2020 Topics

The goal of this issue is to help you identify a topic you are interested in. Once you have selected the topic, post a comment

Climate change and data

Possible topics can include:

• CO2 emissions rate of change for the past 1000 years
• Studying trends in temperature (specific locations, or globally)
• Trends/changes in human behaviour (for example, pre and post industrial revolution)

Ocean monitoring

Possible topics can include:

• Impact of plastics (and microplastics) on marine wildlife
• Impact of industrial fishing practices
• Changes in population dynamics of a given species (you can choose the species you are interested in)

Democracy, voter action and participation, engagement of voters

Possible topics can include:

• Demographics of voters, involvement trends
• Simplify public data on budget and expenses, support understanding of what these quantities mean
• Public involvement in policy

Socio economic status and life quality

Possible topics can include:

• COVID-19 impact in marginalized groups
• Study the relationship between income and educational attainment
• Study the relationship between income and health outcomes
• Identify differences in community access to services like education and internet across Canada

Develop a Jupyter notebook exploring your dataset

Create a Jupyter notebook that explores the dataset.

You can use any of the following notebook templates as inspiration:

Project Template

Final Project Wiki

Sample Projects: Stats Can Notebooks

Statistics Canada https://www.statcan.gc.ca/

Python api for Statistics Canada New Data Model (NDM) https://github.com/ianepreston/stats_can

Birth, death and marriage (1946 - 2019) (hidden code notebook)

Labour force characteristics by Indigenous group and educational attainment (2007 - 2019) (open code notebook)

Participation in outdoor activities

How to interpret data?

The task is to create either YouTube, audio or written content whose focus is on how to interpret graphs.

You can use any of the following resources to help you guide your content. You will need a Google or Microsoft account to access any of our Jupyter notebooks.

Callysto Jupyter Notebooks

• Visualizing COVID-19 total number of cases around the world: Click here to launch

• Labour force characteristics by Indigenous group and educational attainment (2007 - 2019): Click here to launch

• COVID-19 math model Click here to launch

From JupyterCon2020

Covid19 Dashboard https://covid19dashboards.com/

Statistics Canada and Gapminder

• Statistics Canada, Physical flow account for greenhouse gas emissions: Interactive tool https://www150.statcan.gc.ca/n1/pub/71-607-x/71-607-x2020008-eng.htm

• Gapminder offline tools https://www.gapminder.org/tools-offline/

The goal is to use the graphs generated in the notebooks and resources above to guide students through the process of reading and interpreting a graph.

Suggested outline:

1. What are graphs

2. How can we use graphs

3. What does it mean to interpret a graph

4. Example of interpreting a graph (use any of the resources above)

Target audience: students in grades 7 and above (see here for details on the grade system in Canada).

Recommendations: avoid jargon as much as possible, use simple words. Assume people have little to no technical or domain expertise.

Identify open data sources

Pick a dataset associated to the topic you are interested in.

We have compiled a list to get you started. (See table below).

You are welcome to suggest a different dataset. If you are working with a different dataset, please include a link to the dataset in the comments, along with a license to the dataset. Please ensure you select open datasets.

Data set compilation as of Oct 16 2020 (adding to the table as people comment here).

Main Topic	Subtopic	Link to data	License
Climate change and data	CO2 emissions	World Bank climate change data	open data
Climate change and data	Trends in temperatures	monthly temperature readings at canadian sites	Open Government Licence - Canada
Climate change and data	Trends in human behaviour (pre and post industrial revolution	Hyde site data[1]	creative commons 3.0
Ocean monitoring	Impact of (micro)plastics on marine wildlife	Hudson River Herring Stomach Contents[2]	creative commons 4.0
Ocean monitoring	Impact of industrial fishing practices on marine wildlife	Alaska Sea Shelf Groundfish Trawl Survey	open for non-commercial, scientific and educational uses
Ocean monitoring	Changes in population size, behavioural patterns of a given species	grey reef shark movements	open - but must inform OTN of data products created
Citizen participation	Demographics of voters, involvement trends	Electoral Districts for the 2015 AB election and 2016 Census information	Open Government Licence - Alberta
Citizen participation	Visualizing and interpreting data on budget and expenses	Federal Budget Data Sets	Open Government Licence - Canada
Citizen participation	Public involvement in policy	2018 BC Election Referendum Results	Elections BC Open Data Licence
Socio economic status and life quality	COVID-19 impact in marginalized groups	Canadian Confirmed Case details and Perspectives on COVID-19: Impacts on underrepresented populations	Open Government Licence - Canada
Socio economic status and life quality	Income and educational attainment	Labour market outcomes of post secondary graduates	Open Government Licence - Canada
Socio economic status and life quality	Income and health outcomes	World Bank Open Data	Open Data
Socio economic status and life quality	Geographic distribution of access to services like education and internet in Canada	National Broadband Data	Open Government Licence - Canada

[1]: There is an incredible amount of datasets available at this link. This global summary and table of codes. If you decide to use the data sorted by country then be sure to reference these ISO codes.
[2]: The data presented is a bit messy and in .xlxs format. The raw data can be found here as a csv. But the link in the above table will explain the codes.

Curiosity Hackathon: Challenge 4 bug fix

https://github.com/callysto/hackathon/blob/master/SustainabilityOnMars/CuriosityTrack/challenge-4.ipynb

• i.challenge4d() it is buggy. For example, if the user puts in all the Basic Statistics terms and the cell will still prompt to use .describe() again. Or you answer "n" and it still prompts you to put in another Basic Stats term. or sometimes the cells ends and i.challenge4e() does not run.

Books Introduction: Alice's Adventures in Wonderland Sugguestions

https://github.com/callysto/hackathon/blob/notebooks-update/HackathonNotebooks/Books/books-intro.ipynb

WOW great start! It is an inspiration for other hackathon content. Here are a few suggestions for the this first notebook.

• Chapter Lengths Visualization- Maybe prompt the them to see to the comparison of character count to a story structure. Does it look similar, why or why not. Like this one? Character count vs word count , character count works better with this correlation. https://www.researchgate.net/profile/Sara-Elshafie/publication/326720118/figure/fig1/AS:926468729208833@1597898766646/Freytags-pyramid-also-known-as-the-dramatic-arc-showing-a-five-part-story.png
• Adjective, verbs, nouns and proper nouns. It would be nice to see a quick visual on the definitions. Rather than just a link. I created this if you want to share it. https://drive.google.com/drive/folders/1rCUHnGi5Lq-tjqh4bK4eRuqUYVHSFM5P?usp=sharing