nickolasrm / moodlestudentdatascraper

A scraper that extracts student data from moodle!

Hi there, I'm Nickolas!

Social media

   

Who am I?

I'm a person who loves making people's lives easier through IT and data.
Some facts about me:

• ❤️ I'm in love with open source world, specially for data science related stuff
• ♿ I have experience with digital accessibility. Something many people forget about.
• 🎮 Off-topic: I love old-school games. I often play games that made a mark in the game industry.

Languages and tools

SWE

MLEng/DS

Dev

My stats

My best projects

Organizations I belong to

ProjetaAi Ipiranga's MLOps team	CITEI A digital accessibility research group
ATISLabs An HPC and ML research group	My UFRRJ Projects Projects I made for university
Nickoff-Devspace My YouTube channel projects	nickolasrm-Learn Place I use to learn new technologies

moodlestudentdatascraper's Issues

data to scrape

• assignment finishing date

• email

Fix readmel

Add missing information to readme and replace driver section text to chromedriver

config

• create a config file where the user can put needed information to scrape (username, password, course...)

Readme errors

• chromedriver
• find another errors

save data

• write a table file (csv or ods) containing scraped data

• add table header

Note: Pandas is a good way to perform these actions

login

• Make a spider who can perform login into moodle

scrape information

• get participants data

Data to scrape

1. Frequency
2. Grades

modular scraping

• add a section in data.cfg containing boolean fields for selecting whether a data type should be extracted

• change scraper function return to {'headers': [header-string1, ...], 'data': [ [data1, ...] , ... ]}

• change save function to only work with lists

Remove unused libraries

[] removed