blu3r4y / jku-room-search

Search for free rooms on the campus of the Johannes Kepler University Linz

JKU Room Search

Search for free rooms on the campus of the Johannes Kepler University Linz.

Use this App

Go to jkuroomsearch.app

Build this App

Install Node.JS >= 16 and Yarn >= 2 and simply enter

yarn
yarn build

During development, start a local webserver and trigger development builds automatically with

yarn serve

If you use your own local webserver, you can instead use

yarn watch

Create the Index

Once installed, simply enter

yarn scrape

On success, you find a index.json file in the root directory that should go to /data/index.json on the webserver.

Contributing

Before committing, format and lint the code with

yarn format
yarn lint

As a shortcut, you can run both with

yarn fix

You can mimic the checks on the pipeline with

yarn gate

jku-room-search's Issues

Filter by buildings

As a user, I want to fine-tune my search to only a few buildings, i.e. with a drop-down where I can select buildings I am interested in.

This issue depends on #5.

Add loading animation when the results are updated

To indicate that the results section actually changed shortly fade the transparency of that section.

Room capacities are not extracted automatically anymore

It seems this page changed: https://www.jku.at/en/campus/the-jku-campus/buildings/

The log indicates that we already fail to extract the names.

-------------------------------------

GET 200 https://www.jku.at/en/campus/the-jku-campus/buildings/

BUILDING [10](https://github.com/blu3r4y/jku-room-search/actions/runs/3926796283/jobs/6712846272#step:9:11)0% scraped 0 building names
[]

-------------------------------------

Buildings are not scraped anymore

Something did change on https://www.jku.at/en/campus/the-jku-campus/buildings/ so that buildings are not scraped anymore.

GET 200 https://www.jku.at/en/campus/the-jku-campus/buildings/

BUILDING 100% scraped 0 building names
[]

Avoid performing duplicate queries while scraping

Either introduce a cache or first gather all room bookings before querying them.

Upgrade to Webpack 5

https://webpack.js.org/blog/2020-10-10-webpack-5-release/

Re-work entire UI with timeline and building groups

• Parse what rooms belong to which buildings from https://www.jku.at/en/campus/the-jku-campus/buildings/
• Create a wireframe for the new results section with building groups
• Update the results section

Migrate pipeline to GitHub Actions

Travis CI is not free anymore.

Load results automatically, without pressing the button

• Automatically re-render the table if the user changes one of the two time selections, but don't auto-scroll
• Immediately render the table upon page visit, but, think whether it looks good to also trigger the loading animations then
• Still trigger the loading and table fading animation whenever the selection changes
• Rename the button to "Show results"
• The button keeps its functionality: It queries the index, renders the table and scrolls the table into view
• The page shall behave the same on mobile and desktop
• Fix today button
• Test implementation

Show room capacities in results

• Parse room capacities from https://www.jku.at/en/campus/the-jku-campus/buildings/
• Display room capacities in results

Store historic booking data

Right now, the index.json is crawled once every night and re-deployed with GitHub actions:
https://github.com/blu3r4y/jku-room-search/actions/workflows/deploy-github-pages.yml

It would be valuable to also persistently store and query historic data. My first idea would be to change the action to not replace the entire history, but instead append the new file to /data on the gh-pages branch:
https://github.com/blu3r4y/jku-room-search/tree/gh-pages/data

This way, we could utilize GitHub as our storage with minimal changes to the workflow.

Add a "npm run serve" script that helps with local development

• Add webpack-dev-server as a dev dependency
• Add a npm run serve script to package.json that starts the build (in watch-mode) and starts a local web server
• Make sure that content is correctly served from ./dist
• Update the README.md

Remove error messages for unknown rooms in the scraper

• Do not raise errors on unknown rooms, instead, just add a note to the final log message, e.g.

scraped 42 room bookings for course '123456' (of which 12 unknown rooms were ignored: ['tba', 'unknown'])

• Add a statistic counter on numIgnoredBookings which counts all bookings with unknown room names (non-unique)

Just as a note, here is a list of commonly unknown rooms:

-
n.n.
digital
divers
extern
hotel
kein
kusss
live
mail
moodle
offen
online
open
stream
tba
tbd
teams
todo
unbekannt
unknown
video
virtual
virtuell
zoom

If no free rooms are found, the message wrongly says "Sorry, we don't have data for this day"

Change the message and the logic to properly account for the subtle difference between having no scraped data for a day and really having no free rooms for that day.

Optimize the rooms.json storage to ignore 15min breaks

• Investigate what system the break times between slots follow
• Update the scraper to remove break times that nobody will ever query
• Consider if the API needs to changed as well (probably not)
• Compare and test the new storage format
• Report the archived file size reduction

The room capacity of S4 025 is not parsed

There is a table entry for this room, but somehow it isn't parsed correctly.

GET 200 https://www.jku.at/en/campus/the-jku-campus/buildings/science-park-4
ROOM  47% found 0 room entries for building 'Science Park 4 (S4)'

Replace request with got

Since request has been deprecated (request/request#3142), replace it with got (https://www.npmjs.com/package/got).

Make this a progressive web app with offline capabilities

Fix scraping error if there are no courses for a room

The build currently fails on
https://www.kusss.jku.at/kusss/coursecatalogue-search-lvas.action?sortParam0courses=lvaName&asccourses=true&detailsearch=true&advanced=%24advanced&lvaName=&abhart=all&organisationalHint=&lastname=&firstname=&lvaNr=&klaId=&type=all&curriculumContentKey=all&orgid=Alle&language=all&day=all&timefrom=all&timeto=all&room=MT%20226%20%20+&direct=true#result

TypeError: Cannot read property 'trim' of undefined
    at s.<anonymous> (/home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:35744)
    at /home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:227767
    at /home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:230135
    at /home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:200382
    at e.exports (/home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:230101)
    at Object.e.exports [as reduce] (/home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:229903)
    at l.t.map (/home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:227730)
    at e.<anonymous> (/home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:35660)
    at /home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:29144
    at Object.next (/home/travis/build/blu3r4y/jku-room-search/dist/js/scraper.js:235:29249)

Add an auto-formatter to the repository

• Configure prettier with --parser typescript --print-width 120 --tab-width 2 --use-tabs
• Make sure that prettier works with tslint (e.g., use tslint-config-prettier)
• Add scripts to run the auto-formatter and linter
• Update the contribution notes in README.md

Indicate the scraping progress in percent in the log messages

When scraping courses and bookings, we know how many there are in total, because we know the total number of rooms, and courses, respectively, beforehand.

• Modify the logger so that it provides a new "progress" column for a percentage number (right to the status column)
• Fill the progress number when scraping courses
• Fill the progress number when scraping bookins