sawatzkylindsey / workbench

Workbench

Not really a meaningful name. Project for exploring termnets in the context of educational tasks.

Development

Requirements:

# Python 3
python3 -m venv p3
source ./p3/bin/activate
# Libraries (pip based)
pip install markdown
pip install nltk
pip install rake_nltk
pip install requests
pip install networkx
pip install textrank
pip install sphinx
pip install numpy
# Libraries (custom)
curl -LO https://github.com/sawatzkylindsey/pytils/archive/master.zip
unzip master.zip
cd pytils-master/
make install
curl -LO https://github.com/sawatzkylindsey/Wikipedia-API/archive/master.zip
unzip master.zip
cd Wikipedia-API-master/
python setup.py install

Running:

python dev-server.py -v

workbench's Issues

Add control to switch graph indices/metrics

Right now, two metrics are used statically.

1. Clustering coefficient (lighter and darker blue) used to influence the neighbourhood control.
2. (Biased) page rank (eigenvector centrality) used to rank the nodes from the search control.

We should a) allow for more indices/metrics to be used in the termnet, and b) allow these two functions (neighbourhood & search) to be controlled independently by the indices/metrics as applicable.

Experiment with forcing nodes apart more strongly (maybe using user controls)

Allow for term mappings

example:
Dinosaur
Dinosaur Carnivore <- Tyrannosaurus
Dinosaur Herbivore <- Stegosaurus

Add feature to look-ahead on a search

Experiment with link weights

Synonyms, hypernyms, WordNet, etc

Add paragraph vs sentence windowing

Add feature to show/highlight only the nodes/edges of a selected node

A bit like a neighbourhood, but probably temporary and only distance=1.

Change neighbourhood & search terminology

Minimum path between nodes

Perhaps based off the highlights..

Track search history

As search history affects the state of the termnet, we should track this and show the searched terms.

Note, neighbourhood probably doesn't need to be tracked as it doesn't affect the state of the termnet.

Add control to control the sentence window size

As discussed in the lab, we should allow for term co-occurrence to span a user specified number of sentences. We may also want to allow for paragraph breaks to control whether the sentence window terminates, or are treated as longer in some way.

General performance improvements

• Found several areas that could be made more efficient (caching, etc).
• Change positive/negative influence to rely on local maximum (rather than the graph's global maximum).

Colour previously searched (now focused) nodes

Invalid input should produce a visible error

As noted by lab members, if you provide some invalid input to /index.html the tool will simply sit there. We should produce an error, ideally with specific details about the problem, so that the user can know the difference between options:

1. something went wrong, fix and try again.
2. it is still working, wait a little while longer.