archivesunleashed / notebooks

Various examples of notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archives Unleashed Toolkit.

Archives Unleashed Notebooks

Contains various examples of notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archives Unleashed Toolkit.

License

Licensed under the Apache License, Version 2.0.

Acknowledgments

This work is primarily supported by the Andrew W. Mellon Foundation. Other financial and in-kind support comes from the Social Sciences and Humanities Research Council, Compute Canada, the Ontario Ministry of Research, Innovation, and Science, York University Libraries, Start Smart Labs, and the Faculty of Arts and David R. Cheriton School of Computer Science at the University of Waterloo.

Any opinions, findings, and conclusions or recommendations expressed are those of the researchers and do not necessarily reflect the views of the sponsors.

notebooks's Issues

Basic Token Frequency

Could we put some basic token frequency after tokens are generated? Most popular words, etc. If it could be broken down by date that would also be perhaps interesting (most popular words in year1 vs year2). That could flow into the word cloud nicely I think.

Webgraph by Date

Depending on the contours of the collection, maybe an additional webgraph example would be to show how you could filter the web graph based on crawl date (i.e. show a web graph from 2015 and then one from 2016). These sorts of longitudinal changes might be of interest?

(In a forthcoming hyperlink analysis section?)

Display Most Popular Image(s)?

Seeing images displayed in-line in notebooks is always an "ooh" and an "ah" moment, and might help formulate research questions. For the images, could we add a line that displays an image? Could we display the most frequent image? (i.e. the one with the hash b09dc3225d5e1377c52c06feddc33bfe or use the URL to nab it from the Wayback Machine and generate it)

Even if it just descriptive text explaining how you can take it the next step to get to the images.

Display Crawl Dates

Early in the notebook, we get a sense of the most frequent domains. We could consider adding another command to get a senes of the most frequent crawl dates.

arch notebook editing

creating an issue and branch while reviewing ARCH notebooks and as part of work with https://github.com/helgeho/arch/pull/179