Your digital thought mapper.
HyperTag let's humans intuitively express how they think about their files using tags and machine learning. Instead of introducing proprietary file formats like other existing file organization solutions, HyperTag just layers on top of your existing files without any fuss.
Objective Function: Minimize time between a thought and access to all relevant files.
Available on PyPI
$ pip install hypertag
Join the HyperTag matrix chat room to stay up to date on the latest developments or to ask for help.
HyperTag offers a slick CLI but more importantly it creates a directory called HyperTagFS
which is a file system based representation of your files and tags using symbolic links and directories.
Directory Import: Import your existing directory hierarchies using $ hypertag import path/to/directory
. HyperTag converts it automatically into a tag hierarchy using metatagging.
Semantic Text & Image Search (Experimental): Search for images (jpg, png) and text documents (yes, even PDF's) content with a simple text query. Text search is powered by the awesome Sentence Transformers library. Text to image search is powered by OpenAI's CLIP model. Currently only English queries are supported.
HyperTag Daemon (Experimental): Monitors HyperTagFS
and directories added to the auto import list for user changes (see section "Start HyperTag Daemon" below). Also spawns the DaemonService which speeds up semantic search significantly.
Fuzzy Matching Queries: HyperTag uses fuzzy matching to minimize friction in the unlikely case of a typo.
File Type Groups: HyperTag automatically creates folders containing common files (e.g. Images: jpg, png, etc., Documents: txt, pdf, etc., Source Code: py, js, etc.), which can be found in HyperTagFS
.
HyperTag Graph: Quickly get an overview of your HyperTag Graph! HyperTag visualizes the metatag graph on every change and saves it at HyperTagFS/hypertag-graph.pdf
.
Import files with tags inferred from the existing directory hierarchy.
$ hypertag import path/to/directory
$ hypertag add path/to/file https://github.com/SeanPedersen/HyperTag
Manually tag files. Shortcut: $ hypertag t
$ hypertag tag humans/*.txt with human "Homo Sapiens"
Manually remove tag/s from file/s.
$ hypertag untag humans/*.txt with human "Homo Sapiens"
Metatag tag/s to create tag hierarchies. Shortcut: $ hypertag tt
$ hypertag metatag human with animal
Merge all associations (files & tags) of tag A into tag B.
$ hypertag merge human into "Homo Sapiens"
Print file names of the resulting set matching the query. Queries are composed of tags and operands. Tags are fuzzy matched for convenience. Nesting is currently not supported, queries are evaluated from left to right.
Shortcut: $ hypertag q
Print paths: $ hypertag query human --path
Print fuzzy matched tag: $ hypertag query man --verbose
Disable fuzzy matching: $ hypertag query human --fuzzy=0
Default operand is AND (intersection):
$ hypertag query human "Homo Sapiens"
is equivalent to $ hypertag query human and "Homo Sapiens"
OR (union):
$ hypertag query human or "Homo Sapiens"
MINUS (difference):
$ hypertag query human minus "Homo Sapiens"
Only indexed files can be searched.
$ hypertag index
To parse even unparseable PDF's, install tesseract: # pacman -S tesseract tesseract-data-eng
Index only image files: $ hypertag index --image
Index only text files: $ hypertag index --text
Print text file names sorted by matching score.
Performance benefits greatly from running the HyperTag daemon.
Shortcut: $ hypertag s
$ hypertag search "your important text query" --path --score --top_k=10
Print image file names sorted by matching score.
Performance benefits greatly from running the HyperTag daemon.
Shortcut: $ hypertag si
Text to image:
$ hypertag search_image "your image content description" --path --score --top_k=10
Image to image:
$ hypertag search_image "path/to/image.jpg" --path --score --top_k=10
Start daemon process with triple functionality:
- Watches
HyperTagFS
directory for user changes- Maps file (symlink) and directory deletions into tag / metatag removal/s
- On directory creation: Interprets name as set theory tag query and automatically populates it with results
- On directory creation in
Search Images
orSearch Texts
: Interprets name as semantic search query (add top_k=42 to limit result size) and automatically populates it with results
- Watches directories on the auto import list for user changes:
- Maps file changes (moves & renames) to DB
- On file creation: Adds new file/s with inferred tag/s
- Spawns DaemonService to load and expose models used for semantic search, speeding it up significantly
$ hypertag daemon
$ hypertag tags filename1 filename2
$ hypertag metatags tag1 tag2
$ hypertag show
Print names:
$ hypertag show files
Print paths:
$ hypertag show files --path
Visualize the metatag graph hierarchy (saved at HyperTagFS root).
$ hypertag graph
Specify layout algorithm (default: fruchterman_reingold):
$ hypertag graph --layout=kamada_kawai
Generate file system based representation of your files and tags using symbolic links and directories.
$ hypertag mount
Directories added to the auto import list will be monitored by the daemon for new files or changes.
$ hypertag add_auto_import_dir path/to/directory
Default is the user's home directory.
$ hypertag set_hypertagfs_dir path/to/directory
- Python and it's vibrant open-source community power HyperTag
- Many other awesome open-source projects make HyperTag possible (listed in
pyproject.toml
) - SQLite3 serves as the meta data storage engine (located at
~/.config/hypertag/hypertag.db
) - Added URLs are saved in
~/.config/hypertag/web_pages
for websites, others in~/.config/hypertag/downloads
- Symbolic links are used to create the HyperTagFS directory structure
- Semantic text search is powered by the awesome DistilBERT
- Text to image search is powered by OpenAI's impressive CLIP model
- Clone repo:
$ git clone https://github.com/SeanPedersen/HyperTag.git
$ cd HyperTag/
- Install Poetry
- Install dependencies:
$ poetry install
- Activate virtual environment:
$ poetry shell
- Run all tests:
$ pytest -v
- Run formatter:
$ black hypertag/
- Run linter:
$ flake8
- Run type checking:
$ mypy **/*.py
- Run security checking:
$ bandit --exclude tests/ -r .
- Run HyperTag:
$ python -m hypertag
What is the point of HyperTag's existence?
HyperTag offers many unique features such as the import, semantic search for images and texts, graphing and fuzzy matching functions that make it very convenient to use. All while HyperTag's code base staying tiny at <1500 LOC in comparison to TMSU (>10,000 LOC) and SuperTag (>25,000 LOC), making it easy to hack on.
This project is partially inspired by these open-source projects: