Code notebook for python package learning.
emoji | connotation | tag |
---|---|---|
π or β | experienced | :full_moon: |
π or β | advanced | :moon: |
π or β | well-grounded | :first_quarter_moon: |
π or β¬ | understanding | :waxing_crescent_moon: |
π or β¬ | strange | :new_moon: |
integrated machine/deep/reinforcement learning toolkits.
- π pytorch : open source deep learning framework commonly used in academia.
- π tensorflow : open source deep learning framework commonly used in industry.
- π paddle : baidu open source deep learning platform derived from industry practices.
- π ncnn : tencent high performance neural network forward computing framework optimized for mobiles.
- π gym : a toolkit for developing and comparing reinforcement learning algorithms.
- π ACME : a research framework for reinforcement learning by DeepMind.
- π sklearn : machine learning library.
- π autokeras : automl system based on keras.
- π pytorch_geometric : geometric deep learning extension library for pytorch.
- π textbrewer: nlp knowledge distillation toolkit.
- π OpenPrompt : prompt tuning toolkit developed by Tsinghua.
- π OpenAttack : an open-source package for texual adversarial attack.
- π PyCID: a python library for causal influence diagrams.
NLP/CV/etc. packages.
- π torchvision : vision processing toolkit in pytorch.
- π βcv2 : opencv in python.
- π βpillow : python imaging library.
-
π torchtext : natural language data processing for pytorch.
-
π nltk : english natural language toolkit by UPenn.
for chinese, we should import Stanford NLP Chinese package.
-
π ltp : chinese language technology platform by HIT.
-
π gensim : topic modelling for humans (semantic vector).
-
π glove : toy python implementation of glove.
-
π βhugging face : sota pre-trained language model toolkit.
mainly for pre-trained and transformer, etc.
-
π βfairseq : a seq2seq learning toolkit for pytorch by fair.
-
π EDA : easy data augmentation techniques for text.
-
π pattern : simpler to get started than nltk.
-
π pyenchant : easy access to dictionaries
-
π allennlp : natural language processing toolkit based on pytorch by Allen.
mainly for ELMo, KnowBERT, etc.
-
π seqeval : evaluation for sequence labelling.
-
π sentence transformers : compute dense vector representations for sentences, paragraphs, and images.
-
π β(corenlp, spacy, jieba) : chinese processing, language processing, etc.
we put gensim and glove together in this code notebook.
- π βtorchaudio : audio processing toolkit in pytorch.
data processing and computing.
- π pandas : high performance and user-friendly data structure and analysis tool.
- π matplotlib : comprehensive library for creating static, animated and interactive visualizations in python.
- π numpy : scientific computing.
- π scipy : open-source software for mathematics, science, and engineering.
programming aids.
- π βtqdm : progress bar.
- π labml : training visualization on the mobile phone.
- π wandb: experiment results management.
- π TensorSensorοΌvisualization of errors about tensor.
- π optuna: automatic hyperparameter searching.
- π alfred: visualization of annotations.
- π threading : multithreading.
- π multiprocessing : multiprocessing.
- π deepspeed : microsoft training acceleration toolkit.
- π bminf : a low-resource inference package for large-scale pretrained language models (PLMs).
We put threading and multiprocessing together in this code notebook.
- π βglob : file path manager.
- π βshutil : advanced file operation module.
- π βre : regex.
- π βelasticsearch : search and analysis engine.
- π scrapy : web crawler.
- π βpyqt : GUI.
- π βpyinstaller : application packaging.
- π os : access os-related functions.
- π βsys : access interpreter-related variables and functions.
- π unidecode : because ascii is much easier to deal with
- π chardet : character encoding detection
- .py file is not runnable : we view it as a codebook composed by numerous fragments, but not an executable file (there is no strong relation between every two fragments).