artefactory / nlpretext Goto Github PK
View Code? Open in Web Editor NEWAll the goto functions you need to handle NLP use-cases, integrated in NLPretext
Home Page: https://artefactory.github.io/NLPretext/
License: Apache License 2.0
All the goto functions you need to handle NLP use-cases, integrated in NLPretext
Home Page: https://artefactory.github.io/NLPretext/
License: Apache License 2.0
For de-duplication...
Use Cases : BPI (Alexy), Allianz (Samuel)
The idea is to transform the Nautilus_library using SK-learn compatible API, so we can use the Pipeline module of sklearn in order to chain the processing.
https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html
Spacy just realeased v2.1, including training of language model (Bert,UMLFIT like) & 3x faster Tokenization.
Check if no breaking change + make api for pre-training.
os.environ['MALLET_PATH']='/home/shared/Allianz_topic_forcasting/bin/lib'
path2mallet = '$MALLET_PATH/mallet-2.0.8/bin/mallet'
ldamallet = gensim.models.wrappers.LdaMallet(path2mallet,corpus=corpus, num_topics=15, id2word=id2word, prefix='save')
ldamallet.save('monmodele')
Bien préciser le préfixe, et copier coller TOUS les fichiers avec le préfixe, plus le nom du modèle.
L’importer sur l’autre PC, dans ce PC, bien créer la var d’environnement, et
LdaMallet.load(‘/newpath’) avec tous les fichiers dedans.
Attention, il faut placer le fichier d’execution avec le modèle et ses composantes.
Ligne 317 https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/models/wrappers/ldamallet.py
(a mettre en haut) from gensim.models.wrappers import LdaMallet
Git LFS ?
Drive Nautilus
(=comprendre la question, et comment y répondre)
==> BERT, GP2, DialogFlow
Main topic issue. Might be spitted into several issues if needed
All the pipeline to convert english text
Main topic issue. Might be spitted into several issues if needed.
chardet
All the pipeline regarding cleaning and pre-processing french text
Test de load en UTF-8, si unicode error on fait un chardet.
Parametre encoding, param pour activer et désactiver chardet
L'idée de cette abstraction est de créer une classe Document qui contient à la fois le raw text + le résultat des preprocessing.
L'abstraction Corpus étant à prendre littéralement comme un ensemble de document.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.