willkhoza / bible

Data Scrapping using R

bible

Data Scrapping using R

I'm working towards building a machine learning translation programme similar to Google Translate but centered towards languages which are not as prominent. My base language will be Xitsonga.

The task of this programme is to collect data from already existing transaltions between English and Xitsonga. The first source being the bible.

Unfortunately there are no downloadable versions of the Xitsonga bible, so I just wrote an R programmme that will go over the bible version available on a website, page by page and save what all the verses into a single data frame.

It will do the same thing for bible verses in english, then now we have our first dataset of going from a language that is not so promiment to English.

The next programme will be working on sentiment analysis, Followed by the transaltion using recurrent neural networks, Then finally we will do a chatbot.

DISCLAIMER: Webscrapping is really resource extensive on their servers, please keep the requests made minimal and check other laws around doing that.