fra82 / twitter-crawler Goto Github PK

I am using the [REST Keyword crawler], and I only want English tweets.
I did as the snippet in README.md, but I still get tweets of multi Langs. So I checked the file: "crawler.properties", and changed:
####################################################################################
# REST Cralwer of Twitter - by keyword(s)
# Class: org.backingdata.twitter.crawler.rest.TwitterRESTKeywordSearchCrawler
# - Full path of the txt file to read terms from (one term ID per line)
tweetKeyword.fullPathKeywordList=keywords.txt
# - Full path of the output folder to store crawling results
tweetKeyword.fullOutputDirPath=./data/
# - Storage format: "json" to store one tweet per line as tweet JSON object or "tab" to store
# one tweet per line as TWEET_IDTWEET_TEXT
tweetID.outputFormat=json
# - If not empty, it is possible specify a language to retrieve only tweet of a specific language
# (en, es, it, etc.) - if empty all tweet are retrieved, indipendently from their language
# IMPORTANT: The language code may be formatted as ISO 639-1 alpha-2 (en), ISO 639-3 alpha-3 (msa), or ISO 639-1 alpha-2 combined with an ISO 3166-1 alpha-2 localization (zh-tw).
tweetID.languageFilter=en

into:
####################################################################################
# REST Cralwer of Twitter - by keyword(s)
# Class: org.backingdata.twitter.crawler.rest.TwitterRESTKeywordSearchCrawler
# - Full path of the txt file to read terms from (one term ID per line)
tweetKeyword.fullPathKeywordList=keywords.txt
# - Full path of the output folder to store crawling results
tweetKeyword.fullOutputDirPath=./data/
# - Storage format: "json" to store one tweet per line as tweet JSON object or "tab" to store
# one tweet per line as TWEET_IDTWEET_TEXT
tweetKeyword.outputFormat=json
# - If not empty, it is possible specify a language to retrieve only tweet of a specific language
# (en, es, it, etc.) - if empty all tweet are retrieved, indipendently from their language
# IMPORTANT: The language code may be formatted as ISO 639-1 alpha-2 (en), ISO 639-3 alpha-3 (msa), or ISO 639-1 alpha-2 combined with an ISO 3166-1 alpha-2 localization (zh-tw).
tweetKeywod.languageFilter=en

And it worked!
Thanks for your great tool, which is useful and helped a lot!

I use REST Keyword crawler to collect tweets contains multiple keywords.
And my keywords.txt looks like:

love
hello
sentiment

The issue is: it only crawls tweet contains 1 specific keyword e.g. "love" instead of contains "love" | "hello" | "sentiment".
but when it launches, the key word list shows multi keywords to crawl.

<><><><><><><><><><><><><><><><><><><>
List of keywords to crawl:
1 keyword: love
2 keyword: sentiment
3 keyword: hello
<><><><><><><><><><><><><><><><><><><>

How to crawl tweets contains keyword A or B or C... using this tool.