tinybike / argus Goto Github PK

This has a lot of open questions, e.g. how to precisely specify the match / race.

Sources:

• Investigate Yahoo Fantasy - bad, see wiki
• Investigate scraping NBCSports - against ToS
• Investigate Wolfram Alpha
• Investigate http://www.cricapi.com/ for Cricket, specifically

Baseline choice (NBA NFL NHL): https://www.stattleship.com/

Domains:

• soccer
• tennis
• cricket
• basketball
• baseball
• football
• golf
• boxing
• nascar
• horse racing

We have a list of sample finance questions in Google spreadsheet:

https://docs.google.com/spreadsheets/d/1A2itCl8BJ_dc0Td3vzCVM_zmOb3b1K3NRqxiyPzaqi4/edit?usp=sharing

We need to

• have this as a CSV file in git, and
• a tool that'll automatically feed these questions (in JSON format) to panoptes and show results (+ number of answered)

• Stock market (using Yahoo API - NYSE, NASDAQ, LSE and more covered)
• Commodities (oil, gold, silver, copper and more on the U.S. commodities market and exchange, see e.g. CNNMoney)
• Currency exchange rates
• ecurrency prices

Lower priority.

• Investigate Wolfram Alpha
• Investigate forecast.io
• Investigate Yahoo Weather API
• Look for more?

Data types:

• Temperature

For evaluation, we have generated dates of events for the questions in our set (well, at least some of them; XXX) and whenever available, we use that to limit the search to a 14-day period after that date to improve result relevance.

It'd be nice to have an optional date entry in the web interface too, since our original motivation would be that this data is available in Augur anyway.

We should do a train/test split on input data before processing anything, not just when reporting the results, to ensure that proper data hygiene is kept.

(At a later time, we should also further split the train to trainmodel and val and perform learning of sub-classifiers like the sentiment on trainmodel and measure its performance on val rather than test, so that we don't overfit by parameter tuning. However, we have too little data to afford that at this point, so it's just something to bear in mind for now.)

There seems to be some inconsistency between web interface and the training process:

• Will the New England Patriots defeat the Seattle Seahawks in Super Bowl XLIX? in tests/ftrain.tsv is marked as YES 0.521451771259 with "as it happened" as the most relevant article.
• In the web interface right now, with the same model, it's 0.08 for NO and ABC comes first by relevancy, with "as it happened" story as second

Go through the Augur beta and pick out 20-30 sample questions for the sport domain to cover.

When using the benchtest script, the output for the text question regarding aluminium is odd; despite wide date range being selected, both min and max values are the same value and both refer to the same day. Is only a single data point being utilized to describe the entire range?

• Implement proper search for the minimal/maximal day in the range or define special behaviour.