Public Repository for the IARPA Mercury Challenge. Challenge participants should follow this repository to receive notices of code updates and other release information. The files in this directory are provided for IARPA’s Mercury Challenge. You can learn about the challenge and register to compete at https://iarpa.gov/challenges/mercury.html. The discussion forum for the Challenge can be found at http://apps.topcoder.com/forums/?module=ThreadList&forumID=630604 (login required.)
The official scoring package is hosted and the REST endpoints for warning submission and can be found by going to the challenge in Topcoder (see "Making a submission"): http://community.topcoder.com/longcontest/?module=ViewProblemStatement&rd=17231&compid=67293&popup=false
Detailed Submission process can be found here: https://solutions.topcoder.com/mercury-challenge/submission
Leaderboard can be seen here (updated as and when GSRs are completed): http://community.topcoder.com/longcontest/?module=ViewStandings&rd=17231
The ExpressScorer package provides Challenge participants with a lightweight testing engine that does not require a Docker installation. The code is found at https://github.com/planetmercury/mercury-challenge/tree/master/src/ExpressScore and tutorial Jupyter notebooks are found at https://github.com/planetmercury/mercury-challenge/blob/master/src/ExpressScore/notebooks ExpressScore has been tested on MacOS Sierra, Windows 7, and Windows 10 and requires Python 3.4 or greater.
Before beginning, we recommend installing Anaconda (https://www.anaconda.com/download/)
- Anaconda is a free and open source distribution of the Python and R programming languages for data science and machine learning related applications (large-scale data processing, predictive analytics, scientific computing), that aims to simplify package management and deployment.
Install Visual Studio Build Tools 2017
- Installation file found here: https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=15#
- After downloading, open the installer and ensure Visual C++ build tools, Testing tools core features - Build Tools are both checked.
- Depending on which Windows version you're on, ensure that the proper SDK is checked.
- Windows 10 SDK for Windows 10
- Windows 8.1 SDK and UCRT SDK for Windows 7 & 8
Installing CMake is required before continuing.
- If using Ubuntu, run
apt-get install cmake. - If using Centos/RHEL, run
yum install cmake. - You can also build CMake from source by following the instructions here: https://cmake.org/install/
In the terminal of your choice, navigate to src/ExpressScore.
Run conda env create -f expressscore.yml -v to install the required dependencies for the ExpressScorer.
Once complete, activate the environment.
- Run
activate mc_minimalif on Windows - Run
source activate mc_minimalif on Mac or Linux
To ensure successful installation, open the test directory inside src/ExpressScore and run test_ma_scoring.py and test_case_count_scoring.py. If all tests pass, congrats! You now have a working ExpressScorer environment.
The Baserate models make predictions using only ground truth history. The scores for these predictions provide a minimum threshold that Challenge participants must exceed in order to be ranked.
The Mercury Challenge Handbook and Appendix provide complete documentation for the challenge, including definitions of the several event types and scoring rules. They can be found in https://github.com/planetmercury/mercury-challenge/blob/master/doc
- Case Count Scorer Tutorial: https://github.com/planetmercury/mercury-challenge/blob/master/src/ExpressScore/notebooks/Case%20Count%20Scorer.ipynb
- Military Action Scorer Tutorial: https://github.com/planetmercury/mercury-challenge/blob/master/src/ExpressScore/notebooks/Military%20Action%20Scorer.ipynb
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent