fourpartswater / distil

An analytic workbench for user-guided development of model pipelines

distil

Dependencies

• Go programming language binaries with the GOPATH environment variable specified and $GOPATH/bin in your PATH.
• NodeJS JavaScript runtime.
• Docker platform.
• Docker Compose (optional) for managing multi-container dev environments.

Development

Clone the repository:

mkdir -p $GOPATH/src/github.com/uncharted-distil
cd $GOPATH/src/github.com/uncharted-distil
git clone [email protected]:unchartedsoftware/distil.git
cd distil

Install dependencies:

make install

Install datasets:

Datasets are stored using git LFS and can be pulled using the datasets.sh script.

./datasets.sh

To add / remove a dataset modify the $datasets variable:

declare -a datasets=("185_baseball" "LL0_acled" "22_handgeometry")

Generate code (optional):

To regenerate the PANDAS dataframe parser if the api/compute/result/complex_field.peg file is changed, run:

make peg

Docker images:

The application requires:

• ElasticSearch
• PostgreSQL
• TA2 Pipeline Server Stub
• TA2 Pipeline Runner
• D3M Resource Server

Docker images for each are available at the following registry:

docker.uncharted.software

Login to Docker Registry:

sudo docker login docker.uncharted.software

Pull Images:

Pull docker images via Docker Compose:

docker-compose pull

Running the app:

Using three separate terminals:

Terminal 1 - Launch docker containers via Docker Compose:

docker-compose up

Terminal 2 - Build and watch webapp:

yarn watch

Terminal 3 - Build, watch, and run server:

make watch

Common Issues:

"dep: command not found":

• Cause: $GOPATH/bin has not been added to your $PATH.
• Solution: Add export PATH=$PATH:$GOPATH/bin to your .bash_profile or .bashrc.

"../repo/subpackage/file.go:10:2: cannot find package "github.com/company/package/subpackage" in any of":

• Cause: Dependencies are out of date or have not been installed
• Solution: Run make install to install latest dependencies.