Comments (9)
Hi Jacqueline, thanks for your question! The ADAM client was intended to be a Python package that is easily imported to say, a Jupyter notebook. Curious to know if you have any suggestions for improvement though?
from adam_home.
It looks like Docker can be used on Jupyter notebooks, although I'll be the first to admit that I have never used one.
I was watching some of the B612 videos on YT, and Dr Lu mentioned that this was run in the cloud (which makes perfect sense). It looks like Google is one of your partners so I assume that you are using GCP. Containerizing this with docker would allow you to build it in a way that it runs the same way on all machines it runs on (if you're familiar with Docker, please forgive the explanation). Deployment could be automated and would be much quicker with much fewer chance of errors.
If you'd like, I can fork this project and write up a Dockerfile for it. You can try it out and see if it works better for people running it.
from adam_home.
We'd be happy to receive contributions here :)
Yes, we do run ADAM in GCP -- specifically, our backend infrastructure (API, processing engine), which does the actual computations. We've mainly just been using the Jupyter notebooks to script out how we want to send and receive data to/from the API.
from adam_home.
Thank you for being open to suggestions :)
This would only affect deploying to and running in the cloud, not any computer hitting the API. I've used AWS more extensively than I have GCP, but in AWS, at least, it makes it much simpler to deploy if you already have a container; it removes the installation step (if that's how you're deploying to the cloud). I'll write it up and you can see if this works for your flow.
from adam_home.
I wrote this up last night and this morning:
FROM continuumio/miniconda3:latest
RUN mkdir /src
WORKDIR /src
RUN conda update conda
COPY . .
RUN conda create -n adam-dev --file conda-requirements.txt \
&& conda init bash \
&& . /root/.bashrc \
&& conda activate adam-dev \
&& python setup.py develop \
&& pytest adam --cov=adam --ignore=adam/tests/integration_tests
# log in
# RUN adamctl login
# # log into the 'dev' environment as well
# RUN adamctl login dev https://adam-dev-193118.appspot.com/_ah/api/adam/v1
# # set up your workspaces
# RUN adamctl config envs.prod.workspace "uuid-received-from-@AstrogatorJohn"
# RUN adamctl config envs.dev.workspace "uuid-received-from-@AstrogatorJohn"
CMD tail -f /dev/null
but it's failing the unit tests so I wanted to be sure that it would work with your flow. especially since I don't have a login. But there are ways to pass those in in docker.
To run this, paste this into the home folder and run docker build --tag adam .
then docker run adam
to run.
I have a few questions before I can do a pull request: is adam_home just the client, or is it what is running in GCP as well? What does logging in do? Is this to coordinate the separate machines?
from adam_home.
This looks good! Thanks 👍
adam_home (which we should probably, eventually, rename) contains both the Python client, as well as our demo Jupyter notebooks, which have served as starting points for people's analysis.
The API is a frontend for our processing engine. In order to create jobs (e.g. to do computations) in ADAM, one needs to authenticate with ADAM. That's where logging in comes in. We currently don't have open registration, though we are working on that.
A little bit more about the logging in and why adamctl
exists... I vaguely recall that the login process from Jupyter to oauth2 was something that was confusing to users, so the adamctl library was created to initiate the oauth2 flow and store credentials -- meant to be a one-time thing. The GCP cli (gcloud) does something similar, where before you can interact with gcloud and your GCP environment, you're required to authc with GCP and gcloud stores the credentials/handles refreshing access token/etc. I'm pretty sure AWS, Azure, etc. have a similar flow, though it's been a long time since I've used their cli tools.
Regarding the unit tests, they've been broken for awhile unfortunately. We're working on fixing them but there are other projects we've been focusing on, which we are close to wrapping up.
Good questions, and it tells me that it's important that we produce some more documentation :)
from adam_home.
Great, no worries! I modified the Dockerfile accordingly and added instructions to the readme and created a pull request. I tested it as much as I can, but unfortunately, I do not have UUID so I cannot test it further. Let me know if any changes need to be made.
Thanks Carise!
from adam_home.
Thanks! I saw the PR and will have some comments within the next day or so.
from adam_home.
Thank you!
from adam_home.
Related Issues (20)
- Fix test or API: integration_tests/anonymous_test.py HOT 1
- Create a notebook using 2018VP1
- Make it clear from the batch module .run() method whether the run completes or not
- Convert Monte Carlo Examples down to one notebook
- Expose enableLogCloseApproaches setting in the ADAM python
- Investigate: Is there a way to clean up Monte Carlo batch jobs?
- Fix and re-enable integration tests
- Update documentation and provide documentation on how to generate new docs
- Move tests outside of the adam package
- Windows: ConfigManager.set_default_env() will fail when renaming the config
- If adamctl login has more than just the `login` argument, require both the profile name and server url
- Divide-by-zero Error HOT 1
- Investigate: Cartesian state vector + covariance triggers hypercube batch run HOT 1
- The close approach distance in the example notebook is not clear HOT 1
- Invalid SSL Certificate on Domain Link in README.md
- Hypercube: should all parts of batch have same ephem after running the job? HOT 1
- Integration tests take a long time to run HOT 2
- Investigate: Issues setting up adamctl on Windows HOT 7
- Fix test: integration_tests/reference_frame_test.py:test_sun_ememe
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from adam_home.