Comments (9)
Woot Woot 🎊 pip
PR merged pypa/pip#7612
from clearml.
It looks like wheel
is also required for this feature. Updating pip
and installing wheel
before any git source packages solved the problem.
Thanks for your time @bmartinn
from clearml.
Hi @elinep, yes you should totally use trains-agent to do so :)
- I'll check if we could get Trains to auto detect git+ssh packages,
- Regrading the remote execution using trains-agent
- Clone the experiment you want to execute
- The new cloned experiment is in "draft" mode, which means everything is editable
- Go to Execution -> "Installed Packages" section and press on the edit button (it should hover over it, or just double-click on the text)
- Everything you write in the packages section is equivalent to writing it in a "requirements.txt". This means that in your case, just add the missing git+ssh:// line
- Trains-agent will install everything according to what we have in the "installed packages" section, so as long as you have the git/ssh keys installed on the remote machine, it should work out of the box. Notice that if you are running trains-agent in docker mode, your ~/.ssh folder will be mapped into the docker, so the keyring should already be available inside the docker :)
from clearml.
Ok, I confirm it works by manually editing the "installed packages" section.
Would be great if it could be handled automatically though ;)
Thanks
from clearml.
Awesome, I'm happy to hear the problem is solved.
I'm keeping this issue open until we have support for git/ssh package auto-detection :)
from clearml.
Well, solution is on its way... I assume...
Please 👍 on the original PIP issue and the PR
from clearml.
Hi @elinep,
I forgot to update here, starting trains 0.15.0 trains
and trains-agent
fully support this feature
Just don't forget to upgrade pip > 20
😄
from clearml.
Hi @bmartinn
I missed your reply. While testing #219 (comment) I realized that the reported requirements on the UI does not show the git+ssh link. When remotely executing the task, trains-agent tries to install the package from the regular repository.
My requirements.txt is:
git+https://github.com/allegroai/trains.git@6dd7b4e02ec9201e2857ee4bfaa389e4cff04248
hydra-core==1.0.3
The reported requirements.txt is:
# Python 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]
hydra_core == 1.0.3
trains == 0.16.3
# Detailed import analysis
# **************************
# IMPORT PACKAGE hydra_core
# script_hydra_trains_support_pattern_2.py: 3,4
# IMPORT PACKAGE trains
# script_hydra_trains_support_pattern_2.py: 2
And pip version is 20.2.4.
Did I miss something ?
from clearml.
@elinep , what are you getting on pip freeze
? (this is similar to how trains
is analyzing the installed packages)
BTW:
I tested again on a clean environment, and it worked as expected.
One thing to note, pip introduced this capability in v20, that means that if a lower version was used to install trains
, the information regrading the original source (i.e. git) is not stored anywhere.
from clearml.
Related Issues (20)
- Debug sample disappear when experiemnt is running HOT 2
- Dataset uploading error
- Fixed users in `apiserver.conf` receiving `401` (Unauthorized) HOT 2
- How to upload datasets to remote S3 without compressing them? HOT 1
- `api.files_server` without a port causes pipeline to fail HOT 4
- Incorrect ordering of iterations in scalar reporting of metrics HOT 1
- Greater control in downsampling used in plots HOT 2
- Modify report_matplotlib_figure to Omit Iteration Value When iteration=None HOT 3
- Error when calling classes through Fire HOT 2
- Use a function of iterations (e.g. epochs) as the time scale for scalars and plots HOT 2
- Pathlib Path instances in a dataclass do not get tracked by task.connect() HOT 1
- delete datasets after call get_local_copy
- report_matplotlib_figure of subplots HOT 2
- How to use Omegaconf without Hydra? HOT 1
- Scrolling log problem when using tqdm as training process bar HOT 5
- ClearML feature for integration KerasTuner is broken HOT 1
- Fix typo in docs and default sdk config HOT 1
- Executing clearml-task from cli with "-m" modules HOT 1
- Dynamic GPU/Queue Allocation for Workers in ClearML
- Add tag with Clearm-task (cli tools) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clearml.