torrvision / crayon Goto Github PK
View Code? Open in Web Editor NEWA language-agnostic interface to TensorBoard
License: MIT License
A language-agnostic interface to TensorBoard
License: MIT License
This is under a python 3 venv:
$ .venv/bin/pip install pycrayon
Collecting pycrayon
Downloading pycrayon-0.5.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-4hvqb18r/pycrayon/setup.py", line 2, in <module>
from pycrayon.crayon import __version__
File "/tmp/pip-build-4hvqb18r/pycrayon/pycrayon/__init__.py", line 1, in <module>
from .crayon import CrayonClient
File "/tmp/pip-build-4hvqb18r/pycrayon/pycrayon/crayon.py", line 1, in <module>
import requests
ImportError: No module named 'requests'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-4hvqb18r/pycrayon/
$ .venv/bin/pip install requests pycrayon
Collecting requests
Downloading requests-2.13.0-py2.py3-none-any.whl (584kB)
100% |████████████████████████████████| 593kB 2.3MB/s
Collecting pycrayon
Using cached pycrayon-0.5.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-xyawjvaw/pycrayon/setup.py", line 2, in <module>
from pycrayon.crayon import __version__
File "/tmp/pip-build-xyawjvaw/pycrayon/pycrayon/__init__.py", line 1, in <module>
from .crayon import CrayonClient
File "/tmp/pip-build-xyawjvaw/pycrayon/pycrayon/crayon.py", line 1, in <module>
import requests
ImportError: No module named 'requests'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-xyawjvaw/pycrayon/
$ .venv/bin/pip install requests
Collecting requests
Using cached requests-2.13.0-py2.py3-none-any.whl
Installing collected packages: requests
Successfully installed requests-2.13.0
$ .venv/bin/pip install pycrayon
Collecting pycrayon
Using cached pycrayon-0.5.tar.gz
Requirement already satisfied: requests in ./.venv/lib/python3.5/site-packages (from pycrayon)
Building wheels for collected packages: pycrayon
Running setup.py bdist_wheel for pycrayon ... done
Stored in directory: /home/elis/.cache/pip/wheels/f2/39/f0/b7168e93688c8d36ffaae3c0ca5a9a928c5173d01baf69aeff
Successfully built pycrayon
Installing collected packages: pycrayon
Successfully installed pycrayon-0.5
My experiment name is 'myexp+suffix'.
When I call the add_scalar_value function for this experiment, I get the following error message:
Traceback (most recent call last):
File "plot.py", line 126, in main
crayon_experiment.add_scalar_value(key, values[key], step = counter)
File "anaconda/lib/python2.7/site-packages/pycrayon/crayon.py", line 149, in add_scalar_value
raise ValueError(msg.format(r.text))
ValueError: Something went wrong. Server sent: Unknown experiment name 'myexp suffix'.
It has dropped the + sign from the experiment name and complains experiment does not exist.
I started using crayon in my project, and I have been reasonably happy with it although it is a pity that it doesn't support all tensorboard summary types and that it does not seem to be very actively maintained. Later I discovered a different approach of just using a client library to create tensorboard log items (see https://github.com/TeamHG-Memex/tensorboard_logger and, for a particularly simple approach, https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/04-utils/tensorboard/logger.py). I realize that the crayon approach is in a way more flexible since it provides a REST api, but for someone just looking to use tensorboard with pytorch, it is not very clear what the advantages of crayon are over a simpler approach not using a separate server, given that no solution seems to be under active maintenance.
Crayon should use a more random default port number than 8888 since that conflicts with the default for jupyter notebook.
I have a computer with GPU, and run my programs on it. I don't have a server or client machine, just one PC.
How could I install crayon in this situation?
I am glad that #24 is solved, however I find the lastest version is 0.5 which does not contain this fix...
Currently it's a bloody mess. Let's urgently fix that.
requests
)For each client we should add some code examples.
Currently there are only scalar and histogram methods. It would be AWESOME to have embedding methods, as that let's people like me who use PyTorch for unsupervised learning make use of Crayon.
If you think about it, Embedding is the most powerful tool on Tensorboard. Histograms and running averages (scalars) aren't hard to implement. PCA/TSNE interactively are not super easy.
Would be happy to work with other devs on adding embedding functionality if anyone's up! Any leads on how I should approach?
Hello I am trying to install crayon. I have Installed docker. I run docker pull alband/crayon
this command and got this error.
FATA[0000] Post http:///var/run/docker.sock/v1.18/images/create?fromImage=alband%2Fcrayon%3Alatest: dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?
How I can Fixed this?
-Thank You-
That seems to cause the situation where it becomes impossible to remove experiments after a restart. I guess there's a workaround by writing something to an experiment before deleting it, which I'll try out. Still, should probably be improved.
Hi, thanks for making this project open source.
I get the following error,
Traceback (most recent call last): File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap self.run() File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/ajay/PythonProjects/PyT_Neural_Arch_Search_v1_2/train_v1.py", line 201, in train foo.get_scalar_values("Mean Reward") File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/site-packages/pycrayon/crayon.py", line 167, in get_scalar_values return json.loads(r.text) File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
When I run the following in a single thread
cc = CrayonClient( hostname="http://127.0.1.1" , port=8889)
foo = cc.create_experiment("train_" + str(rank))
foo.add_scalar_value("Mean Reward", mean_reward, step = episode_count)
foo.add_scalar_value("Max Reward" , max_reward, step = episode_count)
foo.get_scalar_values("Mean Reward")
foo.get_scalar_values("Max Reward")
Seems to work fine when I run it from the python interpreter though?
Empty (with no point) experiments are not listed in the GET /data
endpoint properly here.
Even though it properly raises an error when trying to create a new one with the same name and opening it works as expected.
I keep getting this error:
Traceback (most recent call last): File "train_viewpoint.py", line 495, in <module> main(args) File "train_viewpoint.py", line 23, in main cc = CrayonClient("focus.eecs.umich.edu") File "/z/home/mbanani/PyTorch2Env/local/lib/python2.7/site-packages/pycrayon/crayon.py", line 29, in __init__ " Server sent: {}.".format(r.text)) RuntimeError: Something went wrong! Server sent: Server: TensorBoard failed to answer request 'logdir'. Done
and I think the reason is that the timeout, specified here is too low when people are training on clusters and trying to connect to a docker on their personal machines. Maybe consider increasing the timeout ?
We should write an interface for visualising PyTorch (or any other NN graph).
Often with the function net.summary.add_scalar_value(...)
, I intermittently get the following error:
requests.exceptions.ConnectionError: HTTPConnectionPool(host=IP, port=8889): Max retries exceeded with url: /data/scalars?xp=VGG&name=Train%20R%5E2 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at MEMORY_ADDRESS>: Failed to establish a new connection: [Errno 4] Interrupted system call',))
My pycrayon server is on my local machine whereas the computation (that is sending this request) is on a remote machine.
This is in the Python 2.7 version of pycrayon. I think that this request should be retried until it succeeds, as per https://stackoverflow.com/questions/14136195/what-is-the-proper-way-to-handle-in-python-ioerror-errno-4-interrupted-syst, which is the default behavior in Python 3.5.
I'm new to pytorch and crayon, so please forgive me if I'm mistaken.
I use foo.add_scalar_dict({'train_acc':trainacc,'test_acc':testacc})
to send training and testing accuracy to the server, but they ended up in separate figures. How can I draw them in a single figure ?
Need to write end points and backend for the image tab in tensorboard. To be done after #8.
We should have a testing suite to run on Travis.
[py2.7, py3.*]
with tox
. For Lua we'll have to check what can be done in terms of test discovery.travis.yml
file.Error occurs when I run the python usage example.
from pycrayon import CrayonClient
import time
cc = CrayonClient(hostname="10.1.12.46")
foo = cc.create_experiment("foo")
ERROR:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-be887cc4349c> in <module>()
----> 1 foo = cc.create_experiment("foo")
/usr/local/lib/python2.7/dist-packages/pycrayon/crayon.pyc in create_experiment(self, xp_name, zip_file)
53 def create_experiment(self, xp_name, zip_file=None):
54 assert(isinstance(xp_name, basestring))
---> 55 return CrayonExperiment(xp_name, self, zip_file=zip_file, create=True)
56
57 def remove_experiment(self, xp_name):
/usr/local/lib/python2.7/dist-packages/pycrayon/crayon.pyc in __init__(self, xp_name, client, zip_file, create)
86
87 elif create:
---> 88 self.__init_empty()
89
90 else:
/usr/local/lib/python2.7/dist-packages/pycrayon/crayon.pyc in __init_empty(self)
94 def __init_empty(self):
95 query = "/data"
---> 96 r = requests.post(self.client.url + query, json=self.xp_name)
97
98 if not r.ok:
/usr/lib/python2.7/dist-packages/requests/api.pyc in post(url, data, **kwargs)
86 """
87
---> 88 return request('post', url, data=data, **kwargs)
89
90
/usr/lib/python2.7/dist-packages/requests/api.pyc in request(method, url, **kwargs)
42
43 session = sessions.Session()
---> 44 return session.request(method=method, url=url, **kwargs)
45
46
TypeError: request() got an unexpected keyword argument 'json'
Maybe a simple key generated by the server at startup, or something you can add to the docker file startup.
Hi,
I tried add_scalar_dict
as following
local foo = cc:create_experiment("foo")
local d = {}
d['bar'] = 3
d['hoo'] = 4
foo:add_scalar_dict(d)
but got the errors
attempt to index field 'scalar_steps' (a nil value)
not sure how to use that.
thanks
It's annoying to write scripts to save / delete runs. We should just run a bootstrap frontend or something.
Probably to do when / after we refactor the server.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.