chainer / chainerui

ChainerUI: User Interface for Chainer

ChainerUI

ChainerUI is a visualization and management tool for Chainer.

Installation

To install ChainerUI, use pip.

$ pip install chainerui

To install ChainerUI from source.

$ git clone https://github.com/chainer/chainerui.git
$ cd chainerui/frontend
$ npm install && npm run build && cd ..
$ pip install -e .

• ChainerUI uses sqlite3 module which is included in the standard Python library. If Python is built from source, sqlite3 must be installed before building Python.
  • On Ubuntu, "libsqlite3-dev" must be installed before building Python ($ apt-get install libsqlite3-dev).
  • On Windows, install Visual C++ Build Tools with the Default Install setting before building Python.

Usage

1. output log file with LogReport extension, see examples/log-file/train_mnist.py
2. output log file manually, see examples/log-file/train_mnist_custom_loop.py
3. post training logs via web API, see examples/web-api/train_mnist.py

Quick start

Initialize ChainerUI database.

$ chainerui db create
$ chainerui db upgrade

Clone examples of train log and create a project.

$ git clone https://github.com/chainer/chainerui.git
$ cd chainerui

$ # create example project
$ chainerui project create -d examples/log-file -n example-project

$ # run ChainerUI server
$ chainerui server

Open http://localhost:5000/ and select "example-project", then shown a chart of training logs.

For more detailed usage, see getting started

Docker start

Get Docker container from DockerHub and start ChainerUI server. The container has installed ChainerUI module, setup a DB and a command to start the server.

$ # replace tag to the latest version number
$ docker pull chainer/chainerui:latest
$ docker run -d -p 5000:5000 --name chainerui chainer/chainerui:latest
$ # then ChainerUI server is running

Open http://localhost:5000/, shown empty project list. examples/web-api/train_mnist.py is an example to setup project and send logs via web API.

For more detailed usage, see getting started or web API

Browser compatibility

ChainerUI is supported by the latest stable version of the following browsers.

• Firefox
• Chrome

License

MIT License (see LICENSE file).

Contribution

Any contribution to ChainerUI is welcome!

• Python codes follow the Chainer contribution guide.

Run tests

Install "pytest" package and run test

$ pytest

Build client side JS

$ cd frontend
$ npm install
$ npm run build:watch

chainerui's Issues

All CI jobs are failed

ERROR: /home/travis/build/chainer/chainerui/setup.cfg:11: requires pytest-3.4, actual pytest-3.10.0'

ChainerUI's server and client crash with large log

I confirmed that ChainerUI crashed with the 120000 iteration (output per 10 iteration) 8MB log file.

Fail to run MNIST example

branch: master (9bd4a961a03b9f7dfc388c6c04024448de8ba0bb)

$ python train_mnist.py -o result_temp
Traceback (most recent call last):
  File "train_mnist.py", line 14, in <module>
    from chainerui.extensions import CommandsExtension
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/extensions/__init__.py", line 1, in <module>
    from chainerui.extensions.commands_extension import CommandsExtension  # NOQA
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/extensions/commands_extension.py", line 8, in <module>
    from chainerui.utils.command_item import CommandItem
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/utils/__init__.py", line 1, in <module>
    from chainerui.utils.command_item import CommandItem  # NOQA
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/utils/command_item.py", line 6, in <module>
    from chainerui.models.command import Command
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/models/__init__.py", line 4, in <module>
    from chainerui.models.project import Project  # NOQA
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/models/project.py", line 11, in <module>
    from chainerui.tasks import collect_results
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/tasks/__init__.py", line 2, in <module>
    from chainerui.tasks.crawl_result import crawl_result  # NOQA
  File "/mnt/c/Development/gopath/src/github.com/chainer/chainerui/chainerui/tasks/crawl_result.py", line 10, in <module>
    from chainerui.utils.command_item import CommandItem
ImportError: cannot import name 'CommandItem'

Toggle options for axis

Hi chainerUI developers!

I want to toggle between display and hide axes because there are many metrics in my experimental setting.

What do you think about it?

multiple graphs

Hi,
great work with chainerui

Is it possible to visualise multiple graphs.

e.g. Losses on one graph and Accuracy on the other?

Support permanent link to share a result with other people

When we use ChainerUI in order to share results of training in our team, we need permanent link feature to share a result in the same graph condition. (Maybe this is also useful to share with oneself after some days or some weeks)

For example, the permanent link should include:

• which metrics is selected as each axis
• which results are selected
  and so on...

(ChainerUI has a lot of parameters to show a graph of results, so is it difficult to design permanent link?)

The current DB schema version is not supported, please setup DB

run the example project command, there is a error message "The current DB schema version is not supported, please setup DB"

what does this message mean?

UI is broken

The vertical positions of the checkbox are broken.

"Invalid Date" when sent command and waiting

After sending a command like 'snapshot' and waiting a response, command history table shows "invalid date". "Waiting..." (or other word) or empty cell is better, I assume.

Result names in the table remain truncated even after the `name` column is expanded

Thank you for making ChainerUI, it is really useful 🍰

Let me just report a couple of things that I noticed.

For some results that have longer names, the name is truncated (ends in ellipsis). Even after dragging the name column to expand it, the name is not shown non-truncated, somewhat counter-intuitive to other table UIs.

update document for v0.5.0

• export log chart
• collapsable table row

Show long result names more readable

In the result table, when a result name or path is long, I want to

• select truncate-right or truncate-left
• see full name on mouse over

Limit of concurrent connections when polling

If API that returns large log is slow, many requests will be stacked.

I think that we should define limit of concurrent connections.

Add stop button on detail page

How to change the default localhost to IP address?

How to change the default localhost to IP address?
I want to visit http://xxx.xxx.xxx.xxx:5000 not localhost

Move chainer/static to frontend

We should manage all assets with webpack.

Occur a warning on starting the server

chainerui==master (9bd4a96)
Flask==1.0.1
on Ubuntu16.04

Add `episode` and `step` key on summery pane

refs #53, #63

Character limit of result names

Currently ChainerUI does not have character limit for result name, and this may cause ugly web UI.

Order of registering result directory is arbitrary

When registering log directories, the order of directories is not fixed, caused by os.walk (os.listdir) spec.

chainerui/chainerui/tasks/collect_results.py

Line 12 in fb199f9

for root, _dirs, _files in os.walk(os.path.abspath(target_path)):

Because of it, test_api.py has tests dependent on the order of registering, so fail in some environment.

Show command error

On sending a command from result page, A server returns 400 error correctly but no message is shown on web UI.

ref: https://github.com/chainer/chainerui/blob/00f1353dc00c03339bbf924cf0e106bfcd2a8e98/chainerui/views/result_command.py

Cannot see the result once unregistered

When unregister a result from result table, the same name result cannot be showed on the table.

[Doc] About line color and enable to change

Add a description about line color and user can change it.

update npm version on CI

API returns 500 when the db is not created yet

I encountered the following error when I see http://localhost:5000 before creating the db.

[2017-12-24 16:25:33,620] ERROR in app: Exception on /api/v1/projects [GET]
Traceback (most recent call last):
  File "/Users/inagaki/.pyenv/versions/anaconda3-4.3.1/envs/ml/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/Users/inagaki/.pyenv/versions/anaconda3-4.3.1/envs/ml/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: project

Managing training jobs

This article mentioned the management of training jobs via the chainerui. Specifically:

To operate training jobs, set CommandsExtension in the training script. This extension supports taking a snapshot and changing hyperparameters such as learning rate while running the training job.

I have been trying to change the learning rate, number of epochs and take snapshots without success. Is this functionality available and if so, how can it be used?

Usage without Trainer

For now there's only one example script, https://github.com/chainer/chainerui/blob/master/examples/train_mnist.py, which depends on Trainer.

Is it possible to use ChainerUI without Trainer? If no, is there any plan to support it?

Strange ellipsis position on right align mode

On "align right" mode, positions of ellipsis (...) seem to be strange.

"align right" looks odd, the names are end with 'log'

"align left" looks correctly

pkg_resources.ContextualVersionConflict between Flask, werkzeug

I installed chainerui by the following commnads,

pip install chainerui
chainerui servser

but I caught the following error

pkg_resources.ContextualVersionConflict: (Werkzeug 0.12.2 (/home/hiroshimatsui/.pyenv/vers
ions/3.6.5/envs/bilstm-chainer/lib/python3.6/site-packages), Requirement.parse('Werkzeug>=
0.14'), {'Flask'})

flask 1.0.2 has requirement Werkzeug>=0.14.

Which version of flask is needed for chainerui?

Better Documentation and more examples

It would be helpful to have better documentation and more examples scripts detailing how to use chainerui.

I am currently not sure which training hyper-parameters can be changed via the CommandsExtension.
Can we only change hyper-parameters to do with the optimiser or can we also change things like the batch size during training?
Can we attach custom hyper-parameters to be tuned via chainerui?

If so, how is this done?

show images and visualise network structure

I am assuming that this will become the main visualisation tool for chainer.

In that case, it would be nice if we had the ability to view images directly in chainerui, this is especially handy when tracking the progress of generative models.

Being able to visualise a network (sub)-structure would be handy too.

Bring together version numbers

Version numbers are written in 3 files.

• chainerui/_version.py
• frontend/package.json
• frontend/src/constants/index.js

should have a unique "key" prop error on result detail page

I've got the following error on result detail page.

Warning: Each child in an array or iterator should have a unique "key" prop. See https://fb.me/react-warning-keys for more information.
    in dl (created by Args)
    in div (created by Args)
    in div (created by Args)
    in Args (created by ResultDetail)
    in div (created by ResultDetail)
    in div (created by ResultDetail)
    in div (created by Container)
    in Container (created by ResultDetail)
    in div (created by ResultDetail)
    in ResultDetail (created by Connect(ResultDetail))
    in Connect(ResultDetail) (created by RouterContext)
    in RouterContext (created by Router)
    in Router (created by Root)
    in Provider (created by Root)
    in Root
    in AppContainer

Cannot see the log including NaN

ChainerUI will stop by the log including NaN.

cf.

$ python
>>> json.dumps([float('nan')])
'[NaN]'

$ node
> JSON.parse('[NaN]')
SyntaxError: Unexpected token N in JSON at position 1

Exception when access GET API with no project on DB

When DB has no project, and access to GET API like /api/v1/projects/1/results , ChainerUI server occurs an exception.

Server log:

127.0.0.1 - - [28/Dec/2017 19:52:15] "GET /api/v1/projects/1/results HTTP/1.1" 500 -
[2017-12-28 19:52:20,226] ERROR in app: Exception on /api/v1/projects/1/results [GET]
Traceback (most recent call last):
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/_compat.py", line 33, in reraise
    raise value
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/root/venv/chainerui/lib/python3.5/site-packages/Flask-0.12.2-py3.5.egg/flask/views.py", line 149, in dispatch_request
    return meth(*args, **kwargs)
  File "/root/venv/chainerui/lib/python3.5/site-packages/chainerui-0.1.1-py3.5.egg/chainerui/views/result.py", line 23, in get
    collect_results(project)
  File "/root/venv/chainerui/lib/python3.5/site-packages/chainerui-0.1.1-py3.5.egg/chainerui/tasks/collect_results.py", line 46, in collect_results
    if (now - project.updated_at).total_seconds() < 4 and (not force):
AttributeError: 'NoneType' object has no attribute 'updated_at'
127.0.0.1 - -[28/Dec/2017 19:52:15] "GET /api/v1/projects/1/results HTTP/1.1" 500 -

Add more key for x-axis

Some train loops do not have epoch and iteration key, for example, reinforcement learning. Current ChainerUI fixes x-axis key (epoch, iteration, and elapsed_time) so, ChainerUI cannot support these training loops. refs #27

It is better that user can select any x-axis keys but this is heavy task a little, so we decided to add more keys on default for x-axis. We got an advice from ChainerRL team, and will add episode and step for x-axis key on default.

Lack npm build on CI script

When build chainerui module manually, require to build JS module, but CI script does not have it.

Visualize images generated in training phase

This issue is for tracking the implements for Image API, which helps users to show images generated in training phase. I split the feature to several PRs.

• Server side API design: #83
• add base view on frontend: #83
• return error on view, ex) not install PIL, not have images or any other
• cache PNG data to DB: #119
• polling to call GET API to automatically update
• lazy load to image
• upload images to show already existed
• more effective UI design
• place a link to access the images page
• user guide
• examples

Wrong project's plot when moving to another page

When I move to project A's page to project B's page,
A's plot remains on the page until the browser finishes loading B's log from the server.

This issue occurs especially when B's log is heavy.

Python version restriction in requirements.txt

According to requirements.txt, python version is restricted to python_version < '3.4' .
Is it typo of python_version >= '3.4' ?.

Show module version

1. should support $ chainerui --version
2. should support $ python -c "import chainerui; print(chainerui.__version__)"

Fix temporary file permission issue

related to this issue chainer/chainer#4528

Some util functions also use temporally file logic, so need to check and fix. such as

chainerui/chainerui/utils/save_args.py

Line 29 in 9e80dc9

fd, path = tempfile.mkstemp(prefix='args', dir=out_path)

Disappear metric on line chart

From v0.6.0, a tooltip displayed on line chart is hidden. There is no way around checking actual values on browser.

Uncaught Error: React is running in production mode

I encountered an error in javascript console.

My environment is:

• Browser: Google Chrome
• ChainerUI version: v0.2.0

Uncaught Error: React is running in production mode, but dead code elimination has not been applied. Read how to correctly configure React for production: https://fb.me/react-perf-use-the-production-build
    at <anonymous>:21:35

DEPRECATED webpack-serve

😢
57157d5#diff-0a6694dd0d6b93a2aa14f32e3d873b4cR45

Cause an error in the frontend

Graph style for yLeftAxis and yRightAxis is the same

This is a feature request. I felt if the style of the graphs of yRightAxis is different than yLeftAxis (e.g. dashed or dotted), it would be easier to distinguish them. Currently both are solid and the same colour which is somewhat confusing.

Stop to use legacy install style

python setup.py install -> pip install . or pip install -e .

ref:
https://docs.python.org/3/install/
chainer/chainer#4366

Overlap order of tool tip and table looks odd

~v0.5.1

master branch, merged #205

(Tool tip showed by hovering lines are disappeared on current v0.6.0 version)

Broken log file halts other log files' rendering

When the project includes the broken log file, I would like to know the log file is broken on ChainerUI.
At worst I want to view other log files that are not broken. ChainerUI should skip to parse broken log files.

Current:

127.0.0.1 - - [02/May/2018 16:17:40] "GET /api/v1/projects/6/results HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1997, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/site-packages/flask/views.py", line 149, in dispatch_request
    return meth(*args, **kwargs)
  File "/Users/ofk/git/chainerui/chainerui/views/result.py", line 36, in get
    result = crawl_result(result.id)
  File "/Users/ofk/git/chainerui/chainerui/tasks/crawl_result.py", line 79, in crawl_result
    crawled_result = crawl_result_path(current_result.path_name, is_updated)
  File "/Users/ofk/git/chainerui/chainerui/tasks/crawl_result.py", line 38, in crawl_result_path
    result['logs'] = load_result_json(result_path, 'log')
  File "/Users/ofk/git/chainerui/chainerui/tasks/crawl_result.py", line 22, in load_result_json
    _list = json.load(json_data)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/Users/ofk/.pyenv/versions/anaconda3-4.2.0/envs/sandbox/lib/python3.5/json/decoder.py", line 342, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 8 column 6 (char 179)