Comments (12)
Posted on Riot.im, but added a post-it (going through all my pending post-its today 🎉 ) to add the script and link here.
#FOR: https://github.com/cylc/cylc/issues/2483
#!/usr/bin/env python3
# coding=utf-8
import sys
import tempfile
# hack until setup.py is done and we are installing it via pip/setuptools
sys.path.append("/home/kinow/Development/python/workspace/cylc/lib/")
from eralchemy import render_er
from cylc.rundb import CylcSuiteDAO
def main():
"""Create cylc public database, then run diagram creation tool."""
with tempfile.NamedTemporaryFile() as tf:
# is_public=False triggers the creation of tables
CylcSuiteDAO(db_file_name=tf.name, is_public=False)
db_name = "sqlite:///{}".format(tf.name)
render_er(db_name, "cylc-database.png")
if __name__ == '__main__':
main()
And a sphinx plugin that does the same without the need for this script (but I have not tested it yet): https://github.com/sphinx-contrib/sqlalchemyviz
The main obstacle right now for this are the database constraints, which are missing in the DB schema. Adding them should be easy, but there is a small risk of causing performance issues as sqlite will do some constraint checking when editing data.
Here's what it looks like at the moment:
from cylc-doc.
Yes, I guess we can add some foreign keys between tables - mostly for mapping task names and cycle points.
from cylc-doc.
Something from the task_jobs
table that wasn't clear (via email):
The intention…
time_submit
: time just before running “qsub” of a job.
time_submit_exit
: time just after “qsub” of a job returns.(However, I am not sure if we have implemented the above correctly or not.)
time_run
: time when the job calls “cylc message started”
time_run_exit
: time when the job calls “cylc message succeeded|failed”.
from cylc-doc.
Note that I'm now writing a bunch of suite-performance reporting scripts which slightly rely on what the database schema is like, so it would be even cooler if it was documented. They can be redone pretty easily however.
from cylc-doc.
I guess we should begin by adding more docs in the code where the schema is defined:
https://github.com/cylc/cylc/blob/master/lib/cylc/rundb.py
I'll work on as soon as I have a chance.
from cylc-doc.
from cylc-doc.
@tfeb Please feel free to raise PRs.
from cylc-doc.
The proposed adoption of the Sphinx documentation engine opens up the possibility for auto-documentation in many areas. Shall we supersede this issue with #2651?
from cylc-doc.
I was doing the reverse engineer with the IDE when I wanted to pick at the class <-> db relationship, but not sure how precise it is. Definitely interested in seeing this one done (and keen to help testing/reviewing).
from cylc-doc.
(Interesting, because I thought most of the tables should have some primary keys.)
from cylc-doc.
Sorry, should have mentioned it is actually the foreign key constraints that are missing. You are correct, primary keys should be there all right.
from cylc-doc.
Would be good, even just for the purposes of standardisation.
from cylc-doc.
Related Issues (20)
- slides: re-enable slides build
- document flow-specific hold and release
- Explain Host Selection HOT 1
- optional outputs: better document how family triggers work with optional outputs
- Document meaning of inactivity timeout HOT 3
- document: cylc set HOT 3
- Request for clarification about CYLC variables for SHARE_DIR and WORK_DIR HOT 2
- add edit this page link
- Document queues better HOT 1
- Problematic overloaded "window" terminology HOT 1
- troubleshooting: diagnosing incorrect task status HOT 4
- document late event expiry
- changes: document cylc-ui#1755
- Troubleshooting Guide: Additional things to say
- task state vs status
- Make the "Use `set -euo pipefail` in Bash scripts" recommendation less ambiguous
- interventions: re-run a workflow from a specified cycle point
- Optional outputs can cause legit partially-satisfied prereqiuisites
- interventions: setting off-flow prereqs
- changes: info view
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cylc-doc.