edgerun / galileo-db Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 1.0 91 KB

Core model and client tools for the Galileo Experiment Database

Makefile 0.94% Shell 1.16% Python 97.90%

galileo-db's People

Contributors

Stargazers

Watchers

Forkers

silv-io

galileo-db's Issues

Manager CLI not available in container

Invoking the galileodb.cli.manager through docker exec results in an error, stating that the file is not available.
I've looked into the Dockerfiles for the recorder and due to the mulit-stage build the scripts are not copied in the final step.

Solving this issue should be as easy as adding COPY --from=builder /src /src to the prod stage.

CLI Recorder does not flush traces on termination

Terminating a galileodb.cli.recorder does not trigger a TraceRecorder flush.
This leads to some traces having their experiment id assigned, while the ones from the last flush, which is invoked by the finally statement upon joining the TraceRecorder process, don't seem to be in the database when exp.finalize_experiment is called.

https://github.com/edgerun/galileo-db/blob/master/galileodb/cli/recorder.py#L65-68

https://github.com/edgerun/galileo-db/blob/master/galileodb/recorder/traces.py#L63

add DB function and CLI command to get experiment IDs of running experiments

update delete_experiment to also remove events

we added experiment events, which are not deleted in

galileo-db/galileodb/sql/adapter.py

Lines 181 to 196 in 364f788

    
           def delete_experiment(self, exp_id: str): 
        
               experiment = self.get_experiment(exp_id) 
        
               if experiment is None: 
        
                   raise ValueError('No such experiment %s' % exp_id) 
        
               stmts = [ 
        
                   "DELETE FROM `telemetry` WHERE EXP_ID = " + self.db.placeholder, 
        
                   "DELETE FROM `traces` WHERE EXP_ID = " + self.db.placeholder, 
        
                   "DELETE FROM `experiments` WHERE EXP_ID = " + self.db.placeholder, 
        
               ] 
        
               for sql in stmts: 
        
                   try: 
        
                       self.db.execute(sql, (exp_id,)) 
        
                   except Exception as e: 
        
                       logger.exception('Exception while executing %s', sql, e)

subsystem of telemd telemetry record is not recorded

The subsystem component of a telemd telemetry record is not yet part of the galileo-db model, so I built a workaround that just appends the subsystem to the metric. But apparently I forgot to pass it to the record:

galileo-db/galileodb/recorder/telemetry.py

Lines 50 to 54 in 364f788

    
           metric = t.metric 
        
           if t.subsystem: 
        
               metric += '/' + t.subsystem 
        
           self.buffer.append(GalileoTelemetry(float(t.timestamp), t.metric, t.node, val, self.exp_id))

edgerun / galileo-db Goto Github PK

galileo-db's People

Contributors

Stargazers

Watchers

Forkers

galileo-db's Issues

Manager CLI not available in container

CLI Recorder does not flush traces on termination

add DB function and CLI command to get experiment IDs of running experiments

update delete_experiment to also remove events

subsystem of telemd telemetry record is not recorded

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	def delete_experiment(self, exp_id: str):
	experiment = self.get_experiment(exp_id)
	if experiment is None:
	raise ValueError('No such experiment %s' % exp_id)

	stmts = [
	"DELETE FROM `telemetry` WHERE EXP_ID = " + self.db.placeholder,
	"DELETE FROM `traces` WHERE EXP_ID = " + self.db.placeholder,
	"DELETE FROM `experiments` WHERE EXP_ID = " + self.db.placeholder,
	]

	for sql in stmts:
	try:
	self.db.execute(sql, (exp_id,))
	except Exception as e:
	logger.exception('Exception while executing %s', sql, e)

	metric = t.metric
	if t.subsystem:
	metric += '/' + t.subsystem

	self.buffer.append(GalileoTelemetry(float(t.timestamp), t.metric, t.node, val, self.exp_id))