Coder Social home page Coder Social logo

deprecated-renku-aqs's Introduction

renku-aqs

params

$ (cd renku-aqs-test-case/; renku aqs params)
+--------------------------------------+-------------------+--------------+
| Run ID                               | AstroQuery Module | Astro Object |
+--------------------------------------+-------------------+--------------+
| 228555e4-151a-4440-919d-f1269132a0fb |    SimbadClass    |   Mrk 421    |
| 76943a72-30bf-4e9e-ad92-3dd068424968 |    SimbadClass    |   Mrk 421    |
| 0320ea0d-eee2-4b50-aa3e-b64c557b9bf2 |    SimbadClass    |   Mrk 421    |
+--------------------------------------+-------------------+--------------+
@prefix local-renku: <file:///home/savchenk/work/oda/renku/renku-aqs/renku-aqs-test-case/.renku/> .
@prefix oda: <http://odahub.io/ontology#> .
@prefix odas: <https://odahub.io/ontology#> .

local-renku:4ab60eb4-d5e7-11eb-a2dc-b5ff8b3b1162 a oda:Run ;
    oda:isRequestingAstroObject odas:AstroObjectMrk_421 ;
    oda:isUsing odas:AQModuleSimbadClass .

local-renku:53e67e80-d5ea-11eb-a2dc-b5ff8b3b1162 a oda:Run ;
    oda:isRequestingAstroObject odas:AstroObjectMrk_421 ;
    oda:isUsing odas:AQModuleSimbadClass .

local-renku:dd481450-d5e4-11eb-a2dc-b5ff8b3b1162 a oda:Run ;
    oda:isRequestingAstroObject odas:AstroObjectMrk_421 ;
    oda:isUsing odas:AQModuleSimbadClass .

Graphical visualization of the graph

Starting from the knowledge graph generated and enriched during the various executions of the notebooks present within the repository, this is queried to retrieve the needed information, perform some inferring and generate a graphical representation.

In particular, two commands are provided:

  • display to generate a representation of the graph over an output image
  • show-graph to start an interactive visualization of the graph over the browser

display command

CLI command to generate a graphical representation of the graph over an image.

In particular, the following information are elaborated:

  • inputs/arguments/outputs of the notebook execution;
  • astroquery modules used and the main query methods called (astroquery api).

Parameters

  • --filename The filename of the output file image, until now, only png images are supported (eg --filename graph.png), default is graph.png
  • --input-notebook Input notebook to process, if not specified, will query for all the executions from all notebooks
  • --no-oda-info Exclude oda related information in the output graph, an output much closer to the lineage graph provided in the renkulab will be generated
$ renku aqs display

Specify executed notebook

$ renku aqs display --input-notebook final-an.ipynb

Do not display oda-related information

$ renku aqs display --input-notebook final-an.ipynb --no-oda-info

show-graph command

CLI command to generate an interactive graphical representation of the graph.

$ renku aqs show-graph

The user can interact with the graph via a single click on one of its nodes: upon clicking, a SPARQL query is dynamically built, and this will retrieve all the nodes and edges directly connected to the clicked node, as shown in the animation below. Once the node has been expanded, the newly added nodes, along with the other leaves nodes that are connected to the previously clicked node, can be re-absorbed by clicking again on the very same node. This is also visible in the animation below.


The interface provides the user with a number of adjustable options:

  • Change graph layout: Two possible layouts are at the moment supported

    • random: where nodes and edges are displayed in a random fashion over the dedicated frame (this layout is used in the above picture)
    • hierarchical: where a hierarchical visualization is applied to the nodes and the edges, an example of this is displayed in the image below
  • Enable/disable selection of subsets of nodes: filtering of certain subset of nodes (eg astroquery-related nodes). This is configurable via a dedicated json file.

  • Apply reductions on the graph: allows to enable the display of certain nodes with the relative child ones absorbed within, and viceversa (eg in the example images, the Plan node has a number of child nodes, those can be visualized inside the Plan, left image, as well as regular ones, right image) This is configurable via a dedicated json file.


  • Enable/disable graphical configurations for the graph: enable/disable a set of graphical configurations for the graph (nodes and edges), where each configuration is loaded from a dedicated json file. In the image displayed below, two json files have been loaded, and the related checkbox will enable/disable that configuration.

The functionalities for the graph drawing and its behavior are developed in javascript and are avaialble at the following repository.

Installation of the plugin

Currently, the plugin is developed using the version 2.2.0 of renku-python. Please make sure such version is installed by running:

renku --version

The versioning of the dependency is guaranteed during the installation of the plugin, and in addition a dedicated check will be performed during the import.

For the installation of the plugin via pip:

pip install renku_aqs

deprecated-renku-aqs's People

Contributors

volodymyrss avatar burnout87 avatar

Stargazers

Rok Roškar avatar

Watchers

James Cloos avatar  avatar  avatar

deprecated-renku-aqs's Issues

exception in renku

    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/savchenk/miniconda-recent/lib/python3.9/site-packages/renkuaqs/plugin.py", line 34, in <module>
    from renku.core.management.command_builder import Command, inject
ModuleNotFoundError: No module named 'renku.core.management.command_builder'

renku_aqs breaks renku installation for python 3.10 and python 3.8

From a fresh virtual environment running python 3.10, I installed renku 2.2.0 and then renku_aqs.

After installing the plugin, renku is giving this error that prevents any further use. I have to redo the virtualenv to get rid of the issue.

renku --version
WARNING:nb2workflow.ontology:unable to import odakb.sparql (No module named 'odakb') but proceeding anyway
WARNING:nb2workflow.ontology:unable to import owlready2: No module named 'owlready2'
Traceback (most recent call last):
File "/home/ferrigno/.venv/renkuVE/bin/renku", line 5, in
from renku.ui.cli import cli
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renku/ui/cli/init.py", line 99, in
from renku.ui.cli.dataset import dataset
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renku/ui/cli/dataset.py", line 825, in
@add_provider_options()
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renku/ui/cli/dataset.py", line 808, in add_provider_options
providers = [p for p in ProviderFactory.get_add_providers() if p.get_add_parameters()] # type: ignore
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renku/core/dataset/providers/factory.py", line 49, in get_add_providers
return [p for p in ProviderFactory.get_providers() if issubclass(p, AddProviderInterface)]
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renku/core/dataset/providers/factory.py", line 43, in get_providers
providers = get_supported_dataset_providers()
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renku/core/plugin/dataset_provider.py", line 43, in get_supported_dataset_providers
pm = get_plugin_manager()
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renku/core/plugin/pluginmanager.py", line 41, in get_plugin_manager
pm.load_setuptools_entrypoints("renku")
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
plugin = ep.load()
File "/usr/lib/python3.10/importlib/metadata/init.py", line 171, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renkuaqs/init.py", line 28, in
import renkuaqs.graph_utils as graph_utils
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renkuaqs/graph_utils.py", line 32, in
from renkuaqs.plugin import AQS
File "/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/renkuaqs/plugin.py", line 35, in
from aqsconverters.io import AQS_DIR, COMMON_DIR
ImportError: cannot import name 'AQS_DIR' from 'aqsconverters.io' (/home/ferrigno/.venv/renkuVE/lib/python3.10/site-packages/aqsconverters/io.py)

trying to upgrade aqsconverters did not help

Not found path(s)

When running the params command, I get the following error:

 "file://~/renku-aqs-test-case/.renku/Mrk 421" does not look like a valid URI, I cannot serialize this as N3/Turtle. Perhaps you wanted to urlencode it?

implement use cases

the library can be used in several places

  1. MMODA frontend (for exploring the result context)
  2. renku session (to help finding terms for developing workflow, and also for exploring context)
    a. with jupyterlab plugin
    b. with jupyter server plugin

I suggest to start with 2b and 2a

edit: to be clear 2a and 2b are alternative implementations of 2. 2b is easier (at least for me it's clear how to do, it's mostly python). 2a has a potential to be nicer but it's a bit of a new teritory. But I saw many jupyter lab plugins and the sdk seems simple enough. So we could explore 2a, and if it hard - resort to 2b. and then if we like the use case - put more effort to implement 2a.
if 2a is not too hard - we can do it without 2b.

make possible to auto-reload graph when the error is returned?

I get this:

"""Error while generating the output graph:

Project version is outdated and a migration is required. Run renku migrate command to fix the issue.""

and when I migrate, the message does not go away. Could it be made so that in this case auto-reload still works?

`Inspect` command

Additional command inspect , to inspect the input notebook and attach those info to the file itself, still using nb2rdf. Add those info in a different place to be added to the graph exported initially and insert also the ontology inside this graph

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.