sirixdb / sirix-svelte-frontend

A GUI console for SirixDB, using Svelte/Sapper.

Project Setup - Setting up SirixDB locally

Unfortunately, this is currently a bit messy, but still rather straightforward.

There are several resources required to run SirixDB, and they can all be found in the resources directory of this repository.

SirixDB depends on keycloak for authentication, so we first need to run (in the directory in which you have the above resources):

docker-compose up -d keycloak

It can take a minute for keycloak to be ready for connections. If you like, you can use the wait.sh script in the above mentioned resources directory to wait until keycloak is ready. A script from the resources folder initializing keycloak with a user "admin" with the password "admin" is run automatically.

Once keycloak is ready, navigate to http://localhost:8080/admin and login using the credentials admin/admin. Click on Users in the sidebar navigation, then click Add user. Enter the name of the new user (admin, for example), select email verified, and click Save. Then, click on the credentials tab and create your password. In the Credentials tab, add all available roles.

Then, run

docker-compose up -d server

The Sirix database is now ready for use.

Alternately, you can simply run

docker-compose up -d

to immediately create both the keycloak and sirix containers. Although the sirix container won't be able to immediately connect, it will continue retrying.

Developing

Code documentation

A lot of documentation is missing. However you can find general descriptions of the files in a directory in the README.md file in that directory. Currently, only some directories these READMEs, they are in the process of being added.

Before developing, run npm run build:tailwind. This command should also be run after running the build or export NPM scripts.

Developing in the browser

1. Start the sapper server: npm run dev.
2. In your browser navigate to http://localhost:3000.

Develop with Tauri

You need to follow the tauri setup guide first. Note that the NPM tauri package is included in the dependencies of this project. This is not yet configured to work on all platforms, however.

1. run npm run dev-for-tauri.
2. in another terminal, run npm run dev:tauri.

To build with Tauri

1. run npm run export:tauri.
2. run npm run build:tauri.

sirix-svelte-frontend's Issues

Use Svench for components

High-level components should be housed in sub-projects, where svench should be used, so as to enable developing components in isolation.

Allow for edits to resources in the /database route

We should possibly allow for updates to resources to be done from the resource view. The possibility to update can only be allowed when is viewing the latest revision. This is independent of #12, which would also edits, as this issue is for a point-and-click option.

Update sirix environment instructions

The sirix-python-client repository contains a resources folder with all files necessary to run a sirix database locally. That file should be copied to this repository, and instructions in the README updated.

To fetch new data, it seems that always two requests are sent

It might be okay, I'm just wondering why :)

Bug in the diffing-view

Precondition:

Execute the following script in the cli:

resource json-diff json resource
create --data '["test", "test"]'
delete --nodekey 2
update --nodekey 1 --data 'diff'
delete --nodekey 3
update --nodekey 1 --data '[{"interesting": "!"}]'

Then in the Svelte-frontend compare revision 4 and 5.

However, the children are swapped.

It's inserted as the first child actually:

default/json-diff/resource (JSON)> diff -s 4 -e 5
[{'insert': {'data': '[{"interesting":"!"}]',
             'depth': 2,
             'deweyID': '1.3.2.3',
             'insertPosition': 'asFirstChild',
             'insertPositionNodeKey': 1,
             'nodeKey': 5,
             'type': 'jsonFragment'}}]

Document all components

Every directory in the src folder should contain a README markdown file describing the purpose and relationship and the components and files therein. If necessary, there should be additional markdown files documenting the components.

Delete messes up the diff view (in the diffing branch)

I've attached the new database/resource.

Comparing revision 1 and 7 results in:

However, the diff is (only the first object has been deleted (and something is completely weird): delete json jn:doc('json-db','business-service-providers.json')=>paths=>"/business_service_providers/search"=>get=>parameters[[0]] :

localhost/json-db/business-service-providers.json (JSON)> diff -s 1 -e 7
[
  {
    "delete": {
      "nodeKey": 34,
      "deweyID": "1.17.113.0.17.0.17.0.49.0.17",
      "depth": 6
    }
  },
  {
    "insert": {
      "nodeKey": 188,
      "insertPositionNodeKey": 86,
      "insertPosition": "asRightSibling",
      "deweyID": "1.17.113.0.17.0.17.0.49.0.97",
      "depth": 6,
      "type": "jsonFragment",
      "data": "{\"name\":\"keyword\"}"
    }
  },
  {
    "insert": {
      "nodeKey": 191,
      "insertPositionNodeKey": 188,
      "insertPosition": "asRightSibling",
      "deweyID": "1.17.113.0.17.0.17.0.49.0.113",
      "depth": 6,
      "type": "jsonFragment",
      "data": "{\"name\":\"keyword\"}"
    }
  },
  {
    "insert": {
      "nodeKey": 194,
      "insertPositionNodeKey": 191,
      "insertPosition": "asRightSibling",
      "deweyID": "1.17.113.0.17.0.17.0.49.0.129",
      "depth": 6,
      "type": "jsonFragment",
      "data": "{\"name\":\"keyword\"}"
    }
  },
  {
    "insert": {
      "nodeKey": 197,
      "insertPositionNodeKey": 194,
      "insertPosition": "asRightSibling",
      "deweyID": "1.17.113.0.17.0.17.0.49.0.145",
      "depth": 6,
      "type": "jsonFragment",
      "data": "{\"name\":\"keyword\"}"
    }
  },
  {
    "insert": {
      "nodeKey": 200,
      "insertPositionNodeKey": 197,
      "insertPosition": "asRightSibling",
      "deweyID": "1.17.113.0.17.0.17.0.49.0.161",
      "depth": 6,
      "type": "jsonFragment",
      "data": "{\"name\":\"keyword\"}"
    }
  }
]

json-db.zip

Layout customization

• Enable resizing the various "sections" in each route,
• as well as allow drag-and-drop to rearrange the layout.
  (a section would be, for example, (in the resource route) the bar showing database/resource names, the history timeline, the resource view. (in the query route) the query box, the query result display, the saved/old queries bar).

I imagine this would require

• using something like svelte-dnd-action to allow rearranging the layout,
• and using the resize CSS property to allow native element resize.
• I imagine the resize property would exist on containers that would each containe a single "region" that is draggable.
• There should be listeners to resize the container region when a large section is moved to a small region (and swap the small section to the other region).
• There should be listeners to accommodate the resize of a region by resizing other regions.

This would require refactoring the layouts to be compatible with drag and drop.

Deleting a resource => Resource tree-view is still visible

After deleting a resource the resource tree-view is still visible. The history view is not shown anymore :)

Ascending/Descending label is always displayed

When I'm scrolling down in the resource tree view, the ascending/descending label text from the history view is always displayed, but the history view in general "scrolls away", such that it's not displayed anymore.

Add a /query route

A route for free-form (not limited to a single resource or database) queries is required. It could probably use many of the components from the /databases route.

XML display

Currently, the /databases route is only capable of displaying JSON. We need to add complete XML functionality.

General visualizations issue

There are many visualizations described in @JohannesLichtenberger's master thesis, these need to be recreated.
A PDF of the master thesis is here.

Add queries to the /database route

Currently, resources can be viewed in the /database route, but the resources cannot be queried. This should be rectified.

Query input should be resizable

The query input field currently is too small for well formatted slightly longer queries. It thus should be resizable.

Switch to new client library

The client library dependency is pinned to an old commit. Now that the client library rewrite has been finished, we should start using the rewritten version. In many ways, the interface of the two versions should be similar.

Database deletion / window still open

After deletion of a database the window to delete the database can not be closed and is not closed automatically in Chrome.

Name of JSON file is not automatically used for the respective field as the resource name

My browser is Chrome, most recent version

Notifications upon user-initiated (network) operations

We should show a notification upon user log in/out, query success/failure, database/resource creation/deletion.

• It should be possible to show multiple notifications at once (and hence, the renderless-svelte notification component would not be suitable), in which case the earlier notifications should be push upward.
• It should also be possible to click and dismiss notifications before they automatically expire.

Sometimes data is not displayed, when expanding a node

It happens in the cityofchicago resource, that the server sends a reply, but the expanded subtree is not displayed:

Auto-expand deleted/updated/inserted and replaced nodes

We should auto-expand all changed nodes, received by the diffs per default. That way we clearly display what has changed :-)

Settings page for lazy loading parameters on Firefox does not work

Firefox can't render the lazy loading input fields, it seems. Furthermore you can't close the window via the X.

Creating database => SVG width/height NaN error

I'm getting this error in the Chrome Console after adding a JSON database.

Bug: Consecutive Diffs (diff not visible in the frontend)

I've attached a database and two resources (only the resource business-service-providers.json is important).

When I compare revision 1 and 2 I should get a diff, but the frontend doesn't show this diff.

localhost/json-db/business-service-providers.json (JSON)> diff -s 1 -e 2
[
  {
    "insert": {
      "nodeKey": 188,
      "insertPositionNodeKey": 86,
      "insertPosition": "asRightSibling",
      "deweyID": "1.17.113.0.17.0.17.0.49.0.97",
      "depth": 6,
      "type": "jsonFragment",
      "data": "{\"name\":\"keyword\"}"
    }
  }
]

json-db.zip

Network errors when clicking on a revision in the history view

When clicking on a revision in the history view, the get the resource tree view on the right, there are Sapper network calls failing:

set up github actions to build and release desktop apps using Tauri

Query then diff two revisions of a resource

Maybe first it would be great to idenfity two revisions to compare as we do already. Then we'd however have a textarea query field to execute a query in order to find a specific subtree in a resource. Ideally the output should somehow be a single nodeKey as input to the serializer on the server GET https://example.org/database/resource/diff?first-revision=3&second-revision=7&startNodeKey=${resultNodeKeyFromQuery} for instance .

Create diff view

We currently have a component to view a json resource for any revision. We need to add functionality to view the resource as a differential between two revisions.

Query: Check for empty String in return value from Server

Some queries return an empty String instead of a valid JSON response for the queries (updating queries and store functions for instance).

Sunburst visualization

Create a zoomable multi-layered sunburst visualization, displaying the density (in terms of descendantCount) of various parts of a resource.

This will probably use D3, but I don't have any further ideas than that.

Bug: Consecutive Diffs

I've attached a database and two resources (only the resource business-service-providers.json is important).

When I compare revision 1 and 3 the diffs have a false depth (+1). Revision 3, the relevant part looks as follows:

          {
            "name": "offset",
            "in": "query",
            "description": "The offset parameter defines the offset from the first result you want to fetch. Unless specified the API returns 10 results at a time.",
            "required": false,
            "type": "string",
            "format": "string"
          },
          {
            "name": "keyword"
          },
          {
            "name": "keyword"
          }

In the frontend it's displayed as:

json-db.zip

Add unit tests as well as e2e tests

We need a good test coverage to avoid regressions.