iaaa-lab / space-ql

TFG Alejandro Magallón

space-ql

TFG Alejandro Magallón

Diagrams

High concept diagram

This diagram is a very high level specification of each piece of the system, so it is easy to explain to people that doesn't know much about technology.

Deployment diagram

This diagram shows the deployment scheme of the application. There are three main modules:

The Client: A KotlinJS app that serves a React web application.

The server: A Spring-Kotlin app that acts as an API and serves the database information to the client. The client uses GraphQL to make the requests to the server.

The database: In a first approach the database will be a folder with different XML files containing some metadata. Later it is supposed to be a graph-oriented database. In the project's final state, the document retrieval process can be automated in a "Harvest" app, that may access the data source and store the data in the database.

Client's design

This diagram shows a first approach to the client's design. It is supposed to be a title-based search, so it wouldn't take any other parameter. On future releases it must let the user search for any possible metadata and filter the different results.

This is the first design for the client. It's a very simple design with an input, and some result cards. That cards show the title and any extra metadata chosen. It is thought to be implemented with react and ElasticSearch.

space-ql's Issues

Create an architecture's diagram

Created Diagrams: https://drive.google.com/file/d/1e8DXZEROsNDWstLCJ0czZ0N2_HjtDGcm/view?usp=sharing

Check apps built with Elasticsearch + React

• Example 1 - web
• Example 1 - code

The app is meant to look somehow like this one

• Example 2
• Example 3

Check the kotlin-redux library

• Getting Started with Redux
• Kotlin-wrappers Redux

EXTRA - Add translation to react for some elements

Use something like this framework

Improve the look of the page

Add an option to allow the user to remove related resources from an element

Use the graphql mutations to allow the user to remove a related document from the list of coupled elements from a service or dataset by using a button from the result view

Create an ElasticSearch instance

Following this video should be enough to serve the API with ElasticSearch.

• See how to configure a local or cloud ElasticSearch instance
• See how to properly retrieve the data from it

Wrong results for the searchbar

Expected Behavior

As you type in the searchbar, suggestions appear. When you click a suggestion, it is written inside the text field. After pressing the "enter" button, the page updates with the new results. The suggestions are all the titles stored into Elasticsearch, so if you click a title, two things could happen:

• The application redirects you to the page of the result with that title
• The title is written in the searchbar and you can search for it, so it is returned as any other search

Current Behavior

When you type, suggestions appear. When you click a suggestion it is written inside the text field and you can edit it. When you press the "enter" button it searches for the selected input. When you look for a long title, it returns an empty list of results, even though it is a title for a result, so we know that at least one of the results should be returned.
After doing some tests, Elasticsearch doesnt't return some of the results when you type the whole title in the match param of the search.

Steps to Reproduce

1. Click on the searchbar from the main page
2. Select a title from the list, for example "Spanish General Directorate for Cadastre - INSPIRE Addresses. - ATOM"
3. Press "enter"
4. No result would be displayed at all, when we know that, at least, one result should appear

Possible Solution

Change the click behaviour of the Autocomplete, so when the user picks a suggestion, it automatically redirects to the Result page. (It is not sure that it could work, because it depends on Elasticsearch and it doesn't seem to return the results properly when the whole title is typed)
A possible solution for the problem may be using another function to get the suggestions instead of a list with all the titles. The function is alredy done, but it is not used because the Autocomplete component does the filtering job, so it seemed like a better idea to use the whole list of titles, so it would be more precise. This function/call to the server, could be used on

Create a script to load the metadata to ElasticSearch

onChange and onSubmit not working with the MUI components

The Searchbar.kt file contains a form and a TextField. Both of them require an event handler. Form should use a onSubmit handler and the TextField should trigger an onChange handler. When I try to set the handlers, the system is specting a handler from an HTMLDivElement, but the handlers are made for a Form and an Input type elements.

I couldn't figure out how to solve this but I will keep working on it.

Facets

• Remove the facets with 0 documents
• Order the facets from higher to lower number of documents

Can't find the JS generated

When i try to run the app it seems to run but the html doesn seem to find the .js generated. I've tried multiple locations such us /resources/space-ql or /static/space-ql or just space-ql but nothing happens.
I've also tried to change the gradle build script to make it look like this (trying to configure the tasks as they appear there to put the JS into every jar) but it only creates more warnings/errors.
The last thing i've tried is to create a CommonMain folder with a resources folder on it containing the Html. I thought that maybe that way both the html and the js are accessible for every part of the app and so it could find the js from the html, but i doesn't work either.

Improve the database system for the test application

Searching directly in folders and files makes things harder. It may be easier if there was a database system that provides the content of each file as plain text, allowing us to attach some info like an id or a name to each file, making them easy to find.

Minimal title-based browser - Bootstrap Implementation

The app consists on a client that makes requests to a simple API. The API is built using Spring and the data are static XML files.

Explore DGS

DGS is a production-ready Netflix framework for building GraphQL servers quickly. It can boost your work!

https://netflix.github.io/dgs/

Config4k not working on KotlinJS

The library Config4k, which is supposed to be a Lightbend Config's wrapper for kotlin, is not being found by IntelliJ when trying to import the package from jsMain. On the other hand, it does work when the package is imported from the jvmMain, so it seems like it isn't compatible with KotlinJS.

Move the generated model to the commonMain

Create a unified model in the commonMain and use only one version of it. It should assume that more null values are possible.
It could be improved using the code generation plugin for DGS.

Learn Kotlin-react

First prototype - Server (ElasticSearch)

• ES searches for the whole content file
• ES searches for specific fields like title and description
• ES can search for fields like the contact points or the type of metadata being served (service or dataset)
• ES and the client work together to serve a dynamic experience for the user, who can navigate very intuitively through the data

Decide the client-side front-end's appearance and functionality

Add suggestions to searchbar

The Elasticsearch dependencies are not working

I have to create a "ElasticsearchRepository<>" to connect the app with the Elasticsearch instance, but it doesn't seem to find the correct dependencies (org.springframework.data) although they should be installed through the starter-data dependency. I've tried to install it directly with the data-elasticsearch dependency but it doesn't find it either.

MUI client

Build a client using MUI

Build the documentation with Asciidoctor

Change the data model

The data model must be adapted in order to truly show how graphql helps this kind of applications.

Modify the appearance of the Documentation so it fits the LaTex document

Also the whole documentation should be checked, It's a first version and some things may be added or removed from It.

General system improvements

• Create aliases for the different formats (gml, csv...)
• Create parent-child relations between organizations and sub-organizations for the contact points
• Make the language on the result info clickable
• Add the formats to the facets

GraphQL builds the Schema two times

Output:

---

APPLICATION FAILED TO START

---

Description:

There are problems with the GraphQL Schema:
* 'Query' type [@1:1] tried to redefine existing 'Query' type [@1:1]
* 'MetaData' type [@5:1] tried to redefine existing 'MetaData' type [@5:1]

Related resources not working

Create some tests for each module