databucket / databucket-server

Open Source Test Data Management Tool

    ██████╗  █████╗ ████████╗ █████╗ ██████╗ ██╗   ██╗ ██████╗██╗  ██╗███████╗████████╗    ██████╗
    ██╔══██╗██╔══██╗╚══██╔══╝██╔══██╗██╔══██╗██║   ██║██╔════╝██║ ██╔╝██╔════╝╚══██╔══╝    ╚════██╗
    ██║  ██║███████║   ██║   ███████║██████╔╝██║   ██║██║     █████╔╝ █████╗     ██║        █████╔╝
    ██║  ██║██╔══██║   ██║   ██╔══██║██╔══██╗██║   ██║██║     ██╔═██╗ ██╔══╝     ██║        ╚═══██╗
    ██████╔╝██║  ██║   ██║   ██║  ██║██████╔╝╚██████╔╝╚██████╗██║  ██╗███████╗   ██║       ██████╔╝
    ╚═════╝ ╚═╝  ╚═╝   ╚═╝   ╚═╝  ╚═╝╚═════╝  ╚═════╝  ╚═════╝╚═╝  ╚═╝╚══════╝   ╚═╝       ╚═════╝

About project

Databucket is an Open Source Test Data Management Tool. It delivers features to effectively create and maintain test data and metadata for extensive tests on multiple projects, environments, and various integrated tools at the same time. Databucket stores data in elastic structures, which gives a ready approach to maintain constantly changing test data in the software development process.

Getting started with Databucket

• First you can read about key features on databucket.pl
• Next, you can dive deeper in the Wiki page.

Contribution rules

1. During development, we are following GitHub flow. In order to provide change or fix in project, one has to:

   1. Create a branch from the repository.
   2. Create, edit, rename, move, or delete files.
   3. Send a pull request from your branch with your proposed changes to kick off a discussion.
   4. Make changes on your branch as needed. Your pull request will update automatically.
   5. Merge the pull request once the branch is ready to be merged.
   6. Tidy up your branches using the delete button in the pull request or on the branches page.

   For more information see Understanding the GitHub flow.

2. Each commit must contain clear and descriptive commit messages.

3. Source code must be properly formatted

4. Source code must be checked according to checkstyle rules

5. Changes must be well-tested. See also Testing strategy chapter.

Setup

Prerequisites

Installed below dependencies:

• Java 17
• NodeJS 18

Useful commands

command	what it does
./gradlew clean assemble	compiles java classes
./gradlew test	runs all tests
./gradlew buildFrontend	installs frontend dependencies and builds frontend
./gradlew copyFrontend	buildFrontend and copies frontend file to java resources
export SPRING_PROFILES_ACTIVE=dev && ./gradlew bootRun	starts app with dev profile
./gradlew clean build	compiles java classes, builds frontend, runs tests and prepares all-in-one runnable jar

databucket-server's Issues

Release v3.5.0 jar asset builded with hardcoded origin set as localhost

Release v3.5.0 jar asset builded with hardcoded origin set as localhost.

We used the jar 3.5.0 from the release page.

After start the application, we tried to log in, but we got a white screen instead of redirect to the databucket server home page.

Looking in the Dev Tools, we found in the UrlBuilder.js file, that the variable origin is hardcoded to localhost:

This behavior is not presented in the source code, only in the generated jar asset.

export const getOrigin = () => {
    const origin = window.location.origin;
    return origin + getContextPath();
}

Could you please generate the jar again?

Thanks for your support.

The "LIKE" operator for properties column

The "LIKE" operator doesn't work for properties column.
This column must be cast to TEXT.

SELECT * FROM "x-bucket-2" b WHERE b.properties::TEXT LIKE '%New%';

Incomplete task list in the event definition

For event created for specific class, global tasks seems to be not showing.

Mapping error

Mapping error when user changes project.

Get data response - totalPages

When limit is 0, then totalPages is 2147483647. This property should be missing or 0.

Rules - allow using only known fields

The API should return failed when I try to use unknown field in a rule.
Eg.
{
"rules": [
["name", "==", "John"]
]
}

The 'name' is not a field, it should be use as property: $.name.

Default sorting for view

Allows to define default soring for view (currect default is by id ASC)

• new column in the view table to keep this information
• allow to set fields, properties and maybe aso functions on properties

Management - change user roles

I'm trying to change super user. Set enabled all roles and save.

java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because the return value of "pl.databucket.server.entity.User.getEmail()" is null
at pl.databucket.server.service.ManageUserService.modifyUser(ManageUserService.java:137)
at pl.databucket.server.controller.ManageUserController.modifyUser(ManageUserController.java:66)
at pl.databucket.server.controller.ManageUserController$$FastClassBySpringCGLIB$$9894c0bb.invoke()

Duplicate data

New button for data that allows to duplicate current data row.

Modified at not updated when pure locking data

It seems, when we make data reservation, the 'modified at' field has not been updated in base data table. In the data history, it seems to be ok.

Problem with reading token

com.fasterxml.jackson.core.JsonParseException: Unexpected character ('�' (code 65533 / 0xfffd))
https://croc.space/a/JSON-BOM-file-import-issue-correction

Unhandled exception on frontend

Defined column as String but value is an json object. On data view we have white screen.

Save filter for active bucket

Save filter configuration per bucket. Load and set configuration after active bucket changed.

Progress bar for tasks execution

3.1.0
When we execute changes on many data rows or the bucket has many data rows, then in the app we wait for a long time for result message. In the waiting time we should see some progress component to be sure someting is in progress.

White screen - class without properties

A bucket is defined based on the class that hasn't got any properties.
When goes to properties on tasks execution or rich filter, then we've got white screen.

Share selected data rows as public

We want to share some data between project members that doesn't have access to Databucket.
It will be a new feature that allow to:

• generate public link from Data Details Panel
• open Data Details by the generated link without authorization

On backend:

• New entity "Link" with fields (id, uuid, bukcet, dataId, jsonPath, expirationDate, createdBy, createdAt)
• New controller "LinkController" (createLink, getDataByLink)
• New repository "LinkRepository" (seach item by uuid)
• New service "LinkService"
• New class CreateDataLinkRequestDto (bucket, dataId)

On frontend:

• New button on Data Details Panel

share 

Defined values for columns - empty list

Default value for column values is null. If I create a list and remove all items, I have empty list. This causes creating empty lookup. Then value in this column can't be modified.

Inconsistent icons issue on Safari

Missing search properties value

I have view with text in the search properties text box.
I go to the project settings and go back to data.
I see, the view is still filtered based on the text from the search properties, but this text is missing in the serarch box.

PostgreSQL compatibility

Test project compatibility with newest version of PostgreSQL

Routing problem for /confirmation/*

It occures only on production build.
The link send as confirmation for 'forgot-password' not working.

User registration

Allows to register new users by itself.
Allows to request for access to new or existing project.

Duplicated requests after logging

Getting incorrect port for API

I use in configuration server port: 8090, but frontend uses 8080 for API.

Add new filter using awesome-query-builder

Add possibility to filter data based on awesome-query-builder component.

Refresh lookups in columns and filter on view list

After class or bucket is changed, refresh lookup for columns and filter.

Task execution - reseved -> false

v.3.0.8
Steps:

1. Open task execution dialog.
2. Select 'Modify data'
3. Check 'Set reserved' (default option is False but request doesn't contain this value)

Data details dialog - sizing

Add buttons for changing dialog size

Search properties - save the text per bucket

When a user searches properties, the search text will be stored in the browser's memory. When user changed active bucket, the search text will be cleared or read from memory if available for current bucket.

Super user with disabled/expired projects

When user has SUPER role and has access to only disabled/expired projects should has access to managment page.
The massage is "Project is not enabled"

User data

Add to user:

• email address (this will allow to prepare password reset process)
• user avatar

Check usage of columns/filter item before deletion

Removing the columns/filter item that is used by any view item cause error.

Changing buckets when not first page is selected

Version: 3.0.16
We have two buckets: Bucket A, Bucket B. Bucket A has 5 pages. Bucket B has 2 pages. When we have selected Bukcket A on e.g. 5 page and change active bucket to Bucket B, then we have white screen on the app.

Project Settings - buckets (uncheck group)

v.3.0.17
Bucket is assingned to 2 groups. One is unchecked and saved. After refreshing, still has 2 groups.

Cannot read properties of undefined

Cursor

The cursor is active in the headings of the following dialog boxes:

Failed to obtain JDBC Connection

Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30020ms.

Fix incopatibility with Gradle 9.0

• What went wrong:
  Execution failed for task ':buildFrontend'.

Process 'command 'npm.cmd'' finished with non-zero exit value 1

• Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
Get more help at https://help.gradle.org.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD FAILED in 34s
4 actionable tasks: 4 executed

White screen - unhandled

v.3.1.0
Scenario:

1. Go to last page for the bucket.
2. Remove at least one page data.
3. You see white screen.

Tag list for view

Add a list of tags available for the active bucket.

Rule with 'not in' operator doesn't work

The example rule doesn't work for data reservation:
["some string","!in","$.parent.array"]

Default icon when creating a new bucket

Default icon is defined with the old scructure.

{
	"icon": "panorama_fish_eye",
	"classId": "19",
	"roleId": null,
	"name": "dev-users",
	"groupsIds": [
		7
	],
	"usersIds": [],
	"teamsIds": [
		8
	]
}

As the result we have an error message:

{
	"timestamp": "2022-10-10T13:24:05.561+00:00",
	"status": 400,
	"error": "Bad Request",
	"path": "/api/buckets"
}

Fix layout on login page

In the version 3.5.0 was correct:

In the version 4.0.0 something has been broken.

Change password on the same password

Block the possibility to change a password on the previous one.

Unlock bundle by event - locked_by

When bundle is unlocked by event, the locked_by field stays with previous value. It must be empty for unlocked bundle.

About data initialization

Hello, kslysz! I think DataBuckets solves some of my problems, so I need to know exactly what DataBuckets does, but I think if there is some initialization data that makes it more intuitive, I would like you to provide some initialization data!