sirnoob97 / storage-service

File storage backend application.

storage-service

File storage backend application.

This service stores files in a PostgreSQL database as BLOB(oid).

Requirements

• Java 17+
• Docker

SQL ERD

The file_size column store the size of the file as bytes.

Usage

You can deploy, build and test the app with the Makefile, add the option -j $(nproc) when convenient.

Depending on the type of 'compose' you are using, change the COMPOSE_CMD variable in the Makefile.

Container

Build the backend image.

make build-container

Or just deploy it, docker automatically builds the image it can't find it.

make deploy

Jar

Before building the jar file, start a postgreSQL instance. This is the configuration for the database connection in the application.yaml. So keep this in mind before building the jar.

spring:
  ...
  datasource:
    url: jdbc:postgresql://${POSTGRES_HOSTNAME:localhost}:${POSTGRES_PORT:5432}/${POSTGRES_DB:sn_storage_service}
    username: ${POSTGRES_USER:sn_user}
    password: ${POSTGRES_PASSWORD:sn_user}

Build the jar.

make build-jar

Tests

Remember to start a Docker daemon first. Because of Testcontainers.

make test

Consume the API

The application has four endpoints.

• Upload a file:

  curl -F attachment=@<file path> -X POST 'http://<host>:<port>/files'

• Update a file:

  Not implemented yet

• Get a list of files:

  limit is the number of elements you want, must be greater than 1 e.g 10, 15, 20, etc.

  offset is the number of skipped elements in other words the page that wou want, must be a apositive number increased by ten e.g 0, 10, 20, 30, etc.

  curl -X GET 'http://<host>:<port>/files?limit=<number>&offset=<number>'

• Download a file:

  curl -X GET -o <your output file path> 'http://<host>:<port>/files/<file id>'

• Delete a file:

  curl -X DELETE 'http://<host>:<port>/files/<file id>'

storage-service's Issues

Better description

Better description in README.md

Refactor service layer tests

Deny invalid `File` info

As expressed here 8a15f11, these values must not be allowed and an exception must be thrown

Improve the SQL test script with the following constraint or use triggers if necessary:

• Filename cannot be an empty string.
• Mime-Type cannot be an empty string.
• File size must be greater than zero.

enable useful options for debug SQL operations

Order criteria when listing files

Create an Enum to manage the order criteria of the method listFiles.

Possible Values:

• fileName
• fileSize
• mimeType

storage-service/src/main/java/com/sirnoob97/storageservice/file/service/FileServiceDefaultImpl.java

Lines 52 to 55 in 1bfcbd7

	@Override
	public Set<FileInfoDto> listFiles(int limit, int offset, String orderBy) {
	return fileRepository.listFileInfoDtos(limit, offset, orderBy);
	}

File entity

Main Code:

• #4
• #6

Tests:

• #7
• #8
• #9

Complete the testcontainer TODO

As described here a55ef48.

Complete the TODO:

storage-service/src/test/resources/db/init_test.sql

Lines 47 to 49 in a55ef48

	-- TODO: replace testcontainers settings in application.yaml by a programatic setup
	-- Try the docker compose integration: https://www.testcontainers.org/modules/docker_compose/
	-- Or normal containers with volumes mapping: https://www.testcontainers.org/features/files/

create `FileDto` class for `File` and `FIleData` classes to avoid exposing ids

set options for spring data-source focus on test containers

Controller test

`FileService` unit tests

Unit test (leveraging mocks) for FileServiceDefaultImpl and fix any errors that arise

File Service

Add service layer. FileService interface and its implementation

Refactor service layer to avoid unnecessary logic in the controller layer

Database description

Update Database diagram and explanation

Tests cases for `FileDataRepositoryTest`

Due to recent changes in 6aeac2a

Perform test cases for the FileDataRepositroryTest to avoid unknown bugs

create `File` entity repository

result mapping annotations in entities classes

Delete `File` by id refactor

As described here, a refactor is necessary to prevent FileData orphans

storage-service/src/main/java/com/sirnoob97/storageservice/file/service/FileServiceDefaultImpl.java

Lines 65 to 79 in 76016c3

	/**
	* BUG: This method should delete both {@link com.sirnoob97.storageservice.file.entity.File} and the {@link com.sirnoob97.storageservice.file.entity.FileData}
	* without a previous search of the {@link com.sirnoob97.storageservice.file.entity.File} entity
	*
	* Refactor the native query to delete the related {@link com.sirnoob97.storageservice.file.entity.FileData}
	* Or add a cascade on delete operations in the sql table
	*/
	@Override
	public void deleteFile(long id) {
	int ret = fileRepository.deleteFileById(id);
	if (ret == 0) {
	throw FILE_NOT_FOUND_EXCEPTION;
	}
	}

create entities classes

Create entities classes:

• File.java
• FileData.java

Based on the database diagram

Update operation

Implement the update operation.

Related Classes:

• FileController
• FileServiceDefaultImpl

Dtos

Main code:

• #5
• #14
• #13

Test:

• #16
• #17

perform unit test for `FileInfoDto` class

native query annotations in entities classes

Update project description

Include steps for deployment and testing.

perform unit test for `File` and `FIleData` relationship

Classes related:

• File
• FileData

Primary keys should be generated as entity

perform unit test for `FileDto` class

Controller class

Complete TODO

storage-service/src/main/java/com/sirnoob97/storageservice/file/service/FileServiceDefaultImpl.java

Lines 63 to 71 in b6505aa

	/**
	* TODO:
	* 1) Create a new native query to return a set of @see com.sirnoob97.storageservice.file.dto.FileInfoDto
	* 2) Create an enum for the order criteria
	*/
	@Override
	public Set<FileInfoDto> listFiles(int limit, int offset, String orderBy) {
	return null;
	}

add SQL script for database initialization

Controller Layer

• #33
• #34
• #35
• #36