quarkiverse / quarkus-minio

Minio (https://min.io) Client Quarkus Extension

Quarkus Minio Extension

Integrate minio sdk for jdk and native build modes.

Get started

All documentation can be found on Quarkiverse//Docs

Anything unclear or missing in the documentation? Please open an issue.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

jtama 💻 🚧 📖	Peter Palaga 💻	JiriOndrusek 💻	dlucasd 💻 📖	Stefan Gürtler 💻	Guillaume Smet 💻	Sébastien Crocquesel 💻
Phillip Krüger 💻	Sylvain METAYER 📖	George Gastaldi 📖	Markus Herhoffer 📖 💻	Alex Katlein 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

WARNING

This project is still in its early stage.

Contributions are always welcome, but this repository is not really ready for external contributions yet, better create an issue to discuss them prior to any contributions.

quarkus-minio's Issues

AllContributors

Add documentation for quarkiverse.github.io

How to config multiple minio client

Prefix artifact ID with `quarkiverse-`

Hello,

Quarkiverse Extensions should have the quarkiverse- prefix. Please rename your extension to quarkiverse-minio-client

Thank you!

Problem with upgrade from 3.0.0.Alpha3 to 3.0.0 in project with Quarkus BOM

Recent effort to upgrade quarkiverse.minio client from 3.0.0.Alpha to 3.0.0 in Camel Quarkus is not possible.

1. Problem happens in case that the maven module uses Quarkus BOM and not explicit dependency versions (which is the case of camel-quarkus)
2. Quarkus BOM (3.0.1.Final currently) forces versions of okhttp (3.14.9) and okio (1.17.2)
3. Quarkiverse.minio 3.0.0 bring minio 8.5.2 which requires okhttp >= 4.8.1 (see the code)

All items combined makes upgrade from 3.0.0.Alpha3 to 3.0.0 impossible.

I created the small reproducer, where the error can be simulated (without camel-quarkus) https://github.com/JiriOndrusek/minioClientQuarkiverseReproducer

MinioAsyncClient configuration support

Hi there,

have you thought about providing MinioAsyncClient instances next to the default blocking MinioClient?

I would find that really useful!

All the best,
Alex

Feature: Enhance client creation

Related to #22

Would it be possible to change behavior of MinioProducer, so it would return null if all configuration properties are empty?

This behavior could be based on another property. For example if we introduce quarkus.minio.nullableClient
If quarkus.minio.nullableClient is true and no configuration properties are present, client producer would return null.
If it is false (default behavior) nothing would change.

Let me explain benefits of this behavior:

• it would allow users to define configuration properties and create client based on them (as is now)
• let say, that there is a library unaware of these properties, wrapper which adds quarkus to it could configure quarkus.minio.nullableClient to true. Library then may act based on the fact that the client is null and can define its own fallback.

This would help camel-quarkus minio extension (see apache/camel-quarkus#2721)

Missing reflective registration (e.g. CopySource)

I was implementing camel-quarkus minio extension by using quarkiverse-minio (apache/camel-quarkus#2116).
I've found that there is some missing registration for reflaction. For example there is a missing constructor for CopySource which is called from its builder.

My solution used in camel-quarkus minio extension is:

    @BuildStep
    ReflectiveClassBuildItem registerForReflection(CombinedIndexBuildItem combinedIndex) {
        IndexView index = combinedIndex.getIndex();
        String[] dtos = index.getAllKnownSubclasses(DotName.createSimple(BaseArgs.class.getName())).stream()
                .map(c -> c.name().toString()).toArray(String[]::new);
        return new ReflectiveClassBuildItem(false, false, dtos);
    }

Upgrade to Quarkus 3.0.0

I started migrating Camel Quarkus to Quarkus 3.0.0. I wonder whether there is any plan or known issues for doing the same in quarkus-minio?

3.7.2 broke "quarkus.minio.url" in devservices mode

Hi and thanks for this extension.
I'm relying on this code to get the current minio URL:

@ConfigProperty(name = "quarkus.minio.url")
String url;

but since 3.7.2 when Minio is started as a devservice (in unit tests) it's only filled by "localhost" instead of http://localhost:6878 (random port chosen by testcontainers).
I'm guessing it has to do with this recent change 204a8ae#diff-ac8ae1ef70059d67a0656e95d75eaeb3482ec3c7ec6a6284c1dd7b05bfe8badeR240

Provide devservices feature

Minio extension should provide devservices with the ability to :

• Enable/Disable the feature
• Enable/Disable mini.io http console.
• Allow multiple client configuration #154
• Provide documentation

Devservice don't expose the minio console

Allow test-container reuse

It would be great if we could reuse the test-container for multiple test runs. This would speedup the tests by a few seconds.

It could be implemented similar to the database-devservices: quarkusio/quarkus#25367

I would suggest, that it can be enabled/disabled by a property (default false?).

Migrate to the new Dev UI

quarkusio/quarkus#31315

Update MinIO client to 8.4.3

There is a related PR from Dependabot - #137

Problem is that new MinIO client requires newer OkHttp dependency.

In gradle.kts this can be worked around as

    implementation(enforcedPlatform("com.squareup.okhttp3:okhttp-bom:4.10.0")) {
        because("MinIO requires OkHttp >= 4.8.1")
    }
    implementation(enforcedPlatform("com.squareup.okio:okio-bom:3.2.0")) {
        // https://github.com/square/okhttp/blob/parent-4.10.0/build.gradle#L20
        because("OkHttp")
    }

native error:

In jvm docker image, the following code works fine:

public String uploadObject(String filename, InputStream inputStream, String mimeType) throws Exception {
        logger.info("uploadObject - bucketName:" + bucketName + ", filename: " + filename + ", mimeType: " + mimeType);
        try {
            minioClient.putObject(
                    PutObjectArgs.builder()
                            .bucket(bucketName)
                            .object(filename)
                            .stream(inputStream, -1, PART_SIZE)
                            .contentType(mimeType)
                            .build());
            return filename;
        } catch (Exception e) {
            throw new Exception("Failed uploading file [" + filename + "]", e);
        }
    }

In native mode, i receive that error:

  Caused by: java.lang.RuntimeException: class io.minio.PutObjectArgs must have no argument constructor
     at io.minio.BaseArgs$Builder.newInstance(BaseArgs.java:135)
     at io.minio.BaseArgs$Builder.build(BaseArgs.java:148)
     at it.ictgroup.portali.messages.service.S3Client.uploadObject(S3Client.java:38)
     ... 45 more

Have you some hacking to UPLOAD file (using InputStream) in native mode?

best regards

Problems with fetching added metadata to a file

ISSUE

When I add metadata to a file, it will correctly store the metadata on minio console but incorrectly retrieve the metadata with minio client.

It's the same problem with that :
minio/minio#15746 (comment)

Version 2.10.3

Workaround available ?
Thx

Feature: Make client configuration optional

It would be very helpful, if configuration to create minio client is optional.

Let me describe use case:
I was implementing camel-quarkus minio extension by using quarkiverse-minio (apache/camel-quarkus#2116).
Camel-quarkus minio extension is capable of using several minio client at once or of defining clients dynamically in runtime.
It would be much easier and consistent if there was no mandatory client creation based on configuration.

In other words, quarkiverse-minio could be used to introduce minio capabilities, but client creation can be left to custom implementation.

The minio extension description on code.quarkus.io shows the generic bom description

CC @maxandersen

Add extension to catalogue

Allow extension to provide client for multiple bucket host.

Configuration should allow creating named minio-client, as possible for databases.

Should be configured as followed :

quarkus.minio.url=https://minio.acme
quarkus.minio.access-key=DUMMY-ACCESS-KEY
quarkus.minio.secret-key=DUMMY-SECRET-KEY
quarkus.minio.region=beyond

quarkus.minio.images.url=https://minio2.acme
quarkus.minio.images.access-key=DUMMY-ACCESS-KEY
quarkus.minio.images.secret-key=DUMMY-SECRET-KEY

quarkus.minio.pdf.url=https://pdf.acme
quarkus.minio.pdf.access-key=DUMMY-ACCESS-KEY
quarkus.minio.pdf.secret-key=DUMMY-SECRET-KEY

See Agroal for inspiration :
https://github.com/quarkusio/quarkus/blob/73ee07970728297003740d5ac896e304bc51f688/extensions/agroal/deployment/src/main/java/io/quarkus/agroal/deployment/AgroalProcessor.java#L71-L131

Micrometer memory leak

Hi all,

We are currently using the quarkus-minio client and once the quarkus micrometer extension is present on the server, we observe a memory leak.
According to the heap dump, the leak is caused by an accumulation of io.micrometer.core.instrument.ImmutableTag.

Those tags are not garbage collected and result in an application crash.
When disabling the metrics production on the minio client with quarkus.minio.produce-metrics we can't see the tags in the heap dump anymore and the application behaves normally.

Unless we missed a configuration somewhere, there a something wrong with those tags.
The micrometer documentation suggest to set a MeterFilter#maximumAllowableTags() to prevent a memory leak. We saw that this has been already been done in the quarkus micrometer extension but maybe the minio client is missing something.

Allow arbitrary env variables passed to the Min.io Dev Service

It is best practice to accept arbitrary env variables being passed to the dev service container. Examples for this can be found at the official Quarkus guide on Dev Services. E.g., quarkus.amqp.devservices.container-env."environment-variable-name"

Use case: I want the dev service to be preconfigured with

ENV MINIO_NOTIFY_KAFKA_ENABLE_LOCAL="on"
ENV MINIO_NOTIFY_KAFKA_BROKERS_LOCAL="localhost:55003"

If the feature gets approved, I could do the PR on my own.

Allow configless programmatic only creation of MinioClient

Before #166 the named scenario was possible and we have been relying on it in Camel Quarkus.

After #166 it does not work anymore

Could you please make config of the default client optional so that we can create the client 100% programmatically without any config in application.properties?

Support for Quarkus integration test with docker

Hi,

could you please add support for the Quarkus Docker integration tests?

We use quarkus.container-image.build=true to build a Docker image with Quarkus before running the integration tests.
In the next step we will do integration tests and Quarkus will create a common network of all dev services and our application as a Docker container. Quarkus documentation: https://quarkus.io/guides/getting-started-testing#quarkus-integration-test

In this case,quarkus.minio.url should not contain the hostname and port of the Docker container on the local machine, but the hostname of the Docker container on the share network, for example: http://minio:9000.

I created this function for quarkus-mockserver: https://github.com/quarkiverse/quarkus-mockserver/blob/main/deployment/src/main/java/io/quarkiverse/mockserver/devservices/DevServicesMockServerProcessor.java#LL95C20-L95C65

Based on "!devServicesSharedNetworkBuildItem.isEmpty()" we can decide which URL we want to configure.

Provide healthBuildItem for minio clients

see https://github.com/quarkusio/quarkus/blob/main/extensions/elasticsearch-rest-client/deployment/src/main/java/io/quarkus/elasticsearch/restclient/lowlevel/deployment/ElasticsearchLowLevelClientProcessor.java

Provide code starter

https://quarkus.io/guides/extension-codestart#writing-an-extension-codestart

PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException

Hello,

I try to upgrade quarkus from 2.14.2.Final to 2.16.4.Final (I have not tested the intermediate versions).
When I try to access to minio client (get, stat, etc...), an exception is throwed :

java.lang.RuntimeException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

My Minio HTTP endpoint is in TLS mode with an auto-signed cert.
Is there a workaround to force the request (even in unsigned SSL mode) ?
Yes, it's a bad practice :)

This option don't work : quarkus.ssl.native = false

Can you support a shh project under minio? sshd Because jsch has not been updated for a long time

Can you support a shh project under minio? sshd
Because jsch has not been updated for a long time

Update region in configuration

Is it possible to add the regions configuration in the extension?

QUESTION: Minio operator

Have you managed to use the minio operator and certificates to use https via minio and quarkus native?

Thanks for your help 😄

Include all contributors