quarkiverse / quarkus-azure-services Goto Github PK
View Code? Open in Web Editor NEWQuarkus extensions for Azure services
License: Apache License 2.0
Quarkus extensions for Azure services
License: Apache License 2.0
https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-storage-blob.html has the following instructions for configuring storage:
Then, in the application.properties file, add the following property:
quarkus.azure.storage.blob.connection-string=DefaultEndpointsProtocol=https;EndpointSuffix=core.windows.net;AccountName=stquarkusazurestorageblo;AccountKey=xxxxxxx==;BlobEndpoint=https://stquarkusazurestorageblo.blob.core.windows.net/;FileEndpoint=https://stquarkusazurestorageblo.file.core.windows.net/;QueueEndpoint=https://stquarkusazurestorageblo.queue.core.windows.net/;TableEndpoint=https://stquarkusazurestorageblo.table.core.windows.net/
I wonder if the instructions should instead suggest setting the QUARKUS_AZURE_STORAGE_BLOB_CONNECTION_STRING
, so there's no risk of committing the connection string to source control? They could even explain that it can be set either in application.properties
or as an environment variable, but environment variable is more secure for this kind of information.
I'd also suggest being explicit that the given string is an example, and the whole connection string is copy-pasted from either the cli or web console. Otherwise people might wonder why they have to set a configuration property, if the docs already know what the value is.
The Camel team has developed a few Azure extensions for Quarkus. Some can be compiled in native, others not yet.
Compiled to native
JVM
Rename integration-tests/blob
to integration-tests/srotage-blob
so it's consistent with the extension name
As Developer,
Would be great to be able to have a Dev Service in Quarkus that allows me to use Azure Event Hub without requiring any internet connection for my local environment.
I would suggest having the following functionalities:
Since Azure Event Hub has compatibility with Apache Kafka, I would recommend using Red panda since is a service already used in Quarkus from another service to support Kafka in Dev Mode.
There are already some functionalities supported by Quarkus community for Kafka https://quarkus.io/guides/kafka#azure-event-hub but this is used only to connect to Azure Event Hub.
Documentation:
https://redpanda.com/
Something like quarkus.azure.app.configuration.enable=false
would be useful.
I renamed the packages io.quarkiverse.azureservices.azure.*
to io.quarkiverse.azureservices.*
according to the Quarkiverse naming conventions. This way we avoid the duplication azureservices.azure
.
@majguo we could also rename our packages io.quarkiverse.azure.*
instead of having io.quarkiverse.azureservices
. It would also fit better with the configuration keys that are called quarkus.azure.
WDYT?
Because the internal libs don't seem to publish the deployment artifacts, it's not possible to include them in outside projects, like the Otel exporter.
See: quarkiverse/quarkus-opentelemetry-exporter#143
I have a project with the latest quarkus version 3.7.4.
This is an excerpt of my dependencies within the project
<dependency>
<groupId>io.quarkiverse.cxf</groupId>
<artifactId>quarkus-cxf</artifactId>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-storage-blob</artifactId>
<version>1.0.2</version>
</dependency>
The following error occur if I execute the command mvn -Pnative install
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.459 s
[INFO] Finished at: 2024-02-28T17:03:17+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.7.4:build (default) on project code-with-quarkus: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR] [error]: Build step io.quarkiverse.azure.jackson.datafromat.xml.deployment.JacksonDataformatXmlSupportProcessor#serviceProviders threw an exception: java.lang.NoClassDefFoundError: com/ctc/wstx/shaded/msv/org_isorelax/verifier/VerifierFactoryLoader
[ERROR] at io.quarkiverse.azure.jackson.datafromat.xml.deployment.JacksonDataformatXmlSupportProcessor.serviceProviders(JacksonDataformatXmlSupportProcessor.java:46)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:568)
[ERROR] at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
[ERROR] at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
[ERROR] at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR] at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
[ERROR] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
[ERROR] at java.base/java.lang.Thread.run(Thread.java:840)
[ERROR] at org.jboss.threads.JBossThread.run(JBossThread.java:501)
[ERROR] Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.shaded.msv.org_isorelax.verifier.VerifierFactoryLoader
[ERROR] at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
[ERROR] at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
[ERROR] at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
[ERROR] at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
[ERROR] at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:518)
[ERROR] at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:468)
[ERROR] ... 12 more
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
Successfully build in native mode
No response
If I remove one of the dependency the build will execute successfully. It doesn't matter which dependency I removed.
Only If both dependencies are "active" I got a problem.
I attached the sample project with the behaviour. code-with-quarkus.zip
No response
uname -a
or ver
No response
java -version
OpenJDK Runtime Environment GraalVM CE 17.0.9+9.1 (build 17.0.9+9-jvmci-23.0-b22)
3.7.4
mvnw --version
or gradlew --version
)Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
No response
Dear team,
With Quarkus 3.9.4 and Azure Quarkiverse 1.0.2 native build fails
Reproducer repo: https://github.com/olivierbeltrandocintoo/azure-quarkiverse-native-build-fails
Which is just a repo that uses latest Quarkus getting started and whose only changes are just adding the Quarkiverse Azure Services dependency (note ignoring or not netty does not change the outcome)
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-storage-blob</artifactId>
<version>${azure-quarkiverse-version}</version>
<exclusions>
<exclusion>
<groupId>com.azure</groupId>
<artifactId>azure-core-http-netty</artifactId>
</exclusion>
</exclusions>
</dependency>
Run this command to reproduce
mvn install -Dnative -DskipTests
Logs
โ azure-quarkiverse-native-build-fails git:(master) mvn install -Dnative -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------< org.acme:getting-started >----------------------
[INFO] Building getting-started 1.0.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ getting-started ---
[INFO] Copying 2 resources from src/main/resources to target/classes
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ getting-started ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ getting-started ---
[INFO] skip non existing resourceDirectory /home/olivier/dev/bugs/azure-quarkiverse-native-build-fails/src/test/resources
[INFO]
[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ getting-started ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- surefire:3.1.2:test (default-test) @ getting-started ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- jar:3.3.0:jar (default-jar) @ getting-started ---
[INFO] Building jar: /home/olivier/dev/bugs/azure-quarkiverse-native-build-fails/target/getting-started-1.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- quarkus:3.9.4:build (default) @ getting-started ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.642 s
[INFO] Finished at: 2024-04-22T18:39:26+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.9.4:build (default) on project getting-started: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR] [error]: Build step io.quarkiverse.azure.jackson.datafromat.xml.deployment.JacksonDataformatXmlSupportProcessor#serviceProviders threw an exception: java.lang.NoClassDefFoundError: com/ctc/wstx/shaded/msv/org_isorelax/verifier/VerifierFactoryLoader
[ERROR] at io.quarkiverse.azure.jackson.datafromat.xml.deployment.JacksonDataformatXmlSupportProcessor.serviceProviders(JacksonDataformatXmlSupportProcessor.java:46)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:568)
[ERROR] at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
[ERROR] at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
[ERROR] at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR] at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
[ERROR] at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
[ERROR] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1495)
[ERROR] at java.base/java.lang.Thread.run(Thread.java:833)
[ERROR] at org.jboss.threads.JBossThread.run(JBossThread.java:483)
[ERROR] Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.shaded.msv.org_isorelax.verifier.VerifierFactoryLoader
[ERROR] at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
[ERROR] at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
[ERROR] at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
[ERROR] at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
[ERROR] at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:518)
[ERROR] at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:468)
[ERROR] ... 13 more
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
As Developer,
Would be great to be able to have a Dev Service in Quarkus that allows me to use Azure Service Bus without requiring any internet connection for my local environment.
My recommendation would be to use Apache ActiveMQ since this supports AMQP 1.0, topics, and queues which is required by Azure Service Bus.
@radcortez I've created a branch (https://github.com/quarkiverse/quarkus-azure-services/tree/agoncal/playback) where I am putting PlayBack Tests in action (see #37).
When I execute my PlayBack test I get the Netty exception below. I set the right properties and get the right connection string to App Configuration (I know it's the right connection string because a simple Hello World using this string connection works).
In the code (https://github.com/quarkiverse/quarkus-azure-services/blob/main/extensions/azure-app-configuration/runtime/src/main/java/io/quarkiverse/azure/app/configuration/AzureAppConfigurationConfigSourceFactory.java#L36) you added the following comment:
// We cannot use the Quarkus Vert.x instance, because the configuration executes before starting Vert.x
2023-02-03 12:26:06,713 ERROR [com.azu.cor.htt.pol.RetryPolicy] (vert.x-eventloop-thread-4) {"az.sdk.message":"Retry attempts have been exhausted.","exception":"Failed to resolve 'appcs-quarkus-azure-app-configuration-true.azconfig.io' and search domain query for configured domains failed as well: [numericable.fr]","tryCount":3}
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.173 s <<< FAILURE! - in io.quarkiverse.azure.app.configuration.pt.AzureAppConfigurationTest
[ERROR] io.quarkiverse.azure.app.configuration.pt.AzureAppConfigurationTest Time elapsed: 8.173 s <<< ERROR!
java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
Caused by: reactor.core.Exceptions$ReactiveException: io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Failed to resolve 'appcs-quarkus-azure-app-conf.azconfig.io' and search domain query for configured domains failed as well: [numericable.fr]
at reactor.core.Exceptions.propagate(Exceptions.java:396)
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:97)
at reactor.core.publisher.Mono.block(Mono.java:1742)
at com.azure.core.implementation.http.rest.AsyncRestProxy.handleRestReturnType(AsyncRestProxy.java:229)
at com.azure.core.implementation.http.rest.AsyncRestProxy.invoke(AsyncRestProxy.java:80)
at com.azure.core.implementation.http.rest.RestProxyBase.invoke(RestProxyBase.java:105)
at com.azure.core.http.rest.RestProxy.invoke(RestProxy.java:92)
at jdk.proxy4/jdk.proxy4.$Proxy66.listKeyValues(Unknown Source)
at com.azure.data.appconfiguration.implementation.ConfigurationClientImpl.listConfigurationSettingsSinglePage(ConfigurationClientImpl.java:757)
at com.azure.data.appconfiguration.implementation.ConfigurationClientImpl.lambda$listConfigurationSettings$24(ConfigurationClientImpl.java:626)
Looks like dependABot does not check the Docker image versions for DevService. Today the Azurite version is 3.19.0 but there is a new one 3.20.1 : https://mcr.microsoft.com/v2/azure-storage/azurite/tags/list.
There are several options that we could think of:
I originally posted here: quarkiverse/quarkus-opentelemetry-exporter#33
Not sure if that is the right place or its should be here since this project is run by MS.
Quarkus OLTP: https://quarkus.io/guides/opentelemetry#exporters
Dear Azure Services community of Quarkiverse,
I've been working on implementing a Quarkus extension for Azure Storage Queue and have made good progress. I'd like to share my thoughts on the current state of my implementation and seek your input on the best way we can move forward to merge it into the official release.
I've based my work on the existing Azure Storage Blob extension, which allowed me to quickly build a working prototype extension for the Azure Storage Queue. In most places, the prototype uses a copy of the code from the Blob extension to establish the basic functionality. The working prototype together with working integration tests and a bit of documentation can be found in my fork.
The prototype is working rather well and meets the initial requirements for interacting with Azure Storage Queues. However, there are certain aspects of my implementation that are duplicated between the two extensions. In particular, the following components are duplicated in most places:
DevServicesConfig
classStorageQueueBuildTimeConfig
classDevServicesStorageQueueProcessor
classQuarkusPortAzuriteContainer
classSome might argue that a functional PoC is sufficient, but due to my strong inclination towards maintaining clean code, the absence of a suitable abstraction for the classes mentioned above is incredibly frustrating to me. ๐
I believe there is a valuable opportunity to deduplicate the code by extracting common components and making them more universally applicable. By doing so, we can achieve cleaner and more maintainable code while reducing redundancy between the Blob and Queue extensions (and most likely a new extension for Storage Table that may potentially be created in the future, since it would also use the same abstraction).
What I would like to propose would involve extracting the common components mentioned above into a shared dependency (either a new or existing one, e.g. quarkus-azure-core-deployment
), which would be utilized by both the Blob and Queue extensions (and most likely Table in future). After performing some preliminary exercises in this approach I found out that this would entail changes to the configuration paths, if we are willing to keep it clear, as well as some structural modifications to accommodate the new shared components.
It's important to note that extracting these common components and making mentioned changes would result in a break of backward compatibility within existing configurations. The reason lies mainly in the fact that we have a blob
fragment in the connection-string
configuration path. When implementing my extension I did use the same approach and placed a queue
in the path as well. However, with the assumption that we have Blob, Queue, and most likely would have a Table extension in a future, we will end up with three different configuration paths to carry the same exact value:
quarkus.azure.storage.blob.connection-string = {a storage account connection here}
quarkus.azure.storage.queue.connection-string = {a storage account connection here}
quarkus.azure.storage.table.connection-string = {a storage account connection here}
^- blob, queue and table with DevServicesConfig for each, but exactly the same value
Similarily, we would end up with three, exactly the same Azurite containers running in dev mode, which is a complete waste of resources (ah, just thinking about this with my mere 16 GB of RAM, annoys me as hell). The only difference between these containers would be port number they exposes, specifically:
This sucks.
What I would like to do is to drop the blob
, queue
or table
from the connection string configuration path, and have it common between the extensions:
quarkus.azure.storage.connection-string = {a storage account connection here}
quarkus.azure.storage.devservices.image-name = {docker image}
^- no blob here
While having a smaller configuration just for the ports, e.g.:
quarkus.azure.storage.devservices.blob.port = {port number for blobs}
quarkus.azure.storage.devservices.queue.port = {port number for queues}
quarkus.azure.storage.devservices.table.port = {port number for tables}
By doing so we can have only one configuration for connection-string
and only one Azurite container instead of three separate ones. However, this would break backward compatibility, since, as you can see, the old configurations users use in the wild would become invalid.
While I understand the significance of maintaining compatibility, I also believe in the long-term benefits of cleaner code and easier maintenance, which may outweigh this concern. But I'm at a crossroads.
At this juncture, I would greatly appreciate the community's insights and opinions on how we can proceed. I'm seeking your thoughts on the following questions:
I understand that this decision may have implications for the broader Quarkus Azure Services ecosystem, and I value your opinions on this matter. Please share your thoughts, concerns, or alternative suggestions regarding the direction we can take. I believe that any feedback in regard to this matter will help us make a well-informed decision that benefits the entire user community.
Thank you for your time and consideration.
The issue is to request support azure-storage-blob
as a Quarkus extension.
As I've been integrating Azure libraries into Quarkus applications the one thing I keep coming back to is azure-core. The core library's use of project reactor/netty prevents me from doing native image builds. Short of writing my own compatible azure-core and replacing those MS dependencies, is there any way to enable native image builds when using Azure services?
As Developer,
Would be great to be able to have a Dev Service in Quarkus that allows me to use Azure IoT Hub without requiring any internet connection for my local environment.
As you know, Azure IoT Hub supports two types of connection:
I would like to be able to make two operations:
When sending messages to a pre-populated device it would be great to define a few properties:
quarkus.azure.iothub.device."quarkus-device1".status=enable/disable
quarkus.azure.iothub.device."quarkus-device1".message.acknowledgment=true/false
quarkus.azure.iothub.device."quarkus-device1".message.acknowledgment.delay=5000 (ms)
Documentation:
https://learn.microsoft.com/en-us/java/api/overview/azure/iot?view=azure-java-stable
Copied from original issue quarkusio/quarkus#36587:
As of now, quarkus does not support Active Directory Authentication in native mode with the MS SQL driver. Since this is a standard authentication method for databases hosted in azure, it would be nice if quarkus supported AD authentication in native mode.
As a quarkus developer, developing a native image application
When I connect to a MS SQL database
Then I want the possibility to use AD authentication
Integrating Azure libs (mainly azure-core dependency) into Quarkus application prevents my native build to complete successfully.
Workaround suggested in #161 does not solve the issue, probably due to the use of quarkus-resteasy-reactive dependency in pom.xml
Native build error logs look like this:
Error: Classes that should be initialized at run time got initialized during image building:
io.netty.buffer.UnpooledDirectByteBuf the class was requested to be initialized at run time (subtype of io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.netty.buffer.UnpooledDirectByteBuf got initialized use --trace-class-initialization=io.netty.buffer.UnpooledDirectByteBuf
...
io.netty.buffer.ByteBufAllocator the class was requested to be initialized at run time (from 'META-INF/native-image/io.netty/netty-buffer/native-image.properties' in 'file:///project/lib/io.netty.netty-buffer-4.1.94.Final.jar' with 'io.netty.buffer.ByteBufAllocator' and from feature io.quarkus.runner.Feature.beforeAnalysis with 'ByteBufAllocator.class'). To see why io.netty.buffer.ByteBufAllocator got initialized use --trace-class-initialization=io.netty.buffer.ByteBufAllocator
Today under the extensions
directory, we have some listed extensions, such as app-configuration
or storage-blob
as well as unlisted ones http-client-vertx
or core
. So, looking at the repo and the structure of the directory, it's not clear of which are extensions that can be used by external projects or not.
Wondering if we could have an extensions
directory and an internal
directory ? Or maybe just prefix the listed extensions with azure
, this way we will have azure-app-configuration
or azure-storage-blob
@ppalaga any thoughts on that ?
As discussed in #61, it would be better to use the quarkus-azure-
prefix for extensions. The azure-
prefix might awake the impression that they are some sort of official Azure artifacts, which they are not.
Hello everyone,
I would like to use especially Azure configuration in my project.
I haven't found this information anywhere.
Thank you
We chatted about this with @agoncal
When executing a mvn integration-test -Dnative
command, the following exception is thrown:
Caused by: java.lang.SecurityException: class "com.azure.storage.blob.BlobServiceClient"'s signer information does not match signer information of other classes in the same package
The class BlobServiceClient
is in the artifact called com.azure:azure-storage-blob
. This artifact is used by io.quarkiverse.azureservices:quarkus-azure-storage-blob
which is in turned used by io.quarkiverse.azureservices:quarkus-azure-storage-blob-deployment
.
I tried verifying JARs with the following Maven plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jarsigner-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<verbose>true</verbose>
<certs>true</certs>
</configuration>
</plugin>
Looking at the azure-storage-blob-12.20.0.jar
inside the file META-INF/MANIFEST.MF
we see:
Name: com/azure/storage/blob/BlobServiceClientBuilder.class
SHA-256-Digest: vknX58uqQ616EHc/doYs/7fte00cZfMdDjyNo4U8gF4=
When using Jarsigner we can see that the JAR is verified:
$ jarsigner -verify azure-storage-blob-12.20.0.jar
jar verified.
$ java -version
openjdk version "11.0.17" 2022-10-18
OpenJDK Runtime Environment GraalVM CE 22.3.0 (build 11.0.17+8-jvmci-22.3-b08)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.0 (build 11.0.17+8-jvmci-22.3-b08, mixed mode)
[1/7] Initializing...
Version info: 'GraalVM 22.3.0 Java 11 CE'
Java version info: '11.0.17+8-jvmci-22.3-b08'
C compiler: cc (apple, arm64, 14.0.0)
Garbage collector: Serial GC
3 user-specific feature(s)
- io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
- io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.jboss.threads] categories
- io.quarkus.runtime.graal.ResourcesFeature: Register each line in META-INF/quarkus-native-resources.txt as a resource on Substrate VM
[2/7] Performing analysis... [WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/Users/agoncal/Documents/Code/Quarkiverse/quarkus-azure-services/integration-tests/target/quarkus-azure-services-integration-tests-999-SNAPSHOT-native-image-source-jar/lib/io.netty.netty-common-4.1.82.Final.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
16:42:46,792 INFO [com.azu.cor.imp.jac.JacksonVersion] Package versions: jackson-core=2.13.4, jackson-databind=2.13.4-2, jackson-dataformat-xml=2.13.4, jackson-datatype-jsr310=2.13.4, azure-core=1.33.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot
] (5.2s @ 1.24GB)
5,006 (82.00%) of 6,105 classes reachable
6,827 (64.24%) of 10,627 fields reachable
18,204 (53.34%) of 34,128 methods reachable
553 classes, 1,423 fields, and 1,163 methods registered for reflection
1 native library: -framework CoreServices
Fatal error: com.oracle.svm.hosted.annotation.AnnotationMetadata$AnnotationExtractionError: java.lang.reflect.InvocationTargetException
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.getRoot(SubstrateAnnotationExtracter.java:460)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.getAnnotationData(SubstrateAnnotationExtracter.java:195)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.extractAnnotation(SubstrateAnnotationExtracter.java:166)
at org.graalvm.nativeimage.base/com.oracle.svm.util.GuardedAnnotationAccess.getAnnotation(GuardedAnnotationAccess.java:82)
at org.graalvm.nativeimage.base/com.oracle.svm.util.GuardedAnnotationAccess.getAnnotation(GuardedAnnotationAccess.java:74)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.platformSupported(SVMHost.java:740)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:461)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.getDeclaredMethods(AnalysisType.java:1194)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.findMethod(AnalysisType.java:1199)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.reflect.ReflectionDataBuilder.registerHidingSubTypeMethods(ReflectionDataBuilder.java:376)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.reflect.ReflectionDataBuilder.registerHidingSubTypeMethods(ReflectionDataBuilder.java:396)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.reflect.ReflectionDataBuilder.processMethodMetadata(ReflectionDataBuilder.java:279)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.reflect.ReflectionDataBuilder.duringAnalysis(ReflectionDataBuilder.java:199)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.reflect.ReflectionFeature.duringAnalysis(ReflectionFeature.java:254)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$10(NativeImageGenerator.java:748)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:85)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$11(NativeImageGenerator.java:748)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.AbstractAnalysisEngine.runAnalysis(AbstractAnalysisEngine.java:162)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:745)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:578)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:535)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:403)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:580)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.getRoot(SubstrateAnnotationExtracter.java:440)
... 23 more
Caused by: java.lang.SecurityException: class "com.azure.storage.blob.BlobServiceClient"'s signer information does not match signer information of other classes in the same package
at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1151)
at java.base/java.lang.ClassLoader.preDefineClass(ClassLoader.java:906)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1015)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asReflectionExecutable(Native Method)
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asReflectionExecutable(CompilerToVM.java:1126)
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJDKReflection.getMethod(HotSpotJDKReflection.java:291)
... 27 more
$ java -version
openjdk version "17.0.4" 2022-07-19
OpenJDK Runtime Environment GraalVM CE 22.2.0 (build 17.0.4+8-jvmci-22.2-b06)
OpenJDK 64-Bit Server VM GraalVM CE 22.2.0 (build 17.0.4+8-jvmci-22.2-b06, mixed mode, sharing)
[1/7] Initializing...
Version info: 'GraalVM 22.2.0 Java 17 CE'
Java version info: '17.0.4+8-jvmci-22.2-b06'
C compiler: cc (apple, arm64, 14.0.0)
Garbage collector: Serial GC
3 user-specific feature(s)
- io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
- io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.jboss.threads] categories
- io.quarkus.runtime.graal.ResourcesFeature: Register each line in META-INF/quarkus-native-resources.txt as a resource on Substrate VM
[2/7] Performing analysis... [16:31:03,513 INFO [com.azu.cor.imp.jac.JacksonVersion] Package versions: jackson-core=2.13.4, jackson-databind=2.13.4-2, jackson-dataformat-xml=2.13.4, jackson-datatype-jsr310=2.13.4, azure-core=1.33.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot
*] (4.9s @ 1.10GB)
5,090 (83.25%) of 6,114 classes reachable
6,894 (61.83%) of 11,150 fields reachable
18,450 (58.96%) of 31,295 methods reachable
569 classes, 1,424 fields, and 342 methods registered for reflection
1 native library: -framework CoreServices
Fatal error: com.oracle.svm.hosted.annotation.AnnotationMetadata$AnnotationExtractionError: java.lang.reflect.InvocationTargetException
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.getRoot(SubstrateAnnotationExtracter.java:454)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.getAnnotationData(SubstrateAnnotationExtracter.java:189)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.extractAnnotation(SubstrateAnnotationExtracter.java:166)
at org.graalvm.nativeimage.base/com.oracle.svm.util.GuardedAnnotationAccess.getAnnotation(GuardedAnnotationAccess.java:81)
at org.graalvm.nativeimage.base/com.oracle.svm.util.GuardedAnnotationAccess.getAnnotation(GuardedAnnotationAccess.java:73)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.platformSupported(SVMHost.java:721)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:449)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.getDeclaredMethods(AnalysisType.java:1158)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.findMethod(AnalysisType.java:1163)
at org.graalvm.nativeimage.builder/com.oracle.svm.reflect.hosted.ReflectionDataBuilder.registerHidingSubTypeMethods(ReflectionDataBuilder.java:386)
at org.graalvm.nativeimage.builder/com.oracle.svm.reflect.hosted.ReflectionDataBuilder.registerHidingSubTypeMethods(ReflectionDataBuilder.java:406)
at org.graalvm.nativeimage.builder/com.oracle.svm.reflect.hosted.ReflectionDataBuilder.processMethodMetadata(ReflectionDataBuilder.java:287)
at org.graalvm.nativeimage.builder/com.oracle.svm.reflect.hosted.ReflectionDataBuilder.duringAnalysis(ReflectionDataBuilder.java:187)
at org.graalvm.nativeimage.builder/com.oracle.svm.reflect.hosted.ReflectionFeature.duringAnalysis(ReflectionFeature.java:238)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$10(NativeImageGenerator.java:734)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:78)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$11(NativeImageGenerator.java:734)
at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis.runAnalysis(PointsToAnalysis.java:755)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:731)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:564)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:521)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:407)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:585)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:128)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.annotation.SubstrateAnnotationExtracter.getRoot(SubstrateAnnotationExtracter.java:434)
... 23 more
Caused by: java.lang.SecurityException: class "com.azure.storage.blob.BlobServiceClient"'s signer information does not match signer information of other classes in the same package
at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1158)
at java.base/java.lang.ClassLoader.preDefineClass(ClassLoader.java:902)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1010)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.asReflectionExecutable(Native Method)
at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJDKReflection.getMethod(HotSpotJDKReflection.java:291)
... 27 more
@all-contributors add @agoncal for code and maintenance
At startup it would be good to create containers. We could do that using a set of properties:
quarkus.azure.storage.blob.container."containerA".generation=create
quarkus.azure.storage.blob.container."containerB".generation=drop-and-create
quarkus.azure.storage.blob.container."containerC".generation=none
And then be able to set some properties:
quarkus.azure.storage.blob.container."containerA".auth-mode=key
quarkus.azure.storage.blob.container."containerA".fail-on-exist=true
quarkus.azure.storage.blob.container."containerA".public-access=blob
quarkus.azure.storage.blob.container."containerA".metadata={key1/value1,key2/value2,key3/value3}
Where:
auth-mode
: The mode in which to run the command. "login" mode will directly use your login credentials for the authentication. The legacy "key" mode will attempt to query for an account key if no authentication parameters for the account are provided. Allowed values: key
, login
.fail-on-exist
: Throw an exception if the container already exists.metadata
: Metadata in space-separated key=value pairs. This overwrites any existing metadata.public-access
: Specifies whether data in the container may be accessed publicly. Allowed values: blob
, container
, off
.Hi,
It will be very useful and secure to have a dependency that allows us to connect our Quarkus Applications properties with Azure KeyVault to retrieve secrets.
The concept is very simple, just add a few properties to enable the extension to list all properties and select those that make a match with a specific placeholder.
Example:
password: ${APPLICATION_USER_PASSWORD:localpassword}
In Spring Cloud there is already an extension for that https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/configure-spring-boot-starter-java-app-with-azure-key-vault but this only works for spring I guess.
Currently, #218 covers creating secrets using sync and async client.
This issue is to cover more APIs.
@all-contributors add @majguo for code and maintenance
The idea/requirement actually comes from #179: produce an async client so it can be used in reactive programming in Quarkus.
The metadata for the keyvault extension lists https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-key-vault.html as a guides link, but that link isn't live yet.
I suggest either
The code that is making the native compilation fail is:
void shouldDownloadATextfile() {
given()
.when().get("/quarkus-azure-storage-blob")
.then()
.statusCode(200)
.body(startsWith("Hello quarkus-azure-storage-blob"));
}
It has been disabled for now. It invokes the following method:
@GET
@Produces(MediaType.TEXT_PLAIN)
public String downloadBlob() {
BlobContainerClient blobContainerClient = blobServiceClient
.createBlobContainerIfNotExists("container-quarkus-azure-storage-blob");
BlobClient blobClient = blobContainerClient.getBlobClient("quarkus-azure-storage-blob.txt");
return blobClient.downloadContent().toString();
}
This returns plain text and should not need any XML parser (as shown in the stack trace below)
2022-11-18 11:21:48,260 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-0) HTTP Request to /quarkus-azure-storage-blob failed, error id: a2fcf69d-510a-43ee-b2af-637943bf67d0-1: org.jboss.resteasy.spi.UnhandledException: reactor.core.Exceptions$ReactiveException: javax.xml.stream.FactoryConfigurationError: Provider for javax.xml.stream.XMLInputFactory cannot be found
at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:105)
at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:359)
at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:218)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:519)
at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:261)
at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:161)
at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:164)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:247)
at io.quarkus.resteasy.runtime.standalone.RequestDispatcher.service(RequestDispatcher.java:82)
at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.dispatch(VertxRequestHandler.java:147)
at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler$1.run(VertxRequestHandler.java:93)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:576)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base@17.0.5/java.lang.Thread.run(Thread.java:833)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)
Caused by: reactor.core.Exceptions$ReactiveException: javax.xml.stream.FactoryConfigurationError: Provider for javax.xml.stream.XMLInputFactory cannot be found
Introduce configuration property for selecting list of labels.
If you search for 'image' in these instructions, it shows how to set an Microsoft logo: https://quarkus.io/guides/extension-metadata
In the discussion of #37, @agoncal mentioned that the Azure SDK integration tests are executed against a test proxy, which will record the real http request/response in record
mode and return the specific recorded response in playback
mode. The benefit of this approach is obvious which can greatly save cost especially there're a lot of dependent Azure services.
The similar approach can be applied to Quarkus Azure Service extensions as they are built on top of Azure SDK. Details pls see:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.