nextflow-io / nf-hello Goto Github PK
View Code? Open in Web Editor NEWNextflow plugin implementation skeleton
Home Page: https://nextflow.io/docs/latest/plugins.html
License: Apache License 2.0
Nextflow plugin implementation skeleton
Home Page: https://nextflow.io/docs/latest/plugins.html
License: Apache License 2.0
When I try to build the nf-hello plugin against Nextflow 23.05.0-edge or later, I get this error:
Could not determine the dependencies of task ':plugins:nf-hello:test'.
> Could not resolve all task dependencies for configuration ':plugins:nf-hello:testRuntimeClasspath'.
> Could not resolve io.nextflow:nextflow:22.10.0.
Required by:
project :plugins:nf-hello
> No matching variant of project :nextflow:nextflow was found. The consumer was configured to find a runtime of a library compatible with Java 17, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally but:
- Variant 'apiElements' capability io.nextflow:nextflow:23.05.0-edge declares a library compatible with Java 11, packaged as a jar, and its dependencies declared externally:
- Incompatible because this component declares an API of a component and the consumer needed a runtime of a component
- Other compatible attribute:
- Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)
- Variant 'runtimeElements' capability io.nextflow:nextflow:23.05.0-edge declares a runtime of a library compatible with Java 11, packaged as a jar, and its dependencies declared externally:
- Other compatible attribute:
- Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)
- Variant 'testFixturesApiElements' capability io.nextflow:nextflow-test-fixtures:23.05.0-edge declares a library, packaged as a jar, and its dependencies declared externally:
- Incompatible because this component declares an API of a component compatible with Java 19 and the consumer needed a runtime of a component compatible with Java 17
- Other compatible attribute:
- Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)
- Variant 'testFixturesRuntimeElements' capability io.nextflow:nextflow-test-fixtures:23.05.0-edge declares a runtime of a library, packaged as a jar, and its dependencies declared externally:
- Incompatible because this component declares a component compatible with Java 19 and the consumer needed a component compatible with Java 17
- Other compatible attribute:
- Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)
To be precise, this error happens during ./gradlew check
. It builds fine, but then the plugin simply doesn't work at runtime:
// main.nf
include { reverse } from 'plugin/nf-hello'
Channel.reverse('hi!')
$ make compile
$ ./launch.sh run main.nf -plugins nf-hello
N E X T F L O W ~ version 23.05.0-edge
Launching `main.nf` [pensive_becquerel] DSL2 - revision: 0124bb0ec6
Plugin 'nf-hello' does not implement any extension point
I think the root cause is this commit in which the Nextflow build was upgraded from Java 17 to Java 19. I understand that it's supposed to be backwards compatible but something isn't working with the plugins. Maybe we need to update the plugin build script as well?
in order to have an example of how to extend ChannelFactory create some ChannelExtensionPoint with use DataflowWriteChannel
and DataflowReadChannel
As per the README
- Run Nextflow with the plugin, using
./launch.sh
as a drop-in replacement for thenextflow
command, and adding the option-plugins nf-hello
to load the plugin:
./launch.sh run nextflow-io/hello -plugins nf-hello
I get the error
Missing '.launch.classpath' file -- create it by running: ./gradlew exportClasspath
and if I try to run the command suggested, I get another error
./gradlew exportClasspath
FAILURE: Build failed with an exception.
* What went wrong:
Task 'exportClasspath' not found in root project 'nf-hello' and its subprojects.
Ideas?
$ gradle --version
------------------------------------------------------------
Gradle 7.6
------------------------------------------------------------
$ java --version
openjdk 11.0.22 2024-01-16 LTS
OpenJDK Runtime Environment Corretto-11.0.22.7.1 (build 11.0.22+7-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.22.7.1 (build 11.0.22+7-LTS, mixed mode)
running against Nextflow version 567f5334ff0d225cd95dc7b5239c4dbd64696a9a
Hi all,
there is a "compile" target in the Makefile
but it raises the following error:
$ make compile
./gradlew :nextflow:exportClasspath compileGroovy
FAILURE: Build failed with an exception.
* What went wrong:
Cannot locate tasks that match ':nextflow:exportClasspath' as project 'nextflow' not found in root project 'nf-hello'.
The plugin build template should follow the same template used in the nextflow core plugins .
The current process for building and testing a plugin requires you to build Nextflow locally, include it in the plugin build, and use ./launch.sh
to test the plugin with Nextflow in dev mode. This process is cumbersome and prone to errors,
An alternative which I first tried in the nf-prov plugin and is also being added to nf-validation here is to build and install the plugin directly to the plugins directory. While it may overwrite an existing plugin installation, it allows you to test the plugin without having to build Nextflow. It also avoids certain limitations of Nextflow dev mode (mainly that other non-core plugins cannot be used).
cross-posted on slack: https://nextflow.slack.com/archives/C02T98BKE2K/p1709894909502359
I tried to compile the hello plugin but I got the following error/stacktrace:
$ ./gradlew check
> Task :plugins:nf-hello:compileGroovy FAILED
java.util.ServiceConfigurationError: Locale provider adapter "CLDR"cannot be instantiated.
at java.base/sun.util.locale.provider.LocaleProviderAdapter.forType(LocaleProviderAdapter.java:200)
(...)
... 12 more
Caused by: java.lang.IllegalArgumentException: newLimit > capacity: (55346993 > 27910144)
at java.base/java.nio.Buffer.createLimitException(Buffer.java:398)
at java.base/java.nio.Buffer.limit(Buffer.java:372)
at java.base/java.nio.ByteBuffer.limit(ByteBuffer.java:1539)
at java.base/java.nio.MappedByteBuffer.limit(MappedByteBuffer.java:331)
at java.base/java.nio.MappedByteBuffer.limit(MappedByteBuffer.java:73)
at java.base/jdk.internal.jimage.BasicImageReader.slice(BasicImageReader.java:213)
at java.base/jdk.internal.jimage.BasicImageReader.readBuffer(BasicImageReader.java:372)
at java.base/jdk.internal.jimage.BasicImageReader.getResourceBuffer(BasicImageReader.java:438)
at java.base/jdk.internal.jimage.ImageReader.getResourceBuffer(ImageReader.java:199)
at java.base/jdk.internal.module.SystemModuleFinders$SystemModuleReader.read(SystemModuleFinders.java:485)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:809)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(BuiltinClassLoader.java:741)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClass(BuiltinClassLoader.java:621)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:633)
at java.base/java.lang.Class.forName(Class.java:584)
at java.base/java.lang.Class.forName(Class.java:559)
at java.base/java.util.ServiceLoader.loadProvider(ServiceLoader.java:859)
at java.base/java.util.ServiceLoader$ModuleServicesLookupIterator.hasNext(ServiceLoader.java:1084)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
at java.base/sun.util.cldr.CLDRLocaleProviderAdapter.lambda$new$0(CLDRLocaleProviderAdapter.java:84)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
at java.base/sun.util.cldr.CLDRLocaleProviderAdapter.<init>(CLDRLocaleProviderAdapter.java:83)
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
... 15 more
FAILURE: Build failed with an exception.
I googled the error message but I don't really understand what I saw. My local javac version is avac 17.0.3.1.
Any idea ?
The plugin should be updated to make it compatible with APIs introduced by Nextflow 22.08.0-edge
We should add a basic integration test to the CI that simply runs a nextflow pipeline with the plugin enabled.
For nf-hello it would probably just be a local pipeline script showing the customer channel factory, operator, etc, but different plugins will have different use cases. The important thing is to create an example here and then encourage plugin developers to do the same for their plugins (and we would do the same for nf-prov and nf-sqldb).
This will give people better confidence that plugin releases actually work with some version of Nextflow.
Create a cookie cutter template based on nf-hello, with a command-line tool that allows the user to enter things like plugin name, target nextflow version, etc. Should make it easier to start a new plugin instead of starting from nf-hello.
Tagging @ewels because I will probably ask you for advice on this at some point
The file HelloExtension.groovy extends PluginExtensionPoint, which no longer exists in Nextflow.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.