Comments (13)
Thank you for documenting this use case! We'll take a look at it.
@koral--, what do you think about this idea?
from shipkit.
I haven't deployed yet such artifacts. However, for me it look like a valid use case for maven POM classifier:
https://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
http://maven.apache.org/pom.html#Dependencies
On linked docs there is an example with jdk8
and jdk11
classifiers so it seems that it also can be applied to versions of the Spark.
If this assumption is true then probably it should be enough to configure maven publication in the project, it may be even opaque to Shipkit. Haven't checked if bintray supports classifiers though.
from shipkit.
I suspect that classifiers will not help here. @jverbus needs to run the build multiple times with different variant. To goal is to make something like that possible with Shipkit:
./gradlew build -Pvariant=2.11
./gradlew build -Pvariant=2.12
./gradlew release // releases all variants
from shipkit.
@jverbus, we haven't had time yet to look into this. We'll try to make more progress sometime this month.
from shipkit.
@mockitoguy, thanks for the update!
from shipkit.
@mockitoguy, @koral-- : Checking in to see if there are any updates on this feature request.
from shipkit.
Hey @jverbus, we did not make tangible progress on this yet. I have a couple of ideas but did not have a chance yet.
from shipkit.
Thanks for the update, @mockitoguy!
from shipkit.
Hey, @mockitoguy!
I was recently thinking about this some more. Here is a bit more detail on my current process and an idea for a solution.
Current process
I'm now building and publishing three artifacts (with different Scala/Spark version dependencies) under the same project version. I expect the number of different builds under the same project version will continue to increase.
Here are the artifacts from a recent commit: https://bintray.com/linkedin/maven/isolation-forest/0.3.3#files/com/linkedin/isolation-forest
I have Travis CI build all three variants on each push to validate that all dependency combinations work. These builds are done in parallel. Here is the Travis CI config:
https://github.com/linkedin/isolation-forest/blob/master/.travis.yml
Due to the Shipkit limitations described earlier in this issue thread, only one artifact variant is automatically released to Bintray via the Travis CI build. I currently publish the others manually via ./gradlew bintrayUpload
on my local machine.
Once all artifact variants for a particular project version have been uploaded to Bintray, I manually sync them to Maven Central.
https://oss.sonatype.org/#nexus-search;quick~isolation-forest
What I'd like
If we can have all of those builds in the Travis CI build matrix publish their results to Bintray under the same project version (0.3.3 in the example links above), that would be awesome. I think this would also allow me to pretty easily automate the subsequent sync with Maven Central in Travis CI as well.
Potential solution
The thing that seems tricky is that having multiple Travis CI build processes running in parallel could potentially result in race conditions where the Shipkit publishing / auto version number updates step on each other.
If the user could have more granular control of the version.properties
update process with Shipkit, this might be a solution. For example, if the user could turn off the auto version update during the build + ciPerformRelease jobs in the build matrix, then all of the build tasks would upload a different artifact with the same project version. I could then use a Travis build stage (https://docs.travis-ci.com/user/build-stages#how-to-define-build-stages) to run a Shipkit command to update the version.properties
in the git repo after all of the jobs in the build matrix succeed.
Perhaps something like this is already possible given the current Shipkit / bintray plugin functionality? Looking forward to hearing what you think. Thanks!
from shipkit.
Hey, @jverbus
Could you look at #869 please? Maybe it will help in your case?
from shipkit.
Thanks, @mstachniuk!
It seems like that solution would require creating a separate sub-module in the project for each additional artifact variant? I could imagine having builds for multiple different combinations of Spark/Scala dependencies, so this doesn't scale too well.
I think I found an inelegant, but seemingly working, solution for my use case using multiple Travis build stages. I should be able to scale up to more artifact variants by adding additional jobs to the "upload" Travis stage in the link below.
Here is the Travis config: https://github.com/linkedin/isolation-forest/blob/master/.travis.yml
Here is an example of a successful run: https://travis-ci.org/github/linkedin/isolation-forest/builds/691954282
It would be nice if there was a built in Shipkit or bintray-gradle-plugin task that could perform something similar to what I am doing for each job in the "upload" Travis stage.
from shipkit.
If the user could have more granular control of the version.properties update process with Shipkit, this might be a solution
The vision of for the "version.properties" is to avoid version bump commits - like the https://github.com/shipkit/shipkit-auto-version
@jverbus, I created #870 to track cleaning up isolation-forest
from shipkit.
#869 I am interested in this feature as well, as I want to use the shadow plugin to additionally upload a fat jar besides the regular publications.
from shipkit.
Related Issues (20)
- javadoc and java plugins are applied in the wrong order HOT 1
- Disabling Release Notes Generation HOT 3
- Unicode in user name of commit HOT 1
- Instructions/compatibility with GitHub Actions HOT 2
- Release to GH and add generated files to release HOT 8
- POM exception when using shipkit with jar archiveClassifier HOT 2
- deprecation warning: The baseName property has been deprecated HOT 3
- BintrayDefaultArtifactUrlResolver does not URL encode slashes in file_path HOT 1
- Adding an archive from a subproject's task causes a build failure HOT 10
- Default POM generated by Shipkit does not meet Maven Central requirements HOT 2
- github access_token query parameter deprecated, breaks after 2020/07/01 HOT 6
- -PskipComparePublications=true doesn't work with ./gradlew -PskipComparePublications=true
- Question: GitHub release provider for Travis needed? HOT 1
- Release normal jar and fat jar at the same time HOT 7
- Shipkit for Scala multi-variant project HOT 4
- Build fails due to upgrade downstream failure on code coverage check
- GradlePortalReleasePlugin: Cannot add task 'validatePlugins' as a task with that name already exists
- Include LICENSE to META-INF rather that to the root of the archives HOT 2
- Will shipkit automatically tag and push to bintray after setup? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from shipkit.