Request: Demo application code about update4j

[martinm1000]

It works !

I needed to add a require xstream in my boot.jar's module-info, and I do specify -add-opens xxx/xxx=xstream.

[mordechaim]

I dedicated today to complete the whole Javadoc. I plan to publish demo code in the near future. Per a launcher; the framework was originally designed in a way that 'you' provide the launch code (via the various service providers) but I do understand that people like things that "just work". I will make a default command-line argument based launcher, will be available on next release.

[martinm1000]

Thank you.

One of the first thing that needs to be done is generating the configuration file. I've seen that you worked on a maven plugin to (I think) do that. Any chance this is currently working ?

[mordechaim]

I started working on a Maven plugin, but diverted focus to complete this framework itself.

You can use Configuration.builder() to generate them until that get's done.

[mordechaim]

Check out the newest release. Just start the framework as a runnable jar file. It should be much clearer and easier to use. Awaiting your response.

[martinm1000]

How do you generate the configuration file with relative paths ?

I can't get the xml result I want, it hardcode the paths to the files it read for computing the crc, and I don't want this.

Collection<Reference> libs = FileMetadata
			.streamDirectory("D:\\_TEMP\\xxxx\\p\\libs")
			.peek(Reference::modulepath)
			.collect(Collectors.toSet());
'''

this generate :

 <file path="D:/_TEMP/xxxx/p/libs/joda-time.jar" size="708990" checksum="ef8e5f5c" modulepath="true"/>

and I want :

 <file path="libs/joda-time.jar" size="708990" checksum="ef8e5f5c" modulepath="true"/>
or 
<file path="joda-time.jar" size="708990" checksum="ef8e5f5c" modulepath="true"/>

[mordechaim]

Use the 'path()' method to set the path to be generated instead of using the real location.

[mordechaim]

You can query the actual path (to work around with in the path() method) with 'getSource()'. Re main class: If you use the default launcher, just add the main class name as a property with key 'default.class.main.launcher'. If you use your own launcher implementation (that gives you more context about the configuration) just specify the class as a provider via:     provides org.update4j.service.Launcher with com.example.MyLauncher; Re adding modules: Add it the same way you would add any other application, namely, start the application with the 'add-modules' option.

[martinm1000]

Thank you, this works !

final Path jarpath = Paths.get("D:\\_TEMP\\xxxx\\p\\");
final List<Reference> libs = FileMetadata.streamDirectory(Paths.get(jarpath.toString(), "libs"))
	.peek(Reference::modulepath)
	.peek(r -> r.path(jarpath.relativize(r.getSource())))
	.collect(Collectors.toUnmodifiableList());

Outputs:

<file path="libs/joda-time.jar" size="708990" checksum="ef8e5f5c" modulepath="true"/>

[mordechaim]

That's great to hear. About demo code and better docs, just understand I am a one man band and not working on this project all day; that's my humble contribution to open source. I just don't have the time/effort to work on them. It would be very nice if the community can take me over with documenting this.

[martinm1000]

I understand, I am also quite busy.
I'll soon open a other issue for discussions, regarding signatures.
If I can get a replacement for java webstart working, I might have some time ;-)

[mordechaim]

Thanks. By the way, reading your code; you can resolve libs path using the parent's 'resolve' method, no need to get a string and build a new path from scratch. Just a side note.

[martinm1000]

EDIT: forget this one, I inverted module and package...

Any idea why I'm getting this (I'm still on Java 10)

<file path="libs/controlsfx.jar" size="1009762" checksum="88dc9807" modulepath="true"/>
        <addOpens>
           <opens package="javafx.base" target="com.sun.javafx.runtime"/>
        </addOpens>
 </file>

Exception in thread "main" java.lang.IllegalStateException: Module 'com.sun.javafx.runtime' is not known to the layer.
at [email protected]/org.update4j.Configuration.lambda$19(Configuration.java:1266)
at java.base/java.util.Optional.orElseThrow(Unknown Source)
at [email protected]/org.update4j.Configuration.launchImpl(Configuration.java:1266)

while I don't get problems with some other addOpens as :

    <file path="libs/xstream.jar" size="646895" checksum="2183e293" modulepath="true">
            <addOpens>
                <opens package="java.awt.font" target="java.desktop"/>
                <opens package="java.lang.reflect" target="java.base"/>
                <opens package="java.text" target="java.base"/>
                <opens package="java.util" target="java.base"/>
            </addOpens>
        </file>

that add-opens for javafx work on the command line for java so...

[mordechaim]

Target takes a module, not a package. In general, add exports/opens/reads operates on packages in that specific library. I see you try to open system packages by listing it on a random library. I doubt what you do really works.  In your case, just use the command-line flags.

[martinm1000]

They are not on random libraries, that's what I needed to do for them to work under Java 9 and my app.

There's one for javafx, and the others are for xstream to be able to do its reflection for some types.

Unless I messed up adding them to your configuration file, they means :

--add-opens java.base/java.util=xstream
--add-opens java.base/java.lang.reflect=xstream
--add-opens java.base/java.text=xstream
--add-opens java.desktop/java.awt.font=xstream
--add-opens javafx.base/com.sun.javafx.runtime=controlsfx

[martinm1000]

Question: are config.xml properties available to my app too ? (using System.getProperty)

[mordechaim]

Ok I see, you are not doing the opening correctly. 'package' should name an explicit package in xstream that you want to open, and target should name the module that you want to open it to. You cannot open an entire module via add opens. You have to list each and every package. Re properties: you can call any of the various getProperty methods in the config class. Read the javadoc to see the differences between them.

[martinm1000]

Oh, these do not directly translate to AddPackage...

reference.opens(new AddPackage(...)) are to be used to open stuff between modules if needed and I should just stick my --add-opens to java.exe as before.

edit: I'm still unsure... these are not exports... damn this modular stuff is complicated...

[mordechaim]

Not exactly, you can use it if the package to be opened is in the module currently referenced. In your case package should be something like 'com.xstream.somepackage' and target 'java.desktop'.

[martinm1000]

As for properties, since they are not visible to java's system properties, I really need to move to a delegate updater to be able to feed them as system properties (webstart's properties could be read as system properties).

[mordechaim]

This could be added in a future version, will think about it. You might feed the properties in bootstrap; no need to make a update handler for this.

[martinm1000]

Not exactly, you can use it if the package to be opened is in the module currently referenced. In your case package should be something like 'com.xstream.somepackage' and target 'java.desktop'.

Not sure it would work then... xstream need access to have access to stuff from java not the other way around... ? anyway, I'll just put them on java.exe.

[martinm1000]

This could be added in a future version, will think about it. You might feed the properties in bootstrap; no need to make a update handler for this.

Umm, i need to re-read the docs... I tough that was the same thing (bootstrap & update handler).

There are security issues regarding setting system properties if the config.xml get hacked. Webstart had to mess around with "secure" properties (starting with jnlp). This might not be a good or easy idea.
https://bugs.openjdk.java.net/browse/JDK-8023821

But there are also other issues regarding security and what's signed / validated / trusted (I need to test and verify some stuff before submitting a issue)

[mordechaim]

Ah! If it's the other way around, then command-line flags is way to go. Bootstrap is completely different from update handler (just a callback class). Bootstrap updates and launches any number of configurations, I.e. A single bootstrap can handle many business applications.

[martinm1000]

Ok, but if the official way is command-line flag, we might have a problem if I need to open some specific jdk packages to one module when I add in an update that new module/library ?

Said in another way, if there is no official xml section for these declarations, and that I use a run.bat file for starting java & update4j, the application won't work as the java runtime is on previous command options.

If you had an official section in the xml for these 'jdk opens', you might be able to dynamically do the updated opens at runtime ?

[mordechaim]

This is a deliberate restriction from the JPMS developers to discourage overriding the module system.

[martinm1000]

Oh. so you can't to these 'reverse' opens at runtime ?
Bummer. unless what I did worked (even if that was not the api intent) ?
I'm not sure of when I tested xstream, but I did try it and didn't get an error when I serialized some data.

I might need to do a hello world app to be sure...

[martinm1000]

Should I extends DefaultBootstrap ? or copy what I need from it ?

What would be the preferred way to have my own for bootstrap code just to move from the config.xml's properties to system properties to my app can read them ?

Right now I can start update4j and it goes to my (implemented) delegate, but there no downloading or nothing else being done.

[mordechaim]

In standard mode, ignore the whole delegate process. Just read the config file yourself using Configuration::read and manually call config.update() and config.launch(). Note: there are many overloads of these 2 methods.

[martinm1000]

Thanks. So I'm still in standard mode... who knew... ;-)

I don't understand... I should not be in a Delegate ?
My app doesn't have update4j as a dependency, so where would that code be ?

[mordechaim]

Well that's what you said in a previous comment. It is all up to you, for most cases you can stick to the default bootstrap and design the business app only, use the default launcher (i.e. Do nothing) and you don't need a dependency at all. (note: you can lay hand on the config instance in default bootstrap only by using a launcher implementation and not the default launcher. This way you can feed the properties in the business app, but now you need a dependency to update4j in the business app). You are technically in delegate mode, so it is started via update4j's main method, but this detail can safely be ignored here. Otherwise, for custom setup in the bootstrap, make a separate application (i.e. Separate jar file) that depends on update4j, and do all update and launch logic there, leaving the business app completely unaware of update4j (or use your own launcher implementation to make the business app aware and depend on update4j in business app). You start the bootstrap just as you would with any other application, namely, your own main method. THIS detail is what makes standard mode different from delegate mode. Leave delegate mode aside for now. It adds an extra layer of complexity. Check it out once you can get standard mode to work. I still feel I didn't explain myself enough; feel free to follow up.

[mordechaim]

I now noticed the wording in last comment, so you are on delegate mode. Sorry for missunderstanding this at first. In delegate mode you still have to do update/launch logic yourself via update/launch methods. So it's all up to you, I wouldn't just copy paste DefaultBootstrap code.

[martinm1000]

In a delegate, I got to set the config properties to system properties, and I can now use my app.
You were rights, the addOpens I added to xstream don't do the job as they are for the inverse case.
Thanks.

[martinm1000]

Otherwise, for custom setup in the bootstrap, make a separate application (i.e. Separate jar file) that depends on update4j, and do all update and launch logic there, leaving the business app completely unaware of update4j

This is what I have done. A separate jar that depends on update4j. Inside is some code that can generate my config.xml and now, a custom delegate. Something simple for now but I might want to customize it more later, or at least play with it.

Now I'll try to sign the files... quick questions about the current signing implementation:

1. This is not about validating signed jars (with jarsigner) right ?

2. Is the public key I'll use validated on a trusted CA certificate from java ?

3. Can an attacker modify a source installation by modifying/adding to the config.xml file, replace my public key certificate and resign all himself ? How would an existing client installation react to that update ?

[martinm1000]

So.. I'm trying to do the java --add-opens module/package=to-module,:

WARNING: Unknown module: xstream specified to --add-opens

but the module system doesn't know those auto-modules at this point (I think).
Does that mean that I can only open them to ALL-UNNAMED and have those libraries on the classpath ?

[mordechaim]

Re signatures: the mechanism is completely independent from jarsigner and jars in general. You can use it for any file. It just makes sure the signature was generated by the private key belonging to that public key. It has no connection to CA certificates in any way, although you can do any type of file validation if you want including checking against the CA store by implemeting an update handler and validating (and throwing exception if failing) at the 'validatingFile()' method using the context object to get the passed certificate. An attacker could not change the trusted certificate if -- and only if -- the certificate is a local file. They cannot replace it by listing it in the config, since they don't have the private key and will be rejected. Unless they inject their cert some other way, it would not be your side that was comprimized. If the cert is a remote resource, the server could be comprimized, yes. Re opening: unnamed should work, but you an still do a slight hack by having 2 versions of xstream, one in bootstrap (never really used, just to tame jpms) and one in business. You will have to mark 'ignoreBootConflict' on that file entry in the config.

[martinm1000]

Re signatures: the mechanism is completely independent from jarsigner and jars in general. You can use it for any file. It just makes sure the signature was generated by the private key belonging to that public key. It has no connection to CA certificates in any way, although you can do any type of file validation if you want including checking against the CA store by implemeting an update handler and validating (and throwing exception if failing) at the 'validatingFile()' method using the context object to get the passed certificate. An attacker could not change the trusted certificate if -- and only if -- the certificate is a local file. They cannot replace it by listing it in the config, since they don't have the private key and will be rejected. Unless they inject their cert some other way, it would not be your side that was comprimized. If the cert is a remote resource, the server could be comprimized, yes.

Thank you, this is clear. I might want to play with this eventually, to get more the experience of webstart (ask the user if he trust the signing certificate, warn if certificate change, etc) and actually validate the jars.

Re opening: unnamed should work, but you an still do a slight hack by having 2 versions of xstream, one in bootstrap (never really used, just to tame jpms) and one in business. You will have to mark 'ignoreBootConflict' on that file entry in the config.

Yeah, I'm really stuck here. I'll try it as I can't think of anything else that could work (my understanding of JPMS isn't that good ;-)

[martinm1000]

I can't seems to make it work.

My business app use xstream on the module-path (xstream isn't a module nor does it have a auto-module-name entry).

I'm trying to also have it on the boot path as a classpath or module-path but anything that let the app start will crash with the error when the jdk module isn't open to xstream.

[martinm1000]

So it worked but its wrong. The way I did it, the boot version of the jar is the one the application use (found out modifying that jar). Which also seems to be against the warning ignoreBootConflict=false says.

But If I try to use the boot classpath for that xstream.jar, the add-opens to ALL-UNNAMED doesn't seems to be doing anything and I will crash in my app (event if my app defines that same xstream.jar in its modulepath.

So I'm not sure of what's going on.

[mordechaim]

Honestly, I myself never tried using ALL-UNNAMED so I can't guarantee anything. About the 2 versions, the way the module-path was designed is, the boot will use the boot version and the business app will use the other. So it looks like you checked the versions in the boot.

[martinm1000]

it was modulepath=true, and I started the updater with a class-path and a module path.

I need to re-evaluate if I still want to use an auto updater at this point. Thank you.