Upgrade the webdav test library / probably others about nio-fs-provider

[laeubi]

@maddingo can you have a look here?

The Updated lib is here: https://search.maven.org/artifact/io.github.atetzner/webdav-embedded-server/0.2.1/jar

[maddingo]

I wonder if it would be better to use a library with less obscure dependencies. Probably a docker container and TestContainers.

[laeubi]

I wonder if it would be better to use a library with less obscure dependencies. Probably a docker container and TestContainers.

If you think it is less effort than using a regular maven dependency... why do you think it is "obscure"? The lib is now available as a regular maven-central one.

[ag88]

hi, i've been trying out some stuff with webdav with this, there are some bummers, but thanks for creating this.
tomcat actually has a webdav servlet
https://github.com/apache/tomcat/blob/main/java/org/apache/catalina/servlets/WebdavServlet.java
https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/servlets/WebdavServlet.html
but not quite "embedded",

edit: oops I made a mistake i used this one
https://sourceforge.net/projects/webdav-servlet/
and patched it to run with Eclipse Jetty, hence 'embedded'.
it works pretty well, but probably not as "industrial strength" as milton's.
I've been wondering what other 'webdav servers' are there, jackrabbit comes to mind (https://jackrabbit.apache.org/archive/wiki/JCR/WebDAV_115513525.html),
wink (retired)
https://wink.apache.org/
https://wink.apache.org/1.4.0/api/org/apache/wink/webdav/server/package-summary.html
etc comes to mind, but that jackrabbit is 'huge', not sure if there are others, but I think atetzner's looks pretty good.
https://github.com/atetzner/webdav-embedded-server
I've not really tried out milton's webdav servlet, but that i'd guess it is one of the "common" one out there.

while I run it in debug, the webdav-nio-fs-provider is actually connecting and retrieving data from my little patched webdav servlet from sourceforge - running on jetty. things are "a litty goofy" as not all methods are implemented, but it certainly looks like quite a lot of work has been done.
for the record e.g. Files.list(path); isn't there yet
Files.isDirectory(path) in turn calls
https://github.com/maddingo/nio-fs-provider/blob/master/webdav/src/main/java/no/maddin/niofs/webdav/WebdavFileSystemProvider.java#L266
readAttributes(), but apparently readAttributes() is expecting a single file, while its
List<DavResource> resources = wfs.getSardine().list(path.toUri().toString());
retrieved the whole directory/folder of files, hence
if (resources.size() != 1) { throw new IllegalArgumentException(); } triggers

well, not complaints but that it is still much work that has been done and this would work for a regular file.
oh and sourceforge's webdav servlet works with
http://www.davexplorer.org/
davexplorer is a little aged, but that it is still an incredible package java1.2 ! that the binary literally still runs today !
i'm halfway hoping that nio-fs-provider could be a 'modern' update to that.

[laeubi]

@ag88 thanks for looking into it, my initial idea was to simply upgrade the item that is already used by the project to get it into a usable state and catch up with latest improvements.
Identifying missing features would then be another (important) topic but if everything is up to date much easier to report and address and I think the nio provider here just deserves some more interest it is really great to see this theoretical "others" providers work, one thing that could also be a cool enhancement would be using jgit to provide access to (local or remote) git repos.

[ag88]

I think atetzner's webdav server looks pretty good. But I'd need to try it out to see how it works
https://github.com/atetzner/webdav-embedded-server
being in maven central certainly helps. At the moment, I'm hitting up some issues with webdav-nio-fs-provider as it seemed the implementation is incomplete, but that i'm using "sourceforge's webdav server"

https://sourceforge.net/projects/webdav-servlet/
this works with http://www.davexplorer.org/
but when I try to use webdav-nio-fs-provider in a different set of codes apparently I run into some stumbling blocks / errors possibly due to incomplete implementations in webdav-nio-fs-provider
I'm not sure how much variance there are between webdav servers functioning as file servers.
http://www.webdav.org/
strictly speaking the "filesystem" stereotype of data structures isn't specified in the webdav RFCs
I'm not sure how much variances are there between the different implementations of "webdav file servers".
I could hardly find a 'reference implementation' to that specifies it and that it could be tested against.
I'm thinking about using tomcat's as it is shipped in the releases. But I remembered encountering some issues trying to use those, it is a reason I've resorted to "sourceforge's webdav server"
https://sourceforge.net/projects/webdav-servlet/
which purpotedly is based on tomcat's. i'm not sure what is 'missing' in tomcat's implementation as I've not tried it for some time and would need to test it again to understand what are the problems.
if tomcat's work well and is a complete "webdav file server" ref implementation, maybe we could switch to that instead.
https://github.com/apache/tomcat/blob/main/java/org/apache/catalina/servlets/WebdavServlet.java
https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/servlets/WebdavServlet.html
However, as the servlet as an app are normally not part of 'tomcat server' itself, they are apps that run in tomcat, i'm not sure how that could be 'embedded'. A way is to fork that code so that the servlet runs independently say with a jetty or embedded tomcat implementation of it.

[maddingo]

It seams milton.io is now available on maven central. "obscure" because I remember there were some transient dependencies that were not on maven central.

[laeubi]

@maddingo thats exactly why I opened this ticket, the library was updated to the latest (central) dependencies and is now also available on central as well.

So the first step could be to simply upgrade to that new version, changes should be minimal.

[ag88]

i've not really explored tomcat's webdav servlet
https://github.com/apache/tomcat/blob/main/java/org/apache/catalina/servlets/WebdavServlet.java
if that fits a 'reference implementation' maybe use that instead?
I remembered i hit up some issues which is why i used that project on source forge for a webdav server, but we'd need to test that (tomcat's webdav servlet) again to see how well it conforms to a webdav server, at least just files and directories

[ag88]

ok I managed to create a WebDAV server based on Apache Tomcat's WebDAV servlet and embedded Tomcat server

https://github.com/ag88/embtomcatwebdav

This is currently alpha, host (localhost) and port 8082 is patched in the codes.
It serves the current working directory using WebDAV at http://localhost:8080/webdav.
No authentication etc, it creates a directory called 'work' on startup.

The codes is 'deceptively' simple, a single class, 65 lines including blanks etc.
App.java

The only dependency is practically just Apache Tomcat itself.

did some tests, nio-fs-webdav with the updated codes actually works ok/well with it.
edit:
I made various updates, added command line arguments and added (BASIC) authentication making it a more complete app. nio-fs-webdav continues to work ok/well with authentication enabled.

edit:
made more updates
The codes is now more developed as an app, nevertheless it can still be easily adapted as an 'embedded' app.
Still a single class
App.java
to use it as an 'embedded' app, set those instance variables e.g.
App.java#L58
and call runserver()
the various tomcat dependencies can be found in the pom.xml

There are pretty verbose javadocs for tomcat
https://tomcat.apache.org/tomcat-8.5-doc/api/index.html
However, that embedded tomcat use isn't well documented
https://tomcat.apache.org/tomcat-8.5-doc/index.html
It mainly centers around the 'driver' class
Tomcat

[ag88]

just some updates, I made quite a lot of updates to my app
WebDAV server based on Apache Tomcat's WebDAV servlet and embedded Tomcat server

https://github.com/ag88/embtomcatwebdav

Added quite a handful of features and make it more usable. At its core it is still the unadulterated Apache Tomcat 8.5 server and its WebDAV servlet, my app is pretty much a facade to get the WebDAV servlet running as a standalone app.
maven-assembly-plugin shrink wrapped it into a nice 3 MB jar file and the 'whole' thing just run, pretty much with the 'full' power of Tomcat. It is hence that I've been able to add authentication BASIC and DIGEST and even SSL, I'd think if any one work the codes, it is possible to bring about more Tomcat features, access logging, virtual hosts etc. The host that Tomcat use is literally a single virtual host, which implies that it'd literally be able to distinguish requested webs based on HTTP 1.1+ domain requests.

The app is still very much a 'single class' app, all the options/parameters e.g. host, port, basic/digest authentication etc are instance variables. I've exposed all that by means of getter/setter methods so that those instance variables can be set by another app and simply call runserver() method after that which would launch the Tomcat server and its WebDAV servlet as appropriate.

I'm taking steps to deploy this to maven central, not familiar with the steps, hope I'd get there soon.

[ag88]

ok i did it, it is now in maven central
https://central.sonatype.com/artifact/io.github.ag88/embtomcatwebdav/0.4.1
updated README.md
https://github.com/ag88/embtomcatwebdav
added some notes for embedding and unit tests (examples) well they are actually used for tests on the server itself
the tests used Apache http client/components and Sardine library to connect to the webdav server during the tests.
the test ran ok/well for various scenarios upload/download/rename file/delete file/create directory, directory listing
https://github.com/ag88/embtomcatwebdav#developmentembedding
In the wiki, I added on how one can add one's own servlet in the same app ;)
https://github.com/ag88/embtomcatwebdav/wiki/Development-Embedding

[ag88]

I've made various updates to the webdav server
https://github.com/ag88/embtomcatwebdav
The version is now 0.5.1 which added an Upload Servlet that allows a user to upload files without a WebDAV server.
I think it is a useful add-on. Based on the tests WebDAV continues to work (well) in the original Apache Tomcat's servlet and the new Upload servlet It can be used as like the original manner for embedded apps.

To use the upload servlet in the embedded app, set upload servlet boolean parameter to true before starting up the server, leaving it as false runs the original Apache Tomcat's WebDAV servlet.

[maddingo]

Sorry for the late reply. I started using Testcontainers for integration tests, e.g. in the branch feature/smb. Maybe it is better to use this instead of embedding the library directly.

[maddingo]

This is fixed in #102

[ag88]

hi thanks, I've been drawn out to other commitments.
I'm rather unfamiliar with Testcontainers, I've not actually worked it and It seemed you have used a sftpgo container implementation for the test. That's really cool ;)
Using docker containers is useful for non-java integrations, but that it do add to complexity.