PROPFIND requests work unauthenticated about docker-nginx-webdav-nononsense

[dgraziotin]

Explorer expects at least these two responses and codes when attempting to login:

nginxwebdav  | 172.30.0.1 - - [17/Dec/2021:15:09:33 +0100] "OPTIONS / HTTP/1.1" 200 0 "-" "Microsoft-WebDAV-MiniRedir/10.0.22000"
nginxwebdav  | 172.30.0.1 - - [17/Dec/2021:15:09:33 +0100] "PROPFIND / HTTP/1.1" 207 424 "-" "Microsoft-WebDAV-MiniRedir/10.0.22000"

OPTIONS is already implemented after #9.
PROPFIND might work if limited to / without auth.

from docker-nginx-webdav-nononsense.

[dgraziotin]

Can't find a decent way to allow PROPFIND / to work unauthenticated and actually return what PROPFIND expects to�, while keeping the rest authenticated.

This is a very minor issue and WONTFIX for the time being.

[Akjo4712]

@dgraziotin: Sorry for writing on an old won't fix Issue. I was just trying to fix the Issue, where Explorer doesn't authenticate after a reboot, and, at least for Windows, I think I found a fix. The Issue seems to be fixed, if I enable authentication for all PROPFIND Requests. Why do you/we need to Support unauthenticated PROPFIND at all? Is this just a regression from the Source of the Propfind Handler, or does this have any purpose?

Windows seems to only authenticate after a reboot if the PROPFIND Requests fail when unauthenticated.

    location @propfind_handler {

      auth_basic "Restricted";
      auth_basic_user_file htpasswd;
    
      internal;

      open_file_cache	off;
      
      if (!-e $webdav_root/$uri) { # Microsoft specific handle.
        return 404;
      }
      
      root			$webdav_root;
      dav_ext_methods		PROPFIND;
    }

PROPFIND currently also works unauthenticated for all Subdirectorys, which reveals all Folders and Files. Although you can't open the Files, solely others enumarating through the Files and Folders feels wrong. Tested with curl -i -X PROPFIND http://localhost:32080/A -H "Depth: 1", where A is a Folder containing other Folders and Files. You can see the File test.txt in the Response.

$ curl -i -X PROPFIND http://localhost:32080/C -H "Depth: 1"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   806  100   806    0     0  69095      0 --:--:-- --:--:-- --:--:-- 80600HTTP/1.1 207 Multi-Status
Server: nginx/1.21.4
Date: Thu, 06 Jan 2022 09:15:38 GMT
Content-Type: text/xml; charset=utf-8
Content-Length: 806
Connection: keep-alive

<?xml version="1.0" encoding="utf-8" ?>
<D:multistatus xmlns:D="DAV:">
<D:response>
<D:href>/C</D:href>
<D:propstat>
<D:prop>
<D:displayname>C</D:displayname>
<D:getlastmodified>Thu, 06 Jan 2022 08:46:31 GMT</D:getlastmodified>
<D:resourcetype><D:collection/></D:resourcetype>
<D:lockdiscovery/>
<D:supportedlock>
</D:supportedlock>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
<D:response>
<D:href>/C/test.txt</D:href>
<D:propstat>
<D:prop>
<D:displayname>test.txt</D:displayname>
<D:getcontentlength>4</D:getcontentlength>
<D:getlastmodified>Thu, 06 Jan 2022 08:46:31 GMT</D:getlastmodified>
<D:resourcetype></D:resourcetype>
<D:lockdiscovery/>
<D:supportedlock>
</D:supportedlock>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>

from docker-nginx-webdav-nononsense.

[dgraziotin]

It's OK, I am very open to discussion and to change things based on them.

First, the password on reboot issue for Windows seems to be recognized around, even by WebDAV storage providers like Koofr. Even them tell you that there is nothing you can do about it other than using another client to connect. They are very likely using Apache for WebDAV, which is more stable than nginx.

My interpretation of RFC4918 section 9.1 and section 9.1.1 is that PROPFIND requests must go through unauthenticated, and the propstat elements in the response must contain appropriate response codes for files and folders.

Following this convention makes nginx and WebDAV work under Windows and macOS Finder (as far as I remember, Finder will complain if PROPFIND requests are blocked).

Now, there is indeed the issue that you are describing. PROPFIND requests against / with Depth: 1 will reveal files and folders. One could repeat this request against all folders and find out the entire hierarchy.

Files are not actually accessible, so the issue is somehow mitigated.

So, we are a bit in deadlock situation. PROPFIND requests should go unauthenticated and single response codes in the response should describe access control.

Section 9.1 of RFC4918 ends with the following paragraph:

Properties may be subject to access control. In the case of
'allprop' and 'propname' requests, if a principal does not have the
right to know whether a particular property exists, then the property
MAY be silently excluded from the response.

Note the MAY. We are not violating the RFC by showing the filesystem structure unauthenticated. It's just concerning and annoying.

If any, it seems to me that there is a bug upstream, at nginx-dav-ext-module that will show <D:status>HTTP/1.1 200 OK</D:status> for all properties even when they are not accessible (this is visible also in your snippets).

It also seems to me that it should be nginx-dav-ext-module to opt to silence properties when unauthenticated. I don't think that I have a way to silence them via nginx.conf.

The workaround would be to block PROPFIND requests when not authenticated, possibly violating RFC4918, and I think I remember that this brings issues to macOS Finder. I can look this up again. If that is true, I have little to do other than opening issues upstream.

TL;DR: your "forgotten password" issue is related to bugs in Windows that seem unrelated to nginx + WebDAV. There is, however, a privacy and security issue that the file system structure is revealed. Going around it might violate the WebDAV RFC, and it might break some clients. I can investigate it. The real fix seems to be made upstream, not at my level.

[dgraziotin]

@Akjo4712 I think I fixed it with yet another workaround. Could you tell me if dgraziotin/nginx-webdav-nononsense:propfind works under Windows? I need three comments:

1. Does everything work as expected, in general?
2. Are you no more able to see the filesystem structure without authentication?
3. Does the password persist after a reboot? <- this one is mostly a curiosity, because I have no fix for it.

[Akjo4712]

Just had some spare minutes to do some testing.

1. Everything is still working as expected with the new Image
2. Requests with Depth: 1 now fail with "401 Authorization Requiered", so I can't Browse through Folders unauthenticated
3. The behaviour in Windows Explorer changed, now I get an Error Message when I try to access the Network Share after reboot.

My assumption is, that Windows connects the Network Drive when I first click on it after reboot. If reconnecting the Network Drive without authnentication is successfull, Windows persists the anonymous credentials for the whole Session with the Network Drive until the next reboot. The first call from Windows when reconnecting a WebDAV Network Drive is with Depth: 0, as this call is successfull without authentication, Statuscode for the response is 207, all subsequent calls are also made without authentication. If the first call with Depth: 0 fails, the stored Credentials are also not sent to the Server, but you get Prompted with a Login Mask which is prefilled with the stored Credentials and you only need to hit okay.

My change to the NGINX Conf described above is now for use for the last 4 Days and working like a charm with Windows.

Although auto reconnect is still not working with my fix, I think it is crucial, that reconnecting a Network Drive is somehow working. Needing to hit okay, or providing Credentials every time I reconnect the Network Drive is the acceptable, but getting a Error Message and needing to remove the Network Drive and reconnect completely is too complicated for end users.

Is there any way, that we requiere all PROPFIND requests from a User Agent associated with the Windows Explorer WebDAV implementation, for instance "Microsoft-WebDAV-MiniRedir/10.0.22000", to include authentication, and otherwise fail with a "401 Authorization Requiered"?

[dgraziotin]

Thanks for checking. In the dgraziotin/nginx-webdav-nononsense:propfind image, I am checking if there is a "Depth" header.

If there is no Depth header, this is the function called with a PROPFIND request:

location @propfind_handler {
    
      internal;

      open_file_cache	off;
      
      
      if (!-e $webdav_root/$uri) { # Microsoft specific handle.
        return 404;
      }
      
      root			$webdav_root;
      dav_ext_methods		PROPFIND;
    }

If there is a Depth header, this is the one used instead

    location @propfind_withdepth_handler { # Same as above but authenticated.
    
      auth_basic "Restricted";
      auth_basic_user_file htpasswd;
    
      internal;

      open_file_cache	off;
      
      
      if (!-e $webdav_root/$uri) { # Microsoft specific handle.
        return 404;
      }
      
      root			$webdav_root;
      dav_ext_methods		PROPFIND;
    }

Given that the logic works well but issues happen at reboot time, I will try keeping this approach and see what Windows does when rebooting. Perhaps I can go around it, like a different HTTP return code or a user agent like you suggest. I will comment here again in case I have something else to try.

from docker-nginx-webdav-nononsense.

[Akjo4712]

Hi! The proposed Solution based on the presence of the depth Header is definitly fixing the Security concerns. I would also try to keep your Solution and add yet another workaround for the Windows Reboot Authentication Issue.

[dgraziotin]

I can replicate this:

The behaviour in Windows Explorer changed, now I get an Error Message when I try to access the Network Share after reboot.

This is no good, I agree.

And now we have #20 that might be related to this (awaiting further details on which codebase/image they are using)

[Bmiest]

@dgraziotin can you open a branch with PROPFIND changes so I can build and test this one with my implementation?

[dgraziotin]

@Bmiest here is the nginx.conf I am using for the image. I will branch it and release it for sure, but I have another solution in mind. No time to implement it yet, hopefully by this week.

daemon off;
user abc abc;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
  worker_connections  1024;
}

http {
  include    mime.types;
  dav_ext_lock_zone zone=foo:10m;
  
  send_timeout 3600;
  client_body_timeout 3600;
  keepalive_timeout 3600;
  lingering_timeout 3600;
  client_max_body_size 10G;
  
  create_full_put_path	on;
	min_delete_depth	0;
  
  server {
    server_name localhost;
    listen 80;
    
    set $destination $http_destination; 
    set $new_path "";
    set $webdav_root "/data";
    set $checkPropfind "";
    
    location ~ \.(_.*|DS_Store|Spotlight-V100|TemporaryItems|Trashes|hidden|localized)$ {
      access_log  off;
      error_log   off;
      
      if ($request_method = PUT) {
        return 403;
      }
      
      return 404;
    }

    location ~ \.metadata_never_index$ {
      return 200 "Don't index this drive, Finder!";
    }
    
    # With fixes for Windows from http://netlab.dhis.org/wiki/ru:software:nginx:webdav
    location / {
     
      auth_basic "Restricted";
      auth_basic_user_file htpasswd;
      
      dav_ext_lock zone=foo;
      dav_methods		PUT MKCOL;
      dav_ext_methods	OPTIONS LOCK UNLOCK;

      autoindex		on;
      autoindex_exact_size	on;
      autoindex_localtime	on;
      
      fancyindex on;
      fancyindex_exact_size off;
      fancyindex_localtime on;
      fancyindex_name_length 255;
      
      root			$webdav_root;
      
      error_page		599 = @propfind_handler;
      error_page		598 = @delete_handler;
      error_page		597 = @copy_move_handler;
      error_page		596 = @propfind_withdepth_handler;
      
      open_file_cache		off;
      
      if ($request_method != OPTIONS) {
        # CORS
        add_header 'Access-Control-Allow-Origin' '*' always;
        add_header 'Access-Control-Allow-Credentials' 'true' always;
        add_header 'Access-Control-Allow-Methods' 'OPTIONS, GET, HEAD, POST, PUT, MKCOL, MOVE, COPY, DELETE, PROPFIND, PROPPATCH, LOCK, UNLOCK' always;
        add_header 'Access-Control-Allow-Headers' 'Authorization,DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Accept-Charset,X-Accept,origin,accept,if-match,destination,overwrite' always;
        add_header 'Access-Control-Expose-Headers' 'ETag' always;
        add_header 'Access-Control-Max-Age' 1728000 always;
      }
      
      if ($request_method = OPTIONS) {
        # CORS
        add_header 'Content-Type' 'text/plain charset=UTF-8';
        add_header 'Access-Control-Allow-Origin' '*';
        add_header 'Access-Control-Allow-Credentials' 'true';
        add_header 'Access-Control-Allow-Methods' 'OPTIONS, GET, HEAD, POST, PUT, MKCOL, MOVE, COPY, DELETE, PROPFIND, PROPPATCH, LOCK, UNLOCK';
        add_header 'Access-Control-Allow-Headers' 'Authorization,DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Accept-Charset,X-Accept,origin,accept,if-match,destination,overwrite';
        add_header 'Access-Control-Expose-Headers' 'ETag';
        add_header 'Access-Control-Max-Age' 1728000;
        
        # WebDAV compliance
        add_header	Allow 'OPTIONS, GET, HEAD, POST, PUT, MKCOL, MOVE, COPY, DELETE, PROPFIND, PROPPATCH, LOCK, UNLOCK';
        add_header	DAV '1, 2';
        return 200;
      }
      
      
      if ($request_method = PROPFIND) { # Normal flow for PROPFIND
        set $checkPropfind "propfind";
      }
      
      if ($http_depth) { # FIX for Depth = 1 allowed unauthenticated
        set $checkPropfind "${checkPropfind}+withDepth";
      }
      
      if ($checkPropfind = "propfind") { # Normal flow for PROPFIND
        return 599;
      } 
      
      if ($checkPropfind = "propfind+withDepth") { # FIX for Depth = 1 allowed unauthenticated
        return 596;
      }   
      
      if ($request_method = PROPPATCH) { # Unsupported, allways return OK.
        add_header	Content-Type 'text/xml';
        return		207 '<?xml version="1.0"?><a:multistatus xmlns:a="DAV:"><a:response><a:propstat><a:status>HTTP/1.1 200 OK</a:status></a:propstat></a:response></a:multistatus>';
      }
      
      if ($request_method = MKCOL) { # Microsoft specific handle: add trailing slash.
        rewrite ^(.*[^/])$ $1/ break;
      }
      
      if ($request_method = DELETE) {
        return 598;
      }
      
      if ($request_method = COPY) {
        return 597;
      }
      
      if ($request_method = MOVE) {
        return 597;
      }
      
      
    }
    
    location @propfind_handler {
    
      internal;

      open_file_cache	off;
      
      
      if (!-e $webdav_root/$uri) { # Microsoft specific handle.
        return 404;
      }
      
      root			$webdav_root;
      dav_ext_methods		PROPFIND;
    }
    
    location @propfind_withdepth_handler { # Same as above but authenticated.
    
      auth_basic "Restricted";
      auth_basic_user_file htpasswd;
    
      internal;

      open_file_cache	off;
      
      
      if (!-e $webdav_root/$uri) { # Microsoft specific handle.
        return 404;
      }
      
      root			$webdav_root;
      dav_ext_methods		PROPFIND;
    }
    
    location @delete_handler {
    
      auth_basic "Restricted";
      auth_basic_user_file htpasswd;
      
      internal;
      open_file_cache	off;
      
      if ($destination ~ ^https?://(.*)$) {
        set $new_path $1;
        more_set_input_headers "Destination: http://$new_path";
      }
      
      if (-d $webdav_root/$uri) { # Microsoft specific handle: Add trailing slash to dirs.
        more_set_input_headers "Destination: http://$new_path/";
        rewrite ^(.*[^/])$ $1/ break;
      }
      
      root			$webdav_root;
      dav_methods		DELETE;
    }

    location @copy_move_handler {
    
      auth_basic "Restricted";
      auth_basic_user_file htpasswd;
    
      internal;
      open_file_cache	off;
      
      if ($destination ~ ^https?://(.*)$) {
        set $new_path $1;
        more_set_input_headers "Destination: http://$new_path";
      }
      
      if (-d $webdav_root/$uri) { # Microsoft specific handle: Add trailing slash to dirs.
        more_set_input_headers "Destination: http://$new_path/";
        rewrite ^(.*[^/])$ $1/ break;
      }
      
      root			$webdav_root;
      dav_methods		COPY MOVE;
    }
  }
}

from docker-nginx-webdav-nononsense.

[dgraziotin]

@Akjo4712 the codebase in the propfind branch seems to work for me (no privacy issues, windows asks for user+password on reboot). The dgraziotin/nginx-webdav-nononsense:propfind image is also updated with these changes.

[Akjo4712]

@dgraziotin: Just did some testing with dgraziotin/nginx-webdav-nononsense:propfind, in general it works like a charm. It definitly already is a big improvement for Windows Users! After Reboot, when I click on the Network Drive in Explorer, I am prompted with a Authentication Window where I just need to press okay, as the Username and Password is already saved.

Unfortunately it still is possible, that the first PROPFIND call is NOT targeting /. You can easily replicate this if you follow the Steps Listed below. This shouldn't be a Problem for normal endusers if they don't enter Paths into Explorer directly, but navigate with point and click, as you need to click on the root Folder to connect the Network Drive. The only way they could be affected by this Issue is, when the WebClient crashes and you need to reauthenticate while moving inside the Network Drive,

When I try to Access the Network Drive through CLI, the Drive Letter is unrecognized until I connect the Network Drive corretly, so this only seems to be affecting Explorer. All other Programms will fail to access Data until you connect the Drive either via a Command or a Click in Explorer.

Steps to Reproduce

1. Mount WebDAV as Network Drive
2. Access some Files, create Folders, move things around --> Everything works fine
3. Keep Explorer open and stay in any Folder served through WebDAV
4. Restart WebClient Service (Click on the Windows Symbol and type "Services"
5. Click on any Folder in the Explorer Window which was left open in Step 3
6. You are prompted with a Error Messages

The Problem is, that the Drive is mounted through a Propfind Request to the Subfolder, which currently doesn't requiere Authentication and therefore the Mount Process failes.

Clicking on a File in the Explorer Window which you left open in Step 3 seems to be unsupported at all, as Windows fails even before Issuing a WebDAV Request to the Image. So my assumption is, opening any Folder in the Network Drive mounts it. Clicking on a File in a not mounted Network Drive is unsupported at all.

The only thing we can do now is to requiere all PROPFIND Requests, at least from Windows, to be authenticated, this would eliminate the Probleme I just found, or we are at a Point where we stop building Workarouds for less common Problems in order to keep the implementation clean and straight forward.

[dgraziotin]

@Akjo4712 just blocking all PROPFIND requests does not work for me, unfortunately. Explorer complains when adding the network share.

This in the logs:

Before entering username/password

nginxwebdav  | 172.18.0.1 - - [12/Jan/2022:14:56:28 +0100] "PROPFIND / HTTP/1.1" 401 179 "-" "Microsoft-WebDAV-MiniRedir/10.0.22000"
nginxwebdav  | 172.18.0.1 - - [12/Jan/2022:14:56:41 +0100] "OPTIONS / HTTP/1.1" 200 0 "-" "Microsoft-WebDAV-MiniRedir/10.0.22000"
nginxwebdav  | 172.18.0.1 - - [12/Jan/2022:14:56:41 +0100] "PROPFIND / HTTP/1.1" 401 179 "-" "Microsoft-WebDAV-MiniRedir/10.0.22000"

After entering username/password
nginxwebdav  | 172.18.0.1 - - [12/Jan/2022:14:56:50 +0100] "PROPFIND / HTTP/1.1" 401 179 "-" "Microsoft-WebDAV-MiniRedir/10.0.22000"

So it's either unauthenticated like it was before opening this issue, or the direction I was working on, with the issues you have just reported...

from docker-nginx-webdav-nononsense.

[Akjo4712]

Seems to behave different on my end, I am still running the nginx.conf with the propfind_handler below in Production and I can connect the Network Drive without any Problems.

I would suggest the following:

• You merge the Version you provided for testing 2 days ago into Master
• You Push a new Image to Dockerhub
• We observe if everything works as expected

As long as the WebClient Service on Windows doesn't crash, there is currently no szenario I can think of where the Edge Case stated by me above would be triggered.

The current State is already a big improvement for all windows users. I would really like to do some long term testing before we invest more time on this one.

By the way we are also Off Topic in Case of this Issue by now. The Issue with unauthenticated PROPFIND was solved, what we are working on now is just another Workaround for Windows and not respecting the RFC.

So I would suggest to merge into Master, Push a new Image and Close this Ticket.

tl;dr: Merge, Push and Observe the Version from 2 days ago. What is your opinion?

propfind_handler

    location @propfind_handler {

      auth_basic "Restricted";
      auth_basic_user_file htpasswd;
    
      internal;

      open_file_cache	off;
      
      if (!-e $webdav_root/$uri) { # Microsoft specific handle.
        return 404;
      }
      
      root			$webdav_root;
      dav_ext_methods		PROPFIND;
    }

from docker-nginx-webdav-nononsense.

[dgraziotin]

@Akjo4712 it's fine, so long as the flow of issues does not grow, we can keep this as a discussion. Also, it is all somehow related to the initial issue. I gave it my best today, developed a very elaborated system to spoof responses to unath PROPFIND when Depth = 1 and auth is in place. Explorer is complaining anyway, and I can't reverse engineer all of it. I give up and will merge this branch into master, push images, and update the latest tag. Thank you SO much for your help!