Coder Social home page Coder Social logo

davrods's Introduction

Davrods - An Apache WebDAV interface to iRODS

Davrods provides access to iRODS servers using the WebDAV protocol. It is a bridge between the WebDAV protocol and the iRODS API, implemented as an Apache HTTPD module.

Davrods leverages the Apache server implementation of the WebDAV protocol, mod_dav, for compliance with the WebDAV Class 2 standard.

Notable features include:

  • WebDAV Class 2 support, with locks local to the Apache server
  • Connects to iRODS server versions 4+
  • PAM and Native (a.k.a. STANDARD) iRODS authentication
  • Optional negotiated SSL encryption for the entire iRODS connection
  • Configurable using Apache configuration directives
  • Optional anonymous access mode for password-less public access
  • Themeable directory listings for read-only web browser access.
  • Partial file up- and downloads and resumes (HTTP byte-ranges)
  • iRODS ticket-based access

Themeable listings and anonymous access were inspired by Simon Tyrrell's work at Earlham Institute.

Download

Please choose the right version for your platform:

  1. If you run Davrods on the same server as your iRODS service, you need a Davrods version built against the same version iRODS runtime.
  2. If you run Davrods separately, on its own server, then the iRODS runtime version does not matter - just pick the newest Davrods you can get. All Davrods packages below should be compatible with any iRODS 4.x server version.
Date Davrods ver. iRODS runtime ver. Packages Remarks
2022-06-22 1.5.0 4.3.0 RPM, DEB Pre-release
2023-07-27 1.5.1 4.2.12 RPM, DEB Pre-release
2021-12-20 1.5.0 4.2.11 RPM, DEB
2021-07-28 1.5.0 4.2.10 RPM, DEB
2021-06-16 1.5.0 4.2.9 RPM, DEB
2020-06-02 1.5.0 4.2.8 RPM, DEB
2019-12-20 1.4.2 4.2.7 RPM, DEB
2019-06-20 1.4.2 4.2.6 RPM, DEB
2019-04-03 1.4.2 4.2.5 RPM, DEB
2018-12-17 1.4.2 4.2.4 RPM, DEB
2018-07-15 1.4.1 4.2.3 RPM, DEB
2018-07-15 1.4.1 4.1.x RPM

(older versions are still available on the releases page)

If you require a certain Davrods/iRODS runtime version combination that is not listed above, you can most likely still build it yourself (see "Building from source").

A log describing which features were added and which bugs were fixed in each version can be found in changelog.txt.

We currently distribute RPM packages for CentOS 7 & RHEL systems and DEB packages for Debian & Ubuntu systems. We test our packages on CentOS 7 and (as of Davrods 1.4.0) Ubuntu 16.04.

Installation

This section describes the installation steps for iRODS 4.2+ based Davrods releases.

To view instructions for iRODS 4.1-based Davrods releases, switch to the irods-4.1-libs branch.

Using the binary distribution

Davrods depends on certain iRODS packages, which as of iRODS 4.2 are distributed at https://packages.irods.org/

After following the instructions for adding the iRODS repository to your package manager at the link above, Davrods can be installed as a binary package using the RPM or DEB file from the releases page (use the table near the top of this README to select the right version).

Download the Davrods package for your platform and install it using your package manager, for example:

yum install davrods-4.2.8_1.5.0-1.rpm
--or--
apt install davrods-4.2.8_1.5.0.deb

Now see the Configuration section for instructions on how to configure Davrods once it has been installed.

Davrods and SELinux

If the machine on which you install Davrods is protected by SELinux, you may need to make changes to your policies to allow Davrods to run:

  • Apache HTTPD must be allowed to connect to TCP port 1247

For example, the following command can be used to resolve this requirement:

setsebool -P httpd_can_network_connect true

Configuration

Davrods is configured in two locations: In a HTTPD vhost configuration file and in an iRODS environment file. The vhost config is the main configuration file, the iRODS environment file is used for iRODS client library configuration, similar to the configuration of icommands.

HTTPD vhost configuration

The Davrods RPM distribution installs two vhost template files:

  1. /etc/httpd/conf.d/davrods-vhost.conf
  2. /etc/httpd/conf.d/davrods-anonymous-vhost.conf

(for Ubuntu, replace /etc/httpd/conf.d with /etc/apache2/sites-available)

These files are provided completely commented out. To enable either configuration, simply remove the first column of # signs, and then tune the settings to your needs.

Note that on Ubuntu, you will additionally need to enable the Davrods module and vhosts, like so:

a2enmod dav
a2enmod davrods
a2ensite davrods-vhost
a2ensite davrods-anonymous-vhost

The normal vhost configuration (1) provides sane defaults for authenticated access.

The anonymous vhost configuration (2) allows password-less public access using the anonymous iRODS account.

You can enable both configurations simultaneously, as long as their ServerName values are unique (for example, you might use dav.example.com for authenticated access and public.dav.example.com for anonymous access).

The iRODS environment file

The binary distribution installs the irods_environment.json file in /etc/httpd/irods. In most iRODS setups, this file can be used as is.

Importantly, options such as irods_host and irods_zone_name are not read from this file (and are omitted for that reason). These settings are taken from their equivalent Davrods configuration directives in the vhost config file instead.

Options in irods_environment.json that are known to affect Davrods behavior are the negotiation, ssl and encryption settings.

See the official documentation for more information on these settings: https://docs.irods.org/4.2.7/system_overview/configuration/#irodsirods_environmentjson

Ticket-based access

Ticket access is disabled by default, and requires special configuration to enable, depending on your use case. Please see README.advanced.md for more information.

Building from source

This repository includes a Vagrant configuration for building Davrods from source on either CentOS 7 (for the RPM package) or Ubuntu 18.04 LTS (for the DEB package). It can be found in vagrant/build. In order to build a package using Vagrant, edit the .env file in the Vagrant build directory. Adjust the BOXNAME and IRODS_VERSION vars as needed. Then run vagrant up to provision the VM. The VM has all dependencies pre-installed, as well as a clone of the Davrods repository. Log in on the VM using vagrant ssh, then generate the build directory and create the package (see below).

To build from source without using the Vagrant configuration, the following build-time dependencies must be installed (package names may differ on your platform):

  • cmake
  • make
  • gcc
  • httpd-devel >= 2.4
  • irods-devel >= 4.2.0
  • openssl-devel
  • irods-runtime >= 4.2.0
  • rpmdevtools (if you are creating an RPM)

Additionally, the following runtime dependencies must be installed:

  • irods-runtime >= 4.2.0
  • openssl-libs
  • httpd >= 2.4

Follow these instructions to build from source:

  • First, browse to the directory where you have unpacked the Davrods source distribution.

  • Check whether your umask is set to a sane value. If the output of umask is not 0022, run umask 0022 to fix it. This is important for avoiding conflicts in created packages later on.

  • Create and generate a build directory.

mkdir build
cd build
cmake ..
  • Compile the project
make

Now you can either build an RPM/DEB or install the project without a package manager.

To create a package:

make package

That's it, you should now have an RPM or DEB in your build directory which you can install using yum or apt.

To install without a package manager on CentOS:

Run the following as user root:

make install
chown apache:apache /var/lib/davrods
chmod 700 /var/lib/davrods

To install without a package manager on Debian:

Run the following as user root:

make install
chown www-data:www-data /var/lib/davrods
chmod 700 /var/lib/davrods

To install without a package manager on other distros:

Linux distributions other than RHEL, Debian and their derivatives may have different HTTPD configuration and directory layouts, which are not currently supported by the build system. For this reason you will need to install the files manually on such Linux distributions:

  • Copy mod_davrods.so to your Apache module directory.
  • Copy davrods.conf to your Apache module configuration/load directory.
  • Copy davrods-vhost.conf and davrods-anonymous-vhost.conf to your Apache vhost configuration directory.
  • Create an irods directory in a location where Apache HTTPD has read access.
  • Copy irods_environment.json to the irods directory.
  • Create directory /var/lib/davrods, and give apache exclusive access to it.

Bugs and feature requests

Please report any issues you encounter on the issues page.

Contact information

For questions or support, please contact the Yoda team at [email protected].

Authors

License

Copyright (c) 2016 - 2021, Utrecht University.

Davrods is licensed under the GNU Lesser General Public License version 3 or higher (LGPLv3+). See the COPYING.LESSER file for details.

The lock_local.c and byterange.c files were adapted from components of Apache HTTPD, and are used with permission granted by the Apache License. See the copyright and license notices in these files for details.

davrods's People

Contributors

cjsmeele avatar lwesterhof avatar stsnel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

davrods's Issues

Does DavRODS support quota requests?

Hello!

Thanks for your development of the davrods application! We've been using it for quite a while now in the Maastricht University and are very happy with it.
Today we got a request from a user to upload large quantities of data through webdav. He cannot do it, because the mounted share in Linux shows that there is only 509G available, when the storage actually still has 14T available.

After a little digging, we found out that the value that Linux shows is just static, and doesn't change based on the value of the underlying resource and unixfilesystem

Is there a way for us to have Linux show the actual size of the unixfilesystem resource, or will it always return the values below?

http://upload.local.dh.unimaas.nl/ 1333333332 800000000 533333332  61% /tmp/test

A Mac will show the following values and will just allow the user to upload as much as he wants:

http://upload.local.dh.unimaas.nl/    0Bi    0Bi    0Bi   100%       0          0  100%   /Volumes/upload.local.dh.unimaas.nl

Thanks in advance!

Debian/Ubuntu package

Hi, I'd be very interested in a Debian/Ubuntu package.

I could compile and make install davrods on Ubuntu 16.04 using the following commands (using the Ubuntu 14.04 iRODS packages, though).

I could not make the package building work for some reason, though.

Would you be interested in also providing packages for .deb-based systems?

# Register irods packages
if [[ ! -f /etc/apt/sources.list.d/renci-irods.list ]]; then
    release=trusty  # no good irods packages for xenial yet
    wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
    echo "deb [arch=amd64] https://packages.irods.org/apt/ $release main" | sudo tee /etc/apt/sources.list.d/renci-irods.list
    sudo apt-get update
fi

# Install dependencies
apt install cmake make gcc apache2-dev irods-dev irods-dev irods-runtime

# Download davrods
mkdir -p ~/davrods
cd ~/davrods
test -f 4.2.1_1.1.1.tar.gz || wget https://github.com/UtrechtUniversity/davrods/archive/4.2.1_1.1.1.tar.gz
tar xzf 4.2.1_1.1.1.tar.gz
cd davrods-4.2.1_1.1.1/
patch -p0 <davrods.patch
mkdir build
cd build
cmake ..
make
make install

Below is the patch I used

--- CMakeLists.txt	2017-08-14 18:18:39.972615385 +0200
+++ CMakeLists.txt	2017-08-14 19:49:48.549653272 +0200
@@ -28,11 +28,22 @@
 
 if(IS_DIRECTORY /etc/httpd/conf.modules.d)
     # This looks like CentOS7's httpd, we know where to put our files on install.
-    set(INSTALLABLE_ON_THIS_SYSTEM TRUE)
+    set(SYSTEM_LOOKS_LIKE "CentOS7")
+    set(SO_PATH "modules")
+elseif(IS_DIRECTORY /etc/apache2/mods-available)
+    # This looks like Debian/Ubuntu, we know where to put our files on install.
+    set(SYSTEM_LOOKS_LIKE "Debian")
+    set(SO_PATH "/usr/lib/apache2/modules/")
+
+# Debian notes, postinstall:
+#
+# a2enmod davrods
+# a2endmod dav
+# apache2ctl restart
 else()
-    set(INSTALLABLE_ON_THIS_SYSTEM FALSE)
+    set(SYSTEM_LOOKS_LIKE "--Unknown--")
     message(WARNING "
-Davrods' build system currently only supports the cmake 'install' target on CentOS7-like systems.
+Davrods' build system currently only supports the cmake 'install' target on CentOS7 and Debian-like systems.
 If you are running CentOS or similar, make sure httpd is installed before running cmake: This build system requires certain HTTPD directories to be in place.
 If you are running a different Linux distribution or if your HTTPD configuration layout differs otherwise, you can install Davrods manually after building. See the instructions in README.md.")
 endif()
@@ -76,14 +87,17 @@
 
 add_library(mod_davrods SHARED ${SOURCES})
 
+configure_file(davrods.conf.in davrods.conf @ONLY)
+
 # Remove "lib" prefix from module SO file.
 set_property(TARGET mod_davrods PROPERTY PREFIX "")
 
-if(INSTALLABLE_ON_THIS_SYSTEM)
+# Enable OS-dependent installation targets
+if(SYSTEM_LOOKS_LIKE STREQUAL "CentOS7")
     install(TARGETS     mod_davrods
             DESTINATION ${HTTPD_BUILDSYS_MODULE_DIR})
 
-    install(FILES       davrods.conf
+    install(FILES       ${CMAKE_BINARY_DIR}/davrods.conf
             DESTINATION /etc/httpd/conf.modules.d
             RENAME      10-davrods.conf)
 
@@ -98,9 +112,28 @@
 
     install(DIRECTORY
             DESTINATION /var/lib/davrods)
+elseif(SYSTEM_LOOKS_LIKE STREQUAL "Debian")
+    install(TARGETS     mod_davrods
+            DESTINATION ${HTTPD_BUILDSYS_MODULE_DIR})
+
+    install(FILES       ${CMAKE_BINARY_DIR}/davrods.conf
+            DESTINATION /etc/apache2/mods-available
+            RENAME      davrods.load)
+
+    install(FILES       davrods-vhost.conf
+            DESTINATION /etc/apache2/conf-available/)
+
+    install(FILES       irods_environment.json
+            DESTINATION /etc/apache2/irods/)
+
+    install(FILES       README.md COPYING COPYING.LESSER
+            DESTINATION /usr/share/doc/davrods-${DAVRODS_VERSION}/)
+
+    install(DIRECTORY
+            DESTINATION /var/lib/davrods)
 endif()
 
-if(INSTALLABLE_ON_THIS_SYSTEM)
+if(SYSTEM_LOOKS_LIKE EQUAL "CentOS7")
     set(CPACK_MONOLITHIC_INSTALL 1)
     set(CPACK_CMAKE_GENERATOR "Unix Makefiles")
     set(CPACK_GENERATOR "RPM")
@@ -137,5 +170,43 @@
 
     set(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_NAME}-${CPACK_PACKAGE_VERSION}-${CPACK_RPM_PACKAGE_RELEASE}")
 
+    include(CPack)
+elseif(SYSTEM_LOOKS_LIKE EQUAL "Debian")
+    set(CPACK_MONOLITHIC_INSTALL 1)
+    set(CPACK_CMAKE_GENERATOR "Unix Makefiles")
+    set(CPACK_GENERATOR "DEB")
+    set(CPACK_PACKAGE_NAME "davrods")
+    set(CPACK_PACKAGE_VENDOR "Utrecht University <[email protected]>")
+    set(CPACK_PACKAGE_CONTACT "Utrecht University <[email protected]>")
+    set(CPACK_PACKAGE_VERSION "${DAVRODS_VERSION}")
+    #set(CPACK_PACKAGE_VERSION_MAJOR "4")
+    #set(CPACK_PACKAGE_VERSION_MINOR "2")
+    #set(CPACK_PACKAGE_VERSION_PATCH "1")
+    set(CPACK_PACKAGE_DESCRIPTION_FILE "${CMAKE_CURRENT_SOURCE_DIR}/package/description.txt")
+    set(CPACK_PACKAGE_DESCRIPTION_SUMMARY "A DAV level 2 compliant Apache interface to iRODS")
+
+    set(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_CURRENT_SOURCE_DIR}/COPYING.LESSER")
+    set(CPACK_RESOURCE_FILE_README  "${CMAKE_CURRENT_SOURCE_DIR}/README.md")
+
+    set(CPACK_RPM_PACKAGE_RELEASE "1")
+    set(CPACK_RPM_PACKAGE_LICENSE "LGPLv3+")
+    set(CPACK_RPM_PACKAGE_REQUIRES "apache2-server >= 2.4, irods-runtime = ${IRODS_VERSION}")
+    set(CPACK_RPM_PACKAGE_URL "https://github.com/UtrechtUniversity/davrods")
+    set(CPACK_RPM_CHANGELOG_FILE "${CMAKE_CURRENT_SOURCE_DIR}/package/changelog.txt")
+    set(CPACK_RPM_PACKAGE_AUTOREQ 0)
+    set(CPACK_RPM_PACKAGE_AUTOPROV 0)
+    set(CPACK_RPM_POST_INSTALL_SCRIPT_FILE "${CMAKE_CURRENT_SOURCE_DIR}/package/postinst.sh")
+
+#    set(CPACK_RPM_USER_FILELIST
+#        "%doc /usr/share/doc/davrods-${DAVRODS_VERSION}/README.md"
+#        "%doc /usr/share/doc/davrods-${DAVRODS_VERSION}/COPYING"
+#        "%doc /usr/share/doc/davrods-${DAVRODS_VERSION}/COPYING.LESSER"
+#        "%config(noreplace) /etc/apache2/mods-available/10-davrods.conf"
+#        "%config(noreplace) /etc/apache2/conf-available/davrods-vhost.conf"
+#        "%config(noreplace) /etc/apache2/irods/irods_environment.json"
+#        "%attr(700,apache,apache) /var/lib/davrods")
+
+    set(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_NAME}-${CPACK_PACKAGE_VERSION}-${CPACK_RPM_PACKAGE_RELEASE}")
+
     include(CPack)
 endif()

Allow access via iRODS tickets

It would be very useful to allow access via iRODS tickets in one way or another. The ticket could be part of the login link (problematic for direct login as modern browsers warn if you get a http://user:password@host link as this is commonly used for phishing).

Another way would require another sort of authentication, which probably implies using a different (sub) domain as WebDAV has to be at the root URL for many client implementation. The ticket could also just be a query parameter as is the (infamous) session ID in early PHP/Java/... session implementations.

I have the following use case in mind.

  • I need to make some files publically available via HTTPS folder/sub tree to a local UCSC genome browser instance independent of a login so it can be shared.
  • Instead, you can add track hubs in the form of a URL.
  • I would like to write a script that creates a collection in iRODS and put a track hub there.
  • This track hub should then be shared for a certain time and only be visible to the UCSC browser server that should access (iRODS tickets are perfect for this).
  • The user can then add this URL to his UCSC session and UCSC browser displays the track hub's content!

E.g., I would call my custom script "publish-ucsc-track.sh local/path". The script uploads things below a predefined location on my iRODS server. It then creates a ticket and dumps a URL to the new collection in DavRods, e.g., https://my-davrods.example.com/track-hubs/random-id?ticket=ticket-value.

I tried to be terse here but I don't know if this request makes sense already. Let me know where this needs more work to be more clear (even if you don't consider this for immediate implementation).

Cannot view or download files after delete files on the cache folder

Hi everyone, I have problems viewing or downloading files using webdav interface after I've deleted the associated files on the cache folder.

Is there any way con configure davrods to access directly to the files store instead of the cache folder?

I don't know if this is a bug or a implemented way of use of the WebDAV interface, but in my case, I need that the files showed in the WebDAV interface will be pulled directly from the archive repository instead of the cache repo. Our archive repository is growing faster and the cache disk is too small to have a copy of ALL the data stored.

PS: I'm using a compound resource in which the cache is stored in a local disk and the archive folder is stored in an Amazon S3 bucket... (the iRODS resource server disk is a virtual machine too, so the disk used in the machine is a virtual disk.) ...double size double cost....

Thank you for your help

Alejandro Lebrero

DavRODS 4.2.9_1.5.0 .deb

Hello All and thank you for the DavRODS package!

I see 4.2.9_1.5.0 is in a "pre-release" state. But I also see there is an .rpm available already.

I'm doing an Ubuntu 18 install and was planning on using your .deb. Will that be available soon or should I work-around?

Thanks,
Blake

Davdrods 4.1. and remote zones

In general when you are working with federated iRODS instances, your users would need read access to the root (/), zone (/ and other zones) and home (//home) collections to list all possible federated zones they might have access to. This does not work with Davrods. When changing the view to

DavRodsExposedRoot  /

users will just see an empty folder as entry point in their webdav client.

SSL_HANDSHAKE_ERROR when trying to connect to irods server

Hey,

I'm currently evaluating davrods and I face the issue, that davrods cannot connect to the irods server. My setup consists of two container (one for irods, one for davrods) in the same network.

When I run icommands inside davrods I can connect to the irods server without any problem. When I use the same settings/credentials with davrods and the provided UI at port 80 (localhost:8888 is mapped to port 80 inside davrods), I receive an internal server error.

Based on the log messages I think the issue starts here. I'm not very familiar with C but as far as I understood the whole (optional) SSL handling happens after the call to rcConnect.

Setup:

  • Version: 4.2.10

Do you have any hints on how to fix this issue? Thanks!

Logs:

[Wed Feb 16 10:23:53.674751 2022] [davrods:debug] [pid 568:tid 140737085699840] auth.c(406): [client 172.19.0.1:44882] Authenticating iRODS username 'rods' using Native auth scheme., referer: http://localhost:8888/
[Wed Feb 16 10:23:53.674761 2022] [davrods:debug] [pid 568:tid 140737085699840] auth.c(128): [client 172.19.0.1:44882] Connecting to iRODS using address <irods:1247>, username <rods> and zone <zone_1>, referer: http://localhost:8888/
[Wed Feb 16 10:23:53.674774 2022] [davrods:debug] [pid 568:tid 140737085699840] auth.c(137): [client 172.19.0.1:44882] Using iRODS env file at </etc/apache2/irods/irods_environment.json>, referer: http://localhost:8888/
remote addresses: 172.19.0.1 ERROR: [-]	/irods/lib/core/src/sockComm.cpp:811:int connectToRhost(rcComm_t *, int, int) :  status [SSL_HANDSHAKE_ERROR]  errno [] -- message [failed to call 'client start']
	[-]	/irods/lib/core/src/sockComm.cpp:64:irods::error sockClientStart(irods::network_object_ptr, rodsEnv *) :  status [SSL_HANDSHAKE_ERROR]  errno [] -- message [failed to call 'client start']
		[-]	/irods/plugins/network/ssl/libssl.cpp:691:irods::error ssl_client_start(irods::plugin_context &, rodsEnv *) :  status [SSL_HANDSHAKE_ERROR]  errno [] -- message [error in SSL_connect]

remote addresses: 172.19.0.1 ERROR: _rcConnect: connectToRhost error, server on irods:1247 is probably down status = -2103000 SSL_HANDSHAKE_ERROR
[Wed Feb 16 10:23:53.777908 2022] [davrods:error] [pid 568:tid 140737085699840] [client 172.19.0.1:44882] Could not connect to iRODS using address <irods:1247>, username <rods> and zone <zone_1>. iRODS says: '_rcConnect: connectToRhost failed\n', referer: http://localhost:8888/

PAM auth not working since irods upgrade to 4.3.X

I upgraded my irods servers to 4.3.1 which are configured with PAM authentication. And then could no more authenticate with webdav.
I upgraded my docker-davrods container to 4.3.0_1.5.0 but no success.

When trying to authenticate, the log of the container reports:

[Wed Mar 27 08:20:22.455920 2024] [davrods:warn] [pid 17] [client 172.17.0.1:45460] rcPamAuthRequest failed: -994000 = PAM_AUTH_PASSWORD_INVALID_TTL
[Wed Mar 27 08:20:22.463480 2024] [auth_basic:error] [pid 17] [client 172.17.0.1:45460] AH01617: user bzizou: authentication failure for "/davrods": Password Mismatch
172.17.0.1 - bzizou [27/Mar/2024:08:20:22 +0000] "GET /davrods HTTP/1.1" 401 381 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"

I tried to change PAM to pam_password into the /etc/httpd/irods/irods_environment.json (as it is a change of 4.2.x -> 4.3.x) with no more success

iRODS 4.x dynamic PEP 'put' not being triggered

Hi,

I'm not sure whether this would be a request for enhancement or a bug report, but in any case I decided to report the following. Due to certain workarounds we had to establish in our environment after iRODS 4.2 upgrade, we had to start using the so called dynamic policy enforcement points, i.e pep_api_data_obj_put_post instead of acPostProcForPut.

For some reason Davrods seems to trigger only acPostProcForPut.

Test scenario using irods-provisioner and spinning up a test cluster.

$ cd /tmp 
$ git clone https://github.com/KTH-PDC/irods-provisioner.git
$ cd irods-provisioner
$ vagrant up
$ ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -b -i hosts-test irods-cluster.yml
$ vagrant ssh rodsmaster1
[vagrant@rodsmaster1 ~]$ sudo su - irods
-bash-4.2$ cat >>/etc/irods/core-override.re 

acPostProcForPut
{
	writeLine("serverLog", "acPostProcForPut: objPath='$objPath'");
}

pep_api_data_obj_put_post(*INSTANCE_NAME, *COMM, *DATAOBJINP, *BUFFER, *PORTAL_OPR_OUT)
{
	*objPath = *DATAOBJINP.obj_path;
   	writeLine("serverLog", "pep_api_data_obj_put_post: objPath = '*objPath'");	
}
^D

At the host one can mount the WebDAV endpoint (macOS commands displayed):

$ mkdir -p /tmp/webdav
$ mount_webdav -i http://rodsfront1:8443/home/rods/ /tmp/webdav
Username: rods
Password:
$ echo blaa > blaa.txt
$ cp blaa.txt /tmp/webdav 

And afterwards we can observe in rodsLog:

Dec 10 10:05:16 pid:12570 NOTICE: writeLine: inString = acPostProcForPut: objPath='/tempZone/home/rods/blaa.txt'
Dec 10 10:05:16 pid:12570 NOTICE: writeLine: inString = acPostProcForPut: objPath='/tempZone/home/rods/blaa.txt'

davrods_merge_dir_config does not set exposed_root

Because compiler remove all 'assert' from release build, assert(set_exposed_root(conf, exposed_root) >= 0); in davrods_merge_dir_config never evaluated and exposed_root always set to "User".
The solution is to move out set_exposed_root from assert.

--- a/src/config.c
+++ b/src/config.c
@@ -123,7 +123,8 @@ void *davrods_merge_dir_config(apr_pool_t *p, void *_parent, void *_child) {
     DAVRODS_PROP_MERGE(anonymous_auth_username);
     DAVRODS_PROP_MERGE(anonymous_auth_password);
 
-    assert(set_exposed_root(conf, exposed_root) >= 0);
+    int rc = set_exposed_root(conf, exposed_root);
+    assert(rc >= 0);
 
     DAVRODS_PROP_MERGE(html_head);
     DAVRODS_PROP_MERGE(html_header);

Using mod_cache: unexpected 304 (NOT MODIFIED) on unconditional GET of valid, expired, cached file

I think I've found a bug in davrods. I have apache configured with mod_cache/mod_cache_disk and mod_dav/davrods. The cache has a copy of an iRODS data object. The copy is valid, meaning the data object in iRODS still exists and hasn't been modified since the copy was cached. The copy is expired though. According to RFC 2616, when a GET request is made for this data object, the apache should check to see if the copy is still valid in iRODS. Since it is, apache should update the expiration time and return the cached copy as the body of a 200 (OK) response. Instead, when I make the GET request, apache returns a 304 (NOT MODIFIED) response with an empty body.

I wasn't certain if this was an issue with davrods or something else. To test if this was a bug in davrods, I added mod_dav_fs based WebDAV repository on the apache server's local filesystem. I performed the same test. When I made the GET request for the file in local WebDAV repository, the cached copy was returned as the body of a 200 response. This implies that the bug is likely in davrods.

I'm using iRODS 4.2.8. For the WebDAV server running on CentOS 7, I'm using apache 2.4.6 and davrods 4.2.8_1.5.0. Here's the virtual host configuration.

<VirtualHost *:80>
  ServerName 128.196.65.41

### MOD_CACHE CONFIGURATION

  CacheDetailHeader  On
  CacheEnable        disk /
  CacheRoot          /var/cache/httpd/proxy
  
  # Have cached files expire quickly
  CacheMaxExpire 1

### 

### MOD_DAV_FS CONFIGURATION

  DavLockDB /var/www/DavLock

  Alias /dav_fs /var/www/webdav

  <Location /dav_fs/>
    AuthType  None
    Require   all granted

    Dav On
  </Location>

###

### DAVRODS CONFIGURATION

  <Location /davrods/>
    AuthType  None
    Require   all granted

    Dav davrods-locallock

    DavRodsEnvFile         /etc/httpd/irods/irods_environment.json
    DavRodsServer          128.196.65.131 1247
    DavRodsZone            cyverse.k8s
    DavRodsAnonymousMode   On
    DavRodsAnonymousLogin  "anonymous" ""
    DavRodsExposedRoot     /cyverse.k8s/home/shared
    DavRodsLockDB          /var/lib/davrods/lockdb_locallock

    DirectoryIndex disabled
  </Location>

###

</VirtualHost>

Here's a curl based example of how mod_dav_fs responds to caching. Notice that when retrieving a file from the cache when the cached copy is expired but still valid, it refreshes the cached copy and returns as the body of a 200 response.

prompt> curl -v 128.196.65.41/dav_fs/MOTD
*   Trying 128.196.65.41...
* TCP_NODELAY set
* Connected to 128.196.65.41 (128.196.65.41) port 80 (#0)
> GET /dav_fs/MOTD HTTP/1.1
> Host: 128.196.65.41
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Tue, 25 Aug 2020 18:58:31 GMT
< Server: Apache/2.4.6 (CentOS)
< Last-Modified: Tue, 25 Aug 2020 18:21:52 GMT
< Content-Length: 20
< ETag: "14-5adb7c6fc1c8e"
< Accept-Ranges: bytes
< X-Cache-Detail: "conditional cache hit: entity refreshed" from 128.196.65.41
< 
Hi from mod_dav_fs!
* Connection #0 to host 128.196.65.41 left intact

Here's one for how davrods responds to caching. Notice that when retrieving a file from the cache when the cached copy is expired but still valid, it returns a 304 response.

prompt> curl -v 128.196.65.41/davrods/MOTD
*   Trying 128.196.65.41...
* TCP_NODELAY set
* Connected to 128.196.65.41 (128.196.65.41) port 80 (#0)
> GET /davrods/MOTD HTTP/1.1
> Host: 128.196.65.41
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 304 Not Modified
< Date: Tue, 25 Aug 2020 18:58:46 GMT
< Server: Apache/2.4.6 (CentOS)
< ETag: "11-01598381793"
< 
* Connection #0 to host 128.196.65.41 left intact

wrong lib name version 4.2.0 / 4.2.1

Update :
I by passed this problem with a link to the dir /usr/lib64 instead of /usr.lib
end.

Using the 4.2_1.1.1 rpm , at the start of httpd, an error came up:

Jul 17 10:57:02 foo.foo.fr httpd[8255]: httpd: Syntax error on line 56 of /etc/httpd/conf/httpd.conf: Syntax error on line 1 of /etc/httpd/conf.modules.d/10-davrods.conf: Cannot load modules/mod_davrods.so into server: libirods_client.so**.4.2.0**: cannot open shared object file: No such file or directory

Jul 17 10:57:02 foo.foo.fr systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE

This is because the lib version installed are : 4.2.1 NOT 4.2.0
Can you fix that / How can i fix that ?

Thx
Cheers
S.

Build fails due to missing dependencies

When I install all the RPMs listed in the readme, I get the following errors

/usr/bin/ld: cannot find -lstdc++
/usr/bin/ld: cannot find -lboost_system
/usr/bin/ld: cannot find -lboost_filesystem
/usr/bin/ld: cannot find -lboost_regex
/usr/bin/ld: cannot find -lboost_thread
/usr/bin/ld: cannot find -lboost_chrono
/usr/bin/ld: cannot find -ljansson
collect2: error: ld returned 1 exit status

I have been able to build after adding the following packages:

  • libstdc++-static
  • boost-static
  • jansson-devel

Is a package for debian 12 planned ?

I'm currently trying to deploy davrods on a debian 12 environment, as a standalone server (irods 4.3.1 is deployed on an external server).

Currently renci only ships irods-runtime 4.3.0 and 4.3.1 for Debian, and unfortunately building from source is not a reliable option for me.

Do you plan to package a DEB for debian 12 in the future ?

Thanks,
technics

Cannot build on RHEL 6

Build fails on RHEL6. I think I have all required packages installed.
Where are AUTHN_PROVIDER_VERSION and AP_AUTH_INTERNAL_PER_CONF supposed to come from?

[root@icat build]# make
[ 12%] Building C object CMakeFiles/mod_davrods.dir/auth.c.o
/root/davrods-4.2.1_1.3.0/auth.c: In function ‘davrods_auth_register’:
/root/davrods-4.2.1_1.3.0/auth.c:442: warning: implicit declaration of function ‘ap_register_auth_provider’
/root/davrods-4.2.1_1.3.0/auth.c:445: error: ‘AUTHN_PROVIDER_VERSION’ undeclared (first use in this function)
/root/davrods-4.2.1_1.3.0/auth.c:445: error: (Each undeclared identifier is reported only once
/root/davrods-4.2.1_1.3.0/auth.c:445: error: for each function it appears in.)
/root/davrods-4.2.1_1.3.0/auth.c:447: error: ‘AP_AUTH_INTERNAL_PER_CONF’ undeclared (first use in this function)
make[2]: *** [CMakeFiles/mod_davrods.dir/auth.c.o] Error 1
make[1]: *** [CMakeFiles/mod_davrods.dir/all] Error 2
make: *** [all] Error 2

Accessing files in IGV (Range header support)

Hi, I have problems with accessing BAM files through HTTPS+Davrods in IGV.

Does Davrods support HTTP range queries?

I'm getting the following access/error logs

==> /var/log/apache2/error.log <==
[Thu Aug 24 14:34:28.938949 2017] [dav:error] [pid 2451:tid 139841358481152] [client 172.16.128.8:57958] Unable to deliver content.  [500, #0]
[Thu Aug 24 14:34:28.939049 2017] [dav:error] [pid 2451:tid 139841358481152] (104)Connection reset by peer: [client 172.16.128.8:57958] Could not write contents to filter.  [500, #0]

==> /var/log/apache2/other_vhosts_access.log <==
cubi-webdav-omics-prototype.bihealth.org:443 172.16.128.8 - holtgrem [24/Aug/2017:14:32:27 +0200] "GET /omicsZone/projects/project8/bio_samples/33900/33900-N1/33900-N1-DNA1/33900-N1-DNA1-WES1/2017-08-23/GRCh37/ngs_mapping/out/bwa.33900-N1-DNA1-WES1.bam.bai HTTP/1.1" 500 519832 "-" "IGV Version 2.3.97 (157)07/10/2017 10:52 PM"
cubi-webdav-omics-prototype.bihealth.org:443 172.16.128.8 - holtgrem [24/Aug/2017:14:34:35 +0200] "HEAD /omicsZone/projects/project8/bio_samples/33900/33900-N1/33900-N1-DNA1/33900-N1-DNA1-WES1/2017-08-23/GRCh37/ngs_mapping/out/bwa.33900-N1-DNA1-WES1.bam.bai HTTP/1.1" 200 2352 "-" "IGV Version 2.3.97 (157)07/10/2017 10:52 PM"
cubi-webdav-omics-prototype.bihealth.org:443 172.16.128.8 - holtgrem [24/Aug/2017:14:34:35 +0200] "HEAD /omicsZone/projects/project8/bio_samples/33900/33900-N1/33900-N1-DNA1/33900-N1-DNA1-WES1/2017-08-23/GRCh37/ngs_mapping/out/bwa.33900-N1-DNA1-WES1.bam HTTP/1.1" 200 277 "-" "IGV Version 2.3.97 (157)07/10/2017 10:52 PM"

IGV is giving me

Error loading https://cubi-webdav-omics-prototype.bihealth.org/omicsZone/projects/project8/bio_samples/33900/33900-N1/33900-N1-DNA1/33900-N1-DNA1-WES1/2017-08-23/GRCh37/ngs_mapping/out/bwa.33900-N1-DNA1-WES1.bam: Premature EOF. Expected 4 but only received 0; BinaryCodec in readmode; streamed file (filename not available)

SegFault when DavrodsExposedRoot is not provided

I used the provided davrods-vhost.conf and received a SegFault when trying to connect. Uncommenting the line

        DavrodsExposedRoot  User

fixed the issue. Havn't tried it for other config values.

Cheers

(gdb) bt full
#0  __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:199
No locals.
#1  0x00007ffff522b59e in set_exposed_root (exposed_root=0x0, conf=<optimized out>, conf=<optimized out>) at /home/vagrant/davrods/src/config.c:31
No locals.
#2  0x00007ffff522b7f0 in davrods_merge_dir_config (p=<optimized out>, _parent=0x7ffff7fa1ea8, _child=0x7ffff7f3c4d0) at /home/vagrant/davrods/src/config.c:140
        ret = <optimized out>
        parent = 0x7ffff7fa1ea8
        child = 0x7ffff7f3c4d0
        conf = 0x7ffff7eb9a58
        exposed_root = <optimized out>
        __PRETTY_FUNCTION__ = "davrods_merge_dir_config"
#3  0x00005555555ae70f in ap_merge_per_dir_configs ()
No symbol table info available.
#4  0x00005555555a6862 in ap_location_walk ()
No symbol table info available.
#5  0x00005555555a6c28 in ap_process_request_internal ()
No symbol table info available.
#6  0x00005555555c6228 in ap_process_async_request ()
No symbol table info available.
#7  0x00005555555c28e0 in ?? ()
No symbol table info available.
#8  0x00005555555b8090 in ap_run_process_connection ()
No symbol table info available.
#9  0x00007ffff1665ecf in ?? () from /usr/lib/apache2/modules/mod_mpm_event.so
No symbol table info available.
#10 0x00007ffff16668e5 in ?? () from /usr/lib/apache2/modules/mod_mpm_event.so
No symbol table info available.
#11 0x00007ffff74e96db in start_thread (arg=0x7fffefe01700) at pthread_create.c:463
        pd = 0x7fffefe01700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737217828608, 6623695043794037053, 140737217826688, 0, 140737353479696, 140737353475416, -6623730396236617411, -6623676774370776771}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#12 0x00007ffff721271f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Can Davrods be used with SSO (SAML/OpenID Connect)

Dear Davrods Developers,

is there a way for Davrods to interact with an SSO integration of iRODS, e.g. SAML/OpenID Connect?

I am not certain how that would work precisely but I could imagine that there are some challenges (and also I'm not certain how well iRODS plays here). I'm aware of this:

This looks like there are some roadblocks...

Have I answered my own question with "not yet"?

Is there any further insight that you could provide?

Best wishes,
Manuel

Paths in multistatus PROPFIND response document have double slash (//)

When a PROPFIND request is made on a collection with the URL https://host.domain/collection/, the paths to the members of /collection/ in the DAV:multistatus response document all have an extra forward slash in the URL immediately after the /collection/ prefix. For example, if object is a member of collection, its URL path in the response will be /collection//object. Here's a real example.

tedgin@dev ~? curl -u tedgin -H 'Depth: 1' -X PROPFIND https://data-dev.cyverse.rocks/dav/cyverse.dev/home/tedgin/IticketTest/
Enter host password for user 'tedgin':
<?xml version="1.0" encoding="utf-8"?>
<D:multistatus xmlns:D="DAV:">
<D:response xmlns:lp1="DAV:" xmlns:lp2="http://apache.org/dav/props/">
<D:href>/dav/cyverse.dev/home/tedgin/IticketTest/</D:href>
<D:propstat>
<D:prop>
<D:creationdate>Wed, 30 Oct 2019 18:58:08 GMT</D:creationdate>
<D:getetag>"01572461888"</D:getetag>
<D:getlastmodified>Wed, 30 Oct 2019 18:58:08 GMT</D:getlastmodified>
<lp1:resourcetype><D:collection/></lp1:resourcetype>
<D:supportedlock>
<D:lockentry>
<D:lockscope><D:exclusive/></D:lockscope>
<D:locktype><D:write/></D:locktype>
</D:lockentry>
<D:lockentry>
<D:lockscope><D:shared/></D:lockscope>
<D:locktype><D:write/></D:locktype>
</D:lockentry>
</D:supportedlock>
<D:lockdiscovery/>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
<D:response xmlns:lp1="DAV:" xmlns:lp2="http://apache.org/dav/props/">
<D:href>/dav/cyverse.dev/home/tedgin/IticketTest//LICENSE.txt</D:href>
<D:propstat>
<D:prop>
<D:creationdate>Wed, 30 Oct 2019 18:58:24 GMT</D:creationdate>
<D:getcontentlength>3491</D:getcontentlength>
<D:getetag>"da3-01572461904"</D:getetag>
<D:getlastmodified>Wed, 30 Oct 2019 18:58:24 GMT</D:getlastmodified>
<lp1:resourcetype/>
<D:supportedlock>
<D:lockentry>
<D:lockscope><D:exclusive/></D:lockscope>
<D:locktype><D:write/></D:locktype>
</D:lockentry>
<D:lockentry>
<D:lockscope><D:shared/></D:lockscope>
<D:locktype><D:write/></D:locktype>
</D:lockentry>
</D:supportedlock>
<D:lockdiscovery/>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>

iRODS replication fails when putting files via Davrods from the OS X built-in WevDAV client

Hi,

I recently noticed that when putting files via Davrods to a replicated resource in iRODS 4.1.10 or iRODS 4.1.11, the replication fails and the new data object lands only in the first resource server it hits.

The really strange thing is that the failure only occurs when connecting via the OS X built-in WebDAV. Linux webdav clients my colleagues tested (GNOME Nautilus or davfs I suppose) worked perfectly. Also CyberDuck works.

For example:

-bash-4.2$ ils -l
/tempZone/home/rods:
  rods              0 replResc;rodsresc2Resource         5606 2017-12-12.16:23 & bash-history-orig.txt
  rods              1 replResc;rodsresc1Resource         5606 2017-12-12.16:23 & bash-history-orig.txt
  rods              0 replResc;rodsresc2Resource         4096 2017-12-12.16:18 & ._SC17 Final Program.pdf
  rods              1 replResc;rodsresc1Resource         4096 2017-12-12.16:18 & ._SC17 Final Program.pdf
  rods              0 replResc;rodsresc2Resource      5500794 2017-12-12.16:18 & SC17 Final Program.pdf
  rods              0 replResc;rodsresc2Resource      2593877 2017-12-12.16:16 & updates.img
-bash-4.2$ 

Here the file updates.img was uploaded via OS X native client, as well as SC17 Final Program.pdf (which had extended attributes). The file bash-history.orig.txt was uploaded from OS X via CyberDuck. To add insult to injury, iRODS replicated the xattr file but not the actual file, as you can see.

My personal guess is that the OS X WebDAV client does something stupid, which Davrods doesn't expect and causes a misbehavior in the iRODS protocol, which breaks replication.

Apache error log gave something possibly related:

[Tue Dec 12 16:18:41.282197 2017] [dav:error] [pid 12128] [client 192.168.56.1:61710] Unable to deliver content.  [500, #0]
[Tue Dec 12 16:18:41.282231 2017] [dav:error] [pid 12128] (32)Broken pipe: [client 192.168.56.1:61710] Could not write contents to filter.  [500, #0]

The iRODS endpoint server (in this case the iCAT) logs were clean as well as the rodsLog.* files on both of the resource servers.

The server environment was a cleanly built virtual cluster via irods-provisioner on CentOS 7 hosts, davrods-1.3.0 and iRODS 4.1.1.11.

support for user impersonation by service account

It is often the case that a web application needs to access a user's data managed by another service. One solution, is to have the web service connect to the data service as the user, using the user's account name and password for authentication. This isn't always viable. For example, if an authenticated user schedules an HPC job that processes some of the user's data, the job may not have access to the user's password for the data service. iRODS supports this case by allowing the web application to impersonate the user. The web application would authenticate with iRODS using a rodsadmin type account and provide the user's iRODS account name as the client user.

I'm requesting that davrods be extended to support iRODS style user impersonation. The web application could connect to a davrods based data service with basic authentication, providing its iRODS account name and password. The client name could be provided in a custom HTTP header, e.g., X-Davrods-Impersonate-User: CLIENT-USERNAME. The client's account name could be passed to iRODS through the iRODS C API similar to how the iCommands pass it in.

Support for mounted tar files

Using the iCommand imcoll, a tar data object can be mounted as a collection, allowing other iCommands to transparently access the contents of the tar data object. For example,

prompt> ls set
file1  file2
prompt> tar -c -C set -f set.tar .
prompt> iput -D tar set.tar /zone/home/user
prompt> imkdir /zone/home/user/set
prompt> imcoll -m tar /zone/home/user/set.tar /zone/home/user/set
prompt> ils set
  file1
  file2

It would be nice if davrods could provide WebDAV support for mounted tar data objects. At the moment, davrods allows navigation of the directories in the mounted tar data object, but it doesn't allow any files to be downloaded. When a download is attempted, it fails and logs a rcDataObjLseek failed: -19000 = SYS_FILE_DESC_OUT_OF_RANGE error.

Timeout with iRODS and Davrods.

Hello,
I'm using irods for physical storage and I have an irods montage (davrods) on my ubuntu 18.04 machine of about 3To size.
The aim is to run python scripts for satellite image processing.
I successfully downloaded and stored my images (~50Go) however when i runned my python code for image processing (stack of some images) the output file never exceeded 32 MB. It increases to 32MB and then empties and refills. It is as if the maximum size was limited to 32 MB...
In the irods_environment.json file, i modified the irods_maximum_size_for_single_buffer_in_megabytes from 32 to 32000 but it didn't solve the problem.

here is a trace of log in irods (v 4.2.5)
Oct 29 17:22:44 pid:8380 remote addresses: 127.0.0.1, 134.158.150.48,
193.50.94.68 ERROR: [-]
/tmp/tmppTB_kL/server/core/src/rsApiHandler.cpp:540:int
readAndProcClientMsg(rsComm_t *, int) : status
[SYS_HEADER_READ_LEN_ERR] errno [] -- message [only read [0] of [4]]

and in davrods (V 4.2.6_1.4.2-1).

[Tue Oct 29 18:00:22.212903 2019] [dav:error] [pid 164681] [client
193.50.94.68:45502] Unable to deliver content. [500, #0]
[Tue Oct 29 18:00:22.212939 2019] [dav:error] [pid 164681]
(104)Connection reset by peer: [client 193.50.94.68:45502] Could not wri
te contents to filter. [500, #0]

Thank's a lot for your advice
P.M.!

Found a minor bug?

Hi,

Thanks for the brilliant WebDAV!

I might have discovered a minor bug, might be configuration too but you might want to check.

If I open http URL:s in a browser like http://ridgehead.pdc.kth.se/home/ilarik without the trailing slash, the directory listing has invalid links like http://ridgehead.pdc.kth.se:8080/home/firstdir/ which should be http://ridgehead.pdc.kth.se:8080/home/ilarik/firstdir/.

If I add the trailing slash to the URL like http://ridgehead.pdc.kth.se:8080/home/ilarik/ everything works peachy in the links in the generated index pages.

Davrods error with irule

Hello,

I am using Davrods to drop files into my irods instance. Every thing is working fine. I can drop images and check through ils -L icommand. I also was working with irules. So I created an irule to check if the file is an image and extract all metadata and insert into irods. My irule works fine when I use iput command, but when I test to drop the images through Davrods I got an error on the httpd server.

tail  /etc/httpd/logs/error_log
[Wed Oct 26 10:41:29.282775 2016] [dav:error] [pid 11405] [client 145.117.144.70:59273] Unable to deliver content.  [500, #0]
[Wed Oct 26 10:41:29.282820 2016] [dav:error] [pid 11405] (32)Broken pipe: [client 145.117.144.70:59273] Could not write contents to filter.  [500, #0]
ERROR: [-]  iRODS/lib/core/src/procApiRequest.cpp:266:readAndProcApiReply :  status [SYS_HEADER_READ_LEN_ERR]  errno [] -- message []
[-] iRODS/lib/core/src/sockComm.cpp:199:readMsgHeader :  status [SYS_HEADER_READ_LEN_ERR]  errno [] -- message [failed to call 'read header']
[-] libssl.cpp:577:ssl_read_msg_header :  status [SYS_HEADER_READ_LEN_ERR]  errno [] -- message [read 0 expected 4]
[Wed Oct 26 10:41:36.192629 2016] [davrods:error] [pid 11389] [client 145.117.144.70:59275] rcDataObjClose failed: -4000 = SYS_HEADER_READ_LEN_ERR
[Wed Oct 26 10:41:36.192643 2016] [dav:error] [pid 11389] [client 145.117.144.70:59275] Could not close the uploaded resource  [500, #0]
ERROR: [-]  iRODS/lib/core/src/rcConnect.cpp:269:rcDisconnect :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message []
[-] iRODS/lib/core/src/sockComm.cpp:1394:sendRodsMsg :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message [failed to call 'write body']
[-] libssl.cpp:1058:ssl_send_rods_msg :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message [Write message header failed.]
[-] iRODS/lib/core/src/sockComm.cpp:473:writeMsgHeader :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message []
[-] libssl.cpp:1001:ssl_write_msg_header :  status [SYS_HEADER_WRITE_LEN_ERR]  errno [Broken pipe] -- message [Wrote -1 expected 140.]
ERROR: [-]  iRODS/lib/core/src/rcConnect.cpp:281:rcDisconnect :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message []
[-] iRODS/lib/core/src/sockComm.cpp:100:sockClientStop :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [failed to call 'client stop']
[-] libssl.cpp:648:ssl_client_stop :  status [SSL_SHUTDOWN_ERROR]  errno [] -- message [error shutting down the SSL connection]
tail  /var/lib/irods/iRODS/server/log/rodsLog.2016.10.26
Authenticated
Oct 26 10:40:30 pid:11611 NOTICE: writeLine: inString = Hello world! A image file was uploaded [/var/lib/irods/iRODS/Vault/home/felipe/nave.jpg].
terminate called after throwing an instance of 'Magick::ErrorCorruptImage'
  what():  irodsAgent: Empty input file `/var/lib/irods/iRODS/Vault/home/felipe/nave.jpg' @ error/jpeg.c/JPEGErrorHandler/316
Oct 26 10:40:30 pid:11315 NOTICE: Agent process 11611 exited with status 6
cat /etc/irods/training_acPostProcForPut.re 
acPostProcForPut {
  if ($filePath like "*.jpg" || $filePath like "*.jpeg" || $filePath like "*.bmp" || $filePath like "*.tif" || $filePath like "*.tiff" || $filePath like "*.rif" || $filePath like "*.gif" || $filePath like "*.png" || $filePath like "*.svg" || $filePath like "*.xpm") {
    writeLine("serverLog", "Hello world! A image file was uploaded [$filePath].");
    msiget_image_meta($filePath, *meta);
    writeLine("serverLog", "image file 1111");
    msiString2KeyValPair(*meta, *meta_kvp);
    writeLine("serverLog", "image file 2222");
    msiAssociateKeyValuePairsToObj(*meta_kvp, $objPath, "-d");
    writeLine("serverLog", "image file 3333");
  } # if
} # acPostProcForPut

uploading large files via davrods

Hi Davrods folks,

We've got davrods deployed but, while we can download large (eg 10GB) files, we cannot upload them.

The upload fails with:
413 Request Entity Too Large

Do you have instructions or advice for increasing the upload file size limit? In the long run, we may need 100GB -> 1TB.

Thanks for this project!

Best,
Blake

Compute checksums on upload?

davrods is an incredibly useful software package, again: thanks for it.

It would be great if davrods would compute checksums for uploaded files. Is this possible?

Read-Only WebDAV

Hello,

Is there a way to make the davrods package give read-only access to iRODS? I would assume some configuration somewhere (apache?) could do that but pointers to where would be greatly appreciated.

Thanks,

Burnce

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.