weiss / ngx_http_upload

Nginx module to handle file uploads and downloads for XMPP servers

upload.pm

This Nginx module implements the HTTP server's part of the XMPP extension XEP-0363: HTTP File Upload. It can be used with either ejabberd's mod_http_upload or Prosody's mod_http_upload_external.

Nginx setup

1. Create a directory and move upload.pm into it, e.g.:

   # mkdir -p /usr/local/lib/perl
   # wget -O /usr/local/lib/perl/upload.pm https://git.io/fNZgL

2. Install the ngx_http_perl_module. On Debian/Ubuntu-based distributions, the package is called libnginx-mod-http-perl, on RedHat/CentOS-based distributions, it's nginx-mod-http-perl. If you're using Docker, make sure you use the image ending in -perl.

3. Add the following snippets to the appropriate sections of your Nginx configuration:

   # This directive was probably added by the distribution package already:
   load_module modules/ngx_http_perl_module.so;
   
   http {
       # Add the following two lines to the existing "http" block.
       perl_modules /usr/local/lib/perl; # Path to upload.pm.
       perl_require upload.pm;
   }
   
   server {
       # Specify directives such as "listen", "server_name", and TLS-related
       # settings for the "server" that handles the uploads.
   
       # Uploaded files will be stored below the "root" directory. To minimize
       # disk I/O, make sure the specified path is on the same file system as
       # the directory used by Nginx to store temporary files holding request
       # bodies ("client_body_temp_path", often some directory below /var).
       root /var/www/upload;
   
       # Specify this "location" block (if you don't use "/", see below):
       location / {
           perl upload::handle;
       }
   
       # Upload file size limit (default: 1m), also specified in your XMPP
       # server's upload module configuration (see below):
       client_max_body_size 100m;
   }

4. Open upload.pm in an editor and adjust the configuration at the top of the file:

   • The $external_secret must match the one specified in your XMPP server's upload module configuration (see below).

   • If the root path of the upload URIs (the location specified in the Nginx server block) isn't / but /some/prefix/, $uri_prefix_components must be set to the number of directory levels. So, for /some/prefix/, it would be 2.

ejabberd setup

Let the mod_http_upload option put_url point to Nginx, and specify exactly the same external_secret as in the upload.pm settings:

modules:
  mod_http_upload:
    put_url: "https://upload.example.com"
    external_secret: "it-is-secret"
    max_size: 104857600 # 100 MiB, also specified in the Nginx configuration.

Prosody setup

Let the mod_http_upload_external option http_upload_external_base_url point to Nginx, and specify exactly the same http_upload_external_secret as in the upload.pm settings:

http_upload_external_base_url = "https://upload.example.com"
http_upload_external_secret = "it-is-secret"
http_upload_external_file_size_limit = 104857600 -- 100 MiB

Contact

If you have any questions, you could ask in the ejabberd room: [email protected] (the maintainer of this module is usually joined as Holger).

ngx_http_upload's Issues

Umlauts of filenames are incorrectly decoded

If there's an umlaut like ä, ü in the uploaded file's name, the perl module does not decode them correctly when saving the file:

Test-ü-ä-ß-Test.txt becomes Test-�_-�_-�_-Test.txt.

For more details see the former discussion on processone/ejabberd#2897

Update for v2 "API"

Prosody just released the v2 api. Can this be updated? :)

Undefined subroutine &upload::handle

Hi!
My current setup:

• Device : Odroid XU4 (ARMv7 32 bits)
• System : Armbian 5.85 Stable Debian Stretch 9
• Linux : 4.14.111-odroidxu4
• nginx : 1.10.3
• perl : 5.24.1
• libnginx-mod-http-perl : 1.10.3
• Prosody : 0.11.2

Whenever I try to upload a file using this module, nginx throws me back the following error:

[error] 2118#2118: *1 call_sv("upload::handle") failed: "Undefined subroutine &upload::handle called.", client: XXX.XXX.XXX.XXX, server: share.example.com, request: "PUT /825223ad-62d6-470c-a23e-0e05493e0851/RGLF37mWRJ26UfeKQkTX4A.jpg?v=3640efdbe186ffdd90dd639fb45a1dbd9d2708ed21d4bb646d0e8159cf022a10 HTTP/1.1", host: "share.example.com"

From what I gathered, this is in fact a Perl error. I tried to look into it myself but unfortunately I really know nothing about Perl. Any thoughts on what could cause this?

Can't locate loadable object for module nginx

Hello @weiss and happy Easter,
my set-up is as follows:

• nginx version: 1.14.2
• perl version: perl 5, version 26, subversion 3 (v5.26.3) built for x86_64-linux-thread-multi
• runtime environment: official nginx:alpine docker (image id 0be75340bd9b) (most likely we don't need that info).
• i have downloaded your upload.pm and modified the secret and the uri_prefix_components (=1)
• I have "load_module /usr/lib/nginx/modules/ngx_http_perl_module.so;"
• have the special "location / {}
• have the settings in ejabberd.yml

the nginx is still starting until this point.

But if I uncomment:

    # perl_modules /usr/local/lib/perl; # Path to upload.pm.
    # perl_require upload.pm;

in the "http {}" part of nginx and then reload the configuration, nginx produces immediately the following error:

Can't locate nginx.pm in @INC (you may need to install the nginx module) (@INC contains: /usr/local/lib/perl /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl).
BEGIN failed--compilation aborted.
2019/04/19 14:16:40 [alert] 17#17: perl_parse() failed: 2
nginx: [alert] perl_parse() failed: 2

I found nginx.pm in /usr/lib/perl5/vendor_perl/x86_64-linux-thread-multi and copied it down one folder to /usr/lib/perl5/vendor_perl.
After this "copying" I encountered the following slightly different error:

Can't locate loadable object for module nginx in @INC (@INC contains: /usr/local/lib/perl /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl) at -e line 0.
Compilation failed in require.
BEGIN failed--compilation aborted.
2019/04/19 14:24:05 [alert] 37#37: perl_parse() failed: 255
nginx: [alert] perl_parse() failed: 255

might it be due to incompatible perl-versions?

cors issue in nginx

Hi , First of all this was nice article . I followed steps as you said i am facing cors in firefox how to overcome this .

mod_http_upload not connecting to the put_url

I have followed your instruction but it is not working, on ejabberd log i see the request slot successfully but nothing uploaded to the destination http nginx server.

@mod_http_upload:create_slot/7:837 Got HTTP upload slot for [email protected]/d5a8f655-9758-4c73-a364-d1eb5db81ede (file: VOICE_896e8e90-01be-4f80-b966-835604bb7020.m4a, size: 14294)

Plz, help to advice.
Thanks.