cmrigney / node-sftp-s3

Node JS module for creating an SFTP server that uses S3 for file storage.

node-sftp-s3

UPDATE: I'm no longer maintaining this project. If you would like to take it over or you know of a solid fork, please reach out!

Node JS module for creating an SFTP server with user isolation that uses S3 for file storage.

Install

npm install node-sftp-s3

Usage

var SFTPS3Server = require('node-sftp-s3').SFTPS3Server;

var AWS = require('aws-sdk');
var path = require('path');
var fs = require('fs');

//aws.json contains credentials for S3 access
AWS.config.loadFromPath(path.join(__dirname, 'aws.json'));

var s3 = new AWS.S3({ params: { Bucket: 'testing' } });

var server = new SFTPS3Server(s3);

//You can generate a key with ssh-keygen
server.addHostKey(fs.readFileSync(path.join(__dirname, 'server_key_rsa')));

//Add users' public keys.  These can also be added after the server has already started
server.addPublicKey(fs.readFileSync(path.join(__dirname, 'client_key_rsa.pub')), 'baruser' /* , 'myapp' (optional path prefix) */);

server.listen(2222, '127.0.0.1', function(port) {
  console.log('Listening on ' + port);
  //server.stop() will stop the server
});

Events

SFTPServer emits several events. Each event passes a dictionary object with the listed parameters.

The path parameter includes the user's subfolder name.

• client-error - { client: <Object>, error: <Error> }
• login - { username: <string> }
• file-uploaded - { path: <string>, username: <string> }
• file-downloaded - { path: <string>, username: <string> }
• file-deleted - { path: <string>, username: <string> }
• directory-deleted - { path: <string>, username: <string> }
• directory-created - { path: <string>, username: <string> }
• file-renamed - { path: <string>, oldPath: <string>, username: <string> }

Notes

Password authentication is NOT currently supported. Only public/private key auth. It should be simple to add this feature. A PR is welcome. :)

Docker

The module also includes a server suitable for running in a Docker container.

1. cp .env.sample .env
2. Edit .env to add appropriate configuration values for all variables
3. mkdir keys && ssh-keygen -t rsa -f keys/server_key_rsa
4. For each user, ssh-keygen -t rsa -f keys/id_<username>
5. docker-compose up

node-sftp-s3's Issues

Include date/time stamp when logging events.

We are using this lib to host SFTP servers and using PM2 to run the node app. It would be great the the event logger include the date/time when writing the log.

Can you update this?

Tests are failing

Tests are failing with the following exception:

  1) SFTP Should Write and Read and Delete Correctly:
     Uncaught TypeError: Cannot read property '_log' of undefined
      at lib/node-sftp-s3.js:29:2873
      at test/s3stub.js:99:28

Advice on using in production

Hi,

I'm trying to deploy my new SFTP server to an AWS EC2 instance.

Works fine running + connecting to it locally - but cannot connect to it when running on the server.

Running, nmap, I see my port is open on AWS EC2, and as far as I can tell "closed" means there is nothing running on that port:

PORT STATE SERVICE
2222/tcp closed EtherNetIP-1

I suspect it could be the hostname that I need to set correctly - but I can't find any documentation about what I should use.

Any clues?

Thanks

Allow password login

This is already mentioned in the README, but I thought I'd best create an issue for it.

As I understand it, this requires:
a) a way of specifying the usernames+passwords (much like addPublicKey does)
b) a way of checking these in client.on('authentication', ...)

I will have a go at this and see where I get. If I've missed something from my plan-of-action above, let me know.

READ returned more bytes than requested

I'm having issues with READ returning bigger packets than the client requested. Both using Transmit and FileZilla on MacOS.

I'm using the package as used in the example with files of size around 10 megabytes.

Is this something you have experienced yourself?

Best Regards
Thomas

Emit error event for server listening error

We should have an error event emitted when the listener fails. See https://github.com/mscdex/ssh2

sample aws.json file?

Can you add a sample aws.json file to /examples ?

Can't view work with directories inside 'user' dir.

Hey there,
Seem to be having an issue working with directories in node-sftp-s3.

sftp> ls
upload
sftp> cd upload
Couldn't stat remote file: No such file or directory

Read 23 byte file, logs show 4MB of reading

Hello,

If I read (using Filezilla in Linux) a small 23 byte file from the SFTP service, the file downloads OK, but the logs contain this:

SFTP OPEN filename=/testfile.txt flags=1 handle=1
Issuing handle 1
SFTP READ handle=1 offset=0 length=32768
SFTP READ handle=1 offset=32768 length=32768
EOF
SFTP READ handle=1 offset=65536 length=32768
EOF
SFTP READ handle=1 offset=98304 length=32768
EOF
SFTP READ handle=1 offset=131072 length=32768
EOF
SFTP READ handle=1 offset=163840 length=32768
EOF

(skipped the middle bit to save space)

SFTP READ handle=1 offset=4128768 length=32768
EOF
SFTP READ handle=1 offset=4161536 length=32768
EOF
Successfully read dir/testfile.txt
SFTP CLOSE handle=1
Handle closed

which doesn't seem right - is it?

AWS SDK breaking changes?

I started working on implementing this today in a project I am working on. I've had quite a bit of trouble, and finally started debugging through the node-sftp-s3 code. I am wondering if the AWS SDK may have changed significantly since the last time this module had a major update?

It looks like the existing code looks for /.dir objects on S3 to determine if an object is a directory. From what I can tell, the way S3 currently works, there are no .dir files. Instead, a directory is simply identified by a trailing / in the name of the object. I am not quite sure how extensive the use of /.dir is, however I have found a few use cases so far. Is there any chance this library could be updated to work with the current version of the AWS SDK?