slnc-dimp / cinch

A project to develop a bulk download service to a central repository that will maintain original file timestamps, virus check, extract file level metadata, create file checksums and periodically validate checksums for continued file integrity. Users merely need to upload a list of URLs to download and when the process completes they can download the requested files and file metadata to their local environment. Funding for the CINCH: Capture, Ingest, & Checksum tool is made possible through an IMLS Sparks! Ignition grant.

As of December 1, 2018, we are no longer offering a hosted version of CINCH. You may still install it locally using the Github files.

A project to develop a bulk download service to a central repository that will maintain original file timestamps, extract file level metadata, create file checksums and periodically validate checksums for continued file integrity.

Users merely need to upload a list of URLs to download and when the process completes they can download the requested files and file metadata to their local environment.

Currently supported file types:

• PDF
• Microsoft Word
• Microsoft Excel
• Microsoft PowerPoint
• Jpeg
• PNG
• GIF
• Text (e.g. files with .txt, .csv extensions, etc.)

Learn more about CINCH.

Full end user instructions

Funding for the CINCH: Capture, Ingest, & Checksum tool is made possible through an IMLS Sparks! Ignition grant.

License: CINCH is released under the Unlicense (http://unlicense.org/)

Requirements

• Currently Cinch will only run on *nix systems
• PHP 5.3+ compiled with --enable-cli flag and curl module. This is probably already setup in your package manager (PHP 5.4+ is recommended).
• MySQL
• ClamAV

After download, you may need to run the setup.sh shell file before initially running application to set file level permissions.

Setting up Cinch on your system:

1. Place the Cinch files in a web accessible directory
2. Create a new database MySQL database and import the project.sql file into it.
3. Open protected/config/main.php
4. Scroll down to the db settings (line 68 or so). Set the database name, the username and password for your new Cinch database.
5. Scroll to the bottom of main.php and set 'adminEmail' email address to your email address.
6. You'll now need to repeat steps four and five in protected/config/console.php.
7. In protected/config/console.php if the setting in date_default_timezone_set() isn't correct you should change it to your timezone setting. For a complete list of timezone settings see: http://us2.php.net/manual/en/timezones.php.
8. Go to http://tika.apache.org/download.html and download the Apache Tika jar file.
9. Place the Apache Tika jar file at the root of the Cinch/protected directory.
10. Configure Cinch cron tasks. See the sample cron.txt file the root of Cinch for suggestions on how you might want to configure it.

You should now be able to login to the web interface as: admin admin. You should then go the change password tab and update your password.

If you don't want to run Cinch via cron you can run it from the command line. If you navigate to Cinch/protected and run the following: path/to/php yiic.php you should be presented with a list of available commands. The general way to run a command is: path/to/php yiic.php command. Several commands such as checksum and purgeystem have subcommands, which have to be run like so from the command line: path/to/php yiic.php command sub-command.

You should run the commands in the following order:

1. readfile
2. download
3. viruscheck
4. checksum create
5. metadata
6. metadatacsv
7. checksum check (optional, recalculates checksum to see if anything has changed between download and current time.)
8. errorcsv
9. zipcreation
10. purgesystem check (optional, Notifies users after 20 days that they have files marked for deletion in 10 days.)
11. purgesystem delete (optional, deletes user files older than 30 days old. Note this deletes upload lists, and all csv file information from the database, but downloaded file, metadata, errors, and event information is retained in the database.)

Useful Notes:

• You should only run the zipcreation command once a day otherwise it will cause conflicts in file processing.
• Uploaded url lists are saved into protected/uploads/"user's username". With the user's directory being created on first upload and being deleted thereafter if it's empty.
• Downloaded user files are saved into protected/curl_downloads/"user's username". With the user's directory being created on first file downloaded and being deleted thereafter if it's empty.
• CINCH API documentation can be viewed at: http://cinch.nclive.org/c_docs/packages/db_Default.html.

Adding New Users:

Currently users can't self-register (This fit our own particular needs.)

• Login as user with admin privileges. The default "admin" user has admin privileges already setup.
• Then go to Admin->User Administration->Create User and add the user. (The user will be sent an email with their username and password. Users may then login and change their password.)
• Next go to Admin->User Rights. Click the user's username and then select the privileges you want them to have.
• You've now successfully added a user.

If you have problems with setting up users feel free to contact us. You might want to take a look at the documentation for the Yii Rights extension used in CINCH: http://yii-rights.googlecode.com/files/yii-rights-doc-1.2.0.pdf

---

Parts of Cinch include:

• Yii Framework http://www.yiiframework.com
• jQuery http://jquery.com
• jQuery UI http://jqueryui.com
• Apache Tika http://tika.apache.org
• ClamAV http://www.clamav.net

cinch's Issues

Non AMQP version

Pull out a 'lite' version that doesn't rely on anything but PHP and MySQL.