master
Build Status:develop
Build Status:- Public Repo: https://github.com/CenterForOpenScience/osf.io/
- Issues: https://github.com/CenterForOpenScience/osf.io/issues?state=open
- Huboard: https://huboard.com/CenterForOpenScience/osf.io#/
- Docs: http://cosdev.rtfd.org/
Solutions to many common issues may be found at the OSF Developer Docs.
These instructions should work on Mac OSX >= 10.7
-
Create your virtualenv.
-
Copy
website/settings/local-dist.py
towebsite/settings/local.py.
NOTE: This is your local settings file, which overrides the settings inwebsite/settings/defaults.py
. It will not be added to source control, so change it as you wish.
$ cp website/settings/local-dist.py website/settings/local.py
-
You will need to:
- Create local.py files for addons that need them.
- Install TokuMX.
- Install libxml2 and libxslt (required for installing lxml).
- Install elasticsearch.
- Install GPG.
- Install requirements.
- Create a GPG key.
- Install npm.
- Install node and bower packages.
- Build assets.
-
To do so, on MacOSX with homebrew (click link for homebrew installation instructions), run:
$ pip install invoke
$ invoke setup
-
On Linux systems, you may have to install python-pip, TokuMX, libxml2, libxslt, elasticsearch, and GPG manually before running the above commands.
-
If invoke setup hangs when 'Generating GnuPG key' (especially under linux), you may need to install some additonal software to make this work. For apt-getters this looks like:
sudo apt-get install rng-tools
next edit /etc/default/rng-tools and set:
HRNGDEVICE=/dev/urandom
last start the rng-tools daemon with:
sudo /etc/init.d/rng-tools start
__source: http://www.howtoforge.com/helping-the-random-number-generator-to-gain-enough-entropy-with-rng-tools-debian-lenny __
- Run your mongodb process.
$ invoke mongo
- Run your local development server.
$ invoke server
- Run your local sharejs server.
$ invoke sharejs
To open the interactive Python shell, run:
$ invoke shell
To run all tests:
$ invoke test --all
To run a certain test method
$ nosetests tests/test_module.py:TestClass.test_method
Run OSF Python tests only:
$ inv test_osf
Run addons Python tests only:
$ inv test_addons
Run Javascript tests:
$ inv karma
By default, inv karma
will start a Karma process which will re-run your tests every time a JS file is changed. To do a single run of the JS tests:
$ inv karma --single
By default, Karma will run tests using a PhantomJS headless browser. You can run tests in other browsers like so:
$ inv karma -b Firefox
If you want to run cross browser tests with SauceLabs, use "sauce" parameter:
$ inv karma --sauce
Addons tests are not run by default. To execute addons tests, run
$ invoke test_addons
First, set MAIL_SERVER
to localhost:1025
in you local.py
file.
website/settings/local.py
...
MAIL_SERVER = "localhost:1025"
...
Sent emails will show up in your server logs.
Optional: fire up a pseudo-mailserver with:
$ invoke mailserver -p 1025
TokuMX is an open-source fork of MongoDB that provides support for transactions in single-sharded environments.
TokuMX supports all MongoDB features as of version 2.4 and adds beginTransaction
, rollbackTransaction
, and
commitTransaction
commands.
$ brew tap tokutek/tokumx
$ brew install tokumx-bin
$ apt-key adv --keyserver keyserver.ubuntu.com --recv-key 505A7412
$ echo "deb [arch=amd64] http://s3.amazonaws.com/tokumx-debs $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/tokumx.list
$ apt-get update
$ apt-get install tokumx
TokuMX and MongoDB use different binary formats. To migrate data from MongoDB to TokuMX:
- Back up the MongoDB data
invoke mongodump --path dump
- Shut down the MongoDB server
- Uninstall MongoDB
- Install TokuMX (see instructions above)
- Restore the data to TokuMX
invoke mongorestore --path dump/osf20130903 --drop
- Verify that the migrated data are available in TokuMX
- Install RabbitMQ. On MacOSX with homebrew,
$ brew update
$ brew install rabbitmq
The scripts are installed to /usr/local/sbin
, so you may need to add PATH=$PATH:/usr/local/sbin
to your .bash_profile
.
For instructions for other OS's, see the official docs.
Then start the RabbitMQ server with
$ invoke rabbitmq
If you want the rabbitmq server to start every time you start your computer (MacOSX), run
$ ln -sfv /usr/local/opt/rabbitmq/*.plist ~/Library/LaunchAgents
$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.rabbitmq.plist
invoke celery_worker
- Install Elasticsearch
$ brew install elasticsearch
note: Oracle JDK 7 must be installed for elasticsearch to run
$ wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.2.1.deb
$ sudo dpkg -i elasticsearch-1.2.1.deb
- In your
website/settings/local.py
file, setSEARCH_ENGINE
to 'elastic'.
SEARCH_ENGINE = 'elastic'
- Start the Elasticsearch server and migrate the models.
$ invoke elasticsearch
$ invoke migrate_search
$ invoke elasticsearch
The Node Package Manager (NPM) is required for installing a number of node-based packages.
# For MacOSX
$ brew update && brew install node
Installing Node on Ubuntu is slightly more complicated. Node is installed as nodejs
, but Bower expects
the binary to be called node
. Symlink nodejs
to node
to fix, then verify that node
is properly aliased:
# For Ubuntu
$ sudo apt-get install nodejs
$ sudo ln -s /usr/bin/nodejs /usr/bin/node
$ node --version # v0.10.25
To install necessary NPM requiremnts, run:
$ npm install
In the OSF root directory. This will install a number libraries for both the front-end and for building the assets (e.g. webpack).
Use the following command to update your requirements and build the asset bundles:
$ inv assets -dw
To download citation styles, run:
$ invoke update_citation_styles
To install the python libraries needed to support the enabled addons, run:
$ invoke addon_requirements
Many addons require application credentials (typically an app key and secret) to be able to authenticate through the OSF. These credentials go in each addon's local.py
settings file (e.g. website/addons/dropbox/settings/local.py
).
For local development, the COS provides test app credentials for a number of services. A listing of these can be found here: https://osf.io/m2hig/wiki/home/ .
If you have all the above services installed, you can start everything up with this sequence
invoke mongo -d # Runs mongod as a daemon
invoke mailserver
invoke rabbitmq
invoke celery_worker
invoke elasticsearch
invoke assets -dw
invoke server