mangalbhaskar / deepsetup

scripts that will bring aimldl system up and running

AI Machine Setup Steps

A curated list of Shell Scripts, Resources, Libraries, Software for AI ML DL . Inspired by linuxscripts

Tested on Ubuntu 16.04 LTS and Ubuntu 18.04 LTS

Remote Access Setup (optional)

• Terminal commands for remote access

  sudo apt-get install openssh-server openssh-client dconf-editor

• Allow ubuntu to do automatic login from system settings panel
• gsettings for remote access:

  sudo gsettings set org.gnome.Vino require-encryption false

• Remote desktop sharing preferences

  sudo vino-preferences

• Check Allow other users to view desktop
• Uncheck You must confirm each access to this machine
• Check Require the user to enter this password: <password>

System Setup

1. Pre-requisites

## 1. Install `git`
sudo apt update
sudo apt install git
#
## 2. Clone linuxscripts
#
mkdir -p ${HOME}/softwares
git clone https://github.com/nikhilbv/deepsetup.git ${HOME}/softwares/deepsetup
alias dsetup="cd ${HOME}/softwares/deepsetup"
#
##3. Copy softwares from `samba5/softwares/packages-for-new-system-install` to local system under `$HOME/Downloads` manually, or use `rsync`
#
smbuser="nikhil"
smbserver="10.4.71.121"
remotepath="/data/samba/software/packages-for-new-system-install"
rsync -r ${smbuser}@${smbserver}:${remotepath} $HOME/Downloads

2. Nvidia driver installation

• WARNING:
  • Do not change/reinstall driver if it is already installed probably in new system
  • Check using command 'nvidia-smi'
  • Reboot happens once the nvidia graphics driver is installed

cd $HOME/softwares/deepsetup
source init-nvidia.sh

• To check the drivers

cd $HOME/softwares/deepsetup
source nvidia-driver-info.sh

3. Install cuda

• WARNING:
  • Add cuda repo key manually on error and again "source cuda.install.sh"
    • Keys for Nvidia CUDA signed repo

    gpg --keyserver keyserver.ubuntu.com --recv-keys F60F4B3D7FA2AF80
    gpg --export --armor F60F4B3D7FA2AF80 | sudo apt-key add -

    source cuda.install.sh

4. Run setup script

• Execute setup.sh which is a binding of all softwares

source setup.sh

* **WARNING:**
   * Setup apache configuration for userdir, wsgi
   * The entries are already in output in terminal
   * Open and edit those two files

• Optional Packages
  • Install cuDNN

    ## Check gcc --version compactibility
    gcc --version
    source cudnn.install.sh

  • Install tensorRT

    source tensorRT.install.sh

• Apache server test
  • Test the apache2, process in browser by:

  http://localhost/~<username>
  http://localhost/~<username>/info.php

• Python environment setup
  • Now install the python environment wrappers for both python2 and python3

  #
  ## For python2
  source python.virtualenvwrapper.install.sh 2
  #
  ## For python3
  source python.virtualenvwrapper.install.sh 3
  #
  ## `deactivate` is the command used to exit from environment
    deactivate

  • Install the required packages for AI only in python3 env:

    workon py_3 <and press tab key>
    pip install -r python.requirements-extras.txt
    pip install -r python.requirements-ai.txt

  • Note: If there is an interrupt and the installation fails, restart the installation

5. Setup AI Directories

• Clone the ai-ml-dl directory

  cd $HOME/Documents
  git clone [email protected]:/home/mapdata/software/git-repo/ai-ml-dl

• Clone MRCNN from github in external folder of aimldl

  cd $HOME/Documents/ai-ml-dl/external
    git clone https://github.com/mangalbhaskar/Mask_RCNN.git

• Copy the model data from AI server to local system

  ##  copy vidteq folder from _/home/alpha/ai-ml-dl-data/data/vidteq to same data structure
  source copydata.sh

• Setup the local AI paths and dnn configurations

  cd $HOME/Documents/ai-ml-dl
  source setup.sh

• Manual checks and changes to be done
  1. Change the ip in index.py i.e., $AI_HOME/www/public_html/od/wsgi-bin/index.py: app.run(debug=True, host='10.4.71.59')
  2. Create directory pixel in $AI_HOME/www/uploads: mkdir -p $AI_HOME/www/uploads/pixel