Machine Learning for iOS

Last Update: June 17, 2017.

Curated list of resources for iOS developers in following topics:

Core ML
Machine Learning Libraries
Deep Learning Libraries
- Deep Learning: Model Compression
Computer Vision
Natural Language Processing
Speech Recognition (TTS) and Generation (STT)
Text Recognition (OCR)
Other AI
Machine Learning Web APIs
Opensource ML Applications
Game AI
Other related staff
iOS ML Blogs
GPU Computing Blogs
Learn Machine Learning
Other Lists

Most of the de-facto standard tools in domains listed above are written in iOS-unfriendly languages (Python/Java/R/Matlab) so finding something appropriate for your iOS application may be a challenging task.

This list consists mainly of libraries written in Objective-C, Swift, C, C++, JavaScript and some other languages if they can be easily ported to iOS. Also, links to some relevant web APIs, blog posts, videos and learning materials included.

Resources are sorted alphabetically or randomly. The order doesn't reflect my personal preferences or anything else. Some of the resources are awesome, some are great, some are fun, and some can serve as an inspiration.

Have fun!

Pull-requests are welcome here.

Core ML

coremltools is a Python package. It contains converters from some popular machine learning libraries to the Apple format.
Core ML is an Apple framework to run inference on device. It is highly optimized to Apple hardware.

Currently CoreML is compatible (partially) with the following machine learning packages:

General-Purpose Machine Learning Libraries

FANN - Fast Artifical Neural Network library; an implementation of neural networks.
lbimproved - DTW + kNN in C
Shark - provides libraries for the design of adaptive systems, including methods for linear and nonlinear optimization (e.g., evolutionary and gradient-based algorithms), kernel-based algorithms and neural networks, and other machine learning techniques. CocoaPods. Official site
YCML - A Machine Learning framework for Objective-C and Swift (OS X / iOS). The following algorithms are currently available: Gradient Descent Backpropagation, Resilient Backpropagation (RProp), Extreme Learning Machines (ELM), Forward Selection using Orthogonal Least Squares (for RBF Net), also with the PRESS statistic, Binary Restricted Boltzmann Machines (CD & PCD, Untested!). YCML also contains some optimization algorithms as support for deriving predictive models, although they can be used for any kind of problem: Gradient Descent (Single-Objective, Unconstrained), RProp Gradient Descent (Single-Objective, Unconstrained), NSGA-II (Multi-Objective, Constrained).
- Github.
Swix - Swift implementation of NumPy.
Brain - Neural networks in JavaScript. Unmaintained.
April-ANN - APRIL-ANN toolkit (A Pattern Recognizer In Lua with ANNs). This toolkit incorporates ANN algorithms (as dropout, stacked denoising auto-encoders, convolutional NNs), with other pattern recognition methods as HMMs among others. Additionally, in experimental stage, it is possible to perform automatic differentiation, for advanced ML research. Potentially can be ported for iOS.
- Official Site.
Recommender - A C library for product recommendations/suggestions using collaborative filtering (CF).
SNNeuralNet - A neural network library for Objective-C based on brain.js, for iOS and Mac OS X.
MLPNeuralNet - Fast multilayer perceptron neural network library for iOS and Mac OS X. MLPNeuralNet predicts new examples by trained neural network. It is built on top of the Apple's Accelerate Framework, using vectorized operations and hardware acceleration if available.
MAChineLearning - An Objective-C multilayer perceptron library, with full support for training through backpropagation. Implemented using vDSP and vecLib, it's 20 times faster than its Java equivalent. Includes sample code for use from Swift.
Swift-AI - 3-layer NN.
EERegression - General purpose multivaritate and quadratic Regression library for Swift 2.1
SwiftSimpleNeuralNetwork - Feed forward and back propagation.
STEM - Swift Tensor Engine for Machine-learning
- Documentation
multilinear-math - Swift library for tensor operations, machine learning and artificial intelligence
MLKit - regressions, cross-validation, data splitting.
dlib is a C++ framework which contains 30+ machine learning algorithms and different useful additions. It's relatively easy to port it to iOS: you just need to get rid of GUI and http-related source files.
Mendel - Genetic algorithms in Swift.
Kalvar Lin's libraries
- ios-BPN-NeuralNetwork - 3-layer neural network + back propagation.
- ios-Multi-Perceptron-NeuralNetwork - Multi-perceptron neural network based on back propagation NN.
- ios-KRDelta - Delta Learning method
- ios-KRHebbian-Algorithm - Hebbian Theory
- ios-KRKmeans-Algorithm - K-Means clustering method.
- ios-KRFuzzyCMeans-Algorithm - Fuzzy C-Means, the fuzzy clustering algorithm.
- ios-KRGreyTheory - Grey Theory / Grey system theory-based models in time series prediction
- ios-KRSVM - Support Vector Machine and SMO.
- ios-KRKNN - kNN implementation.
- ios-KRRBFNN - Radial basis function neural network and OLS.
- ios-KRPerceptron - Perceptron method.
- ios-ML-Recommendation-System - demo of a book recommendation system, built using ios-BPN-NeuralNetwork.

Deep Learning Libraries

Bender - Framework for building fast NNs. Supports TensorFlow models. It uses Metal under the hood.
Birdbrain - RNNs and FF NNs on top of Metal and Accelerate. Not ready for production.
BNNS - Apple Basic neural network subroutines (BNNS) is a collection of functions that you use to implement and run neural networks, using previously obtained training data.
- BNNS usage examples in iOS 10 sampler.
- An example of a neural network trained by tensorflow and executed using BNNS
BrainCore - simple but fast neural network framework written in Swift. It uses Metal framework to be as fast as possible. ReLU, LSTM, L2 ...
Caffe - A deep learning framework developed with cleanliness, readability, and speed in mind. GitHub. [BSD]
- iOS port
- caffe-mobile - another iOS port.
- C++ examples: Classifying ImageNet, Extracting Features
- Caffe iOS sample
Caffe2 - a cross-platform framework made with expression, speed, and modularity in mind.
- Cocoa Pod
- iOS demo app
Convnet.js - ConvNetJS is a Javascript library for training Deep Learning models by Andrej Karpathy. GitHub
- ConvNetSwift - Swift port [work in progress].
Deep Belief SDK - The SDK for Jetpac's iOS Deep Belief image recognition framework
DeepLearningKit - Open Source Deep Learning Framework from Memkite for Apple's tvOS, iOS and OS X.
Espresso - A minimal high performance parallel neural network framework running on iOS.
Forge - A neural network toolkit for Metal.
KSJNeuralNetwork - A Neural Network Inference Library Built atop BNNS and MPS
- Converter for Torch models
MetalPerformanceShaders - CNNs on GPU from Apple.
- MetalCNNWeights - a Python script to convert Inception v3 for MPS.
- MPSCNNfeeder - Keras to MPS models conversion.
MXNet - MXNet is a deep learning framework designed for both efficiency and flexibility.
- Deploying mxnet to smartphone
NNPACK - Acceleration package for neural networks on multi-core CPUs. Prisma uses this library in the mobile app.
Quantized-CNN - compressed convolutional neural networks for Mobile Devices
TensorFlow - an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.
- iOS examples
- another example
- Perfect-TensorFlow - TensorFlow binding for Perfect (server-side Swift framework). Includes only C TF API.
tiny-dnn - header only, dependency-free deep learning framework in C++11.
- iOS example
Torch is a scientific computing framework with wide support for machine learning algorithms.
- Torch4iOS
- Torch-iOS

Deep Learning: Model Compression

TensorFlow implementation of knowledge distilling method
MobileNet-Caffe - Caffe Implementation of Google's MobileNets

Computer Vision

ccv - C-based/Cached/Core Computer Vision Library, A Modern Computer Vision Library
- iOS demo app
OpenCV – Open Source Computer Vision Library. [BSD]
OpenFace – a state-of-the art open source tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
- iOS port
- iOS demo
trackingjs – Object tracking in JS
Vision is an Apple framework for computer vision.

Natural Language Processing

CoreLinguistics - POS tagging (HMM), ngrams, Naive Bayes, IBM alignment models.
GloVe Swift package. Vector words representations.
NSLinguisticTagger
Parsimmon
Twitter text - An Objective-C implementation of Twitter's text processing library. The library includes methods for extracting user names, mentions headers, hashtags, and more – all the tweet specific language syntax you could ever want.
Verbal expressions for Swift, like regexps for humans.
Word2Vec - Original C implementation of Word2Vec Deep Learning algorithm. Works on iPhone like a charm.

Speech Recognition (TTS) and Generation (STT)

Kaldi-iOS framework - on-device speech recognition using deep learning.
- Proof of concept app
MVSpeechSynthesizer
OpenEars™: free speech recognition and speech synthesis for the iPhone - OpenEars™ makes it simple for you to add offline speech recognition and synthesized speech/TTS to your iPhone app quickly and easily. It lets everyone get the great results of using advanced speech UI concepts like statistical language models and finite state grammars in their app, but with no more effort than creating an NSArray or NSDictionary.
- Tutorial (Russian)
TLSphinx, Tutorial

Text Recognition (OCR)

ocrad.js - JS OCR
Tesseract

Other AI

Axiomatic - Swift unification framework for logic programming.
Build Your Own Lisp In Swift
Logician - Logic programming in Swift
Swiftlog - A simple Prolog-like language implemented entirely in Swift.

Machine Learning Web APIs

IBM Watson - Enable Cognitive Computing Features In Your App Using IBM Watson's Language, Vision, Speech and Data APIs.
- Introducing the (beta) IBM Watson iOS SDK
AlchemyAPI - Semantic Text Analysis APIs Using Natural Language Processing. Now part of IBM Watson.
Microsoft Project Oxford
Google Prediction engine
- Objective-C API
Google Translate API
Google Cloud Vision API
Amazon Machine Learning - Amazon ML is a cloud-based service for developers. It provides visualization tools to create machine learning models. Obtain predictions for application using APIs.
- iOS developer guide.
- iOS SDK
PredictionIO - opensource machine learning server for developers and ML engineers. Built on Apache Spark, HBase and Spray.
- Swift SDK
- Tapster iOS Demo - This demo demonstrates how to use the PredictionIO Swift SDK to integrate an iOS app with a PredictionIO engine to make your mobile app more interesting.
- Tutorial on using Swift with PredictionIO.
Wit.AI - NLP API
Yandex SpeechKit Text-to-speech and speech-to-text for Russian language. iOS SDK available.
Abbyy OCR SDK
Clarifai - deep learning web api for image captioning. iOS starter project
MetaMind - deep learning web api for image captioning.
Api.AI - Build intelligent speech interfaces for apps, devices, and web
CloudSight.ai - deep learning web API for fine grained object detection or whole screen description, including natural language object captions. Objective-C API client is available.

Opensource ML Applications

Deep Learning

DeepDreamer - Deep Dream application
DeepDreamApp - Deep Dream Cordova app.
Texture Networks, Lua implementation
Feedforward style transfer, Lua implementation
TensorFlow implementation of Neural Style
Corrosion detection app
ios_camera_object_detection - Realtime mobile visualize based Object Detection based on TensorFlow and YOLO model
TensorFlow MNIST iOS demo - Getting Started with Deep MNIST and TensorFlow on iOS
Drummer App with RNN and Swift
What'sThis
enVision - Deep Learning Models for Vision Tasks on iOS\
GoogLeNet on iOS demo
Neural style in Android
mnist-bnns - TensorFlow MNIST demo port to BNNS
Benchmark of BNNS vs. MPS
VGGNet on Metal
A Sudoku Solver that leverages TensorFlow and iOS BNNS for deep learning.

Traditional Computer Vision

SwiftOCR
GrabCutIOS - Image segmentation using GrabCut algorithm for iOS

NLP

Classical ELIZA chatbot in Swift
InfiniteMonkeys - A Keras-trained RNN to emulate the works of a famous poet, powered by BrainCore

Other

Game AI

Introduction to AI Programming for Games
dlib is a library which has many useful tools including machine learning.
MicroPather is a path finder and A* solver (astar or a-star) written in platform independent C++ that can be easily integrated into existing code.
Here is a list of some AI libraries suggested on OGRE3D website. Seems they are mostly written in C++.
GameplayKit Programming Guide

Other related staff

Linear algebra

Accelerate-in-Swift - Swift example codes for the Accelerate.framework
cuda-swift - Swift binding to CUDA. Not iOS, but still interesting.
Dimensional - Swift matrices with friendly semantics and a familiar interface.
Eigen - A high-level C++ library of template headers for linear algebra, matrix and vector operations, numerical solvers and related algorithms. [MPL2]
Matrix - convenient matrix type with different types of subscripts, custom operators and predefined matrices. A fork of Surge.
NDArray - Float library for Swift, accelerated with Accelerate Framework.
Swift-MathEagle - A general math framework to make using math easy. Currently supports function solving and optimisation, matrix and vector algebra, complex numbers, big int, big frac, big rational, graphs and general handy extensions and functions.
SwiftNum - linear algebra, fft, gradient descent, conjugate GD, plotting.
Surge from Mattt
Upsurge - generic tensors, matrices on top of Accelerate. A fork of Surge.
YCMatrix - A flexible Matrix library for Objective-C and Swift (OS X / iOS)

Statistics, random numbers

SigmaSwiftStatistics - A collection of functions for statistical calculation written in Swift.
SORandom - Collection of functions for generating psuedorandom variables from various distributions
RandKit - Swift framework for random numbers & distributions.

Mathematical optimization

fmincg-c - Conjugate gradient implementation in C
libLBFGS - a C library of Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS)
SwiftOptimizer - QuantLib Swift port.

Feature extraction

IntuneFeatures framework contains code to generate features from audio files and feature labels from the respective MIDI files.
matchbox - Mel-Frequency-Cepstral-Coefficients and Dynamic-Time-Warping for iOS/OSX. Warning: the library was updated last time when iOS 4 was still hot.
LibXtract is a simple, portable, lightweight library of audio feature extraction functions.

Data Visualization

Charts - The Swift port of the MPAndroidChart.
iOS-Charts
Core Plot
Awesome iOS charts
JTChartView
VTK
- VTK in action
D3.js iOS binding

Bioinformatics (kinda)

BioJS - a set of tools for bioinformatics in the browser. BioJS builds a infrastructure, guidelines and tools to avoid the reinvention of the wheel in life sciences. Community builds modules than can be reused by anyone.
BioCocoa - BioCocoa is an open source OpenStep (GNUstep/Cocoa) framework for bioinformatics written in Objective-C. [Dead project].
iBio - A Bioinformatics App for iPhone.

Big Data (not really)

HDF5Kit - This is a Swift wrapper for the HDF5 file format. HDF5 is used in the scientific comunity for managing large volumes of data. The objective is to make it easy to read and write HDF5 files from Swift, including playgrounds.