Comments (21)
@bvallelunga there's no plan to add it soon. But maybe you can try to implement it yourself, by taking a look at some other layer!
from bender.
not what you were looking for - but distantly related albeit this is for server side stuff.
swift + tensorflow + custom models.
https://github.com/PerfectExamples/Perfect-TensorFlow-Demo-Vision
from bender.
@johndpope do you know if it provides iOS support and if the library size doesn't increase the apps binary size to much? Im capped at increasing our apps size by 5mb.
from bender.
@dernster any thoughts on how to run a simple non-image predict on Bender from a TensorFlow file?
from bender.
hi @bvallelunga - no ios support. the native c++ tensorflow stuff out of the box does have support for ios - as of the other week they now have a development cocoapod of 500mb. not sure how this impacts final build size. I've found it a challenge to do the most basic loading of custom models that are not image based though.
It's likely if there's demand that the swift wrapper that @RockfordWei created for desktop tensorflow could be adopted to fit ios down the track. though i suspect that there's no immediate plans. there's a complicated stripping and scaling down that happens when you build for ios, not all ops are supported.
from bender.
@johndpope that interesting, though 500mb is like crazy large. I am also having trouble getting a very basic model to work on bender. I am getting these errors:
"\n\n\nNodes in my graph (pre optimization):"
"inputs/x-input"
"model/W"
"model/MatMul"
"model/b"
"model/add"
"model/y_pred"
"inference/Greater/y"
"inference/Greater"
"inference/inference"
"\n\n\nNodes in my graph (after optimization):"
"inputs/x-input"
"model/W"
"model/b"
"model/add"
"inference/Greater/y"
"inference/Greater"
"inference/inference"
"Bender:: Unsupported layer found: Placeholder"
"Bender:: Unsupported layer found: Const"
"Bender:: Unsupported layer found: Const"
"Bender:: Unsupported layer found: Const"
"Bender:: Unsupported layer found: Greater"
"Bender:: Unsupported layer found: Cast"
This is the TensorFlow python script I used to generate the model:
# This script is used to train the model. It repeats indefinitely and saves the
# model every so often to a checkpoint.
#
# Press Ctrl+C when you feel that training has gone on long enough (since this is
# only a simple model it takes less than a minute to train, but a training a deep l
# earning model could take days).
import os
import numpy as np
import tensorflow as tf
checkpoint_dir = "/tmp/voice/"
print_every = 1000
save_every = 10000
num_inputs = 20
num_classes = 1
# Load the training data.
X_train = np.load("X_train.npy")
y_train = np.load("y_train.npy")
print("Training set size:", X_train.shape)
# Below we'll define the computational graph using TensorFlow. The different parts
# of the model are grouped into different "scopes", making it easier to understand
# what each part is doing.
# Hyperparameters let you configure the model and how it is trained. They're
# called "hyper" parameters because unlike the regular parameters they are not
# learned by the model -- you have to set them to appropriate values yourself.
#
# The learning_rate tells the optimizer how big of a steps it should take.
# Regularization is used to prevent overfitting on the training set.
with tf.name_scope("hyperparameters"):
regularization = tf.placeholder(tf.float32, name="regularization")
learning_rate = tf.placeholder(tf.float32, name="learning-rate")
# This is where we feed the training data (and later the test data) into the model.
# In this dataset there are 20 features, so x is a matrix with 20 columns. Its number
# of rows is None because it depends on how many examples at a time we put into this
# matrix. This is a binary classifier so for every training example, y gives a single
# output: 1 = male, 0 = female.
with tf.name_scope("inputs"):
x = tf.placeholder(tf.float32, [None, num_inputs], name="x-input")
y = tf.placeholder(tf.float32, [None, num_classes], name="y-input")
# The parameters that we'll learn consist of W, a weight matrix, and b, a vector
# of bias values. (Actually, b is just a single value since the classifier has only
# one output. For a classifier that can recognize multiple classes, b would have as
# many elements as there are classes.)
with tf.name_scope("model"):
W = tf.Variable(tf.zeros([num_inputs, num_classes]), name="W")
b = tf.Variable(tf.zeros([num_classes]), name="b")
# The output is the probability the speaker is male. If this is greater than
# 0.5, we consider the speaker to be male, otherwise female.
y_pred = tf.sigmoid(tf.matmul(x, W) + b, name="y_pred")
# This is a logistic classifier, so the loss function is the logistic loss.
with tf.name_scope("loss-function"):
loss = tf.losses.log_loss(labels=y, predictions=y_pred)
# Add L2 regularization to the loss.
loss += regularization * tf.nn.l2_loss(W)
# Use the ADAM optimizer to minimize the loss.
with tf.name_scope("train"):
optimizer = tf.train.AdamOptimizer(learning_rate)
train_op = optimizer.minimize(loss)
# For doing inference on new data for which we don't have labels.
with tf.name_scope("inference"):
inference = tf.to_float(y_pred > 0.5, name="inference")
# The accuracy operation computes the % correct on a dataset with known labels.
with tf.name_scope("score"):
correct_prediction = tf.equal(inference, y)
accuracy = tf.reduce_mean(tf.to_float(correct_prediction), name="accuracy")
init = tf.global_variables_initializer()
# For writing training checkpoints and reading them back in.
saver = tf.train.Saver()
tf.gfile.MakeDirs(checkpoint_dir)
with tf.Session() as sess:
# Write the graph definition to a file. We'll load this in the test.py script.
tf.train.write_graph(sess.graph_def, checkpoint_dir, "graph.pb", False)
# Reset W and b to zero.
sess.run(init)
# Sanity check: the initial loss should be 0.693146, which is -ln(0.5).
loss_value = sess.run(loss, feed_dict={x: X_train, y: y_train, regularization: 0})
print("Initial loss:", loss_value)
# Loop forever:
step = 0
while True:
# We randomly shuffle the examples every time we train.
perm = np.arange(len(X_train))
np.random.shuffle(perm)
X_train = X_train[perm]
y_train = y_train[perm]
# Run the optimizer over the entire training set at once. For larger datasets
# you would train in batches of 100-1000 examples instead of the entire thing.
feed = {x: X_train, y: y_train, learning_rate: 1e-2, regularization: 1e-5}
sess.run(train_op, feed_dict=feed)
# Print the loss once every so many steps. Because of the regularization,
# at some point the loss won't become smaller anymore. At that point, it's
# safe to press Ctrl+C to stop the training.
if step % print_every == 0:
train_accuracy, loss_value = sess.run([accuracy, loss], feed_dict=feed)
print("step: %4d, loss: %.4f, training accuracy: %.4f" % \
(step, loss_value, train_accuracy))
step += 1
# Save the model. You should only press Ctrl+C after you see this message.
if step % save_every == 0:
checkpoint_file = os.path.join(checkpoint_dir, "model")
saver.save(sess, checkpoint_file)
print("*** SAVED MODEL ***")
@dernster or @mats-claassen any thoughts?
from bender.
@bvallelunga Thanks. You can take a look at this test. Here we use an internal Texture
class that can be created from arrays, then it's converted to a MTLTexture
and finally to a MPSImage. In your particular case I see that your model has a "Greater" layer and we don't have support for it at this moment. We are open to receive pull request though :)
from bender.
@dernster Thank you! Yea I just noticed you don't support the Greater layer. Do you plan on adding support for it soon?
Also does code look good for building a MPSImage
?
extension MPSImage {
convenience init(input: [Float], device: MTLDevice) {
// The input data must be placed into the MTLTexture from the MPSImage.
// First we convert it to 16-bit floating point, then copy these floats
// into the texture slices. (Note: this will fail if the number of channels
// is not a multiple of 4 -- in that case you should add padding bytes to
// the example array.)
var input = input
let input16 = float32to16(&input, count: input.count)
let inputImgDesc = MPSImageDescriptor(channelFormat: .float16, width: 1, height: 1, featureChannels: input.count)
self.init(device: device, imageDescriptor: inputImgDesc)
input16.withUnsafeBufferPointer { ptr in
for i in 0..<self.texture.arrayLength {
let region = MTLRegion(origin: MTLOriginMake(0, 0, 0), size: MTLSizeMake(1, 1, 1))
self.texture.replace(region: region, mipmapLevel: 0, slice: i, withBytes: ptr.baseAddress!.advanced(by: i*4), bytesPerRow: MemoryLayout<Float16>.stride * 4, bytesPerImage: 0)
}
}
}
}
from bender.
@bvallelunga Not sure but you can see our Texture class. It already has that functionality and without the multiple of 4 restriction. Maybe we would need to add it to Bender, since right now it's at the Example project.
from bender.
@dernster would you be able to give me an example of using the functionality without the multiple of 4 restriction?
from bender.
Thanks, @johndpope for the comment. For TensorFlow iOS solution, however, I think CoreML maybe a better approach - you can import TensorFlow or even Keras models to CoreML. That's one of the most important reasons that I am not going to migrate Python core code into Swift right now without our sponsors' permission. Sorry about that.
from bender.
Thanks, @johndpope for the comment. For TensorFlow iOS solution, however, I think CoreML maybe a better approach - you can import TensorFlow or even Keras models to CoreML. That's one of the most important reasons that I am not going to migrate Python core code into Swift right now without our sponsors' permission. Sorry about that.
Take into account that Core ML doesn't support TensorFlow without Keras, it only supports versions 1.0.X and 1.1.x and it doesn't support adding custom layeers. Also, it works with iOS 11+, which hasn't been released.
from bender.
from bender.
@RockfordWei would the the completed framework support iOS and be lightweight (under 2mb)?
from bender.
from bender.
@RockfordWei what kind of sponsorship are you looking for?
from bender.
from bender.
@RockfordWei I have checked out the site, is there a specific page you would like me to look at?
from bender.
from bender.
Ahhh, no thank you. I am just a single developer working on a side project.
from bender.
@bvallelunga Sorry for the delay, here's an example:
Create your texture:
let texture = Texture(
data: [
[1.0, 2.0, 3.0, 4.0, 5.0, 6], [9.0, 10.0, 11.0, 12.0, 13.0, 14],
[17.0, 18, 19, 20, 21, 22], [25, 26, 27, 28, 29, 30],
[33, 34, 35, 36, 37, 38], [41, 42, 43, 44, 45, 46]
],
size: LayerSize(f: 6, w: 2, h: 3)
)
Here, each array corresponds to z-values.
Then, you need to create a metal texture:
let metalTexture = texture.metalTexture(with: device)
And finally a MPSImage
instance:
let mpsImage = MPSImage(texture: metalTexture, featureChannels: texture.depth)
After your network returns you can create a computedTexture from the MPSImage returned:
let computedTexture = Texture(metalTexture: returnedImage.texture, size: /* Layer size of returned texture */)
Take into account that the class Texture
isn't included in Bender, for now it's only on the Examples. Maybe you can just copy that class.
from bender.
Related Issues (20)
- Can't compile with Xcode 9.3, iOS 11.3 HOT 4
- Convolution updateWeights function needs to be modified. HOT 1
- Tests fail in iOS 11.3.1 on iPhone 8+ HOT 5
- Pod Issue HOT 1
- Get the weight of each layer of neural network
- macOS support HOT 1
- Unable to find a specification for `MetalPerformanceShadersProxy` HOT 4
- Upsample layer? HOT 3
- about depthwise_conv2d enhance HOT 1
- Cannot create ConcatV2. Missing or invalid attribute axis HOT 2
- how to make bender data loader much fast? HOT 1
- Documentation for efficient storage of models HOT 2
- iOS version requirement for Bender HOT 1
- Batch inference API HOT 1
- Swift 4 Compatibility HOT 1
- pod install, unable to satisfy the following requirements, `MetalBender (~> 0.4)` required by `Podfile` HOT 4
- fatal error: Cannot create ConcatV2. Missing or invalid attribute axis. HOT 3
- Example does not compile HOT 8
- How to use model with 2 input parameters? HOT 2
- Assertion error with Mobilenet on iOS 10 device HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bender.