Unable to Import and Use FlexDelegate in TensorFlowLite in iOS about tensorflow

[sushreebarsa]

@tanpengshi The FlexDelegate functionality is integrated within the TensorFlow Lite interpreter itself when built with Select Ops enabled. Could you please upgrade to the latest TF version and let us know as the older TF versions are not actively supported. Thank you!

[tanpengshi]

I have tried invoking the interpreter and I get the error

        guard let interpreter = tflite else {
            print("TFLite Error: Interpreter is nil.")
            return
        }
        do {
            let inputData = Data(buffer: UnsafeBufferPointer(start: dataBuffer, count: dataBuffer.count))
            try interpreter.copy(inputData, toInputAt: 0)
            try interpreter.invoke()

            let output = getTensorOutput(interpreter: interpreter)
            let detectedActionIndex = output.argmax()
            print("Detected Action Index is: \(detectedActionIndex)")
        } catch {
            print("TFLite Error: \(error.localizedDescription)")
        }

TensorFlow Lite Error: Select TensorFlow op(s), included in the given model, is(are) not supported by this interpreter. Make sure you apply/link the Flex delegate before inference. For the Android, it can be resolved by adding "org.tensorflow:tensorflow-lite-select-tf-ops" dependency. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TensorFlow Lite Error: Node number 76 (FlexTensorListReserve) failed to prepare.
TFLite Error: Must call allocateTensors().

I have already "import TensorFlowLiteSelectTfOps" in my Swift file.

I have accidentally deleted the CoreMLDelegate and also the MetalDelegate from the library. Would it cause this error? I have deleted these libraries and I don't know how to recover them. When I do:

rm -rf Pods Podfile.lock
pod install

The files are still missing. I really appreciate your help! I have come very close to the solution for my App!

[sushreebarsa]

@tanpengshi Directly recovering deleted libraries from within your project isn't possible. Could you try to use the latest TFlite version as newer versions might include these delegates within the framework itself, eliminating the need for separate libraries. After reinstalling/updating, clean and rebuild your project to ensure the changes take effect.

Thank you!

[tanpengshi]

Are you suggesting that I recreate a new project?

Now i proceed to try a more conventional method using the Podfile:

# Uncomment the next line to define a global platform for your project
platform :ios, '17.0'

target 'FacialRecognition' do
  # Comment the next line if you don't want to use dynamic frameworks
  use_frameworks!
  #pod 'TensorFlowLiteSwift', :path => '../../local-podspecs/TensorFlowLiteSwift.podspec'
  # pod 'TensorFlowLiteSelectTfOps', :path => '../../local-podspecs/TensorFlowLiteSelectTfOps.podspec'
  # pod 'TensorFlowLiteSwift'   # or 'TensorFlowLiteObjC'
  # pod 'TensorFlowLiteSelectTfOps', '~> 0.0.1-nightly'


  pod 'TensorFlowLiteSwift'
  pod 'TensorFlowLiteSelectTfOps', '~> 0.0.1-nightly'

  # Pods for FacialRecognition

  target 'FacialRecognitionTests' do
    inherit! :search_paths
    # Pods for testing
  end

  target 'FacialRecognitionUITests' do
    # Pods for testing
  end

  # Add these lines to ensure consistent EXCLUDED_ARCHS settings
  post_install do |installer|
    installer.pods_project.targets.each do |target|
      target.build_configurations.each do |config|
        config.build_settings['EXCLUDED_ARCHS[sdk=iphonesimulator*]'] = 'arm64'
      end
    end
  end

end

But when I build my project, I get:

When I check the 'Pods' directory, I don't see a 'resources-to-copy-FacialRecognition' file

[tanpengshi]

I am now able to solved the issue above and the TensorFlow Lite model with SelectOps is able to successfully run by following the suggestion here:

https://stackoverflow.com/questions/76792138/sandbox-bash72986-deny1-file-write-data-users-xxx-ios-pods-resources-to-co

However my app size is over 200MB because of the library! Hence, I need to do a Selective Build which effectively return me to the first solution. In my Podfile, I did for the first solution:

platform :ios, '17.0'

target 'FacialRecognition' do
  # Comment the next line if you don't want to use dynamic frameworks
  use_frameworks!
  pod 'TensorFlowLiteSwift', :path => '../../local-podspecs/TensorFlowLiteSwift.podspec'

  # Pods for FacialRecognition

  target 'FacialRecognitionTests' do
    inherit! :search_paths
    # Pods for testing
  end

  target 'FacialRecognitionUITests' do
    # Pods for testing
  end

  # Add these lines to ensure consistent EXCLUDED_ARCHS settings
  post_install do |installer|
    installer.pods_project.targets.each do |target|
      target.build_configurations.each do |config|
        config.build_settings['EXCLUDED_ARCHS[sdk=iphonesimulator*]'] = 'arm64'
      end
    end
  end

end

In my TensorFlowLiteSwift.podspec file, I have:

Pod::Spec.new do |s|
  s.name             = 'TensorFlowLiteSwift'
  s.version          = '2.7.0'
  s.authors          = 'Google Inc.'
  s.license          = { :type => 'Apache' }
  s.homepage         = 'https://github.com/tensorflow/tensorflow'
  s.source           = { :git => 'https://github.com/tensorflow/tensorflow.git', :tag => "v#{s.version}" }
  s.summary          = 'TensorFlow Lite for Swift'
  s.description      = <<-DESC

  TensorFlow Lite is TensorFlow's lightweight solution for Swift developers. It
  enables low-latency inference of on-device machine learning models with a
  small binary size and fast performance supporting hardware acceleration.
                       DESC

  s.ios.deployment_target = '17.0'

  s.module_name = 'TensorFlowLite'
  s.static_framework = true

  tfl_dir = 'tensorflow/lite/'
  swift_dir = tfl_dir + 'swift/'

  s.default_subspec = 'Core'

  s.subspec 'Core' do |core|
    # Adjust the path to point to your custom frameworks
    core.vendored_frameworks = [
      'frameworks/TensorFlowLiteC.framework',
      'frameworks/TensorFlowLiteSelectTfOps.framework'
    ]
    
    core.source_files = swift_dir + 'Sources/*.swift'
    core.exclude_files = swift_dir + 'Sources/{CoreML,Metal}Delegate.swift'

    core.test_spec 'Tests' do |ts|
      ts.source_files = swift_dir + 'Tests/*.swift'
      ts.exclude_files = swift_dir + 'Tests/MetalDelegateTests.swift'
      ts.resources = [
        tfl_dir + 'testdata/add.bin',
        tfl_dir + 'testdata/add_quantized.bin',
      ]
    end
  end

  s.subspec 'CoreML' do |coreml|
    coreml.source_files = swift_dir + 'Sources/CoreMLDelegate.swift'
    coreml.dependency 'TensorFlowLiteSwift/Core', "#{s.version}"
  end

  s.subspec 'Metal' do |metal|
    metal.source_files = swift_dir + 'Sources/MetalDelegate.swift'
    metal.dependency 'TensorFlowLiteSwift/Core', "#{s.version}"

    metal.test_spec 'Tests' do |ts|
      ts.source_files = swift_dir + 'Tests/{Interpreter,MetalDelegate}Tests.swift'
      ts.resources = [
        tfl_dir + 'testdata/add.bin',
        tfl_dir + 'testdata/add_quantized.bin',
        tfl_dir + 'testdata/multi_add.bin',
      ]
    end
  end
end

But using this method, with MetalDelegate and CoreMLDelegate libraries deleted, I have FlexDelegate not found error.

[sushreebarsa]

@tanpengshi Thank you for the update. Glad the issue has been resolved.
Could you please let us know if we can close the issue?
Thank you!

[tanpengshi]

Sure, we can close this issue, and I will open a next one regarding the new issue

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[sushreebarsa]

@tanpengshi Thank you!