In this project, we empirically evaluate the performance of two mobile DL frameworks TensorFlow Lite and CoreML , on the inference performance on Android and iOS devices, respectively, using various Convolution Neural Network Architectures.
- TensorFlow Lite (for Android)
- CoreML (for iOS)
- CPU:
- XNNPack Optimized [TF Lite, Android]
- GPU
- Apple Neural Engine (ANE) [CoreML, iOS]
- Neural Network API (NNAPI) [TF Lite, Android]
- In Android Studio, use the Open an Existing Project option and select the
Android-TFLite\TFLitePerformance
folder. - If you wish to use a custom TF-Lite model, copy the
.tflite
file to theapp\assets
folder. Also, update the path in the model's classifier java file (like ClassifierSqueezeNet.java), in the lib_support library. The path is present in thegetModelPath
function. - Use the Device File Explorer to upload the test data onto the device. Default location of data is
/data/local/tmp/DataSet/
. This can be modified by updating the following line of code at line 114 in the MainActivity.java.File dataset_folder = new File(<Dataset location on device>);
- Build the project using
Build > Make Project
. - Run the application on device/emulator using
Run > Run 'app'
. - If you wish to start profiling with application launch, use
Run > Profile 'app'
instead. - On Device/Emulator, ensure the correct model and device is selected.
- Click the
Benchmark
button on the mobile application to initiate the inferencing.
-
These metrics were recorded using Android Studio CPU Profiler.
For evluation purposes Frames Per second, captured in real time video is restriced to 50, following is the FPS processed by the model in real time.
Size vs Accuracy | Inference Time vs Accuracy |
---|---|
- The same AI model on different platform has different accuracy.
- On-board computation device (CPU, GPU) selection is highly dependent on the mobile device.
- Neural network optimized capabilities such as ANE (CoreML, iOS) and NNAPI (TFLite, Android) are not always the best choice for inferencing.
- There is a substantial trade-off between accuracy and throughput. This trade-off must be addressed as per the application requirements.