How to Create a Cartoonizer with TensorFlow Lite?

AI for Art and Design

Written by ML GDEs Margaret Maynard-Reid and Sayak Paul

This is an end-to-end tutorial on how to convert a TensorFlow model to TensorFlow Lite (TFLite) and deploy it to an Android app to cartoonize an image captured by the camera.

We created this end-to-end tutorial to help developers with these objectives:

• Provide a reference for the developers looking to convert models written in TensorFlow 1.x to their TFLite variants using the new features of the latest (v2) converter — for example, the MLIR-based converter, more supported ops, and improved kernels, etc.
  (In order to convert TensorFlow 2.x models in TFLite please follow this guide.)
• How to download the .tflite models directly from TensorFlow Hub if you are only interested in using the models for deployment.
• Understand how to use the TFLite tools such as the Android Benchmark Tool, Model Metadata, and Codegen.
• Guide developers on how to create a mobile application with TFLite models easily, with ML Model Binding feature from Android Studio.

The project repo contains notebooks for saving and converting to .tflite models and the Android code (learn more about the SavedModel format on the TensorFlow doc). The TFLite models are also available for download directly from TensorFlow Hub here.

White-box CartoonGAN (by Xinrui Wang and Jinze Yu) transforms an input image (preferably a natural image) to its cartoonized representation. This tutorial uses the generator of the White-box CartoonGAN for inference in the Android app.

Create the TensorFlow Lite Model

The authors of White-box CartoonGAN provide pre-trained weights that can be used for making inference on images. We convert these pre-trained weights to TFLite which are more suitable to run on a mobile app. Refer to the details on model conversion on GitHub here.

Step-by-step summary of this section:

• Generate a SavedModel out of the pre-trained model checkpoints.
• Convert SavedModel with post-training quantization using the latest TFLiteConverter.