Finetuning Pretrained ResNet for Celebrity Face Search¶
For this example, you will need a GPU machine to enable the best experience.
The Finetuner will work in the following steps:
first, we spawn the Labeler that helps us to inspect the top-K visually similar celebrities face images from original ResNet;
then, with the Labeler UI we accept or reject the results based on their similarities;
finally, the results are collected at the backend by the Tuner, which “tunes” the ResNet and produces better search result.
Hopefully the procedure converges after several rounds; and we get a tuned embedding for better celebrity face search.
Prepare CelebA data¶
Beware that the original CelebA dataset is 1.3GB. In this example, we do not need the full dataset. Here is a smaller version which contains 1000 images from the original dataset. You can download it from here.
Note that Finetuner accepts Jina
DocumentArrayMemmap, so we first load CelebA data into this format using generator:
from jina.types.document.generators import from_files # please change the file path to your data path data = list(from_files('img_align_celeba/*.jpg', size=100, to_dataturi=True)) for doc in data: doc.load_uri_to_image_blob( height=224, width=224 ).set_image_blob_normalization().set_image_blob_channel_axis( -1, 0 ) # No need for changing channel axes line if you are using tf/keras
Load the pretrained model¶
Let’s import pretrained ResNet50 as our base model. ResNet50 is implemented in PyTorch, Keras and Paddle. You can choose whatever framework you feel comfortable:
import torchvision model = torchvision.models.resnet50(pretrained=True)
import tensorflow as tf model = tf.keras.applications.resnet50.ResNet50(weights='imagenet')
import paddle model = paddle.vision.models.resnet50(pretrained=True)
Finally, let’s start the Finetuner. Note that we freeze the weights of the original ResNet and tuned only for the last linear layer, which leverages Tailor underneath:
import finetuner finetuner.fit( model=model, interactive=True, train_data=data, freeze=True, to_embedding_model=True, input_size=(3, 224, 224), output_dim=100 )
Note that how we specify
to_embedding_model=True in the above code to activate Labler and Tailor, respectively.
input_size is not required when you using Keras as the backend.
After running the script, the browser will open the Labeler UI. You can now label the data by mouse/keyboard. The model will get trained and improved as you are labeling. If you are running this example on a CPU machine, it can take up to 20 seconds for each labeling round.
On the backend, you should be able to see the training procedure in the terminal.
[email protected][I]:🎉 Flow is ready to use! 🔗 Protocol: HTTP 🏠 Local access: 0.0.0.0:61622 🔒 Private network: 172.18.1.109:61622 🌐 Public address: 184.108.40.206:61622 💬 Swagger UI: http://localhost:61622/docs 📚 Redoc: http://localhost:61622/redoc UserWarning: ignored unknown argument: ['thread']. (raised from /Users/hanxiao/Documents/jina/jina/helper.py:685) ⠴ Working... ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:00 estimating... [email protected][I]:Finetuner is available at http://localhost:61622/finetuner ⠏ Working... ━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:00 0.0 step/s UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:180.) (raised from /Users/hanxiao/Documents/trainer/finetuner/labeler/executor.py:53) ⠦ DONE ━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:06 1.4 step/s 11 steps done in 6 seconds ⠙ DONE ━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:03 0.3 step/s T: Loss= 0.75