How Does it Work?#
Contrastive metric learning#
From an algorithmic perspective, Finetuner leverages a contrastive metric learning approach to improve your model. How does it work?
Step 1: Convert a model into an embedding model#
Finetuner interprets the backbone model architecture, removes the default head, applies pooling and freezes layers that do not need to be trained. For an image classification task (e.g. cats and dogs), Finetuner is going to remove the classification head (cat-dog classifier) and turn your model into an embedding model.
This embedding model does not make predictions or outputs a probability, but instead outputs a feature vector to represent your data.
Step 2: Triplet construction and training on-the-fly#
During the fine-tuning, Finetuner creates Triplets
(anchor, positive, negative) on-the-fly.
For each anchor,
which can be any training example,
Finetuner looks for a
Document with the same
Document with a different
The objective is to pull
Documents which belong to the same class together,
while pushing the
Documents which belong to a different class away from each other.
From an engineering perspective, we have hidden all the complexity of machine learning algorithms and resource configuration (such as GPUs). All you need to do is decide on your backbone model and prepare your training data.
Once you logged into the Jina Ecosystem with
Finetuner will push your training data into our Cloud Artifact Storage (only visible to you).
At the same time, we will spin-up an isolated computational resource
with proper memory, CPU, GPU dedicated to your fine-tuning job.
Once fine-tuning is done, Finetuner will again push your fine-tuned model to the Cloud Artifact Storage and make it available for you to pull it back to your machine. That’s it!
On the other hand, if you have a certain level of machine learning knowledge, Finetuner gives you enough flexibility to adjust the training parameters. This will be explained in a later section.