Integration#
Fine-tuned model as Executor#
Once fine-tuning is finished, it’s time to actually use the model. Finetuner, being part of the Jina ecosystem, provides a convenient way to use tuned models via Jina Executors.
We’ve created the FinetunerExecutor
which can be added in a Jina Flow and load any tuned model.
More specifically, the executor exposes an /encode
endpoint that embeds Documents using the fine-tuned model.
Loading a tuned model is simple! You just need to provide a few parameters under the uses_with
argument when adding the FinetunerExecutor
to the Flow.
You have three options:
import finetuner
from jina import Flow
finetuner.login()
token = finetuner.get_token()
run = finetuner.get_run(
experiment_name='YOUR-EXPERIMENT',
run_name='YOUR-RUN'
)
f = Flow().add(
uses='jinahub+docker://FinetunerExecutor/v0.9.2', # use v0.9.2-gpu for gpu executor.
uses_with={'artifact': run.artifact_id, 'token': token},
)
from jina import Flow
f = Flow().add(
uses='jinahub+docker://FinetunerExecutor/v0.9.2', # use v0.9.2-gpu for gpu executor.
uses_with={'artifact': '/mnt/YOUR-MODEL.zip'},
volumes=['/your/local/path/:/mnt'] # mount your model path to docker.
)
jtype: Flow
with:
port: 51000
protocol: grpc
executors:
uses: jinahub+docker://FinetunerExecutor/v0.9.2
with:
artifact: 'COPY-YOUR-ARTIFACT-ID-HERE'
token: 'COPY-YOUR-TOKEN-HERE' # or better set as env
As you can see, it’s super easy!
If you did not call save_artifact
,
you need to provide the artifact_id
and token
.
FinetunerExecutor
will automatically pull your model from the cloud storage to the container.
On the other hand,
if you have saved artifact locally,
please mount the zipped artifact to the docker container.
FinetunerExecutor
will unzip the artifact and load models.
You can start your flow with:
with f:
# in this example, we fine-tuned a BERT model and embed a Document..
returned_docs = f.post(
on='/encode',
inputs=DocumentArray(
[
Document(
text='some text to encode'
)
]
)
)
for doc in returned_docs:
print(f'Text of the returned document: {doc.text}')
print(f'Shape of the embedding: {doc.embedding.shape}')
Text of the returned document: some text to encode
Shape of the embedding: (768,)
In order to see what other options you can specify when initializing the executor, please go to the FinetunerExecutor
page and click on Arguments
on the top-right side.
FinetunerExecutor parameters
The only required argument is artifact
. We provide default values for others.
Embed DocumentArray#
Similarly, you can embed the DocumentArray with fine-tuned model:
from docarray import DocumentArray, Document
import finetuner
finetuner.login()
token = finetuner.get_token()
run = finetuner.get_run(
experiment_name='YOUR-EXPERIMENT',
run_name='YOUR-RUN'
)
da = DocumentArray([Document(text='some text to encode')])
da.post(
'jinahub+docker://FinetunerExecutor/v0.9.2/encode',
uses_with={'artifact': run.artifact_id, 'token': token},
)
from docarray import DocumentArray, Document
da = DocumentArray([Document(text='some text to encode')])
da.post(
'jinahub+docker://FinetunerExecutor/v0.9.2/encode,
uses_with={'artifact': '/mnt/YOUR-MODEL.zip'},
volumes=['/your/local/path/:/mnt'] # mount your model path to docker.
)
Text of the returned document: some text to encode
Shape of the embedding: (768,)