finetuner.run module#

class finetuner.run.Run(client, name, experiment_name, config, created_at, description='', task='training', train_data=None)[source]#

Bases: object

Class for a run.

Parameters:

client (FinetunerV1Client) – Client object for sending api requests.
name (str) – Name of the run.
experiment_name (str) – Name of the experiment.
config (dict) – Configuration for the run.
created_at (str) – Creation time of the run.
description (str) – Optional description of the run.
train_data (Optional[str]) – The name of the DocumentArray created if this run is a data synthesis job.

property name: str#

Get the name of the Run.

Return type:: str

property config: dict#

Get the config of the Run.

Return type:: dict

property train_data: str#

Get the data generated by the Run In the case that it was a Synthesis job, if it is a training job, a ValueError is thrown.

Return type:: str

status()[source]#

Get Run status.

Return type:: dict
Returns:: A dict representing the Run status.

logs()[source]#

Check the Run logs.

Return type:: str
Returns:: A string dump of the run logs.

stream_logs(interval=5)[source]#

Stream the Run logs lively.

Parameters:: interval (int) – The time interval to sync the status of finetuner Run.
Yield:: An iterators keep stream the logs from server.
Return type:: Iterator[str]

metrics()[source]#

Get the evaluation metrics of the Run.

Return type:: Dict[str, Dict[str, float]]
Returns:: dictionary with evaluation metrics before and after fine-tuning.

display_metrics()[source]#: Prints a table of retrieval metrics before and after fine-tuning

example_results()[source]#

Get the results of example queries from the evaluation data of the Run.

Return type:: Dict[str, Any]
Returns:: dictionary with results before and after fine-tuning.

display_examples(k=5)[source]#

Prints a table of results of example queries before and after fine-tuning.

Parameters:: k (int) – maximal number of results per query to display

save_artifact(directory='artifacts/')[source]#

Save artifact if the Run is finished.

Parameters:: directory (str) – Directory where the artifact will be stored.
Return type:: str
Returns:: A string object that indicates the download path.

property artifact_id#

Get artifact id of the Run.

An artifact in finetuner contains fine-tuned model and its metadata. Such as preprocessing function, collate function. This id could be useful if you want to directly pull the artifact from the cloud storage, such as using FinetunerExecutor.

Returns:: Artifact id as string object.