azure.ai.ml.model_customization package

class azure.ai.ml.model_customization.EndpointRequestSettings(*, request_batch_size: int | None = None, min_endpoint_success_ratio: float | None = None)[source]

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

>

Initialize EndpointRequestSettings.

Parameters:

  • request_batch_size (Optional[int], optional) – The number of requests to send to the teacher model endpoint as a batch, defaults to None

  • min_endpoint_success_ratio (Optional[float], optional) – The ratio of (successful requests / total requests) needed for the data generation step to be considered successful. Must be a value between 0 and 1 inclusive, defaults to None

items()[source]
property min_endpoint_success_ratio: float | None

Get the minimum ratio of successful inferencing requests.

Returns:

The minimum ratio of successful inferencing requests.

Return type:

Optional[float]

property request_batch_size: int | None

Get the number of inference requests to send to the teacher model as a batch.

Returns:

The number of inference requests to send to the teacher model as a batch.

Return type:

Optional[int]

class azure.ai.ml.model_customization.PromptSettings(*, enable_chain_of_thought: bool = False, enable_chain_of_density: bool = False, max_len_summary: int | None = None)[source]

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

>

Initialize PromptSettings.

Parameters:

  • enable_chain_of_thought (bool, optional) – Whether or not to enable chain of thought which modifies the system prompt used. Can be used for all data_generation_task_type values except SUMMARIZATION, defaults to False

  • enable_chain_of_density (bool, optional) – Whether or not to enable chain of density which modifies the system prompt used. Can only be used for data_generation_task_type of SUMMARIZATION, defaults to False

  • max_len_summary (Optional[int]) – The maximum length of the summary generated for data_generation_task_type` of SUMMARIZATION, defaults to None

items()[source]
property enable_chain_of_density: bool

Get whether or not chain of density is enabled.

Returns:

Whether or not chain of thought is enabled

Return type:

bool

property enable_chain_of_thought: bool

Get whether or not chain of thought is enabled.

Returns:

Whether or not chain of thought is enabled.

Return type:

bool

property max_len_summary: int | None

The number of tokens to use for summarization.

Returns:

The number of tokens to use for summarization

Return type:

Optional[int]

class azure.ai.ml.model_customization.TeacherModelSettings(*, inference_parameters: Dict | None = None, endpoint_request_settings: EndpointRequestSettings | None = None)[source]

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

>

Initialize TeacherModelSettings

Parameters:

  • inference_parameters (Optional[Dict], optional) – The inference parameters inferencing requests will use, defaults to None

  • endpoint_request_settings (Optional[EndpointRequestSettings], optional) – The settings to use for the endpoint, defaults to None

items()[source]
property endpoint_request_settings: EndpointRequestSettings | None

Get the endpoint request settings.

Returns:

The endpoint request settings.

Return type:

Optional[EndpointRequestSettings]

property inference_parameters: Dict | None

Get the inference parameters.

Returns:

The inference parameters.

Return type:

Optional[Dict]

azure.ai.ml.model_customization.distillation(*, experiment_name: str, data_generation_type: str, data_generation_task_type: str, teacher_model_endpoint_connection: WorkspaceConnection, student_model: Input | str, training_data: Input | str | None = None, validation_data: Input | str | None = None, teacher_model_settings: TeacherModelSettings | None = None, prompt_settings: PromptSettings | None = None, hyperparameters: Dict | None = None, resources: ResourceConfiguration | None = None, **kwargs: Any) → DistillationJob[source]

Note

This is an experimental method, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Function to create a Distillation job.

A distillation job is used to transfer knowledge from a teacher model to student model by a two step process of generating synthetic data from the teacher model and then finetuning the student model with the generated synthetic data.

Parameters:

  • experiment_name (str) – The name of the experiment.

  • data_generation_type (str) –

    The type of data generation to perform.

    Acceptable values: label_generation

  • data_generation_task_type –

    The type of data to generate

    Acceptable values: NLI, NLU_QA, CONVERSATION, MATH, SUMMARIZATION

  • teacher_model_endpoint_connection – The kind of teacher model connection that includes the name, endpoint url, and api_key.

  • student_model (Union[Input, str]) – The model to train

  • training_data (Optional[Union[Input, str]], optional) – The training data to use. Should contain the questions but not the labels, defaults to None

  • validation_data (Optional[Union[Input, str]], optional) – The validation data to use. Should contain the questions but not the labels, defaults to None

  • teacher_model_settings (Optional[TeacherModelSettings], optional) –

    The settings for the teacher model. Accepts both the inference parameters and endpoint settings, defaults to None

    Acceptable keys for inference parameters: temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stop

  • prompt_settings (Optional[PromptSettings], optional) – The settings for the prompt that affect the system prompt used for data generation, defaults to None

  • hyperparameters (Optional[Dict], optional) – The hyperparameters to use for finetuning, defaults to None

  • resources (Optional[ResourceConfiguration], optional) – The compute resource to use for the data generation step in the distillation job, defaults to None

Type:

data_generation_task_type: str

Type:

teacher_model_endpoint_connection: WorkspaceConnection

Raises:

ValueError – Raises ValueError if there is no training data and data generation type is ‘label_generation’

Returns:

A DistillationJob to submit

Return type:

DistillationJob