To enable auto rebuild, there has to be some steps done prior that is required as mentioned below:

​
Additional Requirements in AWS Sagemaker for CI/CD

  • The model should be present in a model group and should have a model package.

  • An event bridge webhook is then required to relay the model package updates to Inferless.

  • The following steps will illustrate how we can create a model group, model package ,create an event bridge and approve a model version using the boto3 python library.

​
1. Create a model package group

  • To create a model group by using Boto3, call the create_model_package_group method and specify a name and description as parameters.

  • The response from the create_model_package_group call is the Amazon Resource Name (ARN) of the new model package group.

  • The following example shows how to create a model package group.

sm_client = boto3.client("sagemaker")

model_package_group_input_dict = {
    "ModelPackageGroupName": "",
    "ModelPackageGroupDescription": "",
}

client.create_model_package_group(
    **model_package_group_input_dict
)

Reference: 1) https://.aws.amazon.com/sagemaker/latest/dg/model-registry-model-group.html

​
2. Create a model package version

  • To register a model version by using Boto3, call the create_model_package method.

  • First, you set up the parameter dictionary to pass to the create_model_package method.

  • Then you call the create_model_package method, passing in the parameter dictionary that you just set up.

  • The following example shows the above steps mentioned:

sm_client = boto3.client("sagemaker")

model_url = ""

modelpackage_inference_specification = {
    "InferenceSpecification": {
        "Containers": [
            {
                "Image": "",
                "ModelDataUrl": model_url,
            }
        ],
        "SupportedContentTypes": ["text/csv"],
        "SupportedResponseMIMETypes": ["text/csv"],
    }
}

create_model_package_input_dict = {
    "ModelPackageGroupName": model_package_group_name,
    "ModelPackageDescription": "",
    "ModelApprovalStatus": "PendingManualApproval",
}

create_model_package_input_dict.update(modelpackage_inference_specification)

sm_client.create_model_package(
    **create_model_package_input_dict
)

Reference:https://docs.aws.amazon.com/sagemaker/latest/dg/model-registry-version.html

Post this, you will get a new ARN link generated by running the code below:

create_model_package_response = sm_client.create_model_package(**create_model_package_input_dict)
model_package_arn = create_model_package_response["ModelPackageArn"]
print('ModelPackage Version ARN : {}'.format(model_package_arn))

Note the ARN printed as you will be required to input this into Inferless as part of **"Input Package URL"**

​
3. Create an AWS Event Bridge Rule and API destination.

  • You need to set up an AWS Event Bridge to monitor the AWS events and send a webhook to Inferless whenever there is an update to the package.

  • You can view the video below to understand how to create one.

scrnli_28_02_2023_16-48-52.webm

22MB

Binary

View the video to learn to create a Event bridge

​
4. Approving a model package version

  • Setting the status to Approved can initiate CI/CD deployment for the model.

  • The following code snippet shows how to manually change the approval status to Approved.

sm_client = boto3.client("sagemaker")

model_package_update_input_dict = {
    "ModelPackageArn" : "",
    "ModelApprovalStatus" : "Approved"
}

sm_client.update_model_package(
    **model_package_update_input_dict
)

Reference: https://docs.aws.amazon.com/sagemaker/latest/dg/model-registry-approve.html

Once you have completed the above and saved your file, Inferless would be able to listen to web-hooks and create new versions automatically.

You can view a sample video example below:

Hugging FaceGoogle Vertex AI