CI/CD for Deploying Machine Learning Models on AWS

by Mayank Chamoli

The following is an example of building a simple MLOps pipeline using AWS resources. The pipeline is built with data scientists in mind. A large portion of data scientist’s work revolves around understanding data, building models to solve problems and refining models based on new or updated parameters. It is important for data scientists to be able to complete these primary tasks without worrying about redeploying the model each time it is altered.

Jupyter Notebook Structure

An MLOps pipeline enables automatic model deployment to an endpoint where dependent applications can consume its output. Amazon SageMaker enables data scientists to perform model development, training, tuning, and deployment all from a single notebook instance. As a result, the notebook used for building the model can be leveraged to deploy to higher environments.

MLOps Architecture using AWS Resources

When a user commits the Jupyter notebook to CodeCommit, it triggers the CodePipeline. CodeCommit is configured as the source for CodePipeline. Once the CodePipeline starts, it invokes CodeBuild, which uses the buildspec file to package the notebook and copy its contents to an S3 bucket. As is shown in the image of the CodeCommit repository structure, the buildspec.yml is included within it.

The contents of the buildspec file are shown below for your reference.

version: 0.1

phases: 
  build: 
    commands: 
      - | 
         echo push artifacts to s3 
         zip -r notebook.zip . 
         aws s3 cp notebook.zip s3://s3-for-codepipeline/notebook/notebook.zip

artifacts: 
  files: 
    - "**/*"

Once the build phase in the CodePipeline is complete, the deploy phase is triggered. The deploy phase instructs the CloudFormation to delete any existing SageMaker instances before creating a new SageMaker instance using the latest code from the CodeCommit repository.

The stages in the CodePipeline and the CloudFormation template used in this case are shown below.

CodePipeline
|-Source (CodeCommit)
|-Build (CodeBuild)
|-Deploy (CloudFormation – StackName: sagemaker-test)
    |-delete_test_sagemaker
    |-deploy_test_sagemaker

sagemaker-template-test.yml

Description: "Forecast POC Notebook Instance"
Resources:
  ForecastPOCNotebookInstance:
    Type: "AWS::SageMaker::NotebookInstance"
    Properties:
      InstanceType: "ml.t2.xlarge"
      LifecycleConfigName: !Sub "${AWS::StackName}-lifecycle"
      RoleArn: !GetAtt ExecutionRole.Arn
  ExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          -
            Effect: "Allow"
            Principal:
              Service:
                - "sagemaker.amazonaws.com"
            Action:
                "sts:AssumeRole"
      Path: "/"
      Policies:
        -
          PolicyName: "root"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action: "*"
                Resource: "*"
Outputs:
  BasicNotebookInstanceId:
    Value: !Ref ForecastPOCNotebookInstance

The CloudFormation uses the lifecycle script (below) in SageMaker while creating the notebook instance. The script copies the latest code from the S3 bucket to the notebook instance and unzips it. After granting the necessary permissions to the directory, the Jupyter notebook is executed. When the Jupyter notebook runs, it executes all of the steps in it, then deploys the model. Finally, the model can be tested for accuracy.

sagemaker-test-lifecycle:

#!/bin/bash

set -e

su ec2-user

mkdir -p /home/ec2-user/SageMaker/notebook-test-deploy/
aws s3 cp s3://s3-for-codepipeline/notebook/notebook.zip /home/ec2-user/SageMaker/notebook.zip
unzip /home/ec2-user/SageMaker/notebook.zip -d /home/ec2-user/SageMaker/notebook-test-deploy/

sudo chown -R ec2-user:ec2-user /home/ec2-user/SageMaker/notebook-test-deploy

source /home/ec2-user/anaconda3/bin/activate python3
 
nohup jupyter nbconvert --execute /home/ec2-user/SageMaker/notebook-test-deploy/Breast\ Cancer\ Prediction.ipynb --ExecutePreprocessor.kernel_name=python3 --ExecutePreprocessor.timeout=7200 &

next steps

The CodePipeline process can be further expanded to deploy to production by adding new stages. Once the model testing is complete in the test environment, and the results are as expected, the approval can be given for deployment to the production environment. An example of how CodePipeline would look with additional stages is shown below.

CodePipeline
|-Source (CodeCommit)
|-Build (CodeBuild)
|-Deploy (CloudFormation – StackName: sagemaker-test)
    |-delete_test_sagemaker
    |-deploy_test_sagemaker
|-Approve
|-Deploy (CloudFormation – StackName: sagemaker-prod)
    |-delete_prod_sagemaker
    |-deploy_prod_sagemaker