AutoML on AWS with Example

Learn how to easily train and deploy machine learning models on Amazon Web Services with the help of AutoML and the AWS SDK.

Overview

AutoML, or Automatic Machine Learning, is a rapidly growing field that enables users to train and deploy machine learning models with minimal coding and technical expertise. One popular platform for AutoML is Amazon Web Services (AWS), which offers a variety of tools and services for building, deploying, and managing machine learning models. In this blog post, we will explore how to use AutoML on AWS with an example of how to train and deploy a simple machine learning model.

Example

AutoML with the AWS SDK can be achieved using the Amazon SageMaker service, which provides a variety of tools and APIs for building, deploying, and managing machine learning models. Here’s an example of how to use the Amazon SageMaker SDK to train and deploy an AutoML model in Python:

import boto3
import json

# Create an Amazon SageMaker client
sagemaker = boto3.client('sagemaker')

# Define the dataset location and model type
dataset_location = 's3://my-bucket/dataset.csv'
model_type = 'LinearLearner'

# Start an AutoML job
response = sagemaker.create_auto_ml_job(
    AutoMLJobName='my-automl-job',
    InputDataConfig=[{
        'DataSource': {
            'S3DataSource': {
                'S3DataType': 'S3Prefix',
                'S3Uri': dataset_location
            }
        },
        'TargetAttributeName': 'label'
    }],
    OutputDataConfig={
        'S3OutputPath': 's3://my-bucket/output/'
    },
    ProblemType='BinaryClassification',
    AutoMLJobObjective={
        'MetricName': 'Accuracy'
    },
    AutoMLJobConfig={
        'CandidateSteps': [{
            'CandidateStepType': model_type
        }]
    }
)

# Wait for the AutoML job to complete
job_name = response['AutoMLJobArn']
print('AutoML job started:', job_name)
waiter = sagemaker.get_waiter('auto_ml_job_completed')
waiter.wait(AutoMLJobName=job_name)

# Deploy the best model
best_model = sagemaker.describe_auto_ml_job(AutoMLJobName=job_name)['BestCandidate']
model_name = best_model['FinalAutoMLJobRunnerName']

response = sagemaker.create_model(
    ModelName=model_name,
    ExecutionRoleArn='arn:aws:iam::ACCOUNT_ID:role/service-role/AmazonSageMaker-ExecutionRole-2022-09-28T18:41:46Z',
    PrimaryContainer={
        'Image': best_model['Containers'][0]['Image'],
        'ModelDataUrl': best_model['Containers'][0]['ModelDataUrl']
    }
)

# Create an endpoint configuration
endpoint_config_name = 'my-endpoint-config'
response = sagemaker.create_endpoint_config(
    EndpointConfigName=endpoint_config_name,
    ProductionVariants=[{
        'ModelName': model_name,
        'InitialInstanceCount': 1,
        'InstanceType': 'ml.t2.medium'
    }]
)

# Create an endpoint
endpoint_name = 'my-endpoint'
response = sagemaker.create_endpoint(
    EndpointName=endpoint_name,
    EndpointConfigName=endpoint_config_name
)

# Wait for the endpoint to be ready
waiter = sagemaker.get_waiter('endpoint_in_service')
waiter.wait(EndpointName=endpoint_name)
print("Endpoint ready:", endpoint_name)

# Now you can use the endpoint to make predictions on new data
payload = {
    "instances": [
        {"features": [1, 2, 3, 4]},
        {"features": [5, 6, 7, 8]}
    ]
}
response = sagemaker.invoke_endpoint(EndpointName=endpoint_name, Body=json.dumps(payload))
predictions = json.loads(response['Body'].read())
print("Predictions:", predictions)

This code example shows how to use the Amazon SageMaker SDK to start an AutoML job, wait for it to complete, deploy the best model, and create an endpoint to make predictions on new data. The example uses a binary classification problem with the linear learner model, and the accuracy as the metric to evaluate the performance.

You can use this endpoint to make predictions on new data, the payload variable is the input for the endpoint, it contains the data for the model to make predictions on. The variable ‘predictions’ contains the output of the model, the predictions.

Keep in mind that this is just an example, and the real-world use cases may require more complex configurations, data preprocessing, etc. Additionally, you should replace ACCOUNT_ID in the ExecutionRoleArn with your actual account ID.

Conclusion

AutoML on AWS is a powerful tool for building, deploying, and managing machine learning models. With its user-friendly interface and wide range of tools and services, AWS makes it easy for users of all skill levels to build and deploy high-performing machine learning models. With this example, we have shown how to train and deploy a simple linear regression model on AWS and this is just the beginning, there are many more advanced models, techniques and possibilities.