SageMaker ML Model Deployment with Auto-Scaling

The data science team had a scikit-learn churn prediction model that worked on their MacBooks but failed in every attempt at production deployment. The model used a Pipeline object with custom transformers (a date feature extractor, a categorical encoder with a fitted vocabulary, and a StandardScaler), and the team was serializing the entire pipeline with pickle. The SageMaker SKLearnModel container runs Python 3.10 by default, but the team was developing on Python 3.9. Pickle is not forward-compatible across Python minor versions when custom classes are involved, so the model would load on their machines but throw ModuleNotFoundError or UnpicklingError inside the SageMaker container. API Gateway was returning 502 errors on roughly fifteen percent of requests because the default SageMaker endpoint timeout is 60 seconds, and some prediction payloads containing 500+ rows were exceeding that. The team had no monitoring on the endpoint, so they did not know the model was silently returning incorrect predictions when a feature distribution shifted.

I rebuilt the deployment to decouple the preprocessing pipeline from the model artifact. Instead of pickling the entire Pipeline object, I split it into three separate artifacts stored in S3: the model weights (a joblib-serialized GradientBoostingClassifier), the fitted transformers (saved individually as JSON configuration files), and the feature schema (a JSON document listing expected columns, types, and valid ranges). The custom inference.py implements four SageMaker-required functions. model_fn loads the model weights and transformer configs. input_fn deserializes the incoming JSON or CSV payload and validates against the feature schema. predict_fn applies the transformers and runs prediction. output_fn serializes the response with prediction probability and model version. The SageMaker endpoint uses the SKLearnModel container pinned to Python 3.9. Auto-scaling uses a target tracking policy on InvocationsPerInstance: when the metric exceeds 200 per 5-minute period, SageMaker provisions instances up to a maximum of 10. For API Gateway, the integration timeout is 29 seconds with request validation rejecting payloads over 100 rows. Larger payloads go through SageMaker Batch Transform with S3 output and SNS notification. SageMaker Model Monitor runs hourly, comparing incoming feature distributions against a training baseline. When any feature diverges beyond two standard deviations, a CloudWatch alarm fires. The Model Registry stores versioned artifacts with an approval workflow before production deployment.

Deployed a scikit-learn churn prediction model on AWS SageMaker with a real-time inference endpoint serving under 200 milliseconds p99 latency. The deployment uses a custom inference.py that decouples the preprocessing pipeline from the model artifact, solving a persistent pickle deserialization failure caused by Python version mismatches between the data science team laptops and SageMaker containers. API Gateway fronts the endpoint with a Lambda authorizer for JWT validation. SageMaker auto-scaling maintains between 1 and 10 instances based on InvocationsPerInstance, and SageMaker Model Monitor runs hourly data drift checks against the training baseline distribution.

The endpoint handles roughly 15,000 prediction requests per day with a p99 latency of 180 milliseconds and a p50 of 45 milliseconds. The auto-scaling configuration keeps costs at approximately $340 per month during normal traffic, scaling up to 10 instances during the monthly batch run. Model Monitor detected a distribution shift in the income feature after three weeks in production. The data science team retrained and the new model was deployed through the SageMaker Model Registry with zero downtime using a blue-green endpoint update. Before this deployment, the team was running predictions on a laptop and emailing CSV files to stakeholders, a process that took a full day and frequently produced stale results.