Deploying Hugging Face Models in Sagemaker: 8 Steps to Create Inference End points

The video discusses the way of loading the Hugging Face AI models into AWS Sagemaker, and creating inference endpoints. It starts by introducing the Sagemaker product inside AWS, and then the challenges it solves. We discuss the Hugging Face model loading into S3 bucket for faster access and inference end point creation. The data and the code is located at https://github.com/insightbuilder/python_de_learners_data/blob/main/code_script_notebooks/projects/huggingface_AWS/storing_model_s3.ipynb The code for deploying the inference end point is from sagemaker.huggingface import HuggingFaceModel import sagemaker role = sagemaker.get_execution_role() # Hub Model configuration. https://huggingface.co/models hub = { 'HF_MODEL_ID':'distilbert-base-uncased-finetuned-sst-2-english', 'HF_TASK':'text-classification' } # create Hugging Face Model Class huggingface_model = HuggingFaceModel( transformers_version='4.17.0', pytorch_version='1.10.2', py_version='py38', env=hub, role=role, model = 's3://bucket-name/model.tar.gz ) # deploy model to SageMaker Inference predictor = huggingface_model.deploy( initial_instance_count=1, # number of instances instance_type='ml.m5.xlarge' # ec2 instance type ) predictor.predict({ 'inputs': "Its an impressive sedan" }) The 8 steps required to initiate the Sagemaker instance, pulling the model into the Sagemaker execution environment and finally deploy the endpoint is discussed in this video. I discuss each step to be taken to get the role setup, domain and user created. After the user is created, the application (Studio Notebook) can be created. Hope you like this video, and subscribe to the channel. Further uploads related to Big Data, Large Language models and Artificial Intelligence will be shared to your Youtube Dashboard Directly. The supporting playlists are Mastering NLP with Hugging Face: https://www.youtube.com/playlist?list=PLbzjzOKeYPCpVfe6Up7ga3kDVwRDlEw6k Python Data Engineering Playlist https://www.youtube.com/playlist?list=PLbzjzOKeYPCo_hMXIl2URu7GL33-4_Yy0 Python Ecosystem of Libraries https://www.youtube.com/playlist?list=PLbzjzOKeYPCoNAsZs679iXwsdP44G5SDS ChatGPT and AI Playlist https://www.youtube.com/playlist?list=PLbzjzOKeYPCpp3NCeQioevM0YpZa5VqcS AWS and Python AWS Wrangler https://www.youtube.com/playlist?list=PLbzjzOKeYPCogrhYDBgRNJDPV2CCwGrFT PS: Got a question or have a feedback on my content. Get in touch By leaving a Comment in the video @mail [email protected] @twitter Handle is @KQrios @medium https://medium.com/@kamaljp/about @github https://github.com/Kamalabot