Training Multiple Models in Parallel with Lakeflow Jobs | Hyperpersonalization Use-Cases

Training Multiple Models in Parallel with Lakeflow Jobs | Databricks In this video we leverage Lakeflow Jobs to train multiple models in parallel which we then deploy onto a singular model serving endpoint. The idea is to use this solution for hyper personalization use-cases that require customization on a per model or user-basis. Lakeflow provides the orchestration, Model Serving provides the multi-model hosting capabilities. Video Resources - Notebook Code: https://github.com/RamVegiraju/databricks-samples/tree/master/traditional-ml/MLOps/multi-model-training-pipeline - Lakeflow Jobs Docs: https://docs.databricks.com/aws/en/jobs/ - Model Serving Docs: https://docs.databricks.com/aws/en/machine-learning/model-serving/serve-multiple-models-to-serving-endpoint - Multi-Model Serving Intro Video: https://www.youtube.com/watch?v=hDlBktxkG58 Timestamps 0:00 Introduction 0:53 Use-Cases 4:20 Lakeflow Jobs 10:00 Hands-On #databricks #mlengineering #mlops #mlflow #modeldeployment #modelserving #lakeflow