Scaling TensorFlow 2 models to multi-worker GPUs (TF Dev Summit '20)

This talk showcases multiple performance improvements in TensorFlow 2.2 to accelerate and scale users' ML training workload to multi-worker multi-GPUs. We walk through the optimizations using a BERT fine-tuning task in TF model garden, written using a custom training loop. Speaker: Zongwei Zhou - Software Engineer Resources: XLA: Optimizing Complier for ML → https://goo.gle/3adIDUR GitHub Bert → https://goo.gle/2xhNQgm Mixed precision → https://goo.gle/2PRAEoE GitHub models official → https://goo.gle/3cEZRwj Distributed training with TensorFlow → https://goo.gle/39wMWdY Watch all TensorFlow Dev Summit 2020 sessions → https://goo.gle/TFDS20 Subscribe to the TensorFlow YouTube channel → https://goo.gle/TensorFlow event: TensorFlow Dev Summit 2020; re_ty: Publish; product: TensorFlow - General; fullname: Zongwei Zhou;