L11.5-2: Sequence-to-Sequence Learning, using a Transformer encoder/decoder

This video is the second part of my discussion of sequence-to-sequence learning. In this video I show a complete TransformerEncoder / TransformerDecoder example on the English to Spanish machine translation data. The model does need to be tweaked some as we do not yet get the performance we would hope for. However the basics of how you create a custom subclass to implement a transformer encoder / decoder pair for sequence-to-sequence learning is demonstrated in this video, and I discuss some approaches one might take to improving a sequence-to-sequence translation model. Resources: Textbook: Chollet (2022). "Deep Learning with Python (2ed)". Manning. https://www.amazon.com/dp/1617296864/?bestFormat=true&k=deep%20learning%20with%20python&ref_=nb_sb_ss_w_scx-ent-pd-bk-d_de_k0_1_15 CSci 560 Class Repository: https://github.com/csci560-nndl/nndl Contains video slides and iPython notebooks for this course. 00:00 Introduction 00:31 The Transformer Encoder / Decoder architecture 02:49 Custom TransformerDecoder Keras layer implementation and discussion 05:20 Training and evaluating the Transformer encoder/decoder model 10:22 Summary