L11.5-1: Sequence-to-Sequence Learning, using RNN encoder/decoder

In this video I start our discussion of sequence-to-sequence deep network models. So far in our section on text and sequence processing, we have mostly used a sequence classification task as our example (IMDB sentiment classification). However there are more applications than simple classification. Sequence-to-sequence learning is one of those killer apps of Deep learning sequence processing. It can be applied to many use cases, such as machine translation, text summarization and text generation. I have split this discussion into two parts because of length. In this part we fist look at a typical sequence-to-sequence architecture using the encoder / decoder model. For sequence-to-sequence learning, you need an encoder that turns a source sequence into an intermediate representation. Then the decoder learns to predict the next output N of a target sequence 0..N-1, using the target sequence up to N-1 and the full intermediate representation from the encoder. I use a machine translation dataset and example for these videos, that has pairs of short English phrases and their target translation to Spanish. I show an example of using basic RNN layers for the encoder and decoder (a Bidirectional layer using a GRU recurrent layer). Resources: Textbook: Chollet (2022). "Deep Learning with Python (2ed)". Manning. https://www.amazon.com/dp/1617296864/?bestFormat=true&k=deep%20learning%20with%20python&ref_=nb_sb_ss_w_scx-ent-pd-bk-d_de_k0_1_15 CSci 560 Class Repository: https://github.com/csci560-nndl/nndl Contains video slides and iPython notebooks for this course. 00:00 Introduction 00:50 Sequence-to-sequence learning and model architecture 06:47 Machine translation example corpus: English to Spanish phrase corpus 14:01 Sequence-to-sequence translation with a Bidirectional GRU recurrent layer 26:15 Summary