1. Why Transformer Fully connected self-attention In earlier RNN models, each word could only relate to its neighboring words, whereas with the Attention mechanism in the Transformer model, a word can relate to a word at any position, so it can capture global context information. No vanishing gradient problem An RNN applies repeatedly to the same weight matrix, so when its largest eigenvalue is less than 1 the vanishing gradient problem appears.