Residual connection (ResNet), aka skip connection

A residual neural network (ResNet) is a neural network consisting of one or more residual blocks, i.e., blocks of layers whose output equals the sum of the input into the block plus the output of the final layer in the block . The output of the block is given as

Technically, the term “residual connection” should only be used in the context of a ResNet, which consists of a series of such blocks. However, in practice, “skip connection” and “residual connection” are used interchangeably.

Residual connections are employed in order to prevent gradient collapse. See the excellent Wikipedia page on residual networks for additional details.