The term “deep learning” does not have a single, accepted definition. Originally, just referred to the number of hidden layers: a network with none or just a couple was “shallow,” and a network with several was “deep.”
Today, it usually refers to the set of properties and practices associated with such networks:
- Hierarchical representation of concepts
- Large datasets and training burdens
- Specialized architectures