Distributed representation¶
representation that is composed of many elements that can be set separately form each other
use \(k\) features with \(n\) values to describe \(k^n\) different concepts
each direction in representation space corresponds to a different value of the underlying configuration
The idea is that we transform a symbol (in NLP it would be a word) to a distributed representation, where semantically close concepts are close together in distance.
The distributed representation can represent complex things using a small number of parameters.