The Continuous Bag of Words (CBOW) model is a neural network architecture used in natural language processing to learn word embeddings, most notably implemented in the Word2Vec framework. In this model, the objective is to predict a target word based on its surrounding context words within a given window size.
Unlike traditional bag-of-words approaches, CBOW takes into account the order of words in a probabilistic sense by averaging or summing the embeddings of context words, which are then used to predict the probability distribution of the target word. This approach is computationally efficient and performs well on large datasets, enabling the model to capture semantic and syntactic relationships between words.
CBOW is particularly effective for frequent words and is often contrasted with its counterpart, the Skip-gram model, which instead predicts context words from a given target word.
