Bag of Words: Approach, Python Code, Limitations – Part I

Originally written by Naman Swarnkar and updated by Chainika Thakar.

The world of Natural Language Processing (NLP) is a fascinating field where machines and computers learn to understand and interact with human language. In this journey, we’ll unravel the fascinating realm of NLP and its foundation, the Bag of Words (BoW) technique.

In this comprehensive guide, we’ll take a journey through the basics of NLP and introduce you to a powerful tool within its arsenal: the Bag of Words (BoW).

To start, we’ll discuss a quick overview of NLP and will learn why text analysis is so crucial in this realm. Then, we’ll dive into the Bag of Words approach, breaking down its concept, the essential preprocessing steps you’ll need to follow, and then we will see the step by step guide in building BoW models from scratch.

Next up, we’ll discuss the practical skills to implement BoW in Python. Moreover, we will figure out the key libraries like NLTK and scikit-learn, see how to import data, prepare the text, and construct BoW models. We’ll even walk through visualising the BoW representation and provide a code example with a detailed explanation.

But that’s not all! We’ll also explore the advantages and limitations of the Bag of Words technique, that help make well-informed choices when using it in the NLP projects. We will also discuss some handy tips for supercharging the BoW analysis.

Towards the end of our journey, we’ll see the real-world applications of BoW, demonstrating its relevance in sentiment analysis, text classification, and a host of other NLP uses. By the time we’re done, you will have a solid grasp of BoW, knowing how to put it to work using Python, and appreciate its significance in the world of text analysis.

Ready to get started? Let’s dive in!

Some of the concepts covered in this blog are taken from this Quantra learning track on Natural Language Processing in Trading. You can take a Free Preview of the courses by clicking on the green-coloured Free Preview button.

This blog covers:

• Brief overview of Natural Language Processing
• Importance of text analysis in NLP
• Introduction to the Bag of Words (BoW) technique
• Bag of Words (BoW) and Trading
• Stepwise examples of using Bag of Words with Python
• Advantages of using Bag of Words
• Limitations of using Bag of Words
• Tips while using Bag of Words

---

Let us go through a quick introduction to Bag of Words starting with the brief overview of NLP.

Brief overview of Natural Language Processing

Natural Language Processing (NLP) is the bridge that connects the language we speak and write with the understanding of the language by machines. It’s the technology behind chatbots, language translation apps, and even virtual assistants like Siri or Alexa. NLP enables computers to make sense of human language, making it a pivotal field in the age of information.

---

Importance of text analysis in NLP

Text analysis is the heartbeat of NLP. It’s how we teach machines to read, comprehend, and derive meaning from the vast amount of text data (written in human language such as English) available online. From sentiment analysis of customer reviews to automatically categorising news articles, text analysis is the engine that powers NLP applications.

---

Introduction to the Bag of Words (BoW) technique

Now, let’s get to the star of the show – the Bag of Words technique, often abbreviated as BoW. BoW is like the Lego bricks of NLP. It’s a simple yet powerful method that allows us to convert chunks of text into manageable pieces that machines can work with.

With BoW, we break down sentences and paragraphs into individual words, then count how often each word appears. This creates a “bag” of words, ignoring grammar and word order but capturing the essence of the text. BoW forms the foundation for various NLP tasks, from sentiment analysis to topic modelling, and it’s a great starting point for anyone venturing into the world of NLP.

Let us take a look at a simple example for understanding Bag of Words in depth.

Imagine taking a document, be it an article, a book, or even a tweet, and breaking it down into individual words. Then, we create a “bag” and toss those words in. This is the essence of the Bag of Words technique. BoW represents text data in a way that makes it computationally accessible. It’s like creating a word inventory that machines can count and analyse, forming the basis for various NLP tasks.

So, let’s dive in and explore the ins and outs of this versatile technique!

---

Bag of Words (BoW) and Trading

In the dynamic and information-driven world of trading and finance, staying ahead of market trends and making informed decisions is of paramount importance. To achieve this, professionals in the trading domain harness the power of text data analysis, and one valuable technique in their toolkit is the Bag of Words (BoW).

The Bag of Words approach is a text analysis method that extracts essential insights from textual sources such as financial news articles, social media discussions, and regulatory documents.

The Bag of Words (BoW) technique can be used in various aspects of the trading domain to analyse text data and extract valuable insights. Here are several specific applications of BoW in the trading domain:

• Sentiment Analysis: In trading, BoW can be used to perform sentiment analysis on various sources of text data, such as financial news articles, social media posts, and forum discussions. By analysing sentiment, traders and investors can gain insights into market sentiment trends, helping them make more informed decisions.
• Market News Summarisation: BoW can be applied to summarise and categorise financial news articles efficiently. This summarisation process enables traders and investors to quickly access key information about market trends, mergers and acquisitions, earnings reports, and economic indicators.
• Stock Prediction: BoW can be integrated into stock price prediction models as a feature extraction technique. By incorporating sentiment features derived from news articles and social media, traders can enhance their predictive models, potentially improving their ability to forecast stock price movements.
• Trading Signal Generation: Traders can use BoW to generate trading signals based on sentiment analysis. Positive sentiment in news or social media discussions may trigger buy signals, while negative sentiment may lead to sell signals, aiding in trading decisions.
• Risk Assessment: BoW can assist in assessing and quantifying the risk associated with specific assets or markets. It helps identify and categorise potential risks mentioned in news articles or analyst reports, contributing to better risk management strategies.
• Earnings Call Analysis: BoW can be applied to transcripts of earnings calls conducted by publicly traded companies. Analysing the sentiment expressed by company executives during these calls can provide valuable insights into future financial performance and market expectations.
• Regulatory Compliance: BoW can help financial institutions monitor and analyse regulatory compliance documents effectively. It aids in ensuring that companies adhere to relevant financial laws and regulations.
• Event-Based Trading: Traders can use BoW to detect and respond to significant events in real-time. For instance, the announcement of a merger or acquisition can trigger trading strategies based on news sentiment, allowing for timely market participation.
• Market Commentary Generation: BoW can be leveraged to automatically generate market commentaries and reports. It assists in summarising market conditions, trends, and news for traders and investors, providing them with concise and up-to-date information.
• Customer Feedback Analysis: In the trading domain, BoW can analyse customer feedback and comments on trading platforms or brokerage services. This analysis helps in understanding customer sentiment and can lead to improvements in user experiences and services.
• Risk Management: BoW can be a valuable tool in risk management by analysing textual reports and news for potential risks that may impact portfolios or trading strategies. It aids in identifying and mitigating risks effectively.
• Algorithmic Trading: BoW can be integrated into algorithmic trading strategies as a component for decision-making. Algorithms can consider sentiment signals derived from news and social media when executing trades, potentially enhancing trading performance.

---

Stepwise examples of using Bag of Words with Python

Now, let us see how to use Bag of Words step by step with the help of Python code. First of all, we will see a general example and then we will see an example to showcase using BOW in the trading domain.

Example 1: A General example

Here is a general example of BOW to give you an overview of the same.

Step 1: Import the CountVectorizer class from scikit-learn

from sklearn.feature_extraction.text import CountVectorizer

Import_CountVectorizer.py hosted with ❤ by GitHub

In the first step, you import the CountVectorizer class from scikit-learn, which is a tool for converting text data into a numerical format suitable for machine learning model to read.

Step 2: Define a list of sample documents

You create a list called documents, which contains four sample text documents. These documents will be used to demonstrate the Bag of Words (BoW) technique.

# Sample documents
documents = [
"This is the first document.",
"This document is the second document.",
"And this is the third one.",
"Is this the first document?"
]

Sample_doc.py hosted with ❤ by GitHub

Step 3: Create a CountVectorizer object

Here, you create an instance of the CountVectorizer class. This object will be used to transform the text data into a BoW matrix.

# Create a CountVectorizer object
vectorizer = CountVectorizer()

# Create_CountVectorizer.py hosted with ❤ by GitHub

Step 4: Fit the vectorizer to the documents and transform them into a BoW matrix

This line of code fits the vectorizer to the documents and transforms the text data into a BoW matrix represented by the variable X. Here, each row in this matrix corresponds to a “document”, and each column corresponds to a “unique word” in the vocabulary.

# Fit the vectorizer to the documents and transform them into a BoW matrix
X = vectorizer.fit_transform(documents)

Fit_vectorizer.py hosted with ❤ by GitHub

Step 5: Get the vocabulary (unique words) and the BoW matrix

After transforming the documents, you extract the unique words (vocabulary) using vectorizer.get_feature_names_out(). The BoW matrix is stored in bow_matrix as a NumPy array.

# Get the vocabulary (unique words) and the BoW matrix
vocabulary = vectorizer.get_feature_names_out()
bow_matrix = X.toarray()

Get_vocabulary.py hosted with ❤ by GitHub

Step 6: Display the vocabulary and the BoW matrix

Finally, you print the vocabulary (list of unique words) and the BoW matrix to the console, allowing you to see how the text data has been converted into a numerical representation.

# Display the vocabulary and the BoW matrix
print("Vocabulary (Unique Words):")
print(vocabulary)

print("\nBag of Words Matrix:")
print(bow_matrix)

Display_vocabulary.py hosted with ❤ by GitHub

Output:

Vocabulary (Unique Words):


['and' 'document' 'first' 'is' 'one' 'second' 'the' 'third' 'this']


Bag of Words Matrix:


[[0 1 1 1 0 0 1 0 1]


[0 2 0 1 0 1 1 0 1]


[1 0 0 1 1 0 1 1 1]


[0 1 1 1 0 0 1 0 1]]

The resulting “Bag of Words Matrix” represents the frequency of each word in each document. Each row corresponds to a document, and each column corresponds to a word from the vocabulary. The values in the matrix represent word counts.

The output is a Bag of Words (BoW) matrix generated from a set of documents. Let’s break down the explanation of this specific BoW matrix:

• Each row in the matrix corresponds to a different document from the dataset. There are four rows, meaning there are four documents in this example.
• Each column represents a unique word (or term) found in the vocabulary of the documents. The vocabulary is determined by all the unique words present in the documents. In this case, there are nine unique words.
• The values in the matrix indicate how many times each unique word appears in each document. The numbers represent word frequencies within the respective documents.

Now, let’s analyse the matrix itself below.

First Row:

• The first row corresponds to the first document in this dataset.
• It shows the word frequencies for each of the nine unique words.
• Hence, in this document, the word in the second column appears once, the word in the third column appears once, and so on.

For example:

• This document contains the words ‘document,’ ‘first,’ ‘is,’ ‘the,’ and ‘this.’ The word ‘document’ appears once, ‘first’ appears once, ‘is’ appears once, ‘the’ appears once, and ‘this’ appears once.
• The other words from the vocabulary do not appear in this document.
• So, it appears as follows:

[0 1 1 1 0 0 1 0 1]

Second Row:

• The second row corresponds to the second document.
• It shows word frequencies in this document for the same set of unique words.
• Therefore, the word in the second column appears twice, the word in the fourth column appears once, and so on.

For example:

• In the second document, we see the words ‘and,’ ‘document,’ ‘is,’ ‘the,’ ‘second,’ and ‘this.’
• ‘Document’ appears twice, ‘is’ appears once, ‘the’ appears once, ‘second’ appears once, and ‘this’ appears once.
• The rest of the words from the vocabulary do not appear in this document.
• As a result, the row appears like this:

[0 2 0 1 0 1 1 0 1]

Third Row:

• The third row represents the third document.
• It shows how many times each unique word appears in this document.
• The word in the first column appears once, the word in the fourth column appears once, and so on.

For example:

• The third document contains the words ‘and,’ ‘document,’ ‘is,’ ‘one,’ ‘the,’ ‘third,’ and ‘this.’
• ‘And’ appears once, ‘document’ appears once, ‘is’ appears once, ‘one’ appears once, ‘the’ appears once, ‘third’ appears once, and ‘this’ appears once.
• This results in the following row:

[1 0 0 1 1 0 1 1 1]

Fourth Row:

• The fourth row corresponds to the fourth document.
• It displays word frequencies within this document for the same set of unique words.
• The word in the second column appears once, the word in the third column appears once, and so on.

For example:

• In the fourth document, we observe the words ‘document,’ ‘first,’ ‘is,’ ‘the,’ and ‘this.’
• ‘Document’ appears once, ‘first’ appears once, ‘is’ appears once, ‘the’ appears once, and ‘this’ appears once.
• The other words from the vocabulary do not appear in this document.
• This results in the following row:

[0 1 1 1 0 0 1 0 1]

Summary

Overall, this BoW matrix is a numerical representation of the documents, where each document is represented as a row, and the word frequencies are captured in the columns. It’s a common way to preprocess text data for various natural language processing (NLP) tasks, such as text classification or sentiment analysis.

The code above demonstrates how the CountVectorizer in scikit-learn can be used to perform BoW analysis on a set of sample documents, making it easier to work with text data in machine learning tasks.

Stay tuned for the next installment to learn how to use BOW in trading domain.

Originally posted on QuantInsti blog.