Historical Options & Futures Data using TWS API – Part II

In the previous article, we learnt to build a trading application that fetches historical data for future contracts from the IB servers. We will now see how we can extract historical data for options contracts. We will first cover the easy part which is getting data for live option contracts. Following this, we will discuss a possible approach to arrange historical data for expired option contracts. As discussed in the previous article, I am assuming that you are subscribed to the relevant market data packs. 

Trading Application 

Our trading application need not be changed from what we used to extract futures data. However, in the last article, we simply printed out the historical data and at that time I had promised that I will discuss how to store the extracted data. Therefore, in this article, we will tweak our trading application slightly to store data in Pandas dataframe instead of printing it out on the console. I have used Pandas dataframe and not native python data structures to store data because dataframes allow us some very convenient data handling and processing functions which will help us later in this article. If you are not conversant with Pandas Dataframes then I would highly recommend that you pick up at least some basic proficiency in this data structure. 

The first change in the trading application is adding a class variable hist_data. In Object Oriented Programming, Classes can have variables which define specific attribute of that class. Our TradeApp class is going to have hist_data as an attribute which will be a dataframe having the columns “Date”, “Open”, “High”,” Low”,” Close” and “Volume”. The idea is that as and when TWS return us the historical bar data, we append each bar as a row to this class attribute. 

We have also changed the historicalData Ewrapper function and instead of printing out the bar information, we are using each returned bar’s data to create a python dictionary and then append that to the hist_data class variable. I would like to clarify here that appending dictionaries to a dataframe is not the most efficient way of handling dataframes and that the TWS API has provisions to achieve the same thing more efficiently. However, I have kept that out of scope of this article. 

Creating Option Contract 

Option contracts have a number of properties which are required to identify the exact option contract. In addition to the symbol of the underlying stock, exchange, currency and expiry data we also need to know the strike price of the option and whether the option is a Call or a Put. Therefore, you may have figured out already that we will have to change how we define the parameters of the contract object. Refer the below code for the option contract that I have used in this demo. 

In the above code, I have created the contract object for the call option on Amazon’s stock having a strike price of 3400 and expiry date of 31^st December 2021 (which is a week from when I am writing this article). You can also see that I have defined the exchange as BOX which should work for most US listed option contracts.  

It should be noted that if we define the option contract parameters loosely then TWS will return details for a number of option contracts. For example, if we do not define the strike parameter or define the lastTradeDateorContractMonth as “202201” (i.e. only the year and month) then we will get a number of option contracts satisfying the defined parameters. Therefore, please ensure that you are defining all the relevant parameters which will help TWS pin point the exact option that you are interested in. 

Storing Historical Data 

Finally let us execute the reqHistoricalData function of our Trading Application to extract historical data for the required option contract. TWS will then send us data through the historicalData EWrapper function where we are updating the hist_data class variable. 

There may be some lag between when the function is executed and when the entire data is received and therefore I am adding a 5 second delay before printing out the class variable hist_data. 

Historical Data of Expired Option Contracts 

Before we commence our discussion on this topic please note that the date as of writing of this article is 24^th December 2021. You will need to keep this date in mind and consider the dates presented below in reference to this date.  

Unlike future contracts, where we could use the parameter includeExpired to get historical data for expired futures, there is no such provision for expired option contracts. Therefore, we will have to use a hack which will get us reasonably good historical data for expired options. This hack is based on the Black Scholes Formula which calculates option price based on: 

• Underlying stock price  
• Volatility 
• Strike price 
• Time to maturity  
• Interest rate  

If we have access to these five pieces of information for an expired option, we should be able to derive the option price at any point in the past by feeding this information into the Black Scholes Option Pricing Model. For example, let’s take the expired call option on Amazon’s stock having strike price of 3380 which expired on 1^st October 2021. I want to find the price of this expired option as of 1^st August 2021. To calculate the option price, I would need 

1. The price of Amazon stock on 1^st August 2021  

2. The implied volatility of Amazon’s stock on 1^st August 2021 

3. The strike price which we already know (3380) 

4. Time to expiry which is 61 days (time from 1^st August to 1^st October) or 61/365 years 

5. Interest rate which is usually taken as the long term yield on government debt. Let us take 3% for this demo based on long term treasury yield 

Since we already have the last three pieces of information (strike, time to expiry and interest rate), all that we need is the historical price and the historical implied volatility of the stock to calculate the option price as of 1^st August 2021. 

Deriving the Black Scholes formula is out of scope for this article but I strongly recommend that you gain some familiarity with this very popular formula. Below is the python code which implements the Black Scholes model. We will need to import the Numpy and Scipy.stats libraries to be able to get this function to work.  

I hope you have realized where I am going with this. In order to use the magic of the above function, all that we need to do is request historical data and implied volatility data for Amazon which is quite straight forward. 

Now that we have an intuitive understanding of this approach, let us go about building our trading application. 

Trading Application 2.0 

This time around, we will need two class variables for our TradeApp class. One to store stock data and other to store implied volatility data. In addition, we also need to find a way to segregate the information provided by TWS in the relevant class variables. Thankfully, this could be easily achieved by manipulating the reqId because we will have to have different requestIds for our request for stock data and implied volatility data. Refer below screenshot of the trading application code.  

We are asking our trading application to store the received data to the hist_data variable if the reqId is 0 else store it in the impvol_data variable. Therefore, when we are firing off the app.reqHistoricalData() request, we should be mindful of what reqId to pass as parameters for this trading application to work as desired. 

Creating Contract and Requesting Data 

The contract object this time around is not going to be an option contract but a simple stock contract of the underlying stock. That is because we plan on using the data of this stock to derive the price of the option. I want to extract daily data for the past 300 days. 

We will have to request the historical price of the stock and historical implied volatility of the stock separately. For implied volatility, we will have to change the whatToShow parameter to OPTION_IMPLIED_VOLATILITY as per the TWS API historical data documentation https://interactivebrokers.github.io/tws-api/historical_bars.html 

As discussed above, we will have to use different reqId for the two requests and ensure that the reqId is consistent with our logic in the historicalData EWrapper function. We will have to add some time delay after each request as extracting historical data can take some time. There is a smarter way to get around this issue of adding delay after each request but that topic is for a later date. 

Preparing Data to Feed the Black Scholes Model 

We now have everything that we need. All that is required now is to prepare the data so that the Black Scholes function can process it seamlessly. I prefer to create copies of the class variables into local variables but you may directly use the class variables. It is also a good idea to make the Date column as the index of both the dataframes. 

In the above code snippet, you can see that I have created a dayCount function. This is a utility function which will simply calculate the time to maturity of the option on each day. This is required because time to maturity is one of the requirements of the Black Scholes model. Refer below screenshot which shows how this function transforms the dataframe by adding the maturity column to the dataframe. Please note that maturity is in years, i.e. time to maturity for this option contract as of 24th September 2021 was 7 days which translates to 7/365 or 0.019178 years. 

Calculating Historical Price for Expired Options 

Now all that needs to be done is passing the prepared data to the Black Scholes function. Refer below code snippet. 

I have created an empty dataframe called option_price which will store the option prices calculated by our model. I have removed all bars which happened after the maturity date because there is no option price to be calculated once the option expires. 

We will iterate through each column of our dataframes (open, high, low and close) and calculate option price pertaining to that column. An important thing to note here is that for Open and Close columns, we should get the exact option price since the opening and closing prices of the stock should correspond to the opening and closing implied volatility. However, there is no guarantee that the high and low stock price correspond to the high and low implied volatility. Therefore, let me state the rider here that the high and low prices calculated will not be very accurate. Being mindful of the limitation, let us run the above lines of codes and see what we get. 

So we have got the open, high, low and close prices of the expired option. But are these prices any good? Thankfully, I had taken the screenshot of Amazon’s option chain on 22^nd September 2021 few minutes before the market closed. Refer below screenshot. 

You can see that the bid and ask prices for the call option contract expiring on 1^st October 2021 having strike price of 3380 (i.e. same expired option that we considered in this demo) were 47.55 and 48.55 respectively. Now what price have we calculated? 

From the above, you can see that the close price calculated by us is 48.02 which looks pretty good. I encourage you to test this implementation on other option contracts and drop me a note if you find something interesting or come across any discrepancy. 

The author can be reached at contact@rasuquant.com 

If you found this article interesting, you may consider enrolling in my Interactive Brokers API centric courses on https://learning.rasuquant.com. The courses range from basics of TWS API to advanced concepts that will help you backtest and build fully automated trading applications on Interactive Brokers’ platform.