Requesting Market Data

Study Notes:

Welcome to this lesson on requesting market data in the Trader Workstation API. In this video, we will be highlighting the requirements for requesting market data, how to request delayed data, how to request live market data, and how to request historical bars. Please note that these are the most popular methods of requesting market data; however, Interactive Brokers also offers tick data, histogram data, and market depth.

Discussing Market data subscriptions

Let’s begin by discussing market data subscriptions. In order for clients to subscribe to market data, users must have a funded IBKR account for at least $500 USD in most instances. There are some instances where this is not the case; however, for the average individual at Interactive Brokers, $500 is the minimum. This threshold must be maintained in addition to the cost of any subscriptions held by the account.

For those with a long-time IBKR PRO account, you may have observed that some instruments return market data to your Trader Workstation for free by default. That is because some market data can be provided to users for free while “on-platform”. On-platform simply means that users are observing data directly display through one of Interactive Brokers platforms. Exchanges consider API functionality to be considered off-platform, and as a result, typically have a cost affiliated with them. Some of the most popular market data subscriptions for API use are listed in the API Documentation for Market Data on IBKR Campus. Users can subscribe to market data through the Client Portal. 

It is also worth clarifying that Market Data is affiliated on a per-user basis. Many clients will run a single Trader Workstation instance for monitoring trades; however, it is common to have a separate machine running your trading algorithm on IB Gateway hosted on a virtual machine elsewhere. In order for both of these platforms to retrieve market data, each user consuming market data would need to subscribe to data separately.

Requesting streaming data

With the subscription discussion out of the way, we can start to dive into the actual API requests. Please note that we will be using the same framework from our Essential Components video, so if there are any questions on the initial structure in this video, please be sure to review that lesson first.

The most popular way of requesting and viewing data in the API would be with EClient.reqMktData method, which requests the same data available in the TWS Watchlists.

Requesting delayed data

Clients that do not have a market data subscription for instruments can often request 15 minute delayed data. This is only a single extra step compared to standard market data subscriptions, so I will include it before moving forward.

To clarify if your requests will be Live or delayed, users simply need to call the app.reqMarketDataType function. The only argument this takes is the type of data to retrieve, which can be 1,2,3,4 or Live, Frozen, Delayed, or Delayed Frozen respectively. Frozen data will refer to market data from the most recent close, while delayed frozen data will return yesterday’s closing values. And then as we’ve mentioned, standard delayed data will return 15-minute delayed data.

If I am subscribed to market data on a given instrument, but request delayed market data, live data will still be returned. Interactive Brokers will always try to provide the most up-to-date market data where possible.

Requesting live data

Now, let’s start building out our request for streaming data. We will be focused on requesting price and size data; however, the reqMktData request can also return string, news, generic and even Greek values depending on the tick types provided.

From within our TestApp class, let’s start defining one of our tick functions, tickPrice. This will handle all returning values related to price values. tickPrice takes self, reqId, tickType, price, and attrib as arguments. While we’re already familiar with the first two, and the last two are rather self-explanatory, the tickType argument is used to indicate what kind of data is coming in.

Each tickType is an integer value that correlates to a specific value, be it bid price, last size, closing price, or otherwise. For a full list of all of these tick values, we can look at ticktype.py inside the ibapi source files and see exactly what everything is relating to. Users are welcome to reference the returned integer values directly; however, the enumerator contains a toStr method that converts our tick type integers into the values we see before us. In our file, we can add an import for from ibapi.ticktype import TickTypeEnum. This will allow us to reference the TicKTypeEnum.toStr() method and print out our value directly in a moment.

I will print out all of these values in an ­f-string, including our reference of TickTypeEnum.toStr(). As we discussed before, this would be perfectly fine to print out our price values; however, I also want to see the quantities of our trades effected by this. To do this, we will also add the EWrapper.tickSize function to our TestApp class as well. This function only takes the arguments: self, reqId, tickType, and size.  The sizes returned here will relate to the prices returned in our tickPrice function and allow us to create a clearer picture of the trades taking place.

Now that we have everything in place to receive the data, let’s build out a contract object and a request for market data. Leaping off of our prior video, I’ll make a request for AAPL market data using the symbol, security type, currency, and exchange values.

With a contract now set, I can call app.reqMktData to start requesting my streaming data. For arguments, we’ll need to pass the reqId, which we’ll use our app.nextId() function for. I can pass mycontract for the contract object. For our next argument, the generic tick list, I will pass “232” as a string so I can retrieve the mark price from my request. For users looking to request multiple generic ticks, you would simply comma-separate the values within the string. So maybe you would pass “232, 233, 234” as an example.

The next argument defines if we are requesting a Snapshot value.

This is a single return instance aggregating the last 11 seconds of trading. If no trading has taken place, this will not return any values. And if we do see trades in the last 11 seconds, we will see those values returned in aggregate. Similarly, the next argument determines if we are requesting a regulatory snapshot. This is a snapshot used to determine the price of an instrument before executing a trade.

Regulatory snapshots will cost approximately $0.01 per request, until we reach the cost of the affiliated subscription. If I request market data for AAPL repeatedly, Interactive Brokers will eventually add the subscription to your account, as the cost of the regulatory snapshots equate to the value of the subscription anyway. The final argument takes market data options, which is an argument used only for internal purposes.

If we run this script, we’ll find an initial flood of data depicting the most recent values for several tick types, then over time we will receive all the live data prices and sizes as they come through.

Requesting Historical Data

Requesting Historical Data follows a similar pattern to the live market data requests. The one caveat to this is that market data cannot be retrieved if you do not have a valid market data subscription. Before we begin to dig into historical data, I’d like to first find how far back we can request market data. I’ll start finding this value using a new python file.

Requesting the headTimeStamp Value

In our new file, I will create a new function in the TestApp class to define the headTimeStamp function. This takes three arguments, self, a request ID, and the headTimeStamp string. Within my new function, I will print out my headTimeStamp value. I will also make a request for self.cancelHeadTimeStamp to terminate the request now that I’m done with it, and we can just pass the requestId we received as an argument. With the EWrapper piece out of the way, I will move out of the TestApp class to create my headTimeStamp request. I will copy over my same AAPL contract I used from the Live Data script, because I want to validate how far back I can find AAPL market data.

Next, I will make a call to the app.reqHeadTimeStamp function. This takes an argument for a request ID, which can use our nextId function; and a contract reference, which will take my mycontract object.  After these two, I’m now encountering something known as the “whatToShow” value. This same value is used to denote what data is displayed in your TWS bar charts. In my case, I will use the value for Trades, though the full list of whatToShow values are available in our documentation.

The next argument relates to using regular trading hours. A 1 will indicate that we want the earliest date within trading hours, while a 0 will request the earliest date outside of trading hours. Finally, we have the formatDate parameter. This will indicate whether we want 1, a UTC timestamp in a string format, or 2, an Epoch timestamp. The latter is an integer representation of the same timestamp. You can consider the former better for human consumption, while the latter is best utilized in a programmatic request structure. I will show these off in just a moment by making two requests.

If we run this script using ‘1’ as the date format, we’ll see 19801212-14:30:00. Meaning AAPL’s “Trades” market data can go as far back as December 12, 1980 at 9:30 AM Eastern. Before we move on, I’ll quickly add another print statement to my headTimeStamp method for the datetime.datetime.fromtimestamp function, taking in the integer version of our headTimeStamp. If I change my original request to use 2 as my date format, I’ll print out the original epoch value as well as the python translated datetime, which is automatically converted to my pc’s local time in US/Central time.

Requesting historical bars

Now that we know our historical data range, we can start making a request for historical data. You are welcome to use the same file, but for my demonstration, I’ll be starting from a fresh example of our standard layout but add in our AAPL contract again. As always, we’ll define the EWrapper function inside TestApp using def historicalData. This function takes an argument for self, reqId, and bar. We will finish the function by printing the reqId and bar values.

I will note that we are printing out the full bar object; however, the bar object can be split out, so you may print bar.open for the opening price, bar.close for the closing price, and so on. But just for our presentation here, I’ll print the whole thing.

Each bar size is returned separately, so for us to know we’re done we should reference the EWrapper function for historicalDataEnd. This function takes an argument for self and reqID and is just meant to indicate that all available data has been returned to the user.

With the wrapper functions set, we’ll start our EClient request. To make the request, we’ll call the app.reqHistoricalData function. This takes 10 total arguments, starting with reqid, contract.

The next argument is endDateTime, which takes the value we’d like to end our historical data at. If we leave this as an empty string, the system will assume the current time. Otherwise, we would make a format for year, month, day followed by the 24 hour timestamp, and a timezone. You must pass the timezone for the exchange, available through a Contract Details request, the exact timezone used for your TWS, which is set prior to the homescreen, or using UTC. I will send my request for “20240523 16:00:00 US/Eastern”.

Then, we’ll pass in a duration value, which corresponds to the full interval over which data will be returned. So, if I specify “1 D”, I will receive however many bars in a single day. On the topic of bars, the next argument will take the bar size. In my case, I can pass “1 hour” to receive an hour.

This means I will receive 7 bars for my day, a bit more on that later.

Moving forward in our arguments, we’ll find more familiar values, like the whatToShow value we used before, which I’ll use “TRADES” for once again, then useRth and formatDate, again using 1 in both cases. Now we have the option for “keepUpToDate”, which allows us to build bars as data is available and return these new bars to the historicalDataUpdate function. I’m not interested in this data at the moment, so I’ll go ahead and leave this as False for now. Finally, we end with market data options, which I’ll again leave as an empty list.

Now if we run this script, I will see my request ID, and all of my bar’s values. While most of this is self-explanatory, there are a few points I’d like to mention from the programmatic standpoint. You might notice that we sent the request using US/Eastern, but my data shows America/Chicago. That’s because I’m choosing to print out my “Operator Timezone” even though I made the request with the “Instrument Timezone”. You can modify the time zone returned in TWS by opening the Global Configuration page and opening the API Settings. You’ll notice a section for “Send instrument-specific attributes for dual-mode API Client in”. Specifying Operator Timezone returns your TWS time zone, Instrument time zone returns the time zone for the contract, and UTC is obviously the UTC time zone.

The other piece I’d like to mention is the 7 bars I had referenced earlier. The Date value in the bar references the starting time for the bar. In my case, we can see my bar started at 08:30:00 America/Chicago, which is when NASDAQ opens for trading AAPL. But then we’ll see 09:00 as our next bar, meaning our first bar is only 30 minutes long, before turning into a series of 1-hour bars. This is the same behavior as Trader Workstation, though it may not be as commonly understood when pulling data programmatically. Therefore, it’s best practice to use the next bar as an indicator of approximate bar size.

This concludes our lesson on market data in the TWS API. Thank you for watching. If you have any questions, please be sure to review our documentation or leave a comment below this video. We look forward to having you in the next lesson of our TWS API series.

Code Snippet – headTimeStamp

from ibapi.client import *
from ibapi.wrapper import *
import datetime
import time
import threading

port = 7497


class TestApp(EClient, EWrapper):
    def __init__(self):
        EClient.__init__(self, self)

    def nextValidId(self, orderId: OrderId):
        self.orderId = orderId
    
    def nextId(self):
        self.orderId += 1
        return self.orderId
    
    def error(self, reqId, errorCode, errorString, advancedOrderReject):
        print(f"reqId: {reqId}, errorCode: {errorCode}, errorString: {errorString}, orderReject: {advancedOrderReject}")
    
    def headTimestamp(self, reqId, headTimeStamp):
        print(headTimeStamp)
        print(datetime.datetime.fromtimestamp(int(headTimeStamp)))
        self.cancelHeadTimeStamp(reqId)


app = TestApp()
app.connect("127.0.0.1", port, 0)
threading.Thread(target=app.run).start()
time.sleep(1)

mycontract = Contract()
mycontract.symbol = "AAPL"
mycontract.secType = "STK"
mycontract.exchange = "SMART"
mycontract.currency = "USD"

app.reqHeadTimeStamp(app.nextId(), mycontract, "TRADES", 1, 2)

Code Snippet – historical_market_data.py

from ibapi.client import *
from ibapi.wrapper import *
import datetime
import time
import threading

port = 7497


class TestApp(EClient, EWrapper):
    def __init__(self):
        EClient.__init__(self, self)

    def nextValidId(self, orderId: OrderId):
        self.orderId = orderId
    
    def nextId(self):
        self.orderId += 1
        return self.orderId
    
    def error(self, reqId, errorCode, errorString, advancedOrderReject):
        print(f"reqId: {reqId}, errorCode: {errorCode}, errorString: {errorString}, orderReject: {advancedOrderReject}")
    
    def historicalData(self, reqId, bar):
        print(reqId, bar)
    
    def historicalDataEnd(self, reqId, start, end):
        print(f"Historical Data Ended for {reqId}. Started at {start}, ending at {end}")
        self.cancelHistoricalData(reqId)

app = TestApp()
app.connect("127.0.0.1", port, 0)
threading.Thread(target=app.run).start()
time.sleep(1)

mycontract = Contract()
mycontract.symbol = "AAPL"
mycontract.secType = "STK"
mycontract.exchange = "SMART"
mycontract.currency = "USD"

app.reqHistoricalData(app.nextId(), mycontract, "20240523 16:00:00 US/Eastern", "1 D", "1 hour", "TRADES", 1, 1, False, [])

from ibapi.client import *
from ibapi.wrapper import *
import datetime
import time
import threading
from ibapi.ticktype import TickTypeEnum

port = 7497


class TestApp(EClient, EWrapper):
    def __init__(self):
        EClient.__init__(self, self)

    def nextValidId(self, orderId: OrderId):
        self.orderId = orderId
    
    def nextId(self):
        self.orderId += 1
        return self.orderId
    
    def error(self, reqId, errorCode, errorString, advancedOrderReject):
        print(f"reqId: {reqId}, errorCode: {errorCode}, errorString: {errorString}, orderReject: {advancedOrderReject}")
      
    def tickPrice(self, reqId, tickType, price, attrib):
        print(f"reqId: {reqId}, tickType: {TickTypeEnum.toStr(tickType)}, price: {price}, attrib: {attrib}")
      
    def tickSize(self, reqId, tickType, size):
        print(f"reqId: {reqId}, tickType: {TickTypeEnum.toStr(tickType)}, size: {size}")


app = TestApp()
app.connect("127.0.0.1", port, 0)
threading.Thread(target=app.run).start()
time.sleep(1)

mycontract = Contract()
mycontract.symbol = "AAPL"
mycontract.secType = "STK"
mycontract.exchange = "SMART"
mycontract.currency = "USD"

app.reqMarketDataType(3)
app.reqMktData(app.nextId(), mycontract, "232", False, False, [])