TWS Python API Concurrency Example

Study Notes:

Welcome. In this lesson we will be introducing the most basic concepts behind concurrency as well as further exploring how we can combine our requests. Before we begin, I would like to mention a few disclaimers.

Just like in our initial videos, this is only one way of many to code this request. This is just one example to help showcase multiple EClient connections. I would also mention that this is not the most efficient nor effective method of making these requests. There are several points that could be better handled. Also, this code requires only the IB API and then the Python Standard Library modules to use such as threading, time and datetime.

You are welcome and encouraged to use and implement any packages available at your disposal.

I would also like to briefly discuss what concurrency is. Concurrency means that more than one calculation or operation is taking place simultaneously, and it’s important to note that these tasks are not necessarily dependent on one another, which is precisely what allows them to be completed in parallel. The true potential of concurrency goes far beyond what is represented in this video. Please keep this in mind as we proceed.

Now to begin, first, start by explaining briefly what my plan of this code is.

• The intention of this code is to make a market scanner in which I can then request market data from, compare those values, and then at the end of the market calculations.
• I can then place an order for the highest percent change percentage as well as the highest percent change value.
• And we’ll touch more on what exactly that means later on. To represent this, I’m going to briefly show you what my code looks like after I’ve run it, so that way we can then kind of reverse engineer it and talk our way through it throughout the code.

To begin, I first run my code and connect to my socket. Then I then run my market data scanner and I receive my top 50 results that come from that market data scanner. With that said, in in my market scanner I’m also requesting live market data so that way I could see the bid, ask, last value for whichever rank these contracts might be in. Now, after receiving all of this market data and then I’m then going to calculate the top five contracts comparing them against this morning’s opening.  Though I will also be just showing what they would be calculated against yesterday’s open or the prior trade days open.

With that said, if I look here, I could see my contract as well as my current price. This will match up with the last price from above. Then I can see the bar from today as well as my bar from yesterday and my case yesterday was November 14^th. We can once again see my open high, low and close for those two dates. Now I made a quick calculation to compare my last value against my open values for both today and yesterday and I even in this case calculated the change value like I referenced before as well as the change percent. So that way I can calculate how much they’re shifting by, like an actual dollar amount versus the actual percentage value they have changed on their own.

This is increasingly important for certain contracts that require a change – it might only be a 60 cent change, but that’s 123% difference, whereas a $3 change is 120% difference on a different contract.

So, what my supposed strategy in this case is going to be is that I will look for whatever has the largest increase in my top percent gainers as an increase by integer as well as that which has the highest increase by percentage. And so, I calculated it among those top five. What has the highest of each? And then set the value accordingly.

In my case it’s going to be FDMT and HUYA. Now I could see my two values here and then I offered myself the option. Would I like to buy these? In my case, I would like to buy both of them, so I simply hit yes, and so what my system did is they would place a quick order and then place the order for HUYA and it placed the order for FDMT. Now, as we’d see here, these are pretty standard orders. I just place them as a default market order, and I bought them for a quantity of 100 with a GTC for time in force. So that way I can place these trades at any time.

And that’s what my code here was meant to do. Nothing additional, and it was just supposed to execute my trades based on this sort of baseline strategy that I built here.

Now in my code structure I have three main sections, In the first section I have my typical test app class. This will include all of my IB API components like Eclient and EWrapper. I’m also creating my start investing class. It’s not necessarily essential to have a class for this. This is more of an organizational structure that I prefer. Just that way I can keep all of my methods and functions regarding my own personal requests and how I want them structured and formatted to be all contained within the single class. And then finally I have my main method which calls out to primarily my start investing class and its methods.

Now starting out I’m going to briefly cover just generally what we’ll be working with here today.

To begin, I’m going to be building a market data scanner. I’m going to then print that market scanner After printing all of that data, I will then have my build historical data, calculate the change value and then printing the difference. It’s going to calculate the change, print the difference in the top percentage as well as calculate my best buy options.

I’m going to be touching on each of these quite briefly, just because we’ve covered these in other lessons, and I would advise reviewing those prior lessons to gain a more general understanding of these methods we’re going to be working with, but this will at least showcase how we can integrate them all together and what you can actually do once you start putting these methods of EClient and EWrapper into one piece.

With all that said, a lot of what I’m working with today will be based around my global dictionary called GlobalDict, and what this will do is, it will allow me to pass and move all of m symbols as needed between all of my methods and objects. For those who are newer to coding, it helps to somewhat think of this as each entry into my global dictionary is like a new row in an Excel document.

We’re going to be separating them by the contract, ordering them by the rank or sometimes referred to as my requestID. In this I’ll also be including things like the bid value, ask value, and you can think of it generally like a table or excel sheet.

So, jumping right into the code, I’m just going to be initially starting out by placing a build scanner request and this is almost identical to the one we had in our prior video where I’m requesting the top US stocks and I’m requesting them using the top percent gain as my scan code. And so, after making my initial scan code request, it then returns the values back to me in my scanner data object request and we find that in here where I can then start matching them into my global dictionary.

You’ll see here I’m using my key object as my rank, so in that case I always know that value 0, which is the first in the index and it will always line up with my first contract. And so, it also helps in this case because it’s an integer. I can use that rank as my request ID to always set, let’s say zero as my first contract. It will also denote that anything with rank or request ID of zero will be referring to my first return values, so that’s just an idea to help keep things consistent.

With that said, I’m going to be building out my market scanner a little bit more here by adding on bid, ask last, and so on, and this will actually add on all of these values for all of my market scanners.

At the market scanner end, in addition to just canceling my market scanner, I’m also then going to be using some concurrency, so that way I can actively request market data for all of my 50 scanner symbols at the same time, so that way I don’t have to wait for them to make the request return and then continue to do that. And then serialize it to make it slower. That way, I can just send all of those requests instantly and save a little bit of time.

After getting all of my market scanner details back, both these symbols as well as the values, I can then see the tick price where I will receive these market scanner values and assign them accordingly. It’s worth noting here that I’m using the enumerate functions, so that way I can receive both the index of the key value as well as the actual value that is in the request from before we can get things like bid, ask last and so on. This is particularly relevant because I’m actually going to be comparing the tick type Enum which we had referenced in our market data video. And that way I can compare the two and so that way if I ever see bid, for example returned from my enumerate function here I can compare that against my bid in my dictionary here and then assign it without worrying about assigning to the wrong field because bid would always match up to bid, ask to ask, last to last and so on. And luckily, because things like today, yesterday, today change percent and so on are not part of any tick type, a generic tick type that would never override and so we don’t have to worry about that either.

Similarly, I’m going through making these market data requests as well for my top five symbols here. I’m not going to be going into too much depth on this because just market data request and it’s going to be using the same process for assigning these values.

Going back down to my main method, at this point we’re going to be going on to the fourth step here for calculating the change percentages. The calc change is simply going to be a method, so that way I can clean things up and then put all of my calculations in a single method. So that way I don’t have to bounce around so much. This can also make that request for all of my top five, so that way I don’t need to constantly be recalling my calc change. This will just be a single call.

Clean up all of my data and then let me move on to the next of my methods, which in this case is going to be print top diff. This is just going to be printing out those top differences, and that’s how we’re going to see this sort of formatting we add here, where we could print out the symbol current price and then the bar data for today, yesterday’s bar data and then the change values, both the change integer value as well as the change percentage value and that will also apply to both today and yesterday.

After getting those values, I am actually going to be using this change from today and change from yesterday value as my metrics for calculating the best options for what to buy. And so, if I scroll into my best buys method. We could see here that I’m just iterating through all of my percentages and integer changes or difference values and setting the largest as the best value or the best percentage, and then reporting on them at the end.

Once I do, I use a simple loop so that way I can, after I finish going through all of my values, I end with a simple input asking if I’d like to buy those two symbols. And if I put in a yes, then it immediately sends me over to my ‘buy best’ method.

Now my ‘buy best’ method is just a very simple place order method. It’s just designed to place a market order using GTC as the time and force so that way I could place my order essentially anytime I’d like, and then that way I also have a quantity of 100 sets. That way I can just have a standard value to always enter a position with.

After running that I include my threads for stopping my application. That way I can run all of this code and then beyond myself having to enter whether or not I like to buy the contract, it is entirely automatic and if I wanted to not show that off in this particular case, I couldn’t just simply remove that yes or no prompt and instead put in a call for Req-account summary to check against my equity to see if it’s worth buying those values, or if I do not have the funds.

With all that said, that should cover everything for this quick little introduction here. This concludes our video on combining requests and our introduction to concurrency in the API. Thank you for watching and we look forward to having you join us for more TW’s Python API lessons.

EClient Class List

EWrapper Class List

Market Scanners

Code Snippet – Buy Top Change Scan

from ibapi.client import *
from ibapi.wrapper import *
from ibapi.tag_value import *
from ibapi.contract import *
from ibapi.ticktype import TickTypeEnum

import threading
from time import sleep
from datetime import *


port = 7497

global globalDict
globalDict = {}

global clientId
clientId = 1001

# This is the IBAPI primary EClient and EWrapper class
class TestApp(EClient, EWrapper):
    def __init__(self):
        EClient.__init__(self, self)

    # Overidden error handler
    def error(self, reqId: TickerId, errorCode: int, errorString: str, advancedOrderRejectJson=""):
        pass
        print(reqId, errorCode, errorString, advancedOrderRejectJson)

    def nextValidId(self, orderId: int):
        self.nextOrderId = orderId

    # Returned Market Scanner information (One rank at a time)
    def scannerData(self,reqId: int,rank: int,contractDetails: ContractDetails,distance: str,benchmark: str,projection: str,legsStr: str,):
        global globalDict
        globalDict[rank] = [contractDetails.contract, "BID", "ASK", "LAST", "Today", "Yesterday", "today_CHANGE%", "prior_CHANGE%", "today_CHANGE", "prior_CHANGE"]

    # End of market scanner
    def scannerDataEnd(self, reqId: int):
        # Cancel lingering market scanner
        self.cancelScannerSubscription(reqId)

        # Request market data for all of our scanner values.
        for rank in globalDict:
            x = threading.Thread(target=self.reqMktData(reqId=rank,contract=globalDict[rank][0],genericTickList="",snapshot=True,regulatorySnapshot=False,mktDataOptions=[]))
            x.start()
        

    # Returned market data
    def tickPrice(self, reqId: TickerId, tickType: TickType, price: float, attrib: TickAttrib):
        global globalDict
        for ind,value in enumerate(globalDict[reqId]):
            if TickTypeEnum.to_str(tickType) == value:
                globalDict[reqId][ind] = price

    # End of all market data request
    def tickSnapshotEnd(self, reqId: int):
        if reqId == 49:
            # After my last request, disconnect from socket.
            self.disconnect()

    # Returned Hisotircal Data
    def historicalData(self, reqId: int, bar: BarData):
        global globalDict
        barDate = bar.date.split()[0]
        requestedDate = startInvesting.dateCleanUp()
        
        # Save Todays Bar
        if barDate == requestedDate:
            globalDict[reqId][4] = bar

        # Save the prior bar
        else:
            globalDict[reqId][5] = bar
    
    # End of All Hisotrical Data
    def historicalDataEnd(self, reqId: int, start: str, end: str):
        global clientId
        if reqId  == 4:
            clientId = self.nextOrderId
            self.disconnect()

    # Show order placed
    def openOrder(self, orderId: OrderId, contract: Contract, order: Order, orderState: OrderState):
        print(orderId, contract, order, orderState)

    def stop(self):
        self.done = True
        self.disconnect()
        
def run_loop(app_obj: TestApp):
    print("Run_Loop")
    app_obj.run()

# This is an introduction to start using threads and combining requests with one another
class startInvesting():
    # Normalize Date Values
    def dateCleanUp():
        badDate = date.today().__str__().split('-')
        goodDate = badDate[0] + badDate[1] + badDate[2]
        return goodDate

    # Create the market scanner
    def buildScanner():
        global clientId

        app = TestApp()
        app.connect("127.0.0.1", port, clientId)

        sub = ScannerSubscription()
        sub.instrument = "STK"
        sub.locationCode = "STK.US.MAJOR"
        sub.scanCode = "TOP_PERC_GAIN"

        # Both are lists of TagValue objects: TagValue(tag, value)
        scan_options = []
        filter_options = [
            TagValue("volumeAbove", "10000"),
            TagValue("marketCapBelow1e6", "1000"),
            TagValue("priceAbove", "1"),
        ]
        sleep(3)
        # Request my Market Scanner
        app.reqScannerSubscription(
            reqId=clientId,
            subscription=sub,
            scannerSubscriptionOptions=scan_options,
            scannerSubscriptionFilterOptions=filter_options,
        )
        
        app.run()
        return

    # create all historical data requests
    def buildHistorical():
        global globalDict
        app = TestApp()
        app.connect("127.0.0.1", port, clientId)
        sleep(3)
        for i in range(0,5):
            # threading.Thread(target=app.reqHistoricalData(i, globalDict[i][0], "", "2 D", "1 day", "TRADES", 1, 1, 0, [])).start()
            app.reqHistoricalData(i, globalDict[i][0], "", "2 D", "1 day", "TRADES", 1, 1, 0, [])
        app.run()

    # Create values for change values
    def calcChange():
        global globalDict
        for i in range(0,5):
            yesterday = globalDict[i][5]
            today = globalDict[i][4]
            now = globalDict[i][3]

            globalDict[i][6] = float((now / today.open)*100)
            globalDict[i][7] = float((now / yesterday.open)*100)
            globalDict[i][8] = float(now - today.open)
            globalDict[i][9] = float(now - yesterday.open)
        return

    # Place an order for the best buys
    def bestBuys():
        bestVal = globalDict[0]
        bestPerc = globalDict[0]

        # global globalDict
        for i in range(0,5):
            
            if globalDict[i][8] > bestVal[8]:
                bestVal = globalDict[i]

            if globalDict[i][6] > bestPerc[6]:
                print(globalDict[i][6], ">", bestPerc[6])
                bestPerc = globalDict[i]

        print(f"The largest increase by integer: {bestVal[0].symbol} by {bestVal[8]:.4f} ")
        print(f"The largest increase by percentage: {bestPerc[0].symbol} by {bestPerc[6]:.4f}")

        buyIt = input("\nWould you like to buy these? Y/N: ")
        if buyIt == "Y" or buyIt == "y":
            startInvesting.buyBest([bestVal, bestPerc])
        else:
            return

    # Buy the best percentage and value contracts from the bestBuys()
    def buyBest(steals):
        global clientId
        app = TestApp()
        app.connect("127.0.0.1", port, clientId)
        sleep(3)
        for i in (0, 1):
            order = Order()
            clientId+=1
            order.orderId = clientId
            order.action = "BUY"
            order.orderType = "MKT"
            order.totalQuantity = 100
            order.tif = "GTC"

            app.placeOrder(order.orderId, steals[i][0], order)

        threading.Timer(10, app.stop).start()
        app.run()
        
    # Print Scanner Results
    def printScanner():
        for i in globalDict:
            contract = globalDict[i][0]
            bidPrice = globalDict[i][1]
            askPrice = globalDict[i][2]
            lastPrice = globalDict[i][3]
            print(f"Rank: {i}; Symbol: {contract.symbol}; BID: {bidPrice}; ASK: {askPrice}; LAST: {lastPrice}")
        return

    # print top change
    def printTopDif():
        global globalDict
        print("\nThe top 5 Orders, Compared to this morning's opening:")

        for i in range(0,5):
            symbol = globalDict[i][0].symbol
            yesterday = globalDict[i][5]
            today = globalDict[i][4]
            now = globalDict[i][3]
            

            print(f"Symbol: {symbol}; Current Price: {now} ")
            print(f"Today's bar: Open: {today.open}, High: {today.high}, Low: {today.low}, Close: {today.close};")
            print(f"{yesterday.date}'s bar: Open: {yesterday.open}, High: {yesterday.high}, Low: {yesterday.low}, Close: {yesterday.close}; ")

            print(f"Change from this morning: {globalDict[i][8]:.4f} OR {globalDict[i][6]:.4f}%.")
            print(f"Change from last trade day: {globalDict[i][9]:.4f} OR {globalDict[i][7]:.4f}%. \n")
        return


def main():
    
    startInvesting.buildScanner()

    startInvesting.printScanner()
    
    startInvesting.buildHistorical()
    
    startInvesting.calcChange()
    
    startInvesting.printTopDif()

    startInvesting.bestBuys()


if __name__ == "__main__":
    main()