![[Gamma] Scalping Please](https://ibkrcampus.com/wp-content/smush-webp/2024/04/tir-featured-8-700x394.jpg.webp "[Gamma] Scalping Please")
Study Notes:
Welcome. In this lesson, we will be discussing how to request market scanner parameters, and how to request the market scanner itself.
To begin, we will need to understand which parameters we want to request. This is not technically required, though it would be suggested to be sent before your first ever request. This list is also updated periodically, so you may also wish to make a new call for this list to review it from time to time.
Access Market Scanner Parameters
For this initial program, I will use my simple TestApp class, and in my nextValidId method, I will make a simple call to self.reqScannerParameters(). Next, I can define my scannerParameters method, and I will only receive self and xml therein.
Within my scannerParameters function, I will use some file manipulation to save these details, instead of our usual print. I will save my XML file to my python samples directory, TWS API}samplesPythonTestbed.
I will set that directory to a variable string labelled dir. Then I can type open(dir,’w’).write(xml) to write my whole xml string to that file. Then I can create a quick print statement to say, “Scanner Parameters Received.”
Then I can end my file with my standard app, connect, and run combination I have used in some of our other videos.
I would like to stress that I am saving these details to a file instead of my usual print structure because there is a massive amount of data returned here. And at least in my case, my terminal cannot print all of this data. As a point of reference, this xml file is approximately 2 megabytes.
Now if we run this, we will find the file saved in our Testbed directory. I can open this file up with Visual Studio Code and see all the values present. This list is quite large, so we would encourage you to explore this to find exactly what you are looking for.
If I scroll down a bit, we can see STK.NASDAQ for example, or STK.BATS.
In addition to being able to refine my search for specific value requirements, we can use these location codes to specify the exchanges we would like to operate with. This isn’t unique for just US exchanges.
We can refine this for any country. If I search the document, I will be able to see we have scanners for Hong Kong, Korea, Belgium and more.
Moving in towards the actual filters made available to us, we can see further refinement still. We are able to see values for Volume, or perhaps going through usdVolume so we may see the total value as opposed to share quantity.
I can search nearly any tag from the Trader Workstation, and those values should be made available here. We can search for priceAbove or a variety of 52 Week calculations for stocks, options, and so on.
With the scanner parameters in place, we are set to request the market scanner. I will be using the same values here that I was using before, with my class, nextValidId. I will once again be import TagValue using “from ibapi.tag_value import *”.
With my basic outline created, I can start building my scanner. I want to receive top stock info for the US. I will start by setting sub to ScannerSubscription() to sub as a variable. Then I will set sub.instrument = “STK”, which uses the “Instruments” tag in the XML file. Per my scanner parameters document, I can include sub.locationCode = “STK.US.MAJOR” from the locationCode tag. This will give me all major exchanges in the US, including NYSE, Nasdaq, Amex and more.
I will use the scan code “TOP_OPEN_PERC_GAIN” which is based on the scanCode tag in that scanner parameters document.
I will now create a quick variable labelled scan_options as an empty list. And now I can create another list for filter_options. In the filter options list, I will add a few filters I want to specify for my request.
First, I will add TagValue(“volumeAbove”, “10000”) so I can receive only stocks with a volume over 10000.
Then I can add something like TagValue(“marketCapBelow”, “1000”). This would further refine my list to contracts with a lower market cap still.
And finally, I will set TagValue(“priceAbove”, “1’). This will filter out penny stocks, or anything below 1 USD.
All of these values are listed as a filter option within our scanner parameters xml file we downloaded before.
There is no limit to how much or little refinement you can choose to include here, so feel free to experiment. With that said, not every inquiry will result in a response however, as logic will still apply in this scenario.
For example, if I make a call for “Curr Year Earnings Per Share Above” I can set any earnings threshold I like. If I enter a value of 100, while my program will certainly let me make the request, but I must also understand that a company will not make $100 per share for themselves. And so, I can balance my values to return something more realistic like $10.
With that said, let’s go ahead and create our market scanner request. This requires our self.reqScannerSubscription request, and we will include a request ID, our sub variable for our Scanner Subscription object. Then we can add our scan_options list, and then our filter_options list.
We can quickly mov through our EWrapper object.
As you might expect, this is simply def scannerData. This includes variables for self, reqId, and then the contracts rank, contract details, and then distance, benchmark, projection, and legsStr. And then, as usual, I will print out all of these values. Our variables after contractDetails are values which may or may not return depending on your scanner request. You will not always receive legsStr for example, as you may not receive values for combos in your requests.
Now, moving on, we have our scannerDataEnd method, that simply includes self and the request ID. Typically I note that this is an optional field, though that is not necessary the case for scanner subscriptions. If the subscription is not closed, then you will receive an error regarding a duplicate scanner id on future requests.. And so, I will add in here a quick print that we are at the scanner data end. Then I can add self.cancelScannerSubscription and only include the request ID. After my cancellation, in my case I will make my typical call to self.disconnect().
If I run our code. I can see my usual error codes returned. Now I can also see each rank in my request, starting from rank 0 all the way to rank 49. This will only provide data with respect to the position and contract information.
This is a very basic list to get you started. Keep in mind that you can use market scanners with any other security, like options or futures, but you will need to be sure your Tag Values match up accordingly.
In a future video, we will dive into these issues more in depth to see how we can request market data from the market scanner. This concludes our video on Market Scanners in the TWS API. Thank you for watching, and we look forward to having you join us for more TWS Python API lessons.
Market Scanners – API User Guide & Examples
Scanner Parameters
from ibapi.client import *
from ibapi.wrapper import *
port = 7497
class TestApp(EClient, EWrapper):
def __init__(self):
EClient.__init__(self, self)
def nextValidId(self, orderId: int):
self.reqScannerParameters()
def scannerParameters(self, xml):
dir = "C:\\IBKR\\TWS API\\samples\\Python\\Testbed\\Traders Academy\\scanner.xml"
open(dir, 'w').write(xml)
print("Scanner parameters received!")
app = TestApp()
app.connect("127.0.0.1", port, 1001)
app.run()
Scanner Subscription
from ibapi.client import *
from ibapi.wrapper import *
from ibapi.tag_value import *
port = 7497
class TestApp(EClient, EWrapper):
def __init__(self):
EClient.__init__(self, self)
def nextValidId(self, orderId: int):
sub = ScannerSubscription()
sub.instrument = "STK"
sub.locationCode = "STK.US.MAJOR"
sub.scanCode = "TOP_OPEN_PERC_GAIN"
scan_options = []
filter_options = [
TagValue("volumeAbove","10000"),
TagValue("marketCapBelow1e6", "1000"),
TagValue("priceAbove", '1')
]
self.reqScannerSubscription(orderId, sub, scan_options, filter_options)
def scannerData(self, reqId, rank, contractDetails, distance, benchmark, projection, legsStr):
print(f"scannerData. reqId: {reqId}, rank: {rank}, contractDetails: {contractDetails}, distance: {distance}, benchmark: {benchmark}, projection: {projection}, legsStr: {legsStr}.")
def scannerDataEnd(self, reqId):
print("ScannerDataEnd!")
self.cancelScannerSubscription(reqId)
self.disconnect()
app = TestApp()
app.connect("127.0.0.1", port, 1001)
app.run()
Hello,
I am using the Scanner Subscription example code, but receiving the below error:
ERROR -1 2104 Market data farm connection is OK:hfarm
ERROR -1 2106 HMDS data farm connection is OK:hkhmds
ERROR -1 2106 HMDS data farm connection is OK:ushmds
ERROR -1 2157 Sec-def data farm connection is broken:secdefhk
ERROR 1 165 Historical Market Data Service query message:no items retrieved
ScannerDataEnd!
Could you please help what could be the root cause of problem and how to fix it?
I’m using the latest IBKR GW with the latest API version on Mac.
Hello, thank you for reaching out. Reviewing your error message, we can see the message “ERROR 1 165 Historical Market Data Service query message:no items retrieved” which is indicative that no symbols correspond to your request. Reviewing the code in further detail, it appears that the scanCode “TOP_OPEN_PERC_GAIN” is not valid before regular trading hours. One way to resolve this would be changing your scanCode to “TOP_PERC_GAIN” instead. Otherwise, you could test the request again during regular trading hours to confirm this behavior. If you have any additional questions or concerns, please feel free to contact Client Services: https://www.interactivebrokers.com/sso/resolver?action=NEW_TICKET
Our API Experts would be happy to guide you!
Hi, the results give us a list for only 50 tickers, is there any way we can get more if the criteria is met by more than 50 tickers?
Hello PB, thank you for reaching out. Please create a web ticket for this inquiry; we have a category specifically for “API.” One of our API experts will be happy to guide you! https://www.interactivebrokers.com/sso/resolver?action=NEW_TICKET
Please help with obtaining Earnings Date from Wall Street Horizon Api. I have a subscription to WSH, but I’m having trouble with my Python script. Here is what I have tried:
import threading
import time
import xml.etree.ElementTree as ET
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.common import TickerId
class TWS_API(EWrapper, EClient):
def __init__(self):
EClient.__init__(self, self)
self.earnings_data = None
def error(self, reqId: TickerId, errorCode: int, errorString: str):
print(f”Error: reqId: {reqId}, errorCode: {errorCode}, errorString: {errorString}”)
def wshMetaData(self, reqId: int, dataJson: str):
super().wshMetaData(reqId, dataJson)
print(“WshMetaData.”, “ReqId:”, reqId, “Data JSON:”, dataJson)
def run_loop():
app.run()
def main():
app.connect(“127.0.0.1”, 7496, clientId=7)
contract = Contract()
contract.symbol = “AAPL”
contract.secType = “STK”
contract.exchange = “SMART”
contract.currency = “USD”
app.reqWshMetaData(1100);
event.wait(timeout=3)
if app.earnings_data:
xml_data = app.earnings_data
root = ET.fromstring(xml_data)
earnings_date = None
time_of_day = None
for item in root.iter(“Ratio”):
if item.get(“FieldName”) == “Earnings_Date”:
earnings_date = item.get(“Value”)
elif item.get(“FieldName”) == “Time_of_Day”:
time_of_day = item.get(“Value”)
if earnings_date and time_of_day:
print(f”Earnings Date: {earnings_date}”)
print(f”Time of Day: {time_of_day}”)
else:
print(“Earnings date information not found.”)
else:
print(“Failed to retrieve earnings data.”)
app.disconnect()
if __name__ == “__main__”:
app = TWS_API()
event = threading.Event()
api_thread = threading.Thread(target=run_loop)
api_thread.start()
main_thread = threading.Thread(target=main)
main_thread.start()
api_thread.join()
main_thread.join()
I don’t get any xml data from this script.
Please help.
Hello, thank you for reaching out. The wall street horizons event filters should be returned by reqWshMetaData. One of the returned filters should be “wshe_ed” which is the filter parameter for Wall Street Horizons Earnings Dates. Passing this as a filter into reqWshEventData will result in the earnings dates of AAPL in your instance. If you are in need of additional support, please create a web ticket for this inquiry; we have a category specifically for “API.” One of our API experts will be happy to guide you! https://www.interactivebrokers.com/sso/resolver?action=NEW_TICKET
Hello,
The problem I’m trying to solve is how to obtain a stock’s next earning announcement date and time.
I have subscribed to Wall Street Horizons data in my Interactive Brokers account.
I obtained the Wall Street Horizon Event Types and Fields via Trader Workstation API document.
This document describes specific event types. I am particularly interested in:
wshe_ed Earnings Date
————– ———————-
earnings_date Earnings date for the quarter/fiscal year
time_of_day BEFORE MARKET, DURING MARKET, AFTER MARKET or UNSPECIFIED
I have downloaded the XML for the contract details.
The following section in the XML shows the information from Wall Street Horizons (WSH):
WSH
Upcoming Earnings (WSH)
WSH_NEXT_EARNINGS
STK,STOCK.NA,STOCK.EU,STOCK.ME,STOCK.HK
false
742
Upcoming Earnings
true
m
DATA
0
BA
false
DATA
7
Last
742
Upc Ern
true
2147483647
500
false
unrestricted
Here is what I have done so far with my Python script:
from ibapi.client import *
from ibapi.wrapper import *
from ibapi.scanner import *
from ibapi.tag_value import *
class TestApp(EClient, EWrapper):
def __init__(self):
EClient.__init__(self, self)
def nextValidId(self, orderId: int):
# Define the scanner subscription parameters
subscription = ScannerSubscription()
subscription.instrument = ‘STK’
subscription.locationCode = ‘WSH’
subscription.scanCode = “wshe_ed”
scan_options = []
filter_options = [
TagValue(“earnings_date”, 20240401),
TagValue(“earnings_time”)
]
self.reqScannerSubscription(orderId, subscription, scan_options, filter_options)
def scannerData(self, reqId: int, rank: int, contractDetails):
# Process the scanner data
print(f”Received data for {contractDetails.contract.symbol}”)
print(f”Earnings Date/Time: {contractDetails.contractDetails.nextEarningsDate}”)
def scannerDataEnd(self, reqId: int):
print(“Scanner data received.”)
# Initialize and run the TestApp
app = TestApp()
app.connect(“127.0.0.1”, 7496, clientId=1001)
app.run()
When I run the script, I get:
ERROR -1 2104 Market data farm connection is OK:usfarm.nj
ERROR -1 2104 Market data farm connection is OK:hfarm
ERROR -1 2104 Market data farm connection is OK:jfarm
ERROR -1 2104 Market data farm connection is OK:usfuture
ERROR -1 2104 Market data farm connection is OK:cashfarm
ERROR -1 2104 Market data farm connection is OK:eufarmnj
ERROR -1 2104 Market data farm connection is OK:usfarm
ERROR -1 2106 HMDS data farm connection is OK:hkhmds
ERROR -1 2106 HMDS data farm connection is OK:cashhmds
ERROR -1 2106 HMDS data farm connection is OK:fundfarm
ERROR -1 2106 HMDS data farm connection is OK:ushmds
ERROR -1 2158 Sec-def data farm connection is OK:secdefil
ERROR 1 162 Historical Market Data Service error message:Scanner type with code wshe_ed is disabled.
What am I doing wrong, and I can I fix this Python script?
Charles
Hello, thank you for asking. There are a few visible issues with the request provided. While you are welcome to use the WSH_NEXT_EARNINGS scanCode for Stocks, the locationCode of WSH is not valid. I would also note that the filters used in wall street horizons event requests are not applicable for market scanner requests. As a result, the filter values referring to the WSHE filters would also not be valid in these requests. A simple variant on this request would look something like this:
sub = ScannerSubscription()
sub.instrument = “STK”
sub.locationCode = “STK.US.MAJOR”
sub.scanCode = “WSH_NEXT_EARNINGS”
scan_options = []
filter_options = []
If you are in need of additional support, please create a web ticket for this inquiry; we have a category specifically for “API.” One of our API experts will be happy to guide you! https://www.interactivebrokers.com/sso/resolver?action=NEW_TICKET
Hi,
At the end of the video it says that there will be a future video that explains how to get the data from a scanner for further processing. Are there still plans to release such a video and, if so, is there an ETA for the release?
Thank you.
Hello, we appreciate your question. A new TWS Python course is in the queue to be released. In the meantime, please view this IBKR Campus course:
https://ibkrcampus.com/trading-lessons/tws-python-api-concurrency-example/
It could be a great resource for you!
Hi Guys,
Thanks for the reply.
On the Wall Street Horizons web site there is a sample Python script. It is:
ib = IB()
ib.connect(‘127.0.0.1’, 7496, clientId=1234)
# Get the conId of an instrument (IBM in this case):
ibm = Stock(‘IBM’, ‘SMART’, ‘USD’)
ib.qualifyContracts(ibm)
print(ibm.conId) # is 8314
# Get the list of available filters and event types:
meta = ib.getWshMetaData()
print(meta)
# For IBM (with conId=8314) query the:
# – Earnings Dates (wshe_ed)
# – Board of Directors meetings (wshe_bod)
data = WshEventData(
filter = ”'{
“country”: “All”,
“watchlist”: [“8314”],
“limit_region”: 10,
“limit”: 10,
“wshe_ed”: “true”,
“wshe_bod”: “true”
}”’)
events = ib.getWshEventData(data)
print(events)
This script doesn’t work for me. I get:
8314
Error 10276, reqId 4: News feed is not allowed.
Error 10276, reqId 5: News feed is not allowed.
Press any key to continue . . .
And yes, I am subscribed to Wall Street Horizons in my Trader Workstation account.
As you can see, I am struggling with getting a simple example of how to get an Earnings Date from Wall Street Horizons subscription.
Do you guys have a simple Python script that works with today’s release of Python and the TWS Api?
Any code examples will be greatly appreciated.
Charles
Hello again. Please note that Interactive Brokers does not support IB_Insync. If you are still experiencing issues, please contact Client Services: http://spr.ly/IBKR_ClientServicesCampus