jTWSdump

Historical data download with Interactive Brokers

Thursday, 21 November 2024

About

jTWSdump provides easy download (dump) of historical and intraday data with Interactive Brokers TWS. It generates formatted text files (Date/Time, Open, High, Low, Close, Volume) ready to be imported into any charting or analysis software. Data requests are performed through a graphical interface or through the command-line. Requests can be automated by providing an input file listing multiple contracts. jTWSdump works on Windows, Mac and GNU/Linux.

NEW: support added for Futures Spreads and Stock/Stock Combos.

Screenshot

Screenshot of the graphical interface in the Windows environment.

Features

Historical and intraday data download for Stocks, Futures, Options, Indexes, Forex, Combos, ...
Bar sizes: seconds 1/5/10/15/30, minutes 1/2/3/5/10/15/20/30, hours 1/2/3/4/8, 1 day, 1 week and 1 month
Support for expired futures contracts
Futures Spreads and Stock/Stock Combos support (recently added)
Automated and unattended data requests listed on a input text file
Up to a maximum of 60 batch queries per request
Can work as a command-line script with command-line arguments
Automatic data file naming and creation
Saves successful manual data requests to file dump.txt for later use
Easy to use graphical interface with context sensitive help
Runs on Windows, Mac and GNU/Linux environments

Note the following limitations: a maximum of 5 years historical data is available and it is currently impossible to request tick data. You should also be aware of the limitations imposed by IB on a single data query for the different bar sizes.

jTWSdump can batch several data queries in a single request in order to overcome these limitations. It automatically sets the query duration to maximize the amount of data downloaded for the selected bar size and then you need only to set the Number of Queries. Example: to request 3 months of 1 minute data you would set the bar size to 1 minute (jTWSdump automatically sets the Query Duration to 10 Days for this bar size) and then you would set the Number of Queries to 9 (10 Days x 9 = 90 Days ~ 3 months).

A maximum of 5 years of data can be downloaded for bar sizes: 1 min, 2 mins, 3 mins, 5 mins, 15 mins, ... Here's a breakdown of the maximum amount of downloaded data for smaller bar sizes (24 hour session): 1 sec ~ 2 days, 5 secs ~ 10 days, 15 secs ~ 1 month and 30 secs ~ 3 months (for RTH session twice as much data can be downloaded). By means of an input file, jTWSdump can download even more data for these smaller bar sizes but it will be time consuming.

jTWSdump is able to manage pacing violation errors and can work unattended in the background downloading data while you perform other tasks.

Here is an example of usage in which we feed jTWSdump (through command-line or graphical interface) with an input file with contents:

"EUR" "CASH" "IDEALPRO" "0" "0.0" "" "USD" "" "1 M" "1 day" "0" "MIDPOINT" "1" ""
"GOOG" "STK" "SMART" "0" "0.0" "" "" "" "1 D" "5 mins" "1" "TRADES" "1" ""
"ER2" "FUT" "GLOBEX" "200706" "0.0" "" "" "" "1 D" "2 mins" "0" "TRADES" "1" ""
"QQQ" "OPT" "SMART" "20070615" "45.0" "PUT" "" "" "1 W" "1 hour" "1" "TRADES" "1" ""
"INDU" "IND" "NYSE" "0" "0.0" "" "" "" "1 M" "1 day" "1" "TRADES" "1" ""

The processing of this input file generated the following data files: EUR_CASH_1 day.csv, GOOG_STK_5 mins.csv, ER2_FUT_2 mins.csv, QQQQ_OPT_1 hour.csv and INDU_IND_1 day.csv.

Here is the tail of the QQQQ data file:

20070328  16:00:00,1.92,1.92,1.87,1.91,333
20070328  17:00:00,1.75,1.77,1.75,1.75,28
20070328  18:00:00,1.83,1.83,1.78,1.78,720
20070328  19:00:00,1.85,1.97,1.85,1.93,1944
20070328  20:00:00,1.95,2.03,1.9,2.03,1442
20070328  21:00:00,2.03,2.03,2.03,2.03,5

These are text files which could now be imported into any charting/analysis software or opened with a text editor. The input file had been created by jTWSdump making use of the manual data requests we entered in the graphical interface but it could have been created with a text editor or programmatically.

Here is another example: we manually requested one month of daily data for each Dow Jones Industrials stock using the graphical interface. jTWSdump saved automatically all requests to file dump.txt. After exiting jTWSdump we renamed the file to dump_dow.txt so that it wouldn't be overwritten. The next day we instructed jTWSdump to use the stored dump_dow.txt file and jTWSdump re-generated 30 data files for all Dow stocks in one minute (note that data requests during regular trading hours take more time to complete).

If you are not happy with the default jTWSdump functioning I can customize it prior to delivery.

Requirements

an account with Interactive Brokers with at least one market data subscription
Trader Workstation installed and correctly configured

Agreement

By purchasing the software you agree to use it for personal purposes only (to negotiate a different license contact me). You cannot transfer, rent, lease, lend, copy, share the software and/or documentation. You cannot adapt nor change the software and you cannot re-sell the software. You may not reproduce or distribute any documentation without my permission. You may install a copy of the software on a computer and freely move the software from one computer to another, provided that you are the only individual using the software.

In no event shall I be liable for any special, incidental, indirect, or consequential damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or any other pecuniary loss) arising out of the use of or inability to use the software product or the provision of or failure to provide support services.

Requests

Buy a license.

Free 7-day evaluation.

Share this website.

Questions ? Use the contact form.

Interactive Brokers (IB) is a low cost provider of trade execution and clearing services for individuals, advisors, prop trading groups, brokers and hedge funds. IB's premier technology provides direct access to stocks, options, futures, forex, bonds and funds on over 100 markets worldwide from a single IB Universal account.

Member NYSE, FINRA, SIPC. Visit www.interactivebrokers.com for more information.