CBOE Put/Call Ratio static data provider
Author: Eugene
Creation Date: 5/29/2010 6:45 AM
profile picture

Eugene

#1
CBOE static data provider, which is a part of Community.Providers, has been released for download from our web site. Now you can download the historical CBOE Put/Call Ratio data to your Wealth-Lab Developer/Pro, for free.

CBOE Put/Call Ratio static data provider - installation
CBOE Put/Call Ratio static data provider - home page
profile picture

Eugene

#2
In Wealth-Lab Pro/Developer's "Open Strategy" window, click "Download..." to download an example strategy based on a Trading System Lab article (Active Trader Magazine, 10/2007):

ActiveTrader Mag 10-2007 | CBOE Put/Call Ratio
profile picture

Eugene

#3
CBOE static data provider updated to 2010.09

Summary of changes:

Added: new symbol - CBOE S&P 500 Index (SPX) Put/Call Ratio
Changed: possible to install in Wealth-Lab 6
Fixed: failure to reload chart
Fixed: total volume for VIX Put/Call ratio

Wealth-Lab 6 customers, please install the provider from the Extensions section of our site.
profile picture

gfparker

#4
I downloaded the CBOE extension and it seemed to install ok. I did an update on CBOE and data manager reported adding 1728 bars but when I go to display the data in a new chart I get No data available. I tried the update again in data manager but it reports all data in the provider CBOE is up to date. I'm using WL Pro 6.0, what am I doing wrong?
profile picture

Eugene

#5
Check your data loading range - start with All Data.
profile picture

gfparker

#6
Found the problem, when I opened the CBOE chart window, I already had another chart open with streaming data from Fidelity. The CBOE chart then defaults to streaming and won't show any historical data. Once you turn off streaming for the CBOE chart the historical data shows up,
profile picture

Eugene

#7
Something I haven't thought of but should. ;)

When you turned Streaming on, active Streaming provider backfills the chart (if it has ability to backfill) by requesting the static data from the "companion" historical data provider. For Fidelity it's Fidelity static, for Yahoo RT - it's Yahoo static and so on. Neither Fidelity nor any other provider would recognize the artificial symbol names created by the CBOE provider, that's why the chart will always be blank while streaming is enabled.
profile picture

Eugene

#8
CBOE static data provider updated to version 2011.01 on 01/12/2011.

This is mostly a maintenance release fixing some bugs (like potential removal of penultimate bar when no data was received during update), and adding a MarketInfo.

Update to Wealth-Lab 6.1 to be able to install and use the provider.
profile picture

Eugene

#9
CBOE static data provider updated to version 2011.06.

Note: this update is required for any CBOE provider user: yesterday the vendor's changed their data format, causing the provider to crash Wealth-Lab. This bug is fixed.
profile picture

Eugene

#10
CBOE static data provider updated to version 2011.09.

* Added: Support for volatility indices: VXV, VIX, VXN, VXD (based on request by MrIzzy)
* Changed: possible to install in Wealth-Lab 6.2+
profile picture

Eugene

#11
CBOE static data provider updated to version 2012.01.

* Added: SKEW index (requested by richard1000)
profile picture

Eugene

#12
CBOE static data provider updated to version 2014.12.

* Added: VXMT index (requested by AbsoluteReturn)
* Fix: data not downloaded or downloaded with errors if Windows decimal separator is not "."
* Change: (internal) multithreaded download routine rewritten. Should improve work on modern multi-core CPUs (e.g. servers)
* Change: possible to install in Wealth-Lab 6.6+
profile picture

Eugene

#13
rmandel00 wrote in a thread that no longer exists:


I am interested in monitoring the term structure of the VIX futures. We already have an extension that allows to import and daily update a number of CBOE indicators like the Put/Call Ratio and a few of the VIX ETNs.While the closing values of the daily VIX futures are available on the CBOE website, it is apparently not possible to download them directly to WLP. I have tried using Quandl extension to get the VIX futures but they are always one day behind. Can the CBOE extension be modified to import this data? Do you have any other suggestions about obtaining up to date daily VIX futures.
Thanks,
Ricardo
profile picture

Eugene

#14
Ricardo,

If you take a minute to review the Wiki documentation and this thread, you'll see that VIX is downloadable by this provider since 2011.
profile picture

rmandel00

#15
Eugene,
I think what you are referring to is:
http://www2.wealth-lab.com/wl5WIKI/CBOEStaticProvider.ashx
including VIX, VXD, VXN VXV.

I am interested in the futures that have symbols such as /VXH7, /VXJ7 etc. Each month uses one of
the following letters: F, G, H, J, K, M, N, Q, U, V, X, Z. These are all available on CBOE but apparently not on the
CBOE extension. Any way to get them from CBOE extension?
profile picture

Eugene

#16
Thank you for pointing me in the right direction. I've found what you're looking for on CBOE website: VX - CBOE S&P 500 Volatility Index (VIX) Futures Price and Volume Detail. Yes, this is not supported by the provider currently.

If I were to consider, are you OK with the monthly contracts as is? I do not want to get into the hassle of continuous contract creation and back-adjustment.
profile picture

rmandel00

#17
I would be happy with the current (from the current month up to 4 months into the future). It is more difficult to specify and download all the old /VX futures than to just download the current ones that are trading at any time. It seems like currently traded VIX futures go up to 9 months into the future. I have downloaded present and past futures but can't update until one day late. Hence, trades that require closing prices are delayed by one day.

Would it be possible to get the code for the CBOE extension and modify it myself to give me the desired results?
Thanks,
Ricardo
profile picture

Cone

#18
Suggestion for creating a continuous contract for /VX...

1. Use the splice method. Splicing will create a gap in the contract, but it's easily predicable for strategies.
2. Rollover on the Friday before the 3rd Thursday's expiration.

Here's a nifty routine for finding that rollover day for a specified month and year:
CODE:
Please log in to see this code.
profile picture

Eugene

#19
Thanks for the code Robert. A tweak is to be made to account for offbeat expiration calendar of VX futures (on third Wednesday) with some exceptions. For some odd reason, expiration date this March is on 4th Wednesday (22).

Simple splicing is the way to go; just one VX continuous contract would make things simple. I'll look into it later next month.

Ricardo, this extension is closed source and therefore its code isn't disclosed. The data is indeed delayed by one day which is a CBOE policy: "CFE Data does not become available until approximately 10:00 a.m. C.T. the following business day."
profile picture

rmandel00

#20
Eugene and Cone, Thanks for your information. I do need to compare each monthly future so one continuous contract doesn't really work for me.
Ricardo
profile picture

Eugene

#21
Okay. I'll see what can be done about adding monthly contracts.
profile picture

rmandel00

#22
Eugene,
I really appreciate that you will look to see if the monthly contracts can be added. I don't need the weeklies. The URL page where the daily data is first updated is:
http://cfe.cboe.com/data/dailyvxfutureseodvalues/default.aspx

The historical data at CBOE takes an additional day before it is posted.
Ricardo
profile picture

Eugene

#23
That's just settlement prices. I already looked that up and will not consider them (as an incomplete source). Unfortunately, CBOE doesn't seem to release OHLC until the day after.
profile picture

rmandel00

#24
Thanks for looking it over.
Ricardo
profile picture

Eugene

#25
Ricardo, I will implement it in the upcoming 2017.04 build. Please stay tuned.
profile picture

Eugene

#26
Ricardo, please update the extension and restart WLP. Support for downloading the individual VX Futures contracts has been added. Please refer to the Wiki page for more information:

CBOE static data provider, see section at the bottom
profile picture

Harapa

#27
Symbol Change:
'VXV' is now 'VIX3M'
profile picture

Eugene

#28
Thanks for the heads-up.
profile picture

Eugene

#29
What's new in 2017.12?

* Change: VXV renamed to VIX3M at CBOE
profile picture

Eugene

#30
What's new in 2018.01?

* New: added support for CBOE Bitcoin futures (XBT) as well as some other futures
profile picture

Eugene

#31
Looks like the data for XBT futures hasn't been popular. CBOE has changed its location on the website along with symbology earlier this year and no one noticed. Since it's not a requested feature I will not invest time into it (and simply I just don't have it right now). Old data files (early 2018) can still be downloaded by the provider so nothing's broken.
profile picture

Eugene

#32
What's new in 2019.01?

* Change: VXMT renamed to VIX6M at CBOE
* Change: removed timeout for data requests
* Change (internal): upgraded browser user-agent version and referer
profile picture

Eugene

#33
What's new in 2019.07?

* Fix: gap in the TOTALPC data in between 2003 and 2006
profile picture

Harapa

#34
Provider is not updating PC ratios in addition to 3 and 6 month VIX (VIX3M & VIX6M).
profile picture

Eugene

#35
Updating DataSet CBOE Put/Call Ratio ...
Up-to-date symbols: 0, Update required for: 0, New symbols: 11
Updating new symbols...
Symbol: SPXPC, Bars: 2330, Last date: 04.10.2019
Symbol: INDEXPC, Bars: 3253, Last date: 04.10.2019
Symbol: EQUITYPC, Bars: 3253, Last date: 04.10.2019
Symbol: SKEW, Bars: 7499, Last date: 22.10.2019
Symbol: VIX3M, Bars: 2992, Last date: 22.10.2019
Symbol: VIX6M, Bars: 2624, Last date: 07.06.2018
Symbol: VIX, Bars: 3979, Last date: 22.10.2019
Symbol: VIXPC, Bars: 3427, Last date: 04.10.2019
Symbol: VXN, Bars: 4708, Last date: 22.10.2019
Symbol: VXD, Bars: 5546, Last date: 22.10.2019
Symbol: TOTALPC, Bars: 6041, Last date: 04.10.2019
Update completed (3.272 sec)
----------------

It's not the "provider" but the "website". The raw data for affected symbols ends at 10/04/2019. It's because of this breaking change:

Effective Oct 8, 2019 Cboe Daily Market Statistics will be found at Cboe Daily Market Statistics.

From this date on, the data is presented on a webpage rather than in the CSV archive. I assume it's done to prevent web scraping. The CBOE website contains such a "friendly" disclaimer (and based on my experience, it's not very far from reality):

PLEASE NOTE: IT IS STRICTLY PROHIBITED TO DOWNLOAD DELAYED QUOTE TABLE DATA FROM THIS WEB SITE BY USING AUTO-EXTRACTION PROGRAMS/QUERIES AND/OR SOFTWARE. CBOE WILL BLOCK IP ADDRESSES OF ALL PARTIES WHO ATTEMPT TO DO SO. (...)

Adjusting to this breaking change may be possible but makes little sense. From 10/4/2019 on, each day's data would have to be requested individually. Sadly, the new experience would be worse:

1. As time goes by this would slow down the initial data backfill.
2. Random gaps in the data should web requests and re-requests fail.
3. The user is exposed to IP address ban at CBOE.

Let's revisit it next month. If CBOE returns to update their data archive at least on a monthly schedule I'm fine with this and will not update the provider. Otherwise I may need to figure something out.
profile picture

Eugene

#36
Good news: the provider will be restored in v2019.11. It will use the daily market statistics pages to get the data after 10/04/2019.
profile picture

Eugene

#37
What's new in 2019.11?

* Fix: breaking change at CBOE -- no history after 4 October 2019
* Fix: VIX6M symbol not updated since June 2018

IMPORTANT:

1. You must Update DataSet / Update All Data to get new data. On demand data update does NOT work (the data is stuck in 10/4/2019). This is not a bug.
2. Volume is missing in the data from October 2019 onwards (i.e. zero). This is not a bug.

P.S. Unless CBOE changes their mind and returns to update the data archive, all my concerns will hold true: slowness over time, gaps in the data and IP address ban are all possible.
profile picture

Eugene

#38
Version 2019.11.1:

* Hotfix: noticed a minor change to date format at CBOE website which made the provider skip bars

THIS IS A MANDATORY UPDATE.

Users of v2019.11 should reload the data if already updated.
profile picture

Harapa

#39
Version 2019.11.1, not updating PC data for NOV 01 (last bar).
CBOE https://markets.cboe.com/us/options/market_statistics/daily/ displays NOV 01 data.
profile picture

Eugene

#40
You should restart WLP after installing Extensions. Apparently you haven't done so. It works for me:

Updating DataSet CBOE Put/Call Ratio ...
Up-to-date symbols: 0, Update required for: 0, New symbols: 5
Updating new symbols...
Symbol: SPXPC, Bars: 2330, Last date: 04.10.2019
Symbol: INDEXPC, Bars: 3253, Last date: 04.10.2019
Symbol: EQUITYPC, Bars: 3253, Last date: 04.10.2019
Symbol: VIXPC, Bars: 3427, Last date: 04.10.2019
Symbol: TOTALPC, Bars: 6041, Last date: 04.10.2019

Updating data after 10/04/2019 for supported symbols from Market Summary snapshots...
Downloaded snapshot data...
Symbol: EQUITYPC, Bars: 20, Last date: 01.11.2019
Symbol: INDEXPC, Bars: 20, Last date: 01.11.2019
Symbol: SPXPC, Bars: 20, Last date: 01.11.2019
Symbol: TOTALPC, Bars: 20, Last date: 01.11.2019
Symbol: VIXPC, Bars: 20, Last date: 01.11.2019
Update completed (22.649 sec)
----------------

profile picture

Harapa

#41
I had, nevertheless I restarted WL. But no updates.
QUOTE:
Updating DataSet CBOE Put/Call Ratio ...

Up-to-date symbols: 0, Update required for: 11, New symbols: 0
Symbol: SPXPC, Bars: 0
Symbol: INDEXPC, Bars: 0
Symbol: EQUITYPC, Bars: 0
Symbol: TOTALPC, Bars: 0
Symbol: SKEW, Bars: 7507, Last date: 11/1/2019
Symbol: VIX6M, Bars: 0
Symbol: VIX3M, Bars: 0
Symbol: VIXPC, Bars: 0
Symbol: VIX, Bars: 0
Symbol: VXD, Bars: 0
Symbol: VXN, Bars: 0

Updating data after 10/04/2019 for supported symbols from Market Summary snapshots...
Update completed (2.249 sec)
----------------


For PCs I have 10/31/19 bar after gap since 10/4/19. Extension manager says 2019.11.1. I have only one copy of WL on this machine.
QUOTE:

2. Have you got the bars for Oct 7-9, 2019? If not, you're running the "wrong" version 2019.11.
profile picture

Eugene

#42
1. Have you checked the extension version using the Extension Manager? The update might have been installed into WLP of different bitness, if you have it side by side.

2. Have you got the bars for Oct 7-9, 2019? If not, you're running the "wrong" version 2019.11.

At any rate, clear entire Put/Call data using the Data Tool ("Remove all data"), restart WL, retry update.

Note that the way versions 2019.11+ work they cache those daily snapshot pages to memory. If you ever get something like this in the log, either a WL restart or rollover to a new work day (not weekend) may help to clear the in-memory cache:
QUOTE:
Updating data after 10/04/2019 for supported symbols from Market Summary snapshots...
Update completed (2.249 sec)


As a final suggestion, try a different network connection. I haven't had any issue with my connections during the week of testing but your mileage may vary.
profile picture

Harapa

#43
Issue fixed. Thanks for the help. Can you add OEXPC, may be in the next update?
QUOTE:
At the risk of repeating, clear entire Put/Call data using the Data Tool ("Remove all data"), restart WL, retry update


QUOTE:
Updating DataSet CBOE Put/Call Ratio ...

Up-to-date symbols: 0, Update required for: 0, New symbols: 11
Updating new symbols...
Symbol: EQUITYPC, Bars: 3253, Last date: 10/4/2019
Symbol: INDEXPC, Bars: 3253, Last date: 10/4/2019
Symbol: SPXPC, Bars: 2330, Last date: 10/4/2019
Symbol: SKEW, Bars: 7507, Last date: 11/1/2019
Symbol: VIX3M, Bars: 3000, Last date: 11/1/2019
Symbol: VIX6M, Bars: 2978, Last date: 11/1/2019
Symbol: VIX, Bars: 3987, Last date: 11/1/2019
Symbol: VIXPC, Bars: 3427, Last date: 10/4/2019
Symbol: TOTALPC, Bars: 6041, Last date: 10/4/2019
Symbol: VXD, Bars: 5554, Last date: 11/1/2019
Symbol: VXN, Bars: 4716, Last date: 11/1/2019

Updating data after 10/04/2019 for supported symbols from Market Summary snapshots...
Downloaded snapshot data...
Symbol: EQUITYPC, Bars: 20, Last date: 11/1/2019
Symbol: INDEXPC, Bars: 20, Last date: 11/1/2019
Symbol: SPXPC, Bars: 20, Last date: 11/1/2019
Symbol: TOTALPC, Bars: 20, Last date: 11/1/2019
Symbol: VIXPC, Bars: 20, Last date: 11/1/2019
Update completed (25.873 sec)
----------------
profile picture

Eugene

#44
So as expected, WL restart and data purge have helped. That's good news to hear. Note this line in the log:

Downloaded snapshot data...

Your previous log didn't have it:

Updating data after 10/04/2019 for supported symbols from Market Summary snapshots...
Update completed (2.249 sec)


It may mean that the snapshots were downloaded into memory before but were incorrect (e.g. temporary server error condition) or something. In this case WLP restart is now required to get updates to clear "bad" data from the memory cache of v2019.11+

QUOTE:
Can you add OEXPC, may be in the next update?

I cannot find OEXPC archive data so CBOE must have added it on Oct 7, 2019 as a new data item. Given that it has only 20 bars of history (as of today), do you believe it's worth the trouble?
profile picture

Harapa

#45
QUOTE:
It may mean that the snapshots were downloaded into memory
FYI:
When I first installed 2019.11, I did see update of a single bar for PCs. Next day it didn't update. By than 2019.11.1 was available. I updated to this version via extensions manager (as WL restarts is required by default for the installation) and restarted WL to complete installation but didn't see PCs updating. I than removed CBOE data set (without removing data file), recreated data set a fresh. Updated again but no PCS (as shown in the first snapshot). In between (trying to update) I had restarted WL few times for other reasons. So restarting to clearing the memory didn't work for me. I also looked at the data file(s) and it had only one bar after 10/4/2019.

QUOTE:
OEXPC...Given that it has only 20 bars of history (as of today), do you believe it's worth the trouble?

For some reason many providers have stopped updating this and CBOE is the one of the few available sources. For me this would be very helpful as I have older data and I know how to patch that data in one update-able file in WL.
profile picture

Eugene

#46
I will add support for OEX (OEXPC) and Exchange traded products (ETPPC) Put/Call Ratios starting from v2019.12. It will appear as a new checkbox "Exchange traded products and OEX" in the Wizard.
profile picture

Harapa

#47
Thanks.
profile picture

Eugene

#48
What's new in 2019.11?

* New: added PutCall Ratio for OEX (OEXPC) and Exchange traded products (ETPPC) by request of Harapa
profile picture

Harapa

#49
Thanks.

OT:
Are options under 'Future' for this provider active or a place holder for future feature? For example is VIX under this tab is meant to pull VIX futures data?
profile picture

Eugene

#50
Please refer to provider's online user guide > "Futures for Bitcoin, Volatility Index and other"

Due to lack of demand and cost of maintenance (i.e. changes at CBOE website), the data is provided as is. It may be out of date as of today.
profile picture

Eugene

#51
What's new in 2020.04?

* Fix: cannot finish DataSet creation after checking and unchecking Futures checkboxes in Wizard dialog
* Change: requires .NET 4.6.2+
profile picture

Eugene

#52
What's new in 2020.05?

* Change: Initial data update is made more reliable and considerably faster.
* Change: Daily update will retry on error getting daily snapshots.

It's highly recommended to upgrade to this version if you're just starting to use CBOE data or update it after a long period of inactivity. Also, existing users should upgrade to this version and then reload chart history to make sure it doesn't have gaps!
profile picture

Eugene

#53
Support for downloading CBOE Futures (such as Bitcoin) will be withdrawn in a future build.
profile picture

Eugene

#54
None of the VIX volatility indices are up to date due to a breaking change at CBOE. The data can no longer be updated using the WL6 provider. This will be fixed in upcoming WL7's data provider.

The Put/Call Ratio data works correctly.
profile picture

giorgos

#55
My Put/Call Ratio data stopped updating after the end of March. Has something changed?
profile picture

Eugene

#56
Yes, there was a breaking change on the CBOE website. We've already accounted for this in an extension update for Wealth-Lab 7, the DataExtensions (in Build 5 - to be released probably next week):

https://www.wealth-lab.com/extension/detail/DataExtensions#information

Sorry, there is no plan to work on any fixes for the legacy extensions (WL6).
profile picture

Eugene

#57
Another breaking change to CBOE data. As announced previously, there will be no fix for the legacy extension (WL6). For Wealth-Lab 8, look forward to build 7 of DataExtensions with the fix (soon)!
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).