COT data provider questions and suggestions
Author: andriuking
Creation Date: 3/28/2013 9:14 AM
profile picture

andriuking

#1
Hello.

I used the aplication 'Free COT Data static provider' to load the data and when I plot it the data is not well loaded.

Appears as it:



Is not well loaded. Also, when I open the file is as this:



¿what I´m doing bad?

Thanks!
profile picture

Eugene

#2
Andres,

The provider is working correctly and there's no error.

Here's what you're doing wrong:

1. The file you're looking into (COTC.PRN) represents Briese's %CoT Index for Corn - not the actual COT data. Whereas you should be looking at the files starting with "Q..." i.e. QC.PRN. In other words, you're confusing a 2-3 digit index's value with the actual Commitments of Traders data - which will always appear a much greater number as it represents open interest (i.e. number of actual futures contracts held by traders).

2. The COT provider wasn't meant for charting. I urge you to read "How data is stored" on this page to understand how the provider uses OHLC and named series to store COT data:

Free COT data provider

Hope that helps.
profile picture

andriuking

#3
Hello again Eugene.

I´m sorry, but I´m not sure if I´m understanding well.

Then, COT Data Provider don´t gives commodities futures quotes?

Thanks!
profile picture

Eugene

#4
Bingo! No, it doesn't and never did. It gives very specific futures sentiment series called "COT data" (commitments of traders). Look up in Investopedia etc. what this means.

Futures data can be downloaded using different providers like IQFeed (paid data, complete solution) or TradingBlox/PiTrading (these are free but their vendors have stopped updating the data). Or, you can purchase futures data somewhere else in ASCII or Metastock formats.
profile picture

andriuking

#5
Thanks very much for your help Eugene, now I undestand well.

I used TradingBlox / PiTrading in the past, but as you commented they have stopped updating the data.

I found it in Quandl.

Regards.
profile picture

Eugene

#6
Good for you. Re: Quandl, here's my view on their data quality and restrictions: Quandl data provider request
profile picture

ReneW

#7
I have a question related to the "missing" Open Interest (OI) data.

In my opinion, its supposed to be the sum of all longs + spreads.
The actual COT provider does not deliver OI and Spreads.
Therefore its not possible to calculate the correct OI.

Can that be changed ?

Thank you, Rene


As an example I have attached a very old COT data version (December 2005) , with the "SPREADS" and OI data points :


WHEAT - CHICAGO BOARD OF TRADE Code-001602
FUTURES ONLY POSITIONS AS OF 12/06/05 |
--------------------------------------------------------------| NONREPORTABLE
NON-COMMERCIAL | COMMERCIAL | TOTAL | POSITIONS
--------------------------|-----------------|-----------------|-----------------
LONG | SHORT |SPREADS | LONG | SHORT | LONG | SHORT | LONG | SHORT
--------------------------------------------------------------------------------
(CONTRACTS OF 5,000 BUSHELS) OPEN INTEREST: 296,112
COMMITMENTS
52,967 110,978 22,933 192,723 127,675 268,623 261,586 27,489 34,526

CHANGES FROM 11/29/05 (CHANGE IN OPEN INTEREST: 4,283)
4,413 8,724 -1,673 6,686 1,646 9,426 8,697 -5,143 -4,414

PERCENT OF OPEN INTEREST FOR EACH CATEGORY OF TRADERS
17.9 37.5 7.7 65.1 43.1 90.7 88.3 9.3 11.7

NUMBER OF TRADERS IN EACH CATEGORY (TOTAL TRADERS: 302)
85 127 53 52 64 166 222
profile picture

Eugene

#8
It's highly unlikely that this could be changed. Like I replied earlier to your ticket, the provider processes the "Futures only COT data" in the so called "legacy" format which does not contain neither OI nor spreads. Just the six series as indicated in the Wiki: How data is stored.

I don't know where you got these non-essential fields from but maybe this paid 3rd party extension could provide you with the data you're looking for?

COTCollector Static Data Provider

P.S. Actually, I see no trace of OI even in the "newer" file formats on the commitmentsoftraders.org website - could you point me at them if I'm mistaken?
profile picture

ReneW

#9
Honestly, I had not seen any COT report without Open Interest before,
and therefore I am still surprised, that this data is not contained or delivered here
with the COT Data provider.

When accessing the mentioned website via the View-Menu the reports shown are ok.
Why this data is not contained in the downloadable zip -files or where it is hidden... I dont know...
Couldn't you contact the author because I assume that you must have his allowance
to provide this data ?

Rene

profile picture

Eugene

#10
QUOTE:
Couldn't you contact the author because I assume that you must have his allowance to provide this data ?


We only created the tool whose purpose is basically to use your computer to download and unzip the file directly from the author's website onto your hard drive. From a technical/legal standpoint, your work with the provider is not different from downloading the ZIP file using a browser.

Since we do not redistribute and/or alter the data anyhow, we must not have anyone's allowance to provide the tool (not the data!) to download some publicly available data. If you have some questions or suggestions to the author, the best is to contact him directly.
profile picture

ReneW

#11

I asked the author S.Briese - but I dont expect an answer.

But thank you for the corrected COT provider - the named series
is working now.
profile picture

ReneW

#12
Got an answer by Steve Briese himself:
" We have no use for OI - add the longs and use the non spreading Open Interest".

OK, with this data source the problem of the missing OI does persist and
a potential user has to live with this restriction.
I would recommend adding a note to the COT's data WL wiki entry.

Rene



profile picture

Eugene

#13
Rene,

You can find complete CoT series including Open Interest at Quandl. For now, the data can be exported to a format readable by Wealth-Lab i.e. CSV (ASCII) if you have an account there. I'm happy to announce that in near-term you'll be able to utilize this data in Wealth-Lab directly.

P.S. Observation: while at the CFTC website, the raw files are recent (Nov 2014), at the moment (19 Nov 2014) some CoT data at Quandl is out of date. For example, Brent Crude Oil is actual (15 Nov 2014) but ight Sweet Crude Oil is not (21 Sep 2014).
profile picture

Eugene

#14
Good news:

CoT data in both legacy and new formats, including Open Interest, is downloadable using new Quandl provider suite:

Quandl static provider + Zacks fundamental provider