I have a collection of dividend paying etf's and cef's ...see below. I have been playing around with using dividend adjusted data (Y!) and non dividend adjusted data (fidelity). I use the same multi symbol rotational code, and backtest, and optimize with the same strategies. So the parameters stay the same, though the values may change.

At this point, the Y!data...to my surrpise...is not adding signficant value. (Of course, with fidelity data I include dividends in the back testing; and, turn it off when using Y! adjusted data.)

This is not highly active trading...but typical of tactical asset allocation.

I am curious if others are focusing on this asset class, and if so, how do they trade it?

QUOTE:
AGG AVK AWF BCV BHD BHK BIV BLV BME BND BNZ BOE BQY BSV BWX BWZ CFT CIU COY CVY DBU DEM DGS DHF DNH DNP DTH DVY EBI EDV ELD EMB EMLC ENY EOI ETB FAM FAX FCO FFC FVD FXA FXB FXC FXE FXF FXM FXS FXY GBF GIM GUT GVI HQH HQL HYD HYG IAF IBND IEF IEI IGD ITM ITR JNK JXI LQD MINT MLN MLPI MLPN MUB NRP PCEF PCY PHB PID PLW PST PVR PZA SCD SGL SHM SHY SPY SUB TBT TEI TIP TLO TLT UTF UTG UUP VCIT VCLT VGIT VGLT VGSH VIG VMBS VRD WIP XLU

You are using a selective subset of ETFs + CEFs. To make for more robust trading, and to mimic real life, you should include all of them (note: you may exclude muni bond ETFs & CEFs if you want to exclude federal tax-free income).

Sammmy, HOw big of a data set would that amount to? Personally, for rotational trading my assumption is that ideally it would be best to have a group tickers with low inter-correlation. Large clusters of similar tickers(i.e., low inter correlation) mixed with smaller clusters can make ranking distorted and more difficult for a strategy to determine a rank violation that is consistently effective over time.

>>HOw big of a data set would that amount to? <<
A little over 1100 symbols.

>>my assumption is that ideally it would be best to have a group tickers with low inter-correlation<<
Any sorting or exclusion of symbols should ideally be at the script level. By cherry-picking symbols, you just accentuate the problem of survivorship bias, besides introducing your own bias(es). And what's to stop you from changing the list again after a few months and adding new favorites/survivors who in backtesting will show trades at a time when they were not even in your list? Be smart, play smart; don't cherry pick your symbols.
Incidentally, AGG <-> BND are highly correlated (there may be many others, too).

sammy g,

My list has not been through a tough screen. Your observation is spot on. I wish that I had access to a correlation matrix tool where I could determine time span, and then weed out what I consider duplicates.(I did a search for this here, and as a non tech person who does not code...not much traction...but really really open to suggestions.)

The list is quick shot of getting in a wide representation without clusters of highly correlated tickers becoming too big.

If I may ask, how do you approach this asset class as an investor? Trading frequency model? Degree of diversification? Percentage of your portfolio? etc.

thx

I treat income-producing securities as a separate asset class i.e. as a separate "bucket" for my money, as they frequently march to their own tune rather than in sync with the stock market and tend to have lower volatility. This serves two purposes:
- income
- diversification
both of which are equally important to me. Granted they don't make for exciting dinnertime conversation like stocks such as Apple, Google (and, in the future, Facebook), etc. but the fact is that over a long period of time (reinvested) dividends account for 50% or more of total return from investing; this fact is sadly not realized by most traders. [Roland is one of the few people on this website that understands this.]

If you post some approximation of the kind of results you have got on backtesting on your dataset (limited though it is), I may be able to suggest an existing script that you could look into for further study.

QUOTE:
the fact is that over a long period of time (reinvested) dividends account for 50% or more of total return from investing; this fact is sadly not realized by most traders.

Not to divert the subject, but traders aren't investors, and you can trade highly-profitable strategies that obtain far less than 50% of the returns from dividends. For example, in the upcoming March issue of AT Magazine, the TSL describes a strategy that gained about 230% over the last 10 years (about 12% APR) trading the current S&P 500 symbols, but only a little over 10% of the returns came from dividends. (Some consolation of getting 50% of your returns from S&P 500 dividends had you truly invested in the S&P 500 over the same period!)

Don't get me wrong though. If you have to decide between two stocks to buy and all else were equal, I'd always choose the one that yields most!

Sammy...


What I agree with is having dividend strategies, as well as allocation, as a separate entity. The data set has its own dividend specific symbols.( I have decided to use only dividend adjusted data from Yahoo premium account. Is there a free resource?) And, there would be a separate portfolio income account dedicated to dividend investing.

Speaking to Cone's point( investing vs trading), the strategies that I am applying to the dividend data set can generate more trading than investing IF I allow for multiple positions when trading criteria is achieved. In such cases I allow for 5 positions, mostly to offer myself choices and to use the strategy more as a tool to make executive decisions. How much of the return is dividends I really do not know, but would like to know. As you know, if the return is adjusted for dividends when it comes to back testing the box in preferences for including dividends needs to be unchecked.
Is there a way to find out the contribution of dividends?

Below is an example of a three year data sample for a momentum/rotation/multi-symbol trading strategy. Long only for all three strategy performances. Due to data history requirements actual trading does not begin until 5/08. Data set includes etfs and cefs. No stocks. By going for a longer data sample, which is what I should do for trading confidence, the problem is that inception has not yet occurred for many of the etfs.

I am considering going to a large data set of stocks in order to have a large historical data sample. If you would like to suggest a data set, or criteria for a data set, I would be interested. Typically, the dividend stock list posted are for a specific year, and change year to year. Ideally, these stocks would be representative of the broad market, including global stocks. This, really, is the advantage of etf and cefs, where currency is diversified.

More and more, my approach comes closer to trading.?
My preference is to hold for three months or longer, though in reality I expect to have to exit a few positions before I find one that holds up for longer.
(note: dollar figures in thousands)

Momentum-5 positions at a time
All Trades
Starting Capital $100
Ending Capital $168
Net Profit $68
Net Profit % 68.03%
Annualized Gain % 18.94%
Exposure 56.93%
Total Commission $0.00
Return on Cash $0.00
Margin Interest Paid $0.00
Dividends Received $0.00

Number of Trades 214
Average Profit $317.88
Average Profit % 1.30%
Average Bars Held 10.85

Winning Trades 133
Win Rate 62.15%
Gross Profit $133
Average Profit $1
Average Profit % 4.22%
Average Bars Held 12.45
Max Consecutive Winners 13

Losing Trades 81
Loss Rate 37.85%
Gross Loss ($65
Average Loss ($802.53)
Average Loss % -3.50%
Average Bars Held 8.21
Max Consecutive Losses 8

Maximum Drawdown ($23
Maximum Drawdown Date 11/20/2008
Maximum Drawdown % -23.42%
Maximum Drawdown % Date 11/20/2008

Wealth-Lab Score 25.47
Sharpe Ratio 1.64
Profit Factor 2.05
Recovery Factor 2.86
Payoff Ratio 1.21
------------------------------
Momentum -1 position
All Trades
Starting Capital $100
Ending Capital $222
Net Profit $122
Net Profit % 122.45%
Annualized Gain % 24.78%
Exposure 57.82%
Total Commission $0.00
Return on Cash $0.00
Margin Interest Paid $0.00
Dividends Received $0.00

Number of Trades 10
Average Profit $12
Average Profit % 8.91%
Average Bars Held 48.3

Winning Trades 9
Win Rate 90.00%
Gross Profit $126
Average Profit $14
Average Profit % 10.32%
Average Bars Held 49.67
Max Consecutive Winners 8

Losing Trades 1
Loss Rate 10.00%
Gross Loss ($3
Average Loss ($3
Average Loss % -3.81%
Average Bars Held 36
Max Consecutive Losses 1

Maximum Drawdown ($34
Maximum Drawdown Date 7/6/2010
Maximum Drawdown % -17.09%
Maximum Drawdown % Date 7/6/2010

Wealth-Lab Score 35.54
Sharpe Ratio 1.67
Profit Factor 32.03
Recovery Factor 3.53
Payoff Ratio 2.71
------------------------------
Mean Regression-1 position
All Trades
Starting Capital $100
Ending Capital $248
Net Profit $148
Net Profit % 148.86%
Annualized Gain % 28.72%
Exposure 62.91%
Total Commission $0.00
Return on Cash $0.00
Margin Interest Paid $0.00
Dividends Received $0.00

Number of Trades 13
Average Profit $11
Average Profit % 8.02%
Average Bars Held 38.62

Winning Trades 10
Win Rate 76.92%
Gross Profit $159
Average Profit $15
Average Profit % 11.34%
Average Bars Held 41.8
Max Consecutive Winners 5

Losing Trades 3
Loss Rate 23.08%
Gross Loss ($10
Average Loss ($3
Average Loss % -3.05%
Average Bars Held 28
Max Consecutive Losses 2

Maximum Drawdown ($17
Maximum Drawdown Date 2/24/2010
Maximum Drawdown % -8.64%
Maximum Drawdown % Date 8/29/2008

Wealth-Lab Score 41.71
Sharpe Ratio 1.69
Profit Factor 15.46
Recovery Factor 8.72
Payoff Ratio 3.72

Phew! That's a lot of information to digest, wphill! Let me see how I can help you, if at all; I am by no means an expert. I will try to address your questions, and also some areas of concern I noticed.

...dividend adjusted data from Yahoo premium account. Is there a free resource?
Yahoo itself is a free resource; I don't believe you need a Premium account to d/l Yahoo daily data. Of course, how long Yahoo remains free is another matter. If/when it gets taken over or decides to bump up its revenues & starts charging for daily data, then problems may arise. But the more important question is - should you be using div-adjusted data for this? (see below)

Is there a way to find out the contribution of dividends?
Yes. Run your backtests on non div-adjusted data and in the settings choose 'Apply dividends to backtest results'.
This raises the interesting question: Is it better to run a backtest on div-adjusted data, or on non-div-adjusted data but with Dividend contributions noted? In my humble opinion, its the latter. Here's why: When you create trades on div-adjusted data, you land up buying more of the shares than you could have in real life for the same dollar amount as the effect of div adjustment is to depress past prices. Also, the dividends are only available as of ex-div date, which may be weeks or months later (in other words, that amount was not available for investing till later and could only have been used for future purchases).
One word of caution: Do *not* use weekly or monthly bars in your backtests when choosing 'Apply dividends to backtest results' as in those instances the Dividends Received amount is unreliable; use daily bars only.

If you would like to suggest a data set, or criteria for a data set, I would be interested.
I assume you already have all the standard datasets, like Dow30, S&P500, ETFs (non-leveraged), ETFs (leveraged), CEFs, etc. If you can't find the requisite components, let me know; I have most of them. The dataset you choose for a script depends on the task at hand; if you are looking for federal tax-free income, you would use one containing only muni bond ETFs & CEFs, or if you are looking for high yields then you would want a dataset of ETFs and CEFs with muni bonds excluded, and so on.
But you wrote: the dividend stock list posted are for a specific year, and change year to year If you are changing your dataset every year, then you are not going to get results that you could rely on in the future with any degree of confidence. Its best to have a broad dataset (an ETF dataset with over 1000 symbols is pretty broad, for example) and then do any filtering inside the script to pick potential trading candidates.
Regarding CEFs, there are a couple of issues to be aware of:
- many use leverage; this a double edged sword as it can boost yields (and price performance) but on dividend cutbacks can really slaughter the stock (e.g. AOD last year)
- they may trade at a discount or premium to their NAV; for long-term investing, its important to know this info (unfortunately, this data is not available through Fidelity; you can find it on some websites or in professional quote services) but for short-term trading one can overlook that
*/

Your backtest results are very good (but you should always include commissions and appropriate slippage to better mimic real life); if you can maintain those in the future, you will be all set. Note that the results for S&P 500 for the last 100 years have averaged about 8%/year, including reinvested dividends.
But I am not sure which if any of those strategies are geared towards income (the topic of your thread); it seems to me they are designed for capital appreciation.
*/

You probably don't need this but the script that I referred to in my last post is one designed for dividend symbol rotation: Dogs of the Dow [Rev A] (don't use the original version). Even though it was originally created for the Dow, you can use it for rotating ETFs and CEFs, or any other dataset with dividend paying symbols. Be wary, though; if you backtest it against current S&P500 and S&P400 datasets, the results may be very impressive but I think they reflect a pronounced survivorship bias; also, the drawdown is just horrible. Those two factors are less of a concern with an ETF/CEF dataset.
You wrote above that your preference is to hold for 3 months; you can easily adapt the script to trade quarterly instead of yearly but if you are unable to do so just ask.

QUOTE:
Also, the dividends are only available as of ex-div date, which may be weeks or months later (in other words, that amount was not available for investing till later and could only have been used for future purchases).

This isn't correct. In fact, dividend payouts are known well in advance of their ex-dates - even in WLP.

QUOTE:
One word of caution: Do *not* use weekly or monthly bars in your backtests when choosing 'Apply dividends to backtest results' as in those instances the Dividends Received amount is unreliable; use daily bars only.

I want to clarify this. Counting dividends is a guessing game for stop and limit strategies for scales higher than Daily. That's because a stop and limit orders can trigger on any day of the week or month, but when the bars are weekly or monthly, there's no way to tell if you actually owned the stock for the dividend ex date. In these cases, WLP assumes that the dividend is not paid (more conservative). Consequently, Sammy is correct when referring to stop/limit Strategies.

Sammy..

QUOTE:
This raises the interesting question: Is it better to run a backtest on div-adjusted data, or on non-div-adjusted data but with Dividend contributions noted? In my humble opinion, its the latter.

Your elaboration shows just how complex thing are...items that I had not even thought of.


RE: whether or not use div adjusted data....my concern is that on the day that a ticker has a big drop in stock price entirely due to going ex-dividend my coded script will reflect that there is a violation and needs to be sold at the close. A violation is due either to a (ATR)stoploss or to rank. In fact, I am not even sure that Y! adjusts the price on the close of that day.

RE: how to find out how much in the way of dividends are collected...good suggestion, after I get through backtesting and getting the parameter values of choice, I can then change my preference to include dividends. The performance will be upwardly distorted, but I can at least see how much in the way of dividends are being collected.

Re: Yahoo. Yes, you are probably right in that I can get div adjusted data without going to a premium for fee account. I originally went this route in order to be able to run a strategy at near the close of the mkt day in order to make a trade per strategy outcome and then try to get the price at the close. My experience has been that next day trading can be more expensive...but that is a different topic.

Re: daily data. The coded script trades weekly for buys...a sell can happen any time. All data, though, is daily. Thanks for the heads up because sometimes I do like to just eyeball a chart on weekly data.


Re: data sets. I do want a stable data set. Change the data set, and performance can dramatically change. If this is what you are pointing out, I totally agree. On the other hand,I see this as a major limitation of the symbol rotation approach that I am using. Curve fitting has to be considered as a danger.

Re: leverage or not...my thinking is that cef's and etfs have a lot of underlying factors and that I need to just let the ranking mechanics of the code do the job. However, if the volatility is too great...due to leverage or whatever...then maybe it should be excluded from the data set or rejected vis a vis the code using beta as the metric...if that makes sense.?

Re: an adjustment to the code....I may take you up on your offer.

Meanwhile, I'm going to keep doing a lot of back testing. My confidence in the strategy values increase when I include 3 to 5 positions...as opposed to just one...and winning rate is above 60%, there are few outliers, apr% is above 18%, and the sharpe ratio is above 1.50. The code is only a tool; I do not expect to trade it on an entirely mechanical basis. When a new position is opens it remains up to me to make the decision.

I have a new laptop that runs the backtesting a lot faster than my desktop. I have to do some transferring of wlab files to the laptop. I'll post what I next find.

The more that I get into investing/trading the more that I am impressed with all the balls "experts" juggle. They can wrap their heads around the fundamental and technical factors, point out the pitfalls....yet, when it comes time to posting returns on portfolio's...wow, not necessarily so impressive. That's another topic.

Personally, I'm glad to come around to looking at dividend rich data sets for trading/investing.

Thanks!!!

To wphill:
Re: data sets. I do want a stable data set. Change the data set, and performance can dramatically change. If this is what you are pointing out, I totally agree. On the other hand,I see this as a major limitation of the symbol rotation approach that I am using. Curve fitting has to be considered as a danger.
If you have only a limited number of symbols in your dataset, and they are there probably because they gave good results on backtesting, then you have already involuntarily done 'curve fitting' (or should it be called result fitting?) You also run the risk of omitting symbols that may become important in the future; they may exist now or may represent new ETFs/CEFs released down the road. And just to be clear: Nobody is talking about curve fitting (commonly understood to mean choosing some specific indicators or values because they gave better backtest results than other values, etc.); instead, its about the appropriate filters e.g. I routinely use price and volume filters; its important that the symbol have some minimal price and liquidity before I would trade it.

...or rejected vis a vis the code using beta as the metric...if that makes sense.? Absolutely.
Well, good luck in your trading and post as needed.
-------------------------------

To Cone:
This isn't correct. In fact, dividend payouts are known well in advance of their ex-dates - even in WLP.
I am not quite sure what statement of mine you were referring to as incorrect. Perhaps there's a misunderstanding. I wrote: "Also, the dividends are only available as of ex-div date, which may be weeks or months later " This is true. Dividends are only posted to your account on or soon after the ex-div date, never before. Even if you know the date, you can't get them posted to your account any sooner. [Besides, the info, if available, is only known a few days in advance and even then its not consistent across all the thousands of symbols that pay divs; certainly, its not known months in advance for those symbols that pay only quarterly or six monthly dividends.]
The other half of my sentence was: "(in other words, that amount was not available for investing till later and could only have been used for future purchases)." This is self-evident; you can't use dividends for investing unless and until they have been posted to your account, and even then you will likely use them on next rebalancing.
Perhaps a simple example should clarify things; Say there's a stock that begins the year at a price of 10 and ends the year at 11; at June-end, it pays a dividend of $1/share.
a) Using div-adj prices: If someone used div-adjusted data as a basis for trading and they put $10,000 in the trade, their performance would have been:
Div-adjusted price at year-begin = 10-1 = 9.
Shares bought = 10000/9 = 1111.
Profit on sale at end of year = 1111 x (11 - 9) = 1111 x 2 = 2222 (no dividends received as they went into adjusting the price).
b) Using regular prices:
Shares bought = 10000/10 = 1000
Profit on sale at end of year = 1000 * (11 - 10) = 1000 x 1 = 1000.
Dividends received = 1000 shares x $1/share = 1000.
Total return at end of year = 1000 + 1000 = 2000.
One could factor in some interest earned on the dividend, but even then there's a 10% error in Total Return between the 2 methods, and this would get compounded over time. Also, the dividend received mid-year would sit in the account; it would only be invested when the next rebalancing was performed.
That's what I had meant. Hope this clarifies things.
*/

As for the other point:
Counting dividends is a guessing game for stop and limit strategies for scales higher than Daily. True, but there's a modestly complex solution which the developers can implement, should they choose to do so. [Call this wishful thinking, if you will.] And that is: Since weekly and monthly bars are derived from daily bars, it follows that daily data exists; an algo could be written that could seek out the daily bars and determine when the stop/limit orders filled. Remember the TIntraDay script in WL4 that would access the intraday data when trying to decide which of the (multiple) fills of an EOD script simulation would actually have filled in real life? Its really the same idea expanded to a higher bar scale.