- ago
Using B86.
There are strange things a-happening with Alpha and Beta. I first noticed them a few builds ago. Let me share my observations.

But first, to get everyone up to speed, here's the formula for calculating Alpha showing how it's interlinked with Beta:
Alpha of portfolio = Actual rate of return of portfolio – (Risk-free rate of return + Beta * (Market return – Risk-free rate of return)
----------------------

Test Strategy:
We'll use a very simple Buy@Market test code and use symbol QQQ for testing and SPY as benchmark// Daily scale:
CODE:
using WealthLab.Backtest; using System; using WealthLab.Core; using WealthLab.Indicators; using System.Collections.Generic; namespace WealthScript13 { public class MyStrategy : UserStrategyBase { //create indicators and other objects here, this is executed prior to the main trading loop public override void Initialize(BarHistory bars) { } //execute the strategy rules here, this is executed once for each bar in the backtest history public override void Execute(BarHistory bars, int idx) { if (!HasOpenPosition(bars, PositionType.Long)) { PlaceTrade(bars, TransactionType.Buy, OrderType.Market); } } //declare private variables below } }


Settings: We'll use 5 years as test period:



OBSERVATIONS
1. Aberrant (and incorrect) value of Alpha in Short Only column for a Long only strategy:

There's a value for Alpha in the Short Only column even though there are no Short trades. This value varies with the test period.
If you flip this from a Buy at Market to a Short at Market strategy the identical value will appear in that row under Long Only column.
That row, like adjacent rows in that column, should be blank (-).

My guess is its the Risk-free Rate of Return that's getting displayed here (as a -ve value); if one looks at the equation for Alpha above everything else gets annulled except this.
Note: In Preferences the Interest on Cash/ 'Use UST Yield' box is *not* checked (arrow). Also see #4, below.


2. Checking the box for 'Use UST Yield' causes strange numbers to appear in Shorts Only column, even though its a Long-only strategy, but makes no difference to the Strategy's results.


These values vary with the yield metric chosen and period used.
Oh, and check what I got for a 5-yr backtest using FiveYear yield index:

(Note: I haven't been able to duplicate it since but luckily I took that screenshot.)


3. Beta is being calculated even for short time periods
I always thought WL uses 5 years (60 months), as is typical, for calculating Beta. If one changes the test period under Settings in our test strategy to less than 5 one still gets a value for Beta. Here's a screenshot using only 1yr:


I don't believe Beta can be reliably calculated for such a short time frame so for periods less than the default Beta should return null/blank (-) for the report (not even 0). By extension, Alpha should similarly return null/blank or N/A as it requires a properly calculated beta for its own calculation, the two cannot be disjoined.


4. It appears that Alpha calculation is using some hard-coded Risk-free Rate of Return (probably the average US 3-mo T Bill yield over the test period) even if the 'Use UST Yield' box is unchecked. When it's checked, changing any of the Yield choices makes NO difference to the strategy's results, it only causes the values in Short Only column - which shouldn't even be there in the first place - to change. This is "unexpected" behavior.
Side note: The Sharpe Ratio doesn't change in value either so presumably it is also using the pre-defined, non-modifiable Risk-free RoR.
Please make the 'Use UST Yield' box functional so users can choose whichever yield metric they want - including the option to not choose any.
---------------------------


Fortunately, all these issues are fairly easy to fix. Hope they get fixed soon.
0
190
12 Replies

Reply

Bookmark

Sort
Glitch8
 ( 11.81% )
- ago
#1
Why is the Alpha value in the short only column aberrant? The return is zero, and zero can surely be plugged into the formula yielding a non-aberrant and correct Alpha.

If you don’t want to consider Alpha when there are no trades simply ignore it.

I’m afk now so will weigh in on the other points tomorrow.
0
- ago
#2
Looking into the US TY issue.
0
Glitch8
 ( 11.81% )
- ago
#3
>>2. Checking the box for 'Use UST Yield' causes strange numbers to appear in Shorts Only column, even though its a Long-only strategy, but makes no difference to the Strategy's results.<<

The check box you're referring to is in the "Interest on Cash" section of the Backtest Preferences. It lets you use the specified UST Yield as the cash interest percentage. So, the number in the short column is expected because the cash (uninvested) there is generating interest.
0
Glitch8
 ( 11.81% )
- ago
#4
""3. Beta is being calculated even for short time periods
I always thought WL uses 5 years (60 months), as is typical, for calculating Beta. If one changes the test period under Settings in our test strategy to less than 5 one still gets a value for Beta. Here's a screenshot using only 1yr:""

Where did you learn that 5 years is a typical length of time for calculating beta?

For the record, our calculation is adapted from various online sources, including this code below. If you don't agree with the calculation that's fine, I read that many financial professionals don't agree on beta and other metric calculations. You could always leverage our Concierge service to create a new Scorecard tailored to the exact calculations that you prefer.

https://stackoverflow.com/questions/46733504/c-sharp-calculating-portfolio-beta-between-stock-and-index
0
Glitch8
 ( 11.81% )
- ago
#5
"4. It appears that Alpha calculation is using some hard-coded Risk-free Rate of Return (probably the average US 3-mo T Bill yield over the test period) even if the 'Use UST Yield' box is unchecked. "

Our Risk-Free Rate of Return uses the 1-year Treasury Yield. Again, if you would prefer some different metric we could always build you a customized ScoreCard.
1
Glitch8
 ( 11.81% )
- ago
#6
"Fortunately, all these issues are fairly easy to fix. Hope they get fixed soon."

The only thing I see here that needs to be fixed is the spurious WL Score you pinpointed. I've seen rare glitches like this before in certain metrics and it's on the radar but I haven't identified a cause yet.
0
- ago
#7
QUOTE:
Why is the Alpha value in the short only column aberrant? The return is zero, and zero can surely be plugged into the formula

There is no Shorting being done in this strategy so there should no values in that column. And you cannot plug in 0 as the value for Beta b/c it is actually a valid number but in this context the proper input would be NaN (but of course there shouldn't even be an input for a non-required column).


QUOTE:
the number in the short column is expected because the cash (uninvested) there is generating interest

One is not Shorting so the Short column should be blank. I dug out an old screenshot from many builds ago which shows how unused columns (in this case both Long and Short) looked like in the past (the - representing N/A values), and ought to look like:



QUOTE:
Where did you learn that 5 years is a typical length of time for calculating beta?

If you read around you'll come across it. But here's how Interactive Brokers - a popular and, one might say, the de facto standard - does it:


Basically, they use 60 months by default and will go down to 40 months - but no lower - if history is between 40 & 60 months.
Now just b/c one can calculate for shorter periods doesn't mean one should - there are not enough data points to make the value statistically valid.

QUOTE:
The check box you're referring to is in the "Interest on Cash" section of the Backtest Preferences. It lets you use the specified UST Yield as the cash

Interest on Cash is very closely tied to the Risk-free Rate of Return, there is also a pending Feature Request (https://www.wealth-lab.com/Discussion/Using-a-Benchmark-Rate-with-Offsets-for-Margin-and-Interest-Rates-10953) that would allow further tweaking if/when implemented.
0
Glitch8
 ( 11.81% )
- ago
#8
Because there’s no shorting being done, you’re sitting in cash. The cash generates interest, if enabled. So there is some return, or zero return if no interest. In any event it can be considered for calculating alpha and beta, but you’re free to simply ignore the values.

Let us know if you want a custom ScoreCard developed to mirror how IB is calculating its beta.
1
- ago
#9
Please use the Long and Short columns for exactly what their names imply, putting in any other values will just create confusion.

Instead, if you wish: Create a new column called, say, Cash Only which'll do just that... invest in Cash. Users can choose their choice of yield - preferably offset it by a certain % - to see how they would have fared with an all-cash portfolio and see what was the opportunity cost for *not* investing in the market (or benefit in case of a decades-long bear market).
Of course, creating a blank strategy effectively does the same but this way that column is always there for reference. [Note: After some recent update the 'blank strategy' results are spilling over into both Long and Short columns, check on your PC after choosing your UST Yield; this shouldn't be happening.]

Come to think of it, "Cash Only" column is a pretty good idea....
0
- ago
#10
QUOTE:
how IB is calculating its beta


Its about statistical validity. Basing calculations a small number of observations, especially when talking about a widely dispersed metric like stock market returns, is a big no-no.

Here's a useful reference: https://tsgperformance.com/should-there-be-a-minimum-number-of-months-to-calculate-standard-deviation/
Selected quotes from above article:
- Standard deviation can also be used to measure volatility (or variability, if you prefer) across a time period. GIPS (Global Investment Performance Standards) now requires compliant firms to report an ex post (i.e., backward looking), annualized, three year standard deviation on an annual basis, for the composite and its benchmark. It is measured against the prior 36 months.
- Standard deviation assumes a normal distribution, and in statistics we expect to see a minimum of 30 periods included in the calculation. The problem with a smaller sample size (12 or 24, for example) is that the results may be misleading.
0
Glitch8
 ( 11.81% )
- ago
#11
I don't want to keep going in circles, and I appreciate your zeal in what you'd like to see, but don't agree with your conclusions. But there's always the Concierge avenue to get you a custom crafted ScoreCard to your exact specification.
1
- ago
#12
Nobody uses Long Only or Short Only columns for reporting Cash Only returns. Nobody.

And this is basic statistics, it doesn't merit Concierge service.
0

Reply

Bookmark

Sort