WL7 Sneak Previews
Author: Glitch
Creation Date: 10/24/2020 2:07 PM
profile picture

Glitch

#1
We'll use this thread to show some sneak previews as we develop WL7. One interesting change is that we are focusing on Indicators as a flexible way to present data series. If something can be expressed as an indicator, it will be. This will make it easily usable in the flexible Building Block Strategy structure. Eugene has been working hard on this new paradigm. Look at the FRED and CBOE data series, easily drag and dropped on the chart in the form of an indicator!

profile picture

abegy

#2
One question Glitsh : is it a flotting windows the Indicators ? This is better when you have multiscreen.
profile picture

Glitch

#3
No it’s not, but you can open multiple main windows and each one has its own indicator panel so that should alleviate that concern.
profile picture

Eugene

#4
WL7 will have dynamic DataSets. Sort of what the Wealth-Data provider currently does in WL6 when a symbol leaves a supported DataSet like Dow 30 and is replaced with another. But unlike W-D, these new DataSets can be providerless and use whatever data provider is preferred by the user.

Take these "IPO DataSets" on the screenshot below. For example, the expanded DataSet gets periodically and transparently refreshed with the latest IPOs:

profile picture

mjj3

#5
Can the dynamic datasets be used as a historical scan? (i.e. top down trading approach)

For example:

For a given strategy the process entails:

a) scanning weekly for a combination of fundamentals and indicators for trade candidates
b) running a portfolio of strategies on those trade candidates

In the past I would have to run the strategies on the entire universe of securities (say NYSE) to allow for any candidate to be available through the backtest. Extremely slow and data intensive. If I could dynamically create a DataSet which is a scan of fundamental and technical indicators through time then it makes backtesting this top down approach very doable.
profile picture

Eugene

#6
Technically, a custom DataSet provider could be developed to import the trade candidates from say a file if you mean that.

Also, these DataSets can have dynamic executable ranges. That's how a backtest runs on Wealth-Data on say all Nasdaq 100 constituents through its history but the trades are applied only to the data ranges when an index component was actually part of that index.
profile picture

abegy

#7
One question Eugene : you say that there will be a dynamic executable ranges. How positions will be managed if a symbol is removed ? They will closed automatically the position for this symbol in this case ?
profile picture

Glitch

#8
>>If I could dynamically create a DataSet which is a scan of fundamental and technical indicators through time then it makes backtesting this top down approach very doable.<<

Hmmm, that would be a killer feature! I don't think we will have that out of the box, but it's theoretically possible. We have something called DataSetProviderBase which can, as the name suggests, provide DataSets. This is how I implemented the dynamic WealthData DataSets that account for survivorship bias. There's no reason why we (or someone else) couldn't build a DataSetProvider that creates dynamic DataSets based off some Condition Building Blocks, for example. We could probably make the screen run off a remote service too, like Quantacula's Screener does now. Lots of possibilities!

>>They will closed automatically the position for this symbol in this case ?<<

Yes, the position is closed at market close of the last date the symbol was in the index.
profile picture

mjj3

#9
Happy to take a crack at it myself once I get my hands on WL7 ... or is it the same as UniverseProviderBase in Quantacula ?
profile picture

Glitch

#10
Quantacula's Universes are analogous to WL7's DataSets, yes.
profile picture

abegy

#11
For your information, I have a trading approach similar from mjj3. I'm scanning every month a global exchange to find "best candidates stocks" according Fundamentals (base on estimated earning for example) and technicals (like volume for liquidity) criterias. After that, I take the "candidates stocks" during the month to apply a dip trading strategy. By this way, I have every month a new watchlist.

one question Glitch : is there will be a possibility to create dynamic watchlist programmatically ?
profile picture

Glitch

#12
abegy - for your use case I would personally just update a plain ole' DataSet manually once a month. But ... of course you could create a custom DataSetProvider that reads the symbols, for example, from a file that is saved from your other process. The possibilities are there :)
profile picture

Eugene

#13
I put abegy's idea on my todo list for later since there's some demand.
profile picture

superticker

#14
QUOTE:
a custom DataSet provider could be developed to import the trade candidates from say a file if you mean that.
Well, I guess I could make that work.

What I really want to do is group "some" datasets by earnings dates so I can precisely time updates to their earnings data and trade accordingly. What you're suggesting is the write the stocks with their earnings dates to a file (The EarningsDateCache.txt file is effectively already doing that.), then import that as a dataset. That works. If you could add a Data Manager plug-in to do that automatically or thoroughly document the API for the Data Manager so I could write the plug-in (including calls to update the dataset objects without restarting WL), that would work better.
profile picture

Eugene

#15
It's possible to create a DataSetProvider that would dynamically include symbols based on some condition (for example, upcoming earnings dates) and update them each time WL starts. Even if the process is slow to load (say it makes web requests or database queries) that won't bog down loading WL7. That's how the IPODataSets will work. Hmm maybe there's sense to build an UpcomingEarnings DataSetProvider?
profile picture

Glitch

#16
>>Hmm maybe there's sense to build an UpcomingEarnings DataSetProvider?<<

Yes, I'm looking for stocks to short!
profile picture

superticker

#17
QUOTE:
It's possible to create a DataSetProvider that would dynamically include symbols based on some condition (for example, upcoming earnings dates)
That's great news.

QUOTE:
maybe there's sense to build an UpcomingEarnings DataSetProvider?
Well, that's kind of what the EarningsDate class does now when it creates the EarningsDateCache.txt file. What you could do is integrate them so the DataSetProvider and EarningsDate class can use the same cache file.

I also have a standalone command line program that reads the Excel file created by the Fidelity stock screener and writes a WL dataset. I might consider converting that program to a DataSetProvider. The problem is there are command line parameters passed telling it what to do. For example, ETFs take different command line parameters than stocks (and they write different WL datasets).

I wouldn't try to get every suggestion implemented on the initial WL 7.0 release. Only the breaking changes, like converting to .NET Core and WPF, need to be included on the initial release. As long as WL is extensible, other features can be added later.
profile picture

mjj3

#18
I was thinking more along the lines of this:

A DynamicDataSetProvider that accepts:
Inputs:
a) DataSet (or Universe) superset of symbols to screen against
b) Criteria in the form of a model or (Quantacula Block) where you can specify fundamental/technical criteria (scan criteria)
c) Frequency (Daily, Weekly, Monthly, etc) Frequency that the screen is updated

Iterates through the DataSet and populates a new DataSet using the Quantacula Universe syntax with {symbol}.{StartYYYYMMDD}.{ExitYYYYMMDD} matching when the symbol meets the criteria specified in the inputs.

The only question is can the DataSet/Universe handle a symbol being in the dataset multiple times for different time periods? Such as:
AAPL,19950101,20001231 (Apple from 1995 to 2000')
AAPL,20020101 (Apple from 2002 to current)

The other caveat is what symbols do you want in the execution loop when you run a model against this dynamic dataset? Should it be the distinct symbol list of filtered symbols (any symbol that at one point met the scan criteria)? This would work if you had a Dictionary of <barNumber, bool> that flags that indicate that the symbol meet the scan criteria. You could then incorporate some trade logic that allows you to either exit the trade, trail a stop, etc. if it no longer met the criteria.

Does this make sense?
profile picture

Glitch

#19
Just like WL6 has extendible Performance Visualizers for backtesting, WL7 has extendible Optimization Visualizers for optimizations. I'm neck deep in writing the optimization support in WL7!

profile picture

abegy

#20
Hi Glitch, I take this opportunity to request you some little changes if it is possible.

In the others backtesting platforms, you have the 2D and 3D graphs available for any type of optimizer.
In WLD, this is limited to the exhaustive optimizer. Can you display theses sheets for all optimizers (custom and buld in) ?

Regarding the Results table, can you allow multisort columns and the possibility to add a filter criteria (greater/less and a value) for the selected metric ?

And the best would also be to extend the optimizer features to allow to optimize parameters of the PosSizer selected. If you can add the possibility to push as parameters all available criterias of the posSizer selected, that would facilitate to define best values in a simple way. I saw only few backtesting platform with this build in feature
profile picture

Glitch

#21
Hi abegy - based on your request I made the Surface Graph Optimization Visualizer work with partial optimizations, so consider this done. Multicolumn sort and filtering in the Tabular Optimization Visualizer would, I agree, be valuable. We'll add them to the enhancement list!
profile picture

Carova

#22
Glitch,

A suggestion - Currently Optimization metrics are limited to 2 decimal places during the Optimization process. While that is fine for some metrics (e.g. Profit) it is unsatisfactory for other metrics (e.g. Lake Ratio).

Can you please allow the optimizer to access the metrics with many more significant digits? Thanks!

Vince
profile picture

Glitch

#23
Yes, this is a great point we will address when we develop the "Scorecard" analog!
profile picture

mjj3

#24
Hi Glitch,
I've been messing around with the multi-objective optimizer in Quantacula. It would be good if you could expand that optimizer's fitness conditions to be more like AdaptTrade Builder's where you can specify for each fitness element whether it should be Maximized/Minimized, Weight (which you already have) and if there are conditions for a given element to be removed from the training segment.

Also, they have a nice feature where it can automatically filter for top strategies given a flexible series of conditions.

See attached pics ...
profile picture

mjj3

#25
Additionally, to add some more customization on the Genetic algorithms ... again AdaptTrade does this well. see pic

profile picture

Carova

#26
Glitch,

As long as we are passing along suggestions,,,

Optimizations take time, a lot of time. Consider having a "Pause" feature to allow the user to assess the progress by running some of the solutions found to date, and then a "Resume" feature to continue the process if desired.

Vince
profile picture

Glitch

#27
Thank you mjj3 and Carova for the optimizer suggestions. It's good to be back in the WL forum where there is never a shortage of great ideas. I have a feeling WL7 will break new ground in trading system dev/testing software :)
profile picture

Eugene

#28
Re: post #25. Sorry, couldn't resist reposting this comic having seen that customization on genetic algorithms:




Compare that AdaptTrade screenshot to Aleksey's design in the WL6 version:

profile picture

mjj3

#29
Ha ha ... yes probably a bit overkill ... just food for thought. I do like the flexibility that it provides for the fitness criteria and to auto filter for ideal candidates though.
profile picture

LenMoz

#30
QUOTE:
Compare that AdaptTrade screenshot to Aleksey's design in the WL6 version:
Should there be a highest/lowest parameter? Highest "Net Profit", but lowest "Seykota Lake", for instance.
profile picture

Carova

#31
QUOTE:
Should there be a highest/lowest parameter? Highest "Net Profit", but lowest "Seykota Lake", for instance.


That is a good suggestion Len!
profile picture

mjj3

#32
That's what I was referring to above:

profile picture

mjj3

#33
Have you considered using dockable windows for the Gui to WL7? And potentially using tabbed workspaces?
profile picture

Carova

#34
Glitch,

Will WL7 have the ability to select which variables to optimize when using Optimizers other than Exhaustive? Thanks!

Vince
profile picture

Eugene

#35
@kazuna -> The future of WLD
profile picture

Glitch

#36
Vince, regarding other optimizers, the sky is the limit as each optimizer can specify its own unique configuration UI. To kick things off we will include Exhaustive and Shrinking Window (which lets you specify the metric) and will add more post release and document the API so third parties can build and potentially sell or share their own.
profile picture

Glitch

#37
And yes they run in parallel to take advantage of multi core!
profile picture

Carova

#38
Glitch,

You beat me to it with your last reply! :)

Shrinking Window - I am not familiar with that algorithm (did not find much in a VERY QUICK search). Is it similar to a Bayesian Optimizer?

Vince

EDIT: It seems that "Shrinking Window" might be some adaptation of the Prony algorithm. Need to dig a bunch more.
profile picture

Eugene

#39
@LenMoz -> The future of WLD

Guys, as per post #1: "We'll use this thread to show some sneak previews as we develop WL7." Please use the other thread for general Q&A. Otherwise it becomes impossible to differentiate topics. Thank you.
profile picture

Carova

#40
Glitch,

QUOTE:
It seems that "Shrinking Window" might be some adaptation of the Prony algorithm. Need to dig a bunch more.


Well, I did dig a bunch, and it appears that there are a number of versions and variations of this algorithm. When you get a chance could you post a reference to the version that you are planning to use in WL7? Thanks!

Vince
profile picture

Glitch

#41
"Shrinking Window" is just the name I came up with for a quick and dirty intuitive algorithm I created. It performs a number of passes through the parameter ranges. Each pass has a number of "runs". Each run randomizes the parameters. The first run goes across the entire parameter range. After each pass, the optimizer sees which parameter combination performed best, and then shrinks the range window, centered around that parameter value. After each pass, the window shrinks more, so the optimizer can gradually narrow down on a set of fitter values.
profile picture

Carova

#42
Glitch,

Thanks for the explanation! Is this what you have been using on Quantacula? Are you planning a "standard" Genetic Algorithm or other Evolutionary Optimization, or is "Shrinking Window" just an enhanced version of them?

Vince
profile picture

Glitch

#43
Hey Vince, I’d say my Shrinking Window is probably a less sophisticated optimizer, but the groundwork will be there for others in the future. Right now I’m working 10+ hours a day getting this version up to speed, just like the old days 20 years ago 🙂
profile picture

Carova

#44
Glitch,

Understand!

QUOTE:
Right now I’m working 10+ hours

Does that mean you still have 14 hrs/day to expand into? ;)

Vince
profile picture

Glitch

#45
That’s right, just warming up!
profile picture

Glitch

#46
We just added Help buttons to the top window frames of the child windows. A nice, integrated, way to get context sensitive help from any window. Of course, F1 still works :)


profile picture

jheffez

#47
I assume WL7 is using .NET core. if yes, can it run on a Mac/Linux?
profile picture

Glitch

#48
It’s using .NET Core 3.1, but the user interface library, WPF, is Windows only.
profile picture

Glitch

#49
On this Sunday I got Advanced Position Sizers integrated into WL7! They have basically the same API as WL6. This feature did not make it into Quantacula Studio, so it's good to see it surface again in WL7.

profile picture

Glitch

#50
And here is the new and improved "Meritocratus" PositionSizer ...

profile picture

MatteoCapital

#51
In terms of accuracy, what would it be the difference from a "building block" user and a "coder"?
profile picture

Glitch

#52
The Building Blocks are a way to build a strategy visually by drag and drop. They can also be used to get a jump start to coding, because you can convert a Building Block strategy to a C# Coded strategy . That can be helpful for those who want to learn how to code strategies.
profile picture

abegy

#53
Hi Glitch,

I see with interest that now, it will be possible to specify index benchmark by strategy. This is a very good news.
But, what's about Commissions ? It is possible for you to add also this possibility ? As you know, brokers applies several type of commissions logics depending of the instrument type (or exchange/market).

As a strategy is linked to a dataset (or watchlist), this would be more simple to allow us to specify the commission logic in thestrategy instead of a global parameter. In the case of a global parameter, you need to complexify your commissions logic script to take into account the instrument that you will trade.

Also, I don't see in your screen shot the possibility to attach an account like what we can do in WL 6.9. Why ?
profile picture

Glitch

#54
Currently commission is still a global backtest setting. We will consider making it a strategy-specific setting!

You now assign an account once the orders are sent to the Order Manager. The Order Manager has individual "Order Blocks" that let you easily manage orders from different sources. When you assign an account to an Order Block that comes from a Strategy, WL7 remembers that automatically and will default to that account the next time it sees orders come in from that Strategy.
profile picture

superticker

#55
I have multiple Fidelity accounts that I trade. As a result, I group stocks by trading success and slot them into different datasets associated with different accounts. For example, one account is for lower risk stocks (primarily large cap), another for higher risk (primarily small/micro cap), and another for ETFs. I use the same strategy for everything, but I vary the position size by account/risk.

QUOTE:
WL7 remembers that automatically and will default to that account the next time it sees orders come in from that Strategy.
I'm just wondering how that's going to work with my way of doing things above? Can I still associate specific datasets to specific accounts (and position sizes) like I'm doing with WL6?
profile picture

Glitch

#56
Each Strategy/DataSet combination gets its own Order Block, and you can assign a different account, or even a different broker, to each Order Block. You could also create a copy of the Strategy and then you’d get the benefit of WL7 remembering your assigned accounts automatically.
profile picture

Glitch

#57
Mining for patterns in the Candlestick Genetic Evolver for WL7!


This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).