Neuro-Lab: Using NNIndicator in Strategy
Author: Mockasino
Creation Date: 1/2/2017 2:03 PM
profile picture

Mockasino

#1
Hello,

What exactly happens when i integrate a NN-indicator from a trained NN in a trading strategy?

I trained a NN for a certain portfolio and get back the NN-indicator ranges. The result is stored in an XML-file.
Then i integrate the NN-indicator in a strategy to test via a rule like this : if NN[bar] >= paraNNnumber....

What happens when this statement is triggered/checked in background? I think the xml-file is read and applied to the portfolio? But how?
Why it takes so long?
is there a new Training started when the portfolio differs from the trained one?

This refers also to a former question of mine how the NN-indicator is calculated in the program.
The internal result of a trained NN are the weights assigned to all neurons.
What is the relation of the weights and the NN-indicator?

Is there anybody who can help me?

Thanks in advanced

Christian

profile picture

LenMoz

#2
Make sure you are running Neuro-lab 1.0.3.0. Speed was improved at this release.

See forum topic, "Neuro-Lab compiles the strategy excessively in MSB / Optimization mode", posts 29 to 31.

QUOTE:
What happens when this statement is triggered/checked in background?

Your input script is run to create the input DataSeries. Then (I believe) the XML is parsed. Finally, the weights are applied and the NNIndicator DataSeries is calculated. In my opinion the time consuming component is parsing the XML. MS123 may know more precisely how this works.

QUOTE:
is there a new Training started when the portfolio differs from the trained one?
No

Len
profile picture

Mockasino

#3
i already have Neuro-lab 1.0.3.0. running.......hm.

what does this mean for real live trading? Once the NN is trained and the XML-file with weights is written ,
then a dataset with just the last bars (depending on parameters , lock back periods, etc) is necessary to do the calculation.
the it should not take that Long to calculate und Trigger the alerts.
Am i wrong?
profile picture

LenMoz

#4
QUOTE:
Am i wrong?

I'll defer to Eugene/Cone. They or their developers have access to the NNIndicator code. I wrote my own NNIndicator.Series which is 4-5 times faster than 1.0.3.0. Repeating myself (post #2), "In my opinion the time consuming component is parsing the XML" I believe in their case it happens at every NNIndicator call.
profile picture

Eugene

#5
Our NL developer did attempt to cache those XML requests and deserialization which gave a speed improvement below 10%. His decision was to avoid that since this change could turn out breaking. He hasn't discovered speed issues under profiler.
profile picture

LenMoz

#6
Using my code (essentially a deserialization, caching the weights), my strategy, pre-PosSizers. on a 10-year backtest of 137 symbols, takes 1 to 2 seconds. That is the benchmark speed. NNIndicator.Series, under the same conditions, takes 7 seconds. There is still a 3-to-1, not 10%, speed improvement to be had.
profile picture

Mockasino

#7
now i am confused.
When running a backtest, about 120 symbols, 6 years, it takes about a minute.
any ideas?
profile picture

Eugene

#8
For starters, disable on demand data update.
profile picture

Mockasino

#9
I am just new in NN not in WL:-)
profile picture

Eugene

#10
Well it was worth a try ;)

So what have we got? At least we know from private sources (telemetry) that topic starter's hardware is powerful enough. The mysterious code presumably using NNIndicator runs on a 120-symbol DataSet of about 1000 bars (or is it intraday data?) with a group of unrecognized performance visualizers enabled and it takes a minute to execute. That's intriguing! :)
profile picture

Mockasino

#11
Hi Len,
in Post 6 you speak about your code.
What you are doing to reach this performance-kpi?
And what do you do to get the 7 seconds?

Regards
Christian
profile picture

LenMoz

#12
Simple displays and average a few runs.
At the start of Execute...
CODE:
Please log in to see this code.

and at the end of Execute...
CODE:
Please log in to see this code.
profile picture

Mockasino

#13
Thts how you determine the runtime.
But what have you implemented to reach these good runtimes?
profile picture

LenMoz

#14
Repeating post #4, " I wrote my own NNIndicator.Series which is 4-5 times faster than 1.0.3.0."

I leveraged NN code I had written for previous projects and translated from VBA to C#. I think the big speed difference is that my code processes the XML only once per backtest and stores it as a WL global variable. I can't see the real NNIndicator code. Downsides? It requires hand-tailoring each strategy, inserting the NN Input Script (and keeping both sync'd if the input script changes). And it is limited to two hidden layers (more than enough for me).
profile picture

Mockasino

#15
so for creating an NN-Indicator from the weight of the WL-xml-file, which objective function you are using?
Are you using the sigmoid function?
profile picture

LenMoz

#16
QUOTE:
Are you using the sigmoid function?
Yes