Interpreting Neuro-Lab Evaluate Performance Tab Output
Author: ronc
Creation Date: 10/13/2015 7:48 PM
profile picture

ronc

#1
In Neuro-Lab > Evaluate Performance, can you please provide further description of the last 2 output columns: "% of Average" and the color bars? The User Guide says:

QUOTE:
% of Average
This column displays how much higher or lower the average output of this row is compared to the Average Output of all Observations, displayed above the grid. A color-coded bar is displayed next to this column, which provides a visual representation of how the average output of the row compares to the overall average output.

As shown here, you want to see red bars at the top and green bars at the bottom, meaning that the actual NN output is higher than the average output when the NN predicted output is also higher.


According to a post by LenMoz, describing his output script, which I assume follows the same data definitions as Neuro-Lab:
QUOTE:
The graphic is the average actual for that bucket.

I understand that definition, but a) it seems different than the above definition from the WL User Guide, and b) it does NOT seem to hold true. For example, in the attached screenshot, the last row has a value greater than the previous row, but the last row's bar is to the left of the previous row. If its value is greater I should it not be to the right of the previous row?

profile picture

ronc

#2
To clarify my question, you can see in the image that the Actual Outputs are generally increasing, along with the Predicted Outputs. However, the graphic seems to indicate that the Predicted Outputs are completely inverted from the Actual Outputs, i.e. that the NN is predicting low (long negative/red bars) when it should be outputting high, and predicting high when it should be low.

Any explanation would be helpful.

Thanks,
Ron
profile picture

ronc

#3
Eugene -
Correction to my post above: "does seem to hold true" -> "does NOT seem to hold true." Can you help me understand the NeuroLab graphic output bars? The documentation says the bars are the "average actual for that bucket" but in the image above the red bars seem inversely correlated with the Actual Output column...?
Thanks,
Ron
profile picture

Eugene

#4
QUOTE:
If its value is greater I should it not be to the right of the previous row?

Ron, the values are negative. Its value is less (-22376) than the previous row (-11693), not greater. Does this explain that it's to the left?

QUOTE:
I understand that definition, but a) it seems different than the above definition from the WL User Guide,

I'd stick with the UG definition.
profile picture

ronc

#5
The User Guide says the color bar represents the average actual. In the example above the actual values are 3.08 and 5.92, with the 5.92 bar being to the left of the 3.08 bar. I assume therefore that the User Guide therefore incorrect, and the bars represent "% of Average" rather than "Actual Output", per your post above?

User Guide definition for "% of Average": "how much higher or lower the average output of this row is compared to the Average Output of all Observations." In the last 2 rows, if the last row Actual is greater than the previous row Actual, how can the last row's % of Average be less than the previous row?

It looks to me like the signs in the last colum are inverted. Most of the rows with positive Actuals have negative "% of Average", and most rows with negative Actuals show positive % of Average. I must be missing something here...?
profile picture

Eugene

#6
QUOTE:
In the example above the actual values are 3.08 and 5.92, with the 5.92 bar being to the left of the 3.08 bar.

Correct. Because the 5.92 bar has a -22K % of average and the 3.08 bar has only -11K. -22K is less than -11K. The color-coded bar refers to % of Average Output, not to the Actual Output (i.e. 5.92/3.08). The less the % of Avg Output is, the longer is the red bar and vice versa.