Addressing History Values by datapoint as well as time

Help improve Bet Angel.
Post Reply
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

I'd like to put forward a suggestion to be able to refer to values in a History List by the number of datapoints ago, as well as being able to select time in secs/mins

If trading horse racing I can see the value in a time condition. However, in a sport where there are in play milestones, such as game/set start/end in Tennis, a time condition isn't always relevant.

An example is saving a stats value in a History List in tennis at the end of each selections service game. To compare those history values by time has no real relevance to what is happening in the match as the milestone timing is fluid. However, if we were able to address those values by x points ago it would be straightforward to compare performance to previous service games/sets etc

I would assume there are other sports where this would also be useful
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

sniffer66 wrote:
Mon Mar 08, 2021 1:59 pm
I'd like to put forward a suggestion to be able to refer to values in a History List by the number of datapoints ago, as well as being able to select time in secs/mins

If trading horse racing I can see the value in a time condition. However, in a sport where there are in play milestones, such as game/set start/end in Tennis, a time condition isn't always relevant.

An example is saving a stats value in a History List in tennis at the end of each selections service game. To compare those history values by time has no real relevance to what is happening in the match as the milestone timing is fluid. However, if we were able to address those values by x points ago it would be straightforward to compare performance to previous service games/sets etc

I would assume there are other sports where this would also be useful
stu - great suggestion.

Thinking further on this, would it work if the History lists could optionally be *primed* to store their values using a KeyValuePair scheme??
This would mean that you could create a Stored Value that could act as the key into the History List. This Stored Value would change to represent the data that you were tracking.

For example:

Priming the History List
SV_Name = $currentGamePoint
History List 1 "Key" = $currentGamePoint

Changing the SV_Value to store a keyed entry into the History List
SV_Value (which would change over the game, displaying possible values) =

$currentGamePoint = "Set1:Game2:1530" or "Set1:Game2:3030" or "Set1:Game2:4030" or "Set1:Game2:Finish" etc, etc

So, if you wanted to Retrieve the History List value for a Given set/game/point, then you'd simply ask for History List 1("Set1:Game2:Finish"). You obviously would probably want to use a SV to access the particular History List entry, (i.e. History List 1($svKeyOfInterest)) but that hopefully gives an idea (maybe :D).
User avatar
Dallas
Posts: 22674
Joined: Sun Aug 09, 2015 10:57 pm
Location: Working From Home

Before reading JimiBt's reply I was thinking along the same lines but using Signals

So with a Set/Modify signal rule'
Increment a signal each new service game and at the same time store your value into a History list

Which you could expand and use another 'Set/Modify Signal Rule' with conditions to identify each new game, set etc and store your values into their own history lists
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

Dallas wrote:
Mon Mar 08, 2021 3:30 pm
Before reading JimiBt's reply I was thinking along the same lines but using Signals

So with a Set/Modify signal rule'
Increment a signal each new service game and at the same time store your value into a History list

Which you could expand and use another 'Set/Modify Signal Rule' with conditions to identify each new game, set etc and store your values into their own history lists
The problem then Dallas is that when using an SV Condition Rule and referring to a History List Value, you can only refer to it by time ago i.e x Secs/Mins ago. What you need to do is refer to the value 1 step/datapoint previous, if you want to compare the last 2 service game stats for example
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

sniffer66 wrote:
Mon Mar 08, 2021 1:59 pm
I'd like to put forward a suggestion to be able to refer to values in a History List by the number of datapoints ago, as well as being able to select time in secs/mins

If trading horse racing I can see the value in a time condition. However, in a sport where there are in play milestones, such as game/set start/end in Tennis, a time condition isn't always relevant.

An example is saving a stats value in a History List in tennis at the end of each selections service game. To compare those history values by time has no real relevance to what is happening in the match as the milestone timing is fluid. However, if we were able to address those values by x points ago it would be straightforward to compare performance to previous service games/sets etc

I would assume there are other sports where this would also be useful
I need a little time to digest that Jim, but sounds good :)
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

sniffer66 wrote:
Mon Mar 08, 2021 3:59 pm
sniffer66 wrote:
Mon Mar 08, 2021 1:59 pm
I'd like to put forward a suggestion to be able to refer to values in a History List by the number of datapoints ago, as well as being able to select time in secs/mins

If trading horse racing I can see the value in a time condition. However, in a sport where there are in play milestones, such as game/set start/end in Tennis, a time condition isn't always relevant.

An example is saving a stats value in a History List in tennis at the end of each selections service game. To compare those history values by time has no real relevance to what is happening in the match as the milestone timing is fluid. However, if we were able to address those values by x points ago it would be straightforward to compare performance to previous service games/sets etc

I would assume there are other sports where this would also be useful
I need a little time to digest that Jim, but sounds good :)
lol - assuming that you are able to continue to use the SV-CSV functionality to populate your CSV, then i think you could use the SV idea above as that would guarantee that you supplied the sv value as and when it was relevant. so your csv could set a value to a SV called $currentGamePoint.

This value would represent the current set/game (in this contrived example - i.e.$currentGamePoint = "Game1:Set2" etc).

Once sent back to Guardian, that SV (and value) would create a new entry in the History List that you were targetting. So in effect, you could later query the History List for previous values, based on your Keying pattern... Thus you could retrieve those values based on Keying into the History List with a value along those lines ("Game2:Set4" etc). Obviously, you could track other data in sync with different Keying schemes.

Hopefully a step closer to explaining it - lol, my worst attribute (one of!) -explaining my brain
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

jimibt wrote:
Mon Mar 08, 2021 4:12 pm
sniffer66 wrote:
Mon Mar 08, 2021 3:59 pm
sniffer66 wrote:
Mon Mar 08, 2021 1:59 pm
I'd like to put forward a suggestion to be able to refer to values in a History List by the number of datapoints ago, as well as being able to select time in secs/mins

If trading horse racing I can see the value in a time condition. However, in a sport where there are in play milestones, such as game/set start/end in Tennis, a time condition isn't always relevant.

An example is saving a stats value in a History List in tennis at the end of each selections service game. To compare those history values by time has no real relevance to what is happening in the match as the milestone timing is fluid. However, if we were able to address those values by x points ago it would be straightforward to compare performance to previous service games/sets etc

I would assume there are other sports where this would also be useful
I need a little time to digest that Jim, but sounds good :)
lol - assuming that you are able to continue to use the SV-CSV functionality to populate your CSV, then i think you could use the SV idea above as that would guarantee that you supplied the sv value as and when it was relevant. so your csv could set a value to a SV called $currentGamePoint.

This value would represent the current set/game (in this contrived example - i.e.$currentGamePoint = "Game1:Set2" etc).

Once sent back to Guardian, that SV (and value) would create a new entry in the History List that you were targetting. So in effect, you could later query the History List for previous values, based on your Keying pattern... Thus you could retrieve those values based on Keying into the History List with a value along those lines ("Game2:Set4" etc). Obviously, you could track other data in sync with different Keying schemes.

Hopefully a step closer to explaining it - lol, my worst attribute (one of!) -explaining my brain
I've had coffee now Jim so wrapped my head around it :)

Great suggestion (though I'm not sure how easy it would be to implement). So rather than relying on the value 3 samples ago you can hone in immediately on the value you want to retrieve at a set event in the match, even adding specific values together etc

However, if that wasn't possible, at least being able to retrieve values x samples ago would be an improvement over just time
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

sniffer66 wrote:
Tue Mar 09, 2021 9:52 am
I've had coffee now Jim so wrapped my head around it :)

Great suggestion (though I'm not sure how easy it would be to implement). So rather than relying on the value 3 samples ago you can hone in immediately on the value you want to retrieve at a set event in the match, even adding specific values together etc

However, if that wasn't possible, at least being able to retrieve values x samples ago would be an improvement over just time
yes, if you defined a Keying scheme, you would ALWAYS be able to target a set of History List values at any defined moment during the event. So you could for example, compare the History List when the key was "Game1:Set9" vs "Game2:Set2" etc, etc...

This would however rely on the Keying scheme being adhered to in a uniform way - the benfits are enormous.
Bet Angel
Bet Angel
Bet Angel
Posts: 3999
Joined: Tue Apr 14, 2009 3:47 pm

History Lists are stored as a set of floating point values alongside the chart datapoints (which is why they are accessed in a time based manner). Although unreferenced data is paged out to disk when not required, the fact that Bet Angel can have hundreds of active markets and therefore thousands of selections with active data (any one of which might be being referenced by Automation conditions), memory usage and performance is the top consideration.
There just wouldn't be the memory to associate a key alongside each value stored in the history list.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

Bet Angel wrote:
Tue Mar 09, 2021 1:14 pm
History Lists are stored as a set of floating point values alongside the chart datapoints (which is why they are accessed in a time based manner). Although unreferenced data is paged out to disk when not required, the fact that Bet Angel can have hundreds of active markets and therefore thousands of selections with active data (any one of which might be being referenced by Automation conditions), memory usage and performance is the top consideration.
There just wouldn't be the memory to associate a key alongside each value stored in the history list.
Ok, thanks - understood.

How about referencing them by count, as per my original suggestion then ? My assumption is that wouldn't require a key

As an example, say I save a History value for 1st Serve% In at the end of each selections service game. I want to compare the value 6 service games ago with the current value so I reference the value 6 entries ago in History List x in my SV condition. I can't do that with the current time based compare.
Bet Angel
Bet Angel
Bet Angel
Posts: 3999
Joined: Tue Apr 14, 2009 3:47 pm

sniffer66 wrote:
Tue Mar 09, 2021 1:28 pm
How about referencing them by count, as per my original suggestion then ? My assumption is that wouldn't require a key

As an example, say I save a History value for 1st Serve% In at the end of each selections service game. I want to compare the value 6 service games ago with the current value so I reference the value 6 entries ago in History List x in my SV condition. I can't do that with the current time based compare.
Unfortunately there isn't the concept of 'when you set the value into the history list' only what the value is at any given time.

e.g. Imagine we are capturing from 2pm.
At 14:00:01 you set a history list with the value of 1 and 100ms later at 14:00:01.100 give it the value of 2.
At 14:00:02.800 you give it a value of 3
At 14:00:04.200 you give it a value of 4.

From the history lists point of view (stored as half-second OHLC data against the chart price data) the values from 2pm are:

14:00:00.0 : null
14:00:00.500: null
14:00:01.0: High 2, Low 1, Close 2
14:00:01.500: Close 2 (no need to store OHL as value didn't change)
14:00:02.0: Close 2
14:00:02.500: Open 2, High 3, Low 2, Close 3
14:00:03.0: Close 3
14:00:03.500: Close 3
14:00:04.0: Open 3, High 4, Low 3, Close 4
14:00:04.500: Close 4
14:00:05.0: Close 4
14:00:05.500: Close 4

So you've given it 4 values, but two are in the same data point and there are ten datapoints between 14:00:01 and 14:00:05.500.
To do what you ask, you'd have to hunt back from the most recent datapoint and derive a new list of values regardless of time. You'd probably have to just use the 'close' values as the specific order of multiple values set in the same half second timeframe would be lost.

Not impossible, but not as performant as a direct lookup x half-seconds ago, and generally would have to be treated as a special case for History List interaction. i.e. You'd probably have to do it by Setting another Stored Value with the value of a history list 'x' changes ago, rather than just having it inline in an Automation condition or on display on the ladder/One-click.

Just thought, there's also the issue of if you Set the history list with the Same value. It's not going to know when that happened as nothing changed.

Hope that makes sense.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

Thanks for the detailed reply BA

As a non dev I was assuming a History List was held as an array/table, with the value in the row and the time in the column (or vice versa). So, to address by count it was as simple as specifying a column number to return a previous value.

I get the argument re the issue when writng constantly to the list, but I only want to write a value every 2 mins or so, then retrieve them and run a calc on the sum etc.

Will have to see if I can come up with a workaround using multiple SV's and address them individually - but that has it's own challenges :)
It's a shame as a History List would have been the most elegant solution.
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

sniffer66 wrote:
Wed Mar 10, 2021 8:36 am
Thanks for the detailed reply BA

As a non dev I was assuming a History List was held as an array/table, with the value in the row and the time in the column (or vice versa). So, to address by count it was as simple as specifying a column number to return a previous value.

I get the argument re the issue when writng constantly to the list, but I only want to write a value every 2 mins or so, then retrieve them and run a calc on the sum etc.

Will have to see if I can come up with a workaround using multiple SV's and address them individually - but that has it's own challenges :)
It's a shame as a History List would have been the most elegant solution.
Thinking around this. How about a NEW function (yes, yet another new function), called Snapshots that allow you to save history lists in the state that they are at that point with a key or index value by which to access them??

if memory was an issue, it could be created as a FIFO list that contained a maximum of 40-50 entries. (or could have an expiry time limit set against the item that was input, thus it would drop off after xxx seconds/minutes)
Bet Angel
Bet Angel
Bet Angel
Posts: 3999
Joined: Tue Apr 14, 2009 3:47 pm

jimibt wrote:
Wed Mar 10, 2021 8:56 am
Thinking around this. How about a NEW function (yes, yet another new function), called Snapshots that allow you to save history lists in the state that they are at that point with a key or index value by which to access them??

if memory was an issue, it could be created as a FIFO list that contained a maximum of 40-50 entries. (or could have an expiry time limit set against the item that was input, thus it would drop off after xxx seconds/minutes)
Yes, I see what you mean and it would be something different to the way History Lists and Stored Values are handled.
We'll have to have a think about it, crunch the numbers with regard to memory usage, and generally see how it would fit in to the rest of the application.
FIFO makes sense from a memory management point of view, and that would suggest that the '0' index in the data structure is the most recent rather than the first stored, but I can foresee requests for accessing the data indirectly with the index being held in another stored value. Following that you'd probably have a need to know the 'length' of the data currently stored so that the index can be computed. It soon starts getting more complex from the user interface point of view to work with the data structure.

Perhaps if the rest of this thread could be +1s from the community along with how you would wish to interact/use an index based (rather than time based) storage mechanism, then we can judge the level of demand and consensus.
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

Bet Angel wrote:
Wed Mar 10, 2021 11:25 am
jimibt wrote:
Wed Mar 10, 2021 8:56 am
Thinking around this. How about a NEW function (yes, yet another new function), called Snapshots that allow you to save history lists in the state that they are at that point with a key or index value by which to access them??

if memory was an issue, it could be created as a FIFO list that contained a maximum of 40-50 entries. (or could have an expiry time limit set against the item that was input, thus it would drop off after xxx seconds/minutes)
Yes, I see what you mean and it would be something different to the way History Lists and Stored Values are handled.
We'll have to have a think about it, crunch the numbers with regard to memory usage, and generally see how it would fit in to the rest of the application.
FIFO makes sense from a memory management point of view, and that would suggest that the '0' index in the data structure is the most recent rather than the first stored, but I can foresee requests for accessing the data indirectly with the index being held in another stored value. Following that you'd probably have a need to know the 'length' of the data currently stored so that the index can be computed. It soon starts getting more complex from the user interface point of view to work with the data structure.

Perhaps if the rest of this thread could be +1s from the community along with how you would wish to interact/use an index based (rather than time based) storage mechanism, then we can judge the level of demand and consensus.
ok, i'll set the ball rolling

+1 for implementing something alsong these lines.
Post Reply

Return to “Suggestions”