Back Testing Bots

[Emtaxx]

Hi all,

Got a few automations bits setup and they're near enough doing what I'd like them to be doing.

However just a little bit of guidance I require - How long do you guys back test / run your bots; before coming to the conclusion they don't make profit?

thx

[jamesedwards]

Hi all,

Got a few automations bits setup and they're near enough doing what I'd like them to be doing.

However just a little bit of guidance I require - How long do you guys back test / run your bots; before coming to the conclusion they don't make profit?

thx

Unfortunately it can take thousands, sometimes tens of thousands of markets to take account of variance.

[ShaunWhite]

One thing you could do on the way to a decent sample is to plot your cumulative PL per over time.

Then Right click on the PL line and Add Trendline.
In the trendline options select 'Display R-squared value on chart'

and to save me typing I'll leave the rest to GPT...

R-Squared Guide for Strategy Testing

0 - 10% → Pure noise (no meaningful pattern, likely random).
10 - 30% → Very weak relationship (variance dominates, no real edge visible).
30 - 50% → Weak relationship (some structure, but still unreliable).
50 - 70% → Moderate relationship (clearer trend, but variance still a factor).
70 - 90% → Strong relationship (strategy has predictive value, worth refining).
90 - 100% → Very strong relationship (highly reliable strategy).

For real-world betting/trading, you'd want 0.9+ before trusting it. Lower values mean you’re mostly dealing with randomness.

[Fugazi]

One thing you could do on the way to a decent sample is to plot your cumulative PL per over time.

Then Right click on the PL line and Add Trendline.
In the trendline options select 'Display R-squared value on chart'

and to save me typing I'll leave the rest to GPT...

R-Squared Guide for Strategy Testing

0 - 10% → Pure noise (no meaningful pattern, likely random).
10 - 30% → Very weak relationship (variance dominates, no real edge visible).
30 - 50% → Weak relationship (some structure, but still unreliable).
50 - 70% → Moderate relationship (clearer trend, but variance still a factor).
70 - 90% → Strong relationship (strategy has predictive value, worth refining).
90 - 100% → Very strong relationship (highly reliable strategy).

For real-world betting/trading, you'd want 0.9+ before trusting it. Lower values mean you’re mostly dealing with randomness.

I've been using statistical significance calculations... But this seems much easier (well, I'm sure it's just another measure of the same thing)

[sionascaig]

Screenshot 2025-02-26 064138.png

And here is one (horses) that has been running for almost two years & contains over 200k bets... The drop off is over a period of 5 months...

Depending when you embarked on this strategy could have thought:

- found a golden goose OR
- binned it

It is a long game...

[Fugazi]

Screenshot 2025-02-26 064138.png

And here is one (horses) that has been running for almost two years & contains over 200k bets... The drop off is over a period of 5 months...

Depending when you embarked on this strategy could have thought:

- found a golden goose OR
- binned it

It is a long game...

My struggles with automation in a nutshell

[Emtaxx]

Thanks Shawn, that's a good way to go around it tbf. I think I'll collect a month worth of data and then plot a gradient MA line through it. So far I have a +EV, however its only been 3 days... and today will be the 4th! I'll post back this OP an see what you guys think.

I have some logic behind what I'm doing (and that's the only hope I have at this point!). I believe that over time is should play out well, but still does require some further refinement for retaining and creating better profits.

Damn Siona, that is some extensive data. I just hope that it picks back up again for you to the average line you've drawn else that's gana be a bit of a melon twister to understand what's changed.

[jamesedwards]

Thanks Shaun, that's a really useful feature.

2,200 days and approx 370,000 markets in.

R² = 0.9868.

Proper happy with that

z83.JPG

[jimibt]

2,200 days and approx 370,000 markets in.

R² = 0.9868.

Proper happy with that


z83.JPG

if you have a bed of laurels, you certainly deserve to recline and enjoy - well done, good to see.

[LinusP]

That is a very odd way to calculate EV Shaun.

To put it simply you want to get a price which is better than a price in the future, with sports this is easy to calculate as the end of the event as it's either 1 or 0. However this can take a while to converge so you can speed this up by looking at a price into the future, x seconds/minutes for example.

I like to simplify this with

ev = (1/future_price) - (1/price_matched)

[ShaunWhite]

That is a very odd way to calculate EV Shaun.

To put it simply you want to get a price which is better than a price in the future, with sports this is easy to calculate as the end of the event as it's either 1 or 0. However this can take a while to converge so you can speed this up by looking at a price into the future, x seconds/minutes for example.

I like to simplify this with

ev = (1/future_price) - (1/price_matched)

I agree with all that. This was just meant to be a rough and ready guide. The guy seems new so it's something to do while building up a sample worth looking at. I figured capturing a future time and logging to join it all together later might be have been a push given his experience and Guardian's idiosyncrasies.

[ShaunWhite]

Screenshot 2025-02-26 064138.png

And here is one (horses) that has been running for almost two years & contains over 200k bets... The drop off is over a period of 5 months...

Depending when you embarked on this strategy could have thought:

- found a golden goose OR
- binned it

It is a long game...

I don't usually track the cash, just the EV. They'll always tend towards each other over time. EV is the sanity check that gets you through the more volatile cash.

[ShaunWhite]

Thanks Shaun, that's a really useful feature.

2,200 days and approx 370,000 markets in.

R² = 0.9868.

Given market conditions a straight line is a real result. But I recognise that early up swing and the optimism that it's going to be exponential.

[sionascaig]

I don't usually track the cash, just the EV. They'll always tend towards each other over time. EV is the sanity check that gets you through the more volatile cash.

In my world EV means something very specific (the present value of future net cashflows taking account of interest, mortality & other tiresome expenses) & so I can get a bit confused by what other people mean by it - other than a positive expectation of something..

I tend to focus more on the "margin" as defined by profit / stake for each strategy...

Did you mean something more specific by EV?

[ShaunWhite]

In this context expected value EV is simply the price you get vs the likelihood of the outcome.

A little easier to work out pre race as you can take BSP to be the longterm liklihood. Or you could use a future price as LinusP explained in another thread today......

ev = (1/future_price) - (1/price_matched)

That gives the pure probability difference, but it doesn't account for how much value that difference represents in terms of returns. Multiplying that by (price matched - 1)*100 converts the probability difference into a percentage EV figure.

Neither right or wrong just different.