Back Testing and New Strategies

[MobiusGrey]

Having been involved in the gambling/trading world for the best part of 15yrs my journey has had many turnings, bumps and clear runs. One thing I have discovered (probably later than it should have been) is back testing certain strategies is crucial. I seem to have finally come up with an automated strategy that will trade the indicies spreadbetting but horse racing still remains that elusive nut I just can't crack.

In my quest for automated trading on the financials I use software where you can write an algorithm then back test it over (x) years, if it's profitable you can then tweak certain things and demo it forward testing.

Does anything similar exist for horse racing, I've been using Flat Stats for a couple of days but it's a little clunky.

Secondly, and I'm not sure if anyone will want to answer this or not which is fair enough but am I barking up the wrong tree in thinking you can lay certain horses at certain odds in certain races and expect to have an edge? Is this over an over simplistic approach and it needs to be more nuanced?

I ask that question as I was at one point what I THOUGHT was pretty successful at laying horses using purely fundamental analysis, however in my naievty I was using a fibonacci ratcheting system that naturally caught up with me after close to a year of profitable betting. Needless to say I don't need a lesson on this

[ShaunWhite]

Do you collect any Betfair data? The std approach is to store the API stream, possibly some additional data from other sources in a database (SQL etc) , then you do your analysis, and then run your simulations/backtests on that data. Following that you'd then move to live data but still with simulated trades. If you've survived the slippage that occurs at each stage you're good to go fully live.

It's a fair amount of work but Betfair support an open source solution called Flumine, and also api libraries in C# which is my personal preference, although I did need to learn c# first of course. But like you I'm here for the long haul so a year doing that, and then another year designing/writing software wasn't too bad. Once everything was working well it only took about 6 months to find some sort of edge which has been ultra reliable for close to 3 years now so has repaid all that unpaid investment in time.

The above is the full fat version of the process but the key is to collect your own data so you can do what you want with it rather than being restrained by someone else's functionality.

[MobiusGrey]

This is great thank you. I don't store or collect any data at the moment but will certainly now look into this. I have seen you can buy some data from Betfair, it seems horrifically expensive but I guess it's short term pain for long term gain in that respect.

Think you're probably right though in that collecting then interpreting this data myself is probably the way to go, I have some experience with python and data manipulation so at least I have a jumping off point.

Genuinely I can't believe it's taken me this long to get this point!

[ShaunWhite]

I wouldn't pay for data to start with. You can get some sort of collection working quite quickly first (and it's amazingly useful that you know some python, lucky boy) And by the time you've got your head round everything you'll already have collected say 6 months data and it's starting to get useful.

It's a big commitment and a gamble because it still guarentees nothing, but if you fail it won't be because you didn't try hard enough and if you succeed then the sky's the limit.

[jimibt]

I wouldn't pay for data to start with. You can get some sort of collection working quite quickly first (and it's amazingly useful that you know some python, lucky boy) And by the time you've got your head round everything you'll already have collected say 6 months data and it's starting to get useful.

It's a big commitment and a gamble because it still guarentees nothing, but if you fail it won't be because you didn't try hard enough and if you succeed then the sky's the limit.

hi shaun, hope all well with you. something came to mind when thinking about historical data. I'm kinda out of the loop a bit on BF market data and wondered if you'd noticed any market nuances that indicate that the historical data effectively becomes *stale* at some point in time, either due to a change in market dynamics and/or a change in the events themselves. In the world of crypto, i've noticed that dependng on the timescale of your trade, the relevance of the historical data changes totally, related to your trade timewindow (which is to be expected). In effect, a long term trend may be present, but the micro scale activity is in no way related to that macro scale. Again, i appreciate that sports markets are very different due to the expiry of them after the event.

just wondered as this was definitely something that I struggled with when backtesting BF data and wondered if identifying a gear change in the markets was something that is easily discovered in BF.

[ShaunWhite]

Watcha Jim, yeah all good really. We lost mum at the end of last year so busy with the house etc..happy release though as they say. She went into hospital for the first time in her life with basic TMB (too many birthdays) and she caught covid. We'd been careful for 2yrs and Lincs NHS finished her off in weeks.

Yeah I recognise all of that and hitting a moving target is bloody frustrating. My take on historical data is that yesterday's is relevant but data from 100 years ago isn't.... So there must be a sliding scale of how relevent it is but I haven't been able to think of a way to weigh that relevance of according to age.

Aside from how fresh your data is, last year especially, things which were almost baked in would flip in a matter of days (weaker markets make individuals more impactful). It really comes down to getting that 'feel' about your strategy and being prepared to pull the plug perhaps faster then you would have before.

Big topic though and need to crack on now today, so sorry if I skimmed through that one.

[jimibt]

Watcha Jim, yeah all good really. We lost mum at the end of last year so busy with the house etc..happy release though as they say. She went into hospital for the first time in her life with basic TMB (too many birthdays) and she caught covid. We'd been careful for 2yrs and Lincs NHS finished her off in weeks.

sorry to hear that - TMB is hopefully the way I'll go too, but you know how it is

I guess talking of ages; aging the market to pull out the most relevant and effective data i guess is a moving target across each market, never mind each discipline. I'm just curious as I constanly compare the behaviours (in my head) of sports vs crypto when looking for a novel approach in either.

will catch up proper soon via PM...

[goat68]

Having been involved in the gambling/trading world for the best part of 15yrs my journey has had many turnings, bumps and clear runs. One thing I have discovered (probably later than it should have been) is back testing certain strategies is crucial. I seem to have finally come up with an automated strategy that will trade the indicies spreadbetting but horse racing still remains that elusive nut I just can't crack.

In my quest for automated trading on the financials I use software where you can write an algorithm then back test it over (x) years, if it's profitable you can then tweak certain things and demo it forward testing.

Does anything similar exist for horse racing, I've been using Flat Stats for a couple of days but it's a little clunky.

Secondly, and I'm not sure if anyone will want to answer this or not which is fair enough but am I barking up the wrong tree in thinking you can lay certain horses at certain odds in certain races and expect to have an edge? Is this over an over simplistic approach and it needs to be more nuanced?

I ask that question as I was at one point what I THOUGHT was pretty successful at laying horses using purely fundamental analysis, however in my naievty I was using a fibonacci ratcheting system that naturally caught up with me after close to a year of profitable betting. Needless to say I don't need a lesson on this

hi,
so I use Flumine that Shaun mentioned, really easy if you know Python.
You need Betfairlightweight module as well, and also an API "live key".
My advice would be take a look here: https://github.com/liampauling/betfair
then on that page there's a "Join slack group" link, follow that register, then post a message on that Slack group asking about above...

[Kai]

We lost mum at the end of last year

Sorry to hear it Shaun, not heard the TMB acronym before. Hope you are keeping well yourself

[decomez6]

We lost mum at the end of last year so busy with the house etc..happy release though as they say. She went into hospital for the first time in her life with basic TMB (too many birthdays) and she caught covid. We'd been careful for 2yrs and Lincs NHS finished her off in weeks.

my deepest condolences Shaun.

memories.gif

[ShaunWhite]

Thx dec very thoughtful. It was the right time so that's always easier to deal with. And to be frank a certain selfish relief because the caring/coping had got really tough.

Sorry everyone for the way the thread diverted, it was just a quick news update for Jim really and not supposed to illicit a response, I've had some kind PMs too.

OK thats enough of the love-fest, time to get back to trying to take money from each other