How do you test your automation files?

[the4droogs]

As many of you probably know, building an automation file in BetAngel is a great experience. Testing that automation file - not so much. I've found that most of the bots I create in BetAngel require me to allow for some edge cases which may happen rarely. This means running in practice mode and watching is generally a very inefficient way to test. Similarly, the logs often do not give enough information to work out exactly what happened in a given situation.

My solution has been to set up screen recording which records both the ladder and the output of the logs at the same time so I can replay it and see exactly what has happened in the market and how my bot reacted. I can then fix up any bugs as a result. This avoids me having to sit and watch the market intensely, potentially missing something, but it still requires some work in setting up the screen recording and hoping that the case I am testing occurs in that particular market.

I'd be interested in hearing how others go about making sure their bots are working correctly in all situations.

[ShaunWhite]

If you haven't got a way to replay data then your options are limited. But the standard approach in the software world (and that's what you're testing) would be to first come up with a test plan. Write down a list of every possible scenario you think of that your software might encounter; you can probably think of more wierd and wonderful situations in a few days than you might encounter all year. Then the slightly tedious bit, work through each in turn with your code/Guardian in front of you, and step through each line noting what values/actions each line will produce. It's obviously not fool proof but you should catch the more obvious errors.

That's a long way from being ideal, but unless you're in a position to create a set of test data that contains all the good,bad & ugly situations which you can test and re-test with, then what you're doing together with a standard list of things to be mindful of is probably as good as you'll get, unless someone knows better?

[the4droogs]

Yes coming up with all the scenarios to test is definitely helpful. But as you say, encountering those scenarios in the wild is the hard part.

It would be great if BetAngel provided some sort of unit testing capability, both at the condition level and the rule level. Although it may well be that adding such functionality would not pass a cost benefit analysis required to add it as a feature. However, if it ever aspires to be a genuine alternative to coding your own bots from scratch I think it would be essential.

Personally I feel that I'm getting to the stage of outgrowing BetAngel automation. It is great for quickly getting a bot set up and realising the power of automation when applied to betfair trading. And contrary to the opinion of some people you can definitely build bots that make a tidy profit. However the more I delve into it the more I realise that the limitations. So I'm starting building my own setup from scratch - Being able to take advantage of unit testing is one of the main drivers for that. That is a fair way away though (and I'm not convinced I'll be able to do it well anyway) so in the meantime I intend to do the best I can with BetAngel automation which is still a great product despite the limitations.

[decomez6]

working with BA , has an organic feel.
you can see how it evolved from a manual traders perspective to a more sophisticated algo approach...., its literally a manual traders paradise with whistles , bells and servants that respond to trading what you see .
there is also a little bit of ` survival for the fittest footprint ` in BA.
priority is given to anything that help you replicate the market to its most organic nature or closer E.g the practice mode .
A back testing option would come in very handy ,but , would not survive the test hence the chop/extinction.
who determines that ?
experience is the best teacher and the best teacher will keep you away from all the glitter if its not usefull. BA has a teaching element built in, and it will conform with peter`s experince in sports trading.
i see guardian as the best and easiest tool for traders with no programming backgrounds. and i know its not as easy to as it sounds and thats why its an edge in itself.
VBA and the excel function is another avenue to inco-operate complex formulaue into BA.
and if you have done all that and still fallen short , then its time to go academic and learn some programming languages and be good at it, otherwise you will waste too much time trying to achieve what BA can easily do .

when it comes to testing the outlier occurrence , i completely agree with you.
sometimes you just need to know if the bot is working as its supposed to , but you conditions are rare .
what to do
wait....wait...test in practice mode.....wait....wait...............................................................................................................................wait...........................live mode..............................not enough sample!

[decomez6]

EDIT apologies

[rik]

Important to keep a low amount in your account if your not sure, had an automation what just kept adding bets

[jamesedwards]

I test in practise mode to make sure there are no mistakes, then go live for minimum stakes for testing.