Hey guys,
I've been reading back through some historic posts and it seems to me that definitions can be interpreted in different ways. As I am currently 'testing' some strategies, I realised that my definition may vary compared to others so I was interested to hear what people think testing is and what stages they go through?
For me, there are a few stages of testing. Firstly, it's testing any automation works as expected through a variety of scenarios. Then, it's testing to see if your strategy works in a particular market for example it may give great returns in the Bundesliga but not be as effective in the Championship. I'd probably need 100 occurrences to justify if it's profitable. Then, I'd run another 100 occurrences, this time with small stakes and that would be enough for me to then start increases the stakes gradually. I say gradually, I'd constantly be testing to ensure higher stakes don't impact the outcome.
Does anyone do things differently?
Craig
Testing a Strategy - Definition?
-
jamesedwards
- Posts: 4919
- Joined: Wed Nov 21, 2018 6:16 pm
It's really tough. Markets, countries, stakes are the obvious ones but there are also time of day, day of week, season, weather, to name but a few. So many variables to consider and it is amazing how much impact seemingly innocuous variables can have. One of my automations for example just flat out excludes a couple of periods of time on certain days of the week as these almost always fall behind. No rhyme or reason why, it just happens. Also you really need thousands and thousands of results before you can be really sure of a pattern. Obviously you can get a gauge from many less but I have had lots of automations that look good into the thousands but then suddenly lose their way. You're asking the right questions though so your mind is working in the right direction. Good luck!
Similar but I never start with small stakes. I've always had an issue with telling people to start small and if it works get bigger, because invariably a lot of things that work at small stakes do not work at bigger stakes as you suffer from a lack of fill rate and/or you move the market and reduce your edge.
I should qualify by saying my stakes are generally smallish. And a lot of my automation dips multiple times into a market. So I always use stakes that I would be comfortable to use if the model had already proved successful. What I would change if it proved successful is the number of times it triggers in the market.
Of course stakes that are comfortable for you will be different to mine and to everyone else's. Set your own bar. But you'll save a lot of time starting at a reasonable staking level than with something tiny.
Somethings you know don't work almost immediately, others take a lot of transactions before you know they don't work, and others seem to work fine for months until they don't. For me it comes down to gut feel but the 100 you've quoted is a very small sample.
When I tinker to make improvements I never change more than one thing at a time and then let it run to see if there is improvement. Well that's a lie actually, I often find I can't help myself making a few changes but always tell myself - NEVER make more than one change at a time!!
Good luck.
I should qualify by saying my stakes are generally smallish. And a lot of my automation dips multiple times into a market. So I always use stakes that I would be comfortable to use if the model had already proved successful. What I would change if it proved successful is the number of times it triggers in the market.
Of course stakes that are comfortable for you will be different to mine and to everyone else's. Set your own bar. But you'll save a lot of time starting at a reasonable staking level than with something tiny.
Somethings you know don't work almost immediately, others take a lot of transactions before you know they don't work, and others seem to work fine for months until they don't. For me it comes down to gut feel but the 100 you've quoted is a very small sample.
When I tinker to make improvements I never change more than one thing at a time and then let it run to see if there is improvement. Well that's a lie actually, I often find I can't help myself making a few changes but always tell myself - NEVER make more than one change at a time!!
Good luck.