Losing strategies wanted

[psycho040253]

Suppose also that the system had a high chi-squared value - indicating that the results were probably not a fluke.

The challenge I have with Chi squared values is as follows.

Over many thousands of bets, an equity curve may have substantial periods in which it makes an overall profit, as well as lengthy periods during which it breaks even or makes an overall loss. In each case, you may get a Chi squared result over a statistically significant number of bets that says 'This result cannot realistically have been a fluke'.

So how can you be confident that the random sample you pick is representative of the system's performance more generally, and not an accident of randomness?

Jeff

Jeff

It is thought that the first races to take place in Britain were organised by soldiers of the Roman Empire in Yorkshire around 200 AD, although the first recorded race meeting was during the reign of Henry II at Smithfield, London in 1174 during a horse fair. In addition, who knows how far into the future horse racing will continue. As such, regardless of how much data for a system is collected, it can only represent a small proportion of the total set. And that applies to all systems. Therefore, whenever we test a system, there must be a degree of uncertainty in the value of any metric that we measure.

Because of this, the best we can do is to use the Margin of Error (MoE) which allows for the fact that we can only amass a sub-set of the total data for a system.

For those unfamiliar with the MoE, it is equal to the reciprocal of the square root of the sample size. It is used to modify a metric associated with a system, including the chi-squared value. The modification takes the form of a +/- value, thus giving the metric a range. For example, if the measured strike rate of a system is 20% and the MoE is 3%, the true Strike Rate could be anywhere between 20% +/- 3% i.e between 17% and 23%.

Granted, this solution isn't perfect but it is far better than just hoping and praying that a system will be profitable in the future

What's your solution? To be specific, how do you determine how likely it is that a system will be profitable in the future?

Psycho

[Iron]

Thanks Psycho

I'm not a statisician. I don't have a better solution than Chi, but I'm just exploring how reliable Chi is.

Does a high Chi value tell you that, if the sample is representative, then you can have a very high confidence that you will continue to experience the same results as previously?

If so, does it follow that any overall losses over hundreds of bets are likely to be due to changes in the underlying fundamentals?

This equity curve illustrates what I'm talking about: http://www.betangel.com/blog_wp/2011/09 ... look-like/. It produces a profit overall, but the losses often come in clusters. Does that suggest to you that things are happening on certain days/certain meetings that make the system profitable on some days but not on others?

BTW, something else I struggle with as regards Chi is knowing when my sample size is big enough. Common sense tells you that, if your average odds are 3.0, then a sample of 1000 is more than adequate. But what if your average odds are 30? How big a sample size would you need then to be confident of the Chi result? I've done a bit of research online into that issue, but can't find an answer.

Jeff

What's your solution? To be specific, how do you determine how likely it is that a system will be profitable in the future?

Psycho

[Iron]

PS Psycho - Have a look at the 2011 results for the Racing Synergy system, as that's another case in point:

http://www.bettingautomation.com/results.php

The equity curve went nowhere for about 3000 bets, before taking off and producing an overall 13% ROI.

Would you agree that the period of stagnation was unlikely to be due to randomness?

Jeff

[psycho040253]

Thanks Psycho

I'm not a statisician. I don't have a better solution than Chi, but I'm just exploring how reliable Chi is.

Does a high Chi value tell you that, if the sample is representative, then you can have a very high confidence that you will continue to experience the same results as previously?

If so, does it follow that any overall losses over hundreds of bets are likely to be due to changes in the underlying fundamentals?

This equity curve illustrates what I'm talking about: http://www.betangel.com/blog_wp/2011/09 ... look-like/. It produces a profit overall, but the losses often come in clusters. Does that suggest to you that things are happening on certain days/certain meetings that make the system profitable on some days but not on others?

BTW, something else I struggle with as regards Chi is knowing when my sample size is big enough. Common sense tells you that, if your average odds are 3.0, then a sample of 1000 is more than adequate. But what if your average odds are 30? How big a sample size would you need then to be confident of the Chi result? I've done a bit of research online into that issue, but can't find an answer.

Jeff

What's your solution? To be specific, how do you determine how likely it is that a system will be profitable in the future?

Psycho

The higher the Chi-squared value, the higher the probability that the past results will be repeated in the future. However, there is still a greater than zero probability that they may not.

If a system was profitable in the past and had a high chi-squared value and, at given point in the future, began losing, then there are a number of possibilities:

1. There has been a fundamental change in racing.
2. Statistical variation (bad luck).
3. The result of the chi-squared value being a probability.

The Chi-squared value, essentially, depends upon the difference between the actual and theoretical strike rates and the number of bets. As such, the average odds are irrelevant. The accuracy of the Chi-squared value is determined by the MoE. The MoE depends only on the number of bets. Therefore, a given number of bets will give rise to a given chi-squared accuracy, regardless of the average odds. Therefore, you will not require more 30/1 bets than 3/1 bets to obtain a given level of accuracy. I would like to understand why you think otherwise.

As regards the clustering of losses - yes, this does happen and there is a statistical reason but it is too complex to explain here.

As regards the reliability of Chi-squared - yes, it is reliable. However, let's say that the chi-squared value for a profitable system indicates that there is a 94% probability that the results are no fluke. There is still a 6% probability that the results are a fluke. Therefore, there is a 6% probability that the system could still fail in the future. Now, although 6% is not high, it isn't zero and therefore it could still happen. The fact that it does, does not imply that Chi-squared is unreliable. In fact, quite the reverse.

Let's consider this example: Yesterday, a 200/1 shot romped home. Does this mean that its odds were incorrect? Well, maybe, maybe not. If there are 201 200/1 horses, then 1 of them should win on average and over the long term. If it didn't, their odds should be higher.

Psycho

[Iron]

If a system was profitable in the past and had a high chi-squared value and, at given point in the future, began losing, then there are a number of possibilities:

1. There has been a fundamental change in racing.
2. Statistical variation (bad luck).

Thanks Psycho

Let's say that Chi squared is 99.9% confident about a system. Does that mean that statistical variation is extremely unlikely to be the cause of any lengthy losing periods?

Therefore, you will not require more 30/1 bets than 3/1 bets to obtain a given level of accuracy. I would like to understand why you think otherwise.

Two hypothetical scenarios:

A. I have a sample of 1,000 horses I bet on, with average odds of 2.0. 650 of them win. Because of the sheer number of horses that won over and above the number that would have won had the implied odds been correct, IMHO I can safely say that I have an edge.

B. I have a sample of 1,000 horses I bet on, with average odds of 100. Twelve win. Although my ROI is very nice, I can't say with any confidence that more than 100 would win in a sample of 10,000. It could easily be that 2 of those wins were nothing to do with the system.

Jeff

[psycho040253]

PS Psycho - Have a look at the 2011 results for the Racing Synergy system, as that's another case in point:

http://www.bettingautomation.com/results.php

The equity curve went nowhere for about 3000 bets, before taking off and producing an overall 13% ROI.

Would you agree that the period of stagnation was unlikely to be due to randomness?

Jeff

No, I wouldn't agree but I wouldn't disagree either. There is just no way of determining the reason as to why the equity curve did what it did. The equity curve did what it did because it did is all that I am prepared to agree with. Anything else is pure speculation.

I once saw a system perform way beyond that which could normally be expected for a period of 2 1/2 years. It made a shed load of moolah for its owners. Then, one day, it collapsed in a heap and lost more than it won in its entire history. It never did recover, as far as I am aware.

Why did it fail? Because ultimately and statistically, failure was inevitable. Why did it perform way above expectation for so long? It just did.

Psycho

[Iron]

Thanks Psycho

Let's say, for argument's sake, that you've got a system that's produced a 10% ROI at level stakes for the past 10,000 bets, and the Chi squared is through the roof.

Can we conclude the following?

A. The methodology had an edge for the period in question, ie the system was highly unlikely to be succeeding due to dumb luck.

and that therefore:

B. If the system suddenly stops working, it has to be due to changes in fundamental factors, such as the market getting smarter or changes in racing itself (such as economic changes that impact on field sizes, the quality of horses bought, etc).

Jeff

I once saw a system perform way beyond that which could normally be expected for a period of 2 1/2 years. It made a shed load of moolah for its owners. Then, one day, it collapsed in a heap and lost more than it won in its entire history. It never did recover, as far as I am aware.

Why did it fail? Because ultimately and statistically, failure was inevitable. Why did it perform way above expectation for so long? It just did.

Psycho

[psycho040253]

If a system was profitable in the past and had a high chi-squared value and, at given point in the future, began losing, then there are a number of possibilities:

1. There has been a fundamental change in racing.
2. Statistical variation (bad luck).

Thanks Psycho

Let's say that Chi squared is 99.9% confident about a system. Does that mean that statistical variation is extremely unlikely to be the cause of any lengthy losing periods?

Therefore, you will not require more 30/1 bets than 3/1 bets to obtain a given level of accuracy. I would like to understand why you think otherwise.

Two hypothetical scenarios:

A. I have a sample of 1,000 horses I bet on, with average odds of 2.0. 650 of them win. Because of the sheer number of horses that won over and above the number that would have won had the implied odds been correct, IMHO I can safely say that I have an edge.

B. I have a sample of 1,000 horses I bet on, with average odds of 100. Twelve win. Although my ROI is very nice, I can't say with any confidence that more than 100 would win in a sample of 10,000. It could easily be that 2 of those wins were nothing to do with the system.

Jeff

Jeff

If a system has a ROI of 13% and a chi-squared value of 99.99%, then, it is 99.99% probable that the system will produce a ROI of 13% over the long term future. However, over the short term future, the ROI could rise way above or fall way below the 13%. The reasons could be many and varied, including statistical variation. In fact, even during a testing period, there could be a large variation in the results.

Let's remember what Chi-squared is a measure and what it isn't a measure of.

It is a measure of the statistical significance of a difference between the actual and theoretical strike rates of a system measured over a number of bets.

It isn't a measure of how profitable a system is because, it is quite conceivable that a system has a high chi-squared value and yet loses a fortune.

Also, it isn't a measure of how consistent a system is. In fact, it is conceivable that a system could have some large variations in its results and yet have a high chi-squared value. Therefore, it cannot be assumed that if a system has a high chi-squared value that a large variation in the future results are not the result of statistical variation.

Be also aware that the chi-squared value isn't a constant for a system. It varies with the average odds, the strike rate and the number of bets. However, as the number of bets increases, the variation in the chi-squared value does diminish.

As regards your examples, my feeling is that you are correct. More selections are required to created the same Chi-squared value if the odds are higher than if the odds are lower. I am a layer of low odds horses. I have never backed/layed the long shots. As a result, I have never applied chi-squared to such. This is an aspect of chi-squared that I hadn't fully appreciated.

Having said all this, there isn't a problem with Chi-squared. It is a natural consequence of backing long-odds horses.

Psycho

[Iron]

If a system has a ROI of 13% and a chi-squared value of 99.99%, then, it is 99.99% probable that the system will produce a ROI of 13% over the long term future.

How do you define long-term in this context?

It isn't a measure of how profitable a system is because, it is quite conceivable that a system has a high chi-squared value and yet loses a fortune.

Let's say I have a system with 13% ROI and a very high chi-squared value (say, 99.9% confidence). How, other than by outrageously bad luck or a change in the fundamentals, could I lose money?

Jeff

[psycho040253]

If a system has a ROI of 13% and a chi-squared value of 99.99%, then, it is 99.99% probable that the system will produce a ROI of 13% over the long term future.

How do you define long-term in this context?

It isn't a measure of how profitable a system is because, it is quite conceivable that a system has a high chi-squared value and yet loses a fortune.

Let's say I have a system with 13% ROI and a very high chi-squared value (say, 99.9% confidence). How, other than by outrageously bad luck or a change in the fundamentals, could I lose money?

Jeff

Long term/short term is all relative and without absolute definition.

You misunderstood my point. It is quite conceivable that a system can have a chi-squared value of 99.99% and a ROI of -23.5% for example. In this case, it is 99.99% probable that the loss isn't a fluke and that the system will continue making losses in the future.

It is also conceivable that a system can have a chi-squared value of 50% and a ROI of -10%. In this case, the ROI is probably random. In this case, the system could make a future profit or a future loss which is greater or less than 10%.

Basically, the profitability of a system and the chi-squared value are independent entities. A system can therefore have a high profitability and a high chi-squared value, a high profitability and a low chi-squared value, a low profitability and a high chi-squared value, a low profitability and a low chi-squared value.

This is why I stated that the Chi-squared value is NOT a measure of the profitability of a system. It is only a measure of how its past performance (good or bad), is likely to be repeated in the future over the long term.

Psycho

[Iron]

Thanks Psycho

So let's say that you have a large and representative sample of results data relating to a system, and it shows a 13% ROI and a Chi Squared value that's through the roof.

Would you agree with me that such a system is gold dust, and is only likely realistically likely to fail if the fundamentals change?

Jeff

[psycho040253]

Thanks Psycho

So let's say that you have a large and representative sample of results data relating to a system, and it shows a 13% ROI and a Chi Squared value that's through the roof.

Would you agree with me that such a system is gold dust, and is only likely realistically likely to fail if the fundamentals change?

Jeff

I'd agree with the gold dust part. Failure due to a change in the fundamentals - disagree.

I have a modular laying system that has been running live for the better part of 2 years. The chi-squared value indicates that there is less than a 1 in 10,000 chance of it being random. In fact, one part of the system has a less than 1 in 100,000 chance of it being random. It has a ROI of about 17%. The equity curve still rises and falls like a roller-coaster. Why?

Just like all systems, my system has a LLR. The theoretical LLR is currently 6.01. Because I have only been running live for about 2 years, there is a margin of error in the LLR. As such, I've actually had a LLR of 7. In addition, I've also had numerous shorter losing runs close together.

Such is the roller-coaster effect, there are times when the system looks as though it is failing. Thus far, it has always recovered and gone on to bigger and better things.

As I place more bets using the system, the LLR will increase causing the roller-coaster effect to be amplified.

Can the apparent failure of the system be put down to a fundamental change in racing?

It could be. However, it is more likely that an apparent failure can be put down to a natural consequence of placing bets using a given system because:

More bets > larger LLR > greater amplitude of roller-coaster affect > more frequent and severe 'apparent' failures.

Psycho

[Iron]

Thanks Psycho

So basically, it sounds like a system with a decent ROI and a high Chi Squared value might fail overall over (say) 100 bets, but is unlikely to do so over 100,000 bets. Would that be a fair summary?

Jeff

[psycho040253]

Thanks Psycho

So basically, it sounds like a system with a decent ROI and a high Chi Squared value might fail overall over (say) 100 bets, but is unlikely to do so over 100,000 bets. Would that be a fair summary?

Jeff

Fair - yes.
Accurate - no.

Because we are dealing with probabilities, if something is possible, eventually, it will happen. But, if 'eventually' is in 1,000 years, I don't care. If eventually occurs in the next 5 mins., then I do.

Let's take this statement:

Event X will happen once in 1,000 years.

You'll be surprised how many people take this statement to mean that event x will happen in 1,000 years time.

Strangely, my system looked more reliable over the first 100 bets than it does now.

Psycho

[Iron]

Thanks Psycho

Here's another possible losing strategy. When the race goes in-play, offer to lay each horse substantially below its BSP.

My hunch is that the market will take the value and leave the rest. Also, let's say a horse drops suddenly from (say) 2.2 to 1.8 (perhaps because its rival has just fallen), and you had money waiting to be matched at 2.14. Someone would get to back at 2.14 when the true odds are much lower, at your expense.

I've just been number crunching a spreadsheet relating to ~1,400 horses (which includes min in-play odds), and found that blindly laying the sample at prices substantially lower than the BSP would have led to a hefty loss.

What do you guys think?

Jeff