Trading What I see !?

[ShaunWhite]

I'm a bit stubborn when it comes to admitting defeat, but I suspect that point may come soon as reality of how hard this is sinks in....

It's a familiar phase, lots of people start out thinking it can't be as hard as people say and decide to pick and choose the advice they follow. eg you've known for ages how to guarantee you're not backfitting, but it takes many months to accumulate enough data and you wanted to crack on. Understandable I guess. But eventually it breaks people and they'll just want to follow advice letter by letter and sod making up their own rules.

I'll gladly admit I did that to some extent, it's only when I stoppped trying to second guess information that had layers I couldn't even imagine, and just trusted I'd figure out why it was good advice later that the tide turned. Humility trumps hubris when it comes to learning.

Admitting defeat is fine, the fault comes from making it a battle to begin with when so many other people have already fought those demons and are happy to share the gory stories.

[goat68]

I think after 283 pages and 2820 posts, I think this thread is finally winding up, I think I have to admit defeat in trying to go auto. As Trader Pat and others have pointed out without enough manual background i'm fighting a losing battle. I do think the auto world is very hard indeed, and I can't see a worthwhile way forward.
Thank you to all those that helped me along the way. I'm naturally feeling somewhat dismayed, but I do suspect this journey has helped a number of you other traders.
Good luck all.

[arch4672]

I think after 283 pages and 2820 posts, I think this thread is finally winding up, I think I have to admit defeat in trying to go auto. As Trader Pat and others have pointed out without enough manual background i'm fighting a losing battle. I do think the auto world is very hard indeed, and I can't see a worthwhile way forward.
Thank you to all those that helped me along the way. I'm naturally feeling somewhat dismayed, but I do suspect this journey has helped a number of you other traders.
Good luck all.

Good luck to you. For what it's worth, I think you're making the correct decision.

P.S I don't think a lack of a manual background means you can't succeed. I've never done any manual trading (can't think of anything worse to be honest) and I'm managing to make auto work.

[andy28]

Well AUS horse bot doing rubbish today, every one a loser, arghhh variance!
Screenshot 2021-09-29 at 10.55.23.png

Are you sure it's variance, or it's just that your backfitted strategy isn't very good?

quite possibly!
I think I might have to start not relying on backtesting so much, it's far too easy to just "fit" things it would seem...

I am far from an expert but I think the word back testing is a bit misleading. For me (looking at football stats) I get as much historical data as I can and look for a pattern/edge what ever you want to call it. Then my "Back test" is applying it to the market in practice mode. Then look at the result and hope they are similar to my historical data, if it runs for a month at a loss I bin it and retry with different conditions. My reasoning for this is you can't "Fit" anything, it works or it doesn't.

I must have tried 50-100 times and have 1 that I am using live with small stakes ($2) and is going as expected.

But each to their own

[wearthefoxhat]

I think after 283 pages and 2820 posts, I think this thread is finally winding up, I think I have to admit defeat in trying to go auto. As Trader Pat and others have pointed out without enough manual background i'm fighting a losing battle. I do think the auto world is very hard indeed, and I can't see a worthwhile way forward.
Thank you to all those that helped me along the way. I'm naturally feeling somewhat dismayed, but I do suspect this journey has helped a number of you other traders.
Good luck all.

I'd take a 2 month break and come back refreshed.

That way you can reset and clear away all the noise and clutter that should help you re-focus.

[decomez6]

I think after 283 pages and 2820 posts, I think this thread is finally winding up, I think I have to admit defeat in trying to go auto. As Trader Pat and others have pointed out without enough manual background i'm fighting a losing battle. I do think the auto world is very hard indeed, and I can't see a worthwhile way forward.
Thank you to all those that helped me along the way. I'm naturally feeling somewhat dismayed, but I do suspect this journey has helped a number of you other traders.
Good luck all.

yo L be back.gif

can never Loose can only Learn.

[dm1900]

Are you sure it's variance, or it's just that your backfitted strategy isn't very good?

quite possibly!
I think I might have to start not relying on backtesting so much, it's far too easy to just "fit" things it would seem...

My reasoning for this is you can't "Fit" anything, it works or it doesn't.

This doesn't make much sense. Of course you can "fit" to historical data, that's the whole notion of overfitting (I.e. you make something work on historical data but it doesn't work in unseen time periods because you've overfitted and therefore produced a model that doesn't generalise).

If you cannot wrap your head around overfitting then it becomes a bit dangerous tbh.

[ShaunWhite]

If you cannot wrap your head around overfitting then it becomes a bit dangerous tbh.

Overfitting is easily avoided by using one set of data for analysis and a 2nd set of data for proof of that analysis.

[goat68]

If you cannot wrap your head around overfitting then it becomes a bit dangerous tbh.

Overfitting is easily avoided by using one set of data for analysis and a 2nd set of data for proof of that analysis.

I wouldn't totally agree with that. I was doing that. I was using 1month of data to analyse, then repeat on the other 3months for validation.

[goat68]

Here you go!
This is the essence of "dogbot" a few bits not shown here but this is the key bit, this shows a 98% Rsq trend over 4months:

Code: Select all

           # fav p 1 & 2 only
           if favp[runner.selection_id] > 2:
                continue

            volp1 = ((100.0*runner.total_matched)/total_vol)
            bp1 = get_price(runner.ex.available_to_back, 0) or 1.01
            lp1 = get_price(runner.ex.available_to_lay, 0) or 1000.0
            midp1 = (bp1 + lp1)/2.0

             # max_price 4.5, min_price 2.0
            if midp1 > self.max_price or midp1 < self.min_price:   
                continue
 
            # min_vol 20%
            if volp1 < self.min_vol:
                continue

            vwap1 = market.context["vwap"].get_vwap(runner.selection_id)

            back = True
            if midp1>vwap1:
              # Back
              eprice = bp1
            else:
              # Lay
              back = False
              eprice = lp1

            good = False
            if eprice > 3.24 and eprice < 3.36:
              good = True
            elif eprice > 3.69 and eprice < 3.76:
              good = True

            if not good:
              continue

            vwaptrig1 = False
            if midp1 > (vwap1*1.06) or midp1 < (vwap1*0.94):
                vwaptrig1 = True

            if vwaptrig1 and secToStart >= 170:
                # Early "false" trigger
                continue

            if vwaptrig1 and secToStart < 120 and secToStart >= 70:
                # Early "false" trigger
                continue

            if vwaptrig1:
                    r_sel1 = runner.selection_id
                    runner_context1 = self.get_runner_context(
                        market.market_id, r_sel1, runner.handicap
                    )
                    if runner_context1.live_trade_count == 0:
                        net_lb = get_net_lb(market, r_sel1)
                        if midp1>vwap1:
                            side = "BACK"
                        else:
                            side = "LAY"
                        if side == "BACK":
                            price = get_price(runner.ex.available_to_lay, 0)
                        else:
                            price = get_price(runner.ex.available_to_back, 0)
                        if price:
                            # create trade...
      

[goat68]

By the way my "logic" behind this dogbot strategy was I perceived price movement away from 3.25 and 3.75, towards 3.0,3.5,4.0 in the final stages before the off.
Told you I was bananas!

[andy28]

quite possibly!
I think I might have to start not relying on backtesting so much, it's far too easy to just "fit" things it would seem...

My reasoning for this is you can't "Fit" anything, it works or it doesn't.

This doesn't make much sense. Of course you can "fit" to historical data, that's the whole notion of overfitting (I.e. you make something work on historical data but it doesn't work in unseen time periods because you've overfitted and therefore produced a model that doesn't generalise).

If you cannot wrap your head around overfitting then it becomes a bit dangerous tbh.

Ok I most likely got it wrong or not sure what back testing actually is.

To explain I will quickly run thru my understanding of a Backtest

I have 3000+ football results from the last 3 years, I split those into a group 2, one group (approx 2000) is the matches from the first 2 years and look for an edge. When I find what I think is an edge I apply it the 2nd group last years results, was it profitable? To me it is a yes/no answer. If the answer is no I go back to the FIRST sample only and look closer at them, not the combined sample size of 3000, that is what I think the mistake is, your changing your strategy after you know the results of all 3 years.

The strategy that is running for me now worked in the first sample of 2000 then it worked in the next 1000 and it has had 109 "Live" trades so far this season and is working well.

My personal opinion in Goats thread is the mistake is as time goes on he gathers data and looking at it all and changing things to suit "Fit" to make it work and not tested on a sample group of unknown results before going live.

Like I say I am no expert and I stand to be corrected

[goat68]

My reasoning for this is you can't "Fit" anything, it works or it doesn't.

This doesn't make much sense. Of course you can "fit" to historical data, that's the whole notion of overfitting (I.e. you make something work on historical data but it doesn't work in unseen time periods because you've overfitted and therefore produced a model that doesn't generalise).

If you cannot wrap your head around overfitting then it becomes a bit dangerous tbh.

Ok I most likely got it wrong or not sure what back testing actually is.

To explain I will quickly run thru my understanding of a Backtest

I have 3000+ football results from the last 3 years, I split those into a group 2, one group (approx 2000) is the matches from the first 2 years and look for an edge. When I find what I think is an edge I apply it the 2nd group last years results, was it profitable? To me it is a yes/no answer. If the answer is no I go back to the FIRST sample only and look closer at them, not the combined sample size of 3000, that is what I think the mistake is, your changing your strategy after you know the results of all 3 years.

The strategy that is running for me now worked in the first sample of 2000 then it worked in the next 1000 and it has had 109 "Live" trades so far this season and is working well.

My personal opinion in Goats thread is the mistake is as time goes on he gathers data and looking at it all and changing things to suit "Fit" to make it work and not tested on a sample group of unknown results before going live.

Like I say I am no expert and I stand to be corrected

This sounds reasonable, my issue probably i've only got 4months of data, rather than 1 year + 1 year like you had, I was dealing with 1month + 3months

[Realrocknrolla]

This doesn't make much sense. Of course you can "fit" to historical data, that's the whole notion of overfitting (I.e. you make something work on historical data but it doesn't work in unseen time periods because you've overfitted and therefore produced a model that doesn't generalise).

If you cannot wrap your head around overfitting then it becomes a bit dangerous tbh.

Ok I most likely got it wrong or not sure what back testing actually is.

To explain I will quickly run thru my understanding of a Backtest

I have 3000+ football results from the last 3 years, I split those into a group 2, one group (approx 2000) is the matches from the first 2 years and look for an edge. When I find what I think is an edge I apply it the 2nd group last years results, was it profitable? To me it is a yes/no answer. If the answer is no I go back to the FIRST sample only and look closer at them, not the combined sample size of 3000, that is what I think the mistake is, your changing your strategy after you know the results of all 3 years.

The strategy that is running for me now worked in the first sample of 2000 then it worked in the next 1000 and it has had 109 "Live" trades so far this season and is working well.

My personal opinion in Goats thread is the mistake is as time goes on he gathers data and looking at it all and changing things to suit "Fit" to make it work and not tested on a sample group of unknown results before going live.

Like I say I am no expert and I stand to be corrected

This sounds reasonable, my issue probably i've only got 4months of data, rather than 1 year + 1 year like you had, I was dealing with 1month + 3months

With the money you have spunked, you could of bought some data that goes further than that. Personally 4 months of data is not worth the effort you are putting in.

Did you look at any inplay strategies? There must be 20-30 basic bots that can be applied to various markets. You just need to learn which market to apply them too.

[goat68]

Ok I most likely got it wrong or not sure what back testing actually is.

To explain I will quickly run thru my understanding of a Backtest

I have 3000+ football results from the last 3 years, I split those into a group 2, one group (approx 2000) is the matches from the first 2 years and look for an edge. When I find what I think is an edge I apply it the 2nd group last years results, was it profitable? To me it is a yes/no answer. If the answer is no I go back to the FIRST sample only and look closer at them, not the combined sample size of 3000, that is what I think the mistake is, your changing your strategy after you know the results of all 3 years.

The strategy that is running for me now worked in the first sample of 2000 then it worked in the next 1000 and it has had 109 "Live" trades so far this season and is working well.

My personal opinion in Goats thread is the mistake is as time goes on he gathers data and looking at it all and changing things to suit "Fit" to make it work and not tested on a sample group of unknown results before going live.

Like I say I am no expert and I stand to be corrected

This sounds reasonable, my issue probably i've only got 4months of data, rather than 1 year + 1 year like you had, I was dealing with 1month + 3months

With the money you have spunked, you could of bought some data that goes further than that. Personally 4 months of data is not worth the effort you are putting in.

Did you look at any inplay strategies? There must be 20-30 basic bots that can be applied to various markets. You just need to learn which market to apply them too.

i've read this: https://www.betfairtradingblog.com/in-p ... cing-pace/
which makes sense, but not sure what the 20-30 might be... care to expand please?