Elon Musk's Open AI beat world best dota 2 players

[Dublin_Flyer]

I think football-wise you'd need an incredibly huge amount of games to begin with, include international friendly games, friendly pre-season games that might end 15-0 to one side, add a large amount of these in so there's a full view of teams, games etc.
Include actual league games like Serie B in the last few games of the season, where form/players/expectations don't stand up to the lure of the €€ (possible bungs).
If you want the AI to have a prediction, you need it to have as many of the situations and outcomes as possible......and even then there might be a Grobbelaar in goal!

[marksmeets302]

I looked at training neural nets back in 2007, but it only seems to be trendy now. I'm not sure what has changed, computing power? The methodologies don't seem radically different?

Around that time a couple of improvements came into popularity that enabled training neural networks with a large number of layers. Also, gpus were used to do all the matrix calculations in parallel. The biggest change is probably the availability of much more data. I believe somebody said we now produce more data in a year than we did from the inception of mankind up to ten years ago.

Neural nets are presented as a magic potion but my own experience is that it is difficult to tailor them to specific needs. The standard problems like character recognition, finding the boundaries in a picture filled with x's and o's, etc that works fine. Doing anything that is useful for Mark seems a lot harder. The other family of algorithms that is used for learning to play games is actually the coolest thing I've learned since leaving university. Once I got how a computer can learn to play a game without even stating the rules or objectives it completely blew me away. Unfortunately, for any problem of decent size you need the combination of that algorithm with neural nets so I'm still a bit stuck.

[DoctorEvil]

I looked at training neural nets back in 2007, but it only seems to be trendy now. I'm not sure what has changed, computing power? The methodologies don't seem radically different?

I'm far from an expert, but I think computing power wouldn't have been a problem, at least not in 2007. there was some guy who tried to apply neural networks to make computer recognize objects on images (which is a pretty hard thing to do) and it went great and created lots of hype for NN.

[frogthegambler]

Neural Nets definitely can be used for advantage on Betfair.

I first started using them for to build value models for backing strategies pre-Betfair in 1998.

Since around 2011 I have used neural nets feeding into reinforcement learning algorithms to learn to trade with no human input. So far they work out in smaller markets like USA racing, Aust racing and greyhounds but no luck yet in UK horse racing win markets. After 7 years researching the robot keeps saying no bet!

Watching the robot learn to trade has helped with manual trading as well. It approaches things differently to what I would have expected.

[marksmeets302]

Neural Nets definitely can be used for advantage on Betfair.

I first started using them for to build value models for backing strategies pre-Betfair in 1998.

Since around 2011 I have used neural nets feeding into reinforcement learning algorithms to learn to trade with no human input. So far they work out in smaller markets like USA racing, Aust racing and greyhounds but no luck yet in UK horse racing win markets. After 7 years researching the robot keeps saying no bet!

Watching the robot learn to trade has helped with manual trading as well. It approaches things differently to what I would have expected.

That's really cool! I've been dreaming about this for years, never got it to work. Would be great if one day you get an email from the system "I think I've got something for UK horse races"
Are you using it in the way deepmind did the atari games?

[frogthegambler]

They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.

[marksmeets302]

They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.

Fascinating. Reading your answer I realize I might have misunderstood what experience replay is: I thought it meant feeding old <state, action, reward> tuples to the network. The action and reward is obtained by playing the game.
The number of possible inputs and actions indeed quickly escalates. I restricted it to actions on a single horse, and an indicator whether I had a back/lay or no position. The simulator prohibited stacking positions. Also I used a fixed window for the timeseries, with best-back and size, last and best-lay and size. I only considered the first two horses. For the size of the back and lays I experimented with taking the log of the sizes, the ratio of those two etc. Lots to tweak for sure. Again, great that you cracked it!

[sionascaig]

The logic is harder to define in a real life sport IMHO. In most of the AI applications I have seen so far, it's working to rigid boundaries that can be tested and retested. How would AI know a horse is running loose? But also that it is on a circular track leading back to the start or open countryside?

/quote]

I've been monitoring academic papers over the last few tears for anything on evolutionary algorithms operating within a financial market... Haven't found a single relevant paper....

Rather than the AI approach I thought that a valid alternative could be to create an algorithm that runs an a market where success is defined as making a profit, best performing ones are then combined / mutated and run again - keep going till you get success...

However that fact that there are no academic papers on it suggests to me that someone has proved perhaps at an meta level that this cannot be done. However I cant find a paper on that either )

It would be interesting to determine whether a solution to this problem "exists" and if it does than would at least have a bit of confidence in trying to search for it...

[frogthegambler]

I don't think you will find any papers showing it was possible to make profits using 'AI' for real trading.

When I wrote my masters paper on this showing neural nets with RL could be used to make money in the real world I made sure no copies were kept and it was not published anywhere.

The reason is if you find something that works you do not tell everyone else how to do or the advantage would be lost. (That is one reason you never even really hear Peter Webb or Caan Berry explain exactly how they trade in their videos.)

What is done privately behind the scenes is years ahead of what is published in the journals in AI and finance. There are some very good tools available now (theano, tensorflow etc) but you really do need to know the logic and maths behind it to make it work in environments that are not completely closed like an arcade game.

[Iron]

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Kind of like what some of us may have done when looking at Adrian Massey's database - http://adrianmassey.no-ip.org/web1/db3/thaccess.php - but on a much bigger, more sophisticated scale.

An AI programme became the best chess player in history after a few hours of self-teaching - http://www.bbc.co.uk/news/technology-42251535 - so I wouldn't rule it out.

Jeff

[ShaunWhite]

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Said the CEO of every major bookmakers and every major syndicate. And it's a question that's been worked on for probably 100 yrs by the finest racing brains.

The problem is always that race horses are temperamental skittish creatures and you don't always know what they'll be like from day to day with the degree of certainty you're hoping for.

You'll never calculate true odds for an individual race and what it should have been can never be known. You'll maybe work them out over a long run but if you want that then the numbers are already there in the market.

[SeaHorseRacing]

I think intuition makes a trader and I am not so sure this could be programmed.

[frogthegambler]

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Kind of like what some of us may have done when looking at Adrian Massey's database - http://adrianmassey.no-ip.org/web1/db3/thaccess.php - but on a much bigger, more sophisticated scale.

An AI programme became the best chess player in history after a few hours of self-teaching - http://www.bbc.co.uk/news/technology-42251535 - so I wouldn't rule it out.

Jeff

A key issue with this for working out 'true' win probabilities is insider information in the market for horseracing.

Trainers and owners are allowed to back their own horses when they are fit and ready.

If you look at the price of a horse in the last 5 mins. It can go 3.1 out to 4.3 and then back to 2.8. Its true price cannot vary that much. But which price is right?

I found after about 2004 if I priced something at 3.0 and it was trading at 7.0 it merely meant my model was wrong and the market was right. Insiders knew the horse would not run to form or that another in the race would run better than its form suggested.

[cyxstudio]

I think intuition makes a trader and I am not so sure this could be programmed.

Heres the problem .

Machine learning means machines can develop intuition too after thousands of repitition.

And they are better than us at intuition.

[mcgoo]

I think intuition makes a trader and I am not so sure this could be programmed.

Heres the problem .

Machine learning means machines can develop intuition too after thousands of repitition.

And they are better than us at intuition.

The more I see of AI/ML the more I think one would be crazy not to try it on BF trading. Banks big finance must be using it-BF too.
I am learning to code at a ripe old age -just cos I like it, but I think the leap to machine learning will be a bridge too far for this brain..of course I'll keep tinkering.If nothing else it all has made me realise how far my bots have to go.What else am I gonna do with my time What a time to be alive!