API-NG Direct Access Pros/Cons

[PeterLe]

I came across a web site this evening that has sample code for accessing the new API directly and having read automatic betting exchange betting by Colin Magee in the past, Im still contemplating learning a new language..

I was thinking about learning Python (my son has a raspberry PI for christmas, so i guess Im going to be doing it anyway ) Is that a good language to learn?

I know there are a few people on here who write their own code who may be able to answer some questions I have..

My dilemma is this; I don't mind putting the hundreds of hours needed, but will the effort be worth it?
Ive always used Betangel with the excel interface and will continue to do so, and built some very large successful spreadsheets. Someone recently tested one of them and told me that the (complex) spreadsheet was refreshing at around 80ms and was in effect hanging around waiting (assuming that the minimum (?) refresh is 200ms). So the question is; What advantage (if any) will creating my own software give me over the standard excel interface in BA in the context of speed and/or functionality? (im not really interested in accessing many markets at once, as I understand you can retrieve up to five markets simultaneous?)
Any thoughts from those who have done it ? Thanks in advance
regards
Peter
(PS If i do decide to take a walk down this road, could I use the inbuilt Python in MAC OS X ?)

[LinusP]

Morning Peter,
I, like many others use Python and have found creating my own programs alongside betAngel to work very well. I started with codecademy.com to learn the basics then jumped straight in following the great guide on this blog..

http://markatsmartersig.wordpress.com/

I went from logging in to Betfair all the way to recording IP horse racing data every 100ms in a very short amount of time. Creating simple programs is very easy and intuitive but coding fill/kill/offsets e.t.c can be tricky. I strongly recommend learning but like yourself I still use betAngel to run some of my spreadsheets.

What I like is that I can have loads of py programs running on a VPS without worrying about processing power, especially with the relaxed data requests.

I work off a mac using PyCharm with any .py files (python 3) saved to my dropbox which I then run straight off a VPS.

[marksmeets302]

The api has a couple of quirks that make it hard to use unless you know what you're doing. Because you don't log in and receive updates in streaming matter, but instead have to ask for it things can get messy.

As an example: if you cancel an order, you will get a message back that tells you that the order has been cancelled. However, this message doesn't include how much of the order has been matched. You have to request a marketbook for that. Assuming that you continuously request marketbooks, it can happen that the next marketbook was pushed to you at a time when the cancel wasn't processed yet. So you see the bet as active... Now you have to find a way to notice that this bet should have been gone and you have to wait for the next marketbook that doesn't have the bet anymore. It can show a match for that bet, at which point you finally know that indeed there was a match, and the rest of the bet is gone now. You are now in a stable state and can put all information together to compute your position.

It's these situations where you don't know if you're processing stuff in the right chronological order that's what makes it hard. It requires keeping a lot of administration.

Oh, and then there's the 'place bets with size less than 2' problem. Betfair don't like you to place bets with a really small size. But there is a workaround, Bet Angel can do it for you without you even noticing. To place a back for 0.50 at 10, what you have to do is place one for let's say 2.50 at 1000. You'll get a message back that tells you what the id is of that bet. As soon as you get that, you must reduce the bet by 2, so that 0.50 remains. And you must modify it to price 10. This will give you a new bet, with a different id but with the right size and price. Keeping this administration in sync is a hassle, especially when markets get suspended, turn in play etc. I cannot begin to tell you in how many ways this can go wrong. It's all because of this model where you have to request all information instead of just getting it.

For me this is also a kind of a hobby so I don't really mind (actually I quite enjoy the hunt for bugs in my programs) but if you want to do this be prepared for a lot of frustration. Start small and avoid difficult things like multiplexing information from different accounts, placing orders with small size, and per-bet hedgers. C'est le merde.

[PeterLe]

Thanks gents for the replies.
Ive just done my first hour on codeacademy and found it really interesting..
Just a question Liam; When you record your data, do you use something like an SQL database to retain it?
Regards
Peter

[LinusP]

Thanks gents for the replies.
Ive just done my first hour on codeacademy and found it really interesting..
Just a question Liam; When you record your data, do you use something like an SQL database to retain it?
Regards
Peter

Yeh codecademy is great.

I actually write straight to txt file then load to a sql db, I couldn't get python to write quick enough when recording every 100ms but every second is fine.

[JayBee]

The pros are many.

1) You will be coding applications to your exact specification.

2) The applications will not contain those parts of 3rd party software that you do not want.

3) Guaranteed security.

4) You can code your own bespoke metrics and not give the game away by asking for a 3rd party to code them on their forum.

5) The new API uses JSON rather than SOAP and is very easy to code for.

6) A JSON string is a database in itself. I just save the raw JSON to a file and have a program that can parse the JSON into charts whenever I need it.

Cons.

1) You have to learn how to program.

If you have no intention of being a commercial programmer then I would recommend not using Python. The nerds will always jerk that knee, shout Python and poke fun at anyone who does not do so.

I am a programmer of 30 years plus, to code things quickly and simply in the Windows environment I use Visual Basic or C# (they are essentially the same as they compile to the same run-time). VB is easy to understand for beginners and experts alike, who can port to another language should they wish. Python, especially when written by amateurs, is unintelligible heiroglyphicsand you won't learn anything from looking at it.

If you have a programming problem in VB then someone, somewhere has had the same problem. Coding in VB is simply cutting and pasting other people's code.

The only specifics you need to know is how to parse JSON. That is all done for you with Newtonsoft's JSON parser DLL, which you can include as an import.

Coding to Betfair's old API 6 was tortuous. Coding for API-NG took a couple of weeks. I now spend my time coding precisely the metrics I need.

I don't use Excel because it has a lot of baggage and is slow compared to a bespoke program. Coding an application to handle all race markets simultaneously allows for scalable trading. All races and runners can be loaded and processed in under a second. I have not seen any 3rd party software that comes close to doing that.

You can click on my weblink for a few tutorials on VB for API-NG. Private message me for specifics.

Good luck as it will make you think more about your trading and open your mind to new approaches rather than being confined to what 3rd party software permits you to do.

[LinusP]

Hi JayBee,
Do you have any links that explain how a Trading System should be designed in terms of classes/functions e.t.c?

[PeterLe]

Thanks Jaybee
The last bit of coding I did was assembly lang on an Intel 8085 if I remember correctly about 30 years ago! (it put me off for life think!)

Ive had a good look at this over the Christmas Holidays and whilst I think it would be the preferred route for capturing data, Im not sure my balls are big enough to use it to place in play bets
Im about 6 hours in on the codeacademy site (excellent by the way), so Ill stick it out as I find it really interesting.
Thanks for the replies gents
Regards
Peter

[JayBee]

Hi JayBee,
Do you have any links that explain how a Trading System should be designed in terms of classes/functions e.t.c?

There are a few initial links at the bottom of

http://www.betfairprotrader.co.uk/2012/ ... tware.html

Now that I am free of the drudgery of writing for SmartSigger I'll post a few more programming links there over the week.

If you have anything specific in mind then I'll see what I can do.

[NileVentures]

You can click on my weblink for a few tutorials on VB for API-NG. Private message me for specifics.

JayBee,
Thank you for your very informative and measured post. Really appreciated.

[PeterLe]

Folks
I thought Id provide some feedback on this post as I was appreciative of the initial advice provided. It may also help others if they decide to go down the same path..
I decided to drop the Python idea (I may come back to it later in the future) and settled on C#). The statements seemed to make more sense to me that Python so I thought it may be easier to learn.
Ive read a couple of basic books on C# and also done some self paced learning via Udemy website (recommended).
Ive been able to code some basic stuff (using Visual Studio which makes it easier) and achieved a milestone of being able to log in to betfair and do some basic stuff.
Its not been easy though; and I know Ive a lot to learn but Rome wasnt built in a day!
Regards
Peter

[xitian]

Hi Peter,

I think you've made a good choice. I've been using C# and Visual Studio Express for 5 years now and have been very happy with my setup.

Although I don't know much Python, my initial thoughts are that C# is easier to follow and build something that can potentially get quite complex without creating a lot of spaghetti. That could just be because I come from a Java background though, which is similar to C#.

Feel free to give me a shout if you get stuck on anything and I'll do my best to help if I can.

I also highly recommend downloading the example C# client from the Betfair Developer's area if you haven't already. I actually use that as the starting point of my programs.

[PeterLe]

Hi Xitian,
Thank you for your offer of help. Much appreciated.
Microsoft have just released a code editor that will run directly on my Mac, so will have a play around with it over the next week or so (https://code.visualstudio.com/)
I'm currently running it in a virtual windows environment.
Thanks again,
regards
Peter

[PeterLe]

Hi
Just a question about the API, if someone can help
I know there are limits on data requests via the API, but the information on th BDP website is a little sparse.
If I ran my own programs on the same market (say two instances plus a copy of betangel too) all requesting market data, prices, bet info etc, would be in danger of incurring charges? I know it's a little vague, just tying to get a feel
Thanks
Regards
Peter
Ps the aggregated transactions would be less than 1000 hour, it's the market calls I'm worried about

[marksmeets302]

Peter, I've experienced throttling problems when asking for prices for many markets in one call. Not sure exactly what is the limit, but I have the feeling you can get away with practically anything as long as you stick to one market per call.

If you violate the pacing limits anyway, still not to worry. You will get a result with error TOO_MUCH_DATA. Just wait a bit and make the calls again (in a slower tempo), it's not like you will be blacklisted after a while like on betdaq.

edit: found it, here's the list: https://api.developer.betfair.com/servi ... est+Limits It speaks of limitations per request, but not how many requests you can make (infinite, it seems?).