Betfair Down / Betfair site crash

[Iron]

If that is the case, it's bad leadership.

For any Internet-based business, staying online has to be the number 1 priority. So the directors should insist on being told right away about anything that could result in the site going offline.

Jeff

agree. i dont believe that is a priority for them. i think when in goes wrong, they probably inform each other, their managers etc. no one can decide what to do, until some senior duty manager orders people what to do. i dont think anyone is authorised or as the savy to make a instant decision.

[Iron]

Is it not possible to have instant backup, whereby if I place a bet it's recorded simultaneously on the main server and the backup server?

Jeff

not really. the database can onlt be backed up at certain points. If it go;s down then you either have to roll back to the last backup which on a live site such as betfair is a no go or wait for it to come back up. IMHO there is no way you can just switch servers and carry on from where the previous server crashed.

[andyfuller]

No point in moaning on here if we don't take the chance tonight.

Here is my effort:

_____________________________

To whom it may concern,

Rather unsurprisingly to me as a long term customer of Betfair, your most important assets, your customers, were once again treated as mushrooms during the recent outages

Everyone I am sure accepts that outages happen but how they are dealt with is absolutely shocking and has been for a long time.

At times when outages happen communication with your customers is key but this really does seem to be the last thing you think of!

Therefore in the age of twitter would it be that hard to have a 'live twitter feed' about the service?

Early warning is key so as soon as problems begin a message could be put on your twitter page, then when it is fixed likewise could be done, I am referring here to not complete outages.

During the occasions recently when you experience total outages like we have just had recently we were welcomed by messages that this was planned outages or worse still there was nothing at all said.

Also more often than not the service website fails or it claims there are no issues and when you did put something up eventually on the forum it was gone as fast as the money people lost due to the outage.

So unkown to someone who had logged in after the first outage there was absolutely no warning that the site had been experiencing problems so they would feel safe to use normal stakes but really it was far from safe as the site went down again.

Maybe you could explain why it is you are so slow to put up communications but oh so very fast to remove them?

Understandably once the problem is fixed they should be removed but rather than remove them quickly in my view they should remain for a minimum of 24 hours after the site has been stable so that people logging in can see them and be cautious if needs be.

Surely there is no reason for you to remove them so quickly, please explain why you do?

How was it as well that some people received warning emails about the maintenance on the Monday while others did not. I have opted in for communications and yet never received any such warning - I know of many others in the same circumstances.

Are there reasons why the maintenance was poorly communicated?

Obviously this is another area where Twitter could be used to communicate maintenance. Warnings could be posted on there in advance and then at regular hourly intervals say in the 12 hrs leading up to the maintenance and then updates as to how it is progressing and the likelihood of when the site is going to return.

On the occasions when the site is clearly suffering issues why not take it offline completely until the problem is fixed rather than trying to keep it limping along?

My final point is that you often get a lot of criticism for keeping the site online when it is clearly struggling as people perceive you are only bothered about the commission you earn.

So why don't you donate that money to various sporting charities as a sign of goodwill and to show us you don't just care about the share price!

Kind regards,

Andy Fuller.

Can't say I felt they covered my points/questions at all tonight.

Had a bit of fun hiding a message in the message but I would be surprised if they spotted it

A big gold medal for anyone who can spot it

[jimrobo]

I would have thought the resources to do that would be massive! You would basically have to double up everything. It would probably significantly increase response times and commercially be unworkable. You would also have to have systems on top of systems to ensure the integrity of both databases.

[to75ne]

If that is the case, it's bad leadership.

For any Internet-based business, staying online has to be the number 1 priority. So the directors should insist on being told right away about anything that could result in the site going offline.

Jeff

think its a mixture of large company, therfore things happen slow.

not my job guv.

fear of making a mistake. so time is lost while they all cover their backs and shove the blame/responsibility away somewhere else.

[Iron]

I don't know what the cost would be, but you'd have thought it would be affordable for a stock market listed company with 2,000 employees. And the alternative to instant backup is surely to risk matched bets being lost irretrievably.

Incidentally, I used to work for a former FTSE 100 company, and their IT systems made Betfair's look sophisticated! They were using databases that were literally about 10 years old, and it was rather embarrassing when you spent 15 minutes entering a customer's order into the computer, only for the system to crash!

Jeff

I would have thought the resources to do that would be massive! You would basically have to double up everything. It would probably significantly increase response times and commercially be unworkable. You would also have to have systems on top of systems to ensure the integrity of both databases.

[Iron]

Possibly, but that's where strong leadership from the top is needed.

You need a director to say 'Don't worry about me bollocking you if you tell me that the site is about to go down. I won't. But if you keep me in the dark to cover your back, you can pick up your P45 on the way out'.

Jeff

think its a mixture of large company, therfore things happen slow.

not my job guv.

fear of making a mistake. so time is lost while they all cover their backs and shove the blame/responsibility away somewhere else.

[LeTiss]

Listening to BF's responses reminds me of a Blackadder quote

"There hasn't been a war run this badly since Olaf the hairy, King of all the vikings, ordered 80,000 battle helmets with the horns on the inside"

[jimrobo]

it would just introduce more grey areas!! system go's down one request gets written to the backup but the main database is down so it doesn't get written to the main. which one is correct??? The backup?? When it switches to the backup a customer has a bet matched they didn't think they had which is no different to what happens now. It just wouldn't work! The resources betfair have just to run everything on one system must be massive. There is no way they would be able to simply double everything up and have systems that switch between systems. Even major Financial exchanges do not do this

[to75ne]

Listening to BF's responses reminds me of a Blackadder quote

"There hasn't been a war run this badly since Olaf the hairy, King of all the vikings, ordered 80,000 battle helmets with the horns on the inside"

superb

[Iron]

Betfair's incompetence brings to mind this quote from Fawlty Towers, where Basil is talking with Mr O'Reilly:

Ah, hello, Mr. O'Reilly. How are you? Oh sorry, nearly forgot: Basil Fawlty, remember? The poor sod you do jobs for. So how are things your end? Ah, splendid. Now how would you like to hear how things are my end? Oh, well, up to your usual standard I think the odd hole in the wall, the odd door missing; but nothing you can't be sued for.

It's a shame that Betfair's T&C's prevent us from suing them for their incompetance!

Jeff

Listening to BF's responses reminds me of a Blackadder quote

"There hasn't been a war run this badly since Olaf the hairy, King of all the vikings, ordered 80,000 battle helmets with the horns on the inside"

[to75ne]

[/quote]

Can't say I felt they covered my points/questions at all tonight.

Had a bit of fun hiding a message in the message but I would be surprised if they spotted it

A big gold medal for anyone who can spot it [/quote]

is it to do with fungi?

[Iron]

Is anyone having any problems right now?

My connection is very laggy.

Jeff

[to75ne]

Andy Fuller.

Can't say I felt they covered my points/questions at all tonight.

Had a bit of fun hiding a message in the message but I would be surprised if they spotted it

A big gold medal for anyone who can spot it [/quote]

i think it is "TREAT AS MUSHROOMS"

[andyfuller]

Can't say I felt they covered my points/questions at all tonight.

Had a bit of fun hiding a message in the message but I would be surprised if they spotted it

A big gold medal for anyone who can spot it

is it to do with fungi?

Think you get the medal - best I can manage is a gold medal with an '!' in it though