Football scores to Guardian programatically - Script & Sample BAF

Football, Soccer - whatever you call it. It is the beautiful game.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

Wonkymuffin wrote:
Fri Mar 12, 2021 12:00 pm
I used the URL provided but did not add "$data1 = _INetGetSource($URL)" so I guess this is why I did not get the required data. So in that case, yes, more knowledge would have been useful. ;)
The required stats are now available to me so I will work on getting a suitable csv and will post back here when I've achieved that. Thank you again for all of your help.
Yes, you have to retrieve the data from the new stats URL before you can parse it.

Have fun with the stats 👍
petermikael
Posts: 4
Joined: Fri Jun 19, 2020 9:40 am

I am working on a solution to compare the teamnames with betfair teamnames, if i am correct theres no way to match them if they are not spelled exactly the same. Is there a way to download teamnames from betfair or maybe someone already have them and could share. Please correct me if i am completely wrong, maybe someone already solved this but i missed it. It would be almost magic if it could be done, it opens up a lot of possibilities. Tried other ways to get the correct score ( comparing odds by examples from Dallas) but for me it is not working as good as it should.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

petermikael wrote:
Tue Mar 23, 2021 2:38 pm
I am working on a solution to compare the teamnames with betfair teamnames, if i am correct theres no way to match them if they are not spelled exactly the same. Is there a way to download teamnames from betfair or maybe someone already have them and could share. Please correct me if i am completely wrong, maybe someone already solved this but i missed it. It would be almost magic if it could be done, it opens up a lot of possibilities. Tried other ways to get the correct score ( comparing odds by examples from Dallas) but for me it is not working as good as it should.

What I do with the tennis is just match on the surname by doing a string strip in the script, then I only match the first 10 letters or so. That removes the issue where BF has "Leeds" and the API returns "Leeds United" for example. Then I wildcard the match in BA so it will match the Selection if Leeds is anywhere in the BF name. That should solve most, if not all matching issues. if you aren't up to scripting that let me know and I'll add in the function I'm using to the existing code when I get a sec - taking any string before the first space as the text to match on

I also have a function to remove most special characters - accents, circumflex etc, which will also help

I only posted the original as an example so never got around to testing it fully for selection matching.
Wonkymuffin
Posts: 6
Joined: Wed Nov 09, 2016 8:46 am

I too have encountered a couple of problems with this and have not had the time to try and resolve them, but will state them here so that others are aware.

1. Yes, the team name matching has been a problem and the ideas listed by sniffer66 will be useful and a step in the right direction. I have been using the Sofascore shortname which offers better matching than the long name but does give errors which are either due to strange characters in the name (UK matches have all been okay) or due to similar team names - Note for sniffer66 - using just the first part of the team name, up to the first space, will be of benefit but will not always work (which means that personally I wouldn't rely on it). A for instance is Bristol which is used in four different team names. Bristol City, Rovers, City U23, City WFC.
2. Whilst there are many Sofascore matches with copious statistics there are more without, so for me personally I will miss out on many opportunities. Not alot can be done about that and if you are only interested in the score then it will not affect you.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

Wonkymuffin wrote:
Tue Mar 23, 2021 5:12 pm
I too have encountered a couple of problems with this and have not had the time to try and resolve them, but will state them here so that others are aware.

1. Yes, the team name matching has been a problem and the ideas listed by sniffer66 will be useful and a step in the right direction. I have been using the Sofascore shortname which offers better matching than the long name but does give errors which are either due to strange characters in the name (UK matches have all been okay) or due to similar team names - Note for sniffer66 - using just the first part of the team name, up to the first space, will be of benefit but will not always work (which means that personally I wouldn't rely on it). A for instance is Bristol which is used in four different team names. Bristol City, Rovers, City U23, City WFC.
2. Whilst there are many Sofascore matches with copious statistics there are more without, so for me personally I will miss out on many opportunities. Not alot can be done about that and if you are only interested in the score then it will not affect you.

Good points. The way to use a short string is to actually check that both teams match the short name. So even if you have Bristol as home team, then the other team has to match as well before the data is used in the automation.
That might be an issue in a derby but again you could pull 2 short strings (pre and post space) and match on both... United & City for instance
petermikael
Posts: 4
Joined: Fri Jun 19, 2020 9:40 am

sniffer66 wrote:
Tue Mar 23, 2021 4:03 pm
petermikael wrote:
Tue Mar 23, 2021 2:38 pm
I am working on a solution to compare the teamnames with betfair teamnames, if i am correct theres no way to match them if they are not spelled exactly the same. Is there a way to download teamnames from betfair or maybe someone already have them and could share. Please correct me if i am completely wrong, maybe someone already solved this but i missed it. It would be almost magic if it could be done, it opens up a lot of possibilities. Tried other ways to get the correct score ( comparing odds by examples from Dallas) but for me it is not working as good as it should.

What I do with the tennis is just match on the surname by doing a string strip in the script, then I only match the first 10 letters or so. That removes the issue where BF has "Leeds" and the API returns "Leeds United" for example. Then I wildcard the match in BA so it will match the Selection if Leeds is anywhere in the BF name. That should solve most, if not all matching issues. if you aren't up to scripting that let me know and I'll add in the function I'm using to the existing code when I get a sec - taking any string before the first space as the text to match on

I also have a function to remove most special characters - accents, circumflex etc, which will also help

I only posted the original as an example so never got around to testing it fully for selection matching.
It would be great if you could add that to the existing code, i understand the scripting in most cases but not very good doing them myself. I think this is worth trying out. Thanks for taking the time :D
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

petermikael wrote:
Fri Mar 26, 2021 7:49 am
sniffer66 wrote:
Tue Mar 23, 2021 4:03 pm
petermikael wrote:
Tue Mar 23, 2021 2:38 pm
I am working on a solution to compare the teamnames with betfair teamnames, if i am correct theres no way to match them if they are not spelled exactly the same. Is there a way to download teamnames from betfair or maybe someone already have them and could share. Please correct me if i am completely wrong, maybe someone already solved this but i missed it. It would be almost magic if it could be done, it opens up a lot of possibilities. Tried other ways to get the correct score ( comparing odds by examples from Dallas) but for me it is not working as good as it should.

What I do with the tennis is just match on the surname by doing a string strip in the script, then I only match the first 10 letters or so. That removes the issue where BF has "Leeds" and the API returns "Leeds United" for example. Then I wildcard the match in BA so it will match the Selection if Leeds is anywhere in the BF name. That should solve most, if not all matching issues. if you aren't up to scripting that let me know and I'll add in the function I'm using to the existing code when I get a sec - taking any string before the first space as the text to match on

I also have a function to remove most special characters - accents, circumflex etc, which will also help

I only posted the original as an example so never got around to testing it fully for selection matching.
It would be great if you could add that to the existing code, i understand the scripting in most cases but not very good doing them myself. I think this is worth trying out. Thanks for taking the time :D
I've had a little play around this morning and it's a bit frustrating due to matching team names

Say, Man City were playing Man United

I can split the names in the API to

a: Manchester
b: City

c: Manchester
d: United

If I try to match on "Manchester" both selections will match and would give incorrect results

I could create 4 score SV's, 2 per team, each returning the current score

Team 1:
Score_1 = "Manchester"
Score_2 = "United"

So in the baf you could have (per selection) logically. If Score_1=Score_2 Then Correct_Score = Score_1
If not, ignore the result

That would work but would mean some matches where both teams matched would be ignored. Not ideal

The best solution would be if we were allowed multiple wildcards in the CSV file

i.e *Man*City* (Any String & "Man" & Any String + "City" & Any String)

That would match 100% every time

However, I've tested it today and the current pattern matching only allows a wildcard at the start, end or in the middle (contains). Multiple wildcards don't work.
Might be worth a Suggestion post to request an enhancement to the pattern matching. i.e RegExp or similar
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

i wonder if an alternative option would be to build a rolling lookup table which your code referenced everytime. so, even if you had a direct match, such as Liverpool or Arsenal, you'd still always use the lookup to pick the correct team. this would involve a certain amount of admin as you'd have to create an exceptions table each time that included teams where no match was correlated in the lookup. thereafter, manually (of fuzzy matching) into the lookup file.

not ideal, but would eventually fall into place once coverage across a season.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

jimibt wrote:
Fri Mar 26, 2021 11:58 am
i wonder if an alternative option would be to build a rolling lookup table which your code referenced everytime. so, even if you had a direct match, such as Liverpool or Arsenal, you'd still always use the lookup to pick the correct team. this would involve a certain amount of admin as you'd have to create an exceptions table each time that included teams where no match was correlated in the lookup. thereafter, manually (of fuzzy matching) into the lookup file.

not ideal, but would eventually fall into place once coverage across a season.
That would work Jim but would take some upkeep

However, I just had an epiphany (and can't believe I didnt see it before lol)
Just re-read the CSV help file - we don't need to match the selection name in footy OR tennis. You can match a CSV SV to a row number. We know who the home team are AND we know who the away are. Home team is always Row 1. Can't believe I didnt realise this - doh !

"# Column4: Selection Row Number. The number in this column will be matched against the row (in betfair order) of the selection in the market to decide whether this row applies to the selection or not. Enter a number >= 1 or use a single * character to ignore the filter"

So "Man" in "Row 1" will match Man Utd playing at home, "Man" in "Row 2" will match Man City playing away,
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

sniffer66 wrote:
Fri Mar 26, 2021 12:03 pm
jimibt wrote:
Fri Mar 26, 2021 11:58 am
i wonder if an alternative option would be to build a rolling lookup table which your code referenced everytime. so, even if you had a direct match, such as Liverpool or Arsenal, you'd still always use the lookup to pick the correct team. this would involve a certain amount of admin as you'd have to create an exceptions table each time that included teams where no match was correlated in the lookup. thereafter, manually (of fuzzy matching) into the lookup file.

not ideal, but would eventually fall into place once coverage across a season.
That would work Jim but would take some upkeep

However, I just had an epiphany (and can't believe I didnt see it before lol)
Just re-read the CSV help file - we don't need to match the selection name in footy OR tennis. You can match a CSV SV to a row number. We know who the home team are AND we know who the away are. Home team is always Row 1. Can't believe I didnt realise this - doh !

"# Column4: Selection Row Number. The number in this column will be matched against the row (in betfair order) of the selection in the market to decide whether this row applies to the selection or not. Enter a number >= 1 or use a single * character to ignore the filter"

So "Man" in "Row 1" will match Man Utd playing at home, "Man" in "Row 2" will match Man City playing away,
bingo - embarrassed that this didn't pop off the page at me either. perfect and generic solution (for all head to head sports)

and just to clarify, you would simply apply a 1 or 2 to the value in column #4 and a * in column #3 (selection name)... ;)
fxcarllos
Posts: 14
Joined: Thu Aug 19, 2010 8:58 pm

jimibt wrote:
Fri Mar 26, 2021 12:13 pm


bingo - embarrassed that this didn't pop off the page at me either. perfect and generic solution (for all head to head sports)

and just to clarify, you would simply apply a 1 or 2 to the value in column #4 and a * in column #3 (selection name)... ;)
dont understand how this solves the problem as guardian wouldnt be able to match all these 1 and 2 in the csv file with the correct game. Without the team name idt its not possible to correctly identify the game and score.
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

fxcarllos wrote:
Tue Mar 30, 2021 8:00 am
jimibt wrote:
Fri Mar 26, 2021 12:13 pm


bingo - embarrassed that this didn't pop off the page at me either. perfect and generic solution (for all head to head sports)

and just to clarify, you would simply apply a 1 or 2 to the value in column #4 and a * in column #3 (selection name)... ;)
dont understand how this solves the problem as guardian wouldnt be able to match all these 1 and 2 in the csv file with the correct game. Without the team name idt its not possible to correctly identify the game and score.
Correct. For multiple matches in the CSV you need a short string in the selection name (Col 3) PLUS the row number in Col 4. That would prevent issues with derby matches and mismatches pretty much. I'll adjust the script and repost when I get a sec
fxcarllos
Posts: 14
Joined: Thu Aug 19, 2010 8:58 pm

sniffer66 wrote:
Tue Mar 30, 2021 8:19 am

Correct. For multiple matches in the CSV you need a short string in the selection name (Col 3) PLUS the row number in Col 4. That would prevent issues with derby matches and mismatches pretty much. I'll adjust the script and repost when I get a sec
That would be very wellcome. Cheers :)
sniffer66
Posts: 1666
Joined: Thu May 02, 2019 8:37 am

Ok, had to get a little creative trying to prevent double matches (2 teams playing as City etc)

I've added in a function that will split the entire team name, delimiting by spaces, and then find the first of those split strings with more than 3 characters. It then checks all previous name matches downloaded on that cycle to see if that string exists for another team (City, United etc) and moves on to the next string > 3 chars if another match is found. It will then wildcard that sub string in the CSV. It will also specify the row number in the CSV (1 for Home Team, 2 for Away)

If it cant find any unique string of more than 3 characters it will return a selection name of "NotFound" and not match any selection. I think it better to not have a score returned at all than to return a score for another team. You'll need to cater for a non entry in your baf

Anything less than a 4 character string will cause too many matches IMO

I've also addedd in a function that will strip out most, if not all special characters - accents, circumflex etc

That's about as much as I can do. I think you'll be pretty unlucky to have an incorrect match, in the same row but I could be wrong. Test and let me know

Stu

Tested fine on Mongolia v Japan

30/03/2021 12:57:16: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:16: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:22: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:22: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:27: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:27: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:32: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:32: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:37: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:37: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:43: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:43: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:45: Guardian has detected that the market is suspended

Current CSV looks like this:

* * *Mongolia* 1 S Score 0
* * *Japan* 2 S Score 8
* * *Maziya* 1 S Score 0
* * *United* 2 S Score 0
* * *Nedlands* 1 S Score 1
* * *Gosnells* 2 S Score 4
* * *Bristol* 1 S Score 0
* * *Millwall* 2 S Score 0
* * *Velsao* 1 S Score 1
* * *Churchill* 2 S Score 2
* * *Eves* 1 S Score 1
* * *Hindustan* 2 S Score 1
* * *Sudeva* 1 S Score 2
* * *Rangers* 2 S Score 1
* * *Arema* 1 S Score 1
* * *PSIS* 2 S Score 0
* * *Maccabi* 1 S Score 0
* * *Bnei* 2 S Score 0
* * *Sport* 1 S Score 0
* * *Hapoel* 2 S Score 1
* * *Ironi* 1 S Score 0
* * *Pardes* 2 S Score 0
* * *Hakoach* 1 S Score 0
* * *Marmorek* 2 S Score 2
* * *Shurtan* 1 S Score 0
* * *Buxoro* 2 S Score 0
* * *Becamex* 1 S Score 0
* * *NotFound* 2 S Score 1

Code: Select all

#include <Inet.au3>
#include <json.au3>
#include <Array.au3>
#include <String.au3>
#include <File.au3>

$TitleString = "Team|Score"


;File to connect Guardian to. Will save in same location as script
$File = "score.csv"


While 1

	$URL = "https://api.sofascore.com/api/v1/sport/football/events/live"
	$data = _INetGetSource($URL)
	$object = json_decode($data)

	Local $Count = Json_Get($object, '.events')

	FileDelete($File)
	FileOpen($File)

	$z = 0

	Local $NameArray[1]

	For $i = 0 To UBound($Count) - 1

		;Writing "1" to first line of the CSV. Required by Guardian
		If $z = 0 Then
			FileWriteLine($File, "1")
		EndIf
		$z = 1

		$HomeTeam = _StringReplaceAccent(Strip(Json_Get($object, '.events' & '[' & $i & '].homeTeam.name')))
		$AwayTeam = _StringReplaceAccent(Strip(Json_Get($object, '.events' & '[' & $i & '].awayTeam.name')))

		$HomeString = "*,*," & "*" & $HomeTeam & "*,1,"
		$AwayString = "*,*," & "*" & $AwayTeam & "*,2,"

		$HomeString = $HomeString & "S,Score," & Json_Get($object, '.events' & '[' & $i & '].homeScore.current')
		$AwayString = $AwayString & "S,Score," & Json_Get($object, '.events' & '[' & $i & '].awayScore.current')

		$URL = "https://api.sofascore.com/api/v1/event/" & Json_Get($object, '.events' & '[' & $i & '].id') & "/statistics"
		;MsgBox("", "", $URL)
		$data1 = _INetGetSource($URL)

		If $data1 <> "" Then
			;Json_Dump($data1)
			;MsgBox("", "", $data1)
		EndIf

		;MsgBox($MB_SYSTEMMODAL, "Score", $HomeString & @CRLF & $AwayString)
		FileWriteLine($File, $HomeString)
		FileWriteLine($File, $AwayString)

		_ArrayAdd($NameArray, $HomeTeam)
		_ArrayAdd($NameArray, $AwayTeam)
		;_ArrayDisplay($NameArray)


	Next

	FileClose($File)

	;Wait time between score refresh in milliseconds
	;#######DO NOT SET SHORTER THAN 10S AS YOUR IP MAY BE BLOCKED#####

	Sleep(30000) ;30 seconds


WEnd

Func Strip($Param)
	$f = "NotFound"
	$e = StringSplit($Param, " ")
	;_ArrayDisplay($e)
	For $z = 1 To UBound($e) - 1
		_ArraySearch($NameArray, $e[$z])
		If @error Then
			If StringLen($e[$z]) > 3 Then
				$f = $e[$z]
				ExitLoop
			EndIf
		EndIf
	Next
	;MsgBox($MB_SYSTEMMODAL, "Result", $f )
	Return $f
EndFunc   ;==>Strip


Func _StringReplaceAccent($sString)
	Local $exp, $rep
	Local $Pattern[29][2] = [ _
			["[ÀÁÂÃÅÆ]", "A"], ["[àáâãåą]", "a"], ["Ä", "Ae"], ["[æä]", "ae"], _
			["Þ", "B"], ["þ", "b"], _
			["ÇĆ", "C"], ["[çćč]", "c"], _
			["[ÈÉÊË]", "E"], ["[èéêë]", "e"], _
			["[ÌÍÎÏ]", "I"], ["[ìíîï]", "i"], _
			["Ñ", "N"], ["ñ", "n"], _
			["[ÒÓÔÕÖØ]", "O"], ["[ðòóôõöø]", "o"], _
			["ř", "r"], _
			["[ŠŚ]", "S"], ["[š]", "s"], _
			["ß", "Ss"], _
			["Ț", "T"], _
			["[ÙÚÛ]", "U"], ["[ùúû]", "u"], ["Ü", "Ue"], ["ü", "ue"], _
			["Ý", "Y"], ["[ýýÿ]", "y"], _
			["Ž", "Z"], ["ž", "z"]]

	For $i = 0 To (UBound($Pattern) - 1)
		$exp = $Pattern[$i][0]
		If $exp = "" Then ContinueLoop
		$rep = $Pattern[$i][1]

		$sString = StringRegExpReplace($sString, $exp, $rep)
		If @error == 0 And @extended > 0 Then
			;ConsoleWrite($sString & @LF & "--> " & $exp & @LF)
		EndIf
	Next

	Return $sString
EndFunc   ;==>_StringReplaceAccent

fxcarllos
Posts: 14
Joined: Thu Aug 19, 2010 8:58 pm

sniffer66 wrote:
Tue Mar 30, 2021 1:06 pm


Tested fine on Mongolia v Japan

30/03/2021 12:57:16: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:16: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:22: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:22: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:27: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:27: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:32: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:32: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:37: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:37: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:43: [G_Auto 1] : Store Value for Mongolia: Score = 0
30/03/2021 12:57:43: [G_Auto 1] : Store Value for Japan: Score = 7
30/03/2021 12:57:45: Guardian has detected that the market is suspended

Current CSV looks like this:

* * *Mongolia* 1 S Score 0
* * *Japan* 2 S Score 8
* * *Maziya* 1 S Score 0
* * *United* 2 S Score 0
* * *Nedlands* 1 S Score 1
* * *Gosnells* 2 S Score 4
* * *Bristol* 1 S Score 0
* * *Millwall* 2 S Score 0
* * *Velsao* 1 S Score 1
* * *Churchill* 2 S Score 2
* * *Eves* 1 S Score 1
* * *Hindustan* 2 S Score 1
* * *Sudeva* 1 S Score 2
* * *Rangers* 2 S Score 1
* * *Arema* 1 S Score 1
* * *PSIS* 2 S Score 0
* * *Maccabi* 1 S Score 0
* * *Bnei* 2 S Score 0
* * *Sport* 1 S Score 0
* * *Hapoel* 2 S Score 1
* * *Ironi* 1 S Score 0
* * *Pardes* 2 S Score 0
* * *Hakoach* 1 S Score 0
* * *Marmorek* 2 S Score 2
* * *Shurtan* 1 S Score 0
* * *Buxoro* 2 S Score 0
* * *Becamex* 1 S Score 0
* * *NotFound* 2 S Score 1
I really apreciate your effort. my csv file after i run the new code doesnt mention any teams..

*,*,**,1,S,Score,0
*,*,**,2,S,Score,0
*,*,**,1,S,Score,1
*,*,**,2,S,Score,0
*,*,**,1,S,Score,3
*,*,**,2,S,Score,0
*,*,**,1,S,Score,1
*,*,**,2,S,Score,0
*,*,**,1,S,Score,2
*,*,**,2,S,Score,0
*,*,**,1,S,Score,0
*,*,**,2,S,Score,0
*,*,**,1,S,Score,0
Post Reply

Return to “Football trading”