Football Data (CSV, JSON) - UPDATED 16/08/17

[dm1900]

Corners hold a relationship with goals. So if you plot goals you also plot corners.

Hmm but surely corners can add more information to game state - there may be two games that are 1-0, except the first game has 10 corners by the home team whereas the second only 2 by the home team. If we think about expected goals in the second half, doesn't the 10 corners indicate aggressive play and so more of an expectation of goals in the first game rather than the second?

[Euler]

Describe how a corner would occur.

[dm1900]

Corner = A ball going over the goal line but last touched by the defending team. So usually via clearance or a save by the goalie - both of which indicate that the attacking team are in dangerous positions. So more corners = more attempts to attack by a team, whereas the number of actual goals may not be so indicative as shown by my example.

[Euler]

So if you look at a corner it's more or less linked to an attacking move and can be related to shots on and off target. So it's effectively a derivate of shots (in general) which is a derivative of goals.

[footysystems]

Corners can be misleading imo the best way to predict goals or expect goals is to divide the number of dangerous attacks by the number of attacks to give you a % > than 70% high expectancy of a goal.

[JamieS]

A massive thank you Welshboy for taking the time to do this and share it.

[dm1900]

It is strange since there are no goals past 90' - I imagine everything in extra time is just given a 90' time anyway?

[Dallas]

It is strange since there are no goals past 90' - I imagine everything in extra time is just given a 90' time anyway?

All goals that occur in added time will get counted as either 45 or 90

[Tenable]

I stuck welshboy's data up on a simple website, I'll try and make it a big more searchable when I figure out php7

Enjoying having a play about with this SB,, just wondering if your still going too add more search options too it at some stage?

[elliotarcher1]

Hi Guys,

I'm doing a bit of analysis on stats at halftime/fulltime but I'm falling short trying to find halftime data without scraping it manually. Does anyone have halftime shots/shotsOT/Odds data to hand? Or perhaps know where I might be able to get hold of this? I'm only really interested in the top leagues. Would be seriously useful

P.s Loving this thread, loads of interesting chat going on

[Kafkaesque]

The best part of being a relative newbie on here is these brilliant topics from before my time which pop up every so often

Brilliant work, Welshboy!

An ever so slightly off topic question for ya. Whoscored.com has some stats, I would love to delve more into. Before I try to teach myself to scrape data, though, am I correct in assuming that they've specifically coded the site to make it highly problematic to scrape it for data?

[elliotarcher1]

Whoscored happens to be the website I value the most but yes, unfortunately, it is very difficult to scrape!

Betfair do offer what I am trying to find (historical HT odds) but I am not really sure I am willing to pay £200/month for it at this stage in my trading career!

Ah well...

[Angelbaby]

Thanks to the poster on this one good stuff for us newbies.

I have a question if I am looking to back the draw in a certain match and want to work out roughly when the first 1st goal would be scored.

Would you seasoned football traders look at the specific games involving the 2 teams (as this could be only 2 matches) or would you look at all games played by each team in this league and then take an average from that wider sample?

[rolandimone]

Hi All,

UPDATED 16/08/17
ADDED CSV VERSIONS OF ALL LEAGUES

I've decided to start collecting football data. Mainly because I have an interest in the sport and also because of a certain jonnyg throwing stats around in a very hard to read format.

The data is quite simple and was just scraped from HKJC.

The data is in json format, but you should be able to convert it to csv and import it to Excel. I chose json as it reduces data duplication, is structured and can be read reasonably well by the human eye. It also works really well with Python (Which I'm using to scrape and also analyse the data)
The data contains the following info.

Code: Select all

League
Season
Game Date
Home Team
Away Team
And also Goals and Red cards (Player, Time and Team)
-----
Soon:
Total FT/HT Goals and Red Cards for HT and AT (Included in CSV's)

I've managed to scrape ALL leagues and Seasons from HKJC (Where specific goal time data is provided)
I may decide to scrape more data, but for most leagues there is 10+ years worth of games, with goal and red card time data.

The data is a little too big to post here, and I didn't fancy splitting everything into multiple zip files. So I've uploaded and shared to my Dropbox
Not sure if the mods/admins could increase the file size limit, so I can add directly to the thread.
The below is a direct link to the folder containing all the data I've scraped.
Each league is in a separate text file, and I've also included a .zip of ALL the files.

https://www.dropbox.com/sh/7sif9n90ehys ... bd6ba?dl=0

Contents of the above files

Code: Select all

Number of Countries: 20
Country: Argentine - Leagues: 1
League Name: Argentine Division 1 - Seasons: 11
(2009-10(Au), 2010-11 (Sp), 2010-11(Au), 2011-12(Au), 2012-13 (In), 2012-13(Fi), 2013-14 (In), 2013-14(Fi), 2014-15 (In), 2015, 2016)
Country: Australian - Leagues: 2
League Name: Australian Division 1 Playoffs - Seasons: 7
(2008-09, 2009-10, 2010-11, 2011-12, 2012-13, 2013-14, 2014-15)
League Name: Australian Division 1 - Seasons: 9
(2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Belgian - Leagues: 3
League Name: Belgian Division 1 Championship Playoffs - Seasons: 1
(2016)
League Name: Belgian Division 1 UE Cup Playoffs - Seasons: 1
(2016)
League Name: Belgian Division 1 - Seasons: 2
(2015-2016, 2016-2017)
Country: Brazilian - Leagues: 3
League Name: Brazilian Division 1 - Seasons: 9
(2005, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
League Name: Brazilian Paulista League Knockout stage - Seasons: 1
(2014)
League Name: Brazilian Paulista League - Seasons: 1
(2014)
Country: Chilean - Leagues: 1
League Name: Chilean Division 1 - Seasons: 6
(2014-15(AP), 2014-15(CL), 2015-16(AP), 2015-16(CL), 2016-17(AP), 2016-17(CL))
Country: Dutch - Leagues: 3
League Name: Dutch Cup - Seasons: 7
(2005-06, 2007-08, 2008-09, 2009-10, 2010-11, 2011-12, 2012-13)
League Name: Dutch Division 1 - Seasons: 10
(2005-2006, 2006-2007, 2008-2009, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: Dutch Division 2 - Seasons: 6
(2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Eng - Leagues: 4
League Name: Eng Championship - Seasons: 13
(2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: Eng League 1 - Seasons: 8
(2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: Eng League Cup - Seasons: 6
(2006-07, 2007-08, 2009-10, 2010-11, 2012-13, 2013-14)
League Name: Eng Premier - Seasons: 14
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: French - Leagues: 4
League Name: French Division 1 - Seasons: 12
(2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: French Division 2 - Seasons: 6
(2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: French FA Cup - Seasons: 1
(2005-06)
League Name: French League Cup - Seasons: 1
(2008-09)
Country: German - Leagues: 3
League Name: German Cup - Seasons: 5
(2006-07, 2007-08, 2008-09, 2012-13, 2013-14)
League Name: German Division 1 - Seasons: 14
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: German Division 2 - Seasons: 8
(2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Italian - Leagues: 2
League Name: Italian Cup - Seasons: 2
(2005-06, 2007-08)
League Name: Italian Division 1 - Seasons: 14
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Japanese - Leagues: 3
League Name: Japanese Division 1 - Seasons: 14
(2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015-Stage 1, 2015-Stage 2, 2016-Stage 1, 2016-Stage 2)
League Name: Japanese Division 2 - Seasons: 9
(2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
League Name: Japanese League Cup - Seasons: 9
(2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014)
Country: Korean - Leagues: 1
League Name: Korean Division 1 - Seasons: 1
(2016)
Country: Mexican - Leagues: 1
League Name: Mexican Premier - Seasons: 6
(2014-15 (AP), 2014-15(CL), 2015-16(AP), 2015-16(CL), 2016-17(AP), 2016-17(CL))
Country: Norwegian - Leagues: 1
League Name: Norwegian Division 1 - Seasons: 12
(2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
Country: Portuguese - Leagues: 1
League Name: Portuguese Premier - Seasons: 9
(2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Russian - Leagues: 1
League Name: Russian Premier - Seasons: 3
(2014-2015, 2015-2016, 2016-2017)
Country: Scottish - Leagues: 2
League Name: Scottish League Cup - Seasons: 5
(2005-06, 2007-08, 2008-09, 2009-10, 2012-13)
League Name: Scottish Premier - Seasons: 12
(2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Spanish - Leagues: 1
League Name: Spanish Division 1 - Seasons: 13
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Swedish - Leagues: 1
League Name: Swedish Division 1 - Seasons: 12
(2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
Country: US - Leagues: 2
League Name: US Football League Playoffs - Seasons: 8
(2007, 2009, 2010, 2011, 2012, 2013, 2014, 2015)
League Name: US Football League - Seasons: 10
(2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)

Okay so there are now two sets of CSV files.
One with reds and goals separated, and one where they are together as "Events" The columns for each are as follows:


Goals and Reds Separated

Code: Select all

SEASON, DATE, HOME, AWAY, HHT_GOALS, AHT_GOALS, HFT_GOALS, AFT_GOALS, TOTAL_GOALS, HOME_REDS, AWAY_REDS, TOTAL_REDS, RED_1_TIME, RED_1_TEAM, RED_2_TIME, RED_2_TEAM, RED_3_TIME, RED_3_TEAM, RED_4_TIME, RED_4_TEAM, RED_5_TIME, RED_5_TEAM, RED_6_TIME, RED_6_TEAM, RED_7_TIME, RED_7_TEAM, RED_8_TIME, RED_8_TEAM, RED_9_TIME, RED_9_TEAM, RED_10_TIME, RED_10_TEAM, RED_11_TIME, RED_11_TEAM, RED_12_TIME, RED_12_TEAM, RED_13_TIME, RED_13_TEAM, RED_14_TIME, RED_14_TEAM, RED_15_TIME, RED_15_TEAM, GOAL_1_TIME, GOAL_1_TEAM, GOAL_2_TIME, GOAL_2_TEAM, GOAL_3_TIME, GOAL_3_TEAM, GOAL_4_TIME, GOAL_4_TEAM, GOAL_5_TIME, GOAL_5_TEAM, GOAL_6_TIME, GOAL_6_TEAM, GOAL_7_TIME, GOAL_7_TEAM, GOAL_8_TIME, GOAL_8_TEAM, GOAL_9_TIME, GOAL_9_TEAM, GOAL_10_TIME, GOAL_10_TEAM, GOAL_11_TIME, GOAL_11_TEAM, GOAL_12_TIME, GOAL_12_TEAM, GOAL_13_TIME, GOAL_13_TEAM, GOAL_14_TIME, GOAL_14_TEAM, GOAL_15_TIME, GOAL_15_TEAM

Goals and Reds Together as Events

Code: Select all

SEASON, DATE, HOME, AWAY, HHT_GOALS, AHT_GOALS, HFT_GOALS, AFT_GOALS, TOTAL_GOALS, HOME_REDS, AWAY_REDS, TOTAL_REDS, EVENT_1_TIME, EVENT_1_TEAM, EVENT_1_TYPE, EVENT_2_TIME, EVENT_2_TEAM, EVENT_2_TYPE, EVENT_3_TIME, EVENT_3_TEAM, EVENT_3_TYPE, EVENT_4_TIME, EVENT_4_TEAM, EVENT_4_TYPE, EVENT_5_TIME, EVENT_5_TEAM, EVENT_5_TYPE, EVENT_6_TIME, EVENT_6_TEAM, EVENT_6_TYPE, EVENT_7_TIME, EVENT_7_TEAM, EVENT_7_TYPE, EVENT_8_TIME, EVENT_8_TEAM, EVENT_8_TYPE, EVENT_9_TIME, EVENT_9_TEAM, EVENT_9_TYPE, EVENT_10_TIME, EVENT_10_TEAM, EVENT_10_TYPE, EVENT_11_TIME, EVENT_11_TEAM, EVENT_11_TYPE, EVENT_12_TIME, EVENT_12_TEAM, EVENT_12_TYPE, EVENT_13_TIME, EVENT_13_TEAM, EVENT_13_TYPE, EVENT_14_TIME, EVENT_14_TEAM, EVENT_14_TYPE, EVENT_15_TIME, EVENT_15_TEAM, EVENT_15_TYPE, EVENT_16_TIME, EVENT_16_TEAM, EVENT_16_TYPE, EVENT_17_TIME, EVENT_17_TEAM, EVENT_17_TYPE, EVENT_18_TIME, EVENT_18_TEAM, EVENT_18_TYPE, EVENT_19_TIME, EVENT_19_TEAM, EVENT_19_TYPE, EVENT_20_TIME, EVENT_20_TEAM, EVENT_20_TYPE, EVENT_21_TIME, EVENT_21_TEAM, EVENT_21_TYPE, EVENT_22_TIME, EVENT_22_TEAM, EVENT_22_TYPE, EVENT_23_TIME, EVENT_23_TEAM, EVENT_23_TYPE, EVENT_24_TIME, EVENT_24_TEAM, EVENT_24_TYPE, EVENT_25_TIME, EVENT_25_TEAM, EVENT_25_TYPE, EVENT_26_TIME, EVENT_26_TEAM, EVENT_26_TYPE, EVENT_27_TIME, EVENT_27_TEAM, EVENT_27_TYPE, EVENT_28_TIME, EVENT_28_TEAM, EVENT_28_TYPE, EVENT_29_TIME, EVENT_29_TEAM, EVENT_29_TYPE, EVENT_30_TIME, EVENT_30_TEAM, EVENT_30_TYPE

Please let me know if I should make any changes to the below columns, or the CSV's at all
I've tried, best I can, to go through the data and make sure all games are there. I've not spotted any leagues with missing games as of yet. BUT PLEASE LET ME KNOW IF YOU SPOT ANY ISSUES.

Just to note: The data is all scraped from HKJC, so if there are any errors it would be down to the data they provided. (The dates I've been told are in HK time)

Cheers,
Adam

Link not working.

[Dallas]

Link not working.

I've just tried it and its still active and working