[49yearsold]

Thanks. But i am still curious to know how does ESPN gets all the schedule for all sports in their persistence store? Does NFL, NBA, IPL or other sports league provide them with APIs to fetch data or does ESPN too has web scraping scripts that goes after NBA/NFL/MLB websites to get the data they need or do they have people actually doing manual data entry for all of this information via some UI into their own database?

[JudgePenitent]

I know a guy who works in sports betting software, a pretty large player. They have direct access feeds that they pay a lot for bulk access, to cbs sports, nbc, etc. I dont know how much but its no small sum. ESPN does not use web scraping for live stats- they use high speed apis that get info right from the field (that either their crew gathers or cbs gathers and they pay for access). The overwhelming majority is not manually entered.

I also do some scraping in my spare time and i can tell you that large companies will have... precautions in place to prevent you from scraping their by-the-minute sports stats they pay millions for. Id also say right now scraping is a gray market, airlines in particular have proven that they are willing to fight for their (public) data, so tread carefully.

Scraping is a problematic way to do this too, because how can you verify what is up to date? You have to rely on one single authoritative source, otherwise youll be relying on some stat calc which “guesses” what the right stat is. Additionally, if you take live data that multiple large businesses feel they own, now they can split the lawyering fees on you and even if they lose, it will be cheaper for them together.

The historical content probably would be trivial to get, and its hosts most likely dont care if you have it. its the recent/live data which may be particularly difficult to take. In my professional opinion this is two separate projects- a paid up to date version with live stats and a free historical wiki.

[49yearsold]

Thanks. I do not have any intention to do web scraping. Let's reduce the scope of my request: I want to do two things - 1) Get the schedules of the upcoming (future) matches in most of the popular sports across globe - football, soccer, cricket, tennis etc. How does ESPN or other sources get this? Do they get this information 3 months/6 months in advance from all these sports organizations? Or do they simply have people to enter this data manually in their own system? 2) I want to keep history of who won/lost the particular match and any other interesting tidbits about the match that already took place. So I am not planning on doing any web scraping and neither I plan to offer any live feed API or data. So my question is - how do I accomplish my above stated 2 goals efficiently without doing web scraping? Thanks.

[JudgePenitent]

http://www.espn.com/apis/devcenter/docs/calendar.html#using-...

Thats a start

I tried looking for an nba api but i couldnt find one- but i found nba api projects like https://pypi.org/project/nba-api/

I dont know espn gets their historical data but scraping it wouldnt be problematic. I would think the pro sports orgs would have an api for it

Looks like espn has pretty good historical data records and their api has some documentation. Using the espn api might be your best bet here

[49yearsold]

I think ESPN does not support APIs anymore. http://www.espn.com/apis/devcenter/blog/read/publicretiremen...