Skip to content

Conversation

@dimitrismoustakas
Copy link
Contributor

As discussed on Issues #904 and #905, the understat module has stopped working. Understat no longer supports HTML scraping but instead offers API endpoints where you can access the same data as before. I've made the change so that the package uses the endpoints instead. Understat tests pass (they don't pass on the current main branch), but not tests on the other packages. I've tested and these tests failed to begin with so it's not an issue my pull causes but there's an underlying issue with these modules or the tests are not maintained properly.

API Integration and Data Fetching Improvements:

  • Refactored league, season, and match data retrieval to use Understat's internal API endpoints (/getStatData, /getLeagueData/{league}/{season}, /getMatchData/{match_id}) instead of HTML scraping.
  • Added the _request_api helper method to centralize API requests with appropriate headers, caching, and file storage, improving consistency and reducing code duplication.
  • Introduced the UNDERSTAT_HEADERS constant to ensure all API requests include the required X-Requested-With header.

Session and Cookie Management:

  • Added _ensure_cookies to initialize session cookies from the homepage before making API requests, ensuring authenticated and consistent access. [1] [2]

@probberechts probberechts added bug Something isn't working understat Issue or pull request related to the Understat scraper labels Jan 6, 2026
@probberechts probberechts changed the title fixed Understat scraper to work with the new JSON API endpoints [Understat] Use new JSON API endpoints Jan 6, 2026
@probberechts probberechts merged commit 61c071b into probberechts:master Jan 6, 2026
1 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working understat Issue or pull request related to the Understat scraper

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Understat] KeyError 'statData' on Google Colab (IP Blocking/Cloudflare) [Understat] Team's stat unable to be read

2 participants