Download statistics for release assets

[huonw]

I was curious, so I did an investigation of the download statistics of the released artifacts. I compared:

• the PyPI downloads of pantsbuild.pants in the last 2 years
• the PyPI downloads in the last 30 days
• the GitHub Releases downloads across all time (all assets)

This plot shows the values for all stable releases since the start of 2022:

Summary:

• from the blue series (top), it's clear that 2.13.0 and 2.17.0 at the most popular "ever" releases
• from the red series (middle), it's clear that most people are still on 2.17.0
• from the green series (bottom), there's far fewer downloads of 2.18.0

This plot shows the PyPI downloads per day for the last year, for the versions with the most download (we don't have this info available for GitHub assets, AFAICT):

There's hints to me that either:

• people aren't transitioning to 2.18.0 as quickly as they did earlier releases
• OR, GitHub download stats are fairly different to PyPI ones

The raw data is available here: https://gist.github.com/huonw/112684339a5e6e1e947e75890233ff35

• downloads-2stable.csv: PyPI and GitHub releases stats for all 2.x.y stable versions
• downloads-all.csv: PyPI and GitHub releases stats for all versions
• downloads-pypi-year-per-day.csv: PyPI release stats per day per version for the last year

How I computed this

BigQuery queries following https://packaging.python.org/en/latest/guides/analyzing-pypi-package-downloads/

30 days:

SELECT file.version, COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'pantsbuild-pants'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND CURRENT_DATE()
GROUP BY file.version

2 years:

SELECT file.version, COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'pantsbuild-pants'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 730 DAY)
    AND CURRENT_DATE()
GROUP BY file.version

Downloads per day for the last year:

SELECT file.version, COUNT(*) AS num_downloads, DATE_TRUNC(DATE(timestamp), DAY) AS `day`
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'pantsbuild-pants'
  -- Only query the last 2 years days of history
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 365 DAY)
    AND CURRENT_DATE()
GROUP BY day, file.version
ORDER BY day DESC

Stats for assets:

gh api --paginate -X GET '/repos/pantsbuild/pants/releases' | jq -r '.[] | [.tag_name, ([.assets[].download_count] | add) // 0] | @csv'  |  sort -V

Release dates (from tags)

git tag -l 'release_*' --sort=taggerdate --format='%(refname:short),%(taggerdate:short-local)'

[kaos]

I'll just leave a word of caution if anyone tries the BigQuery thing, make sure you have charge limits set, unless you want a surprise charge if you manage to run a query that requires a lot of data processing.. (I managed to run just a couple of queries in the past, ended up costing me about $50 😬 )

[huonw]

I'm using an account that doesn't have billing details set, for extra defence... but yeah, good call 😅

[kaos]

Oh, I would've loved to see the stats for 2.8.0, as I remember it being another "big" release.. (in terms of features, no idea about adoption)

[huonw]

From https://gist.github.com/huonw/112684339a5e6e1e947e75890233ff35, it's lower than many:

• PyPI last 30 days: 486
• PyPI last 2 years: 99041
• GitHub releases: 371

[huonw]

On slack, @thejcannon raises that he's got automation that queries the GitHub Releases stats per day, to be able to give a time series of those (since the API just exposes the current total number, it seems). It's at https://github.com/thejcannon/pants-download-stats