op3-viewer

[drpancake]

I started playing around with visualising downloads using the OP3 API 😊

🔗 https://op3-viewer.vercel.app

The code is MIT licensed and it lives here: https://github.com/takota-ltd/op3-viewer

@johnspurlock-skymethod I noticed that the API caps the row limit at 1000 — in general is it intended as a low-level API which other parties should use to fetch and store the raw data to derive their own insights on it? Or do you see it directly supporting a tool like this one?

Also I ran into an issue with the ?url=xxx parameter using a wildcard. It often complains that the URL is too short and that makes it difficult to target shows like No Agenda where episode URLs start with only mp3s.nashownotes.com/.

@jamescridland https://op3-viewer.vercel.app/downloads?prefix=podnews.net/audio/podnews&start=-7d

[jamescridland]

@drpancake You want to step through the API results using start time and end time, I think. That's how Dave is doing it.

This is Podnews's total download number for Sep 23, with a log file visible beneath the details tab. I calculate 3,167 downloads for that day. I'm not seeing those numbers reflected in your tool. (I would caption it as "hits" rather than downloads, personally).

Podnews's tool discards all downloads less than 750KB, including all HEAD requests. It doesn't stitch multiple requests from the same user together - so if you download six lots of 400KB as a progressive download, that download will be invisible to Podnews's algorithm. I would expect proper analysis of this data to show a slightly higher number than 3,167 downloads for Sep 23, but not that much bigger.

Edit: I also filter out bots from the user-agent list.

For the user-agents, this filter list will help you. Your current chart is relatively meaningless for analysis - but if you pull in the user-agent list from OPAWG, you may find that helpful. @johnspurlock-skymethod - worth linking to that from somewhere on the OP3 website? I feel it would be useful for developers to know about.

[jamescridland]

(I'm guessing that theoretically if I were to point my data to the OP3 output, it would be identical. I should do that, perhaps.)

[johnspurlock-skymethod]

Yes, you are in a great position to audit the raw hits since you keep such a detailed accounting. I'd be very curious to see if you come across any discrepancies!

[drpancake]

Thanks. Yes the user agent chart is way too simplistic for now. OPAWG seems like the way to go.

[johnspurlock-skymethod]

This is cool! You are going to want to point this new app at OP3's download api once it exists. The raw requests are not really meant to graph directly, rather ingest and do a bunch of processing yourself in your own backend - or to verify OP3 is properly saving all of them, or to audit the privacy claims.

It's like a layer cake. The current api endpoint is the lowest level, it returns everything it captures, but then we'll have higher-level rollups to shows/episodes, then eventually "downloads" in the api, computed using user-agent and other filters to most accurately id downloads for real users, and to de-dup. Then all sorts of widgets/integrations that build on top of those.

I imagine most podcasters are not going to be interested in this project until we get the downloads calculation out there - so that's where I'm heading in the medium-term (after show/episode identification).

[drpancake]

Ah I see. That was my hunch. In that case it makes sense to wait for the new download API before I develop this further.

My goal for this is to make a super easy drop-in React component so that anyone can embed a podcast's analytics like this:

<PodcastAnalytics guid="abc-123" />

And perhaps alternatively with an embeddable <iframe> pointed at the hosted version:

<div data-op3-guid="abc-123" />
<script src="https://example.com/op3-viewer.js" />