Install own server (federation possible?)

[benhartwich]

Hi,

is it possible to install and host my own op3 analytics server? I´ve found the deployment script, but not really any kind of installation documentation. Would be great to self host it, because my organisation would require it.

Looking forward to get some insights and tipps.

[johnspurlock-skymethod]

Hi there,

At the moment, OP3 is open source to serve as a way to audit op3.dev, not to fork or self-host. I try not to mention self-hosting in the project docs to try and frame the discussion, but it's probably good to address it directly here.

I understand this is the first question asked of almost every other open source project (I've asked it myself!), but I'm trying to do something slightly different with OP3. There are economies of scale and utility in comparing shows apples to apples when shows are being analyzed by the same instance, as well as trusting the source.

So in the short/medium term, I'd be really curious to know why your organization would require self-hosting a copy? Again I understand (having worked in enterprise dev) why they would say that in general as a default for most enterprise software, but I'm curious for this particular case of auditable privacy-preserving podcast analytics, what specifically they would require in order to trust OP3 itself. Hoping we could put something in place to make it useful in the common instance.

---

Now...

...in the long-term: say OP3 executes on its entire short/medium-term roadmap - that is, providing high quality common analytics in a trusted, performant, and cost effective way, with plenty of docs around architecture, auditability and management, I could see turning to federation being an interesting problem to tackle at that point. But right now it would be too much of a distraction (we're not even providing download stats yet!) And in fact some of the architecture is going to change as Cloudflare rolls out more of its platform, so running it as-is certainly is possible (everything starts from the GitHub action!), but it would really be a waste of time right now.

I would imagine if all goes well, asking this same question a year from now would be far more actionable.

Hope that helps!

[benhartwich]

Hi,

thank you very much for your explanation. I understand you fully.

I think my question is not about any legal specification, but a basic philosophy of not giving all things to any kind of cloud system - independently if you´re a person or company. Why? Because we need some de-centralised thinking whenever we´re talking about an open podcast service. Net neutrality & de-centralised web seems to me one of the major principles, which is already broken on so much parts (mail server, web hosting, storage....). Technological I understand your need and approach, but it will break the main principles of an open architecture - in my opinion. And there seems to be new desire at my company to not fully trust any cloud infrastructure - because what happens if someone decides to shut down the service? How can you export the data, how can I ensure, that the data is in my property. And this is the main question to us. Our data = our property = our responsibility!

However - great project and I will follow / test it!

Best,

[johnspurlock-skymethod]

Re: Data export - the open access nature of OP3 lends itself well to easy backup over the API. In fact, in some ways it's better since dealing with raw IP addresses is now not a problem that's on your plate at all.

[ve-varun-sharma]

@johnspurlock-skymethod that's actually a fantastic point!

[dellagustin]

@benhartwich , I completely understand the basic philosophy of not giving all things to any kind of cloud system concern here, and I share it.
Even though @johnspurlock-skymethod has done everything marvelously right so far (everything open source and auditable, including terms of service and privacy terms, transparency over costs, etc...), it is still an "all eggs in one basket" kind of situation.
On the other hand, trade-offs: A federated solution makes it harder to audit, not central points of failure, but many points of (smaller) failures to consider.
Food for though, for this kind of service, something that would put users more at ease to consume (it think) would be if it belonged to a non-profit foundation, e.g., Eclipse Foundation, Linux Foundation, etc...
@johnspurlock-skymethod have you consider something like this for the long term vision of op3?

[johnspurlock-skymethod]

Absolutely, I keep the costs in a completely separate account for this reason (easier to hand over the keys to a future foundation).

It's still early for this kind of discussion, as OP3 is not yet even cost-covering month to month. Hopefully it'll be there soon, consider sponsoring at the 100/mo level! Perhaps next year if we 10x the number of shows using OP3 and it has demonstrated break-even for several months in a row, we can pick up this discussion. Otherwise now it would just slow things down.

Right now this service is something I want to exist in the world, and is an order of magnitude safer from a listener privacy point of view than any other similar service. Making sure this is no one's day job is one way to ensure no one is incentivized to loosen these principles.

You don't have all your eggs in one basket, a prefix is easy to add/remove, and you can have multiple.

OP3 is open source to audit, not to fork. One of the key benefits is using a consistent measure across shows, which goes out the window across multiple measurement systems.