Search analytics not working for files and external links

[blueo]

Module version

1.x

Problem statement

When linking to a file with analytics enabled, the analytics is not processed as the link goes directly to the file which bypasses analytics

[chrispenny]

This would also affect search Documents that link off to external sources. This might be a tricky one to solve with backend logic, as we currently require the link to pass through Silverstripe (so that the tracking middleware is triggered).

Internal route instead of middleware?

One option might be to no longer use a middleware, and instead force all links to pass through an internal route before being redirected to their true location. I'm not sure I'm a huge fan of that though...

The rationale behind using the middleware (rather than some internal route that then redirects to the real link) was so that:

• You can hover over a search result and see the (real) link for that search Document
• You can right click > copy link address, and it be the correct link - not a link to some internal route with a redirect
• Using a middleware instead of a route means that errors in the middleware can be logged, but ultimately ignored by the end user, and they still end up in the correct place because there was no redirect logic required.

Both an internal route and middleware?

Maybe another option could be to conditionally pass through an internal route vs the middle, but I see difficulties there as well:

• Discoverer itself doesn't know the difference between any of its search Documents
• A developer might add some sort of "class" field into the search Document, but we'd have no way to know that they will, or that the class name would even be File
• We don't even know (programmatically) what field the link lives in
• I think it would be very difficult for us to programatically determine which search Documents need to route through which mechanisms

JavaScript?

The most consistent solution might actually be to use JavaScript to track clicks. Every search Document can be treated in exactly the same way, and it wouldn't matter where the link pointed.

We can provide an internal API for "click tracking", so that the JS is always calling the same place (regardless of which integration module you are using).

[blueo]

yeah internal route might be the best option here for ensuring tracking. Javascript can be a bit temperamental and we'd perhaps have a challenge of cross domain clicks etc. Most places (google etc) appear to use a route like this - yeah copy pasting links is a bit annoying (looking at you google calendar) but at least it would still resolve to the same place.

[blueo]

an example of a google calendar link https://www.google.com/url?q=https://silverstripe.zoom.us/j/85441916388?pwd%3DZlB4TUQwNVR2YkJDRnpwbUNDSDA3UT09&sa=D&source=calendar&ust=1726549091244715&usg=AOvVaw2Q5wY5LieReRdfGOurLd3I - its just adding the link as a query param - this seems a middle ground - as you can still get the 'real' link out if you need to and visiting it will still take you to the 'final' link. Its not as nice as going directly there but it will allow tracking it. @chrispenny thoughts?

[chrispenny]

I'm probably keen to have a bit of a chat through the options with Jackson as well, to see what he thinks, but yea, I'm open to this option. It's going to mean a refactor for the module either way.

[blueo]

This is a good write up on it - we could make an API endpoint for this - it would decouple tracking from the users' site potentially making it more reliable. The module could offer an OS version too if they want to host it but there are a few downsides to having the tracking within the same application.

https://postmarkapp.com/blog/how-we-built-link-tracking-to-be-reliable-fast-and-secure

Its quite painful to find info on this kind of thing as search results are full of SEO products 😢 but there's generally a few things to consider: reliablility, security and user experience

[chrispenny]

Good read. Thanks for that.

I think Discoverer will need to provide a self-hosted version, but we can make the tracking endpoint (and maybe even the data?) configurable.