Skip to content

Conversation

@kartben
Copy link
Contributor

@kartben kartben commented Feb 10, 2023

Adding CSS and JS code for making sure users opt-in to cookies related to Google Analytics. Fixes issue #51604.

Signed-off-by: Benjamin Cabé [email protected]


Fixes #51604

Adding CSS and JS code for making sure users opt-in to cookies related
to Google Analytics.
Fixes issue zephyrproject-rtos#51604.

Signed-off-by: Benjamin Cabé <[email protected]>
@stephanosio stephanosio added bug The issue is a bug, or the PR is fixing a bug Release Blocker Use this label for justified release blockers labels Feb 10, 2023
@stephanosio stephanosio added this to the v3.3.0 milestone Feb 10, 2023
Copy link
Member

@stephanosio stephanosio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The placement of the "cookie preferences" button seems questionable.

image

Also, some concerns/questions:

  • I am not seeing the "GDPR banner" being displayed on the first visit. Is this solution GDPR-compliant? (IIUC, displaying a banner listing cookie preference options is the norm, if not required for GDPR compliance?)
  • We have "Marketing" and "Personalisation" cookie choices. Does this really make sense for an open source project documentation website?
    image

@kartben
Copy link
Contributor Author

kartben commented Feb 10, 2023

The placement of the "cookie preferences" button seems questionable.

Fair point. I can look into tweaking the CSS.

Also, some concerns/questions:

  • I am not seeing the "GDPR banner" being displayed on the first visit. Is this solution GDPR-compliant? (IIUC, displaying a banner listing cookie preference options is the norm, if not required for GDPR compliance?)

Did you try in "incognito" mode? The cookie consent works across all LF websites, so you might have given consent elsewhere?

  • We have "Marketing" and "Personalisation" cookie choices. Does this really make sense for an open source project documentation website?

I agree this can be confusing. We could use a custom cookie consent script as opposed to the generic LF one to address that.

@stephanosio
Copy link
Member

Did you try in "incognito" mode? The cookie consent works across all LF websites, so you might have given consent elsewhere?

Ah, I did not realise this was across all LF websites. I can confirm that the banner is displayed in incognito mode.

There is a text colour issue with the dark theme though:
image

@henrikbrixandersen
Copy link
Member

I would like someone to justify having a 3rd party tracker built into our technical documentation. What is the purpose and what is the gain for the project?

@nordicjm
Copy link
Contributor

I would like someone to justify having a 3rd party tracker built into our technical documentation. What is the purpose and what is the gain for the project?

Agreed, was this ever agreed by the TSC?

Copy link
Contributor

@nordicjm nordicjm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving a -1 for pending question that needs answering.

@kartben
Copy link
Contributor Author

kartben commented Feb 13, 2023

I would like someone to justify having a 3rd party tracker built into our technical documentation. What is the purpose and what is the gain for the project?

I believe the statement made here still applies #16666 (comment).
I understand it might be perceived as vague, and it's unclear (to me at least) if a lot of people have actually been paying attention to the collected insights until today. That being said, there is in my opinion lots of benefits in looking at the data, both as a "snapshot", and also to identify trends in usage and adoption. In no particular order, things I plan to actively keeping an eye on, and sharing with TSC and Marketing committee:

  • top search queries, to help identify most/least used features, boards/platforms, and typical questions folks need answering when using the documentation
  • identify overly complex navigation paths, i.e people who systematically go through X pages to reach a specific page
  • monitor incoming links to identify cool open source or commercial projects using Zephyr, and help identify new community ambassadors
  • help with identifying accessibility issues (note that there is probably a lot that can be done with "just" static HTML analysis tools, to be complete)

FWIW, I would be happy to look into using another web analytics platform post 3.3, potentially a self-hosted one, if Google Analytics is part of what people are having an issue with.

@nordicjm
Copy link
Contributor

top search queries, to help identify most/least used features, boards/platforms, and typical questions folks need answering when using the documentation

I mean, let's be honest, the search on the documentation is probably the worst search system I've had the misfortune of using, it returns nonsense results, completely unrelated results - and if it does actually return what I'm looking for, sometimes it's way down the page. I usually find it faster to go and clone the repo and do a grep than trying to find the right term that will find what I'm looking for unless I put in the exact text as it appears - so tell me, how is an analytics engine going to explain this (quite common) frustration of use to you?

monitor incoming links to identify cool open source or commercial projects using Zephyr, and help identify new community ambassadors

Why would you be checking what links people take to the documentation?

identify overly complex navigation paths, i.e people who systematically go through X pages to reach a specific page
help with identifying accessibility issues (note that there is probably a lot that can be done with "just" static HTML analysis tools, to be complete)

So not bothering to do the simple thing of just asking people what they think and instead trying to work out their opinion of the documentation through analytics?

Well that's sold me.

@kartben
Copy link
Contributor Author

kartben commented Feb 15, 2023

I mean, let's be honest, the search on the documentation is probably the worst search system I've had the misfortune of using, it returns nonsense results, completely unrelated results [...]

Agreed, and it's my intention to look at tweaking the search "scorer" to improve the quality of the search results.

[...] so tell me, how is an analytics engine going to explain this (quite common) frustration of use to you?

Even if it's broken, people are using it. And looking at the search terms (again, whether the search ends up being successful or not due to poorly ranked results) is IMO valuable, to me at least.
A few things that I noticed by having a quick look. These might be common knowledge for folks but these are I think the kind of insights that are interesting to track, including to see how they evolve over time.

  • Having "esp32" as a top term when it comes to SoCs is an interesting indicator that there seems to be lots of people interested in running Zephyr there
  • Seeing "lvgl" high in the list is also interesting IMO, as this means maybe more people than what we tend to think are interested in running embedded GUIs on Zephyr.

Why would you be checking what links people take to the documentation?

I think I already answered in my previous comment. Examples would be finding active open source projects that "build on" Zephyr, training material being put together by folks, etc. I am not saying that this will ever replace doing everything we (I) can to make sure people know where to find us and let us know about those things, but in my experience complementing this with a look at our inbound traffic is usually very helpful in eventually onboarding people who hadn't realized otherwise they could get more engaged.

So not bothering to do the simple thing of just asking people what they think and instead trying to work out their opinion of the documentation through analytics?

Define "people" :) For better or worse, there will always be downstream adopters of Zephyr that just "consume" whatever is being put out there, and provide hardly any feedback. I definitely plan on trying to find ways to encourage people to participate more, but in the meantime "asking people" might only get us the feedback from people already engaged, or at least already well past their "hello world" and their first time landing on the docs homepage.

@henrikbrixandersen
Copy link
Member

Even if it's broken, people are using it. And looking at the search terms (again, whether the search ends up being successful or not due to poorly ranked results) is IMO valuable, to me at least. A few things that I noticed by having a quick look. These might be common knowledge for folks but these are I think the kind of insights that are interesting to track, including to see how they evolve over time.

  • Having "esp32" as a top term when it comes to SoCs is an interesting indicator that there seems to be lots of people interested in running Zephyr there
  • Seeing "lvgl" high in the list is also interesting IMO, as this means maybe more people than what we tend to think are interested in running embedded GUIs on Zephyr.

Those are both interesting data points, but is a full 3rd party analytics package really needed to find this? Couldn't this be found simply by offline analyses of the httpd logs?

I think I already answered in my previous comment. Examples would be finding active open source projects that "build on" Zephyr, training material being put together by folks, etc. I am not saying that this will ever replace doing everything we (I) can to make sure people know where to find us and let us know about those things, but in my experience complementing this with a look at our inbound traffic is usually very helpful in eventually onboarding people who hadn't realized otherwise they could get more engaged.

Again, couldn't this be found by static analyses of the httpd server logs looking for the referrer?

@nordicjm
Copy link
Contributor

nordicjm commented Feb 15, 2023

Even if it's broken, people are using it. And looking at the search terms (again, whether the search ends up being successful or not due to poorly ranked results) is IMO valuable, to me at least.
A few things that I noticed by having a quick look. These might be common knowledge for folks but these are I think the kind of insights that are interesting to track, including to see how they evolve over time.

Having "esp32" as a top term when it comes to SoCs is an interesting indicator that there seems to be lots of people interested in running Zephyr there

Seeing "lvgl" high in the list is also interesting IMO, as this means maybe more people than what we tend to think are interested in running embedded GUIs on Zephyr.

Except you don't know how reflective these are of real results, anyone blocking these analytics locally or by their network (and given that Europe is the area with the greatest restrictions, we can assume most people blocking them would be in that area) search terms would not appear, so you're seeing what could actually be a completely false view of the data.

Define "people" :) For better or worse, there will always be downstream adopters of Zephyr that just "consume" whatever is being put out there, and provide hardly any feedback. I definitely plan on trying to find ways to encourage people to participate more, but in the meantime "asking people" might only get us the feedback from people already engaged, or at least already well past their "hello world" and their first time landing on the docs homepage.

Well let's start with this question: when and where have people - i.e. anyone - been asked for their feedback on the documentation?

And in regards to referrer tracking, firefox at least has blocked that for a number of years now, so what you would get would be nothing or worthless, would assume other browsers have done similar.

@stephanosio stephanosio modified the milestones: v3.3.0, future Feb 16, 2023
@stephanosio
Copy link
Member

No longer relevant after #54875

@kartben kartben deleted the fix_51604 branch October 24, 2023 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: Documentation Infrastructure bug The issue is a bug, or the PR is fixing a bug Release Blocker Use this label for justified release blockers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

doc: is the documentation GDPR compliant since it uses Google Analytics without prompting the user about tracking?

6 participants