From 268e7140b11b05116e3e5eec7b90fb64fe2f09f2 Mon Sep 17 00:00:00 2001 From: Raphael Kubo da Costa Date: Tue, 13 Dec 2022 17:00:02 +0100 Subject: [PATCH 1/4] Drop own powerful feature, require active local video source. Fixes #79. The idea is that, based on the research on potential attacks on the Ambient Light Sensor API, it is important to prompt users before allowing access to illuminance readouts. This was already mandated by the main Generic Sensors spec, as `Sensor.start()` runs the "Request sensor access" abstract operation. The challenge with the Ambient Light Sensor API is prompting users in a way that they understand what they are being prompted for; the assumption is that most users are not familiar with what an Ambient Light sensor is. We have chosen to solve this issue by dropping our "ambient-light-sensor" powerful feature name altogether and integrating with the Media Capture and Streams specification instead: we consider an Ambient Light Sensor to be a 1x1 camera and require there to be at least one local video source that is not muted or stopped in order for illuminance readouts to be provided. Per the Media Capture and Streams specification, this is only possible if script has called `MediaDevices.getUserMedia()` and granted the "camera" permission. This also means the User Agent has at least indicated to the user that a local video source has started being used. In other words, an Ambient Light Sensor only provides readings if a local video source (such as a camera) is currently active and being used in the same window as the AmbientLightSensor instance, and when all local video sources stop we also stop providing readouts and fire an "error" with a NotReadableError exception. The Use Cases section had to be shortened, as some items described there no longer make much sense when a camera is required. --- index.bs | 129 +++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 117 insertions(+), 12 deletions(-) diff --git a/index.bs b/index.bs index 8915688..0515c83 100644 --- a/index.bs +++ b/index.bs @@ -55,6 +55,13 @@ urlPrefix: https://w3c.github.io/sensors/; spec: GENERIC-SENSOR urlPrefix: https://tc39.es/ecma262/; spec: ECMA-262 type: abstract-op text: abs; url: eqn-abs +urlPrefix: https://w3c.github.io/mediacapture-main/; spec: MEDIACAPTURE-STREAMS + type: dfn + text: stopped; url: source-stopped + +
+spec:generic-sensor; type:attribute; text:[[state]]
+spec:mediacapture-streams; type:dfn; text:source
 
@@ -237,9 +244,13 @@ Works such as [[ALSPRIVACYANALYSIS]], [[PINSKIMMINGVIASENSOR]],
 [[STEALINGSENSITIVEDATA]], and [[VIDEORECOGNITIONAMBIENTLIGHT]] delve further
 into these issues.
 
-To mitigate these threats specific to Ambient Light Sensor, user agents must
-reduce accuracy of sensor readings. User agents may also limit
-maximum sampling frequency.
+To mitigate these threats specific to Ambient Light Sensor, user agents must:
+- [=Reduce accuracy=] of sensor readings. See [[#reduce-sensor-accuracy]].
+- Obtain user consent before providing sensor readings. This is done by
+  integrating with the [[MEDIACAPTURE-STREAMS]] specification. See
+  [[#local-video-source-requirement]].
+
+User agents may also limit maximum sampling frequency.
 
 These mitigation strategies complement the [=mitigation strategies|generic mitigations=]
 defined in the Generic Sensor API [[!GENERIC-SENSOR]].
@@ -283,6 +294,39 @@ experience. Chromium bug 1332536 and Chromium review 3666917 contain more
 information about this.
 
+Active local video source requirement {#local-video-source-requirement}
+-----
+
+Many of the attacks on Ambient Light sensors referenced above rely on being
+able to access illuminance readouts for a certain amount of time without a user
+being aware that the data is being read.
+
+[[STEALINGSENSITIVEDATA]] and [[ALSPRIVACYANALYSIS]] specifically recommend
+requesting user permission before allowing access to illuminance readouts as a
+privacy measure. On the other hand, it can be argued that conveying to users
+what an Ambient Light Sensor is so that they can make an informed choice to
+grant or deny access to it can be difficult.
+
+What this specification does instead is consider an Ambient Light Sensor to be
+a 1x1 grayscale camera, integrate with the [[MEDIACAPTURE-STREAMS]]
+specification and require there to be at least one local video [=source=] that
+is not [=muted=] or [=stopped=] in order for illuminance readouts to be
+provided. In other words, an Ambient Light Sensor only provides readings if a
+local video source (such as a camera) is currently active and being used in the
+same window as the {{AmbientLightSensor}} instance.
+
+Per the [[MEDIACAPTURE-STREAMS]] specification, this is only possible if script
+has called {{MediaDevices/getUserMedia()}} and granted the "camera" permission. This also means the User Agent has at least
+indicated to the user that a local video source has started being used as per
+[[MEDIACAPTURE-STREAMS#privacy-indicator-requirements]].
+
+The goal of this model is to treat an Ambient Light Sensor as potentially as
+invasive as an actual camera device and subject it to the same strict privacy
+requirements together with the Generic Sensor mitigations described in
+[[GENERIC-SENSOR#security-and-privacy]] and the other Ambient Light
+Sensor-specific measures described in this section.
+
 Model {#model}
 =====
 
@@ -292,12 +336,12 @@ subclass is the {{AmbientLightSensor}} class.
 The Ambient Light Sensor has a default sensor,
 which is the device's main light detector.
 
-The Ambient Light Sensor is a [=powerful feature=] that is identified by
-the [=powerful feature/name=] "ambient-light-sensor",
-which is also its associated [=sensor permission name=]. Its
-[=powerful feature/permission revocation algorithm=] is the result of calling
-the [=generic sensor permission revocation algorithm=] with
-"ambient-light-sensor".
+The Ambient Light Sensor's [=sensor permission names=] is an
+[=set/empty=] [=ordered set|set=].
+
+Note: See [[#local-video-source-requirement]]. This specification relies on the
+permission model specified in the [[MEDIACAPTURE-STREAMS]] specification
+instead.
 
 The Ambient Light Sensor is a [=policy-controlled feature=] identified by the string "ambient-light-sensor". Its [=default allowlist=] is `'self'`.
 
@@ -345,6 +389,35 @@ The {{AmbientLightSensor/illuminance}} getter steps are:
    reading=] with [=this=] and "illuminance" as arguments.
 1. Return |illuminance|.
 
+### Media Capture and Streams integration ### {#media-capture-and-streams-integration}
+
+As discussed in [[#local-video-source-requirement]], illuminance readouts are
+provided only if the same {{Window}} with an {{AmbientLightSensor}} object has
+at least one local video [=source=] that is not [=muted=] or [=stopped=].
+
+The [=ambient light pre-activation checks algorithm=] is invoked by
+{{Sensor/start()}} as specified in [[GENERIC-SENSOR]].
+
+Furthermore, whenever an item is added to the {{Window}}.\[[devicesLiveMap]]
+internal slot, or one of its items has its value changed, implementations MUST
+run the following steps:
+
+

+  1. Let |global| be the {{Window}} object of the affected \[[devicesLiveMap]]
+     internal slot.
+  1. Let |result| be the result of invoking [=check for active local video sources=] with |global|.
+  1. If |result| is true, return.
+  1. For each {{AmbientLightSensor}} object |sensor| whose [=relevant global
+     object=] is |global|:
+      1. If |sensor|.{{[[state]]}} is "idle", then [=continue=].
+      1. Invoke [=deactivate a sensor object=] with |sensor|.
+      1.  Let |e| be the result of [=created|creating=]
+          a "{{NotReadableError}}" {{DOMException}}.
+      1.  [=Queue a global task=] on the [=sensor task source=] with |global| to
+          run [=notify error=] with |sensor| and |e| as
+          arguments.
+

+
 Abstract Operations {#abstract-operations}
 ===================
 
@@ -418,13 +491,42 @@ quantization algorithm=]:
   1. Return |quantizedReading|.
 
 
+
Ambient light pre-activation checks algorithm

+
+The [=Ambient Light Sensor=] [=sensor type=] defines the following
+[=pre-activation checks algorithm=]:
+
+

+  : input
+  :: |sensor|, an {{AmbientLightSensor}} object
+  : output
+  :: A [=boolean=] indicating whether the checks have passed and sensor activation may proceed.
+
+  1. Let |global| be |sensor|'s [=relevant global object=].
+  1. Return the result of invoking [=check for active local video sources=] with |global|.
+

+
+
Check for active local video source

+
+

+  : input
+  :: |global|, an {{Window}} object
+  : output
+  :: A [=boolean=] indicating whether there are active local video sources.
+
+  1. If |global| does not have a \[[mediaStreamTrackSources]] internal slot, return false.
+  1. For each |source| in |global|'s \[[mediaStreamTrackSources]] internal slot:
+    1. If |source| is not a video input device, then [=continue=].
+    1. If |source| is not [=stopped=] or [=muted=], then return true.
+  1. Return false.
+

+
 Automation {#automation}
 ==========
 This section extends the [=automation=] section defined in the Generic Sensor API [[GENERIC-SENSOR]]
 to provide mocking information about the ambient light levels for the purposes of testing a user agent's
 implementation of [=Ambient Light Sensor=].
 
-
 
Mock Sensor Type

 
 The {{AmbientLightSensor}} class has an associated [=mock sensor type=] which is
@@ -440,8 +542,6 @@ dictionary is defined as follows:
 Use Cases and Requirements {#usecases-requirements}
 =========
 
-- A Web application provides input for a smart home system to control lighting.
-- A Web application checks whether light level at work space is sufficient.
 - A Web application calculates settings for a camera with manual controls (aperture, shutter speed, ISO).
 - A Web application checks the current light level to determine whether a
   camera stream will contain data that is accurate enough for its purposes
@@ -451,6 +551,11 @@ While some of the use cases may benefit from obtaining precise ambient light mea
 cases that convert ambient light level fluctuations to user input events would benefit from
 higher [=sampling frequency|sampling frequencies=].
 
+Note: A previous version of this specification did not require an active camera and did
+not integrate with the [[MEDIACAPTURE-STREAMS]] specification. It allowed for a wider
+range of use cases, such as providing input smart home systems to control lighting or
+checking whether the light level at a work space is sufficient.
+
 Acknowledgements {#acknowledgements}
 ================
 

From 9fc0b6d64633ce8008118b07192489e942bc9421 Mon Sep 17 00:00:00 2001
From: Raphael Kubo da Costa 
Date: Wed, 14 Dec 2022 14:56:14 +0100
Subject: [PATCH 2/4] Update examples to show Media Capture integration.

The last example was removed because it showed a use case that was deleted
in the previous commit, as it no longer seems very relevant with the new
media capture requirements.
---
 index.bs | 61 ++++++++++----------------------------------------------
 1 file changed, 10 insertions(+), 51 deletions(-)

diff --git a/index.bs b/index.bs
index 0515c83..56c3b0b 100644
--- a/index.bs
+++ b/index.bs
@@ -138,32 +138,27 @@ Examples {#examples}
 ========
 
 

-    In this simple example, ambient light sensor is created with
+    In this simple example, an {{AmbientLightSensor}} instance is created with
     default configuration. Whenever a new [=sensor readings|reading=] is available,
     it is printed to the console.
 
     
-    const sensor = new AmbientLightSensor();
-    sensor.onreading = () => console.log(sensor.illuminance);
-    sensor.onerror = event => console.log(event.error.name, event.error.message);
-    sensor.start();
+    navigator.mediaDevices.getUserMedia({video: true}).then(() => {
+      const sensor = new AmbientLightSensor();
+      sensor.onreading = () => console.log(sensor.illuminance);
+      sensor.onerror = event => console.log(event.error.name, event.error.message);
+      sensor.start();
+    });
     

 

 
 

     In this example, the exposure value (EV) at ISO 100 is calculated from
-    the ambient light [=sensor readings=]. Initially, we check that the user
-    agent has permissions to access ambient light [=sensor readings=]. Then,
-    the {{AmbientLightSensor/illuminance!!attribute}} value is converted to the
-    closest exposure value.
+    the ambient light [=sensor readings=]. The {{AmbientLightSensor/illuminance}}
+    value is converted to the closest exposure value.
 
     
-    navigator.permissions.query({ name: 'ambient-light-sensor' }).then(result => {
-        if (result.state === 'denied') {
-            console.log('Permission to use ambient light sensor is denied.');
-            return;
-        }
-
+    navigator.mediaDevices.getUserMedia({ video: true }).then(() => {
         const als = new AmbientLightSensor({frequency: 20});
         als.addEventListener('activate', () => console.log('Ready to measure EV.'));
         als.addEventListener('error', event => console.log(\`Error: ${event.error.name}\`));
@@ -182,42 +177,6 @@ Examples {#examples}
     

 

 
-

-    This example demonstrates how ambient light [=sensor readings=] can be mapped
-    to recommended workplace light levels.
-
-    
-    const als = new AmbientLightSensor();
-
-    als.onreading = () => {
-        let str = luxToWorkplaceLevel(als.illuminance);
-        if (str) {
-            console.log(\`Light level is suitable for: ${str}.\`);
-        }
-    };
-
-    als.start();
-
-    function luxToWorkplaceLevel(lux) {
-        if (lux > 20 && lux < 100) {
-            return 'public areas, short visits';
-        } else if (lux > 100 && lux < 150) {
-            return 'occasionally performed visual tasks';
-        } else if (lux > 150 && lux < 250) {
-            return 'easy office work, classes, homes, theaters';
-        } else if (lux > 250 && lux < 500) {
-            return 'normal office work, groceries, laboratories';
-        } else if (lux > 500 && lux < 1000) {
-            return 'mechanical workshops, drawing, supermarkets';
-        } else if (lux > 1000 && lux < 5000) {
-            return 'detailed drawing work, visual tasks of low contrast';
-        }
-
-        return;
-    }
-    

-

-
 Security and Privacy Considerations {#security-and-privacy}
 ===================================
 

From 084c6b9dfcb13436b266bbc307eb9cfab76b40cf Mon Sep 17 00:00:00 2001
From: Raphael Kubo da Costa 
Date: Fri, 16 Dec 2022 17:59:27 +0100
Subject: [PATCH 3/4] Reword video source requirement paragraph.

Co-authored-by: Reilly Grant 
---
 index.bs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/index.bs b/index.bs
index 56c3b0b..7440ed4 100644
--- a/index.bs
+++ b/index.bs
@@ -262,9 +262,9 @@ being aware that the data is being read.
 
 [[STEALINGSENSITIVEDATA]] and [[ALSPRIVACYANALYSIS]] specifically recommend
 requesting user permission before allowing access to illuminance readouts as a
-privacy measure. On the other hand, it can be argued that conveying to users
+privacy measure. On the other hand, it can be difficult to convey to users
 what an Ambient Light Sensor is so that they can make an informed choice to
-grant or deny access to it can be difficult.
+grant or deny access to it.
 
 What this specification does instead is consider an Ambient Light Sensor to be
 a 1x1 grayscale camera, integrate with the [[MEDIACAPTURE-STREAMS]]

From 5f7843473d2c8a84304941d27291d76c5bea10a2 Mon Sep 17 00:00:00 2001
From: Raphael Kubo da Costa 
Date: Fri, 16 Dec 2022 19:14:08 +0100
Subject: [PATCH 4/4] Use "local camera source" instead of "local video
 source".

This avoids ambiguity, including the fact that a screen capture session
could qualify as a "local video source".

We really want to tie this API's permission model to the presence of an
active camera specifically.
---
 index.bs | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/index.bs b/index.bs
index 7440ed4..26d0eb7 100644
--- a/index.bs
+++ b/index.bs
@@ -207,7 +207,7 @@ To mitigate these threats specific to Ambient Light Sensor, user agents must:
 - [=Reduce accuracy=] of sensor readings. See [[#reduce-sensor-accuracy]].
 - Obtain user consent before providing sensor readings. This is done by
   integrating with the [[MEDIACAPTURE-STREAMS]] specification. See
-  [[#local-video-source-requirement]].
+  [[#local-camera-source-requirement]].
 
 User agents may also limit maximum sampling frequency.
 
@@ -253,7 +253,7 @@ experience. Chromium bug 1332536 and Chromium review 3666917 contain more
 information about this.
 
-Active local video source requirement {#local-video-source-requirement}
+Active local camera source requirement {#local-camera-source-requirement}
 -----
 
 Many of the attacks on Ambient Light sensors referenced above rely on being
@@ -268,16 +268,16 @@ grant or deny access to it.
 
 What this specification does instead is consider an Ambient Light Sensor to be
 a 1x1 grayscale camera, integrate with the [[MEDIACAPTURE-STREAMS]]
-specification and require there to be at least one local video [=source=] that
+specification and require there to be at least one local camera [=source=] that
 is not [=muted=] or [=stopped=] in order for illuminance readouts to be
 provided. In other words, an Ambient Light Sensor only provides readings if a
-local video source (such as a camera) is currently active and being used in the
-same window as the {{AmbientLightSensor}} instance.
+local video camera is currently active and being used in the same window as the
+{{AmbientLightSensor}} instance.
 
 Per the [[MEDIACAPTURE-STREAMS]] specification, this is only possible if script
 has called {{MediaDevices/getUserMedia()}} and granted the "camera" permission. This also means the User Agent has at least
-indicated to the user that a local video source has started being used as per
+indicated to the user that a local camera source has started being used as per
 [[MEDIACAPTURE-STREAMS#privacy-indicator-requirements]].
 
 The goal of this model is to treat an Ambient Light Sensor as potentially as
@@ -298,7 +298,7 @@ which is the device's main light detector.
 The Ambient Light Sensor's [=sensor permission names=] is an
 [=set/empty=] [=ordered set|set=].
 
-Note: See [[#local-video-source-requirement]]. This specification relies on the
+Note: See [[#local-camera-source-requirement]]. This specification relies on the
 permission model specified in the [[MEDIACAPTURE-STREAMS]] specification
 instead.
 
@@ -350,9 +350,9 @@ The {{AmbientLightSensor/illuminance}} getter steps are:
 
 ### Media Capture and Streams integration ### {#media-capture-and-streams-integration}
 
-As discussed in [[#local-video-source-requirement]], illuminance readouts are
+As discussed in [[#local-camera-source-requirement]], illuminance readouts are
 provided only if the same {{Window}} with an {{AmbientLightSensor}} object has
-at least one local video [=source=] that is not [=muted=] or [=stopped=].
+at least one local camera [=source=] that is not [=muted=] or [=stopped=].
 
 The [=ambient light pre-activation checks algorithm=] is invoked by
 {{Sensor/start()}} as specified in [[GENERIC-SENSOR]].
@@ -364,7 +364,7 @@ run the following steps:
 

   1. Let |global| be the {{Window}} object of the affected \[[devicesLiveMap]]
      internal slot.
-  1. Let |result| be the result of invoking [=check for active local video sources=] with |global|.
+  1. Let |result| be the result of invoking [=check for active local camera sources=] with |global|.
   1. If |result| is true, return.
   1. For each {{AmbientLightSensor}} object |sensor| whose [=relevant global
      object=] is |global|:
@@ -455,23 +455,23 @@ quantization algorithm=]:
 The [=Ambient Light Sensor=] [=sensor type=] defines the following
 [=pre-activation checks algorithm=]:
 
-

+

   : input
   :: |sensor|, an {{AmbientLightSensor}} object
   : output
   :: A [=boolean=] indicating whether the checks have passed and sensor activation may proceed.
 
   1. Let |global| be |sensor|'s [=relevant global object=].
-  1. Return the result of invoking [=check for active local video sources=] with |global|.
+  1. Return the result of invoking [=check for active local camera sources=] with |global|.
 

 
-
Check for active local video source

+
Check for active local camera source

 
-

+

   : input
   :: |global|, an {{Window}} object
   : output
-  :: A [=boolean=] indicating whether there are active local video sources.
+  :: A [=boolean=] indicating whether there are active local camera sources.
 
   1. If |global| does not have a \[[mediaStreamTrackSources]] internal slot, return false.
   1. For each |source| in |global|'s \[[mediaStreamTrackSources]] internal slot: