You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/english/algoprudence/cases/aa202301_bert-based-disinformation-classifier.md
+6-8Lines changed: 6 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,12 +38,12 @@ A visual presentation of this case study can be found in this [slide deck](http
38
38
39
39
#### Normative advice commission
40
40
41
-
* Anne Meuwese, Professor in Public Law & AI at Leiden University
42
-
* Hinda Haned, Professor in Responsible Data Science at University of Amsterdam
43
-
* Raphaële Xenidis, Associate Professor in EU law at Sciences Po Paris
44
-
* Aileen Nielsen, Fellow Law\&Tech at ETH Zürich
45
-
* Carlos Hernández-Echevarría, Assistant Director and Head of Public Policy at the anti-disinformation nonprofit fact-checker [Maldita.es](https://maldita.es/maldita-es-journalism-to-not-be-fooled/)
46
-
* Ellen Judson, Head of CASM and Sophia Knight, Researcher, CASM at Britain’s leading cross-party think tank [Demos](https://demos.co.uk/)
41
+
- Anne Meuwese, Professor in Public Law & AI at Leiden University
42
+
- Hinda Haned, Professor in Responsible Data Science at University of Amsterdam
43
+
- Raphaële Xenidis, Associate Professor in EU law at Sciences Po Paris
44
+
- Aileen Nielsen, Fellow Law\&Tech at ETH Zürich
45
+
- Carlos Hernández-Echevarría, Assistant Director and Head of Public Policy at the anti-disinformation nonprofit fact-checker [Maldita.es](https://maldita.es/maldita-es-journalism-to-not-be-fooled/)
46
+
- Ellen Judson, Head of CASM and Sophia Knight, Researcher, CASM at Britain’s leading cross-party think tank [Demos](https://demos.co.uk/)
47
47
48
48
{{< tab_content_close >}}
49
49
@@ -70,5 +70,3 @@ Our [unsupervised bias detection tool](/technical-tools/bdt/) and this case st
Your reaction will be sent to the team maintaining algoprudence. A team will
12
-
review your response and, if it complies with the guidelines, it will be
13
-
placed in the Discussion & debate section above.
14
-
button_text: Submit
15
-
backend_link: 'https://formspree.io/f/xyyrjyzr'
16
-
id: case-reaction
17
-
questions:
18
-
- label: |
19
-
Name
20
-
id: name
21
-
required: true
22
-
type: text
23
-
- label: |
24
-
Affiliated organization
25
-
id: affiliated-organization
26
-
type: text
27
-
- label: |
28
-
Reaction
29
-
id: reaction
30
-
required: true
31
-
type: textarea
32
-
- label: |
33
-
Contact details
34
-
id: contact-details
35
-
required: true
36
-
type: text
37
-
placeholder: Mail address
8
+
dynamic_form_engine:
9
+
- title: React to this normative judgement
10
+
content: >-
11
+
Your reaction will be sent to the team maintaining algoprudence. A team will
12
+
review your response and, if it complies with the guidelines, it will be
13
+
placed in the Discussion & debate section above.
14
+
id: case-reaction
15
+
icon: fas fa-align-justify
16
+
section:
17
+
- questions:
18
+
- title: |
19
+
Name
20
+
identifier: name
21
+
required: true
22
+
type: text
23
+
- title: |
24
+
Affiliated organization
25
+
identifier: affiliated-organization
26
+
type: text
27
+
- title: |
28
+
Reaction
29
+
identifier: reaction
30
+
required: true
31
+
type: textarea
32
+
- title: |
33
+
Contact details
34
+
identifier: contact-details
35
+
required: true
36
+
type: text
37
+
placeholder: Mail address
38
+
complete_form_options:
39
+
type: submit
40
+
button_text: Submit
41
+
backend_link: "https://formspree.io/f/xyyrjyzr"
38
42
---
39
43
40
44
{{< tab_header width="4" tab1_id="description" tab1_title="Description of algoprudence" tab2_id="actions" tab2_title="Actions following algoprudence" tab3_id="discussion" tab3_title="Discussion & debate" default_tab="description" >}}
@@ -47,15 +51,15 @@ ALGO:AA:2023:02
47
51
48
52
#### Key takeaways normative advice commission
49
53
50
-
* <spanstyle="color:#005aa7; font-weight:600;">Algorithmic profiling is possible under strict conditions</span>\
54
+
- <spanstyle="color:#005aa7; font-weight:600;">Algorithmic profiling is possible under strict conditions</span>\
51
55
The use of algorithmic profiling to re-examine whether social welfare benefits have been duly granted, is acceptable if applied responsibly.
52
-
* <spanstyle="color:#005aa7; font-weight:600;">Profiling must not equate suspicion</span>\
56
+
- <spanstyle="color:#005aa7; font-weight:600;">Profiling must not equate suspicion</span>\
53
57
Re-examination needs to be based more on service and less on distrust.
54
-
* <spanstyle="color:#005aa7; font-weight:600;">Diversity in selection methods</span>\
58
+
- <spanstyle="color:#005aa7; font-weight:600;">Diversity in selection methods</span>\
55
59
To avoid tunnel vision and negative feedback loops, algorithmic profiling ought to be combined with expert-driven profiling and random sampling.
56
-
* <spanstyle="color:#005aa7; font-weight:600;">Well-considered use of profiling criteria</span>\
60
+
- <spanstyle="color:#005aa7; font-weight:600;">Well-considered use of profiling criteria</span>\
57
61
Caring to avoid (proxy) discrimination and other undesirable forms of differentiation, the normative advice commission assessed variables individually on their eligibility for profiling (see Infographic).
58
-
* <spanstyle="color:#005aa7; font-weight:600;">Explainability requirements for machine learning</span>\
62
+
- <spanstyle="color:#005aa7; font-weight:600;">Explainability requirements for machine learning</span>\
59
63
It is necessary that the sampling of residents can be explained throughout the entire decision-making process. Complex training methods for variable selection, such as the xgboost algorithm discussed in this case study, are considered too complex to meet explainability requirements.
60
64
61
65
#### Infographic
@@ -82,11 +86,11 @@ The advice report (ALGO:AA:2023:02:A) has been presented to the Dutch Minister o
82
86
83
87
#### Normative advice commission
84
88
85
-
* Abderrahman El Aazani, Researcher at the Ombudsman Rotterdam-Rijnmond
86
-
* Francien Dechesne, Associate Professor Law and Digital Technologies, Leiden University
87
-
* Maarten van Asten, Alderman Finance, Digitalization, Sports and Events Municipality of Tilburg
88
-
* Munish Ramlal, Ombudsman Metropole region Amsterdam
89
-
* Oskar Gstrein, Assistant Professor Governance and Innovation, University of Groningen
89
+
- Abderrahman El Aazani, Researcher at the Ombudsman Rotterdam-Rijnmond
90
+
- Francien Dechesne, Associate Professor Law and Digital Technologies, Leiden University
91
+
- Maarten van Asten, Alderman Finance, Digitalization, Sports and Events Municipality of Tilburg
92
+
- Munish Ramlal, Ombudsman Metropole region Amsterdam
93
+
- Oskar Gstrein, Assistant Professor Governance and Innovation, University of Groningen
90
94
91
95
{{< tab_content_close >}}
92
96
@@ -98,11 +102,11 @@ The advice report (ALGO:AA:2023:02:A) has been presented to the Dutch Minister o
98
102
99
103
##### Description
100
104
101
-
Council members submitted <ahref="https://amsterdam.raadsinformatie.nl/document/13573898/1/236+sv+Aslami%2C+IJmker+en+Garmy+inzake+toegepaste+profileringscriteria+gemeentelijke+algoritmes"target="_blank">questions</a> whether the machine learning (ML)-driven risk profiling algorithm currently tested by the City of Amsterdam satisfies the requirements as set out in ALGO:AA:2023:02:A, including:
105
+
Council members submitted <ahref="https://amsterdam.raadsinformatie.nl/document/13573898/1/236+sv+Aslami%2C+IJmker+en+Garmy+inzake+toegepaste+profileringscriteria+gemeentelijke+algoritmes"target="_blank">questions</a> whether the machine learning (ML)-driven risk profiling algorithm currently tested by the City of Amsterdam satisfies the requirements as set out in ALGO:AA:2023:02:A, including:
102
106
103
-
* (in)eligible selection criteria fed to the ML model
104
-
* explainability requirements for the used explainable boosting algorithm
105
-
* implications of the AIAct for this particular form of risk profiling.
107
+
- (in)eligible selection criteria fed to the ML model
108
+
- explainability requirements for the used explainable boosting algorithm
109
+
- implications of the AIAct for this particular form of risk profiling.
Copy file name to clipboardExpand all lines: content/english/algoprudence/cases/aa202401_preventing-prejudice.md
+37-33Lines changed: 37 additions & 33 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,36 +6,40 @@ subtitle: >
6
6
Technical audit (TA:AA:2024:01) of risk profiling algorithm used by Dutch Executive Agency of Education
7
7
control process
8
8
image: /images/algoprudence/AA202401/Cover_EN.png
9
-
form1:
10
-
title: React to this technical audit
11
-
content: >-
12
-
Your reaction will be sent to the auditing team. The team will review your
13
-
response and, if it complies with Algorithm Audit's guidelines, the reaction
14
-
will be placed in the Discussion & debate section above.
15
-
button_text: Submit
16
-
backend_link: 'https://formspree.io/f/xyyrjyzr'
17
-
id: case-reaction
18
-
questions:
19
-
- label: |
20
-
Name
21
-
id: name
22
-
required: true
23
-
type: text
24
-
- label: |
25
-
Affiliated organization
26
-
id: affiliated-organization
27
-
type: text
28
-
- label: |
29
-
Reaction
30
-
id: reaction
31
-
required: true
32
-
type: textarea
33
-
- label: |
34
-
Contact details
35
-
id: contact-details
36
-
required: true
37
-
type: email
38
-
placeholder: Mail address
9
+
dynamic_form_engine:
10
+
- title: React to this technical audit
11
+
content: >-
12
+
Your reaction will be sent to the auditing team. The team will review your
13
+
response and, if it complies with Algorithm Audit's guidelines, the reaction
14
+
will be placed in the Discussion & debate section above.
15
+
id: case-reaction
16
+
icon: fas fa-align-justify
17
+
section:
18
+
- questions:
19
+
- title: |
20
+
Name
21
+
identifier: name
22
+
required: true
23
+
type: text
24
+
- title: |
25
+
Affiliated organization
26
+
identifier: affiliated-organization
27
+
type: text
28
+
- title: |
29
+
Reaction
30
+
identifier: reaction
31
+
required: true
32
+
type: textarea
33
+
- title: |
34
+
Contact details
35
+
identifier: contact-details
36
+
required: true
37
+
type: email
38
+
placeholder: Mail address
39
+
complete_form_options:
40
+
type: submit
41
+
button_text: Submit
42
+
backend_link: "https://formspree.io/f/xyyrjyzr"
39
43
---
40
44
41
45
{{< tab_header width="4" tab1_id="description" tab1_title="Description of technical audit" tab2_id="actions" tab2_title="Actions following technical audit" tab3_id="discussion" tab3_title="Discussion & debate" default_tab="description" >}}
@@ -48,7 +52,7 @@ TA:AA:2024:01
48
52
49
53
#### Summary
50
54
51
-
In the period 2012-2022, students who lived close to their parent(s) have been selected significantly more often by Dutch public sector organisation DUO than other students. The algorithm used to support the selection procedure performed as expected. The combination of the algorithm-driven risk scoring and manual selection for the contorl process resulted in a significant overrepresentation of certain groups. Selected students were visited at home to verify whether they were not misusing college allowances. This is the main conclusion of the audit conducted by the Algorithm Audit Foundation on behalf of DUO. DUO's control process came under scrutiny in 2023 following <ahref="https://nos.nl/op3/video/2479701-zo-checkt-duo-of-jij-fraudeert-en-dat-systeem-rammelt"target="_blank">news items</a> from Investico and NOS, which stated that students with a migration background were more often accused of abuse than other students.
55
+
In the period 2012-2022, students who lived close to their parent(s) have been selected significantly more often by Dutch public sector organisation DUO than other students. The algorithm used to support the selection procedure performed as expected. The combination of the algorithm-driven risk scoring and manual selection for the contorl process resulted in a significant overrepresentation of certain groups. Selected students were visited at home to verify whether they were not misusing college allowances. This is the main conclusion of the audit conducted by the Algorithm Audit Foundation on behalf of DUO. DUO's control process came under scrutiny in 2023 following <ahref="https://nos.nl/op3/video/2479701-zo-checkt-duo-of-jij-fraudeert-en-dat-systeem-rammelt"target="_blank">news items</a> from Investico and NOS, which stated that students with a migration background were more often accused of abuse than other students.
52
56
berichtgeving
53
57
54
58
A press release can be found [here](/events/press_room/#DUO).
@@ -71,7 +75,7 @@ Education Executive Agency of The Netherlands (DUO)
71
75
72
76
##### Description
73
77
74
-
Report *Preventing prejudice* has been <ahref="https://www.rijksoverheid.nl/documenten/kamerstukken/2024/03/01/kabinetsreactie-onderzoek-naar-controleproces-uitwonendenbeurs"target="_blank">sent</a> as part of the Internal research documents to Dutch Parliament
78
+
Report _Preventing prejudice_ has been <ahref="https://www.rijksoverheid.nl/documenten/kamerstukken/2024/03/01/kabinetsreactie-onderzoek-naar-controleproces-uitwonendenbeurs"target="_blank">sent</a> as part of the Internal research documents to Dutch Parliament
75
79
76
80
{{< accordion_item_close >}}
77
81
@@ -109,4 +113,4 @@ Therefore, in the case of detecting unduly granted social welfare or misuse of c
0 commit comments