Skip to content

Commit cb1277a

Browse files
shreyanidshreyanid
authored
feat: add Confluence source permission metadata (#434)
* wip almost done * more * edit implies view, delete requires view, separate doc view and edit flags * working * clean lint version * add id to provided space keys case * reformat into entities by role dict * refactor inline fn; only make space perm call once * fix * add limit to returned data * comments * overwrite fixtures * move logic to downloader * record_locator has space key in id field, addtl metadata has id * clean up * add fifo dict based cache (naive lru) * update fixtures * update interface to include dict * update fixtures to dict not list * lru * use union for 3.9 support * import requests differently * change import again * add requires * move import * generic exception * remove dep
1 parent e1b8fcd commit cb1277a

File tree

18 files changed

+546
-114
lines changed

18 files changed

+546
-114
lines changed

‎CHANGELOG.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
## 1.0.3
2+
3+
### Features
4+
5+
**Add permission metadata to Confluence source connector**
6+
17
## 1.0.2
28

39
### Features
@@ -10,7 +16,6 @@
1016

1117
* **Migrate project to use pyproject.toml and uv**
1218

13-
1419
## 0.7.2
1520

1621
### Features

‎test/integration/connectors/expected_results/confluence/downloads/MFS/1540126.html‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -110,15 +110,15 @@ <h1>
110110
</td>
111111
<td class="confluenceTd">
112112
<p>
113-
<span class="status-macro aui-lozenge conf-macro output-inline" data-hasbody="false" data-macro-id="90de2698-62c7-46a5-b6da-fc6adf69ebb4" data-macro-name="status">
113+
<span class="status-macro aui-lozenge aui-lozenge-visual-refresh conf-macro output-inline" data-hasbody="false" data-macro-id="90de2698-62c7-46a5-b6da-fc6adf69ebb4" data-macro-name="status">
114114
NOT STARTED
115115
</span>
116116
/
117-
<span class="status-macro aui-lozenge aui-lozenge-current conf-macro output-inline" data-hasbody="false" data-macro-id="a9840cc7-049c-4ef5-a772-74ccf9680569" data-macro-name="status">
117+
<span class="status-macro aui-lozenge aui-lozenge-visual-refresh aui-lozenge-current conf-macro output-inline" data-hasbody="false" data-macro-id="a9840cc7-049c-4ef5-a772-74ccf9680569" data-macro-name="status">
118118
IN PROGRESS
119119
</span>
120120
/
121-
<span class="status-macro aui-lozenge aui-lozenge-success conf-macro output-inline" data-hasbody="false" data-macro-id="588d71c8-1110-4b3f-81f0-68e4696bd61f" data-macro-name="status">
121+
<span class="status-macro aui-lozenge aui-lozenge-visual-refresh aui-lozenge-success conf-macro output-inline" data-hasbody="false" data-macro-id="588d71c8-1110-4b3f-81f0-68e4696bd61f" data-macro-name="status">
122122
COMPLETE
123123
</span>
124124
</p>
@@ -417,7 +417,7 @@ <h2 id="Template-Projectplan-Timeline">
417417
}
418418
</style>
419419
<div class="roadmap-macro-view" data-roadmap-macro-view='{"id":1540126, "version":1, "hash":"bf72b0d2e5ea24888f6179cf74c6606f", "canUserEditPage":true}'>
420-
<svg color-interpolation="auto" color-rendering="auto" data-roadmap-id="722bec0f-d9d3-4196-bab3-3531ad987456" fill="black" fill-opacity="1" font-family="'Dialog'" font-size="12px" font-style="normal" font-weight="normal" height="447" image-rendering="auto" shape-rendering="auto" stroke="black" stroke-dasharray="none" stroke-dashoffset="0" stroke-linecap="square" stroke-linejoin="miter" stroke-miterlimit="10" stroke-opacity="1" stroke-width="1" text-rendering="auto" width="1007" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
420+
<svg color-interpolation="auto" color-rendering="auto" data-roadmap-id="d181d4a8-ee3e-4892-80dc-96db5cb24e53" fill="black" fill-opacity="1" font-family="'Dialog'" font-size="12px" font-style="normal" font-weight="normal" height="447" image-rendering="auto" shape-rendering="auto" stroke="black" stroke-dasharray="none" stroke-dashoffset="0" stroke-linecap="square" stroke-linejoin="miter" stroke-miterlimit="10" stroke-opacity="1" stroke-width="1" text-rendering="auto" width="1007" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
421421
<!--Generated by the Batik Graphics2D SVG Generator-->
422422
<defs id="genericDefs">
423423
</defs>

‎test/integration/connectors/expected_results/confluence/downloads/MFS/229477.html‎

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ <h1>
44
</h1>
55
<div class="contentLayout2">
66
<style>
7-
[data-colorid=mycepcowt1]{color:#97a0af} html[data-color-mode=dark] [data-colorid=mycepcowt1]{color:#505968}[data-colorid=zraak7v43o]{color:#97a0af} html[data-color-mode=dark] [data-colorid=zraak7v43o]{color:#505968}[data-colorid=zati39hphk]{color:#97a0af} html[data-color-mode=dark] [data-colorid=zati39hphk]{color:#505968}[data-colorid=y6u2v9p1oo]{color:#97a0af} html[data-color-mode=dark] [data-colorid=y6u2v9p1oo]{color:#505968}[data-colorid=j8hnbqe1uo]{color:#97a0af} html[data-color-mode=dark] [data-colorid=j8hnbqe1uo]{color:#505968}[data-colorid=mtlu2s659f]{color:#97a0af} html[data-color-mode=dark] [data-colorid=mtlu2s659f]{color:#505968}[data-colorid=t1xenk5t80]{color:#97a0af} html[data-color-mode=dark] [data-colorid=t1xenk5t80]{color:#505968}[data-colorid=dfm65v88u6]{color:#97a0af} html[data-color-mode=dark] [data-colorid=dfm65v88u6]{color:#505968}[data-colorid=andxc2grn6]{color:#97a0af} html[data-color-mode=dark] [data-colorid=andxc2grn6]{color:#505968}[data-colorid=hzd5euc4xe]{color:#97a0af} html[data-color-mode=dark] [data-colorid=hzd5euc4xe]{color:#505968}[data-colorid=dvrcf99bf7]{color:#97a0af} html[data-color-mode=dark] [data-colorid=dvrcf99bf7]{color:#505968}[data-colorid=rotr82dzml]{color:#97a0af} html[data-color-mode=dark] [data-colorid=rotr82dzml]{color:#505968}[data-colorid=y9caomtyr5]{color:#97a0af} html[data-color-mode=dark] [data-colorid=y9caomtyr5]{color:#505968}[data-colorid=d2ixlh87rd]{color:#97a0af} html[data-color-mode=dark] [data-colorid=d2ixlh87rd]{color:#505968}[data-colorid=r67kuy88ka]{color:#97a0af} html[data-color-mode=dark] [data-colorid=r67kuy88ka]{color:#505968}[data-colorid=pyyxwzkdxi]{color:#97a0af} html[data-color-mode=dark] [data-colorid=pyyxwzkdxi]{color:#505968}[data-colorid=zv7ev6zhm9]{color:#97a0af} html[data-color-mode=dark] [data-colorid=zv7ev6zhm9]{color:#505968}[data-colorid=kcqdyls69j]{color:#97a0af} html[data-color-mode=dark] [data-colorid=kcqdyls69j]{color:#505968}[data-colorid=a4uuyy8lcy]{color:#97a0af} html[data-color-mode=dark] [data-colorid=a4uuyy8lcy]{color:#505968}[data-colorid=twwjz7s257]{color:#97a0af} html[data-color-mode=dark] [data-colorid=twwjz7s257]{color:#505968}[data-colorid=w5ci38x3rg]{color:#97a0af} html[data-color-mode=dark] [data-colorid=w5ci38x3rg]{color:#505968}[data-colorid=ja9gc3y3zi]{color:#97a0af} html[data-color-mode=dark] [data-colorid=ja9gc3y3zi]{color:#505968}
7+
[data-colorid=h9625n7ri4]{color:#97a0af} html[data-color-mode=dark] [data-colorid=h9625n7ri4]{color:#505968}[data-colorid=bnyk47r2q7]{color:#97a0af} html[data-color-mode=dark] [data-colorid=bnyk47r2q7]{color:#505968}[data-colorid=nm77lp9bi2]{color:#97a0af} html[data-color-mode=dark] [data-colorid=nm77lp9bi2]{color:#505968}[data-colorid=v37x6ooyb9]{color:#97a0af} html[data-color-mode=dark] [data-colorid=v37x6ooyb9]{color:#505968}[data-colorid=b0joh54dwx]{color:#97a0af} html[data-color-mode=dark] [data-colorid=b0joh54dwx]{color:#505968}[data-colorid=izy7eq4er2]{color:#97a0af} html[data-color-mode=dark] [data-colorid=izy7eq4er2]{color:#505968}[data-colorid=ugf17ahoqb]{color:#97a0af} html[data-color-mode=dark] [data-colorid=ugf17ahoqb]{color:#505968}[data-colorid=xfxgzt6itd]{color:#97a0af} html[data-color-mode=dark] [data-colorid=xfxgzt6itd]{color:#505968}[data-colorid=qwf6t60b6b]{color:#97a0af} html[data-color-mode=dark] [data-colorid=qwf6t60b6b]{color:#505968}[data-colorid=tkm8o7z0bc]{color:#97a0af} html[data-color-mode=dark] [data-colorid=tkm8o7z0bc]{color:#505968}[data-colorid=dn8jtf335h]{color:#97a0af} html[data-color-mode=dark] [data-colorid=dn8jtf335h]{color:#505968}[data-colorid=edyjgolrfp]{color:#97a0af} html[data-color-mode=dark] [data-colorid=edyjgolrfp]{color:#505968}[data-colorid=of0o4h0ry3]{color:#97a0af} html[data-color-mode=dark] [data-colorid=of0o4h0ry3]{color:#505968}[data-colorid=dgi0blamcz]{color:#97a0af} html[data-color-mode=dark] [data-colorid=dgi0blamcz]{color:#505968}[data-colorid=sge3onm6kh]{color:#97a0af} html[data-color-mode=dark] [data-colorid=sge3onm6kh]{color:#505968}[data-colorid=uobweik769]{color:#97a0af} html[data-color-mode=dark] [data-colorid=uobweik769]{color:#505968}[data-colorid=yapxbdm72r]{color:#97a0af} html[data-color-mode=dark] [data-colorid=yapxbdm72r]{color:#505968}[data-colorid=ars5davzwc]{color:#97a0af} html[data-color-mode=dark] [data-colorid=ars5davzwc]{color:#505968}[data-colorid=ms2zw2fxa1]{color:#97a0af} html[data-color-mode=dark] [data-colorid=ms2zw2fxa1]{color:#505968}[data-colorid=p2bqkao7w9]{color:#97a0af} html[data-color-mode=dark] [data-colorid=p2bqkao7w9]{color:#505968}[data-colorid=ror93wg7b0]{color:#97a0af} html[data-color-mode=dark] [data-colorid=ror93wg7b0]{color:#505968}[data-colorid=my32lhlenh]{color:#97a0af} html[data-color-mode=dark] [data-colorid=my32lhlenh]{color:#505968}
88
</style>
99
<div class="columnLayout fixed-width" data-layout="fixed-width">
1010
<div class="cell normal" data-type="normal">
@@ -59,7 +59,7 @@ <h3 id="Myfirstspace-About">
5959
</h3>
6060
<p>
6161
<em>
62-
<span data-colorid="andxc2grn6">
62+
<span data-colorid="ror93wg7b0">
6363
What is your team all about?
6464
</span>
6565
</em>
@@ -69,7 +69,7 @@ <h3 id="Myfirstspace-Missionandvision">
6969
</h3>
7070
<p>
7171
<em>
72-
<span data-colorid="a4uuyy8lcy">
72+
<span data-colorid="xfxgzt6itd">
7373
What is your team's mission? What is your vision?
7474
</span>
7575
</em>
@@ -79,7 +79,7 @@ <h3 id="Myfirstspace-Meettheteam">
7979
</h3>
8080
<p>
8181
<em>
82-
<span data-colorid="hzd5euc4xe">
82+
<span data-colorid="dn8jtf335h">
8383
Add team members to your space.
8484
</span>
8585
</em>
@@ -94,17 +94,17 @@ <h3 id="Myfirstspace-Meettheteam">
9494
<img class="confluence-embedded-image image-center" data-base-url="https://unstructured-ingest-test.atlassian.net/wiki" data-height="264" data-image-src="https://unstructured-ingest-test.atlassian.net/wiki/download/attachments/229477/angie.svg?version=1&amp;modificationDate=1688145926387&amp;cacheVersion=1&amp;api=v2" data-linked-resource-container-id="229477" data-linked-resource-container-version="1" data-linked-resource-content-type="image/svg+xml" data-linked-resource-default-alias="angie.svg" data-linked-resource-id="229502" data-linked-resource-type="attachment" data-linked-resource-version="1" data-media-id="74d2abe2-81ee-4a73-9a2c-545eff5a4626" data-media-type="file" data-unresolved-comment-count="0" data-width="264" loading="lazy" src="https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/angie.svg?version=1&amp;modificationDate=1688145926387&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257" srcset="https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/angie.svg?version=1&amp;modificationDate=1688145926387&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257 2x, https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/angie.svg?version=1&amp;modificationDate=1688145926387&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257 1x" width="442"/>
9595
</span>
9696
<h3 id="Myfirstspace-Teammember" style="text-align: center;">
97-
<span data-colorid="mycepcowt1">
97+
<span data-colorid="qwf6t60b6b">
9898
Team member
9999
</span>
100100
</h3>
101101
<p style="text-align: center;">
102-
<span data-colorid="dvrcf99bf7">
102+
<span data-colorid="ugf17ahoqb">
103103
Role
104104
</span>
105105
</p>
106106
<p style="text-align: center;">
107-
<span data-colorid="y9caomtyr5">
107+
<span data-colorid="uobweik769">
108108
Responsibility
109109
</span>
110110
</p>
@@ -116,17 +116,17 @@ <h3 id="Myfirstspace-Teammember" style="text-align: center;">
116116
<img class="confluence-embedded-image image-center" data-base-url="https://unstructured-ingest-test.atlassian.net/wiki" data-height="264" data-image-src="https://unstructured-ingest-test.atlassian.net/wiki/download/attachments/229477/gael.svg?version=1&amp;modificationDate=1688145927077&amp;cacheVersion=1&amp;api=v2" data-linked-resource-container-id="229477" data-linked-resource-container-version="1" data-linked-resource-content-type="image/svg+xml" data-linked-resource-default-alias="gael.svg" data-linked-resource-id="229507" data-linked-resource-type="attachment" data-linked-resource-version="1" data-media-id="25a12ef7-1cf7-4aa8-a298-1f75e1a7462a" data-media-type="file" data-unresolved-comment-count="0" data-width="264" loading="lazy" src="https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/gael.svg?version=1&amp;modificationDate=1688145927077&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257" srcset="https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/gael.svg?version=1&amp;modificationDate=1688145927077&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257 2x, https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/gael.svg?version=1&amp;modificationDate=1688145927077&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257 1x" width="442"/>
117117
</span>
118118
<h3 id="Myfirstspace-Teammember.1" style="text-align: center;">
119-
<span data-colorid="j8hnbqe1uo">
119+
<span data-colorid="of0o4h0ry3">
120120
Team member
121121
</span>
122122
</h3>
123123
<p style="text-align: center;">
124-
<span data-colorid="ja9gc3y3zi">
124+
<span data-colorid="tkm8o7z0bc">
125125
Role
126126
</span>
127127
</p>
128128
<p style="text-align: center;">
129-
<span data-colorid="y6u2v9p1oo">
129+
<span data-colorid="my32lhlenh">
130130
Responsibility
131131
</span>
132132
</p>
@@ -138,17 +138,17 @@ <h3 id="Myfirstspace-Teammember.1" style="text-align: center;">
138138
<img class="confluence-embedded-image image-center" data-base-url="https://unstructured-ingest-test.atlassian.net/wiki" data-height="264" data-image-src="https://unstructured-ingest-test.atlassian.net/wiki/download/attachments/229477/claudia.svg?version=1&amp;modificationDate=1688145927764&amp;cacheVersion=1&amp;api=v2" data-linked-resource-container-id="229477" data-linked-resource-container-version="1" data-linked-resource-content-type="image/svg+xml" data-linked-resource-default-alias="claudia.svg" data-linked-resource-id="229512" data-linked-resource-type="attachment" data-linked-resource-version="1" data-media-id="2f2d9b14-9e63-471d-89dd-d26b15903c50" data-media-type="file" data-unresolved-comment-count="0" data-width="264" loading="lazy" src="https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/claudia.svg?version=1&amp;modificationDate=1688145927764&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257" srcset="https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/claudia.svg?version=1&amp;modificationDate=1688145927764&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257 2x, https://unstructured-ingest-test.atlassian.net/wiki/download/thumbnails/229477/claudia.svg?version=1&amp;modificationDate=1688145927764&amp;cacheVersion=1&amp;api=v2&amp;width=256&amp;height=257 1x" width="442"/>
139139
</span>
140140
<h3 id="Myfirstspace-Teammember.2" style="text-align: center;">
141-
<span data-colorid="t1xenk5t80">
141+
<span data-colorid="dgi0blamcz">
142142
Team member
143143
</span>
144144
</h3>
145145
<p style="text-align: center;">
146-
<span data-colorid="mtlu2s659f">
146+
<span data-colorid="h9625n7ri4">
147147
Role
148148
</span>
149149
</p>
150150
<p style="text-align: center;">
151-
<span data-colorid="r67kuy88ka">
151+
<span data-colorid="nm77lp9bi2">
152152
Responsibility
153153
</span>
154154
</p>
@@ -171,7 +171,7 @@ <h3 id="Myfirstspace-Contactus">
171171
</h3>
172172
<p>
173173
<em>
174-
<span data-colorid="rotr82dzml">
174+
<span data-colorid="sge3onm6kh">
175175
How can someone reach out to your team?
176176
</span>
177177
</em>
@@ -180,29 +180,29 @@ <h3 id="Myfirstspace-Contactus">
180180
<li>
181181
<p>
182182
<a class="external-link" href="mailto:[email protected]" rel="nofollow">
183-
<span data-colorid="zati39hphk">
183+
<span data-colorid="bnyk47r2q7">
184184
[email protected]
185185
</span>
186186
</a>
187187
</p>
188188
</li>
189189
<li>
190190
<p>
191-
<span data-colorid="pyyxwzkdxi">
191+
<span data-colorid="ars5davzwc">
192192
Tickets
193193
</span>
194194
</p>
195195
</li>
196196
<li>
197197
<p>
198-
<span data-colorid="dfm65v88u6">
198+
<span data-colorid="yapxbdm72r">
199199
Jira board
200200
</span>
201201
</p>
202202
</li>
203203
<li>
204204
<p>
205-
<span data-colorid="zraak7v43o">
205+
<span data-colorid="edyjgolrfp">
206206
#channel
207207
</span>
208208
</p>
@@ -217,7 +217,7 @@ <h3 id="Myfirstspace-ImportantPages">
217217
</h3>
218218
<p>
219219
<em>
220-
<span data-colorid="kcqdyls69j">
220+
<span data-colorid="b0joh54dwx">
221221
List them here
222222
</span>
223223
</em>
@@ -258,7 +258,7 @@ <h3 id="Myfirstspace-OnboardingFAQs" style="text-align: center;">
258258
</h3>
259259
<p style="text-align: center;">
260260
<em>
261-
<span data-colorid="d2ixlh87rd">
261+
<span data-colorid="p2bqkao7w9">
262262
Add resources for new hires
263263
</span>
264264
</em>
@@ -275,7 +275,7 @@ <h3 id="Myfirstspace-Meetingnotes" style="text-align: center;">
275275
</h3>
276276
<p style="text-align: center;">
277277
<em>
278-
<span data-colorid="twwjz7s257">
278+
<span data-colorid="ms2zw2fxa1">
279279
Add links to meeting notes
280280
</span>
281281
</em>
@@ -292,7 +292,7 @@ <h3 id="Myfirstspace-Teamgoals" style="text-align: center;">
292292
</h3>
293293
<p style="text-align: center;">
294294
<em>
295-
<span data-colorid="zv7ev6zhm9">
295+
<span data-colorid="izy7eq4er2">
296296
List them here
297297
</span>
298298
</em>
@@ -308,7 +308,7 @@ <h2 id="Myfirstspace-Teamnews">
308308
</h2>
309309
<p>
310310
<em>
311-
<span data-colorid="w5ci38x3rg">
311+
<span data-colorid="v37x6ooyb9">
312312
Create a blog post to share team news. It will automatically appear here once it's published.
313313
</span>
314314
</em>

0 commit comments

Comments
 (0)