Skip to content

Commit fc9e1b1

Browse files
potter-potterpotter-potter
authored
RIVET-795: add library option to sharepoint (#529)
Thanks to @Anjum48 we have a nice addition to the Sharepoint connector. We can now download from Libraries other than the built in Documents(Shared Documents)
1 parent 24d6fdc commit fc9e1b1

File tree

15 files changed

+455
-9
lines changed

15 files changed

+455
-9
lines changed

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## 1.0.37
2+
3+
* **Added ability to use libraries in Sharepoint connector**
4+
15
## 1.0.36
26

37
* **Added Notion connector sync block handling by teddysupercuts**

‎test/integration/connectors/expected_results/sharepoint5/directory_structure.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"directory_structure": [
3+
"e2e-library-folder/book-war-and-peace-1p.txt",
4+
"fake-doc.rtf",
5+
"fake-html.html"
6+
]
7+
}

‎test/integration/connectors/expected_results/sharepoint5/downloads/e2e-library-folder/book-war-and-peace-1p.txt‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
CHAPTER I
2+
3+
"Well, Prince, so Genoa and Lucca are now just family estates of the
4+
Buonapartes. But I warn you, if you don't tell me that this means war,
5+
if you still try to defend the infamies and horrors perpetrated by that
6+
Antichrist--I really believe he is Antichrist--I will have nothing more
7+
to do with you and you are no longer my friend, no longer my 'faithful
8+
slave,' as you call yourself! But how do you do? I see I have frightened
9+
you--sit down and tell me all the news."
10+
11+
It was in July, 1805, and the speaker was the well-known Anna Pavlovna
12+
Scherer, maid of honor and favorite of the Empress Marya Fedorovna. With
13+
these words she greeted Prince Vasili Kuragin, a man of high rank and
14+
importance, who was the first to arrive at her reception. Anna Pavlovna
15+
had had a cough for some days. She was, as she said, suffering from la
16+
grippe; grippe being then a new word in St. Petersburg, used only by the
17+
elite.
18+
19+
All her invitations without exception, written in French, and delivered
20+
by a scarlet-liveried footman that morning, ran as follows:
21+
22+
"If you have nothing better to do, Count (or Prince), and if the
23+
prospect of spending an evening with a poor invalid is not too terrible,
24+
I shall be very charmed to see you tonight between 7 and 10--Annette
25+
Scherer."
26+
27+
"Heavens! what a virulent attack!" replied the prince, not in the least
28+
disconcerted by this reception. He had just entered, wearing an
29+
embroidered court uniform, knee breeches, and shoes, and had stars on
30+
his breast and a serene expression on his flat face. He spoke in that
31+
refined French in which our grandfathers not only spoke but thought, and
32+
with the gentle, patronizing intonation natural to a man of importance
33+
who had grown old in society and at court. He went up to Anna Pavlovna,
34+
kissed her hand, presenting to her his bald, scented, and shining head,
35+
and complacently seated himself on the sofa.
36+
37+
"First of all, dear friend, tell me how you are. Set your friend's mind
38+
at rest," said he without altering his tone, beneath the politeness and
39+
affected sympathy of which indifference and even irony could be
40+
discerned.
41+
42+
"Can one be well while suffering morally? Can one be calm in times like
43+
these if one has any feeling?" said Anna Pavlovna. "You are staying the
44+
whole evening, I hope?"
45+
46+
"And the fete at the English ambassador's? Today is Wednesday. I must
47+
put in an appearance there," said the prince. "My daughter is coming for
48+
me to take me there."
49+
50+
"I thought today's fete had been canceled. I confess all these
51+
festivities and fireworks are becoming wearisome."
52+
53+
"If they had known that you wished it, the entertainment would have been
54+
put off," said the prince, who, like a wound-up clock, by force of habit
55+
said things he did not even wish to be believed.
56+
57+
"Don't tease! Well, and what has been decided about Novosiltsev's
58+
dispatch? You know everything."
59+
60+
"What can one say about it?" replied the prince in a cold, listless
61+
tone. "What has been decided? They have decided that Buonaparte has
62+
burnt his boats, and I believe that we are ready to burn ours."

‎test/integration/connectors/expected_results/sharepoint5/downloads/fake-doc.rtf‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{\rtf1\ansi\deff0
2+
{\pard \ql \f0 \sa180 \li0 \fi0 \outlinelevel0 \b \fs36 My First Heading\par}
3+
{\pard \ql \f0 \sa180 \li0 \fi0 My first paragraph.\par}
4+
{\pard \sa180 \li0 \fi0 \b Table Example:\par}
5+
{\trowd\cellx3000\cellx6000
6+
\pard\intbl\qc\fs20 Column 1\cell Column 2\cell\row
7+
\pard\intbl\qc\fs20 Row 1, Cell 1\cell Row 1, Cell 2\cell\row
8+
\pard\intbl\qc\fs20 Row 2, Cell 1\cell Row 2, Cell 2\cell\row
9+
}
10+
}

‎test/integration/connectors/expected_results/sharepoint5/downloads/fake-html.html‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
<!DOCTYPE html>
2+
<html>
3+
<body>
4+
5+
<h1>My First Heading</h1>
6+
<h2>My Second Heading</h2>
7+
<p>My first paragraph. lorem ipsum dolor set amet. if the cow comes home under the sun how do you fault the cow for it's worn hooves?</p>
8+
9+
<ul>
10+
<li>A test list</li>
11+
<ul>
12+
<li>With sub-bullets!</li>
13+
<li>And another</li>
14+
</ul>
15+
<li>Back at the root</li>
16+
</ul>
17+
18+
<h3>A Third Heading</h3>
19+
<table>
20+
<tr>
21+
<th>Column 1</th>
22+
<th>Column 2</th>
23+
</tr>
24+
<tr>
25+
<td>Row 1, Cell 1</td>
26+
<td>Row 1, Cell 2</td>
27+
</tr>
28+
<tr>
29+
<td>Row 2, Cell 1</td>
30+
<td>Row 2, Cell 2</td>
31+
</tr>
32+
</table>
33+
34+
</body>
35+
</html>

‎test/integration/connectors/expected_results/sharepoint5/file_data/0153RHRSBCDXFGQVNK6NCZOQZY3CZ2CCI2.json‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"identifier": "0153RHRSBCDXFGQVNK6NCZOQZY3CZ2CCI2",
3+
"connector_type": "sharepoint",
4+
"source_identifiers": {
5+
"filename": "fake-doc.rtf",
6+
"fullpath": "/fake-doc.rtf",
7+
"rel_path": "fake-doc.rtf"
8+
},
9+
"metadata": {
10+
"url": "/drives/b!BWf1pimvx0aQupoNWnoe7Oqvy6uM3DdAlk4K1H4-5O4Pywn-m3V6S6YnHRnicsui/root:/fake-doc.rtf",
11+
"version": "\"{68CA1D22-AA55-45F3-9743-38D8B3A1091A},1\"",
12+
"record_locator": {
13+
"user_pname": null,
14+
"server_relative_path": "/fake-doc.rtf"
15+
},
16+
"date_created": "1749094807.0",
17+
"date_modified": "1749094807.0",
18+
"date_processed": "1749102848.518254",
19+
"permissions_data": null,
20+
"filesize_bytes": null
21+
},
22+
"additional_metadata": {
23+
"@microsoft.graph.downloadUrl": "https://unstructuredio.sharepoint.com/sites/utic-platform-test-source/_layouts/15/download.aspx?UniqueId=68ca1d22-aa55-45f3-9743-38d8b3a1091a&Translate=false&tempauth=v1.eyJzaXRlaWQiOiJhNmY1NjcwNS1hZjI5LTQ2YzctOTBiYS05YTBkNWE3YTFlZWMiLCJhcHBfZGlzcGxheW5hbWUiOiJzaGFyZXBvaW50LWFwcC1yZWdpc3RyYXRpb24iLCJuYW1laWQiOiI2YzE2MDc1My05YjYzLTQ3MDktYTE0MC0xN2EyN2QzMDg3YTZAM2Q2MGE3ZTUtMWUzMi00MTRlLTgzOWItMWM2ZTY3ODI2MTNkIiwiYXVkIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwL3Vuc3RydWN0dXJlZGlvLnNoYXJlcG9pbnQuY29tQDNkNjBhN2U1LTFlMzItNDE0ZS04MzliLTFjNmU2NzgyNjEzZCIsImV4cCI6IjE3NDkxMDY0NDcifQ.CjIKCmFjdG9yYXBwaWQSJDAwMDAwMDAzLTAwMDAtMDAwMC1jMDAwLTAwMDAwMDAwMDAwMAoKCgRzbmlkEgI2NBILCKrti7Dt_5E-EAUaDTIwLjE5MC4xNTQuMzIqLEd0a2pBRGhQc3R6bXNvc1ducUZDVUpQT0pJUjEyU05VMVYxbDJRNllBM2c9MJ0BOAFCEKGlDUPbkACQDyjEPbBvUYRKEGhhc2hlZHByb29mdG9rZW56ATG6AWVzaGFyZXBvaW50dGVuYW50c2V0dGluZ3MucmVhZHdyaXRlLmFsbCBhbGxzaXRlcy53cml0ZSBhbGxzaXRlcy5tYW5hZ2UgYWxsZmlsZXMud3JpdGUgYWxscHJvZmlsZXMucmVhZMgBAQ.8zbbUT2Cr6e9ixLd3zTDt7IXjIuUStttuRu2vSOCG8I&ApiVersion=2.0",
24+
"eTag": "\"{68CA1D22-AA55-45F3-9743-38D8B3A1091A},1\"",
25+
"id": "0153RHRSBCDXFGQVNK6NCZOQZY3CZ2CCI2",
26+
"name": "fake-doc.rtf",
27+
"webUrl": "https://unstructuredio.sharepoint.com/sites/utic-platform-test-source/_layouts/15/Doc.aspx?sourcedoc=%7B68CA1D22-AA55-45F3-9743-38D8B3A1091A%7D&file=fake-doc.rtf&action=default",
28+
"cTag": "\"c:{68CA1D22-AA55-45F3-9743-38D8B3A1091A},1\"",
29+
"size": 408
30+
},
31+
"reprocess": false,
32+
"local_download_path": "/private/var/folders/85/f389rtdn2c971nv4r3d31d740000gn/T/tmpa097bu5v/fake-doc.rtf",
33+
"display_name": null
34+
}

‎test/integration/connectors/expected_results/sharepoint5/file_data/0153RHRSD4LQFV3VWQRZCY3E724WTG5G6C.json‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"identifier": "0153RHRSD4LQFV3VWQRZCY3E724WTG5G6C",
3+
"connector_type": "sharepoint",
4+
"source_identifiers": {
5+
"filename": "book-war-and-peace-1p.txt",
6+
"fullpath": "e2e-library-folder/book-war-and-peace-1p.txt",
7+
"rel_path": "e2e-library-folder/book-war-and-peace-1p.txt"
8+
},
9+
"metadata": {
10+
"url": "/drives/b!BWf1pimvx0aQupoNWnoe7Oqvy6uM3DdAlk4K1H4-5O4Pywn-m3V6S6YnHRnicsui/root:/e2e-library-folder/book-war-and-peace-1p.txt",
11+
"version": "\"{5D0B5C7C-D0D6-458E-8D93-FAE5A66E9BC2},1\"",
12+
"record_locator": {
13+
"user_pname": null,
14+
"server_relative_path": "e2e-library-folder/book-war-and-peace-1p.txt"
15+
},
16+
"date_created": "1749124915.0",
17+
"date_modified": "1749124915.0",
18+
"date_processed": "1749102861.357167",
19+
"permissions_data": null,
20+
"filesize_bytes": null
21+
},
22+
"additional_metadata": {
23+
"@microsoft.graph.downloadUrl": "https://unstructuredio.sharepoint.com/sites/utic-platform-test-source/_layouts/15/download.aspx?UniqueId=5d0b5c7c-d0d6-458e-8d93-fae5a66e9bc2&Translate=false&tempauth=v1.eyJzaXRlaWQiOiJhNmY1NjcwNS1hZjI5LTQ2YzctOTBiYS05YTBkNWE3YTFlZWMiLCJhcHBfZGlzcGxheW5hbWUiOiJzaGFyZXBvaW50LWFwcC1yZWdpc3RyYXRpb24iLCJuYW1laWQiOiI2YzE2MDc1My05YjYzLTQ3MDktYTE0MC0xN2EyN2QzMDg3YTZAM2Q2MGE3ZTUtMWUzMi00MTRlLTgzOWItMWM2ZTY3ODI2MTNkIiwiYXVkIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwL3Vuc3RydWN0dXJlZGlvLnNoYXJlcG9pbnQuY29tQDNkNjBhN2U1LTFlMzItNDE0ZS04MzliLTFjNmU2NzgyNjEzZCIsImV4cCI6IjE3NDkxMDY0NDgifQ.CjIKCmFjdG9yYXBwaWQSJDAwMDAwMDAzLTAwMDAtMDAwMC1jMDAwLTAwMDAwMDAwMDAwMAoKCgRzbmlkEgI2NBILCPbq6rnt_5E-EAUaDTIwLjE5MC4xNTQuMzIqLHlOY2lzRklNQU5Ja1AyKysvbUhwbVJMSVZ0a3phNklzN1hjMXJlYTM0Zlk9MJ0BOAFCEKGlDUQbYACQKB3ieskYW_pKEGhhc2hlZHByb29mdG9rZW56ATG6AWVzaGFyZXBvaW50dGVuYW50c2V0dGluZ3MucmVhZHdyaXRlLmFsbCBhbGxzaXRlcy53cml0ZSBhbGxzaXRlcy5tYW5hZ2UgYWxsZmlsZXMud3JpdGUgYWxscHJvZmlsZXMucmVhZMgBAQ.KmuBtKQ9Kv92y3fHIHJm3eIn3RjcJRm0_kzm7n92-KM&ApiVersion=2.0",
24+
"eTag": "\"{5D0B5C7C-D0D6-458E-8D93-FAE5A66E9BC2},1\"",
25+
"id": "0153RHRSD4LQFV3VWQRZCY3E724WTG5G6C",
26+
"name": "book-war-and-peace-1p.txt",
27+
"webUrl": "https://unstructuredio.sharepoint.com/sites/utic-platform-test-source/Documents2/e2e-library-folder/book-war-and-peace-1p.txt",
28+
"cTag": "\"c:{5D0B5C7C-D0D6-458E-8D93-FAE5A66E9BC2},1\"",
29+
"size": 3045
30+
},
31+
"reprocess": false,
32+
"local_download_path": "/private/var/folders/85/f389rtdn2c971nv4r3d31d740000gn/T/tmpa097bu5v/e2e-library-folder/book-war-and-peace-1p.txt",
33+
"display_name": null
34+
}

‎test/integration/connectors/expected_results/sharepoint5/file_data/0153RHRSFPJTHUKBK7AFEJO3WOAEA3RC7D.json‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"identifier": "0153RHRSFPJTHUKBK7AFEJO3WOAEA3RC7D",
3+
"connector_type": "sharepoint",
4+
"source_identifiers": {
5+
"filename": "fake-html.html",
6+
"fullpath": "/fake-html.html",
7+
"rel_path": "fake-html.html"
8+
},
9+
"metadata": {
10+
"url": "/drives/b!BWf1pimvx0aQupoNWnoe7Oqvy6uM3DdAlk4K1H4-5O4Pywn-m3V6S6YnHRnicsui/root:/fake-html.html",
11+
"version": "\"{45CF4CAF-5F05-4801-976E-CE0101B88BE3},1\"",
12+
"record_locator": {
13+
"user_pname": null,
14+
"server_relative_path": "/fake-html.html"
15+
},
16+
"date_created": "1749094807.0",
17+
"date_modified": "1749094807.0",
18+
"date_processed": "1749102855.2454588",
19+
"permissions_data": null,
20+
"filesize_bytes": null
21+
},
22+
"additional_metadata": {
23+
"@microsoft.graph.downloadUrl": "https://unstructuredio.sharepoint.com/sites/utic-platform-test-source/_layouts/15/download.aspx?UniqueId=45cf4caf-5f05-4801-976e-ce0101b88be3&Translate=false&tempauth=v1.eyJzaXRlaWQiOiJhNmY1NjcwNS1hZjI5LTQ2YzctOTBiYS05YTBkNWE3YTFlZWMiLCJhcHBfZGlzcGxheW5hbWUiOiJzaGFyZXBvaW50LWFwcC1yZWdpc3RyYXRpb24iLCJuYW1laWQiOiI2YzE2MDc1My05YjYzLTQ3MDktYTE0MC0xN2EyN2QzMDg3YTZAM2Q2MGE3ZTUtMWUzMi00MTRlLTgzOWItMWM2ZTY3ODI2MTNkIiwiYXVkIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwL3Vuc3RydWN0dXJlZGlvLnNoYXJlcG9pbnQuY29tQDNkNjBhN2U1LTFlMzItNDE0ZS04MzliLTFjNmU2NzgyNjEzZCIsImV4cCI6IjE3NDkxMDY0NDcifQ.CjIKCmFjdG9yYXBwaWQSJDAwMDAwMDAzLTAwMDAtMDAwMC1jMDAwLTAwMDAwMDAwMDAwMAoKCgRzbmlkEgI2NBILCMD3nrDt_5E-EAUaDTIwLjE5MC4xNTQuMzIqLEF0SkxkRkZUVVZZaktjRm9vdEFvQW1Qa1lPdFpzQ2VLWjdNVEwrNFJqM0k9MJ0BOAFCEKGlDUPbkACQDyjEPbBvUYRKEGhhc2hlZHByb29mdG9rZW56ATG6AWVzaGFyZXBvaW50dGVuYW50c2V0dGluZ3MucmVhZHdyaXRlLmFsbCBhbGxzaXRlcy53cml0ZSBhbGxzaXRlcy5tYW5hZ2UgYWxsZmlsZXMud3JpdGUgYWxscHJvZmlsZXMucmVhZMgBAQ.QpvOKQ5jZTawuDPh98np-9yER_OpRPPT9W7i1ixTTW0&ApiVersion=2.0",
24+
"eTag": "\"{45CF4CAF-5F05-4801-976E-CE0101B88BE3},1\"",
25+
"id": "0153RHRSFPJTHUKBK7AFEJO3WOAEA3RC7D",
26+
"name": "fake-html.html",
27+
"webUrl": "https://unstructuredio.sharepoint.com/sites/utic-platform-test-source/Documents2/fake-html.html",
28+
"cTag": "\"c:{45CF4CAF-5F05-4801-976E-CE0101B88BE3},1\"",
29+
"size": 665
30+
},
31+
"reprocess": false,
32+
"local_download_path": "/private/var/folders/85/f389rtdn2c971nv4r3d31d740000gn/T/tmpa097bu5v/fake-html.html",
33+
"display_name": null
34+
}

‎test/integration/connectors/expected_results/sharepoint6/directory_structure.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"directory_structure": [
3+
"book-war-and-peace-1p.txt"
4+
]
5+
}

‎test/integration/connectors/expected_results/sharepoint6/downloads/book-war-and-peace-1p.txt‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
CHAPTER I
2+
3+
"Well, Prince, so Genoa and Lucca are now just family estates of the
4+
Buonapartes. But I warn you, if you don't tell me that this means war,
5+
if you still try to defend the infamies and horrors perpetrated by that
6+
Antichrist--I really believe he is Antichrist--I will have nothing more
7+
to do with you and you are no longer my friend, no longer my 'faithful
8+
slave,' as you call yourself! But how do you do? I see I have frightened
9+
you--sit down and tell me all the news."
10+
11+
It was in July, 1805, and the speaker was the well-known Anna Pavlovna
12+
Scherer, maid of honor and favorite of the Empress Marya Fedorovna. With
13+
these words she greeted Prince Vasili Kuragin, a man of high rank and
14+
importance, who was the first to arrive at her reception. Anna Pavlovna
15+
had had a cough for some days. She was, as she said, suffering from la
16+
grippe; grippe being then a new word in St. Petersburg, used only by the
17+
elite.
18+
19+
All her invitations without exception, written in French, and delivered
20+
by a scarlet-liveried footman that morning, ran as follows:
21+
22+
"If you have nothing better to do, Count (or Prince), and if the
23+
prospect of spending an evening with a poor invalid is not too terrible,
24+
I shall be very charmed to see you tonight between 7 and 10--Annette
25+
Scherer."
26+
27+
"Heavens! what a virulent attack!" replied the prince, not in the least
28+
disconcerted by this reception. He had just entered, wearing an
29+
embroidered court uniform, knee breeches, and shoes, and had stars on
30+
his breast and a serene expression on his flat face. He spoke in that
31+
refined French in which our grandfathers not only spoke but thought, and
32+
with the gentle, patronizing intonation natural to a man of importance
33+
who had grown old in society and at court. He went up to Anna Pavlovna,
34+
kissed her hand, presenting to her his bald, scented, and shining head,
35+
and complacently seated himself on the sofa.
36+
37+
"First of all, dear friend, tell me how you are. Set your friend's mind
38+
at rest," said he without altering his tone, beneath the politeness and
39+
affected sympathy of which indifference and even irony could be
40+
discerned.
41+
42+
"Can one be well while suffering morally? Can one be calm in times like
43+
these if one has any feeling?" said Anna Pavlovna. "You are staying the
44+
whole evening, I hope?"
45+
46+
"And the fete at the English ambassador's? Today is Wednesday. I must
47+
put in an appearance there," said the prince. "My daughter is coming for
48+
me to take me there."
49+
50+
"I thought today's fete had been canceled. I confess all these
51+
festivities and fireworks are becoming wearisome."
52+
53+
"If they had known that you wished it, the entertainment would have been
54+
put off," said the prince, who, like a wound-up clock, by force of habit
55+
said things he did not even wish to be believed.
56+
57+
"Don't tease! Well, and what has been decided about Novosiltsev's
58+
dispatch? You know everything."
59+
60+
"What can one say about it?" replied the prince in a cold, listless
61+
tone. "What has been decided? They have decided that Buonaparte has
62+
burnt his boats, and I believe that we are ready to burn ours."

0 commit comments

Comments
 (0)