Skip to content

Commit 88a1316

Browse files
gsaluja9gsaluja9
authored
Merge pull request #298 from aperture-data/release-0.4.9
Release 0.4.9
2 parents 09bc054 + 5fc8a97 commit 88a1316

File tree

32 files changed

+157
-485
lines changed

32 files changed

+157
-485
lines changed

‎.github/workflows/checks.yml‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,19 +9,17 @@ on:
99

1010
jobs:
1111
pre-commit:
12-
1312
runs-on: ubuntu-latest
1413

1514
steps:
16-
17-
- uses: actions/checkout@v2
15+
- uses: actions/checkout@v3
1816

1917
- uses: actions/setup-python@v3
2018
with:
2119
python-version: '3.10'
2220

2321
- uses: pre-commit/[email protected]
2422

25-
- uses: actions/checkout@v2
23+
- uses: actions/checkout@v3
2624

2725
- uses: luisremis/find-trailing-whitespace@master

‎.github/workflows/develop.yml‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ jobs:
1414

1515
steps:
1616

17-
- uses: actions/checkout@v2
17+
- uses: actions/checkout@v3
1818

1919
- name: Login to DockerHub
20-
uses: docker/login-action@v1
20+
uses: docker/login-action@v2
2121
with:
2222
username: ${{ secrets.DOCKER_USER }}
2323
password: ${{ secrets.DOCKER_PASS }}
@@ -35,7 +35,7 @@ jobs:
3535
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
3636
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
3737
GCP_SERVICE_ACCOUNT_KEY: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}
38-
run: 'EXCLUDE_DOCUMENTATION=1 EXCLUDE_DEPLOY=1 ./ci.sh'
38+
run: 'EXCLUDE_DEPLOY=1 ./ci.sh'
3939
shell: bash
4040

4141
build_and_deploy_docs:
@@ -48,10 +48,10 @@ jobs:
4848

4949
steps:
5050

51-
- uses: actions/checkout@v2
51+
- uses: actions/checkout@v3
5252

5353
- name: Login to DockerHub
54-
uses: docker/login-action@v1
54+
uses: docker/login-action@v2
5555
with:
5656
username: ${{ secrets.DOCKER_USER }}
5757
password: ${{ secrets.DOCKER_PASS }}
@@ -74,10 +74,10 @@ jobs:
7474

7575
steps:
7676

77-
- uses: actions/checkout@v2
77+
- uses: actions/checkout@v3
7878

7979
- name: Login to DockerHub
80-
uses: docker/login-action@v1
80+
uses: docker/login-action@v2
8181
with:
8282
username: ${{ secrets.DOCKER_USER }}
8383
password: ${{ secrets.DOCKER_PASS }}
@@ -88,5 +88,5 @@ jobs:
8888
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
8989
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
9090
run: |
91-
EXCLUDE_TESTING=1 EXCLUDE_DOCUMENTATION=1 ./ci.sh
91+
EXCLUDE_TESTING=1 ./ci.sh
9292
shell: bash

‎.github/workflows/main.yml‎

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,18 +6,18 @@ on:
66
- main
77

88
jobs:
9-
build-test:
9+
build_and_test:
1010

1111
runs-on:
1212
- self-hosted
1313
- deployer
1414

1515
steps:
1616

17-
- uses: actions/checkout@v2
17+
- uses: actions/checkout@v3
1818

1919
- name: Login to DockerHub
20-
uses: docker/login-action@v1
20+
uses: docker/login-action@v2
2121
with:
2222
username: ${{ secrets.DOCKER_USER }}
2323
password: ${{ secrets.DOCKER_PASS }}
@@ -35,28 +35,28 @@ jobs:
3535
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
3636
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
3737
GCP_SERVICE_ACCOUNT_KEY: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}
38-
run: 'EXCLUDE_DOCUMENTATION=1 EXCLUDE_DEPLOY=1 ./ci.sh'
38+
run: 'EXCLUDE_DEPLOY=1 ./ci.sh'
3939
shell: bash
4040

41-
build_and_deploy_docs:
41+
build_notebooks_and_publish_pypi:
4242
needs:
43-
- build-test
43+
- build_and_test
4444

4545
runs-on:
4646
- self-hosted
4747
- deployer
4848

4949
steps:
5050

51-
- uses: actions/checkout@v2
51+
- uses: actions/checkout@v3
5252

5353
- name: Login to DockerHub
54-
uses: docker/login-action@v1
54+
uses: docker/login-action@v2
5555
with:
5656
username: ${{ secrets.DOCKER_USER }}
5757
password: ${{ secrets.DOCKER_PASS }}
5858

59-
- name: Build Notebook,docs Docker
59+
- name: Build Notebook
6060
env:
6161
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
6262
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
@@ -83,17 +83,17 @@ jobs:
8383

8484
deploy_web_services:
8585
needs:
86-
- build-test
87-
- build_and_deploy_docs
86+
- build_and_test
87+
- build_notebooks_and_publish_pypi
8888

8989
runs-on: ubuntu-latest
9090

9191
steps:
9292

93-
- uses: actions/checkout@v2
93+
- uses: actions/checkout@v3
9494

9595
- name: Login to DockerHub
96-
uses: docker/login-action@v1
96+
uses: docker/login-action@v2
9797
with:
9898
username: ${{ secrets.DOCKER_USER }}
9999
password: ${{ secrets.DOCKER_PASS }}
@@ -104,13 +104,13 @@ jobs:
104104
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
105105
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
106106
run: |
107-
EXCLUDE_TESTING=1 EXCLUDE_DOCUMENTATION=1 ./ci.sh
107+
EXCLUDE_TESTING=1 ./ci.sh
108108
shell: bash
109109

110110
trigger_demos_buils:
111111
runs-on: ubuntu-latest
112112
needs:
113-
- build_and_deploy_docs
113+
- build_notebooks_and_publish_pypi
114114
steps:
115115
- uses: actions/checkout@v3
116116
- name: repository dispatch
@@ -126,7 +126,7 @@ jobs:
126126
trigger_docs_deploy:
127127
runs-on: ubuntu-latest
128128
needs:
129-
- build_and_deploy_docs
129+
- build_notebooks_and_publish_pypi
130130
steps:
131131
- uses: actions/checkout@v3
132132
- name: repository dispatch

‎.github/workflows/pr.yaml‎

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ jobs:
1414

1515
steps:
1616

17-
- uses: actions/checkout@v2
17+
- uses: actions/checkout@v3
1818

1919
- name: Login to DockerHub
20-
uses: docker/login-action@v1
20+
uses: docker/login-action@v2
2121
with:
2222
username: ${{ secrets.DOCKER_USER }}
2323
password: ${{ secrets.DOCKER_PASS }}
@@ -35,7 +35,7 @@ jobs:
3535
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
3636
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
3737
GCP_SERVICE_ACCOUNT_KEY: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}
38-
run: 'EXCLUDE_DOCUMENTATION=1 EXCLUDE_DEPLOY=1 NO_PUSH=1 BRANCH_NAME=${{ github.event.pull_request.head.ref }} ./ci.sh'
38+
run: 'EXCLUDE_DEPLOY=1 NO_PUSH=1 BRANCH_NAME=${{ github.event.pull_request.head.ref }} ./ci.sh'
3939
shell: bash
4040

4141
run_test_conda_gpu:
@@ -46,10 +46,10 @@ jobs:
4646

4747
steps:
4848

49-
- uses: actions/checkout@v2
49+
- uses: actions/checkout@v3
5050

5151
- name: Login to DockerHub
52-
uses: docker/login-action@v1
52+
uses: docker/login-action@v2
5353
with:
5454
username: ${{ secrets.DOCKER_USER }}
5555
password: ${{ secrets.DOCKER_PASS }}
@@ -86,14 +86,10 @@ jobs:
8686

8787
steps:
8888

89-
- uses: actions/checkout@v2
89+
- uses: actions/checkout@v3
9090

9191
- name: Login to DockerHub
92-
uses: docker/login-action@v1
92+
uses: docker/login-action@v2
9393
with:
9494
username: ${{ secrets.DOCKER_USER }}
9595
password: ${{ secrets.DOCKER_PASS }}
96-
97-
- name: Build documentation
98-
run: 'EXCLUDE_TESTING=1 EXCLUDE_DEPLOY=1 NO_PUSH=1 BRANCH_NAME=${{ github.event.pull_request.head.ref }} ./ci.sh'
99-
shell: bash

‎.github/workflows/release.yaml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ jobs:
1414

1515
steps:
1616

17-
- uses: actions/checkout@v2
17+
- uses: actions/checkout@v3
1818

1919
- name: Login to DockerHub
20-
uses: docker/login-action@v1
20+
uses: docker/login-action@v2
2121
with:
2222
username: ${{ secrets.DOCKER_USER }}
2323
password: ${{ secrets.DOCKER_PASS }}
@@ -35,7 +35,7 @@ jobs:
3535
AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
3636
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
3737
GCP_SERVICE_ACCOUNT_KEY: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}
38-
run: 'EXCLUDE_DOCUMENTATION=1 EXCLUDE_DEPLOY=1 NO_PUSH=1 UPDATE_BRANCH=1 ./ci.sh'
38+
run: 'EXCLUDE_DEPLOY=1 NO_PUSH=1 UPDATE_BRANCH=1 ./ci.sh'
3939
shell: bash
4040

4141
build_and_deploy_docs:
@@ -46,10 +46,10 @@ jobs:
4646

4747
steps:
4848

49-
- uses: actions/checkout@v2
49+
- uses: actions/checkout@v3
5050

5151
- name: Login to DockerHub
52-
uses: docker/login-action@v1
52+
uses: docker/login-action@v2
5353
with:
5454
username: ${{ secrets.DOCKER_USER }}
5555
password: ${{ secrets.DOCKER_PASS }}

‎aperturedb/Connector.py‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,7 @@ def _refresh_token(self):
207207
}
208208
}]
209209

210-
response, _ = self._query(query, [])
210+
response, _ = self._query(query, [], try_resume=False)
211211

212212
logger.info(f"Refresh token response: \r\n{response}")
213213
if isinstance(response, list):
@@ -281,7 +281,7 @@ def _connect(self):
281281

282282
self.connected = True
283283

284-
def _query(self, query, blob_array = []):
284+
def _query(self, query, blob_array = [], try_resume=True):
285285
response_blob_array = []
286286
# Check the query type
287287
if not isinstance(query, str): # assumes json
@@ -328,7 +328,12 @@ def _query(self, query, blob_array = []):
328328
time.sleep(1)
329329
self.conn.close()
330330
self._connect()
331-
self._renew_session()
331+
# Try to resume the session, in cases where the connection is severed.
332+
# For example aperturedb server is restarted, or network is lost.
333+
# While this is useful bit of code, when executed in a refresh token
334+
# path, this can cause a deadlock. Hence the try_resume flag.
335+
if try_resume:
336+
self._renew_session()
332337
if tries == 3:
333338
raise Exception(
334339
f"Could not query apertureDB using TCP.")

‎aperturedb/ConnectorRest.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ def __del__(self):
136136
logger.info("Done with connector")
137137
self.http_session.close()
138138

139-
def _query(self, query, blob_array = []):
139+
def _query(self, query, blob_array = [], try_resume=True):
140140
response_blob_array = []
141141
# Check the query type
142142
if not isinstance(query, str): # assumes json

‎aperturedb/KaggleData.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,12 @@ class KaggleData(Subscriptable):
1111
"""
1212
**Class to wrap around a Dataset retrieved from kaggle**
1313
14-
A DataSet downloaded from kaggle does not implement a standard mechanism to iterate over it's values
15-
This class intends to provide an abstracion like that of a pytorch dataset.
16-
Where the iteration over Dataset elements yields an atomic record.
14+
A DataSet downloaded from kaggle does not implement a standard mechanism to iterate over its values
15+
This class intends to provide an abstracion like that of a pytorch dataset
16+
where the iteration over Dataset elements yields an atomic record.
1717
1818
.. note::
19-
This class should be subclassed with specefic implementations of generate_index and generate_query.
19+
This class should be subclassed with specific implementations of generate_index and generate_query.
2020
2121
Example subclass:
2222
@@ -94,7 +94,7 @@ def generate_query(self, idx: int) -> Tuple[List[dict], List[bytes]]:
9494
```
9595
9696
Args:
97-
dataset_ref (str): URL of kaggle dataset, for example 'https://www.kaggle.com/datasets/crawford/cat-dataset'
97+
dataset_ref (str): URL of kaggle dataset, for example https://www.kaggle.com/datasets/jessicali9530/celeba-dataset
9898
records_count (int): number of records to provide to generate.
9999
100100
"""

‎aperturedb/PolygonDataCSV.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ class PolygonDataCSV(CSVParser.CSVParser):
3232
3333
**_label**: optionally applies a label to the polygon objects.
3434
35-
**polygons**: a JSON array of polygon regions. Each polygon region is itself an array of [x,y] vertices that describe the boundary of a single contiguous polygon. See also https://docs.aperturedata.io/parameters/polygons.html.
35+
**polygons**: a JSON array of polygon regions. Each polygon region is itself an array of [x,y] vertices that describe the boundary of a single contiguous polygon. See also [Polygon API parameter](query_language/Reference/shared_command_parameters/polygons).
3636
3737
Example csv file::
3838

‎aperturedb/Query.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010

1111
class ObjectType(Enum):
1212
"""
13-
Based on : https://docs.aperturedata.io/API-Description.html
13+
Based on : [API description](/query_language/Overview/API%20Description)
1414
1515
"""
1616
BLOB = "_Blob"

0 commit comments

Comments
 (0)