Skip to content

Commit 5103418

Browse files
HadleyKingHadleyKing
authored
Swagger move (#283)
* Remove API app Changes to be committed: deleted: api/__init__.py deleted: api/admin.py deleted: api/apps.py deleted: api/fixtures/bootstrap.json deleted: api/fixtures/metafixtures deleted: api/fixtures/metafixtures.json deleted: api/keys.sh deleted: api/migrations/0001_initial.py deleted: api/migrations/0002_auto_20220124_2356.py deleted: api/migrations/0003_rename_meta_table_prefix_table.py deleted: api/migrations/0004_rename_group_info_groupinfo.py deleted: api/migrations/0005_rename_prefixes_prefix.py deleted: api/migrations/0006_delete_new_users.py deleted: api/migrations/__init__.py deleted: api/model/__init__.py deleted: api/model/groups.py deleted: api/model/prefix.py deleted: api/models.py deleted: api/permissions.py deleted: api/rdb.sh deleted: api/request_definitions/GET.schema deleted: api/request_definitions/POST.schema deleted: api/request_definitions/templates/DELETE_delete_object_by_id.schema deleted: api/request_definitions/templates/GET_activate_account.schema deleted: api/request_definitions/templates/GET_get_object_by_id.schema deleted: api/request_definitions/templates/GET_retrieve_available_schema.schema deleted: api/request_definitions/templates/POST_convert_existing_object_between_schemas.schema deleted: api/request_definitions/templates/POST_convert_payload_to_schema.schema deleted: api/request_definitions/templates/POST_new_account.schema deleted: api/request_definitions/templates/POST_object_listing_by_token.schema deleted: api/request_definitions/templates/POST_objects_draft.schema deleted: api/request_definitions/templates/POST_objects_publish.schema deleted: api/request_definitions/templates/POST_read_object.schema deleted: api/request_definitions/templates/POST_validate_payload_against_schema.schema deleted: api/scripts/__init__.py deleted: api/scripts/method_specific/GET_draft_object_by_id.py deleted: api/scripts/method_specific/GET_published_object_by_id.py deleted: api/scripts/method_specific/GET_published_object_by_id_with_version.py deleted: api/scripts/method_specific/GET_retrieve_available_schema.py deleted: api/scripts/method_specific/POST_api_objects_drafts_create.py deleted: api/scripts/method_specific/POST_api_objects_drafts_delete.py deleted: api/scripts/method_specific/POST_api_objects_drafts_modify.py deleted: api/scripts/method_specific/POST_api_objects_drafts_permissions.py deleted: api/scripts/method_specific/POST_api_objects_drafts_permissions_set.py deleted: api/scripts/method_specific/POST_api_objects_drafts_publish.py deleted: api/scripts/method_specific/POST_api_objects_drafts_read.py deleted: api/scripts/method_specific/POST_api_objects_drafts_token.py deleted: api/scripts/method_specific/POST_api_objects_publish.py deleted: api/scripts/method_specific/POST_api_objects_published.py deleted: api/scripts/method_specific/POST_api_objects_search.py deleted: api/scripts/method_specific/POST_api_objects_token.py deleted: api/scripts/method_specific/POST_validate_payload_against_schema.py deleted: api/scripts/method_specific/__init__.py deleted: api/scripts/utilities/DbUtils.py deleted: api/scripts/utilities/FileUtils.py deleted: api/scripts/utilities/JsonUtils.py deleted: api/scripts/utilities/RequestUtils.py deleted: api/scripts/utilities/ResponseUtils.py deleted: api/scripts/utilities/SettingsUtils.py deleted: api/scripts/utilities/UserUtils.py deleted: api/scripts/utilities/__init__.py deleted: api/serializers.py deleted: api/signals.py deleted: api/templates/api/account_activation_message.html deleted: api/urls.py deleted: api/validation_definitions/IEEE/2791object.json deleted: api/validation_definitions/IEEE/description_domain.json deleted: api/validation_definitions/IEEE/error_domain.json deleted: api/validation_definitions/IEEE/execution_domain.json deleted: api/validation_definitions/IEEE/io_domain.json deleted: api/validation_definitions/IEEE/parametric_domain.json deleted: api/validation_definitions/IEEE/provenance_domain.json deleted: api/validation_definitions/IEEE/usability_domain.json deleted: api/validation_definitions/IEEE_sub/IEEE2791-2020.schema deleted: api/validation_definitions/IEEE_sub/domains/description_domain.json deleted: api/validation_definitions/IEEE_sub/domains/error_domain.json deleted: api/validation_definitions/IEEE_sub/domains/execution_domain.json deleted: api/validation_definitions/IEEE_sub/domains/io_domain.json deleted: api/validation_definitions/IEEE_sub/domains/parametric_domain.json deleted: api/validation_definitions/IEEE_sub/domains/provenance_domain.json deleted: api/validation_definitions/IEEE_sub/domains/usability_domain.json deleted: api/validation_definitions/uri_external deleted: api/views.py new file: config/settings.py modified: config/urls.py Changes not staged for commit: modified: authentication/apis.py modified: authentication/migrations/0001_initial.py deleted: authentication/migrations/0002_newuser.py modified: authentication/services.py modified: docs/refactor.md modified: search/selectors.py modified: tests/fixtures/test_data.json deleted: tests/test_database.py deleted: tests/test_fixtures.py deleted: tests/test_models modified: tests/test_views/test_api_account_activate.py deleted: tests/test_views/test_api_accounts_describe.py modified: tests/test_views/test_api_auth_add.py modified: tests/test_views/test_api_auth_reset_token.py deleted: tests/test_views/test_api_groups_group_info.py deleted: tests/test_views/test_api_groups_modify.py deleted: tests/test_views/test_api_objects.py deleted: tests/test_views/test_api_objects_drafts_create.py deleted: tests/test_views/test_api_objects_drafts_modify.py deleted: tests/test_views/test_api_objects_drafts_publish.py deleted: tests/test_views/test_api_objects_search.py deleted: tests/test_views/test_api_objects_validate.py deleted: tests/test_views/test_api_prefixes_create.py deleted: tests/test_views/test_api_prefixes_token.py deleted: tests/test_views/test_get_object_id_draft.py deleted: tests/test_views/test_get_objectid.py deleted: tests/test_views/test_published_object_by_id.py modified: token.json * Framework for BioCompute model Changes to be committed: new file: biocompute/__init__.py new file: biocompute/admin.py new file: biocompute/apis.py new file: biocompute/migrations/__init__.py new file: biocompute/models.py new file: biocompute/selectors.py new file: biocompute/services.py new file: biocompute/urls.py * Added files for Prefix model Changes to be committed: new file: prefix/__init__.py new file: prefix/admin.py new file: prefix/apis.py new file: prefix/apps.py new file: prefix/migrations/__init__.py new file: prefix/models.py new file: prefix/selectors.py new file: prefix/services.py new file: prefix/urls.py * Tests for authentication Changes to be committed: modified: .gitignore modified: authentication/apis.py modified: authentication/migrations/0001_initial.py deleted: authentication/migrations/0002_newuser.py modified: authentication/services.py new file: biocompute/migrations/0001_initial.py modified: docs/refactor.md new file: prefix/migrations/0001_initial.py modified: search/selectors.py new file: test.json new file: tests/fixtures/old_test_data.json modified: tests/fixtures/test_data.json deleted: tests/test_database.py deleted: tests/test_fixtures.py deleted: tests/test_models modified: tests/test_views/test_api_account_activate.py renamed: tests/test_views/test_api_accounts_describe.py -> tests/test_views/test_api_account_describe.py modified: tests/test_views/test_api_auth_add.py modified: tests/test_views/test_api_auth_reset_token.py deleted: tests/test_views/test_api_groups_group_info.py deleted: tests/test_views/test_api_groups_modify.py deleted: tests/test_views/test_api_objects.py deleted: tests/test_views/test_api_objects_drafts_create.py deleted: tests/test_views/test_api_objects_drafts_modify.py deleted: tests/test_views/test_api_objects_drafts_publish.py deleted: tests/test_views/test_api_objects_search.py deleted: tests/test_views/test_api_objects_validate.py deleted: tests/test_views/test_api_prefixes_create.py deleted: tests/test_views/test_api_prefixes_token.py deleted: tests/test_views/test_get_object_id_draft.py deleted: tests/test_views/test_get_objectid.py deleted: tests/test_views/test_published_object_by_id.py modified: token.json * Move Swagger files and implement BCO draft create Changes to be committed: modified: biocompute/apis.py modified: biocompute/models.py modified: biocompute/services.py modified: biocompute/urls.py new file: config/services.py modified: config/urls.py modified: docs/refactor.md modified: prefix/urls.py new file: tests/fixtures/example_bco.py new file: tests/test_views/test_api_objects_drafts_create.py * Doc fix in Bco.models Changes to be committed: modified: biocompute/models.py
1 parent 6366b0b commit 5103418

File tree

9 files changed

+855
-14
lines changed

9 files changed

+855
-14
lines changed

‎biocompute/apis.py‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,111 @@
1+
#!/usr/bin/env python3
12
#biocompute/apis.py
3+
4+
"""BioCompute Object APIs
5+
"""
6+
7+
from drf_yasg import openapi
8+
from drf_yasg.utils import swagger_auto_schema
9+
from rest_framework.views import APIView
10+
from rest_framework import status
11+
from rest_framework.permissions import IsAuthenticated
12+
from rest_framework.response import Response
13+
from tests.fixtures.example_bco import BCO_000001
14+
from config.services import legacy_api_converter
15+
from biocompute.services import BcoDraftSerializer
16+
17+
class DraftsCreateApi(APIView):
18+
"""
19+
Create BCO Draft
20+
21+
--------------------
22+
23+
Creates a new BCO draft object.
24+
"""
25+
26+
request_body = openapi.Schema(
27+
type=openapi.TYPE_ARRAY,
28+
title="Create BCO Draft Schema",
29+
items=openapi.Schema(
30+
type=openapi.TYPE_OBJECT,
31+
required=["prefix", "contents"],
32+
properties={
33+
"object_id": openapi.Schema(
34+
type=openapi.TYPE_STRING,
35+
description="BCO Object ID.",
36+
example="https://biocomputeobject.org/TEST_000001"
37+
),
38+
"prefix": openapi.Schema(
39+
type=openapi.TYPE_STRING,
40+
description="BCO Prefix to use",
41+
example="BCO"
42+
),
43+
"authorized_users": openapi.Schema(
44+
type=openapi.TYPE_ARRAY,
45+
description="Users which can access the BCO draft.",
46+
items=openapi.Schema(type=openapi.TYPE_STRING, example="None")
47+
),
48+
"authorized_groups": openapi.Schema(
49+
type=openapi.TYPE_ARRAY,
50+
description="Group which can access the BCO draft.",
51+
items=openapi.Schema(type=openapi.TYPE_STRING, example="None")
52+
),
53+
"contents": openapi.Schema(
54+
type=openapi.TYPE_OBJECT,
55+
description="Contents of the BCO.",
56+
example=BCO_000001
57+
),
58+
},
59+
),
60+
description="BCO Drafts to create.",
61+
)
62+
63+
@swagger_auto_schema(
64+
request_body=request_body,
65+
responses={
66+
200: "Creation of BCO draft is successful.",
67+
300: "Some requests failed and some succeeded.",
68+
400: "Bad request.",
69+
403: "Invalid token.",
70+
},
71+
tags=["BCO Management"],
72+
)
73+
74+
def post(self, request) -> Response:
75+
response_data = {}
76+
owner = request.user
77+
data = request.data
78+
all_good = True
79+
if 'POST_api_objects_draft_create' in request.data:
80+
data = legacy_api_converter(request.data)
81+
82+
for index, object in enumerate(data):
83+
list_id = object.get("object_id", index)
84+
bco = BcoDraftSerializer(data=object, context={'request': request})
85+
86+
if bco.is_valid():
87+
bco.create(bco.validated_data)
88+
response_data[list_id] = "bco valid"
89+
90+
else:
91+
response_data[list_id] = bco.errors
92+
all_good = False
93+
94+
if all_good is False:
95+
return Response(
96+
status=status.HTTP_207_MULTI_STATUS,
97+
data=response_data
98+
)
99+
100+
return Response(status=status.HTTP_200_OK, data=response_data)
101+
102+
# def create(self, validated_data):
103+
# # Custom creation logic here, if needed
104+
# return Bco.objects.create(**validated_data)
105+
106+
# def update(self, instance, validated_data):
107+
# # Custom update logic here, if needed
108+
# for attr, value in validated_data.items():
109+
# setattr(instance, attr, value)
110+
# instance.save()
111+
# return instance

‎biocompute/models.py‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,16 +27,20 @@ class Bco(models.Model):
2727
BCO Object Identifier, and primary key
2828
contents: JSONField
2929
BCO JSON contents
30-
authorized_group: ManyToManyField(Group)
31-
String representing the django.contrib.auth.models.Group that 'owns' the object
32-
owner_user = ForeignKey(User)
30+
owner = ForeignKey(User)
3331
String representing the django.contrib.auth.models.User that 'owns' the object
32+
authorized_users: ManyToManyField(User)
33+
String representing the User that has access to the object
34+
authorized_group: ManyToManyField(Group)
35+
String representing the Group that has access to the object
3436
prefix: str
3537
Prefix for the BCO
3638
state:str
3739
State of object. REFERENCED, PUBLISHED, DRAFT, and DELETE are currently accepted values.
3840
last_update: DateTime
3941
Date Time object for the last database change to this object
42+
access_count: Int
43+
number of times this object has been downloaded
4044
4145
"""
4246

‎biocompute/services.py‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
#!/usr/bin/env python3
2+
# biocopmute/services.py
3+
4+
from django.db import transaction
5+
from django.utils import timezone
6+
from biocompute.models import Bco
7+
from prefix.models import Prefix
8+
from django.contrib.auth.models import Group, User
9+
from rest_framework import serializers
10+
11+
"""BioCompute Services
12+
13+
Service functions for working with BCOs
14+
"""
15+
16+
class BcoDraftSerializer(serializers.Serializer):
17+
object_id = serializers.URLField(required=False)
18+
contents = serializers.JSONField()
19+
prefix = serializers.CharField(max_length=5, min_length=3, default="BCO")
20+
authorized_groups = serializers.ListField(child=serializers.CharField(), required=False)
21+
authorized_users = serializers.ListField(child=serializers.CharField(), required=False)
22+
23+
def validate(self, attrs):
24+
errors = {}
25+
request = self.context.get('request')
26+
attrs["owner"] = request.user
27+
28+
#check for groups
29+
if 'authorized_groups' in attrs:
30+
for group in attrs['authorized_groups']:
31+
try:
32+
Group.objects.get(name=group)
33+
except Exception as err:
34+
errors['authorized_groups'] = f"Invalid group: {group}"
35+
# check for users
36+
if 'authorized_users' in attrs:
37+
for user in attrs['authorized_users']:
38+
try:
39+
# import pdb; pdb.set_trace()
40+
User.objects.get(username=user)
41+
except Exception as err:
42+
errors['authorized_users'] =f"Invalid user: {user}"
43+
44+
# Validate Prefix
45+
try:
46+
attrs['prefix_instance'] = Prefix.objects.get(prefix=attrs['prefix'])
47+
except Prefix.DoesNotExist as err:
48+
errors['prefix'] = 'Invalid prefix.'
49+
50+
# Validate object_id match
51+
if 'object_id' in attrs and attrs['object_id'] != attrs['contents'].get('object_id', ''):
52+
errors["object_id"] = "object_id does not match object_id in contents."
53+
54+
# Validate that object_id is unique
55+
object_id = attrs['contents'].get('object_id', '')
56+
57+
if not Bco.objects.filter(object_id=object_id).exists():
58+
pass
59+
else:
60+
errors["object_id"] = f"That object_id, {attrs['object_id']}, already exists."
61+
62+
if errors:
63+
raise serializers.ValidationError(errors)
64+
65+
return attrs
66+
67+
@transaction.atomic
68+
def create(self, validated_data):
69+
# Remove the non-model field 'prefix' and use 'prefix_instance' instead
70+
prefix_instance = validated_data.pop('prefix_instance', None)
71+
validated_data.pop('prefix')
72+
authorized_group_names = validated_data.pop('authorized_groups', [])
73+
authorized_usernames = validated_data.pop('authorized_users', [])
74+
75+
bco_instance = Bco.objects.create(**validated_data, prefix=prefix_instance, last_update=timezone.now())
76+
77+
# Set ManyToMany relations
78+
if authorized_group_names:
79+
authorized_groups = Group.objects.filter(name__in=authorized_group_names)
80+
bco_instance.authorized_groups.set(authorized_groups)
81+
82+
if authorized_usernames:
83+
authorized_users = User.objects.filter(username__in=authorized_usernames)
84+
bco_instance.authorized_users.set(authorized_users)
85+
86+
return bco_instance

‎biocompute/urls.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,12 @@
11
# biocompute/urls.py
2+
"""BioCompute URLs
3+
"""
24

35
from django.urls import path
6+
from biocompute.apis import (
7+
DraftsCreateApi
8+
)
49

510
urlpatterns = [
6-
11+
path("objects/drafts/create/", DraftsCreateApi.as_view())
712
]

‎config/services.py‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/usr/bin/env python3
2+
# config/services.py
3+
4+
"""DB Level Services
5+
6+
Service functiontions for the entire DB
7+
"""
8+
9+
def legacy_api_converter(data:dict) ->dict:
10+
"""Legacy API converter
11+
12+
Used to remove the `POST_` object from requests.
13+
"""
14+
15+
_, new_data = data.popitem()
16+
return new_data

‎config/urls.py‎

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,57 @@
22
33
Top level URL configuration for BCO DB. See `api.urls` for APIs
44
"""
5+
import configparser
6+
from django.conf import settings
57
from django.contrib import admin
6-
from django.urls import path, include
8+
from django.urls import path, include, re_path
9+
from drf_yasg.views import get_schema_view
10+
from drf_yasg import openapi
11+
from rest_framework import permissions
712
from rest_framework_jwt.views import obtain_jwt_token, verify_jwt_token
813

14+
# Load the server config file.
15+
server_config = configparser.ConfigParser()
16+
server_config.read(settings.BASE_DIR + "/server.conf")
17+
18+
PUBLISH_ONLY = server_config["PUBLISHONLY"]["publishonly"]
19+
VERSION = server_config["VERSION"]["version"]
20+
21+
ShcemaView = get_schema_view(
22+
openapi.Info(
23+
title="BioCompute Object Data Base API (BCODB API)",
24+
default_version=VERSION,
25+
description="A web application that can be used to create, store and "
26+
"edit BioCompute objects based on BioCompute schema described "
27+
"in the BCO specification document.",
28+
terms_of_service="https://github.com/biocompute-objects/bco_api/blob/master/LICENSE",
29+
contact=openapi.Contact(email="[email protected]"),
30+
license=openapi.License(name="MIT License"),
31+
),
32+
public=True,
33+
permission_classes=(permissions.AllowAny,),
34+
)
35+
936
urlpatterns = [
37+
re_path(
38+
r"^api/doc(?P<format>\.json|\.yaml)$",
39+
ShcemaView.without_ui(cache_timeout=0),
40+
name="schema-json",
41+
),
42+
path(
43+
"api/docs/",
44+
ShcemaView.with_ui("swagger", cache_timeout=0),
45+
name="schema-swagger-ui",
46+
),
47+
path(
48+
"api/redocs/",
49+
ShcemaView.with_ui("redoc", cache_timeout=0),
50+
name="schema-redoc",
51+
),
1052
path("api/admin/", admin.site.urls),
1153
path("api/token/", obtain_jwt_token),
1254
path("api/verify/", verify_jwt_token),
1355
path("api/", include("authentication.urls")),
1456
path("api/", include("search.urls")),
57+
path("api/", include("biocompute.urls")),
1558
]

‎docs/refactor.md‎

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,41 @@
1-
# Notes for refactor
1+
# Planned Changes for 24.04 release
22

3-
## Changed items
4-
- new_user -> NewUser
3+
## Proposed changes
4+
5+
### Provide `One Click` examples that work for Swagger
6+
- GlyGen and ARGOS do this already.
7+
8+
### simplify API models and processing
9+
- previous model was based on multiple DB requests per object and each request could have buld sumissions
10+
- We still want bulk submissions but validations and permissions should be checked by classes and serializers before pining DB
11+
#### Examples:
12+
- POST_api_objects_draft_create
13+
14+
### Handeling what will become `Legacy` requests
15+
1. maintain the old code, and not publicize it
16+
2. develope converter functions to process
17+
18+
19+
### Refactor the groups user permissions
20+
- previous model was based on additional objects for Groups and Users, This required the use of `signals` and meant that there were many additional objects in the DB each time a new user was created. This also led to dependancy issues which prohibits deleting anything.
21+
- propose to elimiate the extra models
22+
23+
### Refactor the Prefix permission system
24+
- Prefix required it's own two groups and the creation of 5 permissions for each prefix. Look up for authentication was time consuming and taxing on DB. Users also had no idea how to use the system, just what to do to make it work.
25+
- Propose to add `authorized groups` to the prefix model. if it is empty then anyone can use it. If populated than only those in list can use it
26+
27+
### Refactor the BCO permission system
28+
- same situation as prefix
529

630
## Items to look at later
731
- `authentication.apis.RegisterUserNoVerificationAPI` has no swagger or tests
832
- fix email and secrets
9-
<<<<<<< Local Changes
1033
- install a `test_template` for Swagger responses
1134
- provide example values that are usable for testing APIs.
1235
- certifying key for prefix as a JWT?
1336
- owner = models.ForeignKey(
1437
User,
1538
on_delete=models.CASCADE,
16-
39+
- need tests for token
40+
- unwanted swagger endpoints
1741
- need tests for token

‎prefix/urls.py‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
1-
"""Prefix Admin Pannel
1+
#!/usr/bin/env python3
2+
3+
"""Prefix URLs
24
"""
35

4-
from django.contrib import admin
5-
from prefix.models import Prefix
66

7-
admin.site.register(Prefix)
7+
from biocompute.apis import DraftsCreateApi
8+

0 commit comments

Comments
 (0)