Skip to content

Commit 5e120f6

Browse files
sjer-akamaiCopilot
sjer-akamai
and
Copilot
authored
Aclp logs support (#686)
* Add destination API models for monitor objects * Add destination API support in monitor group * Destination.history client call fix * `access_key_secret` parameter name fixes * Add unit tests * Rename Destination to LogsDestination * Add integration tests * Typo fix * Integration tests tweaks * Documentation tweaks * Add negative integration test cases * Fix assertion * Formatting tweaks * ACLP Logs stream - add model and group * ACLP Logs Stream - unit tests * ACLP Logs stream - add integration tests * ACLP Logs Stream - ensure update reverts on failed assertions * ACLP Logs Stream - Formatting tweaks * ACLP Logs Stream - copilot review tweaks * ACLP Logs Stream - review tweaks - pt. 2 * ACLP Logs Stream - review tweaks - pt. 3 * ACLP Logs Stream - remove redundant comma * ACLP Logs Stream - Merge save() method integration tests, add skip guard if stream already exists * Update test/unit/objects/monitor_test.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * ACLP Logs Stream - refactor invalid destination test to use session-scoped fixture for deterministic execution * ACLP Logs - update assertion - stream status not tracked by version history * ACLP Logs - update e2e tests workflow * Update .github/workflows/e2e-test.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update linode_api4/objects/monitor.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * ACLP Logs - Clarify LogsDestinationType supporting only one value * ACLP Logs - Fix formatting * ACLP Logs - Extend destination API with new updates * ACLP Logs - Extend Stream API with lke_audit_logs_type, reorder tests * ACLP Logs - fix __all__ statement * ACLP Logs - Stabilize stream tests * ACLP Logs - Reduce integration tests runtime - remove unnecessary updates * ACLP Logs - Remove integration tests capability check * ACLP Logs - Stabilize integration tests - loosen status update assertions * ACLP Logs - Increase timeout for destinations and buckets cleanup * ACLP Logs - Increase timeout for destinations and buckets cleanup * ACLP Logs - stabilize integration tests - fix teardown * ACLP Logs - rename fixture --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent fcc41e5 commit 5e120f6

13 files changed

Lines changed: 1945 additions & 4 deletions

‎.github/workflows/e2e-test-pr.yml‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@ on:
22
pull_request:
33
workflow_dispatch:
44
inputs:
5+
run_aclp_logs_stream_tests:
6+
description: 'Set this parameter to "true" to run ACLP logs stream related test cases'
7+
required: false
8+
default: 'false'
9+
type: choice
10+
options:
11+
- 'true'
12+
- 'false'
513
run_db_fork_tests:
614
description: 'Set this parameter to "true" to run fork database related test cases'
715
required: false
@@ -104,7 +112,7 @@ jobs:
104112
run: |
105113
timestamp=$(date +'%Y%m%d%H%M')
106114
report_filename="${timestamp}_sdk_test_report.xml"
107-
make test-int RUN_DB_FORK_TESTS=${{ github.event.inputs.run_db_fork_tests }} RUN_DB_TESTS=${{ github.event.inputs.run_db_tests }} TEST_ARGS="--junitxml=${report_filename}" TEST_SUITE="${{ github.event.inputs.test_suite }}"
115+
make test-int RUN_DB_FORK_TESTS=${{ github.event.inputs.run_db_fork_tests }} RUN_DB_TESTS=${{ github.event.inputs.run_db_tests }} RUN_ACLP_LOGS_STREAM_TESTS=${{ github.event.inputs.run_aclp_logs_stream_tests }} TEST_ARGS="--junitxml=${report_filename}" TEST_SUITE="${{ github.event.inputs.test_suite }}"
108116
env:
109117
LINODE_TOKEN: ${{ secrets.LINODE_TOKEN }}
110118

‎.github/workflows/e2e-test.yml‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,14 @@ name: Integration Tests
33
on:
44
workflow_dispatch:
55
inputs:
6+
run_aclp_logs_stream_tests:
7+
description: 'Set this parameter to "true" to run ACLP logs stream related test cases'
8+
required: false
9+
default: 'false'
10+
type: choice
11+
options:
12+
- 'true'
13+
- 'false'
614
run_db_fork_tests:
715
description: 'Set this parameter to "true" to run fork database related test cases'
816
required: false
@@ -99,7 +107,7 @@ jobs:
99107
run: |
100108
timestamp=$(date +'%Y%m%d%H%M')
101109
report_filename="${timestamp}_sdk_test_report.xml"
102-
make test-int RUN_DB_FORK_TESTS=${{ github.event.inputs.run_db_fork_tests }} RUN_DB_TESTS=${{ github.event.inputs.run_db_tests }} TEST_SUITE="${{ github.event.inputs.test_suite }}" TEST_ARGS="--junitxml=${report_filename}"
110+
make test-int RUN_DB_FORK_TESTS=${{ github.event.inputs.run_db_fork_tests }} RUN_DB_TESTS=${{ github.event.inputs.run_db_tests }} RUN_ACLP_LOGS_STREAM_TESTS=${{ github.event.inputs.run_aclp_logs_stream_tests }} TEST_SUITE="${{ github.event.inputs.test_suite }}" TEST_ARGS="--junitxml=${report_filename}"
103111
env:
104112
LINODE_TOKEN: ${{ env.LINODE_TOKEN }}
105113

‎linode_api4/groups/monitor.py‎

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,21 @@
88
AlertDefinition,
99
AlertDefinitionEntity,
1010
AlertScope,
11+
LogsDestination,
12+
LogsDestinationType,
13+
LogsStream,
14+
LogsStreamStatus,
15+
LogsStreamType,
1116
MonitorDashboard,
1217
MonitorMetricsDefinition,
1318
MonitorService,
1419
MonitorServiceToken,
1520
)
21+
from linode_api4.objects.monitor import (
22+
AkamaiObjectStorageLogsDestinationDetails,
23+
CustomHTTPSLogsDestinationDetails,
24+
LogsStreamDetails,
25+
)
1626

1727
__all__ = [
1828
"MonitorGroup",
@@ -332,3 +342,206 @@ def alert_definition_entities(
332342
*filters,
333343
endpoint=endpoint,
334344
)
345+
346+
def destinations(self, *filters) -> PaginatedList:
347+
"""
348+
List available logs destinations.
349+
350+
Returns a paginated collection of :class:`LogsDestination` objects which
351+
describe logs destinations. By default, this method returns all available
352+
destinations; you can supply optional filter expressions to restrict
353+
the results, for example::
354+
355+
# Get destinations created by username and with id 111
356+
destinations = client.monitor.destinations(LogsDestination.created_by == "username",
357+
LogsDestination.id == 111)
358+
359+
API Documentation: https://techdocs.akamai.com/linode-api/reference/get-destinations
360+
361+
:param filters: Any number of filters to apply to this query.
362+
See :doc:`Filtering Collections</linode_api4/objects/filtering>`
363+
for more details on filtering.
364+
365+
:returns: A list of :class:`LogsDestination` objects matching the query.
366+
:rtype: PaginatedList of LogsDestination
367+
"""
368+
369+
return self.client._get_and_filter(LogsDestination, *filters)
370+
371+
def destination_create(
372+
self,
373+
label: str,
374+
type: Union[LogsDestinationType, str],
375+
details: Union[
376+
AkamaiObjectStorageLogsDestinationDetails,
377+
CustomHTTPSLogsDestinationDetails,
378+
],
379+
) -> LogsDestination:
380+
"""
381+
Creates a new :any:`LogsDestination` for logs on this account.
382+
383+
For an ``akamai_object_storage`` destination::
384+
385+
client = LinodeClient(TOKEN)
386+
387+
new_destination = client.monitor.destination_create(
388+
label="OBJ_logs_destination",
389+
type="akamai_object_storage",
390+
details=AkamaiObjectStorageLogsDestinationDetails(
391+
access_key_id="1ABCD23EFG4HIJKLMNO5",
392+
access_key_secret="1aB2CD3e4fgHi5JK6lmnop7qR8STU9VxYzabcdefHh",
393+
bucket_name="primary-bucket",
394+
host="primary-bucket-1.us-east-12.linodeobjects.com",
395+
path="audit-logs",
396+
)
397+
)
398+
399+
For a ``custom_https`` destination::
400+
401+
new_destination = client.monitor.destination_create(
402+
label="custom_logs_destination",
403+
type="custom_https",
404+
details=CustomHTTPSLogsDestinationDetails(
405+
endpoint_url="https://my-site.com/log-storage/basicAuth",
406+
authentication=DestinationAuthentication(
407+
type="basic",
408+
details=BasicAuthenticationDetails(
409+
basic_authentication_user="user",
410+
basic_authentication_password="pass",
411+
),
412+
),
413+
data_compression="gzip",
414+
content_type="application/json",
415+
)
416+
)
417+
418+
API Documentation: https://techdocs.akamai.com/linode-api/reference/post-destination
419+
420+
:param label: The name for this logs destination.
421+
:type label: str
422+
:param type: The type of destination — ``akamai_object_storage`` or ``custom_https``.
423+
:type type: str or LogsDestinationType
424+
:param details: A typed details object matching the destination type.
425+
Use :class:`AkamaiObjectStorageLogsDestinationDetails` for
426+
``akamai_object_storage`` or :class:`CustomHTTPSLogsDestinationDetails`
427+
for ``custom_https``.
428+
:type details: AkamaiObjectStorageLogsDestinationDetails or CustomHTTPSLogsDestinationDetails
429+
430+
:returns: The newly created logs destination.
431+
:rtype: LogsDestination
432+
"""
433+
434+
params = {
435+
"label": label,
436+
"type": type,
437+
"details": details.dict,
438+
}
439+
440+
result = self.client.post("/monitor/streams/destinations", data=params)
441+
442+
if "id" not in result:
443+
raise UnexpectedResponseError(
444+
"Unexpected response when creating destination!",
445+
json=result,
446+
)
447+
448+
return LogsDestination(self.client, result["id"], result)
449+
450+
def streams(self, *filters) -> PaginatedList:
451+
"""
452+
List available logs streams.
453+
454+
Returns a paginated collection of :class:`LogsStream` objects which
455+
describe logs streams. By default, this method returns all available
456+
streams; you can supply optional filter expressions to restrict
457+
the results, for example::
458+
459+
# Get all streams with status ``provisioning``
460+
provisioning_streams = client.monitor.streams(LogsStream.status == "provisioning")
461+
462+
API Documentation: https://techdocs.akamai.com/linode-api/reference/get-streams
463+
464+
:param filters: Any number of filters to apply to this query.
465+
See :doc:`Filtering Collections</linode_api4/objects/filtering>`
466+
for more details on filtering.
467+
:returns: A list of :class:`LogsStream` objects matching the query.
468+
:rtype: PaginatedList of LogsStream
469+
"""
470+
471+
return self.client._get_and_filter(LogsStream, *filters)
472+
473+
def stream_create(
474+
self,
475+
destinations: list[int],
476+
label: str,
477+
type: Union[LogsStreamType, str],
478+
status: Optional[Union[LogsStreamStatus, str]] = None,
479+
details: Optional[LogsStreamDetails] = None,
480+
) -> LogsStream:
481+
"""
482+
Creates a new :any:`LogsStream` for logs on this account. For example::
483+
484+
client = LinodeClient(TOKEN)
485+
486+
# audit_logs stream (no details required)
487+
new_stream = client.monitor.stream_create(
488+
destinations=[1234],
489+
label="Linode_services",
490+
status="active",
491+
type="audit_logs"
492+
)
493+
494+
# lke_audit_logs stream with specific clusters
495+
lke_stream = client.monitor.stream_create(
496+
destinations=[1234],
497+
label="LKE_audit_stream",
498+
type="lke_audit_logs",
499+
details=LogsStreamDetails(
500+
cluster_ids=[1111, 2222],
501+
is_auto_add_all_clusters_enabled=False,
502+
)
503+
)
504+
505+
API Documentation: https://techdocs.akamai.com/linode-api/reference/post-stream
506+
507+
:param destinations: The unique identifier for the sync point that will receive logs data.
508+
Run the List destinations operation and store the id values for each applicable destination.
509+
At the moment only single destination is supported.
510+
:type destinations: list[int]
511+
:param label: The name of the stream. This is used for display purposes in Akamai Cloud Manager.
512+
:type label: str
513+
:param type: The type of stream — ``audit_logs`` for Linode control plane logs,
514+
or ``lke_audit_logs`` for LKE enterprise cluster audit logs.
515+
:type type: str or LogsStreamType
516+
:param status: (Optional) The availability status of the stream. Possible values are: ``active``, ``inactive``.
517+
Defaults to ``active``.
518+
:type status: str
519+
:param details: (Optional) Additional stream details. Only applicable for
520+
``lke_audit_logs`` streams. Omit for ``audit_logs`` streams.
521+
:type details: LogsStreamDetails
522+
523+
:returns: The newly created logs stream.
524+
:rtype: LogsStream
525+
"""
526+
527+
params = {
528+
"label": label,
529+
"type": type,
530+
"destinations": destinations,
531+
}
532+
533+
if status is not None:
534+
params["status"] = status
535+
536+
if details is not None:
537+
params["details"] = details.dict
538+
539+
result = self.client.post("/monitor/streams", data=params)
540+
541+
if "id" not in result:
542+
raise UnexpectedResponseError(
543+
"Unexpected response when creating logs stream!",
544+
json=result,
545+
)
546+
547+
return LogsStream(self.client, result["id"], result)

0 commit comments

Comments
 (0)