Skip to content

Conversation

de-sh
Copy link
Contributor

@de-sh de-sh commented Jan 3, 2025

Fixes #765.

Description

Audit logs are sent to another Parseable instance everytime a request is received over http or when new logs are received over kafka.

While we did explore the option, we chose to perform audit logging without relying on tracing, given the complexity of the setup.

In the PR we have been as careful as possible to not cause unnecessary allocations if we don't end up logging anyways.


This PR has:

  • been tested to ensure log ingestion and log query works.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.

@coveralls
Copy link

coveralls commented Jan 3, 2025

Pull Request Test Coverage Report for Build 12627709898

Details

  • 0 of 271 (0.0%) changed or added relevant lines in 8 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage decreased (-0.2%) to 12.331%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/handlers/http/modal/mod.rs 0 1 0.0%
src/rbac/mod.rs 0 3 0.0%
src/rbac/map.rs 0 5 0.0%
src/handlers/http/middleware.rs 0 7 0.0%
src/kafka.rs 0 10 0.0%
src/cli.rs 0 27 0.0%
src/handlers/http/audit.rs 0 74 0.0%
src/audit.rs 0 144 0.0%
Files with Coverage Reduction New Missed Lines %
src/rbac/mod.rs 1 0.0%
Totals Coverage Status
Change from base Build 12627677014: -0.2%
Covered Lines: 2467
Relevant Lines: 20007

💛 - Coveralls

@de-sh de-sh changed the title feat: perform audit logging from the auth middleware feat: audit logging Jan 3, 2025
Copy link
Contributor

@hippalus hippalus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Few comments

@nitisht
Copy link
Member

nitisht commented Jan 4, 2025

proposed format:

{
  "parseable_server_version": "1.6.3",
  "parseable_server_deploymentId": "01JGR7PS9K28JRKNS4V6F2WNAK",
  "audit_version": "V1",
  "audit_generation_timestamp": "2025-01-04T14:16:28.483+05:30",
  "audit_id": "01JGR7X443QX5N45B4ZPNN5QEG",
  "actor_authorizationMethod": "Basic Auth",
  "actor_remote_host": "127.0.0.1",
  "actor_username": "admin",
  "actor_user_agent": "PostmanRuntime/7.43.0",
  "request_headers_accept": "*/*",
  "request_headers_accept-encoding": "gzip, deflate, br",
  "request_headers_connection": "keep-alive",
  "request_headers_content-length": "100",
  "request_headers_content-type": "application/json",
  "request_headers_host": "localhost:8000",
  "request_headers_postman-token": "6a1bd4ae-0395-4e5c-90df-b7b5105bf0b5",
  "request_method": "POST",
  "request_start_time": "2025-01-04T14:16:28.480+05:30",
  "request_end_time": "2025-01-04T14:16:28.483+05:30",
  "request_path": "/api/v1/ingest",
  "request_protocol": "http",
  "request_body_size": "1234",
  "response_status_code": 400,
  "response_error_message": "",
  "response_body_size": "1234",
  "stream": "app",
}

@de-sh de-sh requested a review from hippalus January 5, 2025 17:28
@nitisht nitisht merged commit 04a92b0 into parseablehq:main Jan 6, 2025
9 checks passed
@de-sh de-sh deleted the audit-logging branch January 6, 2025 09:39
parmesant pushed a commit to parmesant/parseable that referenced this pull request Jan 13, 2025
Add remote audit logging with the below schema:

```
{
  "parseable_server_version": "1.6.3",
  "parseable_server_deploymentId": "01JGR7PS9K28JRKNS4V6F2WNAK",
  "audit_version": "V1",
  "audit_generation_timestamp": "2025-01-04T14:16:28.483+05:30",
  "audit_id": "01JGR7X443QX5N45B4ZPNN5QEG",
  "actor_authorizationMethod": "Basic Auth",
  "actor_remote_host": "127.0.0.1",
  "actor_username": "admin",
  "actor_user_agent": "PostmanRuntime/7.43.0",
  "request_headers_accept": "*/*",
  "request_headers_accept-encoding": "gzip, deflate, br",
  "request_headers_connection": "keep-alive",
  "request_headers_content-length": "100",
  "request_headers_content-type": "application/json",
  "request_headers_host": "localhost:8000",
  "request_headers_postman-token": "6a1bd4ae-0395-4e5c-90df-b7b5105bf0b5",
  "request_method": "POST",
  "request_start_time": "2025-01-04T14:16:28.480+05:30",
  "request_end_time": "2025-01-04T14:16:28.483+05:30",
  "request_path": "/api/v1/ingest",
  "request_protocol": "http",
  "request_body_size": "1234",
  "response_status_code": 400,
  "response_error_message": "",
  "response_body_size": "1234",
  "stream": "app",
}
```

Fixes parseablehq#765
---------

Signed-off-by: Devdutt Shenoi <[email protected]>
Co-authored-by: Nikhil Sinha <[email protected]>
@de-sh de-sh mentioned this pull request Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: audit logging for Parseable
5 participants