-
Notifications
You must be signed in to change notification settings - Fork 539
[silentpush-enrichment] Verify connector #5431
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,16 @@ | ||
| OPENCTI_TOKEN=changeme | ||
| SILENTPUSH_API_KEY=YOUR-SILENT-PUSH-API-KEY | ||
| # Generic parameters (connection with OpenCTI) | ||
| OPENCTI_URL=http://opencti:8080 | ||
| OPENCTI_TOKEN=CHANGEME | ||
|
|
||
| # Common parameters for connectors | ||
| # CONNECTOR_ID=9955194d-d000-4507-bc9d-19f851c20de9 | ||
| # CONNECTOR_NAME=Silent Push - Enrichment | ||
| # CONNECTOR_SCOPE=Indicator,URL,IPv4-Addr,IPv6-Addr,Domain-Name | ||
| # CONNECTOR_LOG_LEVEL=error | ||
| # CONNECTOR_AUTO=false | ||
|
|
||
| # Custom parameters for connector-silentpush | ||
| # SILENTPUSH_API_BASE_URL=https://app.silentpush.com/api/v2/ | ||
| SILENTPUSH_API_KEY=CHANGEME | ||
| # SILENTPUSH_MAX_TLP_LEVEL=TLP:CLEAR # Available values: TLP:CLEAR, TLP:WHITE, TLP:GREEN, TLP:AMBER, TLP:AMBER+STRICT, TLP:RED | ||
| # SILENTPUSH_VERIFY_CERT=true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,23 +1,16 @@ | ||
| FROM python:3.12-alpine as silentpush-enrichment | ||
|
|
||
| LABEL authors="[email protected]" | ||
|
|
||
| ENV CONNECTOR_TYPE=INTERNAL_ENRICHMENT | ||
| ENV CONNECTOR_SCOPE=Indicator,URL,IPv4-Addr,IPv6-Addr,Domain-Name,Hostname | ||
| FROM python:3.12-alpine | ||
|
|
||
| # Copy the connector | ||
| COPY src /opt/silentpush-enrichment | ||
| COPY src /opt/opencti-connector-silentpush-enrichment | ||
| WORKDIR /opt/opencti-connector-silentpush-enrichment | ||
|
|
||
| # Install Python modules | ||
| # hadolint ignore=DL3003 | ||
| RUN apk update && apk upgrade && \ | ||
| apk --no-cache add git build-base libmagic libffi-dev libxml2-dev libxslt-dev | ||
|
|
||
| RUN cd /opt/silentpush-enrichment && \ | ||
| RUN cd /opt/opencti-connector-silentpush-enrichment && \ | ||
| pip3 install --no-cache-dir -r requirements.txt && \ | ||
| apk del git build-base | ||
|
|
||
| # Expose and entrypoint | ||
| COPY entrypoint.sh / | ||
| RUN chmod +x /entrypoint.sh | ||
| ENTRYPOINT ["/entrypoint.sh"] | ||
| CMD ["python", "main.py"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,45 +28,17 @@ We apply unique behavioral fingerprints to attacker activity and search across o | |
|
|
||
| ### Requirements | ||
|
|
||
| - OpenCTI Platform >= 6.4.2 | ||
| - Python >= 3.11 | ||
| - OpenCTI Platform >= 6.8.13 | ||
| - [`pycti`](https://pypi.org/project/pycti/) library matching your OpenCTI version | ||
| - [`connectors-sdk`](https://github.com/OpenCTI-Platform/connectors.git@master#subdirectory=connectors-sdk) library matching your OpenCTI version | ||
|
|
||
| ## Configuration variables | ||
|
|
||
| There are a number of configuration options, which are set either in `docker-compose.yml` (for Docker) or | ||
| in `config.yml` (for manual deployment). | ||
| Find all the configuration variables available here: [Connector Configurations](./__metadata__/CONNECTOR_CONFIG_DOC.md) | ||
|
|
||
| ### OpenCTI environment variables | ||
|
|
||
| Below are the parameters you'll need to set for OpenCTI: | ||
|
|
||
| | Parameter | config.yml | Docker environment variable | Mandatory | Description | | ||
| |---------------|------------|-----------------------------|-----------|------------------------------------------------------| | ||
| | OpenCTI URL | url | `OPENCTI_URL` | Yes | The URL of the OpenCTI platform. | | ||
| | OpenCTI Token | token | `OPENCTI_TOKEN` | Yes | The default admin token set in the OpenCTI platform. | | ||
|
|
||
| ### Base connector environment variables | ||
|
|
||
| Below are the parameters you'll need to set for running the connector properly: | ||
|
|
||
| | Parameter | config.yml | Docker environment variable | Default | Mandatory | Description | | ||
| |-----------------|-----------------|-----------------------------|---------|-----------|------------------------------------------------------------------------------------------| | ||
| | Connector ID | id | `CONNECTOR_ID` | / | Yes | A unique `UUIDv4` identifier for this connector instance. | | ||
| | Connector Type | type | `CONNECTOR_TYPE` | | Yes | Should always be set to `INTERNAL_ENRICHMENT` for this connector. | | ||
| | Connector Name | name | `CONNECTOR_NAME` | | Yes | Name of the connector. | | ||
| | Connector Scope | scope | `CONNECTOR_SCOPE` | | Yes | The scope or type of data the connector is importing, either a MIME type or Stix Object. | | ||
| | Log Level | log_level | `CONNECTOR_LOG_LEVEL` | info | Yes | Determines the verbosity of the logs. Options are `debug`, `info`, `warn`, or `error`. | | ||
| | Connector Auto | connector_auto | `CONNECTOR_AUTO` | True | Yes | Must be `true` or `false` to enable or disable auto-enrichment of observables | | ||
|
|
||
| ### Connector extra parameters environment variables | ||
|
|
||
| Below are the parameters you'll need to set for the connector: | ||
|
|
||
| | Parameter | config.yml | Docker environment variable | Default | Mandatory | Description | | ||
| |-----------------------------------|--------------|-----------------------------|------------------------------------|-----------|-------------------------------------------------------------------------| | ||
| | API base URL | api_base_url | SILENTPUSH_API_BASE_URL | https://app.silentpush.com/api/v1/ | No | The Silent Push API URL | | ||
| | API key | api_key | SILENTPUSH_API_KEY | | Yes | You need an API key, sign up at https://explore.silentpush.com/register | | ||
| | Signed or self signed Certificate | verify_cert | SILENTPUSH_VERIFY_CERT | True | No | | | ||
| | TLP classification | max_tlp | | | No | | | ||
| _The `opencti` and `connector` options in the `docker-compose.yml` and `config.yml` are the same as for any other connector. | ||
| For more information regarding these variables, please refer to [OpenCTI's documentation on connectors](https://docs.opencti.io/latest/deployment/connectors/)._ | ||
|
|
||
| ## Deployment | ||
|
|
||
|
|
@@ -106,7 +78,7 @@ Install the required python dependencies (preferably in a virtual environment): | |
| pip3 install -r requirements.txt | ||
| ``` | ||
|
|
||
| Then, start the connector from silentpush-enrichment/src: | ||
| Then, start the connector from `src` directory: | ||
|
|
||
| ```shell | ||
| python3 main.py | ||
|
|
@@ -128,15 +100,13 @@ download of data by re-running the connector. | |
| This connector enriches Domains, IPv4, IPv6 and URLs observables. | ||
| Also enriches indicators containing those types of observables. | ||
|
|
||
|
|
||
| ## Debugging | ||
|
|
||
| The connector can be debugged by setting the appropiate log level. | ||
| Note that logging messages can be added using `self.helper.connector_logger,{LOG_LEVEL}("Sample message")`, i. | ||
| e., `self.helper.connector_logger.error("An error message")`. | ||
|
|
||
| <!-- Any additional information to help future users debug and report detailed issues concerning this connector --> | ||
|
|
||
| ## Additional information | ||
|
|
||
| Anything needed please talk to us at [[email protected]](mailto:[email protected]) | ||
| Anything needed please talk to us at [[email protected]](mailto:[email protected]) | ||
19 changes: 19 additions & 0 deletions
19
internal-enrichment/silentpush-enrichment/__metadata__/CONNECTOR_CONFIG_DOC.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Connector Configurations | ||
|
|
||
| Below is an exhaustive enumeration of all configurable parameters available, each accompanied by detailed explanations of their purposes, default behaviors, and usage guidelines to help you understand and utilize them effectively. | ||
|
|
||
| ### Type: `object` | ||
|
|
||
| | Property | Type | Required | Possible values | Default | Description | | ||
| | -------- | ---- | -------- | --------------- | ------- | ----------- | | ||
| | OPENCTI_URL | `string` | ✅ | Format: [`uri`](https://json-schema.org/understanding-json-schema/reference/string#built-in-formats) | | The base URL of the OpenCTI instance. | | ||
| | OPENCTI_TOKEN | `string` | ✅ | string | | The API token to connect to OpenCTI. | | ||
| | SILENTPUSH_API_KEY | `string` | ✅ | Format: [`password`](https://json-schema.org/understanding-json-schema/reference/string#built-in-formats) | | API key for authentication. | | ||
| | CONNECTOR_NAME | `string` | | string | `"Silent Push - Enrichment"` | The name of the connector. | | ||
| | CONNECTOR_SCOPE | `array` | | string | `["Indicator", "IPv4-Addr", "IPv6-Addr", "Domain-Name", "Hostname", "URL"]` | The scope of the connector | | ||
| | CONNECTOR_LOG_LEVEL | `string` | | `debug` `info` `warn` `warning` `error` | `"error"` | The minimum level of logs to display. | | ||
| | CONNECTOR_TYPE | `const` | | `INTERNAL_ENRICHMENT` | `"INTERNAL_ENRICHMENT"` | | | ||
| | CONNECTOR_AUTO | `boolean` | | boolean | `false` | Whether the connector should run automatically when an entity is created or updated. | | ||
| | SILENTPUSH_API_BASE_URL | `string` | | Format: [`uri`](https://json-schema.org/understanding-json-schema/reference/string#built-in-formats) | `"https://app.silentpush.com/api/v2/"` | External API base URL. | | ||
| | SILENTPUSH_MAX_TLP | `string` | | `TLP:WHITE` `TLP:CLEAR` `TLP:GREEN` `TLP:AMBER` `TLP:AMBER+STRICT` `TLP:RED` | `"TLP:AMBER"` | Max TLP level of the entities to enrich. | | ||
| | SILENTPUSH_VERIFY_CERT | `boolean` | | boolean | `true` | Whether to verify SSL certificates when connecting to the API. | |
98 changes: 98 additions & 0 deletions
98
internal-enrichment/silentpush-enrichment/__metadata__/connector_config_schema.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "$id": "https://www.filigran.io/connectors/silentpush-enrichment_config.schema.json", | ||
| "type": "object", | ||
| "properties": { | ||
| "OPENCTI_URL": { | ||
| "description": "The base URL of the OpenCTI instance.", | ||
| "format": "uri", | ||
| "maxLength": 2083, | ||
| "minLength": 1, | ||
| "type": "string" | ||
| }, | ||
| "OPENCTI_TOKEN": { | ||
| "description": "The API token to connect to OpenCTI.", | ||
| "type": "string" | ||
| }, | ||
| "CONNECTOR_NAME": { | ||
| "default": "Silent Push - Enrichment", | ||
| "description": "The name of the connector.", | ||
| "type": "string" | ||
| }, | ||
| "CONNECTOR_SCOPE": { | ||
| "default": [ | ||
| "Indicator", | ||
| "IPv4-Addr", | ||
| "IPv6-Addr", | ||
| "Domain-Name", | ||
| "Hostname", | ||
| "URL" | ||
| ], | ||
| "description": "The scope of the connector", | ||
| "items": { | ||
| "type": "string" | ||
| }, | ||
| "type": "array" | ||
| }, | ||
| "CONNECTOR_LOG_LEVEL": { | ||
| "default": "error", | ||
| "description": "The minimum level of logs to display.", | ||
| "enum": [ | ||
| "debug", | ||
| "info", | ||
| "warn", | ||
| "warning", | ||
| "error" | ||
| ], | ||
| "type": "string" | ||
| }, | ||
| "CONNECTOR_TYPE": { | ||
| "const": "INTERNAL_ENRICHMENT", | ||
| "default": "INTERNAL_ENRICHMENT", | ||
| "type": "string" | ||
| }, | ||
| "CONNECTOR_AUTO": { | ||
| "default": false, | ||
| "description": "Whether the connector should run automatically when an entity is created or updated.", | ||
| "type": "boolean" | ||
| }, | ||
| "SILENTPUSH_API_BASE_URL": { | ||
| "default": "https://app.silentpush.com/api/v2/", | ||
| "description": "External API base URL.", | ||
| "format": "uri", | ||
| "maxLength": 2083, | ||
| "minLength": 1, | ||
| "type": "string" | ||
| }, | ||
| "SILENTPUSH_API_KEY": { | ||
| "description": "API key for authentication.", | ||
| "format": "password", | ||
| "type": "string", | ||
| "writeOnly": true | ||
| }, | ||
| "SILENTPUSH_MAX_TLP": { | ||
| "default": "TLP:AMBER", | ||
| "description": "Max TLP level of the entities to enrich.", | ||
| "enum": [ | ||
| "TLP:WHITE", | ||
| "TLP:CLEAR", | ||
| "TLP:GREEN", | ||
| "TLP:AMBER", | ||
| "TLP:AMBER+STRICT", | ||
| "TLP:RED" | ||
| ], | ||
| "type": "string" | ||
| }, | ||
| "SILENTPUSH_VERIFY_CERT": { | ||
| "default": true, | ||
| "description": "Whether to verify SSL certificates when connecting to the API.", | ||
| "type": "boolean" | ||
| } | ||
| }, | ||
| "required": [ | ||
| "OPENCTI_URL", | ||
| "OPENCTI_TOKEN", | ||
| "SILENTPUSH_API_KEY" | ||
| ], | ||
| "additionalProperties": true | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
16 changes: 16 additions & 0 deletions
16
internal-enrichment/silentpush-enrichment/config.yml.sample
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| opencti: | ||
| url: 'http://opencti:8080' | ||
| token: 'changeme' | ||
|
|
||
| connector: | ||
| # id: '9955194d-d000-4507-bc9d-19f851c20de9' | ||
| # name: 'Silent Push - Enrichment' | ||
| # scope: 'Indicator,IPv4-Addr,IPv6-Addr,Domain-Name,Hostname,URL' | ||
| # log_level: 'error' | ||
| # auto: false # Enable/disable auto-enrichment of observables | ||
|
|
||
| silentpush: | ||
| # api_base_url: 'https://app.silentpush.com/api/v2/' | ||
| api_key: 'YOUR-SILENT-PUSH-API-KEY' | ||
| # verify_cert: true | ||
| # max_tlp: "TLP:AMBER" # Available values: TLP:CLEAR, TLP:WHITE, TLP:GREEN, TLP:AMBER, TLP:AMBER+STRICT, TLP:RED |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
17 changes: 0 additions & 17 deletions
17
internal-enrichment/silentpush-enrichment/src/config.yml.sample
This file was deleted.
Oops, something went wrong.
7 changes: 7 additions & 0 deletions
7
internal-enrichment/silentpush-enrichment/src/connector/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| from .connector import SilentpushConnector | ||
| from .settings import ConnectorSettings | ||
|
|
||
| __all__ = [ | ||
| "SilentpushConnector", | ||
| "ConnectorSettings", | ||
| ] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.