diff --git a/docs/add-extension/add-extension-layer-widget.asciidoc b/docs/add-extension/add-extension-layer-widget.asciidoc deleted file mode 100644 index d7d4e555..00000000 --- a/docs/add-extension/add-extension-layer-widget.asciidoc +++ /dev/null @@ -1,118 +0,0 @@ -++++ -
-
- - - - - - -
-
-++++ - -include::add-extension-layer.asciidoc[tag=console-{layer-section-type}] - -++++ -
- - - - - -
-++++ \ No newline at end of file diff --git a/docs/add-extension/add-extension-layer.asciidoc b/docs/add-extension/add-extension-layer.asciidoc deleted file mode 100644 index 2c9d9b28..00000000 --- a/docs/add-extension/add-extension-layer.asciidoc +++ /dev/null @@ -1,191 +0,0 @@ -// tag::console-extension-only[] - -To add a layer to a Lambda function through the AWS Management Console: - -1. Navigate to your function in the AWS Management Console -2. Scroll to the Layers section and click the _Add a layer_ button image:images/config-layer.png[image of layer configuration section in AWS Console] -3. Choose the _Specify an ARN_ radio button -4. Copy and paste the following ARN of the {apm-lambda-ext} layer in the _Specify an ARN_ text input: + -+++EXTENSION_ARN+++ -image:images/choose-a-layer.png[image of choosing a layer in AWS Console] -5. Click the _Add_ button - -// end::console-extension-only[] - -// tag::console-with-agent[] - -To add the layers to your Lambda function through the AWS Management Console: - -1. Navigate to your function in the AWS Management Console -2. Scroll to the Layers section and click the _Add a layer_ button image:images/config-layer.png[image of layer configuration section in AWS Console] -3. Choose the _Specify an ARN_ radio button -4. Copy and paste the following ARNs of the {apm-lambda-ext} layer and the APM agent layer in the _Specify an ARN_ text input: + -APM Extension layer: + -+++EXTENSION_ARN+++ + -APM agent layer: + -+++AGENT_ARN+++ -image:images/choose-a-layer.png[image of choosing a layer in AWS Console] -5. Click the _Add_ button - -// end::console-with-agent[] - -// tag::cli-extension-only[] - -To add the {apm-lambda-ext} Layer ARN through the AWS command line interface execute the following command: - -[source,bash] ----- -aws lambda update-function-configuration --function-name yourLambdaFunctionName \ ---layers EXTENSION_ARN ----- - -// end::cli-extension-only[] - -// tag::cli-with-agent[] - -To add the Layer ARNs of the {apm-lambda-ext} and the APM agent through the AWS command line interface execute the following command: - -[source,bash] ----- -aws lambda update-function-configuration --function-name yourLambdaFunctionName \ ---layers EXTENSION_ARN \ -AGENT_ARN ----- - -// end::cli-with-agent[] - -// tag::sam-extension-only[] - -In your SAM `template.yml` file add the {apm-lambda-ext} Layer ARN as follows: - -[source,yml] ----- -... -Resources: - yourLambdaFunction: - Type: AWS::Serverless::Function - Properties: - ... - Layers: - - EXTENSION_ARN -... ----- - -// end::sam-extension-only[] - -// tag::sam-with-agent[] - -In your SAM `template.yml` file add the Layer ARNs of the {apm-lambda-ext} and the APM agent as follows: - -[source,yml] ----- -... -Resources: - yourLambdaFunction: - Type: AWS::Serverless::Function - Properties: - ... - Layers: - - EXTENSION_ARN - - AGENT_ARN -... ----- - -// end::sam-with-agent[] - -// tag::serverless-extension-only[] - -In your `serverless.yml` file add the {apm-lambda-ext} Layer ARN to your function as follows: - -[source,yml] ----- -... -functions: - yourLambdaFunction: - handler: ... - layers: - - EXTENSION_ARN -... ----- - -// end::serverless-extension-only[] - -// tag::serverless-with-agent[] - -In your `serverless.yml` file add the Layer ARNs of the {apm-lambda-ext} and the APM agent to your function as follows: - -[source,yml] ----- -... -functions: - yourLambdaFunction: - handler: ... - layers: - - EXTENSION_ARN - - AGENT_ARN -... ----- - -// end::serverless-with-agent[] - -// tag::terraform-extension-only[] -To add the {apm-lambda-ext} Layer to your function add the ARN to the `layers` property in your Terraform file: - -[source,terraform] ----- -... -resource "aws_lambda_function" "your_lambda_function" { - ... - layers = ["EXTENSION_ARN"] -} -... ----- - -// end::terraform-extension-only[] - -// tag::terraform-with-agent[] -To add the{apm-lambda-ext} and the APM agent to your function add the ARNs to the `layers` property in your Terraform file: - -[source,terraform] ----- -... -resource "aws_lambda_function" "your_lambda_function" { - ... - layers = ["EXTENSION_ARN", "AGENT_ARN"] -} -... ----- - -// end::terraform-with-agent[] - -// tag::container-extension-only[] -To add the {apm-lambda-ext} to your container-based function extend the Dockerfile of your function image as follows: - -[source,Dockerfile] ----- -FROM docker.elastic.co/observability/apm-lambda-extension-IMAGE_ARCH:latest AS lambda-extension - -# FROM ... <-- this is the base image of your Lambda function - -COPY --from=lambda-extension /opt/elastic-apm-extension /opt/extensions/elastic-apm-extension - -# ... ----- -// end::container-extension-only[] - -// tag::container-with-agent[] -To add the {apm-lambda-ext} and the APM agent to your container-based function extend the Dockerfile of your function image as follows: - -[source,Dockerfile] ----- -FROM docker.elastic.co/observability/apm-lambda-extension-IMAGE_ARCH:latest AS lambda-extension -AGENT_IMPORT - -# FROM ... <-- this is the base image of your Lambda function - -COPY --from=lambda-extension /opt/elastic-apm-extension /opt/extensions/elastic-apm-extension -AGENT_COPY - -# ... ----- -// end::container-with-agent[] diff --git a/docs/docset.yml b/docs/docset.yml new file mode 100644 index 00000000..3e443564 --- /dev/null +++ b/docs/docset.yml @@ -0,0 +1,490 @@ +project: 'APM Architecture for AWS Lambda' +cross_links: + - apm-agent-java + - apm-agent-nodejs + - apm-agent-php + - apm-agent-python + - docs-content +toc: + - toc: reference + - toc: release-notes +subs: + ref: "https://www.elastic.co/guide/en/elasticsearch/reference/current" + ref-bare: "https://www.elastic.co/guide/en/elasticsearch/reference" + ref-8x: "https://www.elastic.co/guide/en/elasticsearch/reference/8.1" + ref-80: "https://www.elastic.co/guide/en/elasticsearch/reference/8.0" + ref-7x: "https://www.elastic.co/guide/en/elasticsearch/reference/7.17" + ref-70: "https://www.elastic.co/guide/en/elasticsearch/reference/7.0" + ref-60: "https://www.elastic.co/guide/en/elasticsearch/reference/6.0" + ref-64: "https://www.elastic.co/guide/en/elasticsearch/reference/6.4" + xpack-ref: "https://www.elastic.co/guide/en/x-pack/6.2" + logstash-ref: "https://www.elastic.co/guide/en/logstash/current" + kibana-ref: "https://www.elastic.co/guide/en/kibana/current" + kibana-ref-all: "https://www.elastic.co/guide/en/kibana" + beats-ref-root: "https://www.elastic.co/guide/en/beats" + beats-ref: "https://www.elastic.co/guide/en/beats/libbeat/current" + beats-ref-60: "https://www.elastic.co/guide/en/beats/libbeat/6.0" + beats-ref-63: "https://www.elastic.co/guide/en/beats/libbeat/6.3" + beats-devguide: "https://www.elastic.co/guide/en/beats/devguide/current" + auditbeat-ref: "https://www.elastic.co/guide/en/beats/auditbeat/current" + packetbeat-ref: "https://www.elastic.co/guide/en/beats/packetbeat/current" + metricbeat-ref: "https://www.elastic.co/guide/en/beats/metricbeat/current" + filebeat-ref: "https://www.elastic.co/guide/en/beats/filebeat/current" + functionbeat-ref: "https://www.elastic.co/guide/en/beats/functionbeat/current" + winlogbeat-ref: "https://www.elastic.co/guide/en/beats/winlogbeat/current" + heartbeat-ref: "https://www.elastic.co/guide/en/beats/heartbeat/current" + journalbeat-ref: "https://www.elastic.co/guide/en/beats/journalbeat/current" + ingest-guide: "https://www.elastic.co/guide/en/ingest/current" + fleet-guide: "https://www.elastic.co/guide/en/fleet/current" + apm-guide-ref: "https://www.elastic.co/guide/en/apm/guide/current" + apm-guide-7x: "https://www.elastic.co/guide/en/apm/guide/7.17" + apm-app-ref: "https://www.elastic.co/guide/en/kibana/current" + apm-agents-ref: "https://www.elastic.co/guide/en/apm/agent" + apm-android-ref: "https://www.elastic.co/guide/en/apm/agent/android/current" + apm-py-ref: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-py-ref-3x: "https://www.elastic.co/guide/en/apm/agent/python/3.x" + apm-node-ref-index: "https://www.elastic.co/guide/en/apm/agent/nodejs" + apm-node-ref: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-node-ref-1x: "https://www.elastic.co/guide/en/apm/agent/nodejs/1.x" + apm-rum-ref: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-dotnet-ref: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref: "https://www.elastic.co/guide/en/apm/agent/php/current" + apm-ios-ref: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-lambda-ref: "https://www.elastic.co/guide/en/apm/lambda/current" + apm-attacher-ref: "https://www.elastic.co/guide/en/apm/attacher/current" + docker-logging-ref: "https://www.elastic.co/guide/en/beats/loggingplugin/current" + esf-ref: "https://www.elastic.co/guide/en/esf/current" + kinesis-firehose-ref: "https://www.elastic.co/guide/en/kinesis/{{kinesis_version}}" + estc-welcome-current: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome-all: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions" + hadoop-ref: "https://www.elastic.co/guide/en/elasticsearch/hadoop/current" + stack-ref: "https://www.elastic.co/guide/en/elastic-stack/current" + stack-ref-67: "https://www.elastic.co/guide/en/elastic-stack/6.7" + stack-ref-68: "https://www.elastic.co/guide/en/elastic-stack/6.8" + stack-ref-70: "https://www.elastic.co/guide/en/elastic-stack/7.0" + stack-ref-80: "https://www.elastic.co/guide/en/elastic-stack/8.0" + stack-ov: "https://www.elastic.co/guide/en/elastic-stack-overview/current" + stack-gs: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + stack-gs-current: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + javaclient: "https://www.elastic.co/guide/en/elasticsearch/client/java-api/current" + java-api-client: "https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current" + java-rest: "https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current" + jsclient: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + jsclient-current: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + es-ruby-client: "https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current" + es-dotnet-client: "https://www.elastic.co/guide/en/elasticsearch/client/net-api/current" + es-php-client: "https://www.elastic.co/guide/en/elasticsearch/client/php-api/current" + es-python-client: "https://www.elastic.co/guide/en/elasticsearch/client/python-api/current" + defguide: "https://www.elastic.co/guide/en/elasticsearch/guide/2.x" + painless: "https://www.elastic.co/guide/en/elasticsearch/painless/current" + plugins: "https://www.elastic.co/guide/en/elasticsearch/plugins/current" + plugins-8x: "https://www.elastic.co/guide/en/elasticsearch/plugins/8.1" + plugins-7x: "https://www.elastic.co/guide/en/elasticsearch/plugins/7.17" + plugins-6x: "https://www.elastic.co/guide/en/elasticsearch/plugins/6.8" + glossary: "https://www.elastic.co/guide/en/elastic-stack-glossary/current" + upgrade_guide: "https://www.elastic.co/products/upgrade_guide" + blog-ref: "https://www.elastic.co/blog/" + curator-ref: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + curator-ref-current: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + metrics-ref: "https://www.elastic.co/guide/en/metrics/current" + metrics-guide: "https://www.elastic.co/guide/en/metrics/guide/current" + logs-ref: "https://www.elastic.co/guide/en/logs/current" + logs-guide: "https://www.elastic.co/guide/en/logs/guide/current" + uptime-guide: "https://www.elastic.co/guide/en/uptime/current" + observability-guide: "https://www.elastic.co/guide/en/observability/current" + observability-guide-all: "https://www.elastic.co/guide/en/observability" + siem-guide: "https://www.elastic.co/guide/en/siem/guide/current" + security-guide: "https://www.elastic.co/guide/en/security/current" + security-guide-all: "https://www.elastic.co/guide/en/security" + endpoint-guide: "https://www.elastic.co/guide/en/endpoint/current" + sql-odbc: "https://www.elastic.co/guide/en/elasticsearch/sql-odbc/current" + ecs-ref: "https://www.elastic.co/guide/en/ecs/current" + ecs-logging-ref: "https://www.elastic.co/guide/en/ecs-logging/overview/current" + ecs-logging-go-logrus-ref: "https://www.elastic.co/guide/en/ecs-logging/go-logrus/current" + ecs-logging-go-zap-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-go-zerolog-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-java-ref: "https://www.elastic.co/guide/en/ecs-logging/java/current" + ecs-logging-dotnet-ref: "https://www.elastic.co/guide/en/ecs-logging/dotnet/current" + ecs-logging-nodejs-ref: "https://www.elastic.co/guide/en/ecs-logging/nodejs/current" + ecs-logging-php-ref: "https://www.elastic.co/guide/en/ecs-logging/php/current" + ecs-logging-python-ref: "https://www.elastic.co/guide/en/ecs-logging/python/current" + ecs-logging-ruby-ref: "https://www.elastic.co/guide/en/ecs-logging/ruby/current" + ml-docs: "https://www.elastic.co/guide/en/machine-learning/current" + eland-docs: "https://www.elastic.co/guide/en/elasticsearch/client/eland/current" + eql-ref: "https://eql.readthedocs.io/en/latest/query-guide" + extendtrial: "https://www.elastic.co/trialextension" + wikipedia: "https://en.wikipedia.org/wiki" + forum: "https://discuss.elastic.co/" + xpack-forum: "https://discuss.elastic.co/c/50-x-pack" + security-forum: "https://discuss.elastic.co/c/x-pack/shield" + watcher-forum: "https://discuss.elastic.co/c/x-pack/watcher" + monitoring-forum: "https://discuss.elastic.co/c/x-pack/marvel" + graph-forum: "https://discuss.elastic.co/c/x-pack/graph" + apm-forum: "https://discuss.elastic.co/c/apm" + enterprise-search-ref: "https://www.elastic.co/guide/en/enterprise-search/current" + app-search-ref: "https://www.elastic.co/guide/en/app-search/current" + workplace-search-ref: "https://www.elastic.co/guide/en/workplace-search/current" + enterprise-search-node-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/enterprise-search-node/current" + enterprise-search-php-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/php/current" + enterprise-search-python-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/python/current" + enterprise-search-ruby-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/ruby/current" + elastic-maps-service: "https://maps.elastic.co" + integrations-docs: "https://docs.elastic.co/en/integrations" + integrations-devguide: "https://www.elastic.co/guide/en/integrations-developer/current" + time-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#time-units" + byte-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#byte-units" + apm-py-ref-v: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-node-ref-v: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-rum-ref-v: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref-v: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref-v: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref-v: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-ios-ref-v: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-dotnet-ref-v: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref-v: "https://www.elastic.co/guide/en/apm/agent/php/current" + ecloud: "Elastic Cloud" + esf: "Elastic Serverless Forwarder" + ess: "Elasticsearch Service" + ece: "Elastic Cloud Enterprise" + eck: "Elastic Cloud on Kubernetes" + serverless-full: "Elastic Cloud Serverless" + serverless-short: "Serverless" + es-serverless: "Elasticsearch Serverless" + es3: "Elasticsearch Serverless" + obs-serverless: "Elastic Observability Serverless" + sec-serverless: "Elastic Security Serverless" + serverless-docs: "https://docs.elastic.co/serverless" + cloud: "https://www.elastic.co/guide/en/cloud/current" + ess-utm-params: "?page=docs&placement=docs-body" + ess-baymax: "?page=docs&placement=docs-body" + ess-trial: "https://cloud.elastic.co/registration?page=docs&placement=docs-body" + ess-product: "https://www.elastic.co/cloud/elasticsearch-service?page=docs&placement=docs-body" + ess-console: "https://cloud.elastic.co?page=docs&placement=docs-body" + ess-console-name: "Elasticsearch Service Console" + ess-deployments: "https://cloud.elastic.co/deployments?page=docs&placement=docs-body" + ece-ref: "https://www.elastic.co/guide/en/cloud-enterprise/current" + eck-ref: "https://www.elastic.co/guide/en/cloud-on-k8s/current" + ess-leadin: "You can run Elasticsearch on your own hardware or use our hosted Elasticsearch Service that is available on AWS, GCP, and Azure. https://cloud.elastic.co/registration{ess-utm-params}[Try the Elasticsearch Service for free]." + ess-leadin-short: "Our hosted Elasticsearch Service is available on AWS, GCP, and Azure, and you can https://cloud.elastic.co/registration{ess-utm-params}[try it for free]." + ess-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elasticsearch Service\"]" + ece-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud_ece.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elastic Cloud Enterprise\"]" + cloud-only: "This feature is designed for indirect use by https://cloud.elastic.co/registration{ess-utm-params}[Elasticsearch Service], https://www.elastic.co/guide/en/cloud-enterprise/{ece-version-link}[Elastic Cloud Enterprise], and https://www.elastic.co/guide/en/cloud-on-k8s/current[Elastic Cloud on Kubernetes]. Direct use is not supported." + ess-setting-change: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"{ess-trial}\", title=\"Supported on {ess}\"] indicates a change to a supported https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html[user setting] for Elasticsearch Service." + ess-skip-section: "If you use Elasticsearch Service, skip this section. Elasticsearch Service handles these changes for you." + api-cloud: "https://www.elastic.co/docs/api/doc/cloud" + api-ece: "https://www.elastic.co/docs/api/doc/cloud-enterprise" + api-kibana-serverless: "https://www.elastic.co/docs/api/doc/serverless" + es-feature-flag: "This feature is in development and not yet available for use. This documentation is provided for informational purposes only." + es-ref-dir: "'{{elasticsearch-root}}/docs/reference'" + apm-app: "APM app" + uptime-app: "Uptime app" + synthetics-app: "Synthetics app" + logs-app: "Logs app" + metrics-app: "Metrics app" + infrastructure-app: "Infrastructure app" + siem-app: "SIEM app" + security-app: "Elastic Security app" + ml-app: "Machine Learning" + dev-tools-app: "Dev Tools" + ingest-manager-app: "Ingest Manager" + stack-manage-app: "Stack Management" + stack-monitor-app: "Stack Monitoring" + alerts-ui: "Alerts and Actions" + rules-ui: "Rules" + rac-ui: "Rules and Connectors" + connectors-ui: "Connectors" + connectors-feature: "Actions and Connectors" + stack-rules-feature: "Stack Rules" + user-experience: "User Experience" + ems: "Elastic Maps Service" + ems-init: "EMS" + hosted-ems: "Elastic Maps Server" + ipm-app: "Index Pattern Management" + ingest-pipelines: "ingest pipelines" + ingest-pipelines-app: "Ingest Pipelines" + ingest-pipelines-cap: "Ingest pipelines" + ls-pipelines: "Logstash pipelines" + ls-pipelines-app: "Logstash Pipelines" + maint-windows: "maintenance windows" + maint-windows-app: "Maintenance Windows" + maint-windows-cap: "Maintenance windows" + custom-roles-app: "Custom Roles" + data-source: "data view" + data-sources: "data views" + data-source-caps: "Data View" + data-sources-caps: "Data Views" + data-source-cap: "Data view" + data-sources-cap: "Data views" + project-settings: "Project settings" + manage-app: "Management" + index-manage-app: "Index Management" + data-views-app: "Data Views" + rules-app: "Rules" + saved-objects-app: "Saved Objects" + tags-app: "Tags" + api-keys-app: "API keys" + transforms-app: "Transforms" + connectors-app: "Connectors" + files-app: "Files" + reports-app: "Reports" + maps-app: "Maps" + alerts-app: "Alerts" + crawler: "Enterprise Search web crawler" + ents: "Enterprise Search" + app-search-crawler: "App Search web crawler" + agent: "Elastic Agent" + agents: "Elastic Agents" + fleet: "Fleet" + fleet-server: "Fleet Server" + integrations-server: "Integrations Server" + ingest-manager: "Ingest Manager" + ingest-management: "ingest management" + package-manager: "Elastic Package Manager" + integrations: "Integrations" + package-registry: "Elastic Package Registry" + artifact-registry: "Elastic Artifact Registry" + aws: "AWS" + stack: "Elastic Stack" + xpack: "X-Pack" + es: "Elasticsearch" + kib: "Kibana" + esms: "Elastic Stack Monitoring Service" + esms-init: "ESMS" + ls: "Logstash" + beats: "Beats" + auditbeat: "Auditbeat" + filebeat: "Filebeat" + heartbeat: "Heartbeat" + metricbeat: "Metricbeat" + packetbeat: "Packetbeat" + winlogbeat: "Winlogbeat" + functionbeat: "Functionbeat" + journalbeat: "Journalbeat" + es-sql: "Elasticsearch SQL" + esql: "ES|QL" + elastic-agent: "Elastic Agent" + k8s: "Kubernetes" + log-driver-long: "Elastic Logging Plugin for Docker" + security: "X-Pack security" + security-features: "security features" + operator-feature: "operator privileges feature" + es-security-features: "Elasticsearch security features" + stack-security-features: "Elastic Stack security features" + endpoint-sec: "Endpoint Security" + endpoint-cloud-sec: "Endpoint and Cloud Security" + elastic-defend: "Elastic Defend" + elastic-sec: "Elastic Security" + elastic-endpoint: "Elastic Endpoint" + swimlane: "Swimlane" + sn: "ServiceNow" + sn-itsm: "ServiceNow ITSM" + sn-itom: "ServiceNow ITOM" + sn-sir: "ServiceNow SecOps" + jira: "Jira" + ibm-r: "IBM Resilient" + webhook: "Webhook" + webhook-cm: "Webhook - Case Management" + opsgenie: "Opsgenie" + bedrock: "Amazon Bedrock" + gemini: "Google Gemini" + hive: "TheHive" + monitoring: "X-Pack monitoring" + monitor-features: "monitoring features" + stack-monitor-features: "Elastic Stack monitoring features" + watcher: "Watcher" + alert-features: "alerting features" + reporting: "X-Pack reporting" + report-features: "reporting features" + graph: "X-Pack graph" + graph-features: "graph analytics features" + searchprofiler: "Search Profiler" + xpackml: "X-Pack machine learning" + ml: "machine learning" + ml-cap: "Machine learning" + ml-init: "ML" + ml-features: "machine learning features" + stack-ml-features: "Elastic Stack machine learning features" + ccr: "cross-cluster replication" + ccr-cap: "Cross-cluster replication" + ccr-init: "CCR" + ccs: "cross-cluster search" + ccs-cap: "Cross-cluster search" + ccs-init: "CCS" + ilm: "index lifecycle management" + ilm-cap: "Index lifecycle management" + ilm-init: "ILM" + dlm: "data lifecycle management" + dlm-cap: "Data lifecycle management" + dlm-init: "DLM" + search-snap: "searchable snapshot" + search-snaps: "searchable snapshots" + search-snaps-cap: "Searchable snapshots" + slm: "snapshot lifecycle management" + slm-cap: "Snapshot lifecycle management" + slm-init: "SLM" + rollup-features: "data rollup features" + ipm: "index pattern management" + ipm-cap: "Index pattern" + rollup: "rollup" + rollup-cap: "Rollup" + rollups: "rollups" + rollups-cap: "Rollups" + rollup-job: "rollup job" + rollup-jobs: "rollup jobs" + rollup-jobs-cap: "Rollup jobs" + dfeed: "datafeed" + dfeeds: "datafeeds" + dfeed-cap: "Datafeed" + dfeeds-cap: "Datafeeds" + ml-jobs: "machine learning jobs" + ml-jobs-cap: "Machine learning jobs" + anomaly-detect: "anomaly detection" + anomaly-detect-cap: "Anomaly detection" + anomaly-job: "anomaly detection job" + anomaly-jobs: "anomaly detection jobs" + anomaly-jobs-cap: "Anomaly detection jobs" + dataframe: "data frame" + dataframes: "data frames" + dataframe-cap: "Data frame" + dataframes-cap: "Data frames" + watcher-transform: "payload transform" + watcher-transforms: "payload transforms" + watcher-transform-cap: "Payload transform" + watcher-transforms-cap: "Payload transforms" + transform: "transform" + transforms: "transforms" + transform-cap: "Transform" + transforms-cap: "Transforms" + dataframe-transform: "transform" + dataframe-transform-cap: "Transform" + dataframe-transforms: "transforms" + dataframe-transforms-cap: "Transforms" + dfanalytics-cap: "Data frame analytics" + dfanalytics: "data frame analytics" + dataframe-analytics-config: "'{dataframe} analytics config'" + dfanalytics-job: "'{dataframe} analytics job'" + dfanalytics-jobs: "'{dataframe} analytics jobs'" + dfanalytics-jobs-cap: "'{dataframe-cap} analytics jobs'" + cdataframe: "continuous data frame" + cdataframes: "continuous data frames" + cdataframe-cap: "Continuous data frame" + cdataframes-cap: "Continuous data frames" + cdataframe-transform: "continuous transform" + cdataframe-transforms: "continuous transforms" + cdataframe-transforms-cap: "Continuous transforms" + ctransform: "continuous transform" + ctransform-cap: "Continuous transform" + ctransforms: "continuous transforms" + ctransforms-cap: "Continuous transforms" + oldetection: "outlier detection" + oldetection-cap: "Outlier detection" + olscore: "outlier score" + olscores: "outlier scores" + fiscore: "feature influence score" + evaluatedf-api: "evaluate {dataframe} analytics API" + evaluatedf-api-cap: "Evaluate {dataframe} analytics API" + binarysc: "binary soft classification" + binarysc-cap: "Binary soft classification" + regression: "regression" + regression-cap: "Regression" + reganalysis: "regression analysis" + reganalysis-cap: "Regression analysis" + depvar: "dependent variable" + feature-var: "feature variable" + feature-vars: "feature variables" + feature-vars-cap: "Feature variables" + classification: "classification" + classification-cap: "Classification" + classanalysis: "classification analysis" + classanalysis-cap: "Classification analysis" + infer-cap: "Inference" + infer: "inference" + lang-ident-cap: "Language identification" + lang-ident: "language identification" + data-viz: "Data Visualizer" + file-data-viz: "File Data Visualizer" + feat-imp: "feature importance" + feat-imp-cap: "Feature importance" + nlp: "natural language processing" + nlp-cap: "Natural language processing" + apm-agent: "APM agent" + apm-go-agent: "Elastic APM Go agent" + apm-go-agents: "Elastic APM Go agents" + apm-ios-agent: "Elastic APM iOS agent" + apm-ios-agents: "Elastic APM iOS agents" + apm-java-agent: "Elastic APM Java agent" + apm-java-agents: "Elastic APM Java agents" + apm-dotnet-agent: "Elastic APM .NET agent" + apm-dotnet-agents: "Elastic APM .NET agents" + apm-node-agent: "Elastic APM Node.js agent" + apm-node-agents: "Elastic APM Node.js agents" + apm-php-agent: "Elastic APM PHP agent" + apm-php-agents: "Elastic APM PHP agents" + apm-py-agent: "Elastic APM Python agent" + apm-py-agents: "Elastic APM Python agents" + apm-ruby-agent: "Elastic APM Ruby agent" + apm-ruby-agents: "Elastic APM Ruby agents" + apm-rum-agent: "Elastic APM Real User Monitoring (RUM) JavaScript agent" + apm-rum-agents: "Elastic APM RUM JavaScript agents" + apm-lambda-ext: "Elastic APM AWS Lambda extension" + project-monitors: "project monitors" + project-monitors-cap: "Project monitors" + private-location: "Private Location" + private-locations: "Private Locations" + pwd: "YOUR_PASSWORD" + esh: "ES-Hadoop" + default-dist: "default distribution" + oss-dist: "OSS-only distribution" + observability: "Observability" + api-request-title: "Request" + api-prereq-title: "Prerequisites" + api-description-title: "Description" + api-path-parms-title: "Path parameters" + api-query-parms-title: "Query parameters" + api-request-body-title: "Request body" + api-response-codes-title: "Response codes" + api-response-body-title: "Response body" + api-example-title: "Example" + api-examples-title: "Examples" + api-definitions-title: "Properties" + multi-arg: "†footnoteref:[multi-arg,This parameter accepts multiple arguments.]" + multi-arg-ref: "†footnoteref:[multi-arg]" + yes-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png[Yes,20,15]" + no-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png[No,20,15]" + es-repo: "https://github.com/elastic/elasticsearch/" + es-issue: "https://github.com/elastic/elasticsearch/issues/" + es-pull: "https://github.com/elastic/elasticsearch/pull/" + es-commit: "https://github.com/elastic/elasticsearch/commit/" + kib-repo: "https://github.com/elastic/kibana/" + kib-issue: "https://github.com/elastic/kibana/issues/" + kibana-issue: "'{kib-repo}issues/'" + kib-pull: "https://github.com/elastic/kibana/pull/" + kibana-pull: "'{kib-repo}pull/'" + kib-commit: "https://github.com/elastic/kibana/commit/" + ml-repo: "https://github.com/elastic/ml-cpp/" + ml-issue: "https://github.com/elastic/ml-cpp/issues/" + ml-pull: "https://github.com/elastic/ml-cpp/pull/" + ml-commit: "https://github.com/elastic/ml-cpp/commit/" + apm-repo: "https://github.com/elastic/apm-server/" + apm-issue: "https://github.com/elastic/apm-server/issues/" + apm-pull: "https://github.com/elastic/apm-server/pull/" + kibana-blob: "https://github.com/elastic/kibana/blob/current/" + apm-get-started-ref: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-server-ref: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-v: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-m: "https://www.elastic.co/guide/en/apm/server/master" + apm-server-ref-62: "https://www.elastic.co/guide/en/apm/server/6.2" + apm-server-ref-64: "https://www.elastic.co/guide/en/apm/server/6.4" + apm-server-ref-70: "https://www.elastic.co/guide/en/apm/server/7.0" + apm-overview-ref-v: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-overview-ref-70: "https://www.elastic.co/guide/en/apm/get-started/7.0" + apm-overview-ref-m: "https://www.elastic.co/guide/en/apm/get-started/master" + infra-guide: "https://www.elastic.co/guide/en/infrastructure/guide/current" + a-data-source: "a data view" + icon-bug: "pass:[]" + icon-checkInCircleFilled: "pass:[]" + icon-warningFilled: "pass:[]" diff --git a/docs/images/choose-a-layer.png b/docs/images/choose-a-layer.png deleted file mode 100644 index 49cfd991..00000000 Binary files a/docs/images/choose-a-layer.png and /dev/null differ diff --git a/docs/images/config-layer.png b/docs/images/config-layer.png deleted file mode 100644 index ec6c045d..00000000 Binary files a/docs/images/config-layer.png and /dev/null differ diff --git a/docs/images/images.asciidoc b/docs/images/images.asciidoc deleted file mode 100644 index 6ff9fc3b..00000000 --- a/docs/images/images.asciidoc +++ /dev/null @@ -1,11 +0,0 @@ -tag::java-env-vars[] -image:./java-lambda-env-vars.png[Java environment variables configuration section in AWS Console] -end::java-env-vars[] - -tag::python-env-vars[] -image:./python-lambda-env-vars.png[Python environment variables configuration section in AWS Console] -end::python-env-vars[] - -tag::nodejs-env-vars[] -image:./nodejs-lambda-env-vars.png[Node.js environment variables configuration section in AWS Console] -end::nodejs-env-vars[] \ No newline at end of file diff --git a/docs/images/java-lambda-env-vars.png b/docs/images/java-lambda-env-vars.png deleted file mode 100644 index 028e840e..00000000 Binary files a/docs/images/java-lambda-env-vars.png and /dev/null differ diff --git a/docs/images/nodejs-lambda-env-vars.png b/docs/images/nodejs-lambda-env-vars.png deleted file mode 100644 index 79476738..00000000 Binary files a/docs/images/nodejs-lambda-env-vars.png and /dev/null differ diff --git a/docs/images/python-lambda-env-vars.png b/docs/images/python-lambda-env-vars.png deleted file mode 100644 index cbf4a25d..00000000 Binary files a/docs/images/python-lambda-env-vars.png and /dev/null differ diff --git a/docs/index.asciidoc b/docs/index.asciidoc deleted file mode 100644 index 7da6e8e2..00000000 --- a/docs/index.asciidoc +++ /dev/null @@ -1,11 +0,0 @@ -include::{asciidoc-dir}/../../shared/versions/stack/current.asciidoc[] -include::{asciidoc-dir}/../../shared/attributes.asciidoc[] - -ifdef::env-github[] -NOTE: For the best reading experience, -please view this documentation at https://www.elastic.co/guide/en/ecs-logging/go-zap/current/index.html[elastic.co] -endif::[] - -include::monitoring-aws-lambda.asciidoc[] - -include::release-notes.asciidoc[] diff --git a/docs/lambda-selector/extension-arn-replacement.asciidoc b/docs/lambda-selector/extension-arn-replacement.asciidoc deleted file mode 100644 index 4916a668..00000000 --- a/docs/lambda-selector/extension-arn-replacement.asciidoc +++ /dev/null @@ -1,7 +0,0 @@ -++++ - -++++ \ No newline at end of file diff --git a/docs/lambda-selector/lambda-attributes-selector.asciidoc b/docs/lambda-selector/lambda-attributes-selector.asciidoc deleted file mode 100644 index 1fd709eb..00000000 --- a/docs/lambda-selector/lambda-attributes-selector.asciidoc +++ /dev/null @@ -1,180 +0,0 @@ -++++ - - - - -

Pick the right ARN from this release table for the APM Lambda Extension Layer.

-

In addition, pick the right ARN from this release table for the APM Agent Layer.

-
-
Select the AWS region and architecture of your Lambda function. This documentation will update based on your selections.
-
-
-
region:
- -
-
-
architecture:
- -
-
-
-++++ - -WARNING: The selected _AWS region_ and the _architecture_ must match the AWS region and architecture of your AWS Lambda function! diff --git a/docs/monitoring-aws-lambda.asciidoc b/docs/monitoring-aws-lambda.asciidoc deleted file mode 100644 index b792749e..00000000 --- a/docs/monitoring-aws-lambda.asciidoc +++ /dev/null @@ -1,254 +0,0 @@ -[[monitoring-aws-lambda]] -= Monitoring AWS Lambda Functions - -Elastic APM lets you monitor your AWS Lambda functions. -The natural integration of {apm-guide-ref}/apm-distributed-tracing.html[distributed tracing] into your AWS Lambda functions provides insights into the functions' execution and runtime behavior as well as their relationships and dependencies to other services. - -To get started with the setup of Elastic APM for your Lambda functions, checkout the language-specific guides: - -* {apm-node-ref}/lambda.html[Quick Start with APM on AWS Lambda - Node.js] -* {apm-py-ref}/lambda-support.html[Quick Start with APM on AWS Lambda - Python] -* {apm-java-ref}/aws-lambda.html[Quick Start with APM on AWS Lambda - Java] - -Note that AWS Step Functions are currently not supported by any of the languages. - -Learn more about the <> of Elastic APM for AWS Lambda. - -[[aws-lambda-arch]] -== APM Architecture for AWS Lambda - -AWS Lambda uses a special execution model to provide a scalable, on-demand compute service for code execution. In particular, AWS freezes the execution environment of a lambda function when no active requests are being processed. This execution model poses additional requirements on APM in the context of AWS Lambda functions: - -1. To avoid data loss, APM data collected by APM agents needs to be flushed before the execution environment of a lambda function is frozen. -2. Flushing APM data must be fast so as not to impact the response times of lambda function requests. - -To accomplish the above, Elastic APM agents instrument AWS Lambda functions and dispatch APM data via an https://docs.aws.amazon.com/lambda/latest/dg/using-extensions.html[AWS Lambda extension]. - -Normally, during the execution of a Lambda function, there's only a single language process running in the AWS Lambda execution environment. With an AWS Lambda extension, Lambda users run a _second_ process alongside their main service/application process. - -image:images/architecture-white.png[image showing data flow from lambda function, to extension, to APM Server] - -By using an AWS Lambda extension, Elastic APM agents can send data to a local Lambda extension process, and that process will forward data on to APM Server asynchronously. The Lambda extension ensures that any potential latency between the Lambda function and the APM Server instance will not cause latency in the request flow of the Lambda function itself. - -[[aws-lambda-overhead]] -== Performance impact and overhead -As described in <>, using Elastic APM with AWS Lambda requires adding both the Elastic APM AWS Lambda extension and a corresponding Elastic APM agent to the Lambda runtime. -These components may introduce a small overhead on the size of your function's deployment package as well as the execution duration of your function's invocations. - -[float] -=== Impact on the deployment package size -These components contribute a little to the uncompressed deployment package size of your Lambda function. Overall, the impact of using Elastic APM on the uncompressed deployment package size of your Lambda function is less than 30MB. - -[float] -=== Performance impact -An advantage of the Elastic APM AWS Lambda extension architecture is that APM data dispatching is decoupled from your function's request processing. -The Elastic APM AWS Lambda extension flushes APM data to the Elastic backend _after_ your function responds to the client's request. Thus, it does not affect the latency of the client's request. -However, the extension's flushing of APM data contributes to the overall execution time of the function invocation. -The <> config option with the related _exponential backoff algorithm_ limits and allows to control the impact the extension may have on the function's overall execution time. - -When your function experiences a cold start, the Elastic APM AWS Lambda extension needs to be initialized and, thus, slightly increases the cold start duration (in the range of tens of milliseconds) of your function. - -APM agents enrich your application's code with measurement code that collects APM data. This measurement code introduces a small performance overhead to your application, which is usually in a negligible range. The same is true with Lambda functions. The concrete performance overhead introduced by APM agents highly depends on the configuration of the agent and on the characteristics of your function's code. The following agent-specific documentation pages provide insights and instructions on tuning the performance the APM agents: - -* {apm-node-ref}/performance-tuning.html[Performance Tuning - Node.js] -* {apm-py-ref}/tuning-and-overhead.html[Performance Tuning - Python] -* {apm-java-ref}/tuning-and-overhead.html[Performance Tuning - Java] - -Similar to the Elastic APM AWS Lambda extension, APM agents are initialized at cold start time. As a consequence, the APM agent's overhead will be higher for cold starts as compared to their overhead on _warm_ invocations. This effect is especially relevant for the Java APM agent on AWS Lambda. Learn more about corresponding tuning options in the {apm-java-ref}/aws-lambda.html#aws-lambda-caveats[Java Agent's AWS Lambda documentation]. - -[[aws-lambda-config-options]] -== Configuration options - -The recommended way of configuring the {apm-lambda-ext} and the APM agents on AWS Lambda is through the Lambda function's environment variables. - -The configuration options for the APM agents are documented in the corresponding language agents: - -* {apm-node-ref}/configuration.html[Configuration options - Node.js APM agent] -* {apm-py-ref}/configuration.html[Configuration options - Python APM agent] -* {apm-java-ref}/configuration.html[Configuration options - Java APM agent] - -NOTE: Some APM agent configuration options don't make sense when the APM agent is running in a Lambda environment. -For example, instead of using the Python APM agent configuration variable, `verify_server_cert`, you must use the -`ELASTIC_APM_LAMBDA_VERIFY_SERVER_CERT` variable described below. - -NOTE: APM Central configuration is not supported when using the Elastic APM AWS Lambda extension - -[float] -[[aws-lambda-config-relevant]] -=== Relevant configuration options - -A list of relevant configuration options for the {apm-lambda-ext} is below. - -[float] -[[aws-lambda-extension]] -==== `ELASTIC_APM_LAMBDA_APM_SERVER` -This required config option controls where the {apm-lambda-ext} will ship data. This should be the URL of the final APM Server destination for your telemetry. - -[float] -==== `ELASTIC_APM_LAMBDA_AGENT_DATA_BUFFER_SIZE` -The size of the buffer that stores APM agent data to be forwarded to the APM server. The _default_ is `100`. - -[float] -[[aws-lambda-config-authentication-keys]] -==== `ELASTIC_APM_SECRET_TOKEN` or `ELASTIC_APM_API_KEY` -One of these (or, alternatively, the corresponding settings for the AWS Secrets Manager IDs) needs to be set as the authentication method that the {apm-lambda-ext} uses when sending data to the URL configured via `ELASTIC_APM_LAMBDA_APM_SERVER`. Alternatively, you can store your APM Server credentials <> and use the <> config options, instead. Sending data to the APM Server if none of these options is set is possible, but your APM agent must be allowed to send data to your APM server in https://www.elastic.co/guide/en/apm/guide/current/configuration-anonymous.html[anonymous mode]. - -[float] -[[aws-lambda-config-secrets-manager-options]] -==== `ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` or `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID` -Instead of specifying the <> as plain text in your Lambda environment variables, you can <> to securely store your APM authetication keys. The `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID` or `ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` config options allow you to specify the Secrets Manager's secret id of the stored APM API key or APM secret token, respectively, to be used by the {apm-lambda-ext} for authentication. - -`ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` takes precedence over <>, and `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID` over <>, respectively. - -[float] -==== `ELASTIC_APM_SERVICE_NAME` -The configured name of your application or service. The APM agent will use this value when reporting data to the APM Server. If unset, the APM agent will automatically set the value based on the Lambda function name. Use this config option if you want to group multiple Lambda functions under a single service entity in APM. - -[float] -[[aws-lambda-config-data-receiver-timeout]] -==== `ELASTIC_APM_DATA_RECEIVER_TIMEOUT` - -[small]#Added in: v1.2.0. Replaces `ELASTIC_APM_DATA_RECEIVER_TIMEOUT_SECONDS`.# - -The {apm-lambda-ext}'s timeout value, for receiving data from the APM agent. The _default_ is `15s`. - -[float] -==== `ELASTIC_APM_DATA_RECEIVER_SERVER_PORT` -The port on which the {apm-lambda-ext} listens to receive data from the APM agent. The _default_ is `8200`. - -[float] -[[aws-lambda-config-data-forwarder-timeout]] -==== `ELASTIC_APM_DATA_FORWARDER_TIMEOUT` - -[small]#Added in: v1.2.0. Replaces `ELASTIC_APM_DATA_FORWARDER_TIMEOUT_SECONDS`.# - -The timeout value, for the {apm-lambda-ext}'s HTTP client sending data to the APM Server. The _default_ is `3s`. If the extension's attempt to send APM data during this time interval is not successful, the extension queues back the data. Further attempts at sending the data are governed by an exponential backoff algorithm: data will be sent after a increasingly large grace period of 0, then circa 1, 4, 9, 16, 25 and 36 seconds, provided that the Lambda function execution is ongoing. - -[float] -==== `ELASTIC_APM_SEND_STRATEGY` -Whether to synchronously flush APM agent data from the {apm-lambda-ext} to the APM Server at the end of the function invocation. -The two accepted values are `background` and `syncflush`. The _default_ is `syncflush`. - -* The `background` strategy indicates that the {apm-lambda-ext} will not flush when it receives a signal that the function invocation -has completed. It will instead send any remaining buffered data on the next function invocation. The result is that, if the -function is not subsequently invoked for that Lambda environment, the buffered data will be lost. However, for lambda functions -that have a steadily frequent load pattern the extension could delay sending the data to the APM Server to the next lambda -request and do the sending in parallel to the processing of that next request. This potentially would improve both the lambda -function response time and its throughput. -* The other value, `syncflush` will synchronously flush all remaining buffered APM agent data to the APM Server when the -extension receives a signal that the function invocation has completed. This strategy blocks the lambda function from receiving -the next request until the extension has flushed all the data. This has a negative effect on the throughput of the function, -though it ensures that all APM data is sent to the APM server. - -[float] -==== `ELASTIC_APM_LOG_LEVEL` -The logging level to be used by both the APM Agent and the {apm-lambda-ext}. Supported values are `trace`, `debug`, `info`, `warning`, `error`, `critical` and `off`. - -[float] -==== `ELASTIC_APM_LAMBDA_CAPTURE_LOGS` - -preview:[] -Starting in Elastic Stack version 8.5.0, the Elastic APM lambda extension supports the collection of log events by default. -Log events can be viewed in {kib} in the APM UI. Disable log collection by setting this to `false`. - -[float] -==== `ELASTIC_APM_LAMBDA_VERIFY_SERVER_CERT` - -[small]#Added in: v1.3.0.# - -Whether to enable {apm-lambda-ext} to verify APM Server's certificate chain and host name. - -[float] -==== `ELASTIC_APM_LAMBDA_SERVER_CA_CERT_PEM` - -[small]#Added in: v1.3.0.# - -The certificate passed as environment variable. To be used to verify APM Server's certificate chain if verify server certificate is enabled. - -[float] -==== `ELASTIC_APM_SERVER_CA_CERT_FILE` - -[small]#Added in: v1.3.0.# - -The certificate passed as a file name available to the extension. To be used to verify APM Server's certificate chain if verify server certificate is enabled. - -[float] -==== `ELASTIC_APM_SERVER_CA_CERT_ACM_ID` - -[small]#Added in: v1.3.0.# - -The ARN for Amazon-issued certificate. To be used to verify APM Server's certificate chain if verify server certificate is enabled. - -[NOTE] -==== -You may see errors similar to the following in {stack} versions less than 8.5: - -[source,text] ----- -client error: response status code: 400 -message: log: did not recognize object type ----- - -Users on older versions should disable log collection by setting `ELASTIC_APM_LAMBDA_CAPTURE_LOGS` to `false`. -==== - -[float] -[[aws-lambda-config-deprecated]] -=== Deprecated options - -[float] -[[aws-lambda-config-data-receiver-timeout-seconds]] -==== `ELASTIC_APM_DATA_RECEIVER_TIMEOUT_SECONDS` - -[small]#Deprecated in: v1.2.0. Use <> instead.# - -The {apm-lambda-ext}'s timeout value, in seconds, for receiving data from the APM agent. The _default_ is `15`. - -[float] -[[aws-lambda-config-data-forwarder-timeout-seconds]] -==== `ELASTIC_APM_DATA_FORWARDER_TIMEOUT_SECONDS` - -[small]#Deprecated in: v1.2.0. Use <> instead.# - -The timeout value, in seconds, for the {apm-lambda-ext}'s HTTP client sending data to the APM Server. The _default_ is `3`. If the extension's attempt to send APM data during this time interval is not successful, the extension queues back the data. Further attempts at sending the data are governed by an exponential backoff algorithm: data will be sent after a increasingly large grace period of 0, then circa 1, 4, 9, 16, 25 and 36 seconds, provided that the Lambda function execution is ongoing. - -[[aws-lambda-secrets-manager]] -== Using AWS Secrets Manager to manage APM authentication keys -When using the config options <> for authentication of the {apm-lambda-ext}, the corresponding keys are specified in plain text in the environment variables of your Lambda function. If you prefer to securely store the authentication keys, you can use the AWS Secrets Manager and let the extension retrieve the actual keys from the AWS Secrets Manager. Follow the instructions below to set up the AWS Secrets Manager with the extension. - -[float] -[[aws-lambda-secrets-manager-create-secret]] -=== Step 1: Create a secret in the AWS Secrets Manager. -https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html[Create a secret in the AWS Secrets Manager] for the {apm-guide-ref}/secret-token.html[APM Secret Token] or the {apm-guide-ref}/api-key.html[APM API key], depending on which one you prefer to use. Make sure to create the secret as a *Plaintext* typed secret and ensure it is created *in the same AWS region* as your target Lambda function that will use the secret. - -We recommend using the AWS-managed encryption key `aws/secretsmanager`. However, you can optionally create and select a custom KMS key for encryption. Note that with a custom encryption key, you will need additional key permissions on your Lambda function (see <>). - -include::./secrets-manager/create-secret-widget.asciidoc[] - -Remember your chosen secret name. You will use the secret name as the value for the config options <> when configuring your {apm-lambda-ext}. - -[float] -[[aws-lambda-secrets-manager-permissions]] -=== Step 2: Add permissions to your AWS Lambda function -For your Lambda function to be able to retrieve the authentication key from the AWS Secrets Manager, you need to provide the following permissions to your Lambda function. - -include::./secrets-manager/add-permissions-widget.asciidoc[] - -[float] -=== Step 3: Configure the {apm-lambda-ext} -Finally, you will need to configure the {apm-lambda-ext} to use the secret from the Secrets Manager _instead of_ the value provided through <>. - -Provide the name of the secret you created in <> as the value for the <> config option, respectively, depending on whether you want to use the {apm-guide-ref}/secret-token.html[APM Secret Token] or the {apm-guide-ref}/api-key.html[APM API key]. - -The language-specific instructions describe how to set environment variables for configuring AWS Lambda for Elastic APM: - -* {apm-node-ref}/lambda.html#_step_3_configure_apm_on_aws_lambda[Configure APM on AWS Lambda - Node.js] -* {apm-py-ref}/lambda-support.html#_step_3_configure_apm_on_aws_lambda[Configure APM on AWS Lambda - Python] -* {apm-java-ref}/aws-lambda.html#_step_3_configure_apm_on_aws_lambda[Configure APM on AWS Lambda - Java] - -That's it. With the first invocation (cold start) of your Lambda function you should see a log message from the {apm-lambda-ext} indicating that a secret from the secrets manager is used: - -[source, yml] ----- -"Using the APM secret token retrieved from Secrets Manager." ----- diff --git a/docs/reference/aws-lambda-config-options.md b/docs/reference/aws-lambda-config-options.md new file mode 100644 index 00000000..b626a500 --- /dev/null +++ b/docs/reference/aws-lambda-config-options.md @@ -0,0 +1,152 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/lambda/current/aws-lambda-config-options.html +--- + +# Configuration options [aws-lambda-config-options] + +The recommended way of configuring the {{apm-lambda-ext}} and the APM agents on AWS Lambda is through the Lambda function’s environment variables. + +The configuration options for the APM agents are documented in the corresponding language agents: + +* [Configuration options - Node.js APM agent](apm-agent-nodejs://reference/configuration.md) +* [Configuration options - Python APM agent](apm-agent-php://reference/configuration.md) +* [Configuration options - Java APM agent](apm-agent-java://reference/configuration.md) + +::::{note} +Some APM agent configuration options don’t make sense when the APM agent is running in a Lambda environment. For example, instead of using the Python APM agent configuration variable, `verify_server_cert`, you must use the `ELASTIC_APM_LAMBDA_VERIFY_SERVER_CERT` variable described below. +:::: + + +::::{note} +APM Central configuration is not supported when using the Elastic APM AWS Lambda extension +:::: + + + +## Relevant configuration options [aws-lambda-config-relevant] + +A list of relevant configuration options for the {{apm-lambda-ext}} is below. + + +### `ELASTIC_APM_LAMBDA_APM_SERVER` [aws-lambda-extension] + +This required config option controls where the {{apm-lambda-ext}} will ship data. This should be the URL of the final APM Server destination for your telemetry. + + +### `ELASTIC_APM_LAMBDA_AGENT_DATA_BUFFER_SIZE` [_elastic_apm_lambda_agent_data_buffer_size] + +The size of the buffer that stores APM agent data to be forwarded to the APM server. The *default* is `100`. + + +### `ELASTIC_APM_SECRET_TOKEN` or `ELASTIC_APM_API_KEY` [aws-lambda-config-authentication-keys] + +One of these (or, alternatively, the corresponding settings for the AWS Secrets Manager IDs) needs to be set as the authentication method that the {{apm-lambda-ext}} uses when sending data to the URL configured via `ELASTIC_APM_LAMBDA_APM_SERVER`. Alternatively, you can store your APM Server credentials [using the AWS Secrets Manager](/reference/aws-lambda-secrets-manager.md) and use the [`ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` or `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID`](#aws-lambda-config-secrets-manager-options) config options, instead. Sending data to the APM Server if none of these options is set is possible, but your APM agent must be allowed to send data to your APM server in [anonymous mode](docs-content://solutions/observability/apps/configure-anonymous-authentication.md). + + +### `ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` or `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID` [aws-lambda-config-secrets-manager-options] + +Instead of specifying the [`ELASTIC_APM_SECRET_TOKEN` or `ELASTIC_APM_API_KEY`](#aws-lambda-config-authentication-keys) as plain text in your Lambda environment variables, you can [use the AWS Secrets Manager](/reference/aws-lambda-secrets-manager.md) to securely store your APM authetication keys. The `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID` or `ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` config options allow you to specify the Secrets Manager’s secret id of the stored APM API key or APM secret token, respectively, to be used by the {{apm-lambda-ext}} for authentication. + +`ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` takes precedence over [`ELASTIC_APM_SECRET_TOKEN`](#aws-lambda-config-authentication-keys), and `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID` over [`ELASTIC_APM_API_KEY`](#aws-lambda-config-authentication-keys), respectively. + + +### `ELASTIC_APM_SERVICE_NAME` [_elastic_apm_service_name] + +The configured name of your application or service. The APM agent will use this value when reporting data to the APM Server. If unset, the APM agent will automatically set the value based on the Lambda function name. Use this config option if you want to group multiple Lambda functions under a single service entity in APM. + + +### `ELASTIC_APM_DATA_RECEIVER_TIMEOUT` [aws-lambda-config-data-receiver-timeout] + +Added in: v1.2.0. Replaces `ELASTIC_APM_DATA_RECEIVER_TIMEOUT_SECONDS`. + +The {{apm-lambda-ext}}'s timeout value, for receiving data from the APM agent. The *default* is `15s`. + + +### `ELASTIC_APM_DATA_RECEIVER_SERVER_PORT` [_elastic_apm_data_receiver_server_port] + +The port on which the {{apm-lambda-ext}} listens to receive data from the APM agent. The *default* is `8200`. + + +### `ELASTIC_APM_DATA_FORWARDER_TIMEOUT` [aws-lambda-config-data-forwarder-timeout] + +Added in: v1.2.0. Replaces `ELASTIC_APM_DATA_FORWARDER_TIMEOUT_SECONDS`. + +The timeout value, for the {{apm-lambda-ext}}'s HTTP client sending data to the APM Server. The *default* is `3s`. If the extension’s attempt to send APM data during this time interval is not successful, the extension queues back the data. Further attempts at sending the data are governed by an exponential backoff algorithm: data will be sent after a increasingly large grace period of 0, then circa 1, 4, 9, 16, 25 and 36 seconds, provided that the Lambda function execution is ongoing. + + +### `ELASTIC_APM_SEND_STRATEGY` [_elastic_apm_send_strategy] + +Whether to synchronously flush APM agent data from the {{apm-lambda-ext}} to the APM Server at the end of the function invocation. The two accepted values are `background` and `syncflush`. The *default* is `syncflush`. + +* The `background` strategy indicates that the {{apm-lambda-ext}} will not flush when it receives a signal that the function invocation has completed. It will instead send any remaining buffered data on the next function invocation. The result is that, if the function is not subsequently invoked for that Lambda environment, the buffered data will be lost. However, for lambda functions that have a steadily frequent load pattern the extension could delay sending the data to the APM Server to the next lambda request and do the sending in parallel to the processing of that next request. This potentially would improve both the lambda function response time and its throughput. +* The other value, `syncflush` will synchronously flush all remaining buffered APM agent data to the APM Server when the extension receives a signal that the function invocation has completed. This strategy blocks the lambda function from receiving the next request until the extension has flushed all the data. This has a negative effect on the throughput of the function, though it ensures that all APM data is sent to the APM server. + + +### `ELASTIC_APM_LOG_LEVEL` [_elastic_apm_log_level] + +The logging level to be used by both the APM Agent and the {{apm-lambda-ext}}. Supported values are `trace`, `debug`, `info`, `warning`, `error`, `critical` and `off`. + + +### `ELASTIC_APM_LAMBDA_CAPTURE_LOGS` [_elastic_apm_lambda_capture_logs] + +[preview] Starting in Elastic Stack version 8.5.0, the Elastic APM lambda extension supports the collection of log events by default. Log events can be viewed in {{kib}} in the APM UI. Disable log collection by setting this to `false`. + + +### `ELASTIC_APM_LAMBDA_VERIFY_SERVER_CERT` [_elastic_apm_lambda_verify_server_cert] + +Added in: v1.3.0. + +Whether to enable {{apm-lambda-ext}} to verify APM Server’s certificate chain and host name. + + +### `ELASTIC_APM_LAMBDA_SERVER_CA_CERT_PEM` [_elastic_apm_lambda_server_ca_cert_pem] + +Added in: v1.3.0. + +The certificate passed as environment variable. To be used to verify APM Server’s certificate chain if verify server certificate is enabled. + + +### `ELASTIC_APM_SERVER_CA_CERT_FILE` [_elastic_apm_server_ca_cert_file] + +Added in: v1.3.0. + +The certificate passed as a file name available to the extension. To be used to verify APM Server’s certificate chain if verify server certificate is enabled. + + +### `ELASTIC_APM_SERVER_CA_CERT_ACM_ID` [_elastic_apm_server_ca_cert_acm_id] + +Added in: v1.3.0. + +The ARN for Amazon-issued certificate. To be used to verify APM Server’s certificate chain if verify server certificate is enabled. + +::::{note} +You may see errors similar to the following in {{stack}} versions less than 8.5: + +```text +client error: response status code: 400 +message: log: did not recognize object type +``` + +Users on older versions should disable log collection by setting `ELASTIC_APM_LAMBDA_CAPTURE_LOGS` to `false`. + +:::: + + + +## Deprecated options [aws-lambda-config-deprecated] + + +### `ELASTIC_APM_DATA_RECEIVER_TIMEOUT_SECONDS` [aws-lambda-config-data-receiver-timeout-seconds] + +Deprecated in: v1.2.0. Use [`ELASTIC_APM_DATA_RECEIVER_TIMEOUT`](#aws-lambda-config-data-receiver-timeout) instead. + +The {{apm-lambda-ext}}'s timeout value, in seconds, for receiving data from the APM agent. The *default* is `15`. + + +### `ELASTIC_APM_DATA_FORWARDER_TIMEOUT_SECONDS` [aws-lambda-config-data-forwarder-timeout-seconds] + +Deprecated in: v1.2.0. Use [`ELASTIC_APM_DATA_FORWARDER_TIMEOUT`](#aws-lambda-config-data-forwarder-timeout) instead. + +The timeout value, in seconds, for the {{apm-lambda-ext}}'s HTTP client sending data to the APM Server. The *default* is `3`. If the extension’s attempt to send APM data during this time interval is not successful, the extension queues back the data. Further attempts at sending the data are governed by an exponential backoff algorithm: data will be sent after a increasingly large grace period of 0, then circa 1, 4, 9, 16, 25 and 36 seconds, provided that the Lambda function execution is ongoing. + diff --git a/docs/reference/aws-lambda-overhead.md b/docs/reference/aws-lambda-overhead.md new file mode 100644 index 00000000..216f89f1 --- /dev/null +++ b/docs/reference/aws-lambda-overhead.md @@ -0,0 +1,29 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/lambda/current/aws-lambda-overhead.html +--- + +# Performance impact and overhead [aws-lambda-overhead] + +As described in [*APM Architecture for AWS Lambda*](/reference/index.md), using Elastic APM with AWS Lambda requires adding both the Elastic APM AWS Lambda extension and a corresponding Elastic APM agent to the Lambda runtime. These components may introduce a small overhead on the size of your function’s deployment package as well as the execution duration of your function’s invocations. + + +## Impact on the deployment package size [_impact_on_the_deployment_package_size] + +These components contribute a little to the uncompressed deployment package size of your Lambda function. Overall, the impact of using Elastic APM on the uncompressed deployment package size of your Lambda function is less than 30MB. + + +## Performance impact [_performance_impact] + +An advantage of the Elastic APM AWS Lambda extension architecture is that APM data dispatching is decoupled from your function’s request processing. The Elastic APM AWS Lambda extension flushes APM data to the Elastic backend *after* your function responds to the client’s request. Thus, it does not affect the latency of the client’s request. However, the extension’s flushing of APM data contributes to the overall execution time of the function invocation. The [`ELASTIC_APM_DATA_FORWARDER_TIMEOUT`](/reference/aws-lambda-config-options.md#aws-lambda-config-data-forwarder-timeout) config option with the related *exponential backoff algorithm* limits and allows to control the impact the extension may have on the function’s overall execution time. + +When your function experiences a cold start, the Elastic APM AWS Lambda extension needs to be initialized and, thus, slightly increases the cold start duration (in the range of tens of milliseconds) of your function. + +APM agents enrich your application’s code with measurement code that collects APM data. This measurement code introduces a small performance overhead to your application, which is usually in a negligible range. The same is true with Lambda functions. The concrete performance overhead introduced by APM agents highly depends on the configuration of the agent and on the characteristics of your function’s code. The following agent-specific documentation pages provide insights and instructions on tuning the performance the APM agents: + +* [Performance Tuning - Node.js](apm-agent-nodejs://reference/performance-tuning.md) +* [Performance Tuning - Python](apm-agent-python://reference/performance-tuning.md) +* [Performance Tuning - Java](apm-agent-java://reference/overhead-performance-tuning.md) + +Similar to the Elastic APM AWS Lambda extension, APM agents are initialized at cold start time. As a consequence, the APM agent’s overhead will be higher for cold starts as compared to their overhead on *warm* invocations. This effect is especially relevant for the Java APM agent on AWS Lambda. Learn more about corresponding tuning options in the [Java Agent’s AWS Lambda documentation](apm-agent-java://reference/aws-lambda.md#aws-lambda-caveats). + diff --git a/docs/reference/aws-lambda-secrets-manager.md b/docs/reference/aws-lambda-secrets-manager.md new file mode 100644 index 00000000..481ab6e4 --- /dev/null +++ b/docs/reference/aws-lambda-secrets-manager.md @@ -0,0 +1,278 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/lambda/current/aws-lambda-secrets-manager.html +--- + +# Using AWS Secrets Manager to manage APM authentication keys [aws-lambda-secrets-manager] + +When using the config options [`ELASTIC_APM_SECRET_TOKEN` or `ELASTIC_APM_API_KEY`](/reference/aws-lambda-config-options.md#aws-lambda-config-authentication-keys) for authentication of the {{apm-lambda-ext}}, the corresponding keys are specified in plain text in the environment variables of your Lambda function. If you prefer to securely store the authentication keys, you can use the AWS Secrets Manager and let the extension retrieve the actual keys from the AWS Secrets Manager. Follow the instructions below to set up the AWS Secrets Manager with the extension. + + +## Step 1: Create a secret in the AWS Secrets Manager. [aws-lambda-secrets-manager-create-secret] + +[Create a secret in the AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.md) for the [APM Secret Token](docs-content://solutions/observability/apps/secret-token.md) or the [APM API key](docs-content://solutions/observability/apps/api-keys.md), depending on which one you prefer to use. Make sure to create the secret as a **Plaintext** typed secret and ensure it is created **in the same AWS region** as your target Lambda function that will use the secret. + +We recommend using the AWS-managed encryption key `aws/secretsmanager`. However, you can optionally create and select a custom KMS key for encryption. Note that with a custom encryption key, you will need additional key permissions on your Lambda function (see [Step 2](#aws-lambda-secrets-manager-permissions)). + +:::::::{tab-set} + +::::::{tab-item} AWS Web Console +In the Web Console of the AWS Secrets Manager, navigate to `Store a new secret`, select `Other type of secret` as the secret type, and choose the `Plaintext` tab for entering the secret value. Enter your APM secret token or APM API key value as a plain string (**not** as a JSON key value pair). + +![image showing the creation of a plaintext secret in Secrets Manager](../images/secrets-manager.png "") + +Click `Next`, choose a secret name, and finalize the creation of the secret. +:::::: + +::::::{tab-item} AWS CLI +Execute the following command to create a secret in the AWS Secrets Manager for your APM secret token or the APM API key: + +```bash +aws secretsmanager create-secret \ + --name MyAPMSecretToken \ <1> + --description "The Elastic APM secret token." \ + --secret-string "YOUR_APM_AUTHENTICATION_KEY" <2> +``` + +1. Choose a secret name +2. APM secret token or APM API key +:::::: + +::::::{tab-item} SAM +The AWS Serverless Application Model (SAM) does not provide means to create a secret in the AWS Secrets Manager. Use one of the other options to create the secret. +:::::: + +::::::{tab-item} Serverless +The Serverless Framework does not provide means to create a secret in the AWS Secrets Manager. Use one of the other options to create the secret. +:::::: + +::::::{tab-item} Terraform +Add the following resources to your terraform file to create a secret in the AWS Secrets Manager for your APM secret token or the APM API key: + +```yaml +... +resource "aws_secretsmanager_secret" "apm_secret_token" { + name = "MyAPMSecretToken" <1> +} + +resource "aws_secretsmanager_secret_version" "apm_secret_token_version" { + secret_id = aws_secretsmanager_secret.apm_secret_token.id + secret_string = "YOUR_APM_AUTHENTICATION_KEY" <2> +} +... +``` + +1. Choose a secret name +2. APM secret token or APM API key +:::::: + +::::::: +Remember your chosen secret name. You will use the secret name as the value for the config options [`ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` or `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID`](/reference/aws-lambda-config-options.md#aws-lambda-config-secrets-manager-options) when configuring your {{apm-lambda-ext}}. + + +## Step 2: Add permissions to your AWS Lambda function [aws-lambda-secrets-manager-permissions] + +For your Lambda function to be able to retrieve the authentication key from the AWS Secrets Manager, you need to provide the following permissions to your Lambda function. + +:::::::{tab-set} + +::::::{tab-item} AWS Web Console +* In the Web Console of your AWS Lambda function navigate to `Configuration` → `Permissions` and click on the link in the `Execution Role` section. This will lead you to the Web Console for the corresponding IAM role. +* Select `Create inline policy` in the `Add permission` dropdown button and select the `JSON` tab to define the following JSON-based policy: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "secretsmanager:GetSecretValue" + ], + "Resource": [ + "THE_ARN_OF_YOUR_SECRET" + ] + }, + { <1> + "Effect": "Allow", + "Action": [ + "kms:Decrypt" + ], + "Resource": [ + "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY" + ] + } + ] + } + ``` + + 1. This permission is ONLY needed if you use a custom KMS encryption key for your secret + +* Review and save the policy. +:::::: + +::::::{tab-item} AWS CLI +* Create a policy file (`smElasticApmPolicy.json`) with the following content: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "secretsmanager:GetSecretValue" + ], + "Resource": [ + "THE_ARN_OF_YOUR_SECRET" + ] + }, + { <1> + "Effect": "Allow", + "Action": [ + "kms:Decrypt" + ], + "Resource": [ + "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY" + ] + } + ] + } + ``` + + 1. This permission is ONLY needed if you use a custom KMS encryption key for your secret + +* Retrieve the execution role name of your Lambda function with the following command: + + ```bash + aws lambda get-function-configuration --function-name THE_NAME_OF_YOUR_FUNCTION | grep "Role" + ``` + +* Attach the new policy to the execution role: + + ```bash + aws iam put-role-policy \ + --role-name NAME_OR_ARN_OF_THE_EXECUTION_ROLE \ + --policy-name SecretsManagerAPMKeyPolicy \ + --policy-document file://smElasticApmPolicy.json + ``` +:::::: + +::::::{tab-item} SAM +In your SAM `template.yml` file add the following policy to give the Lambda function access to the stored secret: + +```yaml +... +AWSTemplateFormatVersion: '2010-09-09' +Transform: 'AWS::Serverless-2016-10-31' +Resources: + yourLambdaFunction: + Type: 'AWS::Serverless::Function' + Properties: + ... + Policies: + - Statement: + - Sid: SecretsManagerGetSecretValue + Effect: Allow + Action: + - secretsmanager:GetSecretValue + Resource: 'THE_ARN_OF_YOUR_SECRET' + - Sid: KMSDecrypt <1> + Effect: Allow + Action: + - kms:Decrypt + Resource: 'THE_ARN_OF_YOUR_CUSTOM_KMS_KEY' +... +``` + +1. This permission is ONLY needed if you use a custom KMS encryption key for your secret +:::::: + +::::::{tab-item} Serverless +In your serverless file add the following policy to give the Lambda function access to the stored secret: + +```yaml +... +provider: + iam: + role: + statements: + - Effect: Allow + Action: 'secretsmanager:GetSecretValue' + Resource: 'THE_ARN_OF_YOUR_SECRET' + - Effect: Allow <1> + Action: 'kms:Decrypt' + Resource: 'THE_ARN_OF_YOUR_CUSTOM_KMS_KEY' +... +``` + +1. This permission is ONLY needed if you use a custom KMS encryption key for your secret +:::::: + +::::::{tab-item} Terraform +Add the following policy and policy attachement resources to your terraform file to give the Lambda function access to the stored secret: + +```yaml +... +resource "aws_iam_role" "lambda_role" { + // Here is your Lambda execution role + ... +} + +resource "aws_iam_policy" "secrets_manager_elastic_apm_policy" { + name = "secrets_manager_elastic_apm_policy" + description = "Allows the lambda function to access the APM authentication key stored in AWS Secrets Manager." + + policy = < + "Action": [ + "kms:Decrypt" + ], + "Effect": "Allow", + "Resource": "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY" + } + ] +} +EOF +} + +resource "aws_iam_policy_attachment" "secrets_manager_elastic_apm_policy_attach" { + role = aws_iam_role.lambda_role.name + policy_arn = aws_iam_policy.secrets_manager_elastic_apm_policy.arn +} +... +``` + +1. This permission is ONLY needed if you use a custom KMS encryption key for your secret +:::::: + +::::::: + +## Step 3: Configure the {{apm-lambda-ext}} [_step_3_configure_the_apm_lambda_ext] + +Finally, you will need to configure the {{apm-lambda-ext}} to use the secret from the Secrets Manager *instead of* the value provided through [`ELASTIC_APM_SECRET_TOKEN` or `ELASTIC_APM_API_KEY`](/reference/aws-lambda-config-options.md#aws-lambda-config-authentication-keys). + +Provide the name of the secret you created in [Step 1](#aws-lambda-secrets-manager-create-secret) as the value for the [`ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID` or `ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID`](/reference/aws-lambda-config-options.md#aws-lambda-config-secrets-manager-options) config option, respectively, depending on whether you want to use the [APM Secret Token](docs-content://solutions/observability/apps/secret-token.md) or the [APM API key](docs-content://solutions/observability/apps/api-keys.md). + +The language-specific instructions describe how to set environment variables for configuring AWS Lambda for Elastic APM: + +* [Configure APM on AWS Lambda - Node.js](apm-agent-nodejs://reference/lambda.md#_step_3_configure_apm_on_aws_lambda) +* [Configure APM on AWS Lambda - Python](apm-agent-python://reference/lambda-support.md#_step_3_configure_apm_on_aws_lambda) +* [Configure APM on AWS Lambda - Java](apm-agent-java://reference/aws-lambda.md#_step_3_configure_apm_on_aws_lambda) + +That’s it. With the first invocation (cold start) of your Lambda function you should see a log message from the {{apm-lambda-ext}} indicating that a secret from the secrets manager is used: + +```yaml +"Using the APM secret token retrieved from Secrets Manager." +``` + diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 00000000..0476d64b --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,21 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/lambda/current/aws-lambda-arch.html + - https://www.elastic.co/guide/en/apm/lambda/current/index.html +--- + +# APM Architecture for AWS Lambda [aws-lambda-arch] + +AWS Lambda uses a special execution model to provide a scalable, on-demand compute service for code execution. In particular, AWS freezes the execution environment of a lambda function when no active requests are being processed. This execution model poses additional requirements on APM in the context of AWS Lambda functions: + +1. To avoid data loss, APM data collected by APM agents needs to be flushed before the execution environment of a lambda function is frozen. +2. Flushing APM data must be fast so as not to impact the response times of lambda function requests. + +To accomplish the above, Elastic APM agents instrument AWS Lambda functions and dispatch APM data via an [AWS Lambda extension](https://docs.aws.amazon.com/lambda/latest/dg/using-extensions.md). + +Normally, during the execution of a Lambda function, there’s only a single language process running in the AWS Lambda execution environment. With an AWS Lambda extension, Lambda users run a *second* process alongside their main service/application process. + +![image showing data flow from lambda function](../images/architecture-white.png "") + +By using an AWS Lambda extension, Elastic APM agents can send data to a local Lambda extension process, and that process will forward data on to APM Server asynchronously. The Lambda extension ensures that any potential latency between the Lambda function and the APM Server instance will not cause latency in the request flow of the Lambda function itself. + diff --git a/docs/reference/toc.yml b/docs/reference/toc.yml new file mode 100644 index 00000000..1dcfbe50 --- /dev/null +++ b/docs/reference/toc.yml @@ -0,0 +1,6 @@ +project: 'APM architecture for AWS Lambda reference' +toc: + - file: index.md + - file: aws-lambda-overhead.md + - file: aws-lambda-config-options.md + - file: aws-lambda-secrets-manager.md \ No newline at end of file diff --git a/docs/release-notes.asciidoc b/docs/release-notes.asciidoc deleted file mode 100644 index 49a370da..00000000 --- a/docs/release-notes.asciidoc +++ /dev/null @@ -1,6 +0,0 @@ -[[aws-lambda-release-notes]] -== Release notes - -This section summarizes the changes in each release. - -include::../CHANGELOG.asciidoc[] diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md new file mode 100644 index 00000000..0fe3bba4 --- /dev/null +++ b/docs/release-notes/index.md @@ -0,0 +1,110 @@ +--- +navigation_title: "Elastic APM AWS Lambda Extension" +mapped_pages: + - https://www.elastic.co/guide/en/apm/lambda/current/aws-lambda-release-notes.html +--- + +# Elastic APM AWS Lambda Extension release notes [elastic-apm-aws-lambda-extension-release-notes] + +Review the changes, fixes, and more in each version of Elastic APM AWS Lambda Extension. + +To check for security updates, go to [Security announcements for the Elastic stack](https://discuss.elastic.co/c/announcements/security-announcements/31). + +% Release notes include only features, enhancements, and fixes. Add breaking changes, deprecations, and known issues to the applicable release notes sections. + +% ## version.next [elastic-apm-aws-lambda-extension-versionext-release-notes] +% **Release date:** Month day, year + +% ### Features and enhancements [elastic-apm-aws-lambda-extension-versionext-features-enhancements] + +% ### Fixes [elastic-apm-aws-lambda-extension-versionext-fixes] + +## 1.5.1 [elastic-apm-aws-lambda-extension-151-release-notes] +**Release date:** October 6, 2023 + +### Fixes [elastic-apm-aws-lambda-extension-151-fixes] +* Fix incorrect proxy transaction handling at shutdown due to not flushing the data before processing shutdown event. [412](https://github.com/elastic/apm-aws-lambda/pull/412). + +## 1.5.0 [elastic-apm-aws-lambda-extension-150-release-notes] +**Release date:** September 13, 2023 + +### Features and enhancements [elastic-apm-aws-lambda-extension-150-features-enhancements] +* Use User-Agent header with Lambda extension version and propagate info from apm agents [404](https://github.com/elastic/apm-aws-lambda/pull/404) + +### Fixes [elastic-apm-aws-lambda-extension-150-fixes] +* Log a warning, instead of failing a Lambda function, if auth retrieval from AWS Secrets Manager fails. Reporting APM data will not work, but the Lambda function invocations will proceed. [401](https://github.com/elastic/apm-aws-lambda/pull/401) + +## 1.4.0 [elastic-apm-aws-lambda-extension-140-release-notes] +**Release date:** May 3, 2023 + +### Features and enhancements [elastic-apm-aws-lambda-extension-150-features-enhancements] +* [preview] Allow metadata in register transaction [384](https://github.com/elastic/apm-aws-lambda/pull/384) + +## 1.3.1 [elastic-apm-aws-lambda-extension-131-release-notes] +**Release date:** April 4, 2023 + +### Fixes [elastic-apm-aws-lambda-extension-131-fixes] +* Print response body on error if decoding fails [382](https://github.com/elastic/apm-aws-lambda/pull/382) + +## 1.3.0 [elastic-apm-aws-lambda-extension-130-release-notes] +**Release date:** April 22, 2023 + +### Features and enhancements [elastic-apm-aws-lambda-extension-130-features-enhancements] +* [preview] Create proxy transaction with error results if not reported by agent [315](https://github.com/elastic/apm-aws-lambda/pull/315) +* Wait for the final platform report metrics on shutdown [347](https://github.com/elastic/apm-aws-lambda/pull/347) +* Process platform report metrics when extension is lagging [358](https://github.com/elastic/apm-aws-lambda/pull/358) +* Add TLS support [357](https://github.com/elastic/apm-aws-lambda/pull/357) + +## 1.2.0 [elastic-apm-aws-lambda-extension-120-release-notes] +**Release date:** November 1, 2022 + +### Features and enhancements [elastic-apm-aws-lambda-extension-120-features-enhancements] +* Parse and log APM Server error responses, and backoff on critical errors [281](https://github.com/elastic/apm-aws-lambda/pull/281) +* Disable CGO to prevent libc/ABI compatibility issues [292](https://github.com/elastic/apm-aws-lambda/pull/292) +* Deprecate `ELASTIC_APM_DATA_RECEIVER_TIMEOUT_SECONDS` in favour of `ELASTIC_APM_DATA_RECEIVER_TIMEOUT` [294](https://github.com/elastic/apm-aws-lambda/pull/294) +* Log shutdown reason on exit [297](https://github.com/elastic/apm-aws-lambda/pull/297) +* Add support for collecting and shipping function logs to APM Server [303](https://github.com/elastic/apm-aws-lambda/pull/303) +* Batch data collected from lambda logs API before sending to APM Server [314](https://github.com/elastic/apm-aws-lambda/pull/314) + +### Fixes [elastic-apm-aws-lambda-extension-120-fixes] +* Fix possible data corruption while processing multiple log events [309](https://github.com/elastic/apm-aws-lambda/pull/309) + +## 1.1.0 [elastic-apm-aws-lambda-extension-110-release-notes] +**Release date:** August 24, 2022 + +### Features and enhancements [elastic-apm-aws-lambda-extension-110-features-enhancements] +* Added support for Secret Manager [208](https://github.com/elastic/apm-aws-lambda/pull/208) +* Added support for Lambda platform metrics [202](https://github.com/elastic/apm-aws-lambda/pull/202) +* Migrated to AWS SDK for Go v2 [232](https://github.com/elastic/apm-aws-lambda/pull/232) +* Make buffer size for agent data configurable [262](https://github.com/elastic/apm-aws-lambda/pull/262) +* Add support for reproducible builds [237](https://github.com/elastic/apm-aws-lambda/pull/237) +* Improve extension client error messages [259](https://github.com/elastic/apm-aws-lambda/pull/259) + +### Fixes [elastic-apm-aws-lambda-extension-110-fixes] +* Log a warning when authentication with APM Server fails [228](https://github.com/elastic/apm-aws-lambda/pull/228) +* Handle http.ErrServerClosed correctly [234](https://github.com/elastic/apm-aws-lambda/pull/234) +* Handle main loop errors correctly [252](https://github.com/elastic/apm-aws-lambda/pull/252) +* Avoid sending corrupted compressed data to APM Server [257](https://github.com/elastic/apm-aws-lambda/pull/257) +* Avoid creating http transports on each info request [260](https://github.com/elastic/apm-aws-lambda/pull/260) +* Randomise the initial grace period to avoid collisions [240](https://github.com/elastic/apm-aws-lambda/pull/240) +* Handle metadata errors correctly [254](https://github.com/elastic/apm-aws-lambda/pull/254) +* Always flush data to APM server before shutting down and avoid concurrent access to data channel [258](https://github.com/elastic/apm-aws-lambda/pull/258) + +## 1.0.2 [elastic-apm-aws-lambda-extension-102-release-notes] +**Release date:** June 9, 2022 + +### Fixes [elastic-apm-aws-lambda-extension-102-fixes] +* Only add executables to extension [216](https://github.com/elastic/apm-aws-lambda/pull/216) + +## 1.0.1 [elastic-apm-aws-lambda-extension-101-release-notes] +**Release date:** June 3, 2022 + +### Features and enhancements [elastic-apm-aws-lambda-extension-101-features-enhancements] +* Add support for building and pushing docker images [199](https://github.com/elastic/apm-aws-lambda/pull/199) + +## 1.0.0 [elastic-apm-aws-lambda-extension-100-release-notes] +**Release date:** April 26, 2022 + +### Features and enhancements [elastic-apm-aws-lambda-extension-100-features-enhancements] +* First stable release of the Elastic APM AWS Lambda extension. + diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md new file mode 100644 index 00000000..2005c561 --- /dev/null +++ b/docs/release-notes/known-issues.md @@ -0,0 +1,20 @@ +--- +navigation_title: "Elastic APM AWS Lambda Extension" + +--- + +# Elastic APM AWS Lambda Extension known issues [elastic-apm-aws-lambda-extension-known-issues] + +% Use the following template to add entries to this page. + +% :::{dropdown} Title of known issue +% **Details** +% On [Month/Day/Year], a known issue was discovered that [description of known issue]. + +% **Workaround** +% Workaround description. + +% **Resolved** +% On [Month/Day/Year], this issue was resolved. + +::: \ No newline at end of file diff --git a/docs/release-notes/toc.yml b/docs/release-notes/toc.yml new file mode 100644 index 00000000..3bbe7c82 --- /dev/null +++ b/docs/release-notes/toc.yml @@ -0,0 +1,3 @@ +toc: + - file: index.md + - file: known-issues.md \ No newline at end of file diff --git a/docs/secrets-manager/add-permissions-content.asciidoc b/docs/secrets-manager/add-permissions-content.asciidoc deleted file mode 100644 index 5b4f7222..00000000 --- a/docs/secrets-manager/add-permissions-content.asciidoc +++ /dev/null @@ -1,191 +0,0 @@ -// tag::console[] -- In the Web Console of your AWS Lambda function navigate to `Configuration` -> `Permissions` and click on the link in the `Execution Role` section. -This will lead you to the Web Console for the corresponding IAM role. -- Select `Create inline policy` in the `Add permission` dropdown button and select the `JSON` tab to define the following JSON-based policy: -+ --- -[source,json] ----- -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "secretsmanager:GetSecretValue" - ], - "Resource": [ - "THE_ARN_OF_YOUR_SECRET" - ] - }, - { <1> - "Effect": "Allow", - "Action": [ - "kms:Decrypt" - ], - "Resource": [ - "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY" - ] - } - ] -} ----- -<1> This permission is ONLY needed if you use a custom KMS encryption key for your secret --- -- Review and save the policy. - -// end::console[] - -// tag::cli[] -- Create a policy file (`smElasticApmPolicy.json`) with the following content: -+ --- -[source,json] ----- -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "secretsmanager:GetSecretValue" - ], - "Resource": [ - "THE_ARN_OF_YOUR_SECRET" - ] - }, - { <1> - "Effect": "Allow", - "Action": [ - "kms:Decrypt" - ], - "Resource": [ - "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY" - ] - } - ] -} ----- -<1> This permission is ONLY needed if you use a custom KMS encryption key for your secret --- -- Retrieve the execution role name of your Lambda function with the following command: -+ --- -[source,bash] ----- -aws lambda get-function-configuration --function-name THE_NAME_OF_YOUR_FUNCTION | grep "Role" ----- --- -- Attach the new policy to the execution role: -+ --- -[source,bash] ----- -aws iam put-role-policy \ - --role-name NAME_OR_ARN_OF_THE_EXECUTION_ROLE \ - --policy-name SecretsManagerAPMKeyPolicy \ - --policy-document file://smElasticApmPolicy.json ----- --- -// end::cli[] - -// tag::sam[] - -In your SAM `template.yml` file add the following policy to give the Lambda function access to the stored secret: - -[source,yml] ----- -... -AWSTemplateFormatVersion: '2010-09-09' -Transform: 'AWS::Serverless-2016-10-31' -Resources: - yourLambdaFunction: - Type: 'AWS::Serverless::Function' - Properties: - ... - Policies: - - Statement: - - Sid: SecretsManagerGetSecretValue - Effect: Allow - Action: - - secretsmanager:GetSecretValue - Resource: 'THE_ARN_OF_YOUR_SECRET' - - Sid: KMSDecrypt <1> - Effect: Allow - Action: - - kms:Decrypt - Resource: 'THE_ARN_OF_YOUR_CUSTOM_KMS_KEY' -... ----- -<1> This permission is ONLY needed if you use a custom KMS encryption key for your secret - -// end::sam[] - -// tag::serverless[] -In your serverless file add the following policy to give the Lambda function access to the stored secret: -[source,yml] ----- -... -provider: - iam: - role: - statements: - - Effect: Allow - Action: 'secretsmanager:GetSecretValue' - Resource: 'THE_ARN_OF_YOUR_SECRET' - - Effect: Allow <1> - Action: 'kms:Decrypt' - Resource: 'THE_ARN_OF_YOUR_CUSTOM_KMS_KEY' -... ----- -<1> This permission is ONLY needed if you use a custom KMS encryption key for your secret - -// end::serverless[] - -// tag::terraform[] -Add the following policy and policy attachement resources to your terraform file to -give the Lambda function access to the stored secret: -[source,terraform] ----- -... -resource "aws_iam_role" "lambda_role" { - // Here is your Lambda execution role - ... -} - -resource "aws_iam_policy" "secrets_manager_elastic_apm_policy" { - name = "secrets_manager_elastic_apm_policy" - description = "Allows the lambda function to access the APM authentication key stored in AWS Secrets Manager." - - policy = < - "Action": [ - "kms:Decrypt" - ], - "Effect": "Allow", - "Resource": "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY" - } - ] -} -EOF -} - -resource "aws_iam_policy_attachment" "secrets_manager_elastic_apm_policy_attach" { - role = aws_iam_role.lambda_role.name - policy_arn = aws_iam_policy.secrets_manager_elastic_apm_policy.arn -} -... ----- -<1> This permission is ONLY needed if you use a custom KMS encryption key for your secret - -// end::terraform[] diff --git a/docs/secrets-manager/add-permissions-widget.asciidoc b/docs/secrets-manager/add-permissions-widget.asciidoc deleted file mode 100644 index 609d1485..00000000 --- a/docs/secrets-manager/add-permissions-widget.asciidoc +++ /dev/null @@ -1,99 +0,0 @@ -++++ -
-
- - - - - -
-
-++++ - -include::add-permissions-content.asciidoc[tag=console] - -++++ -
- - - - -
-++++ \ No newline at end of file diff --git a/docs/secrets-manager/create-secret-content.asciidoc b/docs/secrets-manager/create-secret-content.asciidoc deleted file mode 100644 index d778fb37..00000000 --- a/docs/secrets-manager/create-secret-content.asciidoc +++ /dev/null @@ -1,58 +0,0 @@ -// tag::console[] - -In the Web Console of the AWS Secrets Manager, navigate to `Store a new secret`, select `Other type of secret` as the secret type, and choose the `Plaintext` tab for entering the secret value. -Enter your APM secret token or APM API key value as a plain string (*not* as a JSON key value pair). - -image:images/secrets-manager.png[image showing the creation of a plaintext secret in Secrets Manager] - -Click `Next`, choose a secret name, and finalize the creation of the secret. - -// end::console[] - -// tag::cli[] -Execute the following command to create a secret in the AWS Secrets Manager for your APM secret token or the APM API key: - -[source,bash] ----- -aws secretsmanager create-secret \ - --name MyAPMSecretToken \ <1> - --description "The Elastic APM secret token." \ - --secret-string "YOUR_APM_AUTHENTICATION_KEY" <2> ----- -<1> Choose a secret name -<2> APM secret token or APM API key - -// end::cli[] - -// tag::sam[] - -The AWS Serverless Application Model (SAM) does not provide means to create a secret in the AWS Secrets Manager. Use one of the other options to create the secret. - -// end::sam[] - -// tag::serverless[] - -The Serverless Framework does not provide means to create a secret in the AWS Secrets Manager. Use one of the other options to create the secret. - -// end::serverless[] - -// tag::terraform[] -Add the following resources to your terraform file to create a secret in the AWS Secrets Manager for your APM secret token or the APM API key: - -[source,terraform] ----- -... -resource "aws_secretsmanager_secret" "apm_secret_token" { - name = "MyAPMSecretToken" <1> -} - -resource "aws_secretsmanager_secret_version" "apm_secret_token_version" { - secret_id = aws_secretsmanager_secret.apm_secret_token.id - secret_string = "YOUR_APM_AUTHENTICATION_KEY" <2> -} -... ----- -<1> Choose a secret name -<2> APM secret token or APM API key - -// end::terraform[] diff --git a/docs/secrets-manager/create-secret-widget.asciidoc b/docs/secrets-manager/create-secret-widget.asciidoc deleted file mode 100644 index 7ad716a1..00000000 --- a/docs/secrets-manager/create-secret-widget.asciidoc +++ /dev/null @@ -1,99 +0,0 @@ -++++ -
-
- - - - - -
-
-++++ - -include::create-secret-content.asciidoc[tag=console] - -++++ -
- - - - -
-++++ \ No newline at end of file