Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/actions/rust-build/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ runs:
if [ "${{ inputs.toolchain }}" != stable ]; then
rm -fv Cargo.lock
fi
cargo build --all-features --verbose
cargo build --verbose ${{ inputs.flags }}
- name: Run tests
shell: bash
run: cargo test --all-features --verbose
run: |
cargo test --verbose ${{ inputs.flags }}
6 changes: 6 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ jobs:
toolchain:
- "1.85" # Current MSRV due to Cargo MSRV feature
- stable
flags:
- "--all-features"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was surprised to see that these passed out of box, but I guess we don't actually test any network calls? Looks like we validate metadata parsing only, and nothing to do with s3?

Probably having an integration test that validates these directly e2e using real auth would be nice, but separate scope. I can cut an issue (low priority).

Without that, can you confirm that you've manually validated that the existing functionality (without the new features) works properly? IE no regression for old APIs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, the tests don't have network access. I'm not in the mood of setting up an AWS account so I can do AWS-in-the-loop tests.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, we can maybe do #47 someday

From offline - thanks for confirming that you tested locally e2e and existin gfunctionality works.

- "--no-default-features"
- "--no-default-features --features=s3-no-defaults"
- "--no-default-features --features=aws-metadata-no-defaults"
env:
RUST_BACKTRACE: 1
steps:
Expand All @@ -18,6 +23,7 @@ jobs:
uses: ./.github/actions/rust-build
with:
toolchain: ${{ matrix.toolchain }}
flags: ${{ matrix.flags }}
build-for-testing:
name: Build for testing
runs-on: ubuntu-latest
Expand Down
32 changes: 16 additions & 16 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 12 additions & 7 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,19 @@ edition = "2021"

[dependencies]
async-trait = "0.1"
# not really an AWS crate so no need to disable default features
aws-arn = { version = "0.3", optional = true }
aws-config = { version = "1", optional = true }
aws-sdk-s3 = { version = "1", optional = true }
aws-config = { version = "1", optional = true, default-features = false }
aws-sdk-s3 = { version = "1", optional = true, default-features = false }
chrono = "0.4"
futures = { version = "0.3", default-features = false }
futures = { version = "0.3", default-features = false, features = ["alloc"] }
libloading = "0.8"
reqwest = { version = "0.12", default-features = false, optional = true, features = ["charset", "http2", "rustls-tls"] }
reqwest = { version = "0.12", default-features = false, optional = true, features = ["charset", "http2"] }
serde_json = "1"
serde = { version = "1", features = ["derive"] }
tempfile = "3"
thiserror = "2"
tokio = { version = "1", features = ["macros"], optional = true }
tokio = { version = "1", features = ["fs", "macros", "rt", "sync", "time"] }
tracing = "0.1"
zip = { version = "3", default-features = false, features = ["deflate"] }

Expand All @@ -38,5 +39,9 @@ name = 'simple'

[features]
default = ["s3", "aws-metadata"]
s3 = ["dep:aws-config", "dep:aws-sdk-s3", "dep:tokio"]
aws-metadata = ["dep:reqwest", "dep:aws-config", "dep:aws-arn"]
s3 = ["s3-no-defaults", "aws-config/default", "aws-sdk-s3/default"]
# A version of the s3 feature that does not enable AWS default features
s3-no-defaults = ["dep:aws-config", "dep:aws-sdk-s3"]
aws-metadata = ["aws-metadata-no-defaults", "aws-config/default", "reqwest/rustls-tls"]
# A version of the aws-metadata feature that does not enable AWS default features
aws-metadata-no-defaults = ["dep:reqwest", "dep:aws-config", "dep:aws-arn"]
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@ emit `tokio.PollCatchV1` events this way:
}
```

### Not enabling the AWS SDK / Reqwest default features

The `aws-metadata-no-defaults` and `s3-no-defaults` feature flags do not enable feature flags for the AWS SDK and `reqwest`.

If you want things to work, you'll need to enable features for these crates to allow at least a TLS provider. This can be used to use a TLS provider other than the default Rustls

## Decoder

The `decoder` directory in the Git repository contains a decoder that can be used to view JFR files, especially with PollCatch information.
Expand Down
72 changes: 46 additions & 26 deletions examples/simple/main.rs
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0

#[cfg(feature = "s3-no-defaults")]
use async_profiler_agent::reporter::s3::{S3Reporter, S3ReporterConfig};
use async_profiler_agent::{
metadata::AgentMetadata,
profiler::{ProfilerBuilder, ProfilerOptionsBuilder},
reporter::{
local::LocalReporter,
s3::{S3Reporter, S3ReporterConfig},
},
reporter::local::LocalReporter,
};
use std::time::Duration;

#[cfg(feature = "s3-no-defaults")]
use aws_config::BehaviorVersion;
use clap::{ArgGroup, Parser};

Expand All @@ -29,20 +29,27 @@ pub fn set_up_tracing() {
.init();
}

#[derive(Clone, Debug, Default, Parser)]
struct S3BucketArgs {
#[arg(long, requires = "bucket_owner", requires = "profiling_group")]
bucket: Option<String>,
#[arg(long)]
bucket_owner: Option<String>,
#[arg(long)]
profiling_group: Option<String>,
}

/// Simple program to test the profiler agent
#[derive(Parser, Debug)]
#[derive(Debug, Parser)]
#[command(group(
ArgGroup::new("options")
.required(true)
.args(["local", "bucket"]),
))]
struct Args {
#[arg(long)]
profiling_group: Option<String>,
#[arg(long)]
bucket_owner: Option<String>,
#[arg(long, requires = "bucket_owner", requires = "profiling_group")]
bucket: Option<String>,
#[cfg(feature = "s3-no-defaults")]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these feature flags make the example very hard to follow. Can we move all s3 features into their own struct, #[derive(Arg)] on it, and flatten it from the parent struct? That way we can just feature gate the one field in the top-level struct.

#[command(flatten)]
bucket_args: S3BucketArgs,
#[arg(long)]
local: Option<String>,
#[arg(long)]
Expand All @@ -57,6 +64,18 @@ struct Args {
native_mem: Option<String>,
}

impl Args {
#[cfg(feature = "s3-no-defaults")]
fn s3_bucket_args(&self) -> S3BucketArgs {
self.bucket_args.clone()
}

#[cfg(not(feature = "s3-no-defaults"))]
fn s3_bucket_args(&self) -> S3BucketArgs {
S3BucketArgs::default()
}
}

#[allow(unexpected_cfgs)]
pub fn main() -> anyhow::Result<()> {
let args = Args::parse();
Expand All @@ -82,23 +101,24 @@ async fn main_internal(args: Args) -> Result<(), anyhow::Error> {

let profiler = ProfilerBuilder::default();

let profiler = match (
args.local,
args.bucket,
args.bucket_owner,
args.profiling_group,
) {
(Some(local), _, _, _) => profiler
let profiler = match (&args.local, args.s3_bucket_args()) {
(Some(local), S3BucketArgs { .. }) => profiler
.with_reporter(LocalReporter::new(local))
.with_custom_agent_metadata(AgentMetadata::Other),
(_, Some(bucket), Some(bucket_owner), Some(profiling_group)) => {
profiler.with_reporter(S3Reporter::new(S3ReporterConfig {
sdk_config: &aws_config::defaults(BehaviorVersion::latest()).load().await,
bucket_owner: bucket_owner,
bucket_name: bucket,
profiling_group_name: profiling_group,
}))
}
#[cfg(feature = "s3-no-defaults")]
(
_,
S3BucketArgs {
bucket: Some(bucket_name),
bucket_owner: Some(bucket_owner),
profiling_group: Some(profiling_group_name),
},
) => profiler.with_reporter(S3Reporter::new(S3ReporterConfig {
sdk_config: &aws_config::defaults(BehaviorVersion::latest()).load().await,
bucket_owner,
bucket_name,
profiling_group_name,
})),
_ => unreachable!(),
};

Expand Down
3 changes: 2 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@
//!
//! You can use the [`S3Reporter`], which uploads the reports to an S3 bucket, as follows:
//!
//! ```no_run
#![cfg_attr(feature = "s3-no-defaults", doc = "```no_run")]
#![cfg_attr(not(feature = "s3-no-defaults"), doc = "```compile_fail")]
//! # use async_profiler_agent::profiler::{ProfilerBuilder, SpawnError};
//! # use async_profiler_agent::reporter::s3::{S3Reporter, S3ReporterConfig};
//! # use aws_config::BehaviorVersion;
Expand Down
2 changes: 1 addition & 1 deletion src/metadata/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ pub struct ReportMetadata<'a> {
pub reporting_interval: Duration,
}

#[cfg(feature = "aws-metadata")]
#[cfg(feature = "aws-metadata-no-defaults")]
pub mod aws;

/// [private] dummy metadata to make testing easier
Expand Down
5 changes: 3 additions & 2 deletions src/profiler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

use crate::{
asprof::{self, AsProfError},
metadata::{aws::AwsProfilerMetadataError, AgentMetadata, ReportMetadata},
metadata::{AgentMetadata, ReportMetadata},
reporter::Reporter,
};
use std::{
Expand Down Expand Up @@ -311,7 +311,8 @@ enum TickError {
#[error(transparent)]
AsProf(#[from] AsProfError),
#[error(transparent)]
Metadata(#[from] AwsProfilerMetadataError),
#[cfg(feature = "aws-metadata-no-defaults")]
Metadata(#[from] crate::metadata::aws::AwsProfilerMetadataError),
#[error("reporter: {0}")]
Reporter(Box<dyn std::error::Error + Send>),
#[error("broken clock: {0}")]
Expand Down
2 changes: 1 addition & 1 deletion src/reporter/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ use crate::metadata::ReportMetadata;

pub mod local;
pub mod multi;
#[cfg(feature = "s3")]
#[cfg(feature = "s3-no-defaults")]
pub mod s3;

/// Abstraction around reporting profiler data.
Expand Down
Loading