fix(profiling): Ingest profile file path #4060
Conversation
`ingest-profiles` is now using vroomrs to ingest profiles instead of writing through vroom. For self-hosted, we need to make sure filestore for profiles is properly configured so vroom can find the ingested profiles.
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## master #4060 +/- ##
===========================================
- Coverage 99.49% 88.83% -10.66%
===========================================
Files 3 3
Lines 197 197
===========================================
- Hits 196 175 -21
- Misses 1 22 +21 ☔ View full report in Codecov by Sentry. |
BYK
left a comment
BYK
left a comment
There was a problem hiding this comment.
Looks okay but without any tests, hard to be sure. Also added some comments I'd like to see addressed.
| # 3. Point `filestore-profiles` and vroom to the SeaweedFS "profiles" bucket. | ||
|
|
||
| start_service_and_wait_ready seaweedfs | ||
| $dc exec -e "HTTP_PROXY=${HTTP_PROXY:-}" -e "HTTPS_PROXY=${HTTPS_PROXY:-}" -e "NO_PROXY=${NO_PROXY:-}" -e "http_proxy=${http_proxy:-}" -e "https_proxy=${https_proxy:-}" -e "no_proxy=${no_proxy:-}" seaweedfs apk add --no-cache s3cmd |
There was a problem hiding this comment.
@aldy505 we should have a shortcut for this monstrosity:
$dc exec -e "HTTP_PROXY=${HTTP_PROXY:-}" -e "HTTPS_PROXY=${HTTPS_PROXY:-}" -e "NO_PROXY=${NO_PROXY:-}" -e "http_proxy=${http_proxy:-}" -e "https_proxy=${https_proxy:-}" -e "no_proxy=${no_proxy:-}"
There was a problem hiding this comment.
Fun fact, there's a little problem. Surprisingly seaweedfs only reads the lowercased http_proxy :)
curses at myself
| # 3. Point `filestore-profiles` and vroom to the SeaweedFS "profiles" bucket. | ||
|
|
||
| start_service_and_wait_ready seaweedfs | ||
| $dc exec -e "HTTP_PROXY=${HTTP_PROXY:-}" -e "HTTPS_PROXY=${HTTPS_PROXY:-}" -e "NO_PROXY=${NO_PROXY:-}" -e "http_proxy=${http_proxy:-}" -e "https_proxy=${https_proxy:-}" -e "no_proxy=${no_proxy:-}" seaweedfs apk add --no-cache s3cmd |
There was a problem hiding this comment.
Fun fact, there's a little problem. Surprisingly seaweedfs only reads the lowercased http_proxy :)
curses at myself
| SENTRY_KAFKA_BROKERS_PROFILING: "kafka:9092" | ||
| SENTRY_KAFKA_BROKERS_OCCURRENCES: "kafka:9092" | ||
| SENTRY_BUCKET_PROFILES: file:///var/vroom/sentry-profiles | ||
| SENTRY_BUCKET_PROFILES: "s3://profiles?region=us-east-1&endpoint=seaweedfs:8333&s3ForcePathStyle=true&disableSSL=true" |
There was a problem hiding this comment.
Just realized I need to add AWS_ACCESS_KEY and AWS_SECRET_KEY too
Co-authored-by: Burak Yigit Kaya <[email protected]>
![sentry[bot]](https://avatars.githubusercontent.com/u/1396951?s=60&v=4){kind=small}
| echo "'profiles' bucket already exists on SeaweedFS. Skipping creation." | ||
| fi | ||
|
|
||
| if [[ -z "${APPLY_AUTOMATIC_CONFIG_UPDATES:-}" || "$APPLY_AUTOMATIC_CONFIG_UPDATES" == 1 ]]; then |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
This is right, but you're missing a context. This will not be executed if the user set APPLY_AUTOMATIC_CONFIG_UPDATES to 0.
ingest-profilesis now using vroomrs to ingest profiles instead of writing through vroom. For self-hosted, we need to make sure filestore for profiles is properly configured so vroom can find the ingested profiles.Fixes the issue in #4012 (comment)