Skip to content

Commit aeb6577

Browse files
authored
[Step 2/4] Queue Time Histgram (#6467)
# Description Add two main class for queue time histogram lambda: - TimeIntervalGenerator: generate list of time intervals based on the latest timestamp from source table and target table - generate a single data from source table if no timestamp found in target table - throw error if the interval is greater than maxi-interval threshold - WorkerPoolProcessor: - input the interval list from TimeIntervalGenerator, and process histogram data in parrallel - It process each interval by calling the QueueTimeProcessor.process(), and in the end write all result to destination # Others - added unit tests for all utilization methods - clickhouse_client is not thread-safe, make sure each thread has its own client in thread pool # Next steps step 3 - Add HistogramGenerator - Add runner_label logic in HistogramGenerator - Fix the issue with GithubConfigRetriever - Change the target db to misc. instead of fortesting step 4 - setup AWS resources - setup deployment
1 parent 4e8d1bd commit aeb6577

File tree

2 files changed

+989
-221
lines changed

2 files changed

+989
-221
lines changed

0 commit comments

Comments
 (0)