What is the meaning of setting custom value for Time Range in "Set Sync Policy"

[AnkeshThakur]

I want to understand how does setting Time Range value impact overall data collection and analysis. Specifically,

• Does it set the beginning date for data collection in each sync cycle? In other word it sets the look-back period's initial date in each sync
• Or is it just for the first time sync?
• Would it capture the data if I set this period to be 3 months and PRs which are older than 3 months are now getting merged?

[dosubot[bot]]

Setting a custom Time Range in "Set Sync Policy" defines the earliest date from which DevLake will collect data during syncs. This Time Range acts as the starting point for the initial historical data collection window. After the first successful sync, DevLake switches to incremental syncs, using the last successful sync time as the new lower bound for future data collection, not the original Time Range start date. So, the Time Range is mainly relevant for the first sync or if you reset the sync policy to an earlier date source.

For most data sources (like PRs), the Time Range filter is applied to the creation time. This means if you set the period to 3 months, PRs created more than 3 months ago but merged now will not be collected by the main collector, because their creation date is outside the window source.

However, DevLake has a secondary mechanism: if incremental sync is enabled and the plugin supports it, there is a secondary collector that can pick up updates to "unfinished" or recently updated records (like open PRs), even if they were created before the Time Range. This helps capture late merges or updates, but only for records that are still open or have recent activity source.

Here's a simplified diagram of the sync logic:

graph TD
    A["Set Time Range (Start Date)"] --> B["First Sync: Collect data created after Start Date"]
    B --> C["Incremental Syncs: Use last successful sync time as new lower bound"]
    C --> D["Secondary Collector: Picks up updates to old records if supported"]

Loading

If you want to ensure you capture late updates to old PRs, consider setting a wider initial Time Range or check if your plugin supports the secondary collector for unfinished/updated records.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}