Skip to content

Commit 7877237

Browse files
gabriellegtwgabriellegtw
authored
[#2373] Support datetime for since and until repo-config.csv (#2458)
Support datetime (previously just date) for since and until in repo-config.csv 3 supported formats for since and until dates: 1. DD/MM/YYYY 2. DD/MM/YYYY'T'HH:mm 3. DD/MM/YYYY'T'HH:mm:ss Behavior: For since date, if hour/minute/seconds not supplied then default to 0 for each. For until date, if hour/minute/seconds not supplied then default to 23, 59, 59 respectively.
1 parent 681a756 commit 7877237

11 files changed

+393
-27
lines changed

‎docs/ug/configFiles.md‎

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -31,22 +31,22 @@ Given below are the details of the various config files used by RepoSense.
3131
**`repo-config.csv` file contains repo-level config data.** Each row represents a repository's configuration ([example](repo-config.csv)).
3232

3333

34-
| Column Name | Explanation |
35-
|---------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
36-
| Repository's Location {{ mandatory }} | The `Remote Repo URL` or `Disk Path` to the Git repository e.g., `https://github.com/foo/bar.git` or `C:\Users\user\Desktop\GitHub\foo\bar` |
37-
| Branch | The branch to analyze in the target repository e.g., `master`. Default: the default branch of the repo |
38-
| File formats<sup>*+</sup> | The file extensions to analyze. Binary file formats, such as `png` and `jpg`, will be automatically labelled as the file type `binary` in the generated report. Default: all file formats |
39-
| Find Previous Authors | Enter **`yes`** to utilize Git blame's ignore revisions functionality, RepoSense will attempt to blame the line changes caused by commits in the ignore commit list to the previous authors who altered those lines (if available). |
40-
| Ignore Glob List<sup>*+</sup> | The list of file path globs to ignore during analysis for each author e.g., `test/**;temp/**`. Refer to the [_glob format_](https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob) for the path glob syntax. |
41-
| Ignore standalone config | To ignore the standalone config file (if any) in target repository, enter **`yes`**. If the cell is empty, the standalone config file in the repo (if any) will take precedence over configurations provided in the csv files. |
42-
| Ignore Commits List<sup>*+</sup> | The list of commits to ignore during analysis. For accurate results, the commits should be provided with their full hash. Additionally, a range of commits can be specified using the `..` notation e.g. `abc123..def456` (both inclusive). |
43-
| Ignore Authors List<sup>*+</sup> | The list of authors to ignore during analysis. Authors should be specified by their [Git Author Name](#a-note-about-git-author-name). |
44-
| Shallow Cloning | Enter **`yes`** to clone the repository using Git's shallow cloning functionality. This option can significantly reduce the time taken to clone large repositories. However, the option should ideally be disabled for smaller repositories where the `.git` file is smaller than 500 MB, as it would create overhead. |
45-
| File Size Limit<sup>+</sup> | Enter a file size limit for the repository in bytes as a single number without units (for a size limit of 1MB for example, enter 1000000). This file size limit will override the default file size limit (500KB). Files exceeding the file size limit will be marked as ignored and only the file name and line count will be reflected in the report. |
46-
| Ignore File Size Limit | Enter **`yes`** to ignore both the default file size limit and the file size limit possibly set by the user in `repo-config.csv`. |
47-
| Skip Ignored File Analysis | Enter **`yes`** to ignore analysis of files exceeding the file size limit entirely. If file analysis is skipped, all information about the file will be omitted from the generated report. This option can significantly improve report generation time. |
48-
| Since Date | Enter since date in the format of `yyyy/mm/dd` to signify the start date of analysis. If the field is ignored, the date will be set to the default one or the date indicated in CLI flags |
49-
| Until Date | Enter until date in the format of `yyyy/mm/dd` to signify the end date of analysis. If the field is ignored, the date will be set to the default one or the date indicated in CLI flags |
34+
| Column Name | Explanation |
35+
|---------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
36+
| Repository's Location {{ mandatory }} | The `Remote Repo URL` or `Disk Path` to the Git repository e.g., `https://github.com/foo/bar.git` or `C:\Users\user\Desktop\GitHub\foo\bar` |
37+
| Branch | The branch to analyze in the target repository e.g., `master`. Default: the default branch of the repo |
38+
| File formats<sup>*+</sup> | The file extensions to analyze. Binary file formats, such as `png` and `jpg`, will be automatically labelled as the file type `binary` in the generated report. Default: all file formats |
39+
| Find Previous Authors | Enter **`yes`** to utilize Git blame's ignore revisions functionality, RepoSense will attempt to blame the line changes caused by commits in the ignore commit list to the previous authors who altered those lines (if available). |
40+
| Ignore Glob List<sup>*+</sup> | The list of file path globs to ignore during analysis for each author e.g., `test/**;temp/**`. Refer to the [_glob format_](https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob) for the path glob syntax. |
41+
| Ignore standalone config | To ignore the standalone config file (if any) in target repository, enter **`yes`**. If the cell is empty, the standalone config file in the repo (if any) will take precedence over configurations provided in the csv files. |
42+
| Ignore Commits List<sup>*+</sup> | The list of commits to ignore during analysis. For accurate results, the commits should be provided with their full hash. Additionally, a range of commits can be specified using the `..` notation e.g. `abc123..def456` (both inclusive). |
43+
| Ignore Authors List<sup>*+</sup> | The list of authors to ignore during analysis. Authors should be specified by their [Git Author Name](#a-note-about-git-author-name). |
44+
| Shallow Cloning | Enter **`yes`** to clone the repository using Git's shallow cloning functionality. This option can significantly reduce the time taken to clone large repositories. However, the option should ideally be disabled for smaller repositories where the `.git` file is smaller than 500 MB, as it would create overhead. |
45+
| File Size Limit<sup>+</sup> | Enter a file size limit for the repository in bytes as a single number without units (for a size limit of 1MB for example, enter 1000000). This file size limit will override the default file size limit (500KB). Files exceeding the file size limit will be marked as ignored and only the file name and line count will be reflected in the report. |
46+
| Ignore File Size Limit | Enter **`yes`** to ignore both the default file size limit and the file size limit possibly set by the user in `repo-config.csv`. |
47+
| Skip Ignored File Analysis | Enter **`yes`** to ignore analysis of files exceeding the file size limit entirely. If file analysis is skipped, all information about the file will be omitted from the generated report. This option can significantly improve report generation time. |
48+
| Since Date | Enter since date (with optional time specification) in one of the following formats: <br>• `DD/MM/YYYY`<br>• `DD/MM/YYYY'T'HH:mm`<br>• `DD/MM/YYYY'T'HH:mm:ss`<br><br> This signifies the start date of analysis. If the field is ignored, the date will be set to the default one or the date indicated in CLI flags <br><br> If hours/ minutes/ seconds are not provided, each will default to `00` |
49+
| Until Date | Enter until date (with optional time specification) in one of the following formats: <br>• `DD/MM/YYYY`<br>• `DD/MM/YYYY'T'HH:mm`<br>• `DD/MM/YYYY'T'HH:mm:ss`<br><br> This signifies the end date of analysis. If the field is ignored, the date will be set to the default one or the date indicated in CLI flags <br><br> If hours/ minutes/ seconds are not provided, they will default to `23`, `59`, and `59`, respectively. |
5050

5151
<box type="info" seamless>
5252
The Shallow Cloning option is incompatible with the "--last-modified-date" CLI flag.
@@ -67,13 +67,13 @@ Behavior of since dates and until dates specified in CSV and CLI flags:
6767

6868
**Example**:
6969

70-
CLI Dates: Since: 21/09/2024; Until: 29/01/2025
70+
CLI Dates: Since: 21/09/2024T12:30:00; Until: 29/01/2025T17:50:59
7171

7272
CSV Date Ranges:
7373

74-
Invalid: [01/02/2025, 02/02/2025] (exceeds the CLI "until" date), [01/09/2024, 01/01/2025] (starts before the CLI "since" date)
74+
Invalid: [01/02/2025T14:00:00, 02/02/2025T14:00:00] (exceeds the CLI "until" date), [01/09/2024T13:00:00, 01/01/2025T13:00:00] (starts before the CLI "since" date)
7575

76-
Valid: [30/09/2024, 01/01/2025]
76+
Valid: [30/09/2024T16:30:00, 01/01/2025T16:30:00]
7777

7878
2. CSV Dates Are Fully Provided, but One CLI Flag Is Missing
7979

‎src/main/java/reposense/parser/RepoConfigCsvParser.java‎

Lines changed: 35 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@
44
import java.nio.file.Path;
55
import java.time.LocalDateTime;
66
import java.time.format.DateTimeFormatter;
7+
import java.time.format.DateTimeFormatterBuilder;
78
import java.time.format.DateTimeParseException;
9+
import java.time.temporal.ChronoField;
810
import java.util.List;
911

1012
import org.apache.commons.csv.CSVRecord;
@@ -31,9 +33,6 @@ public class RepoConfigCsvParser extends CsvParser<RepoConfiguration> {
3133
private static final String SHALLOW_CLONING_CONFIG_KEYWORD = "yes";
3234
private static final String FIND_PREVIOUS_AUTHORS_KEYWORD = "yes";
3335

34-
private static final String LOCAL_DATETIME_FORMAT = "dd/MM/yyyy HH:mm:ss";
35-
private static final String DEFAULT_START_TIME = " 00:00:00";
36-
private static final String DEFAULT_END_TIME = " 23:59:59";
3736
private static final String MESSAGE_SINCE_DATE_LATER_THAN_TODAY_DATE =
3837
"\"Since Date\" should not be later than \"Until Date\"";
3938
private static final String MESSAGE_PARSING_INVALID_FORMAT =
@@ -240,10 +239,25 @@ private void checkValidDatesWithCli(LocalDateTime date) throws InvalidDatesExcep
240239
private LocalDateTime extractCsvSinceDate(CSVRecord record) throws InvalidDatesException {
241240
String sinceDateStr = get(record, SINCE_HEADER);
242241
boolean hasUpdatedSinceDateTime = !sinceDateStr.isEmpty();
242+
243+
DateTimeFormatter flexFormatter = new DateTimeFormatterBuilder()
244+
.appendPattern("dd/MM/yyyy")
245+
.optionalStart()
246+
.appendLiteral('T')
247+
.appendPattern("HH:mm")
248+
.optionalStart()
249+
.appendPattern(":ss")
250+
.optionalEnd()
251+
.optionalEnd()
252+
.parseDefaulting(ChronoField.HOUR_OF_DAY, 0)
253+
.parseDefaulting(ChronoField.MINUTE_OF_HOUR, 0)
254+
.parseDefaulting(ChronoField.SECOND_OF_MINUTE, 0)
255+
.toFormatter();
256+
243257
try {
244258
if (hasUpdatedSinceDateTime) {
245-
return LocalDateTime.parse(sinceDateStr + DEFAULT_START_TIME,
246-
DateTimeFormatter.ofPattern(LOCAL_DATETIME_FORMAT));
259+
return LocalDateTime.parse(sinceDateStr,
260+
flexFormatter);
247261
} else {
248262
return null;
249263
}
@@ -260,10 +274,24 @@ private LocalDateTime extractCsvSinceDate(CSVRecord record) throws InvalidDatesE
260274
private LocalDateTime extractCsvUntilDate(CSVRecord record) throws InvalidDatesException {
261275
String untilDateStr = get(record, UNTIL_HEADER);
262276

277+
DateTimeFormatter flexFormatter = new DateTimeFormatterBuilder()
278+
.appendPattern("dd/MM/yyyy")
279+
.optionalStart()
280+
.appendLiteral('T')
281+
.appendPattern("HH:mm")
282+
.optionalStart()
283+
.appendPattern(":ss")
284+
.optionalEnd()
285+
.optionalEnd()
286+
.parseDefaulting(ChronoField.HOUR_OF_DAY, 23)
287+
.parseDefaulting(ChronoField.MINUTE_OF_HOUR, 59)
288+
.parseDefaulting(ChronoField.SECOND_OF_MINUTE, 59)
289+
.toFormatter();
290+
263291
try {
264292
if (!untilDateStr.isEmpty()) {
265-
return LocalDateTime.parse(untilDateStr + DEFAULT_END_TIME,
266-
DateTimeFormatter.ofPattern(LOCAL_DATETIME_FORMAT));
293+
return LocalDateTime.parse(untilDateStr,
294+
flexFormatter);
267295
} else {
268296
return null;
269297
}

0 commit comments

Comments
 (0)