-
Notifications
You must be signed in to change notification settings - Fork 2
Feature/csvstream #115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/csvstream #115
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMHO, the current implementation does not meet the goal of the class/methods that were defined in the design document.
Blackboard: Team_5_GroupProjectProposal_2025_as_submitted.docx
| @@ -0,0 +1,1001 @@ | |||
| id,name,email | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you need to push this output file to the repo?
It does not look related or required.
|
|
||
| IntStream.rangeClosed(1, 1000).forEach(i -> { | ||
| try { | ||
| String line = i + ",User_" + i + ",user" + i + "@mail.com"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why hard code it?
The stream could process anything, not only this use case.
| writer.write("id,name,email"); | ||
| writer.newLine(); | ||
|
|
||
| IntStream.rangeClosed(1, 1000).forEach(i -> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should not generate a large CSV file here.
The repository already has many sample CSV files at main/src/main/resources/sample-csvs
Check these:
- snakes_count_10000.csv
- hw_25000.csv
- mlb_players.csv
|
|
||
| import static org.junit.jupiter.api.Assertions.*; | ||
|
|
||
| class CsvStreamTest { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests are empty
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good for the Start!
I suggest integrating your CsvStream class with the existing CSV library. Its main role is to be a lightweight wrapper around CsvReader and provide a Java Stream<Row> interface to make processing rows easy.
A minimal recommended API could look like this:
public final class CsvStream implements Closeable {
private CsvStream(CsvReader reader);
/** Wraps an existing CsvReader into a CsvStream */
public static CsvStream from(CsvReader reader);
/** Opens a CSV file with the given configuration */
public static CsvStream fromPath(Path path, CsvConfig config) throws Exception;
/** Opens a CSV file with configuration and explicit headers */
public static CsvStream fromPath(Path path, CsvConfig config, Headers headers) throws Exception;
/** Returns a Stream of Row objects for lazy processing */
public Stream<Row> stream();
/** Closes the underlying CsvReader */
public void close();
}How a user would use CsvStream:
Path file = Paths.get("data.csv");
CsvConfig config = CsvConfig.builder().setHasHeader(true).build();
try (CsvStream csvStream = CsvStream.fromPath(file, config)) {
csvStream.stream()
.filter(row -> row.get("age").asInt() > 30)
.forEach(row -> System.out.println(row.get("name")));
}
// CsvStream automatically closes the underlying CsvReaderKey Points for Implementation:
CsvStreamshould wrap aCsvReaderinternally.stream()should lazily read rows so you don’t load the entire file into memory.close()should release resources from the underlyingCsvReader.- Use
try-with-resourceswhen consumingCsvStreamto ensure proper cleanup.
b537c72 to
49fedc3
Compare
|
Working on it, to continue from @quadri00 version and implement @bogdansharp suggestions. |
- Wraps CsvReader for lazy, memory-efficient streaming - Implements Closeable for resource management - Factory methods: from(CsvReader), fromPath(Path, CsvConfig), fromPath(Path, CsvConfig, Headers) - Supports all Stream operations (filter, map, limit, forEach, count, collect) - 9 comprehensive unit tests covering all functionality - Extensive Javadoc with usage examples - Implements all suggestions from Bogdan's code review - All tests pass, build successful
da533d5 to
b43f6bd
Compare
|
Lots of commits too, let me squash it. |
|
Local build pass: |
csv stream puahed to main for review