Update generative-proof-of-concept-CPU-preprocessing-in-memory.py

david-thrower · web-flow · commit b9a42cdc3882 · 2025-10-14T20:06:19.000-04:00
Remove .repeat()
diff --git a/generative-proof-of-concept-CPU-preprocessing-in-memory.py b/generative-proof-of-concept-CPU-preprocessing-in-memory.py
@@ -1382,7 +1382,7 @@ def create_dataset(raw_text_samples, tokenizer, sample_expansion_batch_size=10)
                 )
             )
             # Set dataset to allow multiple epochs:
-            dataset = dataset.repeat()
+            # dataset = dataset.repeat()
             # Batch it
             dataset = dataset.batch(batch_size)
             return dataset

Original file line number	Diff line number	Diff line change
`@@ -1382,7 +1382,7 @@ def create_dataset(raw_text_samples, tokenizer, sample_expansion_batch_size=10)`
`1382`	`1382`	`)`
`1383`	`1383`	`)`
`1384`	`1384`	`# Set dataset to allow multiple epochs:`
`1385`		`- dataset = dataset.repeat()`
	`1385`	`+ # dataset = dataset.repeat()`
`1386`	`1386`	`# Batch it`
`1387`	`1387`	`dataset = dataset.batch(batch_size)`
`1388`	`1388`	`return dataset`