Skip to content

Commit 6223504

Browse files
committed
#687 Fix PR suggestions including typos in comments, exception messages, and documentation for consistency.
1 parent 39a9f34 commit 6223504

File tree

5 files changed

+12
-12
lines changed

5 files changed

+12
-12
lines changed

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3473,13 +3473,13 @@ execution Pramen will do the following:
34733473
- `2` means backfill for the info date and the date before.
34743474
- Check for retrospective updates of the source data according to `track.days` of corresponding tables in the metastore.
34753475
For this check Pramen will query sources for record counts for each of previous days.
3476+
- The option has the following meaning (similar to `backfill.days``):
3477+
- `-1` means backfill only up until the last successfull attempt. Never track retrospective updates.
3478+
- `0` means never track retrospective updates.
3479+
- `1` means track retrospective only for the same info date.
3480+
- `2` means track retrospective for the info date and the date before.
34763481
- If a mismatch is found (as at `2020-07-16` on this diagram), the data is reloaded and dependent transformers are
34773482
recomputed (if `trigger.updates = true`)
3478-
- The option has the following meaning (similar to `backfill.days``):
3479-
- `-1` means backfill only up until the last successfull attempt. Never track retrospective updates.
3480-
- `0` means never track retrospective updates.
3481-
- `1` means track retrospective only for the same info date.
3482-
- `2` means track retrospective for the info date and the date before.
34833483
- Check for late data by querying sources for records for previous days if none were loaded. If such data is found, it
34843484
is loaded and related transformations will be ran.
34853485
- Run the pipeline for the day - check new data and run dependent transformers and sink jobs.

pramen/core/src/main/resources/reference.conf

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,13 +68,13 @@ pramen {
6868
wait.for.output.table.enabled = false
6969
wait.for.output.table.seconds = 600
7070

71-
# How many days to check back for late data (when there is not data at the source for that day)
72-
# 0 - never check previous days, even if no data at the source happenned
71+
# How many days to check back for late data (when there is no data at the source for that day)
72+
# 0 - never check previous days, even if no data at the source happened
7373
# 1 - check only the current info date if you run the job more than once per day
7474
# 2 - check the latest info date and the date before
7575
# etc...
7676
# You can also set this parameter for individual tables in the metastore.
77-
# NOTE. By default it is the same as track days, so it won't take effect. In order for it to take effect you
77+
# NOTE. By default it is the same as track.days, so it won't take effect. In order for it to take effect you
7878
# need to either increase 'backfill.days' and/or decrease 'track.days'
7979
backfill.days = 5
8080

pramen/core/src/main/scala/za/co/absa/pramen/core/bookkeeper/BookkeeperBase.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ abstract class BookkeeperBase(isBookkeepingEnabled: Boolean, batchId: Long) exte
134134
}
135135

136136
private[pramen] override def getOffsetManager: OffsetManager = {
137-
throw new IllegalArgumentException(s"This implementation of bookeeping does not support offset management and incremental pipelines. " +
137+
throw new IllegalArgumentException(s"This implementation of bookkeeping does not support offset management and incremental pipelines. " +
138138
"Please, use JDBC for bookkeeping to enable this.")
139139
}
140140

@@ -180,7 +180,7 @@ abstract class BookkeeperBase(isBookkeepingEnabled: Boolean, batchId: Long) exte
180180
val foundDataAvailable = new ListBuffer[DataAvailability]
181181

182182
while (date.isBefore(dateEndPlus)) {
183-
val chunks = getTransientDataChunks(table, Option(date), None, None)
183+
val chunks = getTransientDataChunks(table, Option(date), Option(date), None)
184184

185185
if (chunks.nonEmpty) {
186186
val totalRecords = chunks.map(_.outputRecordCount).sum

pramen/core/src/main/scala/za/co/absa/pramen/core/rdb/PramenDb.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ object PramenDb {
155155
slick.jdbc.MySQLProfile
156156
case "com.microsoft.sqlserver.jdbc.SQLServerDriver" | "net.sourceforge.jtds.jdbc.Driver" =>
157157
slick.jdbc.SQLServerProfile
158-
case other => throw new IllegalArgumentException(s"Unknown driver for the bookeeping database: $other")
158+
case other => throw new IllegalArgumentException(s"Unknown driver for the bookkeeping database: $other")
159159
}
160160
}
161161

pramen/core/src/test/scala/za/co/absa/pramen/core/mocks/bookkeeper/SyncBookkeeperMock.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ class SyncBookkeeperMock(batchId: Long = 123L) extends Bookkeeper {
9797

9898
foundDataAvailable += DataAvailability(
9999
date,
100-
chunks.size,
100+
chunksForDate.size,
101101
totalRecordCount
102102
)
103103
}

0 commit comments

Comments
 (0)