support converting null TTLs to default TTL relative to write timestamp

[tarzanek]

No description provided.

[tarzanek]

before merging
please add a disclaimer that for safety reasons such a migration where we tamper with writetimestamps and TTLs should be always done into an empty table

[tarzanek]

this merge is delayed, this approach uncovered some bugs in scylla, so merge will happen once Scylla has all the patches to support this injection

[tarzanek]

all officially supported branches have now fixes, we should revisit this and merge

[julienrf]

It would be interesting to explain in more details the motivation behind this feature: why would one create a TTL in the target table if it did not exist in the source table?

Also, would it be possible to add tests for this feature? You can draw inspiration from the tests in tests/src/test/scala/com/scylladb/migrator/scylla.

Last, you need to document the defaultTTL option in the documentation. See files docs/source/migrate-from-cassandra-or-parquet.rst and docs/source/configuration.rst.

[julienrf]

Suggested change

	# final write TTL set will be relative to writetimestamp, so: defaultTTL - now - writetimestamp , rounded to milliseconds
	# final write TTL set will be relative to writetimestamp, so: defaultTTL - (now - writetimestamp), rounded to milliseconds

[julienrf]

I wonder if it would be clearer to keep the Option in the defaultTTL parameter.

So, the parameter would have type defaultTTL: Option[Long], and here you would write val baseTTL = defaultTTL.getOrElse(0).

[julienrf]

This looks suspicious because writetime has type Option[Long] and CassandraOption.Unset has a type that is disjoint from Long. See my comment below for a suggestion.

[julienrf]

I recommend to avoid using asInstanceOf. I suggest the following changes, which I believe are more readable:

Suggested change

	val deltaTTL = if (writetimestamp == CassandraOption.Unset) {
	0
	} else {
	baseTTL - (System.currentTimeMillis - writetimestamp.asInstanceOf[Long] / 1000) / 1000
	}
	val deltaTTL = writetime match {
	case None => 0
	case Some(writetimestamp) =>
	baseTTL - (System.currentTimeMillis - writetimestamp / 1000) / 1000
	}

[julienrf]

Here use assume that deltaTTL == 0 is always the case when the write timestamp wasn’t found. But it seems it could also happen e.g. if baseTTL == 5 and the write timestamp was 5 seconds ago.

Maybe use the type Option again to model the fact that the deltaTTL might be unset. So, you would have something like:

Suggested change

	if (deltaTTL > 0) deltaTTL else if (deltaTTL == 0) baseTTL else 1L
	deltaTTL.map(_.max(1L)).getOrElse(baseTTL)