Skip to content

Commit f39e82c

Browse files
mgaido91cloud-fan
authored andcommitted
[SPARK-23986][SQL] freshName can generate non-unique names
## What changes were proposed in this pull request? We are using `CodegenContext.freshName` to get a unique name for any new variable we are adding. Unfortunately, this method currently fails to create a unique name when we request more than one instance of variables with starting name `name1` and an instance with starting name `name11`. The PR changes the way a new name is generated by `CodegenContext.freshName` so that we generate unique names in this scenario too. ## How was this patch tested? added UT Author: Marco Gaido <[email protected]> Closes apache#21080 from mgaido91/SPARK-23986.
1 parent 3990daa commit f39e82c

File tree

2 files changed

+13
-8
lines changed

2 files changed

+13
-8
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -572,14 +572,9 @@ class CodegenContext {
572572
} else {
573573
s"${freshNamePrefix}_$name"
574574
}
575-
if (freshNameIds.contains(fullName)) {
576-
val id = freshNameIds(fullName)
577-
freshNameIds(fullName) = id + 1
578-
s"$fullName$id"
579-
} else {
580-
freshNameIds += fullName -> 1
581-
fullName
582-
}
575+
val id = freshNameIds.getOrElse(fullName, 0)
576+
freshNameIds(fullName) = id + 1
577+
s"${fullName}_$id"
583578
}
584579

585580
/**

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -489,4 +489,14 @@ class CodeGenerationSuite extends SparkFunSuite with ExpressionEvalHelper {
489489
assert(!ctx.subExprEliminationExprs.contains(ref))
490490
}
491491
}
492+
493+
test("SPARK-23986: freshName can generate duplicated names") {
494+
val ctx = new CodegenContext
495+
val names1 = ctx.freshName("myName1") :: ctx.freshName("myName1") ::
496+
ctx.freshName("myName11") :: Nil
497+
assert(names1.distinct.length == 3)
498+
val names2 = ctx.freshName("a") :: ctx.freshName("a") ::
499+
ctx.freshName("a_1") :: ctx.freshName("a_0") :: Nil
500+
assert(names2.distinct.length == 4)
501+
}
492502
}

0 commit comments

Comments
 (0)