[SPARK-55530][Geo][SQL] Support Geo result sets in Hive and Thrift server#54325
Open
uros-db wants to merge 7 commits intoapache:masterfrom
Open
[SPARK-55530][Geo][SQL] Support Geo result sets in Hive and Thrift server#54325uros-db wants to merge 7 commits intoapache:masterfrom
uros-db wants to merge 7 commits intoapache:masterfrom
Conversation
uros-db
force-pushed
the
geo-thrift-server
branch
from
9bb3128 to
73e7ab9
Compare
uros-db
commented
Feb 15, 2026
Contributor
Author
uros-db
left a comment
uros-db
left a comment
There was a problem hiding this comment.
@cloud-fan Please review.
cloud-fan
reviewed
Feb 16, 2026
| case (g: Geometry, dt: GeometryType) => | ||
| val internalGeom = STUtils.serializeGeomFromWKB(g, dt) | ||
| val s = STUtils.stAsEwkt(internalGeom).toString | ||
| if (nested) "\"" + s + "\"" else s |
Contributor
There was a problem hiding this comment.
not related to this PR, but seems we forget to do the same for VariantType
cloud-fan
reviewed
Feb 16, 2026
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala
Show resolved
Hide resolved
cloud-fan
reviewed
Feb 17, 2026
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala
Outdated
Show resolved
Hide resolved
cloud-fan
reviewed
Feb 17, 2026
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/RowSetUtils.scala
Outdated
Show resolved
Hide resolved
…/thriftserver/RowSetUtils.scala Co-authored-by: Wenchen Fan <cloud0fan@gmail.com>
Contributor
Author
|
CI failure looks unrelated. Otherwise, this should be ready @cloud-fan. |
szehon-ho
approved these changes
Feb 20, 2026
Member
szehon-ho
left a comment
szehon-ho
left a comment
There was a problem hiding this comment.
mostly lgtm, small suggestions
| WkbReader reader = new WkbReader(true); | ||
| GeometryModel model = reader.read(Arrays.copyOfRange( | ||
| getBytes(), WKB_OFFSET, getBytes().length)); | ||
| return model.toString(); |
Member
There was a problem hiding this comment.
i think it would have been cleaner to have explicit toWKT in GeometryModel, as toString may change. It can be separate from this pr though
| public byte[] toEwkt() { | ||
| // Once EWKT conversion is implemented, it should support various precisions. | ||
| throw new UnsupportedOperationException("Geography EWKT conversion is not yet supported."); | ||
| String ewkt = "SRID=" + srid() + ";" + toWktInternal(); |
Member
There was a problem hiding this comment.
ok, so all wkt is valid ewkt, and we add extra SRID=... make sense to me.
| public byte[] toWkt() { | ||
| // Once WKT conversion is implemented, it should support various precisions. | ||
| throw new UnsupportedOperationException("Geometry WKT conversion is not yet supported."); | ||
| return toWktInternal().getBytes(StandardCharsets.UTF_8); |
Member
There was a problem hiding this comment.
we could make this in Geo base class, but its quite small
| } | ||
| } | ||
| case gt: GeometryType => | ||
| val possibleGeometriesWKB = Seq( |
Member
There was a problem hiding this comment.
can you comment what these values are?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Implement geospatial data output display in Extended Well-Known Text (EWKT) format and add the appropriate geospatial type mapping in Hive Thrift Server.
Why are the changes needed?
Display Geometry and Geography values in a human-readable format, which allows projecting geospatial data in Spark SQL and Thrift Server.
Does this PR introduce any user-facing change?
Yes, geospatial data is now displayed using EWKT.
How was this patch tested?
Was this patch authored or co-authored using generative AI tooling?
No.