Skip to content

Conversation

wendigo
Copy link
Contributor

@wendigo wendigo commented Jan 31, 2025

This makes the API much easier to consume in the segments cursor style:

cur = conn.cursor('segment')
cur.execute(sql)
segments = cur.fetchall()
total_row_count = 0
row_mapper = RowMapperFactory().create(columns=cur._query.columns, legacy_primitive_types=False)
for segment in segments:
  rows = list(SegmentIterator(segment, row_mapper))
  print ("rows length is " + str(len(rows)) + " " + segment.encoding)
  total_row_count += len(rows)
print(total_row_count)

This will work the same:

segments = cur.fetchall()
total_row_count = 0
row_mapper = RowMapperFactory().create(columns=cur._query.columns, legacy_primitive_types=False)

rows = list(SegmentIterator(segments, row_mapper))
print ("rows length is " + str(len(rows)))
total_row_count += len(rows)

print(total_row_count)

This is part of the contract which is not used in the JSON
encoding scheme but will be in the future for other formats.
@cla-bot cla-bot bot added the cla-signed label Jan 31, 2025
@wendigo wendigo requested review from aalbu, hashhar and mdesmet and removed request for hashhar January 31, 2025 20:06
@wendigo wendigo force-pushed the serafin/fix-tuple branch 2 times, most recently from 6114f7a to 0fc43cc Compare January 31, 2025 20:14
@wendigo
Copy link
Contributor Author

wendigo commented Jan 31, 2025

@mdesmet I've dropped the Tuple and the API feels more natural now

This makes the API much easier to consume in the `segments` cursor style:

```
cur = conn.cursor('segment')
cur.execute(sql)
segments = cur.fetchall()
total_row_count = 0
row_mapper = RowMapperFactory().create(columns=cur._query.columns, legacy_primitive_types=False)
for segment in segments:
  rows = list(SegmentIterator(segment, row_mapper))
  print ("rows length is " + str(len(rows)) + " " + segment.encoding)
  total_row_count += len(rows)
print(total_row_count)
```

This will work as well:

```
cur = conn.cursor('segment')
cur.execute(sql)
segments = cur.fetchall()
total_row_count = 0
row_mapper = RowMapperFactory().create(columns=cur._query.columns, legacy_primitive_types=False)

rows = list(SegmentIterator(segments, row_mapper))
print ("rows length is " + str(len(rows)))
total_row_count += len(rows)

print(total_row_count)
```
Sending authorization headers will make S3 presigned uris
to fail due to additional headers that cannot be present.
@wendigo
Copy link
Contributor Author

wendigo commented Feb 2, 2025

@hashhar ptal, alas it's a breaking change but it was broken API from the beginning and since it was in a single release so far, I think we should just fix it and move along

@wendigo wendigo merged commit 9b28484 into master Feb 3, 2025
12 checks passed
@wendigo
Copy link
Contributor Author

wendigo commented Feb 3, 2025

@hashhar can we release new version?

@hashhar hashhar deleted the serafin/fix-tuple branch February 3, 2025 15:04
GaetanLepage pushed a commit to NixOS/nixpkgs that referenced this pull request Apr 24, 2025
* Improve handling of query results containing `null` values for
  the error field.
  ([#512](trinodb/trino-python-client#512))
* Fix accessing spooled segments on S3 when authentication to the
  Trino cluster is used.
  ([#525](trinodb/trino-python-client#525))
* Simplify `segment` cursor API to make it easier to iterate over spooled
  segments.
  ([#525](trinodb/trino-python-client#525))
mkg20001 pushed a commit to mkg20001/nixpkgs that referenced this pull request Apr 29, 2025
* Improve handling of query results containing `null` values for
  the error field.
  ([NixOS#512](trinodb/trino-python-client#512))
* Fix accessing spooled segments on S3 when authentication to the
  Trino cluster is used.
  ([NixOS#525](trinodb/trino-python-client#525))
* Simplify `segment` cursor API to make it easier to iterate over spooled
  segments.
  ([NixOS#525](trinodb/trino-python-client#525))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants