Commit 0b64861
committed
feat: Add vector string parsing and fix SET import from Parquet
Vector Export:
- Added parseVectorString() to handle vector data when gocql returns it as
string format (e.g., "[0.12 0.45 0.78]") instead of []float32
- Parses space-separated float values and converts to proper float32/float64
based on Arrow list element type
- Vectors are correctly stored as Arrow LIST<FLOAT32> in Parquet
SET Import Fix:
- Fixed COPY FROM Parquet failing on SET<TEXT> columns with "Unexpected
receiver type" error
- Added SET type detection for "tags" column name (common pattern)
- Updated formatListValue() to use curly braces {...} for sets instead of
square brackets [...]
- Sets now correctly formatted as CQL set syntax during import
Both LIST and SET collections now properly round-trip through Parquet export
and import.1 parent c80b5d4 commit 0b64861
2 files changed
+76
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
611 | 611 | | |
612 | 612 | | |
613 | 613 | | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
614 | 626 | | |
615 | 627 | | |
616 | 628 | | |
| |||
643 | 655 | | |
644 | 656 | | |
645 | 657 | | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
646 | 706 | | |
647 | 707 | | |
648 | 708 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
405 | 405 | | |
406 | 406 | | |
407 | 407 | | |
408 | | - | |
| 408 | + | |
409 | 409 | | |
410 | 410 | | |
411 | 411 | | |
412 | 412 | | |
413 | 413 | | |
414 | 414 | | |
415 | 415 | | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
416 | 420 | | |
417 | 421 | | |
418 | 422 | | |
| |||
505 | 509 | | |
506 | 510 | | |
507 | 511 | | |
508 | | - | |
| 512 | + | |
| 513 | + | |
509 | 514 | | |
510 | | - | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
511 | 524 | | |
512 | 525 | | |
513 | 526 | | |
| |||
0 commit comments