Skip to content

Conversation

@andygrove
Copy link
Member

Which issue does this PR close?

Workaround for #1182 but we really need to fix the underlying issue or have Ballista disable string views.

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@andygrove
Copy link
Member Author

@milenkovicm fyi

@milenkovicm
Copy link
Contributor

for this release could we just disable them ?

datafusion.execution.parquet.schema_force_view_types=false at

// Ballista disables this option
& (which needs to be refactored)

   let session_config = SessionConfig::new()
            .with_information_schema(true)
            .with_option_extension(config.clone())
            .set_bool("datafusion.execution.parquet.schema_force_view_types", false)
            // Ballista disables this option
            .with_round_robin_repartition(false);

@andygrove
Copy link
Member Author

Yes, disabling string views would be ideal. I wasn't sure where to do that.

@milenkovicm
Copy link
Contributor

Unfortunately two places at the moment, I've missed duplicated code

@milenkovicm
Copy link
Contributor

I had some time to push #1185 which covers this pr and #1183

@andygrove andygrove closed this Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants