Change `serde` dependency to `serde_core` where applicable

[crepererum]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently some arrow crates depend on serde, which may take a long time to compile if -- due to feature unification -- arrow users opted into the derive features. See serde-rs/serde#2608 . However we often don't need derive and hence we could depend on the actual core part of serde called serde_derive.

Describe the solution you'd like
Check our dependencies and where we don't use derive, change the dependency from serde to serde_derive.

Describe alternatives you've considered
-

Additional context
serde-rs/serde#2608

[AdamGS]

I think you meant migrating to serde_core

[AdamGS]

take

[AdamGS]

After spending just under an hour on this, the main issue is that arrow-schema makes heavy use of derived serialization, and basically everything else depends on it, as far as I can tell the only direct dependency on serde that can be removed is in arrow-json.
There are two ways forward here:

1. Hand-write Serialize/Deserialize implementation (or generate and include the expanded ones) for all the relevant types.
2. Accept the the benefit here is limited, and live with the existing compile time cost.

I've started going down 1 for arrow-schema, but would love some consensus/support before I go down that path as its pretty boilerplate intensive.

The crates the depend on the derive feature are, including their overall state:

• arrow-json - can actually be removed from here, in favor of serde_core
• arrow-schema - Derives 14 implementation, and is the highest value target
• arrow-avro - Derives serialization 13 different types
• arrow - its a dev dependency
• arrow-integration-test - derives 9 implementations, but is probably not a very high value target
• arrow-integration-testing - includes it as a dependency but doesn't use it as far as I can tell!
• parquet - used in tests and in binaries behind a feature, so probably doesn't make much of a difference here.

[crepererum]

I think the two integration test crates are not super relevant, I would view them as dev dependencies. arrow-json and arrow-avro are optional features. So I agree that this is mostly gonna make sense for arrow-schema. Having a quick look at this, it seems that most (or even all?) derives are gated behind the serde feature and I think it's fine if feature = serde = serde_derive. So my main question is mostly: can one compile arrow (i.e. the core crate) with default_features = false so that it doesn't require serde_derive?

[AdamGS]

That's already possible as far as I can tell. I opened #8558 which shows what I got, I don't think it makes 0 improvement but it is pretty minor.