Skip to content

Conversation

@zm711
Copy link
Contributor

@zm711 zm711 commented Sep 13, 2024

I want to do this in a couple parts. First remove copy. Then ensure tests are passing (currently I've commented out the tests, but my plan is to delete the copy=True tests). Just so people can monitor progress I'll just have this open as a PR.

Fixes #1534
Fixes #1529

@zm711
Copy link
Contributor Author

zm711 commented Sep 13, 2024

Things I found for us to discus:

  1. we can't change the units or dtype during construction since we only get a view of the array. I deleted all tests changing units/dtype

  2. time_shift required a different strategy so I implemented it

  3. deleted all tests checking for copy vs view

  4. I need help with the pickle tests. Reading from pickle just returns a None instead of the appropriate array. Someone who knows pickle better should check this for me. Currently they are commented out.

@zm711
Copy link
Contributor Author

zm711 commented Sep 13, 2024

IO tests failing due to a CI issue right now.... We can check that later.

So this issue seems to actually be due to failure on python 3.11. I can't find a reason google other than saying the wrong *.so file was probably downloaded. Maybe there will be a fix, but not sure.

@h-mayorquin if you have two minutes sometime today could you look at the CI error for IO tests here. I've tried googling but it just seems like there is a package misalignment. I'm not sure from where. You definitely don't have to try to debug yourself, but if you've seen this before on any of your CIs I would appreciate your insights.

@zm711 zm711 added this to the 0.14.0 milestone Sep 16, 2024
@h-mayorquin
Copy link
Contributor

h-mayorquin commented Sep 16, 2024

Try flushing the conda cashes in the CI just to see if the dependency issue goes away.

@zm711
Copy link
Contributor Author

zm711 commented Sep 16, 2024

So now we have an issue in nix that I have propagated the fix for quantities here.

I just have pickleio that is broken and I don't know why. @samuelgarcia or @apdavison can either of you walk me through the pickling story a bit more.

@zm711
Copy link
Contributor Author

zm711 commented Sep 30, 2024

@apdavison, if you have the brain power to look over this. We are running into one stumbling block with pickleio not working. Sam and I looked at it and we aren't quite sure why. The nix test failure needs to be fixed with my PR at quantities level.

(the original :class:`IrregularlySampledSignal` is not modified).
"""
new_sig = deepcopy(self)
# As of numpy 2.0/quantities 0.16 we need to copy the array itself
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be discussed.

units = pq.quantity.validate_dimensionality(units)

new_st = self.__class__(signal, t_start=t_start, t_stop=t_stop, waveforms=waveforms, units=units)
signal = deepcopy(signal)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be discussed


class TestAnalogSignalFunctions(unittest.TestCase):

## someone with more pickle knowledge needs to work on this
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.



class TestAnalogSignalFunctions(unittest.TestCase):
# pickle help needed
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samuelgarcia
Copy link
Contributor

@zm711 : I fixes the pickleio tsuff it was on the copy=True was still here in the __reduce__ (used by pickle)

@zm711
Copy link
Contributor Author

zm711 commented Oct 16, 2024

Perfect thanks for the pickleio. I'll uncomment the core pickle and then we have the nix fix that needs to happen in quantities.

@zm711
Copy link
Contributor Author

zm711 commented Oct 16, 2024

@apdavison, I convinced Sam to help with the pickle a bit more. So last thing we need for tests to pass is for a release of quantities with python-quantities/python-quantities#242 merged in. Tests have passed there. Then you can do the final review for this!

@apdavison
Copy link
Member

@h-mayorquin
Copy link
Contributor

Great!

Copy link
Member

@apdavison apdavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests are now passing. Other than the fixes to version numbers in pyproject.toml, this looks fine.


dependencies = [
"packaging",
"numpy>=1.22.4,<2.0.0",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dependency versions need to be fixed ("numpy>=1.22.4", "quantities>=0.16.1").

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quantities yes. But do we want to do the numpy in 2 PRs? This one to get quantities ready and then if numpy isn't ready we mark the limit and fix the separate numpy issues? Let's free it and see what happens :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was more the lower limit I was talking about, which was going back to 1.19 when I reviewed this.

@apdavison apdavison marked this pull request as ready for review October 16, 2024 15:42
@apdavison apdavison changed the title [WIP] - Deprecate Copy Behavior to support Quantities 16.0 Deprecate Copy Behavior to support Quantities 16.0 Oct 16, 2024
@zm711
Copy link
Contributor Author

zm711 commented Oct 16, 2024

can we keep the numpy limit and do that in a second pull request? we have some numpy api issues that I would prefer to do in a separate testing framework. We also can't use our IO tests with 2.0 since 2.0 requires more recent versions to work. So I would need to rewrite our core action for the divide created by 2.0.

@apdavison apdavison merged commit 2d688a8 into NeuralEnsemble:master Oct 16, 2024
@zm711 zm711 deleted the remove-copy branch December 13, 2024 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix tests for quantities 0.16.0 Deprecation in pq.Quantity of the copy argument

4 participants