updating numpy dependencies

[helenmacdonald]

Testing to see if we are able to upgrade to a more recent version of numpy.
As requested here by @rbeucher

[helenmacdonald]

@ashjbarnes and @manishvenu - can you please help me with this one?
I am updating numpy by forcing it to be >2 so we can check it works but getting some odd errors in the testing:

Expected:
    1000.0
Got:
    np.float64(1000.0)

There are a few more similar errors. These are the same number but are not recognised as such. The test pass when I run on a local environment setup on Gadi. Both my local environment and the test environment are using numpy version 2.3.5 so I am wondering if there is something subtle that I need to change about the tests on GitHub when updating the environment.

[manishvenu]

@ashjbarnes and @manishvenu - can you please help me with this one? I am updating numpy by forcing it to be >2 so we can check it works but getting some odd errors in the testing:

Expected:
    1000.0
Got:
    np.float64(1000.0)

There are a few more similar errors. These are the same number but are not recognised as such. The test pass when I run on a local environment setup on Gadi. Both my local environment and the test environment are using numpy version 2.3.5 so I am wondering if there is something subtle that I need to change about the tests on GitHub when updating the environment.

Hey @helenmacdonald,

We might have to change tests. That's actually pretty common.

It could be that the env is slightly different. Since the github and your environment have the exact same version of numpy, it might be tricky to figure out exactly what's going on. I like your solution to force the output of the functions to be in a specific form (like numpy.float), or the reverse.

We could do a bit of messing around to see what the problem is. I'm not sure it's worth the fuss though!

I'd also recommend removing the numpy requirements from the environment-ci and pyproject file. If the environment solves numpy naturally above v2, that would be great (We also need to check if the ESMF regridding is still working).

[helenmacdonald]

Thanks @manishvenu - good to hear I am not the only one who encountered these issues!

I like your solution to force the output of the functions to be in a specific form (like numpy.float), or the reverse.

The problem is, that I did that and it didn't work 😞. It was sucsessfull in forcing them to be the same type on my gadi environment, but was not for the testing here. I'm going to keep digging tomorrow.

I'd also recommend removing the numpy requirements from the environment-ci and pyproject file. If the environment solves numpy naturally above v2, that would be great (We also need to check if the ESMF regridding is still working).

Agreed, thanks. I will leave it forced to v2 until I have passed the testing, and then remove when I make this PR non draft. Do we need to write a test for the ESMF regridding - or will this be tested when I get past the current failing tests?

[manishvenu]

Thanks @manishvenu - good to hear I am not the only one who encountered these issues!

I like your solution to force the output of the functions to be in a specific form (like numpy.float), or the reverse.

The problem is, that I did that and it didn't work 😞. It was sucsessfull in forcing them to be the same type on my gadi environment, but was not for the testing here. I'm going to keep digging tomorrow.

I'd also recommend removing the numpy requirements from the environment-ci and pyproject file. If the environment solves numpy naturally above v2, that would be great (We also need to check if the ESMF regridding is still working).

Agreed, thanks. I will leave it forced to v2 until I have passed the testing, and then remove when I make this PR non draft. Do we need to write a test for the ESMF regridding - or will this be tested when I get past the current failing tests?

I think the regridding is in the notebooks! So probably should be good, but we should check, because the reason we had numpy<2 was because of the regridding I think.

Sounds good on the testing! Let me know if you want me to take a stab as well. Happy to help.