ImportError

[MitjaCernko]

Hello,

when trying to run the train function (after successfully establishing the reticulate connection with a conda environment based on the provided rmidas-env.yml file), I receive the following error:

Error in py_module_import(module, convert = convert) :
ImportError: cannot import name 'AutoGraphError' from partially initialized module 'tensorflow.python.autograph.impl.api' (most likely due to a circular import) (C:\Users\[user]\.conda\envs\rmidas\lib\site-packages\tensorflow\python\autograph\impl\api.py)

I would greatly appreciate any suggestions on resolving this issue and am happy to provide additional information that might be needed to pinpoint the cause of the error.

[edvinskis]

Hi @MitjaCernko,
This is interesting. Could you please tell us which rMIDAS version you are currently using and share the output of py_config()?

[MitjaCernko]

Gladly, and thank you for a swift response!

I'm using the 0.5.0 version of rMIDAS (and 1.2.4 version of MIDASpy).

Here's the output of the py_config function:

python: C:/Users/[user]/.conda/envs/rmidas/python.exe
libpython: C:/Users/[user]/.conda/envs/rmidas/python38.dll
pythonhome: C:/Users/[user]/.conda/envs/rmidas
version: 3.8.16 | packaged by conda-forge | (default, Feb 1 2023, 15:53:35) [MSC v.1929 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/[user]/.conda/envs/rmidas/Lib/site-packages/numpy
numpy_version: 1.24.2

NOTE: Python version was forced by use_python() function

It might be relevant to point out, that I have used the rMIDAS package in the past successfully, with a very minimalistic Python setup (without using Anaconda) - I believe it was simply a clean installation of Python 3.8, running the pip install MIDASpy command.

When trying this setup, I get the same error as well...

py_config for the simplified setup:

python: C:/Program Files/Python38/python.exe
libpython: C:/Program Files/Python38/python38.dll
pythonhome: C:/Program Files/Python38
version: 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Program Files/Python38/Lib/site-packages/numpy
numpy_version: 1.24.4
sys: [builtin module]

The full error message (with the screenshot of the final output of the py_last_error() function attached):

Initialising Python connection
Error in py_module_import(module, convert = convert) :
ImportError: cannot import name 'AutoGraphError' from partially initialized module 'tensorflow.python.autograph.impl.api' (most likely due to a circular import) (C:\PROGRA~1\Python38\lib\site-packages\tensorflow\python\autograph\impl\api.py)
Run reticulate::py_last_error() for details.

[edvinskis]

Thank you for the detailed response @MitjaCernko. As the error message suggests there's a circular import issue within TensorFlow's internal modules. This could be due to a variety of reasons, but to further understand and debug this issue could you also tell us:

1. Which version of R are you currently using?
2. Which version of reticulate is installed?
3. Could you provide a list of the installed Python dependencies when using the .yml file? You can just type conda list -n rmidas in your terminal.

[MitjaCernko]

Thank you very much for your help and involvement!

1. When submitting the issue, I've been using R version 4.2.3, but have updated it to 4.3.1 since - still receive the same error.
2. Reticulate version 1.32.0.
3. The terminal output screenshot is attached below.

[edvinskis]

Hi @MitjaCernko, I was able to emulate the error on my end, and it appears to be Windows specific. This behaviour does not occur on Linux or macOS. I believe that the issue is more involved and we should investigate this further. In the meantime, I've prepared this .yml file that might serve as a temporary solution. Please give it a try and let us know if it resolves the issue for you.

rmidas-env-windows.yml.zip

[MitjaCernko]

Hello!

Thank you very much for your reply and providing a swift solution - setting up a new conda environment with this windows specific .yml file resolves the initial error, and I can now run the overimputation / training functions in R :)

I noticed a new minor issue (unless this is intended behavior) when stopping either of these functions before they fully complete. If I stop their execution prematurely, the next time I run them, the "Initializing Python connection" message appears, but then unlocks access to the console.

When issuing any new command, the size index message appears, but the process seems to stop at that point.

If I try to run either of these functions again, I get the following error:

This can be mitigated by restarting the R session and allowing the functions to run their full course.