Skip to content

The concept of exactPython as a constant in version.py is fataly flawed for release packages #5389

@adamnovak

Description

@adamnovak

Toil generates a version.py that contains both the Git revision Toil was built from and the Python version it was built on, as constants. exactPython is meant to hold the exact Python interpreter in use (like python3.9). Here is the template code for it:

toil/version_template.py

Lines 78 to 85 in 05257fe

def exactPython():
"""
Returns the Python command for the exact version of Python we are installed
for. Something like 'python2.7' or 'python3.6'.
"""
import sys
return f"python{sys.version_info[0]}.{sys.version_info[1]}"

But the generated version.py gets shipped in the sdist and wheel packages on PyPI. And those aren't Python-version-specific and are meant to be installable on any supported Python version. So we end up installing a version.py that describes the Python version used to build Toil, not the one we're actually installed on.

The current Python version also feeds into version.py in the Docker container IDs in there, which include the Python version we want installed in the container.

I think we need to:

  1. Get rid of any code that processes version.py to extract versions via anything other than actually running it (i.e. make sure we're not grepping through it somewhere).
  2. Change anything that is supposed to involve the "current" Python version from being a constant to generating the value using an expression, based on asking the running Python interpreter about its version. So e.g. the Docker container names will include the GIt hash in a string but concatenate that with version numbers fished out of sys.

The hard part is that this file is codegen'd by version_template.py, so we need to adjust that codegen process to be able to produce the active code we need and not just a list of constants.

┆Issue is synchronized with this Jira Story
┆Issue Number: TOIL-1783

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions