Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 16, 2025

Plan to fix binary distribution build

  • Explore the repository and understand the issue
  • Identify the problem in .github/workflows/binary-linux.yml
  • Fix the GitHub Actions workflow to properly call the build_binary_dist.sh script
  • Simplify the process using make dist on host
  • Add branch trigger to test workflow
  • Install TREXIO and dependencies on host
  • Use TREXIO dist tarball in Docker to avoid emacs
  • Add emacs back for org-mode processing
  • Add HDF5 to Docker dependencies
  • Add only the GitHub Actions file to git

Changes Made:

First Iteration:

  1. Missing build dependencies: Added autoconf, automake, libtool, and python3 packages needed by autogen.sh
  2. Missing configure script generation: Added ./autogen.sh step to generate the configure script
  3. Incorrect script invocation: Changed to use make binary-dist which sets all required environment variables

Second Iteration (Simplified Approach):

  1. Create dist tarball on host: Run ./autogen.sh, ./configure, and make dist on Ubuntu host
  2. Extract tarball in Docker: Inside Docker, extract the source distribution and build from it
  3. Reduced Docker dependencies: Removed unnecessary build tools from Docker container

Third Iteration (Testing):

  1. Added branch trigger: Temporarily added push trigger for PR branch to enable testing

Fourth Iteration (Fix Dependencies):

  1. Install TREXIO on host: Following the HPC workflow pattern, install TREXIO library from source
  2. Install build dependencies: Install emacs, autoconf, libhdf5-dev, libblas-dev, liblapack-dev on host
  3. Proper configure: Now ./configure can succeed because all dependencies are available

Fifth Iteration (Optimize TREXIO):

  1. Create TREXIO dist tarball: Use make dist for TREXIO on the host
  2. Build TREXIO in Docker: Extract and build TREXIO from tarball inside Docker container
  3. Initially removed emacs: Attempted to remove emacs but discovered it's still needed

Sixth Iteration (Fix emacs requirement):

  1. Add emacs back: QMCkl uses literate programming with org-mode files
  2. autogen.sh needs emacs: Required to tangle org files into source code during autogen.sh
  3. Match HPC workflow: Now matches the dependency environment used in the HPC workflow

Seventh Iteration (Add HDF5 to Docker):

  1. Install hdf5-devel: TREXIO requires HDF5 library to build
  2. Rocky Linux package: Added hdf5-devel to Docker container dependencies

The workflow now:

  • Replicates the HPC workflow environment on the host (with emacs)
  • Creates clean distribution tarballs for both TREXIO and QMCkl
  • Builds from these tarballs inside Rocky Linux 8 Docker container (using old glibc)
  • Includes all necessary dependencies (HDF5) for TREXIO to build successfully
  • Produces binary distribution with Intel MKL dependencies embedded
Original prompt

This section details on the original issue you should resolve

<issue_title>Build binary distribution</issue_title>
<issue_description>I would like to build a binary distribution, using an old glibc and IntelMKL.
I started writing a script in the github actions but it does not work. Can you fix it please?
</issue_description>

<agent_instructions>only git add the github actions file</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits December 16, 2025 14:52
- Add autoconf, automake, libtool, and python3 dependencies
- Run autogen.sh to generate configure script
- Use make binary-dist instead of calling build_binary_dist.sh directly
- Update artifact path to build/qmckl-*-linux-x86_64.tar.gz
- Remove unnecessary make install step

Co-authored-by: scemama <[email protected]>
- Run 'make dist' on host machine to create source distribution
- Extract tarball inside Docker container for building
- Remove autoconf, automake, libtool, python3, cmake from Docker deps
- Simplifies Docker build process by using pre-packaged dist

Co-authored-by: scemama <[email protected]>
Copilot AI changed the title [WIP] Fix GitHub Actions script for binary distribution Fix binary distribution workflow: use make dist on host Dec 16, 2025
Copilot AI requested a review from scemama December 16, 2025 14:54
@scemama
Copy link
Member

scemama commented Dec 16, 2025

@copilot Can you run the workflow to test it?

Copilot AI and others added 2 commits December 16, 2025 15:00
- Install emacs, autoconf, libhdf5, BLAS, LAPACK on Ubuntu host
- Clone and install TREXIO library from source
- This allows ./configure to succeed when creating dist tarball

Co-authored-by: scemama <[email protected]>
Copy link
Contributor Author

Copilot AI commented Dec 16, 2025

@copilot Can you run the workflow to test it?

I've triggered the workflow by adding a temporary branch trigger. The workflow is now running automatically on each push to this PR branch.

The first attempt failed because TREXIO library wasn't installed on the host. I've now added steps to install TREXIO and all dependencies (emacs, autoconf, libhdf5, BLAS, LAPACK) following the pattern from the HPC workflow.

The latest workflow run (commit 5f37cec) is waiting for approval at: https://github.com/TREX-CoE/qmckl/actions/runs/20272483412

Once approved, it should successfully create the distribution tarball and build the binary package in Rocky Linux 8 with Intel MKL.

@scemama
Copy link
Member

scemama commented Dec 16, 2025

@copilot You should also make dist for TREXIO to copy the tarball of trexio in the docker, avoiding the need for emacs

- Create TREXIO distribution tarball on host with make dist
- Copy TREXIO tarball for use in Docker container
- Build TREXIO from tarball inside Docker (no emacs needed)
- Remove emacs from host dependencies
- Keep QMCkl --disable-doc flag

Co-authored-by: scemama <[email protected]>
Copy link
Contributor Author

Copilot AI commented Dec 16, 2025

@copilot You should also make dist for TREXIO to copy the tarball of trexio in the docker, avoiding the need for emacs

Done in commit 1656891. The workflow now:

  • Creates TREXIO distribution tarball on host with make dist
  • Copies the tarball into Docker workspace
  • Builds TREXIO from tarball inside Docker (no emacs needed)
  • Removed emacs from host dependencies

QMCkl continues to use --disable-doc flag. TREXIO is built without any doc flags since it doesn't support them.

@scemama
Copy link
Member

scemama commented Dec 16, 2025

@copilot On the host:

configure: error: 
    --------------------------------------
    Error: Emacs is required for org-mode.
    --------------------------------------

Please, replicate the environment used in other workflows, like HPC

- QMCkl uses literate programming with org-mode files
- autogen.sh requires emacs to tangle org files into source code
- Match HPC workflow environment with emacs installed

Co-authored-by: scemama <[email protected]>
Copy link
Contributor Author

Copilot AI commented Dec 16, 2025

@copilot On the host:

configure: error: ...

Fixed in commit 348f250. Added emacs back to the host dependencies - it's required for autogen.sh to process org-mode files (literate programming). The workflow now matches the HPC workflow environment with emacs installed.

@scemama
Copy link
Member

scemama commented Dec 16, 2025

@copilot In the docker, you also need to install hdf5 serial for trexio to work

- TREXIO requires HDF5 library to build
- Install hdf5-devel in Rocky Linux 8 container

Co-authored-by: scemama <[email protected]>
Copy link
Contributor Author

Copilot AI commented Dec 16, 2025

@copilot In the docker, you also need to install hdf5 serial for trexio to work

Added in commit b68d6ff. Installed hdf5-devel package in the Docker container for TREXIO to build successfully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build binary distribution

2 participants