Adding G4 Test cases

[LAVEEN]

This PR introduces G4 integration test for g4 slurm blueprint

Submission Checklist

NOTE: Community submissions can take up to 2 weeks to be reviewed.

Please take the following actions before submitting this pull request.

• Fork your PR branch from the Toolkit "develop" branch (not main)
• Test all changes with pre-commit in a local branch #
• Confirm that "make tests" passes all tests
• Add or modify unit tests to cover code changes
• Ensure that unit test coverage remains above 80%
• Update all applicable documentation
• Follow Cluster Toolkit Contribution guidelines #

[gemini-code-assist]

Summary of Changes

Hello @LAVEEN, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a series of significant updates aimed at modernizing and expanding the toolkit's capabilities. Key changes include upgrading the supported Ubuntu version to 22.04 LTS across examples and documentation, and integrating new features such as Spot VM support for G4 instances and Google Cloud Storage Anywhere Cache. The internal module structure has been improved by refactoring the service-account module, and the overall continuous integration and testing framework has been bolstered with new and updated test cases for GKE and Slurm environments.

Highlights

• Ubuntu 22.04 LTS Upgrade: Many examples, documentation, and test configurations have been updated to use Ubuntu 22.04 LTS, replacing the older 20.04 LTS version.
• G4 Instance Spot VM Support: New test cases and configurations have been added for G4 instances, including the ability to provision G4 VMs as Spot instances within Slurm blueprints.
• Anywhere Cache Integration for Cloud Storage: The Cloud Storage bucket module now supports Google Cloud Storage Anywhere Cache, enhancing data access performance, along with new tests to validate this functionality.
• Service Account Module Refactoring: The service-account module has been refactored and moved from the community directory to the main modules directory, with all references updated accordingly.
• Enhanced CI/Testing for GKE/Slurm: New CI tests, Ansible playbooks, and updated validation logic have been introduced for various GKE and Slurm configurations, including specific tests for TPU jobs and A3/A4 Mega/UltraGPU spot instances.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/pr-precommit.yml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds new variables to the ml-slurm-g4.yaml blueprint to support Spot VMs and reservations, and introduces a new integration test for G4 instances using Spot VMs. My review focuses on the implementation of the new test. I've found a critical issue in the Cloud Build configuration where a sed command incorrectly modifies the blueprint, leading to a duplicate key. I've also identified a variable name mismatch in the test configuration file that would prevent the test from running as intended. My suggestions aim to fix these issues by removing the unnecessary sed command and correcting the variable name, aligning the test with the project's existing patterns for test configuration.