-
Notifications
You must be signed in to change notification settings - Fork 1.9k
OSDOCS Document how to perform boot image updates on marketplace clusters AWS #108736
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mburke5678
wants to merge
8
commits into
openshift:main
Choose a base branch
from
mburke5678:mco-manual-boot-image-aws
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
07fd1fd
OSDOCS Document how to perform boot image updates on marketplace clus…
ee5772e
proofread
0cd711f
proofread
f5d39a5
Add note on auto update
526f36f
edits per djoshy
d7babf1
edit
0943245
proofread
50ba7b1
Added Manually updating the boot image assembly
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,292 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * machine_configuration/mco-update-boot-images.adoc | ||
|
|
||
| :_mod-docs-content-type: PROCEDURE | ||
| [id="mco-update-boot-images-aws_{context}"] | ||
| = Manually updating the boot image on an {aws-short} cluster | ||
|
|
||
| [role="_abstract"] | ||
| You can manually update the boot image for your {aws-first} cluster by configuring your machine sets to use the latest {product-title} image as the boot image to ensure that new nodes can scale up properly. | ||
|
|
||
| The following procedure, which includes steps to create environment variables that facilitate running the required commands, shows how to identify the correct Amazon Machine Image (AMI) to use as the new boot image and how to modify your compute machine sets to use that image. | ||
|
|
||
| The process differs for clusters that use a default {op-system-first} image and clusters that use a custom {op-system-first} image from the {aws-short} Marketplace. The following procedure helps determine which type of image you use. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| For clusters that use a default {op-system-first} image, you can configure the cluster to automatically update the boot image each time the cluster is updated. If you are using the following procedure, ensure that automatic boot image updates are disabled and skew enforcement is in manual mode. For more information, see "Boot image management" and "Boot image skew enforcement". | ||
| ==== | ||
|
|
||
| .Prerequisites | ||
|
|
||
| * You have completed the general boot image prerequisites as described in the "Prerequisites" section of the link:https://access.redhat.com/articles/7053165#prerequisites-2[{product-title} Boot Image Updates knowledgebase article]. | ||
|
|
||
| * You have installed the {oc-first}. | ||
|
|
||
| * You have set boot image skew enforcement to the manual or none mode. For more information, see "Configuring boot image skew enforcement". | ||
|
|
||
| * You have disabled boot image management for the cluster. For more information, see "Disabling boot image management". | ||
|
|
||
| * You have installed the link:https://aws.amazon.com/cli/[{aws-short} CLI]. | ||
|
|
||
| * You configured an AWS account to host the cluster. For information, see "Configuring an AWS account". | ||
|
|
||
| * For a cluster that uses a default {op-system} image, you have downloaded the latest version of the {product-title} installation program, openshift-install, from the {cluster-manager-url}. For more information, see "Obtaining the installation program." | ||
|
|
||
| * For a cluster that uses a default {op-system} image, you have installed the link:https://stedolan.github.io/jq/[`jq`] program. | ||
|
|
||
| .Procedure | ||
|
|
||
| . Determine if your cluster uses a default {op-system} image or a custom {op-system} image from the {aws-short} Marketplace image: | ||
|
|
||
| .. Obtain the current {aws-short} region where the cluster is installed and set the value in an environment variable by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ export REGION=$(oc get infrastructure cluster -o jsonpath='{.status.platformStatus.aws.region}') | ||
| ---- | ||
|
|
||
| .. Obtain the current Amazon Machine Image (AMI) ID for your region and set the value in an environment variable by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ export CURRENT_AMI=$(oc get machineset -n openshift-machine-api -o jsonpath='{.items[0].spec.template.spec.providerSpec.value.ami.id}') | ||
| ---- | ||
|
|
||
| .. Obtain the product ID for your AMI and set the value in an environment variable by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ export PRODUCT_ID=$(aws ec2 describe-images --image-ids "$CURRENT_AMI" --region "$REGION" \ | ||
| --query 'Images[0].Name' --output text | \ | ||
| grep -oE '[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}') | ||
| ---- | ||
| + | ||
| `CURRENT_AMI` and `REGION` are environment variables you created in previous steps. | ||
|
|
||
| .. Display the contents of the `PRODUCT_ID` environment variable by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ echo $PRODUCT_ID | ||
| ---- | ||
| + | ||
| * If the output for the `PRODUCT_ID` environment variable is empty, as shown in the following example, your cluster uses a standard {product-title} image. | ||
| + | ||
| .Example with empty output | ||
| [source,terminal] | ||
| ---- | ||
| $ echo $PRODUCT_ID | ||
|
|
||
| $ | ||
| ---- | ||
| + | ||
| * If the output for the `PRODUCT_ID` environment variable is not empty, as shown in the following example, your cluster uses an {aws-short} Marketplace image. | ||
| + | ||
| .Example with non-empty output | ||
| [source,terminal] | ||
| ---- | ||
| $ echo $PRODUCT_ID | ||
| 59ead7de-2540-4653-a8b0-fa7926d5c845 | ||
| $ | ||
| ---- | ||
| + | ||
| * If the command returns an error, and you are unable to determine your cluster variant, contact Red Hat Support. If Red Hat Support determines that your cluster uses an {aws-short} Marketplace image, you can set the `PRODUCT_ID` environment variable with the appropriate product ID from the following table. | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ export PRODUCT_ID=<Product_ID_from_table> | ||
| ---- | ||
| + | ||
| [cols="1,1",options="header"] | ||
| |=== | ||
| | Variant | Product ID | ||
| | link:https://aws.amazon.com/marketplace/procurement/?productId=59ead7de-2540-4653-a8b0-fa7926d5c845[{product-title} on x86 - NA]| `59ead7de-2540-4653-a8b0-fa7926d5c845` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=963b36c3-de6f-48ed-b802-2b38b2a2cdeb[{oke} on x86 - NA]| `963b36c3-de6f-48ed-b802-2b38b2a2cdeb` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=f5da01a6-d046-487c-9072-42fe53b1cad4[{opp} on x86 - NA]| `f5da01a6-d046-487c-9072-42fe53b1cad4` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=abc249f8-7440-45f7-a4b1-c026baff64c1[{product-title} on ARM - NA]| `abc249f8-7440-45f7-a4b1-c026baff64c1` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=d2d3ebcd-c1ca-43d8-bf0a-530433200f35[{oke} on ARM - NA]| `d2d3ebcd-c1ca-43d8-bf0a-530433200f35` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=be6d3e94-c8dc-4a3e-9218-4b449b11f06f[{opp} on ARM - NA]| `be6d3e94-c8dc-4a3e-9218-4b449b11f06f` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=962791c7-3ae5-46d1-ba62-c7a5ebac54fd[{product-title} on x86 - EU, ME and Africa]| `962791c7-3ae5-46d1-ba62-c7a5ebac54fd` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=7026c8d7-392c-4010-b93c-f93f7bc5495f[{oke} on x86 - EU, ME and Africa]| `7026c8d7-392c-4010-b93c-f93f7bc5495f` | ||
| | link:https://aws.amazon.com/marketplace/procurement?productId=628c9df3-0254-4f91-bc1f-8619d1b8eaa8[{opp} on x86 - EU, ME and Africa]| `628c9df3-0254-4f91-bc1f-8619d1b8eaa8` | ||
| |=== | ||
|
|
||
| . Determine the AMI for the new boot image by using one of the following steps, depending upon the type of images used in your cluster: | ||
|
|
||
| * For a cluster that uses a default {op-system} image, perform the following steps: | ||
| + | ||
| .. Set an environment variable with your cluster architecture by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ export ARCH=<architecture_type> | ||
| ---- | ||
| + | ||
| Replace `<architecture_type>` with one of the following values: | ||
| + | ||
| -- | ||
| * Specify `aarch64` for the AArch64 or ARM64 architecture. | ||
| * Specify `ppc64le` for the {ibm-power-name} (ppc64le) architecture. | ||
| * Specify `s390x` for the {ibm-z-name} and {ibm-linuxone-name} (s390x) architecture. | ||
| * Specify `x86_64` for the x86_64 or AMD64 architecture. | ||
| -- | ||
| + | ||
| You can find the architecture as a label in any `MachineSet` object. | ||
| + | ||
| .Example machine set with an architecture label | ||
| [source,terminal] | ||
| ---- | ||
| apiVersion: machine.openshift.io/v1beta1 | ||
| kind: MachineSet | ||
| metadata: | ||
| annotations: | ||
| capacity.cluster-autoscaler.kubernetes.io/labels: kubernetes.io/arch=amd64 | ||
| # ... | ||
| ---- | ||
|
|
||
| .. Obtain the AMI for the new boot image and set an environment variable with the AMI by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ export AMI_ID=$(openshift-install coreos print-stream-json | jq -r ".architectures.\"${ARCH}\".images.aws.regions.\"${REGION}\".image") | ||
| ---- | ||
| + | ||
| `ARCH` and `REGION` are environment variables you created in previous steps. | ||
|
|
||
| .. View the {op-system-first} version of the new boot image by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ openshift-install coreos print-stream-json | jq -r ".architectures.\"${ARCH}\".images.aws.regions.\"${REGION}\".release" | ||
| ---- | ||
| + | ||
| .Example output | ||
| [source,terminal] | ||
| ---- | ||
| 9.6.20251212-1 | ||
| ---- | ||
| + | ||
| Make note of the {op-system} version for later use. | ||
|
|
||
| * For a cluster that uses a custom {op-system} image, perform the following steps: | ||
| + | ||
| .. Obtain a list of valid AMI images by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ aws ec2 describe-images --region "${REGION}" --filters "Name=name,Values=*${PRODUCT_ID}*" \ | ||
| --query 'reverse(sort_by(Images, &CreationDate))[].[CreationDate,ImageId,Name]' --output table | ||
| ---- | ||
| + | ||
| `REGION` and `PRODUCT_ID` are environment variables you created in previous steps. | ||
| + | ||
| This command returns the AMIs ordered by creation date, with the latest images first. The {op-system} version of each AMI is contained in the AMI name. Choose the latest image version available. | ||
| + | ||
| Make note of the {op-system-first} version for later use. | ||
|
|
||
| .. Set an environment variable with the AMI of the new boot image by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ export AMI_ID=<ami-value> | ||
| ---- | ||
|
|
||
| . Update each of your compute machine sets to include the new boot image: | ||
|
|
||
| .. Obtain the name of your machine sets for use in the following step by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc get machineset -n openshift-machine-api | ||
| ---- | ||
| + | ||
| .Example output | ||
| [source,terminal] | ||
| ---- | ||
| NAME DESIRED CURRENT READY AVAILABLE AGE | ||
| rhhdrbk-b5564-4pcm9-worker-0 3 3 3 3 123m | ||
| ci-ln-xj96skb-72292-48nm5-worker-d 1 1 1 1 27m | ||
| ---- | ||
|
|
||
| .. Edit a machine set to update the `image` field in the `providerSpec` stanza to add your boot image by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc patch machineset <machineset-name> -n openshift-machine-api --type merge -p '{"spec":{"template":{"spec":{"providerSpec":{"value":{"ami":{"id":"'${AMI_ID}'"}}}}}}}' | ||
| ---- | ||
| + | ||
| Replace `<machineset_name>` with the name of your machine set. | ||
| + | ||
| `AMI_ID` is the environment variable you created in a previous step. | ||
|
|
||
| . If boot image skew enforcement in your cluster is set to the manual mode, update the boot image version in the `MachineConfiguration` object as described in "Updating the boot image skew enforcement version." | ||
|
|
||
| .Verification | ||
|
|
||
| . Scale up a machine set to check that the new node is using the new boot image: | ||
|
|
||
| .. Increase the machine set replicas by one to trigger a new machine by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc scale --replicas=<count> machineset <machineset_name> -n openshift-machine-api | ||
| ---- | ||
| where: | ||
|
|
||
| `<count>`:: Specifies the total number of replicas, including any existing replicas, that you want for this machine set. | ||
| `<machineset_name>`:: Specifies the name of the machine set to scale. | ||
|
|
||
| .. Optional: View the status of the machine set as it provisions by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc get machines.machine.openshift.io -n openshift-machine-api -w | ||
| ---- | ||
| + | ||
| It can take several minutes for the machine set to achieve the `Running` state. | ||
|
|
||
| .. Verify that the new node has been created and is in the `Ready` state by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc get nodes | ||
| ---- | ||
|
|
||
| . Verify that the new node is using the new boot image by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc debug node/<new_node> -- chroot /host cat /sysroot/.coreos-aleph-version.json | ||
| ---- | ||
| + | ||
| Replace `<new_node>` with the name of your new node. | ||
| + | ||
| .Example output | ||
| [source,terminal] | ||
| ---- | ||
| { | ||
| # ... | ||
| "ref": "docker://ostree-image-signed:oci-archive:/rhcos-9.6.20251212-1-ostree.x86_64.ociarchive", | ||
| "version": "9.6.20251212-1" | ||
| } | ||
| ---- | ||
| where: | ||
|
|
||
| `version`:: Specifies the boot image version. | ||
|
|
||
| . Verify that the boot image is the same the {op-system} version as the image you noted in a previous step by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ echo ${AMI_ID} | ||
| ---- | ||
| + | ||
| `AMI_ID` is the environment variable you created in a previous step. | ||
| + | ||
| .Example output | ||
| [source,terminal] | ||
| ---- | ||
| 9.6.20251212-1 | ||
| ---- | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.