copy: add option to strip sparse manifest lists #655

aguidirh · 2026-02-11T10:28:39Z

Add option to strip sparse manifest lists when copying specific images

This PR adds the ability to strip non-copied instances from manifest lists when using CopySpecificImages, implementing the functionality proposed in containers/image#1707.

Motivation

Relates to #227

When copying a subset of images from a multi-architecture manifest list using CopySpecificImages, the current behavior always produces a sparse manifest list - a manifest list that references platforms that weren't actually copied. While this maintains the original digest, many registries reject sparse manifests with "blob unknown to registry" errors.

This PR adds a new SparseManifestListAction option that gives users control over this behavior:

KeepSparseManifestList (default, existing behavior) - keeps all references even for non-copied images
StripSparseManifestList (new) - removes non-copied instances and generates a manifest list with only copied instances

This is particularly important for use cases like selective mirroring where only specific architectures need to be copied to registries that don't support sparse manifests.

Changes

1. New `SparseManifestListAction` option in `image/copy`

Added SparseManifestListAction enum with two values:
- KeepSparseManifestList (default): preserves existing behavior - keeps all manifest references
- StripSparseManifestList (new): removes non-copied instances from the manifest list
Integrated into Options struct to give users explicit control

2. New `ListOpDelete` operation in `image/internal/manifest`

Extended ListEdit with ListOpDelete operation and DeleteIndex field to support removing instances
Implemented deletion logic in both OCI1Index and Schema2List
Uses index-based deletion (not digest-based) to handle cases where multiple platforms share the same digest (e.g., amd64 variants v2/v3)
Requires deletion from highest to lowest index to prevent index shifting issues

3. Sparse manifest stripping in `copyMultipleImages` (`image/copy/multiple.go`)

Implemented stripping logic that activates when SparseManifestListAction == StripSparseManifestList
Identifies which instances were not copied by comparing against the original manifest list
Removes skipped instances using the new EditInstances API with ListOpDelete

4. Comprehensive test coverage

Added TestListDeleteInstances covering both OCI and Docker manifest formats
Tests both successful deletion and error handling for invalid indices
Validates high-to-low deletion order requirement

Testing

All existing tests pass. New tests verify:

✅ Successful deletion of instances by index
✅ Error handling for invalid indices
✅ Both OCI Index and Docker Schema2 List formats
✅ Manually tested with oc-mirror and local registry

Credits

This implementation is based on the original work by @bertbaron and @mtrmac in containers/image#1707, adapted for the container-libs monorepo structure with the following changes:

Moved from manifest package to internal/manifest (per container-libs directory structure)
Switched from digest-based to index-based deletion (per review feedback in original PR about platform variants)
Simplified test coverage while maintaining comprehensive validation of both OCI and Docker formats
Integrated with the existing EditInstances API pattern used throughout the codebase

@bertbaron

Add the ability to strip non-copied instances from manifest lists when using CopySpecificImages. This implements the functionality originally proposed in containers/image#1707. When copying a subset of images from a multi-architecture manifest list using CopySpecificImages, the current behavior always produces a sparse manifest list - a manifest list that references platforms that weren't actually copied. While this maintains the original digest, many registries reject sparse manifests with "blob unknown to registry" errors. This commit adds a new SparseManifestListAction option that gives users control over this behavior: - KeepSparseManifestList (default): preserves existing behavior - StripSparseManifestList (new): removes non-copied instances The implementation includes: 1. New SparseManifestListAction enum in image/copy 2. New ListOpDelete operation in image/internal/manifest with support for both OCI1Index and Schema2List formats 3. Index-based deletion (not digest-based) to handle platform variants that share the same digest 4. Stripping logic in copyMultipleImages that activates when StripSparseManifestList is set 5. Comprehensive test coverage for deletion operations Based on original work by @bertbaron and @mtrmac in containers/image#1707, adapted for the container-libs monorepo structure. Relates to containers#227 Signed-off-by: Alex Guidi <[email protected]>

packit-as-a-service · 2026-02-11T10:30:58Z

Packit jobs failed. @containers/packit-build please check.

mtrmac

Thanks! Just a brief first look: generally the structure and choice of affected sub packages look correct.

This is impressively small, I was expecting a that a larger change to internal/manifest would be necessary.

mtrmac · 2026-02-11T13:51:14Z

image/copy/multiple.go

+		}
+
+		// Find which indices were skipped
+		var indicesToDelete []int


(If the structure remains: I think this variable is unnecessary — we can do something like for range slices.Backwards(instanceDigests) { if !copiedDigests.Contains(…) { deleteEdits = append(…) } in one loop)

mtrmac · 2026-02-11T13:52:23Z

image/internal/manifest/docker_schema2_list.go

+				return fmt.Errorf("Schema2List.EditInstances: invalid delete index %d (list has %d instances)", editInstance.DeleteIndex, len(list.Manifests))
+			}
+			// Remove the element by appending slices before and after the target index
+			list.Manifests = append(list.Manifests[:editInstance.DeleteIndex], list.Manifests[editInstance.DeleteIndex+1:]...)


slices.Delete would be a bit better, (also in the other copy)

mtrmac · 2026-02-11T13:54:31Z

image/internal/manifest/list_test.go

 }
+
+// TestListDeleteInstances tests the ListOpDelete functionality for both OCI and Docker manifest formats.
+func TestListDeleteInstances(t *testing.T) {


I’d prefer this to be tested in the existing Test…EditInstances methods (but the two can share a helper if that is beneficial)

mtrmac · 2026-02-11T13:55:52Z

image/copy/multiple.go

+		// Remove skipped instances from the manifest list using EditInstances
+		if len(deleteEdits) > 0 {
+			logrus.Debugf("Removing %d instances from manifest list", len(deleteEdits))
+			if err := updatedList.EditInstances(deleteEdits, false); err != nil {


Maybe we can only call EditInstances once, appending to existing instanceEdits.

mtrmac · 2026-02-11T14:00:02Z

image/copy/multiple.go

+		// Build a set of digests that were copied
+		copiedDigests := set.New[digest.Digest]()
+		for _, instance := range instanceCopyList {
+			copiedDigests.Add(instance.sourceDigest)
+		}


What happens if the same digest is included multiple times in the index? That’s not very likely with per-platform images, but with CopySpecificImages, the caller can choose to copy one of the duplicates and not the other.

prepareInstanceCopies knows exactly which instances are copied and which are not, so I think it can also prepare some version of the deletion instructions.

Also, prepareInstanceCopies is intentionally a ~pure function, so that it is unit-testable. Unit tests for the logic here, ensuring we ask for deleting the correct instances, seem worthwhile.

mtrmac · 2026-02-11T14:00:57Z

image/copy/copy.go

+
+	// StripSparseManifestList will strip missing images from the manifest
+	// list. When images are stripped the digest will differ from the original.
+	StripSparseManifestList


Suggested change

StripSparseManifestList

StripSparseManifestList SparseManifestListAction

IIRC, otherwise it doesn’t have the specialized type.

mtrmac · 2026-02-11T14:01:59Z

image/copy/copy.go


+	// When only a subset of images of a list is copied, this action indicates if the manifest should be kept or stripped.
+	// See CopySpecificImages.
+	SparseManifestListAction SparseManifestListAction


I think this would be better placed closer to the ImageListSelection field.

github-actions bot added the image Related to "image" package label Feb 11, 2026

mtrmac reviewed Feb 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

copy: add option to strip sparse manifest lists #655

copy: add option to strip sparse manifest lists #655

Uh oh!

aguidirh commented Feb 11, 2026

Uh oh!

packit-as-a-service bot commented Feb 11, 2026

Uh oh!

mtrmac left a comment

Uh oh!

mtrmac Feb 11, 2026

Uh oh!

mtrmac Feb 11, 2026

Uh oh!

mtrmac Feb 11, 2026

Uh oh!

mtrmac Feb 11, 2026

Uh oh!

mtrmac Feb 11, 2026

Uh oh!

mtrmac Feb 11, 2026

Uh oh!

mtrmac Feb 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	StripSparseManifestList
	StripSparseManifestList SparseManifestListAction

copy: add option to strip sparse manifest lists #655

Are you sure you want to change the base?

copy: add option to strip sparse manifest lists #655

Uh oh!

Conversation

aguidirh commented Feb 11, 2026

Add option to strip sparse manifest lists when copying specific images

Motivation

Changes

1. New SparseManifestListAction option in image/copy

2. New ListOpDelete operation in image/internal/manifest

3. Sparse manifest stripping in copyMultipleImages (image/copy/multiple.go)

4. Comprehensive test coverage

Testing

Credits

Uh oh!

packit-as-a-service bot commented Feb 11, 2026

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

mtrmac Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

mtrmac Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

1. New `SparseManifestListAction` option in `image/copy`

2. New `ListOpDelete` operation in `image/internal/manifest`

3. Sparse manifest stripping in `copyMultipleImages` (`image/copy/multiple.go`)