Skip to content

Conversation

@cgwalters
Copy link
Contributor

This is a redo of containers/storage#1933 but this time Assisted-by: OpenCode (Opus 4.6). Main motivation is just helping people who need to look at this (e.g. debugging, forensics) without digging deep into source code, and obviously would be useful for PoC work like containers/composefs-rs#218

Add doc comments to the overlay, vfs, btrfs, and zfs storage driver
packages describing their implementation approach:

- overlay: Uses Linux OverlayFS with composefs option for integrity
- vfs: Copies directories, attempting reflinks first for efficiency
- btrfs: Uses native subvolumes and snapshots with qgroup quotas
- zfs: Uses datasets and clones with mountpoint=legacy

Assisted-by: OpenCode (Opus 4.5)
Signed-off-by: Colin Walters <[email protected]>
Create dedicated documentation files for each storage driver:
- containers-storage-driver-overlay.md (includes zstd:chunked and composefs)
- containers-storage-driver-vfs.md
- containers-storage-driver-btrfs.md
- containers-storage-driver-zfs.md

The composefs and zstd:chunked documentation is consolidated into the
overlay driver doc since these are overlay-specific features.

Assisted-by: OpenCode (Opus 4.5)
Signed-off-by: Colin Walters <[email protected]>
@github-actions github-actions bot added the storage Related to "storage" package label Feb 9, 2026
@packit-as-a-service
Copy link

Packit jobs failed. @containers/packit-build please check.

The on-disk file layout is an internal implementation detail and may change between versions. The only stable interface is the Go library API.
The description below is intended to aid debugging and recovery, but changing content directly is not supported.

The top-level overlay directory holds layers keyed by a [chain ID](https://github.com/opencontainers/image-spec/blob/main/config.md#layer-chainid) which identifies the precise sequence of parent layers leading to this one. A layer with the same DiffID can have multiple physical objects in this directory if it was created in different contexts (e.g. with or without zstd:chunked).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The top-level overlay directory holds layers keyed by a [chain ID](https://github.com/opencontainers/image-spec/blob/main/config.md#layer-chainid) which identifies the precise sequence of parent layers leading to this one. A layer with the same DiffID can have multiple physical objects in this directory if it was created in different contexts (e.g. with or without zstd:chunked).
The top-level overlay directory holds layers keyed by a [chain ID](https://github.com/opencontainers/image-spec/blob/main/config.md#layer-chainid), which identifies the precise sequence of parent layers leading to this one. A layer with the same DiffID can have multiple physical objects in this directory if it was created in different contexts (e.g., with or without zstd:chunked).


## RUNTIME

When a container needs its filesystem, the driver performs a `mount(2)` with type `overlay`, passing the layer's `diff` directory as the upperdir and all parent layers' `diff` directories as lowerdirs. The kernel's overlayfs merges these at access time — no data is copied, and layers remain independent on disk. Writes go to the upperdir via copy-up. The mount is placed at the layer's `merged` directory, and the `work` directory is used internally by overlayfs for atomic operations like rename.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When a container needs its filesystem, the driver performs a `mount(2)` with type `overlay`, passing the layer's `diff` directory as the upperdir and all parent layers' `diff` directories as lowerdirs. The kernel's overlayfs merges these at access time — no data is copied, and layers remain independent on disk. Writes go to the upperdir via copy-up. The mount is placed at the layer's `merged` directory, and the `work` directory is used internally by overlayfs for atomic operations like rename.
When a container needs its filesystem, the driver performs a `mount(2)` with type `overlay`, passing the layer's `diff` directory as the upperdir and all parent layers' `diff` directories as lowerdirs. The kernel's overlayfs merges these at access time — no data is copied, and layers remain independent on disk. Writes go to the upperdir via copy-up. The mount is placed in the layer's `merged` directory, and the `work` directory is used internally by overlayfs for atomic operations such as rename.


## RUNTIME

There is no mount involved. When a container needs its filesystem, `Get()` simply returns the layer's directory path. All layer merging happened at create time when the parent was copied, so the directory is already a complete filesystem tree. `Put()` is a no-op since there is nothing to unmount.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
There is no mount involved. When a container needs its filesystem, `Get()` simply returns the layer's directory path. All layer merging happened at create time when the parent was copied, so the directory is already a complete filesystem tree. `Put()` is a no-op since there is nothing to unmount.
There is no mount involved. When a container needs its filesystem, `Get()` simply returns the layer's directory path. All layer merging occurs at create time, when the parent is copied, so the directory is already a complete filesystem tree. `Put()` is a no-op since there is nothing to unmount.

Copy link
Member

@TomSweeneyRedHat TomSweeneyRedHat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nitty suggestions for consideration or to be ignored. Otherwise:
LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

storage Related to "storage" package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants