CS repo directory problems in container deployment

[sfayer]

Hi,

We're working on the non-k8s diracx container deployment recipe and have run into some issues with loading the CS store... We've exported our existing CS into the diracx format and have it in a local git repo directory which we're initially bind mounting in as a read-only mount at /cs_store and setting DIRACX_CONFIG_BACKEND_URL="git+file:///cs_store?branch=master".

This doesn't work: The git+file handler runs into a number of issues:

• The bind mount point files are owned by "root", whereas the diracx container uses "mambauser" which trips out git's insecure file permissions check for the rev-parse command:

  diracx/diracx-core/src/diracx/core/config/sources.py

  Lines 251 to 252 in c265a47

  	sh.git(
  	"rev-parse",

• If we bypass the security checks, the checkout fails because we made it a bare repo.
• If we convert it to a normal repo, the checkout fails because the volume is read-only.
• If we make the volume read-write, the checkout fails because of the different file ownership.

We modified the code, to use the git+https "RemoteGitConfigSource" handler for the git+file case too and this works (and I suspect would also work better if this directory was shared across multiple hosts).

Our questions for discussion are:

• Is there actually a good use case for the LocalGitConfigSource class? It seems to me that in the majority of deployments RemoteGitConfigSource would work the same or better, even if the source is actually a local directory.
• If removing LocalGitConfigSource isn't an option, perhaps we could make using it optional instead? (For example offering 3 protocol types, say git+https and git+file using RemoteGitConfigSource and a new git+local option if you really want the LocalGitConfigSource behavior)?

We're not really sure how this is all going to work out if it needs to start pushing commits back: Getting the permissions all correct for writing a bind-mount on any platform seems to be its own challenge (but perhaps that isn't planned for a while and we can worry about that later?).

Regards,
Simon

Mentioning @marianne013

[chrisburr]

Cloning a local directory with different ownership requires this delightful incantation:

git -c safe.directory="*" \
    clone \
    -u 'git -c safe.directory="*" upload-pack' \
    ...

See https://lore.kernel.org/git/[email protected]/t/#u

[sfayer]

Hmm, I'm not sure that's still the case... It seems like a directory owned by another user is not that different to cloning over https: from a security perspective they're both untrusted. I suspect that blocking cloning from untrusted local directories was the initial workaround for the CVE and the behaviour was changed again since then. Our testing shows that it does work with the current version of git in the dirac-services container (I don't think that contains any special git config?):

(base) mambauser@fe21e32c1425:/tmp$ git -v
git version 2.51.0
(base) mambauser@fe21e32c1425:/tmp$ whoami
mambauser
(base) mambauser@fe21e32c1425:/tmp$ ls -ld /diracx_sources/
drwxr-xr-x. 16 root root 4096 Nov 13 18:17 /diracx_sources/
(base) mambauser@fe21e32c1425:/tmp$ git clone file:///diracx_sources
Cloning into 'diracx_sources'...
remote: Enumerating objects: 11695, done.
...
(base) mambauser@fe21e32c1425:/tmp$ ls -l /tmp/diracx_sources/
total 132
-rw-r--r--. 1 mambauser mambauser   677 Nov 14 11:16 CHANGELOG.md
-rw-r--r--. 1 mambauser mambauser 32452 Nov 14 11:16 LICENSE
...

[chrisburr]

Nice! I'm glad that has been fixed, it seems like file URLs started working in 2.48.0 or 2.48.1.

$ git --version
git version 2.45.1

$ git -c safe.directory="*" clone /cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git /tmp/cburr/xxx/
Cloning into '/tmp/cburr/xxx'...
fatal: detected dubious ownership in repository at '/cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git'
To add an exception for this directory, call:

	git config --global --add safe.directory /cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

$ git clone file:///cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git /tmp/cburr/xxx/
Cloning into '/tmp/cburr/xxx'...
fatal: detected dubious ownership in repository at '/cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git'
To add an exception for this directory, call:

	git config --global --add safe.directory /cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

$ git --version
git version 2.51.2

$ git -c safe.directory="*" clone /cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git /tmp/cburr/xxx/
Cloning into '/tmp/cburr/xxx'...
done.

$ env GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=safe.directory GIT_CONFIG_VALUE_0='*' git clone /cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git /tmp/cburr/xxx/
Cloning into '/tmp/cburr/xxx'...
done.

$ git clone file:///cvmfs/lhcb.cern.ch/lib/lhcb/lcg-toolchains/.git /tmp/cburr/xxx/
Cloning into '/tmp/cburr/xxx'...
remote: Enumerating objects: 4030, done.
remote: Counting objects: 100% (4030/4030), done.
remote: Compressing objects: 100% (2598/2598), done.
remote: Total 4030 (delta 3285), reused 1529 (delta 1360), pack-reused 0 (from 0)
Receiving objects: 100% (4030/4030), 3.06 MiB | 5.33 MiB/s, done.
Resolving deltas: 100% (3285/3285), done.

[chrisburr]

I'm inclined to drop the LocalGitConfigSource but I'll discuss with Chris on Monday.

[chrisburr]

Looks like the fix was git/git@0ffb5a6