Add test for uploading system CNB

[sethboyles]

Are you submitting this PR against the develop branch?

Yes

What is this change about?

New system CNB feature

Please provide contextual information.

https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0031-system-cnb.md
cloudfoundry/cloud_controller_ng#4342

What version of cf-deployment have you run this cf-acceptance-test change against?

v49.0.0, but you need pre-release CF CLI

Please check all that apply for this PR:

• introduces a new test --- Are you sure everyone should be running this test?
• changes an existing test
• requires an update to a CATs integration-config

Did you update the README as appropriate for this change?

• YES
• N/A

If you are introducing a new acceptance test, what is your rationale for including it CATs rather than your own acceptance test suite?

Tests basic system CNB workflow

How many more (or fewer) seconds of runtime will this change introduce to CATs?

120s

What is the level of urgency for publishing this change?

• Urgent - unblocks current or future work
• Slightly Less than Urgent

Currently blocked on CLI release

Tag your pair, your PM, and/or team!

[sethboyles]

Do not merge until we have a new version of CF CLI

[sethboyles]

We have a new version of the CF CLI, so this is ready to merge: https://github.com/cloudfoundry/cli/releases/tag/v8.14.0

[beyhan]

I was just wondering wether we need to specify the buildpack name or let CF figure it out. In this way we could also test the auto detection with system buildpacks. Or does this makes the test flaky?

[davewalter]

I don't think it would make the test flakier than it might otherwise be, but it would certainly make it take longer to run. Most of the tests in the buildpack detect suite rely on auto-detection so I don't think we need to also test it here.

[beyhan]

Based on my current understanding, enabling the –b option does not appear to affect execution time significantly, as the detect command is executed in both cases with an order containing a single buildpack (assuming my interpretation of this code section is accurate). Additionally, there seems to be a minor distinction between using the –b option and not, as indicated in this pull request. However, since we already have a comprehensive buildpack test suite that will be executed against CNBs, I also think that it is not necessary to add further tests about auto detection here.

Thank you for the thoughtful discussion!

[oliver-heinrich]

Hi @sethboyles , the changes are looking good to me. I have the following questions:

• Do we need also tests for CNB auto-detection?
• Is the CNB auto-detection behavior the same like for the Classic buildpacks
• are there any plan to provide the CNB's as part of the cf deployment like for the classic buildpacks in the cf-deployment.yml see e.g. the following references:
  ** https://github.com/cloudfoundry/cf-deployment/blob/main/cf-deployment.yml#L927 (install order)
  ** https://github.com/cloudfoundry/cf-deployment/blob/main/cf-deployment.yml#L2883 (buildpacks added in the release section)
  Do you want to join the next ARD community meeting on 12th June 5:30 PM CEST time to discuss this?

@dimivel : FYI

[sethboyles]

1. AFAIK the process should be basically the same--the cnbapplifecycle lifecycle will run through the various detect commands (or equivalent, I'm not sure if CNBs commands are named similarly). I believe this would be testing the cnbapplifecycle + individual buildpacks, so if we think that is valuable, I think we can add it. @tomkennedy513 or @pbusko may have more context on whether or not this worth adding.
2. We'd like to eventually add default system CNBs, but as far as I know there are no CNBs packaged in bosh releases yet. I'm not sure if anyone is working this at the moment.

I'm happy to join the ARD meeting, but I don't have much context beyond the CAPI side of this work.

[oliver-heinrich]

Hi @beyhan , maybe we can this test and later take care about CMB build-pack detection.
CNB's are not added so far to the complete lifecycle ( means CNB's are not available on Bosh.io and also not part of cf-deployment.yml,...). Aren't this pre-conditions to enhance the CATs testsuite with auto-detection tests for CNB buildpacks?

[beyhan]

Point 2.) should be out of scope here and a separate discussion but for 1.) I don't think that we have to address 2.). My understanding is that the uploaded CNB buildpack in the preparation step of this pr is enough to test auto detection. I would be also interested in additional opinion here.

[tomkennedy513]

1. AFAIK the process should be basically the same--the cnbapplifecycle lifecycle will run through the various detect commands (or equivalent, I'm not sure if CNBs commands are named similarly). I believe this would be testing the cnbapplifecycle + individual buildpacks, so if we think that is valuable, I think we can add it. @tomkennedy513 or @pbusko may have more context on whether or not this worth adding.
2. We'd like to eventually add default system CNBs, but as far as I know there are no CNBs packaged in bosh releases yet. I'm not sure if anyone is working this at the moment.

I'm happy to join the ARD meeting, but I don't have much context beyond the CAPI side of this work.

I think it makes sense to have a test that runs through autodetect vs specifying (ideally multiple) buildpacks just to ensure that the order logic is sound (though this could probably also be lifecycle specific tests as well)

[davewalter]

Does the CNB auto-detect logic follow a different path to the "regular buildpack" detection process that we already test?

[tomkennedy513]

Does the CNB auto-detect logic follow a different path to the "regular buildpack" detection process that we already test?

Not that I’m aware of. I believe the lifecycle just gets a list of buildpacks and whether or not it is autodetect vs a requested list. Under the hood that changes how the order is set up for the cab lifecycle