Skip to content

Fix build caching: remove generateSecretKeys dependency from test#1028

Merged
jacopoc merged 4 commits intoapache:trunkfrom
ribafish:fix/generate-secret-keys-caching
Apr 5, 2026
Merged

Fix build caching: remove generateSecretKeys dependency from test#1028
jacopoc merged 4 commits intoapache:trunkfrom
ribafish:fix/generate-secret-keys-caching

Conversation

@ribafish
Copy link
Copy Markdown
Contributor

@ribafish ribafish commented Mar 25, 2026

Summary

The test task depends on generateSecretKeys, which writes new random values to framework/security/config/security.properties on every build. Since this file is part of the main resource source set (config/ dirs are included via getDirectoryInActiveComponentsIfExists('config')), changing it causes cascading Gradle build cache misses for:

  • :compileTestGroovy — classpath includes main resources
  • :checkstyleMain — classpath includes sourceSets.main.output
  • :checkstyleTest — same reason
  • :test — classpath + test classes differ

This PR:

  1. Removes dependsOn 'generateSecretKeys' from the test task — the main security.properties is no longer mutated during build
  2. Adds a test-specific security.properties in framework/security/src/test/resources/ with fixed keys — this shadows the main config on the test classpath so unit tests that require JWT keys (e.g. ModelFormTest) continue to work

The generateSecretKeys task is unchanged and remains available for manual use (./gradlew generateSecretKeys) and as a dependency of loadAll.

ribafish added a commit to gradle/develocity-oss-projects that referenced this pull request Mar 25, 2026
The generateSecretKeys task writes new random keys to security.properties
on every build, causing cascading cache misses for compileTestGroovy,
checkstyleMain, checkstyleTest, and test. Use a git hook to remove the
test dependency and provide fixed keys instead.

Upstream fix: apache/ofbiz-framework#1028
@jacopoc jacopoc self-assigned this Mar 25, 2026
@jacopoc
Copy link
Copy Markdown
Contributor

jacopoc commented Mar 25, 2026

Hi @ribafish,
your report is valid, but I addressed it using a different strategy than the one you proposed.

I’ve merged my changes in PR #1029.
Please take a look and let me know if this resolves the issues you were experiencing.

Thank you!

@ribafish ribafish closed this Mar 26, 2026
@ribafish ribafish reopened this Mar 26, 2026
The test task previously depended on generateSecretKeys, which writes
new random values to security.properties on every build. Since this
file is part of the main resource source set, it causes cascading cache
misses for compileTestGroovy, checkstyleMain, checkstyleTest, and test.

Instead, provide a test-specific security.properties with fixed keys
in framework/security/src/test/resources/. This shadows the main config
on the test classpath, so unit tests that need JWT keys (e.g.
ModelFormTest) still work. The generateSecretKeys task remains available
for manual use and for the loadAll task.
@ribafish ribafish force-pushed the fix/generate-secret-keys-caching branch from d92eb60 to 4054eb6 Compare March 26, 2026 11:51
@ribafish
Copy link
Copy Markdown
Contributor Author

Thanks for the quick fix in #1029!

It does resolve the issue for local build cache reuse (two consecutive builds on the same machine) — the first build generates keys, the second sees them and skips.

However, it doesn't fully address cross-machine build cache reuse. On a fresh checkout the keys are empty, so generateSecretKeys still generates random values and mutates security.properties. Two different machines checking out the same commit will each generate different random keys, producing different processResources output and different cache keys for :compileTestGroovy, :checkstyleMain, :checkstyleTest, and :test.

This PR complements #1029 by removing generateSecretKeys from the build task graph entirely (it remains available for manual use and loadAll), and providing fixed test keys via a test resource file so that security.properties is never mutated during builds.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of creating a copy of security.properties that is only used by tests, however it would be nice to keep it minimal (i.e., do not include any property that is not required for testing).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A minimal security.properties file like the following should be enough for the test task to succeed:

##############################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
###############################################################################
####
# OFBiz Security Settings
####

# -- Security key used to encrypt and decrypt the autogenerated password in forgot password functionality.
#    Read Passwords and JWT (JSON Web Tokens) usage documentation to choose the way you want to store this key
#    The key must be 512 bits (ie 64 chars) as we use HMAC512 to create the token, cf. OFBIZ-12724
#    Run './gradlew generateSecretKeys' to generate a cryptographically secure random key.
login.secret_key_string=ZdxxqjX87e6CR9Rf5pAzA+GTX4DP2T0H3bimGaDGA59OytU/tVQ7AC7N9PUA9e17

# -- The secret key for the JWT token signature.
#    Read Passwords and JWT (JSON Web Tokens) usage documentation to choose the way you want to store this key
#    The key must be 512 bits (ie 64 chars) as we use HMAC512 to create the token, cf. OFBIZ-12724
#    Run './gradlew generateSecretKeys' to generate a cryptographically secure random key.
security.token.key=TzWl/rNavz7VYMc6sxDpf8Yon6NnkbUfrL4JrscJBdTEx3vtwQQ7Mt0wk/TtZXUn

#-- To accept the execution on some groovy script who match the deniedScriptletsTokens regExp, put their hash here.
#-- like allowedScriptletHashes={SHA}59f8ab616b3878ddf825ea50c13ce603a3a6c5a9,{SHA}59f5ab516b3878ddf825ea50c13ce603a3a6c5a9
allowedScriptletHashes= {SHA}4e025676cfa6df142e3457099271ecdcd1c1f5f9,{SHA}d8451d7509ae73421974f47752b6e9eef7503041,{SHA}edf12cf95597d52eacc14020a85a8df2abb34ab7

#-- RegExp to secure groovy script execution. If the regExp match a script, it would be disabled and OFBiz run nothing.
#-- In this case, you will have on log the original script with it hash. The hash can be added on allowedScriptletHashes
#-- properties to accept it on the next execution.
deniedScriptletsTokens=java\\s*\.|import\\s|embed[^\\w]|process[^\\w]|class[^\\w]|require[^\\w]\
|\.\\s*.exec.*[\(|\\s]|\.\\s*calc.*[\(|\\s]|\.\\s*.eval.*[\(|\\s]|Eval\\s*\.|\\s+File\
|System\\s*\.|\.\\s*codehaus|\.\\s*groovy[^:]|\.\\s*runtime\|groovyx\\s*\.

#-- If you want to deactivate the security control on each groovy script set to false.
# Warn ensure to be sure on what you do because this can open the door for code injection
useDeniedScriptletsTokens=true

@jacopoc
Copy link
Copy Markdown
Contributor

jacopoc commented Apr 2, 2026

@ribafish Thank you again for your work.

I really like the goal of properly supporting cross-machine build cache (and making better use of the Develocity cache, see #1033).
However, before enabling the remote cache, I should mention that while our CI GitHub Action for Gradle is currently configured to run the tasks

./gradlew check javadoc,

we are not able to run instead

./gradlew pullAllPluginsSource check javadoc loadAll testIntegration,

which would be our preferred setup. Frankly speaking, I’m not sure what issues are preventing us from running these tasks... it still needs to be investigated.

Hence, ideally, we should make the loadAll task cache-compliant, as it currently depends on generateSecretKeys. One possible approach could be to copy hardcoded "demo" keys into framework/security/config/security.properties before executing the loadAll task: always using the same keys should not compromise cache hits.

Any advice would be more than welcome!

@jacopoc jacopoc merged commit 73d0d37 into apache:trunk Apr 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants