gradle discrepancies

[zeners]

i found the following needed actions, please correct me, if i'm wrong

• SerenityTask need removing clearHistory and history as mentioned by @petrandreev
• build.gradle should use tasks.register
• .all should be replaces with configureEach in build.gradle
• test-Task need useJUnitPlatform() to run the tests
• what's happen to remove newly introduced and unnecessary generateOutcomes property #30 ?
• JUnit5-test-classes should be package-private
• contains-assertions should be written as assertThat(...).contains(...); (currently assertThat(....contains(...));
• create README-file with detailed usage descriptions

i tested the actions with a simple test-project and with active --build-cache --configuration-cache
getting the following result:

• clean test without change: create empty result

   > Task :test FROM-CACHE
   > Task :aggregate
  

• test after change: create index with no results pages listed (after clean test)

  > Task :test
  > Task :aggregate UP-TO-DATE
  

this looks like test create output, that aggregate uses without the knowledge of gradle-cache

gradle writes output into build, serenity into target/site/serenity, clean only remove serenity inside target/site

[petrandreev]

hi @zeners, regarding your Q about generateOutcomes, not needed before- it's been changed here:3424830
It used to be always set, now one needs to set it to "true" on PluginExtension explicitely.
Thanks for summing up

[zeners]

i can't test the deployment, so i hope that i didn't broke something

[wakaleo]

Just published the new version, feel free to test

[zeners]

@wakaleo thank you

the cache doesn't work as expected. I'm afraid that we have to use the complete outputDirectory from test.

• gradle clean test -x aggregate --build-cache --configuration-cache should create the test-result without report:

  ...
  > Task :test 
  
  

• gradle clean test --build-cache --configuration-cache should build the reports, based on cached test output

  > Task :test FROM-CACHE
  > Task :aggregate
  Generating Serenity Reports
  

but the generated report is empty, because the test-results are missing
will try to write a test addressing this

[tnielens]

Hi here,
Sorry to jump in. Was #19 reverted? From what I tested only #30 to fix the missing test results needed to be merged for it to work properly. Internally we're using a published version including #30 and we observe no issues with the plugin.

[zeners]

c7e0310 was an undo of #19, but 3424830 makes a redo, but with differences

currently i'm afraid that's not possible to make a consistent result, if you test without aggegate (test -x aggregate) and after it clean test will run aggregate without test-results, not restored from cache.

i have a plan in mind, but need some time to implement and test

[tnielens]

Thanks for clarifying.

As for aggregate, we don't understand well its behavior as it doesn't play well with the project-scoped expectation of gradle. That task seems to collect outputs from all subprojects in a multimodule project although the task exists in each project. Also the static/global state of serenity prevents use of gradle parallelism and running parallel tasks in a multi-module project leads to concurrency issues in serenity tasks and outputs.

Aside of these issues, #19 was a welcome improvement 👍.

[zeners]

That task seems to collect outputs from all subprojects in a multimodule project although the task exists in each project.

assuming the gradle-plugin should work like the maven-plugin, as outlined in serenity-bdd/serenity-maven-plugin#22 (comment) you have to set a common output-directory in all subprojects, to get one report for all subproject tests, correct?

[petrandreev]

Solved reports generation issue in a muli-module project with v 4.1.10 (happens on RH UBI, e.g. linux, not on Mac though...) on my side.
IMHO the issue has nothing to do with gradle cache(s) -

1. test task (JSONTestOutcomeReporter actually) generates reports into serenity.conf's outputDirectory (build/site/serenity)
2. on aggregate task, Gradle decides that the whole directory, configured by serenity plugin extension's outputDirectory is stale and deletes it!:

> Task :serenity:aggregate
Deleting stale output file: ......../build/site/serenity

3. aggregator has nothing to work with since no JSON outcomes - no reports in the site HTML!

The both configured outputDirectory s must point to the same location- otherwise JSON reports are not picked up for further processing, not sure why....

Solution:
declare either

• input of aggregate task to be build/site/serenity directory or
• output of the test task to be build/site/serenity directory

Gradle then takes into account that there could be some other output, besides aggregate's one, and doesn't delete the entire build/site/serenity directory- aggregate picks JSONs up and produces HTML report.

P.S. thanks to @tnielens - yes i forgot to mention that this setup is not recommended by gradle - but I currently I don't see any other way to generate reports- it doesn't work otherwise...

[tnielens]

Gradle incremental build and task cache (to be distinguished from the configuration cache) don't work reliably if multiple tasks output results in the same directory. You'll see warnings in the log about this.

[zeners]

• test-results are not only the json-files, but also can (at least) include downloadable and (screenshot)pictures
• these outputs should be known to gradle and connected via input/output between the task
• the connection of output and input creates a direct dependency between these 2 tasks (test and aggregate)
• i like to see test and aggregate be separate to call (now you have to exclude aggregate, when running test, but that's ok)

we can copy the test-result-files to a aggregate-working directory to split the test-output from the aggregate output - but this would be a breaking change

[wakaleo]

I'm reverting all of these changes as they have caused a number of regressions.

[zeners]

wich ones?

[wakaleo]

serenity-bdd/serenity-core#3468