WinTPS v2 ideas

[RadekCap]

Current status

• V1
• Installation, uninstallation, default, all - demonstrate the format

We are covering several main areas

• install & uninstall
• checking of installed files on the filesystem
• checking of proper system variables setup
• checking of proper registry values

Why
I would like to use a more robust testing approach than bash. It's possible to write it in nUnit/junit or something similar. Aside from robustness, we'll get significantly higher testing coverage due to the creation of testing installation variants.

• current testing isn't robust
• a new addition needs a lot of boilerplate code
• significantly easier maintenance

Draft of implementation in nunit with TestCaseSource

public class WinTPSTests
{
   // SetUp
   // installation for every combination


   [TestCaseSource(typeof(VendorConfiguration))]
   public void CheckRegistryInstallation(VendorConfiguration configuration)
   {
       // foreach RegistyPaths in configuration && current JavaVersion
       Assert.AreEqual(configuration.Registry, actualRegistryValue);
   }


   [TestCaseSource(typeof(VendorConfiguration))]
   public void CheckFilesInstallation(VendorConfiguration configuration)
   {
       // foreach Files in configuration && current JavaVersion
       Assert.AreTrue(File.Exists(configuration.File));
   }


   // TearDown for every combination
}


public class VendorConfiguration : IEnumerable
{
   public IEnumerator GetEnumerator()
   {
       yield return new VendorSpecificConfiguration[]
       {
         Feature = "FeatureEnvironment",
         new List<Path>() { "ABC", "XYZ"},
         new List<String>() {"RegistryPath1", "RegistryPath2"}
         JavaVersion = 20
       };


       yield return new VendorSpecificConfiguration[]
       {
         Feature = "FeatureJavaHome",
         new List<Path>() { "ABC", "XYZ"},
         new List<String>() {"RegistryPath1", "RegistryPath2"} 
         JavaVersion = 20
       };


       yield return new VendorSpecificConfiguration[]
       {
         Feature = "FeatureJarFileRunWith",
         new List<Path>() { "ABC", "XYZ"},
         new List<String>() {"RegistryPath1", "RegistryPath2"} 
         JavaVersion = 20
       };
   }
}


public class VendorSpecificConfiguration
{
 public IEnumerator GetEnumerator()
 {
     string Feature;
     List<Path> Files;
     List<String> RegistyPaths;
     int JavaVersion;
 }
}

What’s missing?
https://github.com/adoptium/installer/blob/master/wix/README.md#features-available

• Reinstall?
• Per user installation?
• Upgrade?
• Do we need some runnable application?

[judovana]

Dont forget that to run junit, you need java. Do you really wan to test jdk isntall on system polluted with java?
On contrary, we already have jdk in opt to connect jenkins agents.

To get away from languages needein massive runtime (junit, xunit...) I was trying to find some other unittesting, sumamry can be found there: adoptium/temurin-build#3741

That is leading us to perl/python. I had choosen python over bash/java once (jsf) and I would never ever repeat that mistake. But pyUnit may be not-invetigated path here (in acursed JSF, I went by to muchnon-standart path)

So gennerally spoken, I do not see reasoning to do so. Current stuff seems to be working well and is extendable (If you accept the boiler plate).
If you insists it is good spending of time for future, then my choice would be junit5 eith 3rd party java. But be aware, that setup of 3rd party java (aka our opt) may be tricky on eg adoptium machines, as the test would need to unpack the 3rd party jdk beforehead.

Thanx a lot for writing it down. It really gave sense, only I'm not sure if it is worthy.

[judovana]

The upgrade is tricky part. In our old infra rpms-old were there to test upgrade to future one. But it is not stable. eg rerun do not work for obvious reasons. To download previous binaries is extremly hard. Zdenek tried that fro cjc, and it is extremly fagile, and vendor/os/arch specific

[judovana]

What is runnable application?

[RadekCap]

What is runnable application?

executable

[judovana]

What is runnable application?

executable

Executable is any file with +x :) so runnable application means jsut that test wrapper? I was imagining some top level real world app run as part of msi tests which would run after install to verify it works.