The org.ajoberstar.stutter plugin ( https://github.com/ajoberstar/gradle-stutter ) is a tool for compatibility testing.
But I cannot make it run any compatibility tests.
It did not work with Gradle 8.9, it did not work with Gradle 8.6 (8.6 is the most recent version mentioned in the plugin docs).
- I create a project with
~/Downloads/tmpgradle/8.6/gradlew init:
type of project to generate: 4: Gradle plugin
implementation language: 1: Groovy
build script DSL: 2: Groovy
Project name (default: greetingPlugin): greetingPlugin
Generate build using new APIs and behavior: no
- I insert a
prinln()into the plugin’sapply()function, and aprintln()into the (generated) functional test. I copy thefunctionalTestdirectory to thecompatTestdirectory and rename the class.
I modifybuild.gradleto include thestutterplugin:
plugins {
id 'java-gradle-plugin'
id 'groovy'
id 'org.ajoberstar.stutter' version '1.0.0'
}
repositories {
mavenCentral()
}
dependencies {
testImplementation libs.spock.core
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
gradlePlugin {
plugins {
greeting {
id = 'org.example.greeting'
implementationClass = 'org.example.GreetingPluginPlugin'
}
}
}
sourceSets {
functionalTest {
}
}
configurations.functionalTestImplementation.extendsFrom(configurations.testImplementation)
configurations.functionalTestRuntimeOnly.extendsFrom(configurations.testRuntimeOnly)
// Add a task to run the functional tests
tasks.register('functionalTest', Test) {
testClassesDirs = sourceSets.functionalTest.output.classesDirs
classpath = sourceSets.functionalTest.runtimeClasspath
useJUnitPlatform()
}
gradlePlugin.testSourceSets.add(sourceSets.functionalTest)
tasks.named('check') {
// Run the functional tests as part of `check`
dependsOn(tasks.functionalTest)
}
tasks.named('test') {
// Use JUnit Jupiter for unit tests.
useJUnitPlatform()
}
tasks.withType(Test) {
testLogging.showStandardStreams = true
}
dependencies {
// Use the awesome Spock testing and specification framework
compatTestImplementation libs.spock.core
compatTestRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
stutter {
// if true, only match min/max within that otherwise matches your compatibility specs in each Gradle major version
// sparse = false // defaults to true
matrices {
java13 {
javaToolchain {
languageVersion = JavaLanguageVersion.of(13)
}
gradleVersions {
compatible '6.2.1'
compatible '8.6'
}
}
// // a matrix of tests running against Java 8
// java8 {
// javaToolchain {
// languageVersion = JavaLanguageVersion.of(8)
// }
// gradleVersions {
// compatibleRange '3.0', '4.0' // include 3.0 <= version < 4.0
// compatibleRange '4.2' // include 4.2 <= version
// compatible '2.14', '1.2' // include 2.14 and 1.12 specifically
// incompatible '3.3' // exclude 3.3 even if included above
// }
// }
//
//
// // name can be anything
// jdkNine {
// javaToolchain {
// languageVersion = JavaLanguageVersion.of(9)
// }
// gradleVersions {
// compatibleRange '4.0' // include 4.0 <= version
// }
// }
}
// You do have to specify compatible Gradle versions for all Java versions you run Gradle with
// If you have a lot of tests, or otherwise just don't want to test every Gradle version that you say is compatible,
// use sparse = true. This will greatly limit the number of versions you test against, but should do the job of
// verifying compatibility.
// e.g. compatible '2.14' and compatibleRange '3.0'
// matches '2.14', '3.0', '3.5.1', '4.0', '4.7' (presuming 4.7 is the latest available 4.x)
}
//does not help:
//configurations.compatTestImplementation.extendsFrom(configurations.testImplementation)
//configurations.compatTestRuntimeOnly.extendsFrom(configurations.testRuntimeOnly)
- I run
./gradlew stutterWriteLocksand./gradlew clean compatTest --warning-mode all.
The project directory contains:
~/stackoverflow/greetingPlugin$ tree
.
├── gradle
│ ├── libs.versions.toml
│ └── wrapper
│ ├── gradle-wrapper.jar
│ └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
├── plugin
│ ├── build
│ │ ├── classes
│ │ │ └── groovy
│ │ │ ├── compatTest
│ │ │ │ └── org
│ │ │ │ └── example
│ │ │ │ └── GreetingPluginPluginCompatTest.class
│ │ │ └── main
│ │ │ └── org
│ │ │ └── example
│ │ │ ├── GreetingPluginPlugin$_apply_closure1$_closure2.class
│ │ │ ├── GreetingPluginPlugin$_apply_closure1.class
│ │ │ └── GreetingPluginPlugin.class
│ │ ├── generated
│ │ │ └── sources
│ │ │ └── annotationProcessor
│ │ │ └── groovy
│ │ │ ├── compatTest
│ │ │ └── main
│ │ ├── pluginDescriptors
│ │ │ └── org.example.greeting.properties
│ │ ├── pluginUnderTestMetadata
│ │ │ └── plugin-under-test-metadata.properties
│ │ ├── reports
│ │ │ └── tests
│ │ │ ├── compatTestJava13Gradle6.2.1
│ │ │ │ ├── css
│ │ │ │ │ ├── base-style.css
│ │ │ │ │ └── style.css
│ │ │ │ ├── index.html
│ │ │ │ └── js
│ │ │ │ └── report.js
│ │ │ └── compatTestJava13Gradle8.6
│ │ │ ├── css
│ │ │ │ ├── base-style.css
│ │ │ │ └── style.css
│ │ │ ├── index.html
│ │ │ └── js
│ │ │ └── report.js
│ │ ├── resources
│ │ │ └── main
│ │ │ └── META-INF
│ │ │ └── gradle-plugins
│ │ │ └── org.example.greeting.properties
│ │ ├── test-results
│ │ │ ├── compatTestJava13Gradle6.2.1
│ │ │ │ └── binary
│ │ │ │ ├── output.bin
│ │ │ │ ├── output.bin.idx
│ │ │ │ └── results.bin
│ │ │ └── compatTestJava13Gradle8.6
│ │ │ └── binary
│ │ │ ├── output.bin
│ │ │ ├── output.bin.idx
│ │ │ └── results.bin
│ │ └── tmp
│ │ ├── compatTestJava13Gradle6.2.1
│ │ │ ├── jar_extract_11168663823107129072_tmp
│ │ │ ├── jar_extract_1507282802988250170_tmp
│ │ │ └── jar_extract_7058108455941420678_tmp
│ │ ├── compatTestJava13Gradle8.6
│ │ │ ├── jar_extract_1158481335806899552_tmp
│ │ │ ├── jar_extract_5215173060414151702_tmp
│ │ │ └── jar_extract_7036217941087640325_tmp
│ │ ├── compileCompatTestGroovy
│ │ │ └── groovy-java-stubs
│ │ └── compileGroovy
│ │ └── groovy-java-stubs
│ ├── build.gradle
│ ├── src
│ │ ├── compatTest
│ │ │ └── groovy
│ │ │ └── org
│ │ │ └── example
│ │ │ └── GreetingPluginPluginCompatTest.groovy
│ │ ├── functionalTest
│ │ │ └── groovy
│ │ │ └── org
│ │ │ └── example
│ │ │ └── GreetingPluginPluginFunctionalTest.groovy
│ │ ├── main
│ │ │ ├── groovy
│ │ │ │ └── org
│ │ │ │ └── example
│ │ │ │ └── GreetingPluginPlugin.groovy
│ │ │ └── resources
│ │ └── test
│ │ ├── groovy
│ │ │ └── org
│ │ │ └── example
│ │ │ └── GreetingPluginPluginTest.groovy
│ │ └── resources
│ └── stutter.lockfile
└── settings.gradle
63 directories, 39 files
The gradle output is:
$ ./gradlew clean compatTest --warning-mode all
Path for java installation '/usr/lib/jvm/openjdk-17' (Common Linux Locations) does not contain a java executable
Path for java installation '/usr/lib/jvm/openjdk-19' (Common Linux Locations) does not contain a java executable
Path for java installation '/usr/lib/jvm/openjdk-21' (Common Linux Locations) does not contain a java executable
> Task :plugin:clean
> Task :plugin:compileJava NO-SOURCE
> Task :plugin:compileGroovy
> Task :plugin:pluginDescriptors
> Task :plugin:processResources
> Task :plugin:classes
> Task :plugin:compileCompatTestJava NO-SOURCE
> Task :plugin:compileCompatTestGroovy
> Task :plugin:processCompatTestResources NO-SOURCE
> Task :plugin:compatTestClasses
> Task :plugin:pluginUnderTestMetadata
> Task :plugin:compatTestJava13Gradle6.2.1
The automatic loading of test framework implementation dependencies has been deprecated. This is scheduled to be removed in Gradle 9.0. Declare the desired test framework directly on the test suite or explicitly declare the test framework implementation dependencies on the test's runtime classpath. Consult the upgrading guide for further information: https://docs.gradle.org/8.6/userguide/upgrading_version_8.html#test_framework_implementation_dependencies
No test executed. This behavior has been deprecated. This will fail with an error in Gradle 9.0. There are test sources present but no test was executed. Please check your test configuration. Consult the upgrading guide for further information: https://docs.gradle.org/8.6/userguide/upgrading_version_8.html#test_task_fail_on_no_test_executed
> Task :plugin:compatTestJava13Gradle8.6
> Task :plugin:compatTestJava13
> Task :plugin:compatTest
BUILD SUCCESSFUL in 2s
8 actionable tasks: 8 executed
You see that Gradle says that no tests were executed. This is true, I would see the messages from the test and the plugin, I enabled them via tasks.withType(Test) { testLogging.showStandardStreams = true } and I see the messages when I run integrationTest.
There are messages about missing java executables, but AFAIK Gradle would dounload them if they are necessary.
$ ls -lF /usr/lib/jvm/openjdk-17
total 0
lrwxrwxrwx 1 root root 11 окт 19 2023 src.zip -> lib/src.zip
(All three poin to the same lib/src.zip)
What am I doing wrong?