A Gradle Plugin to determine which modules were affected by a set of files in a commit. One use case for this plugin is for developers who would like to only run tests in modules which have changed in a given commit.
The AffectedModuleDetector will look at the last commit and determine which files have changed, it will then build a dependency graph of all the modules in the project. The detector exposes a set of APIs which can be used to determine whether a module was considered affected.
The module detector assumes that it is being applied to a project stored in git and a git client is present on the system. It will query the last commit on the current branch to determine the list of files changed.
The tracker will evaluate the project and find all modules and their dependencies for all configurations.
The detector allows for three options for affected modules:
- Changed Projects: These are projects which had files changed within them – enabled with
-Paffected_module_detector.changedProjects) - Dependent Projects: These are projects which are dependent on projects which had changes within them – enabled with
-Paffected_module_detector.dependentProjects) - All Affected Projects: This is the union of Changed Projects and Dependent Projects (this is the default configuration)
These options can be useful depending on how many tests your project has and where in the integration cycle you would like to run them. For example, Changed Projects may be a good options when initially sending a Pull Requests, and All Affected Projects may be useful to use when a developer merges their pull request.
The detector exposes APIs which will be helpful for your plugin to use. In particular, it exposes:
- AffectedModuleDetector.configureTaskGuard - This will apply an
onlyIfguard on your task and can be called either during configuration or execution - AffectedModuleDetector.isProjectAffected - This will return a boolean if the project has been affected. It can only be called after the project has been configured.
In the example below, we're showing a hypothetical project graph and what projects would be considered affected if the All Affected Projects option was used and a change was made in the :networking module.
// settings.gradle(.kts)
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
// root build.gradle(.kts)
plugins {
id("com.dropbox.affectedmoduledetector") version "<latest-version>"
}Note that the plugin is currently published to Maven Central, so you need to add it to the repositories list in the pluginsManagement block.
Alternatively, it can be consumed via manual buildscript dependency + plugin application.
Apply the project to the root build.gradle:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath "com.dropbox.affectedmoduledetector:affectedmoduledetector:<LATEST_VERSION>"
}
}
//rootproject
apply plugin: "com.dropbox.affectedmoduledetector"If you want to develop a plugin using the APIs, add this to your buildSrc's dependencies list:
implementation("com.dropbox.affectedmoduledetector:affectedmoduledetector:<LATEST_VERSION>")
You can specify the configuration block for the detector in the root project:
affectedModuleDetector {
baseDir = "${project.rootDir}"
pathsAffectingAllModules = [
"buildSrc/"
]
logFilename = "output.log"
logFolder = "${project.rootDir}/output"
compareFrom = "PreviousCommit" //default is PreviousCommit
excludedModules = [
"sample-util", ":(app|library):.+"
]
ignoredFiles = [
".*\\.md", ".*\\.txt", ".*README"
]
includeUncommitted = true
top = "HEAD"
customTasks = [
new AffectedModuleConfiguration.CustomTask(
"runDetektByImpact",
"detekt",
"Run static analysis tool without auto-correction by Impact analysis"
)
]
}baseDir: The root directory for all of thepathsAffectingAllModules. Used to validate the paths exist.pathsAffectingAllModules: Paths to files or folders which if changed will trigger all modules to be considered affectedlogFilename: A filename for the output detector to uselogFolder: A folder to output the log file inspecifiedBranch: A branch to specify changes against. Must be used in combination with configurationcompareFrom = "SpecifiedBranchCommit"ignoredFiles: A set of files that will be filtered out of the list of changed files retrieved by git.compareFrom: A commit to compare the branch changes against. Can be either:- PreviousCommit: compare against the previous commit
- ForkCommit: compare against the commit the branch was forked from
- SpecifiedBranchCommit: compare against the last commit of
$specifiedBranchusinggit rev-parseapproach. - SpecifiedBranchCommitMergeBase: compare against the nearest ancestors with
$specifiedBranchusinggit merge baseapproach.
Note: specify the branch to compare changes against using the specifiedBranch configuration before the compareFrom configuration
excludedModules: A list of modules that will be excluded from the build process, can be the name of a module, or a regex against the module gradle pathincludeUncommitted: If uncommitted files should be considered affectedtop: The top of the git log to use. Must be used in combination with configurationincludeUncommitted = falsecustomTasks: set of CustomTask
By default, the Detector will look for assembleAndroidDebugTest, connectedAndroidDebugTest, and testDebug. Modules can specify a configuration block to specify which variant tests to run:
affectedTestConfiguration {
assembleAndroidTestTask = "assembleAndroidReleaseTest"
runAndroidTestTask = "connectedAndroidReleaseTest"
jvmTestTask = "testRelease"
}The plugin will create a few top level tasks that will assemble or run tests for only affected modules:
./gradlew runAffectedUnitTests- runs jvm tests./gradlew runAffectedAndroidTests- runs connected tests./gradlew assembleAffectedAndroidTests- assembles but does not run on device tests, useful when working with device labs
- SpecifiedBranchCommit using
git rev-parsecommand for getting sha. - SpecifiedBranchCommitMergeBase using
git merge basecommand for getting sha.
What does it mean? When we run any AMD command we compare the current branch with the specified parent branch. Consider an example when, during the development of our feature, another developer merged his changes (9 files) into our common remote parent branch - "origin/dev".
Please, look at picture:
Suppose we have changed 6 files in our "feature" branch.
- Behaviour of SpecifiedBranchCommit: AMD will show the result that 15 files were affected. Because our branch is not updated (pull) and AMD will see our 6 files and 9 files that were merged by another developer.
- Behaviour of SpecifiedBranchCommitMergeBase: AMD will show the result that 6 files were affected. And this is the correct behavior.
Hence, depending on your CI settings you have to configure AMD appropriately.
Running the plugin generated tasks is quite simple. By default, if affected_module_detector.enable is not set,
the generated tasks will run on all the modules. However, the plugin offers three different modes of operation so that it
only executes the given task on a subset of projects.
To run all the projects affected by a change, run one of the tasks while enabling the module detector.
./gradlew runAffectedUnitTests -Paffected_module_detector.enable
To run all the projects that changed, run one of the tasks (while enabling the module detector) and with -Paffected_module_detector.changedProjects
./gradlew runAffectedUnitTests -Paffected_module_detector.enable -Paffected_module_detector.changedProjects
To run all the dependent projects of projects that changed, run one of the tasks (while enabling the module detector) and with -Paffected_module_detector.dependentProjects
./gradlew runAffectedUnitTests -Paffected_module_detector.enable -Paffected_module_detector.dependentProjects
To run this on the sample app:
- Publish the plugin to local maven:
./gradlew :affectedmoduledetector:publishToMavenLocal
- Try running the following commands:
cd sample
./gradlew runAffectedUnitTests -Paffected_module_detector.enable
You should see zero tests run. Make a change within one of the modules and commit it. Rerunning the command should execute tests in that module and its dependent modules.
If you want to add a custom gradle command to execute with impact analysis
you must declare AffectedModuleConfiguration.CustomTask
which is implementing the AffectedModuleTaskType interface in the build.gradle configuration of your project:
// ...
affectedModuleDetector {
// ...
customTasks = [
new AffectedModuleConfiguration.CustomTask(
"runDetektByImpact",
"detekt",
"Run static analysis tool without auto-correction by Impact analysis"
)
]
// ...
}NOTE: Please, test all your custom commands.
If your custom task doesn't work correctly after testing, it might be that your task is quite complex
and to work correctly it must use more gradle api's.
Hence, you must create buildSrc module and write a custom plugin manually like AffectedModuleDetectorPlugin
Special thanks to the AndroidX team for originally developing this project at https://android.googlesource.com/platform/frameworks/support/+/androidx-main/buildSrc/src/main/kotlin/androidx/build/dependencyTracker
Copyright (c) 2021 Dropbox, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent