Coder Social home page Coder Social logo

jenkinsci / ecutest-plugin Goto Github PK

View Code? Open in Web Editor NEW
23.0 101.0 16.0 6.7 MB

This plugin integrates Jenkins with ecu.test and generates reports on automated test execution.

Home Page: https://plugins.jenkins.io/ecutest

License: Other

ANTLR 0.08% Java 93.09% HTML 3.53% CSS 0.13% Groovy 2.65% JavaScript 0.17% Python 0.35%
jenkins jenkins-plugin ecu-test testautomation embedded-systems tracetronic hacktoberfest deprecated

ecutest-plugin's Introduction

tracetronic ecu.test plugin

Build Status Third Party License Check Jenkins Plugin Jenkins Release Plugin Installs License

⚠️ End of Life ⚠️
This plugin is deprecated and the support ends at 2024-12-31.
You can migrate your test execution to test.guide or still use Jenkins with the ecu.test Execution Plugin.
After the compatibility update for ecu.test 2024.4 at the end of 2024, there will never be an update for this plugin again - not for compatibility updates and also not for security fixes.
The plugin will be deleted from the Update-Center at 2025-12-31.



This plugin integrates Jenkins with ecu.test and generates reports on automated test execution.

ecu.test Logo

ecu.test allows for a consistent and effective automation of the entire test environment. It supports and connects so many different software tools, that it easily fits in numerous existing test and validation processes. The test automation tool can be used in all phases of development up to the finished vehicle itself. It allows for a high reusability of the test cases in the various test areas MiL, SiL, HiL, PiL and ViL (vehicle), through generic test description and tool-access abstraction.

ecu.test Jenkins Plugin project is part of the Automotive DevOps Platform by tracetronic. With the Automotive DevOps Platform, we go from the big picture to the details and unite all phases of vehicle software testing – from planning the test scopes to summarizing the test results. At the same time, continuous monitoring across all test phases always provides an overview of all activities – even with several thousand test executions per day and in different test environments.

Please consider other open-source automation solutions by tracetronic, especially Jenkins Library and CX Templates.

Table of Contents

Features

  • Provides an easy integration and control of ecu.test and the Tool-Server with Jenkins
  • Enables the execution of ecu.test packages and projects with their respective configurations

Moreover publishing test results is included in form of:

  • Linking the TRF reports
  • Generating various report formats
  • Displaying test result trend graphs
  • Uploading ATX test reports to test.guide
  • Parsing the ecu.test logs for warnings and errors

Advanced workflows integrated for:

Configuration

ecu.test configuration

ecu.test installations are administrated in the Jenkins system configuration or starting with Jenkins 2.0 in the global tool configuration at section "ecu.test". An installation entry is specified by an arbitrary name and the path to the installation directory. The execution on a Jenkins agent requires the adaption of the ecu.test installation directory on the agent configuration page.

ecu.test ecu.test_AGENT

test.guide configuration

test.guide is also configured in the Jenkins system configuration at section "test.guide". A new entry requires both an arbitrary name and the selection of a previously configured ecu.test installation. Further settings like the server or ATX specific configuration can be found in the advanced block.

test.guide

Job configuration

The plugin provides several new build steps:

Build Steps

Note:

  • All build steps have an "Advanced" button through which more options can be accessed
  • All text fields can be parameterized and are resolved at build execution
    • Syntax: $parameter or ${parameter}

The post-build actions are extended by these test report related post-build steps:

Post-Build Steps

Job execution

  • After job configuration is done a new build can be triggered by clicking on the "Build now" button
  • The test results are published both on the build page and the job page based on the latest build after the step [TT] Publish UNIT Reports has been executed.
  • The console log contains detailed information per build step execution.

Demo Job Demo Build Demo Log

Environment variables

  • This plugin also exposes the tool and test related information of the current build as environment variables.
  • These variables are composed of ${TT_<TOOL | TEST>__n} where n is the index of the related build steps in the job configuration, counting from 0.
  • The summary of all generated environment variables is also shown on the build page.
Tool related variables
Environment variable Description
TT_TOOL_NAME Name of the selected ecu.test installation
TT_TOOL_INSTALL Full path to related ecu.test executable
TT_TOOL_VERSION Program version of selected ecu.test installation
TT_TOOL_WORKSPACE ecu.test workspace directory
TT_TOOL_SETTINGS ecu.test settings directory
TT_TOOL_DEBUG ecu.test debug mode
TT_TOOL_TIMEOUT Maximum execution time
TT_TOOL_LAST_TBC Last loaded TBC file path
TT_TOOL_LAST_TCF Last loaded TCF file path
Test related variables
Environment variable Description
TT_TEST_NAME Name of the executed test
TT_TEST_FILE Full path to related test file
TT_TEST_TYPE Type of related test (PACKAGE, PROJECT)
TT_TEST_DESCRIPTION Test description
TT_TEST_TBC Full path to test bench configuration file
TT_TEST_TCF Full path to test configuration file
TT_TEST_TIMEOUT Maximum execution time
TT_TEST_RESULT Test result (SUCCESS, INCONCLUSIVE, FAILED, ERROR, NONE)
TT_TEST_REPORT Full path to test report directory
TT_TEST_RETVAL_<VARIABLE NAME> Test related package output parameter (only for package execution)

Downstream report generation

The concept of downstream report generation allows to execute the report generators like ATX, UNIT, HTML etc. in a downstream-based project. Therefore the report artifacts like TRF reports etc. must be archived after the test execution stage and will then be copied into a separate workspace of the triggered downstream project.

This downstream project acts like a central reporting service which waits for inputs from other test executions. It will be an ease to scale this service with help of Jenkins parallelization and distribution features. The big advantage of this approach is to save time for the report generation and uploading process on expensive testing resources like HiL test benches. These resources can be directly used for the next test.

Downstream Report Generation

Example configurations

Preconditions

To prevent duplicate reports the build workspace should be cleaned before running the tests and before copying artifacts inside of the downstream project. This can be achieved by using clean checkouts from repository or using the Workspace Cleanup Plugin when preparing the build environment.

To transfer the report artifacts from one node to another the Copy Artifact Plugin is required for freestyle projects. When using pipelines also stash/unstash commands can be used.

Freestyle configuration

Executing tests and archiving the artifacts (upstream)

Upstream Build Steps Upstream Post-Build Steps

Copying artifacts and generating ATX report (downstream)

Downstream Build Steps Downstream Post-Build Steps

Pipeline configuration

Upstream pipeline definition
node('windows') {
    deleteDir()

    startET 'ecu.test'
    testPackage testFile: 'sample.pkg', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf']
    stopET 'ecu.test'

    archiveArtifacts 'TestReports/**/*.trf'
    build job: 'downstream', wait: false
}
Downstream pipeline definition
node('windows') {
    deleteDir()

    copyArtifacts filter: 'TestReports/**/*.trf', projectName: 'upstream', selector: lastSuccessful()

    // Class notation is required for publisher steps inside downstream wrapper!
    // Available: ATXPublisher, ETLogPublisher, JUnitPublisher, ReportGeneratorPublisher, TMSPublisher, TRFPublisher
    downstreamPublisher workspace: '', reportDir: 'TestReports', publishers: [
        [$class: 'ATXPublisher', atxName: 'test.guide'],
        [$class: 'JUnitPublisher', toolName: 'ecu.test']]
}

Downstream trace analysis

In addition to downstream report generation the plugin allows to run trace analyses in a downstream-based project. Therefore the test cases are executed without trace analysis at first. Afterwards all trace analysis artifacts like analysis job files, traces, trace step templates and TRF reports etc. must be archived and will then be copied into a separate workspace of the triggered downstream project.

This downstream project acts like a central trace analysis service which waits for inputs from other test executions and is designed to be highly scalable. The huge advantage of better ressource efficiency is the same as for downstream report generation and can also be used in combination.

Downstream Trace Analysis

Example configurations

Preconditions

Same as in downstream report generation.

Freestyle configuration

Executing test cases without trace analysis and archiving the artifacts (upstream)

Upstream Build Steps Upstream Post-Build Steps

Copying artifacts and running the trace analysis (downstream)

Downstream Build Steps Downstream Post-Build Steps

Pipeline configuration

Upstream pipeline definition
node('windows') {
    deleteDir()

    startET 'ecu.test'
    testPackage testFile: 'sample.pkg', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf'], packageConfig: [runTraceAnalysis: false]
    stopET 'ecu.test'

    archiveArtifacts 'TestReports/**/*.trf, TestReports/**/*.ajob, Packages/**, Traces/**, TraceStepTemplates/**'
    build job: 'downstream', wait: false
}
Downstream pipeline definition
node('windows') {
    deleteDir()

    copyArtifacts filter: 'TestReports/**/*.trf, TestReports/**/*.ajob, Packages/**, Traces/**, TraceStepTemplates/**', projectName: 'upstream', selector: lastSuccessful()

    // Class notation is required for publisher steps inside downstream wrapper!
    downstreamPublisher workspace: '', reportDir: 'TestReports', publishers: [
        [$class: 'TraceAnalysisPublisher', toolName: 'ecu.test', mergeReports: true]]
}

Extensions

Job DSL

The Job DSL Plugin allows the programmatic creation of projects using a DSL. This plugin provides some specific DSL extensions.

Furthermore, if there is a method which is currently not supported then Job DSL also provides an auto-generator.

Controlling tools
steps {
    // Start ecu.test
    startET(String installation) {
        workspaceDir(String directory)
        settingsDir(String directory)
        timeout(int | String timeout = 120)
        debugMode(boolean debugMode = false)
        keepInstance(boolean keepInstance = false)
    }
    // Stop ecu.test
    stopET(String installation) {
        timeout(int | String timeout = 30)
    }
    // Start Tool-Server
    startTS(String installation) {
        toolLibsIni(String toolLibsPath)
        tcpPort(int | String port = 5017)
        timeout(int | String timeout = 120)
        keepInstance(boolean keepInstance = false)
    }
    // Stop Tool-Server
    stopTS(String installation)
    timeout(int | String timeout = 30)
}
Test execution
steps {
    // Run package
    testPackage(String packageFile) {
        testConfig {
            tbcFile(String tbcFile)
            tcfFile(String tcfFile)
            forceReload(boolean forceReload = false)
            loadOnly(boolean loadOnly = false)
            constants {
                constant(String name, String value)
                constant {
                    name(String name)
                    value(String value)
                }
            }
        }
        packageConfig {
            runTest(boolean runTest = true)
            runTraceAnalysis(boolean runTraceAnalysis = true)
            parameters {
                parameter(String name, String value)
                parameter {
                    name(String name)
                    value(String value)
                }
            }
        }
        executionConfig {
            timeout(int | String timeout = 3600)
            checkTestFile(boolean checkTestFile = true)
            stopOnError(boolean stopOnError = true)
        }
    }
    // Run project
    testProject(String projectFile) {
        testConfig { (...) }
        projectConfig {
            execInCurrentPkgDir(boolean currentPkgDir = false)
            filterExpression(String expression)
            // modes: NO_EXECUTION(0), SEQUENTIAL_EXECUTION(1),
            // PARALLEL_EXECUTION(2), SEPARATE_SEQUENTIAL_EXECUTION(5),
            // SEPARATE_PARALLEL_EXECUTION(6), NO_TESTCASE_EXECUTION(9)
            jobExecutionMode(int | String execMode = 1)
        }
        executionConfig { (...) }
    }
    // Run test folder
    testFolder(String testFolder) {
        // modes: PACKAGES_ONLY, PROJECTS_ONLY, PACKAGES_AND_PROJECTS
        scanMode(String mode = 'PACKAGES_AND_PROJECTS')
        recursiveScan(boolean recursive = false)
        testConfig { (...) }
        packageConfig { (...) }
        projectConfig { (...) }
        executionConfig { (...) }
    }
}
Test management
steps {
    // Import projects
    importProjects {
        importFromArchive(String archivePath, String importPath, String importConfigPath, boolean replaceFiles)
        importFromArchive(String archivePath) {
            importPath(String importPath)
            importConfigPath(String importConfigPath)
            replaceFiles(boolean replaceFiles)
        }
        importFromTMS(String credentialsId, String projectPath, String importPath, int | String timeout = 60, String tmProjectId)
        importFromTMS(String credentialsId, String projectPath, {
            importPath(String importPath)
            timeout(int | String timeout = 60)
        }, String tmProjectId)
        importFromTMSDir(String credentialsId, String projectDirPath, String importPath, int | String timeout = 60)
        importFromTMSDir(String credentialsId, String projectDirPath) {
            importPath(String importPath)
            timeout(int | String timeout = 60)
        }
        importAttributesFromTMS(String credentialsId, String projectPath, int | String timeout = 60)
        importAttributesFromTMS(String credentialsId, String projectPath) {
            timeout(int | String timeout = 60)
        }
    }
    // Import packages
    importPackages {
        importFromTMS { (...) }
        importFromTMSDir { (...) }
        importAttributesFromTMS { (...) }
    }
    // Export projects
    exportProjects {
        exportToTMS(String credentialsId, String projectPath, String exportPath, boolean createNewPath, int | String timeout = 60)
        exportToTMS(String credentialsId, String projectPath) {
            exportPath(String exportPath)
            createNewPath(boolean createNewPath)
            timeout(int | String timeout = 60)
        }
        exportAttributesToTMS(String credentialsId, String projectPath, int | String timeout = 60)
        exportAttributesToTMS(String credentialsId, String projectPath) {
            timeout(int | String timeout = 60)
        }
    }
    // Export packages
    exportPackages {
        exportToTMS { (...) }
        exportAttributesToTMS { (...) }
    }
}
Publishing reports
publishers {
    // Publish ATX reports
    publishATX(String atxInstallation) {
        // Archive options
        allowMissing(boolean allowMissing = false)
        runOnFailed(boolean runOnFailed = false)
        archiving(boolean archiving = true)
        keepAll(boolean keepAll = true)
    }
    // Publish TRF reports
    publishTRF() {
        // Archive options...
    }
    // Publish UNIT reports
    publishUNIT(String installation) {
        unstableThreshold(double threshold = 0)
        failedThreshold(double threshold = 0)
        // Archive options...
    }
    // Publish ecu.test logs
    publishETLogs() {
        unstableOnWarning(boolean markUnstable = false)
        failedOnError(boolean markFailed = false)
        testSpecific(boolean testSpecific = false)
        // Archive options...
    }
    // Publish Report Generators
    publishGenerators(String installation) {
        generators {
            generator(String generatorName) {
                settings {
                    setting(String name, String value)
                    setting {
                        name(String name)
                        value(String value)
                    }
                }
            }
        }
        customGenerators {
            customGenerator(String generatorName) { (...) }
        }
        // Archive options...
    }
    // Publish to Test Management System
    publishTMS(String installation, String credentialsId) {
        timeout(int | String timeout = 60)
        // Archive options...
    }
}

Pipeline

The Pipeline Plugin allows to orchestrate automation, simple or complex, using a DSL. This plugin supports the use of all provided build steps and post-build actions from within a Jenkins Pipeline build. The appropriate DSL syntax for these steps and actions can be easily generated with help of the Pipeline Snippet Generator using the related custom pipeline step or the 'step' command, when using plugin version prior to 1.13.

Basic pipeline example
node('windows') {
    stage('Start Tools') {
        startET 'ecu.test'
    }
    stage('Test Execution') {
        testPackage testFile: 'sample.pkg', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf']
        testProject testFile: 'sample.prj', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf']
    }
    stage('Publish Reports') {
        publishATX 'test.guide'
    }
    stage('Stop Tools') {
        stopET 'ecu.test'
    }
}
Dynamic ecu.test pipeline example
node('windows') {
    // ecu.test installation using global variable ET
    def et = ET.newInstallation toolName: 'ecu.test', installPath: 'C:\\Program Files\\ECU-TEST X.Y'

    // or getting existing installation from global tool configuration
    def et = ET.installation('ecu.test')

    // Start ecu.test
    et.start('C:\\Data\\workspace', 'C:\\Data\\settings')

    // Stop ecu.test
    et.stop()
}
Dynamic test.guide pipeline example

Simple
For simple use cases there is the newServer method of the global ATX variable.

node('windows') {
    // Start tools, execute tests
    ...

    // test.guide server instantiation using global variable ATX
    def atx = ATX.newServer atxName: 'test.guide', toolName: 'ecu.test',
        fullServerURL: 'http://localhost:8085', uploadToServer: false,
        uploadAuthenticationKey: 'xxx', projectId: '1'

    // or getting existing instance from global configuration
    def atx = ATX.server('test.guide')

    // Override one or multiple settings explicitly
    atx.overrideSetting('useHttpsConnection', true)
    atx.overrideSettings([serverURL: '127.0.0.1', useHttpsConnection: true])

    // Publish ATX reports directly
    atx.publish()
}

Complex configurations
For more complex configurations it is recommended to use the newATXServer step, which allows to pass an ATXConfig object. For some more information and/or examples refer the help section of the newATXServer step in your Jenkins instance.
If there is a setting in ecu.test which is not supported by this plugin yet, you can still configure them as customSettings.

node('windows') {
    // Start tools, execute tests, generate reports
    ...

    // dynamic test.guide server instantiation using newATXServer step
    def atx = newAtxServer atxName: 'test.guide', toolName: 'ecu.test',
        config: atxConfig(
            settings: [
                atxTextSetting(group: 'CONNECTION', name: 'serverURL', value: 'localhost'),
                atxTextSetting(group: 'CONNECTION', name: 'serverPort', value: '8085'),
                atxTextSetting(group: 'CONNECTION', name: 'projectId', value: '1'),
                atxSecretSetting(group: 'CONNECTION', name: 'uploadAuthenticationKey', value: 'xxx'),
                atxBooleanSetting(group: 'UPLOAD', name: 'uploadToServer', value: true),
            ],
            customSettings: [
                atxCustomTextSetting(name: 'customSetting', value: 'customValue'),
                atxCustomBooleanSetting(name: 'boolSetting', checked: true)
            ]
        )
    
    // Publish ATX reports
    publishATX atxName: atx.installation.getName(), atxInstallation: atx.installation
}

Configuration as Code

The Jenkins Configuration as Code Plugin (a.k.a. JCasC) allows to configure Jenkins based on human-readable declarative configuration files. This plugin supports the external configuration of ecu.test tool installations and of test.guide server instances in the Jenkins global configuration.

Existing configurations of both types can be exported to according YAML file using the JCasC export view.

ecu.test installation
tool:
    ecu-test:
        installations:
            - name: "ecu.test X.Y"
              home: "C:\\Program Files\\ECU-TEST X.Y"
            - name: "ecu.test X2.Y2"
              home: "C:\\Program Files\\ECU-TEST X2.Y2"
              properties:
                  - ecuTestProperty:
                        progId: "ECU-TEST.Application.X2.Y2"
                        registerComServer: true
                        timeout: 60
test.guide server
unclassified:
    testGuide:
        installations:
            - name: "test.guide"
              toolName: "ecu.test"
              config:
                  settings:
                      - atxTextSetting:
                            group: CONNECTION
                            name: "serverURL"
                            value: "127.0.0.1"
                      - atxTextSetting:
                            group: CONNECTION
                            name: "serverLabel"
                      - atxBooleanSetting:
                            group: CONNECTION
                            name: "useHttpsConnection"
                            value: false
                      - atxBooleanSetting:
                            group: CONNECTION
                            name: "ignoreSSL"
                            value: false
                      - atxTextSetting:
                            group: CONNECTION
                            name: "serverPort"
                            value: "8085"
                      - atxTextSetting:
                            group: CONNECTION
                            name: "serverContextPath"
                      - atxSecretSetting:
                            group: CONNECTION
                            name: "httpProxy"
                      - atxSecretSetting:
                            group: CONNECTION
                            name: "httpsProxy"
                      - atxTextSetting:
                            group: CONNECTION
                            name: "projectId"
                            value: "1"
                      - atxSecretSetting:
                            group: CONNECTION
                            name: "uploadAuthenticationKey"
                      - atxTextSetting:
                            group: CONNECTION
                            name: "useSettingsFromServer"
                            value: "Never"
                      - atxBooleanSetting:
                            group: UPLOAD
                            name: "uploadToServer"
                            value: true
                      - atxTextSetting:
                            group: UPLOAD
                            name: "uploadThroughResourceAdapter"
                            value: "0"
                      - atxBooleanSetting:
                            group: UPLOAD
                            name: "uploadAsync"
                            value: true
                      - atxTextSetting:
                            group: UPLOAD
                            name: "maxUploadTries"
                            value: "42"
                      - atxBooleanSetting:
                            group: UPLOAD
                            name: "compressUpload"
                            value: false
                      - atxBooleanSetting:
                            group: UPLOAD
                            name: "cleanAfterSuccessUpload"
                            value: true
                      - atxBooleanSetting:
                            group: UPLOAD
                            name: "uploadOnlyProjectReport"
                            value: false
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "enableArchive"
                            value: true
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveExternalLinks"
                            value: false
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveTrf"
                            value: true
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archivePkg"
                            value: false
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveTcf"
                            value: false
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveTbc"
                            value: false
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveMapping"
                            value: false
                      - atxTextSetting:
                            group: ARCHIVE
                            name: "archiveRecordings"
                            value: "False"
                      - atxTextSetting:
                            group: ARCHIVE
                            name: "archiveRecordingsByAttributes"
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveRecordingMetadata"
                            value: true
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archivePlots"
                            value: true
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveImages"
                            value: true
                      - atxTextSetting:
                            group: ARCHIVE
                            name: "archiveMiscFiles"
                      - atxTextSetting:
                            group: ARCHIVE
                            name: "archiveMiscFilePrefix"
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveMiscFilesOnlyInTestReportDir"
                            value: true
                      - atxTextSetting:
                            group: ARCHIVE
                            name: "archiveFilesPerPackage"
                      - atxTextSetting:
                            group: ARCHIVE
                            name: "archiveBy"
                            value: "NONE;SUCCESS;INCONCLUSIVE;FAILED;ERROR"
                      - atxBooleanSetting:
                            group: ARCHIVE
                            name: "archiveDescriptionImages"
                            value: true
                      - atxTextSetting:
                            group: ATTRIBUTE
                            name: "coveredAttributes"
                            value: "Testlevel;Designer;Execution Priority;Estimated Duration [min];"
                      - atxTextSetting:
                            group: ATTRIBUTE
                            name: "attributeDelimiter"
                      - atxBooleanSetting:
                            group: ATTRIBUTE
                            name: "mapIsTestCaseAsAttribute"
                            value: true
                      - atxBooleanSetting:
                            group: ATTRIBUTE
                            name: "mapTestCaseVersionAsAttribute"
                            value: true
                      - atxBooleanSetting:
                            group: ATTRIBUTE
                            name: "mapRootPrjAttrToPkgAttr"
                            value: true
                      - atxTextSetting:
                            group: ATTRIBUTE
                            name: "excludePrjAttrPrefixFor"
                      - atxBooleanSetting:
                            group: ATTRIBUTE
                            name: "includePkgSVNRevision"
                            value: true
                      - atxBooleanSetting:
                            group: ATTRIBUTE
                            name: "mapSwkIdsAsAttribute"
                            value: true
                      - atxTextSetting:
                            group: TBC_CONSTANTS
                            name: "coveredConstants"
                            value: "*; SWK_Version; TT_*; TBC_*; TCF_*"
                      - atxBooleanSetting:
                            group: TBC_CONSTANTS
                            name: "mapTbcToolAsConstant"
                            value: true
                      - atxBooleanSetting:
                            group: TCF_CONSTANTS
                            name: "mapTcfTesterAsConstant"
                            value: false
                      - atxBooleanSetting:
                            group: TCF_CONSTANTS
                            name: "mapTCFPropertyAsConstant"
                            value: true
                      - atxBooleanSetting:
                            group: TCF_CONSTANTS
                            name: "mapUserDefinedReportDataAsConstant"
                            value: true
                      - atxTextSetting:
                            group: TCF_CONSTANTS
                            name: "setConstants"
                      - atxTextSetting:
                            group: TCF_CONSTANTS
                            name: "setAttributes"
                      - atxTextSetting:
                            group: REVIEW
                            name: "setReviewTags"
                      - atxBooleanSetting:
                            group: REVIEW
                            name: "reviewUseAbortCodeAsCustomEvaluation"
                            value: false
                      - atxTextSetting:
                            group: REVIEW
                            name: "detectReviewTags"
                      - atxTextSetting:
                            group: REVIEW
                            name: "detectReviewDefects"
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "autoATXGeneratorUpdate"
                            value: true
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "includeToolIdentifier"
                            value: false
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "includePkgTestSteps"
                            value: true
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "onlyIncludePkgTestCases"
                            value: false
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "overrideParamSetNameMapping"
                            value: false
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "mapProjectElementNameAsTestCaseName"
                            value: false
                      - atxTextSetting:
                            group: SPECIAL
                            name: "mapSubPackageAsTestCaseLevel"
                            value: "0"
                      - atxTextSetting:
                            group: SPECIAL
                            name: "captureSubPackageOnVerdict"
                      - atxTextSetting:
                            group: SPECIAL
                            name: "captureSpecialTestSteps"
                            value: ""
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "mapSeparateProjectExecutionAsSingleTestplan"
                            value: false
                      - atxTextSetting:
                            group: SPECIAL
                            name: "mapAttributeAsConstant"
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "mapTestReportPathAsConstant"
                            value: false
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "includeResourceAdapterInfo"
                            value: true
                      - atxBooleanSetting:
                            group: SPECIAL
                            name: "captureExecutionTime"
                            value: true
                  customSettings:
                      - atxCustomBooleanSetting:
                            name: "customOption"
                            checked: true
                      - atxCustomTextSetting:
                            name: "customLabel"
                            value: "test"

Debugging

To change the job console log level to debug, the system property ecutest.debugLog should be set to true. This could be done either at startup

java -Decutest.debugLog=true -jar jenkins.war

or at runtime in the console under Jenkins -> Manage Jenkins -> Script Console

System.setProperty("ecutest.debugLog", "true")

To get a more debug output about plugin COM API communication a new log recorder with following logger instances could be created under Manage Jenkins -> System Log -> New Log Recorder. Set a preferable name, add loggers de.tracetronic.jenkins.plugins.ecutest.wrapper.com.ETComClient, de.tracetronic.jenkins.plugins.ecutest.wrapper.com.ETComDispatch and set the log level to at least FINE.

Create new COM API log recorder

Issues

To report a bug or request an enhancement to this plugin please raise a new GitHub issue or create a ticket in JIRA.

Known limitations

When using the plugin in a agent-based setup (especially running the agent at command line via JNLP) you also need to restart the agent when restarting the master. This is due to already loaded libraries, for more information see this blog post and related issue JENKINS-31961.

Release 1.8 implemented an extension point that will workaround this issue. In order to work a new task has to be created in the Windows Task Scheduler named RESTART_JENKINS_AGENT and configured with actions how to restart the agent.

Please note that from ecu.test Version 2022.3 on, invalid escape sequences in expressions within ecu.test are disallowed. This means, in particular, that backslashes in Windows paths need to be escaped properly (e.g. C:\\my\\path instead of C:\my\path) - otherwise, this will lead to an error within ecu.test.

Since release 2.39 we test only against ecu.test versions, that are inside the support window of the tracetronic tool support terms. Please consult the compatibility matrix for detailed information.

FAQ

Frequently asked questions, common issues and error messages about the ecu.test Jenkins Plugin and ecu.test itself can be found in the tracetronic Knowledge Base.

Compatibility

  • Jenkins LTS 2.319.3 or higher
  • Java SE Runtime Environment 17 or higher
ecu.test 2023.1 or higher
Version latest - 2.44 2.43 - 2.39 2.38 - 2.26 2.25 - 2.23 2.22 - 2.19 2.18 - 2.15 2.15 2.14 - 2.13 2.12 - 2.8 2.7 - 2.5 2.4 - 2.2 2.1 - 2.0
2024.1 ✔️
2023.4 ✔️ ✔️
2023.3 ✔️ ✔️
2023.2 ✔️ ✔️
2023.1 ✔️ ✔️
2022.4 ✔️ ✔️
2022.3 ✔️ ✔️
2022.2 ✔️ ✔️
2022.1 ✔️ ✔️
2021.4 ✔️ ✔️
2021.3 ✔️ ✔️
2021.2 ✔️ ✔️
2021.1 ✔️ ✔️
2020.4 ✔️ ✔️
2020.3 ✔️ ✔️ ✔️
2020.2 ✔️ ✔️ ✔️ ✔️
2020.1 ✔️ ✔️ ✔️ ✔️ ✔️
8.1.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
8.0.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
7.2.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
7.1.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
7.0.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️

Explanation: ✔️ -> compatible | ❔ -> not tested, might be compatible | ❌ -> not compatible

optional: test.guide 1.87.0 or higher
Version latest - 2.44 2.43 - 2.26 2.25 - 2.23 2.22 2.21 - 2.18 2.17 - 2.16 2.15 - 2.14 2.13 - 2.12 2.11 2.10 - 2.8 2.7 - 2.6
1.169.1 ✔️
1.129.0 ✔️ ✔️
1.120.0 ✔️ ✔️
1.113.0 ✔️ ✔️
1.104.0 ✔️ ✔️
1.95.0 ✔️ ✔️
1.87.0 ✔️ ✔️ ✔️
1.84.2 ✔️ ✔️ ✔️
1.82.0 ✔️ ✔️ ✔️ ✔️
1.80.0 ✔️ ✔️ ✔️ ✔️ ✔️
1.76.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
1.72.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
1.67.0 ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
1.64.1 ✔️ ✔️ ✔️ ✔️
* Note that some lacking version incompatibilities in the test.guide - ecu.test-Plugin compatibility matrix are due to configuration fields which are not natively supported. You may still add these lacking configurations via the *custom fields* in the test.guide configuration on the Jenkins server.

License

This plugin is licensed under 3-clause BSD license. More information can be found inside the LICENSE file or within the LICENSES folder. Using the REUSE helper tool, you can run reuse spdx to get a bill of materials.

Changelog

ecutest-plugin's People

Contributors

cpoenisch avatar daniel-beck avatar dependabot-preview[bot] avatar dependabot[bot] avatar erikrehmtt avatar hans-jakobholtztt avatar jangu-tt avatar mariofuchstt avatar martingroschett avatar mxeh-tt avatar tbeu avatar timja avatar tt-cx-user avatar tt-marcel avatar tt-s3-d1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ecutest-plugin's Issues

Read TEST-GUIDE settings from persisted settings file

Describe the feature request
Allow to read TEST-GUIDE server and ATX related settings from a persisted XML file placed next to the TRF report when publishing ATX reports using the ATXPublisher.

The same mechanism is already implemented in ReportGeneratorPublisher but has the drawback that linked ATX reports in the build only consist of report artifacts but do not redirect to TEST-GUIDE when upload is enabled.

Therefore the same option to read from persisted settings file should be added to the ATXPublisher and will supersede existing TEST-GUIDE settings (global or dynamic configuration) when checked.

Additional context
Related to #39

Debug mode option is ignored using dynamic ECU-TEST installation

Describe the bug
When starting a dynamically created ECU-TEST instance in pipelines with debug mode set this option will be ignored throwing following warning to console output:

WARNING: Unknown parameter(s) found for class type 'de.tracetronic.jenkins.plugins.ecutest.tool.StartETBuilder': debug

How to reproduce

def et = ET.installation('ECU-TEST')
et.start('wsDir', 'settingsDir', 0, true, false, false)

Expected behavior
Debug mode option should be passed properly to the according build step.

Versions

  • Jenkins: unrelated
  • Plugin: 2.12
  • ECU-TEST: unrelated
  • TEST-GUIDE: unrelated

Additional context

  • Option name should match with property of StartETBuilder class
    (debugMode instead of debug)

Ensure ECU-TEST 8.1 compatibility

Describe the feature request
Ensure compatibility with ECU-TEST 8.1 (especially COM API) while retaining backward compatibility with ECU-TEST 7.x and above.

Additional context
Estimated release date is on 28.11.2019.

Add generator option to read from persisted settings file

Starting with ECU-TEST 8.0 report generator settings can be persisted to XML file placed next to the TRF which can be used for downstream report generation. An option should be added to report generator publisher in order to use persisted settings.

According COM API method: COMTestEnvironment: GenerateTestReportDocument(String trfFile, String outDir, String fmtConfigFile, boolean waitUntilFinished=false) : boolean

Add option to fail the build when TEST-GUIDE is not reachable

With activated TEST-GUIDE upload option and in case the server is not reachable during the publisher execution only a warning will be logged:

-> ATX upload will be skipped because selected TEST-GUIDE server is not reachable!

Nevertheless the publisher continues and only generates the ATX report files for download and manual upload to TEST-GUIDE.

To mark the build as failed in this situation a new publisher option is requested.

Add more debug logs

Provide more detailed debug logs for all build steps and publishers. Debug logging can be enabled by setting system property ecutest.debugLog=true.

Extended debug logging for:

  • Open tool processes
  • COM programmatic ID
  • COM version
  • Loaded ECU-TEST patches
  • Current workspace settings
  • Last loaded TBC and TCF
  • Full stacktrace in case of COM exceptions

Allow dynamic inline ECU-TEST instances in pipelines

In order to reduce maintenance efforts for multiple ECU-TEST installations in the Jenkins global tool configuration there should be a dynamic way for pipelines to define ECU-TEST installation configurations inline. The existing ECU-TEST build step must be extended to accept such installation configurations.

Method signatures

Define new installation

ET.newInstallation(String toolName, String installPath, ETToolProperty property) : ETInstance
or
ET.newInstallation(String toolName, String installPath, String progId, int timeout) : ETInstance

Get configured installation

ET.installation(String toolName) : ETInstance
and
ETInstance.getInstallation() : ETInstallation

Pipeline steps

ECU-TEST

startET(ETInstallation installation, String workspaceDir, String settingsDir, int timeout, boolean debug, boolean keepInstance, boolean updateUserLibs) : void
or
ETInstance.start(String workspaceDir, String settingsDir, int timeout, boolean debug, boolean keepInstance, boolean updateUserLibs) : void

stopET(String toolName | ETInstallation installation, int timeout) : void
or
ETInstance.stop(int timeout) : void

Tool-Server

startTS(ETInstallation installation, String workspaceDir, String settingsDir, int timeout, boolean debug, boolean keepInstance, boolean updateUserLibs) : void
or
ETInstance.startTS(String workspaceDir, String settingsDir, int timeout, boolean debug, boolean keepInstance, boolean updateUserLibs) : void

stopTS(ETInstallation installation, int timeout) : void
or
ETInstance.stopTS(int timeout) : void

UNIT Publisher

publishUNIT(ETInstallation installation, double unstableThreshold, double failedThreshold, boolean allowMissing, boolean runOnFailed) : void
or
ETInstance.publishUNIT(double unstableThreshold, double failedThreshold, boolean allowMissing, boolean runOnFailed) : void

Report Generator Publisher

publishGenerators(ETInstallation installation, List<ReportGeneratorConfig> generators, List<ReportGeneratorConfig> customGenerators, boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void
or
ETInstance.publishGenerators(List<ReportGeneratorConfig> generators, List<ReportGeneratorConfig> customGenerators, boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void

Testmanagement-System Publisher

publishTMS(ETInstallation installation, String credentialsId, int timeout, boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void
or
ETInstance.publishTMS(String credentialsId, int timeout, boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void

Traceanalysis Publisher

publishTraceAnalysis(ETInstallation installation, boolean mergeReports, boolean createReportDir, int timeout, boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void
or
ETInstance.publishTraceAnalysis(boolean mergeReports, boolean createReportDir, int timeout, boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void

ATX trend view link redirects to deprecated page

The ATX publisher creates dynamic TEST-GUIDE links for each test case (a prepared search filter link and a trend view link for package executions only).

Starting with TEST-GUIDE 1.65.0 the Wicket based link for test case details including the trend view has been changed from wicket/bookmarkable/de.tracetronic.ttstm.web.detail.TestReportViewPage to wicket/bookmarkable/de.tracetronic.ttstm.web.detail.TestCaseDetailPage.

Published ATX trend view links from Jenkins will not redirect to TEST-GUIDE properly anymore and therefore either both link types must be handled as long as older TEST-GUIDE versions are supported (currently 1.55.0 and higher) or the minimum supported version will be increased to at least 1.65.0.

Allow individual Windows task name

In order prevent naming conflicts when running multiple Jenkins agents on the same machine, the default task name 'RESTART_JENKINS_SLAVE' for the Windows Task Scheduler should be changeable via system property optionally.

Handle starting ECU-TEST with invalid license

Beginning with ECU-TEST 8.1 the case of starting ECU-TEST without or with an invalid license will no longer result in a blocking license dialog that will prevent further automation. It will then be handled properly by returning exit code 99 when using the command line option --startupAutomated.
Starting via COM API will throw a proper exception message instead.

The plugin needs to handle that process exit code and should log an appropriate error message.

Your .dependabot/config.yml contained invalid details

Dependabot encountered the following error when parsing your .dependabot/config.yml:

The property '#/update_configs/0/ignored_updates/0' contains additional properties ["dependency_name"] outside of the schema when none are allowed
The property '#/update_configs/0/ignored_updates/0/match' of type null did not match the following type: object
The property '#/update_configs/0/ignored_updates/1' contains additional properties ["dependency_name"] outside of the schema when none are allowed
The property '#/update_configs/0/ignored_updates/1/match' of type null did not match the following type: object
The property '#/update_configs/0/ignored_updates/2' contains additional properties ["dependency_name"] outside of the schema when none are allowed
The property '#/update_configs/0/ignored_updates/2/match' of type null did not match the following type: object
The property '#/update_configs/0/ignored_updates/3' contains additional properties ["dependency_name"] outside of the schema when none are allowed
The property '#/update_configs/0/ignored_updates/3/match' of type null did not match the following type: object

Please update the config file to conform with Dependabot's specification.

You can mention @dependabot in the comments below to contact the Dependabot team.

Warning about serialization of anonymous class

While starting ECU-TEST or the Tool-Server following warning messages are printed to agent's remoting log:

org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARNUNG: Attempt to (de-)serialize anonymous class de.tracetronic.jenkins.plugins.ecutest.tool.installation.AbstractToolInstallation$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/

and

org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARNUNG: Attempt to (de-)serialize anonymous class de.tracetronic.jenkins.plugins.ecutest.tool.installation.ETInstallation$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/

The referenced wiki page explains how to avoid such serialization issues.

Add more debug logs to COM API calls

Add more logging in ETComClient for debug purposes for COM-API calls
Add more console log output in debug mode where needed
Add path to ECU-TEST log file to every error log entry in console logger - TTConsoleLogger
Document setting up of debug modus in FAQ.

ECU-TEST 8.0 compatibility

  • Ensure compatibility with ECU-TEST 8.0 (especially COM API)
  • Retain backward compatibility with ECU-TEST 7.x
  • Drop support for ECU-TEST 6.x

Check whether configurations are started

Add a pipeline step that allows to check whether the currently selected test configuration and test bench configuration are started. This feature will be only available on ECU-TEST 8.0 and later.

According COM API method: COMApplication#IsStarted() : boolean

Method signature

ETInstance.isConfigStarted() : boolean

Pipeline Snippet

def et = ET.installation('ECU-TEST')
boolean isConfigStarted = et.isConfigStarted()

Add timeout option using Quit and Exit via COM API

Starting with ECU-TEST 8.0 an optional timeout parameter was added to COM API methods in order to handle safe shut down of connected tools or ECU-TEST itself.

According COM API method:
COMApplication#Quit(timeout=60) : boolean
COMApplication#Exit(timeout=60) : boolean

Add support for analysis packages

Starting with ECU-TEST 8.0 stimulation packages and analysis packages (with .ta extension) can be used to separate stimulation and analysis in a project context. This workflow should be supported in downstream trace analysis publisher. In particular, searching for generated analysis job files (with .ajob extension) needs to be adapted to support prefix of stimulation package name (regex: **/*Job_*.ajob or simply **/*.ajob)

TEST-GUIDE 1.67.0 compatibility

  • Ensure compatibility with TEST-GUIDE 1.67
  • Update config.xml storing available ATX settings
  • Drop support for TEST-GUIDE 1.54 and below

Add Warnings NG support for package and project checks

Starting with ECU-TEST 2020.3 package and projects can be represented in the Warnings Plugin Native Format as JSON response.
Next to the current option by simply printing check warnings to the build console a new option is requested to feed the Warnings NG plugin with compatible input in order to record these issues within the build directly.

Additional context
According COM API method:

  • COMPackage.CheckNG() : String
  • COMProject.CheckNG() : String

Move changelogs from Wiki to GitHub

Follow-up of #87:

We have recently introduced support of GitHub documentation on the Jenkins Plugin Site. See this Developer thread for the announcement.

Jenkins users seeking documentation will have a better user experience on plugins.jenkins.io than they do on wiki.jenkins.io. At the same time, maintainers can follow the documentation-as-code approach and make documentation changes a part of pull requests. Documentation changes will be reviewed as part of pull requests and documentation contributors will be recognized, especially when combined with Release Drafter. We recommend all plugins to migrate documentation to GitHub.

Once documentation is moved, the plugin metadata needs to be updated to point to GitHub: documentation URL in the project file (guide) . Then the change will be applied after the release.

Move changelogs from Wiki page to separate CHANGELOG.md and link from README.md.
Recent changes (2.9+) are automatically logged by Release Drafter and do not need to be included.

Controlled timeout allowing report to finish

Executing test projects/packages should allow an option to perform "Cancel after next project step" if execution time is reached. Additionally test step should wait for some time to let the last project step to finish and completed report generation. Both timeouts should be configurable.

Add GitHub Actions based CI workflow

Due to recent instabilities of the Jenkins CI a parallel CI workflow based on GitHub Actions should be configured as a backup CI next to the internal Jenkins tests.

The workflow should should run on all push and pull request events and trigger a Maven build including unit tests for following matrix configuration: Java 1.8/11, based/latest Jenkins, Ubuntu/Windows

Support generation and re-use of ECU-TEST caches

Starting with ECU-TEST 7.2 cache files for different cache types (A2L, ELF, BUS, MODEL, SERVICE) can be pre-generated und re-used when starting the configuration. The plugin should provide a build step to generate these cache files beforehand.

Allow dynamic inline TEST-GUIDE instances in pipelines

In order to reduce maintenance efforts for multiple TEST-GUIDE servers in the Jenkins global configuration there should be a dynamic way for pipelines to define TEST-GUIDE server configurations inline. The existing ATX publisher step must be extended to accept such server configurations.

Method signatures

Define new server

ATX.newServer(String atxName, String toolName, ATXConfig config) : ATXServer
or
ATX.newServer(String atxName, String toolName, String serverUrl, boolean uploadToServer, String authKey, String projectId) : ATXServer

Get configured server

ATX.server(String atxName) : ATXServer
and
ATXServer.getInstallation() : ATXInstallation

Publish ATX reports

publishATXReports(ATXInstallation installation, boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void
or
ATXServer.publish(boolean allowMissing, boolean runOnFailed, boolean archiving, boolean keepAll) : void

Improve TMS error handling

Actually all ETComExceptions while calling TMS are labeled with "Enable TMS with feature flag".
The feature flag doesn't exist since ET6.5.

Fix the error message and provide possible solutions in the FAQ.

TEST-GUIDE upload response is not recognized

Responses (errors and success) from ATX upload to TEST-GUIDE are written to a JSON file that will be read from the plugin afterwards to handle upload issues. This only works when the 'Clean After Success Upload' upload setting is disabled in order to parse the response files.

Starting with TEST-GUIDE 1.64.1 the error log file name has been changed from error.log.raw.json to error.raw.json and upload errors are not recognized properly.
Additionally, the response format has been changed in success.json when asynchronous upload is enabled. Therefore either both response formats must be handled as long as older TEST-GUIDE versions are supported (currently 1.55.0 and higher) or the minimum supported version will be increased to at least 1.65.0 (see #101).

ATX publisher ignores ECUTEST var for parametrized instance

  • mehrere ECU-TEST-Instanzen als Hilfstools konfiguriert
  • TG-Instanz mit parametrierter ECU-TEST-Instanz konfigurieren
  • in Pipeline ECUTEST = 'ET_ToolLabel' setzen
  • startET/stopET/testProject/etc funktionieren
  • publishATX kann die ECU-TEST-Instanz nicht starten
  • startET - publishATX - stopET funktioniert
  • TG-Konfiguration mit festem ET - publishATX funktioniert

--> publishATX sollte auch parametriert ECU-TEST korrekt starten können

Allow named arguments for all dynamic pipeline steps

Describe the feature request
Follow-up of #121 implementing the usage of named parameters for all dynamic pipeline steps.

Upstream changes
#124

Additional context
Update documentation and sample code in help files
Add more unit tests checking passed parameters

Use more generic report link index

If a publisher pipeline step (e.g. ReportGeneratorPublisher) is called multiple times, the starting index for linking the generated report(s) will be reset to 1. This results in wrong report linking for all consecutive reports generated by further publisher calls.

Usage of parameterized ECU-TEST installation overwrites value in freestyle job

When using parameterized ECU-TEST installation in e.g. "[TT] Start ECU-TEST" step in freestyle job configurations, the initial option value (${ECUTEST} (parameterized)) is changed to the selected value which has been passed as ECUTEST job parameter with each build.
The modification is just temporarily, after a Jenkins restart the selection gets reverted to the parameterized value.

Return final value of package variables

Describe the feature request
After test execution sometimes the final value of a package variable is needed in order to deal with special test execution flows (e.g. retry the test in case of an initialization failure). Therefore a new option is requested to specify multiple package variables that are written to test related environment variables if test execution has finished: TT_TEST_RETVAL_<VARIABLE NAME>_<INDEX>

Additional context
According COM API method:
COMTestExecutionInfo#GetReturnValue(varname) : dynamic

Migrate the plugin documentation from Wiki to GitHub

We have recently introduced support of GitHub documentation on the Jenkins Plugin Site. See this Developer thread for the announcement.

Jenkins users seeking documentation will have a better user experience on plugins.jenkins.io than they do on wiki.jenkins.io. At the same time, maintainers can follow the documentation-as-code approach and make documentation changes a part of pull requests. Documentation changes will be reviewed as part of pull requests and documentation contributors will be recognized, especially when combined with Release Drafter. We recommend all plugins to migrate documentation to GitHub.

Once documentation is moved, the plugin metadata needs to be updated to point to GitHub: documentation URL in the project file (guide) . Then the change will be applied after the release.

P.S: In Jenkins project we track the documentation migration as one of the Hacktoberfest featured project: https://jenkins.io/events/hacktoberfest/ . Please feel free to put a Hacktoberfest label in this repo if you are interested to suggest it for external contributions

Allow multiple Tool-Server instances

Feature request for parallel Tool-Server instances on different ports considering keepInstance and tcpPort settings. Possible configurations:

keepInstance=false:

  • kill all instances with the same configured port and start a new one

keepInstance=true:

  • re-use existing process if tcpPort setting matches running port
  • otherwise keep other instances and start a new one

Running instances without explicit port argument will be treated as instance with default port 5017.

[JENKINS-57571] Configuration-as-Code compatibility

Add full support for the Jenkins Configuration as Code Plugin. This mainly includes the global configurations for ECU-TEST installations and TEST-GUIDE servers.

Example configurations:

ECU-TEST

tool:
  ecu-test:
    installations:
    - home: "C:\\Program Files\\ECU-TEST 8.0"
      name: "ECU-TEST 8.0"

TEST-GUIDE

unclassified:
  test-guide:
    installations:
      - config:
        settings:
          - atx-boolean-setting:
            group: UPLOAD
            name: "uploadToServer"
            value: true
          - atx-text-setting:
            group: UPLOAD
            name: "serverURL"
            value: "127.0.0.1"
          - ...
        customSettings:
          - atx-custom-boolean-setting:
            name: "customOption"
            checked: true
          - atx-custom-text-setting:
            name: "customLabel"
            value: "test"
          - ...
      name: "TEST-GUIDE"
      toolName: "ECU-TEST"

Tracking: jenkinsci/configuration-as-code-plugin#809

Fix order of generated reports using ReportGeneratorPublisher

Describe the bug
The order of test reports is undetermined when using default or custom report generator publishers. This concerns the order of report generation (sub reports before main reports) as well as the order of linked generated reports in the build page.

To Reproduce
Run multiple tests (packages or projects) and configure any ReportGeneratorPublisher, e.g.:

publishGenerators toolName: 'ECU-TEST',
                  generators: [[name: 'JSON']],
                  customGenerators: [[name: 'CUSTOM']]

Expected behavior
The reports are generated and linked in the order of test execution.

Versions

  • Jenkins: unrelated
  • Plugin: 2.13
  • ECU-TEST: unrelated
  • TEST-GUIDE: unrelated

Request API endpoint to check TEST-GUIDE availability

Starting with TEST-GUIDE 1.55.0 all APIs will be moved to /api and /rest to improve SSO compatibility. Therefore the TEST-GUIDE availability and verification checks from the plugin needs no be moved to the new /api endpoint. This also have the effect to increase the minimum required TEST-GUIDE version to 1.55.0.
Furthermore the ATX report template settings should be updated to latest version.

Rename pipeline symbols to camel case names

All @Symbol pipeline annotations should follow the camel case convention because hyphenated symbol names are not supported in pipeline runs. Pipeline scripts from snippet generator are misleading due to wrong syntax, though.

⚠️ Existing pipeline scripts and configurations (e.g. CasC) must be adapted thereby, i.e.:

  • ecu-test -> ecuTest
  • ecu-test-property -> ecuTestProperty
  • test-guide -> testGuide
  • atx-config -> atxConfig
  • atx-boolean-setting -> atxBooleanSetting
  • atx-text-setting -> atxTextSetting
  • atx-custom-boolean-setting -> atxCustomBooleanSetting
  • atx-custom-text-setting -> atxCustomTextSetting

Workaround: Using class based notation [$class: <Builder/Publisher>, ...]

Soft cancel for timeout

Test execution should try a soft canceling before a hard abortion.

  • reports should be copied
  • execution of report generators should work for the aborted reports

Register COM server on each start of ECU-TEST

Create a new option in ECU-TEST installation configuration that enables automatic registration of COM API server every time ECU-TEST is started.

Command: <ECU-TEST installDir>\ECU-TEST_COM.exe /register /peruser

Wrong encoded tool variables using snippet generator

  • mehrere TG-/ET-Instanzen konfigurieren
  • SnippetGenerator für bspw startET oder publishATX verwenden
  • Auswahl der parametrierten ToolInstanz - Dropdown-Wert ist nicht korrekt encoded
  • Generat enthält '${ECUTEST}' oder '${TESTGUIDE}' -> Variable kann in der Pipeline nicht aufgelöst werden
    --> Generator liefert "${ECUTEST}" oder "${TESTGUIDE}"

Reference filter validation throws ANTLR exception

When validating the filter expression to filter package references in 'Reference Filter' form field in 'Project Settings' section for 'Run Project' step following exception occurs:

java.io.InvalidClassException: org.antlr.v4.runtime.atn.ATN; Could not deserialize ATN with UUID 59627784-3be5-417a-b9eb-8131a7286089 (expected aadb8d7e-aeef-4415-ad2b-8204d6cf042e or a legacy UUID).
Caused: java.lang.UnsupportedOperationException
	at org.antlr.v4.runtime.atn.ATNDeserializer.deserialize(ATNDeserializer.java:154)
	at de.tracetronic.jenkins.plugins.ecutest.filter.RefFilterLexer.<clinit>(RefFilterLexer.java:202)
Caused: java.lang.ExceptionInInitializerError
	at de.tracetronic.jenkins.plugins.ecutest.util.validation.TestValidator$FilterExpressionValidator.validate(TestValidator.java:213)
	at de.tracetronic.jenkins.plugins.ecutest.util.validation.TestValidator.validateFilterExpression(TestValidator.java:168)
	at de.tracetronic.jenkins.plugins.ecutest.test.config.ProjectConfig$DescriptorImpl.doCheckFilterExpression(ProjectConfig.java:236)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:145)
	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:537)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:739)
Caused: javax.servlet.ServletException

There seems to be a dependency conflict between loaded antlr-runtime 4.5 in credentials-plugin and recent version 4.7 in ecutest-plugin.

Convenient methods to override dynamic ATX settings

Allow to override existing ATX settings that were dynamically instantiated as an ATXServer object via ATX.newServer or ATX.server in order to alter some specific default values instead to pass the full configuration map via ATXConfig.

Proposed method signatures:

Default setting:
ATX.newServer(String atxName, String toolName) : ATXServer

Explicit settings configuration via ATXConfig:
ATX.newServer(String atxName, String toolName, ATXConfig config) : ATXServer

Commonly used settings: fullServerURL is parsed to equivalent URL settings
ATX.newServer(String atxName, String toolName, String fullServerURL, boolean uploadToServer, String authKey, String projectId) : ATXServer

Settings via named arguments:
ATXServer.newServer(Map<String, Object> serverArgs) : ATXServer

Additional methods:
ATXServer.overrideSetting(String settingName, Object settingValue) : void
ATXServer.overrideSettings(final Map<String, Object> settings) : void

ATXServer.getSetting(String settingName) : ATXSetting
ATXServer.getSettings() : Map<String, Object>

SQL exception when parsing TRF

SQL parsing exception occured when uploading ATX report to TEST-GUIDE in Windows Docker environment.

Console log:

[TT] INFO: - Generating and uploading ATX reports...
[TT] INFO: -> Generating and uploading ATX report: C:\Workspaces\TestWorkspace\TestReports\Package_xxx\Package.trf
ERROR: Step ‘[TT] Publish ATX Reports’ aborted due to exception: 
java.sql.SQLException: ResultSet closed
	at org.sqlite.RS.checkOpen(RS.java:63)
	at org.sqlite.RS.findColumn(RS.java:108)
	at org.sqlite.RS.getString(RS.java:317)
	at de.tracetronic.jenkins.plugins.ecutest.report.atx.ATXReportUploader$ParseTRFCallable.call(ATXReportUploader.java:580)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip/ip:49175
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:957)
		at de.tracetronic.jenkins.plugins.ecutest.report.atx.ATXReportUploader.upload(ATXReportUploader.java:120)
		at de.tracetronic.jenkins.plugins.ecutest.report.atx.ATXPublisher.publishReports(ATXPublisher.java:158)
		at de.tracetronic.jenkins.plugins.ecutest.report.atx.ATXPublisher.performReport(ATXPublisher.java:119)
		at de.tracetronic.jenkins.plugins.ecutest.report.AbstractReportPublisher.perform(AbstractReportPublisher.java:262)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1841)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: java.io.IOException
	at de.tracetronic.jenkins.plugins.ecutest.report.atx.ATXReportUploader$ParseTRFCallable.call(ATXReportUploader.java:589)
	at de.tracetronic.jenkins.plugins.ecutest.report.atx.ATXReportUploader$ParseTRFCallable.call(ATXReportUploader.java:553)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

Dynamic ECU-TEST installation not found when stopping instance

When stopping an ECU-TEST instance that was referenced dynamically before the following error occurs:

[TT] ERROR: The selected ECU-TEST installation is not configured for this node!

Code to reproduce:

def et = ET.installation('ECU-TEST') // same for ET.newInstallation
et.stop()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.