Create, delete and update Jobs for VectorCAST/Manage projects.
This plugin allows the user to create, delete and update Jobs to build and run VectorCAST projects. Coverage is displayed using the VectorCAST Coverage Plugin.
Jobs can be created as a single job or split into multiple jobs for a Manage project, with one job for each environment and an overall job to combine the results.
This plugin adds a new top-level menu item to the Jenkins sidebar.
that provides job control for VectorCAST/Manage projects
There are 2 types of supported jobs and 1 deprecated job type
- Single Job
- Pipeline Job
- Multi-Job (Deprecated)
Single creates a single Jenkins job to build/execute and (optionally) report on all environments in a VectorCAST/Manage project.
Pipeline creates a Jenkins Pipeline job to build/execute and (optionally) report on all environments in a VectorCAST/Manage project in parallel.
There are 2 options for running tests
- Use an SCM system (any that is supported by Jenkins)
- In this case, Jenkins will check out the code and tests into the workspace for each Jenkins job from your repository
- The pipeline job will them combine the coverage and test results from all these individual machines/nodes
- In this case, the VectorCAST/Manage project should be specified as relative to the root of the checkout
- There is now an option to use the main Pipeline Job's Workspace as a dedicated single checkout directory. This checkout directory must be available to all executors across all nodes either by having all executors running on the same computer or have the main Pipeline Job's Workspace on a shared network drive.
- Use an existing drive/directory for VectorCAST/Manage project
- In this case, the VectorCAST/Manage project should be specified as an absolute path that is available on all machines/nodes
- Each job can optionally clean up the working directory which will have no effect on the VectorCAST/Manage project since it is located elsewhere
- The reports are generated into the workspace and archived as part of the Jenkins job
The user will be able to disable the use of Change Based Testing to perform a complete run of their VectorCAST Project. By default Change Based Testing is enabled, but this option can be disabled by unchecking the Use Change Based Testing box
For users with Continuous Integration Licneses, you will be able to access those licenses by checking the Use Continuous Integration License checkbox. If you do not have Continuous Integration License, do not check this box as you will encounter licensing errors.
If the user wishes to call the Jenkins job from another Pipeline job, check the box to
parameterize the Jenkinsfile. This will add parameters to the pipeline job
that will be used by the VectorCAST pipeline job to locate the VectorCAST/Manage project (VCAST_PROJECT_DIR)
and for forcing the VectorCAST Jobs to be executed on a specific node (VCAST_FORCE_NODE_EXEC_NAME) instead of using
the compiler as a node label.
Calling the build command will return Failed, Unstable, Success cooresponding to the results of the VectorCAST Pipeline job. To enabled the main pipeline job to continue, the user can surround the build command without a catchError block as demonstrated below
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', catchInterruptions : false ) {
build job: 'UnitTestingProject_vcast_pipeline', parameters: [string(name: 'VCAST_PROJECT_DIR', value: 'C:\\UnitTesting\\Project'), string(name: 'VCAST_FORCE_NODE_EXEC_NAME', value: 'MyTestNode')]
}
Additionally, if a shared artifact directory is specified, VectorCAST/Manage will execute jobs independently, but have a central location for storing the build artifacts. This option can be used to accelerate testing by use of VectorCAST's Change Based Testing feature.
For Pipeline jobs, the plugin processes the build log to determine which tests have been skipped by VectorCAST's Change Based Testing feature. Any test cases that had previously pass and were skipped on the current execution will be marked as skipped for JUnit and display as such in yellow on the test result trend chart:
and denoted as Skipped in the test results list
Multi-Job (Deprecated) creates a top-level Jenkins job to combine the results from individual Jenkins jobs created for each environment in the VectorCAST/Manage project. The options for a multi-job are the same as single apart from supplying a copy of the Manage project.
Jenkins jobs normally build and run in independent workspaces. This is the case with this integration. However, there are 2 options for running tests
- Use an SCM system (any that is supported by Jenkins)
- In this case, Jenkins will check out the code and tests into the workspace for each Jenkins job from your repository
- The top-level job will them combine the coverage and test results from all these individual machines/nodes
- In this case, the VectorCAST/Manage project should be specified as relative to the root of the checkout
- Each job can optionally clean up the working directory. If the working directory is not cleaned, then the results from the previous run allow VectorCAST/Manage to optimize the execution phase based on any code changes
- Use a common, shared drive/directory
- In this case, the VectorCAST/Manage project should be specified as an absolute path that is available on all machines/nodes
- Note: Some network drives/shares do not fully implement file locking which cause SQLite, used by VectorCAST/Manage, problems that can result in corruption of the results. If this happens, you may need to use a different network drive/share or consider using an SCM system. Using VectorCAST vc2019 SP3 eliminates these errors
- Each job can optionally clean up the working directory which will have no effect on the VectorCAST/Manage project since it is located elsewhere
- The reports are generated into the workspace and archived as part of the Jenkins job
When using Multi-Jobs, the jobs are created to run on specific nodes related to the compiler chosen for the environment. E.g.
Make sure to set the labels on the relevant Jenkins nodes. Manage Jenkins -> Manage Nodes -> configure appropriate node and set the ‘Labels’ field. In this example the labels have been set to GNU_Native_5.1_C and GNU_Native_4.8_C
The requirements for using this plugin with VectorCAST are
- VectorCAST needs to be installed and setup on each node to be used
- VECTORCAST_DIR and VECTOR_LICENSE needs to be set correctly
- Jenkins needs to be installed and setup
- in particular BUILD_URL needs to be defined (in Jenkins->Manage Jenkins->Configure System and define 'Jenkins URL' and save the settings
An existing multi-job can be updated using the Update Multi-job setup manually, or by creating an auto-update multi-job Job. The auto-update job may require username/password to be supplied depending on your Jenkins configuration.
See Configuring Content Security Policy
"Jenkins 1.641 / Jenkins 1.625.3 introduce
the Content-Security-Policy header to static files served by Jenkins
(specifically, DirectoryBrowserSupport). This header is set to a very
restrictive default set of permissions to protect Jenkins users from
malicious HTML/JS files in workspaces, /userContent, or archived
artifacts."
What this means is that the aggregate coverage report will show everything as black rather than red/green/amber coverage colors.
The link above gives details of how to configure Jenkins to relax its security.
For non-pipeline jobs, JUnit publisher will fail any environments published with no test results. If you have an environment with no test results, you will manually need to check the box "Do not fail the build on empty test results" in the Publish JUnit test result report configuration.
- Fixed bug caught by updated to Python 3
- Fixed command capture for processing later
- Changed side panel display of VectorCAST to not be shown when anonymous user is using Jenkins
- Single jobs not displaying full report and rebuild report correctly in summary (VC2020)
- Capturing stdout for skipped analysis causing console output to be delayed
- Pipeline job fails unless user specific .vcm for manage project
- Problem with CBT analysis when not using single checkout directory
- Using lowercase %workspace% causes single checkout directory error
- Updated managewait.py to overcome race condition causing script to hang on readline()
- Added skipped test analysis to Single Jobs
- Added detection of disabled environment to DataAPI generated results
- Added addition error detection
- Skipped tests not detected for cover environments
- Check for illegal characters in pipeline job names
- Add Option to not use CBT in Jenkins pipeline plugin
- VC2020 Manage reports have no formatting in Jenkins job summary
- Remove xUnit reporting
- Remove truncated output during the build
- "Could not acquire a read lock on the project's vcm file" should raise an error, but it does not
- Add option to active CI licenses
- Regression Scripts with identical ending directories show testcase as skipped
- Add option to make job parameterized for Manage project location and Force Node Execution rather than compiler
- Update error thrown when user creates absolute path with SCM snippet
- Need to add more error detection
- Expand JUnit test case name display to include file.subprogram
- Change reports to not use VCAST_RPTS_SELF_CONTAINED=FALSE
- Support Multi-Branch Pipelines
- Include plugin version in Pipeline Job Jenkins Script and console log
- Check for illegal characters in Pipeline Job names
- Update Single and Multi-Jobs to use same reporting as Pipeline Jobs
- Remove CombinedReport.html from previous Pipeline job build at beginning of new build
- Make aggregate coverage results consistent with VectorCAST/Manage aggregate coverage results
- Always use new VectorCAST report API for VectorCAST/2019 and later
- Fix potential file access race condition when generating xml
- Fix exception when building VectorCAST/Unit regression script environments
- Support for VectorCAST/vpython 3
- Introduce pipeline job
- Updated support for Jenkins Pipeline
- Updated python scripts to work with VectorCAST 2019 SP3
- Added fix for Ada environments not building properly
- Update support for Manage API and VCAST_RPTS_SELF_CONTAINED
- Update for VECTORCAST_DIR rollback
- Update for different drive than workspace
- Updates for duplicate results in multijob
- Revert changes from 0.43 and 0.44 to remove need for VECTORCAST_DIR. VECTORCAST_DIR will need to be defined when running Jenkins for now.
- Add support for new VCAST_RPTS_SELF_CONTAINED option added in VC19 SP2
- Add support for using Manage API to generate XML reports if available
- Correct variable expression on Linux platforms
- Updated scripts to handle compound only in compound case
- Updates for F+FC coverage and updating database pathing
- Updates for changed to aggregate coverage report metrics heading change
- Additional debug logging
- Updates for scripts not handling Ada operator "+"
- Update exception handling when using DataAPI
- Updated to support VectorCAST 2019 SP1
- Update to get complete display name for C++ functions
- Update to fix characters that need escaping in XML attributes
- Fix for disabled environments
- Fix for printing non UTF-8 compliant failure code
- Updates for using external CSS/images
- Removed xUnit dependency and converted to JUnit.
- Added better legacy support for VectorCAST installations that do not have their executables on the system PATH
- Removed all uses of the environment variable VECTORCAST_DIR. From now on it is assumed that VectorCAST executables are on the system PATH. Legacy support is still maintained for older versions of VectorCAST.
- Additional cleaning up of old files
- Updates for corner cases, verbose out issue, and cleaning up previous build's files
- Problem when function coverage enabled, but not function call
- Function coverage format incorrect in XML causing plugin to throw an error
- Added catch for additional licensing errors
- Added catch for all (E) Line: errors in the console log
- Fix for function and basis path coverage when using VectorCAST 2019
- Update to fix auto job updates (where path to Manage project was being removed)
- Update to make the management report generate for a cover project
- Fix for spurious newline characters in report title in XML for Jenkins with VectorCAST 2019
- Corrected missing " that may affect running multi-job on Linux
- Added support for generating reports using VectorCAST 2019
- Support overlapping version 17 Manage projects
- Updates to support long directory paths in VectorCAST/Manage reporting
- Support newer versions of xUnit plugin
- Support MultiJob plugin up to 0.29 and later, 0.30 onwards
-
Don't create intermediate CSV file for bad test case management report
-
Raise post-groovy alert for bad test case management report
- Improve support for unit without coverage, avoiding corrupt xml files
- Removed spurious " in Linux single job
- Corrected link from xUnit graph to report
- Correct regression with report naming for archived artifacts with shorter names
- Added environment variable (VCAST_VC_SCRIPTS) to provide optional source of vc_scripts
- Improve support for long Manage project names, environment names and compiler names
- Correct regression with windows variable names being used in Unix script
- Correct regression with missing space in commands for single job
- Option to set the name of the single job or multi-job (name is pre-pended to sub-job in the case of multi-jobs)
- Option to configure (at creation/update time) the node to run the single job or top-level multi-job on
- Allow license retries for single jobs
- Update summary/detailed text written by the groovy scripts
- Update to retry functionality to support jobs created with earlier plugin versions
- Update to store and use job details when auto-updating
- Added (optional) ability to retry a command if it fails due to unavailable license
- Added a job that can be used to update an existing multi-job
- Added support for new version of VectorCAST Manage that uses 2 levels instead of 4
- Improved groovy script to mark failing builds as failed rather than unstable
- Allow conversion script to accept report that has a missing or incomplete Function Coverage column
- Added --force option to use of --release-locks
- Added option to use either HTML or TEXT format for the build description
- Add execution report link to all test cases
- Added update to pulling in both the full report and incremental build report into the job build description
- Added update to pulling in both the full report and incremental build report into the job build description
- Always display the VectorCAST menu and leave permission checking/reporting to Jenkins
- Corrected processing checking if BUILD_URL has been set
- Corrected processing to support function and function call coverage
- Corrected typos in Diagnostics job and pattern for files to copy
- Add support for spaces in paths
- Add support to keep or clean the working directory
- Support added for multi-job with SCM and for calculating correctly aggregated coverage for the top-level display in the VectorCAST coverage plugin
- Initial release (no support for using SCM with multi-job)