01 - Programming (EN)01e - Programming miscellanea (EN)

Static analysis of Cobol and RPG source code on IBM i with SonarQube

Last Updated on 4 January 2023 by Roberto De Pedrini

In computer science, static program analysis (or, simply, static analysis) is the analysis of source code performed without executing programs (as opposed to dynamic analysis of programs, which is performed during their execution).

This process provides an understanding of the structure of the source code and can help ensure that the code adheres to industry standards (compliance, safety, and security).

Some tools can help programmers and developers perform static analysis by inspecting code for potential programming errors (bugs), coding standard violations, security vulnerabilities, and more.

SonarQube

SonarQube is capable of analyzing different languages depending on the edition in use. Cobol and RPG for IBM i are only available in Enterprise and Data Centers editions.

Before you can analyze source code, SonarQube must, of course, have access to it. SonarQube is unable to access members of your physical source files stored on the IBM i library file system, as generically stated in the “Source code extraction” section of SonarQube’s Cobol language documentation:

In order to analyze your source code with SonarQube you need to first extract it onto a filesystem. You can use your own tool or an open-source tool; SonarSource does not provide any connectors or source code extraction tools.

On IBM i systems, a filesystem accessible from SonarQube is the Integrated File System. All we have to do, then, is copy our source code to IFS and make it available to external systems (specifically, the system on which the code analysis with the sonar-scanner command will be run, see below).

To export the source code to IFS, we can use the CPYSRC2IFS command while to make it available to external systems we can use the IBM i NetServer.

CPYSCR2IFS and IBM i NetServers

CPYSRC2IFS is a command that allows source code to be copied from physical source file members to stream files in IFS directories where applications from other platforms, via network sharing provided by the NetServer component of IBM i, can access them.

CPYSCR2IFS and IBM i NetServer provide the bridge between the traditional IBM i development environment (PDM/SEU) and open tools like SonarQube.

Once the source code is made available, analysis can proceed.

Using SonarQube

Before starting a scan, you need to create a project.

The project can use your reference DevOps platform (Azure, Bitbucket, GitHub, or GitLab) as a source code repository or use a local repository (this is our case):

Here is some general information about the project:

Select the type of source code repository (in our case, it is a local repository):

Create the project identification token:

After pressing on “Generate” the identification token is displayed:

Provide additional project information needed by the analysis tool:

Following the indications entered, SonarQube provides the sonar-scanner command to be executed on a Windows system to scan the source code for the newly created CBLPRJ01 project:

sonar-scanner.bat -D"sonar.projectKey=CBLPRJ01" -D"sonar.sources=." -D"sonar.host.url=http://{SonarQube_hostname}:{port}" -D"sonar.login=sqp_2508f4480e88c8bcc75aa7ec9ed8a41aa75c04e1"

If we had selected Linux as the operating system then the suggested command would be as follows:

sonar-scanner \
-Dsonar.projectKey=CBLPRJ01 \
-Dsonar.sources=. \
-Dsonar.host.url=http://{SonarQube_hostname}:{port} \
-Dsonar.login=sqp_2508f4480e88c8bcc75aa7ec9ed8a41aa75c04e1

{SonarQube_hostname} is the hostname or IP address of your SonarQube instance and {port} the port the SonarQube instance is listening on (default port is 9000).

By clicking on the official documentation of the scanner link on the screen, we will be redirected to the download page of SonarScanner, the SonarQube code scanning and analysis tool, where we will be able to download the version based on the operating system on which it will be run:

In our case, the platform used for source code analysis is Windows 64-bit and the downloaded SonarScanner version is 4.7.0.2747.

Note. The SonarScanner download page also contains helpful guidance on configuring the project (sonar-project.properties file), on running SonarScanner from the zip archive or from Docker image, on possible alternatives to using the sonar-project.properties file and other information.

At the end of the download, we will have to extract the contents of the archive and set the path of the bin directory just extracted in the PATH environment variable:

  • On Windows systems, you can set the Path environment variable from the Windows command prompt using the set or setx command. The set command just sets the environment variable for the current session. The setx command sets it permanently, but not for the current session (if you want to set it for current and future sessions, use both setx and set). Alternatively, you can also set the environment variable from the Power User task menu by right-clicking the Windows Start menu then selecting “System”, then “Advanced system settings”, “Environment variables” and selecting the “Path” environment variable then pressing on “Edit”.
  • On Linux systems, you can add the directory /var/opt/sonar-scanner-4.7.0.2747-linux/bin to the PATH environment variable:
$ export PATH=$PATH:/var/opt/sonar-scanner-4.7.0.2747-linux/bin

$ ln -s /var/opt/sonar-scanner-4.7.0.2747-linux/bin/sonar-scanner /usr/local/bin/

Preparing for source code analysis

In the article, we assume that our source code (programs and copies) is stored in stream files on IFS in the Src and Cpy directories and that the directory containing them is shared via NetServer.

Source code analysis with SonarScanner

sonar-scanner is the command we can use to analyze our source code and send the result of the analysis to SonarQube. Once the tool is installed, we need to configure it by creating the sonar-project.properties file at the root level of the project:

The sonar-project.properties file contains the SonarQube instance hostname, the project key to identify the project in SonarQube, and source file information (such as file extension suffixes, source location, etc.). Let’s see an example:

# ----------------------------------------
# SonarQube Host
# Note: Default port is 9000
# ----------------------------------------
sonar.host.url=http://{SonarQube_hostname}:{port}

# ----------------------------------------
# Project identification
# Note: The key is generated by the SonarQube project admin.
# ----------------------------------------
sonar.projectKey=CBLPRJ01
sonar.projectName=CBLPRJ01

# ----------------------------------------
# Cobol dialect
# ----------------------------------------
# Cobol/400
sonar.cobol.dialect=ibm-cobol/400
# Cobol ILE
#sonar.cobol.dialect=ibm-ile-cobol

# ----------------------------------------
# Language-dependent attributes
# ----------------------------------------
#sonar.cobol.sourceFormat=fixed
#sonar.cobol.tab.width=8

# ----------------------------------------
# Source information
# Note: This assumes that your source is downloaded into a directory
# named "Src" 
# ----------------------------------------
sonar.sources=./Src
sonar.cobol.file.suffixes= CBL,SQLCBL,CPY

# ----------------------------------------
# Copy information
# Note: This assumes that your source is downloaded into a directory
# named "Cpy"
# ----------------------------------------
sonar.cobol.copy.directories=./Cpy
sonar.cobol.copy.suffixes= CBL,SQLCBL,CPY

# ----------------------------------------
# Generic attributes
# ----------------------------------------
# Excludes all files from the test coverage metrics
sonar.coverage.exclusions=**/*.*

Note. Code coverage describes the percentage of code affected by automated tests. In other words, it controls which pieces of code are executed during testing and which are not. In essence, it is a metric used to verify the quality of tests, because it represents the percentage of production code tested and executed. SonarQube is used in integration with JaCoCo, a free code coverage library for Java. Since this feature is not used I prefer to turn it off to avoid annoying signaling while browsing the source code from the SonarQube web interface.

Once the sonar-project.properties file is configured, we can scan the source code using the sonar-scanner command from the command prompt window:

> m:
> cd \CBLPRJ01
> sonar-scanner.bat -D"sonar.login=sqp_2508f4480e88c8bcc75aa7ec9ed8a41aa75c04e1"

In our case, m: is the disk pointing to the network share from the IBM i system where the source code to be analyzed is located, and exported with the CPYSRC2IFS command.

Note. The configurations in the sonar-project.properties file can be passed as parameters to the sonar-scanner command as follows, thus eliminating the need to use the file:

> sonar-scanner.bat -D"sonar.login=sqp_2508f4480e88c8bcc75aa7ec9ed8a41aa75c04e1" -D"sonar.host.url=http://{SonarQube_hostname}:{port}" -D"sonar.projectKey=CBLPRJ01" -D"sonar.projectName=CBLPRJ01" -D"sonar.cobol.dialect=ibm-cobol/400" -D"sonar.sources=./Src" -D"sonar.cobol.file.suffixes= CBL,SQLCBL,CPY" -D"sonar.cobol.copy.directories=./Cpy" -D"sonar.cobol.copy.suffixes= CBL,SQLCBL,CPY" -D"sonar.coverage.exclusions=**/*.*"

Below is the output of the sonar-scanner command:

INFO: Scanner configuration file: C:\MyPrograms\sonar-scanner-4.7.0.2747-windows\bin\..\conf\sonar-scanner.properties
INFO: Project root configuration file: m:\CBLPRJ01\sonar-project.properties
INFO: SonarScanner 4.7.0.2747
INFO: Java 11.0.14.1 Eclipse Adoptium (64-bit)
INFO: Windows 10 10.0 amd64
INFO: User cache: C:\Users\xxxxxxxx\.sonar\cache
INFO: Scanner configuration file: C:\MyPrograms\sonar-scanner-4.7.0.2747-windows\bin\..\conf\sonar-scanner.properties
INFO: Project root configuration file: m:\CBLPRJ01\sonar-project.properties
INFO: Analyzing on SonarQube server 9.7.1.62043
INFO: Default locale: "it_IT", source code encoding: "windows-1252" (analysis is platform dependent)
INFO: Load global settings
INFO: Load global settings (done) | time=66ms
INFO: Server id: 147B411E-AYRcQ0q0f13ET8K16xxa
INFO: User cache: C:\Users\xxxxxxxx\.sonar\cache
INFO: Load/download plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=55ms
INFO: Load/download plugins (done) | time=7608ms
INFO: Loaded core extensions: developer-scanner
INFO: Process project properties
INFO: Process project properties (done) | time=23ms
INFO: Execute project builders
INFO: Execute project builders (done) | time=2ms
INFO: Project key: CBLPRJ01
INFO: Base dir: M:\CBLPRJ01
INFO: Working dir: m:\CBLPRJ01\.scannerwork
INFO: Load project settings for component key: 'CBLPRJ01'
INFO: Load project settings for component key: 'CBLPRJ01' (done) | time=39ms
INFO: Load project branches
INFO: Load project branches (done) | time=22ms
INFO: Load project pull requests
INFO: Load project pull requests (done) | time=11ms
INFO: Load branch configuration
INFO: Load branch configuration (done) | time=3ms
WARN: SCM provider autodetection failed. Please use "sonar.scm.provider" to define SCM of your project, or disable the SCM Sensor in the project settings.
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=69ms
INFO: Load active rules
INFO: Load active rules (done) | time=3211ms
INFO: Load analysis cache
INFO: Load analysis cache (404) | time=9ms
INFO: Load project repositories
INFO: Load project repositories (done) | time=28ms
INFO: Indexing files...
INFO: Project configuration:
INFO: 1 file indexed
INFO: Quality profile for cobol: Sonar way
INFO: ------------- Run sensors on module CBLPRJ01
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=55ms
INFO: Sensor CobolSquidSensor [cobol]
INFO: About to analyse COBOL Programs using 'ibm-cobol/400' dialect and 'fixed' source code format.
INFO: Found empty cache on server
INFO: 1 source file to be analyzed
INFO: 1/1 source file has been analyzed
INFO: Analysis time: 454ms
INFO: 1 source file to be highlighted and measured
INFO: 1/1 source file has been highlighted and measured
INFO: Decoration time: 58ms
INFO: Cache: 0/1 hits, 0/1 decorator hits, 12226 bytes
INFO: Sensor CobolSquidSensor [cobol] (done) | time=514ms
INFO: Sensor C# Project Type Information [csharp]
INFO: Sensor C# Project Type Information [csharp] (done) | time=1ms
INFO: Sensor C# Analysis Log [csharp]
INFO: Sensor C# Analysis Log [csharp] (done) | time=10ms
INFO: Sensor C# Properties [csharp]
INFO: Sensor C# Properties [csharp] (done) | time=0ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=2ms
INFO: Sensor Text Sensor [text]
INFO: 1 source file to be analyzed
INFO: 1/1 source file has been analyzed
INFO: Sensor Text Sensor [text] (done) | time=7ms
INFO: Sensor VB.NET Project Type Information [vbnet]
INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=1ms
INFO: Sensor VB.NET Analysis Log [vbnet]
INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=10ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=10ms
INFO: Sensor CSS Rules [javascript]
INFO: No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
INFO: Sensor CSS Rules [javascript] (done) | time=2ms
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=3ms
INFO: Sensor Serverless configuration file sensor [security]
INFO: 0 Serverless function entries were found in the project
INFO: 0 Serverless function handlers were kept as entrypoints
INFO: Sensor Serverless configuration file sensor [security] (done) | time=9ms
INFO: Sensor AWS SAM template file sensor [security]
INFO: Sensor AWS SAM template file sensor [security] (done) | time=0ms
INFO: Sensor AWS SAM Inline template file sensor [security]
INFO: Sensor AWS SAM Inline template file sensor [security] (done) | time=0ms
INFO: Sensor javabugs [dbd]
INFO: Reading IR files from: m:\CBLPRJ01\.scannerwork\ir\java
INFO: No IR files have been included for analysis.
INFO: Sensor javabugs [dbd] (done) | time=3ms
INFO: Sensor pythonbugs [dbd]
INFO: Reading IR files from: m:\CBLPRJ01\.scannerwork\ir\python
INFO: No IR files have been included for analysis.
INFO: Sensor pythonbugs [dbd] (done) | time=6ms
INFO: Sensor JavaSecuritySensor [security]
INFO: Reading type hierarchy from: m:\CBLPRJ01\.scannerwork\ucfg2\java
INFO: Read 0 type definitions
INFO: Reading UCFGs from: m:\CBLPRJ01\.scannerwork\ucfg2\java
INFO: No UCFGs have been included for analysis.
INFO: Sensor JavaSecuritySensor [security] (done) | time=7ms
INFO: Sensor CSharpSecuritySensor [security]
INFO: Reading type hierarchy from: m:\CBLPRJ01\ucfg_cs2
INFO: Read 0 type definitions
INFO: Reading UCFGs from: m:\CBLPRJ01\ucfg_cs2
INFO: No UCFGs have been included for analysis.
INFO: Sensor CSharpSecuritySensor [security] (done) | time=3ms
INFO: Sensor PhpSecuritySensor [security]
INFO: Reading type hierarchy from: m:\CBLPRJ01\.scannerwork\ucfg2\php
INFO: Read 0 type definitions
INFO: Reading UCFGs from: m:\CBLPRJ01\.scannerwork\ucfg2\php
INFO: No UCFGs have been included for analysis.
INFO: Sensor PhpSecuritySensor [security] (done) | time=3ms
INFO: Sensor PythonSecuritySensor [security]
INFO: Reading type hierarchy from: m:\CBLPRJ01\.scannerwork\ucfg2\python
INFO: Read 0 type definitions
INFO: Reading UCFGs from: m:\CBLPRJ01\.scannerwork\ucfg2\python
INFO: No UCFGs have been included for analysis.
INFO: Sensor PythonSecuritySensor [security] (done) | time=4ms
INFO: Sensor JsSecuritySensor [security]
INFO: Reading type hierarchy from: m:\CBLPRJ01\.scannerwork\ucfg2\js
INFO: Read 0 type definitions
INFO: Reading UCFGs from: m:\CBLPRJ01\.scannerwork\ucfg2\js
INFO: No UCFGs have been included for analysis.
INFO: Sensor JsSecuritySensor [security] (done) | time=4ms
INFO: ------------- Run sensors on project
INFO: Sensor Analysis Warnings import [csharp]
INFO: Sensor Analysis Warnings import [csharp] (done) | time=1ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=16ms
INFO: SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
INFO: CPD Executor Calculating CPD for 1 file
INFO: CPD Executor CPD calculation finished (done) | time=10ms
INFO: Analysis report generated in 168ms, dir size=241.3 kB
INFO: Analysis report compressed in 3084ms, zip size=42.9 kB
INFO: Analysis report uploaded in 63ms
INFO: ANALYSIS SUCCESSFUL, you can find the results at: http://{SonarQube_hostname}:{port}/dashboard?id=CBLPRJ01
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://{SonarQube_hostname}:{port}/api/ce/task?id=AYSaBuDP2lfWAptP6Djs
INFO: Analysis total time: 12.356 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 22.474s
INFO: Final Memory: 37M/124M
INFO: ------------------------------------------------------------------------

Once the scan is complete, you can navigate through the project from the SonarQube web interface and view the results (the sonar-scanner output provides the URL to copy into the browser, in our case: http://{ SonarQube_hostname }:{ port }/dashboard?id=CBLPRJ01):

The source code passed the analysis but work needs to be done on reliability (5 bugs found).

Using static code analysis tools you can ensure that your code complies with style and coding guidelines and, more importantly, that it is “clean” and bug-free before it is deployed.

Appendix: SonarQube project configuration for RPG language

For the RPG language, the sonar-project.properties file should look like this:

# ----------------------------------------
# SonarQube Host 
# Note: Default port is 9000
# ----------------------------------------
sonar.host.url=http://{SonarQube_hostname}:{port}

# ----------------------------------------
# Project identification
# Note: The key is generated by the SonarQube project admin.
# ----------------------------------------
sonar.projectKey=RPGPRJ01
sonar.projectName=RPGPRJ01

# ----------------------------------------
# Language-dependent attributes
# ----------------------------------------
sonar.rpg.leftMarginWidth=0

# ----------------------------------------
# Source information
# Note: This assumes that your source is downloaded into a directory
# named "Src" 
# ----------------------------------------
sonar.sources=./Src
# sonar.rpg.suffixes=.rpg,.rpgle,.sqlrpgle,.RPG,.RPGLE,.SQLRPGLE

# ----------------------------------------
# Generic attributes
# ----------------------------------------
# Excludes all files from the test coverage metrics
sonar.coverage.exclusions=**/*.*
Related Posts
DB2 for i SQL – String Manipulation – POSSTR-LOCATE-LOCATE_IN_STRING (EN)

Introduction Often, in our applications, we need to work with text strings, and DB2 SQL can come in very useful Read more

DB2 for i – FAQ & Howtos (EN)

DB2 Database and SQL ... maybe the most important things on IBM i platform: here's a collection of FAQs, tips Read more

IBM i 7.4 Announcement (En)

Comes directly with the Easter egg this IBM announcement for the news of the IBM i 7.4 version, iNext version Read more

Generated Always Columns (EN)

Introduction "Generated Always Column": are columns, table fields, filled by DB2 engine: something like columns with a default value but Read more

About author

IBM i System Administrator

Leave a Reply

Your email address will not be published. Required fields are marked *