Author: Michael Baron, Emenda<\/p>\n
Three key areas of the software verification process (release management, technical debt and management reporting) often go underutilized and unvalued. With increasing numbers of free and open source tools able to improve productivity, reduce costs and provide an insight into a projects' development cycle, this article looks at what is required of a modern verification process and how to make the most of what is available.<\/strong><\/p>\n When Continuous Integration (CI) tools, such as Jenkins, first appeared in the industry the focus was on build management, however since then the industry has transitioned to the more complicated and involved task of release management and associated Development Operations (DevOps).<\/p>\n Release management concerns the process from development to release, with optimal processes being automated through CI, with constant feedback to both the developer(s) and Quality Assurance (QA) teams. A well-designed release management process would mean that if one line of code was changed then the automation system should be able to handle the verification of the new release quickly and with no added human interaction. We should also aim to design a release management process that; is simple in design, functions as quickly as possible to reduce the time to release, is reliable, should handle the configuration of the build automatically, optimize the developers' time, optimize the quality assurance activities, and enforce software quality across the project. The result of a well-functioning release management system is reduced time for product verification and shorter time to market.<\/p>\n To automate the whole release lifecycle, development teams have been adding in analysis and testing tools to achieve the quality assurance and verification of the product pre-release, thus, ensuring no surprises downstream. Generally, these tools are most effective when used in a 'per check-in' process but with an increased number of activities being integrated with the automation system, and combined with ever growing codebase sizes, the result is often a shift from fast 'per check-in' runs to periodic runs, every few hours, nightly or even in some cases over a weekend. The net result is that the release cycle is then slowed down significantly.<\/p>\n Some other common limitations when adding automated analysis and testing tools into the release cycle are:<\/p>\n Making most of CI, and tools like Jenkins, allows us to outsource the test and analysis execution, and provide near real time feedback to the developer of the result. But, to succeed, additional test and analysis tools and steps also need to seamlessly connect into the CI platform provide appropriate and relevant data, and be both fast and scalable.<\/p>\n In addition, to make the most of Jenkins and SonarQube for release management, defining relevant quality gates and thresholds, ensures that we have a clear understanding of the progress towards the final release and allows us to continually ensure that we are compliant with the required quality standards \u2013 continuous compliance.<\/p>\n In an ideal world, there wouldn't be any technical debt, however in practice that isn't realistic. Managing it effectively is the next best option.<\/p>\n Like other forms of debt, the cost of technical debt increases the longer it is held, and, of course, higher than if it were never acquired in the first place. Thus, preferably, the debt is resolved as soon as possible and by placing emphasis on providing fast feedback, closer to the development, when the debt is increased, it ensures that the costs are managed as best as they can be.<\/p>\n For example, if a defect is introduced into the code and the developer fixes it before check-in the relative technical debt is very low as it has only resulted in their time to re-work the code in a context they understand. If, however, the defect makes its way into the system and is discovered before release in the quality assurance phase, the cost of the debt is both to recode and retest. Finally, if the defect makes its way into the release and it is discovered by the customer, the debt value is significantly higher due to it requiring recoding, possible branching from in development code, retesting and any financial implications or reputational costs of the software failure with the customer.<\/p>\n Strategies such as architectural de-coupling and change-based testing can help to reduce the impact of technical debt. For example, if build, test and analysis tools and processes can be configured to rebuild, retest and re-analyze only the modules and packages that need be \u2013 rather than everything \u2013 then removing technical debt becomes cheaper and thus the technical debt itself is reduced.<\/p>\n In addition to the CI platforms, such as Jenkins, tools like Docker, can help enable quicker analysis and configuration management.<\/p>\n However, there will still be technical debt, so using visualization tools, such as SonarQube, that will help development and QA teams understand and share the technical debt results is vital. SonarQube also provides impact metrics to help find the \u201elow hanging fruit\u201c enabling better allocation of resources for debt reduction.<\/p>\n Management Reporting<\/strong><\/p>\n What is required from management reporting is an instant answer to \u201eCan the product be released?\u201c If the answer is no, then clear details on the metrics that failed are needed, along with technical debt values to enable estimation of release dates and the resourcing required to get there.<\/p>\n Typically, the result from CI runs is used as the basis for many of these reports, and the results of each step and each tool within the cycle is collated in the relevant project page, for review, or alternatively failures are raised within a bug tracking system.<\/p>\n There are a wide range of limitations with using this type of reporting, for example:<\/p>\n A dedicated \u201edash-boarding tool\u201c, such as SonarQube, will pool together all the metrics from the various test and analysis tools, helping to mitigate these limitations. It also allows the thresholds and quality gates to be defined in one place and outputs one pass or fail result for clarity. The focus of SonarQube is on the results related to the quality (and security) of the code, meaning that it provides management and QA teams with only the data that they need, and not on build failures, etc.<\/p>\n SonarQube calculates technical debt from supplemented information and metrics provided by the test and analysis tools that it connects to, and allows for totals and breakdowns to be calculated across the full data set. Management and QA teams can then drill down into the results to focus in on the causes.<\/p>\n As the data from the external tools are collated and processed into a single project view, a change in verification tools generally has a lesser impact on the top-level results. The central dashboard also provides the ability to generate one overall compliance report based on the metrics from multiple tools.<\/p>\n Michael Baron works as a technical consultant at Emenda, and is a long-term contributor of plugins to the Jenkins platform. Michael is experienced in providing DevOps consultancy to large businesses on the setup, configuration, and process design for the implementation of verification and validation tools within continuous integration (CI) and delivery (CD) development cycles. Michael has been a long-term supporter and evangelist for the utilization of free and open source tools within the software development life cycle where possible and sensitive, and he is also involved in numerous large-scale projects for Emenda around enterprise-wide technical debt (and technical wealth) reporting to board-level management teams.<\/p>\n Download the article as a PDF<\/strong><\/a><\/p>\n Do you want to bring yourself up to date with the latest technology?<\/strong><\/p>\n Then find out more\u00a0here<\/strong>\u00a0<\/a>MircoConsult offers training courses\/seminars\/workshops and individual coaching on the topic of Open Source \/ Embedded Software Engineering.<\/p>\n Training & coaching on the other topics in our portfolio can be found here.\u00a0here<\/a>.<\/strong><\/p>\n Valuable expertise in the field of Open Source \/ Embedded Software Engineering is available.\u00a0here\u00a0<\/strong><\/a>Available for you to download free of charge.<\/p>\nRelease Management<\/h2>\n
\n
Technical Debt<\/h2>\n
\n
Author<\/h2>\n
\nOpen Source \u2013 our training & coaching<\/h2>\n
\nOpen Source \u2013 Expertise<\/h2>\n