10 biggest mistakes in using static analysis

Static analysis should help us to discover all the potential problems and raise a red flag whenever the codebase didn’t meet the rules. The biggest mistake you can do is producing warnings only and not let the build fail when violations against the rules were thrown. It leads to the immediate start off ignoring mode for anybody to pay attention to static analysis. When you are following a strategy that you can ship regardless of static analysis results, it’s bad, see mistake #10.

Engagement of the developers itself is essential for having static analysis effective. If the developers can’t affect the tools, are unable to cooperate with the rules, engagement will be slow, none or both. Developers are the ones who are taking care of code quality in everyday life, so you need to be friendly to allow them to participate in the rules, tools and overall process.

Setting up static code analysis and removal of the problems after decent work on the project is done, can be nightmare. It’s so much easier to set up tools for checking the quality of the code at the start of the project and enforce them from the outset. For pet projects probably doesn’t make sense to set up these tools but I have seen a pretty significant number of projects which started as pet projects and ended as a regular project. So maybe is the solution recognizing the right time for adding static analysis to the project.

If coding standards are speaking differently than rules in static analysis, it’s wrong and can be very misleading. These two guys should be playing the same game. Being careful about coding standards and having them aligned with static analysis it’s essential to have clear intention what’s useful for project and what’s not. Also lack of documentation about static analysis itself - how to

This lack of documentation is a problem because it can easily break the engagement by team members and proper use of all the tools.

You can believe me; there is always something special in your software which should be checked as early as possible by static analysis.

There are things which static analysis can’t discover - for an example configuration of the product, some specific runtime behavior. What you should provide is the good suite of quality assurance tests. These tests don’t test the functionality of the software, but they are ensuring that the codebase is following coding standards in the different way than static analysis. It’s common mistake to believe that static analysis is the only one tool which can get rid of all the problems.

When you are using some unified IDEs for developing software, you should provide default project configuration for such IDE. For example settitngs.jar for the Intellij IDEA. There is list of code inspections set for almost each programming language and environment, and you should carefully set all the rules and disable/enable the rules according to coding standards and static analysis. Within every new rule in static analysis or coding standards, these settings needs be aligned as well.

Code coverage is one of the tools from the static analysis toolset. It verifies that unit and integration tests check your production codebase. It’s common mistake if there isn’t set threshold meaning failing build whenever the code coverage is under the threshold. And it’s even worse not increasing of this threshold regularly. It doesn’t need to be increased by some percents but whenever we are increasing the code coverage, the threshold should be increased as well to ensuring that new code added won’t be covered by the tests.

Yes, there are pretty obvious possibilities that analysis will throw false positives. You should take care about all the violations, and those which are false positives should be handled inside code with some suppression with a comment why it’s false positives and why we can safely ignore them.

Fewer rules we are following are better than many that aren’t. Adding static analysis to the product after some development is tough. But not having static analysis on your side it’s worse. So if you haven’t sufficient time to incorporate a comprehensive set of tools and rules, start with some minimal number and work on improvements continuously. It’s even the standard for the static analysis for greenfield projects - static analysis is the continuous process, and you should take care about that in a regular manner. Adding rules, avoiding inconvenient ones, this is the product of a careful way of having ensured quality.

If you have not management on your side, there is no simple way how to have sufficient time for taking care about static analysis. It’s not the short run, it’s the long one and not the easy one. And time-consuming as well so you need to have management which has an understanding of return on such investment, have set allowed time, requirements and goals.