A recent article from fortune.com implies that perhaps the Russian race to be the first to announce an approved vaccine for COVID-19 might circumvent the level of rigorous safety and efficacy testing that is typically done in other countries. At the risk of taking the analogy of human and computer viruses way too far, I find it interesting to compare the concerns over this “rushed unorthodox approach” to the way that software vendors often rush out software patches after a new vulnerability is detected. Should we expect some sort of careful “Phase 3” testing for a new software patch?
Today, when a new software vulnerability is found, vendors rush to develop and distribute updates for users to patch their systems and make them “immune” to the threat. These updates get tested, approved, and deployed quite quickly using a mature software validation process, but the scope of that validation is confined to a pre-defined set of acceptance criteria. The resulting patch is shipped out and network operators are expected to trust that it will be “safe” to deploy.
There are many reasons why critical infrastructure operators are often slow to apply patches to their systems. One often cited reason is that they are uncertain that the update will “work” in their specific environment without errors or issues that cause additional downtime; or worse, cause damage to their operational systems. They often will avoid applying a patch unless they see an explicit threat to their network. If they do decide to apply a patch, they will do extensive in-house testing before they trust it on their production systems. They know that a critical failure in a nuclear power plant or power distribution network could lead to loss of life and they need to take extra precautions.
This is a little like asking physicians to pull a Jonas Salk and “test” a new vaccine themselves before they allow their own patients to be inoculated. In most developed countries we have created a robust and extensive centralized process such that once a vaccine is approved it can reasonably be assumed to be safe and effective. Regional doctors are provided with detailed guidance on where and how to use the vaccine, potential side-effects, allergies, and complications. This information allows doctors to make an informed decision about who should receive the vaccine and who deserves special care or attention after administration. They typically don’t spend a lot of time doing their own in-house testing to see if the vaccine is safe.
While most software updates and patches do come out with release notes that might call-out potential issues, these are rarely standardized. As a result, many network operators will go through the slow, tedious process of doing offline testing before they trust a new patch. Would that change if they were provided with clear, unambiguous guidance from a trusted authority that had already run the code through the equivalent of a “Phase 3” trial? Would a Software Bill of Materials (SBoM) help to automate the process of detecting potential “side-effects” that a new patch might cause in their systems?
I believe that the answer to both of these questions is yes, but it will require the cooperation of software vendors who make patches and critical infrastructure network operators who need to trust and apply them. It will also require the development of rigorous testing and documentation standards, comparable to the FDA standards for vaccine trials. And, it will likely require the introduction of a standard SBoM as a trusted and reliable source of information about the components that are present deep within a software package.
So why don’t we raise safety concerns when a quickly developed patch is rushed out to critical infrastructure networks for immediate deployment? If there are valid concerns over the “rushed unorthodox approach” to Russian COVID-19 vaccine development, then why doesn’t the same logic apply when we rush to deploy a software patch for a safety-critical system before it has been rigorously tested, validated, and documented? If software patches are the “vaccines” for cyber vulnerabilities, perhaps critical infrastructure operators should require a parallel set of “Phase 3” trials and standardized release documentation that we expect from a new medical vaccine.