100% Test Coverage
Continuous Testing in the DevOps Age
Whether building software products for customers or developing and configuring software in-house, today’s software development organizations are investing significantly in automated and continuous testing.
However, they are often not able to achieve 100% application test coverage manually or automated. Many problems are not uncovered or identified until a complex production situation arises that was not included in test planning.
Billing System Testing Example
How do you test your billing system?
Typical billing systems have a 1-4% error rate – but you do not know which bills are wrong. It could be your largest customer. It could be the local government. A wrong bill is a customer service nightmare for everyone.
Frequently there are actually half-a-dozen or more billing applications cobbled together through corporate acquisitions and mergers.
The real “gotcha” is change. The days of static pricing and rate structures that stay the same for years is long gone. Today’s business requirement is a billing capability that handles continuously changing prices, rate structures, promotions, and customization for individual customers.
Managing your billing system(s) in a dynamic, changing business environment is hard, it is expensive, and it produces more errors than anyone wants to admit.
A Transformative Approach To Testing
Cloud-Sliver provides a safe and efficient path to test and transform ancient billing system with no business disruption.
Cloud-Sliver has helped partners deploy a new species of app for testing.
This testing app replicates the billing calculation functionality of your billing system runs in parallel with your existing billing system. Nothing changes for your existing system-of-record.
Cloud-Sliver’s parallel billing App-Sliver reconciles EVERY line item on every bill to make sure the bills are 100% accurate.
If two billing applications, written with different technologies, both come up with the same amount for a customer’s bill, the odds of the bill being wrong are vanishingly small.
Out goes that 1-4% error rate. Away go those customer service problems.
The parallel billing App-Sliver also provides a real time customer portal where customers can interact with information that impacts their bills instantly. They can do “what if” analysis and receive notifications and alarms.
You now have a billing system that is an asset for deepening your customer relationships, not a liability.
Three Levels of Validation
In a typical testing deployment scenario, our testing partner provides three levels of independent validation:
Independent Validation of Requirements
Sometimes the application portfolio that Cloud-Sliver tackles has little or no source code. In other instances, the vendor is no longer supporting or available to maintain the legacy app.
Far too often, what the business users think the legacy app is doing and what it is actually doing are remarkably different.
Our independent validation partners provide QA as a service to validate that the capabilities Cloud-Sliver is providing are indeed what needs to be delivered.
This is accomplished via systematic audit of application requirements for testability and comparative analysis to the current system, a review of current QA processes and test assets, and application inspection.
Continuous Parallel Testing and Coverage
One of the primary advantages of using Cloud-Sliver in the development of enterprise software is its ability to rapidly process significant amounts of data.
Our validation partners use this feature to run entire days, months, or years of input data through the system under test and Cloud-Sliver in parallel.
Here, two systems, with different code bases, are performing what should be identical tasks. The independent testing as a service vendor measures every transaction to determine, line by line if they are identical.
This is a two step process that first validates that the Cloud-Sliver application accurately matches the existing legacy system. Through this process multiple years of data are run and reconciled for correctness.
Once the Cloud-Sliver application is validated, the second step is to make the suggested software changes to both the legacy system and the Cloud-Sliver application.
Then the data sets are re-run looking for expected and unexpected outcomes. The outcome differences are analyzed and confirmed for defects.
Our experience has shown a 100% accuracy rate in identifying production defects when there is a mismatch between the two systems.
When calculations from two independent systems, each with different code bases reach the same conclusion, the probability that the final product — here a telco or utility bill — is correct approaches 100%.
Validating a System of Record is Correct with New Features or Data
As enhancements to the system of record become available, Cloud-Sliver builds them into a version of the parallel billing system.
Because the Cloud-Sliver technology enables complex applications to be modified, configured, and delivered rapidly, it is practical to construct parallel systems for each set of feature enhancements.
With this approach, every line item is reconciled for every calculation in real time.
The system of record can be checked against an established parallel system and also reconciled against one or more additional parallel systems to ensure that all of them agree.
This process is continuous.
The parallel systems are being tested 24/7.
When an anomaly appears, there is immediate feedback to the customer.
As each new system is validated and then moved over to the role of “primary parallel system” it is possible to introduce significant updates into a production system and test them at the level of every line item, on every bill, for every customer for 100% test coverage of the output of the updated system.
Since Cloud-Sliver applications are more efficient than traditional applications, the hardware requirements are minimal.
For the first time it is economical to run multiple systems in parallel without breaking the program budget.
It also makes it possible to do extensive retrospective testing – for example, running three years of transactions through each of the parallel systems on a regular basis to find anomalies.
Cloud-Sliver has developed a new way to implement and deliver enterprise class applications.
With independent testing partners, we have jointly created a new way to test and validate critical applications on an ongoing continuous basis to deliver a higher level of quality than what has been previously possible.