Inverting the Test Pyramid, by Joel Masset

Great Article from Joel Masset, Global Head of Product Assurance and Chief Quality Officer in the Financial Services Industry

invertedpyramid
Inverjoelmassetting the Test Pyramid
by Joel  Masset
https://www.linkedin.com/pulse/inverting-test-pyramid-joel-masset

Although most organizations focus on what it means for development teams to be agile, what that changes for them, and how to make it happen, testing is still very important in an agile delivery model. It is even more critical than in a traditional model.

I want here to highlight the big change that agility represents from a testing point of view. This is highly inspired from Mike Cohn’s well known test pyramid. However, I thought it was worth sharing this once more, as it is so close to what many major software delivery organizations need to go through nowadays.

In a traditional software delivery approach (waterfall, V cycle, iterative V cycle, washing machine…), all activities are serialized. Business requirements, design, development, test, debugging, […], release. The focus is put on finding bugs.

InvertedPyramid1

This leads to very little test effort to happen at the beginning of the cycle, and massive focus at the end, after developers have integrated their components into a product.

Roles are very clear and distinct. Developers’ role is to code and debug, testers’ is to find bugs, and check they are fixed.

Automated tests developed after the integration has happened will be essentially based on the UI. Their development cost is very high and since they are very fragile, maintenance costs are very high too. Any update to the code is likely to break the tests, which will require automation code to be updated prior to being ran. Since most of the testing focus is happening so close to the expected delivery date, there is a lot of pressure on the team to give its test campaign results. In most cases, it results into massive manual testing. In the best case, automated test development will happen in parallel, so that these can run post the release, but very regularly, the team will just give up on automating.

As a result, automation is extremely painful, costly, and inefficient, with a very low Return On Investment. The manual testing in this model is very costly, and happening under stress.

Don’t get me wrong, I am not saying this model cannot work, I have myself been using it for years. But oh my god, is it stressful…
Agile testing takes the opposite approach, by inverting the test pyramid.

AgileAutomatedPyramid

The focus is now put on preventing bugs from existing. This means that most of the test effort will happen at the beginning, at the code and UI level. In this context, automation will be extremely efficient. Unit tests will be written alongside development, or even before the actual code is written or updated (this is the Test Driven Development model).

Acceptance tests will be created at the API layer level during development and integrated in the continuous build and integration tools and processes.

At the time the developer checks the code in, it has already been successfully tested. Post-build and integration tests will automatically be ran and results checked.

Since all those tests happen during the sprint, a failed test will immediately lead to a code change fixing the issue. Not only the cost of fixing a problem is much smaller at this stage, but the risk associated to re-opening code that was already pushed, days or weeks ago is gone. The team then rarely has to switch context from the current version of the code to an older one. This is a much more reliable process.

Some testing will still be required after the integration steps. Some will still be automated at the UI level. This represents very boring and repetitive tests like links checks, or basic user workflow steps. The essential part of testing at that stage is exploratory testing. Based on end user workflows, these are tests that only a human person can do. But the effort will be limited here.

In this new approach, you end up with developers and testers both testing and coding together. A much higher synergy, and very little stress, because the delivery process is made much more reliable.

Results are impressive on the Quality of the software, the predictability of the releases, and as well on the motivation of the teams.

This change requires a big cultural change obviously, hence very high senior leadership, this will certainly be the focus of a future article…

Bill Fulbright
770-880-0959
Bill@2100solutions.com

Advertisements

Vendor Management Service Transformation: Entry 1 – Re-Factoring, Businss Architecture

metodo_pratiche_agile_chart_manifesto_itaEntry 1    3.22.2015

I recently was invited to join a project for a Vendor Management Service (VMS) in Mid-March 2015. The project is to provide in Phase 1 a re-factoring of our Client’s code by replacing the hardcoded middleware with services, and adding new client facing features, along with a new UI. All needing new documentation, of which there is now verylittle.

Our client provides a turnkey service for managing IT vendors who need to outsource their HR, Recruiting, Accounting, and Financial Services for this aspect of their business.

My role is to document the present legacy Business Processes, the new Processes, the new Services and the newly re-factored APIs, processes and added features by providing the Requirements, Use Cases, Workflows and Processes.

The leadership on this project is not only setting the pace, but shining a bright light into the future vision for this client, and for the VMS industry. It is a privilege to work with them.

 

Presently, I am awash in the project ramp-up and assimilation of the many layers, features and infrastructure required to successfully launch a program as complex as this.

We have two teams: one is onsite with FTE EEs of the customer, and a fly-in contingent of our leadership. The other is an offsite team in Atlanta, that is providing an AGILE based component for delivery of the new code which provides the new Service APIs and integration; as well as Leadership, Business Architecure, Process Articulation & Documentation. The client will observe the present SDLC based approach for now.

We have defined the primary users and their roles, the features – both new and old – associated with their roles. The functionality of these features some of which, for now, will remain as legacy, while others are new. There are around 400 of these. Some are Epic, requiring some of the features to support the workflows.

For the new and replacement pieces (in AGILE) we have defined the primary “Day in the Life” from the “need to the ass in the seat” E2E process to establish a critical Happy Path. Variations and UCM will be modeled based upon this primary structure.

The software and coding will be the same, albeit updated. The specific usage of the system will vary based upon the needs and systems of client-users of this system.

The SDLC pieces for things like the DATA, and QA will be driven from the client sites.

I will be updating this log at various points along the way…. so STAY TUNED!!

Bill Fulbright

Words That Sell Software Testing

Here is a very helpful article by Simon Knight, who has done his homework on powerful words that can help you in your  career:

Words That Sell Software Testing
by Simon Knight

Some time ago I decided to re-write my About Me page so as to incorporate some lessons learnt from research into sales, marketing and in particular – copywriting. While doing so it made sense to look for words that would lend weight to the message I wanted to convey. I turned to the book Words That Sell for inspiration and as a result, developed my lists of Words That Sell Software Testing below:

Technical words that dazzle the listener or reader with the cutting-edge possibilities of a product or service:

Powerful
Functionality
Performance
Transforms
Maximises
High-capacity
High-performance
Advanced
Sets the standard

Cerebral words that appeal to the head and that carry a tone of maturity and competence:

Assurance
Collaborative
Continuous
Control
Effective
Essential
Integral
Investigate
Logical
Continue reading Words That Sell Software Testing

Internet of Things – The new User Interface – Do we need new test tools?

Internet of Things – The new User Interface – Do we need new test tools?

Director Test Strategy and Consulting

JwristPADust wondering. Internet of Things will be massive. Wearable devices for Health, Medicine, Communication, Entertainment, Functional Workplace Applications, etc. There are as many applications under development and those we haven’t seen, that will challenge the test methodology we use to test our present systems and environments.

Imagine the test required for a brain wave synchronizer, being driven by an application and data residing in the cloud, that will capture the experiential responses as well as govern them for the user. The uses in this case are vast. Relaxation, Accelerated Learning, Medical monitoring of Brain Wave activity, treatment of ADHD, transmission of said data to and from subscribers, etc. I can imagine the Test Strategy Document, Test Plan, the Lab work, the logistics and Planning. Test resources with the skills to run the full gamut of tests? This was a product I developed back in the 80’s, but I was the testing guinea pig!!

intel-wearable-featWe will need to step it up, to keep up with the variety and depth of new applications. Creative thinking, innovative approaches to capturing the device dynamics, and reporting those as metrics… I think it is a very exciting time, and we will see this explosion happen over the next 15 years. It is inevitable.

You might want to consider: What does this mean to you? How will you remain relevant? Does this mean your present skills are already obsolete, or that you will have to learn something new (I certainly hope so!)

Let me know how and why you think this will impact your testing career!

Bill

BPM Testing – Get the Best Bang for your Buck!

QA2100-BPMTesting-16MP

Interested in getting the best bang for your buck with BPM Product Design, Development, Strategy, Testing, and Implementation?

Need a lift?  We can help!

Give us the opportunity to provide you with our assessments.  We have USA resources, and fully experienced offshore capacities for development, testing and delivery.

We have lived it for over 8 years and provided some of the finest products in the Insurance and Banking Industries.

Ccropped-facebookcover.jpgontact Bill Fulbright
Company: QA 2100 Test Strategy and Consulting
Website: http://qa2100.com
Email: bill@qa2100.com

BPM Testing in Today’s Market – QA 2100’s Testing BPM Testing Toolkits

QA 2100’s BPM Open Source and Web Service Testing Toolkits

The behavior of many BPM service based applications are governed by business process and workflows which are defined by business rules. These business rules must be validated during application testing. For many firms, testing business rules is a costly and complicated process which involves business users and testers. QA 2100 has invested in state-of-the-art automated BPM test methods and tools integrated by Pega Systems into PegaRULES Process Commander®
(PRPC) V.X and Test Management Framework, Bonita, and other opensource BPM products. Within the framework of PRPC, and BPM products is a process of design which utilizes not only business process, but a Requirements Definition tool which clarifies the Requirement process. This process turns use cases based on requirements into design, thus providing fundamental testing paths for automated testing of the BPM framework. This allows you to develop an application using a design based upon business rules, use cases, best practice development and quality principles.

Automated business rules and workflow validations can lower your testing time by 95%

Test Automation Using QA 2100 BPM Testing Toolkits
QA 2100 takes business rules validation testing one step further by automating the creation of test scripts using parameterized data and automating the execution of test cases. For example, QA 2100’s accelerator can execute 65 rule validations in 1.1 minutes using automation, versus 32.5 hours for manual execution. We use the Automated Unit Testing functionality within Pega PRPC to help you build a series of test cases to satisfy test requirements defined by the business requirements and use cases. These test cases are the foundation for automated test scripts. Automated test scripts can be built to pass from workflow to workflow, thus describing a partial or complete path through the application for scenario or end-to-end testing.

With the use of Test Management Framework (TMF), and other Test Repository tools, use case steps and parameters as described within the automated test scripts can be satisfied using the Scenarios and Suites features. The Scenarios and Suites test the behavior of the application and verify compliance with the original requirements. Besides providing significant savings in cost, time and efforts, automation lets you run many more tests during your testing process as a suite to provide hands off BPM Testing results.

Boundary Testing
QA 2100 provides boundary or negative testing of the business rules in the BPM framework and process to confirm the effectiveness of rule sets by requesting conditions that don’t exist. This helps ensure the business rules engine returns the correct value or an appropriate error. These boundary tests are set up as part of the actual application within each workflow.
QA 2100 has experience with automated tools to accelerate testing and improve accuracy
Employing automation tools to test and validate business rules adds breadth and depth to your testing efforts. By using pre-defined testing parameters, hands-off automation methodologies, and innovative solutions, you can accelerate and simplify a complex process.

XMLServiceTestToolExecutionTiming

32.5 hours to perform 65 rules tests manually
1.15 minutes to perform 65 rules tests using automation

Continue reading BPM Testing in Today’s Market – QA 2100’s Testing BPM Testing Toolkits

%d bloggers like this: