Inverting the Test Pyramid, by Joel Masset

Great Article from Joel Masset, Global Head of Product Assurance and Chief Quality Officer in the Financial Services Industry

invertedpyramid
Inverjoelmassetting the Test Pyramid
by Joel  Masset
https://www.linkedin.com/pulse/inverting-test-pyramid-joel-masset

Although most organizations focus on what it means for development teams to be agile, what that changes for them, and how to make it happen, testing is still very important in an agile delivery model. It is even more critical than in a traditional model.

I want here to highlight the big change that agility represents from a testing point of view. This is highly inspired from Mike Cohn’s well known test pyramid. However, I thought it was worth sharing this once more, as it is so close to what many major software delivery organizations need to go through nowadays.

In a traditional software delivery approach (waterfall, V cycle, iterative V cycle, washing machine…), all activities are serialized. Business requirements, design, development, test, debugging, […], release. The focus is put on finding bugs.

InvertedPyramid1

This leads to very little test effort to happen at the beginning of the cycle, and massive focus at the end, after developers have integrated their components into a product.

Roles are very clear and distinct. Developers’ role is to code and debug, testers’ is to find bugs, and check they are fixed.

Automated tests developed after the integration has happened will be essentially based on the UI. Their development cost is very high and since they are very fragile, maintenance costs are very high too. Any update to the code is likely to break the tests, which will require automation code to be updated prior to being ran. Since most of the testing focus is happening so close to the expected delivery date, there is a lot of pressure on the team to give its test campaign results. In most cases, it results into massive manual testing. In the best case, automated test development will happen in parallel, so that these can run post the release, but very regularly, the team will just give up on automating.

As a result, automation is extremely painful, costly, and inefficient, with a very low Return On Investment. The manual testing in this model is very costly, and happening under stress.

Don’t get me wrong, I am not saying this model cannot work, I have myself been using it for years. But oh my god, is it stressful…
Agile testing takes the opposite approach, by inverting the test pyramid.

AgileAutomatedPyramid

The focus is now put on preventing bugs from existing. This means that most of the test effort will happen at the beginning, at the code and UI level. In this context, automation will be extremely efficient. Unit tests will be written alongside development, or even before the actual code is written or updated (this is the Test Driven Development model).

Acceptance tests will be created at the API layer level during development and integrated in the continuous build and integration tools and processes.

At the time the developer checks the code in, it has already been successfully tested. Post-build and integration tests will automatically be ran and results checked.

Since all those tests happen during the sprint, a failed test will immediately lead to a code change fixing the issue. Not only the cost of fixing a problem is much smaller at this stage, but the risk associated to re-opening code that was already pushed, days or weeks ago is gone. The team then rarely has to switch context from the current version of the code to an older one. This is a much more reliable process.

Some testing will still be required after the integration steps. Some will still be automated at the UI level. This represents very boring and repetitive tests like links checks, or basic user workflow steps. The essential part of testing at that stage is exploratory testing. Based on end user workflows, these are tests that only a human person can do. But the effort will be limited here.

In this new approach, you end up with developers and testers both testing and coding together. A much higher synergy, and very little stress, because the delivery process is made much more reliable.

Results are impressive on the Quality of the software, the predictability of the releases, and as well on the motivation of the teams.

This change requires a big cultural change obviously, hence very high senior leadership, this will certainly be the focus of a future article…

Bill Fulbright
770-880-0959
Bill@2100solutions.com

Advertisements

Vendor Management Service Transformation: Entry 1 – Re-Factoring, Businss Architecture

metodo_pratiche_agile_chart_manifesto_itaEntry 1    3.22.2015

I recently was invited to join a project for a Vendor Management Service (VMS) in Mid-March 2015. The project is to provide in Phase 1 a re-factoring of our Client’s code by replacing the hardcoded middleware with services, and adding new client facing features, along with a new UI. All needing new documentation, of which there is now verylittle.

Our client provides a turnkey service for managing IT vendors who need to outsource their HR, Recruiting, Accounting, and Financial Services for this aspect of their business.

My role is to document the present legacy Business Processes, the new Processes, the new Services and the newly re-factored APIs, processes and added features by providing the Requirements, Use Cases, Workflows and Processes.

The leadership on this project is not only setting the pace, but shining a bright light into the future vision for this client, and for the VMS industry. It is a privilege to work with them.

 

Presently, I am awash in the project ramp-up and assimilation of the many layers, features and infrastructure required to successfully launch a program as complex as this.

We have two teams: one is onsite with FTE EEs of the customer, and a fly-in contingent of our leadership. The other is an offsite team in Atlanta, that is providing an AGILE based component for delivery of the new code which provides the new Service APIs and integration; as well as Leadership, Business Architecure, Process Articulation & Documentation. The client will observe the present SDLC based approach for now.

We have defined the primary users and their roles, the features – both new and old – associated with their roles. The functionality of these features some of which, for now, will remain as legacy, while others are new. There are around 400 of these. Some are Epic, requiring some of the features to support the workflows.

For the new and replacement pieces (in AGILE) we have defined the primary “Day in the Life” from the “need to the ass in the seat” E2E process to establish a critical Happy Path. Variations and UCM will be modeled based upon this primary structure.

The software and coding will be the same, albeit updated. The specific usage of the system will vary based upon the needs and systems of client-users of this system.

The SDLC pieces for things like the DATA, and QA will be driven from the client sites.

I will be updating this log at various points along the way…. so STAY TUNED!!

Bill Fulbright

Want To Start a Testing / Delivery Practice?

cropped-qa2100_gravatar.pngSo many of us do.  However, looking at starting something like this from scratch, is truly starting a new business and requires much thought, planning and many “hats”.  Here are some points I have used to assist others in grasping the needed vision, depth of effort, types of talent, and sustainability of the effort that will be required.

Visit my Linked In group at:  QA 2100 Testing: Financial Services

Roles and Resources
It requires full time attention to each of these Roles and Resources:
1.  Experienced and Qualified Prospecting and Sales force
2.  Pre-Sales Preparation, RFP/RFI Replies, as well as Attendance at Presentation (Orals) – for each opportunity.
3.  Attained Status as Preferred Vendor, or Vendor
a.  Requires relationship with Client Companies
b.  Requires meeting their criteria for becoming a vendor
1.  Capacity to perform
2.  Liability Insurance
3.  Fiduciary Responsibility
4.  Legal Compliance
4. Developers for the domains you wish to pursue, with the appropriate skill sets, and experience
5. Business Analysts and Business Architects for the domains you wish to pursue, with the appropriate skill sets, and experience
6.  QA Leaders, Managers, and Testers with domain experience
7.  Project / Engagement / Delivery Managers to connect with the Client and the Teams
8.  Financial Resources to sustain the ramp-up of business, infrastructure, and delivery of services
9.  Financial Resources to sustain Market Research, Marketing Strategies
10.Partnership Alliances – to give you value added leverage when positioning for new business
11.Budget that will include all the above, balanced with enough sales to justify the effort, or a plan for ROI over 5 years for investors.
12.Business plan and road-map that demonstrates all the above, and it’s veracity.
13.Staff on the ground – available, ready to work on-site, near-shore, no visa issues, or proven offshore teams that have a solid delivery history.

I am writing this not to discourage those interested in pursuing such a goal, rather to create curiosity and and interest in taking on the challenge.   These concepts and work efforts are real elements that have to be in place before it can be a successful venture…. even for an established company wanting to build a new practice.

Business Plans:
In other words, this effort requires at least:
Experts to Initiate, Drive and Deliver
Business Concept(s) meeting Market needs; Business Need/Justifications
Planning: Initial Steps to Initiate, Milestones at 3 mo., 6 mo., 1 yr, 18 mos., 2 yrs., etc. till 5 years
Identified Services and Products
Multi-Phase Financing: Start-Up Money, Mid-Term Money, Long-Term Money, Planned ROI’s for each phase
Market Research for Service / Product Viability,
Strategy for Launching the Business,
Scope of Ramp-up,
Planned New Business Market(s)
Planned Costs
Ability to Staff and Deliver
Motivation, Determination, Desire, Mission and Purpose
Commitment to see it through

None of these comes pre-packaged, or comes easily.  It all requires vision, leadership, experience, clarity, strategy, good communication, follow through.

I am certainly not giving away the store here, but sharing some thought work I have used to help prepare business not only in the IT world, but as part of my 12 yrs of Management Consulting before my last 19 years in the IT business!  These principles hold true in any enterprise!  I have written many accurate and successful business plans for new or international or established companies seeking investment money to fund a start-up.

As a thought leader and architect of Quality Assurance, and Business Process, I have learned through experience these tenets, which will not be learned so much at school, but by living it and delivering it – with successful outcomes.  That does not mean there weren’t failures, or massive challenges along the way – that is where the real learning takes place.

The Insapience of Anti-Automationism – Context Driven Testing

Wow! That is a mouthful, but well worth reading in this great and stirring article on Context Driven Testing by Cem Kaner   http://context-driven-testing.com/?p=69

This is an outstanding article on the DISTINCTIONS and False Dichotomies that need to be clarified and observed when planning and writing test cases: manual vs automated,  checking vs testing …. or ‘sapient’ (is this a new test term or is it an attempt to marketize a concept?)

“…According to the ?new doctrine?, only manual testing can be sapient. Automated tests are done by machines and machines are not sapient. Therefore, automated testing cannot be sapient. Therefore, if a tester is enthusiastic about sapient testing (whatever that is), automated testing probably won’t be of much interest to them.”

Hmmmm……

It also Invites ond, as Ron Jeffries puts it in the comments:   “…to look at the question from different angles, and the message that I get is that there is no bright line anywhere to be seen. That matches my deepest intuitions and gives me a kind of comfort.”

 

Security in 2015 – What measures will be implemented to improve it?

by ,  Sr. Quality Strategy and Delivery Advisor

Is Security in your environment covered?

  • Do you have a full rundown/analysis of the gaps you may have in your system?
  • Have you created a checklist of all:

    • touchpoints
    • protocols
    • profiles
    • methods of transmission
    • firewalls
    • frequency of sweeps
    • frequency of security monitoring status reports?
    • Do you have a network ops monitoring application?

imageThis is not the year to be avoiding the security risks afoot … not only from your own employees, random local hacker, but serious international hacking as pro-active attacks on your system. 2014 demonstrated an increase in security leaks – or might i say exposure of weak security by upstream hackers with malicious intent. Expect more of it. Breaches have been happening every year for some time. We are no longer surprised by them. It is another overload of input that we as consumers can do little to prevent.

Prevention of security leaks are up to those responsible for maintaining our private accounts and data. That they have allowed weaknesses that are gaps, and hackable, is irresponsible, unacceptable, and once leaked causes much damage financially, and personally.

What is a Threat Agent?

The term Threat Agent is used to indicate an individual or group that can manifest a threat. It is fundamental to identify who would want to exploit the assets of a company, and how they might use them against the company.  You can read more about it here:
https://www.owasp.org/index.php/Category:Threat_Agent

Here are some Highlights from Open Web Application Security Project “Attacks” references:
https://www.owasp.org/index.php/Category:Attack

Looking forward to seeing deeper security measures, and fewer assailable gaps by our financial institutions and retailers.

All comments invited.

2015 Hospital Hacks, Banking and Retail Hacks, Entertainment Hacks – Attack On Financial Services

This post on my blog QA2100.com is in reference to a great post highlighted by Jaden Turner’s share on 2015 Hospital Hacks: and posted into our Group on Linked In:  QA2100 Testing Strategy: Financial Services

Every week we are hearing about another leak, hack or break-in and millions of credit card holders are exposed, at risk, or invaded.   Who are these hackers?  Why are they hacking?  Money.  Greed. Something for nothing.  Retribution.    All of which is Vicious, Criminal and
destructive to infrastructure, commerce, and consumer confidence

Security – is this an oxymoron? We hear it, and aretaught to believe it, so we truWhyItMattersst that others are responsible about implementing it. Real Security means real testing dollars are spent beyond the boundaries of a new project launch… Usually only the minimal security testing is considered if at all. If it is, is usually not part of projects, rather it is part of the ‘network’ group, or ‘infrastructure’ group.

So, this is not about the kind of job our network guys are up to, rather the kind of budget that gets allocated to supporting enough security measures, plus the budget to ensure it is being implemented and maintained at a deep enough or broad enough level. This means maintenance, and keeping up with the latests shenanigans by our nefarious ‘hackers’.

I have the same issues with performance testing. and automation for regression.

So I could go on, but these areas are allowed to get weak due to higher priority profit making budgets. and on and on until an emergency effort to shore up security is done again. Security = Insurance. If you don’t spend the money on the protection, it won’t be there when you need it.

This is the tip of the IceBerg and we need to be vigilant, and attentive to the looming prospects of risk.