From Fire Fighting to 0 Incidents: How Jira Components Changed Product Culture

QA can change your product culture

They were 100% in a firefighting mode. After patching multiple times every day, things would constantly break again. The phone of their customer service team was ringing off the hooks, the account managers were putting out fires, it was management chaos.

When I came in I said: I have this method and I know it works. Just bear with me.

As a QA Manager, Eliana was used to fighting for the value of her work. She would often be asked, point blank, what her job was about and why it was important.

The answer she developed overtime is a planning methodology that demonstrates the business results of Quality Assurance done right. When she finally got leadership behind her it was so refined that she was able to turn around company culture.

At one of our quarterly meetings our Director of Client Services said they only had to fix one bug over the entire quarter. The rest of their effort was spent in creating feature requests, onboarding customers, creating tutorial videos... This quality process had changed the product, how it was perceived, brought to market, and charged for. I'm so proud with having driven that change.

But how could the transformation of a company possibly start from a QA manager? And what does it have to do with components in Jira?

How to promote the business value of QA

One of the core ideas around Eliana’s methodology is that QA is not about testing at the end, when it’s already too late and too expensive, but rather about setting a quality process.

There are several implications of this basic idea for team operations:

• Aim at a 100% test coverage for your code. You won’t always get there, but it surely means scratching manual testing to a minimum
• Developers should own testing automations, because they will get a great coverage of their code at a very low level and leave very few blind spots
• In the end, it’s about DevOps: developers must be involved throughout the entire lifecycle of their code
• The number of bugs should be predictable and included in the planning process
• Ultimately, QA KPIs can be used to inform better business decisions.

Now that you’re starting to get into the mindset, let’s look at how you can use your Jira components to predict the quality of your code… and the time it will spend on the testing and debugging bench.

How to use Jira components for QA planning

Step 1: Assess the complexity of each Jira component

At some point Eliana started noticing a simple but important correlation: the more complex a component was, the more bugs it would generate on the long run. Since any modern software application is already broken down into components, with specific functions and a code base of their own, finding a diversity of complexity levels is not hard.

Step 2: Measure the test coverage

Complexity is not all that counts. Given the same complexity for two components, the test coverage with automated testing can really make a difference. If the automated test coverage is very high, even the most complex component should not generate too many unknown issues.

Step 3: Calculate code risk

The Risk Index is a simple score that combines the complexity and test coverage into a simple metric so you can rank components. The higher the number, the more likely it is that something unexpected has to be solved before taking the code to production.

Step 4: Report on the historic trends in the bug repository

At least every quarter, Eliana would look at how many bugs had been reported for each of the components and share the metrics with the business.

Simply by looking at this report, the product owner or any other business stakeholder could easily understand how new features would impact the product’s time-to-market. It wasn’t uncommon to postpone work in a component of the application that was very complex but was only catered to a small fraction of the user base.

Step 5: Apply QA metrics to every story

Until now, we’ve only talked about measuring.

Now, ask your developers to assess how complex each story they are about to start is and adjust their assessment with what you know of that component based on the historic metrics in steps 1 to 4.

Step 6: Predict the number of bugs

You might not get it right at first, but when your team and QA culture stabilizes you will be able to predict exactly how many bugs will that story generate when it’s manually tested by the QA team before going into production.

I was so accurate in my estimations, it was ridiculous. They often joked about it. When I handed them to the project manager, she would always be like "how the hell did you do that". And I was like: "I have my formula!"

Step 7: Adjust your release plans

When you know how many bugs you will get and how hard it will be to fix them, you’ll also be able to predict when that story is finally ready to deploy.

Yes, you heard right: you will get your release times right from the time code is done for the developers until its pushed to the customer.

What tools do you need do a QA report with Jira?

• Jira Software as dev and bug tracker is useful because it supports the components.
• Octo for Jira will help you manage and share components across projects in Jira and ultimately create a standardized structure
• A code analysis tool like NCover for .NET to have objective data about the complexity of your code
• A tool like Selenium to automate regression automation
• Swanly will also help you create customized release timelines in Jira that include an elastic QA phase
• Depending on your skillset, you will need access to an expert who can integrate your tools so you’re looking at accurate, updated and visual data on a BI tool like Power BI

The benefits of measuring components in Jira

Implementing this methodology will require an invested team with a clear direction. You will need to get the data right, set up testing automation, get developers involved in QA… Remember that the end game is not just doing great Quality Assurance, but changing the product culture in your organization.

These are the main arguments you can use to drive your efforts:

• A common language to bridge the dev/business divide. The model communicates the diversity of your code base in very simple terms and at a high level to business actors.
• A great fit for microservice architectures. Although you can try this approach to break down into pieces your monolithic application, if you do microservices then the question of what your components are will be much easier to answer!
• The trade-off for business decisions is clear: the higher the risk, the longer the development cycles and the higher the chances that customers will engage with support because that component fails in their hands.
• Better business decisions. On the long run, the business slowly internalized the component model and started having more conscious conversations about the ratio between the value delivered by a feature and its quality risks - simply based on the components involved.
• You will know the exact dates for your releases. Forget about agile orthodoxy for a second. Of course, you have to take this with a grain of salt until you learn to master the method, but the business impact of having this type of certainty can be transformational.
• Skip the fire-fighting mode. By getting your code clean first, your entire organization will be able to switch from reactive incident support to managing customer success.

If you’d like to use the same Jira components in different projects for a common reporting structure, make sure to try out Octo.