Blog : bug reports

How well do you advocate for your bugs?

How well do you advocate for your bugs?

If you do testing, and recognize how cognitively rich the activities involved in testing are, you probably also recognize the importance of testing skills.

On all the projects I’ve contributed to, good testing, deep testing, involved skills. Asking a random person from the street to test on the project would probably not have led to spectacular results (unless, of course, they happen to be an exquisite tester with awesome testing skills!). Developing those skills requires a lot of work.

On each project, stakeholders expected from me to be able to provide relevant information. This included sometimes:

  • the status of the product,
  • progress of the testing effort,
  • feedback on the development process,
  • and risks worth considering.

This information can be conveyed in different forms, and one of the prevalent ones is creating bug reports. In many of my testing experiences, bug reports have been a main way of communicating relevant information on the project.

Replicate, Isolate, Maximize

I quickly understood that “bug advocacy” did not mean insisting to get EVERY ONE of the bugs I found fixed. It also did not mean coming up with unreasonable arguments for fixing a bug because I wanted to prove something or look good in the face of a developer. I associated the idea of bug advocacy with presenting relevant information in a bug report to help others decide if they want to fix the bug or not.

As I accumulated more experience in doing testing, I realized more and more that I needed to learn new, specific skills to do this work well. For some reason though, bug advocacy skills were not at the center of my learning focus at the beginning. I observed how others around me reported issues, and tried to mimic that. I chose examples from people I thought were doing this well, and tried to follow along. From this process I gathered ideas like including the browser and OS version in my bug report when it was relevant, writing a clear summary, organizing the relevant information in steps to reproduce, include the result I’m getting and the oracle I’m using to decide that a behavior may be a problem. I slowly built a template for that.

In terms of bug investigation, I learned to try to reproduce the issue more than once, that it was important to understand what triggers the problem, and what impacts it could have. I didn’t have a template for that though, so I followed my gut instinct and decided to apply my analytical thinking to guide me through. That was the level of detail I got into analyzing how I do bug investigation.

And then the idea of “bug advocacy” started to take shape in my head. I took the Bug Advocacy course from AST in 2012 and realized there was more to that than I had initially thought. There was a wide array of skills related to doing effective Bug Advocacy. I got familiarized with a framework for investigating bugs and dealing with excuses to not fixing them.

That was when I consciously started considering Isolation, Maximization, Generalization, and Externalization of bugs. So I started practicing on my projects using a structure. Through feedback I understood that a template for bug reports does not always work, that it’s important to find ways to effectively tailor the reports to the context.

I practiced with a follow-up testing framework that would help me understand if the bug was more serious than I initially thought. I practiced with what would be relevant to vary and to consider in order to manage to reproduce the hard-to-reproduce bugs, and to uncorner my corner cases. I started grasping just how powerful bug reports are as a tool for communication. And how being more skillful at this makes a big difference. It influenced how I was perceived by the programmers, how much the project managers trusted my suggestions and appreciated testing efforts.

Generalize, Externalize, Neutral tone

As with other arrays of skills I use in testing, I need to clarify my understanding of the concepts and to practice with bug advocacy skills in order to improve them. And as I go deeper, I find there’s more and more interesting skills to work on. Why, then, have I initially overlooked this area of skills in testing?

I think it’s because I thought I’m doing bug advocacy as good as any other fellow tester. A multitude of skills involved were not visible to me. They were not explicit, but incorporated in the reports, and in the testers’ credibility. I hope they are visible to testers around me now, and I strive to make them so.

As part of that, I got involved in actually teaching the Bug Advocacy course. As part of Altom’s collaboration with Cem, I’m lucky enough to be able to participate in teaching the course alongside him. And it has been enlightening so far. One central feature which I missed when I did the course as a student was the interactive grading session halfway through the course. Since this course focuses on developing bug advocacy skills, it contains practical assignments. Students get to actually contribute to the bug reports of an open source project. After their first contribution to such a report, they have an interactive grading session in which they get feedback on what the did well, what to improve, and how. Afterwards, they get the chance to work on a different bug report and incorporate the feedback. I think this loop of doing a task, getting good feedback, then trying again, is essential to developing skills. And I’ve had the chance to see the students’ evolution in this context. So that’s one of the elements I’m looking for in my own learning process: after advocating for bugs, getting useful feedback, then getting the chance to tweak the process. A challenging aspect is that contexts change all the time, as I work with different teams, and encounter different types of issues. Which is also fun, because there’s always something new to learn in the process.

I hope to internalize the bug advocacy frameworks I’ve learned about more and more as I consciously practice with them and think critically about them.

So, what about you? What’s your way of improving your bug advocacy skills?

PS: You may have noticed that the bugs drawings represent the RIMGEN bug advocacy framework. Thanks to my colleague Levi for creating them! You can download the printable RIMGEN postcards for free.

What to expect from Bug Advocacy: the hands on approach and revisions

What to expect from Bug Advocacy: the hands on approach and revisions

If you have completed BBST Foundations, congratulations! You can now move on to a more hands-on part of the BBST series, starting with evaluating bug reports in the Bug Advocacy course module.

Compared to Foundations, this module is much more focused on practical exercises. You get to work on live bug reports of open-source applications. You can actually contribute to the documentation of these bugs.

The most appreciated feature of the course is the interactive grading session.  In contrast to Foundations, this session happens halfway through the course: you get feedback for an assignment, instead of the final exam. This way, your instructors will provide feedback that you can apply immediately on a subsequent assignment.

The interactive grading session will be on Skype – a call with one of your instructors.

The assignment you will discuss is your actual contribution to a bug report in an open-source project. After you get constructive feedback on your work, a second assignment will give you the opportunity to apply that feedback by contributing to another bug report. This practice-feedback-practice flow has proved to be a very valuable learning technique during the course development over the years and from Cem’s teaching experience. We learn new skills by practicing and getting actionable feedback on our work, then practicing some more. The skills you practice here are:

  • investigating a behavior based on the available information.
  • communicating your findings through arguments that motivate others to take the appropriate action – either fix it or close it/focus on something more important.

Remember, the point of using interactive grading is to let the student focus on learning. That’s why your grade cannot go lower during the session. When you have no reason to worry about the grade you can fully focus on applying the feedback and learn from it.

What’s new in this edition

Some of the lessons in the workbook were adapted and the chapter order was rearranged to avoid confusions that some students had in the previous years.

Different exercises and assignments were changed for the same reasons: students were confused or could not complete the assignments the way it was intended.

Most recently, new quizzes were introduced. Based on feedback from students in older sessions, these new quizzes aim to be more clear.

 

Register now for the next course starting on August 13!