Blog : Posts

How well do you advocate for your bugs?

How well do you advocate for your bugs?

If you do testing, and recognize how cognitively rich the activities involved in testing are, you probably also recognize the importance of testing skills.

On all the projects I’ve contributed to, good testing, deep testing, involved skills. Asking a random person from the street to test on the project would probably not have led to spectacular results (unless, of course, they happen to be an exquisite tester with awesome testing skills!). Developing those skills requires a lot of work.

On each project, stakeholders expected from me to be able to provide relevant information. This included sometimes:

  • the status of the product,
  • progress of the testing effort,
  • feedback on the development process,
  • and risks worth considering.

This information can be conveyed in different forms, and one of the prevalent ones is creating bug reports. In many of my testing experiences, bug reports have been a main way of communicating relevant information on the project.

Replicate, Isolate, Maximize

I quickly understood that “bug advocacy” did not mean insisting to get EVERY ONE of the bugs I found fixed. It also did not mean coming up with unreasonable arguments for fixing a bug because I wanted to prove something or look good in the face of a developer. I associated the idea of bug advocacy with presenting relevant information in a bug report to help others decide if they want to fix the bug or not.

As I accumulated more experience in doing testing, I realized more and more that I needed to learn new, specific skills to do this work well. For some reason though, bug advocacy skills were not at the center of my learning focus at the beginning. I observed how others around me reported issues, and tried to mimic that. I chose examples from people I thought were doing this well, and tried to follow along. From this process I gathered ideas like including the browser and OS version in my bug report when it was relevant, writing a clear summary, organizing the relevant information in steps to reproduce, include the result I’m getting and the oracle I’m using to decide that a behavior may be a problem. I slowly built a template for that.

In terms of bug investigation, I learned to try to reproduce the issue more than once, that it was important to understand what triggers the problem, and what impacts it could have. I didn’t have a template for that though, so I followed my gut instinct and decided to apply my analytical thinking to guide me through. That was the level of detail I got into analyzing how I do bug investigation.

And then the idea of “bug advocacy” started to take shape in my head. I took the Bug Advocacy course from AST in 2012 and realized there was more to that than I had initially thought. There was a wide array of skills related to doing effective Bug Advocacy. I got familiarized with a framework for investigating bugs and dealing with excuses to not fixing them.

That was when I consciously started considering Isolation, Maximization, Generalization, and Externalization of bugs. So I started practicing on my projects using a structure. Through feedback I understood that a template for bug reports does not always work, that it’s important to find ways to effectively tailor the reports to the context.

I practiced with a follow-up testing framework that would help me understand if the bug was more serious than I initially thought. I practiced with what would be relevant to vary and to consider in order to manage to reproduce the hard-to-reproduce bugs, and to uncorner my corner cases. I started grasping just how powerful bug reports are as a tool for communication. And how being more skillful at this makes a big difference. It influenced how I was perceived by the programmers, how much the project managers trusted my suggestions and appreciated testing efforts.

Generalize, Externalize, Neutral tone

As with other arrays of skills I use in testing, I need to clarify my understanding of the concepts and to practice with bug advocacy skills in order to improve them. And as I go deeper, I find there’s more and more interesting skills to work on. Why, then, have I initially overlooked this area of skills in testing?

I think it’s because I thought I’m doing bug advocacy as good as any other fellow tester. A multitude of skills involved were not visible to me. They were not explicit, but incorporated in the reports, and in the testers’ credibility. I hope they are visible to testers around me now, and I strive to make them so.

As part of that, I got involved in actually teaching the Bug Advocacy course. As part of Altom’s collaboration with Cem, I’m lucky enough to be able to participate in teaching the course alongside him. And it has been enlightening so far. One central feature which I missed when I did the course as a student was the interactive grading session halfway through the course. Since this course focuses on developing bug advocacy skills, it contains practical assignments. Students get to actually contribute to the bug reports of an open source project. After their first contribution to such a report, they have an interactive grading session in which they get feedback on what the did well, what to improve, and how. Afterwards, they get the chance to work on a different bug report and incorporate the feedback. I think this loop of doing a task, getting good feedback, then trying again, is essential to developing skills. And I’ve had the chance to see the students’ evolution in this context. So that’s one of the elements I’m looking for in my own learning process: after advocating for bugs, getting useful feedback, then getting the chance to tweak the process. A challenging aspect is that contexts change all the time, as I work with different teams, and encounter different types of issues. Which is also fun, because there’s always something new to learn in the process.

I hope to internalize the bug advocacy frameworks I’ve learned about more and more as I consciously practice with them and think critically about them.

So, what about you? What’s your way of improving your bug advocacy skills?

PS: You may have noticed that the bugs drawings represent the RIMGEN bug advocacy framework. Thanks to my colleague Levi for creating them! You can download the printable RIMGEN postcards for free.

What to expect from Bug Advocacy: the hands on approach and revisions

What to expect from Bug Advocacy: the hands on approach and revisions

If you have completed BBST Foundations, congratulations! You can now move on to a more hands-on part of the BBST series, starting with evaluating bug reports in the Bug Advocacy course module.

Compared to Foundations, this module is much more focused on practical exercises. You get to work on live bug reports of open-source applications. You can actually contribute to the documentation of these bugs.

The most appreciated feature of the course is the interactive grading session.  In contrast to Foundations, this session happens halfway through the course: you get feedback for an assignment, instead of the final exam. This way, your instructors will provide feedback that you can apply immediately on a subsequent assignment.

The interactive grading session will be on Skype – a call with one of your instructors.

The assignment you will discuss is your actual contribution to a bug report in an open-source project. After you get constructive feedback on your work, a second assignment will give you the opportunity to apply that feedback by contributing to another bug report. This practice-feedback-practice flow has proved to be a very valuable learning technique during the course development over the years and from Cem’s teaching experience. We learn new skills by practicing and getting actionable feedback on our work, then practicing some more. The skills you practice here are:

  • investigating a behavior based on the available information.
  • communicating your findings through arguments that motivate others to take the appropriate action – either fix it or close it/focus on something more important.

Remember, the point of using interactive grading is to let the student focus on learning. That’s why your grade cannot go lower during the session. When you have no reason to worry about the grade you can fully focus on applying the feedback and learn from it.

What’s new in this edition

Some of the lessons in the workbook were adapted and the chapter order was rearranged to avoid confusions that some students had in the previous years.

Different exercises and assignments were changed for the same reasons: students were confused or could not complete the assignments the way it was intended.

Most recently, new quizzes were introduced. Based on feedback from students in older sessions, these new quizzes aim to be more clear.


Register now for the next course starting on August 13!

Why BBST® Foundations is Not Only For Novice Testers

The first course in the Black Box Software Testing series is called BBST® Foundations, but don’t let the title fool you: this course is not only for beginner testers. How would you, no longer a novice to software testing, benefit from taking this course? – To help you find an answer to this question, we have compiled a list – tailored for the experienced tester – of valuable takeaways from the course.


Reviewing and deepening your knowledge of the fundamentals of software testing.

BBST® Foundations is an opportunity to go over the essentials and, most importantly, a space facilitating a deeper reflection on these. How often do you stop and reflect on your testing notions and activities? What is the mission of testing? How would you define your role as a tester? What is a bug? – All these are basic, “simple” questions participants in the course are encouraged to ponder, with the ensuing discussions being wonderfully complex and enlightening.  

If we were to take as reference a scientific model to describe this transition to another level of knowledge, we could invoke Anderson and Krathwohl’s revision of Bloom’s classic cognitive taxonomy. The revised taxonomy emphasizes how learning occurs over several – both knowledge and cognitive process – levels. The model includes four knowledge dimensions – factual, conceptual, procedural, metacognitive – and six cognitive processes: remember, understand, apply, analyze, evaluate, create. Anderson and Krathwohl’s model could be represented like this:


Anderson and Krathwohl's revision to Bloom’s classic cognitive taxonomy


Bringing that to our context, a non-novice tester might have a factual and conceptual knowledge but still have yet to achieve procedural and metacognitive. They might remember and understand, but still have issues with apply and evaluate, or have not reached the create yet. This course can come in helpful in this process of transitioning to a superior cognitive layer. For instance, you know what heuristics are but you do not apply them in your work, or you do not come up with heuristics of your own (or you do but are not aware of that).


Preparing for the next course/s in the Black Box Software Testing series.

The BBST® series consists of three courses: Foundations, Bug Advocacy, and Test Design. Each course in the series is designed as a prerequisite for the next. Starting with Foundations, students get a grasp of the structure of the courses, and become familiar with the online platform that hosts them, Canvas. This helps with future organization and time management.


Canvas - BBST Foundations online learning platform


Interacting with your peers and learning from their experience.

BBST® Foundations is a course which places emphasis on practical work and on interaction. Taking part in it is a great way of reminding yourself that testing is a social activity. One of our students has pointed out in the feedback form that BBST® Foundations is an “online course more social than any regular courses I’ve ever attended”.

Students in the course are encouraged to communicate, to connect, to debate, to express their points of view, to share stories and ideas. Topics are discussed with input from both the other students and the instructors. There is so much to be learned from these daily exchanges. For instance, not once has it happened that students have found solutions to testing problems they were facing by participating in the discussions. A testing technique that you are not familiar with described in a post, or a way of organizing the test team that you have not thought of – participants often find inspiration to deal with their daily challenges in experiences described by the others. There is a great deal of learning that comes from this sharing of personal stories and experiments, with their victories and their mistakes.


Working on your review skills and on giving productive feedback.

The course offers plenty of opportunities to exercise these valuable skills: there are assignments that involve reviewing the work submitted by your peer students. Correspondingly, your work will also be reviewed, by your peers and by your instructors. This exposure to different ways of giving and receiving feedback promotes understanding and an increased awareness of the various mechanisms at play, of the dos and the don’ts. We are sure you will find these very helpful when next reviewing the performance of a colleague at work.   


BBST students discussing


Developing your argumentation skills and discovering new ideas for structuring your arguments.

Besides being engaged in reviewing activities, all students are encouraged to participate in forum discussions over the various items in the course. Helpful information is gained from these exchanges. This is good practice for the discussions that will follow in the workplace, helping cement argumentation skills and providing new ideas for you to use and expand upon.

This is one of the most valuable takeaways of the course: you do not simply get to revisit concepts and exercise your skills, you will actually use what you learn in your work as a tester. You will find, for instance, you have more solid guidelines for evaluating the heuristics you use in a particular context, or for searching suitable oracles in different situations.


Taking a valuable course, highly recommended by specialists in the software testing field.

One of the content owners of the course is Cem Kaner, a noted figure in the field of software testing. Professor of Computer Sciences and Cybersecurity at the Florida Institute of Technology, Cem Kaner is also the author of some of the most insightful writings on software testing. He is senior author of the classic Testing Computer Software (with Jack Falk and Hung Quoc Nguyen), and also of Bad Software (with David Pels), and Lessons Learned in Software Testing (with James Bach and Bret Pettichord), the first self-described context-driven book.

Cem Kaner is the primary creator of the BBST® courses and the series has received much acclaim, being considered a first-rate option in terms of meaningful courses for software testers.


Software testing is about exploring, thinking critically, communicating – and a healthy dose of curiosity. There is a lot for the curious tester, whatever their level of experience, to explore in BBST® Foundations: from questions such as “what is testing?” and “how do we measure testing?”, to a diverse assortment of practical assignments, to a multitude of – more or less divergent – points of view on the various aspects of testing. Sometimes there is no need for an additional reason: a tester might simply indulge their curiosity, follow the spirit of exploration and take the class :)

Canvas – A new online learning platform for BBST® Foundations

Canvas – A new online learning platform for BBST® Foundations

Starting with this next BBST® Foundations class, we will be using a new online learning platform – Canvas from Instructure.

Canvas is a new and modern platform, allowing both students and instructors to easily navigate through the course modules and intuitively use its online features.

Amongst our favorite features, besides the intuitive user interface and modern look, we would like to mention:

  • An excellent mobile application, allowing you to check your progress, read and contribute to discussions and read assignments and lectures straight from your mobile phone or your tablet (either from iOS or Android devices)Canvas App on iTunes


  • A calendar view with all the deadlines for each of the assignments in the class and your scheduled meetings with the instructors, all setup in your own time zone. The calendar feed allows you to easily add all these to your own personal calendar, so you have all the important dates saved on your computer or on your mobile phone.

Launching the blog!


Welcome to the Altom BBST courses blog!


We are excited to start a collaboration with Kaner, Fiedler & Associates, LLC and to be offering a new generation of public BBST® courses.

We will try to gather here extra information useful for the courses, as well as impressions and tips from other people involved in the course, instructors and former students.

Together with the course and the BBST community, we would like this blog to help with building a platform for enabling learning in the testing area.
Testing is a complex activity, requiring a lot of knowledge and skills, when performed with passion and dedication. This is an idea we want to spread, and if you are inspired by it, we want to encourage you to further share it and build on it.

We also work with the idea that testing can be a valuable tool in software development projects. We enjoy learning experiences that we find useful in our work. This is what we would like to give further: an effective learning process, through which testing can be learned at a level which renders it very useful in the software development process.


Let’s make the most out of learning and testing!