(Agile) Testing

In one of my current projects, I’m using Postman to check an API product which is used by multiple clients, where each client has its own URL and specific test data to be used.
It’s about a dozen postman collections (each modelling one specific use case), and those need to be tested for about 20 different clients (number is constantly growing).

For execution in our continuous integration server, we’re using Newman (the Postman command line runner). You install the newman npm package, export the collections and environment files from Postman to your hard disk, and then can run the postman checks on your command line, or on a CI system.

A Newman command basically looks like this: newman run "collection.postman_collection.json" -e "environment.postman_environment.json"

So telling newman to run the collection collection.postman_collection.json using the data from the environment file environment.postman_environment.json.
Pretty simple and straightforward.

But remember, I need this for about 12 collections for 20 different clients. Also, each client has a sandbox, user acceptance test and production environments. In addition, the commands are slightly different in my local Windows machine, compared to the CI server, which is running on Linux.
12 collections x 20 clients x 3 environments x 2 operating systems = 1,440 commands to be written!
That’s quite a lot!

So I’m using a spreadsheet to generate the commands, since this is something that can be done pretty easily with formulas. A simplified version of my spreadsheet looks like the following:

• Column A holds the name of the collections (without the “.postman_collection.json” extension)
• Row 1 holds the name of the postman environments (without the “.postman_environment.json” extension)

The formula in cell B2 is ="newman run """&$A2&".postman_collection.json"" -e """&B$1&".postman_environment.json""", which is the very same as the basic Newman command I explained earlier, but it is pulling the value for the collection from cell A2, and the value for the environment from cell B1.
The $ signs are telling the spreadsheet, to not change the value followed by it, when extending the formula to other cells. When extending the formula to cell C2, we still want to get the value from A2 as the environment.
I then can easily copy the commands from the spreadsheet and paste them into my command line window or wherever I need them.
The example spreadsheet is available here: Spreadsheets to create test execution commands

Hope that this might help anyone out there to maybe save some manual work and would be interested to hear how you are using spreadsheets to ease your lifeas a tester (besides from putting your test cases in there).

Yesterday there was another meetup with Alex Schladebeck, Huib Schoots and Bart Knaack, where I again participated in the lean coffee facilitated by Alex.
Jesper Ottosen raised the topic of „visualizing test strategies“. Since I found that topic pretty interesting, here are my (quick) notes I took during the session, for later reference.

Mirjana Andovska is creating one page test strategies/ plans, that generally „should fit into one page“, containing answers to questions like „What type of questions are answered by which (types of) tests?“, „Which problems are being solved by doing this?“ and „When are those tests run?“
She also proposes that for reporting one nowadays can easily pull data from different sources (eg. Atlassian)

Jantien van der Meer is neither creating lengthy documents (never be read) nor mind maps (not everybody like them). She says that very often architecture diagrams of the system already exists, but usually not detailed enough as you would like them for testing. She draws similar diagrams, especially containing the things relevant for testing (eg. interfaces), then highlights those parts and shares the drawings.
Jantien mentions Sonarcube and pipelines as additional sources to pull data from for reporting.

Alex Schladebeck recommends to ask „Who’s it for? Who reads it? What info do they need?“ and proposes that different versions might be needed.

Update: Jesper adds the following two models via Twitter: Visualizing Pipelines to Help Build a Quality Culture by Lisa Crispin and Ashley Hunsberger and A new model for test strategies… (An update to the Heuristic Test Strategy Model) by Dan Ashby.

For the second time during the corona lockdown, Alex Schladebeck, Huib Schoots and Bart Knaack organized an online get-together for testers. They offered and facilitated various activities, and during the last session, I participated in the lean coffee facilitated by Alex.
I was quite happy that one of my proposed topics got selected, but before getting to the details of the discussion, let me explain some background:

About 14 months ago, I joined Finologee as a Senior Software Tester. The role is pretty technical, with a strong focus on technical automated tests (checks). The most technical role I ever had in my 15+ years of being in testing – and I absolutely love it!
But: It’s also very challenging, which as such is not a problem – I love challenges!
It’s so many new frameworks, tools and other various technical things to discover and learn!

I think compared to most software testers, my technical background is quite strong: I did a 3 year traineeship as application developer (german “Fachinformatiker Anwendungsentwicklung”), studied informatics and during my career my focus has been test automation most of the time.Nevertheless, in that new role I feel overwhelmed from time to time: What is that framework for? How do I configure this in Maven? What does this method do? What is the most elegant way to solve that problem in code? And so on…

Quite often I’m searching Google and Stack Overflow for solutions to problems, that probably others have solve a thousand times before – or not. But I don’t want to spend hours looking for something, where there is probably good solution existing and I’m just not finding it because I don’t know the right terms to enter into that search field.
Because one important lesson I learned is “to reach out when stuck!” (shoutout to Deborah Preuss), after an unlucky search on the web, I enter my question into our company’s #dev Slack channel, and most of the times, I get a sufficient answer really quick.

Problem solved, end of story – one might think!
But everytime I ask a question there, I’m asking myself: Is this a stupid question? Should I have known that myself? What will the developers think of me for asking such a “stupid” question? And this is not really making me feel good.Yes, I know, being a software tester and being a programmer are too different roles, with somewhat overlapping, but still very different skill sets. So there’s no shame in not knowing everything, in not being as skilled in programming topics as programmers are.
Still it is bothering me way too often.

So I was happy that my proposed topic “How to not feel ‘stupid’ because as a testers we don’t have the same skillset as programmers?” was discussed during the lean coffee session.
Following things have been discussed in order to overcome this:

Find a mentor

• find one person in the company who mentors you
• it’s not “embarrassing” if you’re officially mentored and asking something compared to asking your question in a slack channel with every developer

Embrace the fact you have „gefährliches Halbwissen“ (knowing just enough to be dangerous)

• it might be enough knowledge about a tool/ framework to get done what you need to do at the moment

Embrace that you have different skills

• Testers and programmers have different skill sets, both are valuable
• You probably know more when it comes to testing, and programmers can learn about testing from you

You’ve been hired for a reason

• The company hiring you did see something in you
• They have seen other candidates, and decided you’re the best to get the job done

Embrace what you don’t know

• It’s ok to not know everything

Pair with people

• It’s a great chance for learning.
• Even if you don’t know much about the code, you will learn something.
• Also the programmer will learn by the questions that you ask

On Agile Testing Days 2019, Damian Synadinos gave a keynote with the topic „More Than That“. His basic message was, everybody could put more than one „label“ on herself. We’re more then testers or developers, we’re also human beings, parents, children, husbands, wifes, players, musicians… and so on.

Another important advice Damian gave is „un-just yourself“. And he means that we should not make ourselves small. We’re not „just“ testers; we are testers. That resonated strongly with myself, already during that talk.

Why? I think I’m using „just“ way too often. For example at work, when I’ve filed a bug, and a developer or project manager approaches me and asks: „How the heck did you spot that? That’s such a difficult thing to find!“, I used to reply „I’ve just done my job.“.  And thanks to Damian I now have a totally new perspective on that: I found that bug by doing my job, by being a tester. I’m in that field for nearly 15 years, I learned it, from the ground, attented trainings, workshops, meetups & conferences, I’m reading books & blogs, I’m practicing testing to constantly get better at it.

I’m a professional software tester, that’s why I’m able to spot that difficult-to-find-bug, that a lot of other people would not have found. I’m a tester, not „just“ a tester.

After that talk, during visiting other talks at the conference, I noticed how often other speakers were using „just“ in their talks, and by that diminuishing something they did, they achieved. By that, people are also making their own achievements make appear smaller than they actually are. Also, most probably, this is not good for one’s own self-confidence.

I even think that’s my most important takeaway from this year’s Agile Testing Days: Un-just yourself!

See also what labels Stephan Kämper is putting on himself: Calling Me Names — New Labels