During our latest book study group session some of the members of the group stated that they aren’t interested in reading the whole book, since they would not benefit from every chapter. So we decided to read the (at first sight) most interesting (looking) chapters first and then see how to go on.
So as next chapter to study and discuss, we decided on „Chapter 18, Coding and Testing“.
First of all, I have to admit, that from the title I would have expected a more technical content; I expected to see things like examples of code and unit tests, see automated tests and stuff like this, but this wasn’t the case, and at the beginning I was a bit surprised. 😉
But here we go, my notes & summary on the chapter:
Chapter 18, Coding and Testing
- Start simple First tests to write should follow the happy path, to check wether the core functionality is working
- Why test automation: Cause manual tests are too time consuming! Failing tests should show _why_ they’re failing
- Creating automted tests might be huge effort, but by starting with the simple cases you „keep focus on designing the automation solution“
- Add complexity:
- When the „happy path“ is working, add test cases for boundary and edge conditions, to challenge programmers, testers, requirements, the important thing is „talking“.
- Make notes about potential scenarios for further manual exploratory testing
- Asses risk: Classic „risk based“ testing approach?
- Agile development may already consider high risk by giving high-risk stories higher estimates. (!)
- Might be helpful to add all the risks (e.g. „ilities) to a complex story and rate them by impact to the business
- Do same for likelihood, multiply both to get total risk
- -> Common sense? We’re already doing something similiar: POs/PMs, DEVs & testers are rating the importance of a test case, combined result is overall prio of that test case
- Business Domain makes the big difference here -> e.g. in our case things that effect „patient safety“
- History is a good teacher.
- -> Whom else to ask about (input for) estimation/ evaluation about impact and probability?
- Coding and Testing progress together(!): All team members make sure that a story is finished
- Identify Varations
- Add working variations, add variations that will fail/ end up in exceptions -> „iterative back and forth between coding and testing“
- „Different teams take different approaches“ -> Point is: Coding and testing are part of one (development) process in which all team members participate
- Ongoing process to add more and more test cases with more and more variations (which may not all end up in the regression test suite (-> What if the costs for automating such tests are (nearly) zero?)
- Power of three Having three different point of views is a great (starting) point to clearify if things (e.g. requirements) aren’t unclear.
- E.g.: programmer, tester, product owner
- Maybe it’s necessary to pull more people (up to whole team) into the discussion
- Focus on one story (at one time)
- Try to get one story done before starting the next one
- -> Not sure if this is feasible all the time in praxis (but of course great approach/ advice)
- Tests that critique the product
- When automated tests are passing, explore functionality more deeply
- use task cards for business& technology-facing tests (see chpt. 10 & 11 for more)
- Try to execute end-to-end tests to identify gaps (as early as possible in the process!)
- You might find „nice to have“ issues/ features now
- If „significant functionality“ is missing, write stories for later iterations
- Collaborate with programmers -> !Really close collaboration!
- Pair testing (possible approach: programmer „drives“ during the session)
- „Show me“
-
- Better then entering defect into bug tracker
- Difficult in distributed (location, time) teams -> Maybe screen videos might help/ assist?
- „Speaking out load“ very often helps („Rubber Ducking“)
- Talk to customers
- Show customers
- Understand the business (-> how to compensate if (nearly) impossiple? -> get business experts in your team might be one solution)
- Completing testing tasks -> „Agile testers are proactive.“
- Remember that anyone on the team can sign up for testing tasks.“
- Better to drop one or two stories and finish others completely (including testing) then having a couple of unfinished (= untested) stories.
- „Everyone on the team also must be willing to take on manual testing tasks.“ -> „this task provides great motivation for learning how to design the application to facilitate test automation“ -> 😉 -> See chapter 21 „Key success factor“. 1. Whole team approach „testable code will be designed, when the whole team is responsible for testing“
- Dealing with bugs
- When to track/ fix a bug in which way?
- „What is a bug“ -> Customer decides, so it’s not important how to name it (bug, feature, ’needs to be fixed‘)
- Customer decides on priority
- Technical debt: Not fixed bugs have (long term) negative influnce on whole system
-
- Zero Bug Tolerance:
- Communication !
- Co-Location
- developers and testers need to be commited
-> Does not mean „bug free“!
- It’s all about choices Approach might be different for every team, following things to be considered to make the right choice
- Unit test failures: don’t log
- Failures in higher level regression tests: Failing test is type of recorded bug, but maybe more info needs to be added
-
- Story bugs within the Current Iteration:
- Don’t log bugs that can be fixed immediately
- Adjust your tests
-
- Post iteration bugs (or thise that can’t be fixed immediately)
- Log if can’t be fixed immediately
- Might be missed requirement -> story for future iteration
-
- From the legacy system
- Log! (if PO decides that it is worth to fix them)
- Log only if they will be fixed
- Found in production: Log!
- Decide which bugs to log
-
- Choose when to fix your bugs
- Fix now (in general: cheapest)
- Fix later
- Never fix: Close it, don’t keep it open, cause that would simulate that you gonna fix it
- -> Feasible for big organizations with hundrets of customers?
-
- Choose the media you should use to log a bug
-
- Index cards
- disciplined agile team wants to fix bugs within iteration
- make bugs visible for the team
- can be combined with DTS
-
- Defect tracking system
- distributed team
- track bugs for audit purposes/ capture in release notes
- bugs that escaped iteration
- legacy system (with high number of bugs)
- Does not mean all bugs have to be logged; decision on which ones to log should be smart 😉
- None at all: Cover the bug with an automated test
-
-
- Alternatives and Suggestions for Dealing with bugs: Teams evolve while getting mature in using agile, so bug handling shoul/ will evolve
- Set Rules
-
- Fix all bugs
- correlation „low prio“ and „quick to fix“
- small isolated bugs may evolve to large, tangled bugs
- Combine bugs -> New story
- Treat it as a story (if it is missed functionality)
- Blue, green and red stickers
- Start simple: When finding lots of bugs in new code, analyze cause!
- Daily standup, Visibility, Even overhead of having story board eletronically and physically might be worth the effort
- Testers facilitate communication: Power of three
- Distributed teams (see Chpt. 9)
- -> work harder at communications!
- use retrospectives to improve
- Regression tests Should run automated on continual/ daily build process
-
- Keep the build green
- Fixing the cause for a failing unit test, should be the team’s highest prio
- Stop development of new features to do so
- -> „Keeping the build running is serious busisness.“
- Keep the build quick, to provide immediate feedback: tests that take too long should run in a seperate build process
-
- Building a regression suite
- Add new automated tests to the regression test suite as soon as they pass
- Regression tests should be kept under version control, maybe together with the production code; at least daily backuped
- When moved to regression test suite, purpose of test changes: No longer to find new bugs, but to prove existing functionality is still working -> Common sense
- Checking the „big picture“
- Resources
- test environments, test data, test tools
- Collaboration with other team members, db experts, sys admins to setup needed additional infrastructure
- Cooperate closely with external resources brought in to support specific tasks
- Iteration metrics Before starting to measure, know what problem you’re gonna solve!
- Measuring progress: You need to know about the „status quo“, tools: burndown chart, estimated vs. actual time, number of test task cards in the „to do“ column -> „Visibility!“
-
- Defect metrics: Consider goal before starting to measure
- Try to find root cause for (groups of) defects and solve it
- „go for simplicity“
- drop metrics if they aren’t useful anymore
- if your organization desires metrics, use the simplest way to satisfy there needs (examples on page 440)
Thoughts/ Ideas/ Questions aka „my two cents“
-
- Two recurrent themes in the chapter:
- It’s often stated to „start simple“, and add complexity later -if needed -> This reminded me to the KISS-principle: Keep it simple, stupid! http://en.wikipedia.org/wiki/KISS_principle
- „Visibility“: It’s stated very often, that things (issues, problems, progress) should be visible. This is imho very closely tight to „feedback“ and „short iteration“, which are both improved by visibility. As I understood it, these are key-factors of „agile“.
- Keeping the customer in the loop is a big challenge in big global organizations which have dozens of teams and hundrets of customers, each spread over the whole globe.