ISTQB - Foundation Level Agile Tester Recap
This is a summary that will help you to approve the Agile Tester certification test.
1 - Agile Software Development
1.1 - The fundamentals of Agile Software Development
The Agile Manifesto has 4 values:
- Individuals and interactions over processes and tools (people-centered)
- Working software over big documentations (time to market advantage)
- Customer collaboration over contract negotiation (customer requirements)
- Responding to change over following plan (change is more important)
- Satisfy the customer with continuous delivery
- Changing requirements
- Deliver software frequently (few weeks or months)
- Business people and developer must work together
- Build projects around motivated individuals
- Face to face conversation
- Working software is the primary measure of progress
- The team should maintain a constant pace indefinitely
- Technical excellence and good design
- Simplicity
- Self-organizing team
- The team reflects on how to become more effective and adjust
The whole-team approach means involving everyone with the knowledge and skills necessary to ensure project success (small team, same workspace, daily, stand-up meetings), promoting more effective and efficient team dynamics. Its benefits include:
- Better communication and collaboration in the team
- Enabling several skill sets
- Making quality everyone's responsibility (tester will work closely with developers and PO to ensure that the desired quality levels are achieved, to agree on the testing strategy and test automation approaches)
Early and frequent feedback helps the team focus on the features with the highest business value or associated risk, and these are delivered to the customer first. Its benefits include:
- Avoiding requirements misunderstanding
- Clarifying customer features requests
- Resolving quality problems early
- Providing information regarding its productivity and ability to deliver
- Promoting consistent project momentum
1.2 - Aspects of Agile Approaches
There are 3 Agile approaches:
1) Extreme Programming (XP):
It is defined by 5 values (communication, simplicity, feedback, courage
and respect), some principles (humanity, self-similarity, improvement,
diversity, reflection, flow, opportunity, failure, quality and more) and
some practices (sit together, whole team, pair programming, stories,
weekly cycles, slack, continuous integration, test first programing,
etc)
2) Scrum: It has the following instruments and practices: sprint, product increment, product backlog, sprint backlog, definition of done, time boxing, and transparency. It has 3 roles: Scrum Master, Product Owner and development team.
3) Kanban: It has 3 instruments: Kanban board, working in progress limit (amount of parallel active tasks is limited) and lead time.
Poor specifications are often a major reason for project failure. The User Stories (US) must address both functional and non-functional characteristics, including acceptance criteria to consider the task finish. It is the conjunction of 3 elements: Card, Conversation (how the software will be used) and Confirmation (acceptance criteria to confirm that story is done).
A tester may use the INVEST technique: Independent, Negotiable, Valuable, Estimable, Small and Testable.
A Retrospective is a meeting help at the end of each iteration to discuss what was successful, that could be improved and how to incorporate the improvements and retain the successes in the future iterations (only a few improvements per iteration). All team members, testers and non-testers, can provide input on both testing and non-testing activities.
Delivery of a product increment requires reliable, working, integrated software at the end of every sprint. A Continuous Integration process has the next automated activities: static code, compile, uni tests, deploy, integration tests and report. It allows to run automated tests regularly, send feedback to the team on the quality of the code, and reduce the time taken to deliver. Continuous integration can provide these benefits:
- Early detection and root cause analysis of any problem
- Gives regular feedback
- Keeps the version
- Reduces regression risk
- Provides confidence
- Makes progress toward the completion
- Eliminates the schedule risks
- Provide constant availability of the executable software
- Reduce the repetitive manual testing activities
- Provides quick feedback
The Release Planning defines and re-defines the product backlog (few months ahead of the start of a project). The testers are involved on the following activities:
- Defining testable user stories
- Quality risk analyses
- Estimating testing
- Defining the necessary test levels
- Planning the testing for the release
In the Iteration Planning, the team selects user stories from the prioritized release backlog. The number of stories selected is based on established team velocity and the estimated size of the stories. The testers are involved on the following activities:
- Participating on the risk analysis
- Determining the testability of the US
- Creating acceptance tests
- Breaking down US into tasks
- Estimating testing effort
- Identifying functional and non-functional aspects of the system
- Participating in test automation
2 - Fundamental Agile Testing Principles, Practices and Process
2.1 - The difference between Testing in Traditional and Agile Approaches
The tester need to understand the differences between the traditional lifecycle models and agile lifecycle.
At the beginning of the project, there is a release planning meeting period (high-level risk analysis and estimation of the test effort required for each feature), followed by a lot iterations, at the beginning of each iteration, there is an iteration planning period. Testing activities occur throughout the iteration.
Best practices: No feature is considered done until is has been integrated and test with the whole system; address defect from the previous iteration at the beginning of the next iteration; in some cases delivery occurs at the end of each iteration.
Test automation occurs at all levels of testing. Developers will focus on creating unit tests, and testers should focus on creating automated integration, system, and system integration tests.
Project work products has 3 categories: business-oriented work products, development work products and test work products.
In Agile project, it should avoid producing many documentation, focus is more on having working software. User stories are the Agile form of requirements specifications, it should define a feature small enough to be completed in a single iteration. Epics may include user stories from different development teams.
There is often a parallel process of regression testing occurring throughout the iteration. Once, the first user story is ready for such testing, this can involve executing functional test and non-functional tests as well. Also, close of each iteration, internal alpha tests and external beta may occur.
In some cases, due to the duration of the functional tests, these are separated from the unit tests and run less frequently. If any automated test fails, the team should fix the underlying defect in time for the next code check-in.
Independent testers (e.i contractor company) are often more effective at finding defects. Developers working together with testers could do many of the testing tasks, but the testers could loss the focus and objective evaluation. Other approach is fully independent, assigning testers on-demand during the final days of each sprint. The last option, is where testers are assigned to Agile teams on a long-term basis at the beginning of the project.
2.2 - Status of Testing in Agile projects
Agile teams may use tools that automatically generate reports and burndown charts to track the progress across the entire release (amount of work left to be done against time allocated to the release or iteration). Also, the agile teams perform customer satisfaction surveys to receive feedback.
Any issue that may block test progress are communicated during the daily stand-up meeting. Some metrics that can be used: test pass/fail rates, defect rates, test results, defect density, defect found and fixed, requirements coverage, risk coverage, code coverage, and code churn to improve.
When the code change in the current iteration, the testers need to verify no regression has been introduced on features that were developed and tested on previous sprints. So, to maintain velocity, it is critical that the team invest on automation as early as possible and kept them up-to-day/maintained with each iteration, also to provide rapid feedback on the product quality.
Results from the build verification test will provide instant feedback on the software after deployment, so the team don't waste time testing an unstable build. As soon as an automated regression test fails, the team stops and investigate the reasons for the failing test (to fix the defect prior to progressing with new features).
It is critical that testers have the ability to quickly identify and update test cases from previous iterations that are affected by the changes made in the current sprint.
Testing tasks may be automated: data generation, loading test data, deployment, restoration and comparison of data outputs.
2.3 - Role and Skills of a tester in an Agile team
Each team member is responsible for product quality. The testers must closely collaborate with all other team members and with the business stakeholders. Testers skills on an Agile team:
- Be positive and solution-oriented
- Display critical and quality-oriented
- Acquire information from stakeholders
- Evaluate, report test results and progress
- Define testable user stories
- Collaborate within the team
- Respond to change quickly
- Plan and organize their own work
The testers activities (role) in an Agile team are:
- Implement and update the test strategy
- Measuring and report the test coverage
- Proper use of testing tools
- Managing test environment and test data
- Reporting defects
- Coaching
- Testing tasks are scheduled during the release and iteration planing
- Collaborating with developers and business stakeholders
- Participating in team retrospectives
Some test-related organizational risks:
- The testers may lose the appropriate tester mindset
- Testers become tolerant
- Testers can not keep pace (rhythm) within the incoming changes
3 - Agile Testing Methods, Techniques and Tools
3.1 - Agile Testing Methods
Testing practices to followed in every development project: writing tests in advance, focusing on early defect prevention, detention and ensuring the right test types are run at the right time.
Test-Driven Development (TDD): the test written are primarily unit level (UTs coded by developers), executed on the continuous integration and it is popular on XP methodology. It helps developers focus on clearly-defined expected results. The process is:
- Add a test that that capture the concept of the desired function
- Run the test (fail since the code does not exist)
- Write the code until the test passes
- Refactor the code
- Repeat the process
Acceptance Test-Driven Development (ATDD): it defines acceptance criteria and tests during the creation of user stories (the TCs can be manual or automated), it is a collaborative approach that allows every stakeholder to understand how the software has to behave. The ATDD create reusable tests from regression testing with specific tools that support creation and execution of such tests (understandable for every stakeholder), for example: Robot Framework. The first tests are the positive tests (happy path), then negative or non-functional tests.
Behavior-Driven Development (BDD): it allows a developer to focus on testing the code based on the expected behavior of the software (generate code that can be used to create test cases, for example: Cucumber), the tests are easier for everyone to understand, the BDD helps to collaborate with other stakeholders focusing on business needs.
The Testing Pyramid methods defined that the software may be tested at different levels (eliminating defects as early as possible), from the base of the pyramid to the top, unit, integration, system and acceptances test, having the large number of tests at the lower (bottom of the pyramid) and, as development moves to the upper levels, the number of tests decreases.
Teamwork emphasizes the whole-team approach consisting of developers, testers and business representatives working together. Best practices in Scrum teams:
- Cross-functional: a different set of skills to the team
- Self-organizing
- Co-located: whole sit together
- Collaborative
- Empowered: technical decisions are made by the team
- Committed: to question and evaluate the product's behavior
- Transparent: task visible in a board
- Credible: credibility in a test strategy
- Open to feedback: retrospectives allow teams to learn from successes and failures
- Resilient: respond to change
The Sprint Zero is the first iteration of the project, it sets the direction for what/how testing needs to achieve. The tester collaborates on the following activities:
- Identify the scope of the project
- Create an initial architecture
- Acquire and install needed tools
- Create an initial test strategy
- Initial quality risk analysis
- Define test metrics to measure
- Specify the definition of "done"
- Create the task board
- Define when to continue or stop testing before delivering
The test planing should start during the release planing session and be updated during each sprint.
Some agile testing practices include:
- Pairing
- Incremental test design: starting with simple tests and moving toward more complex ones
- Mind mapping tool
3.2 - Assessing Quality Risk in Agile projects
The goal on Agile or traditional is to reduce the risk of product quality to an acceptable level prior to release. Testers use the same techniques to identify quality risks, estimate the effort required to reduce those risks and then mitigate them through test design, implementation and execution.
The risk is the possibility of a negative or undesirable outcome or event. The level of risk is found by the assessing the likelihood (probability) of occurrence of the risk and the impact of the risk.
The quality risk analysis takes at 2 places:
- Release planning: business representatives provide a high-level overview of the risks. Also, estimate the testing effort
- Iteration planning: the whole team identifies the quality risks
Tasks with higher risks should start earlier and involve more testing effort then tasks with lower risk. After, the risk is identified for each task and selected the appropriate test technique(s) to mitigate it, then the tester designs, implements and executes tests to mitigate the risks (the risk can also be mitigated before test execution starts).
A common estimation technique used is the planning poker, after the PO reads the user story (the team ask questions to the PO), each estimator defined a Fibonacci sequence value (it can represent story points or effort days) at the same time. A high estimation usually means that the story is not well understood or should be broken down into multiples smaller stories.
3.3 - Techniques in Agile projects
The user story is an important test basis, but there are other test bases:
- Experience from previous projects
- Existing functions
- Code, architecture and design
- User profiles
- Information on defects
- Applicable standards
- Quality risks
To be testable, the acceptance criteria should address the following topics:
- Functional behavior
- Quality characteristics: how the system performs the specified behavior. Common quality characteristics are performance, reliability, usability, etc
- Scenarios: sequence of actions between the actor and the system
- Business rules: activities that can be performed under certain conditions
- External interfaces: connections between the system and the outside world
- Constrains: any restriction
- Data definition: format of the data, data type, allowed values, default values, etc
Each test level has its own definition of Done. Some examples could be:
1) Unit tests: 100% decision coverage, static analysis, all unit tests automated, no unresolved major defects
2) Integration tests: all functional requirements tested, all quality risks covered, no unresolved major defects, all regression tests automated
3) System tests: e2e tests of US, all user personas covered, all quality risks covered, all regression tests automated, all defects found reported, no unresolved major defects
Also, there are definition of Done for user stories (the stories selected are complete), features (design, code and documentation is complete, no major defects), iteration (any critical defect, all features for the iteration are completed and tested) and release (coverage, quality, time and cost).
Testers can apply traditional black box test design techniques such as equivalence partition, boundary value analysis, decision tables, and state transition testing to create tests. Non-functional requirements can be documented as user stories.
The exploratory testing is important in Agile projects due to the limited time available for test analysis and the limited details of the user stories, it should be combined with other techniques (analytical risk-based, analytical requirement), the test design and the test execution occur at the same time by a test charter.
A test charter provides the test conditions to cover during the a time-boxed testing session or session-based test management (from 60 to 120 minutes). It may include the following information:
- Actor: user of the system
- Purpose: objective the actor want to achieve
- Setup: what needs to put in place before starting the test execution
- Priority: based on the associated user story or risk level
- Reference: specifications (US), risks, etc
- Data: whatever data needed to carry out the test charter
- Activities: list of ideas of what the actor may want to do or would be interesting to test
- Oracle notes: how to evaluate the product to determine correct results
- Variations: other actions to complement the ideas described under activities
The quality of the tests depends on the testers' ability to ask relevant questions about what to test on the exploratory testing. During the test execution, the tester uses creativity, intuition, and skills to find possible problems with the product. They also needs to have good knowledge and understanding of the software under test, the business domain, and how to determine when the system fails. A set of heuristics can be applied when testing (boundaries, CRUD, configuration, interruptions).
Information that may be useful to document during the exploratory testing (save it in a way that makes it easy to understand the current status for all the testing on any test management tool):
- Test coverage: how much remains to be tested
- Evaluation notes: observations during the testing
- Risk/strategy list: risks have been covered
- Issues, questions and anomalies
- Actual behavior: recording the actual behavior of the system (video, screenshots)
3.4 - Tools in Agile projects
The Task Management and Tracking Tools (JIRA) have the following purposes:
- Record stories and test tasks
- Capture team members' estimations (calculate the effort required for the stories)
- Associate development tasks and test tasks with the same story
- Aggregate developer and tester updates to the task status
- Provide a visual representation and quickly check status
- Integrate with configuration management tools (automatically status updates for tasks)
Some tools to support communication and information sharing (complement and extend, not replace, face-to-face communication):
- Wikis: diagrams, feature discussions, photos, metrics, charts, dashboards
- Instant messaging, audio teleconferencing and video: real time communication, involve distributed teams and reduce telephone bills
- Desktop sharing and capturing tools
The configuration management tools (Git, Xray, TestRail) may be used not only to store source code and automated tests, but manual tests and other test work products too. It provides traceability between which version of the software were tested with a particular version of the tests.
Some additional tools used on Agile teams:
- Test design tools: like mind maps
- Test data preparation and generation tools
- Test data load tools
- Automated test execution tools: these tools allow testers and business stuff to express the expected system behavior in tables or natural language using keywords.
- Exploratory testing tools: capture and log activities performed on an application during an exploratory test session, like: Test & Feedback, Exploratory Testing, Evernote, etc
Visualization allows a single physical resource (server) to operate as many separate, smaller resources.
4 - References and practices
- http://www.erikvanveenendaal.nl/site/wp-content/uploads/CFTL-FA-Sample-Exam-v1.0a-Justifications.pdf