TMMi-P_Syll2020 TMMi - Test Maturity Model Integration Professional Question and Answers

As part of theProduct Quality Evaluationprocess area (4.2), the specific goal"Project Goals for Product Quality and their Priorities are Established"includes the practice ofidentifying product quality needs. This involves understanding and defining what quality means for the product, determining quality characteristics (such as reliability, usability, etc.), and establishing priorities for these characteristics based on project goals.

Other practices such asestablishing test process measuresandmeasuring product quality quantitativelycome later in the lifecycle or are part of other specific goals.

TMMi References:

InProduct Quality EvaluationatTMMi Level 4, identifying product quality needs is critical for setting up measurable and prioritized goals for product quality​​.

During theDiagnosing phaseof a test improvement cycle, in addition to conducting an assessment to determine the current maturity status, the activity of developing recommendations is also performed. These recommendations provide guidance on what actions should be taken to achieve the desired improvements. The assessment helps establish the organisation's current state, and the recommendations serve as a roadmap for moving forward.

In theActing phaseof the IDEAL model, the focus is on the implementation of the proposed solutions. Activities such as refining the solution and implementing the solution are key components of this phase. These actions are based on the plan developed in the earlier phases, and they are designed to apply the solutions to achieve the desired improvements. Specifically:

Refine solutions (i)refers to improving or adjusting the solution based on feedback from pilots or initial tests.

Implement Solution (iii)refers to the full implementation of the refined solution across the test process or organisation​​.

TheAdvanced Reviewsprocess area, which is part ofTMMi Level 4, is concerned with measuring product quality early in the lifecycle. It aims to enhance thetest strategyandtest approachby aligningstatic testing(reviews and inspections) withdynamic testing(execution of test cases). The results and data from static testing are used to optimize the dynamic testing approach, making the overall testing process more efficient and effective.

TMMi References:

Advanced Reviewsat TMMi Level 4 focuses on integrating static and dynamic testing approaches to optimize test strategy and product quality evaluation​​.

The specific practice"Establish the Organization’s Test Process Database"is found within theTest Lifecycle and Integrationprocess area, which is part ofTMMi Level 3. This process area aims to establish standardized test processes across the organization, including maintaining atest process databasethat holds process-related information and data for reuse and continuous improvement.

TMMi References:

TheTest Lifecycle and Integrationprocess area is responsible for establishing and maintaining organizational test process assets, including the test process database​​.

The description provided—where testing ismulti-level, with distinct test levels (component, integration, system, and acceptance), and where the processes of testing and debugging are differentiated—applies toTMMi Level 2 (Managed). At this maturity level, testing is separated from debugging, and there are formalized processes for different testing levels. A company-wide or program-wide test strategy is established, and specific testing objectives are defined for each test level.

TMMi References:

TheTMMi Level 2framework defines structured testing withmultiple levels, including component, integration, system, and acceptance testing, as well as the separation of testing and debugging​​.

The absence of amaster test plan spanning multiple test levelspoints to a weakness in theTest Lifecycle and Integrationprocess area, which is aTMMi Level 3process area. This process area focuses on ensuring that testing is integrated throughout the development lifecycle and includes the development of comprehensive test plans (including master test plans) that cover multiple test levels and align with the overall testing strategy.

TMMi References:

TheTest Lifecycle and Integrationprocess area atTMMi Level 3addresses the need for comprehensive and integrated test planning across all test levels​​.

The description provided, "Testing is a thoroughly defined, well-founded, and measurable process. Testing is perceived as evaluation; it consists of all lifecycle activities concerned with checking products and related work products," corresponds to TMMi Level 4, which is labelled "Measured." At this level, testing processes are quantitatively measured, and there is a strong focus on evaluating product and process quality using quantitative metrics. Testing is integrated throughout the lifecycle and becomes a predictable, measurable process that supports organisational goals.

In the scenario described, testing is performed ad hoc by business analysts who do not have defined test roles or a structured career path within the organisation. The process area that needs attention here isTest Organization(TMMi Level 3). This process area is crucial for establishing a structured test group with dedicated roles and responsibilities for testing, rather than relying on business analysts for testing as a secondary task.

TheTest Organizationprocess area focuses on establishing a formalised test organisation within the company, defining test functions and responsibilities, and promoting testing as a recognised profession with defined career paths. This would address the shortcomings of the current setup by creating a dedicated, skilled testing team, thus improving the consistency and quality of testing activities within the bank​​.

CMMI and TMMi complement each other in many ways. While CMMI focuses on improving broader software development processes,TMMispecifically addresses testing. Implementing CMMI provides value to test organizationsby enabling a structured and continuous improvement approach that can incorporate TMMi as part of the overall process.

CMMI’s continuous representation allows organizations to implement specific process areas related to testing (like verification and validation) and integrate TMMi practices forcontinuous improvementin their testing processes. Organizations followingTMMimay also refer to CMMI practices for complementary process improvements in areas such as configuration management, planning, and performance measurement.

TMMi References:

TMMi outlines how CMMI’s process areas, such asVerification and Validation, provide critical value for testing organizations​​.

CMMI’s structure supports continuous process improvement that aligns well with TMMi’s structured approach to test process maturity​.

The generic practice"Establish a defined process"addresses the need to tailor a managed process from the organization’s set of standard processes. This practice is part ofGeneric Goal 3 (GG 3), which aims to ensure that processes are standardized and defined across the organization. By tailoring processes based on established guidelines, organizations can create specific implementations that fit their projects while maintaining consistency and control across the organization.

TMMi References:

In TMMi, establishing a defined process involves tailoring standard processes to suit specific project needs, ensuring the institutionalization of best practices across projects​​.

The specific goal "Project Goals for Product Quality and their Priorities are Established" belongs to theProduct Quality Evaluationprocess area, which is part of TMMi Level 4. This process area focuses on defining measurable and prioritised project goals for product quality and managing actual progress towards those goals throughout the lifecycle. Product quality goals typically include both functional and non-functional aspects of the system.

Planning pokeris an Agile technique commonly used to estimate effort, and it can be adapted for use in theTest Planningprocess area to help estimate test efforts (SG3: Establish Test Estimates). This collaborative technique brings stakeholders together to estimate the effort required for tasks by assigning numerical values in a consensus-driven manner. It is particularly effective in Agile environments where estimation is iterative and frequently updated based on evolving project needs​.

Informal assessments, as described in the TMMi framework, are often quicker and cheaper as they do not require the extensive corroboration of evidence that formal assessments do. One key characteristic of informal assessments is that they may rely solely on interviews, without using other forms of corroboration such as documentation reviews or observations. These informal assessments provide an indicative view of the organization's maturity level but do not result in formal certification​

The Learning Phase in the IDEAL improvement framework is the final phase in the cycle and plays a crucial role in ensuring continuous improvement. Contrary to the belief that it is less important because most work has been done in previous phases, this phase is critical for learning from the experience gained during the improvement program. The Learning Phase ensures that the organisation refines its ability to implement change effectively.

The key activities in this phase are:

Analyze and Validate:This involves reviewing what has been achieved, whether the goals have been met, and identifying lessons learned. It focuses on what worked well and what could be done better.

Propose Future Actions:Based on the analysis, recommendations are made to improve future programs. These proposals are often forwarded to management for consideration.

Thus, the Learning Phase is essential for organisational growth as it solidifies the learning from past improvements and applies it to future cycles, fostering continuous refinement of processes.

Regularcode reviewsandarchitectural reviewsare part of theAdvanced Reviewsprocess area, which belongs toTMMi Level 4. This process area emphasizes conducting static reviews (e.g., code and design reviews) early in the lifecycle to measure product quality and adjust the testing strategy based on the insights gained from these reviews.

TMMi References:

TheAdvanced Reviewsprocess area in TMMi Level 4 aims to ensure that static testing activities such as code and architectural reviews are used to measure product quality early in the lifecycle​​.

In a staged model like TMMi, an organisation follows a predefined improvement path that includes a set of process areas organised by maturity levels. Each level builds upon the previous one, and the organisation progresses in a step-by-step manner through these maturity levels to improve its processes. The staged model is well-structured and prescriptive, offering a clear improvement roadmap for the organisation.

Management commitment is highlighted as one of the critical success factors in the context of implementing and sustaining test process improvements. Within the TMMi framework, senior management support is crucial for ensuring that necessary resources, funding, and priority are allocated to the improvement initiatives. This commitment must be sustained throughout the process to overcome potential resistance and ensure that the improvements are institutionalised. Without strong management commitment, the efforts may lose momentum, and the improvement goals may not be achieved.

This factor is critical across several levels of TMMi, as seen in processes related to institutionalising the test process (such as defining organisational policies, resource allocation, and performance evaluation).

At TMMi Level 5, theQuality Controlprocess area becomes relevant as it focuses on statistically managing and controlling the test process. Since the organisation is already at TMMi Level 4, moving to Level 5 involves ensuring that the test process is fully predictable, measured, and statistically controlled.Quality Control is a key process area in achieving TMMi Level 5 compliance, alongside other areas such as Defect Prevention and Test Process Optimization.

The three TMMi Level 5 process areas are as follows:

Defect Prevention(A): The key specific goal here is todetermine common causes of defects. This process involves analysing defect patterns and establishing actions to prevent similar defects in the future.

Quality Control(B): This area focuses onevaluating new testing technologies to determine their impact on the testing process. Statistical methods help predict product quality, and these evaluations drive improvements in the test process.

Test Process Optimization(C): The goal here is toestablish a statistically controlled test process, where baselines are set, and improvements are implemented in an orderly manner​​​.

An indirect benefit of a test improvement programme is improved staff motivation. While direct benefits, such as shortened lead time or higher defect detection rates, are focused on measurable improvements, indirect benefits include enhanced morale and engagement of the staff involved. A well-structured test improvement programme can lead to better recognition of the testing function within the organisation, leading to increased satisfaction and motivation among testing professionals.

Test Design and Execution is a key process area at TMMi Level 2, not Level 3. However, other process areas at TMMi Level 3 include Non-functional Testing, Peer Reviews, and Test Lifecycle and Integration. Test Design and Execution plays a foundational role in laying down the practices for creating and executing test cases, especially at earlier maturity levels. By TMMi Level 3, processes become more sophisticated with a broader range of testing techniques and integration into the overall lifecycle.

TheActing phaseof a test improvement cycle focuses on executing the planned improvements. Key activities during this phase includecreating solutionsto address identified problems,planning actionsto implement those solutions, andpiloting solutionsbefore full-scale deployment. Thesesteps ensure that the improvements are well-defined, tested in a controlled environment, and refined before being fully implemented across the organisation.

Required componentsdescribe what an organizationmustcomply with to satisfy a process area in the TMMi model. These components includespecific goalsandgeneric goals, and they form the basis for determining whether a process area has been successfully implemented during assessments.Expected componentsprovide guidance on how to achieve the required goals, but they are not mandatory.

TMMi References:

TMMi definesrequired componentsas those that organizations must achieve to satisfy a process area​​.

TheTMMi process area Test Planningis indeed supported by severalCMMI process areas, includingProject Planning,Requirements Management, andRisk Management. These CMMI areas provide foundational support for ensuring that the test planning process incorporates adequate project management, requirement traceability, and risk identification, all of which are critical for effective testing.

Specific Goal 2 of the CMMI Validation process arearelates to validation activities, not peer reviews, and therefore does not support the TMMi Peer Reviews process area.

Causal Analysis and Resolutionis linked toDefect Preventionin TMMi, not Quality Control.

Performance Managementdoes not directly support the Non-Functional Testing process area in TMMi.

TMMi References:

TheTMMi Test Planningprocess area is supported by relevant CMMI process areas, such asProject Planning, which helps in aligning project goals with test planning activities​​.

Thedebugging-oriented phaseof the evolutionary testing model aligns closely with TMMi Level 1 (Initial). At TMMi Level 1, testing is often chaotic and is not formally separated from debugging. The primary focus is on finding and fixing bugs, which reflects the characteristics of the debugging-oriented phase described in the Gelperin and Hetzel evolutionary testing model. Testing at this level lacks structure and is mainly an ad hoc activity aimed at removing defects from the software.

This statement is related to adefined process(Generic Goal 3), not amanaged process(Generic Goal 2). A managed process (GG2) refers to processes that are planned, executed inaccordance with policy, involve relevant stakeholders, are monitored and controlled, and are reviewed for adherence. Tailoring from the organisation's set of standard processes applies to a defined process, which is established under GG3.

The specific practice"Define product risk categories and parameters"is part of theTest Policy and Strategyprocess area in TMMi. This practice involves identifying and categorizing the product risks to guide testing efforts and establish a clear understanding of potential product issues. Product risk categories help in determining the focus areas for testing, ensuring that testing is prioritized based on business and technical risks.

TMMi References:

The practice of defining product risk categories is directly linked to theTest Policy and Strategyprocess area at TMMiLevel 2, where risk-based testing approaches are established​​.

TheTMMi modelis designed to be versatile and applicable acrossall test levels(including static and dynamic testing) andall types of development methodologies, such aswaterfall,Agile, andDevOps. It provides comprehensive coverage of the testing process, ensuring that all critical aspects of testing—such as lifecycle processes, techniques, infrastructure, and organization—are addressed.

TMMiis not limited to specific development methods or organizational sizes; it islifecycle-independentand can be used effectively with various development approaches, making it widely applicable in bothsmallandlarge organizations.

TMMi References:

TMMi coversall test levels, including bothstatic testing(such as reviews) anddynamic testingat multiple stages, and is applicable across different software development lifecycles​​.

Agile organisations often focus on rapid iterations and minimal documentation, which can lead to the omission or underuse of structured testing practices.TMMiserves as a valuable guide for Agile teams by highlightingcritical testing practicesthat might be overlooked, ensuring that essential quality processes such as risk analysis, peer reviews, and test planning are not neglected​​​.

This reminder ensures that teams maintain a disciplined approach to testing while still benefiting from Agile flexibility.