TMMi-P_Syll2.1 Questions and Answers

In the Initiating phase of a test improvement cycle, one of the key activities is to Charter Infrastructure. This activity involves setting up the structure needed to support the improvement effort, such as establishing a project board and defining responsibilities. This phase focuses on creating the foundational support for the test improvement initiative.

In the context of TMMi, the focus of test process improvement is usually driven by business needs that aim to improve the efficiency, effectiveness, and predictability of the testing process. The typical business reasons for test improvement include:

Increase predictability of testing (Option B): This refers to making the testing process more reliable and consistent, allowing better planning and forecasting of testing activities.

Reduce the costs of failure by improving effectiveness of testing (Option C): This is a direct goal of test process improvement, as it helps catch defects earlier, reducing costs associated with defects in later stages of the lifecycle.

Reduce time to market by increasing efficiency of testing activities (Option D): By making testing more efficient, organizations can release products faster while maintaining or improving quality.

Option A, Introduce a new process improvement framework, is NOT a typical business reason for test improvement. This refers to the introduction of a framework, which could be part of a process improvement strategy but is not a direct business reason for improving testing. Test improvement efforts are generally focused on achieving tangible business benefits, such as cost reduction, risk mitigation, or faster delivery, rather than the implementation of a specific framework for its own sake.

TMMi References:

TMMi emphasizes business-driven goals for process improvement that align with reducing risks, enhancing quality, and improving testing efficiency and effectiveness​.

Agile organisations often focus on rapid iterations and minimal documentation, which can lead to the omission or underuse of structured testing practices. TMMi serves as a valuable guide for Agile teams by highlighting critical testing practices that might be overlooked, ensuring that essential quality processes such as risk analysis, peer reviews, and test planning are not neglected​​​.

This reminder ensures that teams maintain a disciplined approach to testing while still benefiting from Agile flexibility.

Management commitment is highlighted as one of the critical success factors in the context of implementing and sustaining test process improvements. Within the TMMi framework, senior management support is crucial for ensuring that necessary resources, funding, and priority are allocated to the improvement initiatives. This commitment must be sustained throughout the process to overcome potential resistance and ensure that the improvements are institutionalised. Without strong management commitment, the efforts may lose momentum, and the improvement goals may not be achieved.

This factor is critical across several levels of TMMi, as seen in processes related to institutionalising the test process (such as defining organisational policies, resource allocation, and performance evaluation).

ISO/IEC 15504, which is known for its process assessment model, was not one of the sources used during the development of the TMMi model. The TMMi model was developed using inputs from sources such as the Capability Maturity Model Integration (CMMI), international testing standards (like those from ISO and IEEE), and industry testing best practices surveys. While the ISO/IEC 15504 model has relevance to general process assessments, it was not specifically referenced in the creation of the TMMi.

This statement is false because informal assessments do not provide an official result on test process maturity. Informal assessments are designed to give indicative information only and focus on identifying key areas for improvement without leading to a formal maturity rating or certification. In contrast, formal assessments are more comprehensive and provide an official maturity rating when conducted under an accredited assessment method.

The statement that all defects should be analyzed to prevent as many defects as possible is FALSE. In the Defect Prevention process area at TMMi Level 5, only a subset of defects—those that are recurring or considered to be of significant impact—are analyzed. Analyzing all defects would be inefficient and not always necessary. Instead, a targeted approach is used to identify common causes of defects and implement actions to prevent their recurrence.

TMMi References:

The Defect Prevention process area is based on the analysis of a subset of defects to find and eliminate common causes​​.

The TMMi model is designed to support test process improvement in both software engineering and systems engineering. This is critical because it addresses the needs of total system development, which may include hardware and software. The model covers various testing activities across different disciplines, allowing organisations to improve their test processes regardless of whether they focus solely on software or integrated systems.

In the Defect Prevention process area at TMMi Level 5, one of the key specific goals (SG2) is to prioritise and define actions to systematically eliminate the root causes of defects. This involves proposing solutions to address common defect causes that have been identified during causal analysis. These solutions aim to prevent defects from reoccurring across the development lifecycle.

With a 40% achievement in the Test Planning process area, the rating would be "Partly achieved." According to TMMi, a process area is rated "Partly achieved" when the percentage of achievement is between 15% and 50%. This indicates that some practices have been implemented, but significant improvement is still required to meet the full requirements of the process area.

The description provided, "Testing is a thoroughly defined, well-founded, and measurable process. Testing is perceived as evaluation; it consists of all lifecycle activities concerned with checking products and related work products," corresponds to TMMi Level 4, which is labelled "Measured." At this level, testing processes are quantitatively measured, and there is a strong focus on evaluating product and process quality using quantitative metrics. Testing is integrated throughout the lifecycle and becomes a predictable, measurable process that supports organisational goals.

The process area Test Lifecycle and Integration at TMMi Level 3 focuses on establishing a coherent test process across multiple test levels and integrating the test lifecycle with the development lifecycle. This process area addresses the creation of a master test plan, which is essential for ensuring consistency and avoiding redundancy or omissions across different test levels.

The specific goal and practice for establishing a master test plan are well documented within the Test Lifecycle and Integration process area. Practices include performing a product risk assessment, establishing the test approach, and developing the master test plan itself​​.

In the scenario described, where an organisation suffers from poor test estimations despite many practices being in place, the process area that requires attention is Test Planning. Specifically, SG 3 (Establish Test Estimates) focuses on developing well-founded estimates for testing effort and cost. This includes creating a top-level work breakdown structure, defining the test lifecycle, and determining estimates for the test effort and costs associated with different test activities. By addressing these practices, the organisation can improve its ability to generate accurate test estimations, leading to better project planning and resource allocation.

The TMMi model is designed to be versatile and applicable across all test levels (including static and dynamic testing) and all types of development methodologies, such as waterfall, Agile, and DevOps. It provides comprehensive coverage of the testing process, ensuring that all critical aspects of testing—such as lifecycle processes, techniques, infrastructure, and organization—are addressed.

TMMi is not limited to specific development methods or organizational sizes; it is lifecycle-independent and can be used effectively with various development approaches, making it widely applicable in both small and large organizations.

TMMi References:

TMMi covers all test levels, including both static testing (such as reviews) and dynamic testing at multiple stages, and is applicable across different software development lifecycles​​.

CMMI and TMMi complement each other in many ways. While CMMI focuses on improving broader software development processes, TMMi specifically addresses testing. Implementing CMMI provides value to test organizations by enabling a structured and continuous improvement approach that can incorporate TMMi as part of the overall process.

CMMI’s continuous representation allows organizations to implement specific process areas related to testing (like verification and validation) and integrate TMMi practices for continuous improvement in their testing processes. Organizations following TMMi may also refer to CMMI practices for complementary process improvements in areas such as configuration management, planning, and performance measurement.

TMMi References:

TMMi outlines how CMMI’s process areas, such as Verification and Validation, provide critical value for testing organizations​​.

CMMI’s structure supports continuous process improvement that aligns well with TMMi’s structured approach to test process maturity​.

The three TMMi Level 5 process areas are as follows:

Defect Prevention (A): The key specific goal here is to determine common causes of defects. This process involves analysing defect patterns and establishing actions to prevent similar defects in the future.

Quality Control (B): This area focuses on evaluating new testing technologies to determine their impact on the testing process. Statistical methods help predict product quality, and these evaluations drive improvements in the test process.

Test Process Optimization (C): The goal here is to establish a statistically controlled test process, where baselines are set, and improvements are implemented in an orderly manner​​​.

In the Acting phase of the IDEAL improvement model, the focus is on implementing the changes that were identified and planned in the earlier phases. The two main activities in this phase are Create Solution and Implement Solution. This phase generally consumes the most effort, as creating and deploying solutions to improve the processes are resource-intensive activities. These activities align with option D, making it the correct answer .

"Provide Necessary Test Training" is part of the Test Training Program process area, not the Test Organization process area. Test Organization focuses on establishing a structured group responsible for testing, defining roles, career paths, and implementing process improvements. Therefore, providing test training is not a specific goal of Test Organization but rather falls under the Test Training Program process area.

A direct benefit of a test improvement program is higher productivity. When test processes are streamlined, better test planning, design, and execution can lead to reduced testing times, fewer defects, and quicker turnaround of quality products. This has a direct impact on organisational efficiency, allowing for more effective use of resources and time.

For a formal TMMi assessment, both staff interviews and document study are required to collect sufficient evidence. The assessment must adhere to the TMMi Assessment Method Accreditation Requirements (TAMAR), which state that formal assessments require the corroboration of interview data with documentary evidence (artifacts). These can include documents, templates, or screenshots, ensuring that the evaluation is robust, repeatable, and auditable​​.

This statement is FALSE because TMMi is flexible and does not require the imposition of a predefined set of practices in Agile organizations. Instead, it encourages the adaptation of practices to fit the Agile context. TMMi allows for alternative practices that achieve the same goals as the expected practices, making it possible to integrate testing improvements in a way that aligns with Agile principles. The focus is on continuous improvement, not compliance with rigid practices.

TMMi References:

TMMi emphasizes flexibility in adapting practices, particularly in Agile environments where imposed processes would contradict Agile values of adaptability and continuous improvement​​.

For safety-critical systems, reliability testing is a key concern. The identified shortcoming relates to the lack of formal techniques and approaches for reliability testing. The Non-Functional Testing process area, at TMMi Level 3, addresses testing non-functional aspects like reliability, performance, and usability. This process area would need specific attention as it provides a structured approach to assess non-functional risks, define testing techniques, and execute tests to verify the reliability of the product, which is critical for medical systems.

At TMMi Level 3, Test Lifecycle and Integration builds on the practices of Test Planning introduced at TMMi Level 2. This process area focuses on establishing a standardised test lifecycle, integrating the test process with the development lifecycle, and coordinating the testing effort across multiple levels. It also includes creating a master test plan that builds upon the risk assessments, test estimates, and approaches defined at Level 2 during Test Planning.

By further refining these practices and introducing lifecycle integration, Test Lifecycle and Integration ensures that the testing processes are consistent, well-defined, and fully embedded into the overall project lifecycle​​.

One of the key differences between informal and formal assessments is that informal assessments are not as rigorous. Informal assessments are more flexible, quicker, and less costly; however, they do not provide the same level of detail or accuracy as formal assessments. The statement that "informal assessments provide a result just as accurate as with a formal assessment" is therefore false. Formal assessments involve multiple types of evidence and a more thorough evaluation, while informal assessments typically rely on fewer types of evidence (often only interviews)​​.

Both Test Organization and Test Training Program are process areas associated with TMMi Level 3 (Defined). At this level, the organisation establishes a formal test organisation and ensures that there is a dedicated test training program. This helps to institutionalise testing as a profession and ensures the systematic development of skills needed for testing activities across the organisation.

The description provided—where testing is multi-level, with distinct test levels (component, integration, system, and acceptance), and where the processes of testing and debugging are differentiated—applies to TMMi Level 2 (Managed). At this maturity level, testing is separated from debugging, and there are formalized processes for different testing levels. A company-wide or program-wide test strategy is established, and specific testing objectives are defined for each test level.

TMMi References:

The TMMi Level 2 framework defines structured testing with multiple levels, including component, integration, system, and acceptance testing, as well as the separation of testing and debugging​​.

At TMMi Level 5, the process areas Defect Prevention, Quality Control, and Test Process Optimization work together to support continuous process improvement. Specifically:

Defect Prevention focuses on identifying and analysing common causes of defects and proposing corrective actions to prevent them from recurring. This includes submitting test improvement proposals, which help Test Process Optimization fine-tune the testing process through these insights​​.

While Quality Control supports Defect Prevention through statistical methods and analysis, the correct relationship between Defect Prevention and Test Process Optimization involves submitting test improvement proposals to optimise the process.

Test Design and Execution is a key process area at TMMi Level 2, not Level 3. However, other process areas at TMMi Level 3 include Non-functional Testing, Peer Reviews, and Test Lifecycle and Integration. Test Design and Execution plays a foundational role in laying down the practices for creating and executing test cases, especially at earlier maturity levels. By TMMi Level 3, processes become more sophisticated with a broader range of testing techniques and integration into the overall lifecycle.