Assessment Procedures and Test Requirements
To facilitate validation of the software vendor’s software lifecycle management practices, software vendors must produce appropriate evidence that confirms they have satisfied the control objectives defined within this standard. The test requirements identified for each control objective describe the expected activities to be performed to validate whether the software vendor has met the objective. Where sub-bullets are specified in a control objective or test requirement, each bullet must be satisfied as part of the validation. In addition, where terms such as “periodic,” “appropriate,” and “reasonable” are used in the test requirement, it is the software vendor’s responsibility to define and defend its decisions regarding the frequency, robustness, and maturity of the implemented controls or processes.
Test requirements typically include the following activities:
- Examine: The assessor critically evaluates data evidence. Common examples of evidence include software design and architecture documents (electronic or physical), source code, configuration and metadata files, bug tracking data and other output from software development systems, and security-testing results.
- Observe: The assessor watches an action or views something in the environment. Examples of observation subjects include personnel performing tasks or processes, software or system components performing a function or responding to input, system configurations/settings, environmental conditions, and physical controls.
- Interview: The assessor converses with individual personnel. The purpose of such interviews may include determining how an activity is performed, whether an activity is performed as defined, and whether personnel have particular knowledge or understanding of applicable policies, processes, responsibilities, or concepts.
The test requirements provide both software vendors and assessors with a common understanding of the expected validation activities to be performed. The specific items or processes to be examined or observed and the personnel to be interviewed should be appropriate for the control objective being validated as well as for each software vendor’s unique organizational structure, culture, and business practices.
When documenting the assessment results, the assessor identifies the testing activities performed and the result of each activity. While it is expected that an assessor will perform all the test requirements identified for each control objective, it may also be possible for a control objective to be validated using different or additional testing methods. In such cases, the assessor should document and justify why other testing methods were used and how those methods provide at least the same level of assurance as would have been achieved using the test requirements defined in this standard.
