SDLC STANDARDS

VENDOR TEST STANDARDS

Version 0.2            July 9, 2004

This page contains the published standards for the Ministry's approach to the Vendor Test Phase of the System Development Life Cycle (SDLC).

Table of Contents

VERSION CONTROL

1.       INTRODUCTION

1.1     Purpose
1.2     Audience
1.3     Assumptions
1.4     Other Standards

2.       VENDOR TESTING
2.1     Unit Testing
2.2     System Testing
2.3     Integration Testing
2.4     Load Testing
2.5     Confidence Level

3.      VENDOR SITE USER TEST

4.      CONCLUSION

 VERSION CONTROL

This section of the document records the various versions or releases of this document.

Table 1: Version Control

[TOC]

1.     INTRODUCTION

This section outlines the Purpose, Audience, Assumptions and Other Standards associated with this standards document.

1.1      Purpose

The purpose of this standard is to document the expectations the Ministry has around testing during the Build Phase of a development project.  While it is recognized that each vendor may have their own development and test process, the Ministry requires and expects a certain level of testing to occur prior to delivery. This standard documents those testing requirements.

All recommendations or suggestions for the improvement of this standard should be sent for review to: wwwimbstds@srmwww.gov.bc.ca  

1.2      Audience

This document is directed at anyone building applications for the Ministry including vendors and Ministry staff. 

1.3      Assumptions

This standard assumes that anyone performing development work for the Ministry has a proven development and test strategy that they use for all development projects.

1.4      Other Standards

This standard is intended to be used in conjunction with other SDLC standards, especially the User Acceptance Testing (UAT) standard.

[TOC]

2.       VENDOR TESTING

Although this section is entitled Vendor Testing anyone developing applications for the Ministry (including Ministry staff) must follow these standards. 

It is expected that all applications will undergo the following test cycles in the Development Environment (normally at the Vendor Site) prior to the application being delivered to the Ministry.  Each and every delivery must go through these test cycles (including both new applications, enhancement releases or patches).  A brief description of the test cycle is provided, but each vendor may follow their own standards and processes for these cycles, as long as they meet the expectations described below.

2.1      Unit Testing

Each team member who is taking part in development should test his or her code thoroughly.  At the end of Unit Testing it is expected that the development team member has performed rigorous testing on all new or revised code.  For maintenance releases, any code which has not been modified but which may be impacted by changed code must also be tested. The individual developer must also keep sufficient testing documentation for future reference.

2.2      System Testing

After Unit Testing is completed by all team members it is expected that the various components of the system will be put together (built) to form a complete system.  At this point System Testing needs to be performed. Individuals who have not developed the code should test that the functions perform according to the specifications.  All aspects of any new or modified functions should be tested. Any modified code that may affect areas of the system that have not been modified must be tested as well.

2.3      Integration Testing

The vendor should perform an Integration Test by running through a set of standard business processes as provided by the business experts. Standards business processes should be tested to ensure that all components of the system needed for a single process perform according to the specification.  Testing must also satisfy multi-user access requirements and reasonable response time requirements.

2.4      Load Testing

While it is recognized that load testing may not be an easy task without an automated load test tool; the vendor must perform some level of Load Testing on all new applications or applications which have major functional or technical architecture changes.

2.5      Confidence Level

Upon delivery of a new version of an application the Ministry expects that the vendor is confident that the system will perform according to the specification.

[TOC]

3.     VENDOR SITE USER TEST

At the request of the Application Manager and agreement by the vendor, it is highly recommended that prior to delivery (and after all vendor testing is complete) that a Vendor Site User Test (VSUT) be performed.  The scope of this test would be for the Application Manager and/or designated user to come to the vendor site and spend one or several days (depending on the scope of changes made) testing that the application performs according to their expectations.  

Note: The VSUT is not a duplication of the User Acceptance Test (UAT). The VSUT should ensure that sufficient systems functions are working properly so that UAT can proceed after delivery. Examples of these critical system functions include: opening main screens, adding/updating/deleting records, ensuring user roles have the required authority, table look ups, and reporting capability.

Since the delivery process requires significant resource commitment from both the vendor and the Ministry, it is in everyone's best interest that this test be performed prior to delivery.  Experience has shown that through this process simple errors can be found and fixed at the vendor's site, thereby reducing errors during UAT and the number of deliveries before signoff.

[TOC]

4.     CONCLUSION

This standard outlines the tests to be performed during the Build Phase of the System Development Life Cycle.  Although the Vendor implements these tests, the Ministry requires that each test cycle described in this standard be completed prior to delivery of a new version or patch of any application.

[TOC]