In-Depth

The next step: Validating year 2000 compliance

• July 6, 2001

Many large I/S organizations are attacking the year 2000 problem in segments -- working through the inventory analysis, impact analysis and remediation phases of projects. As the year 2000 projects pass through these phases, managers face the critical task of verifying the fixed applications are year 2000 compliant.

This article sets out to define the essential components of year 2000 compliance and then describes how verification can be accomplished. The objective of the author is to shed light on a variety of year 2000 concepts, including Regression Testing, Date Simulation, Time Machine, Data Aging, Test Coverage Analysis and Test Data Generation. The first goal of the process is to define year 2000 compliance. An application is year 2000 compliant if it functions correctly when executed during the year 2000 and beyond with data files containing dates of year 2000 and beyond. To verify that an application is year 2000 compliant, it must meet the following conditions:

No Regression: If executed today, the modified program performs the same way as the unmodified program;

Year 2000 Execution: If executed under year 2000 dates, the modified program performs correctly, producing the expected results;

Year 2000 Data Handling: If executed under year 2000 dates with data files containing year 2000 dates the modified program performs correctly; and

Sufficiency of Test Data: Sufficient test data is available to exercise all of the modified application code.

Organizations must ensure developers on year 2000 projects understand these conditions and what work must be done to meet them. A more complete explanation of each condition follows.

Regression

To meet the "No Regression" condition, experts advise that, except in certain circumstances, no non-year 2000 functional changes be made when applications are modified for year 2000 compliance. Under these conditions, modified applications executed under current dates (1998 or 1999) should produce the same results as unmodified programs. In that case, the changed application has not regressed.

One possible exception to that rule is the use of four characters instead of two to display or store dates. In addition, this may shift the location of other fields on a line or the offset of other fields in a file record. For the purpose of verifying "No Regression" for year 2000 compliance, these differences should be intelligently ignored.

Since most applications require continuous maintenance and enhancements, it is essential that, after year 2000 compliance changes are made, the application is returned to production as soon as possible. To get the modified application into production mode, developers must ensure that it functions identically to current production code, except for the changes made to gain year 2000
compliance. If maintenance and enhancements were made to the production code while an application is undergoing year 2000 compliance changes, then the dual changes must be reconciled before the modified application is returned to production. Experts warn, though, the reconciliation is a difficult, time-consuming, and error-prone process and should be avoided. To avoid dual projects, developers should enact a code-freeze for a brief period to make year 2000 compliance changes
and verify there is no regression of the basic functionality. Once that project is completed, the application can be returned to production and the code freeze can be vacated.

If emergency fixes must be made to an application during a year 2000 compliance project, then the same changes must be made to both sets of code to avoid any difference in functionality. These differences should clearly be attributable to the emergency fixes.

Successful regression requires several components, including:

*Test Suites, which consist of Input information (Test Data Files, and Captured User Screen Input), Output information (Resulting Data Files, Resulting online screens, and resulting reports) and execution parameters;

*An ability to capture all screen input and output. This requirement includes the capacity to automatically feed the input to the application when a test needs to be re-run;

*A facility for comparing the old and new outputs. Such a comparison facility should be able to handle output screens and output reports as well as output data files. Each of the comparisons has slightly varying requirements. When comparing screens, developers must ignore the position of the screens (or Windows), and any fields, such as the time of execution, that are expected to change. Developers on the project must also ignore similar differences in reports. In case of data files, the comparison utility must be capable of handling the multitudes of file types, and multiple record types that may reside within the same file.

Some regression testing products shipping today include, in alphabetical order by company: Computer Associates International Inc. (CA-Verify and CA-Verify/EEO); Compuware Corp. (QA Hiperstation); Mercury Interactive (TestSuite 2000); Seec Inc. (AccuTest Suite); and Viasoft Inc.(VIA/AutoTest). The list of available file comparison products include: Computer Associates (CA-Accucheck); Compuware (FileAid); Viasoft Inc. (Comparex); and IBM (Super C).

Year 2000 execution
Once developers can verify that an application has not regressed, the software must be executed with the current date set to year 2000 dates. Since waiting until January 1, 2000 to test applications is not a viable option, organizations should create a simulated year 2000 environment. Correct execution under the simulated dates further validates the year 2000 compliance modifications. Experts suggest the following dates are best for testing: the rollover from December 31, 1999 to January 1, 2000; February 29, 2000 (it is a leap year); March 31, 2000 (end of the quarter); and other significant dates.

To execute applications under year 2000 dates, developers must build a simulated environment that
represents dates in the year 2000. Such an environment can be created via one of the following three techniques.

Time Machine -- Utilizing a standalone machine that can duplicate the environment under which the application executes. The machine is initialized with the appropriate year 2000 dates and all applications running on that machine see the same date. Overall, the time machine is the best solution, but it does tend to be fairly expensive due to the cost of the dedicated machine, software,
and resources required for set up.

Logical Time Machine -- This technique calls for the logical partitioning of a machine to set it up with year 2000 dates. This is primarily applicable for large mainframe computers. All applications running under that logical partitioning see the same date.

Date Simulation Products -- Date simulation products intercept system date requests and provide selected applications with dates in a specified range, such as the year 2000 and beyond. For the most part, date simulation products from various vendors provide the same basic capability. The products differ in the user interface, and in the ability to select specific users, jobs, steps, tasks and
transactions. In addition some products cannot handle some exceptional coding practices of retrieving the system date.

Simulation products are shipping today from several suppliers, including Computer Associates (CA-Impact/2000); Compuware (Xpediter/Xchange); Isogon Corp. (TICTOC); MainWare Inc. (HourGlass); Prince Software Inc. (Simulate 2000); Seec Inc., Journey); and Viasoft (VIA/ValidDate).

Year 2000 data handling

Once the year 2000 Execution Environment is set up and verified, developers should next verify that the application handles data files containing dates for the year 2000. These would include screen input, databases and transaction files. Since a significant date related functionality in most applications manipulates dates in data files, it is necessary that the data files used for the testing
contain year 2000 related dates. Without this, a significant part of the application's functionality may not get verified.

One example of a lack of verification could be a credit card application designed to compute payments due from customers, that is run with the current date set to January 1, 2000, while the dates in the data file are left untouched. Such an application would determine that no payment was
made for a long period of time (March 1998 to January 1, 2000) and will mark every account as past due and thus delinquent. In reality, though, on January 1, 2000, the data files would reflect payments and transactions throughout 1998 and 1999. This process needs to be simulated.

The simulation process is accomplished by a concept called Data Aging, where all the date files are
uniformly aged (for example, adding a certain number of days). Once that process is completed, applications executed under a date of January 1, 2000 will display payments made in prior months and would compute accounts correctly.

To ensure that applications can correctly handle data files that contain year 2000 dates requires modification of the data files. This process can generally be undertaken utilizing a concept called Data Aging, which ages all dates in the data files by a given number of days to simulate dates around year 2000. An example would be an application tested on April 15, 1998, using a simulation date of January 1, 2000.

The difference between these two dates, 625 days, is added to all the dates in the data to simulate the data that would be found around January 1, 2000. In selecting Data Aging products it is necessary to verify that the product handles the different file formats and databases that are currently in use at your site. Most bridging software also contains the built-in capability to age the data that is
being read-in.

Companies selling products designed for data aging include: Compuware (File-AID/Data Ager); Mercury Interactive (Ages Screen Input); Platinum Technology Inc. (TransCentury File Age); Seec (Ages Screen Input); and Serena Software International (StarWarp).

Sufficiency of test data or test coverage analysis

With the next step, it is essential developers ensure sufficient test data is used to exercise the different pieces of applications. For the purpose of year 2000 compliance verification, at a minimum, all statements that were modified must be executed.

Test Coverage Analysis is used to identify the ratio of executed lines of code to total lines of code, expressed as a percentage. For the purpose of year 2000 Compliance Validation, Test Coverage will be the number of modified lines that are executed divided by the total number of modified lines.

If the Test Coverage of the modified statements is close to 100%, there is sufficient test data and testing is satisfactory. On the other hand, if the Test Coverage is significantly below 100%, then it is essential that additional Test Data is created that can specifically execute the modified statements that were not exercised by the existing Test Data.

To ensure there is sufficient test data, test coverage analysis must be performed. If 100% of the
modified statements are executed by the test data, then there is sufficient coverage. Otherwise, additional test data must be generated to ensure 100% coverage of all modified statements.

Test coverage analysis capabilities are incorporated in the products of several vendors, including: Computer Associates (CA-TestCoverage/2000); Seec Inc. (AccuFix): and Viasoft (VIA/SmartTest and TCA).

Test data can be generated in one of two ways. The first approach is to take the record format of the input records and randomly generate data values corresponding to the format of the different fields but ensuring that the data fits the pre-specified data ranges. Although this approach creates additional records, it does not necessarily ensure the statements that need to be executed will be executed by using this additional data. Test data generation tools are essential for creating additional test data.

Companies selling this type of test data generation products include: Computer Associates
(CA-Datamacs/II); and Mercury Interactive (TestBytes).

The second approach to test data generation is to identify the specific input data necessary to execute specific statement(s). This method understands the program logic and identifies the conditions under which the specific statements will be executed. Examples of companies selling targeted test data generation products include: Seec Inc. (AccuTest).

For year 2000 compliance validation, Correct Performance is defined as the set of business expectations from the applications, or what the production code produces. Hence, the modified applications would have Correct Performance if their results (resulting screens, reports, data files) were identical to what is produced by the current production version. This definition, if followed
rigorously, will eliminate the tendency to fix pre-existing problems in production code that are unrelated to the year 2000 problem.

Compliance testing goal

Since significant time, effort and resources are required to perform compliance testing, applications must be prioritized and the level of compliance verification identified. Applications can require Extensive, Normal or Minimal Level testing based on the business value, the impact on the bottom line, the magnitude of change required, and the extent of automation in the remediation process.

The Extensive testing level calls for all testing processes -- regression testing, year 2000 date simulation, data aging to simulate year 2000 data, test coverage analysis -- for verification of 100% coverage of all modified statements and generation of additional test data, to reach 100% coverage.

Normal testing includes regression testing, year 2000 date simulation and data aging. The test coverage analysis and generation of additional test case processes may be minimized.

The Minimal testing level calls for the regression testing, year 2000 date simulation and data aging activities. The other jobs can be skipped.

This article described the requirements for undertaking a year 2000 compliance validation project and offered examples of commercially available products that can be used in the process. Although the paper concentrated specifically on the testing process, it is essential that organizations have a complete methodology for managing the source, identifying units of work, tracking the changes, the use of automated tools for remediation, year 2000 compliance validation, and return of code to production. Depending on the level of sophistication of a company's enhancement and testing process, it may require minimal or extensive changes in the way year 2000 compliance is implemented.

In conclusion, the estimated cost for testing and validation is a significant factor in the overall cost of year 2000 compliance. In addition, most organizations have never made such extensive changes to almost every application in a shop and do not have the experience of testing and verifying every application. Hence, it is essential that the year 2000 compliance validation be approached in a systematic way, such as is described here.