Implementation of a Multi-Site Enterprise System in a Regulated Environment using Cloud Services

From a regulated perspective, changes made by the CSP affecting the application and database layers, could indirectly impact on data integrity. So the client and the CSP established communication channels to mitigate any impact. Additionally, the client created multiple environments for the application (Development, Sandbox, Training, Validation, Production) such that changes could be tested under controlled circumstances before being rolled out. For example, the client was able to control the time window when operating system updates were applied, such that they could be fully vetted prior to being applied to the production environment.

Implementation

While the program was to be rolled out to dozens of sites, the client initially assembled a team with representatives from five sites that defined data formats, data movements, and changes to standardized workflows necessary to accommodate a regulated environment. Some customization of the CMMS was inevitable due to the diverse nature of their regulated manufacturing facilities. Most of this customization was related to workflow configuration and application security profiles. Good project management was crucial for success as there were a large number of decisions to be made.

Of the initial sites, the one with the most efficient maintenance operations was selected as the pilot site. We extracted data from the existing CMMS at the pilot site and reformatted it for the new CMMS—revealing a host of issues—the old data was not hierarchically organized, the new data had to be. The old data did not account for portable equipment and location changes, the new data needed different location codes. The old data did not include job plans, safety plans, work instructions, or spare parts and for the new system, much of this data had to be created. Additionally, once the old data could be examined holistically, data discrepancies were everywhere— usually in the form of similar equipment with dissimilar PMs, job plans, calibration data points, and so forth. All these problems needed correction.

Data Migration and Validation

We created a Data Migration Plan that detailed the process steps, data to be migrated, created, and how it would be verified. Much of the maintenance history existed already as paper printouts, and as the old CMMS would not be retired for many years, electronic historical maintenance records did not need to be transferred to the new system.

Planning was crucial, as the entire data creation and migration process had to occur in a very short time window. This is because the data in the old system had to be frozen— no new assets, no new spare parts, etc. Realistically, it is impossible to continue to maintain a facility while the data is frozen so we created a separate process whereby we could track all new data changes, to be added after the new CMMS go-live.

For this relatively small facility, there were over 50,000 records to be created and/or migrated. We used large spreadsheets that aligned with the table structures of the new CMMS to organize the exported data. Changes and additions were highlighted. Corrections were made programmatically via filters or formulae to maintain consistency. Dry-runs of data imports helped catch parent-child errors (where the child data is loaded before the parent data in the same datasheet) and other errors. Once the data was correct and ready for loading into the new system, we printed and reviewed the spreadsheets with the Quality Assurance group.

After approval by QA and the data owners, the spreadsheets became the document of record. We then loaded the data into the new CMMS and generated data extractions from the database. We electronically compared the spreadsheets to the data extractions. Of course, there were still errors—but ultimately, we resolved all the errors and loaded all data successfully.

Bringing New Sites Online

Validation used an E2500 risk-based approach that focused testing on the elements that had been configured or changed. Out-of-the-box elements were indirectly tested as part of workflow functionality. Approximately 120 use cases were directly challenged, as opposed to about 600 use cases for the entire system. Most testing occurred in FAT or SAT and then was leveraged in the OQ. The PQ challenged high-level workflow processes as defined in the operating procedures.

As new sites prepared to come online, they too had to go through the same structured data migration process, but the validation was much simpler and limited to a small IQ (for environment creation and networking) and the PQ. Functional testing was not repeated.

Additionally, new sites were brought online one environment at a time, so it was possible to work through the go-live checklists without being in the production environment. This allowed us to perform other CSP benchmark tests, such as latency checking, security overrides, and so forth.