21 CFR Part 11/Audit guidelines and checklist

The following guidelines and checklist items provide a frame of reference for vendors and auditors to better determine potential compliance issues with Title 21 Code of Federal Regulations Part 11 and a variety of other regulatory guidelines.

All items in the checklist for general IT controls should also be checked for individual systems, especially where those systems use different control measures (e.g., they have an independent authentication system).

If this checklist is used by software vendors, then certain elements may or may not apply depending on the circumstances. For instance, validation is technically the responsibility of the entity acquiring the software. However, in the case of SaaS, a greater practical responsibility to validate the system may lie with the vendor. In all cases, the vendor should assume responsibility for ensuring that their software operates as intended within the targeted environments. Failure to do so may result in a lack of willingness of potential customers to obtain the system.

References will be provided for each checklist item to indicate where the requirement comes from. These references are either to the regulation itself, Agency responses in the Final Rule, or from the guidance document "General Principles of Software Validation; Final Guidance for Industry and FDA Staff" (GPSV).

General IT

Following is a list of questions that either apply to the larger IT environment, or to both the larger environment and to individual systems. The auditor must be sure to evaluate both where necessary. For instance, an organization may have a robust password policy which is managed by a centralized identity management tool. This is important evaluate in terms of general security around the systems in scope. At the same time, the specific system may or may not leverage the corporate IDM and thus it’s identity management should be evaluated on its own merits.

Computer Systems Validation - 21 CFR 11.10(a)

• Does a defined computer system validation policy exist? - 21 CFR 11.10(a)
• Are all computer systems involved in activities covered by predicate regulations validated? - 21 CFR 11.10(a), 21 CFR 211.68(b), 21 CFR 820.30(g)
• Does the computer system validation cover the current deployed version of the system? - GPSV 4.7
• Validation Assessment
  • Does the software developer have a defined systems development life-cycle (SDLC)? - GPSV 4.4
  • Does the SDLC reflect a generally recognized life cycle approach?
  • Is the SDLC followed? - GPSV 4.4
  • Is the software well documented from a design/development/implementation perspective? - GPSV 3.3
  • Is there evidence of design review activities (what this entails will depend on the nature of the SDLC - for instance, Agile methodologies will involve daily standup meetings,while a waterfall approach may reflect formal design review steps)? - GPSV 3.5
  • Does the level of validation coverage reflect the risk from system failure? - GPSV 6.1
  • Is there sufficient level of independence in the validation/verification activities? - GPSV 4.9
  • Are sufficient resources and personnel provided for software development and validation? - 21 CFR 211.25(c), 21 CFR 820.25(a)
  • Are records maintained of defects and failures identified in the development process? - GPSV 5.2.6
  • For any software system, is there a set of approved requirements which drove the design (note: the name can vary based on the SDLC in use). - GPSV 6.1
  • For iterative development approaches, are previous versions of deliverables (such as requirements lists) archived in some fashion? - GPSV 5.2.1
  • Is there an audit trail for modifications to system documentation? - 21 CFR 11.10(k)(2)
  • For commercial off-the-shelf (COTS), has the vendor been evaluated for its quality systems? - GPSV 6.3
  • Is there some form of traceability that permits tracking of test results and verification activities to specific requirements? - GPSV 5.2.2
  • Are adequate change control systems in place during the development and implementation processes? GPSV 3.3
  • For each of the other elements of this checklist that apply directly to an electronic record system, has appropriate validation work been undertaken to establish that the system complies with the checklist item?

Identity Management Systems

• Do any identity management systems have minimum password complexity/strength requirements? Do these minimums seem reasonable? - 21 CFR Part 11 Final Rule Section 130
• Do these id systems have policies regarding password change frequency? - 21 CFR 11.300(b)
• Do identity management systems prevent the creation of duplicate user ID’s?

Access Controls

• Do formal procedures exist governing user account creation for electronic records systems.
• Do formal procedures exist governing access to network and server resources that are used to operate electronic records systems?

Cloud Computing Policies

• Are policies in place governing the selection and use of cloud vendors for electronic record systems?
• Do these policies include Service Level Agreements(SLA's) regarding such things as up time, and support responsiveness?
• Are cloud vendors evaluated for security and compliance with appropriate regulations?
• Do policies governing record retention specifically apply to cloud vendors?
• Are systems for transmitting electronic records configured to do so in a secure manner? 21 CFR 11.30

Training and Personnel

• Is an organizational chart available covering personnel involved in the design, development, administration or use of electronic records systems?
• Are job descriptions available for these individuals, indicating their specific responsibilities regarding electronic record systems?
• Is there a defined training program around authentication practices? Electronic signatures?21 CFR 11.10(i)
• Are system administrators and developers trained in Part 11 and related regulations? 21 CFR 11.10(i)
• Are users trained on the use of electronic records systems? 21 CFR 11.10(i)

Change Control Systems

• Is there a formal change control system for modifications to the production electronic records system?
• Is there a formal change control system for changes to requirements and design elements of the system during the development process?
• Does the change control system require an assessment of impact, risk, and require authorization before proceeding?
• Do change control systems in use for Agile development models require product owner and development team approvals?

Electronic Signature Certification

• If the organization is using electronic signatures, have they filed a certification with the FDA indicating so?

Records Retention Policy

• Does the organization have a records retention policy covering records per the predicate regulations?

System Specific

Fraud Detection

• Is the system designed to either prevent record alteration or make such alteration apparent?

Audit Trails

• Does the system maintain an audit trail that tracks changes to electronic records?
• Are the audit trail records time stamped?
• Are the audit trail records system generated, such that human intervention is not required?
• Are audit trail records secured such that they cannot be modified by users of the system?
• Is the audit trail data available for export (printing or electronic) to support agency review?

Access Controls

• Does the identity management systems have minimum password complexity/strength requirements? Do these minimums seem reasonable?
• Do these id systems have policies regarding password change frequency?
• Do identity management systems prevent the creation of duplicate user ID’s?

Open Systems Controls

• Are records transmitted by the system sent in a secure manner, such that their authenticity, integrity and confidentiality are ensured?
• Is access to the system appropriately managed to prevent unauthorized external access?
• Has the system been evaluated for susceptibility to intrusion?
• Is there a system in place to evaluate current IT security threats that have been identified (by the National Cyber Awareness System via NIST, or other appropriate organization)?

Electronic Signatures

• Is the electronic signature system engineered in such a way as to ensure that the signatures cannot be attached to other records, or cannot be removed from the records they are attached to?
• Is the system engineered such that in order to apply someone else’s signature to a file that collaboration is required between two or more individuals? (this is largely covered by the identity management controls).
• If a signature event only requires one signature element, is it only in the case of being part of a continuous period of system access?
• Are their suitable loss management procedures in place to address compromised passwords, or lost/stolen authentication devices (such as RSA ID tokens)?
• Is the system designed to alert security and/or management in the event of an apparent attempt at unauthorized use of electronic signatures? Does the system automatically take steps to lock out users associated with these attempts?
• Is there a system for the periodic testing of tokens and cards to ensure that they are still operating as expected and have not been altered? If not, is there something in the nature of the tokens/cards that would render them unusable should alteration be attempted?
• Is there a password reset method that does not require system administrators to know a user’s password?
• Are user passwords suitably encrypted in any persistent data store, such that elucidating the original password would require extraordinary means?
• Are controls in place to ensure that password reset instructions are sent to the correct individual?

Export of Records for Agency Review

• Does the system support exporting records in a format that is readable by the agency?
• If the agency hasn’t been specifically consulted with regard to acceptable formats, does the system support export into common formats such as XML or JSON?

Records Retention Support

• Does the system have sufficient controls to ensure that the records stored within it will be available throughout the period specified in the records retention policy?

Process Controls

• Does the system have a mechanism to establish differing levels of authority to perform tasks in the system?
• Does the system have a mechanism for preventing steps being taken out of sequence (e.g., signing a record before data has been entered, or releasing a record before the review step was completed)?

Reference material

21 CFR Part 11

Subpart A — General Provisions

§ 11.1 Scope

§ 11.2 Implementation

§ 11.3 Definitions

Subpart B — Electronic Records

§ 11.10 Controls for closed systems

§ 11.30 Controls for open systems

§ 11.50 Signature manifestations

§ 11.70 Signature/record linking

Subpart C — Electronic Signatures

§ 11.100 General requirements

§ 11.200 Electronic signature components and controls

§ 11.300 Controls for identification codes/passwords

General Principles of Software Validation

The full name of this FDA guidance document is "General Principles of Software Validation; Final Guidance for Industry and FDA Staff," referenced on here as "GPSV."

Section 1. Purpose

Section 2. Scope

2.1. Applicability

2.2. Audience

2.3. The Least Burdensome Approach

2.4. Regulatory Requirements for Software Validation

2.4. Quality System Regulation vs Pre-Market Submissions

Section 3. Context for Software Validation

3.1. Definitions and Terminology

3.1.1 Requirements and Specifications

3.1.2 Verification and Validation

3.1.3 IQ/OQ/PQ

3.2. Software Development as Part of System Design

3.3. Software is Different from Hardware

3.4. Benefits of Software Validation

3.5 Design Review

Section 4. Principles of Software Validation

4.1. Requirements

4.2. Defect Prevention

4.3. Time and Effort

4.4. Software Life Cycle

4.5. Plans

4.6. Procedures

4.7. Software Validation After a Change

4.8. Validation Coverage

4.9. Independence of Review

4.10. Flexibility and Responsibility

Section 5. Activities and Tasks

5.1. Software Life Cycle Activities

5.2. Typical Tasks Supporting Validation

5.2.1. Quality Planning

5.2.2. Requirements

5.2.3. Design

5.2.4. Construction or Coding

5.2.5. Testing by the Software Developer

5.2.6. User Site Testing

5.2.7. Maintenance and Software Changes

Section 6. Validation of Automated Process Equipment and Quality System Software

6.1. How Much Validation Evidence Is Needed?

6.2. Defined User Requirements

6.3. Validation of Off-the-Shelf Software and Automated Equipment

Others

• 21 CFR Part 211: Current Good Manufacturing Practice for Finished Pharmaceuticals

• 21 CFR Part 820: Quality System Regulation