How Risk-Based TMF Quality Checks and Quality Review Improve Inspection-Readiness

For the kickoff of Phlexglobal’s “TMF Summer Shorts” program July 18, 2023, we purposely chose a hot topic: how to manage risk and ensure quality in a Trial Master File using quality checks (QC) and quality review (QR). 

Designed to be a series of educational briefs (sorry ;) on key TMF challenges, the first of our Summer Shorts was attended by more than 140 TMF professionals from around the globe, representing study sponsors, Contract Research Organizations, and Academic Research Organizations. 

During the webinar, we asked our audience two questions. The responses were revealing as to why TMF quality remains an elusive goal in many companies.  

A couple of key takeaways? Only around one-third of respondents have a methodical approach to assessing TMF quality, and less than 20% say their current quality processes are “very effective.” 

How a Proactive, Risk-Based Approach Improves Quality While Saving Time and Effort 

So how can organizations improve their TMF quality processes and reduce risk?  

In our experience supporting hundreds of organizations across every aspect of Trial Master File management – in eTMF systems from Phlexglobal’s own PhlexTMF to Veeva Vault and Trial Interactive – one common factor emerges: the best-performing companies proactively implement and continually refine their quality processes. They identify and avoid problems before they escalate, and perform better on inspections while freeing busy team members to focus on other critical tasks. 

Taking a proactive approach can be hard, especially with today’s resource constraints and demands on study teams. Yet companies that build quality into the TMF lifecycle, whether alone or with the help of expert outside resources, consistently have high-quality Trial Master Files. 

To make the most effective use of resources, companies are increasingly taking a risk-based approach to TMF quality processes. This risk-based approach has long been supported by the ICH and all leading regulatory authorities. 

Where to start? First, understand the three primary risks of TMF documentation: 

1. The document does not meet Good Clinical Practice (GCP) and/or internal requirements (poor quality document) 
2. The document is not filed correctly (poor indexing) 
3. The document is not filed in the TMF (missing document) 

Then, assess the level of relative risk according to four critical factors to develop a proactive, risk-based QC approach: 

1. Impact of the artifact on patient safety, patient rights and data integrity 
2. Effort to demonstrate compliance where document quality issues exist 
3. Likelihood of quality issues arising (based on empirical data) 
4. Robustness of document review processes

Quality Checks vs. Quality Review: A Critical Distinction 

It’s important to know the key differences between TMF quality checks and TMF quality review, and when each is appropriate in a risk-based TMF quality management approach: 

TMF Quality Checks: A document-level review for content/metadata alignment, this is ongoing risk management at the document level, rather than looking at the TMF as a whole (see Figure 1). Where to focus checks is determined by looking at relative risk as noted above, as well as historical performance by sites and CROs. 

TMF Quality Review: A comprehensive evaluation providing critical details on the quality, completeness, and timeliness of the Trial Master File. It includes a complete inventory of all content, line-by-line details for all required actions, and a comprehensive gap analysis identifying missing documentation. Think of it as a “deep-dive” inspection into the health of the TMF. 

Quality Checks: Selected Best Practices 

Figure 1. An example of TMF quality check processes at the upload stage supporting GCP. 

An effective quality check process will also check for duplicates after document upload. One proven method is to identify identical “fingerprints” of optical character recognition (OCR) documents submitted to the same assignment. Doing this on all OCR documents saves a duplicate document from going through time-consuming and costly steps, only to ultimately be deleted. It’s a relatively simple step that produces big dividends (see Figure 2). 

Figure 2. Automating the flagging of potentially duplicate documents in the TMF can save hours of unnecessary effort later. 

Note: We recommend contracting for a minimum of 95% document processing accuracy (“right-first-time") if using any external service provider such as a CRO. Doing so minimizes time, cost, and effort remediating quality issues later – as well as avoiding recriminations and finger-pointing when problems occur. 

Most eTMF systems also provide built-in completeness reporting, which is a good way to get an overview of TMF document status. These built-in tools (e.g. Completeness Reporting in PhlexTMF, Expected Document Lists in Veeva Vault) consider the locations of documents and provide a gap analysis of the TMF – for example by leaving placeholders for documents expected but not yet present. 

It’s important to note that these reports are limited to what has been filed into the eTMF, and can’t provide insight into potential errors within the document itself, or conflicts with other documents. For that, we need to turn to a TMF quality review. 

TMF Quality Review: Shining a Bright Light Into Trial Master File Quality 

As noted previously, a TMF quality review is a deep dive by knowledgeable TMF professionals into the Trial Master File, helping ensure that it meets GCP standards and regulatory requirements. An effective quality review utilizes several proven methods to uncover potential quality problems, such as more in-depth completeness checks using TMF heatmaps. 

Note: TMF quality review is an excellent method to implement and demonstrate ongoing oversight of a study to inspectors, and can be conducted in your system or the CRO’s. 

When conducting a TMF quality review, experts also rely on one of the most useful methods: a detailed logic and crosscheck analysis. 

For example, for site personnel we will check the latest delegation log and all copies of the 1572 to get a list of names and roles at the site. We then crosscheck that information with expected documents, from CVs and Financial Disclosures to study-specific training expected for each role, to identify gaps and inaccuracies.  

This approach uncovers hard-to-spot but serious quality errors such as a new name being shown on the delegation log but without a matching CV available in the TMF (see Figure 3). Knowing the Principal Investigator (PI) in particular can help identify misfilings elsewhere as well, such as monitoring reports being filed in the wrong site. 

Figure 3. Logic checks and cross-checks to validate the quality of individual documents and whether documents all relate to each other properly in the Trial Master File. 

 Another good example is looking beyond the number of documents to actual document versions. You can have version one of the document in the TMF, and later on version four – where are versions two and three? A basic completeness report will not pick up on that subtlety. 

 As with TMF quality checks, it’s helpful to take a risk-based approach to TMF quality review, focusing on critical areas to save time and effort. Based on our experience, we know where the most serious quality problems are likely to occur in the TMF that have the most impact. Companies can start with a smaller percentage of key sections and artifacts, and also in areas where they’ve experienced quality issues with specific sites or service providers. Organizations can then increase the scope of the review based on the initial findings. 

As shown in Figure 4, there are different types of TMF quality review to match specific situations: 

Ongoing Quality Review: Performed in pre-determined intervals across the whole study, assessing every new document in the TMF starting from the beginning of a trial through the end. This is the best way to avoid experiencing significant issues with TMF quality. 

Inspection-Readiness Quality Review: Conducted when companies have received the notice of inspection, or are preparing a product for submission. Often with tight deadlines (see also “How to Survive a TMF Inspection When You’re Not Prepared”).  

Periodic Risk-Based Quality Review: A planned check of selected documents, for example every quarter. Helpful to proactively spot quality issues before they escalate. 

End-of-Study Quality Review: Since inspections can occur even years after the study is closed, ensuring that your final TMF can pass inspection is critical – especially since it is far more difficult to recreate the “story” of the trial and demonstrate GCP as more time has passed. 

 Figure 4. Different types of TMF quality review available, depending on an organization’s specific requirements. 

We hope you found this information helpful in your quest towards improving TMF quality and inspection-readiness. Want to be sure about TMF quality? Request an expert TMF quality assessment. 

Like to learn more? Visit https://www.phlexglobal.com/tmf-quality-review