Manual of Diagnostic Tests for Aquatic Animals (2003)

CHAPTER 1.1.3.

VALIDATION AND QUALITY CONTROL OF POLYMERASE CHAIN REACTION METHODS USED FOR THE DIAGNOSIS OF INFECTIOUS DISEASES

---

SUMMARY

The diagnosis of infectious diseases is performed by direct and/or indirect detection of infectious agents. By direct methods, the particles of the agents and/or their components, such as nucleic acids, structural or nonstructural proteins, enzymes, etc., are detected. The indirect methods demonstrate the antibodies induced by the infections.
 
The most common direct detection methods are virus isolation, bacteria cultivation (the gold standards), electron microscopy, immunofluorescence, immunohistochemistry, antigen enzyme-linked immunosorbent assay (ELISA), nucleic-acid hybridisation (NAH) and nucleic acid amplification, such as the polymerase chain reaction (PCR). As NAH and PCR assays have nucleic acid molecules as targets, they are also termed methods of molecular diagnosis.
 
The most common indirect methods of infectious agent detection are virus neutralisation, antibody detection by ELISA and haemagglutination inhibition tests. In general, diagnostic laboratories simultaneously apply both the direct and the indirect methods, in order to assure the certainty of a diagnosis.
 
To date, OIE principles of validation have been developed for indirect detection methods, i.e. for antibody ELISA (see Chapter 1.1.2.). The purpose of this chapter is to extend the rules to a direct method of infectious agent detection, i.e. to adapt the principles of validation to the PCR assays.
 
The experiences of the last decade indicate that the PCR techniques will eventually supersede many of the classical direct methods of infectious agent detection. It is clear that in many laboratories, the PCR is replacing virus isolation or bacteria cultivation for the detection of agents that are difficult or impossible to culture. There are several reasons for this trend, including that virus isolation requires: i) the presence of replicating viruses; ii) expensive cell culture and maintenance facilities; iii) as long as several weeks to complete the diagnosis; and iv) special expertise, which is missing or diminishing today in many laboratories. Although PCR assays were initially expensive and cumbersome to use, they have now become relatively inexpensive, safe and user-friendly tools in diagnostic laboratories (2, 7).
 

A. PCR METHODS USED IN ROUTINE MOLECULAR DIAGNOSTICS

1.	The principles of the PCR
	Polymerase chain reaction (PCR) implies that there is an amplification reaction in the assay. The term 'chain reaction' refers to several cycles of copying a specified stretch of DNA from a target nucleic acid, in this case from the genome of an infectious agent. The amplified region is defined by two (or more) short oligonucleotides, and two primers that are complementary to DNA regions flanking the target sequence. Using a heat-stable DNA polymerase, which is not denatured during heat cycling, it is possible to copy the DNA sequence between the primers. By repeating 20-40 times a heat-cycling regime, the amount of copied target DNA gained is enough for further operations, such as detection, cloning or sequencing. The diagnostic sensitivity of the PCR is very high because several million copies of the selected target are produced. The specificity of the reaction may also be very high, as determined by the specific nucleotide sequences of the oligonucleotides (primers). The primers are designed to detect specific nucleotide sequences in the genomes of the selected target infectious agents.
	a)	DNA amplification
		If the genome of the infectious agent is DNA, the amplification is performed directly, with or without previous purification of the target DNA.
	b)	RNA amplification (reverse-transcription PCR)
		The genomes of many infectious agents contain ribonucleic acid (RNA) that cannot be amplified directly by the PCR. For PCR amplification, a double-stranded DNA target is necessary, and this is not available in the case of RNA viruses. This problem can be solved by the addition of a step before the PCR is begun. Using reverse transcriptase it is possible to transcribe the RNA into complimentary DNA (cDNA), which is double-stranded DNA and hence can be used in a PCR assay (the procedure is termed reverse transcriptase PCR: RT-PCR). Traditionally, the reverse transcription reaction is performed in a separate reaction vessel and the cDNA produced is then transferred to a new tube for the PCR. However, heat-stable DNA polymerases with reverse transcriptase activity are now readily available. When used in specific buffers, these enzymes enable the RT-PCR reaction to take place in the same tube and in direct sequence without any further handling.
		Some examples of PCR methods currently used are given below.
2.	Single PCR
	'Single PCR' (or simply PCR) uses one pair of oligonucleotide primers to amplify a small part of the genome of the infectious agent. Analytical sensitivity is relatively high with a minimum number of 100 to 1000 copies of the genome detectable. Analytical specificity is also rather high. Both analytical sensitivity and specificity can be further improved by applying nested PCR (see point 3 below). Using Southern blotting and checking the PCR products with a labelled probe can further improve specificity, but this is time-consuming and is not a common practice in diagnostic laboratories today.
3.	Nested PCR
	Nested PCR assays use two amplification cycles with four primers, termed external and internal primers. In general, nested PCR assays provide higher analytical sensitivity and specificity compared with single PCR. Analytical sensitivity is typically <10 genomic copies of the infectious agent, and analytical specificity is also enhanced because in the nested PCR, four oligonucleotides have to bind specifically to the selected targets in order to yield a positive reaction (2).
4.	Real-time PCR
	Real-time PCR is an amplification where the PCR products are detected directly during the amplification cycles using fluorescence-labelled probes. Various real-time methods, such as TaqMan or Molecular Beacon assays, have become popular tools for detection of infectious agents. Real-time PCR has been used for the detection of bacteria, viruses or parasites from a range of animal species (2, 6, 8). These new assays have several advantages over the 'classical' single or nested PCR methods. Only one primer pair is used, providing sensitivity often close or equal to traditional nested PCR but with a much lower risk of contamination. Fluorescence, indicating the presence of the amplified product, is measured through the lid or side of the reaction vessel thus there is no need for post-PCR handling of the products. These procedures are considerably less time-consuming compared with traditional PCR product detection in agarose gels followed by ethidium bromide staining and again, the risk of contamination is reduced. The use of a 96-well microtitre plate format, without the need for nested PCR, allows the procedure to be automated (5). Diagnosis can be further automated by using robots for DNA/RNA extractions and pipetting. Compared with classical amplification methods, a further advantage of the real-time PCR is that it is possible to perform quantitative assays (6).
5.	Multiplex PCR
	Probes for real-time PCR can be labelled with a large number of different fluorophores, which function as reporter dyes. The use of fluorescent probes emitting different colours enables 'multiplexing' of the assays. In multiplex PCR, various infectious agents can be detected and differentiated in a single reaction vessel at the same time. The 'classical' PCR technique was also found to be suitable for the development of multiplex systems. However, the use of 'classical' nested PCR methods for the construction of a multiplex assay is complicated because of the large number of primers that may 'compete' with each other in the same reaction mix. In contrast, the concept of real-time PCR (single primer pairs) provides excellent possibilities for the construction of highly sensitive multiplex systems (2, 4).

B. VALIDATION OF MOLECULAR DIAGNOSTIC ASSAYS

When performing diagnostic analyses of clinical material it is important to produce data of good quality. For this, some key criteria have to be fulfilled. The establishment of quality assurance (QA) and quality control (QC) systems is required, i.e. a set of quality protocols, including the use of control samples, that ensure that the system is working properly and confirms data quality. QA and QC systems, together with trained and competent personnel, have already been established in many laboratories world-wide. Assay validation is another essential factor for assuring that test results reflect the true status of the samples (3).
 
To predict the performance of a diagnostic assay, it is necessary to use a validation methodology to validate the assay in question. Validation is the evaluation of a diagnostic assay for the purpose of determining how fit the assay is for a particular use. The general principles of assay validation can be found in Chapter 1.1.2. Principles of Validation of Diagnostic Assays for Infectious Disease. This chapter extends these validation principles to molecular diagnostic assays. For explanations of terms and definitions please consult Chapter 1.1.2.
 

C. MEASURES OF VALIDITY

Performance characteristics (or assay parameters) give information about how a method functions under specified conditions. Some typical performance characteristics are given in Chapter 1.1.2. and some others, important to PCR methods, are given here.
 

D. STAGES OF ASSAY VALIDATION

In Chapter 1.1.2., the five stages of assay validation are described in detail. In this chapter, these stages are presented briefly with special emphasis on molecular diagnostic assays.
 

STAGE 1. FEASIBILITY STUDIES

A feasibility study is a preliminary step in validating a new assay. The goal is to determine whether or not a new assay can detect a range of target concentrations without background activity. At least ten samples (for example, infectious agents produced in the laboratory in cell or bacterial culture) are chosen, ranging from low to high levels of the infectious agent. It is also necessary to include at least ten samples containing no target. Usually it is difficult to separate this stage from stage 2, as preliminary optimisation is necessary before further studies can be conducted. Assays that look promising are subjected to further development in stage 2. Note that it is sometimes possible to substantially improve an assay by proper optimisation schemes and thus exclusion of non-optimal assays should be done with caution.
 
Primer selection is critical, and account should be taken of the nature of the infectious agent, its genome structure and the diversity of genetic sequences among different strains.
 
The result obtained by PCR may be influenced by the performance of the thermocycler, which should therefore be monitored on a continual basis. Regular temperature calibration is crucial and, for real-time PCR instruments, the optical systems must be controlled regularly. Assays developed and validated using a specific brand of thermocycler should be revalidated or otherwise controlled when new equipment is used.
 

STAGE 2. ASSAY DEVELOPMENT AND STANDARDISATION

1.	Selection of Optimal Reagent Concentrations, Protocol Parameters and Equipment
	Sample collection, preparation and transport (see Chapter I.1.) and nucleic acid extraction methods (see Chapter I.1.) are all critical parameters in test performance and should be optimised for disease diagnosis. Suitable methods vary depending on sample and organism type. In general, blood and serum are suitable samples for easy extraction of target nucleic acids, while faeces and semen samples are more difficult to handle. Extraction of RNA targets differs from extraction of DNA targets, and RNA is more prone to degradation. Both commercial (robotic, spin columns, etc.) and in-house methods are used for DNA or RNA extraction. It is crucial to determine the most suitable method before further validation of the assay is performed. If the method of extraction is changed, the entire validation procedure should be repeated.
	All equipment used during the process must be properly maintained. Apparatus (heating blocks, refrigerators, freezers, thermocyclers, pipettes, etc.) that require calibration must be calibrated according to the laboratory's quality protocols.
	When developing 'classical' or real-time PCR assays, all parameters, protocols and reagents need to be optimised. A standardised assay is a method that consistently gives the same result for a given sample when repeated several times.
2.	Repeatability - Preliminary Estimates
	Agreement between replicates within and between runs of the assay should be accessed at this stage. This gives important information about the assay before further validation is carried out. If excessive variability is encountered, it should be corrected before continuing the validation process.
3.	Determination of Critical Control Parameters
	During the optimisation of the PCR assay, it is also possible to estimate the capacity of the method to remain unaffected by small changes in the main parameters. Introduction of intentional variations in the validation process will characterise critical parameters in the assay. Examples of such parameters include: incubation times and temperatures, concentrations of buffers, primers, MgCl2, etc., pH, amounts of other components added (e.g. dNTP, bovine serum albumin, etc.). The characterisation of critical control parameters is crucial for identifying critical points that must be properly controlled in the assay.
4.	Analytical Sensitivity and Specificity
	Analytical sensitivity (or limit of detection) is defined as the smallest number of genome copies of the infectious agent that can be detected and distinguished from a zero result. To determine analytical sensitivity, an end-point dilution is used until the assay can no longer detect the target in question in more than 5% of the replicates (2 standard deviations). Cloned fragments of the PCR products in question can be used as standard samples, either as DNA or for RNA targets, the RNA being transcribed in vitro into DNA.
	Analytical specificity is defined as the ability of an assay to distinguish the target agent from other infectious agents. This ability is determined by analysing closely related pathogens using the assay in question.
5.	Range
	Analytical techniques can rarely be scaled up or down arbitrarily; the assay should be optimised in the linear phase of the dose-response curve. The range of an assay is defined as the interval between the upper and lower concentration of an infectious agent in a sample in which the agent can be reliably detected.

STAGE 3. DETERMINING ASSAY PERFORMANCE CHARACTERISTICS

1.	Diagnostic Sensitivity and Specificity
	Diagnostic sensitivity (D-SN; proportion of known infected reference animals that are tested positive in the assay) and specificity (D-SP; proportion of known uninfected reference animals that are tested negative in the assay) are the most important parameters obtained during the validation of an assay. They form the basis for calculating other parameters and hence they are critical to the whole validation process. The number of reference samples required to determine estimates and allowable error of both D-SN and D-SP can be calculated. To do this, a reasonable prediction of both D-SN and D-SP must be used. Generally, confidence in the estimate is set at 95%. However, no formula can account for the numerous host/organism factors that can affect the outcome of the test. The number of samples to determine estimates of D-SN and D-SP is outlined in Chapter 1.1.2. It is recognised that achieving these numbers for molecular assays might be difficult and costly. Testing smaller numbers will result in a reduction in confidence of the estimate. The status of known infected and uninfected animals should be established using comparisons with other assays. The use of spiked samples in PCR is not appropriate as these might not be representative of naturally infected samples and thus the whole validation process could potentially be jeopardised.
2.	Repeatability and Reproducibility
	Repeatability and reproducibility are both important parameters in assay precision. Repeatability is measured as both the amount of agreement between replicates within the same run or between replicates tested in different runs. Reproducibility is determined in several laboratories using the identical assay (protocol, reagents and controls).
	Currently, OIE stage 3 is rarely performed to its full extent in veterinary diagnostic laboratories carrying out PCR assays. Traditionally, many laboratories have used tests developed in-house, probably for practical reasons. Where there are published standardised and validated methods, these should be followed. Inter-laboratory validation processes have to be carried out even if they are costly and labour intensive. This work will lead to standardised assays, allowing harmonised diagnostic activity in various countries.

STAGE 4. MONITORING VALIDITY OF ASSAY PERFORMANCE

The estimation of the prevalence of a virus in the population is necessary for calculating the predictive value of positive (PV+) or negative (PV-) test results. This applies equally to molecular test methods as it does to other methods such as the enzyme-linked immunosorbent assay.
 
Reference Laboratories are encouraged to determine values for D-SN and D-SP as accurately as possible, as these are extremely important for judging the real performance of an assay when used in the field. It is also important to estimate the predictive values (PV+ or PV-) in the local situation.
 

STAGE 5. MAINTENANCE AND ENHANCEMENT OF VALIDATION CRITERIA

When the assay is used as a routine test it is important to maintain the internal QC. The assay needs to be consistently monitored for repeatability and accuracy. Reproducibility between laboratories (ring tests) is recommended by the OIE to be estimated at least twice a year (9).
 
If the assay is to be applied in another geographical region and/or population, it might be necessary to revalidate it under the new conditions. This is especially true for PCR assays as it is very common for point mutations to occur in many infectious agents (i.e. RNA viruses). Mutations, which may occur within the primer sites, can affect the performance of the assay and by doing so the established validation is no longer valid. It is also advisable to regularly sequence the selected genomic regions in the national isolates of the infectious agents. This is especially true for the primer sites, to ensure that they remain stable so that the validation of the assay cannot be questioned.
 

1.	Precautions and Controls
	Considering the uncertainty about the safety and reliability of the PCR in routine diagnosis, special precautions should be applied in any laboratory using PCR for detecting infectious agents, in order to avoid false-positive or false-negative results. These, together with internal controls (mimics) assure the safe evaluation of the results.
	a)	Precautions to avoid false-positive results
		False-positive results (negative samples showing a positive reaction), may arise from either product carryover from positive samples or, more commonly, from cross-contamination by PCR products from earlier experiments. Various practices and tools have been applied to prevent false-positive results. Samples and mixes should be handled in laminar air-flow hoods, which are regularly decontaminated using UV light and bleach. Constructing and using special tube-holders and openers help prevent false-positive PCR. In addition, safe laboratory practices should be applied, i.e. to perform the basic steps of nested PCR (mix and primer preparation, sample preparation, etc.) in separated laboratory areas (1, 2). It is also very important to include negative controls, i.e. samples that are as similar to the test samples as possible but without having the target. At least one negative control per five diagnostic samples should be used.
	b)	Internal controls (mimics) to avoid false-negative results
		False-negative results (infected samples tested as negative) are mostly due to inhibitory effects and/or pipetting errors. Therefore, internal controls (termed 'mimics') are used as indicators of amplification efficiency. The mimics have the same primer-binding sequences as the template of the agent, but flank a heterologous DNA fragment of a different size. The identical primer-binding nucleotide sequences allow co-amplification of template and mimic in the same tube with minimal competition. The size differences provide easy discrimination. The internal controls increase the reliability of the diagnostic PCR (1, 2). Caution must be used when designing and validating mimics. Extensive testing is necessary to ensure that the added mimic and its amplification is not competing with the diagnostic PCR and thus lowering the analytical sensitivity. The mimic is used in a concentration slightly higher than the detection limit of the diagnostic PCR to ensure the test's performance.
		In real-time PCR assays, it is also possible to apply internal controls by detecting a selected fragment of the host animal's genome. By including such an intrinsic control with a specifically coloured reporter fluorophore, it is possible to check the sample quality and to control pipetting errors simultaneously as the target agent is detected (8).
2.	Preparation of Standards
	Reference laboratories should provide standard samples representative of a given infectious agent. Such samples can be cultivated infectious agents, clinical specimens, etc., which are distributed in such a manner that the infectious agent is well preserved. Thus, the samples are distributed frozen, in organic solvents (e.g. Trizol) or other suitable ways. The samples can also be sent as nucleic acids (frozen, freeze-dried or in ethanol). For specific details, see the individual disease chapters.

REFERENCES

1.	Ballagi-Pordany A. & Belak S. (1996). The use of mimics as internal standards to avoid false negatives in diagnostic PCR. Mol. Cell. Probes, 10, 159-164.
2.	Belak S. & Thoren P. (2001). Molecular diagnosis of animal diseases. Expert Rev. Mol. Diagn., 1, 434-444.
3.	Burkhardt H.J. (2000). Standardization and quality control of PCR analyses. Clin. Chem. Lab. Med., 38, 87-91.
4.	Elnifro E.M., Ashshi A.M., Cooper R.J., & Klapper P.E. (2000). Multiplex PCR: optimization and application in diagnostic virology. Clin. Microbiol. Rev., 13, 559-570.
5.	Jungkind D. (2001). Automation of laboratory testing for infectious diseases using the polymerase chain reaction - our past, our present, our future. J. Clin. Virol., 20, 1-6.
6.	Heid C.A., Stevens J., Livak K.J. & Williams P.M. (1996). Real-time quantitative PCR. Genome Res., 6, 986-994.
7.	Louie M., Louie L. & Simor A.E. (2000). The role of DNA amplification technology in the diagnosis of infectious diseases. CMAJ., 163, 301-309.
8.	Mackay I.M., Aarde K.E. & Nitsche A. (2002). Real-time PCR in virology. Nucleic Acids Res., 30, 1292-1305.
9.	Office International Des Epizooties (2002). OIE Guide 3: Laboratory Proficiency Testing. In: OIE Quality Standard and Guidelines for Veterinary Laboratories: Infectious Diseases. OIE, Paris, France, 53-63.

[retour au début]

Copyright Office international des épizooties (OIE) - scientific.dept@oie.int