Manual of Diagnostic Tests for Aquatic Animals (2003)

CHAPTER 1.1.4.

REQUIREMENTS FOR SURVEILLANCE
FOR INTERNATIONAL RECOGNITION OF
FREEDOM FROM INFECTION

---

PART 1

A. INTERNATIONAL RECOGNITION OF FREEDOM FROM INFECTION

1.	General Principles
	General principles are provided below for declaring a country, zone or aquaculture establishment free from infection in relation to the time of last occurrence, and in particular for the recognition of historical freedom.
	An essential prerequisite to provide the guarantees required for the recognition of freedom from infection is that the particular Member Country complies with the requirements of Chapter 1.4.3 of the Aquatic Code for the evaluation of the Competent Authorities.
	The general principles are:
	.	in the absence of infection or vaccination, the animal population would be] susceptible to clinical disease, or infection, over a period of time;
	.	the disease agents to which these provisions apply are likely to produce identifiable clinical or pathological signs in susceptible animals;
	.	an animal population may be free from some specified pathogens but not from others
	.	there are competent and effective personnel of the Competent Authority able to investigate, diagnose and report disease or infection, if present;
	.	the absence of infection over a long period of time in susceptible populations can be substantiated by effective disease investigation and reporting by the Competent Authority of the Member Country.
2.	Requirements to Declare a Country, Zone or Aquaculture Establishment Free from Infection with a Specified Pathogen
	The requirements to declare a country, zone or aquaculture establishment free from infection differ depending on the previous infection status of the country, zone or aquaculture establishment, namely:
	.	Absence of susceptible species;
	.	Historically free;
	.	Last known occurrence within the previous 25 years;
	.	Previously unknown infection status.
	2.1.	Absence of susceptible species
		Unless otherwise specified in the relevant disease chapter, a country, zone or aquaculture establishment may be recognised as being free from infection without applying targeted surveillance if there are no susceptible species (as listed in the relevant chapter of the Aquatic Code, or in the scientific literature) present in that country, zone or aquaculture establishment, provided that the prescribed biosecurity conditions have been in place continuously in the country, zone or aquaculture establishment for at least the previous 10 years.
	2.2.	Historically free
		Unless otherwise specified in the relevant disease chapter, a country, zone or aquaculture establishment may be recognised as being free from infection without formally applying targeted surveillance when:
		.	there has never been any observed occurrence of disease;
		or
		.	eradication has been achieved or the disease has ceased to occur for at least 25 years,
		provided that the prescribed biosecurity conditions have been in place continuously in the country, zone or aquaculture establishment for at least the previous 10 years.
	2.3.	Last known occurrence within the previous 25 years
		For countries or zones that have achieved eradication (or in which the disease has ceased to occur) within the previous 25 years, in addition to the prescribed biosecurity conditions, appropriate targeted surveillance must have been applied to demonstrate the absence of the infection, consistent with the provisions of Section B of this chapter.
	2.4.	Previously unknown infection status
		For countries or zones with previously unknown infection status, or which have not previously met the requirements of the Sections A.2.1, A.2.2 or A.2.3 above, the prescribed biosecurity conditions must be introduced in addition to targeted surveillance consistent with the provisions of Section B of this chapter.
3.	Guidelines for the Maintenance of Continued Recognition of Freedom from Infection
	A country, zone or aquaculture establishment that has been recognised free from infection following the provisions of Sections A.2.1 or A.2.2, may maintain its official status as infection-free provided that the prescribed biosecurity conditions are continuously maintained.
	A country, zone or aquaculture establishment that has been recognised free from infection following the provisions of Sections A.2.3 or A.2.4, may discontinue targeted surveillance and maintain its official status as infection-free provided that the prescribed biosecurity conditions are continuously maintained.
	The different paths to recognition of freedom from infection are summarised in the diagram below.

B. TARGETED SURVEILLANCE FOR DEMONSTRATION OF FREEDOM FROM INFECTION

1.	Introduction
	This section provides standards to be applied when demonstrating country, zone or aquaculture establishment freedom from infection, in accordance with the principles of Section A. Standards described in this section may be applied to all diseases, their agents and susceptible species as listed in the Aquatic Code, and are designed to assist with the development of surveillance methodologies. More detailed information in each disease chapter (where it exists) of this Aquatic Manual may be used to further refine the general approaches described in this chapter. Where detailed disease/infection-specific information is not available, suitable values should be chosen based on the guidelines in this chapter.
2.	General Principles
	Demonstrating freedom from infection involves providing sufficient evidence to demonstrate that infection with a specified agent is not present in a specified population. In practice, it is not possible to definitively prove that a population is free from infection (unless every member of the population is examined simultaneously with a perfect test with both sensitivity and specificity equal to 100%). Instead, the aim is to provide adequate evidence (to an acceptable level of confidence), that infection, if present, is present in less than a specified proportion of the population.
	Methodologies to demonstrate freedom from infection should be flexible to deal with the complexity of real life situations. No single method is applicable in all cases. Methodologies must be able to accommodate the variety of aquatic animal species, the multiple diseases of relevance, varying production and surveillance systems, and types and amounts of data and information available.
	The methodology used should be based on the best available information that is in accord with current scientific thinking. The methodology should be well documented and supported with references to the scientific literature and other sources, including expert opinion.
	Consistency in methodologies should be encouraged and transparency is essential in order to ensure fairness and rationality, consistency in decision making and ease of understanding by all the interested parties. Applications for recognition of infection-free status should document the uncertainties, the assumptions made, and the effect of these on the final estimate.
3.	General Requirements for Demonstration of Freedom from Infection
	3.1.	Population
		The target population to which the demonstration of freedom from infection applies is all individuals of all species susceptible to the infection in a country, zone or aquaculture establishment.
		The study population may be the same as the target population or a subset of it. The study population should be (in order of preference):
		.	The appropriate study population as defined in the relevant disease chapter of the Aquatic Code (if such a definition exists),
		.	A subset of the target population that defines a group of animals which, if infection were present, would be most likely to have a higher prevalence of infection than the target population. This subset should be defined in terms of:
			.	species;
			.	time (e.g. season or month of year);
			.	stage of life-cycle or growth period;
			.	production system and/or management characteristics;
			.	location;
			.	readily identifiable physical or behavioural characteristics.
		.	The same as the target population,
		.	A subset of the target population with the same or lower probability of infection. The nature and impact of any biases on the results of the analysis must be considered, documented and taken into account in the analysis.
	3.2.	Sources of evidence
		Evidence of freedom from infection may be based on a number of different sources, including:
		.	structured, population-based surveys using one or more tests for the presence of the agent;
		.	other surveillance, including structured non-random surveillance, such as:
			.	sentinel sites;
			.	disease notifications and laboratory investigation records;
			.	academic and other scientific studies;
		.	a knowledge of the biology of the agent, including environmental, host population distribution, known geographical distribution, vector distribution and climatic information;
		.	history of imports of potentially infected material;
		.	biosecurity measures in place;
		.	evaluation of the official services; or
		.	any other sources that provide contributory evidence that infection is not present in the country, zone or aquaculture establishment.
		The sources of evidence used to demonstrate freedom from infection must be fully described. In the case of a structured survey, this must include a description of the sampling strategy used for the selection of units for testing. For complex surveillance systems, a full description of the system is required including consideration of any biases that may be inherent in the system.
	3.3.	Statistical methodology
		Analysis of data for evidence of freedom from infection involves estimating the probability (á) that the evidence observed (the results of surveillance) could have been produced under the null hypothesis that infection is present in the population at a specified prevalence(s) (the design prevalence[s]). The confidence in (or, equivalently, the sensitivity of) the surveillance system that produced the evidence is equal to 1-á. If the confidence level exceeds a pre-set threshold, the evidence is deemed adequate to demonstrate freedom from infection.
		The required level of confidence in the surveillance system (probability that the system would detect infection if infection were present at the specified level) must be greater than or equal to 95%.
		The power (probability that the system would report that no infection is present if infection is truly not present) may be set to any value. By convention, this is often set to 80%, but may be adjusted according to the country's or zone's requirements.
		Different statistical methodologies for the calculation of the probability á, including both quantitative and qualitative approaches, are acceptable as long as they are based on accepted scientific principles.
		The methodology used to calculate the confidence in the surveillance system must be scientifically based and clearly documented, including references to published work describing the methodology.
	3.4.	Clustering of infection
		Infection in a country, zone or aquaculture establishment usually clusters rather than being uniformly distributed through a population. Clustering may occur at a number of different levels (e.g. a cluster of moribund fish in a pond, a cluster of ponds in a farm, or a cluster of farms in a zone). Except when dealing with demonstrably homogenous populations, approaches to demonstrating freedom must take this clustering into account in the design and the statistical analysis of the data, at least at what is judged to be the most significant level of clustering for the particular animal population and infection.
	3.5.	Design prevalence
		Calculation of the confidence of a surveillance system is based on the null hypothesis that infection is present in the population. The level of infection is specified by the design prevalence. In the simplest case, this is the prevalence of infection in a homogenous population. More commonly, in the presence of disease clustering, two design prevalence values are required, for instance, the animal-level prevalence (proportion of fish infected in an infected farm) and the group-level prevalence (proportion of infected farms in the country, zone or aquaculture establishment). Further levels of clustering may be considered, requiring further design prevalence values.
		The values for design prevalence used in calculations must be those specified in the relevant disease chapter (if present) of this Aquatic Manual. If not specified for the particular disease, justification for the selection of design prevalence values must be provided, and should be based on the following guidelines:
		.	At the individual animal level, the design prevalence is based on the biology of the infection in the population. It is equal to the minimum expected prevalence of infection in the study population, if the infection had become established in that population. It is dependent on the dynamics of infection in the population and the definition of the study population (which may be defined to maximise the expected prevalence in the presence of infection).
		.	A suitable design prevalence value at the animal level (e.g. prevalence of infected animals in a cage) may be:
			.	between 1% and 5% for infections that are transmitted slowly; and
			.	over 5% for more contagious infections.
		.	At higher levels (e.g. cage, pond, farm, village, etc.) the design prevalence usually reflects the prevalence of infection that is practically and reasonably able to be detected by a surveillance system. Detection of infection at the lowest limit (a single infected unit in the population) is rarely feasible in large populations. The expected behaviour of the infection may also play a role. Infections that have the ability to spread rapidly between farms may have a higher farm-level design prevalence than slow-moving infections.
			A suitable design prevalence value for the first level of clustering, (e.g. proportion of infected farms in a zone) may be up to 2%.
	3.6.	Test characteristics
		All surveillance involves performing one or more tests for evidence of the presence of current or past infection, ranging from detailed laboratory examinations to farmer observations. The performance level of a test at the population level is described in terms of its sensitivity and specificity. Imperfect sensitivity and/or specificity impact on the interpretation of surveillance results and must be taken into account in the analysis of surveillance data.
		All calculations must take the performance level (sensitivity and specificity) of any tests used into account. The values of sensitivity and specificity used for calculations must be specified, and the method used to determine or estimate these values must be documented. Where values for sensitivity and/or specificity for a particular test are specified in this Aquatic Manual, these values may be used without justification.
		Where more than one test is used in a surveillance system (sometimes called using tests in series or parallel), the overall test system sensitivity and specificity must be calculated using a scientifically valid method.
		Pooled testing involves the pooling of specimens from multiple individuals and performing a single test on the pool. Pooled testing is an acceptable approach. Where pooled testing is used, the results of testing must be interpreted using sensitivity and specificity values that have been determined or estimated for that particular pooled testing procedure and for the applicable pool sizes being used. Analysis of the results of pooled testing must, where possible, be performed using accepted, statistically based methodologies, which must be fully documented, including published references.
	3.7.	Multiple sources of evidence
		Where multiple different data sources providing evidence of freedom from infection exist or are generated, each of these data sources may be analysed according to the provisions of Sections B.3, B.4 (for structured surveys) and B.5 (for complex data sources). The resulting estimates of the confidence in each data source may be combined to provide an overall level of confidence for the combined data sources.
		The methodology used to combine the estimates from multiple data sources:
		.	must be scientifically valid, and fully documented, including references to published material; and
		.	should, where possible, take into account any lack of statistical independence between different data sources.
		Surveillance information gathered from the same country, zone or aquaculture establishment at different times may provide cumulative evidence of freedom from infection. Such evidence gathered over time may be combined into an overall level of confidence. For instance, repeated annual surveys may be analysed to provide a cumulative level of confidence. However, a single (larger) survey may be able to achieve the same level of confidence in just 1 year.
		Analysis of surveillance information gathered intermittently or continuously over time should, where possible, incorporate the time of collection of the information to take the decreased value of older information into account.
4.	Specific Requirements for Structured Survey Design and Analysis
	One method of generating evidence for freedom from infection is the use of structured, population-based, targeted surveys. In addition to the requirements specified in Section B.3, the following guidelines should be used when implementing and analysing surveys to demonstrate freedom from infection.
	4.1.	Survey design
		The most important unit of diagnosis is the epidemiological unit. The population of epidemiological units must be clearly defined.
		The design of the survey will depend on the size and structure of the population being studied. If the population is relatively small and can be considered to be homogenous with regards to risk of infection, a single-stage survey can be used.
		In larger populations where a sampling frame is not available, or when there is a likelihood of clustering of disease, multi-stage sampling is required. In two-stage sampling, at the first stage of sampling, groups of animals (e.g. ponds, farms or villages) are selected. At the second stage, animals are selected for testing from each of the selected groups.
		Stratification may be used in survey design.
	4.2.	Sampling
		The objective of sampling from a population is to select a subset of units from the population that is representative of the population with respect to the characteristic of interest (in this case, the presence or absence of infection). Sampling should be carried out in such a way as to provide the best likelihood that the sample will be representative of the population, within the practical constraints imposed by different environments and production systems.
		Biased or targeted sampling in this context involves sampling from a defined study population that has a different probability of infection than the target population of which it is a subpopulation. Once the study population has been identified, the objective is still to select a representative sample from this subpopulation.
	4.3.	Sampling methods
		The survey design may involve sampling at several levels.
		For sampling at the level of the epidemiological units or higher units, a formal probability sampling (e.g. simple random sampling) method must be used.
		When sampling below the level of the epidemiological unit (e.g. individual animal), the sampling method used should provide the best practical chance of generating a sample that is representative of the population of the chosen epidemiological unit. Collecting a truly representative sample of individual animals (whether from a pond, cage or fishery) is often very difficult. To maximise the chance of finding infection, the aim should be to bias the sampling towards infected animals, e.g. selecting moribund animals, life stages with a greater chance of active infection, etc.
		The sampling method used at all levels must be fully documented and justified.
	4.4.	Sample size
		The number of units to be sampled from a population should be calculated using a statistically valid technique that takes at least the following factors into account:
		.	The sensitivity and specificity of the diagnostic test, or test system;
		.	The design prevalence (or prevalences where a multi-stage design is used);
		.	The level of confidence that is desired of the survey results.
		Additionally, other factors may be considered in sample size calculations, including (but not limited to):
		.	The size of the population (but it is acceptable to assume that the population is infinitely large);
		.	The desired power of the survey;
		.	Uncertainty or variability in estimates of sensitivity and specificity.
		The specific sampling requirements will need to be tailor-made for each individual disease, taking into account its characteristics and the specificity and sensitivity of the accepted testing methods for detecting the disease agent in host populations.
		Detailed guidelines are to be provided in the next (fifth) edition of the Aquatic Manual. In the meantime, the sampling procedures given in Chapters I.1, I.2 and I.3 may be applied.
	4.5.	Data analysis
		Analysis of test results from a survey shall be in accordance with the provisions of Section B.3 and take at least the following considerations into account:
		.	The survey design;
		.	The sensitivity and specificity of the test, or test system;
		.	The design prevalence (or prevalences where a multi-stage design is used);
		.	The results of the survey.
	4.6.	Quality assurance
		Surveys should include a documented quality assurance system, to ensure that field and other procedures conform to the specified survey design. Acceptable systems may be quite simple, as long as they provide verifiable documentation of procedures and basic checks to detect significant deviations of procedures from those documented in the survey design.
5.	Specific Requirements for Complex Non-survey Data Sources
	Data sources that provide evidence of freedom from infection, but are not based on structured population-based surveys may also be used to demonstrate freedom, either alone or in combination with other data sources. Different methodologies may be used for the analysis of such data sources, but the methodology must comply with the provisions of Section B.3. The approach used should, where possible, also take into account any lack of statistical independence between observations.
	Analytical methodologies based on the use of step-wise probability estimates to describe the surveillance system may determine the probability of each step either by:
	.	the analysis of available data, using a scientifically valid methodology; or where no data are available,
	.	the use of estimates based on expert opinion, gathered and combined using a formal, documented and scientifically valid methodology.
	Where there is significant uncertainty and/or variability in estimates used in the analysis, stochastic modelling or other equivalent techniques should be used to assess the impact of this uncertainty and/or variability on the final estimate of confidence.

PART 2

EXAMPLE SURVEILLANCE SYSTEMS

The following examples describe surveillance systems and approaches to the analysis of evidence that are able to meet the requirements of Part 1 of this chapter. The purpose of these examples is:
 

.	to illustrate the range of approaches that may be acceptable;
.	to provide practical guidance and models that may be used for the design of specific surveillance systems; and
.	to provide references to available resources that are useful in the development and analysis of surveillance systems.

While these examples demonstrate ways in which freedom from infection may be successfully demonstrated, they are not intended to be prescriptive. Countries are free to use different approaches, as long as they meet the requirements of Part 1 of this chapter.
 
The examples deal with the use of structured surveys and are designed to illustrate different survey designs, sampling schemes, the calculation of sample size, and analysis of results. It is important to note that alternative approaches to demonstrating freedom using complex non-survey-based data sources are also currently being developed and may soon be published (International EpiLab, Denmark, Research Theme 1: Freedom from disease. http://www.vetinst.dk/high_uk.asp?page_id=196).
 

EXAMPLE 1 - ONE-STAGE STRUCTURED SURVEY (FARM ACCREDITATION)

Context
 
A freshwater aquaculture industry raising fish in tanks has established a farm accreditation scheme. This involves demonstrating farm-level freedom from a particular (hypothetical) disease (Disease X). The disease does not spread very quickly, and is most common during the winter months, with adult fish at the end of the production cycle being most severely affected. Farms consist of a number of grow-out tanks, ranging from 2 to 20, and each tank holds between 1000 and 5000 fish.
 
Objective
 
The objective is to implement surveillance that is capable of providing evidence that an individual farm is free from Disease X. (The issue of national or zone freedom, as opposed to farm freedom, is considered in the next example.)
 
Approach
 
The accreditation scheme establishes a set of standard operating procedures and requirements for recognition of freedom, based on the guidelines given in this chapter. These require farms to undertake a structured survey capable of producing 95% confidence that the disease would be detected if it were present. Once farms have been surveyed without detecting disease, they are recognised as free, as long as they maintain a set of minimum biosecurity standards. These standards are designed to prevent the introduction of Disease X into the farm (through the implementation of controls specific to the method of spread of that disease) and to ensure that the disease would be detected rapidly if it were to enter the farm (based on evidence of adequate health record keeping and the prompt investigation of unusual disease events). The effective implementation of these biosecurity measures is evaluated with annual on-farm audits conducted by independent auditors.
 
Survey standards
 
Based on the guidelines given in this chapter, a set of standards are established for the conduct of surveys to demonstrate freedom from infection with causative agent of Disease X. These standards include:
 

.	The level of confidence required of the survey is 95% (i.e. Type I error = 5%).
.	The power of the survey is arbitrarily set at 95% (i.e. Type II error = 5%, which means that there is a 5% chance of concluding that a non-diseased farm is infected).
.	The target population is all the fish on the farm. Due to the patterns of disease in this production system, in which only fish in the final stages of grow-out, and only in winter are affected, the study population is defined as grow-out fish during the winter months.
.	The issue of clustering is considered. As fish are grouped into tanks, this is the logical level at which to consider clustering. However, when a farm is infected, the disease often occurs in multiple tanks, so there is little evidence of strong clustering. Also, the small number of tanks on a single farm means that it is difficult to define a design prevalence at the tank level (i.e. the proportion of infected tanks that the survey should be able to detect on the farm). For these reasons, it is decided to treat the entire grow-out population of each farm as a single homogenous population.
.	Stratification is also considered. In order to ensure full representation, it is decided to stratify the sample size by tank, proportional to the population of each tank.
.	The design prevalence at the animal level is determined based on the epidemiology of the disease. The disease does not spread quickly, however, in the defined target population, it has been reported to affect at least 10% of fish, if the population is infected. In order to take the most conservative approach, an arbitrarily low design prevalence of 2% is used. A prevalence of 10% may have been used (and would result in a much smaller sample size), but the authorities were not convinced by the thought that the population could still be infected at a level of say 5%, and disease still not be detected.
.	The test used involves destructive sampling of the fish, and is based on an antigen-detection enzyme-linked immunosorbent assay (ELISA). Disease X is present in some parts of the country (hence the need for a farm-level accreditation programme). This has provided the opportunity for the sensitivity and the specificity of the ELISA to be evaluated in similar populations to those on farms. A recent study (using a combination of histology and culture as a gold standard) estimated the sensitivity of the ELISA to be 98% (95% confidence interval 96.7-99.2%), and the specificity to be 99.4% (99.2-99.6%). Due to the relatively narrow confidence intervals, it was decided to use the point estimates of the sensitivity and specificity rather than complicate calculations by taking the uncertainty in those estimates into account.

Sample size
 
The sample size required to meet the objectives of the survey is calculated to take the population size, the test performance, the confidence required and the design prevalence into account. As the population of each farm is relatively large, differences in the total population of each farm have little effect on the calculated sample size. The other parameters for sample size calculation are fixed across all farms. Therefore, a standard sample size (based on the use of this particular ELISA, in this population) is calculated. The sample size calculations are performed using the FreeCalc software (FreeCalc - Cameron, AR. Software for the calculation of sample size and analysis of surveys to demonstrate freedom from disease. Available for free download from http://www.ausvet.com.au). Based on the parameters listed above, the sample size required is calculated to be 410 fish per farm. In addition, the program calculates that, given the imperfect specificity, it is still possible for the test to produce up to five false-positive reactors from an uninfected population using this sample size. The authorities are not comfortable with dealing with false-positive reactors, so it is decided to change the test system to include a confirmatory test for any positive reactors. Culture is selected as the most appropriate test, as it has a specificity that is considered to be 100%. However, its sensitivity is only 90% due to the difficulty of growing the organism.
 
As two tests are now being used, the performance of the test system must be calculated, and the sample size recalculated based on the test system performance.
 
Using this combination of tests (in which a sample is considered positive only if it tests positive to both tests), the specificity of the combined two tests can be calculated by the formula:

 
which produces a combined specificity of 1 + 0.994 - (1 x 0.994) = 100%
 
The sensitivity may be calculated by the formula:

 

which produces a combined sensitivity of 0.9 x 0.98 = 88.2%
 
These new values are used to calculate the survey sample size yielding a result of 169 fish. It is worth noting that attempts to improve the performance of a test (in this case increase specificity) generally result in a decrease in the performance of the other aspect of the test performance (sensitivity in this example). However, in this case, the loss of sensitivity is more than compensated for by the decreased sample size due to the improved specificity.
 
It is also worth noting that, when using a test system with 100% specificity, the effective power of the survey will always be 100%, regardless of the figure used in the design. This is because it is not possible to make a Type II error, and conclude that the farm is infected when it is not.
 
A check of the impact of population size on the calculated sample size is worthwhile. The calculated sample size is based on an infinitely large population. If the population size is smaller, the impact on sample size is shown in the following table:
 

Population size   Sample size   1000   157   2000   163   5000   166   10,000   169

Based on these calculations, it is clear that, for the population sizes under consideration, there is little effect on the sample size. For the sake of simplicity, a standard sample size of 169 is used, regardless of the number of grow-out fish on the farm.
 
Sampling
 
The selection of individual fish to include in the sample should be done in such a manner as to give the best chance of the sample being representative of the study population. A fuller description of how this may be achieved under different circumstances is provided in Survey Toolbox (Survey Toolbox for Aquatic Animal Diseases - A Practical Manual and Software Package. Cameron A.R. [2002]. Australian Centre for International Agricultural Research [ACIAR], Monograph No. 94, 375 pp. ISBN 1 86320 350 8. Printed version available from ACIAR [[www.aciar.gov.au]. Electronic version available for free download from http://www.ausvet.com.au). An example of a single farm will be used to illustrate some of the issues.
 
One farm has a total of eight tanks, four of which are used for grow-out. At the time of the survey (during winter), the four grow-out tanks have 1850, 4250, 4270 and 4880 fish, respectively, giving a total population of 15,250 grow-out fish.
 
Simple random sampling from this entire population is likely to produce sample sizes from each tank roughly in proportion to the number of fish in each tank. However, proportional stratified sampling will guarantee that each tank is represented in proportion. This simply involves dividing the sample size between tanks in proportion to their population. The first tank has 1850 fish out of a total of 15,250, representing 12.13%. Therefore 12.13% of the sample (21 fish) should be taken from the first tank. Using a similar approach the sample size for the other three tanks is 47, 47 and 54 fish, respectively.
 
Once the sample for each tank is determined, the problem remains as to how to select 21 fish from a tank of 1850 so that they are representative of the population. Several options exist.
 

.	If the fish can be handled individually, random systematic sampling may be used. This is likely to be the case if, for example:
	.	fish are harvested during winter and samples can be collected at harvest; or
	.	routine management activities involving handling the fish (such as grading or vaccination) are conducted during the winter.
	If fish are handled, systematic sampling simply involves selecting a fish at regular intervals. For instance, to select 21 from 1850, the sampling interval should be 1850/21 = 88. This means that every 88th fish from the tank should be sampled. To ensure randomness, it is good practice to use a random number between 1 and 88 (in this case) to select the first fish (e.g. using a random number table), and then select every 88th fish after that.
.	If fish cannot be handled individually (by far the most common, and more difficult, circumstance) then the fish to be sampled must be captured from the tanks. Fish should be captured in the most efficient and practical way possible, however every effort should be made to try to ensure that the sample is representative. In this example, a dip net is the normal method used for capturing fish. Using a dip net, convenience sampling would involve capturing 21 fish by repeatedly dipping at one spot and capturing the easiest fish (perhaps the smaller ones). This approach is strongly discouraged. One method of increasing the representativeness is to sample at different locations in the tank - some at one end, some at either side, some at the other end, some in the middle, some close to the edge. Additionally, if there are differences among the fish, an attempt should be made to capture fish in such a way as to give different groups of fish a chance of being caught (i.e. do not just try to catch the small ones, but include big ones as well).
	This method of collecting a sample is far from the ideal of random sampling, but due to the practical difficulties of implementing random sampling of individual fish, this approach is acceptable, as long as the efforts made to increase the representativeness of the sample are both genuine and fully documented.

Testing
 
Specimens are collected, processed and tested according to standardised procedures developed under the accreditation programme and designed to meet the requirements of this Aquatic Manual. The testing protocol dictates that any specimens that test positive to ELISA be submitted for culture, and that any positive culture results indicate a true positive specimen (i.e. that the farm is not free from disease). It is important that this protocol be adhered to exactly. If a positive culture is found, then it is not acceptable to retest it, unless further testing is specified in the original testing protocol, and the impact of such testing accounted for in the test system sensitivity and specificity estimates (and therefore the sample size).
 
Analysis
 
If the calculated sample size of 169 is used, and no positive reactors are found, then the survey will have a confidence of 95%. This can be confirmed by analysing the results using the FreeCalc software mentioned above (which reports a confidence level of 95.06%).
 
It may happen in some cases that the survey is not conducted exactly as planned, and the actual sample size is less than the target sample size. However, the size of the farm may also be smaller. In these cases, it is advisable to analyse the farm data on a farm-by-farm basis. For example, if only 165 specimens were collected from a farm with only 2520 fish, the resulting confidence would still be 95%. If only 160 fish were collected, the confidence is only 94.5%. If a rigid target of 95% confidence is used, then this survey would fail to meet that target and more evidence would be required.
 

EXAMPLE 2 - TWO-STAGE STRUCTURED SURVEY (NATIONAL FREEDOM)

Context
 
A country aims to declare freedom from Disease Y of crustaceans. The industry in this country is based largely on small-holder ponds, grouped closely together in and around villages. The disease is reasonably highly contagious, and causes mass mortality mid to late in the production cycle, with affected animals becoming moribund and dying in a matter of days. Affected animals show few characteristic signs, but an infected pond will almost invariably break down with mass mortality unless harvested beforehand. It is more common in late summer, but can occur at any time of year. It also occurs occasionally early in the production cycle. In this country, there are some limitations to the availability of laboratory facilities and the transport infrastructure. However, there is a relatively large government structure, and a comprehensive network of fisheries officers.
 
Objective
 
The objective is to establish national freedom from Disease Y. The surveillance system must meet the requirements of Part 1 of this chapter, but must also be able to be practically implemented in this small-holder production system.
 
Approach
 
The aquaculture authorities decide to use a survey to gather evidence of freedom, using a two-stage survey design (sampling villages at the first level, and ponds at the second). Laboratory testing of specimens from a large number of farms is not considered feasible, so a combined test system is developed to minimise the need for expensive laboratory tests.
 
The unit of observation and analysis is, in this case, the pond, rather than the individual animal. This means that the diagnosis is being made at the pond level (an infected pond or a non-infected pond) rather than at the animal level.
 
The survey is therefore a survey to demonstrate that no villages are infected (using a random sample of villages and making a village-level diagnosis). The test used to make a village-level diagnosis is, in fact, another survey, this time to demonstrate that no ponds in the village are affected. A test is then performed at the pond level (farmer observation followed, if necessary, by further laboratory testing).
 
Survey standards
 

.	The confidence to be achieved by the survey is 95%. The power is set at 95% (but is likely to be virtually 100% if the test system used achieves nearly 100% specificity, as demonstrated in the previous example).
.	The target population is all ponds stocked with shrimp in the country during the study period. The study population is the same, except that those remote areas to which access is not possible are excluded. As outbreaks can occur at any time of year, and at any stage of the production cycle, it is decided not to further refine the definition of the population to target a particular time or age.
.	Three tests are used. The first is farmer observation, to determine if mass mortality is occurring in a particular pond. If a pond is positive to the first test (i.e. mass mortality is detected), a second test is applied. The second test used is polymerase chain reaction (PCR). Cases positive to PCR are further tested using transmission experiments.
	.	Farmer observation can be treated as a test just like any other. In this case, the observation of mass mortality is being used as a test for the presence of Disease Y. As there are a variety of other diseases that are capable of causing mass mortality, the test is not very specific. On the other hand, it is quite unusual for Disease Y to be present, and not result in mass mortality, so the test is quite sensitive. A standard case definition is established for 'mass mortality' (for instance, greater than 20% of the pond's population of shrimp observed dead in the space of less than 1 week). Based on this definition, farmers are able to 'diagnose' each pond as having mass mortality. Some farmers may be over-sensitive and decide that mass mortality is occurring when only a small proportion of shrimp are found dead (false positives, leading to a decrease in specificity) while a small number of others fail to recognise the mortalities, decreasing sensitivity.
		In order to quantify the sensitivity and specificity of farmer observation of mass mortalities, as a test for Disease Y, a separate study is carried out. This involves both a retrospective study of the number of mass mortality events in a population that is thought to be free from disease, as well as a study of farmers presented with a series of mortality scenarios, to assess their ability to accurately identify a pond with mass mortality. By combining these results, it is estimated that the sensitivity of farmer-reported mass mortalities as a test for Disease Y is 87% while the specificity is 68%.
	.	When a farmer detects a pond with mass mortality, specimens are collected from moribund shrimp following a prescribed protocol. Tissue samples from 20 shrimp are collected, and pooled for PCR testing. In the laboratory, the ability of pooled PCR to identify a single infected animal in a pool of 20 has been studied, and the sensitivity of the procedure is 98.6%. A similar study of negative specimens has shown that positive results have occasionally occurred, probably due to laboratory contamination, but maybe also because of the presence of non-viable genetic material from another source (shrimp-based feed stuffs are suspected). The specificity is therefore estimated at 99%.
	.	Published studies in other countries have shown that the sensitivity of transmission tests, the third type of test to be used, is 95%, partly due to variability in the load of the agent in inoculated material. The specificity is agreed to be 100%.
	.	Based on these figures, the combined test system sensitivity and specificity are calculated using the formulae presented in Example 1, first with the first two tests, and then with the combined effect of the first two tests and the third test. The result is a sensitivity of 81.5% and a specificity of 100%.
.	The design prevalence must be calculated at two levels. First, the pond-level design prevalence (the proportion of ponds in a village that would be infected if disease were present) is determined. In neighbouring infected countries, experience has shown that ponds in close contact with each other are quickly infected. It is unusual to observe an infected village with fewer than 20% of ponds infected. Conservatively, a design prevalence of 5% is used. The second value for design prevalence applies at the village level, or the proportion of infected villages that could be identified by the survey. As it is conceivable that the infection may persist in a local area without rapid spread to other parts of the country, a value of 1% is used. This is considered to be the lowest design prevalence value for which a survey can be practically designed.
.	The population of villages in the country is 65,302, according to official government records. Those with shrimp ponds number 12,890, based on records maintained by the aquaculture authorities. These are generated through a five-yearly agricultural census, and updated annually based on reports of fisheries officers. There are no records available of the number of ponds in each of these villages.

Sample size
 
Sample size is calculated for the two levels of sampling, first the number of villages to be sampled and then the number of ponds to be sampled. The number of villages to be sampled depends on the sensitivity and the specificity of the test used to classify villages as infected or not infected. As the 'test' used in each village is really just another survey, the sensitivity is equal to the confidence and the specificity is equal to the power of the village-level survey. It is possible to adjust both confidence and power by changing the sample size in the village survey (number of ponds examined), which means that we can determine, within certain limits, what sensitivity and specificity we achieve.
 
This allows a flexible approach to sample size calculation. If a smaller first-stage sample size is desired (a small number of villages), a high sensitivity and specificity are needed, which means that the number of ponds in each village that need to be examined is larger. A smaller number of ponds will result in lower sensitivity and specificity, requiring a larger number of villages. The approach to determining the optimal (least cost) combination of first- and second-stage sample sizes is described in Survey Toolbox.
 
A further complication is presented by the fact that each village has a different number of ponds. In order to achieve the same (or similar) confidence and power (sensitivity and specificity) for each village, a different sample size may be required. The authorities choose to produce a table of sample sizes for the number of ponds to sample in each village, based on the total ponds in each village.
 
An example of one possible approach to determining the sample size follows:
 
The target sensitivity (confidence) achieved by each village-level survey is 95%. The target specificity is 100%. Using the FreeCalc software, with a design prevalence of 1% (the survey is able to detect disease if 1% or more villages are infected), the first-stage sample size is calculated as 314 villages. Within each village, the test used is the combined test system described above with a sensitivity of 81.5% and a specificity of 100%. Based on these figures the following table is developed, listing the number of ponds that need to be sampled in order to achieve 95% sensitivity.
 

Population   Sample size   30   29   40   39   60   47   80   52   100   55   120   57   140   59   160   61   180   62   200   63   220   64   240   64   260   65   280   65   300   66   320   66   340   67   360   67   380   67   400   67   420   68   440   68   460   68   480   68   500   68   1000   70

Sampling
 
First-stage sampling (selection of villages) is done using random numbers and a sampling frame based on the fisheries authorities list of villages with shrimp ponds. The villages are listed on a spreadsheet with each village numbered from 1 to 12,890. A random number table (such as that included in Survey Toolbox) or software designed for the generation of random numbers (such as EpiCalc [http://www.myatt.demon.co.uk/epicalc.htm]) is used.
 
The second stage of sampling involves random selection of ponds within each village. This requires a sampling frame, or list of each pond in the village. The fisheries authorities use trained local fisheries officers to coordinate the survey. For each selected village, the officer visits the village and convenes a meeting of all shrimp farmers. At the meeting, they are asked how many ponds they have and a list of farmers' names and the number of ponds is compiled. A simple random sample of the appropriate number of ponds (between 29 and 70, from the table above, depending on the number of ponds in the village) is selected from this list. This is done either using software (such as Survey Toolbox's RandomAnimal program), or manually with a random number table or decimal dice for random number selection. Details of this process are described in Survey Toolbox. This selection process identifies a particular pond in terms of the name of the owner, and the sequence number amongst the ponds owned (e.g. Mr Smith's 3rd pond). Identification of the actual pond is based on the owners own numbering system for the ponds.
 
Testing
 
Once ponds have been identified, the actual survey consists of 'testing those ponds'. In practice, this involves the farmers observing the ponds during one complete production cycle. The local fisheries officer makes weekly visits to each farmer to check if any of the selected ponds have suffered mass mortality. If any are observed (i.e. the first test is positive), 20 moribund shrimp are collected for laboratory examination (first PCR, and then, if positive, transmission experiments).
 
Analysis
 
Analysis is performed in two stages. First, the results from each village are analysed to ensure that they meet the required level of confidence. If the target sample size is achieved (and only negative results obtained), the confidence should be 95% or greater in each village. At the second stage, the results from each village are analysed to provide a country level of confidence. Again, if the target sample size (number of villages) is achieved, this should exceed 95%.
 

EXAMPLE 3 - SPATIAL SAMPLING AND THE USE OF TESTS WITH IMPERFECT SPECIFICITY

Context
 
A country has an oyster culture industry, based primarily on rack culture of oysters in 23 estuaries distributed along the coastline. In similar regions in other countries, Disease Z causes mortalities in late summer/early autumn. During an outbreak a high proportion of oysters are affected, however, it is suspected that the agent may be present at relatively low prevalence in the absence of disease outbreaks.
 
Objective
 
The national authorities wish to demonstrate national freedom from Disease Z. If the disease should be detected, a secondary objective of the survey is to collect adequate evidence to support zoning at the estuary level.
 
Approach
 
The authorities conclude that clinical surveillance for disease outbreaks is inadequate because of the possibility of low level subclinical infections. It is therefore decided to base surveillance on a structured two-stage survey, in which sampled oysters are subjected to laboratory testing. The first stage of the survey is the selection of estuaries. However, due to the objective of providing evidence for zoning (should disease be found in any of the estuaries), it is decided to use a census approach and sample every estuary. In essence this means that there will be 23 separate surveys, one for each estuary. A range of options for sampling oysters are considered, including sampling at harvest or marketing, or using farms (oyster leases) as a level of sampling or stratification. However the peak time of activity of the agent does not correspond to the harvest period, and the use of farms would exclude the significant numbers of wild oysters present in the estuaries. It is therefore decided to attempt to simulate simple random sampling from the entire oyster population in the estuary, using a spatial sampling approach.
 
Survey standards
 

.	The target population is all of the oysters in each of the estuaries. The study population is the oysters present during the peak disease-risk period in late summer early autumn. Wild and cultured oysters are both susceptible to disease, and may have associated with them different (but unknown) risks of infection. They are therefore both included in the study population. As will be described below, sampling is based on mapping. Therefore the study population can more accurately be described as that population falling within those mapped areas identified as oyster habitats.
.	A design prevalence value is only required at the oyster level (as a census is being used at the estuary level). While the disease is often recognised with very high prevalence during outbreaks, a low value is used to account for the possibility of persistence of the agent in the absence of clinical signs. A value of 2% is selected.
.	The test used is histopathology with immuno-staining techniques. This test is known to produce occasional false-positive results due to nonspecific staining, but is very sensitive. Published studies indicate values of 99.1% for sensitivity and 98.2% for specificity. No other practical tests are available. This means that it is not possible to definitively differentiate false positives from true positives, and that in a survey of any size, a few false positives are expected (i.e. 1.8%).
.	The confidence is set at 95% and the power at 80%. In the previous examples, due to the assumed 100% specificity achieved by use of multiple tests, the effective power was 100%. In this case, with imperfect specificity, there will be a risk of falsely concluding that a healthy estuary is infected, so the power is not 100%. The choice of a relatively low figure (80%) means that there is a 1 in 5 chance of falsely calling an estuary infected when it is not infected, but it also dramatically decreases the survey costs, through a lower sample size.

Sample size
 
Based on the assumption that the sampling procedure will mimic simple random sampling, the sample size (number of oysters to sample per estuary) can be calculated with FreeCalc. The population size (number of oysters per estuary) is assumed to be very large. The calculated sample size, using the sensitivity, specificity and design prevalence figures given above, is 450. FreeCalc also reports that, based on this sample size and the specificity of the test, it is possible to get 10 or fewer false-positive test results, and still conclude that the population is free from disease. This is because, if the population were infected at 2% or greater, the anticipated number of positive reactors from a sample of 450 would be greater than 10. In fact, we would expect 9 true positives (450 x 2% x 99.1%) and 8 false positives (450 x 98% x 1.8%) or a total of 17 positives if the population were infected at a prevalence of 2%.
 
This illustrates how probability theory and adequate sample size can help differentiate between true- and false-positive results when there is no alternative but to use a test with imperfect specificity.
 
Sampling
 
The aim is to collect a sample of 450 oysters that represent an entire estuary. Simple random sampling depends on creating a sampling frame listing every oyster (not possible) and systematic sampling depends on being able to (at least conceptually) line up all the oysters (again, not possible). The authorities decide to use spatial sampling to approximate simple random sampling. Spatial sampling involves selecting random points (defined by coordinates), and then selecting oysters near the selected points. In order to avoid selecting many points with no oysters nearby, the estuary is first mapped (the fisheries authorities already have digital maps defining oyster leases available). To these maps areas with significant concentrations of wild oysters are also added, based on local expertise. Pairs of random numbers are generated such that the defined point falls within the defined oyster areas. Other schemes are considered (including using a rope marked at regular intervals, laid out on a lease to define a transect, and collecting an oyster adjacent to each mark on the rope) but the random coordinate approach is adopted.
 
Survey teams then visit each point by boat (using a GPS [Global Positioning System] unit to pinpoint the location). A range of approaches is available for selecting which oyster to select from a densely populated area, but it should involve some effort at randomness. Survey staff opt for a simple approach: when the GPS receiver indicates that the site has been reached, a pebble is tossed in the air and the oyster closest to the point where it lands is selected. Where oysters are arranged vertically (e.g. wild oysters growing up a post), a systematic approach is used to determine the depth of the oyster to select. First, an oyster at the surface, next, an oyster halfway down, and thirdly, an oyster as deep as can be reached from the boat.
 
This approach runs the risk of bias towards lightly populated areas, so an estimate of the relative density of oysters at each sampling point is used to weight the results (see Survey Toolbox for more details).
 
Testing
 
Specimens are collected, processed, and analysed following a standardised procedure. The results are classified as definitively positive (showing strong staining in a highly characteristic pattern, possibly with associated signs of tissue damage), probably positive (on the balance of probabilities, but less characteristic staining), and negative.
 
Analysis
 
The interpretation of the results when using a test with imperfect specificity is based on the assumption that, in order to conclude that the population is free from infection, any positive result identified is really a false positive. With a sample size of 450, up to 10 false positives may be expected while still concluding that the population is free from disease. However, if there is reasonable evidence that there is even a single true positive, then the population cannot be considered free. This is the reason for the classification of positive results into definitive and probable positives. If there are any definitive positives at all, the population in that estuary must be considered infected. The probable positives are consistent with false positives, and therefore up to 10 may be accepted. Using FreeCalc the actual confidence achieved based on the number of (presumed) false positives detected can be calculated. For instance, if 8 'probably positive' results were detected from an estuary, the confidence level for the survey would be 98.76%. On the other hand, if 15 'probably positive' results were detected, the confidence is only 61.9%, indicating that the estuary is likely to be infected.
 
Discussion
 
Normally, it may be safely assumed that a surveillance system aimed at demonstrating freedom from disease is 100% specific. This is because any suspected occurrence of disease is investigated until a definitive decision can be made. If the conclusion is that the case is truly a case of disease, then there is no issue of declaring freedom - the disease is known to be present. This example presents a different situation where, due to lack of suitable tests, it is not possible for the surveillance system to be 100% specific. This may represent an unusual situation in practice, but illustrates that methods exist for dealing with this sort of problem. In practice, a conclusion that a country (or estuary) is free from infection, in the face of a small (but statistically acceptable) number of positive results, will usually be backed up by further evidence (such as the absence of clinical disease).
 

[retour au début]

Copyright Office international des épizooties (OIE) - scientific.dept@oie.int