Three Times And We're Alright? Replicates In Validation

Three Times And We’re Alright? Replicates In Validation By Tim Sandle Mar 31, 2021 7:00 am EDT Patterns of three occur regularly in science. For example, there are three main types of stable particles: the proton, the neutron, and the electron. These are the three building blocks of atoms. All solid matter consists of atoms built entirely from these three particles. INTRODUCTION Numbers are essential to validation, just as numbers are critical to much of human activity. Numbers represent the continuous variables that we use to describe the world. Numbers can be something directly observable, or they can be theoretical constructs, used as parameters within a model. While both play a part in ‘validation’, it is with the former that this article toys with. The issue at hand is with how much validation is required at the performance qualification stage for a given validation stage? This may be a cleaning cycle on a vessel or an autoclave load. How many times should such validation exercises be conducted in order to show that the results are reliable? While the number of runs should rest with the outcome of a risk assessment, it is generally the rule that the number of runs is seldom less than three. This begs two questions: why is three considered acceptable?, and is three enough? Before entering this debate it is interesting to consider what is so special about the number 3. After this, this article looks at the issue from the process validation perspective and the analytical assay perspective, concluding that three runs or assays is rarely enough. Moreover, with process validation greater weight should be put onto the lifecycle approach for trending and hence understanding process performance. THE PATTERN OF ‘THREE’ There is, in both society and with the natural world, something particular about ‘three’. Whether humanity has simply become obsessed with three or whether our common use of things in patterns of three is in our subconscious, reflecting order within science and nature, is uncertain. Coincidental or inherent, there is a regular occurrence of patterns of three in how we have constructed our world and how our world is directed by a natural order. Societal Resonance In terms of society and social abstraction, three is the smallest number to which we can describe the meaning “all” (1). Three straddles the line between some and a lot, between finite and, for all practical purposes, infinite. Moreover, three is the smallest number needed to create a pattern, providing both a combination of brevity and rhythm. This is captured in the Latin phrase omne trium perfectum: every set of three is complete. In ancient societies, three has tended to carry religious, spiritual, or superstitious significance (2). In music, one of humanity’s civilizing forces, the number three has a significant role with regard to the spatial characteristics of the tone (3). Making three points is the recommended approach for a presentation. In connection to this, politicians tend to use three-of-something to make a point, to capture the take home message. Former US President Barack Obama used the phrase ‘Yes We Can’ (composed of three words) and his speeches were often peppered with groups of three. From further back in US political history, Abraham Lincoln used three phrases when expressing his vision for office: ‘Government of the people by the people for the people’. British Prime Minister Tony Blair set out his electoral strategy in 1997 as being based on: ‘education, education and education’. William Shakespeare would often use sets of three in the soliloquies to be spoken by his characters, as with ‘Friends, Romans, Countrymen’ from Julius Caesar. Three As A Pattern In Science Patterns of three occur regularly in science. For example, there are three main types of stable particles: the proton, the neutron, and the electron. These are the three building blocks of atoms. All solid matter consists of atoms built entirely from these three particles. In terms of other examples, there are three dimensions of space: height, width, and depth. There are three main types of matter: gaseous, liquid, and solid. There are three main types of massive objects: planets, stars, and galaxies (4). From the biological perspective, cell growth, cell differentiation, morphogenesis are required in developmental biology. A global positioning system (GPS) only needs three points to find a location; and in construction, a triangle shape section is stronger than any other shape. The list can go on (5): Three laws of motion by Sir Isaac Newton; Three thermodynamics laws; Three Kepler’s laws on planets’ orbital movement around the Sun; The number of means of heat transfer: heat connection, heat conduction and heat convection; The number of different classifications of rock: igneous, metamorphic (or pyroclastic) and sedimentary etc. VALIDATION AND THE OBJECTIVE OF AVOIDING MAKING DECISIONS BASED ON CHANCE Every decision made in science has a degree of uncertainty, it is a matter of how much uncertainty we are prepared to accept. This degree of limited knowledge is referred to as epistemic uncertainty. This relates to some phenomena that we currently do not know enough about, and this appears, in the validation context, in the assumptions we begin with, the observations we note, to the extrapolations and the generalizations that we make. This means that all final decisions have some level of epistemic uncertainty. If we just validated according to n = 1, this is not science, as one run or one sample has not been shown to be reproducible (6). We can minimize the degree of uncertainty by running the same activity more than once under the same set of conditions. But is doing so just once again sufficient? That something being repeated a second time might be simply due to chance whereas a third set of similar data provides the accepted minima for validation activities. Long before validation was a realizable concept, the same rang true in reportage. In journalism it was said by editors that two is but a fluke, but the third time something happens, it is a trend and hence a story (7). Or to put the entire sequence together: “Once is chance, twice is coincidence, third time is a trend.” Let’s look at this further. Running something once is meaningless, be that an experiment or process validation. This is because if we have conducted a good pre-assessment it is relatively easy to obtain a set of data that conforms with our target range; however, it is just as easy to miss the target or for the next data set to be out of range. Running something twice and where we obtain a data set similar to the fist may simply be coincidence. Whereas three or more provides a degree of confidence that the results are not due to chance if all three independent runs provide parameters within the same range. The use of three, as in three replicates, is also central to analytical methods validation. For example, with linearity to determine the response relationship within the linear range, it is recommended that three replicates (are carried out at each concentration level (8). Furthermore, the limit of detection is typically expressed as the analyte concentration corresponding to the sample blank plus three sample standard deviations, based on as series of independent analyses of sample blanks (9). But is three sufficient? Do we simply default to three because it is convenient or embodied in much of society and in terms of scientific rules? By running three, are we simply resorting to outmoded thinking? It also stands that what is applicable to process validation will be different to what is acceptable for assay development. With process validation, since 2011, the FDA guidance on process validation is required a more detailed consideration than simply ‘default three’. Yet the persistence of three continues to exist in many organizations. In relation to the number of runs and number of samples required, the FDA guidance states (10): “In addition, the CGMP regulations regarding sampling set forth a number of requirements for validation: samples must represent the batch under analysis (§ 211.160(b)(3)); the sampling plan must result in statistical confidence (§ 211.165(c) and (d)); and the batch must meet its predetermined specifications (§ 211.165(a)).” And in a different section: “The number of samples should be adequate to provide sufficient statistical confidence of quality both within a batch and between batches. The confidence level selected can be based on risk analysis as it relates to the particular attribute under examination.” In other words, there should be some kind of scientific, risk based, or statistical basis for the number of runs selected. This often leads to more than three, since the more narrowly focused the information then the more difficult to see patterns and linkages (11). There is ordinarily a requirement for the number of batches selected to be consecutive. As to what is the appropriate answer, approaches like design of experiments (12), supported by a risk assessment (13), to account for the range of process variables present a suitable framework for consideration and to identify, report, and assess the potential failure modes of a product or process and its effects. With assays, a degree of rigor is required. The probability of achieving a reasonable result depends on understanding the true standard error (s) per unit, the number of replicates (r), and experimental error (residual) in relation to the degrees of freedom. The standard error is the measure of the magnitude of the experimental error of an estimated statistic (e.g. average) (14). Approaches to sample numbers will vary upon what aspect of assay criteria requires assessing. For example, to obtain a sample size to achieve a particular precision of the estimated mean, the standard deviation of the mean is the standard error (of the mean) - SE. The value of the SE can be estimated from the standard deviation (SD) of the data and the sample size (N) used to estimate the mean as SE = SD / sqrt(N). To find the N to get a particular SE, it is necessary to solve the equation for N: N = (SD / SE)² For example if the SD is 532, and the objective is to obtain an SE of less than 100, the laboratory will need to measure at least 29 values: N = (532 / 100)² = 28.3 (it is typical to always round up to the whole number). It is important to note that the number of samples or the number of replicates are in themselves insufficient to determine whether an assay is robust. Samples and replicate measurements are used to monitor the performance of the experiment. However, the use of replicates do not function as independent tests of the hypothesis, and so they cannot provide evidence of the reproducibility of the main results. What is needed is running the assay several times. SUMMARY Doing something three times to prove that it works or saying something three times to get a point across, or establishing three laws to explain natural phenomena, works because it is easy to understand and recall. However, in the world of validation three runs may not be sufficient to assess whether a process is robust and consistent. For validating a manufacturing process, or to establish a new assay, this can rarely be reduced to so simplistic an approach as the completion of three successful full scale batches If three is not enough, how many runs are required? To answer this a level of process understanding is required and a scientific and risk based approach adopted, assessing the variables that might lead to inconsistent data and ensuring that the number of runs executed are sufficient to cover these variables. Even when the number of runs is established, the validation process never really stops. The life-cycle approach to validation means that data needs to be consistently reviewed and trended. Science is, after all, knowledge gained through repeated experiment or observation. The conclusion is that the approach of three-times-and-its-over no longer holds firm. REFERENCES 1. Ifrah, G. (1998) The Universal History of Numbers: From Prehistory to the Invention of the Computer transl. London: The Harvill Press, pp393 2. Lease, E. B. (1919) The Number Three, Mysterious, Mystic, Magic, Classical Philology 14 (1): 56-73 3. Mincheva, P.P. (2015) The Phenomenon of the Number 3 in Music, International Journal of Literature and Arts, 3 ( 5): 37-42 4. Mahin, M. (2014) Nature Seems to Love the Number Three, Future and Cosmos, at: https://futureandcosmos.blogspot.com/2014/01/nature-seems-to-love-number... 5. Anon. Number 3 in science, Numberopedia at: https://sites.google.com/site/numberopedia/number3inscience 6. Vaux, D. L., Fidler, F., & Cumming, G. (2012). Replicates and repeats--what is the difference and is it significant? A brief discussion of statistics and experimental design. EMBO reports, 13(4), 291–296. https://doi.org/10.1038/embor.2012.36 7. Newman, A. (2008) Blessed in Triplicate, New York Times, 10th October 2008 at: https://www.nytimes.com/2008/10/12/fashion/sundaystyles/12three.html 8. Belouafa, S., Habti, F., Benhar, S., Belafkih, B., Tayane, S., Hamdouch, S., Bennamara, A. and Abourriche, A. (2017) Statistical tools and approaches to validate analytical methods: methodology and practical examples, Int. J. Metrol. Qual. Eng., 8 (9) DOI: https://doi.org/10.1051/ijmqe/2016030 9. Belafkih, B., Belouafa, S., Charrouf, M., Bennamara, A. Skalli, A. Slaoui, F. and Abourriche, A. (2015) Validation of a method for the quantitation of MDMA in seized materials by spectrophotometric method, Anal. Chem. 15 (6), 219–224 10. FDA Guidance for Industry Process Validation: General Principles and Practices, January 2011, Revision 1 at: Process Validation: General Principles and Practices, FDA, Bethesda, MD, USA 11. Avellanet, J. (2010) Get to Market Now! Turn FDA Compliance into a Competitive Edge in the Era of Personalized Medicine, Logos Press: Washington, D.C., p. 11. 12. Durivage, M.A., 2016, Practical Design of Experiments (DOE), Milwaukee, ASQ Quality Press, USA 13. Durivage, M.A., 2016, Risk-Based Approaches To Establishing Sample Sizes For Process Validation, Life Science Connect, USA 14. Ellison, S., Barwick, V., Duguid Farrant, T. (2009) Practical Statistics for the Analytical Scientist: A Bench Guide, 2nd ed., Royal Society of Chemistry, UK