Cut points are the threshold values for the distinction of positive and negative results in immunogenicity assays. Establishing appropriate cut points is crucial to ensuring acceptable assay sensitivity, and, although there are assay-specific characteristics to be considered when setting the cut point, some fundamental aspects should be considered when setting cut points in immunogenicity assays.
1. Statistical Methods
Cut points should be calculated applying newest statistical tools using samples from treatment-naïve subjects to obtain robust numbers and take the variability of the assays into account. The cut point should be determined with results from tests using a minimum of 50 samples on at least three different days. It is recommended that the cut point for ADA screening assays be determined by a 90% one-sided lower confidence interval for the 95th percentile of the negative control population. This will assure at least a 5% false-positive rate with a 90% confidence level. As the assessment of immunogenicity should be performed in a tiered approach, a confirmatory assay should follow the screening assay. The confirmatory assay cut point should be calculated using an 80% one-sided lower confidence interval for the 99th percentile allowing a 1% false-positive rate.
While these algorithms are straight forward, there are multiple factors that need to be considered. Among these are the distribution of the values and the variability of the data. Particularly, if the mean of the signals varies between assays, plates, or analysts but the variance around the mean is constant, a normalization factor should be applied. The majority of immunogenicity screening assays apply a floating cut point, even if both mean and variance are constant. Secondly, the determination of outliers is crucial for a statistical evaluation of the data. The outlier elimination can be challenging and the method is still divergent among laboratories. The elimination should be performed iteratively with removal of data points being justified.
2. Target Disease Cut Points
Different populations may lead to different immune response due to the different levels of immune responses. For example, it is generally expected that patients with autoimmune diseases may have a higher background immunoreactivity than the normal population or immune compromised patients. Consequently, samples from different target populations and disease states may have components that can cause the background signal from the assay to vary, and different cut points may be needed for discrete populations being studied. Moreover, some disease entities are often classified as distinct disease entities but in fact they are syndromes with different etiologies and a variety of clinical signs and symptoms. In consequence, one target disease population will differ from another, or in other a pre-study target disease population may differ in immune reactivity from an in-study population of the same target disease. Hence, a target disease specific cut point might be needed to confirm that the cut point determined during assay validation is suitable for the population being studied.
Besides the intrinsic differences in immune stimulation, commercial samples used during the pre-study validation may be handled differently than study samples. In these cases, a sufficient number of samples from the study population should be used to calculate an in-study target disease cut point. In addition, an in-study cut point may be needed if the overall signal values within the study population differ >30% from the signals in the pre-study population. In addition, consideration of the possible need of a study-specific cut point may arise if the incidence of false positives diverges too greatly from the desired 5% rate, i.e.e less than 2% or greater than 10% can lead to a request from regulatory agencies to assess/consider an in-study cut point.
3. Pre-existing Antibodies
Natural antibodies may be present in treatment-naïve subjects due to previous exposure to similar or related structures or due to the presence of product-related compounds such excipients or impurities/contaminants. While some pre-existing antibodies are frequently encountered in the context of autoimmune diseases (e.g. rheumatoid factors, insulin), others may be cross-reactive or directed towards non-protein moieties of the biological drug. A specific focus should be directed to pre-existing antibodies against protein modification (e.g PEGylation) as assays needed to be in place to determine the specificity of ADA for the protein component as well as the modification to the therapeutic protein product, i.e. multiple assays need to be in place to measure immune responses to different domains of the molecules.
In any case, pre-existing antibodies need to be considered when calculating the assay cut point and an alternative to the qualitative screening assay approach may be needed in case of the presence of pre-existing antibodies. For example, testing samples for an increase in ADA using a semi-quantitative assay type such as a titering assay can provide information on the impact of a therapeutic protein product on product immunogenicity that is not provided by a qualitative assay. In particular, the prevalence of subjects whose baseline samples were tested (with reportable results) need to be determined and the percentage of baseline ADA-positive subjects with significant increases in ADA titer after drug administration need to be calculated. Following this approach, ADA titers that are greater than the baseline titer by a scientifically reasonable margin, such as four- or nine-fold, is considered to be treatment boosted. It should be noted, that a low serial dilution scheme for titering such as twofold or threefold is recommended. Due to the format of titration assays, close titer values ought to be compared cautiously based on a reasonable margin of the imprecision of the titration method.