Monitoring of patient safety is an indispensable part of clinical trial planning and conduct. Proactive safety signal monitoring using blinded data in on-going clinical trials enables pharmaceutical sponsors to monitor patient safety closely while maintaining the study blind. Bayesian methods, by their nature of updating knowledge based on accumulating data and synthesis of prior knowledge, provide an excellent framework for carrying out such monitoring of safety. This short communication summarizes a straightforward Bayesian framework which can be applied to safety monitoring for one or more adverse events of special interest in real clinical trial settings. This framework is general enough to allow adaptation to a number of different Bayesian models appropriate for application to different clinical settings and types of data (such as rare events, exposure-dependent events or continuous laboratory parameters). An instructive case study is presented to demonstrate the utility of the proposed method.

•    Safety signal monitoring
•    Interim analysis
•    Bayesian monitoring
•    Blinded clinical trial

DMC: Data Monitoring Committee; AESI: Adverse Event of Special Interest

Wen S, Ball G, Dey J (2015) Bayesian Monitoring of Safety Signals in Blinded Clinical Trial Data. Ann Public Health Res 2(2): 1019.

In randomized clinical trials, interim reviews of safety data are usually planned during the design stage and conducted periodically by a data monitoring committee (DMC). For these planned interim analyses, unblinded safety data are used to compare the safety profile of the experimental treatment to that of the control group. In order to minimize any operational bias on the part of the sponsor, the DMC would typically be independent and have minimal contact with the study team. Since safety signals may emerge at any time during a trial, it is also imperative for pharmaceutical sponsors to monitor patient safety in a real-time fashion during an on-going trial while maintaining the study blind, and engage the DMC as appropriate. However, formal statistical methods for such monitoring are not commonly available or in use. While clearly less informative than an unblinded review, blinded analysis of safety data, where the treatment group assignment is not revealed, can meaningfully augment and is logistically simpler than the DMC review process [1]. It can help identify potential safety issues ahead of scheduled DMC meetings and can help prevent such issues from becoming serious concerns. Moreover, for studies without a DMC, blinded safety monitoring could be used to assess the need for performing an unblinded safety analysis and/or establishing a DMC.

By virtue of incorporating prior knowledge about the safety profile of the control group and updating knowledge based on accumulating data, Bayesian methods provide an excellent framework for carrying out efficient and effective monitoring of blinded safety data. This short communication summarizes a simple Bayesian framework which can be applied to safety monitoring for one or more adverse events of special interest (AESI) in a real clinical trial setting. A simulation study is used to demonstrate the value for this method [2].

The Bayesian framework proposed in this paper is based on evaluating the probability that a clinical parameter of interest exceeds a pre-specified critical value, given the observed blinded data. The critical value will typically be selected based on historical data about the control group or medical judgment. If this probability were to ever get big enough, it would signal a potential safety concern, leading to further investigation to confirm whether there is truly a safety problem related to the experimental treatment or not. Mathematically, this is formulated as checking the following inequality involving a Bayesian posterior probability (1):

Pr(θ > θc | blinded data) > P cut-off …... (1),

where θ is a particular clinical parameter or metric of interest (such as pooled proportion, risk difference or odds-ratio), θ c represents the critical value for comparison, and P cut-off is a probability threshold (such as 90% or 99%) representing the desired confidence needed to identify a potential safety signal.

In a blinded data setting, a common choice for θ is the pooled incidence rate (overall proportion) of an adverse event of special interest (AESI) among all subjects, since this is the rate that is directly estimable. In this case, a conjugate beta-binomial model would be a natural choice for Bayesian probability calculations. As noted above, other models within the general framework can be chosen depending on the inferential setting and data type, such as a gamma-Poisson model for incidence of rare events or to account for drug exposure at the time of analysis, or a conjugate normal-normal model when θ is continuous, such as the mean of a particular lab parameter.

A typical process for blinded safety monitoring of a particular AESI in a clinical trial setting can be described in the following steps.

1. The choice of the probability model, prior parameters and probability threshold should be pre-specified before the analysis begins. Usually several rounds of trial simulations are run to fine tune the decision parameters and fully understand the operating characteristics (OC) of the decision criterion prior to finalizing.

2. Assuming subjects are continuously enrolled into the study, after a certain number of subjects have been enrolled, start evaluating the posterior probability as described in inequality (1) based on the pre-defined decision criterion.

3. Apply the signal identification criterion at the desired predetermined frequency until the end of the trial or until the prespecified threshold is crossed. If the threshold is crossed, carry out additional investigations and/or consider an unblinded safety review, as appropriate.

We provide an illustrative example based on a real clinical trial setting to demonstrate the utility of the safety signal detection criterion and the blinded monitoring process. Consider a double blind, parallel group, randomized clinical trial of an experimental treatment versus placebo to prevent complications in patients undergoing major cardiovascular surgery. A total of 240 subjects were randomized to receive active drug or placebo in a 3:1 ratio. All randomized subjects were supposed to receive one dose of experimental study drug or placebo on day 1. The AESI under consideration was an early-onset event with a 4 week observation period and majority of events expected to occur within 10 days of dosing. The proposed safety monitoring process was to begin when 50 subjects overall had received study drug. Based on historical data, the background rate of the AESI was expected to be around 0.4% and therefore, the decision criterion was set as Pr (θ > 0.004 | the blinded data) > 0.99. Thus, a safety alert would be triggered if the posterior probability suggested that the overall incidence rate exceeding 0.4% was almost certain (exceeded 99%). A beta-binomial model was chosen as the primary probabilistic model and clinical trial simulations were conducted to understand the operating characteristics – essentially the power curve – of this decision criterion. Table 1 describes how parameter choices were matched with the clinical information for simulations to mimic the trial setting as closely as possible.

Since an incidence rate more than 2% was determined to be of serious clinical concern, sensitivity was mainly assessed under this assumption. To understand the robustness of findings, simulations were also performed for different prior settings and probability models as described in the next section. For each choice of model and decision criterion, a graphical monitoring chart can be created (as in Figure 1), showing the decision boundary in terms of the number of events observed by subjects treated which simplifies real-time application of the process. This chart is particularly helpful for study physicians.

Figure 1 shows the decision boundary for the primary model with a “flat” Uniform (0,1), i.e. Beta (1, 1), prior used to reflect the high uncertainty about the overall incidence rate on the trial. In particular, all incidence rates from 0% to 100% are equally likely under this prior, and there is a very high (98%) prior probability of the overall incidence rate exceeding 2%. With this prior, if 3 or more AESIs were to be observed out of 100 treated subjects, then an alert would be warranted (Figure 1).

Figure 2 shows the OC of the decision criterion in terms of statistical power (percentage of trials with at least one safety alert per the criterion) as a function of the fixed true overall incidence rate based on trial simulations under different choices of priors. Figure 2(a) shows that under the Beta (1, 1) prior, probability of seeing a signal would be high (>80%) when the true rate was 2% or more, and fairly low (<7%) when the true rate was 0.4% or less. Thus the procedure is seen to have at least 93% specificity and more than 80% sensitivity.

Figure 2(b) shows the OC with a slightly more informative prior, Beta (0.1, 5), which has a mean event rate of 0.02 and approximately 18% probability that the overall rate exceeds 0.02. As may be expected, the specificity under this prior would increase to close to 98% but the sensitivity would decrease to less than 70%. Figure 2(c) shows the OC when slight uncertainty about the background rate was incorporated into the model by setting θc to be distributed as Beta (1,249) with the mean at 0.004. The sensitivity would decrease further, but only slightly. Figure 2(d) shows the OC for the safety signal decision criterion using a gamma-Poisson model with a less informative prior distribution of Gamma (0.001, 0.001). The purpose of this less informative prior is just to let the data dominate the posterior distribution. When incidence rates are low, the Poisson distribution provides a good approximation to the binomial. The performance of the procedure under this model is seen to be very similar to that in Figure 2(b).

Table 1: Choice of simulation parameters to match the corresponding clinical trial information.

These results show that it is feasible to meaningfully implement a formal process of continuously monitoring blinded data to augment the current practice of periodic unblinded safety reviews, as by a DMC. The performance of such blinded signal monitoring depends heavily on reliability of prior knowledge about the background incidence rate of the outcome of interest in the study population. If prior information can be appropriately incorporated, potential safety issues can be detected quicker and with more certainty. This is especially valuable for blinded studies without a formal DMC, to determine if and when a DMC may be needed. The proposed Bayesian approach can be easily adapted to different data types and decision criterion, and can provide useful information in a straightforward way for monitoring one or a few well-defined adverse events of special interest. Also, instead of posterior probabilities, as illustrated in this short communication, one may consider predictive probabilities for decision making, such as, how likely it would be for two more AESI to occur when an additional 20 subjects were enrolled. Although the mathematical evaluation of these probabilities could be different, the safety signal monitoring process and the decision framework would be the same.

AbbVie Inc. funded the study. AbbVie was responsible for the study design, research, analysis, data collection, interpretation of data, and writing, reviewing, and approving of the publication. Shihua Wen, Greg Ball, and Jyotirmoy Dey are employees of AbbVie, Inc.

1. Ball G, Piller LB, and Silverman MH. Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Contemporary Clinical Trials 2011; 32: S2-S4.

2. Wen S, Dey J, Ball G, Kracht K. Application of a Solo-Bayesian Method in Blinded Safety Monitoring and Signal Detection in Clinical Trials. Joint Applied Statistics Symposium of International Chinese Statistical Association & Korean International Statistical Society (ICSA/KISS). 2014.