The FDA IND Safety Reporting final rule highlights the importance of aggregate analyses for determining reasonable possibility of an association with study drug for serious adverse events. Clinical safety professionals have been strong qualitative thinkers, focused on individual case review and case series. New guidance requires more quantitative methods, especially for disease-related events; safety data scientists have a lot to offer in this area. Successful implementation will require a partnership between qualitative and quantitative thinkers.
Statistical approaches offer promise for making aggregate comparisons. The FDA supports the involvement of statisticians, but a multidisciplinary approach is needed. Frameworks around aggregate review and level of evidence are what’s needed – not statistical decision rules. Rather than “quantitative”, we need “aggregate” safety evaluations to inform when sponsors identify emerging safety signals. In addition, visual and exploratory approaches are encouraged, especially since they are often so helpful with clarity in uncovering safety issues.
Statistical decision rules are challenging, because those rules may be appropriate for one trial, but not another. One size does not fit all. For events that are common in the background, there needs to be a judgment. We don’t say how to do it, just that you need to consider it – we lay it on the sponsor. These worrisome events, look at them in the whole program. Have an external group doing it, shielded from everything. This can be done. Lilly [the model for the SAC in the 2015 draft guidance] had committees like this 25 years ago.
Assessments should be product specific and decisions driven by medical judgment. There are no specific recommendations on thresholds to be utilized, as sponsors are experts in their programs, indications, and therapeutic areas. Sponsors employ and have access to experts needed to define thresholds based upon this clinical expertise. Thresholds will apply as SAC’s deem them relevant for a compound/indication instead of in general. CTTI speaks to thresholds (Wittes et al, 2015); however, thresholds alone can be problematic. There is an understanding that medical judgment is needed – especially for events that are common in the background – and sponsors will not be faulted for their choices.
When speaking to the IND rule and discussing safety monitoring, the FDA prefers to speak in terms of aggregate data review and not quantitative (statistically driven) analyses. Although this involves less statistical methodology, statistical engagement is still vital for helping the team to determine how best to perform aggregate reviews. Statisticians can be helpful in determining how to implement clinically driven rules defining when a comparison reaches an imbalance that warrants referral to a Safety Assessment Committee (SAC).