Recently, there have been published reports stating that a large proportion of studies published in high-impact scientific journals could not be reproduced outside of the originator laboratories. These findings are particularly problematic since many of the drugs screened are being developed for human testing and ultimately FDA approval.
Despite these reproducibility concerns, there is still a strong case for standardization and single-laboratory studies since models, techniques and processes can vary greatly from one lab to another. In addition to this, reproduction of the exact experimental conditions under which another lab performed an experiment is nearly impossible. This is especially true for immuno-oncology studies.
For example, the same cancer cell lines may have fundamentally different characteristics between laboratories such as varying immunogenicity and acquired mutations. Other factors to consider include the microbiome, animal strain, animal vendor, as well as variations in tumor model systems and technician expertise. Even differences in animal feed and lighting conditions can impact the study outcome.
If third party validation is important yet standardization still plays a role, how do companies design preclinical trials that garner data diversity to achieve high reproducibility? Since duplicating the exact conditions under which another lab performed an experiment is not always feasible, it is recommended to use models that are well validated at each lab. It should also be clear how reproducible the controls are from study to study within each lab. Trying to reproduce data from another lab is always going to be a challenge and scientists must make sure all variables that could drive reproducibility are considered. Researchers should optimize study conditions within the new lab and plan to run more than one experiment to increase confidence.
Studies should also be designed to answer bigger development questions – what model systems will show that a compound works and what combinations should be explored that have clinical development relevance? The goal is to determine if, how and where the drug works. If a drug has a clear signal of activity, scientists should be able to demonstrate that signal of activity in another lab through correctly designed experiments. If a drug only works in one testing laboratory, it is likely that the drug will encounter challenges in development in the future.
It is also important to understand what is at risk when attempting to include environmental diversity to a drug program. Additional unwanted variables can be introduced due to lack of technical expertise, inadequate drug handling and failure to follow formulation protocols, to name a few. These unwanted variables can confound the data of even the strongest drug candidate.
Partner with a CRO that takes all factors into account early in the preclinical trial process and offers service and consulting throughout a drug’s journey to inform the most direct clinical path. For organizations exploring either multi-lab and single-lab approaches, a CRO like TD2 can recommend the best strategy and process to achieve high reproducibility and a promising path forward for a particular cancer drug.
