"Human Gene Therapy" Systematically Evaluates 14 CRISPR-Cas9 Off-Target Prediction Tools: Why Comput
"Human Gene Therapy" systematically evaluates 14 CRISPR-Cas9 off-target prediction tools: Why can computational predictions not replace experimental validation?
As CRISPR gene editing technology gradually enters the clinic, the assessment of off-target risks has shifted from "whether it needs to be conducted" to "how to conduct it as comprehensively as possible." Currently, computational predictions are often used to screen potential off-target sites: after inputting the sgRNA sequence, software ranks genome-wide candidate sites based on sequence similarity, mismatch position, PAM type, or machine learning models. However, a long-standing problem remains: Do the top-ranked sites actually undergo editing? Will real off-target sites with lower rankings be missed? Can the prediction scores represent the actual off-target frequency?
In 2026, Kaufmann et al. published a study in "Human Gene Therapy", systematically comparing 14 CRISPR-Cas9 off-target prediction tools. The research results indicate that existing computational tools can help researchers narrow down the candidate range but still struggle to fully identify real off-target sites and cannot accurately predict the intensity of off-target editing. Therefore, the authors emphasize that preclinical evaluation still requires a strategy of "combining computational predictions with experimental validation."
The research team included 14 common CRISPR-Cas9 off-target prediction tools, which can be roughly divided into three categories.
The first category consists of tools based on sequence alignment and mismatch rules, including Cas-OFFinder, CCTop, FlashFry, CRISPRitz_mismatch, and GuideScan2. The main task of these tools is to search the whole genome for sites similar to the sgRNA sequence.
The second category consists of tools that provide continuous scores, including CRISPOR, CRISPRitz_cfd, CRISPRoff, MOFF, and CRISOT. They not only find candidate sites but also rank the risks of these candidate sites based on mismatch position, cleavage frequency, thermodynamic features, or molecular interactions.
The third category consists of machine learning models, including CnnCrispr, CRISPR-OFFT, CRISPR-IP, and CRISPert.
Figure 1 | Overview of the CRISPRoffT benchmark dataset used in this study. A: Distribution of the number of deep-sequenced off-target sites per gRNA/Cas9 combination; B: Distribution of indel frequencies across all sequenced sites, where sites with indel frequency ≥ 0.1% are defined as true off-targets; C: Relationship between the number of mismatches between the gRNA and off-target sites and the indel frequency.
The study usedhuman cell data from the CRISPRoffT database, ultimately including 26 gRNA/Cas9 combinations and 3,827 genomic sites subjected to targeted deep sequencing. The authors defined sites with an indel frequency of not less than 0.1% as true off-targets. After removing duplicate sites, a total of 1,806 unique sites were obtained, of which 118 were true off-target sites. Since most machine learning tools do not support DNA bulges or RNA bulges, the machine learning models actually used a further reduced dataset; therefore, their results cannot be directly compared horizontally with traditional tools.
It is worth noting that this study primarily evaluatedDNA double-strand breaks and indel results caused by wild-type SpCas9.Its conclusions cannot be directly applied to base editors, prime editors, or RNA editors, but they are significant for understanding the role of computational predictions in safety evaluation.
Result 1
Whether an off-target site can be "found" is the first bottleneck
To fairly compare different tools, the research team first combined candidate sites from multiple genome-wide search tools to create a shared candidate library containing 324,375 unique sites. However, even after integrating tools such as Cas-OFFinder, CRISPRitz, FlashFry, GuideScan2, CCTop, and CRISPOR, 46 experimentally validated true off-target sites were still not found. The study found that approximately 84.8% of these were high-mismatch sites, and some sites contained DNA or RNA bulges.
Figure 2 | Score distribution of different off-target prediction tools for different types of sites. The figure compares the score distribution of true off-target sites (indel ≥ 0.1%), low-frequency/undetected sites (indel < 0.1%), and candidate sites not sequenced and validated in the dataset. A–J are traditional tools, K–N are machine learning tools.
This reveals an easily overlooked issue in computational predictions:A site must first enter the candidate set to be scored by subsequent models. If it is missed during the search phase, no matter how accurate the subsequent scoring model is, it cannot be identified as an off-target site.
Therefore, evaluating a software tool should not only look at its ranking performance but also consider how many mismatches it allows, whether it supports non-canonical PAMs, whether it considers bulges, and whether the candidate site space is sufficiently comprehensive. The study also clearly points out that metrics like precision, recall, and PR-AUC need to be interpreted in conjunction with the candidate site coverage. For safety evaluation, compared to generating many candidate sites that require subsequent validation, missing a true off-target site may pose a greater risk.
Result 2
Prediction scores can be used for ranking but cannot represent actual off-target frequency
Overall, the 14 tools could assign relatively higher scores to true off-target sites, indicating that these tools do possess a certain ability to prioritize candidate sites. However, the problem is that there is significant overlap in the score distribution between true off-target sites and non-off-target sites. That is:
High-scoring sites are not necessarily true off-targets;
Low-scoring sites are not necessarily completely safe;
A single score threshold makes it difficult to clearly distinguish true off-targets from non-off-targets.
Regarding the correlation between prediction scores and actual indel frequency, the relatively better-performing tools include CRISPert, CRISPRoff, MOFF, and CRISOT, with Spearman correlation coefficients of approximately 0.606, 0.587, 0.570, and 0.513, respectively. However, the coefficient of determination R² for all tools was generally low, meaning that prediction scores can only explain a limited portion of the variation in actual editing frequency.
Figure 3 | Correlation between prediction scores and actual indel frequency.
In other words, computational tools might determine that a site is "relatively more worthy of attention," but they struggle to answer:
Is the actual off-target frequency of this site 0.1% or 10%;
Is the editing frequency consistent across different cell types;
Does editing still occur at this site under the final process, dose, and delivery conditions.
The reason is that off-target activity is not only determined by the sequence similarity between the sgRNA and DNA but is also influenced by factors such as chromatin accessibility, local DNA structure, cell type, editor exposure time, and DNA repair pathways. Current models still primarily rely on sequence features and cannot fully describe these biological variables.
Result 3
Machine learning models have improved, but still have not reached the level of "independently replacing experiments"
The authors used PR-AUC to evaluate the overall performance of different tools on imbalanced datasets. Among traditional continuous scoring tools, MOFF and CRISOT performed relatively well, with PR-AUCs of 0.31 and 0.30, respectively. Among machine learning models, CRISPert achieved a PR-AUC of 0.48, making it the best-performing tool in this group.
Figure 4 | Precision-Recall (PR) performance of different off-target prediction tools.
However, it should be noted that machine learning models like CRISPert were evaluated on a reduced dataset excluding bulge sites, so one cannot simply conclude that "machine learning tools are necessarily superior to traditional tools." Even the best-performing model had only a moderate PR-AUC, and stable, clear differentiation between true off-targets and non-off-targets remains unattainable.
Machine learning models also showed a more pronounced tendency towards polarized scoring, where many sites were assigned scores close to 0 or 1.This phenomenon might improve ranking efficiency, but it could also mean that the model gives "overconfident" judgments when training data is insufficient or when actual samples deviate from the training distribution. Especially in projects involving primary cells, patient-derived cells, different delivery methods, or novel editors, the applicability of the model is still limited by the scope of the training data.
Result 4
Increasing the number of validated sites can improve recall, but still cannot achieve complete coverage
In preclinical evaluation, a very practical question is: How many predicted sites actually need to be validated? The research results show that for the better-performing traditional tools:
When validating the top 250 sites, the recall rate was approximately 72%;
When validating the top 500 sites, the recall rate was approximately 77%;
When expanding to the top 1,250 sites, the recall rate was approximately 83%.
This means that even if 500 computationally predicted sites are validated per gRNA, approximately one-quarter of true off-targets may still be missed. Even expanding the validation scope to 1,250 sites, about 17% of true off-targets remain uncovered.
Figure 5 | Coverage of experimentally validated off-target sites by different tools.
The trend was similar for machine learning tools. In the dataset excluding bulge sites, when validating the top 1,250 sites with CRISPert and CRISPR-IP, the recall rate was approximately 81%, meaning about one-fifth of true off-targets could still be missed.
As the number of validated sites increases, the recall rate continues to rise, but the marginal benefit gradually decreases. The study estimates that to cover approximately 70% of true off-target events, hundreds of candidate sites per gRNA often need to be experimentally validated. Therefore, "only selecting the top 10 or top 20 sites for sequencing" is cost-effective but does not represent a systematic off-target safety evaluation. The specific number of sites to validate needs to be determined comprehensively based on the product type, edited cells, expected clinical dose, functional annotation of candidate sites, and overall risk level.
Result 5
Combining multiple prediction software tools yields less improvement than expected
The research team further tested whether combining multiple tools could improve recall. For sites without bulges, using CRISPert alone achieved a recall rate of approximately 69% for the top 50 sites.Combining CRISPert with tools like MOFF, CRISOT, or CRISPR-OFFT only increased the recall rate to approximately 71%.
Figure 6 | Changes in the recall rate of true off-target sites as the number of top-ranked candidate sites included for validation increases.
Even when integrating up to 5 prediction tools, the improvement was still very limited; when the validation scope expanded to 1,250 sites, the multi-tool combination offered almost no additional benefit compared to the best single tool. This suggests that although different software uses different algorithms, their core information sources are still highly overlapping, mostly based on sequence mismatches, PAMs, and existing experimental training data. Simply increasing the number of software tools cannot compensate for all the missing biological information. Therefore, the reasonable value of combining multiple tools is to reduce the bias introduced by a single algorithm, not to treat it as a replacement for experimental detection.
Figure 7 | Impact of combining different prediction tools on the recall rate of true off-target sites.
What implications does this study have for preclinical off-target evaluation?
The most important conclusion of this study is not to select a "best prediction software," but to redefine the position of computational predictions within the entire off-target evaluation system.
Computational predictions are suitable for candidate site screening, not final safety determination
Computational tools can quickly scan the whole genome and rank a large number of candidate sites, making them suitable for early sgRNA screening and subsequent validation site design. However, a low predicted risk does not mean the absence of off-target editing experimentally, and a high predicted risk does not guarantee that editing will occur.
Candidate site coverage is as important as the ranking algorithm
If the software cannot search for high-mismatch sites, bulges, or non-canonical PAMs, no scoring model can recover these sites. When evaluating a tool, both "how many sites are found" and "how accurate the ranking is" should be examined.
Experimental discovery and candidate site validation serve different purposes
Unbiased or semi-unbiased experimental methods are used to discover actual genomic sites where cleavage or editing occurs; computational predictions supplement the sequence-based candidate range and can consider population genetic variations. Subsequently, targeted deep sequencing is needed for validation and quantification under product-relevant cell and process conditions.
Should not focus only on small indels
CRISPR-Cas9 off-target cleavage can cause small insertions or deletions, as well as large deletions, chromosomal rearrangements, and insertion of exogenous DNA or endogenous sequences. Relying solely on conventional short-amplicon sequencing may not cover all genomic safety risks.
Different editors require different evaluation systems
This study targeted wild-type SpCas9. CBEs, ABEs, and PEs produce different molecular intermediates and types of unintended edits, so the sequence similarity prediction models for Cas9 nucleases cannot be directly applied. The authors also point out that future prediction tools need to be further adapted for base editors and prime editors.
Overall, a more reasonable preclinical off-target evaluation pathway should be:
Comprehensive computational candidate site prediction
Experimental genome-wide off-target discovery adapted to the editor type
Targeted deep validation of candidate sites
Evaluation of genomic structure and genetic stability
Risk analysis combining site function and product exposure
GeneRulor – One-stop Gene Editing Safety Evaluation Platform
GeneRulor
Zhuhai GeneRulor Medical Technology Co., Ltd. (GeneRulor) focuses on the safety evaluation of cell and gene therapy and small nucleic acid products. It can establish technical solutions covering initial design, off-target discovery, validation, and genetic stability evaluation based on the editor type, delivery method, cell source, and project stage.
[Service Module 1] sgRNA Design and Computational Prediction
GeneRulor can provide:
sgRNA design;
sgRNA sequence accuracy verification;
Off-target prediction using CRISPRme software;
Gene editing efficiency detection.
This module is suitable for early screening of candidate sgRNAs, preliminary nomination of potential off-target sites, and designing the scope of subsequent experimental validation. Computational prediction results should be interpreted in conjunction with the actual editing system and should not be used alone as a conclusion for off-target safety.
[Service Module 2] CRISPR Nuclease Off-Target Detection
For editing systems that produce DNA cleavage, such as Cas9, the following can be conducted based on project characteristics:
E-GUIDE in vivo off-target detection;
AID-seq in vitro off-target detection;
CRISPR editing WGS off-target detection;
Candidate off-target site validation and editing efficiency analysis.
Different methods reflect different biological information. In vitro methods typically have a higher capacity for discovering candidate sites, while in-cell methods can further reflect the impact of chromatin state, editor delivery, and cellular environment. Cross-analyzing computational predictions, in vitro discovery, and in-cell detection helps build a more complete set of candidate sites.
[Service Module 3] Specialized Evaluation for Base Editors and Prime Editors
Targeting the specific risks of different editors, Shutong Technology has established corresponding detection services:
Detect-seq: Genome-wide off-target detection for CBE base editors;
Ino-seq: Genome-wide off-target detection for ABE base editors;
PE-tag: Prime editor off-target detection;
In addition to potential sgRNA-dependent DNA off-target editing, CBEs and ABEs may also have deaminase-mediated sgRNA-independent unintended DNA or RNA editing; PEs require attention to error editing, bystander editing, and unintended products. Therefore, different editors should be paired with evaluation methods that match their mechanisms of action.
[Service Module 4] Genomic Stability and Extended Safety Evaluation
Off-target sites are not the entirety of gene editing product safety evaluation. Shutong Technology can also conduct the following around edited cells and products:
Chromosomal structural abnormality detection;
Vector integration site detection;
iPSC WES genetic stability detection;
iPSC WGS genetic stability and off-target detection;
Epigenetic detection;
Single-cell sequencing, etc.
Additionally, GeneRulor providesASO/siRNA off-target safety evaluation and genetically modified microorganism safety evaluation, designing targeted safety evaluation strategies based on different product mechanisms.
This systematic evaluation once again demonstrates that the core value of CRISPR off-target prediction tools is to nominate and rank candidate sites, not to directly prove that a site is safe or dangerous.
For preclinical gene editing products, relying solely on a single prediction software, or only validating a small number of top-ranked sites, may underestimate the true off-target risk. A more robust approach is to integrate computational predictions, experimental genome-wide off-target discovery, deep validation of candidate sites, and genomic stability evaluation, based on the editor mechanism and product attributes.
As gene editing technology expands from traditional Cas9 nucleases to CBEs, ABEs, and PEs, off-target safety evaluation also needs to shift from a "unified method" to a comprehensive strategy that is "editor-specific, product-specific, and risk-oriented."
