Press Releases in 2010:
Independent user study ranks eHiTS at the top
TORONTO, ON - Sep 3rd 2010: SimBioSys, Inc. is pleased to inform the scientific community about the following recent paper in Wiley's Journal of Computational Chemistry:
"Can we trust docking results? Evaluation of seven commonly used programs on PDBbind database"
by Plewczynski et.al, from the University of Warsaw (http://onlinelibrary.wiley.com/doi/10.1002/jcc.21643/abstract).
They performed a comparison of 7 docking and scoring programs to
evaluate pose prediction and score accuracy on a large set of 1300 PDB
complexes. They performed a fairly thorough study asking some important
questions, such as how the starting ligand conformations influence the
results and how the results differ for small or large ligands, mostly
hydrophobic or mostly polar interaction. The good news they report is
that, statistically, overall results do not seem to be influenced by
the starting conformations, although there is a slight advantage in
some programs for the X-ray conformation, which is understandable. The
bad news is that ligand size does matter: while we are very successful
with small, fairly rigid molecules, large floppy ones still prove to be
hard to handle for all programs. The really ugly news is that none of
the scoring functions provided adequate correlation with binding energy.
The results are divided into 3 major sections: pose prediction
accuracy, score correlation with experimental binding energy and
score-rmsd correlation (ranking performance of the scoring functions).
The authors' conclusion of the pose prediction exercise can be
summarized by the following quote:
"On the basis of those results, we can order programs in the following way: GOLD ~ eHiTS > Surflex > Glide > LigandFit > FlexX > AutoDock. The
best programs have the average RMSD top score around 2.7 A, and it
increases to nearly 4.5 A for the weakest FlexX. As expected, better
results were observed for best pose conformations. For those
poses, the mean RMSD value was even below 2 A for GOLD, eHiTS, and
Surflex. ... Moreover, the percentage of pairs for which top score
conformation is below 2 A shows that even for the best programs the
success rate is below 60%, and in some cases even below 40%."
Based on the score-energy correlation performance, the authors divided
the programs into three categories. The best one is "composed of
functions implemented in eHiTS and in Surflex, which gave Pearson
correlation 0.38 and 0.33, respectively. Moreover, for eHiTS scoring
function very high-Spearman correlation was obtained..." The Pearson
correlations for the middle and worse categories are in the range
of 0.17-0.25 and less than 0.1 respectively. The authors rightly
conclude that the score-energy correlation results are inadequate even
Finally, in the ranking performance comparison (correlation of score with
quality of poses) AutoDock achieved the highest 0.32 correlation with eHiTS as close second with ~0.3 correlation.
So, what is the final conclusion of the authors with regards to
answering the question in the title ? Here is the quote with the answer:
"Thus, can we trust docking programs? The answer must be given individually for two aspects of docking programs. In terms of pose prediction, we can say that GOLD and eHiTS performance is accurate enough
... In the case of scoring functions, the answer must be negative, as
virtually no correlations could be observed between docking score and
in vitro binding affinities ... the empirically derived functions have
now reached the saturation of year-to-year improvement ... The future
direction should be either to use statistical approach based on
increasing number of X-ray protein-ligand complexes, as can be
determined from the results obtained by eHiTS scoring functions, or to
develop completely new approaches in terms of predicting in vivo
activity of the ligand."
Zsolt Zsoldos SimBioSys' chief scientist and main eHiTS developer says: "I am very happy to see that eHiTS came out at the top, i.e. among the
best-2 contenders for all 3 aspects of the comparison while the
other-best were three different programs for the 3 aspects. On the
other hand, I agree with the authors that there is still a lot of room
and need for significant improvements both in terms of pose prediction
(~60% success rate) and score accuracy (~0.4 correlation). Furthermore,
we definitely need such thorough and large-scale performance
comparisons as this one in the future to continuously assess the state
of the art until some programs (hopefully eHiTS remaining on the lead)
will reach adequate performance."
eHiTS is a flexible ligand docking and virtual screening software
developed by SimBioSys, Inc. By utilizing an exhaustive search
algorithm, that rapidly enumerates mappings of interacting atoms
between receptor and ligand the software employs new approaches to the
classical problem of flexible ligand docking. The eHiTS software
provides energy optimized 3D coordinates of docked poses and
conformations of ligand molecules in the active site of the receptor.
The binding energy of each pose is calculated and reported as a score.
At the end of each run the ligand that produced the lowest binding
energy (i.e. best score) is reported. The software is not only fast,
but is also highly scalable, it's able to efficiently run from single
desk-top PC's to multiCPU clusters, super-computers and computer
About SimBioSys: Privately
owned, SimBioSys is a recognized leader in the field of cutting edge
drug discovery software. Providing a wide range of software
solutions, the company is focused on the development of scientific
tools to facilitate the drug discovery process. It retains a constant
focus on the innovation of algorithms to provide improved throughput
and accuracy in the fields of flexible docking, virtual screening and
de-novo structure design. SimBioSys is also a pioneer in the field of
computer-aided retrosynthetic analysis where it supports chemists
through the challenges of organic synthesis. With attention to detail,
ease-of-use and improved productivity, SimBioSys has built a strong
reputation of delivering state-of-the-art scientific solutions to
biotechnology and pharmaceutical companies.