Performance of a popular docking code

This post is a quick note on a performance of a commercial docking code, as measured across the entire DUDE dataset. For around a 100 protein targets, the code is supposed to rank active compounds higher than decoys compounds – separate poppy seeds from sand if you like.

Let me start by saying that I'm pretty impressed with what I've seen. Starting this side-project, I assumed that any docking code can be expected to have an AUC of around 0.6-0.7 measured on a standard benchmarking set (such as DUDE). I think that's largely true of free codes such as Autodock vina or rdock. But here we're looking at a piece of code from a major commercial vendor and it performs beyond my expectation.

I'm not disclosing the name of the code, since there may have been something in the academic license that prevents such benchmarking studies from being published (a "gag order" effectively, in case somebody is measuring the code "incorrectly").


AUC is estimated for each target by using only 10% of the decoys available in the DUDE dataset and all of the actives. This increases the error bar per-target but I'm more interested in AUC performance across many targets than any particular one.

The error bars on the boxplots are estimated from 100 randomly selected subslices, half the size of the ranked list of actives and decoys. On the right, boxplots of median target AUCs are plotted for the entire DUDE dataset, the older DUD39 subslice and the non-DUD39 targets. The aim of this last plot is to check if on older, more familiar dataset, DUD39, outperforms something more modern (which doesn't appear to be the case here).

Comments

Post a Comment

Popular posts from this blog

React.js – edit and delete comments in CommentBox

It appears customary for high-intelligence, high-skills groups of engineers to "roll their own" whenever they have a chance to. Counter-intuitively to most managerial/MBA types out there, re-inventing the wheel can give you... a better wheel. Success is not guaranteed though – but these people may leave sooner rather than later if you don't let them... Having looked at number of JS frameworks (ember, backbone, angular) there was no clear winner. They were all a mess, at least to me. So I approached React.js with no hopes – how much better could it possible be? It turns out, a lot! This post will be an extension of the official React.js tutorial . The tutorial stops at the stage where you can add comments to a dynamic lists, that will grow as you add comments. This falls short of the functionality displayed on facebook: once you add a comment you can edit or delete it, so that's what're going to do. Full disclosure, I only started using React this weekendo so m...
Read more

Example slurm cluster on your laptop (multiple VMs via vagrant)

To anybody who's considering a switch to modern queuing systems, like slurm, this will be an useful guide. Rather than doing a roll out on production nodes, we'll try things out in vagrant. This text assumes you have vagrant installed on your local machine (it's very easy). The ultimate aim will be to setup 2 servers (worker nodes) and one master/controller, and run a simple job on them. There are multiple great guides out there: most of them are out of date, and won't work without some tweaking on Ubuntu 16.04 Xenial. The entirety of the code is hosted at https://github.com/jandom/gromacs-slurm-openmpi-vagrant The README.md file will show you how to set it up, with checks on every step. References https://mussolblog.wordpress.com/2013/07/17/setting-up-a-testing-slurm-cluster/ https://github.com/gabrieleiannetti/slurm_cluster_wiki/wiki/Installing-a-Slurm-Cluster http://philipwfowler.me/2016/04/14/how-to-setup-a-gramble/ https://github.com/dakl/slurm-clus...
Read more

Wrapping openbabel in python - using cython

The previous post showed a contrived example of how one could access the openbabel functionality in python using the boost libraries. There is alternative to boost to wrap around c/c++ code, it's called cython. Here is an example openbabel wrapper, exposing some very simple functionality. Just before diving in, you should know that there already exists a wrapper around openbabel in python (it is called pybel). What I'm showing here doesn't even come close to pybel in terms of usefulness, the idea here is to show a demo of how easy wrapping things in python is. #! /usr/bin/python from ez_setup import use_setuptools use_setuptools() from setuptools import setup, Extension, find_packages from Cython.Distutils import build_ext import sys, os import glob import subprocess as sub def get_include(name="openbabel"): p = sub.Popen('locate %s' % name ,stdout=sub.PIPE,stderr=sub.PIPE, shell=True) output, errors = p.communicate() ...
Read more