Model development

Web deployed model

In order to refine the GAF homologs down to the subgroup level, ideally we needed a model that could identity patterns in protein sequences which affect the subgroup identity. In searching for existing programs, we discovered over twenty programs dating back to 1996 that were designed for a similar purpose. However, many were not maintained over the years or open access. Of the 12 programs our data was tested set on, most failed to correctly separate the protein family based on subgroup. Since we were unable to find a program which fit our needs, we developed our own.

Historically, consensus sequences and pro-site maps were used to identify influential patterns in sequences. While these methods are effective for highly conserved sequences and regions, they require arbitrary data cut-offs which makes their methods a bit coarse. One method which does not require these cut-offs are profile Hidden Markov Models, or profile HMMs. Profile HMM’s are statistical models that give the probability of any amino acid or DNA base at any position in the sequence. For example, if there is a lysine at position 2 in the sequence alignment this informs not only the probability of lysine at position 2, but the probability of other similar amino acids at that position. Profile HMM’s have become widely used in identifying protein families, even Pfam the protein family database used HMM’s to curate their database (24).

Specifically, the model began with about 100 previously characterized GAF protein sequences provided by Clark Lagarias’s laboratory which we then divided into blue, yellow, and red subgroups. The blue subgroup represents any GAF sequence that absorbs in the red spectra, and thus reflects and appears blue. For the purposes of this model, GAF domains absorbs from 550nm and below are considered in the yellow subgroup, from 550-600nm are considered in the red subgroup, and from from 600nm and above are considered in the blue subgroup. For every subgroup we created multiple sequence alignments that were influenced by the protein structure using Promals3D. Using the protein structure in the multiple alignment allows relatively divergent clusters to be aligned with more accuracy because it takes into account sequence-based constraints derived from the 3D structures (20) . After creating the three multiple sequence alignments, a profile HMM was created using HMMER. The model then aligns the user given sequence and compares it to all three profile HMMs.By calculating the log2 odds ratio of the observed to expected E value, a quantifiable score is produced. This model now has a web interface and is downloadable on github here.

To assess the accuracy of our model a four-fold cross validation was performed. This meaning a random quarter of the GAF sequences were removed and only the remaining 3/4ths were used for the profile HMM creation. This ensures that the model is not populated with the sequences which will be used for assessing accuracy, and thus our accuracy measurement is not biased. The reduced model constructed from just 3/4ths of the original sequences would then be queried by the randomized quarter of previously removed sequences and calculated how accurately the model could blindly predict the sequences. We repeated this three other times until all random and no overlapping quarters were used as the query.Through this validation we calculated our model has a 75% accuracy which is 42% more accurate than would occur by random chance.

Future Goals

In the future, our team would like to refine this model to have more specificity. Instead of our model predicting three subgroups, we would like to alter it to predict the wavelength range the GAF sequence will absorb at. This level of specificity would be beneficial in distinguishing more nuanced differences such as orange from yellow and blue from green.