Tuesday, July 28, 2015

New package version with bug fix for evol.vcv

I just posted a new version of phytools to my webpage - though not to CRAN. This version contains only the minor update over the previous version of containing a bug fix for evol.vcv.

It is pretty easy to install this version from source, however - and this can be done even if dependency packages are not yet installed.

The simplest way to do this (in my opinion) is to first install the most recent CRAN version, which will automatically install all the packages upon which phytools depends. Then you can go ahead and install from source using the URL of the latest package build I have posted.

install.packages("phytools",repos="http://cran.us.r-project.org")
## Installing package into 'C:/Users/Liam/Documents/R/win-library/3.2'
## (as 'lib' is unspecified)
## also installing the dependency 'ape'
## package 'ape' successfully unpacked and MD5 sums checked
## package 'phytools' successfully unpacked and MD5 sums checked
## 
## The downloaded binary packages are in
##  C:\Users\Liam\AppData\Local\Temp\RtmpgZn2Oa\downloaded_packages
## or set a mirror repository near you
packageVersion("phytools")
## [1] '0.4.60'
install.packages("http://www.phytools.org/nonstatic/phytools_0.4-62.tar.gz")
## Installing package into 'C:/Users/Liam/Documents/R/win-library/3.2'
## (as 'lib' is unspecified)
## inferring 'repos = NULL' from 'pkgs'
packageVersion("phytools")
## [1] '0.4.62'
library(phytools)
## Loading required package: ape
## Loading required package: maps

That's it.

Important bug fix in evol.vcv

A phytools user by the name of Jurriaan de Vos recently identified an important bug in the phytools function evol.vcv. evol.vcv implements the method of Revell & Collar (2009) for fitting two or more among-trait evolutionary covariance matrices for the Brownian process to different parts of the tree. The problem stems from a bug introduced in which I inadvertently assumed that the order of the input data matrix matched the order of the tip labels in tree.

Here is an example of the problem.

First, I'll simulate data with different correlation structure in different parts of the tree:

library(phytools)
tree<-pbtree(n=26,tip.label=LETTERS,scale=2)
Q<-matrix(c(-1,1,1,-1),2,2)
rownames(Q)<-colnames(Q)<-letters[1:2]
tree<-sim.history(tree,Q,anc="a")
## Done simulation(s).
plotSimmap(tree,colors=setNames(c("blue","red"),letters[1:2]),ylim=c(-1,27))
add.simmap.legend(colors=setNames(c("blue","red"),letters[1:2]),prompt=FALSE,
    x=0.1*max(nodeHeights(tree)),y=0,vertical=FALSE)

plot of chunk unnamed-chunk-1

vcv<-list(matrix(c(1,0,0,1),2,2),
    matrix(c(2,1.8,1.8,2),2,2))
names(vcv)<-letters[1:2]
vcv
## $a
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1
## 
## $b
##      [,1] [,2]
## [1,]  2.0  1.8
## [2,]  1.8  2.0
X<-sim.corrs(tree,vcv=vcv)
phylomorphospace(tree,X,colors=setNames(c("blue","red"),letters[1:2]),
    xlab="trait 1",ylab="trait 2")
add.simmap.legend(colors=setNames(c("blue","red"),letters[1:2]),prompt=FALSE,
    x=par()$usr[1]+0.02*(par()$usr[2]-par()$usr[1]),
    y=par()$usr[4]-0.04*(par()$usr[4]-par()$usr[3]))

plot of chunk unnamed-chunk-2

OK, now let's fit the models:

fit1<-evol.vcv(tree,X)
fit1
## ML single-matrix model:
##  R[1,1]  R[1,2]  R[2,2]  k   log(L)
## fitted   1.1928  0.7186  0.9892  5   -57.9027    
## 
## ML multi-matrix model:
##  R[1,1]  R[1,2]  R[2,2]  k   log(L)
## a    0.3866  -0.1434 0.4604  8   -53.3203    
## b    1.9253  1.467   1.4903  
## 
## P-value (based on X^2): 0.0272 
## 
## R thinks it has found the ML solution.
## rows scrambled:
fit2<-evol.vcv(tree,X[sample(rownames(X)),])
fit2
## ML single-matrix model:
##  R[1,1]  R[1,2]  R[2,2]  k   log(L)
## fitted   1.1928  0.7186  0.9892  5   -131.4336   
## 
## ML multi-matrix model:
##  R[1,1]  R[1,2]  R[2,2]  k   log(L)
## a    0.9387  0.6829  1.4885  8   -86.0687    
## b    5.4343  2.7888  4.7515  
## 
## P-value (based on X^2): 0 
## 
## R thinks it has found the ML solution.

There are obviously multiple weird things going on here. Firstly, the single-matrix model is fit correctly, though the log-likelihood is obviously totally wrong. Secondly, the two-matrix models are completely messed up….

Interestingly, the bug does not affect evolvcv.lite which fits the same model - as well as other ones - but only for datasets with just two continuous traits. (Unfortunately, there is no fancy print method for evolvcv.lite).

fit3<-evolvcv.lite(tree,X[sample(rownames(X)),])
fit3$model1
## $description
## [1] "common rates, common correlation"
## 
## $R
##           [,1]      [,2]
## [1,] 1.1927796 0.7185548
## [2,] 0.7185548 0.9891478
## 
## $logLik
## [1] -57.90274
## 
## $convergence
## [1] 0
## 
## $k
## [1] 5
## 
## $AIC
## [1] 125.8055
fit3$model4
## $description
## [1] "no common structure"
## 
## $R
## $R$a
##            [,1]       [,2]
## [1,]  0.3862680 -0.1435202
## [2,] -0.1435202  0.4596485
## 
## $R$b
##          [,1]     [,2]
## [1,] 1.924823 1.466919
## [2,] 1.466919 1.490986
## 
## 
## $logLik
## [1] -53.32028
## 
## $convergence
## [1] 0
## 
## $k
## [1] 8
## 
## $AIC
## [1] 122.6406

The fix - also pointed out by Jurriaan is extremely simple & merely involved moving one line of code (in which X) is vectorized two lines below (to be below where the rows of X were already sorted)! I have posted a new version of evol.vcv with this fix implemented here and will obviously be in the next update to phytools.

Here's a demo showing that it is fixed:

source("http://www.phytools.org/evol.vcv/v0.7/evol.vcv.R")
library(numDeriv)
fit4<-evol.vcv(tree,X[sample(rownames(X)),])
fit4
## ML single-matrix model:
##  R[1,1]  R[1,2]  R[2,2]  k   log(L)
## fitted   1.1928  0.7186  0.9892  5   -57.9027    
## 
## ML multi-matrix model:
##  R[1,1]  R[1,2]  R[2,2]  k   log(L)
## a    0.3866  -0.1434 0.4604  8   -53.3203    
## b    1.9253  1.467   1.4903  
## 
## P-value (based on X^2): 0.0272 
## 
## R thinks it has found the ML solution.

That's it.

Sunday, July 26, 2015

New version of phytools with co-phylogenetic plotting function, ltt methods, and bug-fix for plotSimmap (plus, how to install a source package directly from a URL)

I just posted a new version of phytools (phytools_0.4-61). This version has a few updates over the previous phytools version posted to CRAN - most especially, the addition of the function cophylo for plotting 'co-phylogenetic' trees - that is, trees in which associated tips can be rotated to maximize vertical alignment and then linked. So, for instance, from a recent demo of the function:

library(phytools)
## Loading required package: ape
## Loading required package: maps
packageVersion("phytools")
## [1] '0.4.61'
tr1<-rtree(n=40)
tr2<-rtree(n=26,tip.label=LETTERS)
assoc<-cbind(sample(tr1$tip.label,20),sample(LETTERS,20))
assoc
##       [,1]  [,2]
##  [1,] "t14" "Y" 
##  [2,] "t19" "I" 
##  [3,] "t26" "B" 
##  [4,] "t17" "D" 
##  [5,] "t39" "J" 
##  [6,] "t29" "P" 
##  [7,] "t13" "W" 
##  [8,] "t37" "V" 
##  [9,] "t36" "R" 
## [10,] "t10" "X" 
## [11,] "t1"  "Q" 
## [12,] "t28" "S" 
## [13,] "t3"  "M" 
## [14,] "t23" "K" 
## [15,] "t21" "C" 
## [16,] "t11" "H" 
## [17,] "t33" "U" 
## [18,] "t31" "E" 
## [19,] "t34" "O" 
## [20,] "t25" "G"
obj<-cophylo(tr1,tr2,assoc=assoc,rotate=TRUE)
## Rotating nodes to optimize matching...
## Done.
obj
## Object of class "cophylo" containing:
## 
## (1) 2 (possibly rotated) phylogenetic trees in an object of class "multiPhylo".
## 
## (2) A table of associations between the tips of both trees.
plot(obj)

plot of chunk unnamed-chunk-1

The function also fixes a 'bug' (or arguably, 'feature') of the function plot.ltt which caused plot(obj,log.lineages=FALSE,log="y",show.tree=TRUE) to space the tips of the plotted tree on a log-scale. This is no longer an issue:

tree<-pbtree(n=40,scale=10)
obj<-ltt(tree,plot=FALSE)
plot(obj,log.lineages=FALSE,log="y",show.tree=TRUE)

plot of chunk unnamed-chunk-2

Finally, the function fixes a weird bug in plotSimmap for objects of class "multiPhylo" which was basically due to not all the arguments being correctly passed to internal calls of plotSimmap for all loops corresponding to each tree in the list. This meant that a call of plotSimmap on an object of class "multiPhylo" basically just didn't work.

The updated package can be downloaded from the phytools page and installed from source - or you can just copy & execute the following lines:

detach("package:phytools",unload=TRUE) ## if phytools is loaded
install.packages("http://www.phytools.org/nonstatic/phytools_0.4-61.tar.gz",
    type="source",repos=NULL)
## Installing package into 'C:/Users/Liam/Documents/R/win-library/3.2'
## (as 'lib' is unspecified)

Saturday, July 25, 2015

Integrating stochastic character maps across multiple character transition models

I recently fielded an interesting question by Oscar Inostroza from the Universidad de Concepción how to choose among alternative models for the substitution process for stochastic mapping when different models have comparable support. Presently, it is possible to condition on a particular model (such as, for example, model="ER", the “equal rates” model), then either fix the transition matrix Q at it's empirical ML value; or sample Q from its posterior distribution conditioned on the selected model. It is not possible, however, to sample character histories conditioned on multiple alternative models for character change between states.

Ideally what we might like to do in this case is design a reversible-jump MCMC to sample among models for character transition in proportion to their posterior probabilities; however unfortunately this is not practical at the current time. What I suggested instead is that we use Akaike weights to generate stochastic maps under all of the alternative models under consideration, in proportion to (or, approximately in proportion to - since we must use a finite number of simulations) the weight of evidence in support of that model.

There are a few tricks to doing this. In the following, I'll illustrate how it can be done using phytools.

First, some preliminaries. Let's start by loading phytools & simulating some data for the present case. I will simulate data under a "SYM" (symmetric) transition model, but in which the rates are not too dissimilar between states - which I hope will create fairly even support for the "SYM" & "ER" models on a modest sized phylogeny.

## load packages
library(phytools)

## simulate some data
tree<-pbtree(n=40,scale=1)
Q<-matrix(c(-1,0.75,0.25,
        0.75,-1.25,0.5,
        0.25,0.5,-0.75),3,3)
rownames(Q)<-colnames(Q)<-letters[1:3]
Q
##       a     b     c
## a -1.00  0.75  0.25
## b  0.75 -1.25  0.50
## c  0.25  0.50 -0.75
x<-sim.history(tree,Q)$states
## Done simulation(s).
x
## t24 t33 t34 t10  t9 t25 t26 t21 t22  t1 t39 t40 t20 t28 t29 t31 t32 t19 
## "c" "c" "c" "c" "c" "a" "a" "b" "c" "b" "b" "b" "b" "c" "b" "b" "b" "b" 
## t13 t23 t30 t35 t36 t27 t14 t15  t3  t6  t7  t2 t17 t18  t8  t4  t5 t37 
## "b" "a" "a" "a" "a" "a" "a" "a" "a" "c" "a" "a" "a" "a" "a" "a" "a" "c" 
## t38 t16 t11 t12 
## "c" "b" "a" "a"

Next, here are some simple functions that we'll use later to compute the AIC scores and the Akaike weights for each model:

aic<-function(logL,k) 2*k-2*logL
aic.w<-function(aic){
    d.aic<-aic-min(aic)
    exp(-1/2*d.aic)/sum(exp(-1/2*d.aic))
}

Now, we need to fit & (more importantly) obtain the log-likelihood of each fitted model. We could actually do this with multiple functions in R (ace, fitDiscrete, & optim.pml, among others); however here I will use make.simmap, which actually computes the likelihood using code modified from ace.

## compute log likelihoods
logL<-sapply(c("ER","SYM","ARD"),
    function(model,tree,x) make.simmap(tree,x,model)$logL,
    tree=tree,x=x)
## make.simmap is sampling character histories conditioned on the transition matrix
## Q =
##            a          b          c
## a -0.7389255  0.3694628  0.3694628
## b  0.3694628 -0.7389255  0.3694628
## c  0.3694628  0.3694628 -0.7389255
## (estimated using likelihood);
## and (mean) root node prior probabilities
## pi =
##         a         b         c 
## 0.3333333 0.3333333 0.3333333
## Done.
## make.simmap is sampling character histories conditioned on the transition matrix
## Q =
##            a         b          c
## a -0.5566061  0.000000  0.5566061
## b  0.0000000 -1.158184  1.1581840
## c  0.5566061  1.158184 -1.7147901
## (estimated using likelihood);
## and (mean) root node prior probabilities
## pi =
##         a         b         c 
## 0.3333333 0.3333333 0.3333333
## Done.
## make.simmap is sampling character histories conditioned on the transition matrix
## Q =
##            a           b          c
## a -0.5197351  0.03634396  0.4833911
## b  0.0000000 -0.65052948  0.6505295
## c  1.4205225  0.95975112 -2.3802736
## (estimated using likelihood);
## and (mean) root node prior probabilities
## pi =
##         a         b         c 
## 0.3333333 0.3333333 0.3333333
## Done.
logL
##        ER       SYM       ARD 
## -28.55567 -26.84571 -26.38334
## now let's compute AIC values & weights
## (in a real study, we might use AICc)
AIC<-mapply(aic,logL,c(1,3,6))
AIC
##       ER      SYM      ARD 
## 59.11134 59.69141 64.76668
AIC.W<-aic.w(AIC)
AIC.W
##         ER        SYM        ARD 
## 0.55328515 0.41398768 0.03272717

In our next step, we can “normalize” our weights to the number of simulations we want to use for stochastic mapping. The trick here is to remember that if we just round the product of the weights (which are currently normalized to sum to one) × the desired total number of simulations, we might end up with a total number of simulations less than or greater than our desired number. In the code below I (arbitrarily) add (or subtract) the deficit (or surplus) to models chosen at random; but this need not be our strategy.

## now let's normalize this to our number of stochastic
## mapping simulations
nsim<-1000
Nsim<-round(nsim*AIC.W)
d<-if(sum(Nsim)>nsim) -1 else 1
nsim<-Nsim+d*sample(c(rep(1,abs(nsim-sum(Nsim))),
    rep(0,length(Nsim)-abs(nsim-sum(Nsim)))))
nsim
##  ER SYM ARD 
## 553 414  33

Finally, I will perform nsim stochastic mapping simulations for each model & then combine stochastic maps conducted across all of our models into one object of class "multiPhylo":

## remove any with nsim==0
nsim<-nsim[nsim!=0]
trees<-list()
class(trees)<-"multiPhylo"
for(i in 1:length(nsim)){
    obj<-make.simmap(tree,x,model=names(nsim)[i],nsim=nsim[i])
    ## we could also use:
    # obj<-make.simmap(tree,x,model=names(nsim)[i],nsim=nsim[i],Q="mcmc")
    ## but it would be slower
    if(nsim[i]==1){ 
        obj<-list(obj)
        class(obj)<-"multiPhylo"
    }
    trees<-c(trees,obj)
}
## make.simmap is sampling character histories conditioned on the transition matrix
## Q =
##            a          b          c
## a -0.7389255  0.3694628  0.3694628
## b  0.3694628 -0.7389255  0.3694628
## c  0.3694628  0.3694628 -0.7389255
## (estimated using likelihood);
## and (mean) root node prior probabilities
## pi =
##         a         b         c 
## 0.3333333 0.3333333 0.3333333
## Done.
## make.simmap is sampling character histories conditioned on the transition matrix
## Q =
##            a         b          c
## a -0.5566061  0.000000  0.5566061
## b  0.0000000 -1.158184  1.1581840
## c  0.5566061  1.158184 -1.7147901
## (estimated using likelihood);
## and (mean) root node prior probabilities
## pi =
##         a         b         c 
## 0.3333333 0.3333333 0.3333333
## Done.
## make.simmap is sampling character histories conditioned on the transition matrix
## Q =
##            a           b          c
## a -0.5197351  0.03634396  0.4833911
## b  0.0000000 -0.65052948  0.6505295
## c  1.4205225  0.95975112 -2.3802736
## (estimated using likelihood);
## and (mean) root node prior probabilities
## pi =
##         a         b         c 
## 0.3333333 0.3333333 0.3333333
## Done.

With this object, it is possible to do any of the other standard type of things, such as use describe.simmap to summarize the results of our analysis, including posterior probabilities at nodes, etc. For instance:

obj<-describe.simmap(trees)
obj
## 1000 trees with a mapped discrete character with states:
##  a, b, c 
## 
## trees have 11.26 changes between states on average
## 
## changes are of the following types:
##        a,b   a,c   b,a   b,c   c,a   c,b
## x->y 1.238 3.593 0.254 1.816 1.667 2.692
## 
## mean total time spent in each state is:
##              a         b         c    total
## raw  8.5813823 2.6669467 2.6141229 13.86245
## prop 0.6190378 0.1923864 0.1885758  1.00000
plot(obj)

plot of chunk unnamed-chunk-6

and so on.

That's all for now on this topic.