  To identify novel dynamic patterns of gene expression, we develop a
statistical method to cluster noisy measurements of gene expression collected
from multiple replicates at multiple time points, with an unknown number of
clusters. We propose a random-effects mixture model coupled with a
Dirichlet-process prior for clustering. The mixture model formulation allows
for probabilistic cluster assignments. The random-effects formulation allows
for attributing the total variability in the data to the sources that are
consistent with the experimental design, particularly when the noise level is
high and the temporal dependence is not strong. The Dirichlet-process prior
induces a prior distribution on partitions and helps to estimate the number of
clusters (or mixture components) from the data. We further tackle two
challenges associated with Dirichlet-process prior-based methods. One is
efficient sampling. We develop a novel Metropolis-Hastings Markov Chain Monte
Carlo (MCMC) procedure to sample the partitions. The other is efficient use of
the MCMC samples in forming clusters. We propose a two-step procedure for
posterior inference, which involves resampling and relabeling, to estimate the
posterior allocation probability matrix. This matrix can be directly used in
cluster assignments, while describing the uncertainty in clustering. We
demonstrate the effectiveness of our model and sampling procedure through
simulated data. Applying our method to a real data set collected from
Drosophila adult muscle cells after five-minute Notch activation, we identify
14 clusters of different transcriptional responses among 163 differentially
expressed genes, which provides novel insights into underlying transcriptional
mechanisms in the Notch signaling pathway. The algorithm developed here is
implemented in the R package DIRECT, available on CRAN.
