text: Linear mixed models (LMMs) are widely used for heritability estimation in genome-wide association studies (GWAS). In standard approaches to heritability estimation with LMMs, a genetic relationship matrix (GRM) must be specified. In GWAS, the GRM is frequently a correlation matrix estimated from the study population's genotypes, which corresponds to a normalized Euclidean distance kernel. In this paper, we show that reliance on the Euclidean distance kernel contributes to several unresolved modeling inconsistencies in heritability estimation for GWAS. These inconsistencies can cause biased heritability estimates in the presence of linkage disequilibrium (LD), depending on the distribution of causal variants. We show that these biases can be resolved (at least at the modeling level) if one adopts a Mahalanobis distance-based GRM for LMM analysis. Additionally, we propose a new definition of partitioned heritability -- the heritability attributable to a subset of genes or single nucleotide polymorphisms (SNPs) -- using the Mahalanobis GRM, and show that it inherits many of the nice consistency properties identified in our original analysis. Partitioned heritability is a relatively new area for GWAS analysis, where inconsistency issues related to LD have previously been known to be especially pernicious.
  @cite_0: Single nucleotide polymorphism (SNP)-heritability estimation is an important topic in several research fields, including animal, plant and human genetics, as well as in ecology. Linear mixed model estimation of SNP-heritability uses the structures of genomic relationships between individuals, which is constructed from genome-wide sets of SNP-markers that are generally weighted equally in their contributions. Proposed methods to handle dependence between SNPs include, “thinning” the marker set by linkage disequilibrium (LD)-pruning, the use of haplotype-tagging of SNPs, and LD-weighting of the SNP-contributions. For improved estimation, we propose a new conceptual framework for genomic relationship matrix, in which Mahalanobis distance-based LD-correction is used in a linear mixed model estimation of SNP-heritability. The superiority of the presented method is illustrated and compared to mixed-model analyses using a VanRaden genomic relationship matrix, a matrix used by GCTA and a matrix employing LD-weighting (as implemented in the LDAK software) in simulated (using real human, rice and cattle genotypes) and real (maize, rice and mice) datasets. Despite of the computational difficulties, our results suggest that by using the proposed method one can improve the accuracy of SNP-heritability estimates in datasets with high LD.
summary: Recently, in independent work, @cite_0 proposed using the Mahalanobis kernel in a similar way for heritability esitmation with GWAS data. 's paper primarily focuses on empirical analysis, using both simulated and real datasets to illsutrate advantages of the Mahalanobis kernel. The present work contains more precise mathematical and statistical justification for much of the work in @cite_0 , and introduces statistical principles (e.g. @math -heritability in Section ) that can be extended to other targeted application areas and genetics (like partitioning heritability).