structural:summarization
Selection bias and meta-analysis

Selective reporting has been offered by critics as an explanation for the positive results reported by parapsychologists. Selective reporting is sometimes called a "file drawer" problem, which arises when only positive study results are made public, while studies with negative or null results are not made public. Selective reporting has a compounded effect on meta-analysis, which is a statistical technique that aggregates the results of many studies to generate sufficient statistical power to demonstrate a result that the individual studies themselves could not demonstrate at a statistically significant level. For example, a recent meta-analysis combined 380 studies on psychokinesis, including data from the PEAR lab. It concluded that, although there is a statistically significant overall effect, it is inconsistent, and relatively few negative studies would cancel it out. Consequently, biased publication of positive results could be the cause.

Numerous researchers have criticized the popularity of meta-analysis in parapsychology, and is often seen as troublesome even within parapsychology. Critics have said that parapsychologists misuse meta-analysis to create the incorrect impression that statistically significant results have been obtained that indicate the existence of psi phenomena. Physicist Robert Park states that parapsychology's reported positive results are problematic because most such findings are invariably at the margin of statistical significance and that might be explained by a number of confounding effects; Park states that such marginal results are a typical symptom of pathological science as described by Irving Langmuir.

Researcher J. E. Kennedy has said that concerns over meta-analysis in science and medicine also apply to problems present in parapsychological meta-analysis. As a post-hoc analysis, critics emphasize the opportunity the method presents to produce biased outcomes via selecting cases chosen for study, methods employed, and other key criteria. Critics say that analogous problems with meta-analysis have been documented in medicine, where it has been shown different investigators performing meta-analyses of the same set of studies have reached contradictory conclusions.