  False-positive results and bias may be common features of the biomedical
literature today, including risk factor-chronic disease research. A study was
undertaken to assess the reliability of base studies used in a meta-analysis
examining whether carbon monoxide, particulate matter 10 and 2.5 micro molar,
sulfur dioxide, nitrogen dioxide and ozone are risk factors for asthma
exacerbation (hospital admission and emergency room visits for asthma attack).
The number of statistical tests and models were counted in 17 randomly selected
base papers from 87 used in the meta-analysis. P-value plots for each air
component were constructed to evaluate the effect heterogeneity of p-values
used from all 87 base papers The number of statistical tests possible in the 17
selected base papers was large, median=15,360 (interquartile range=1,536 to
40,960), in comparison to results presented. Each p-value plot showed a
two-component mixture with small p-values less than .001 while other p-values
appeared random (p-values greater than .05). Given potentially large numbers of
statistical tests conducted in the 17 selected base papers, p-hacking cannot be
ruled out as explanations for small p-values. Our interpretation of the
meta-analysis is that the random p-values indicating null associations are more
plausible and that the meta-analysis will not likely replicate in the absence
of bias. We conclude the meta-analysis and base papers used are unreliable and
do not offer evidence of value to inform public health practitioners about air
quality as a risk factor for asthma exacerbation. The following areas are
crucial for enabling improvements in risk factor chronic disease observational
studies at the funding agency and journal level: preregistration, changes in
funding agency and journal editor (and reviewer) practices, open sharing of
data and facilitation of reproducibility research.
