  Consider the problem of testing $s$ hypotheses simultaneously. The usual
approach restricts attention to procedures that control the probability of even
one false rejection, the familywise error rate (FWER). If $s$ is large, one
might be willing to tolerate more than one false rejection, thereby increasing
the ability of the procedure to correctly reject false null hypotheses. One
possibility is to replace control of the FWER by control of the probability of
$k$ or more false rejections, which is called the $k$-FWER. We derive both
single-step and step-down procedures that control the $k$-FWER in finite
samples or asymptotically, depending on the situation. We also consider the
false discovery proportion (FDP) defined as the number of false rejections
divided by the total number of rejections (and defined to be 0 if there are no
rejections). The false discovery rate proposed by Benjamini and Hochberg [J.
Roy. Statist. Soc. Ser. B 57 (1995) 289--300] controls $E(FDP)$. Here, the goal
is to construct methods which satisfy, for a given $\gamma$ and $\alpha$,
$P\{FDP>\gamma\}\le \alpha$, at least asymptotically. In contrast to the
proposals of Lehmann and Romano [Ann. Statist. 33 (2005) 1138--1154], we
construct methods that implicitly take into account the dependence structure of
the individual test statistics in order to further increase the ability to
detect false null hypotheses. This feature is also shared by related work of
van der Laan, Dudoit and Pollard [Stat. Appl. Genet. Mol. Biol. 3 (2004)
article 15], but our methodology is quite different. Like the work of Pollard
and van der Laan [Proc. 2003 International Multi-Conference in Computer Science
and Engineering, METMBS'03 Conference (2003) 3--9] and Dudoit, van der Laan and
Pollard [Stat. Appl. Genet. Mol. Biol. 3 (2004) article 13], we employ
resampling methods to achieve our goals. Some simulations compare finite sample
performance to currently available methods.
