  We study online learning in repeated first-price auctions where a bidder,
only observing the winning bid at the end of each auction, learns to adaptively
bid in order to maximize her cumulative payoff. To achieve this goal, the
bidder faces a censored feedback: if she wins the bid, then she is not able to
observe the highest bid of the other bidders, which we assume is \textit{iid}
drawn from an unknown distribution. In this paper, we develop the first
learning algorithm that achieves a near-optimal $\widetilde{O}(\sqrt{T})$
regret bound, by exploiting two structural properties of first-price auctions,
i.e. the specific feedback structure and payoff function.
  The feedback in first-price auctions combines the graph feedback across
actions (bids), the cross learning across contexts (private values), and a
partial order over the contexts; we generalize it as the partially ordered
contextual bandits. We establish both strengths and weaknesses of this
framework, by showing a curious separation that a regret nearly independent of
the action/context sizes is possible under stochastic contexts, but is
impossible under adversarial contexts. In particular, this framework leads to
an $O(\sqrt{T}\log^{2.5}T)$ regret for first-price auctions when the bidder's
private values are \emph{iid}.
  Despite the limitation of the above framework, we further exploit the special
payoff function of first-price auctions to develop a sample-efficient algorithm
even in the presence of adversarially generated private values. We establish an
$O(\sqrt{T}\log^3 T)$ regret bound for this algorithm, hence providing a
complete characterization of optimal learning guarantees for first-price
auctions.
