  First-price auctions have very recently swept the online advertising
industry, replacing second-price auctions as the predominant auction mechanism
on many platforms. This shift has brought forth important challenges for a
bidder: how should one bid in a first-price auction, where unlike in
second-price auctions, it is no longer optimal to bid one's private value
truthfully and hard to know the others' bidding behaviors? In this paper, we
take an online learning angle and address the fundamental problem of learning
to bid in repeated first-price auctions, where both the bidder's private
valuations and other bidders' bids can be arbitrary. We develop the first
minimax optimal online bidding algorithm that achieves an
$\widetilde{O}(\sqrt{T})$ regret when competing with the set of all Lipschitz
bidding policies, a strong oracle that contains a rich set of bidding
strategies. This novel algorithm is built on the insight that the presence of a
good expert can be leveraged to improve performance, as well as an original
hierarchical expert-chaining structure, both of which could be of independent
interest in online learning. Further, by exploiting the product structure that
exists in the problem, we modify this algorithm--in its vanilla form
statistically optimal but computationally infeasible--to a computationally
efficient and space efficient algorithm that also retains the same
$\widetilde{O}(\sqrt{T})$ minimax optimal regret guarantee. Additionally,
through an impossibility result, we highlight that one is unlikely to compete
this favorably with a stronger oracle (than the considered Lipschitz bidding
policies). Finally, we test our algorithm on three real-world first-price
auction datasets obtained from Verizon Media and demonstrate our algorithm's
superior performance compared to several existing bidding algorithms.
