Kemp had been a captain of the Duke of Lancaster's Own Yeomanry since July 1891. In early February 1900, Kemp volunteered for active service in South Africa during the Second Boer War. He was appointed a captain of the Imperial Yeomanry, in command of the 23rd company (the Yeomanry detachment of the Duke of Lancaster's Own Yeomanry), to serve as part of the 8th Battalion Imperial Yeomanry. His company left Liverpool on the SS Africa on 12 February, and arrived in Cape Town the following month. For his service he was mentioned in despatches. He left again for South Africa in May 1902, as temporary lieutenant-colonel in command of the 32nd Battalion of the Imperial Yeomanry, including a machine-gun section which he had helped raise.
