According to Saxo Grammaticus, Haraldsborg was built by Harald Kesja. It was most likely a ring fortification with palisades. Saxo mentions a revolving wooden tower, but such structures are not known from archeological excavations. Harald Kesja plundered far and wide, which greatly contributed to his not being elected king after his father died in 1103. Instead, his uncle Niels of Denmark was elected king in 1104. In 1133, Haraldsborg came under siege by Harald's half-brother Eric Emune and fell after local German craftsmen from Roskilde had assisted Eric in the construction of a trebuchet. Haraldsborg was part of Queen Dorothea's dower lands.
