  Understanding the algebraic structure underlying a manifold with a general
affine connection is a natural problem. In this context, A. V. Gavrilov
introduced the notion of framed Lie algebra, consisting of a Lie bracket (the
usual Jacobi bracket of vector fields) and a magmatic product without any
compatibility relations between them. In this work we will show that an affine
connection with curvature and torsion always gives rise to a post-Lie algebra
as well as a $D$-algebra. The notions of torsion and curvature together with
Gavrilov's special polynomials and double exponential are revisited in this
post-Lie algebraic framework. We unfold the relations between the post-Lie
Magnus expansion, the Grossman-Larson product and the $K$-map, $\alpha$-map and
$\beta$-map, three particular functions introduced by Gavrilov with the aim of
understanding the geometric and algebraic properties of the double-exponential,
which can be understood as a geometric variant of the Baker-Campbell-Hausdorff
formula. We propose a partial answer to a conjecture by Gavrilov, by showing
that a particular class of geometrically special polynomials is generated by
torsion and curvature. This approach unlocks many possibilities for further
research such as numerical integrators and rough paths on Riemannian manifolds.
