  In this paper, we start from an extension of the notion of holonomy on
diffeological bundles, reformulate the notion of regular Lie group or
Fr\"olicher Lie groups, state an Ambrose-Singer theorem that enlarges the one
stated in \cite{Ma2}, and conclude with a differential geometric treatment of
KP hierarchy. The examples of Lie groups that are studied are principally those
obtained by enlarging some graded Fr\"olicher (Lie) algebras such as formal
$q-$series of the quantum algebra of pseudo-differential operators. These
deformations can be defined for classical pseudo-differential operators but
they are used here on formal pseudo-differential operators in order to get a
differential geometric framework to deal with the KP hierarchy that is known to
be completely integrable in the sense of Frobenius. Here, we get an integration
of the Zakharov-Shabat connection form by means of smooth sections of a
(differential geometric) bundle with structure group some groups of
$q-$deformed operators. The integration obtained by Mulase \cite{M1}, and the
key tools he developped, are totally recovered on the germs of the smooth maps
of our construction.
  The tool coming from (classical) differential geometry used in this
construction is the holonomy group, on which we have an Ambrose-Singer-like
theorem: the Lie algebra is spanned by the curvature elements. This result is
proved for any connection a diffeological principal bundle with structure group
a regular Fr\"olicher Lie group. The case of a (classical) Lie group modelled
on a complete locally convex topological vector space is also recovered and the
work developped in \cite{Ma,Ma2} is completed.
