  Using non-commutative differential forms, we construct a complex called
singular Hochschild cochain complex for any associative algebra over a field.
The cohomology of this complex is isomorphic to the Tate-Hochschild cohomology
in the sense of Buchweitz. By a natural action of the cellular chain operad of
the spineless cacti operad, introduced by R. Kaufmann, on the singular
Hochschild cochain complex, we provide a proof of the Deligne's conjecture for
this complex. More concretely, the complex is an algebra over the (dg) operad
of chains of the little $2$-discs operad. By this action, we also obtain that
the singular Hochschild cochain complex has a $B$-infinity algebra structure
and its cohomology ring is a Gerstenhaber algebra.
  Inspired by the original definition of Tate cohomology for finite groups, we
define a generalized Tate-Hochschild complex with the Hochschild chains in
negative degrees and the Hochschild cochains in non-negative degrees. There is
a natural embedding of this complex into the singular Hochschild cochain
complex. In the case of a self-injective algebra, this embedding becomes a
quasi-isomorphism. In particular, for a symmetric algebra, this allows us to
show that the Tate-Hochschild cohomology ring, equipped with the Gerstenhaber
algebra structure, is a Batalin-Vilkovisky algebra.
