  Direct manipulation interfaces and programmatic systems have distinct and
complementary strengths. The former provide intuitive, immediate visual
feedback and enable rapid prototyping, whereas the latter enable complex,
reusable abstractions. Unfortunately, existing systems typically force users
into just one of these two interaction modes.
  We present a system called Sketch-n-Sketch that integrates programmatic and
direct manipulation for the particular domain of Scalable Vector Graphics
(SVG). In Sketch-n-Sketch, the user writes a program to generate an output SVG
canvas. Then the user may directly manipulate the canvas while the system
immediately infers a program update in order to match the changes to the
output, a workflow we call live synchronization. To achieve this, we propose
(i) a technique called trace-based program synthesis that takes program
execution history into account in order to constrain the search space and (ii)
heuristics for dealing with ambiguities. Based on our experience with examples
spanning 2,000 lines of code and from the results of a preliminary user study,
we believe that Sketch-n-Sketch provides a novel workflow that can augment
traditional programming systems. Our approach may serve as the basis for live
synchronization in other application domains, as well as a starting point for
yet more ambitious ways of combining programmatic and direct manipulation.
