  The increasing complexity and scale of machine learning (ML) has led to the
need for more efficient collaboration among multiple teams. For example, when a
research team invents a new architecture like "ResNet," it is desirable for
multiple engineering teams to adopt it. However, the effort required for each
team to study and understand the invention does not scale well with the number
of teams or inventions. In this paper, we present an extension of our PyGlove
library to easily and scalably share ML ideas. PyGlove represents ideas as
symbolic rule-based patches, enabling researchers to write down the rules for
models they have not seen. For example, an inventor can write rules that will
"add skip-connections." This permits a network effect among teams: at once, any
team can issue patches to all other teams. Such a network effect allows users
to quickly surmount the cost of adopting PyGlove by writing less code quicker,
providing a benefit that scales with time. We describe the new paradigm of
organizing ML through symbolic patches and compare it to existing approaches.
We also perform a case study of a large codebase where PyGlove led to an 80%
reduction in the number of lines of code.
