  Software and hardware co-design and optimization of HPC systems has become
intolerably complex, ad-hoc, time consuming and error prone due to enormous
number of available design and optimization choices, complex interactions
between all software and hardware components, and multiple strict requirements
placed on performance, power consumption, size, reliability and cost. We
present our novel long-term holistic and practical solution to this problem
based on customizable, plugin-based, schema-free, heterogeneous, open-source
Collective Mind repository and infrastructure with unified web interfaces and
on-line advise system. This collaborative framework distributes analysis and
multi-objective off-line and on-line auto-tuning of computer systems among many
participants while utilizing any available smart phone, tablet, laptop, cluster
or data center, and continuously observing, classifying and modeling their
realistic behavior. Any unexpected behavior is analyzed using shared data
mining and predictive modeling plugins or exposed to the community at
cTuning.org for collaborative explanation, top-down complexity reduction,
incremental problem decomposition and detection of correlating program,
architecture or run-time properties (features). Gradually increasing
optimization knowledge helps to continuously improve optimization heuristics of
any compiler, predict optimizations for new programs or suggest efficient
run-time (online) tuning and adaptation strategies depending on end-user
requirements. We decided to share all our past research artifacts including
hundreds of codelets, numerical applications, data sets, models, universal
experimental analysis and auto-tuning pipelines, self-tuning machine learning
based meta compiler, and unified statistical analysis and machine learning
plugins in a public repository to initiate systematic, reproducible and
collaborative research, development and experimentation with a new publication
model where experiments and techniques are validated, ranked and improved by
the community.
