  The conventional model aggregation-based federated learning (FL) approach
requires all local models to have the same architecture, which fails to support
practical scenarios with heterogeneous local models. Moreover, frequent model
exchange is costly for resource-limited wireless networks since modern deep
neural networks usually have over a million parameters. To tackle these
challenges, we propose a novel knowledge-aided FL (KFL) framework, which
aggregates light high-level data features, namely knowledge, in the per-round
learning process. This framework allows devices to design their
machine-learning models independently and reduces the communication overhead in
the training process. We then theoretically analyze the convergence bound of
the proposed framework, revealing that scheduling more data volume in each
round helps to improve the learning performance. In addition, large data volume
should be scheduled in early rounds if the total scheduled data volume during
the entire learning course is fixed. Inspired by this, we define a new
objective function, i.e., the weighted scheduled data sample volume, to
transform the inexplicit global loss minimization problem into a tractable one
for device scheduling, bandwidth allocation, and power control. To deal with
unknown time-varying wireless channels, we transform the considered problem
into a deterministic problem for each round with the assistance of the Lyapunov
optimization framework. Then, we derive the optimal bandwidth allocation and
power control solution and develop an efficient online device scheduling
algorithm to achieve an energy-learning trade-off in the learning process.
Experimental results on MNIST and CIFAR-10 show that the proposed KFL is
capable of reducing over 99% communication overhead while achieving better
learning performance than the conventional model aggregation-based algorithms.
