Real Tesbed FL Performance Analysis published in IEEE QRS 2024
Congratulation to Jose, Bradely and Dr. Ahmed of SAYED Systems group along with the collabrators from HKUST, Sun Yat-Sen, HUST for the publication and presentation of work titled “Performance Profiling of Federated Learning Across Heterogeneous Mobile Devices” in IEEE QRS 2024. In this work, we showcase that as mobile devices increasingly become more widely used in data analytics and machine learning, Federated Learning (FL) offers a promising decentralised approach that maintains data privacy and reduces data transmission costs. This work analyses FL performance across heterogeneous mobile devices in real deployment. We examine the impacts of device heterogeneity on the training efficiency of FL systems by conducting a series of experiments involving real and emulated smartphones with various computational capabilities and network conditions. Through performance profiling, we identify bottlenecks in mobile settings to overcome the unique challenges mobile environments pose and design an FL system that effectively uses heterogeneous devices. Our experiments make use of a federated CIFAR10 dataset. Our results indicate higher setup times on physical devices than emulated ones, regardless of the network connection type. While data loading from storage is the primary bottleneck, consuming up to 88% of setup time, with a Broadband connection, downloading the model from the server becomes another major bottleneck with a 4G/LTE connection. Our findings also reveal that the fit function of the training phase takes up to 94% of training time. These insights can hopefully aid in designing FL systems that adapt effectively to heterogeneous resources in FL environments.