Federated Learning Without Labels

We introduce a generalized federated self-supervised learning (FedSSL) framework and conduct in-depth empirical study of FedSSL based on the framework. Our study uncovers unique insights of FedSSL: 1) stop-gradient operation, previously reported to be essential, is not always necessary in FedSSL; 2) retaining local knowledge of clients in FedSSL is particularly beneficial for non-IID data. Inspired by the insights, we propose a new approach for model update, FedEMA.

We propose a novel federated unsupervised learning framework, FedU, to learn visual representation from decentralized data while preserving data prviacy. To tackle non-IID challenge, we propose two simple but effective methods: 1) We design the communication protocol to upload and update only the online encoders; 2) We introduce a new module to dynamically decide how to update predictors based on the divergence caused by non-IID.

We present FedUReID, a federated unsupervised person ReID system to learn person ReID models without any labels while preserving privacy. FedUReID enables in-situ model training on edges with unlabeled data. A cloud server aggregates models from edges instead of centralizing raw data to preserve data privacy. Moreover, to tackle the problem that edges vary in data volumes and distributions, we personalize training in edges with joint optimization of cloud and edge. Specifically, we propose personalized epoch to reassign computation throughout training, personalized clustering to iteratively predict suitable labels for unlabeled data, and personalized update to adapt the server aggregated model to each edge.