Edge devices use neural network models learnt on cloud to predict labels of its data records, which may lead to incorrect predictions especially for records that are different from the data involved in the training process, i.e., out-of-distribution (OOD) records. However, recent efforts in OOD detection either require the retraining of the model or assume the existence of a certain amount of OOD records, thus limiting their application in practice. In this work, we propose a novel OOD detection method (named as NeuralFP) without requiring any access to OOD records, which constructs non-linear fingerprints of neural network models memorizing the information of data observed during training. The key idea of NeuralFP is to exploit the difference in how the neural network model responds to data records in its training set versus data records that are anomalous. Specifically, NeuralFP builds autoencoders for each layer of the neural network model and then carefully analyzes the error distribution of the autocoders in reconstructing the training set to identify OOD records. Through extensive experiments on multiple real-world datasets, we show the effectiveness of NeuralFP in detecting OOD records as well as its advantages over previous approaches. Furthermore, we provide useful guidelines for parameter selection in the practical adoption of NeuralFP.

Many image and vision applications require a large amount of data for model training. Collecting all such data at a central location can be challenging due to data privacy and communication bandwidth restrictions. Federated learning is an effective way of training a machine learning model in a distributed manner from local data collected by client devices, which does not require exchanging the raw data among clients. A challenge is that among the large variety of data collected at each client, it is likely that only a subset is relevant for a learning task while the rest of data has a negative impact on model training. Therefore, before starting the learning process, it is important to select the subset of data that is relevant to the given federated learning task. In this paper, we propose a method for distributedly selecting relevant data, where we use a benchmark model trained on a small benchmark dataset that is task-specific, to evaluate the relevance of individual data samples at each client and select the data with sufficiently high relevance. Then, each client only uses the selected subset of its data in the federated learning process. The effectiveness of our proposed approach is evaluated on multiple real-world image datasets in a simulated system with a large number of clients, showing up to 25% improvement in model accuracy compared to training with all data.