\section*{Acknowledgments}
\section*{Acknowledgement}
This paper is supported by  NSFC (Grant No.: U1836207) and the National Key R\&D Program of China (Grant No.: 2019YFE0103800).  We thank Dr. Xiaoning Du for her significant and timely helps. Dr. Weili Han is the corresponding author.

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi
\balance
\bibliographystyle{IEEEtran}

\section{Cryptographic Primitives}

\balance
\section{Machine Learning Models}
\label{sec.ml}

In this section, we briefly review the state-of-the-art machine learning models mentioned in Section \ref{sec:frameworks}: linear regression, logistic regression, deep neural network (DNN for short), convolutional neural network (CNN for short), support vector machine (SVM for short) and decision tree.

\begin{itemize}

    \item \textbf{Linear Regression:}  Linear regression is a fundamental building block of many machine learning algorithms where the relationship among independent and dependent variables is linear. Given a set of data points, it produces a model by fitting a linear curve through the data points. Formally, the linear function can be represented as $g(\mathbf{x}) = \mathbf{x} \cdot \mathbf{w}$, where $\mathbf{x}$ is the input vector and $\mathbf{w}$ is the parameter vector.

    The parameters of fitting function are set by continuously calculating the gap between the current fitted value and the actual value, then updating the parameters according to the gradient generated by the gap. Linear regression repeats this iteration several times (until convergence).
    \item \textbf{Logistic Regression:} In classification problems with two classes, logistic regression introduces the logistic function $f(u) = \frac{1}{1+e^{-u}}$ as the activation function to bound the output of the prediction between 0 and 1. Thus the relationship of logistic regression is expressed as $g(\mathbf{x}) = f(\mathbf{x} \cdot \mathbf{w})$.
    \item \textbf{Support Vector Machine:} SVM is a supervised machine learning model that uses classification algorithms for two-group classification problems. The main idea of SVM is find a hyperplane which has the maximum margin to all data samples. For the data samples which are linear inseparable, SVM use kernel methods to project low dimension samples to high dimension space to make all samples could be separated by a high dimension hyperplane.
    \item \textbf{Deep Neural Network:} DNN is one of the supervised learning model, which can learn complex and non-linear relationships among high dimensional data. DNN consists of one input layer, one output layer and multiple hidden layers, where the output of each layer is the input to the next layer. Each unit in the network is named as a neuron. The value of each neuron except for those in the input layer is calculated by a linear function with the input from parameters and the value of neurons from former layers. After the linear function, each neuron is also processed by an activation function (such as ReLu, sigmod).

    \item \textbf{Convolutional Neural Network:} CNN is widely used in image processing and computer vision, which takes pictures represented as matrices as input. The structure of CNN is similar to DNN, but it has multiple additional layers, convolutional layers and pooling layers. In a CNN, the amount of information of input pictures can get well abridged by the convolutional layers and pooling layers, which utilize the stationary property of images and can greatly reduce the amount of parameters while preserving a promising accuracy.

    \item \textbf{Decision Tree:} Decision tree is a non-parametric model, containing nodes and edges. Each interior node corresponds to partitioning rule; the edges leaving a node correspond to the possible values taken on by that partitioning rule and leaf nodes  correspond to class labels. Given a decision tree, a feature vector is classified by  walking the tree starting from the root node, and using the partitioning rule represented by each interior node to decide which edge to take  until reach a leaf. The class result will be found in the leaf node.
\end{itemize}

\section{Discussion}
\label{sec:disc}

\begin{figure*}[htb]
    \centering
    \includegraphics[scale=1]{pictures/development.pdf}
    \caption{The development history to resolve \texttt{TMMPP}\xspace}
    \label{fig.development}
\end{figure*}

\texttt{TMMPP}\xspace enables multiple {data controller}s to collaboratively learn a machine learning model while keeping all the training data private. \hq{This solves} the open problem of data isolated islands, \hq{which is} limited by privacy protection laws and regulations.
In recent years, this topic has aroused great interest and attention, both in academia and industry.

In this section, we \hq{focus on} the development \hq{history} of \texttt{TMMPP}\xspace, \hq{and then take a step forward to discuss} its future directions.

\subsubsection{Development History}
The development history of \texttt{TMMPP}\xspace is summarized in chronological order, as shown in Figure \ref{fig.development}.
After the concept of MPC was proposed in 1986, the development of MPL was mainly to improve the underlying protocols, although there are some frameworks for training linear regression models or for secure neural network inference. Until in 2017, Mohassel et al. proposed secureML~\cite{mohassel2017secureml}, MPC based machine learning frameworks that support the training phase became hotter, especially in 2019. The concept of FL was proposed in 2016 and is still in its infancy. Since 2019, a large number of platforms about \texttt{TMMPP}\xspace have been developed by many companies to deal with some real-world problems or for researches.

\subsubsection{Future Directions}
In the future, the study of \texttt{TMMPP}\xspace would cover the topics of privacy protection, efficiency optimization, and supporting more data partitioning and formats, and system implementations.

\begin{itemize}

    \item \textbf{Privacy protection.} Privacy protection is a first-priority concern in \texttt{TMMPP}\xspace, including two types, the least leakage of privacy and privacy compliance. We can only try our best to reduce the leakage of privacy. MPC would provide high \sls{security} guarantees. Many current MPL frameworks can resist semi-honest models well, but these frameworks are still not efficient enough for malicious models. ABY3~\cite{mohassel2018aby3} can theoretically support the malicious model, but it has not been implemented.
    FL frameworks towards protecting {data subject}s' privacy by sharing model updates (e.g., gradient information), instead of the raw data. However, the updates communicated throughout the training process can nonetheless reveal sensitive information to the centralized server. While there are some methods that aim to enhance the privacy of FL using HE, MPC or DP, these approaches usually provide privacy at the cost of reducing model performance or framework efficiency.
    In many practical scenarios, we only need to do not violate the privacy compliance of each \emph{data controller}. Under the circumstances permitted by laws and regulations, that is, under the compliance, to relax privacy restrictions can maximize the utilization of data of each \emph{data controller} to train machine learning models with better performance.

    \item \textbf{Efficiency optimization.} Efficiency is a challenge that has always existed in \texttt{TMMPP}\xspace. When the number of {data controller}s is larger, the communication overhead and computational complexity will be much larger, especially in MPL. In fact, the efficiency of \texttt{TMMPP}\xspace is closely related to the technological route chosen, so it is essentially important to choose an appropriate protocol or privacy method to deal with the different application scenarios. For example, if there are millions of {data controller}s to train a machine learning model, it is suitable to choose an FL framework now.
   Under the compliance, the trade-off between security and efficiency can be balanced, thereby sacrificing some security to improve the efficiency of the framework.

    \item \textbf{Supporting more data partitioning and formats.} From the discussion of Section \ref{sec.comparison}, the training data in both MPL frameworks and FL frameworks are usually cut horizontally. However, vertical data partitioning~\cite{vaidya2005privacy} is also common and important in the real-world, especially between/among different organizations. Thus, more efforts should be paid to vertical MPL frameworks and FL frameworks. What's more, the problems of statistical heterogeneity and data unbalance arise when training models from data that are not identically distributed across devices. Currently, MPL has no effective solution for Non-IID data so far. For FL, the effective solution is multi-task learning~\cite{smith2017federated}\cite{corinzia2019variational} and meta-learning~\cite{li2019differentially}\cite{jiang2019improving}\cite{fallah2020personalized}. Both the multi-task and meta-learning enable personalized or device-specific modeling, which is often a more natural approach to handling the statistical heterogeneity of the data. How to make the MPL based frameworks support Non-IID data or combine with multi-task learning or meta-learning would be interesting research directions.

    \item \textbf{System implementations.} In this paper, we assume that the network is reliable and without system heterogeneity. However, in the real-world when online training a machine learning model over multiple sources, the computational capabilities of each \emph{data controller}'s device may be different due to the variability in hardware (CPU, memory), wireless channel conditions (3G, 4G, 5G, WiFi) and power (battery level). Even, during the training process, clients may drop out  (due to unreliable networks, device battery issues, etc.).

\end{itemize}

\section{Introduction}
\label{introduction}
In the era of big data, almost all online activities are driven by data. As is said by IBM's Chief Executive Officer that ``big data is the new oil'', the data would bring us huge benefits, therefore have become the key in current business competitions. In addition, these data are pushing the advances of machine learning which have brought us such breakthroughs in various scenarios, such as medical diagnosis~\cite{esteva2017dermatologist}\cite{fakoor2013using}, image classification~\cite{rastegari2016xnor}, and facial recognition~\cite{schroff2015facenet}. In such aforementioned widely-applied scenarios, massive amounts of high-quality data are the dominating factor in the performance of the machine learning models.

However, researchers are faced with a tough predicament where the needed data are hard to be collected and shared directly to jointly train high-performance machine learning models, because these data protected by privacy protection policies or regulations might contain much sensitive or private information, and are usually stored in multiple locations.
Recently, the released laws, such as General Data Protection Regulation (GDPR)~\cite{voigt2017eu} which is presented by the European Union, aggravates this predicament during data sharing. Here, GDPR defines three key roles in data sharing: the \emph{data subject}, which refers that ``\emph{an identified or identifiable natural person who owns the personal data}'';
the \emph{data controller}, which refers that ``\emph{the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data}'';
and the \emph{data processor}, which refers that ``\emph{a natural or legal person, public authority, agency or other body which processes personal data on behalf of the controller}''.
The role considered in this paper is the \emph{data controller}.

How to utilize the decentralized data in multiple \emph{data controller}s to train high-performance machine learning models efficiently with privacy preservation is a key challenge.
In this challenging scenario, the \emph{data controller}s own valuable data under their controls, which can support the training process. However, the potential leakage risk of sensitive information incurs regulations enforced by related laws, which further leads to the phenomenon of isolated data islands.
The solutions to this challenge are essential for current small innovative businesses, which is usually the main-stream force of innovations in our modern society, because, compared to the big Internet vendors, each small company in the innovative businesses usually controls a much smaller volume of data in its initial step.

Motivated to counter this challenge, in this paper, we deeply investigate the current advances of the problem of \underline{T}raining  machine learning \underline{M}odels over \underline{M}ultiple data sources with \underline{P}rivacy \underline{P}reservation, referred to as \texttt{TMMPP}\xspace,
from two technical solutions: 1) \sls{Secure Multi-party Learning~\cite{bogdanov2008sharemind}\cite{demmler2015aby}\cite{mohassel2017secureml} (MPL for short), i.e., Secure Multi-party Computation (MPC for short) based machine learning, whose architecture is typically the Peer-to-Peer model.} 2) Federated Learning (FL for short), whose architecture is typically the Client-Server model~\cite{mcmahan2016federated}\cite{konevcny2016federated1}\cite{konevcny2016federated2}\cite{mcmahan2017communication}.
Note that, researchers and engineers also propose trust execution environment technical solutions that we do not investigate in this paper. In this solution, they use the technologies of trusted execution environment to train the models over a central data source from distributed locations. The privacy is preserved by the trustworthiness of the data process environment, where executors can only obtain the final results and cannot know the details of raw data.

MPC~\cite{yao1986generate}, proposed by Yao in 1986, is originated from the millionaire problem.
Two millionaires can use Yao's protocol to figure out who is richer without revealing their concrete assets to each other or third-party.
MPC provides a security mechanism by which a group of mutually distrusting parties can jointly compute functions without revealing any private information beyond the function result. Especially, MPC can be implemented without a trusted third party.
The identification of MPC is groundbreaking, creating opportunities in which semi-trusted parties can jointly calculate results of mutual interest without a trusted third party.
Owing to the powerful \sls{security} guarantees, MPC has a wide range of potential application scenarios.
With the supports of MPC protocols, researchers have developed several machine learning frameworks to keep the training data private.
But in the real-world scenarios, the expensive communication overhead and computation complexity of its underlying protocols are still two issues.

In 2016, Google proposed the concept of  FL~\cite{konevcny2016federated1}\cite{konevcny2016federated2}, which made distributed privacy-preserving machine learning a hot topic once again.
FL can enable \emph{data controller}s` efficiently train machine learning models without collecting raw data.
In the FL framework proposed by Google, each \emph{data controller} trains a model locally with their local data, and upload the intermediate training results such as gradients rather than raw data to a centralized server.
That is, the \emph{data controller}s and centralized server work together only by exchanging model parameters (e.g., gradients) and global parameters. Thus, FL usually has lower communication overhead and computation complexity than \sls{MPL}.
Briefly, FL iteratively executes the four steps as follows: the \emph{data controller}s 1) download current global model parameters from the centralized server; 2) update the models using their local data, 3) upload the new model parameters to the centralized server;
the centralized server 4) computes a new global model by aggregating the updates from the other \emph{data controller}s.
In past five year, FL becomes a hot topic to resolve the problem of \texttt{TMMPP}\xspace. A few practical frameworks~\cite{smith2017federated}\cite{cheng2019secureboost}\cite{liang2020think}\cite{xu2019verifynet} and applications~\cite{wang2019edge}\cite{kim2019blockchained}\cite{nishio2019client}\cite{zhao2020local} extended
from the FL framework proposed by Google in 2016 have been presented.

\subsection{Related Surveys}
The problem of \texttt{TMMPP}\xspace has been studied for a few years. Academic researchers are motivated to present protocols or frameworks, while industrial vendors build up practical platforms.
\hq{Several related surveys have also been conducted concerning the topic of MPL and FL respectively.}

Archer et al.~\cite{archer2016maturity} investigated the state-of-the-art secure computation technologies, described three paradigms of programmable secure computation, including Homomorphic Encryption (HE for short), Garbled Circuit (GC for short), and Secret Sharing (SS for short).
Then they evaluated schemes based on existing applications and benchmarks. Mood et al.~\cite{mood2016frigate} presented a brief survey about existing secure computation compilers, and analyzed these compilers' correctness. Hastings et al.~\cite{hastings2019sok} surveyed general-purpose compilers for MPC, which focused on their usability features, along with container-based virtual environments to run example programs.

Yang et al.~\cite{yang2019federated} provided a seminal survey of existing works on FL, generally introduced the definition, architecture and techniques of FL, and further discussed its potential in various applications. Li et al.~\cite{li2019federateda} discussed the challenges faced by FL and gave a broad overview of current works about FL around these challenges. Finally, they outlined several open directions worth the future research effort.
A comprehensive survey written by Kairouz et al.~\cite{kairouz2019advances}, summarized the recent advances and challenges on FL from various research topics.
The literature~\cite{li2019federatedb} surveyed FL systems while provided a categorization with six different aspects and gave the comparison among existing FL systems.
Lim et al.'s~\cite{lim2020federated} survey highlighted the issues and solutions regarding the FL implementation to Mobile Edge Computation.

The above surveys and reviews merely focus on \sls{MPL} or FL separately. Despite the rapid development of
the solutions to \texttt{TMMPP}\xspace, a comprehensive and systematic survey of \texttt{TMMPP}\xspace is still absent so far.
Thus, we are motivated to conduct a \hq{comprehensive} and systematic literature review of the technical routes, frameworks and platforms of \texttt{TMMPP}\xspace.
Our study can help researchers choose suitable \texttt{TMMPP}\xspace frameworks and platforms for various scenarios, and further identify research gaps and improve the weaknesses in the approaches.
\subsection{Our Contributions}
In this paper, our contributions are as follows:

\begin{itemize}
    \item We present a definition of the problem of \texttt{TMMPP}\xspace, and list the challenges to solve the technical issues faced by \texttt{TMMPP}\xspace, including statistical challenges, efficiency challenges and security challenges.
    \item We investigate the state-of-the-art works proposed to resolve \texttt{TMMPP}\xspace, and classify the various frameworks based on the \hq{underlying} techniques they built upon. For the solutions of \sls{MPL}, we divide the frameworks into four categories: HE-based \sls{MPL} frameworks, GC-based \sls{MPL} frameworks, SS-based \sls{MPL} frameworks, and Mixed-protocol based \sls{MPL} frameworks.
    For the solutions of FL, we divide FL frameworks into three categories according to the type of privacy mechanisms, including Non-cryptographic FL frameworks, DP-based FL frameworks, and SC-based FL frameworks.

    \hq{Then, we provide an analysis of the differences between these two routes, along with the strengths and weakness, respectively.}
    \item We sketch the platforms about the solutions of \texttt{TMMPP}\xspace. For the open-source platforms, we compare them based on a series of indicators, such as the type of FL, privacy mechanisms. For the closed-source platforms, we compare them with other platforms according to their application scenarios.
    \item We present a comprehensive overview of \texttt{TMMPP}\xspace's history and discuss its future directions, including privacy protection, efficiency optimization, supporting more data partitioning and type, and system implementations.

\end{itemize}

The rest of this paper is organized as follows: Section \ref{sec:pc} presents the \hq{problem} definition of \texttt{TMMPP}\xspace and identifies the challenges to solve the problem;
Section \ref{sec:frameworks} investigates the state-of-the-art works on \texttt{TMMPP}\xspace, summarizes the protocols and frameworks based on various cryptographic primitives, and analyzes their characteristics.
We also introduce the platforms about \texttt{TMMPP}\xspace and compare their advantages and disadvantages with each other in Section \ref{sec:platforms}.
Finally, we discuss the future directions of \texttt{TMMPP}\xspace in Section \ref{sec:disc}.

\section{Platforms}
\label{sec:platforms}
In recent years, many platforms related to \texttt{TMMPP}\xspace have been developed by international and well-known business enterprises, including some open-source projects \texttt{FATE}, \texttt{TFF}, \texttt{PaddleFL}, \texttt{Pysyft}, \texttt{coMind}, as well as closed-source projects \texttt{Clara FL}, \texttt{FMPC}.
In this section, we introduce eight platforms and give a comparison among them according to a series of indicators.

\subsection{Overview}
We give an overview of the eight projects about \texttt{TMMPP}\xspace, including five open-source systems (\texttt{FATE}, \texttt{TFF}, \texttt{PaddleFL}, \texttt{Pysyft}, \texttt{coMind}) and three closed-source systems (\texttt{Clara FL}, \texttt{FMPC}, \texttt{Shared Machine Learning}).

\subsubsection{Open-Source Platforms}
Table \ref{open-source} showed the basic information of five open-source platforms.
Overall, all five platforms support horizontal data partitioning. \texttt{FATE} and \texttt{PaddleFL} also support vertical data partitioning and transfer learning. \texttt{FATE} provides two privacy technologies, i.e HE and MPC, and \texttt{PaddleFL} provides MPC and DP. \texttt{Pysyft} provides all these three privacy technologies, while \texttt{TFF} and \texttt{coMind} do not provide any privacy technologies. Only \texttt{TFF} and \texttt{coMind} utilize GPUs, but they cannot support cluster deployment.
Because \texttt{FATE} and \texttt{PaddleFL} provide algorithm-level interfaces,  users can implement their owned algorithms. TFF, \texttt{Pysyft} and \texttt{coMind} provide building blocks which are easier for non-expert users to use.

\begin{table*}[]
    \caption{The comparison among five open-source platforms}
    \label{open-source}
    \centering
    \renewcommand\arraystretch{1.3}
    \begin{tabular}{|c|c|c|c|c|c|c|}
    \hline
    \multicolumn{2}{|c|}{Indicators/Platforms}              & \texttt{FATE}  & \texttt{TFF}  & \texttt{PaddleFL}  & \texttt{Pysyft}  & \texttt{coMind}  \\ \hline
    \multirow{3}{*}{Data Partitioning}  & Horizontal        & \checkmark     & \checkmark    & \checkmark         & \checkmark       & \checkmark       \\ \cline{2-7}
                                        & Vertical          & \checkmark     & $\times$      & \checkmark         & $\times$         & $\times$         \\ \cline{2-7}
                                        & Transfer Learning & \checkmark     & $\times$      & \checkmark         & $\times$         & $\times$         \\ \hline
    \multirow{3}{*}{Privacy Technology} & HE                & \checkmark     & $\times$      &  $\times$          & \checkmark       & $\times$         \\ \cline{2-7}
                                        & MPC               & \checkmark     & $\times$      & \checkmark         & \checkmark       & $\times$         \\ \cline{2-7}
                                        & DP                & $\times$       & $\times$      & \checkmark         & \checkmark       & $\times$         \\ \hline
    \multirow{2}{*}{Type of Deployment} & Stand-alone       & \checkmark     & \checkmark    & \checkmark         & \checkmark       & \checkmark       \\ \cline{2-7}
                                        & Cluster           & \checkmark     & $\times$      & \checkmark         & \checkmark       & $\times$         \\ \hline
    \multirow{2}{*}{Hardware}           & CPU               & \checkmark     & \checkmark    & \checkmark         & \checkmark       & \checkmark       \\ \cline{2-7}
                                        & GPU               & $\times$       & \checkmark    & $\times$           & $\times$         & \checkmark       \\ \hline
    \multicolumn{2}{|c|}{Custom Model}                     & \checkmark     & $\times$      & \checkmark         & $\times$         & $\times$         \\ \hline

    \end{tabular}
\end{table*}

\subsubsection{Closed-Source Platforms}
We can only collect the information of these closed-source platforms from some documents provided by the official websites without \hq{detailed} implementation details.
Closed-source platforms are all enterprise-level platforms designed for specific scenarios, \hq{thus we here} only compare the application scenarios of three closed-source platforms as shown in Table \ref{closed-source}.

\begin{table}[h]
    \caption{The comparison among three closed-source platforms}
    \label{closed-source}
    \centering
    \renewcommand\arraystretch{1.3}
    \scalebox{0.9}{
    \begin{tabular}{|c|c|c|c|}
    \hline
    Application Scenarios   & \texttt{Clara FL}     & \texttt{FMPC}         & \begin{tabular}[c]{@{}c@{}}\texttt{Shared} \\ \texttt{Machine Learning}\end{tabular} \\ \hline
    Financial Field         & $\times$              & \checkmark             & \checkmark                                                   \\ \hline
    Medical Field           & \checkmark            & \checkmark             & $\times$                                                     \\ \hline
    Government Field        & $\times$              & \checkmark             & $\times$                                                     \\ \hline
    \end{tabular}
    }
\end{table}

\subsection{Technical Details }
In this section, we give the details of eight platforms.
\subsubsection{\texttt{FATE}}

To the best of our knowledge, \texttt{FATE} (Federated AI Technology Enable)~\footnote{\url{https://github.com/FederatedAI/FATE}} is the first open-source industrial level FL framework presented by WeBank's AI Department in February 2019. It enables multiple companies and institutions to effectively collaborate on training machine learning models in compliance with data security and data protection regulations.

\texttt{FATE} provides algorithm level APIs detailed documents on installation and usage for users to use directly.
It provides a secure computing framework to support various machine learning algorithms, such as linear regression, logistic regression, boosting tree \cite{cheng2019secureboost}, NNs, and so on,  which support heterogeneous and homogeneous styles. \texttt{FATE}  also provides various model evaluations, including binary classification,  multi-classification, regression evaluation, and local vs federated comparison. Currently, \texttt{FATE} supports all the three FL architectures, including vertical FL, horizontal FL, and federated transfer learning. Additionally, \texttt{FATE} supports both standalone and cluster deployments.

\subsubsection{\texttt{TFF}}
\texttt{TFF} (TensorFlow Federated)~\footnote{\url{https://github.com/tensorflow/federated}}, developed by Google, is an open-source framework for federated machine learning and other computation on decentralized data.

\texttt{TFF} provides a flexible, open framework for locally simulating decentralized computations into the hands of all TensorFlow users.
It enables developers to simulate the included FL algorithms on their models and data, as well as to experiment with novel algorithms.
\texttt{TFF} also supports non-learning computation, such as aggregated analytics over decentralized data.
\texttt{TFF} provides two APIs of different layers: Federated Learning (FL) API and Federated Core (FC) API. FL API offers a set of high-level interfaces, which plug existing Keras or non-Keras machine learning models into the TFF framework.
With FL API, users can perform FL or evaluation to their existing TensorFlow models, without studying the details of FL algorithms.
FC API comes with a set of lower-level interfaces, is the core of the framework, and also serves as the foundation to built FL.
The interfaces concisely express custom federated algorithms by combining TensorFlow with distributed communication operators within a strongly-typed functional programming environment.

With \texttt{TFF}, developers can declaratively express federated computations, so they could be deployed to diverse runtime environments.
However, the latest version of \texttt{TFF} currently released only supports horizontal FL without underlying privacy technologies (e.g., HE, MPC, and DP) to protect data security.
Therefore, \texttt{TFF} is only suitable for experimental testing and simulation, and cannot be deployed in a real environment. What's more, \texttt{TFF} only supports single-machine simulation of multiple machines for training models while cannot support cluster deployment.

\subsubsection{\texttt{PaddleFL}}
\texttt{PaddleFL}~\footnote{\url{https://github.com/PaddlePaddle/PaddleFL}} is an open-source FL framework based on PaddlePaddle~\footnote{\url{https://github.com/PaddlePaddle/Paddle}}, which is a machine learning framework developed by Baidu.
It is mainly designed for deep learning, providing several FL strategies and applications in the fields of computer vision, natural language processing, recommendation, and so on.

\texttt{PaddleFL} provides a basic programming framework for researchers and encapsulates some public FL datasets that researchers to easily replicate and compare different FL algorithms.
With the help of the rich model library and pre-trained models of PaddlePaddle, it is also easy to deploy a federated learning system in distributed clusters.
In the design of \texttt{PaddleFL}, it implements secure training and inference tasks based on ABY3~\cite{mohassel2018aby3}, and uses DP as one of the privacy mechanisms.
As for horizontal FL strategies, \texttt{PaddleFL} implements a variety of different optimization algorithms, such as DP-SGD, FedAvg, and Secure Aggregate, etc.
For vertical FL strategies, it provides two algorithms, including logistic regression and NN.  Currently, \texttt{PaddleFL} supports Kubernetes to deploy it easily and open-sources a relatively complete horizontal federated learning version, but vertical federated learning and transfer federated learning is still in the early stages.

\subsubsection{\texttt{PySyft}}
\texttt{PySyft}~\footnote{\url{https://github.com/OpenMined/PySyft}} is an open-source Python library built for FL and privacy-preserving, which was originally outlined by Pyffel et al.~\cite{ryffel2018generic} and its first implementation was lead by \sls{OpenMined}, one of the leading decentralized AI platforms.

\texttt{PySyft} is a flexible, easy-to-use library and enables perform private and secure computation on deep learning models. \texttt{PySyft} provides interfaces for developers to implement their algorithms.
It decouples private data from model training, using FL, secure computation techniques (like MPC and HE), and privacy-preserving techniques (like DP) within different deep learning frameworks, such asiPyTorch, Keras, and TensorFlow.
Moreover, \texttt{PySyft} provides a comprehensive step-by-step list of tutorials, designed for complete beginners. These tutorials cover how to perform techniques such as FL, MPC, and DP using \texttt{PySyft}.
With tutorials, users can learn about all the ways \texttt{PySyft} can be used to bring privacy and decentralization to the deep learning ecosystem.

\subsubsection{\texttt{coMind}}
\texttt{coMind} \footnote{\url{https://github.com/coMindOrg/federated-averaging-tutorials}} is an open-source project for jointly training machine learning models with privacy preservation base on TensorFlow.

It develops a custom optimizer, which implements federated averaging for the Tensorflow to train NNs easily. Besides, it provides a series of tutorials and examples to help users on how to use TensorFlow and config FL.
There are two types of examples provided, including three basic examples and three advanced examples, which introduces how to train and evaluate TensorFlow machine learning models in a local, distributed, and federated way, respectively.
In this project, both message passing interface and python sockets can implement federated averaging. They take the communication out of TensorFlow and average the weights by a custom hook.
In addition, Keras framework is used in \texttt{coMind} as the basis of distributed and federated averaging.
However, similar to \texttt{TFF}, \texttt{coMind} does not provide any encryption method.

\subsubsection{\texttt{Clara FL}}

\texttt{Clara FL} (Clara Federated Learning)\footnote{\url{https://developer.nvidia.com/clara}} is an application mainly for the medical field, first introduced by NVIDIA on December 1, 2019. It addresses the concerns about protecting the security of the huge volumes of patient data which are required to train artificial intelligence (AI) models.

Running on NVIDIA EGX intelligent edge computing platform, \texttt{Clara FL} facilitates distributed, collaborative AI model training utilized FL technology without the need for sharing personal data, therefore keeping patient data private and secure.
\texttt{Clara FL} has been deployed healthcare giants around the world, such as King’s College London, Peking University First Hospital, American College of Radiology MGH and BWH Center for Clinical Data Science, and UCLA Health.
While \texttt{Clara FL} provides official documents, \texttt{Clara FL} is not open source, so some training details and privacy technologies it used are not accessible.

\subsubsection{\texttt{FMPC}}
\texttt{FMPC}~\footnote{\url{https://www.fudata.cn/}} is a secure computation platform developed by Fushu Technology.
It builds both \texttt{FMPC} secure computation product and \texttt{FMPC} open platform, providing the capabilities of secure computation and FL for big data applications from all walks of life.\texttt{FMPC} secure computation product implements secure big data integration and FL with the characteristics of industrial level and out of the box. The product has multiple functional versions, such as \texttt{FMPC Federated Learning} and \texttt{FMPC MPC}, which both support private deployment.

The users of \texttt{FMPC Federated Learning} Platform \sls{could manage their owned projects} and data through a visual interface to complete the secure federated modeling. When running distributed machine learning algorithms among collaborative parties, all operations and calculations are performed in the party's owned private environments, ensuring that the data do not go out in the private domain. The \texttt{FMPC Federated Learning} Platform supports a wealth of machine learning algorithms to meet the needs of various application scenarios.
\texttt{FMPC MPC} Platform realizes multiple operators such as secure statistics, and secure matrix operations, by SS, GC, HE, and other technologies. The user initiates MPC projects in their owned private environment, adds collaborative partners, agrees on secure multi-party computing rules, and start the task.
\texttt{FMPC MPC} Platform has a friendly user interface and development interface, suitable for a variety of scenarios such as finance, medical, government, and industry.

\subsubsection{\texttt{Shared \sls{Machine} Learning}}
\texttt{Shared} \texttt{Machine Learning}, proposed by Ant Financial, is a learning paradigm that aggregates multi-party information while protecting the privacy of individual in the context of mutual distrust between data owners and platform.
There are two solutions, one is TEE-based centralized solution, and the other MPC-based distributed solution.

The foundation layer used in \texttt{Shared Machine Learning} is Intel's SGX, which can be compatible with other TEE implementations.
Each participant realizes data security and privacy protection by encrypting and uploading local data to a TEE for calculation.
This solution supports both online prediction and offline training.

The framework of the MPC solution is divided into there layers: security technology layer, basic operator layer, and secure machine learning algorithm. The security technology layer provides basic security technologies, such as GC, HE, DP, etc.
Based on the security technology layer, the encapsulation of basic operators is done, including the computation of matrix (such as addition, multiplication),  the computation of function (such as ReLU, sigmoid), and the basic computation (such as comparison).
This framework supports a variety of machine learning algorithms, such as GBDT, NNs.

\subsubsection{Others}
There are other systems related to MPTML.
JUGO~\footnote{\url{https://jugo.juzix.net/home}}, developed by JUZIX, provides a MPC underlying algorithm platform and integrates an SDK for general MPC algorithms.
However, JUGO only supports two parties to collaborate calculating with basic operation, such as additation and comparison.
Hive is a federated learning platform build by Ping An Technology for the financial industry. So far, the platform has not been open-sourced, and the official documents are lacking.
Except for PaddleFL, Baidu develpes MesaTEE \footnote{\url{https://anquan.baidu.com/product/mesatee}}, a universal secure computation platform. MesaTEE builds a FaaS (Function-as-a-Service) general computing framework, providing strict and practical privacy and security capabilities.
TF-Encrypted \footnote{\url{https://github.com/tf-encrypted/tf-encrypted}} is a library for privacy-preserving machine learning in TensorFlow. It makes use of the ease-of-use of the Keras API while training and predicting encrypted data with MPC and HE technologies.

\section{Problem Definition and Challenges}
\label{sec:pc}
In this section, we define the problem of \texttt{TMMPP}\xspace. Then, we identify the main technical challenges when solving it.

\subsection{Problem Definition}
\label{problem}

As is shown in Figure~\ref{fig.problem}, when we consider the following scenario: a group of \emph{data controller}s, who may be semi-honest or even malicious, plan to jointly train a machine learning model over their owned data with \sls{security} assurance (\sls{we only consider the secure training phase and ignore the secure inference}).
\hq{Note that, we here focus on the secure training phase which is more generic since the secure inference is naturally implied when training is done.}
These \emph{data controller}s communicate with or without a centralized server during the training. But, they do not upload their raw data to a centralized party (or other \emph{data controller}s).

Here, we present a \hq{formal} definition of the above problem. Without loss of generality, let $\lbrace\mathcal{D}_1, \mathcal{D}_2, ... , \mathcal{D}_n \rbrace$ respectively be the raw data set held by $n$ \emph{data controller}s $\lbrace\mathcal{P}_1, \mathcal{P}_2, ... , \mathcal{P}_n \rbrace$ who may be mutually distrusted. We use $\mathcal{M}$ to denote the machine learning model cooperatively trained by \emph{data controller}s.

Then, the problem of \texttt{TMMPP}\xspace can be stated as follows:

\textbf{Input:} Each \emph{data controller} $\mathcal{P}_i$ takes its owned raw data $\mathcal{D}_i$ as the input.

\textbf{Output:} A global model $\mathcal{M}$ jointly trained by all the \emph{data controller}s without exposing any information about the raw data of any \emph{data controller}s to others in the process.

\begin{figure}[ht]
\centering
\includegraphics[scale=0.7]{pictures/problem.pdf}
\caption{The structure of training machine learning models over multiple data sources with privacy preservation. $\mathcal{P}_i$ denotes the \emph{data controller}.}
\label{fig.problem}
\end{figure}

Basically, we assume that the network is reliable and the system is homogeneous.
Specifically, in the solutions of \sls{MPL}, a group of \emph{data controller}s cooperatively compute a function on their inputs without revealing any information including intermediate results, beyond the result of the computation.
\sls{Besides, MPL schemes can potentially keep the model private.}
In the solutions of FL, the aim is to learn the global machine learning model under the constraint that each data controller's data are stored and processed locally, with only model parameters being communicated through a centralized server.
Both the centralized server and \emph{data controller}s could be distrusted.

\sls{Note that, in this paper, the involved \emph{data controller}s in both two series of solutions of \sls{MPL} and FL are both input parties and compute parties. However, there are other settings in the solution of MPL that the computation can be outsourced to a small number of compute parties. And in the real business scenarios of the FL, a \emph{data controller} might keep his or her data in private but obtained the trained model, i.e. this data controller is not a compute party. }

\subsection{Challenges}
\label{sec.challenge}
We summary three key challenges, i.e. statistical challenge, efficiency challenge, and security challenge, when we solve the problem of \texttt{TMMPP}\xspace.

\subsubsection{Statistical Challenge}

The data held by \emph{data controller}s are usually generated or collected in a non-IID (not independent and identically distributed) manner, i.e. the data may not be independent, or have a distinct distribution, even neither independent nor identically distributed.
Moreover, the volume of data distributed across \emph{data controller}s may also vary significantly, in other words, the data are unbalanced.
The non-IID and unbalanced data make it difficult to train a high-quality machine learning model and may add complexity in terms of analysis, and evaluation.
For instance, the \emph{data controller}s who account for a large portion of the total data volume will play a decisive role in the phase of the centralized server aggregation model parameters, which will affect the performance of the model.

\subsubsection{Efficiency Challenge} Efficiency is a significant bottleneck in \texttt{TMMPP}\xspace, including communication overhead and computational complexity.
Training machine learning models over multiple data sources involves a massive number of data controllers, e.g., in the scenario of FL, there are millions of \emph{data controller}s.
Due to the security requirements to protect each data controller's privacy, the communication among them brings in additional overhead compared to local computation on raw data by many orders of magnitude.
Besides, in \sls{MPL} frameworks, the costs of communication and computation greatly depends on underlying protocols.
For instance, HE-based protocols usually lead to high computation complexity, and GC-based protocols generally lead to expensive communication overhead. Generally, to improve the efficiency of the \sls{MPL} frameworks, a trade-off is supposed to be made between communication overhead and computation complexity.

\sls{\subsubsection{Security Challenge} Each \emph{data controller} and the centralized server \hwl{cannot be fully trusted}.
\hq{Some of them may be adversaries, who can somehow impose attacks on the private information or interfere the normal execution of the training algorithms.}
What's more, FL protects the data of each data controller by exchanging model parameters (e.g., local gradient), instead of the raw data.
However, these gradients may leak sensitive information of the raw data if manipulated considerately, leading to privacy leakage during the aggregation of these gradients by the centralized server.}

\sls{In this paper, we consider two threat models: the semi-honest model and malicious model:
(1) The semi-honest (also known as honest-but-curious or passive) adversaries attempt to gain as much information as possible from the data they are not entitled to during the protocol execution. But they do not deviate from the protocol specification. (2) Malicious (also known as active) adversaries can break the protocol arbitrarily, such as sending incorrect messages to
\hq{other} \emph{data controller}s. Protocols that achieve security in this model can provide a very high guarantee of security.}

\section{Protocols and Frameworks}
\label{sec:frameworks}
In this section, we summarize the state-of-the-art academic works about \texttt{TMMPP}\xspace, sketch the different technical routes, and analyze the advantages and disadvantages of these technical routes.

Taking the fundamental technologies utilized during the machine learning training process into consideration, we divide the MPL frameworks into four categories, including HE-based MPL frameworks, GC-based MPL frameworks, SS-based MPL frameworks and Mixed-protocol based MPL frameworks.
Besides, we divide FL frameworks into three categories according to the type of privacy mechanisms, including Non-cryptographic FL frameworks, DP-based FL frameworks, and SC-based FL frameworks.
Besides, Appendix \ref{sec.ml} shows the brief introduction of machine learning models commonly trained in the MPL frameworks and FL frameworks.

\sls{\subsection{Secure Multi-party Learning}}
\label{sec.mpc}
\sls{Several MPC protocols and MPL frameworks} have been proposed that utilize one or more cryptographic primitives such as HE, GC and SS to train machine learning models over multi-party with privacy preservation.
Each of these cryptographic primitives offers its owned characteristics and trade-offs, so the frameworks based on them also have corresponding advantages and disadvantages.

\subsubsection{HE-based MPL Frameworks}
\label{secsec.HE_MPC}
HE~\cite{rivest1978data} is a form of encryption where one can perform a specific algebraic operation on ciphertext directly, without decrypting it and knowing any information about the private key.
Then it generates an encrypted result, whose decryption result exactly matches the result of the same operation that performed on plaintext.

\sls{
HE can be categorized into three types~\cite{acar2018survey}. Partially Homomorphic Encryption (PHE)~\cite{rivest1978method}\cite{elgamal1985public} allows only one type of operation (addition or multiplication) with an unlimited number of times. One popular example is Additively Homomorphic Encryption (AHE) of Paillier scheme~\cite{paillier1999public}, which can only perform additive operations.
To do both additions and multiplications among ciphertexts, Somewhat Homomorphic Encryption (SWHE)~\cite{boneh2005evaluating} and Fully Homomorphic Encryption (FHE)~\cite{van2010fully}\cite{gentry2009fully}\cite{smart2010fully}\cite{coron2011fully} can be used.
SWHE can perform some types of operations within a limited number of times while FHE can handle all the operations without a limited number of times.
\sls{However, FHE's computation complexity is much more expensive than SWHE and PHE.}
}

By performing calculations on encrypted data directly, HE can be used to ensure the security of the computing process.
Naturally, HE can be adopted to protect data privacy through calculating and communicating on ciphertexts directly when training machine learning models.
Here, as shown in Figure~\ref{fig.HE}, take secure two-party computation as an example, one party $P_0$ generates a key-pair $(P_k,S_k)$ for the homomorphic cryptosystem and sends the public key $P_k$ together with his encrypted message $M_0$ to the other party $P_1$, who then evaluate the Arithmetic circuit under encryption using the homomorphic properties of the cryptosystem with $M_0$.
Finally, $P_1$ sends back the encrypted result $r$ which $P_0$ can decrypt using its private key.
Thus, the unencrypted data themselves are not transmitted, nor can they be guessed by other parties, there is only a little possibility of leaking the original data.
\sls{Moreover, HE methods are widely applied in MPL frameworks to generate multiplication triples in the pre-computation phase~\cite{mohassel2017secureml}, which can efficiently reduce the communication overhead in the online phase. }
Note that non-linear functionalities such as the ReLU, Sigmod activation functions cannot be supported by HE.

\begin{figure}[ht]
    \centering
    \includegraphics[scale=0.7]{pictures/HE.pdf}
    \caption{Two parties calculate a function with HE, where $(P_k,S_k)$ is a key-pair generated by $P_0$, $m_1$, $M_0$ and $m_1$, $M_1$ is the raw data and encrypted data of $P_0$ and $P_1$ respectively, r and R is the encrypted and unencrypted result, respectively.}
    \label{fig.HE}
\end{figure}

\sls{Wu et al.~\cite{wu2013privacy} trained a privacy-preserving logistic regression model using linear HE. However, the time complexity of linear HE increases exponentially with the number of parameters.
CryptoNets~\cite{gilad2016cryptonets}, CryptoDL~\cite{hesamifard2017cryptodl} and SEALion~\cite{van2019sealion} are HE-based frameworks for Convolutional Neural Networks (CNN for short) inference while cannot support training process.
Giacomelli et al.~\cite{giacomelli2018privacy} proposed a approach that can train a linear regression model using only linearly HE in the vertically-partitioned setting.}

\subsubsection{GC-based MPL Frameworks}
GC~\cite{yao1986generate}\cite{huang2011faster}, also known as Yao's garbled circuit, is an underlying technology of secure two-party computation, originally proposed by Andrew Yao. GC provides an interactive protocol for two parties (a garbler and an evaluator) obliviously evaluating an arbitrary function which is represented as a Boolean circuit.

In the construction of classical GC includes three main phases, garbling, transferring, and evaluation. Firstly, for each wire $i$ of the circuit, the garbler generates two random strings $k_i^0$ and $k_i^1$ as labels to represent the two possible bit values “0” and “1” for the wire, respectively.
For each gate in the circuit, the garbler creates a truth table.  Each output of the truth table is encrypted using two labels corresponding to its input.
This is done by the garbler to choose a key derivation function that generates symmetric keys using the two labels. Then the garbler permutes the rows of the truth table. After the garbling phase, the garbler transfers the garbled tables, together with the input wire labels corresponding to his input bits to the evaluator. Moreover, the evaluator acquires the labels corresponding to her input securely by Oblivious Transfer~\cite{beaver1996correlated}\cite{ishai2003extending}\cite{asharov2013more}.
With the garbled tables and labels of the input wires, the evaluator is in charge of decrypting the garbled tables iteratively, until getting the final result of the function.

\sls{The computation and communication costs of GC protocol only depend on the total number of AND gates in the circuit as the local computation for XOR gates can be ignored. Regardless of the depth of the circuit and the functionality,
GC executes in a constant number of rounds.
However, GC based protocols employ a complex mechanism to transform a function into a Boolean circuit. And these protocols calculate a function bit-by-bit, which leads to significant overhead in communication costs for several operations, especially multiplication.}
Recently, researchers have proposed many variants of original GC protocol to improve the performance of GC.
The point-and-permute mechanism proposed by Beaver et al.~\cite{beaver1990round} reduce the number of encryptions. The free XOR methods presented by Kolesnikov~\cite{kolesnikov2008improved} improve the computational efficiency of XOR gates and reduce the size of garbled circuits. Pinkas et al.~\cite{pinkas2009secure} proposed a garbled-row reduction method called 4-2 GRR to reduce the size of a garbled table from four to two ciphertexts, which is not compatible with free XOR. Half gates~\cite{zahur2015two} reduces the number of ciphertexts from four to two with free XOR in AND gates.

The first implementation of GC protocol, which supports evaluating generic function securely,  was Fairplay~\cite{malkhi2004fairplay} framework in the semi-honest setting.
Fairplay allowed developers to specify the function to be computed securely with a high-level language, SFDL, which was compiled and optimized into a Boolean circuit stored in a file.
FairplayMP~\cite{ben2008fairplaymp} extended the original Fairplay framework to multiple parties.
Mohassel et al.~\cite{mohassel2015fast} introduced a GC-based protocol for three-party computation with \sls{security} guarantee for malicious adversaries.
TinyGarble~\cite{songhori2015tinygarble} used a sequential circuit description to generate compact and efficient Boolean circuits.
ObliVM~\cite{liu2015oblivm} offered a programming framework for secure computation.
\cite{gascon2017privacy} designed a MPL framework based on Yao's GC to train a linear regression model over vertically partitioned data distributed among an arbitrary number of
parties.
Rouhani et al. presented DeepSecure~\cite{rouhani2018deepsecure} framework for deep learning inference based on GC protocol.

\subsubsection{SS-based MPL Frameworks}
\sls{SS schemes are crucial tools in cryptography and are used as building blocks in many security protocols.
SS-based protocols such as Additive SS~\cite{bogdanov2008sharemind}, Shamir's SS~\cite{shamir1979share}, GMW (Goldreich-Micali-Wigderson)~\cite{goldreich2019play}, BGW (Ben-Or-Goldwasser-Wigderson)~\cite{ben2019completeness}, and SPDZ~\cite{damgaard2012multiparty}, \hq{are} suitable for multiple parties.
In SS-based MPL frameworks, parties initially \hq{split their secret inputs as shares} using a SS scheme.}

The main idea of SS is to break a secret value into shares, each of which is distributed to a party.
As each party only owns part of the secret value, multiple parties must cooperate to reconstruct the secret value. The \sls{Additive} SS refers that the sum of the shares is the secret value. As a threshold protocol, Shamir's SS ensures that knowledge of any at least $k$ shares can reconstruct the secret, and nothing can be inferred about the secret if you obtain fewer than $k$ shares, where $k$ is the threshold. Beimel~\cite{beimel2011secret} gave a survey about
SS schemes.

GMW protocol is the first secure multi-party computation protocol, allowing an arbitrary number of parties to securely compute a function that can be represented as a Boolean circuit or Arithmetic circuit.
Take the Boolean circuit as an example.
All the parties share their inputs using the XOR-based SS scheme and parties interact to compute the result gate-by-gate.
GMW protocol evaluates a Boolean circuit as follows. Similar to Yao’s GC, for the XOR gates in the circuit, each party can locally XOR their shares using SS separately. The local calculations of each party can be ignored.
As for AND gates, evaluate each gate requires communicating among the parties and multiplication triples can be pre-computed using OT or its extensions~\cite{schneider2013gmw}. Thus, the performance of the GMW protocol depends on both the total number of AND gates (the number of OTs) in the circuit and the depth of the circuit.
GMW based protocols does not need to garble the truth table, and need only to perform XOR and AND operations for the calculation, so there is no symmetric encryption and decryption operations.
Besides, GMW based protocols allow to pre-compute all cryptographic operations, but require several rounds of interactions between multiple parties in online phase. Therefore, GMW achieves good performance in low-latency networks.

BGW protocol is the protocol for secure multi-party computation of Arithmetic circuits \sls{for 3+ parties}. The general structure of the protocol is similar to the GMW.
Parties initially share their inputs using a linear SS scheme (ususlly Shamir's SS), then compute the result gate-by-gate, with the invariant that parties hold random shares of the values of the internal wires of the circuit.
In general, BGW can be used to compute any Arithmetic Circuit. Similar to GMW protocol,  for the addition gates in the circuit, the computation is free while for the multiplication gates, parties require interactions.
However, GMW and BGW are different in the form of interaction. Rather than using OTs to communicate among parties, BGW relies on linear SS (such as Shamir’s SS) which supports multiplication operations.
\sls{But BGW relies on honest-majority.}
BGW protocol can against semi-honest adversaries for up to $t < n/2$ corrupt parties except the scenarios trivial for two-parties and against malicious adversaries for up to $t < n/3$ corrupt parties.

SPDZ~\cite{damgaard2012multiparty}\cite{damgaard2013practical} is a protocol for dishonest majority multiparty computation presented by Damgård et al., which capable of supporting more than two parties in computing arithmetic circuits. It is split into an offline phase and an online phase.
The advantage of SPDZ is that expensive, public-key machinery can be offloaded in the offline phase while the online phase purely uses cheap, information-theoretically secure primitives.
SWHE is used to perform secure multiplication in the offline phase with constant-round. The online phase of SPDZ is linear-round, following the GMW paradigm, using secret sharing over a finite field to ensure security.
SPDZ could against malicious adversaries for up to $t\leq n$ corrupted parties, where $t$ is the number of adversaries, $n$ is the number of parties.

Bogdanov et al. presented Sharemind~\cite{bogdanov2008sharemind}, a \sls{MPL} framework based on additive SS, which was secure in the semi-honest adversary model.
To the best of our knowledge, Choiet et al.~\cite{choi2012secure} firstly implemented the GMW protocol for any number of parties in a semi-honest setting, rather than requiring an honest majority.
A runtime environment for executing secure programs via the SPDZ protocol in the preprocessing model was presented in~\cite{keller2013architecture}.
Cramer et al. presented SPD$\mathbb{Z}_{2^k}$~\cite{cramer2018spd} for malicious adversaries with a dishonest majority, that works over rings instead of fields. Damg{\aa}rd et al.~\cite{damgaard2019new} optimized~\cite{cramer2018spd} and implemented their protocols to FRESCO~\cite{fresco} framework to support private classification using decision trees and support vector machines (SVMs).
Makri et al. proposed EPIC~\cite{makri2017pics}\cite{makri2019epic}, which is secure against malicious adversaries and involve more than two parties. They used SPDZ to implement private image classification based on SVM.
\sls{But both \cite{damgaard2019new} and \cite{makri2019epic} only focused on secure inference.
Wagh et al. proposed SecureNN~\cite{wagh2019securenn}, which combined various SS schemes for training deep neural network (DNN for short) and CNN.
FLSAH~\cite{byali2020flash}, a 4-party framework designed by Byali et al., using Additive SS and two variants of SS to train multiple machine learning models.
Wagh et al.~\cite{wagh2020falcon} presented an efficient 3-party MPL framework that utilized SS technologies to support the training of DNN and CNN.}

\subsubsection{Mixed-Protocol based MPL Frameworks}
In addition to the aforementioned single-protocol \sls{MPL} frameworks, a few commonly used frameworks typically adopt the hybrid protocols, which combine two or more protocols to utilize the advantages and avoid the disadvantages of each.
For instance, essential idea behind the mixed protocol that combines HE and GC is to calculate operations that have an efficient representation as Arithmetic circuits (e.g., additions and multiplications) using HE and
operations that have an efficient representation as Boolean circuits (e.g., comparisons) using GC.
However, the conversion \hq{between different schemes of shares is not trivial and the} costs are relatively expensive.
\sls{Besides, there are also some frameworks which combine MPC with Differential Privacy (DP for short, \hq{introduced in} Section \ref{sec.dp}).}

TASTY compiler~\cite{henecka2010tasty} could automatically generate protocols based on HE and GC as well as combinations of both.
The framework of~\cite{kolesnikov2013systematic}, implemented in the TASTY combined AHE with GC to the application of  face-recognition.
\cite{nikolaenko2013privacy-preserving} combined HE and GC to learn a linear regression model.
Demmler et al.~\cite{demmler2015aby} proposed ABY, a mixed-protocol framework combining additive SS, GC and GMW for two-party computation in the semi-honest adversary model.
ABY3~\cite{mohassel2018aby3} extended ABY to three-party scenarios.
Trident~\cite{rachuri2019trident} improved ABY3 performance and extended it to the four-party setting.
SecureML framework~\cite{mohassel2017secureml}, presented by Mohassel et al., combined additive SS and GC in the online phase. It focused on both secure training and inference of various machine learning models like linear regression, logistic regression and neural networks (NN for short).
Chase et al.~\cite{chase2017private} combined SS and DP to train  DNN models.
Riazi et al. presented a mixed-protocol machine learning framework named Chameleon~\cite{riazi2018chameleon} that utilized GC, GMW, and additive SS for secure two-party computation.
QUOTIENT~\cite{agrawal2019quotient} implemented SS and Yao's GC to train DNN models with two parties in the semi-honest model.
\sls{BLAZE~\cite{patra2020blaze} combined GC and SS to perform secure training and secure inference for linear regression and logistic regression models and secure inference alone for DNN.}
Besides, several mixed-Protocol \sls{MPL} Frameworks were proposed only for secure NN inference, such as MiniONN~\cite{liu2017oblivious}, GAZELLE~\cite{juvekar2018gazelle}.

\subsection{Federated Learning}
\label{sec.fl}
Although the local data of \emph{data controllers} are not exchanged in FL frameworks, the parameters transmitted between the server and \emph{data controller}s may leak sensitive information~\cite{aono2017privacy}\cite{melis2019exploiting}\cite{zhu2019deep}\cite{li2019quantification}.
To protect the \emph{data controller}s' local data from being leaked and to protect the privacy of the intermediate data during the training process, a few privacy technologies are applied in the frameworks of FL to privately exchange parameters when \emph{data controller}s interact with the server.
In this paper, we classify FL framework as Non-cryptographic FL framework, DP-based FL framework, and SC-based FL framework in terms of the privacy protection mechanisms used in FL frameworks.

\subsubsection{Non-cryptographic FL framework}
There are lots of FL frameworks that focus on improving efficiency or solving the challenge of statistical heterogeneity, while ignoring the potential risk brought by exchanging plaintext parameters, i.e. following the original privacy-preserving ideas proposed by Google~\cite{mcmahan2016federated}.

Smith et al. proposed a systems-aware optimization framework named MOCHA~\cite{smith2017federated}, which combined FL with multi-task learning to handle the statistical challenges.
FedCS~\cite{nishio2019client}, a mobile edge computing framework for machine learning presented by Nishio et al., aimed to perform FL efficiently under the setting of heterogeneous \emph{data controller}s.
Liang et al. proposed LG-FEDAVG~\cite{liang2020think} combing local representation learning with FL. They showed that local models better deal with heterogeneous data and effectively learn fair representations that obfuscate protected attributes.

\subsubsection{DP-based FL frameworks}
\label{sec.dp}

DP~\cite{dwork2008differential}\cite{dwork2014algorithmic}\cite{dwork2006calibrating} is a privacy technique with strong information-theoretic guarantees to add noises to the data\cite{chaudhuri2009privacy}\cite{song2013stochastic}~\cite{abadi2016deep}. A dataset satisfying DP can resist any analysis of private data, in other words, the data adversaries obtained is almost useless for speculating other data in the same dataset.
By adding random noises to the raw data or the model parameters, DP provides statistical privacy guarantees for individual records, thereby making the data impossible to be restored to protect \emph{data controller}s' privacy.

\begin{figure}[htb]
\centering
\subfigure[central differential privacy (CDP)]{
\label{fig.DP_central}
\includegraphics[scale=0.7]{pictures/DP_central.pdf}
}
\subfigure[local differential privacy (LDP)]{
\label{fig.DP_local}
\includegraphics[scale=0.7]{pictures/DP_local.pdf}
}
\centering
\caption{The CDP and LDP. $M$ denotes the DP mechanism. $W_i$ denotes the model updates for each \emph{data controller} $P_i$. With CDP (a), the model updates aggregated by the centralized server (must be honest) and then perturb them. With LDP (b), the model updates are applied DP mechanisms then aggregated by the centralized server (may be dishonest). }
\label{fig.DP}
\end{figure}

Client-level DP can prevent any \emph{data controller}s from trying to reconstruct the private data of another \emph{data controller} by exploiting the global model.
In FL frameworks, the applications of DP can be classified into two categories: the central model of DP (CDP for short) and the local model of DP (LDP for short). As demonstrated in Figure \ref{fig.DP}, in the setting of CDP, the model update parameters generated in each round are firstly aggregated by the centralized server and then perturbed by centralized server. \sls{Simply put}, the model parameters are private to all other \emph{data controller}s except the centralized server, i.e. the server must be trusted.
In the setting of LDP, each \emph{data controller} applies a differentially private transformation to their data prior to sharing it with the centralized server. So the model parameters are both private to other \emph{data controller}s and the server, and the server may be dishonest.
LDP affords stronger privacy guarantees than CDP, but achieving it while maintaining utility is a challenge~\cite{bassily2017practical}.
Additionally, there exists a trade-off between privacy and model accuracy, as adding more noises results in a greater privacy guarantee but may compromise the model accuracy significantly. \cite{sweeney2002k}\cite{el2008protecting}, l-diversity~\cite{machanavajjhala2007diversity}\cite{zhou2011k} and t-closeness~\cite{li2007t} can also apply to FL for data privacy protection, that may be applicable to different machine learning problems.

~\cite{geyer2017differentially} applied CDP to FL frameworks which can maintain client-level DP by hiding the client's contributions during the training and made a trade-off between privacy and performance.
McMahan et al.~\cite{mcmahan2017learning} used a similar method to train Long Short-Term Memory (LSTM) models.
Bhowmick et al.~\cite{bhowmick2018protection} proposed new optimal LDP mechanisms to protect the parameters in FL. They considered a practical threat model, limiting the power of potential adversaries. This method had better model performance than strict local privacy and provided stronger privacy guarantees than CDP.
A novel framework based on LDP was proposed by Pihur et al.~\cite{pihur2018differentially} in federated settings, which provides both client-side and server-side privacy protections and improved model quality.

\subsubsection{SC-based FL frameworks}
Secure computation methods such as HE and MPC are widely used in FL frameworks, which only reveal the result of the computation to the involved parties, without revealing any additional information.

The basic knowledge about HE has already been mentioned in Section \ref{secsec.HE_MPC}.
In fact, the ways HE applied to FL frameworks are similar to that applied to \sls{MPL} frameworks except for some details.
In FL frameworks, HE is used to protecting the privacy of the model parameters (such as gradients) interacted between the party and the server rather than the data interacted between parties as HE applied in \sls{MPL} frameworks. \cite{aono2017privacy} applied AHE to preserve the privacy of gradients for providing security against the semi-honest centralized server in FL models.

MPC involves multiple parties and retains the original accuracy with a very high \sls{security} guarantee. It guarantees each party knows nothing except the results and can be applied in FL models for secure aggregation and protect local models. In the FL frameworks based on MPC, the centralized server cannot obtain any local information and local updates, but observe the exact aggregated results at each round. However, MPC technologies applied in FL frameworks will incur significant extra communication and computation costs. At present,  SS as a basic protocol in MPC is the most widely used in FL frameworks, especially, Shamir's SS.
\cite{bonawitz2017practical} presented a protocol based on Shamir's SS for securely aggregating updates.

\subsection{Comparisons}

\subsubsection{Comparison of technical routes}

\begin{table*}[]
    \centering
    \caption{Various technical routes along with their characteristics.}
    \label{table.protocols}
    \renewcommand\arraystretch{1.4}
    \begin{tabular}{|c|c|c|p{11cm}|}
        \hline
                                                                                & \multicolumn{2}{c|}{Technical Routes}                                                                                & \multicolumn{1}{c|}{Characteristics}                                                                                \\ \hline
        \multirow{28}{*}{\multirow{4}{*}{\sls{\begin{tabular}[c]{@{}c@{}}Secure Multi-party \\ Learning\end{tabular}}}}         & \multicolumn{2}{c|}{\multirow{6}{*}{\begin{tabular}[c]{@{}c@{}}HE-based \sls{MPL} \\ framework\end{tabular}}}                   &

        \begin{itemize}
            \item \textbf{Low} communication overhead.
            \item \textbf{Constant} number of communication rounds.
            \item \textbf{Huge} computational complexity due to expensive public-key operations in the online phase.
            \item \textbf{No} support of non-linear functions.
        \end{itemize} \\ \cline{2-4}
                                                                                & \multicolumn{2}{c|}{\multirow{8}{*}{\begin{tabular}[c]{@{}c@{}}GC-based \sls{MPL} \\ framework\end{tabular}}}                   &

        \begin{itemize}
            \item \textbf{Mostly} used to the scenario of \textbf{two-party} with \textbf{Low} computational complexity.
            \item \textbf{Constant} number of communication rounds which only depends on the number of \textbf{multiplication (AND) gates} in the circuit.
            \item \textbf{Expensive} communication overhead.
            \item \textbf{Requirement of OTs} for every input bit of each party and \textbf{symmetric cryptographic operations} in the online phase.

        \end{itemize}  \\ \cline{2-4}
                                                                                & \multirow{16.5}{*}{\begin{tabular}[c]{@{}c@{}}SS-based \sls{MPL} \\ framework\end{tabular}}   & \multirow{6}{*}{GMW}            &
        \begin{itemize}
            \item Suitable for \textbf{arbitrary} number of parties.
            \item \textbf{Pre-computation} of all cryptographic operations in the offline phase.
            \item The number of communication rounds depends on the number of \textbf{multiplication (AND) gates} in the circuit and \textbf{the depth} of the circuit.
            \item \textbf{Merely rely on OTs} to support multiplication operations.

        \end{itemize}  \\ \cline{3-4}
                                                                                &                                                                                & \multirow{5.5}{*}{BGW}          &
        \begin{itemize}
            \item Suitable for \sls{\textbf{3+}} number of parties.

            \item \textbf{Better performance} owing to performing multiplication using LSS rather than OTs or other expensive public-key operations.
            \item \textbf{Threshold SS mechanism} which requires the \textbf{honest majority} assumption.
        \end{itemize} \\ \cline{3-4}
                                                                                &                                                                                & \multirow{6}{*}{SPDZ}           &
        \begin{itemize}
            \item Suitable for \textbf{arbitrary} number of parties.
            \item Perform secure multiplication by \textbf{SWHE} in the offline phase, and the online phase following the GMW paradigm.

            \item Supporting \textbf{dishonest majority} but leading to more overhead than GMW due to the necessity to enforce correctness.
        \end{itemize}\\ \cline{2-4}
                                                                                & \multicolumn{2}{c|}{\multirow{3.5}{*}{\begin{tabular}[c]{@{}c@{}}Mixed-protocol based \\ \sls{MPL} framework\end{tabular}}}     &
        \begin{itemize}
            \item \textbf{Trade-off} between computational complexity and communication overhead.

            \item Relatively \textbf{expensive} conversion costs between different protocols.
        \end{itemize}\\ \hline
        \multirow{9.5}{*}{Federated Learning}                                                                            & \multicolumn{2}{c|}{\multirow{3.5}{*}{\begin{tabular}[c]{@{}c@{}}Non-cryptographic \\ FL framework\end{tabular}}}         &
        \begin{itemize}
            \item \textbf{No additional security guarantees} are provided for model parameters, which are exchanged in plaintext .
            \item \textbf{More efficient} than the other two technical routes.
        \end{itemize}\\ \cline{2-4}
                                                                                & \multicolumn{2}{c|}{\multirow{3}{*}{\begin{tabular}[c]{@{}c@{}}SC-based FL\\ framework\end{tabular}}}                   &
        \begin{itemize}
            \item \textbf{Significant extra} communication or computational cost.
            \item \textbf{Lossless} result and accuracy with a very high security guarantee.
        \end{itemize}\\ \cline{2-4}
                                                                                & \multicolumn{2}{c|}{\multirow{3}{*}{\begin{tabular}[c]{@{}c@{}}DP-based FL\\ framework\end{tabular}}}                   &

        \begin{itemize}
            \item Adding \textbf{noise} to the model parameters.
            \item \textbf{Cheap} computational cost and communication cost, but need a compromise of \textbf{accuracy}.
        \end{itemize} \\ \hline
    \end{tabular}
\end{table*}

Firstly, we discuss the advantages and disadvantages between \sls{MPL} and FL from the aspects of privacy preservation, ways of communication, communication overhead, format of data, the accuracy of trained models and application scenarios. Then we summarize the characteristics of different technical routes of \texttt{TMMPP}\xspace in Table \ref{table.protocols}.

\begin{itemize}
     \item \textbf{Privacy preservation}. The MPC protocols used in \sls{MPL} frameworks provide parties a much high guarantee of security. However, non-cryptographic FL frameworks exchange the model parameters between the \emph{data controller} and the server are in plaintext, the sensitive information may also be leaked.
     \item \textbf{Ways of communication.} The communication among \emph{data controller}s in \sls{MPL} is usually in the form of Peer-to-Peer without a trusted third party, while FL is generally in the form of Client-Server with a centralized server. In other words, every \emph{data controller}s in \sls{MPL} are equal in status, while the \emph{data controller} and the centralized server in FL are unequal.
    \item \textbf{Communication overhead.} Due to the communication among \emph{data controller}s can be coordinated by a centralized server, the communication overhead is smaller than the Peer-to-Peer form of \sls{MPL}, especially when the number of \emph{data controller}s is very large.
    \item \textbf{Format of data.} Currently, in the solution of \sls{MPL}, the non-IID settings are not considered. However, in the solution of FL, as each \emph{data controller} trains the model locally, it is easier to suitable the non-IID setting.
    \item \textbf{The Accuracy of trained models.} In \sls{MPL}, there is usually no loss of accuracy in the global model. But if FL utilize DP to protect privacy, the global model will usually have a certain loss of accuracy.
    \item \textbf{Application scenarios.} Combined with the above analysis, we can find that \sls{MPL} is more suitable for scenarios with higher security and accuracy, while FL is more suitable for scenarios with the requirement of higher performance for more \emph{data controller}s.
\end{itemize}

\subsubsection{Comparison of frameworks}
\label{sec.comparison}

\begin{table*}[]
    \caption{The comparison among fourteen MPL frameworks}
    \label{table.frameworks}
    \centering
    \renewcommand\arraystretch{1.3}
    \begin{tabular}{|c|c|c|c|c|c|c|}
    \hline
                                                    & Technical routes                                                                               & Parties supported & Data partitioning                                              & Threat model                                                      & Models supported                                                                                & Year \\ \hline
    \cite{wu2013privacy}                            & \multirow{3}{*}{\begin{tabular}[c]{@{}c@{}}HE-based MPL \\ framework\end{tabular}}             & 2                 & \textbackslash{}$^{(1)}$                                          & \textbackslash{}$^{(2)}$                                               & Logistic Regression                                                                                & 2013 \\ \cline{1-1} \cline{3-7}
    \cite{giacomelli2018privacy}                    &                                                                                & 2+                & \begin{tabular}[c]{@{}c@{}}Horizontal,\\ Vertical\end{tabular} & Semi-honest                                                       & Linear Regression                                                                                & 2018 \\ \hline
    \cite{gascon2017privacy}                        & \begin{tabular}[c]{@{}c@{}}GC-based MPL \\ framework\end{tabular}                              & 2                 & Vertical                                                       & Semi-honest                                                       & Linear Regression                                                                                & 2017 \\ \hline
    SecureNN~\cite{wagh2019securenn}                & \multirow{5}{*}{\begin{tabular}[c]{@{}c@{}}SS-based MPL \\ framework\end{tabular}}             & 3                 & Horizontal                                                     & \begin{tabular}[c]{@{}c@{}}Semi-honest, \\ Malicious\end{tabular} & DNN, CNN                                                                                & 2019 \\ \cline{1-1} \cline{3-7}
    FLASH~\cite{byali2020flash}                     &                                                                                & 4                 & Horizontal                                                     & \begin{tabular}[c]{@{}c@{}}Semi-honest,\\ Malicious\end{tabular}  & \begin{tabular}[c]{@{}c@{}}Linear Regression\\ Logistic Regression\\ DNN, BNN$^{(3)}$\end{tabular}              & 2020 \\ \cline{1-1} \cline{3-7}
    Falcon~\cite{wagh2020falcon}                    &                                                                                & 3                 & Horizontal                                                     & \begin{tabular}[c]{@{}c@{}}Semi-honest, \\ Malicious\end{tabular} & DNN, CNN                                                                                & 2020 \\ \hline
    \cite{nikolaenko2013privacy-preserving}         & \multirow{15}{*}{\begin{tabular}[c]{@{}c@{}}Mixed-protocol based \\ MPL framework\end{tabular}} & 2+                & Horizontal                                                     & \begin{tabular}[c]{@{}c@{}}Semi-honest, \\ Malicious\end{tabular} & Linear Regression                                                                                & 2013 \\ \cline{1-1} \cline{3-7}
    SecureML~\cite{mohassel2017secureml}            &                                                                                & 2                 & Horizontal                                                     & Semi-honest                                                       & \begin{tabular}[c]{@{}c@{}}Linear Regression\\ Logistic Regression\\ DNN\end{tabular}                       & 2017 \\ \cline{1-1} \cline{3-7}
    \cite{chase2017private}                         &                                                                                & 2+                & Horizontal                                                     & Semi-honest                                                       & DNN                                                                                & 2017 \\ \cline{1-1} \cline{3-7}
    Chameleon~\cite{riazi2018chameleon}             &                                                                                & 2                 & Horizontal                                                     & Semi-honest                                                       & DNN, CNN, SVM                                                                                & 2019 \\ \cline{1-1} \cline{3-7}
    ABY3~\cite{mohassel2018aby3}                    &                                                                                & 3                 & Horizontal                                                     & \begin{tabular}[c]{@{}c@{}}Semi-honest, \\ Malicious\end{tabular} & \begin{tabular}[c]{@{}c@{}}Linear Regression,  \\ Logistic Regression, \\ DNN, CNN\end{tabular}             & 2018 \\ \cline{1-1} \cline{3-7}
    QUOTIENT~\cite{agrawal2019quotient}             &                                                                                & 2                 & Horizontal                                                     & Semi-honest                                                       & DNN, CNN                                                                                & 2019 \\ \cline{1-1} \cline{3-7}
    Trident~\cite{rachuri2019trident}               &                                                                                & 4                 & Horizontal                                                     & \begin{tabular}[c]{@{}c@{}}Semi-honest, \\ Malicious\end{tabular} & \begin{tabular}[c]{@{}c@{}}Linear Regression,  \\ Logistic Regression, \\ DNN, CNN\end{tabular}             & 2019 \\ \cline{1-1} \cline{3-7}
    BLAZE~\cite{patra2020blaze}                     &                                                                                & 3                 & Horizontal                                                     & \begin{tabular}[c]{@{}c@{}}Semi-honest, \\ Malicious\end{tabular} & \begin{tabular}[c]{@{}c@{}}Linear Regression, \\ Logistic Regression,\\ (DNN$^{(4)}$)\end{tabular}              & 2020 \\ \hline
\multicolumn{7}{l}{$^{(1)}$ \cite{wu2013privacy} trains logistic regression model between two parties where one party holds feature values and another holds label velues.} \\
\multicolumn{7}{l}{$^{(2)}$ There is no threat model mentioned in \cite{wu2013privacy}.} \\
\multicolumn{7}{l}{$^{(3)}$ BNN refers to Binarized Neural Network.} \\
\multicolumn{7}{l}{$^{(4)}$ \cite{patra2020blaze} performs both secure training and secure inference for linear regression and logistic regression and secure inference alone for DNN.}
\end{tabular}
\end{table*}

\begin{table*}[]
    \caption{The comparison among eleven FL based machine learning frameworks}
    \label{table.flframeworks}
    \centering
    \renewcommand\arraystretch{1.4}
    \begin{tabular}{|c|c|c|c|c|c|}
        \hline
                                                            & Technical routes                       & Parties supported & Data partitioning & Models supported                                                                        & Year \\ \hline
        FedAvg~\cite{mcmahan2016federated}                                              & \multirow{6}{*}{\begin{tabular}[c]{@{}c@{}} Non-cryptographic  \\ FL framework\end{tabular}}                      & 2+                & Horizontal        & NN, CNN, LSTM                                                                           & 2016 \\ \cline{1-1} \cline{3-6}
        FSVRG~\cite{konevcny2016federated1}                 &                                        & 2+                & Horizontal        & \begin{tabular}[c]{@{}c@{}}Linear Regression,\\ Logistic Regression,\\ SVM\end{tabular} & 2016 \\ \cline{1-1} \cline{3-6}
        MOCHA~\cite{smith2017federated}                                               &                                        & 2+                & Horizontal        &  \textbackslash{}                                                                              & 2017 \\ \cline{1-1} \cline{3-6}
        FedCS~\cite{nishio2019client}                                               &                                        & 2+                & Horizontal        & CNN                                                                                & 2018 \\ \cline{1-1} \cline{3-6}
        LG-FEDAVG~\cite{liang2020think}                                           &                                        & 2+                & Horizontal        & CNN                                                                                & 2019 \\ \cline{1-1} \cline{3-6}
        FedBCD~\cite{liu2019communication}                                              &                                        & 2+                & Vertical          & NN                                                                                & 2019 \\ \hline
        Federated Secure aggregation~\cite{bonawitz2017practical}                        & \multirow{3}{*}{SC-based FL framework} & 2+                & Horizontal        & NN                                                                                & 2017 \\ \cline{1-1} \cline{3-6}
        SecureBoost~\cite{cheng2019secureboost}                                         &                                        & 2+                & Vertical          & Decision Trees                                                                          & 2019 \\ \cline{1-1} \cline{3-6}
        VerifyNet~\cite{xu2019verifynet}                                           &                                        & 2+                & Horizontal        & CNN                                                                                & 2019 \\ \hline
        DP-FedAvg~\cite{mcmahan2017learning}                                           & \multirow{4}{*}{DP-based FL framework} & 2+                & Horizontal        & LSTM                                                                                & 2018 \\ \cline{1-1} \cline{3-6}
        LDP-FedSGD~\cite{zhao2020local}                      &                                        & 2+                & Horizontal        & \begin{tabular}[c]{@{}c@{}}Linear Regression,\\ Logistic Regression,\\ SVM\end{tabular} & 2020 \\ \hline
    \end{tabular}
\end{table*}

We investigated fourteen \sls{MPL} frameworks and eleven FL frameworks, all of which implement multiple parties training machine learning models with privacy preservation. We group these \sls{MPL} frameworks into four technical routes according to the description in Section \ref{sec.mpc} as well as grouping the FL frameworks into three technical routes. We also note that the number of parties supported, the types of machine learning model supported, and the way the data are partitioned in a framework.

The above information about thirteen \sls{MPL} frameworks and eleven FL frameworks are summarized in Table \ref{table.frameworks} and Table \ref{table.flframeworks}, respectively. From Table \ref{table.frameworks} and Table \ref{table.flframeworks}, we can conclude as follows.

\begin{itemize}
    \item \sls{MPL} frameworks usually use tailored protocols and only support a certain number of participants, that is, poor scalability. In addition, the participating parties involved in the current framework are generally two or three parties, when there are more parties even dozens of them, the efficiency of the frameworks drops sharply.

    \item Non-cryptographic FL frameworks exchange the raw model parameters without any \sls{security} guarantees which might leak sensitive information. The privacy of \emph{data controller}s is under threat if there is a powerful attack on the intermediate results during the training of machine learning models.

    \item The existing frameworks including both \sls{MPL} frameworks and FL frameworks usually consider a scenario of horizontal data partitioning while very few works consider the vertically partitioned data. We argue that the processing of training data and the design of the framework in horizontal data partitioning is relatively easier than vertical data partitioning. In addition, those frameworks that claim to support vertical data partitioning can only train simple machine learning models such as linear regression. Considering the fact that vertical data partitioning is also common and important in the real-world, it remains promising to train complex machine learning models efficiently upon vertical data partitioning.

    \item Most of the current frameworks are  targeted at training parametric models, such as linear regression, logistic regression, NN, while a few frameworks are designed to train non-parametric models such as decision trees. In the real-world, these non-parametric models are also widely used, so extending these frameworks to support more non-parametric models is also an interesting direction in the future.
\end{itemize}