\section{Introduction}

It has been shown that neural networks are vulnerable to adversarial examples~\citep{szegedy2016rethinking,goodfellow2015explaining,carlini2017towards,athalye2018obfuscated}.
Given a victim neural network model and a correctly classified example, an adversarial attack aims to compute a small perturbation such that with this perturbation added, the original example will be misclassified.
Many adversarial attacks have been proposed in the literature.
Most of them consider the white-box setting, where the attacker has full knowledge about the victim model, and thus gradient based optimization can be used for attack. Popular Examples include
C\&W~\citep{carlini2017towards} and PGD~\citep{madry2017towards} attacks.
On the other hand,
some more recent attacks have considered the probability black-box  setting where the attacker does not know the victim model's structure and weights, but can iteratively query the model and get the corresponding probability output.
In this setting, although gradient (of output probability to the input layer) is not computable, it can still be estimated using finite differences, and
algorithms many attacks are based on this~\citep{chen2017zoo, ilyas2018black,tu2018autozoom,jun2018adversarial}.

In this paper, we consider the most challenging and practical attack setting -- hard-label black-box setting -- where the model is hidden to the attacker and the attacker can only make  queries and get the corresponding hard-label decisions (e.g., predicted labels) of the model.  A commonly used algorithm proposed in this setting, also called Boundary attack~\citep{brendel2017decision}, is based on random walks on the decision surface, but it does not have any convergence guarantee. More recently, \cite{cheng2018queryefficient} showed that finding the minimum adversarial perturbation in the hard-label setting can be reformulated as  another optimization problem (we call this Cheng's formulation in this paper). This new formulation enjoys the benefit of having a smooth boundary in most tasks and the function value is computable using hard-label queries. Therefore, the authors of~\citep{cheng2018queryefficient}
are able to use standard zeroth order optimization to solve the new formulation.  Although their algorithm converges quickly, it still requires large number of queries (e.g., 20,000) for attacking a single image since every function evaluation of Cheng's formulation has to be computed using binary search requiring tens of queries.

In this paper, we follow the same optimization formulation of~\citep{cheng2018queryefficient} which has the advantage of smoothness, but instead of using finite differences to estimate the magnitude of directional derivative,
we propose to evaluate its sign using only {\bf a single query}. With this single-query sign oracle, we design novel algorithms for solving the Cheng's formulation, and we theoretically prove and empirically demonstrate the significant reduction in the number of queries required for hard-label black box attack.

Our contribution are summarized below:
\begin{compactitem}
 \item  {\bf Novelty in terms of adversarial attack.} We elucidate an efficient approach to compute the sign of directional derivative of Cheng's formulation using a single query, and based on this technique we develop a novel optimization algorithm called Sign-OPT for hard-label black-box attack.
    \item {\bf Novelty in terms of optimization.}
 Our method
can be viewed as a new zeroth order optimization algorithm that features fast
convergence of signSGD. Instead of directly taking the sign of gradient estimation, our algorithm utilizes the scale of
random direction. This make existing analysis inappropriate to our case, and we provide a new recipe to prove the convergence of this new optimizer.

    \item We conduct comprehensive experiments on several datasets and models. We show that the proposed algorithm consistently reduces the query count by 5--10 times across different models and datasets, suggesting a practical and query-efficient robustness evaluation tool. Furthermore, on most datasets our algorithm can find an adversarial example with smaller distortion compared with previous approaches.
\end{compactitem}

\section{Related Work}
\label{sec:related}
\vspace{-5pt}
\paragraph{White-box attack}
Since it was firstly found that neural networks are easy to be fooled by adversarial examples \citep{goodfellow2015explaining}, a lot of work has been proposed in the white-box attack setting, where the classifier $f$ is completely exposed to the attacker. For neural networks, under this assumption, back-propagation can be conducted on the target model because both network structure and weights are known by the attacker.
Algorithms including \citep{goodfellow2015explaining, kurakin2016adversarial, carlini2017towards, chen2018ead, madry2017towards} are then proposed based on gradient computation.
Recently, the BPDA attack introduced by \cite{athalye2018obfuscated} bypasses some models with obfuscated gradients and is shown to successfully circumvent many defenses. In addition to typical attacks based on small $\ell_p$ norm perturbation, non-$\ell_p$ norm perturbations such as scaling or shifting have also been considered~\citep{zhang2019limitations}.

\paragraph{Black-box attack}
Recently, black-box setting is drawing rapidly increasing attention. In black-box setting, the attacker can query the model but
has no (direct) access to any internal information inside the model. Although there are some works based on transfer attack \citep{papernot2017practical}, we consider the query-based attack in the paper.  Depending on the model's feedback for a given query, an attack can be classified as a soft-label or hard-label attack. In the soft-label setting, the model outputs a probability score for each decision.
\cite{chen2017zoo} uses a finite difference in a coordinate-wise manner to approximately estimate the output probability changes and does a coordinate descent to conduct the attack. \cite{ilyas2018black} uses Neural evolution strategy (NES) to approximately estimate the gradient directly. Later, some variants \citep{ilyas2018prior,tu2018autozoom} were proposed to utilize the side information to further speed up the attack procedure. \cite{alzantot2019genattack} uses a evolutionary algorithm as a black-box optimizer for the soft-label setting.
Recently, \cite{al2019there} proposes SignHunter algorithm based on signSGD \citep{pmlr-v80-bernstein18a} to achieve faster convergence in the soft-label setting.
The recent work  \citep{al2019there} proposes SignHunter algorithm to achieve a more query-efficent sign estimate when crafting black-box adversarial examples through soft-label information.

In the hard-label case, only the final decision, i.e. the top-1 predicted class, is observed. As a result, the attacker can only make queries to acquire the corresponding hard-label decision instead of the probability outputs. \cite{brendel2017decision} first studied this problem and proposed an algorithm based on random walks near the decision boundary. By selecting a random direction and projecting it onto a boundary sphere in each iteration, it aims to generate a high-quality adversarial example.  Query-Limited attack~\citep{ilyas2018black} tries to estimate the output probability scores with model query and turn the hard-label into a soft-label problem. \cite{cheng2018queryefficient} instead re-formalizes the hard-label attack into an optimization problem that finds a direction which could produce the shortest distance to decision boundary.

The recent arXiv paper \citep{chen2019boundary} applied the zeroth-order sign oracle to improve Boundary attack, and also demonstrated significant improvement. The major differences to our algorithm are that we propose a new zeroth-order gradient descent algorithm,  provide its algorithmic convergence guarantees, and aim to improve the query complexity of the attack formulation proposed in~\citep{cheng2018queryefficient}.  For completeness, we also compare with this method in Section \ref{ssec:hsja-sign-comp}.
Moreover, \citep{chen2019boundary}
 uses one-point gradient estimate, which is unbiased but may encounter larger variance compared with the gradient estimate in our paper. Thus, we can observe in Section \ref{ssec:hsja-sign-comp} that although they are slightly faster in the initial stage, Sign-OPT will catch up and eventually lead to a slightly better solution.

\section{Proposed Method}
\label{sec:proposed}

We follow the same formulation in
\citep{cheng2018queryefficient} and consider the hard-label attack as the problem of finding the direction with shortest distance to the decision boundary.
Specifically, for a given example $\bx_0$, true label $y_0$ and the hard-label black-box function $f: \RR^d \rightarrow \{1, \dots, K\}$, the objective function $g: \RR^d \rightarrow \RR$ (for the untargeted attack) can be written as:
\begin{equation}
    \min_{\btheta} g(\btheta) \ \text{ where } \ g(\btheta) = \arg\min_{\lambda>0} \bigg( f(x_0+\lambda\frac{\btheta}{\|\btheta\|})\neq y_0 \bigg).
    \label{eq:g_theta}
\end{equation}
It has been shown that this objective function is usually smooth and the objective function $g$ can be evaluated by a binary search procedure locally. At each binary search step, we query the function $f(\bx_0 + \lambda \frac{\btheta}{\|\btheta\|})$ and determine whether the distance to decision boundary in the direction $\btheta$ is greater or smaller than $\lambda$
based on the hard-label prediction\footnote{Note that binary search only works in a small local region; in more general case $g(\btheta)$  has to be computed by a fine-grained search plus binary search, as discussed in  \cite{cheng2018queryefficient}.}.

As the objective function is computable, the directional derivative of $g$ can be estimated by finite differences:
\begin{align}
    \hat{\nabla} g(\btheta;\ub):= \frac{g(\btheta+\epsilon \ub)-g(\btheta)}{\epsilon}\ub
    \label{eq:grad_test}
\end{align}
where $\bu$ is a random Gaussian vector and $\epsilon > 0$ is a very small smoothing parameter. This is a standard zeroth order oracle for estimating directional derivative and based on this we can apply many different zeroth order optimization algorithms to minimize $g$.
\begin{wrapfigure}{r}{0.4\textwidth}
    \centering
    \includegraphics[width=0.3\textwidth]{figures/illu.pdf}
    \caption{Illustration}
    \label{fig:illustration}
\end{wrapfigure}
For example, \cite{cheng2018queryefficient} used the Random Derivative Free algorithm~\cite{nesterov2017random} to solve problem~\eqref{eq:g_theta}.
However, each computation of  \eqref{eq:grad_test}
requires many hard-label queries due to binary search, so  \cite{cheng2018queryefficient} still requires a huge number of queries despite having fast convergence.

In this work, we introduce an algorithm that hugely improves the query complexity over~\cite{cheng2018queryefficient}. Our algorithm is based on the following key ideas: (i) one does not need very accurate values of directional derivative in order to make the algorithm converge, and (ii) there exists an {\bf imperfect but informative estimation} of directional derivative of $g$ that can be computed by a single query.

\subsection{A single query oracle}

As mentioned before, the previous approach requires computing $g(\btheta+\epsilon \bu) - g(\btheta)$ which consumes a lot of queries.
However, based on the definition of $g(\cdot)$,
we can compute the sign of this value $\text{sign}(g(\btheta + \epsilon \bu)-g(\btheta))$
using a single query. Considering the untargeted attack case, the sign can be computed by
\begin{equation}
\text{sign}(g(\btheta+\epsilon \ub)-g(\btheta))=
\begin{cases}
+1, & f(x_0 + g(\btheta)\frac{(\btheta+\epsilon \ub)}{\|\btheta+\epsilon \ub\|})=y_0 ,\\
-1, & \text{Otherwise.}
\end{cases}
\label{eq:est_sign}
 \end{equation}
This is illustrated in Figure \ref{fig:illustration}.
Essentially, for a new direction $\btheta+\epsilon\ub$, we test whether a point at the original distance $g(\btheta)$ from $x_0$ in this direction lies inside or outside the decision boundary,
i.e. if the produced perturbation will result in a wrong prediction by classifier. If the produced perturbation is outside the boundary i.e. $f(x_0 + g(\btheta)\frac{(\btheta+\epsilon \ub)}{\|\btheta+\epsilon \ub\|})\neq y_0$, the new direction has a smaller distance to decision boundary, and thus giving a smaller value of $g$. It indicates that $\ub$ is a descent direction to minimize $g$.
\subsection{Sign-OPT attack}
By sampling random Gaussian vector $Q$ times, we can estimate the imperfect gradient by
\begin{equation}
     \hat{\nabla} g(\btheta)\approx \hat{\bg}:= \sum\nolimits_{q=1}^Q\text{sign}(g(\btheta+\epsilon \ub_q) - g(\btheta))\ub_q,
     \label{eq:our_estimator}
\end{equation}
which only requires $Q$ queries.
We then use this imperfect gradient estimate to update our search direction $\btheta$ as $\btheta \leftarrow \btheta-\eta \hat{\gb}$ with a step size $\eta$ and use the same search procedure to compute $g(\btheta)$ up to a certain accuracy. The detailed procedure is shown in Algorithm \ref{alg:signopt}.

\begin{algorithm}[t]

\SetAlgoLined
\textbf{Input}: Hard-label model $f$, original image $x_0$, initial $\btheta_0$ \;
\For{$t=1, 2, \dots, T$}{
Randomly sample $u_1, \dots, u_Q$ from a Gaussian or Uniform distribution\;
Compute $\hat{\bg} \leftarrow \frac{1}{Q}\sum_{q=1}^Q \text{sign}(g(\btheta_t+\epsilon \ub_q) - g(\btheta_t))\cdot \ub_q$ \;
Update $\btheta_{t+1} \leftarrow \btheta_t - \eta \hat{\bg}$ \;
Evaluate $g(\btheta_t)$ using the same search algorithm in \cite{cheng2018queryefficient} \;
}
\caption{Sign-OPT attack}
\label{alg:signopt}
\end{algorithm}
We note that \cite{liu2018signsgd}
designed a Zeroth Order SignSGD
algorithm for soft-label black box attack (not hard-label setting).
They use $\hat{\nabla} g(\btheta)\approx \hat{\bg}:= \sum_{q=1}^Q\text{sign}(g(\btheta+\epsilon \ub_q) - g(\btheta)\ub_q)$ and
shows that it could achieve a comparable or even better convergence rate than zeroth order stochastic gradient descent by using only sign information of gradient estimation.
Although it is possible to combine ZO-SignSGD with our proposed single query oracle for solving hard-label attack,
their estimator will take sign of the whole vector and thus ignore the direction of $\ub_q$, which leads to slower convergence in practice (please refer to Section 4.4 and Figure 5(b) for more details).

To the best of our knowledge, no previous analysis can be used to prove convergence of Algorithm~\ref{alg:signopt}.
In the following, we show that Algorithm~\ref{alg:signopt} can in fact converge and furthermore, with similar convergence rate compared with~\citep{liu2018signsgd} despite using a different gradient estimator.
\begin{assumption}
Function $g(\theta)$ is L-smooth with a finite value of L.
\end{assumption}
\begin{assumption}
At any iteration step t, the gradient of the function g is upper bounded by $\|\nabla g(\btheta_t)\|_2 \leq \sigma$.
\end{assumption}
\begin{theorem}
Suppose that the conditions in the assumptions hold, and the distribution of gradient noise is
unimodal and symmetric. Then, Sign-OPT attack with learning rate $\eta_t = O(\frac{1}{Q\sqrt{dT}})$ and $\epsilon = O(\frac{1}{dT})$ will give following bound on $\EE[\|\nabla g(\btheta)\|_2]$:
\begin{align*}
     \EE[\|\nabla g(\btheta)\|_2] = O(\frac{\sqrt{d}}{\sqrt{T}} + \frac{d}{\sqrt{Q}}).
\end{align*}
\end{theorem}
The proof can be found in \autoref{ssec:proof}. The main difference with the original analysis provided by~\cite{liu2018signsgd} is that they only only deal with sign of each element, while our analysis also takes the magnitudes of each element of $\bu_q$ into account.

\subsection{Other gradient estimations}

Note that the value  $\text{sign}(g(\btheta+\epsilon \bu)-g(\btheta))$ computed by our single query oracle is actually the sign of the directional derivative:
\begin{equation*}
    \text{sign}(\langle \nabla g(\btheta), \bu\rangle ) = \text{sign}(\lim_{\epsilon\rightarrow \infty} \frac{g(\btheta+\epsilon \bu)-g(\btheta)}{\epsilon}) = \text{sign}(g(\btheta+\epsilon \bu) - g(\btheta))  \text{ for a small $\epsilon$.}
\end{equation*}
Therefore, we can use this information to estimate the original gradient. The Sign-OPT approach in the previous section uses $\sum_{q} \text{sign}(\langle \nabla g(\btheta), \bu_q \rangle) \bu_q$ as an estimation of gradient. Let $y_q := \text{sign}(\langle \nabla g(\btheta), \bu_q \rangle)$,
a more accurate gradient estimation can be cast as the following constraint optimization problem:
\begin{align*}
\text{Find a vector $\bz$  such that } \text{sign}(\langle \bz, \bu_q\rangle) = y_q \ \ \forall q=1, \dots, Q.
\end{align*}
Therefore, this is equivalent to a hard constraint SVM problem where each $\bu_q$ is a training sample and $y_q$ is the corresponding label. The gradient can then be recovered by solving
the following quadratic programming problem:
\begin{equation}
    \min_{\bz} \ \bz^T \bz \ \ \text{ s.t. } \ \  \bz^T \bu_q \geq y_q,\ \  \forall q=1,\dots, Q.
    \label{eq:svm}
\end{equation}
By solving this problem, we can get a good estimation of the gradient. As explained earlier, each $y_q$ can be determined with a single query. Therefore, we propose a variant of Sign-OPT, which is called SVM-OPT attack. The detailed procedure is shown in Algorithm \ref{alg:svmopt}. We will present an empirical comparison of our two algorithms in \autoref{ssec:svm-sign-comp}.
\begin{algorithm}[h]
\SetAlgoLined
\textbf{Input}: Hard-label model $f$, original image $\bx_0$, initial $\btheta_0$ \;
\For{$t=1, 2, \dots,T$}{
Sample $\bu_1, \dots, \bu_Q$ from Gaussian or orthogonal basis \;
Solve $\bz$ defined by \eqref{eq:svm} \;
Update $\btheta_{t+1} \leftarrow \btheta_t - \eta \bz$ \;
Evaluate $g(\btheta_t)$ using search algorithm in \citep{cheng2018queryefficient} \;
}
\caption{SVM-OPT attack}
\label{alg:svmopt}
\end{algorithm}

\vspace{-8pt}

\section{Experimental Results}
\label{sec:exp}
\vspace{-6pt}

\begin{figure}
     \centering
     \begin{subfigure}[b]{\textwidth}
         \centering
         \includegraphics[width=0.9\textwidth]{figures/example_mnist.pdf}

     \end{subfigure}

     \begin{subfigure}[b]{0.9\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/example_cifar_sign.pdf}

     \end{subfigure}

     \begin{subfigure}[b]{0.9\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/example_imgnt_car.pdf}

     \end{subfigure}
  \captionsetup{justification=centering}
  \caption{Example of Sign-OPT targeted attack. $L_2$ distortions and queries used are shown above and below the images. First two rows: Example comparison of Sign-OPT attack and OPT attack. Third and fourth rows: Examples of Sign-OPT attack on CIFAR-10 and ImageNet}
  \label{fig:untarget-example}

\end{figure}

We evaluate the SIGN-OPT algorithm for attacking black-box models in a hard-label setting on three different standard datasets - MNIST~\citep{lecun1998gradient}, CIFAR-10~\citep{cifar10} and ImageNet-1000~\citep{deng2009imagenet} and compare it with existing methods. For fair and easy comparison, we use the CNN networks provided by \citep{carlini2017towards}, which have also been used by other previous hard-label attacks as well. Specifically, for both MNIST and CIFAR-10, the model consists of nine layers in total - four convolutional layers, two max-pooling layers and two fully-connected layers. Further details about implementation, training and parameters are available on \citep{carlini2017towards}. As reported in \citep{carlini2017towards} and \citep{cheng2018queryefficient}, we were able to achieve an accuracy of 99.5\% on MNIST and 82.5\% on CIFAR-10. We use the pretrained Resnet-50 \citep{he2016deep} network provided by torchvision \citep{torchvision} for ImageNet-1000,  which achieves a Top-1 accuracy of 76.15\%.

In our experiments, we found that Sign-OPT and SVM-OPT perform quite similarly in terms of query efficiency. Hence we compare only Sign-OPT attack with previous approaches and provide a comparison between Sign-OPT and SVM-OPT in \autoref{ssec:svm-sign-comp}. We compare the following attacks:
\begin{itemize}
    \item \textbf{Sign-OPT attack} (black box): The approach presented in this paper.
    \item \textbf{Opt-based attack} (black box): The method proposed in \cite{cheng2018queryefficient} where they use Randomized Gradient-Free method to optimize the same objective function.
    We use the implementation provided at {\small \url{https://github.com/LeMinhThong/blackbox-attack}}.
    \item \textbf{Boundary attack} (black box):
    The method proposed in \cite{brendel2017decision}. This is compared only in $L_2$ setting as it is designed for the same. We use the implementation provided in Foolbox ({\small\url{https://github.com/bethgelab/foolbox}}).

    \item \textbf{Guessing Smart Attack} (black box): The method proposed in \citep{brunner2018guessing}. This attack enhances boundary attack by biasing sampling towards three priors. Note that one of the priors assumes access to a similar model as the target model and for a fair comparison we do not incorporate this bias in our experiments. We use the implementation provided at {\small \url{https://github.com/ttbrunner/biased_boundary_attack}}.
    \item \textbf {C\&W attack} (white box):
    One of the most popular methods in the white-box setting proposed in
     \cite{carlini2017towards}. We use C\&W $L_2$ norm attack as a baseline for the white-box attack performance.

\end{itemize}

For each attack, we randomly sample 100 examples from validation set and generate adversarial perturbations for them. For untargeted attack, we only consider examples that are correctly predicted by model and for targeted attack, we consider examples that are already not predicted as target label by the model.
To compare different methods,
we mainly use \textit{median distortion}
as the metric.
Median distortion for $x$ queries is the median adversarial perturbation of all examples achieved by a method using less than $x$ queries.
Since all the hard-label attack algorithms will start from an adversarial exmample and keep reduce the distortion, if we stop at any time they will always give an adversarial example and medium distortion will be the most suitable metric to compare their performance.
Besides, we also show
\textit{success rate (SR)} for $x$ queries for a given threshold ($\epsilon$), which is the percentage of number of examples that have achieved an adversarial perturbation below $\epsilon$ with less than $x$ queries.
We evaluate success rate on different thresholds which depend on the dataset being used.
For comparison of different algorithms in each setting, we chose the same set of examples across all attacks.

\textbf{Implementation details}: To optimize \autoref{alg:signopt}, we estimate the step size $\eta$ using the same line search procedure implemented  in~\cite{cheng2018queryefficient}. At the cost of a relatively small number of queries, this provides significant speedup in the optimization. Similar to \cite{cheng2018queryefficient}, $g(\theta)$ in last step of \autoref{alg:signopt} is approximated via binary search. The initial $\theta_0$ in \autoref{alg:signopt} is calculated by evaluating $g(\theta)$ on 100 random directions and taking the best one.
We provide our implementation publicly\footnote{https://github.com/cmhcbb/attackbox}.
\vspace{-5pt}
\subsection{Comparison between Sign-OPT and SVM-OPT}
\label{ssec:svm-sign-comp}
\vspace{-5pt}
In our experiments, we found that the performance in terms of queries of both these attacks is remarkably similar in all settings (both $L_2$/$L_\infty$ \& Targeted/Untargeted) and datasets. We present a comparison for MNIST and CIFAR-10 ($L_2$ norm-based) for both targeted and untargeted attacks in
\autoref{fig:svm_sign_comparison}. We see that the median distortion achieved for a given number of queries is quite on part for both Sign-OPT and SVM-OPT.

\textbf{Number of queries per gradient estimate}: In \autoref{fig:svm_sign_comparison}, we show the comparison of Sign-OPT attack with different values of $Q$. Our experiments suggest that $Q$ does not have an impact on the convergence point reached by the algorithm. Although, small values of $Q$ provide a  noisy gradient estimate and hence delayed convergence to an adversarial perturbation. Large values of $Q$, on the other hand, require large amount of time per gradient estimate. After fine tuning on a small set of examples, we found that $Q=200$ provides a good balance between the two. Hence, we set the value of $Q=200$ for all our experiments in this section.

\subsection{Untargeted attack}
\vspace{-5pt}
\begin{figure}
     \centering
     \includegraphics[width=0.9\textwidth]{figures/svm-param.pdf}
    \captionsetup{justification=centering}
    \caption{Median $L_2$ distortion vs Queries. First two: Comparison of Sign-OPT and SVM-OPT attack for MNIST and CIFAR-10. Third: Performance of Sign-OPT for different values of $Q$.}
    \label{fig:svm_sign_comparison}
\end{figure}

In this attack, the objective is to generate an adversary from an original image for which the prediction by model is different from that of original image. \autoref{fig:untargeted_distortion} provides an elaborate comparison of different attacks for $L_2$ case
for the three datasets. Sign-OPT attack consistently outperforms the current approaches in terms of queries. Not only is Sign-OPT more efficient in terms of queries, in most cases
it converges to a lower distortion than what is possible by other hard-label attacks. Furthermore, we observe Sign-OPT  converges to a solution comparable with C\&W white-box attack (better on CIFAR-10, worse on MNIST, comparable on ImageNet). This is significant for a hard-label attack algorithm since we are given very limited information.

We highlight some of the comparisons of Boundary attack, OPT-based attack and Sign-OPT attack ($L_2$ norm-based) in \autoref{u-l2-table}. Particularly for ImageNet dataset on ResNet-50 model, Sign-OPT attack reaches a median distortion below 3.0 in less than $30k$ queries while other attacks need more than $200k$ queries for the same.

\begin{figure}
     \centering
         \includegraphics[width=\textwidth]{figures_iclr/u_l2.pdf}

        \caption{Untargeted attack: Median distortion vs Queries for different datasets. }
        \label{fig:untargeted_distortion}
\end{figure}

\begin{figure}
     \centering
         \includegraphics[width=\textwidth]{figures_iclr/fig5.pdf}

        \caption{(a) Targeted Attack: Median distortion vs Queries of different attacks on MNIST and CIFAR-10. (b) Comparing Sign-OPT and ZO-SignSGD with and without single query oracle (SQO).}
        \label{fig:targeted_distortion}
\end{figure}

\subsection{Targeted attack}
\vspace{-5pt}
In targeted attack, the goal is to generate an adversarial perturbation for an image so that the prediction of resulting image is the same as a specified target. For each example, we randomly specify the target label, keeping it consistent across different attacks. We calculate the initial $\theta_0$ in \autoref{alg:signopt} using 100 samples in target label class from training dataset and this $\theta_0$ is the same across different attacks. \autoref{fig:untarget-example} shows some examples of adversarial examples generated by Sign-OPT attack and the Opt-based attack. The first two rows show comparison of Sign-OPT and Opt attack respectively on an example from MNIST dataset. The figures show adversarial examples generated at almost same number of queries for both attacks. Sign-OPT method generates an $L_2$ adversarial perturbation of 0.94 in $\sim6k$ queries for this particular example while Opt-based attack requires $\sim35k$ for the same. \autoref{fig:targeted_distortion} displays a comparison among different attacks in targeted setting. In our experiments, average distortion achieved by white box attack C\&W for MNIST dataset is 1.51, for which Sign-OPT  requires $\sim 12k$ queries while others need $> 120k$ queries. We present a comparison of success rate of different attacks for CIFAR-10 dataset in \autoref{fig:success_rate} for both targeted and untargeted cases.

\begin{figure}
     \centering
     \begin{subfigure}[b]{0.24\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures_iclr/success_u_l2_e1.pdf}

     \end{subfigure}

     \begin{subfigure}[b]{0.24\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures_iclr/success_u_l2_e2.pdf}

     \end{subfigure}
     \begin{subfigure}[b]{0.24\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures_iclr/success_t_l2_e1.pdf}

     \end{subfigure}
     \begin{subfigure}[b]{0.24\textwidth}
         \centering
         \includegraphics[width=\textwidth]{figures_iclr/success_t_l2_e2.pdf}

     \end{subfigure}

        \captionsetup{justification=centering}
        \caption{Success Rate vs Queries for CIFAR-10 ($L_2$ norm-based attack). First two and last two depict untargeted and targeted attacks respectively. Success rate threshold is at the top of each plot.}
        \label{fig:success_rate}

\end{figure}

\subsection{The power of single query oracle}
\vspace{-5pt}
In this subsection, we conduct several experiments to prove the effectiveness of our proposed single query oracle in hard-label adversarial attack setting. ZO-SignSGD algorithm  \citep{liu2018signsgd} is proposed for soft-label black box attack and we extend it into hard-label setting.
A straightforward way is simply applying ZO-SignSGD to solve the hard-label objective proposed in~\cite{cheng2018queryefficient}, estimate the gradient using binary search as \citep{cheng2018queryefficient} and take its sign. In Figure 5(b), we clearly observe that simply combining ZO-SignSGD and \cite{cheng2018queryefficient} is not efficient.
With the proposed single query sign oracle, we can also reduce the query count of this method, as demonstrated in Figure 5(b). This
verifies the effectiveness of single query oracle, which can universally improve many different optimization methods in the hard-label attack setting. To be noted, there is still improvement on Sign-OPT over ZO-SignSGD with single query oracle because instead of directly taking the sign of gradient estimation, our algorithm utilizes the scale of random direction $u$ as well. In other words, signSGD's gradient norm is always 1 while our gradient norm takes into account the magnitude of $u$. Therefore, our signOPT optimization algorithm is fundamentally different \citep{liu2018signsgd} or any other proposed signSGD varieties. Our method can be viewed as a new zeroth order optimization algorithm that features fast convergence in signSGD.

\section{Conclusion}
\vspace{-6pt}
We developed a new and ultra query-efficient algorithm for adversarial attack in the hard-label black-box setting. Using the same smooth reformulation in \cite{cheng2018queryefficient}, we design a novel zeroth order oracle that can compute the sign of directional derivative of the attack objective using single query. Equipped with this single-query oracle, we design a new optimization algorithm that can dramatically reduce number of queries compared with \cite{cheng2018queryefficient}. We prove the convergence of the proposed algorithm and show our new algorithm is overwhelmingly better than current hard-label black-box attacks.

\begin{table}[tbp]

  \centering
  \captionsetup{justification=centering}
    \caption{$L_2$ Untargeted attack - Comparison of average $L_2$ distortion achieved using a given number of queries for different attacks. SR stands for success rate.}
    \centering

  \resizebox{\textwidth}{!}{
  \begin{tabular}{lccc|ccc|ccc}
    \toprule
    \cmidrule(r){1-2}
    &  \multicolumn{3}{c}{MNIST} &\multicolumn{3}{c}{CIFAR10} & \multicolumn{3}{c}{ImageNet (ResNet-50)}\\
    &\#Queries &Avg $L_2$& SR($\epsilon=1.5$)&  \#Queries  & Avg $L_2$ & SR($\epsilon=0.5$)&\#Queries & Avg $L_2$ & SR($\epsilon=3.0$)\\
    \midrule
    \multirow{3}{*}{Boundary attack}
    &4,000      &4.24 &1.0\%    &4,000    &3.12 &2.3\%    &4,000   &209.63     &0\% \\
    &8,000      &4.24 &1.0\%    &8,000    &2.84 &7.6\%    &30,000    &17.40     &16.6\%\\
    &14,000     &2.13 &16.3\%   &12,000    &0.78 &29.2\%  &160,000    &4.62     &41.6\%\\
    \hline
    \multirow{2}{*}{OPT attack}
    &4,000  &3.65 &3.0\%    &4,000  &0.77   &37.0\% &4,000      &83.85     &2.0\%\\
    &8,000  &2.41 &18.0\%   &8,000  &0.43   &53.0\% &30,000     &16.77     &14.0\%\\
    &14,000 &1.76 &36.0\%   &12,000 &0.33   &61.0\% &160,000    &4.27     &34.0\%\\
   \hline
    \multirow{2}{*}{Guessing Smart}
    &4,000  &1.74 &41.0\%    &4,000  &0.29   &75.0\% &4,000      &16.69     &12.0\%\\
    &8,000  &1.69 &42.0\%   &8,000  &0.25   &80.0\% &30,000     &13.27     &12.0\%\\
    &14,000 &1.68 &43.0\%   &12,000 &0.24   &80.0\% &160,000    &12.88     &12.0\%\\
   \hline
    \multirow{2}{*}{\textbf{Sign-OPT attack}}
    &4,000  &1.54  &46.0\%  &4,000  &0.26  &73.0\%  &4,000      &23.19  &8.0\%\\
    &8,000  &1.18  &84.0\%  &8,000  &0.16  &90.0\%  &30,000     &2.99   &50.0\%\\
    &14,000 &1.09  &94.0\%  &12,000 &0.13  &95.0\%  &160,000    &1.21   &90.0\%\\
   \hline
      C\&W (white-box)
    &-   &0.88      &99.0\%  &-    &0.25 &85.0\%  &- &1.51& 80.0\%   \\
    \bottomrule
  \end{tabular}
}

  \label{u-l2-table}

\end{table}

\section*{Acknowledgement}
This work is based  upon  work  supported by the Department of Energy National Energy Technology Laboratory under Award Number DE-OE0000911 and by NSF under IIS1719097.

\newpage