\section{Introduction}

It can occur in some physical systems that they evolve into a state for which the coefficient of the highest derivative in the differential equation that governs the evolution of the system vanishes. When such state is reached the dynamical evolution experiences an abrupt change {----} the evolution may become unpredictable, some degrees of freedom may cease to exist, and information about the initial state can be lost \cite{Dds}. This may happen, for example, in compressible fluids when a wave front exceeds the speed of sound, as well as in shock-wave solutions of Burgers’ equation \cite{Choodnovsky}. In Lagrangian mechanics this corresponds to a globally non-invertible relation between velocities and momenta, which leads to multiple Hamiltonians for a given Lagrangian \cite{HTZ} and has recently been used  to  construct ``time crystals" \cite{Wilczek}. This problem is also a generic condition of gravitation theories in more than four spacetime dimensions \cite{TZ}.

Although in mechanics the problem of degeneracy usually appears as the multivaluedness of the velocity as a function of momentum, in its simplest form degeneracy appears in autonomous first order equations such as those describing a Hamiltonian flow.
\vskip.3cm

Recall the usual form of continuous dynamical systems
\begin{equation}
\frac{dx}{dt}=f(x),\quad x\in \bR^n, \quad t\in \bR.\label{eq:1}
\end{equation}
The {\it degenerate dynamical systems} have a modified form of (cf. \cite{Dds})
\begin{equation}
 A(x)\cdot \frac{dx}{dt}=f(x),\label{eq:2}
\end{equation}
where the matrix $A$ on the left-hand side can become singular on {certain {\it degeneracy set}, defined by}
\begin{equation*}
{D} = \{x\in \bR^n\,:\, \det A(x)=0\}.
\end{equation*}

A characteristic feature of degeneracy is that around degeneracy points, magnitude of the velocity $|\frac{dx}{dt}|$ can be infinitely large. This can be seen from the following $1$D example
\[
x\cdot \frac{dx}{dt}=1,\quad x\in \bR,
\]
which degenerates at $x=0$. This degeneracy point marks the sign-change of $\det A(x)=x$, which results in the velocity $\frac{dx}{dt}$ switching from $+\infty$ to $-\infty$ as $x$ moves from the positive to the negative. In general, for higher dimensional phase space ($n>1$), degeneracy sets ${D}$ typically form codimension-1 surfaces in phase space (cf. \cite{Dds}).

Mathematical studies of degenerate dynamical systems have been sparse. In piecewise smooth or discontinuous dynamical systems, it is generally assumed that the right-hand side $f$ of (\ref{eq:1}), defined on disjoint open domains of the phase space, can be continuously extended to their border (cf. \cite{Filippov_1988,Bernardo_2008,Hogan_2016}). On the other hand, degenerate dynamical systems {in which $\det A(x)$ has a simple zero are equivalent to those of the form} (\ref{eq:1}) whose right-hand side switches from $+\infty$ to $-\infty$ across the border of domain. In singular perturbation theory, ordinary differential equations  that contain vanishing coefficients being a parameter multiplying the highest order terms are studied. The zero-tending parameter is, however, generally assumed to be constant throughout the phase space and cannot capture the kind of degenerate dynamics caused by non-constant rank of pre-symplectic multipliers (cf. \cite{PS_2007}).

In this paper, we are interested in studying $2$D degenerate dynamical systems, where $A$  arises as a symplectic form
\[
A=\left(\begin{array}{cc} 0 &f\\ -f&0\end{array}\right),
\]
for a smooth function $f:\bR^2\to \bR$. The symplectic nature of $A$ is not a fundamental requirement for the present discussion, but it is motivated by physial considerations (cf. \cite{Dds}).

That is, we are interested in studying
\begin{equation}\label{eq:sym2d_gen}
\left(\begin{array}{cc} 0 & f\\ -f & 0\end{array}\right)
\left(\begin{array}{c} \dot x_1\\\dot x_2 \end{array}\right)
=\left(\begin{array}{c} E_1\\ E_2 \end{array}\right)=E,
\end{equation}
for a smooth function $E:\bR^2\to \bR^2$. The degeneracy points in this case, are precisely zeros of $f$, which under a regularity assumption, form a codimension-1 submanifold in the plane, being either an infinitely extending line or a circle.

The goal of this paper is to define a topological index {for the flow (\ref{eq:sym2d_gen}) in the presence of} these co-dimension one lines of degeneracy, which can be used to classify degenerate dynamics in the plane and on the two-dimensional sphere $S^2$. {Naturally}, degenerate dynamical systems with an empty degeneracy set become ordinary dynamical systems. As we will see, the introduced index gives an extension of the usual topological index for ordinary flows and provides a parallel of the Poincar\'e-Hopf {theorem} on the sphere for degenerate flows.

\section{Ring Index} \label{sec:ri}

Let $f:\bR^2\to \bR$ be a smooth map having $0$ as a regular value. Assume that $f^{-1}(0)\ne \emptyset$. Then,  by the Implicit Function Theorem, the pre-image
\[\gamma:=\{(x_1,x_2)\in\bR^2\,:\, f(x_1,x_2)=0\}\]
is a submanifold of co-dimension one. Every connected component of $\gamma$ is either  homeomorphic to an infinitely extending line $\bR$ or to a circle $S^1$.

The reason {for requiring regularity of} $\gamma$ is to avoid singular cases such as self-intersections, which can be thought of as an intermediate transition between two topologically distinct sets. See Figure \ref{F:self} for example, where a figure-eight curve can be perturbed to either a disjoint union of two circles ($S^1\sqcup S^1$) or to one circle ($S^1$).

	\begin{figure}[!htb]
\centerline{
		\includegraphics[width=1\textwidth]{self}
}

	\caption{{A curve with self-intersection (left) can be either perturbed to a disjoint union of two circles  (middle) or  one circle  (right).}}
	\label{F:self}
\end{figure}

Thus, no topological index that {remains} constant under small perturbations can be defined directly to such singular cases like the figure-eight. These can be studied in the context of topological bifurcations {and lies beyond} the scope of this paper.

We introduce first an index {for} compact $\gamma$ and then extend it to non-compact $\gamma$ using its compactfication on the sphere $S^2$.

Denote by $J=\left(\begin{array}{cc} 0 & -1\\ 1 & 0\end{array}\right)$. Then, (\ref{eq:sym2d_gen}) can be reformulated as
\begin{equation}\label{eq:sym2d_gen_JE}
f
\left(\begin{array}{c} \dot x_1\\\dot x_2 \end{array}\right)
=JE.
\end{equation}

\subsection{Definition}\label{subsec:def_ri}
\begin{Def}\rm A compact connected $1$-manifold $\gamma$ without boundary is called a {\it ring}. If $\gamma=f^{-1}(0)$ for a smooth map $f:\bR^2\to \bR$, then $f$ is chosen (with an appropriate sign) so that $J\nabla f$ coincides with the counter-clock wise orientation on $\gamma$.

\hfill\mbox{\raggedright$\Diamond$}
\end{Def}

\begin{ex}\label{ex:f_2}\rm
\begin{itemize}
	\item[(a)]	Let $\gamma=\{(x_1,x_2)\in \bR^2: x_1^2+x_2^2=1\}$ and $f=x_1^2+x_2^2-1$. Then, $\gamma=f^{-1}(0)$ and $J\nabla f$ is counter-clock wise on $\gamma$. See Figure \ref{F:f_ex_2} (left).
	\begin{figure}[!htb]
		\centerline{
			\includegraphics[width=.8\textwidth]{f_ex_2}
		}

		\vskip-1cm
		\caption{Left: Positions of $J\nabla f$ and $JE^-$ on $\gamma=\{(x_1,x_2)\in \bR^2: x_1^2+x_2^2=1\}$, where $f=x_1^2+x_2^2-1$ and $E^-=(E_1,E_2)=(-x_2,x_1)$. Right: Dynamics of (\ref{eq:sym2d_gen_JE}) around $\gamma$. }
		\label{F:f_ex_2}
	\end{figure}

\item [{(a)}] Consider $E^-:=(E_1,E_2)=(-x_2,x_1)$ for $(x_1,x_2)\in \bR^2$. Then, $JE^-=(-x_1, -x_2)$. The dynamics of (\ref{eq:sym2d_gen_JE}) around $\gamma$ can be obtained directly. It is characterized by an in-flow from both sides of $\gamma$ with an infinite velocity $(\dot x_1,\dot x_2)$ along $JE^-$. See Figure \ref{F:f_ex_2} (right).
\item [(b)] Consider $E^+=(E_1,E_2)=(x_2,-x_1)$ for $(x_1,x_2)\in \bR^2$. Then, $JE^+=(x_1, x_2)$. The dynamics of (\ref{eq:sym2d_gen_JE}) around $\gamma$ can be obtained similarly. It is characterized by an out-flow from both sides of $\gamma$ with an infinite velocity $(\dot x_1,\dot x_2)$ {along $JE^+$}. See Figure \ref{F:f_ex_3} (right).

 	\begin{figure}[!htb]
 	\centerline{
		\includegraphics[width=.8\textwidth]{f_ex_3.eps}
	}

	\vskip-1cm
 	\caption{Left: Positions of $J\nabla f$ and $JE^+$ on $\gamma=\{(x_1,x_2)\in \bR^2: x_1^2+x_2^2=1\}$, where $f=x_1^2+x_2^2-1$ and $E^+=(E_1,E_2)=(x_2,-x_1)$. Right: Dynamics of (\ref{eq:sym2d_gen_JE}) around $\gamma$.}
 	\label{F:f_ex_3}
 \end{figure}

\end{itemize}
\hfill\mbox{\raggedright$\Diamond$}
\end{ex}

\begin{rmk}\rm
	Note that the two vector fields $JE^\pm$ in Example \ref{ex:f_2}(a)-(b) are homotopic on $\gamma$ by a rotation of $180^\circ$. Indeed, they both define a map $\gamma\to S^1$ that has winding number $1$. However, since the degenerate dynamics they define through (\ref{eq:sym2d_gen_JE}) differs dramatically, we want to  introduce a topological index that can distinguish the two cases, by putting restrictions on the allowed deformations.
\hfill\mbox{\raggedright$\Diamond$}
\end{rmk}

The idea is to refrain $JE$ from taking the tangent direction of $\gamma$. For convenience, denote  by
$$m_\gamma(JE)=\big|\{(x_1,x_2)\in \gamma: \langle JE(x_1,x_2), \nabla f (x_1,x_2)\rangle=0\}\big|,$$
which is the number of points on $\gamma$ where $JE$ becomes tangent to $\gamma$ {and $\langle \cdot , \cdot \rangle$ is the Euclidean scalar product in $\bR$}. This number is by no means always finite. An extreme example is when $JE$ becomes tangent everywhere on $\gamma$, a case of {\it reducible degeneracy} (cf. \cite{Qds,Dds}). In example 2.2, $\langle JE^{{\pm}} , \nabla f \rangle=\pm 2$ and therefore $m_\gamma(JE)=0$.

\begin{lem}\label{lem:even} Consider the map $\langle JE, \nabla f\rangle:\gamma\to \bR$, which is a differentiable map between manifolds of the same dimension. If $0$ is a regular value of $\langle JE, \nabla f\rangle$, then there are an even number of zeros of $\langle JE, \nabla f\rangle$ on $\gamma$.

\end{lem}
\begin{proof} Consider the map $\langle JE, \nabla f\rangle$ as a map $g:[a,b]\to \bR$ such that $g(a)=g(b)$ {Here we used the {implicit assumption} that $\gamma$ is a closed curve, $\gamma \sim S^1$}. Then, the graph of $g$ must cross even number of times on the $x$-axis to come back the initial value $g(a)=g(b)$.

	Formally, $\bR$ is contractible, so any continuous map from $S^1\to\bR$ is homotopic to a (non-zero) constant map, whose number of zeros is zero. By the modulo $2$ degree of maps between manifolds of the same dimension, homotopic maps have the same number of zeros modulo $2$. Thus, $\langle JE, \nabla f\rangle$ must have an even number of zeros on $S^1$. See \cite{Milnor}.

\end{proof}

\begin{Def}\rm
A homotopy $H:[0,1]\times \gamma\to \bR^2\setminus\{0\}$ on vector fields $E$ is called {\it admissible}, if $m_{\gamma}(JH(t,\cdot))\equiv\text{\,constant\,}$ for all $t\in[0,1]$. Two vector fields $E_0,E_1$ are called {\it admissibly homotopic}, if there is an admissible homotopy $H$ connecting the two: $E_i=H(i,\cdot)$ for $i=0,1$.

Similarly, a homotopy $H:[0,1]\times S^1\to \bR^2$ on rings $\gamma$ is called {\it admissible}, if $m_{H(t,\cdot)} (JE)\equiv\text{\,constant\,}$ for all $t\in[0,1]$.
	\hfill\mbox{\raggedright$\Diamond$}
\end{Def}

\begin{Def}\rm\label{def:ring_ind} Let $\gamma=f^{-1}(0)\subset\bR^2$ be a ring and $E:\gamma\to \bR^2\setminus\{0\}$ be a vector field such that $0$ is a regular value of $\langle JE,\nabla f\rangle$. Let $m_\gamma(JE)$ be the (even) number of zeros of $\langle JE,\nabla f\rangle$.
	\begin{itemize}
		\item [(a)] If $m_\gamma(JE)=0$, then $\langle JE, \nabla f\rangle\ne 0$ for all points on $\gamma$. Define the {\it ring index} of $\gamma$ by
		\[\text{\,rind\,}_{\gamma}(E):=\frac{1}{l(\gamma)}\underset{s\in \gamma}{\oint}\text{\,sign\,} \langle JE, \nabla f\rangle ds,\]
		where $l(\gamma)$ is the length of $\gamma$ and {$\text{\,sign\,} (c)=\pm 1$ for $\pm c>0$.}

	\item[(b)]	If $m_\gamma(JE)>0$, then $\gamma$ is divided {into $m:=m_\gamma(JE)$ intervals $\gamma_1,\dots, \gamma_m$, between consecutive zeros}. Define the {\it ring index} of $\gamma$ by
	\[\text{\,rind\,}_\gamma(E)=\overset{m}{\underset{i=1}{\sum}} \text{\,rind\,}_{\gamma_i} (E).\]
	\end{itemize}
	\hfill\mbox{\raggedright$\Diamond$}
\end{Def}

\begin{ex}\rm\label{ex:rind_E_pm}
Let $E^\pm$ be the two vector fields given in Example \ref{ex:f_2}(a)-(b). Then, we have $m_\gamma(JE^\pm)=0$ and $\text{\,rind\,}_{\gamma}(E^\pm)=\pm 1$.
\hfill\mbox{\raggedright$\Diamond$}
\end{ex}

\begin{rmk}\rm
It is interesting to notice that
\[\langle JE,\nabla f\rangle=\langle E, -J\nabla f\rangle\]
by the ortho-normal property $-J^2=1$. Thus, the ring index is also equal to
\begin{equation*
\text{\,rind\,}_\gamma(E)=- \frac{1}{l(\gamma)} \oint_\gamma \text{\,sign\,} \langle E, J\nabla f \rangle ds.
\end{equation*}
If $m_\gamma(JE)=0$, then $\langle E, J\nabla f \rangle$ does not change sign on $\gamma$. Thus, we have
\[\text{\,rind\,}_\gamma(E)=- \text{\,sign\,}\big (\frac{1}{l(\gamma)} \oint_\gamma \langle E, J\nabla f \rangle ds\big).
\]
Since $J\nabla f$ is the unit tangent vector in alignment of the orientation of $\gamma$, the ring index expresses the work done {against $E$ along $\gamma$}.
\hfill\mbox{\raggedright$\Diamond$}
\end{rmk}

\begin{lem}\label{lem:r1} The ring index satisfies the following.
	\begin{itemize}
		\item [(a)] If $m_\gamma(JE)=0$, then $\text{\,rind\,}_\gamma(E) =\text{\,sign\,} \langle JE, \nabla f\rangle\in\{\pm 1\}$.
		\item[(b)] If $m_\gamma(JE)>0$, then $\text{\,rind\,}_\gamma(E) =0$.
	\end{itemize}
\end{lem}
\begin{proof} (a) Since $m_\gamma(JE)=0$, $\langle JE, \nabla f\rangle$ vanishes nowhere on $\gamma$ and $\text{\,sign\,}\langle JE, \nabla f\rangle \equiv \pm 1$ is a constant function. Thus,
	\[\text{\,rind\,}_\gamma(E)=\text{\,sign\,}\langle JE, \nabla f\rangle\frac{1}{l(\gamma)}\int_{s\in \gamma}ds=\text{\,sign\,}\langle JE, \nabla f\rangle.\]

\noindent(b) By Lemma \ref{lem:even}, $m_\gamma(JE)$ is an even number. Thus, $\gamma$ is divided into even number of  intervals, on each of which $\text{\,rind\,}_{\gamma_i}(E)\in\{\pm 1\}$ has an alternating sign. Therefore,  $\text{\,rind\,}_{\gamma}(E)=0$.

\end{proof}

\begin{proposition}\label{prop:ring_ind} The ring index is a homotopy invariant under all admissible homotopies of vector fields $E$ and of rings $\gamma$.
\end{proposition}
\begin{proof}
It follows from Lemma \ref{lem:r1}, since admissible homotopies keep the number $m_\gamma(JE)$ constant, under which condition the value of $\text{\,rind\,}_\gamma(E)$ remains the same.

\end{proof}

\begin{rmk}\rm \label{rmk_rev_rind}
The reverse of Proposition \ref{prop:ring_ind} does not hold in general. {That is, two admissibly homotopic vector fields do {\it not} necessarily have the same ring index.} Indeed, if $E_1,E_2$ are vector fields with $m_\gamma(JE_i)=2i>0$. Then, $\text{\,rind\,}_\gamma(E_i)=0$ for $i=1,2$. However, the two vector fields are not admissibly homotopic to each other, due to their different numbers $m_\gamma(JE_1)\ne m_\gamma(JE_2)$.
\hfill\mbox{\raggedright$\Diamond$}
\end{rmk}

It turns out that the reverse statement of Proposition \ref{prop:ring_ind} holds for vector fields with $m_\gamma(JE)=0$.

\begin{proposition}\label{prop:ring_ind_2}  Let $E^\pm$ be the two vector fields given in Example \ref{ex:f_2}(a)-(b). Then,  for any vector field $E$ with $m_\gamma(JE)=0$, we have
	\begin{itemize}
		\item[(i)] $\text{\,rind\,}_\gamma(E)=-1$ if and only if $E$ is admissibly homotopic to $E^-$;
		\item[(ii)] $\text{\,rind\,}_\gamma(E)=1$ if and only if $E$ is admissibly homotopic to $E^+$.
	\end{itemize}

\end{proposition}
\begin{proof} (i) If $\text{\,rind\,}_\gamma(E)=-1$, then by Lemma \ref{lem:r1}, $\langle JE,\nabla f\rangle<0$ on the whole $\gamma$. Thus, the map $\langle JE,\nabla f\rangle<0$ can be continuously deformed to the constant map $-1$ on $\gamma$, by an admissible homotopy. Since $\langle JE^-, \nabla f\rangle \equiv -1$, this homotopy leads to an admissible homotopy from $E$ to $E^-$. Conversely, if $E$ is admissibly homotopic to $E^-$, then at every $t\in [0,1]$, we have $\text{\,sign\,} \langle JH(t,\cdot), \nabla f\rangle=-1$. Thus, by Lemma \ref{lem:r1}(i), $\text{\,rind\,}_\gamma(H(t,\cdot))=-1$ for  all $t\in [0,1]$. Especially, we have $\text{\,rind\,}_\gamma(E)=-1$.

	The part (ii) is parallel.
\end{proof}

\begin{Def}\rm
	A vector field $E:\gamma \to \bR^2\setminus\{0\}$ is called {\it rotating}, if $m_\gamma(JE)=0$.
	\hfill\mbox{\raggedright$\Diamond$}
\end{Def}

By Proposition \ref{prop:ring_ind_2}, the ring index gives a topological classification of rotating fields under admissible homotopies.

\subsection{Relation to the Winding Number}\label{subsec_rel}

Consider $E:\gamma\to \bR^2\setminus\{0\}$. Recall that the {\it winding number}  $w(E,\gamma)$ of $E$ along $\gamma$ counts how many times the image $E(\gamma)\subset \bR^2\setminus\{0\}$ has gone around the origin counterclockwise. It can be calculated by
\[w(E,\gamma)=\frac{1}{l(\gamma)}\underset{(x_1,x_2)\in \gamma}\oint E_1dE_2-E_2dE_1. \]

\begin{ex}\rm\label{ex:wind_E_pm}
	Let $E^\pm$ be the two vector fields given in Example \ref{ex:f_2}(a)-(b). Then, the winding number of $E^\pm$ is in both cases,  $w(E^\pm, \gamma)=1$. To the contrary, the ring index distinguishes the two by being  $\text{\,rind\,}_\gamma(E^\pm)=\pm 1$ (cf. Example \ref{ex:rind_E_pm}).
	\hfill\mbox{\raggedright$\Diamond$}
\end{ex}

\begin{lem}\label{lem:rw} For rotating fields $E$,
we have $\text{\,rind\,}_\gamma(E) =\text{\,sign\,} \langle JE, \nabla f\rangle\cdot w(E,\gamma)$.
\end{lem}
\begin{proof} If $\langle JE, \nabla f\rangle\ne 0$ for all $(x_1,x_2)\in \gamma$, then $\langle JE, \nabla f\rangle <0$ or $>0$ always. Assume the first case.  By the homotopy invariance of the winding number, we have
\[w(E,\gamma)=w(\tilde E,\gamma),\]
where $\tilde E$ can be chosen to be a map such that $\langle JE, \nabla f\rangle \equiv -1$ on the whole $\gamma$. One such choice is  $\tilde E=E^-$. Thus, by  Proposition \ref{prop:ring_ind_2},
\[\text{\,rind\,}_\gamma(E) =\text{\,rind\,}_\gamma(E^-)=-1=-w(E^-,\gamma)=-w(E,\gamma).\]
The other case $\langle JE, \nabla f\rangle >0$ is similar.
 \end{proof}

\begin{cor}\label{cor:w1} For rotating fields $E$, we have $w(E,\gamma)=1$.
\end{cor}
\begin{proof} It is a consequence of Corollary \ref{lem:r1} and Lemma \ref{lem:rw}.
\end{proof}

\begin{lem}\label{lem:rw_ne_1} Every vector field with $w(E,\gamma)\ne 1$ has a ring index zero.

\end{lem}
\begin{proof}
Let $E$ be a vector field with $w(E,\gamma)\ne 1$. By Corollary \ref{cor:w1}, $E$ is not a rotating field and thus,  $m_\gamma(JE)>0$. By Lemma \ref{lem:r1}(b), $\text{\,rind\,}_\gamma(E)=0$.
\end{proof}

\subsection{Robustness of Degeneracy Rings}\label{subsec:robust}

By Remark \ref{rmk_rev_rind} and Lemma \ref{lem:rw_ne_1}, the ring index does not distinguish among vector fields having winding numbers other than $1$. These include vector fields of winding number $0$ (constant field), $-1$ (saddle) or $2$ (dipole).

We will show that vector fields with winding numbers other than $1$ cannot sustain a stable ring of degeneracy on a simply connected region. Thus, the only rings of degeneracy that exist robustly in $\bR^2$ or $S^2$ are those with non-zero ring index and supported by rotating fields.

\begin{ex}\rm \label{ex:E_pm_n}
Consider the representative vector fields for winding numbers $\pm n$ with $n=0,1,\dots$. For a parametrization $(\cos t, \sin t)$ of $\gamma=S^1$ with $t\in [0,2\pi]$, the vector fields $E_n=(-\sin(nt), \cos(nt))$ and $E_{-n}=(\sin(nt),\cos(nt))$ have winding numbers $w(E_n)=n$ and $w(E_{-n})=-n$, respectively. Moreover,
\begin{align*}
\langle JE_n, \nabla f \rangle&=-\cos(n-1) t\\
\langle JE_{-n}, \nabla f \rangle&=-\cos(n+1) t,
\end{align*}
where $f$ is chosen so that $J\nabla f=(-\sin t,\cos t)$ coincides with the tangent direction of $S^1$ in the counter-clock wise sense.
Thus,  the vector fields $E_{n+2}$ and $E_{-n}$ have the same number of zeros of $\langle JE,\nabla f\rangle$ on $S^1$, being equal to
\[m_\gamma(JE_{n+2})=m_\gamma(JE_{-n})=2(n+1).\]

{Conversely}, one can show that a vector field $E$ with $m_\gamma(JE)=2(n+1)$ is homotopic to either  $E_{n+2}$ or $E_{-n}$ by properties of winding numbers.
\hfill\mbox{\raggedright$\Diamond$}
\end{ex}

\begin{ex}\rm
	Consider a vector field $E$ with $m_\gamma(JE)=2$ for $\gamma=S^1$ and $f$ being chosen so that $J\nabla f=$ points at the tangent of $S^1$ in the counter-clock wise direction. The degenerate dynamics of (\ref{eq:sym2d_gen_JE}) is depicted in Figure \ref{F:2_zeros}.
	\begin{figure}[!htb]
	\centerline{
		\includegraphics[width=.25\textwidth]{2_zeros.eps}
	}
\vskip0cm
	\caption{Dynamics of (\ref{eq:sym2d_gen_JE}) for a vector field $E$ having $m_\gamma(JE)=2$ on  $\gamma=S^1$.}
	\label{F:2_zeros}
\end{figure}

The winding number of $E$ along $\gamma$ is either $0$ or $2$, being  homotopically to either the constant field $E_{0}$ or to the dipole $E_2$, respectively. See Figure \ref{F:2_zero_a}, where  $\tilde E_2$ is a variation of $E_2$ that has only simple zeros.

	\begin{figure}[!htb]
	\centerline{
		\includegraphics[width=1.1\textwidth]{2_zero_2.eps}
	}
	\vskip0cm
	\caption{Three kinds of vector fields $E_0$ (left), $E_2$ (middle) and $\tilde E_2$ (right) that have degenerate dynamics from Figure \ref{F:2_zeros}, where $\tilde E_2$ is homotopic to $E_2$ admissibly. All lead to unstable degeneracies. See Figure \ref{F:2_zero_b}.}
	\label{F:2_zero_a}
\end{figure}

The zeros on the ring in all cases {allow} the whole ring to {be continuously deformed collapsing it to a point.} The one side of in-flow finds its way out to the other side of out-flow. See figures \ref{F:2_zero_b}-\ref{F:2_zero_d}.

\begin{figure}[!htb]
	\centerline{
		\includegraphics[width=1\textwidth]{2_zero_b.eps}
	}
	\vskip0cm
	\caption{The disappearance of degeneracy for $E_0$.}
	\label{F:2_zero_b}
\end{figure}

\begin{figure}[!htb]
	\centerline{
		\includegraphics[width=1\textwidth]{2_zero_c.eps}
	}
	\vskip0cm
	\caption{The disappearance of degeneracy for $E_2$.}
	\label{F:2_zero_c}
\end{figure}

\begin{figure}[!htb]
	\centerline{
		\includegraphics[width=1\textwidth]{2_zero_d.eps}
	}
	\vskip0cm
	\caption{The disappearance of degeneracy for $\tilde E_2$.}
	\label{F:2_zero_d}
\end{figure}
\hfill\mbox{\raggedright$\Diamond$}
\end{ex}

In general, a non-zero even number of zeros of $\langle JE, \nabla f\rangle$ on the ring gives rise to the collapse of the  degeneracy ring, by contracting the in- and out-flow pair-wise. Therefore, we have

\begin{proposition}\label{prop:robust}
The only degeneracy rings that are robust against deformations are those with ring index $\pm 1$.
\end{proposition}
\begin{proof} Let $E$ be a vector field with a ring index different from $\pm 1$. Then, by Lemma \ref{lem:r1}, it has $\text{\,rind\,}_\gamma(E)=0$ and $m_\gamma(JE)>0$. By Lemma \ref{lem:even}, $m:=m_\gamma(JE)$ is even. By properties of winding number, $E$ is homotopic to either $E_{\frac m2+1}$ or $E_{-\frac m2+1}$ (cf. Example \ref{ex:E_pm_n} for notations). In either case, the degeneracy ring can be deformed away by pairing up in- and out-flow inside the ring.
\end{proof}

\section{2D Sphere}

Recall that the Euler characteristic of {$S^2$} can be realized by the sum of indices of all isolated zeros of any vector field on $S^2$  (having only isolated zeros), as stated by the Poincar\'e-Hopf theorem. Based on this result, flows on $S^2$ can be classified by the indices of their zeros and the sum of all indices is globally constrained by the topology of $S^2$.

We would like to establish a parallel of the Poincar\'e-Hopf theorem for degenerate flows on $S^2$ using degeneracy indices, which in the case of absence of degeneracy, reduces to the classical Poincar\'e-Hopf theorem.

Given a degenerate flow on $S^2$, we assume that the flow has only isolated zeros besides degeneracy rings. We also assume that the degeneracy rings do not intersect any of these zeros, nor do they intersect each other.

We will define the {\it degeneracy index} for degeneracy rings and for isolated zeros, respectively. {We use the symbol $(S^1)$ to label isolated zeros for their degeneracy index, to distinguish these $0$-dimensional manifolds from the $1$-dimensional degeneracy rings, which will be labeled by the symbol $({\mathbb{Z}}_1)$.  These notations are borrowed from the $S^1$-equivariant degree, where zero orbits of an $S^1$-equviariant map are labeled by their symmetries (orbit types) (cf. \cite{Ruan}). The idea is that a zero orbit of an $S^1$-equivariant map is $1$-dimensional if it has a discrete symmetry $({\mathbb{Z}}_m)$ for some finite subgroup ${\mathbb{Z}}_m\subset S^1$; or it is $0$-dimensional if it has the full  continuous symmetry $(S^1)$. }

 As we will see, this degeneracy index for isolated zeros differs from the classical index for isolated zeros. In particular, a source and a sink will have different degeneracy indices.

\subsection{Degeneracy Rings}\label{subsec:dr}

In Subsection \ref{subsec:def_ri}, {we} introduced a ring index for degeneracy rings, which takes value from $\pm 1$ for rotating fields $E$, or else it is equal to $0$. Also, as it has been shown in Subsection \ref{subsec:robust}, degeneracy rings enclosing a simply connected region are robust if and only if they have non-zero ring index.

Thus, there are in effect only two kinds of degeneracy rings on the two-dimensional sphere $S^2$, having ring index either $1$ or $-1$. To distinguish the degeneracy index for rings and for isolated zeros  later, we use symbols $({\mathbb{Z}}_1)$ to label them.

More precisely, a ring $\gamma$ is called an {\it annihilation} ring, if $\text{\,rind\,}_{\gamma}(E)=-1$; it is called a {\it creation} ring, if $\text{\,rind\,}_{\gamma}(E)=1$. Define {\it degeneracy index of $\gamma$} by
\begin{align}
\text{\rm Ind\,}: \quad \text{annihilation $\gamma$}\quad&\mapsto\quad -({\mathbb{Z}}_1)
\label{eq:di_ring}\\
\text{creation $\gamma$}\quad&\mapsto\quad \;\;\; ({\mathbb{Z}}_1).\notag
\end{align}
\\
See Figure \ref{fig:z1_pm} for the degenerate flows it implies.

\begin{figure}[!htb]
	\centerline{
		\includegraphics[width=.7\textwidth]{z1_pm.eps}
	}

\caption{Index of degeneracy rings: (left) $({\mathbb{Z}}_1)$ for creation ring; (right) $-({\mathbb{Z}}_1)$ for annihilation ring. The sign is determined by the ring index.}
\label{fig:z1_pm}
\end{figure}

\begin{rmk}\rm\label{rmk:ss_diff} Notice that a creation ring can naturally enclose a sink inside by extending all the in-flow arrows. Similarly, an annihilation ring can enclose a source inside. However, if a creation ring were to enclose a source inside, there needs to be some additional structure mediating the two.

\hfill\mbox{\raggedright$\Diamond$}
\end{rmk}

\subsection{Isolated Zeros}

By Subsection \ref{subsec:robust}, degeneracy rings that  enclose other isolated zeros  than sources or sinks cannot be robust. Thus, it is sufficient to define degeneracy index for sources and sinks.

The following observation provides a foundation for the definition.
\begin{lem}\label{lem:ss_ah} A sink is admissibly homotopic to an annihilation ring enclosing a source; a source is admissibly homotopic to a creation ring enclosing a sink (cf. Figures \ref{fig:ss_1} and \ref{fig:ss_2}).
\end{lem}

\begin{proof} To prove the first part, suppose that we have an annihilation ring enclosing a source. Example \ref{ex:f_2}(a) gives a vector field $E^-$ together with $f=x_1^2+x_2^2-1$ that carries such a degenerate flow. By Proposition \ref{prop:ring_ind_2}(i), every such degenerate flow is indeed admissibly homotopic to $E^-$. Define a homotopy $H(s,x_1,x_2)=x_1^2+x_2^2-s$ for $s\in [-1,1]$ of $f$. The zero set of $H_s:=H(s,\cdot)$ forms a ring of radius $\sqrt{s}$ for $s>0$, which disappears for $s<0$. It gives rise to an admissible homotopy on rings $\gamma_s:=H_s^{-1}(0)$ of degeneracy, since $m_{\gamma(s)}(JE^-)=0$ for all $s\in [-1,1]$. The deformation of flow is described by the following system
	\[(x_1^2+x_2^2-s)\left(\begin{array}{c} \dot x_1\\\dot x_2 \end{array}\right)=JE^-=\left(\begin{array}{c} -x_1\\-x_2 \end{array}\right),\quad s\in[-1,1],\]
which has a sink for $s=-1$ and an annihilation ring enclosing a source for $s=1$. See Figure \ref{fig:ss_1}.
\begin{figure}[!htb]
	\vskip-.5cm
	\centerline{
		\includegraphics[width=.8\textwidth]{ss_1.eps}
	}

	\caption{A sink (left) and an annihilation ring enclosing a source (right) are admissibly homotopic.}
	\label{fig:ss_1}
\end{figure}

The other case is analogous and can be proved using $E^+$ from Example \ref{ex:f_2}(b). See Figure \ref{fig:ss_2} for the dynamics.
\begin{figure}[!htb]
	\vskip-.3cm
	\centerline{
		\includegraphics[width=.8\textwidth]{ss_2.eps}
	}

	\caption{A source (left) and a creation ring enclosing a sink (right) are admissibly homotopic.}
	\label{fig:ss_2}
\end{figure}
\end{proof}

In other words, a sink can be viewed as an annihilation ring collapsed to a point. Similarly, a source can be viewed as a creation ring collapsed to a point.

It follows from Lemma \ref{lem:ss_ah} that degeneracy indices for a source and for a sink differ by a degeneracy ring with an appropriate sign. For example, if we associate $(S^1)$ to a source, then the index for a sink is equal to ($(S^1)-({\mathbb{Z}}_1)$), since it is the index of a source substracted by the index of a creation ring which is $({\mathbb{Z}}_1)$.

Define the {\it degeneracy index} of a source and a sink by the following.
\begin{align}
\text{\rm Ind\,}: \quad \text{source}\quad&\mapsto\quad (S^1)\label{eq:di_zero}\\
\text{sink}\quad&\mapsto\quad (S^1)-({\mathbb{Z}}_1).\notag
\end{align}

\subsection{Generalized  Poincar\'e-Hopf Theorem}

Based on the degeneracy index defined by (\ref{eq:di_ring})  and (\ref{eq:di_zero}), we have the following extension of Poincar\'e-Hopf theorem.

\begin{proposition}\label{prop:ind}
For every flow of (\ref{eq:sym2d_gen_JE}) generated by $f$ and $JE$ on $S^2$, we have
\begin{equation}\textcolor{red}{\label{eq:gt}}
\sum \text{\rm Ind\,}=2(S^1)-({\mathbb{Z}}_1),
\end{equation}
where {the sum} is taken over all isolated zeros and degeneracy rings of the flow.
\end{proposition}

\begin{proof} A {special} case that we consider first is when the flow does {\it not} possess any degeneracy rings, that is, a case of regular flow. Then, such flow is homotopic to a flow that contains only one source and one sink, for which we have
\[\sum \text{\rm Ind\,}=(S^1)+(S^1)-({\mathbb{Z}}_1)=2(S^1)-({\mathbb{Z}}_1).\]
\vskip.3cm
	Otherwise, consider a flow that has degeneracy rings. Assume, without loss of generality, that they are robust and do not intersect each other on $S^2$. It follows that they form parallel rings on $S^2$ and by Proposition \ref{prop:robust}, they have ring index $\pm 1$, {corresponding} to degeneracy index $\pm ({\mathbb{Z}}_1)$.
\vskip.3cm

Let $\gamma$ be a degeneracy ring that {encloses} no other degeneracy rings, that is the first or the last parallel ring on $S^2$. Depending on its degeneracy index, it contains either a source (for $-({\mathbb{Z}}_1)$-ring) or a sink (for $({\mathbb{Z}}_1)$-ring). By Lemma \ref{lem:ss_ah}, one can then deform the ring to the contained fixed point by an admissibly homotopy, which results in a sink or a source, in the respective two cases. Notice that throughout the deformation, the sum of the degeneracy index {does not change}.
\vskip.3cm
Repeat this process for the {remaining} degeneracy rings, deforming them one after another to their contained fixed points by admissible homotopies, which every time makes a ring disappear by {switching between a source and a sink}. {At the end}, one obtains a flow without degeneracy rings, {whose sum of degeneracy indices equals} $2(S^1)-({\mathbb{Z}}_1)$. Since the deformation procedure is achieved by {admissible} homotopies, the sum of degeneracy index {is preserved}. Thus, the sum of degeneracy index for the original flow is also equal to $2(S^1)-({\mathbb{Z}}_1)$.
\end{proof}

\subsection{Interpretation of  $2(S^1)-({\mathbb{Z}}_1)$}

The  sum ($2(S^1)-({\mathbb{Z}}_1)$) of degeneracy indices in Proposition \ref{prop:ind} describes the global topology of $S^2$ in the following way. Recall that the sum of ordinary indices  amounts to the Euler characteristic of $S^2$, which is given by an alternating sum using cells in a cellular decomposition of $S^2$.

In a similar way, one can use degeneracy index to distinguish different cells for the construction of $S^2$. See Figure \ref{fig:gt2}, where indices for $2$D cells are defined by their enclosed singularities (cf. Lemma \ref{lem:ss_ah}).
\begin{figure}[!htb]
	\vskip-.3cm
	\centerline{
		\includegraphics[width=.7\textwidth]{gt2.eps}
	}

				\vskip.2cm
	\caption{Cells of dimension $0$ and $2$ with their indices. The upper resp. lower row represents a cellular decomposition of $S^2$ (cf. (\ref{eq:dc_sk})-(\ref{eq:dc_sc})).}
	\label{fig:gt2}
\end{figure}

For example, consider a (cellular) decomposition of $S^2$ given by
\begin{equation}\label{eq:dc_sk}
S^2=S^2\setminus\{\text{sink}\}\sqcup \{\text{sink}\},
\end{equation}
which is a combination of the two cells in the upper row of Figure \ref{fig:gt2}.  Alternatively, one can consider the decomposition of $S^2$ given by
\begin{equation}\label{eq:dc_sc}
S^2=S^2\setminus\{\text{source}\}\sqcup \{\text{source}\},
\end{equation}
which corresponds to the combination of the two  cells in the lower row of Figure \ref{fig:gt2}.
Other decompositions involving $1$D cell are also possible. Consider the decomposition of $S^2$ by two $2$D cells and one $1$D (closed) cell
\begin{equation}\label{eq:dc_sn1}
S^2= B_N^+\sqcup B_S^+\sqcup S^{-},
\end{equation}
where $B_N^+$ resp. $B_S^+$ denotes the open northern resp. southern hemisphere of $S^2$ with {\it outgoing} flow on the boundary and $S^{-}$ denotes the equator of {\it incoming} flow from both sides (cf. Figure \ref{fig:gt3}, the upper row).
\begin{figure}[!htb]
	\vskip-.3cm
	\centerline{
		\includegraphics[width=.9\textwidth]{gt3.eps}
	}

	\vskip.2cm
	\caption{Cells of dimension $1$ and $2$ with their indices.  The upper resp. lower row represents a cellular decomposition of $S^2$ (cf. (\ref{eq:dc_sn1})-(\ref{eq:dc_sn2})).}
	\label{fig:gt3}
\end{figure}
Alternatively,
\begin{equation}\label{eq:dc_sn2}
S^2= B_N^-\sqcup B_S^-\sqcup S^{+},
\end{equation}
is another decomposition, where $B_N^-$ resp. $B_S^-$ denotes the open northern resp. southern hemisphere of $S^2$ with {\it incoming} flow on the boundary and $S^{+}$ denotes the equator of {\it outgoing} flow from both sides (cf. Figure \ref{fig:gt3}, the lower row).  Notice that in all the decompositions (\ref{eq:dc_sk})-(\ref{eq:dc_sn2}), the sum of indices of cells is equal to  $2(S^1)-({\mathbb{Z}}_1)$.

\section{Conclusion}

We have introduced a degeneracy index for $2$D flows and shown that it can be used to extend the Poincar\'e-Hopf Theorem for degenerate flows on $S^2$. But it remains unclear how it can be extended to other $2$D compact surfaces such as $g$-surfaces in a straighrforward way. The analysis of the robustness of degeneracy rings benefits from the simplicity of the topology of $S^2$ (cf. Subsection \ref{subsec:robust}). Also, the current discussion in $2$D is motivated by physics applications, but it is also interesting to explore higher dimensions. Careful readers may have noticed we have used the symbols $(S^1)$ and $({\mathbb{Z}}_1)$ to distinguish zeros and degeneracy rings on $S^2$. It is not clear, however, how this choice can be made in a canonical way and how to systematically define a degeneracy index for degenerate flows in  higher dimensional phase spaces.

\textbf{Acknowledgments}\\
The work of JZ has been partially funded by Fondecyt Grant No.1180368. The	Centro de Estudios Científicos (CECs) is funded by the Chilean	Government through the Centers of Excellence Base Financing Program of	Conicyt. HR thanks JZ for his invitation to visit CECs, during which the work has found its fast convergence.

\vskip.3cm


\section{Introduction}

With the rapid development of public available datasets (e.g. spoken term classification~\cite{Warden_GoogleSpeechCommand}, speaker identification~\cite{Nagrani17,Chung18b}, acoustic event classification/detection~\cite{AudioSet,DCASE2017challenge}, etc.), state-of-the-art models for various acoustic applications can be trained with a large amount of annotated data.
CNN-based architectures have achieved state-of-the-art performance in keyword spotting~\cite{Sainath15}, speech recognition~\cite{AbdelHamid2012CNN_SR,Qian16}, speaker identification~\cite{Nagrani17,Chung18b}, acoustic event classification~\cite{Takahashi2016DeepCN,Hershey2017,Shi2019,Tang2019}, and rare acoustic event detection~\cite{Lim2017,Kao18}.
CNNs have shown performance superior to feed-forward Deep Neural Networks (DNNs) in various acoustic applications due to the following reasons.
First, DNN is not good at modeling the strong correlations in time and frequency of acoustic signal.
Second, DNNs are not able to model shift of formants in speech signals.
Instead, CNNs are able to capture local patterns that model the correlations properly and detect shift of formants by sharing the weights of 2D kernels (\textit{time}$\times$\textit{feature}) across different locations in the input feature space.

An important property of CNN is shift invariance (also known as spatial invariance), which allows CNN to detect patterns even if it does not appear at exactly the same location as samples seen in the training set.
Weights of learned 2D convolutional kernels are shared across different locations in the input feature map.
This property is desirable for visual detection tasks, where the physical meaning of two dimensions in the input feature map are the same (i.e. $x$-axis and $y$-axis in an image).
However, the spatial invariance property of 2D kernels does not fit acoustic applications well since the physical meaning of a specific 2D kernel holds only along the time axis in a input feature map, but varies a lot along the feature axis.
To keep the shift invariance property locally, we propose an architecture of sub-band CNN by limiting the weight sharing within a certain sub-band on the feature axis.

\begin{figure*}[t!]
    \centering
    \begin{subfigure}[b]{0.28\textwidth}
        \includegraphics[width=\textwidth]{Fig/fullband.pdf}
        \caption{Full band (baseline)}
        \label{fig:fullband}
    \end{subfigure}
    ~

    \begin{subfigure}[b]{0.33\textwidth}
        \includegraphics[width=\textwidth]{Fig/multiband_3b.pdf}
        \caption{Full band plus non-overlapped sub-band}
        \label{fig:multiband}
    \end{subfigure}
    ~

    \begin{subfigure}[b]{0.28\textwidth}
        \includegraphics[width=\textwidth]{Fig/subband_3b.pdf}
        \caption{Overlapped sub-band (proposed)}
        \label{fig:subband}
    \end{subfigure}
    \caption{CNN models with different weight sharing methods. (a) The baseline model proposed in~\cite{Sainath15}. (b) Applying the multi-band approach proposed in~\cite{Takahashi2017} to the baseline model. (c) The proposed overlapped sub-band CNN. For the easiness of illustration, the $x$-axis is set as feature in this figure, which is different from conventional settings.}\label{fig:architectures}

    \vspace{-0.5cm}
\end{figure*}

In this work, we experimented the proposed architecture on Google Speech Commands dataset~\cite{Warden_GoogleSpeechCommand}, which provides a common benchmark for keyword spotting and spoken term classification.
The goal of both tasks is to detect a relatively small set of predefined keywords in an utterance.
Different from keyword spotting that has been widely used for virtual assistants (e.g. Amazon Alexa, Google Assistant, Apple Siri), spoken term classification does not have the low-latency constraint since the classification is done at utterance level.
Previous works~\cite{Chen2014,He2017,Arik2017,Tang2018} showed that neural networks are very effective in keyword spotting.
As tremendous efforts are dedicated into the discovery of effective CNN architectures for further advancing the performance, we argue that it is also important to investigate into effective ways for utilizing computational resource at inference time.
Since most of the applications mentioned above run on mobile devices or smart speakers, a model with small memory footprint and low computational budget is required.
While previous works used low-rank SVD~\cite{Tucker2016,Sun2017} and knowledge distillation~\cite{Lu2017,Pang2018,Shi2019IS} to make neural networks more compact, this work focuses on how to utilize the computational resource efficiently for CNNs.
Compared with residual network for small-footprint keyword spotting (5.65M multiplications)~\cite{Tang2018}, we explored a regime with much lower computational resource for spoken term classification, which is more than 20${\times}$ reduction in the number of multiplications.
\footnote{We use FLOPS as the measurement of computation complexity in this paper, while the number of multiplication is used in~\cite{Tang2018}. For the proposed model, about half of the FLOPS are multiplications. We use this rate to convert FLOPS to the number of multiplication.}

We propose a simple approach to utilize the computational resource efficiently for CNNs that are dedicated for acoustic applications.
Our approach applies different sets of convolutional kernels at each feature sub-band.
Feature maps extracted from each sub-band are concatenated along the channel axis, and then fed into the next convolutional layer.
Limited weight sharing (LWS) for convolutional layers has been proven very effective in speech recognition~\cite{AbdelHamid2012CNN_SR,AbdelHamid2013ExploringCNN}.
LWS is explored for models with a single convolutional layer in these two works.
A CNN that consists of one pair of convolution and max-pooling layers, and two fully connected hidden layers is used in ~\cite{AbdelHamid2012CNN_SR}.
LWS in either time or feature axis for models with a single layer convolutional layer are discussed in~\cite{AbdelHamid2013ExploringCNN}.
Although multi-layers CNN has been tested in~\cite{AbdelHamid2013ExploringCNN}, full weight sharing was used so that more than one convolutional layers can be stacked.
Different from these works, we apply LWS to CNNs with multiple convolutional layers, and explore different ways to concatenate features extracted from each sub-band.

Most similar to our approach of using sub-band CNN with multiple convolutional layers is the work of~\cite{Takahashi2017}, which combines multiple non-overlapped sub-band sub-networks with a full band sub-network for audio source separation.
Different from~\cite{Takahashi2017}, we use overlapped sub-band networks without an extra full band sub-network.
Overlapping between sub-bands helps to avoid information loss at the boundary between sub-bands.
\cite{Takahashi2017} used a full band sub-network to avoid this information loss brought by non-overlapped sub-bands.
However, it may introduce redundant information as well as excessive computational costs.
Our experimental results show that the proposed overlapped sub-band CNN performs better than the multi-band architecture proposed in~\cite{Takahashi2017} on spoken term classification.
Recently, Phaye et. al~\cite{Phaye2019} proposed an architecture using sub-spectrogram based CNN for acoustic scene classification.
Different from these two works~\cite{Takahashi2017,Phaye2019} of applying LWS to CNNs with multiple convolutional layers, this work is the first one to concatenate features within a CNN rather than concatenating the feature extracted from the top most layer of a CNN.
Experimental results in Sec.~\ref{sec:concat} show that concatenating features along channel axis within CNN outperforms other concatenation methods for sub-band CNN.

In the rest of this paper, we discuss our sub-band CNN approach in Section 2, demonstrate them on two tasks (commands and digits classification) of Speech Commands dataset~\cite{Warden_GoogleSpeechCommand} in Section 3, and provide conclusion remarks in Section 4.

\begin{table}
\begin{center}
    \caption[Table caption text]{Detailed architecture of full band CNN (Fig.~\ref{fig:fullband}).}
    \label{table:arch_baseline}
    \begin{tabular}{| l | l | }
    \hline
    Layer & Full band \\ \hline
    Conv 1 (t{$\times$}f, ch, stride) &20$\times$8, $K$, 1$\times$1 \\ \hline
    Activation 1 &ReLU\\\hline
    Dropout 1 &$P$\\\hline
    Maxpool (t{$\times$}f, stride) &2$\times$2, 2$\times$2\\\hline
    Conv 2 (t{$\times$}f, ch, stride) &10$\times$4, $K$, 1$\times$1 \\ \hline
    Activation 2 &ReLU\\\hline
    Dropout 2 &$P$\\\hline
    Dense (\# outputs)&12\\
    \hline
    \end{tabular}
    \vspace{-0.8cm}
\end{center}
\end{table}

\begin{table}
\begin{center}
    \caption[Table caption text]{Detailed architecture of full band plus non-overlapped sub-band CNN (Fig.~\ref{fig:multiband}).}
    \label{table:arch_multiband}
    \begin{tabular}{| p{1.8cm} | l | l | l | l |}
    \hline
    Layer & Band 1 & Band 2 & Band 3 & Full band \\ \hline
    Conv 1 &20$\times$8,&20$\times$8, &20$\times$8, &20$\times$8,\\
    (t{$\times$}f, ch, str.) & $K$,1$\times$1 & $K$,1$\times$1 & $K$,1$\times$1 & $K$,1$\times$1 \\\hline
    Activ. 1 &ReLU&ReLU&ReLU&ReLU\\\hline
    Dropout 1 &$P$&$P$&$P$&$P$\\\hline
    Maxpool &2$\times$2,&2$\times$2,&2$\times$2,&2$\times$2,\\
    (t{$\times$}f, str.)  &2$\times$2&2$\times$2&2$\times$2&2$\times$2\\\hline
    Conv 2 &10$\times$4,&10$\times$4, &10$\times$4, &10$\times$4,\\
    (t{$\times$}f, ch, str.) & $K$,1$\times$1 & $K$,1$\times$1 & $K$,1$\times$1 & $K$,1$\times$1 \\\hline
    Activ. 2 &ReLU&ReLU&ReLU&ReLU\\\hline
    Dropout 2 &$P$&$P$&$P$&$P$\\\hline
    Concat. (axis)&\multicolumn{3}{|c|}{Feature} & -\\\hline
    Concat. (axis)&\multicolumn{4}{|c|}{Channel}\\\hline
    Dense (\# o/p)&\multicolumn{4}{|c|}{12}\\

    \hline
    \end{tabular}
    \vspace{-0.8cm}
\end{center}
\end{table}

\begin{figure*}[t!]
    \centering
    \begin{subfigure}[b]{0.48\textwidth}
        \includegraphics[width=\textwidth]{Fig/commands_main.png}
        \caption{Commands classification}
        \label{fig:commands.main}
    \end{subfigure}
    ~

    \begin{subfigure}[b]{0.48\textwidth}
        \includegraphics[width=\textwidth]{Fig/digits_main.png}
        \caption{Digits classification}
        \label{fig:digits.main}
    \end{subfigure}
    \vspace{-0.2cm}
    \caption{Accuracy curve of different weight sharing methods on subsets of Google Speech Commands dataset~\cite{Warden_GoogleSpeechCommand}. Each data point represents an average of five trials, and the error bar is the sample standard deviation of five trials.}
    \vspace{-0.5cm}
    \label{fig:acc.main}
\end{figure*}

\section{Sub-band CNN}
We show implementation details of the proposed sub-band CNN in this section.
We chose the ``\texttt{cnn-trad-fpool3}" model proposed in~\cite{Sainath15} as our baseline model.
We used the implementation of ``\texttt{cnn-trad-fpool3}" in Tensorflow official package~\cite{tensorflow2015-whitepaper} as the baseline, which is slightly different from the original model described in ~\cite{Sainath15}.
As shown in Fig.~\ref{fig:fullband}, it consists of two convolutional layers followed by a dense layer.
The detailed architecture of the baseline model is shown in Table~\ref{table:arch_baseline}.
The convolutional layers are applied to the full band input feature map, which is equivalent to full weight sharing mentioned earlier.

We applied the proposed sub-band CNN idea to the baseline model, and Fig.~\ref{fig:subband} shows the architecture of the resulting model.
First, the input feature map is split into $B$ overlapped sub-bands ($B$=3 in Fig.~\ref{fig:subband}), and each sub-band has its own set of kernels at the first convolutional layer.
The feature extracted from each sub-band after the first convolutional layer are concatenated along the channel axis, and then fed into the second convolutional layer.
The high-level feature extracted by the second convolutional layer is then fed into a dense layer to generate the final prediction.
Note that we set the number of kernels ($K$) in each convolutional block to be the same (i.e. there are $K$ kernels in `\texttt{Conv1\_1}', `\texttt{Conv1\_2}', ..., `\texttt{Conv1\_B}', and `\texttt{Conv2}' respectively).
The detailed architecture is shown in Table~\ref{table:arch_subband}.

We experimented with different number of sub-bands (\{2,3,4\}) on commands classification task.
As shown in Fig.~\ref{fig:commands.layers}, the model with 3 bands is comparable to the one with 4 bands, and both of them outperform the model with 2 bands.
We chose the model with 3 bands for further experiments per Occam's razor.
For comparison, we applied the multi-band approach proposed in~\cite{Takahashi2017} to the baseline model.
It is shown in Fig.~\ref{fig:multiband}, and the detailed architecture is shown in Table~\ref{table:arch_multiband}.

\begin{table}
\begin{center}
    \caption[Table caption text]{Detailed architecture of overlapped sub-band CNN (Fig.~\ref{fig:subband}).}
    \label{table:arch_subband}
    \begin{tabular}{| p{2.3cm} | l | l | l |}
    \hline
    Layer & Band 1 & Band 2 & Band 3 \\ \hline
    Conv 1 &20$\times$8, $K$,&20$\times$8, $K$,&20$\times$8, $K$, \\
    (t{$\times$}f, ch, stride)  &  1$\times$1  & 1$\times$1 & 1$\times$1 \\\hline
    Activation 1 &ReLU&ReLU&ReLU\\\hline
    Dropout 1 &$P$&$P$&$P$\\\hline
    Maxpool &2$\times$2,&2$\times$2,&2$\times$2,\\
    (t{$\times$}f, stride)  &2$\times$2&2$\times$2&2$\times$2\\\hline
    Concat. (axis) &\multicolumn{3}{|c|}{Channel}\\\hline
    Conv 2 &\multicolumn{3}{|c|}{10$\times$4, $K$, 1$\times$1}\\
    (t{$\times$}f, ch, stride) &\multicolumn{3}{|c|}{}\\\hline
    Activation 2 &\multicolumn{3}{|c|}{ReLU}\\\hline
    Dropout 2 &\multicolumn{3}{|c|}{$P$}\\\hline
    Dense (\# outputs)&\multicolumn{3}{|c|}{12}\\
    \hline
    \end{tabular}
    \vspace{-0.8cm}
\end{center}
\end{table}

\begin{figure*}[t!]
    \centering
    \begin{subfigure}[b]{0.48\textwidth}
        \includegraphics[width=\textwidth]{Fig/commands_layers.png}
        \caption{Different number of bands}
        \label{fig:commands.layers}
    \end{subfigure}
    ~

    \begin{subfigure}[b]{0.48\textwidth}
        \includegraphics[width=\textwidth]{Fig/commands_concat.png}
        \caption{Different concatenation methods for sub-band features}
        \label{fig:commands.concat}
    \end{subfigure}
    \vspace{-0.2cm}
    \caption{Accuracy curve of experiments on number of sub-bands and concatenation methods for sub-band features. Commands classification is used as the testbed. Each data point represents an average of five trials, and the error bar is the sample standard deviation of five trials.}
    \vspace{-0.1cm}
    \label{fig:acc.ablation}
\end{figure*}
\section{Experimental Results}

\subsection{Datasets}
We tested the proposed model on Google Speech Commands dataset~\cite{Warden_GoogleSpeechCommand}, which has 35 words in the latest version (v0.02).
We chose two subsets as our testbed for spoken term classification tasks.
For the formulation of subsets, we use the same setup as the ``Audio Recognition" tutorial in the official Tensorflow package~\cite{tf-asr}.
The task is formulated as a twelve-way classification.
For each task, the subset consists of ten keywords, silence, and unknown (i.e. words not belong to the ten selected keywords).
The first task is commands classification, which contains ten keywords as: ``yes", ``no", ``up", ``down", ``left", ``right", ``on", ``off", ``stop", or ``go".
The second task is digits classification, which uses digits zero to nine as ten keywords.
There are 36,923, 4,445, 4,890 samples for commands classification, and  37,390, 4,373, 4,929 samples for digits classification in train, dev, test sets respectively.

\noindent
\textbf{Feature extraction}
Each utterance is an one second clip with mono audio signals sampled at 16kHz.
The acoustic features used in this work are mel-frequency cepstral coefficients (MFCCs), and the inputs fed to each sub-band sub-network are actually MFCC sub-vectors.
For each one-second clip, we extract 40 dimensional MFCCs from frames of 30 ms duration with shifts of 10 ms.

\subsection{Experimental Setups}
We compare three weight sharing methods for CNN on spoken term classification: (1) full band (Fig.~\ref{fig:fullband}), (2) full band plus non-overlapped sub-band (Fig.~\ref{fig:multiband}), (3) overlapped sub-band (Fig.~\ref{fig:subband}).
To investigate the performance of different model sizes, we experimented with different number of kernels ($K$) in each convolutional block.
Every curve in Fig.~\ref{fig:acc.main} and Fig.~\ref{fig:acc.ablation} consists of data points generated with different values of $K$: \{8, 16, 24, 32, 40, 48, 56, 64\}.
For each model with a specific $K$, we use floating point operations per second (FLOPS) as the measure of model complexity.
Number of FLOPS is measured by \textit{float\_operation} function in the official Tensorflow profiler tool.
We apply the same dropout probability ($P$=0.5) to all the models in this paper.
All models are trained with stochastic gradient descent (SGD) optimizer with a minibatch size of 100.
We train the models with an intial learning rate of 0.001 for 24k iterations, and drop the learning rate to 0.0001 for another 3k iterations.
For the overlapped sub-band models shown in Fig.~\ref{fig:commands.layers}, the bands are \{[0,26), [14,40)\} for 2 bands, \{[0,16), [12,28), [24,40)\} for 3 bands, and \{[0,14), [8,22), [16,30), [26,40)\} for 4 bands, respectively.

The evaluation metric used for spoken term classification in this work is accuracy.
All the accuracies reported in this paper are the average of five random trials to reduce the effect caused by randomness during the training of CNN models.
Error bars in all figures are the sample standard deviation of five trials.
We chose accuracy as the evaluation metric rather than a Detection Error Tradeoff (DET) of false reject and false accept rate for the easiness to compare a large number of trained models in a plot.
It's more succinct to represent five trials using one data point with an error bar compared to plotting tens of DETs in a figure.

\subsection{Results}
Fig.~\ref{fig:commands.main} and Fig.~\ref{fig:digits.main} show the accuracy curves for different weight sharing methods on commands and digits classification.
We have two major observations from these plots.
First, overlapped sub-band outperforms the other two methods in for both cases.
It shows that using overlapped sub-band CNNs to limit weight sharing within a narrow band works better on spoken term classification.
This observation is aligned with the motivation of this work: spatial invariance property of CNN does not fit acoustic applications well, and feature axis should be treated different from time axis.
Interestingly, full band plus non-overlapped sub-band CNN does not work better than the baseline model.
We hypothesize that the full band sub-network and the non-overlapped sub-band sub-network may extract similar features, which causes redundancy in computation.
Second, the computational efficiency brought by overlapped sub-band CNN is more beneficial for small-footprint models (i.e. in the region with lower FLOPS).
If we set the target accuracy as the baseline model (full band) with 500k FLOPS, the proposed sub-band CNN architecture reduces the computation (in terms of FLOPS) by 39.7\% on commands classification, and 49.3\% on digits classification.
Similarly, if the target accuracy is set as the baseline model with 1,000k FLOPS, the reduction is 23.7\% on commands classification, and 50.1\% on digits classification.
Given the same $K$, we found that the decrease in FLOPS of the proposed method comes from the decrease of number of points in the feature map generated by \textit{conv2}.
Feed inputs with less number of points to the final dense layer significantly reduce the required computation.
From the trend of curves shown in Fig.~\ref{fig:acc.main}, we suspect that full band model and overlapped sub-band model may have similar performance when unlimited computational budget is given.

\subsection{Concatenation of Sub-band Features}
\label{sec:concat}
To investigate the effect of different methods to concatenate feature maps from each sub-band, we tested three settings as following:
(1) \textit{concat-c@conv1}: Concatenate along channel axis after the first convolutional layer. This is the \textit{overlapped sub-band CNN} in all other sections throughout this paper.
(2) \textit{concat@conv2}: Each sub-band sub-network has two convolutional layers, and we concatenate all feature maps after the second convolutional layer.  The axis for concatenation does not matter under this setting since the concatenated feature is further fed into a dense layer.
(3) \textit{concat-f@conv1}: Concatenate along feature axis after the first convolutional layer.
As shown in Fig.~\ref{fig:commands.concat}, \textit{concat-c@conv1} outperforms the other two concatenation methods.
By concatenating along channel axis, the receptive field of each point in the concatenated feature map after the first convolutional layer has been tripled in feature axis (from 21$\times$9 to 21$\times$27).
Larger receptive field enables the feature map responds to large enough areas in the input feature map to capture information about acoustic signals spread across different feature dimensions.
Traditionally, larger receptive field is achieved by stacking more convolutional layers, which is less feasible for mobile devices or smart speakers.
The proposed sub-band CNN provides an alternative way to achieve larger receptive field, which is suited for small-footprint model on spoken term classification.

\section{Conclusions}
In this paper, we proposed a sub-band CNN architecture and explored it for spoken term classification.
We compare the proposed sub-band CNNs to full band CNNs and another weight sharing approach on two spoken term classification tasks.
The proposed architecture of sub-band CNNs reduces the computation by 39.7\% on commands classification, and 49.3\% on digits classification, to achieve the same accuracy as the baseline full band CNN with 500k FLOPS.
We found that the computational efficiency brought by sub-band CNN is more beneficial for small-footprint models.
Potential applications for the proposed architecture include on-device speech command recognition, acoustic event detection, etc.

\section{Acknowledgement}
The authors would like to thank Weiran Wang, Krishna Puvvada, and Wei-Ning Hsu for useful discussions.

\bibliographystyle{IEEEtran}