\section{INTRODUCTION}

Modern space and ground-based instruments allow spatially resolved observations of interstellar matter in extragalactic star-forming regions (SFR), which opens up opportunities for studying the features of the star formation process in a significantly wider range of parameters than is possible in our Galaxy. In particular, it is possible to study star-forming in detail in those galaxies in which the birth of stars is significantly affected by some external influence or even stimulated by such influence, for example, in interacting systems.

Observations show that the characteristics of star formation in galaxies of different morphological types differ, and interstellar matter in them is distributed differently: the star-forming regions can be located both chaotically and ordered. Of particular interest from this point of view are those galaxies that show signs of recent triggered star formation, for example, as a result of tidal interaction or merging with another galaxy. Such objects may include so-called polar-ring galaxies—a rather rare class of objects whose subsystems (disk and ring) rotate in almost orthogonal planes. In some close galaxies of this type it is possible to trace differences between star-forming regions and stellar clusters in the ring and in the disk \cite{2004A&A...421..833K,2016A&A...585A.156O}. Although the reasons for the formation of polar subsystems may be different, and there are ongoing disputes about the nature of polar rings in specific galaxies  \cite{BournaudCombes2003,2019MNRASEgorovMoiseev}, in any
case, we are talking about the fact that a galaxy captures external matter with a moment of rotation, which is very different from that of the disk.

In \cite{Smirnova2017,2019ARep...63..445S} we considered various relationships between the observed characteristics of “normal ” galaxies. The subject of this study is similar relations for those galaxies, in the history of which there were episodes that led to certain features in their modern morphology and kinematics. This study was started by us in \cite{2017SWM}, devoted to the study of star formation processes in the ring and disk of the polar-ring galaxy NGC 660. We have revealed some patterns in the emission parameters of the ring and disk SFRs in the near- and mid-IR ranges. In particular, it turned out that the infrared flux from SFRs in the ring and disk of the galaxy differ by orders of magnitude.

Unfortunately, we were not able to find at least one more polar-ring galaxy to continue this analysis, since it requires that the ring be sufficiently extended to allow the detection of as many SFRs as possible, which would not blend with each other when observed by the Spitzer and Herschel Space IR Observatories with an angular resolution of $2-12''$.

In other words, it is necessary that in a galaxy it is possible to fulfill photometry of regions corresponding in size to individual SFRs, or at least to a complex of close SFRs. In
addition, there should be sufficient observational data in other ranges for the selected galaxy. To date, there are several hundred known polar-ring galaxies, but they are all too far away. Moreover, only for a part of the candidates selected by their peculiar morphology, there are reliable kinematic confirmations of the presence of polar rings \cite{1990Whitmore, 2011Moiseev, 2019MNRASReshetnikov}.

So we had the idea to supplement the analysis with other types of galaxies. In the work by \cite{2017SWM} it was concluded that the differences in the fluxes from the disk and ring SFRs of NGC 660 are related to differences in their ages, namely: the regions of the ring in which the near-and mid-IR fluxes are 1‒2 orders of magnitude less than in the disk SFRs of NGC 660 (as well as in the SFRs of “normal” highmetal galaxies) are younger. Based on this assumption, we looked for galaxies that had dynamic processes in the recent past that could cause a wave of star formation. Such galaxies should be located at a short distance (no more than 15 Mpc), so that individual SFRs can be resolved in them, and it is desirable to have a “face on” orientation, in order to simplify the allocation of SFRs and reduce the line-of-sight confusion. Based on the above criteria, we included in consideration the interacting galaxy NGC 1512, for which data is available not only in the near-infrared, but also in the far-infrared ranges.

In addition, we paid attention to asymmetric (lopsided) galaxies \cite{2009PhR...471...75J}. Numerical simulations show that asymmetry can occur when a galaxy interacts with a companion \cite{2014MNRAS.439.1948Y} or as a result of the interaction of two dwarf galaxies \cite{2016Pardy}, resulting in the galaxy’s disk becoming “lopsided”. Disk asymmetry in this model persists for almost 2 billion years. One type of lopsided galaxy is a galaxy with an offset bar \cite{2017MNRAS.469.3363K}, in which the position of the bar does not coincide with the photometric center of the galaxy disk. Unfortunately, among galaxies with an offset bar only the galaxy NGC 4618 meets our criteria \cite{1991AJ....101..829O}. Another asymmetric galaxy with the necessary data set was the Seifert galaxy NGC 4395 \cite{1995ApJS...98..477H}. Both of these galaxies belong to the SBm type.

In the previous work \cite{Smirnova2017} we investigated the relationship between the various components of the interstellar medium (ISM) (atomic and molecular hydrogen, dust particles of various types), which are directly involved in the star formation process, in eleven galaxies, which were divided into two groups: with high and low metallicity (metallicity was taken from the article \cite{moustakas2010}). In this paper, for comparison with our previous results, we consider the high-metallicity galaxy NGC 628. It is further called the comparison galaxy, while the galaxies NGC 660, NGC1512, NGC 4395 and NGC 4618 are collectively called new galaxies.

To compare the surface brightness of SFRs in different galaxies, it is necessary to know the distances to them. For uniformity, we have accepted estimates from the Cosmicflows-3 catalog \footnote{http://edd.ifa.hawaii.edu} \cite{2016AJ....152...50T}. They are shown in Table~\ref{distances} along with the corresponding angular scales. The table also provides estimates of the stellar mass in the galaxies under consideration, derived from fluxes at wavelengths of 3.6 and 4.5 $\mu$m (the Spitzer telescope) using the method described in the paper \cite{2012AJ....143..139E}. Since for our study it is sufficient to know the approximate mass values (see Section 4), we did not attempt to perform accurate photometry. However, our estimates are close to the values obtained for NGC 628 and NGC 660 in  \cite{2019PASJ...71S..14S} and for NGC 4395 and NGC4618 in \cite{2016MNRAS.463.2092Y}. In addition, Table~\ref{distances} gives an estimate of the total star formation rate derived from the flux at a wavelength of 24 $\mu$m according to the calibration given by \cite{2013seg..book..419C}, for all galaxies except NGC 660, for which data at a wavelength of 24 $\mu$m  are not available.

The metallicity of the galaxies NGC 660 and NGC 4618 was calculated using calibration ratios for determination of the oxygen content from R-calibrations from the work \cite{2016Pilyugin}
\begin{eqnarray}
R_2 = I_{{\rm O}_{\rm II}} \lambda 3727+\lambda 3727/I_{{\rm H}_{\beta}}, \nonumber\\
N_2 = I_{{\rm N}_{\rm II}} \lambda 6548+\lambda 6584/I_{{\rm H}_{\beta}}, \nonumber\\
S_2 = I_{{\rm S}_{\rm II}} \lambda 6717+\lambda 6731/I_{{\rm H}_{\beta}}, \nonumber\\
R_3 = I_{{\rm O}_{\rm III}} \lambda 4959+\lambda 5007/I_{{\rm H}_{\beta}}.
\label{main_difeq}
\end{eqnarray}
The line intensities for these galaxies were taken in the article \cite{2006Moustakas}.

The last column of Table~\ref{distances} indicates whether the galaxies in question belong to groups or clusters. The opinions of different authors on the attribution of the galaxies NGC 660 and NGC 628 to one or to different groups differ. However, even if both of these galaxies belong to the same group, they are spatially far apart, and the effects of the environment should not be large.

\begin{table*}
\caption{Characteristics of the studied galaxies}
\label{distances}
\begin{tabular}{lccccccc}
\hline
Galaxy & Distance, & Angular & 12 + & Morph. & $\lg(M_*/M_\odot)$ & Rate &Group,\\
 & Мпк & scale, &  lg(O/H) &type &  &  of SF, &cluster\\
 &   & pc/arcsec  &  &  &  & $M_{\odot}$/year &\\
\hline
NGC 660  & 13.55 & 66   & 8.89  & SB(s)a pec & 10.5 & & LGG29 \cite{1993Garcia},\\
         &       &      &       &             &   & & NGC 660 \cite{2011Makarov}\\
\hline
NGC 1512 & 12.25 & 60   & 8.56 \cite{moustakas2010}  & SB(r)a; HII & 10.2 & 0.4 & LGG108 \cite{1993Garcia},\\
         &       &      &       &             &   & &  NGC 1512 \cite{2011Makarov}\\
\hline
NGC 4395 & 4.76  & 23   & 8.33 \cite{1996Roy} & SA(s)m; & 9.4 & 0.03 & LGG291 \cite{1993Garcia},\\
         &       &      &       & LINER Sy1.8  & & & USGCU480 \\
\hline
NGC 4618 & 6.52  & 32  & 8.35 & SB(rs)m; HII & 9.7 & 0.05 & LGG290 \cite{1993Garcia}\\
\hline
NGC 628  & 9.77  & 47  & 8.45 \cite{moustakas2010} & SA(s)c; HII  & 10.4 & 1.0 & LGG29 \cite{1993Garcia},\\
         &       &      &       &             &   & & NGC 628 \cite{2011Makarov}\\
\hline
\end{tabular}
\end{table*}

\section{OBSERVATIONAL DATA AND REDUCTION}

We used the same sets of observations of the NGC 660 galaxy in the near- and mid-IR ranges, as well as
in the H$\alpha$ line as \cite{2017SWM}. In this article, in addition to the integral intensity of the line H$\alpha$ we also consider the spectra of this emission to estimate the velocity dispersion of the ionized gas. The observations were performed on December 22–23, 2003, at the SAO RAS 6-meter telescope using a Fabry–Perot scanning interferometer in the multimode focal reducer SCORPIO \cite{Afanasiev2005}. The width of the interfringe was about $1290 km/s$ around the red-shifted emission line H$\alpha$. This range was evenly filled with 32 interferograms with exposures of 240 s each. The result of processing performed using the software package \cite{MoiseevEgorov2008}, is a data cube where each pixel of size 0.71" in the field of view $6.1'\times6.1'$ contains a 32-channel spectrum. The spectral resolution was $120 km/s$, and the angular resolution was 2.9".

Data sources for the galaxy NGC628 are given in \cite{Smirnova2017,2019ARep...63..445S}. In them, we imposed a limitation on the size of the studied areas (at least 12) due to the use of observational
data in the CO line. As a result, we selected 65 SFRs bright in IR in the galaxy NGC 628. In this paper, we do not consider the CO radiation, and, in addition, for comparison with the systems under consideration, we need more faint ones in the IR band. Therefore, we additionally identified 19 more regions in the galaxy NGC 628. The entire set of regions under consideration is shown in Fig.~\ref{NGC628_dopreg}. Different colors show the areas of the central part of the galaxy and the areas on its periphery (see below).

\begin{figure}[t!]
    \begin{subfloat}
    \centering{\includegraphics[width=0.8\linewidth]{Smirnova_fig01.png}}
    \end{subfloat}
\caption{Star-forming regions of the galaxy NGC 628: the black circles are regions from the central part of the galaxy, the purple circles are peripheral regions. An image of the galaxy at a wavelength of 24 $\mu$m from the Spitzer telescope’s science archive is used as the background.}
\label{NGC628_dopreg}
\end{figure}
Near-IR observations for the galaxies NGC1512, NGC 4395 and NGC 4618 (at wavelengths of 3.6, 4.5, 8 and 24 $\mu$m) were taken from the Spitzer telescope’s scientific archive. For the galaxy NGC 660, Spitzer observations at a wavelength of 24 $\mu$m are not available, so we used data obtained using the WISE telescope instead. Data from observations of all galaxies in the ultraviolet range were downloaded from the GALEX telescope archive. For the galaxy NGC 1512, data are also available in the far IR range from the KINGFISH survey\footnote{http://herschel.esac.esa.int/Science\_Archive.shtml} \cite{kingfish}, performed on the Herschel telescope.

The observational data used had to be reduced (if possible) to a single angular resolution. In the near and middle IR ranges, the worst angular resolution was obtained from observations of NGC 660 with the  WISE telescope at a wavelength of 22 $\mu$m, so observations of all galaxies at wavelengths of 8 and 24 $\mu$m, as well as observations of the galaxy NGC1512 at wavelengths of 70 and 100 $\mu$m, were reduced to the resolution in the photometric band at a wavelength of 22$\mu$m ($\sim12''$). The convolution was made using kernels from the work of \cite{2011Aniano}. Observations of the galaxy NGC 1512 at a wavelength of 160 $\mu$m were used in their original form, since their angular resolution is close to the angular resolution of the data at a wavelength of 22 $\mu$m.

We also have data in the HI line for the galaxy NGC 660. It was observed in the L band using the Westerbork Synthesis Radio Telescope (WSRT) for three epochs in December 2009, 12 hours for each epoch. The total bandwidth was 10 MHz and was divided into 1024 channels. In accordance with standard practice for WSRT, the primary calibrators CTD93 and 3C147 were observed for approximately 30 minutes before and after the main object was observed. Standard data processing was performed using the software {\sc Miriad} \cite{1995ASPC...77..433S}. After performing initial cross-calibration of the gain and bandwidth using the observed calibration sources, we used an iterative self-calibration procedure to improve the gain calibration in the continuum: first, the image was constructed in the continuum using homogeneous data weighting, which was used for subsequent iterative cleaning by the method of CLEAN components \cite{1974A&AS...15..417H,1980A&A....89..377C}. After each CLEAN procedure, we defined CLEAN areas by applying a cut-off at some level to the resulting image, which allowed us to allocate CLEAN components only within these areas. Using the resulting set of CLEAN components as the input model, we applied a self-calibration procedure to the data and repeated the cleaning of the image by the CLEAN method again. The process was repeated until the desired image quality was achieved (with the cut-off level gradually decreasing). The obtained values of complex gain coefficients in the continuum were then transferred to the original data set, and the resulting model was subtracted in the UV region. After additional subtraction of the continuum {\sc uvlin }in the UV region, the Hanning smoothing was applied to the data. The data cube was created using robust weighing \cite{briggs} and tapering. Due to the small declination of NGC 660, the projections of the WSRT bases in the East-West direction were greatly shortened, so it was necessary either to make the synthesized directional diagram symmetrical, thereby worsening the angular resolution (but losing in sensitivity significantly less than for sources with large declinations), or to leave it strongly elongated.
The data cube analyzed here had an RMS noise of
$\sigma_{\rm rms}\,=0.68\,\mathrm{mJy}\,\mathrm{beam}^{-1}$ and a resolution of $64^{\prime\prime}\times 12^{\prime\prime}$ (HPBW). Observations of neutral hydrogen were used without convolution and without taking into account the elongation of the beam. Our preliminary calculations showed that this does not change our conclusions.

The regions selected in the new galaxies are shown in Fig.~\ref{regions}. After preparing the observation material and selecting the studied regions, we performed aperture photometry with background subtraction. The photometry procedure and estimation of measurement errors are described in \cite{pahcycle}. The background in all cases was estimated by a six-pixelwide ring surrounding the study region. Given the pixel size for IR images, this means that the background detection ring is about twice as wide as the aperture itself. This wide ring allows you to smooth out the effect of neighboring bright areas and correctly take into account the diffuse background in the aperture.

\begin{figure*}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig02a.png}}
    \end{subfloat}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig02b.png}}
    \end{subfloat}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig02c.png}}
    \end{subfloat}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig02d.png}}
    \end{subfloat}
\caption{Star-forming regions in the galaxies considered in this paper, against the background of emission maps at a wavelength of 8 $\mu$m, convolved to the resolution of the WISE telescope at a wavelength of 22 $\mu$m.}
\label{regions}
\end{figure*}

As a result, radiation fluxes in the near and middle IR range, as well as in the UV range, were estimated for all galaxies. The fluxes in bands at wavelengths of 8 and 24 (22) $\mu$m were corrected for the contribution of stellar radiation. In \cite{pahcycle}, the notations $F_{8}^{\rm afe}$ and $F_{24}^{\rm ns}$ were used for corrected fluxes. Further, the upper indexes are omitted for simplicity, but the comparison is made with the corrected fluxes. For the galaxy NGC 1512, the fluxes at wavelengths of 70, 100, and 160 $\mu$m were also estimated.

\section{RESULTS}

\subsection{Aperture Photometry}

The main object of our research is the observed properties of dust and gas in the star-forming regions. We assume that the source of radiation in the near-IR range is small aromatic particles (polycyclic aromatic hydrocarbons, PAHs), whereas radiation at a wavelength of 24 (22) $\mu$m is generated by larger hot dust grains. In both cases, the generation of IR radiation requires heating the dust particle with ultraviolet radiation, so it can be assumed that the regions of intense radiation in the near and middle IR ranges are star-forming complexes.

The common nature of emission at wavelengths of 8 and 24 $\mu$m is emphasized by the diagram in Fig.~\ref{F8_F22(24)}. It can be seen that the position of the SFRs in NGC 628 and the disk of the galaxy NGC 660 in this diagram differs from the position of the SFRs in the galaxies NGC 1512, NGC 4395, NGC 4618, as well as in the ring of the galaxy NGC 660. In the SFRs of new galaxies, the fluxes at wavelengths of 8 and 24 $\mu$m also correlate well with each other, but their values are significantly lower than the fluxes in the SFRs of the “ordinary” galaxy of high metallicity (especially its central part) and of the disk of the galaxy NGC 660.

\begin{figure}
    \begin{subfloat}
    \centering{\includegraphics[width=0.8\linewidth]{Smirnova_fig03.png}}
    \end{subfloat}
\caption{The ratio of radiation fluxes at wavelengths of 8 and 24 (22) $\mu$m for SFRs in the considered galaxies NGC1512, NGC 4395, NGC 4618, NGC 660 (the ring and disk are marked with different symbols) and in the comparison galaxy NGC628.}
\label{F8_F22(24)}
\end{figure}

Since the Herschel data are also available for the NGC 1512 galaxy, we can see how its near and far IR range fluxes compare with “normal” galaxies. In Fig.~\ref{F8_24_FIR}, the total flux in the far IR range (the sum of fluxes at wavelengths of 70, 100, and 160 $\mu$m) is compared with fluxes at wavelengths of 8 and 24 $\mu$m. It is obvious that the SFRs in the galaxy NGC 1512 in the far IR range are also significantly dimmer than the SFRs in the comparison galaxy (both central and peripheral), although in comparison with the fluxes at wavelengths of 8 and 24 $\mu$m, they show the same trend.

\begin{figure*}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig04a.png}}
    \end{subfloat}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig04b.png}}
    \end{subfloat}
\caption{The ratio of radiation fluxes at wavelengths of 8 and 24 $\mu$m and the total flux in the far IR range for SFRs in the galaxies NGC 1512 and NGC 628.}
\label{F8_24_FIR}
\end{figure*}

A direct interpretation of Figs.~\ref{F8_F22(24)} and \ref{F8_24_FIR} is that the infrared fluxes of SFRs in new galaxies are systematically lower than similar fluxes from SFRs in the comparison galaxy. However, this may be partly due to the fact that we have isolated SFRs of different sizes in different galaxies. It should be noted that in all cases this was done by eye, without using any formal algorithm (see the discussion in \cite{2017ASPC..510..102S}), however, we tried to select the size of the apertures so that they correspond to the size of the selected SFR as much as possible. Fig.~\ref{aperdiam}a shows histograms of aperture diameters for the galaxies in question. It is obvious that in the new galaxies, we systematically chose more compact SFRs than in the comparison galaxy. In NGC 628, a significant part of the SFRs has a diameter greater than $20''$, whereas in new galaxies, the diameters of the SFRs are usually significantly less than this value. Note that both in the center and on the periphery of the galaxy NGC 628, we have selected regions of the same size on average, except for some excess of small regions in the outer part of the galaxy, which may be due to selection effects.

\begin{figure}
    \begin{subfloat}
    \centering{\includegraphics[clip=,width=0.7\linewidth]{Smirnova_fig05.jpg}}
    \end{subfloat}
\caption{Distributions of the studied SFRs by aperture diameters (a) and by linear diameters (b).}
\label{aperdiam}
\end{figure}

The histogram in Fig.~\ref{aperdiam}a is more representative of the process of isolating SFRs than of their physical characteristics. Figure ~\ref{aperdiam}b shows the distribution of SFRs over linear diameters calculated for the distance estimates above, and here we see a slightly different picture. The largest SFRs ($\gtrsim 1$ kpc) are observed in the galaxies NGC 628 and NGC 660 (both in the disk and in the ring), while in the interacting galaxies the size of the SFRs is significantly smaller ($\lesssim 700$ pc). What was said above about the angular sizes of the central and peripheral SFRs in the galaxy NGC 628, is obviously true for their linear sizes. It should be clarified that the almost complete absence of identified SFRs in NGC 628, which are comparable in size to the SFRs in the galaxies NGC 4395 and NGC 4618, may be a selection effect. At a distance of NGC 628, a SFR diameter of about 250 pc corresponds to an aperture diameter of $5''$. In the images we used, we could have missed such small objects. Therefore, we can only say that the average size of SFRs in NGC 628 exceeds the average size of SFRs in NGC 1512, NGC 4395 and NGC 4618.

Another parameter that can be used to compare the systems in question is the surface brightness. Figure~\ref{F8_24_surfbri} compares the surface brightness (in Jy/pc$^2$)) of the studied SFRs at wavelengths of 8 and 24 $\mu$m separately for each galaxy studied in this work. The surface brightness of three SFRs of the NGC 660 galaxy disk at a wavelength of 8 $\mu$m exceeds the surface brightness of all other considered SFRs, but this may be due to the uncertain isolation of SFRs in this subsystem. Two more SFRs of the NGC 660 disk do not differ in surface brightness from the SFRs in the comparison galgalaxy. In contrast, the SFRs of the NGC 660 ring have significantly lower surface brightness at wavelengths of 8 and 24 $\mu$m, less than $10^{-3}$ Jy/pc$^2$, than the SFRs of the center and periphery of NGC628. The same low values of $F_{8}/S_{\rm ap}$ and $F_{24}/S_{\rm ap}$ are observed in the galaxy NGC 1512. In contrast to these two systems, the SFRs in the galaxies NGC 4395 and NGC 4618 practically do not differ in surface brightness from the SFRs from the comparison galaxy.

The low surface brightness of IR radiation of the SFRs in NGC 1512 and in the NGC 660 ring may be due to both a small number of sources (aromatic particles) and a low intensity of ultraviolet radiation. To compare the radiation in the infrared and ultraviolet ranges, in Fig.~\ref{F8_24_GALEX} we compare the results of aperture photometry of data from the archives of observations with the Spitzer and GALEX telescopes. The data of ultraviolet observations in the FUV filter and radiation at a wavelength of 8 $\mu$m are presented (for fluxes in the NUV bands and at a wavelength of 24 $\mu$m, the results are essentially the same).

\begin{figure*}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig06a.png}}
    \end{subfloat}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig06b.png}}
    \end{subfloat}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig06c.png}}
    \end{subfloat}
        \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig06d.png}}
    \end{subfloat}
\caption{The ratio of the surface brightness of radiation at wavelengths of 8 and 24 $\mu$m for SFRs from the galaxies studied in this paper.}
\label{F8_24_surfbri}
\end{figure*}

As in Fig.~\ref{F8_24_surfbri}, we see a significant difference between the galaxies NGC 660 and NGC 1512 on the one hand and the galaxies NGC 4395 and NGC 4618 on the other hand. It is obvious that the intensity of the aromatic bands in the NGC 660 ring and in NGC 1512 correlates well with the UV emission in the filter FUV. In the NGC 660 ring, the UV flux from SFRs is significantly lower than in the comparison galaxy. The SFRs in NGC 1512 overlap somewhat with the SFRs in NGC 628, but on average the surface brightness of SFRs in this galaxy in both IR and UV is lower than in NGC 628.

\begin{figure*}
\begin{subfloat}
\centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig07a.png}}
\end{subfloat}
\begin{subfloat}
\centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig07b.png}}
\end{subfloat}
\begin{subfloat}
\centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig07c.png}}
\end{subfloat}
\begin{subfloat}
\centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig07d.png}}
\end{subfloat}
\caption{The ratio of surface brightness at a wavelength of 8 $\mu$m and in the UV range for SFRs in the galaxies studied in this paper.}
\label{F8_24_GALEX}
\end{figure*}

There is a temptation to link the observed surface brightness in the IR and UV ranges, but we must take into account that, firstly, in the galaxies NGC 4395 and NGC 4618, we observe a different picture, namely: a significant shift of the SFR positions in Fig.~\ref{F8_24_GALEX} up along the Y axis, which means a higher surface brightness in the FUV filter at the same surface brightness in the band 8 $\mu$m. Secondly, since the UV range was not taken into account when isolating SFRs, there is no noticeable UV radiation in some of the regions we have selected, as illustrated in Fig.~\ref{NGC4395_NGC1512_GALEX}, where the positions of SFRs in the NGC 4395 galaxy are marked in its images in the UV range. It can be seen that the UV radiation is not associated with some SFRs or is significantly shifted relative to them (for example, SFRs 6, 31, 87, 92).

\begin{figure}
\includegraphics[width=0.8\linewidth]{Smirnova_fig08.png}
\caption{A fragment of the image of the galaxy NGC 4395 in the UV range (GALEX FUV) with the studied SFRs.}
\label{NGC4395_NGC1512_GALEX}
\end{figure}

Radiation in the near-IR, UV bands, and radiation in the H$\alpha$ line are associated with active star formation. Due to the availability of data on observations of neutral hydrogen in the galaxy NGC 660, we are able to correlate the emission in the IR range with the emission in the line at a wavelength of 21 cm, just as in the work \cite{Smirnova2017}, normalizing the flux in the IR range to the flux in the line HI. The results are shown in Fig.~\ref{ngc660_HI}.

\begin{figure*}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig09a.png}}
    \end{subfloat}
    \begin{subfloat}
    \centering{\includegraphics[width=0.4\linewidth]{Smirnova_fig09b.png}}
    \end{subfloat}
\caption{The ratio of fluxes $F_8/F_{\rm HI}$ and $F_{24}/F_{\rm HI}$ for SFRs in the galaxy NGC660 and the comparison galaxy as a function of the
flux in the neutral hydrogen line.}
\label{ngc660_HI}
\end{figure*}

Both in the diagram for a wavelength of 8 $\mu$m, and in the diagram for a wavelength of 22 $\mu$m, the SFRs of the NGC 660 ring as a group fall on the same (anti)correlation as the SFRs of the galaxies studied in \cite{Smirnova2017}: the brighter the region in the line HI, the lower its relative brightness in the IR range. \cite{Smirnova2017} suggested that the existence of such anticorrelation may be due to age in the sense that neutral hydrogen is particularly rich in young regions where conditions have not yet developed for generating intense infrared radiation. However, there is no correlation between $F_{8,22}/F_{\rm HI}$ and $F_{\rm HI}$ within these regions. Low fluxes in the IR range are observed both in SFRs rich in hydrogen and in SFRs poor in it.

\subsection{Kinematics}

In \cite{2019ARep...63..445S}, the value of the velocity scatter $\Delta V$, equal to the velocity difference of the extreme channels in the full spectrum, in which the signal exceeds the tripled standard deviation, was introduced as a characteristic of internal motions in the SFRs. The standard deviation was estimated for the part of the spectrum where there is no signal from the SFRs. Of the new galaxies, these data are only available for NGC 660. Also, only for this galaxy there are spectra in the  H$\alpha$  line. For them, the procedure described above for estimating $\Delta V$ does not work, and we used a more traditional approach, determining the width of the line by fitting the Gaussian into those spectra that are dominated by a single line. In this case, $\Delta V$ is estimated as FWHM for the fitted pro file, subtracting the width of the tool profile.

To estimate the contribution of the large-scale velocity field (beam smearing), we assumed that atomic hydrogen moves in the disk and ring in the same way as HII, i.e. it shows almost regular rotation.
The smoothed velocity field H$\alpha$ was used to analyze the distribution of velocities in pixels that fell into the elongated beam of WSRT observations, and the dispersion of these velocities in our apertures was calculated. As expected, in the outer regions of the ring, the correction is small and is 8–12 km/s. In the disk, its value is greater, 30–60 km/s, with maximum values near the intersections of the disk with the ring. In all cases, the estimated contribution of the large-scale velocity field is significantly less than the value $\Delta V$ in the same aperture, so we ignored it in the further analysis. Unfortunately, in the absence of statistical interpretation of the value $\Delta V$, the procedure for subtracting the contribution of the large-scale velocity field from it is unclear, which makes our conclusions somewhat uncertain.

The values $\Delta V$ for H$\alpha$ and HI for NGC 660 are compared in Fig.~\ref{NGC660kincompar} and compared to the corresponding fluxes in Fig.~\ref{NGC660kinematique}.

\begin{figure}[t!]
    \begin{subfloat}
    \centering{\includegraphics[width=0.8\linewidth]{Smirnova_fig10.png}}
    \end{subfloat}
\caption{The ratio of velocity scatters in the lines H$\alpha$ and HI.}
\label{NGC660kincompar}
\end{figure}

Figure~\ref{NGC660kincompar} shows that the velocity scatters in the SFRs of the NGC 660 ring in both lines are enclosed in the range from 100 to 200 km/s, but there is no correlation between them. The only region where the velocity scatter in the H$\alpha$ line exceeds 400 km/s, is characterized by a two-peak profile. Given its large size, this may mean that two regions fell into the aperture (interestingly, the velocity scatter in the line HI has a moderate value).

\begin{figure}
    \begin{subfloat}
    \centering{\includegraphics[width=0.8\linewidth]{Smirnova_fig11b.png}}
    \end{subfloat}
   \begin{subfloat}
    \centering{\includegraphics[width=0.8\linewidth]{Smirnova_fig11a.png}}
    \end{subfloat}
\caption{The ratio between the velocity scatter and the radiation flux in the SFRs from the galaxy NGC 660: HI (a) and H$\alpha$ (b).}
\label{NGC660kinematique}
\end{figure}

The ratio between flux and velocity scatter in the H$\alpha$  spectrum (Fig.~\ref{NGC660kinematique}a) shows a known correlation: the flux growth is accompanied by an increase in velocity scatter, which may indicate a relationship between velocity scatter and mass, although the nature of this correlation remains a subject of discussion \cite{2015MNRAS.449.3568M}. But there is almost no correlation between the velocity scatter in the lines HI and the corresponding flux (Fig.~\ref{NGC660kinematique}b).

\section{DISCUSSION}

Initially, the motivation for this work was the study of the polar-ring galaxy NGC660, which revealed a significant difference in the properties of infrared radiation in the SFRs of the disk and the ring of this galaxy. The assumption that this difference is due to the different nature of star formation in the two subsystems of NGC 660, was the incentive for this study, in which we expected to find similar differences in other galaxies that have recently experienced an episode of enhanced star formation due to some external influence. However, the real picture was more complicated.

A direct comparison of IR fluxes does show that in all new galaxies the SFRs are dimmer than the SFRs in the comparison galaxy. However, the reasons for this are different. The SFRs in the NGC 660 ring are
comparable in physical size to the SFRs in the “normal” galaxy NGC 628 and to the SFRs in the NGC 660 disk and their small IR flux is associated with low surface brightness. The SFRs in the galaxies NGC 4395 and NGC 4618 are similar in surface brightness to the SFRs in the NGC 628 galaxy, and their low fluxes are associated with smaller physical sizes. Finally, SFRs in the galaxy NGC 1512 are characterized by both small size and low surface brightness. These differences must obviously be related to the evolutionary differences between the galaxies in question. Let us look at them in more detail.

The galaxy NGC 1512 is a system with two rings—external and internal,—the morphology of which is largely determined by the interaction with the nearby dwarf galaxy NGC 1510. The distribution of gas and dust in the NGC 1512/NGC 1510 system was considered in \cite{2009MNRAS.400.1749K} and \cite{2010AJ....139.1190L}, respectively. \cite{2009MNRAS.400.1749K} showed that atomic hydrogen extends to a distance at least four times the optical size of the galaxy ($B_{25}$), and is largely concentrated in two spiral arms (Arm1 and Arm2). Individual clouds of HI are traced to distances of the order of $\sim30^\prime$ (for the adopted distance, this corresponds to 100 kpc). Up to about the same distance, UV radiation spots are also visible, probably tracking the position of star-forming regions (their more detailed analysis is presented in \cite{2015MNRAS.450.3381L}). At the same time, \cite{2009MNRAS.400.1749K} note that in the outer region of the NGC 1512 disk the radiation at a wavelength of 8 $\mu$m is not detected, which is consistent with our results: according to radiation maps at a wavelength of 8 $\mu$m, we were not able to isolate any SFRs in the outer region of this galaxy. In \cite{2010AJ....139.1190L}, the outer regions of NGC 1512 were also not considered. Thus, there is a similarity between the galaxies NGC 660 and NGC 1512: the SFRs in the rings of NGC 660 and in NGC 1512 are characterized by reduced surface brightness at a wavelength of 8 $\mu$m. Moreover, on the
periphery of NGC 1512, exposed to the tidal action of NGC 1510, the surface brightness at a wavelength of 8 $\mu$m is so low that radiation is not detected at all, although the content of HI there, as in the NGC 660 ring (see Fig.~\ref{ngc660_HI}, is high. Thus, the common characteristic of the galaxies NGC 660 and NGC 1512 can be considered the presence of subsystems (the NGC 660 ring, external spiral arms of NGC 1512), characterized by a significant content of atomic hydrogen, the presence of UV radiation sources and low surface brightness at a wavelength of 8 $\mu$m.

It should be noted that the tidal structures in NGC 1512, as well as the NGC 660 ring, are not longlived structures, whereas in lopsided galaxies the characteristic time of star formation initiation may be longer. However, the problem of the age of the considered SFRs requires a separate study.

A common feature of the galaxies NGC 4395 and NGC 4618 is their lopsidedness. There is no clear answer to the question about the nature of the asymmetric structure of disk galaxies. Numerical calculations
show that in some cases such asymmetry may occur during the dynamic evolution of the disk in barred spiral galaxies.
Among the more common explanations are tidal interactions with other galaxies or an asymmetric inflow of external matter \cite{2009PhR...471...75J}. One is tempted to associate the above-mentioned features of NGC 4395 and NGC 4618 with their asymmetry. However, they differ from other galaxies considered in this article by at least three other parameters: distance, mass, and star formation rate, and the differences in mass and star formation rate may be related.

The close location of NGC 4395 and NGC 4618 could lead to the fact that in these galaxies, due to the better spatial resolution, we isolated smaller SFRs. Indeed, in the galaxy NGC 4395, it is possible to specify several groups of SFRs, which at a lower resolution would be identified by us as one larger SFR. However, there are few such groups, and they would not change the overall picture of the distribution of SFRs by size. In addition, the relatively small physical size is also characteristic of SFRs in the galaxy NGC 1512, located much further away. We have already noted the possible effects of selection when isolating regions in NGC 628, but even with their consideration, the main conclusion remains the same: the average size of SFRs in NGC 4395 and NGC 4618 is inferior to the average size of SFRs in other galaxies.

Both of the above reasons of galaxy asymmetry can cause increased star formation, and we do see some indications of this. The total luminosity of NGC 4395 and NGC 4618 in the mid-IR range indicates that the integrated star formation rate in them is an order of magnitude or more inferior to the star formation rate in the galaxies NGC 628 and NGC 1512 (see Table~\ref{distances}), however, this difference indicates a smaller size of NGC 4395 and NGC 4618: the total surface brightness of these galaxies at a wavelength of 24 $\mu$m (according to our estimates) and in the B band (according to the HyperLEDA database) exceeds the similar parameters of the galaxies NGC 628 and NGC C1512. In other words, in the two lopsided galaxies considered in this paper, the star formation is actually somewhat more intense than in the other galaxies. In addition, the SFRs in NGC 4395 and NGC 4618 have a higher surface brightness in the UV range than the SFRs in the comparison galaxy, which have the same brightness in the near IR range. Thus, although the asymmetry of disk galaxies is a fairly common property (\cite{2009PhR...471...75J}), in two specific cases (NGC 4395 and NGC 4618) we see noticeable differences from both interacting and “normal” galaxies. It should be noted that the galaxy NGC 4395 is isolated, and the galaxy NGC 4618 although usually considered in conjunction with the galaxy NGC 4625, may also not be interacting (\cite{2012AJ....144...67K}). It is obvious that the interaction is not the only reason for the formation of SFRs with a small flux in the IR range.

It should be noted that, if in the galaxy NGC 628 we separately consider SFRs in the center and on the periphery, the above diagrams also show differences between them: central SFRs exhibit higher luminosities and surface brightnesses in the near IR region than peripheral SFRs, however, the differences noted above between the galaxies under consideration and the galaxy NGC 628 turn out to be significantly larger than the differences between the central and peripheral SFRs in NGC 628. In the UV range, as well as in the emission of neutral hydrogen, there are no systematic differences between the central and peripheral SFRs in NGC 628.

The metallicities of all the galaxies under consideration are close, so we cannot expect any significant differences related to the chemical composition. The galaxy NGC 660 stands somewhat apart in this respect. Its metallicity (as well as other integral characteristics) refers mainly to the disk. We have not found any determinations of the chemical composition of the ring in the literature, except for the work \cite{2004A&A...421..833K}, the authors of which showed that the stellar population of the ring is best described by isochrones with a rather low metallicity, $Z=0.008$. In addition, \cite{2000A&A...357..443A} estimated the ratio of dust and gas masses in the NGC 660 ring. It turned out that its value is 2 to 3 times lower than in the solar neighborhood. If we assume that this ratio is a measure of metallicity, it also indicates the content of heavy elements just a few times lower than the solar ($12+\lg({\rm O}/{\rm H})\sim8.1\div8.3$) Thanks to the observations in HI presented in the press release\footnote{https://www.astron.nl/dailyimage/main.php?date=20100923} \cite{Jozsa2010} it is clear that the ring in NGC 660 was formed as a result of gas capture from the disk of a rather massive dwarf irregular galaxy UGC 1195. Direct measurements of the metallicity of this galaxy are not available in the literature, but the using of the “metallicity luminosity” relation from \cite{2004A&A...425..849P} gives $M_B({\rm UGC~1195)}=-17.67$ for (according to the HyperLeda database) the value $12+\lg({\rm O}/{\rm H})=8.33$ in good agreement with the above estimate of metallicity of the NGC 660 polar ring. At the same time, as our research also shows \cite{Smirnova2017}, differences that are related to the metallicity become significant at values of $12+\lg({\rm O}/{\rm H})<8$.

\section{CONCLUSIONS}

The paper considers the parameters of SFRs in several galaxies and possible relationships between these parameters. Based on the results obtained, the following conclusions are made:
\begin{enumerate}
\item{In the galaxies with signs of recent interaction NGC 660 and NGC 1512, the surface brightness of star-forming complexes in the UV and IR ranges is significantly lower than in the comparison galaxy. At the same time, in the peripheral region of NGC 1512, there are no SFRs emitting in the near-IR range at all.}
\item{In the asymmetric galaxies NGC 4395 and NGC 4618, the surface brightness of SFRs in the IR range does not differ from the surface brightness of SFRs in “normal” galaxies. The surface brightness of SFRs in these galaxies in the UV range exceeds the similar brightness of SFRs in comparison galaxies. However, the physical size of the SFRs in these two galaxies is smaller than the size of the SFRs in both the interacting galaxies and the comparison galaxy.}
\item{In the SFRs of the ring of the NGC 660 galaxy, the HI flux at 21 cm is comparable to or exceeds the flux in the SFRs from the comparison galaxy (at significantly lower IR fluxes). The velocity scatter in the line H$\alpha$  increases with the flux in this line; the velocity scatter in the line HI does not depend on the flux in it.}
\end{enumerate}

The results of the study may indicate that the parameters of star-forming complexes differ in galaxies with signs of recent interaction, in galaxies with asymmetric disks and in normal galaxies. The differences observed in both the physical dimensions and the surface brightness of the SFRs indicate that the course of star formation process depends on many factors, and not only on the belonging of the galaxy to one or another morphological type.

The authors thank the referee for remarks and comments that allowed a deeper understanding of the results presented, as well as A. Alakoz and D. Makarov for useful discussions. This work is based in part on observations made with the Spitzer Space Telescope, which was operated by the Jet Propulsion Laboratory, California Institute of Technology under a contract with NASA and Herschel Space Observatory that is an ESA space observatory with science instruments provided by European-led Principal Investigator consortia and with important participation from NASA. The astronomical database HYPERLEDA
(http://leda.univ-lyon1.fr) was used in the work.

The research was supported by Russian Foundation for Basic Research as part of the science project no.~19-32-50063. The work is based on the use of observations from 6-meter telescope of SAO RAS, supported by the Ministry of Science and Higher Education of the Russian Federation (agreement no.~05.619.21.0016, project ID RFMEFI61919X0016).


\section{Introduction}
One-shot Video Object Segmentation~(VOS) aims to segment an object of interest in a video sequence, where the object mask in the first frame is provided.
The objective of this task is to track a target object in a pixel-wise manner.
It has various applications such as robotics, autonomous driving, and video editing to name a few.
VOS is a challenging task, and generating quality segmentation masks requires addressing inevitable real-world difficulties such as unconstrained camera motion, occlusion, fast motion, and motion blur as well as handling objects with different sizes.

VOS has been extensively studied in the Computer Vision community with several works based on classical techniques such as energy minimization and utilizing superpixels~\cite{chang2013video,marki2016bilateral,grundmann2010efficient}.
However, learning-based methods~\cite{perazzi2017learning,Man+18b} have proved to be more successful by significantly surpassing the traditional approaches.

Amongst the wide variety of the suggested learning-based methods, some works approach the problem by processing the frames independently and learning an object model~\cite{perazzi2017learning,Man+18b}, while others utilize temporal information~\cite{xu2018youtub,ventura2019rvos}.
Tokmakov \etal~\cite{tokmakov2017learning} propose utilizing optical flow to propagate the object mask throughout the sequence and make use of the motion cues as well as the spatial information.
However, flow-based models need an additional component for flow estimation~\cite{ilg2017flownet}, which is usually trained separately, and the performance of the whole system is dependent on the accuracy of this module.
With the same motivation of using temporal data,~\cite{xu2018youtub,ventura2019rvos,azimi2020revisiting} utilize Recurrent Neural Networks~(RNNs) to track the target object in a temporally consistent way.
These models are trained end-to-end and rely on learning the spatio-temporal features to track the object and to propagate the object mask across time.
A disadvantage of this category is the performance drop in longer sequences caused by drift and error propagation in the RNN.

In this work, we study S2S~\cite{xu2018youtub}, a common RNN-based model for VOS due to the effectiveness of RNNs in utilizing the spatio-temporal features and providing a motion model of the target object, resulting in good segmentation accuracy.
Inspired by~\cite{faktor2014video,wug2018fast,yang2019anchor}, we propose a dual propagation strategy by augmenting the spatio-temporal features obtained from the RNN with correspondence matching to reduce the impact of drift.
Utilizing the features obtained from similarity matching provides a robust measurement for segmentation, improves the segmentation quality, and reduces the error propagation.
This aspect is especially beneficial for the model in long sequences where the RNN performance declines.
Additionally, we integrate the first frame features into the model throughout the whole sequence as a reliable source of information~\cite{ebert2017self,wug2018fast,oh2019video,yang2019anchor}.
By employing these reference features, the model can better handle challenging scenarios such as occlusion~\cite{ebert2017self}, since, by definition, the object is present in the first frame.
\autoref{fig:occlusion} shows an illustration of how correspondence matching together with utilizing the first frame can be helpful in better handling the occlusion.
We hypothesize that the RNN also plays a complementary role in correspondence matching.
Imagine a scenario where multiple instances of similar objects are present in the scene;
in this case, the spatio-temporal model learned by the RNN can act as a location prior and aid the model to distinguish between the target object and the other similar instances.

We evaluate our hybrid sequence-to-sequence~(HS2S) method on the Youtube-VOS~\cite{xu2018youtub} and DAVIS2017~\cite{pont20172017} datasets and demonstrate that our model significantly improves the independent RNN-based models' segmentation quality \cite{xu2018youtub,ventura2019rvos}.
\begin{figure}[t]
\centering
  \includegraphics[width=0.9\linewidth]{pics/occlusion.pdf}
   \caption{This figure indicates how utilizing the first frame as the reference can help the model recover from occlusion. Here, the object of interest is a bear overlaid with the red mask, which is absent from the middle row frames (from $t=30$ to $t=70$).
   We observe that the model can detect the animal after it appears again, and by looking at the saliency map of the first frame, we note that the model has correctly captured the correspondence between the bear in the first frame and the frame right after the occlusion.
   }
\label{fig:occlusion}
\end{figure}
\section{Related Work}
A large body of research in Computer Vision literature has studied VOS during the last decade.
The classical methods for solving VOS were mainly based on energy minimization~\cite{brox2010object,faktor2014video,papazoglou2013fast,shankar2015video}.
Brox \etal~\cite{brox2010object} propose a model based on motion clustering and segment the moving object via the analysis of the point trajectories throughout the video.
They also use motion cues to distinguish foreground from background.
Faktor \etal~\cite{faktor2014video}
present a method based on consensus voting.
They extract the superpixels in each frame, and by computing the similarity of the superpixel descriptors, then use the nearest neighbor method to cluster the most similar superpixels together in a segmentation mask.
\cite{jain2014supervoxel} addresses the problem of fast motion and appearance change in the video by extending the idea of using superpixels to using super-voxels (adding the time dimension) and taking into account the long-range temporal connection during the object movement.

Since the advent of Deep Learning~\cite{krizhevsky2012imagenet}, the Computer Vision community has witnessed a significant progress in the accuracy of VOS methods~\cite{Man+18b,perazzi2017learning,tokmakov2017learning}.
The success of learning based methods can largely be accounted to progress made in learning algorithms~\cite{krizhevsky2012imagenet, he2016deep} and the availability of large-scale VOS datasets such as Youtube-VOS~\cite{xu2018youtub}.

In one-shot VOS, there exist two training schemes, namely offline and online training. Offline training is the standard training phase in learning-based techniques.
As the segmentation mask of the first object appearance is available at test time, online training refers to further fine-tuning the model on this mask with extensive data augmentation.
This additional step considerably improves the segmentation quality at the expense of slower inference.

Considering offline training, one can divide the proposed solutions into multiple categories.
Some methods focus on learning the object masks using only the frame-wise data~\cite{perazzi2017learning, Man+18b}.
In~\cite{Man+18b}, authors extended a VGG-based architecture designed for retinal image understanding~\cite{maninis2016deep} for VOS.
They start with the pre-trained weights on ImageNet~\cite{deng2009imagenet}, and then further train the \textit{parent network} on a specialized VOS dataset~\cite{Perazzi2016}.
This model relies on online training and boundary snapping for achieving good performance.
\cite{voigtlaender2017online} further improves this method by employing online adaption to handle drastic changes in the object's appearance.
Perazzi \etal~\cite{perazzi2017learning} provide a solution based on guided instance segmentation.
They utilize a DeepLab architecture~\cite{chen2017deeplab} and modify the network to accept the previous segmentation mask as an additional input.
Therefore, a rough guidance signal is provided to the model to mark the approximate location where the object of interest lies.
Yang \etal~\cite{yang2018efficient} take a meta-learning approach and train an additional modulator network that adjusts the middle layers of a generic segmentation network to capture the appearance of the target object.

In~\cite{wug2018fast} a Siamese architecture is used to segment the object based on its similarity to the mask template in the first frame.
Similarly, \cite{yang2019anchor} proposes a zero-shot VOS model, where the object mask at every time step is detected based on the similarity of the current frame to the anchor frames~(first frame and the frame at $t-1$).
Following this idea, \cite{johnander2019generative} suggests a generative approach for segmenting the target object, introducing an appearance module to learn the probabilistic model of the background and the foreground object.
In~\cite{zhang2020transductive}, the authors develop a model that propagates the segmentation mask based on an affinity in the embedding space.
They propose to model the local dependencies via using motion and spatial priors and the global dependencies based on the visual appearance learned by a convolutional network.
Although these methods obtain good performance on the standard benchmarks~\cite{Perazzi2016,pont20172017}, they do not utilize temporal information and motion cues.

Another line of work relies on region proposal techniques such as \cite{he2017mask}.
For example, \cite{luiten2018premvos} takes a multi-step approach, in which they first generate the region proposals and then refine and merge promising regions to produce the final mask.
Furthermore, they use optical flow to maintain the temporal consistency.
In~\cite{li2017video}, an additional re-identification method based on template-matching is used.
This way, the model can recapture objects lost at some point in the sequence.
These methods are quite complex in architecture design and relatively slow at inference time.

A different group of methods focus on utilizing a memory module to process motion and compute spatio-temporal features.
In order to obtain temporally consistent segmentation masks,~\cite{xu2018youtub, azimi2020revisiting,tokmakov2017learning,ventura2019rvos} employ a ConvLSTM~\cite{xingjian2015convolutional} (or ConvGRU) memory module while~\cite{oh2019video} resorts to using an external memory to process the space-time information.

In this work, we build on top of the S2S~\cite{xu2018youtub} architecture, which is an RNN-based method, on account of exploiting the spatio-temporal features, good performance, and the simple architecture.

We study some of this model's shortcomings stemming from the finite memory and error propagation in RNNs.
To address these limitations, we propose a hybrid design that combines the spatio-temporal features from the RNN with similarity matching information, as it will be elaborated in the next section.
Unlike~\cite{oh2019video}, our model does not require any form of external memory.
This is advantageous since using external memory results in additional constraints in the inference phase (e.g. memory overflow for long video sequences).
\section{Method}
In this section, we explain our hybrid architecture for VOS.
We build on top of the S2S model~\cite{xu2018youtub}, which is an RNN-based architecture and employ a dual mask propagation strategy that utilizes the spatio-temporal features from the RNN as well as correspondence matching to propagate the mask from time step $t-1$ to $t$.
Moreover, we integrate the features from the first frame as a reference throughout the sequence.

The S2S model is composed of an encoder-decoder architecture with a memory module in the bottleneck to memorize the target object and obtain temporal consistency in the predicted segmentation masks.
The overall design of this method is illustrated in \autoref{fig:architecture}.
In this model, the object masks are computed as in~\cite{xu2018youtub}:
\begin{equation}
    h_{0}, c_{0} = \text{Initializer}(x_{0}, y_{0})
\label{eq:1}
\end{equation}
\begin{equation}
    \Tilde{x}_{t}= \text{Encoder}(x_{t})
\label{eq:2}
\end{equation}
\begin{equation}
    h_{t}, c_{t} = \text{RNN}(\Tilde{x}_{t}, h_{t-1}, c_{t-1})
\label{eq:3}
\end{equation}
\begin{equation}
    \hat{y}_{t} = \text{Decoder}(h_{t})
\label{eq:4}
\end{equation}
where $x$ and $y$ refer to the RGB input image and the binary mask of the target object in the first frame.
\begin{figure*}[t!]
    \centering

    \includegraphics[width=16cm]{pics/architecture.pdf}
    \caption{In this figure, we depict the overall architecture of S2S \cite{xu2018youtub}~(\Cref{eq:1,eq:2,eq:3,eq:4}) and our HS2S method~(\cref{eq:5,eq:x0,eq:xt_1,eq:6,eq:7,eq:8}).
    In HS2S, we initialize the RNN hidden states~($h_{0}$ and $c_{0}$) with \textit{zeros}, instead of using the initializer network.
    We keep track of the target object by feeding the previous segmentation mask~($y_{t-1}$) to the encoder as an additional input channel, similar to~\cite{perazzi2017learning}.
    Furthermore, we use a separate reference encoder to process the input to the matching branch.
    We highlight that the functions approximated by these two encoders differ, as the inputs to the  \textit{Reference Encoder} are aligned in time, but this is not the case for the \textit{Encoder} network.
    Finally, the hidden state of the RNN ($h_{t}$) is combined with
    the encoded features from the matching branch via a merge layer and passed to the decoder to predict the segmentation mask.
    The skip connections between the encoder and the decoder networks are not shown for simplicity.}
    \label{fig:architecture}
\end{figure*}

One of the main limitations of RNN-based models, such as S2S, is the fixed-sized memory, which can be insufficient to capture the whole sequence and long-term dependencies~\cite{bahdanau2014neural}.
Therefore, as the sequence length grows, access to information from earlier time steps decreases.
This issue, together with the vanishing gradient problem, adversely impacts the segmentation quality in longer sequences.
This problem is especially critical in sequences with occlusion, where the object of interest can be absent for an extended period.

Another obstacle with this category of approaches is drift and error propagation.
Due to the recurrent connection, the model output is fed back to the network;
as a result, the prediction error propagates to the future, and erroneous model predictions affect the performance for future time steps.
This issue is another contributing factor to the performance drop in later frames.

\vspace{+1mm}
\textbf{Hybrid Mask Propagation}.
Based on the challenges in the RNN-based models, we propose a hybrid architecture, combining the RNN output with information derived from correspondence matching.
In our model, the segmentation mask is predicted using the location prior obtained from the RNN, as well as similarity matching between the video frames at $t-1$ and $t$.
Our intuition is that the merits of using the spatio-temporal model from RNN-based models and the matching-based methods are complementary.
In situations where multiple similar objects are present in the scene, the matching-based approaches struggle to distinguish between the different instances.
Hence, the location prior provided by the spatio-temporal features from the  RNN can resolve this ambiguity.
Moreover, the information obtained from similarity matching provides a reliable measurement for propagating the segmentation mask to the next time step (as investigated in~\cite{yang2019anchor} for zero-shot VOS). Using this additional data helps the model reduce the prediction error, improving the drift problem, and obtaining better segmentation quality for longer sequences.

To encode the frame at $t-1$, we redefine the initializer network's task in S2S to a reference encoder (as shown in \autoref{fig:architecture}), initializing the hidden states of the RNN module with $zeros$.
In our experiments, we observed that the initializer network does not play a crucial role, and it is possible to replace it with zero-initialization with little change in the performance.

To perform the similarity matching between the RNN hidden state ($h_{t}$) and the reference encoder's output features, one can use different techniques such as using the cosine distance between the feature vectors.
Here, we follow the design in~\cite{wug2018fast} and use a Global Convolution~\cite{peng2017large} to accomplish the task~(merge layer in \autoref{fig:architecture}).
Global Convolution~(GC) approximates a large kernel convolution layer efficiently with less number of parameters.
The large kernel size is essential to model both the local connections (as required for localization) and the dense global connections required for accurate classification (foreground, background).
This way, the model directly accesses the features from time steps $0$ and $t-1$.
We note that this operation can also be interpreted as self-attention; as,
the features at the current time step, which share a higher similarity to the object features from the reference frames, get a higher weight via the convolution operation in the merge layer.

As shown in \autoref{fig:architecture}, we do not use weight sharing between the Reference Encoder and the Encoder, as we observed a considerable performance drop in doing so.
We believe the underlying reason is that the functions approximated by these two modules are different;
the inputs to the Reference Encoder are aligned in time while the inputs to the Encoder are not.
We highlight that compared to S2S~\cite{xu2018youtub}, the only added element is the light-weight Merge Layer~(\autoref{fig:architecture}).
The rest of the components remain unchanged, by modifying the task of the Initializer Network to Reference Encoder.

\textbf{Attention to the First Frame}.
As suggested in~\cite{ebert2017self} for the Video Prediction task, the first frame of the sequence is of significant importance as it contains the reference information which can be utilized for recovering from occlusion.
We note that by definition, the target object is present in the first frame.
By computing the correspondences between the object appearance after occlusion and in the first frame, the model is able to re-detect the target.
Additionally, \cite{yang2019anchor,wug2018fast} demonstrate the effectiveness of using the first frame as an anchor or reference frame.
In~\cite{wug2018fast}, the authors propose a Siamese architecture that learns to segment the object of interest by finding the feature correspondences between the target object in the first frame and the current frame.
Although this model's performance suffers in scenarios with drastic appearance change, it reveals the importance of rigorously using the data in the first frame.
We use the same reference encoder and merge layer for integrating the first frame features.
We hypothesize that this modification can be considered as an attention mechanism~\cite{bahdanau2014neural}, where the attention span is limited to the first frame.
Using attention is a standard solution to address this finite memory in the RNNs, by providing additional context to the memory module.
The context vector is usually generated from a weighted combination of the embeddings from all the time steps.
However, in high-dimensional data such as video, it would be computationally demanding to store the features and compute all the frames' attention weights.

The resulting architecture is shown in~\autoref{fig:architecture} and can be formulated as:
\begin{equation}
    h_{0}, c_{0} = \bm{0}
\label{eq:5}
\end{equation}
\begin{equation}
    \Tilde{x}_{0} = \text{Reference\_Encoder}(x_0, y_0)
\label{eq:x0}
\end{equation}
\begin{equation}
    \Tilde{x}_{t-1} = \text{Reference\_Encoder}(x_{t-1}, y_{t-1})
\label{eq:xt_1}
\end{equation}
\begin{equation}
    \Tilde{x}_{t} = \text{Encoder}(x_{t}, y_{t-1})
\label{eq:6}
\end{equation}
\begin{equation}
    h_{t}, c_{t} = \text{RNN}(\Tilde{x}_{t}, h_{t-1}, c_{t-1})
\label{eq:7}
\end{equation}
\begin{equation}
    \hat{y} = \text{Decoder}(\Tilde{x}_{0}, \Tilde{x}_{t-1}, h)
\label{eq:8}
\end{equation}
where \textit{x} and \textit{y} are the RGB image and the binary segmentation mask, and $\bm{0} \in R^{d}$ with $d$ as the feature dimension.
Here the merge layer is considered as part of the decoder.

\vspace{+1mm}
\textbf{Training Objective}.
For the loss function, we utilize a linear combination of the balanced Binary Cross-Entropy~(BCE) loss and an auxiliary loss~\cite{azimi2020revisiting}:
\begin{equation}
 L_\text{total} = \lambda\ L_\text{seg} + (1-\lambda)\ L_\text{aux}
 \label{eq:total}
\end{equation}
The auxiliary task employed here is border classification.
For this task, a border class is assigned to each pixel based on its location with respect to the object boundary, where the boundary target classes are assigned based on a distance transform~\cite{hayder2017boundary}.
This term provides fine-grained location information for each pixel resulting in improved boundary detection $F$-score.
For more details, please refer to~\cite{azimi2020revisiting}.

The balanced BCE loss is computed as in~\cite{caelles2017one} :
\begin{equation}
  \begin{aligned}
    L_\text{seg}(\textbf{W}) & = \sum_{t=1}^{T}(-\beta\sum\limits_{j\in Y_{+}}\log P(y_{j}=1|X;\textbf{W})\\
      & -(1-\beta)\sum\limits_{j\in Y_{-}}\log P(y_{j}=0|X;\textbf{W}))
  \end{aligned}
  \label{eq:seg_loss}
\end{equation}
with $X$ as input, $\textbf{W}$ as the model parameters, $Y_{+}$ and $Y_{-}$ standing for the foreground and background groundtruth labels, $\beta=|Y_{-}|/|Y|$, and $|Y|=|Y_{-}|+|Y_{+}|$.
This loss addresses the data imbalance between the foreground and the background classes by the weighting factors $\beta$.
\section{Implementation Details}
In this section, we explain the implementation details of our hybrid model.
The code and the pre-trained models will be made publicly available  \footnote{\url{https://github.com/fatemehazimi990/HS2S}}.
\subsection{Encoder Networks}
In the S2S model, a VGG network~\cite{simonyan2014very} is used as the backbone for the initializer and encoder networks.
In this work, we utilize a ResNet50~\cite{he2016deep} architecture, pre-trained on ImageNet~\cite{deng2009imagenet}.
We remove the last average pooling and the fully connected layers, which are specific for image classification.
Furthermore, we add an extra $1\times1$ convolution layer to reduce the number of output channels from $2048$ to $1024$.
The number of input channels is altered to $4$, as we feed the RGB image and the binary segmentation mask to the encoder.
We utilize skip connections~\cite{ronneberger2015u} between the encoder and the decoder at every spatial resolution of the feature map ($5$ levels in total) to capture the fine details lost in the pooling operations and reducing the spatial size of the feature map.
Moreover, we use an additional RNN module in the first skip connection, as suggested in~\cite{azimi2020revisiting}.
The impact of changing the backbone network in the S2S model from VGG to ResNet on the segmentation accuracy is studied in \autoref{tab:component}.
\subsection{RNN and Merge Layer}
For the RNN component, we use a ConvLSTM layer~\cite{xingjian2015convolutional}, with a kernel-size of $3\times3$ and $1024$ filters.
As suggested in~\cite{xu2018youtub}, $Sigmoid$ and $ReLU$ activations are used for the gate and state outputs, respectively.

The merging layer's role is to perform correspondence matching between the RNN hidden state (the spatio-temporal features) and the outputs from the reference encoder.
There are different ways that can be used for this layer based on similarity matching and cosine distance.
Similar to~\cite{wug2018fast}, we utilize Global Convolution (GC) layers~\cite{peng2017large} for this function.
Two GC layers with an effective kernel size of $7\times7$ are employed to combine the RNN hidden state with the reference features and the features from the previous time step ($\Tilde{x}_{0}$ and $\Tilde{x}_{t-1}$ as in \Cref{eq:x0,eq:xt_1}).
The output of these two layers are then merged using a $1\times1$ convolution and then fed into the decoder network.
\subsection{Decoder}
The decoder network consists of five up-sampling layers followed by $5\times5$ convolution layers with $512$, $256$, $128$, $64$, and $64$ number of filters, respectively.
In the last layer, a $Conv_{1\times1}$ maps the $64$ channels to $1$ and a $Sigmoid$ activation is used to generate the binary segmentation scores (for the foreground and background classes).
The features from the skip connections are merged into the decoder using a $1\times1$ convolution layers.
$ReLU$ activation is used after each convolution layer, except for the last layer, where we use $Sigmoid$ activation to generate the segmentation output.
\subsection{Training Details}
For data augmentation, we apply random horizontal flipping as well as affine transformations.
The $\lambda$ in \autoref{eq:total} is set to $0.8$.
We use Adam optimizer~\cite{kingma2014adam} with an initial learning rate of $10^{-4}$.
We gradually lower the learning rate in the final phase of training when the loss is stable.
During the training, we use video snippets with $5$ to $10$ frames and a batch size of $16$.

Additionally, we apply a curriculum learning method as suggested for sequence prediction tasks~\cite{bengio2015scheduled}.
To this end, we use the ground-truth for the segmentation mask input in the earlier stages of training where the model output is not yet satisfactory.
This phase is known as \textit{teacher forcing}.
Next, with a pre-defined probabilistic scheme~\cite{bengio2015scheduled}, we randomly choose between using the ground-truth or the model-generated segmentation mask, on a per-frame basis.
This process helps to close the gap between the training and inference data distributions (during the inference, only the model-generated masks are used).
\section{Experiments}
This section provides the experimental results for our hybrid model and a comparison with other state-of-the-art methods.
Additionally, we analyze our hybrid model's behavior on the two challenging scenarios occlusion and long sequences.
\subsection{Evaluation on Youtube-VOS and DAVIS2017}
We evaluate our model on the Youtube-VOS~\cite{xu2018youtub} dataset (the largest for Video Object Segmentation), as well as the DAVIS2017 dataset~\cite{pont20172017}.

We report the standard metrics of the task, namely \textit{Region Similarity} and \textit{Boundary Accuracy} ($F\&J$)~\cite{perazzi2016benchmark}.
The $F$ score measures the quality of the estimated segmentation boundaries and the Jaccard index $J$ measures the intersection over union area between the model output and the ground-truth segmentation mask.

\autoref{tab:yvos} shows a comparison of our model with other state-of-the-art methods.
The upper and lower sections include the methods with and without online training.
During the online training, the model is further fine-tuned on the first frame (where the segmentation mask is available) at test time;
Although this stage significantly improves the segmentation accuracy, it results in slow inference which is not practical for real-time applications.
Despite this, we see that our model without online training still outperforms the S2S model with online training.
The performance improvement compared to RGMP~\cite{wug2018fast}~(matching-based) and S2S~\cite{xu2018youtub}~(RNN-based) models strongly indicates that both propagation and matching information are required for better segmentation quality.
Moreover, our method achieves similar performance to STM~\cite{oh2019video} when training on the same amount of data~(not using synthetic data generated from image segmentation datasets) without relying on an external memory module;
Therefore, our model is less memory-constrained during the inference stage compared to methods using external memory that are prone to memory overflow for longer sequences~(in ~\cite{oh2019video}, authors save every 5th frame to the memory to avoid the GPU memory overflow during the test phase).

\autoref{fig:sample1} illustrates some visual examples from our model.
As we see, our model can properly track the target object in the presence of similar object instances as well as occlusion.
More visual samples are provided in the supplementary material.
\begin{table}
\caption{A comparison with the state-of-the-art methods on the Youtube-VOS dataset~\cite{xu2018youtub}. The upper part of the table shows models with online training, the lower part without.
All scores are in percent.
RVOS, S2S, and S2S++ are the RNN-based architectures.
As shown in this table, our hybrid model outperforms the S2S(no-OL) baseline model with an average improvement of 11.2 pp.
STM- refers to results in~\cite{oh2019video}, with the same amount of training data for a fair comparison.
We can see that our method can achieve similar results to STM-, without requiring an external memory module.
}
\centering
\begin{tabular}{|c c c c| }
\hline
Method & $\mathit{J}$ & $\mathit{F}$ & $\mathit{F\&J}$\\
\hline\hline
OSVOS~\cite{Man+18b} & 57.0 & 60.6 & 58.8\\
MaskTrack~\cite{perazzi2017learning} & 52.5 & 53.7 & 50.6\\
S2S(OL)~\cite{xu2018youtub} & \textbf{63.25} & \textbf{65.6} & \textbf{64.4}\\
\hline
OSMN~\cite{yang2018efficient} & 50.3 & 52.1 & 51.2 \\
RGMP~\cite{wug2018fast} & 52.4 & 55.3 & 53.8\\
RVOS~\cite{ventura2019rvos} & 54.6 & 59.1 & 56.8\\
A-GAME~\cite{johnander2019generative} & 64.3 & 67.9 & 66.1\\
S2S(no-OL)~\cite{xu2018youtub} & 57.5 & 57.9 & 57.7\\
S2S++~\cite{azimi2020revisiting} & 58.8 & 63.2 & 61.0\\
STM-~\cite{oh2019video} & - & - & 68.2 \\
TVOS~\cite{zhang2020transductive} & 65.4 & 70.5 & 67.2 \\
HS2S~(ours)& \textbf{66.1} & \textbf{71.7} & \textbf{68.9}\\
\hline
\end{tabular}
\label{tab:yvos}
\end{table}
\begin{figure*}[t!]
    \centering
    \includegraphics[width=\linewidth]{pics/samples_3.pdf}
    \caption{Visual samples of our model on Youtube-VOS validation set. As can be observed, our method can successfully segment sequences with similar object instances, even in the presence of occlusion.}
    \label{fig:sample1}
\end{figure*}

To assess the generalization of our model after training on Youtube-VOS, we freeze the weights and evaluate the model on DAVIS2017 dataset~\cite{pont20172017}.
The results can be seen in \autoref{tab:davis}.
We observe that our hybrid model outperforms the independent RNN-based and matching-based methods, even without fine-tuning on this dataset.
\begin{table}
\caption{A comparison between the independent RNN-based~(RVOS) and matching-based~(RGMP) models and our hybrid method on the DAVIS2017 dataset~\cite{pont20172017}~(test-val).
HS2S- shows the results of our model trained on Youtube-VOS without fine-tuning on DAVIS2017.
The results of the S2S model on DAVIS2017 were not available.
}
\centering
\begin{tabular}{|c c c c|}
\hline
Method & $\mathit{J}$ & $\mathit{F}$ & $\mathit{F\&J}$\\
\hline\hline
S2S~\cite{xu2018youtub} & - & - & - \\
RVOS~\cite{ventura2019rvos} & 52.7 & 58.1 & 55.4\\
RGMP~\cite{wug2018fast} & 58.1 & 61.5 8 & 59.8\\
HS2S-~(ours) & \textbf{58.9} & \textbf{63.4} & \textbf{61.1} \\
\hline
\end{tabular}
\label{tab:davis}
\end{table}
\subsection{Analysis of Sequence Length and Occlusion}
To quantitatively assess our model's effectiveness, we evaluate it in challenging scenarios such as occlusion and longer sequences.
As the validation set of the Youtube-VOS dataset is not released, we use the 80:20-splits of the training set from~\cite{ventura2019rvos} for training and evaluation.
For the S2S model results, we further used our re-implementation as the code for their work is not publicly available.
Furthermore, we use the ResNet50 architecture as backbone for both models for a fair comparison (to our disadvantage, as it improves the overall evaluation score of $57.3\%$ for S2S (as reported in \cite{xu2018youtub}) to $60\%$ for our re-implementation S2S*).

\autoref{fig:lenhistogram} shows the sequence length distribution of the Youtube-VOS training set (one sequence per object in each video).
As can be seen, the length varies between $1$ to about $35$ frames in a very non-uniform fashion.
To study the impact of the video length on the segmentation scores, we pick the sequences longer than $20$ frames and measure the scores for frames with $t<10$ (considered as early frames) and frames with $t>20$ (considered as late frames), separately.
\begin{figure}[t]
  \centering
  \includegraphics[width=1.\linewidth]{pics/len_hist.pdf}
   \caption{Distribution of the sequence length (per object) in the Youtube-VOS dataset.
   In Youtube-VOS, the video frame rate is reduced to $30$ fps, and the annotations are provided every fifth frame~($6$ fps).
   Therefore, a sequence with $36$ labeled frames spans $180$ time steps in the original frame rate.}
\label{fig:lenhistogram}
\end{figure}
\begin{table}
\caption{A study on the impact of sequence length on the segmentation accuracy.
For this experiment, we picked the video sequences with more than $20$ frames.
Then we compute the $F$ and $J$ scores for frames earlier ($t<10$) and later ($t>20$) in the sequence.
As the results show, there is a performance drop as the time step increases.
However, our hybrid model's performance drops a lot less than the baseline's.}
\centering
\begin{tabular}{|l c c c c |}
\hline
Method & $\mathit{F_{l<10}}$ & $\mathit{J_{l<10}}$ & $\mathit{F_{l>20}}$ & $\mathit{J_{l>20}}$  \\
\hline\hline
S2S* & 74.4 & 73.7 & 54.5 & 54.6  \\
HS2S~(ours) & \textbf{77.1} & \textbf{76.3} & \textbf{65.5} & \textbf{64.2}  \\
\hline
\end{tabular}
\label{tab:len}
\end{table}
As presented in \autoref{tab:len}, we observe that the hybrid model improves the late frame accuracy significantly and reduces the performance gap between the early and late frames.
This observation confirms the effectiveness of the hybrid path for utilizing the information from spatio-temporal features as well as the correspondence matching.

The histogram in \autoref{fig:occlusionhistogram} shows the number of sequences with occlusion in Youtube-VOS training set.
Each bin in the histogram shows the occlusion duration, and the $y$ axis indicates the number of sequences that belong to each bin.
As can be seen from this plot, the occlusion duration varies between $1$ to $25$ frames.
\begin{figure}[t]
\centering
  \includegraphics[width=1.\linewidth]{pics/occ_hist.pdf}
   \caption{The number of occluded sequences (per object) in Youtube-VOS train set, for different occlusion lengths and with three occlusion thresholds (shifted by 1/3 for better visibility).}
\label{fig:occlusionhistogram}
\end{figure}
\begin{table}
\caption{A study on the impact of occlusion on the segmentation quality.
The scores presented in this table are the average of $F$ and $J$ scores in percentages, when considering different thresholds (in pixels) for occlusion.
The $avg$ score refers to the average result for all the sequences in the 20-split.
For the other columns, we only considered the frames after ending the first occlusion period~(when the target object re-appears in the scene).
}
\centering
\begin{tabular}{|l c c c c|}
\hline
Method & $\mathit{avg}$ & $\mathit{th: 0}$ & $\mathit{th: 50}$ & $\mathit{th: 100}$ \\
\hline\hline
S2S* & 63.3 & 33.6 & 30.8 & 33.1\\
HS2S~(ours) & \textbf{69.0} & \textbf{40.2} & \textbf{39.3} & \textbf{47.7}\\
\hline
\end{tabular}
\label{tab:occlusion}
\end{table}
To study our model's effectiveness in handling occlusion, we report the scores for frames appearing \textit{after} a first occlusion in \autoref{tab:occlusion}.
An occlusion is considered a scenario where the object leaves the scene entirely and re-appears again.
As the areas below $100$ pixels are almost not visible (and could be considered as labeling noise), we also consider occlusions at three different thresholds of 0, 50, and 100 pixels.
As we can see in the table, occlusion is a challenging scenario with significantly lower scores than the average sequence scores.
However, our proposed approach again succeeds in defending its considerable improvement over the S2S baseline.
\section{Ablation Study}
\label{ablation}
In this section, we present an ablation study on the impact of different components of our model.
In addition, we provide the results for a variant of our model where we use cosine similarity~\cite{wang2018non} for the merge layer instead of global convolution (referred to as HS2S$_{sim}$).

\autoref{tab:component} presents the segmentation scores when different components in our model are added one at a time.
The results for \textit{S2S*} are obtained from our re-implementation of the S2S model with ResNet50 backbone.
As it can be seen from the results, utilizing the first frame as the reference~(HS2S$_{0}$) and using the hybrid match-propagate strategy~(HS2S$_{t-1}$) both improve the segmentation quality.
Moreover, the enhancements add up when they are integrated into a single model~(HS2S).

\begin{table}
\caption{An ablation study on the impact of different components in our model.
S2S* is our re-implementation of the S2S method with the same backbone as our model, for a fair comparison (this version achieves a better segmentation accuracy).
S2S$_{0}$ refers to our model without the hybrid propagation, only using the first frame as reference.
S2S$_{t-1}$ is our model with hybrid propagation and without utilizing the first frame.
In HS2S$_\text{sim}$, we implemented the merge layer (\autoref{fig:architecture}) using cosine similarity instead of Global Convolution.}
\centering
\begin{tabular}{|c c c c|}
\hline
Method & $\mathit{J}$ & $\mathit{F}$ & $\mathit{F\&J}$\\
\hline\hline
S2S~\cite{xu2018youtub} & 57.5 & 57.9 & 57.7\\
S2S* & 59.1 & 63.7 & 61.4 \\
HS2S$_{0}$ & 64.0 & 68.95 & 66.5\\
HS2S$_{t-1}$ & 63.6 & 68.7 & 66.2\\
HS2S & \textbf{66.1} & \textbf{71.7} &\textbf{68.9} \\
HS2S$_{sim}$& 64.35 & 69.35 & 66.9\\
\hline
\end{tabular}
\label{tab:component}
\end{table}
\section{Conclusion}
In this work, we presented a hybrid architecture for the task of one-shot Video Object Segmentation.
To this end, we combined the merits of RNN-based approaches and
models based on correspondence matching.
We showed that the advantages of these two categories are complementary, and can assist each other in challenging scenarios.
Our experiments demonstrate that both mechanisms are required for obtaining better segmentation quality.

Furthermore, we provided an analysis of two challenging scenarios: occlusion and longer sequences.
We observed that our hybrid model achieves a significant improvement in robustness compared to the baselines that rely on RNNs~\cite{xu2018youtub} and reference guidance~\cite{wug2018fast}.
However, occlusion remains an open challenge for future investigation, as the performance in this scenario is considerably lower than the average.
Moreover, we believe that integrating global information and modeling the interactions between the objects in the scene is a promising direction for future work.

{\small
\bibliographystyle{ieee_fullname}