\section{Introduction}
\label{intro}

A distinctive common feature of the giant planets within the Solar
System is the presence of ring systems orbiting the planet. Rings
systems have been detected and studied extensively for each of Jupiter
\citep{showalter1987,porco2003}, Saturn
\citep{pollack1975d,porco2005a}, Uranus \citep{elliot1977c,tyler1986},
and Neptune \citep{lane1989,showalter2020}. In particular, the
prominent rings of Saturn have been the source of numerous
investigations with regards to their formation
\citep{goldreich1978c,charnoz2009b} and dynamics
\citep{goldreich1978b,bridges1984}. For example, density waves
detected within Saturn's rings have been utilized as an effective
means to infer oscillations within the planetary interior
\citep{hedman2013c} and determine differential rotation of the outer
envelope \citep{mankovich2019} and a diffuse core
\citep{mankovich2021}. Additionally, the age of Saturn's rings has a
range of estimated values, from several hundred Myrs
\citep{zhang2017h,dubinski2019,iess2019} to as old as the Solar System
\citep{crida2019}. Age determinations depend on many factors, such as
the velocity dispersion and interaction between ring particles
\citep{salo1995}. Rings also stand as a possible record of past
collision events, and indeed the formation of Saturn's rings has been
suggested as the result of a substantial moon being desiccated by
tidal forces as it spiraled into the planet \citep{canup2010}.

By comparison, the Jupiter system contains a substantially more modest
ring system that has been extensively studied via data from such
missions as Voyager and Galileo, as well as ground-based observations
\citep{smith1979b,ockbertbell1999}. Theories regarding the origin and
evolution of the Jovian rings vary, such as their possible formation
along with the Galilean satellites \citep{prentice1979b}, and the
contributions of collisional ejecta lost from inclined satellites
\citep{burns1999} and escaping ejecta from the Galilean satellites
and/or the inner small moons
\citep{burns1999,esposito2002,krivov2002a}. Further possible sources
of potential ring material originate from impact debris
\citep{ahrens1994a,hueso2013,sankar2020} and the tidal disruption of
satellites \citep{hyodo2017b} or large passing Kuiper belt objects
\citep{hyodo2017a}. Such events are likely stochastic in nature whose
frequency is an age-dependent phenomenon
\citep{horner2008a}. Moreover, the dynamical evolution of Jupiter's
rings can have complicated explanations, partially with interactions
between ions and the Jovian magnetosphere \citep{horanyi1996b} and the
incorporation of dissipation effects \citep{greenberg1983a}. The
formation and dynamical evolution of Jupiter's rings have important
consequences regarding the prevalence of rings of giant
exoplanets. Exoplanetary rings can be a source of confusion when
evaluating the true nature of the planet and its properties
\citep{piro2018c,piro2020a} and their successful detection may reveal
important information regarding the formation of the planet and its
local environment \citep{arnold2004c,zuluaga2015a,sucerquia2020b}.
The Jupiter and Saturn systems demonstrate that the presence and
sustainability of rings may be an intricate function of the
architecture of planetary moons, as well as the intrinsic properties
of the planet itself. The discoveries and/or limits on exoplanet rings
and moons will provide crucial statistical data to further understand
how rings may have formed and evolved in our Solar System
\citep{kenworthy2015b}.

In this work, we provide the results of an extensive dynamical
simulation that tests regions of long-term dynamical stability for
ring systems near the plane of the Jovian equator, and in the presence
of the Galilean moons. In Section~\ref{arch}, we describe the
architectures of the Jupiter and Saturn systems, specifically the
structure of their rings with respect to the orbits of their
respective moons. Section~\ref{dynamics} provides the methodology and
results for our dynamical simulation of stable orbits at locations
that extend from within the rigid satellite Roche limit to 10\% of the
Jupiter Hill radius. The implications of our simulation results are
discussed in Section~\ref{discussion}, both in terms of the potential
for past/future Jovian ring systems and within the context of giant
exoplanets. Concluding remarks and suggestions for observational tests
are provided in Section~\ref{conclusions}.

\section{Architecture of Jupiter and Saturn Systems}
\label{arch}

At the time of writing, the Solar System is known to contain over 200
moons, with Jupiter and Saturn harboring at least 79 and 82 moons,
respectively\footnote{\tt
  https://solarsystem.nasa.gov/moons/in-depth/}. The prevalence of
moons within the Solar System has been a primary motivator behind the
study of and search for exomoons
\citep[e.g.,][]{hinkel2013b,kipping2013d,heller2014c,hill2018}. It has
further been suggested that Solar System regular moons may serve as
analogs of compact exoplanetary systems in terms of their formation
and architectures
\citep{kane2013e,makarov2018,dobos2019,batygin2020b}. Regular moons
are particularly notable in that they likely formed with the planet,
as evidenced by their equatorial prograde orbits, and are often large
enough to exhibit hydrostatic equilibrium, resulting in a near-round
morphology. For example, the four Galilean moons likely formed from
the protoplanetary disk surrounding Jupiter
\citep{ogihara2012,heller2015e}, possibly catalyzed by migration of
Saturn \citep{ronnet2018}, and now contain $\sim$99.997\% of the total
mass orbiting the planet. The mass contained within the regular moons
will therefore have a significant influence on the dynamics of ring
formation and sustainability.

\begin{figure*}
  \begin{center}
    \includegraphics[angle=270,width=16.0cm]{f01.ps}
  \end{center}
  \caption{The regular moons and rings of the Jupiter and Saturn
    systems, where the relative sizes of the moons are shown, and
    their semi-major axes are provided in units of the host planet
    radii. The extent of the current ring systems, including dense and
    tenuous rings, are shown as gray regions, and the vertical red
    dashed lines are the fluid satellite Roche limit for each planet.}
  \label{fig:radii}
\end{figure*}

Figure~\ref{fig:radii} provides a scaled view of the Jupiter and
Saturn systems, where the separation from the planetary centers,
located at zero, are provided in units of the respective host planet
radius. The regular moons are shown as blue circles, where their sizes
are relative to each other rather than in units of planetary radii,
and the names of the major moons are provided. It is worth noting that
all of Jupiter's regular moons, including the Galilean moons, have
semi-major axes that are within 1/25 of the Jupiter Hill radius,
possibly a result of the mass distribution of the circumplanetary disk
and moon migration processes that may have occurred. The extent of the
rings in units of the planetary radii are shown as gray regions, and
the vertical red dashed lines indicate the location of the fluid
satellite Roche limit for each planet. The depiction of Jupiter's
rings include the Halo to Thebe gossamer rings, which span a distance
of 1.29--3.16 Jupiter radii. Similarly, the depiction of Saturn's
rings include rings D--E, which extend a distance of 1.11--7.96 Saturn
radii. The fluid satellite Roche limits are 2.76 and 2.22 planet radii
for Jupiter and Saturn, respectively. Note that the rings depicted in
Figure~\ref{fig:radii} contain both ``dense'' and ``tenuous'' rings,
both of which have different sources of material and physical
processes acting upon them \citep{daisaka2001}. For example, Saturn's
E ring originates mainly from the cryovolcanic plumes of Enceladus,
and is considerably more tenuous than the main ring
\citep{horanyi2009,cuzzi2010a}.

Figure~\ref{fig:radii} highlights the differences in the distribution
of moon mass with respect to the rings for each of the planets. In
particular, the rings of Saturn contain numerous small moons, whose
presence can both feed the rings with new material, and also
``shepherd'' the rings through their gravitational influence
\citep{petit1988c,charnoz2011b,cuzzi2014b,nakajima2020}. Indeed, moons
can accrete relatively rapidly from ring material beyond the Roche
limit, and such rings are thought to have been a source of numerous
moons within the present Saturn architecture
\citep{charnoz2010,crida2012,salmon2017}. Gaps in Saturn's rings form
through several processes, among them orbital resonances with moons,
such as the relationship between the Cassini Division and Mimas
\citep{goldreich1978c,iess2019,noyelles2019}. Titan is relatively far
from the main ring structure, but its presence does result in a
ringlet within the inner C ring through the effect of orbital
resonance \citep{porco1984a}. However, there is evidence to suggest
that Titan has experienced an outward migration through tidal
dissipation \citep{lainey2020}. This is somewhat in contrast to the
effects of the more massive Galilean moons on the dynamical
environment around Jupiter, particularly as Io, Europa, and Ganymede
are located in a 4:2:1 Laplace resonance
\citep{malhotra1991,peale2002b}. Their orbital configuration is
interpreted as strong evidence that the moons migrated inward either
during formation or soon thereafter
\citep{greenberg1987,peale2002b,sasaki2010,ogihara2012}. This may have
resulted in a significant gravitational truncation of a potential
massive ring system for Jupiter, such as that seen for Saturn,
depending on the formation and migration timescales of the Galilean
moons. Overall, there are significant differences between the ways in
which dense and tenuous rings interact with the planet and satellite
system, partially depending on the location of the rings relative to
the Roche limits. The simulations described in this paper consider
only dense rings for a range of semi-major axes. The physical
processes acting upon the rings are described further in
Section~\ref{rings}.

\section{Dynamical Simulations}
\label{dynamics}

This section describes the dynamical simulations, including their
configuration, results for the Galilean moons, and injection of ring
particles.

\subsection{Dynamics of the Galilean Moons}
\label{galilean}

As discussed in Section~\ref{arch}, an important factor in the
formation and evolution of planetary rings is the moons present in the
system. In this context, the Galilean moons are by far the largest
gravitational influence on the presence of rings within the inner part
of the Jupiter system. Thus, the dynamical evolution of the Galilean
moons is a crucial component of evaluating potential ring
sustainability. There are substantial data that contribute toward
accurate ephemerides of the moons, and their dynamics have previously
been studied in detail
\citep{lieske1980,greenberg1987,lainey2004a,lari2018}. However, the
vast majority of these studies focus on the short-term
($\sim$100~year) dynamics of the moons, whereas this study is
concentrated on timescales related to the sustainability of planetary
rings (1--10 million years).

\begin{figure*}
  \begin{center}
    \includegraphics[angle=270,width=16.0cm]{f02.ps}
  \end{center}
  \caption{Eccentricity as a function of time for the Galilean
    satellites: Io, Europa, Ganymede, and Callisto. The eccentricity
    variations were recorded every 100 years during a $10^6$~year
    simulation, as described in Section~\ref{dynamics}.}
  \label{fig:ecc}
\end{figure*}

The simulations carried out for this work were conducted within the
dynamical simulation package REBOUND \citep{rein2012a} with the
symplectic integrator WHFast \citep{rein2015c}. The initial conditions
of the system were constructed to reproduce the configuration of the
Galilean moon system, incorporating the current orbital elements
extracted from the Horizons DE431 ephemerides \citep{folkner2014}, and
including the effects of Jupiter's oblateness \citep{tamayo2020a}. The
dynamical simulation presented in this work provides an independent
assessment for the eccentricity evolution of the Galilean moons, the
results of which are shown in Figure~\ref{fig:ecc} for a duration of
$10^6$~years. The combination of the resonance configuration and tidal
dissipation present in the Galilean system ensures that the moons have
exceptionally stable orbits through time and that they remain largely
circular. Note that tidal dissipation was not included in our
dynamical simulation. Figure~\ref{fig:ecc} shows that the
eccentricities remain below 2\%, 3\%, 1\%, and 1\% for Io, Europa,
Ganymede, and Callisto, respectively. The short-term analyses of the
Galilean moon eccentricities, such as the semi-analytical model
provided by \citet{lari2018}, reveal a transfer of angular momentum
within the Laplace resonance of the three inner moons with a period in
the range 400-500 days. The long-term eccentricity evolution
represented in Figure~\ref{fig:ecc} does not reveal periodic behavior
at significantly longer timescales, but does demonstrate that angular
momentum transfer results in higher eccentricity variations for the
less massive inner two moons than the more massive outer two
moons. The slightly higher eccentricity variations for Io and Europa
increases regions of dynamical instability in their vicinity, and thus
has consequences for long-term ring stability close to Jupiter.

\subsection{Particle Injection and Ring Stability}
\label{particle}

To explore the gravitational constraints that the Galilean moons
impose upon potential ring particles orbiting Jupiter, we conducted a
suite of dynamical simulations for the system based upon the
architecture framework described in Section~\ref{galilean}. The
stability of an extensive ring system and possible moon forming
material around Jupiter was tested by introducing a series of test
particles to the Galilean moon system and evaluating their
survivability at different locations within the system. The test
particles were assumed to have a density of water ice
(0.917~g/cm$^3$), the value for which is representative of most of the
materials in Saturn's ring system, and a spherical shape with radius
of $\sim$1~meter, yielding a total particle mass of
$\sim$3841~kg. Thus, we are considering only dense ring material and
the gravitational perturbations acting upon them (see
Section~\ref{rings}). The particle orbits were set to be circular with
an inclination of $\sim$0\degr\ with respect to the Jupiter equator
(coplanar with the Galilean moons). The test particles were placed at
different locations from Jupiter, extending from inside the rigid
satellite Roche limit (see Figure~\ref{fig:radii}) to $1/10$ of the
Jupiter Hill radius. This region was sampled with 1000 evenly spaced
locations, resulting in a location step size of $\sim$4963~km. The
simulations were carried out with both $10^6$~year and $10^7$~year
integration times for each of the separation cases, which translates
to $4.6 \times 10^6$ and $4.6 \times 10^7$ orbits at the outer edge of
our sample space, respectively. For each simulation, the time step was
set to be 0.05 of Io's orbital period (2.1~hours), except for cases
where the test particle locations were inside Io's orbit, where the
time step was adjusted to 0.05 of the orbital period at each particle
location.  This ensured an adequate time resolution to properly sample
the dynamical interactions between test particles and the Galilean
moons. The orbital properties of the particles and Galilean moons were
output every 100 years during the simulation and the survival rates of
the test particle were calculated at each orbital separation. Test
particle survival was based upon the elapsed time to be either ejected
from the system or suffering a collision with one of the large system
bodies during the entire simulation.

\begin{figure*}
  \begin{center}
    \includegraphics[angle=270,width=16.0cm]{f03a.ps} \\
    \includegraphics[angle=270,width=16.0cm]{f03b.ps} \\
    \includegraphics[angle=270,width=16.0cm]{f03c.ps}
    \end{center}
  \caption{Results of particle injection and survival for the $10^6$
    year (top panel) and $10^7$ year (middle panel) dynamical
    simulations. The horizontal axis is the separation from Jupiter in
    planetary radii, and the vertical axis shows the percentage of the
    total dynamical simulation that particles survived at that
    location, represented by the green line. The vertical dotted lines
    represent the semi-major axes of the Galilean moons, and the
    vertical dashed lines represent the locations of the rigid and
    fluid Roche limits. The bottom panel shows the change in
    eccentricity that occurs for each particle as a function of their
    initial semi-major axis during the course of the $10^7$ year
    simulations.}
  \label{fig:sim}
\end{figure*}

The primary results of our simulations that include particle
injections are shown in the top two panels of Figure~\ref{fig:sim},
and are represented as the percentage of the total simulation survived
by particles for each separation location. The simulation durations of
$10^6$~years and $10^7$ years are shown in the top and bottom panels,
respectively, and the location of the Galilean moons are indicated by
vertical dotted lines. The vertical dashed lines indicate the location
of both the rigid and fluid body Roche limits. The top panel shows
that, even for $10^6$ years, the region surrounding the Laplace
resonance of Io, Europa, and Ganymede is rendered unstable,
exacerbated by the slight eccentricity variations of Io and Europa
described in Section~\ref{galilean}. The inner limits of dynamical
stability are located at $\sim$3.55 Jupiter radii and $\sim$3.12
Jupiter radii for the $10^6$~year and $10^7$~year simulations,
respectively. The current Jupiter ring system, depicted in
Figure~\ref{fig:radii}, extends to $\sim$3.16 Jupiter radii,
indicating that the ring is young ($< 10^7$~years) and/or is being
sustained with additional material to the Thebe extension
\citep{borisov2021c}. The structure of the dynamical stability beyond
the orbit of Callisto is a complicated result of Ganymede and Callisto
resonance locations. For example, the dynamical instability located at
$\sim$31.2 Jupiter radii, partially present at $10^6$~years and
significantly more pronounced at $10^7$~years, is the 1:3 resonance
location with Ganymede. Furthermore, there is an island of stability
located at $\sim$41.8 Jupiter radii, which is not evident in the
$10^6$~year results but becomes apparent at $10^7$~years, which is a
result of a 1:2 resonance with Callisto. The effect of resonances is
further emphasized in the bottom panel of Figure~\ref{fig:sim}, which
shows the change in eccentricity, $\delta e$, that occurs during the
course of the $10^7$ year simulations. It is worth noting that,
although the bottom panel of Figure~\ref{fig:sim} implies that
particles close to the Roche limits are predominantly scattered into
high eccentricity orbits, stable low-eccentricity orbits can exist
within that region. For example, the inner Jovian moons of Metis,
Adrastea, Amalthea, and Thebe have semmi-major axes that lie within
the range 1.8--3.2 Jupiter radii and are known to have eccentricities
that are below 0.02 \citep{cooper2006,borisov2020}, although dynamical
interactions with Io have inflated the eccentricity of Thebe
\citep{burns2004a}. We conducted additional simulations for the
specific cases of those four inner moons where, as for the Galilean
moons, current orbital elements were extracted from the Horizons DE431
ephemerides. The results of these simulations show that their mean
eccentricities remained small throughout the entire simulation, with
mean eccentricities of 0.008, 0.008, 0.005, and 0.016 for Metis,
Adrastea, Amalthea, and Thebe, respectively. As noted in
Section~\ref{galilean}, tidal dissipation was not included in our
simulations, and so these dynamically induced eccentricities may be
considered upper limits to their expected values.

The rate at which particles are
lost from within the investigated semi-major region is shown in
Figure~\ref{fig:lossrate}. Almost 30\% of the particles are lost
within the first $10^5$~years of simulation time. After $10^6$~years,
$\sim$65\% of particles remain and, after $10^7$~years, $\sim$52\% of
particles remain. As noted above, lost particles are generally
considered to be the result of ejection or collisions. However, as
described in Section~\ref{arch}, particles beyond the fluid body Roche
limit may also coalesce into moons on relatively short timescales, and
the diversity of eccentricities exhibited in the bottom panel of
Figure~\ref{fig:sim} for many of the surviving particles supports this
possible outcome. Another source of particle loss can be capture by
the Galilean moons, forming circumsatellital rings
\citep{sucerquia2021}. Overall, the Galilean moons carve a substantial
area of instability into the region around Jupiter that may only allow
relatively short-lived ring systems to co-exist with the orbital
architectures.

\begin{figure}
  \includegraphics[angle=270,width=8.5cm]{f04.ps}
  \caption{The loss rate for all particles within the investigated
    semi-major axis region around Jupiter, shown as a function of time
    (fractions of $10^7$~years) and the percentage of particles that
    survive to that time.}
  \label{fig:lossrate}
\end{figure}

\section{Discussion}
\label{discussion}

\subsection{Implications for Jovian Rings}
\label{rings}

As the dominant planetary mass in the Solar System, Jupiter has
undoubtedly had a complex and eventful dynamical history. Such a
history includes significant impacts and other events that have
resulted in raw material for substantial ring formation. The major
question regarding these ring structures pertain to their long-term
viability within the architecture of the system. The results of our
simulations described in Section~\ref{particle} demonstrate that ring
structures within the separation range $\sim$3--29 Jupiter radii are
mostly removed within $10^6$~years. What remains are a system of
ringlets between Ganymede and Callisto ($\sim$20--24 Jupiter radii),
and rings that coincide with the orbit of Callisto. Rings beyond 29
Jupiter radii are viable for $10^6$~years, but such rings would likely
consist of remaining material from planet and moon formation processes
when the Solar System was relatively young. After $10^7$~years, the
dynamical influence of the Galilean moons further desiccates the
material in the region of $\sim$3--29 Jupiter radii, including the
ringlet structure between Ganymede and Callisto. As discussed in
Section~\ref{particle}, orbital resonances with Ganymede and Callisto
also compromise the region beyond 29 Jupiter radii, ensuring that the
primordial material would not have remained beyond a few tens of
millions of years.

The results presented in this work are based primarily on dynamical
interactions within the Jupiter system. However, there are numerous
other factors that have not been taken into account. Events that may
increase the lifetime of ring structures, such as impacts and
outgassing of moons that release solid particles, have not been
considered. Jupiter is well known to suffer relatively frequent
impacts \citep{zahnle2003a,horner2008a,hueso2013,hueso2018b}, such as
the Comet Shoemaker-Levy-9 event \citep{zahnle1994,asphaug1996b}, with
observations of recent impact events helping to place constraints on
ejection material that achieves escape velocity
\citep{ahrens1994a,sankar2020}, and thus may contribute to ring
material. It is worth noting that contributions from isotropic
impactors (compared with ecliptic impactors) may result in a
stochastic distribution of ring material, whereas our simulations
specifically investigate rings coplanar with Jupiter's
equator. Furthermore, orbital precession and collisions will naturally
cause any orbiting debris to collapse into the planet's equatorial
plane. Additional sources of ring material are the grinding down of
small moons and outgassing from the Galilean moons
\citep{burns1999,esposito2002}.

On the other hand, there are numerous processes that serve to decrease
the ring lifetime, such as Poynting-Robertson drag, the Yarkovsky
effect, and the electromagnetic influence of the Jovian magnetosphere
\citep{burns1999,rubincam2006,kobayashi2009a}. Such non-gravitational
forces primarily govern particles that are significantly smaller
($\mu$m--mm) than those used in our simulations \citep{burns1979c},
and so are unlikely to substantially impact the results presented
here. For dense rings, such as those that are considered in our
simulations, a major source of ring material loss beyond the Roche
limit is the accretion into small moons \citep{crida2012}. Such
accretion can happen on relatively small timescales, such that the
loss of ring material in this manner may occur well within the periods
of time considered by our simulations. In fact, our results provided
in Section~\ref{particle} demonstrate that the excitation of particles
into eccentric orbits may further promote collision
scenarios. However, moonlets that form in the investigated region of
semi-major axes will likely remain subject to the instability caused
by the Galilean moons, resulting in possible ejection from the
system. Migration of moons have also played a role in shaping the
dynamical environment around Jupiter. Resonances between moons can
promote inward and outward migration and is thought to have occurred
for both the Jupiter and Saturn moon systems \citep{fuller2016}. The
mechanics of resonance trapping for moons is the same process that
takes place for planetary migration \citep{wyatt2003e} and can greatly
affect their composition, such as the case of the TRAPPIST-1 planets
\citep{unterborn2018a}. Indeed, interaction with the rings themselves
can result in the migration of small moons \citep{bromley2013}, which
may in turn result in their contribution to the rings near the Roche
limit. Furthermore, given the relatively small mass of the test
particles used in our simulations, the calculated life expectancy of
those particles may be considered an upper limit in many cases.

With the various above described competing factors in ring
contributions, the sustainability of planetary rings may be sensitive
to the architecture of the local environment. Based on our
simulations, it is possible that the Galilean moons may be a
significant reason that, integrated over time, Jupiter is unable to
harbor substantial rings. However, there may also have been periods of
Jupiter history during which impact and small moon collision rates
exceeded dynamical disruption by the Galilean moons, allowing
sustained rings significantly more massive than those observed at the
present epoch. Although the sustainability of an extended ring system
beyond the Roche limit may be dominated by the combination of
dynamical effects and moon coalescence, the lifetime of rings within
the Roche limit is substantially more complicated. For example, in the
case of Saturn, an examination of the Pallene dusty ring found that
non-gravitational forces dominate over the dynamical effects with
regards to the sustainability of the ring, although the moon mass
within the Saturnian system is significantly less than that of the
Galilean moons \citep{munozgutierrez2022}. Therefore, the relative
weighting of ring sustainability and dessication factors close to the
planet have yet to be determined, and it is unclear that the dynamical
effects are the primary factor in that region.

\subsection{Implications for Exoplanets}
\label{exoplanets}

\begin{figure}
  \includegraphics[angle=270,width=8.5cm]{f05.ps}
  \caption{Planet masses (Jupiter masses) and Hill radii (planetary
    radii) for the known exoplanets with measured masses and
    radii. The locations of Earth, Jupiter, and Saturn are marked by
    E, J, and S, respectively. The gray region indicates where
    formation of massive moons may be limited by disk migration,
    whilst a larger Hill radius maximizes primordial and new material
    to contribute to ring formation.}
  \label{fig:exo}
\end{figure}

As noted in Section~\ref{intro}, rings are an important component of
planetary evolution, and there are active efforts to search for their
presence around exoplanets
\citep{arnold2004c,zuluaga2015a,sucerquia2020b}. It is expected that
primordial ring structures, inherited from the circumplanetary disk,
contain a mass that scales with the feeding zone, and hence the Hill
radius, of the planet \citep{heller2014c,kaib2015a}. There is also
evidence that moons play a significant role in this early ring
evolution, and that the disks are truncated well inside the extent of
the Hill radius \citep{shabram2013,fujii2017a}. At the present epoch,
the respective prevalence of rings around Jupiter and Saturn suggest
that post-primordial ring formation and evolution does not scale with
planet mass, but rather with the particular moon architecture
associated with the host planet. Indeed, the circumplanetary mass
required to form a Galilean moon system analog may result in a natural
inhibitor to ring formation around massive giant planets. In
particular, moon migration into a Laplace resonance, such as that
described in Section~\ref{arch}, produces strong destabilizing
perturbations that quickly desiccate disk particles within several
tens of planetary radii of the planet. It is further suggested by
\citet{canup2006} that disk migration may limit the maximum size of
moons, beyond which the moons may migrate through the circumplanetary
disk and into the planet. The implication is that Saturn may possibly
represent a ``sweet spot'' of ring formation in terms of mass, Hill
radius, and moon-forming capacity, although this is quite difficult to
gauge fully without more complete knowledge of the Jupiter and Saturn
architecture evolutions. Moreover, careful consideration must be given
to the various processes acting upon ring formation and dessication,
described in Section~\ref{rings}. In particular, there are numerous
non-gravitational forces acting upon ring material within or close to
the Roche limit that can have a dominating effect on ring evolution
(see Section~\ref{rings}). It is also worth noting that these implied
correlations between planet mass and ring/moon prevalence are based
upon the limited inventory of giant planets present in the Solar
System, for which exoplanet studies will provide a much needed
statistical validation.

However, if indeed the presence of relatively massive moon systems is
correlated with planet mass and Hill radius, then the prevalence of
ring systems may be likewise rare amongst many of the discovered giant
exoplanets. Shown in Figure~\ref{fig:exo} are planetary masses
(Jupiter masses) and calculated Hill radii (planetary radii) for known
exoplanets with relavant data available. The data were extracted from
the NASA Exoplanet Archive \citep{akeson2013} and are current as of
2021 December 18 \citep{nea}. For comparison, the locations of Earth,
Jupiter, and Saturn are also marked on the diagram. The gray box
provides a rough guide for what may be considered an optimization
region where the harboring of substantial planetary rings could be
favorable. The planet mass range of the gray box is 10 Earth masses to
1 Jupiter mass to approximately capture planets within the ice/gas
giant regime
\citep{weiss2013,lammer2014a,lopez2014,rogers2015a,chen2017} whilst
limiting the potential for substantial moon formation. The Hill radius
range is for all Hill radii beyond 50 planetary radii since small Hill
radii truncates both the extent of the circumplanetary disk
\citep{shabram2013} and the dynamical viability of moons
\citep{barnes2002b,hinkel2013b,kane2017c}. Saturn is deeply embedded
within the gray region, whilst Uranus and Neptune (also fulfilling the
planet mass criteria) are located at Hill radii of 2613 and 4644
planetary radii, respectively. As described above, planets more
massive than Jupiter, with larger mass proto-satellite disks and a
larger Hill radius in which to engage in moon formation, may
experience significantly reduced timescale of ring
sustainability. Such a timescale reduction would result from the rapid
formation of large moons, whose gravitational presence would, in turn,
either eject the remaining ring material or excite their
eccentricities resulting in further enhancement of moon formation (see
Section~\ref{particle}). The size of the box shown in
Figure~\ref{fig:exo} is empirical in nature and requires further
investigation of the various competing factors toward ring formation
and sustainability, but may serve as a useful guide for testing models
regarding how the presence of substantial moons could potentially
influence the long-term presence of rings around giant planets.

\section{Conclusions}
\label{conclusions}

Planetary rings and moons are very important features of our Solar
System, both in their intrinsic geologic and dynamical properties, and
as crucial signposts of planetary formation and
evolution. Understanding the complex interactions between moons and
rings, and how these vary with planetary mass, composition, Hill
radius, and time, remains one of the most intricate research topics in
planetary science. The dynamical evolution of giant planet systems is
one of the primary ways in which tracing of rings systems and their
ages may be undertaken.

The results of our dynamical simulations demonstrate that the presence
of massive moons, especially systems that have migrated into resonance
traps as for the Galilean moons, can create significant dynamical
constraints on ring systems comprised of dense material. This
indicates that, although Jupiter may have had intermittent periods of
substantial rings systems, their long-term sustainability may be
severely truncated by the presence of the Galilean moons and
associated resonances. Ring material beyond the Roche limit that
remain in stable orbits may also experience eccentricity excitation
that enhances moonlet coalescence. Furthermore, we have shown that the
outer edge of the present ring system must be relatively young ($<
10^7$~years) in order to have survived dynamical scattering
processes. The balance between planet mass, the formation of massive
moons, and the sustainability of significant ring mass, means that
Saturn may be near the optimal region for the formation and long-term
survival of substantial rings. A useful extension to this work could
thus include longer timescale simulations combined with migration
effects that fully explore the interaction between moons and rings
during periods of formation, as well as moon formation from ring
material and the inclusion of non-gravitational forces near the Roche
limit.

Although the inventory of Solar System giant planets and their
associated rings and moons is limited, they yet provide the best clues
to the formation and evolution of such systems
\citep{horner2020b,kane2021d}, as well as a guide toward detecting
their exoplanet analogs
\citep{dalba2015,mayorga2016,mayorga2020,wakeford2020b}. The detailed
data available for local giant planet systems must necessarily be
balanced by the statistical knowledge that will be gained through the
discovery of exomoons and exorings. Such discoveries will provide the
means to fully explore the above described potential correlation of
moons and ring properties with those of their host planet.

\section*{Acknowledgements}

The authors thank Paul Dalba and the anonymous referees for useful
feedback on the manuscript. This research has made use of the NASA
Exoplanet Archive, which is operated by the California Institute of
Technology, under contract with the National Aeronautics and Space
Administration under the Exoplanet Exploration Program. The results
reported herein benefited from collaborations and/or information
exchange within NASA's Nexus for Exoplanet System Science (NExSS)
research coordination network sponsored by NASA's Science Mission
Directorate.

\software{REBOUND \citep{rein2012a}}


\section{Introduction}
\label{sec:introduction}

Moving Object Detection (MOD) is a crucial problem in computer vision for applications such as video surveillance, pose estimation, and intelligent visual observation of animals, to name just a few \cite{giraldo2019camera,garcia2020background}.
MOD mainly aims to separate the moving objects known as foreground from the static component known as background \cite{bouwmans2014traditional}.
In the literature, MOD has been considered as a binary classification problem where each pixel is predicted for either background or foreground component in a sequence taken from a static or moving camera. Therefore, this problem is also known as background-foreground segregation \cite{bouwmans2017decomposition}.
MOD becomes a very challenging problem because of the presence of dynamic variations in the background scene such as illumination variations, swaying bushes, camouflage, intermittent object motion, shadows, and jittering effects, to name a few \cite{wang2014cdnet}.
In addition, videos that are taken from PanTiltZoom (PTZ) and moving cameras also pose more challenges for efficient MOD.
Several methods have been proposed to improve the performance of MOD under challenging scenarios such as those presented in \cite{oreifej2012simultaneous,pham2014detection,giraldo2020graph}.
However, there is no unique unsupervised or supervised method that can effectively handle all the aforementioned challenges in real scenarios \cite{tezcan2020bsuv,wang2014cdnet}.

\begin{figure*}
    \centering
    \includegraphics[width=0.86\textwidth]{figures/introduction_figure.pdf}
    \caption{Comparisons of the visual results of the proposed GraphMOD-Net algorithm with two state-of-the-art methods for MOD on six very challenging video sequences taken from CDNet2014 \cite{wang2014cdnet} and UCSD \cite{mahadevan2009spatiotemporal} datasets, including moving camera sequences plus dynamic backgrounds. From left to right: zoom in zoom out, two-position PTZ cameras, intermittent pan, surfers, skiing, and surf sequences. From top to bottom: input frame, the ground truth of MOD, SuBSENSE \cite{st2014subsense}, BSUV-Net \cite{tezcan2020bsuv}, and our proposed GraphMOD-Net.}
    \label{fig:introductory_image}
\end{figure*}

Many state-of-the-art MOD methods such as SuBSENSE \cite{st2014subsense} work well for sequences taken from a static camera; however, they show performance degradation in the presence of moving camera sequences as shown in Fig. \ref{fig:introductory_image}. And in fact, none of the methods proposed so far can address all challenges posed by static as well as moving cameras.
In the current work, we contribute to bridging these gaps by formulating the MOD problem on Graph Convolutional Neural Networks (GCNNs).
We propose a novel semi-supervised algorithm dubbed Graph MOD Network (GraphMOD-Net) based on GCNNs \cite{kipf2017semi}.
GraphMOD-Net models the instances in videos as nodes embedded in a graph.
The instances are obtained with a Mask Region-CNN (Mask R-CNN) \cite{he2017mask} or Cascade Mask R-CNN \cite{cai2019cascade}.
The representation of the nodes is obtained with background initialization, optical flow, intensity, and texture features \cite{lucas1981iterative,ojala2002multiresolution}.
Moreover, these nodes are associated either with the class foreground or background using ground-truth information and finally, a GCNN is trained with a small percentage of labeled nodes to perform a semi-supervised learning classification \cite{kipf2017semi} with an unseen scheme \cite{tezcan2020bsuv}.
GraphMOD-Net outperforms state-of-the-art methods in several challenges of the Change Detection 2014 (CDNet2014) \cite{wang2014cdnet} and UCSD background subtraction \cite{mahadevan2009spatiotemporal} datasets. The main contributions of this paper are summarized as follows:
\begin{itemize}[leftmargin=*]
\itemsep0em
\item We pose the problem of MOD as a binary classification problem on the graph where each node is classified into two distinct components including background and foreground using GCNNs
\item We perform rigorous experiments using the CDNet2014 dataset \cite{wang2014cdnet} for MOD. Our results demonstrate that GraphMOD-Net uses a limited amount of labeled data and outperforms some state-of-the-art classical and deep learning methods.
\end{itemize}

The rest of the paper is organized as follows. Section \ref{sec:related_works} presents the related works. Section \ref{sec:algorithm} explains the basic concepts and the proposed GraphMOD-Net. Section \ref{sec:experimental_framework} introduces the experimental framework. And finally, Sections \ref{sec:results_discussion} and \ref{sec:conclusions} present the results and conclusions, respectively.

\section{Related Work}
\label{sec:related_works}

The MOD methods can be categorized into unsupervised, semi-supervised, and supervised learning methods. The unsupervised methods can be classified as statistical \cite{bouwmans2008background,narayana2014background,zuo2019moving}, subspace learning \cite{bouwmans2009subspace}, and robust principal component analysis models \cite{bouwmans2014robust,rodriguez2016incremental}. These methods do not successfully adapt to complex scenarios in MOD. Inspired by the success of deep CNNs on a wide variety of visual recognition tasks \cite{russakovsky2015imagenet}, several studies have also been proposed to handle the MOD problem in a fully supervised manner \cite{garcia2019foreground,bouwmans2019deep}. However, these deep learning methods usually require a large amount of training data. The annotation of such data is an intensive task and can be very expensive to obtain the required degree of quality.
Even though some works have tried to explain the success of deep learning methods for MOD \cite{minematsu2018analytics}, there is no concrete evidence in the literature about the sample complexity required in the deep learning regimen \cite{du2018many}.
Furthermore, being trained and tested on sequences derived from the same videos, most of the deep learning methods for MOD do not show good generalization to unseen videos exhibiting unpredictable changes. The performance degradation was confirmed in CDNet2014 (FgSegNetv2 \cite{lim2019learning}) for which the performance of leading methods dropped dramatically when evaluated on unseen videos as reported in \cite{tezcan2020bsuv}.
For a complete review of supervised and unsupervised methods, we invite the readers to the survey papers \cite{bouwmans2008background,bouwmans2014traditional,bouwmans2014robust,bouwmans2019deep}.

\begin{figure*}
    \centering
    \includegraphics[width=0.89\textwidth]{figures/pipeline.pdf}
    \caption{GraphMOD-Net uses background initialization and instance segmentation. Each instance represents a node in a graph using motion, intensity, and texture features. Finally, a GCNN classifies if each node is a moving or static object with an unseen scheme.}
    \label{fig:pipeline}
\end{figure*}

Giraldo and Bouwmans \cite{giraldo2020graphbgs} have recently proposed a Graph-based semi-supervised learning method for BackGround Subtraction (GraphBGS). Leveraging the theory of sampling and graph signal reconstruction, this framework found applications in MOD \cite{ortega2018graph}.
GraphBGS exploits a variational approach to solve the semi-supervised learning problem \cite{pesenson2009variational}, assuming that the underlying signals corresponding to the background/foreground nodes are smooth in the graph \cite{chen2015discrete}. Therefore, this method shows performance degradation if the smooth prior assumption is not fully satisfied. In this work, we propose a semi-supervised learning algorithm dubbed as GraphMOD-Net. Our algorithm extends the work of Giraldo and Bouwmans \cite{giraldo2020graphbgs} by increasing modeling capacity and avoiding the explicit smoothness assumption with GCNNs. GraphMOD-Net has the advantage of requiring less training data than deep learning methods while adapting to complex MOD scenarios, unlike unsupervised methods.

\section{Proposed Algorithm}
\label{sec:algorithm}

Our proposed algorithm consists of several components including instance segmentation, background model initialization, features extraction, graph construction, and GCNN training, as shown in the block diagram in Fig. \ref{fig:pipeline}.
In the first step, we compute the instance segmentation mask of the input sequence using the Mask R-CNN method \cite{he2017mask}.
The instance segmentation mask assists our algorithm to get the prior knowledge of the object instances in the training videos.
We also compute the initial background model using the temporal median filtering method.
In the second step, the instances and background model are used to derive the features to be used in the graph construction.
We use texture features, motion features using optical flow estimation \cite{lucas1981iterative}, and intensity features. However, other types of features can be used to represent the nodes in the graph \cite{giraldo2020graph}.
In the second step, we construct the graph where a node is assigned to each object instance represented by the group of the aforementioned features. With the so obtained graph representation, we train a GCNN to classify the nodes into either background or foreground. The different components of the algorithm will be described next.

\subsection{Instance Segmentation Description}

We used the Mask R-CNN \cite{he2017mask}, with a Residual Network of 50 layers (ResNet50) \cite{he2017mask} and Feature Pyramid Network (FPN) \cite{lin2017feature} as the backbone, for instance segmentation in the CDNet2014. While for the UCSD dataset we used Cascade Mask R-CNN \cite{cai2019cascade} with ResNeSt of 200 layers \cite{zhang2020resnest}.
Both instance segmentation networks are trained with the Common Objects in Context (COCO) 2017 dataset \cite{lin2014microsoft}. Each output (mask) of the instance segmentation method represents a node in our graph.
We note that super-pixels can be used as an alternative for object instances in GraphMOD-Net.
However, super-pixels and pixel-based approaches, in general, produce a huge-sized graph, making the computation cost-prohibitive and the implementation impractical \cite{parada2019blue,lu2019fast}.

\subsection{Graph Construction}

A graph $G$ can be represented with two sets as $(\mathcal{V,E})$, where $\mathcal{V}=\{1,\dots,N\}$ is the set of nodes, with cardinality $\vert \mathcal{V} \vert = N$ is the number of nodes of $G$. $\mathcal{E}=\{(i,j)\}$ is the set of edges, where $(i,j)$ represents an edge between the nodes $i$ and $j$. The adjacency matrix of $G$ is $\mathbf{A} \in \mathbb{R}^{N \times N}$, where $\mathbf{A}(i,j)$ is the weight of the edge $(i,j)$. A graph is called unweighted when $\mathbf{A}(i,j)=1~\forall~(i,j) \in \mathcal{E}$. For undirected graphs $\mathbf{A}$ is symmetric, \ie, the weights between the edges $(i,j)$ and $(j,i)$ are the same. Finally, $\mathbf{D} \in \mathbb{R}^{N \times N}$ is the degree matrix of $G$ defined as a diagonal matrix such that $\mathbf{D}(i,i)=\sum_{j=1}^N \mathbf{A}(i,j)~\forall~i \in \mathcal{V}$.

In our graph, each node is represented by a $853$-dimensional vector $\mathbf{x}_v$ for $v \in \mathcal{V}$, and grouped in the matrix $\mathbf{X}=[\mathbf{x}_1,\dots,\mathbf{x}_N]^{\mathsf{T}}$. The feature vector $\mathbf{x}_v$ is obtained using a temporal median filter for background initialization, optical flow, intensity, and texture features, after converting the videos to gray-scale. Let $\mathbf{I}_v^t$ be the Region of Interest (RoI) image of node $v \in \mathcal{V}$ in the current ($t$) frame. The RoI is the group of pixels belonging to an object instance produced by the instance segmentation method (the bounding box output of Mask R-CNN). Let $\mathcal{P}_v$ be the set of pixel-indices corresponding to the mask object $v$ (the mask output of Mask R-CNN). Let $\mathbf{B}$ be the output image of the temporal median filter and let $\mathbf{B}_v$ be the RoI image of the background of node $v$, \ie, we are extracting the bounding box of $v$ in $\mathbf{B}$.
We compute the optical flow vectors, of the current frame with support in $\mathcal{P}_v$, and their amplitudes and orientations. Afterward, we construct histograms and their descriptive statistics: minimum, maximum, mean, standard deviation, mean absolute deviation, and range. We also use
the RoI images $\mathbf{I}_v^t$, $\mathbf{I}_v^{t-1}$, $\mathbf{B}_v$ and $\vert \mathbf{I}_v^t - \mathbf{B}_v \vert$ to derive a texture representation, with local binary patterns \cite{ojala2002multiresolution}, and the intensity histograms. All the previous features are concatenated into thee feature vector $\mathbf{x}_v$
representing the node $v$.

The construction of the undirected graph $G$ is performed using the k-nearest neighbors algorithm with $\text{k}=30$ in the matrix $\mathbf{X}$. The elements of the adjacency matrix are computed with a Gaussian kernel $\mathbf{A}(i,j) = \exp{({-\frac{d(i,j)^2}{\rho^2}})}$, where $d(i,j)=\Vert \mathbf{x}_i - \mathbf{x}_j \Vert$, and $\rho$ is the standard deviation of the Gaussian function, set to $\rho = \frac{1}{\vert \mathcal{E} \vert + N}\sum_{(i,j) \in \mathcal{E}} d(i,j)$. Furthermore, the full set of labels $\mathbf{Y}\in \mathbb{R}^{N\times F}$ of our algorithm is a matrix encoding the classes of all nodes in $\mathcal{V}$, where $F=2$ (background and foreground). Each row of $\mathbf{Y}$ is represented with the so-called one-hot vector: background $[1,0]$, and foreground $[0,1]$.

\subsection{Graph Semi-Supervised Learning Algorithm}

The previous semi-supervised algorithm GraphBGS \cite{giraldo2020graphbgs} relies on the sampling and reconstruction of graph signals \cite{chen2015discrete,ortega2018graph,anis2018sampling,parada2019blue}. As a consequence, Giraldo and Bouwmans \cite{giraldo2020graphbgs} defined the sampled nodes (or training set in our case) as a subset of nodes $\mathcal{S} \subset \mathcal{V}$ with $\mathcal{S}=\{s_1,s_2,\dots,s_m\}$, where $m = \vert \mathcal{S}\vert \leq N$ is the number of sampled nodes. GraphBGS applied a variational method in graphs \cite{pesenson2009variational} to solve the semi-supervised learning problem, the application of this variational method assumes that the background/foreground nodes are smooth in the graph. Unlike GraphBGS, our algorithm avoids the smoothness assumption by solving the learning problem with GCNNs, and we refer to the set $\mathcal{S}$ as the training set in GraphMOD-Net.

The layer-wise propagation rule of our GraphMOD-Net, inspired from Kipf and Welling \cite{kipf2017semi} method, is given as follows:
\begin{equation}
    \mathbf{H}^{(l+1)}=\sigma(\tilde{\mathbf{D}}^{-\frac{1}{2}}\tilde{\mathbf{A}}\tilde{\mathbf{D}}^{-\frac{1}{2}}\mathbf{H}^{(l)}\mathbf{W}^{(l)})
    \label{eqn:propagation_rule}
\end{equation}
where $\tilde{\mathbf{A}}=\mathbf{A}+\mathbf{I}$ is the adjacency matrix of $G$ with added self-connections\footnote{The self-connections given by $\tilde{\mathbf{A}}=\mathbf{A}+\mathbf{I}$ were introduced by Kipf and Welling \cite{kipf2017semi} to avoid numerical instabilities and exploding/vanishing gradients.}, $\mathbf{I}$ is the identity matrix, $\tilde{\mathbf{D}}$ is the degree matrix of $\tilde{\mathbf{A}}$, $\mathbf{W}^{(l)}$ is the matrix of trainable weights in layer $l$, $\sigma(\cdot)$ denotes an activation function, and $\mathbf{H}^{(l)}$ is the matrix of activations in layer $l$ such that $\mathbf{H}^{(0)}=\mathbf{X}$ is the input matrix denoting the representation of the nodes. The propagation rule in Eqn. (\ref{eqn:propagation_rule}) is motivated by the first-order approximation of localized spectral filters on graphs \cite{hammond2011wavelets,defferrard2016convolutional}. For further details, the reader is referred to Kipf and Welling paper \cite{kipf2017semi}.

GraphMOD-Net uses a GCNN with one hidden layer to solve the semi-supervised learning problem. The forward of our model is computed using the propagation rule of Eqn. (\ref{eqn:propagation_rule}) as follows:
\begin{align}
    \nonumber
    \mathbf{Z}=f(\mathbf{X,A}) = \softmax \left(\tilde{\mathbf{D}}^{-\frac{1}{2}}\tilde{\mathbf{A}}\tilde{\mathbf{D}}^{-\frac{1}{2}} \right. & \\
    \label{eqn:forward_gcnn}
    \left. \relu\left(\tilde{\mathbf{D}}^{-\frac{1}{2}}\tilde{\mathbf{A}}\tilde{\mathbf{D}}^{-\frac{1}{2}} \mathbf{XW}^{(0)}\right)\mathbf{W}^{(1)}\right) &,
\end{align}
where $\mathbf{Z}$ is the output of the GCNN; $\mathbf{W}^{(0)} \in \mathbb{R}^{C\times H}$ is the weights of the first hidden layer such that $C$ is the dimension of vector $\mathbf{x}_v$, \ie, $C=853$, and $H$ is the number of feature maps; $\mathbf{W}^{(1)} \in \mathbb{R}^{H\times F}$ is the weights of the output layer such that $F$ is the number of classes (\ie, $F=2$: background and foreground); $\relu(\cdot)=\max(0,\cdot)$ is the rectified linear unit; and finally the softmax activation function is defined as $\softmax(x_i)=\frac{1}{S}\exp{(x_i)}$ where $S=\sum_i \exp{(x_i)}$. Here the softmax function is applied row-wise. We set the number of feature maps $H$ in the hidden layer to $16$ in our model. The loss function of this GCNN is defined as the cross-entropy error over all labeled nodes:
\begin{equation}
    \mathcal{L}=-\sum_{i \in \mathcal{S}} \sum_{j=1}^F \mathbf{Y}(i,j) \ln{\mathbf{Z}(i,j)}.
    \label{eqn:cross_entropy}
\end{equation}
We use the Adam optimizer \cite{kingma2015adam} for training and set the training batch to the full training set in each iteration (this is possible since the whole database fits in memory).

\section{Experimentation}
\label{sec:experimental_framework}

This section introduces the databases used in this paper, the experimentation protocol, the evaluation metrics, the experiments, and the implementation details of GraphMOD-Net.

\subsection{Databases}

We use the Change Detection 2014 (CDNet2014) \cite{wang2014cdnet} and the UCSD background subtraction \cite{mahadevan2009spatiotemporal} datasets in this work. CDNet2014 contains $11$ challenges including, bad weather, low frame rate, night videos, PTZ, turbulence, baseline, dynamic background, camera jitter, intermittent object motion, shadow, and thermal. Each challenge has between $4$ to $6$ videos, and each video has from $600$ up to $7999$ images. Certain frames in each video contain ground-truth images, consisting of pixel-level annotations of foreground and background. On the other hand, UCSD contains $18$ videos mainly composed of moving camera sequences, with $30$ up to $246$ frames each sequence. Each video of UCSD is partially or fully annotated with pixel-level ground-truth images of foreground and background.

\subsection{Experimentation protocol}
\label{sec:training_validation_test}

The selection of the training, validation, and test nodes is based on an agnostic-video evaluation methodology in order to best evaluate the performance of GraphMOD-Net on unseen videos.
To that end, the testing nodes should come from different videos with respect to the training nodes. Formally, let $\mathcal{V}_k$ be a partition of the whole set of nodes $\mathcal{V}_k \subset \mathcal{V}$, and let $\mathcal{U}_k \subset \mathcal{V}_k$ be the set of nodes corresponding to the unseen videos for that specific partition $k$. Then, the training set $\mathcal{S} \subset \mathcal{V}_k$ is randomly chosen from a subset of possible nodes $\mathcal{V}_k \setminus \mathcal{U}_k$. In other words, the training set is $\mathcal{S}$, the test set is $\mathcal{U}_k$, and the validation set $\mathcal{T}_k$ is randomly chosen from the subset $\mathcal{T}_k \subset \mathcal{V}_k \setminus (\mathcal{S} \bigcup \mathcal{U}_k)$.

For CDNet2014 we adopt the same partition of the database as Tezcan \etal \cite{tezcan2020bsuv}. The database is divided in 18 sets, \ie, $k=\{1,\dots,18\}$, where each split contains a list of training and test sequences. GraphMOD-Net randomly selects a small subset of nodes $\mathcal{S} \subset \mathcal{V}_k$ and a small set $\mathcal{T}_k$ for validation from the training sequences to train and then evaluate the GCNN on unseen videos. For example, the first partition is given such that $\mathcal{V}_1$ contains the nodes corresponding to the challenges: baseline, bad weather, intermittent object motion, low framerate, and shadow; while the set of unseen videos $\mathcal{U}_1$ contains the sequences: office, and PETS2006 from baseline; and backdoor, copy machine, and people in shade from shadow. The cardinality of $\mathcal{T}_k$ is fixed for all experiments such that $\vert \mathcal{T}_k \vert = N\times 0.01$, \ie, we are using $1\%$ of the dataset as validation. The 18 partitions for the CDNet2014 is given in the supplementary material of Tezcan \etal paper \cite{tezcan2020bsuv}.

For UCSD we train our GraphMOD-Net in a subset of nodes of CDNet2014 and we test in UCSD dataset. For the training and validation sets we randomly choose nodes from the challenges baseline, dynamic background, shadow, and PTZ of CDNet2014.

\subsection{Evaluation Metrics}

GraphMOD-Net is evaluated using the F-measure defined as follows:
\begin{gather}
    \label{eqn:f-measure}
    \text{F-measure} = 2\frac{\text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}},\\
    \text{Recall} = \frac{\text{TP}}{\text{TP}+\text{FN}}, \text{ Precision} = \frac{\text{TP}}{\text{TP}+\text{FP}},
\end{gather}
with TP, FP, and FN the number of True Positives, False Positives, and False Negatives pixels, respectively.

\subsection{Experiments}

For CDNet2014, GraphMOD-Net constructs a graph for the whole dataset, resulting in an undirected graph of $258956$ nodes. For each partition of the dataset a percentage of the total amount of nodes in the set $\mathcal{M}=\{ 0.001, 0.005, 0.05, 0.1\}$ is used for training and then evaluated with unseen videos. For example, $\mathcal{S}$ contains $259$ nodes for $0.001 \in \mathcal{M}$ (keep in mind that usually each image contains several nodes as shown in Fig. \ref{fig:pipeline}). The training of our GCNN is validated with a Monte Carlo cross-validation with $3$ repetitions for each training density and split of the database. As a consequence, we train $18 \times 3 \times \vert \mathcal{M} \vert = 216$ GCNNs for the validation of GraphMOD-Net.

For UCSD, GraphMOD-Net constructs a graph with the whole UCSD dataset plus the challenges baseline, dynamic background, shadow, and PTZ of CDNet2014; resulting in a graph of $104261$ nodes. In this case, the test set is the whole UCSD dataset, and a small number of nodes in the set $\mathcal{M}$ is used for training. A Monte Carlo cross-validation with 5 repetitions is performed for each training density, \ie, we train $5 \times \vert \mathcal{M} \vert = 20$ GCNNs for UCSD dataset. For the sake of clarification, the metrics in the experiments are computed on the unseen set of nodes in each experiment. The CDNet2014 and UCSD experiments follow the guidelines for the training, validation, and testing sets of Section \ref{sec:training_validation_test}.

\begin{table*}
\centering
\caption{Comparison of the average F-measure over nine challenges of the CDNet2014 dataset, namely: bad weather (BWT), baseline (BSL), camera jitter (CJI), dynamic background (DBA), intermittent object motion (IOM), low frame rate (LFR), PTZ, shadow (SHW), and thermal (THL). The best and second best performing methods for each challenge are shown in {\color{red}\textbf{red}} and {\color{blue}\textbf{blue}}, respectively.}
\label{tbl:CDNet_results}
\begin{scriptsize}
\begin{tabular}{lcccccccccc}
\hline
\multicolumn{1}{l|}{Method} & BWT & BSL & CJI & DBA & IOM & LFR & PTZ & SHW & \multicolumn{1}{c|}{THL} & Overall \\
\hline
\multicolumn{1}{l|}{GraphCutDiff (unsupervised)} & {\color{red}\textbf{0.879}} & 0.715 & 0.549 & 0.539 & 0.402 & 0.513 & 0.372 & 0.723 & \multicolumn{1}{c|}{0.579} & 0.5857 \\
\multicolumn{1}{l|}{SuBSENSE (unsupervised)} & 0.862 & 0.950 & {\color{blue}\textbf{0.815}} & 0.818 & 0.657 & {\color{blue}\textbf{0.645}} & 0.348 & 0.899 & \multicolumn{1}{c|}{{\color{blue}\textbf{0.817}}} & 0.7567 \\
\multicolumn{1}{l|}{WisenetMD (unsupervised)} & 0.862 & 0.949 & {\color{red}\textbf{0.823}} & {\color{blue}\textbf{0.838}} & {\color{blue}\textbf{0.726}} & 0.640 & 0.337 & 0.898 & \multicolumn{1}{c|}{0.815} & {\color{blue}\textbf{0.7653}} \\
\hline
\multicolumn{1}{l|}{GraphBGS (semi-supervised)} & 0.837 & 0.942 & 0.700 & 0.743 & 0.405 & 0.558 & {\color{blue}\textbf{0.749}} & {\color{red}\textbf{0.966}} & \multicolumn{1}{c|}{0.729} & 0.7366 \\
\multicolumn{1}{l|}{GraphMOD-Net (semi-supervised, {\color{red}ours})} & 0.839 & {\color{blue}\textbf{0.955}} & 0.720 & {\color{red}\textbf{0.851}} & 0.554 & 0.521 & {\color{red}\textbf{0.770}} & {\color{blue}\textbf{0.942}} & \multicolumn{1}{c|}{0.682} & 0.7593 \\
\hline
\multicolumn{1}{l|}{BSUV-net (supervised)} & {\color{blue}\textbf{0.871}} & {\color{red}\textbf{0.969}} & 0.774 & 0.797 & {\color{red}\textbf{0.750}} & {\color{red}\textbf{0.680}} & 0.628 & 0.923 & \multicolumn{1}{c|}{{\color{red}\textbf{0.858}}} & {\color{red}\textbf{0.8056}} \\
\multicolumn{1}{l|}{FgSegNet v2 (supervised)} & 0.328 & 0.693 & 0.427 & 0.363 & 0.200 & 0.248 & 0.350 & 0.530 & \multicolumn{1}{c|}{0.604} & 0.4159 \\
\hline
\end{tabular}
\end{scriptsize}
\end{table*}

\begin{table*}
\centering
\caption{Comparison of the F-measure results over the videos of UCSD background subtraction dataset. The best and second best performing methods for each video are shown in {\color{red}\textbf{red}} and {\color{blue}\textbf{blue}}, respectively. DECOLOR and GRASTA are subspace learning methods.}
\label{tbl:UCSD_results}
\begin{scriptsize}
\begin{tabular}{l|cccccccc}
\hline
Sequence & \tiny{MoG} & \tiny{DECOLOR} & \tiny{ViBe} & \tiny{GRASTA} & \tiny{SuBSENSE} & \tiny{BSUV-Net} & \tiny{GraphBGS} & \tiny{GraphMOD} \\
\hline
Birds & 0.1427 & 0.1457 & 0.3354 & 0.1320 & 0.4832 & 0.2625 & \color{red}\textbf{0.7495} & \color{blue}\textbf{0.7391} \\
Boats & 0.0881 & 0.2179 & 0.1854 & 0.0678 & 0.4550 & 0.6621 & \color{blue}\textbf{0.7765} & \color{red}\textbf{0.8007} \\
Bottle & 0.1856 & 0.4765 & 0.4512 & 0.1159 & \color{blue}\textbf{0.6570} & 0.5039 & \color{red}\textbf{0.8741} & \color{red}\textbf{0.8741} \\
Chopper & 0.3237 & 0.6214 & 0.4930 & 0.0842 & 0.6723 & 0.3020 & \color{blue}\textbf{0.7766} & \color{red}\textbf{0.7844} \\
Cyclists & 0.0915 & 0.2224 & 0.1211 & 0.1243 & 0.1445 & 0.4138 & \color{blue}\textbf{0.7417} & \color{red}\textbf{0.7479} \\
Flock & 0.2706 & 0.2943 & 0.2306 & 0.1612 & 0.2492 & 0.0025 & \color{red}\textbf{0.5903} & \color{blue}\textbf{0.5871} \\
Freeway & 0.2622 & \color{blue}\textbf{0.5229} & 0.4002 & 0.0814 & \color{red}\textbf{0.5518} & 0.1185 & 0.3719 & 0.3516 \\
Hockey & 0.3867 & 0.3449 & 0.4195 & 0.3149 & 0.3611 & 0.6908 & \color{blue}\textbf{0.7664} & \color{red}\textbf{0.7804} \\
Jump & 0.2679 & 0.3135 & 0.2636 & 0.4175 & 0.2295 & \color{red}\textbf{0.8697} & \color{blue}\textbf{0.7734} & 0.7602 \\
Landing & 0.0335 & 0.0640 & 0.0433 & 0.0414 & 0.0026 & 0.0012 & \color{red}\textbf{0.2452} & \color{blue}\textbf{0.1840} \\
Ocean & 0.1113 & 0.1315 & 0.1648 & 0.1144 & 0.2533 & \color{blue}\textbf{0.5335} & \color{red}\textbf{0.8593} & \color{red}\textbf{0.8593} \\
Peds & 0.3731 & 0.7942 & 0.5257 & 0.4653 & 0.5154 & 0.6738 & \color{red}\textbf{0.8518} & \color{blue}\textbf{0.8512} \\
Skiing & 0.2038 & 0.3473 & 0.1441 & 0.0927 & 0.2482 & 0.0602 & \color{red}\textbf{0.5963} & \color{blue}\textbf{0.5953} \\
Surf & 0.0489 & 0.0647 & 0.0462 & 0.0523 & 0.0467 & 0 & \color{blue}\textbf{0.5851} & \color{red}\textbf{0.6139} \\
Surfers & 0.0542 & 0.1959 & 0.1189 & 0.0742 & 0.1393 & 0.4776 & \color{red}\textbf{0.6719} & \color{blue}\textbf{0.6611} \\
Trafic & 0.2188 & \color{blue}\textbf{0.2732} & 0.1445 & 0.0368 & 0.1165 & 0 & \color{red}\textbf{0.5722} & \color{red}\textbf{0.5722} \\
\hline
Overall & 0.1914 & 0.3144 & 0.2555 & 0.1485 & 0.3203 & 0.3483 & \color{red}\textbf{0.6751} & \color{blue}\textbf{0.6727} \\
\hline
\end{tabular}
\end{scriptsize}
\end{table*}

\subsection{Implementation Details}

The Cascade and Mask R-CNN were implemented using Pytorch and Detectron2 \cite{wu2019detectron2,zhang2020resnest}. The semi-supervised learning GCNN was implemented using TensorFlow \cite{abadi2016tensorflow}. The GCNN has a dropout rate of $0.5$ \cite{srivastava2014dropout}, an initial learning rate of $0.01$, a weight decay of $5\times 10^{-4}$, and a maximum number of $600$ epochs. We also implemented an early stopping with a window size of 10, \ie, the training is stopped when the validation loss does not decrease for 10 consecutive epochs.

For the comparison with GraphMOD-Net, the background subtraction algorithms MoG \cite{stauffer1999adaptive}, SuBSENSE \cite{st2014subsense}, and ViBe \cite{barnich2010vibe} for USCD were implemented with the BGSLibrary \cite{sobral2014bgs}; while GRASTA \cite{he2012incremental} and DECOLOR \cite{zhou2012moving} were implemented using the LRSLibrary \cite{lrslibrary2015}.

\section{Results and Discussion}
\label{sec:results_discussion}

GraphMOD-Net is compared, either in CDNet2014 or UCSD, with the following state-of-the-art algorithms for MOD: MoG \cite{stauffer1999adaptive}, GRASTA \cite{he2012incremental}, DECOLOR \cite{zhou2012moving}, Graph Cut Difference (GraphCutDiff) \cite{miron2015change}, ViBe \cite{barnich2010vibe}, SuBSENSE \cite{st2014subsense}, WisenetMD \cite{lee2019wisenetmd}, GraphBGS \cite{giraldo2020graphbgs}, BSUV-Net \cite{tezcan2020bsuv}, and FgSegNet v2 \cite{lim2019learning}. The results of FgSegNet v2 \cite{lim2019learning} for CDNet2014 are reported using unseen videos for the evaluation of the network, the performance comes from Tezcan \etal paper \cite{tezcan2020bsuv}.

Tables \ref{tbl:CDNet_results} and \ref{tbl:UCSD_results} show the comparison of GraphMOD-Net with several state-of-the-art methods in CDNet2014 and UCSD, respectively. The results of GraphMOD-Net and GraphBGS are computed with the best F-measures in each sequence from the experiments (both algorithms have the same instance segmentation method). For CDNet2014 in Table \ref{tbl:CDNet_results}, GraphMOD-Net shows the best results in the challenges baseline, dynamic background, PTZ, and shadow. The results also show that our algorithm has an overall increment in performance with respect to GraphBGS. For UCSD in Table \ref{tbl:UCSD_results}, GraphMOD-Net show either the best or the second-best results across almost all the videos of UCSD. In the overall performance, GraphMOD-Net comes second lagging slightly behind GraphBGS. We believe it is a consequence of increasing the representational power when solving the semi-supervised learning problem with GCNNs leading to a slightly worse generalization. Similarly, GraphMOD-Net outperforms subspace learning methods such as DECOLOR and GRASTA. Subspace learning methods generally have problems when dealing with moving camera sequences.

We can also notice the degradation of the deep learning method FgSegNet v2 compared to the original results \cite{lim2019learning} when tested on unseen videos in Table \ref{tbl:CDNet_results}.
Similarly, the performance of BSUV-Net drops when applied on UCSD, these results could suggest a weakness of deep learning methods in generalization for MOD (keep in mind that BSUV-Net uses the output of a fully CNN trained in ADE20K dataset \cite{zhou2017scene} as input of the network). In Table \ref{tbl:UCSD_results}, we evaluated the BSUV-Net model provided by the authors of \cite{tezcan2020bsuv} that was trained in the CDNet2014. In our case, we trained GraphMOD-Net in a small subset of nodes of CDNet2014 including the challenges baseline, dynamic background, shadow, and PTZ. This suggests that GraphMOD-Net generalizes better than BSUV-Net when applied on a different dataset from which it was trained. For the sake of clarification, we did not use the ground truth of UCSD in the training procedure.

\begin{figure*}
    \centering
    \includegraphics[width=0.815\textwidth]{figures/results.pdf}
    \caption{Average F-measure vs training percentage in the nine challenges of CDNet2014, using GraphBGS, and our GraphMOD-Net. Each point in the plots is a Monte Carlo cross-validation experiment with 3 repetitions.}
    \label{fig:experimental_results}
\end{figure*}

Figure \ref{fig:experimental_results} depicts the results of our GraphMOD-Net compared with the previous semi-supervised MOD method GraphBGS \cite{giraldo2020graphbgs}. The results of the challenges turbulence and night videos are not displayed since our Mask R-CNN fails to detect several instances in these videos. As a result, it is not possible to detect the moving objects in some sequences for the GCNN. GraphMOD-Net is better than GraphBGS in the challenges of bad weather, baseline, camera jitter, dynamic background, intermittent object motion, low frame rate, and PTZ; while GraphBGS shows better results than GraphMOD-Net in the challenges shadow and thermal. GraphMOD-Net benefits from the higher modeling capacity of GCNNs by improving upon the GraphBGS as shown in Tables \ref{tbl:CDNet_results}, \ref{tbl:UCSD_results}, and in Figure \ref{fig:experimental_results}.

\begin{table*}
\centering
\caption{Some visual results of GraphMOD-Net in CDNet2014 compared with state-of-the-art algorithms, from left to right: original images, ground-truth images, BSUV-Net \cite{tezcan2020bsuv}, GraphBGS \cite{giraldo2020graphbgs}, and the proposed GraphMOD-Net.}
\label{tbl:visual_results}
\makebox[\linewidth]{
\scalebox{0.88}{
\begin{tabular}{|lccccc|}
\hline
\textbf{Categories} & Original & Ground Truth  & BSUV-Net & GraphBGS & GraphMOD-Net \\% Column names row
\hline
\hline
\begin{tabular}[l]{@{}l@{}}Bad Weather\\ wetSnow\\ in000500\end{tabular} & \parbox[c][16mm][c]{18mm}{\includegraphics[height=13.5mm]{figures/visual_results/badWeather_in000500.jpg}} & \parbox[c][16mm][c]{18mm}{\includegraphics[height=13.5mm]{figures/visual_results/badWeather_gt000500.png}} & \parbox[c][16mm][c]{18mm}{\includegraphics[height=13.5mm]{figures/visual_results/bin000500_BSUV_Net.png}} & \parbox[c][16mm][c]{18mm}{\includegraphics[height=13.5mm]{figures/visual_results/bin000500_GraphBGS.png}} &  \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=13.5mm]{figures/visual_results/bin000500_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Baseline\\ Pedestrians\\ in000496\end{tabular} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/baseline_in000496.jpg}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/baseline_gt000496.png}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/bin000496_BSUV_Net.png}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/bin000496_GraphBGS.png}} &  \parbox[c][15mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results/bin000496_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Camera Jitter\\ Boulevard\\ in000915\end{tabular} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12.2mm]{figures/visual_results/cameraJitter_in000915.jpg}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12.2mm]{figures/visual_results/cameraJitter_gt000915.png}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12.2mm]{figures/visual_results/bin000915_BSUV_Net.png}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12.2mm]{figures/visual_results/bin000915_GraphBGS.png}} &  \parbox[c][15mm][c]{18.5mm}{\includegraphics[height=12.2mm]{figures/visual_results/bin000915_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Dynamic Back.\\ Fall\\ in002789\end{tabular} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/dynamic_in002789.jpg}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/dynamic_gt002789.png}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/bin002789_BSUV_Net.png}} & \parbox[c][15mm][c]{18mm}{\includegraphics[height=12mm]{figures/visual_results/bin002789_GraphBGS.png}} &  \parbox[c][15mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results/bin002789_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}PTZ\\ Two Position.\\ in002789\end{tabular} & \parbox[c][14mm][c]{18mm}{\includegraphics[height=10.8mm]{figures/visual_results/PTZ_in001218.jpg}} & \parbox[c][14mm][c]{18mm}{\includegraphics[height=10.8mm]{figures/visual_results/PTZ_gt001218.png}} & \parbox[c][14mm][c]{18mm}{\includegraphics[height=10.8mm]{figures/visual_results/bin001218_BSUV_Net.png}} & \parbox[c][14mm][c]{18mm}{\includegraphics[height=10.8mm]{figures/visual_results/bin001218_GraphBGS.png}} &  \parbox[c][14mm][c]{18.5mm}{\includegraphics[height=10.8mm]{figures/visual_results/bin001218_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Low-F Rate\\ TramCross\\ in000607\end{tabular} & \parbox[c][13mm][c]{18mm}{\includegraphics[height=9.8mm]{figures/visual_results/lowFramerate_in000607.jpg}} & \parbox[c][13mm][c]{18mm}{\includegraphics[height=9.8mm]{figures/visual_results/lowFramerate_gt000607.png}} & \parbox[c][13mm][c]{18mm}{\includegraphics[height=9.8mm]{figures/visual_results/bin000607_BSUV_Net.png}} & \parbox[c][13mm][c]{18mm}{\includegraphics[height=9.8mm]{figures/visual_results/bin000607_GraphBGS.png}} &  \parbox[c][13mm][c]{18.5mm}{\includegraphics[height=9.8mm]{figures/visual_results/bin000607_GraphMOD.png}} \\
\hline
\end{tabular}
}
}
\end{table*}

\begin{table*}
\centering
\caption{Some visual results of GraphMOD-Net in UCSD compared with state-of-the-art algorithms, from left to right: original images, ground-truth images, DECOLOR \cite{zhou2012moving}, SuBSENSE \cite{st2014subsense}, and the proposed GraphMOD-Net.}
\label{tbl:visual_results_UCSD}
\makebox[\linewidth]{
\scalebox{0.88}{
\begin{tabular}{|lccccc|}
\hline
\textbf{Videos} & Original & Ground Truth & DECOLOR & SuBSENSE & GraphMOD-Net \\% Column names row
\hline
\hline
\begin{tabular}[l]{@{}l@{}}Birds\\ frame\_4\end{tabular} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/frame_4.jpg}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/birds_frame_4_GT.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/birds_frame_4_DECOLOR.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/birds_frame_4_SuBSENSE.png}} &  \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/birds_frame_4_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Boats\\ frame\_12\end{tabular} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.1mm]{figures/visual_results_UCSD/frame_12.jpg}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.1mm]{figures/visual_results_UCSD/boats_frame_12_GT.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.1mm]{figures/visual_results_UCSD/boats_frame_12_DECOLOR.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.1mm]{figures/visual_results_UCSD/boats_frame_12_SuBSENSE.png}} &  \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.1mm]{figures/visual_results_UCSD/boats_frame_12_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Bottle\\ frame\_14\end{tabular} & \parbox[c][16.5mm][c]{18.5mm}{\includegraphics[height=13.6mm]{figures/visual_results_UCSD/frame_14.jpg}} & \parbox[c][16.5mm][c]{18.5mm}{\includegraphics[height=13.6mm]{figures/visual_results_UCSD/bottle_frame_14_GT.png}} & \parbox[c][16.5mm][c]{18.5mm}{\includegraphics[height=13.6mm]{figures/visual_results_UCSD/bottle_frame_14_DECOLOR.png}} & \parbox[c][16.5mm][c]{18.5mm}{\includegraphics[height=13.6mm]{figures/visual_results_UCSD/bottle_frame_14_SuBSENSE.png}} &  \parbox[c][16.5mm][c]{18.5mm}{\includegraphics[height=13.6mm]{figures/visual_results_UCSD/bottle_frame_14_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Chopper\\ frame\_47\end{tabular} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/frame_47.jpg}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/chopper_frame_47_GT.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/chopper_frame_47_DECOLOR.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/chopper_frame_47_SuBSENSE.png}} &  \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12mm]{figures/visual_results_UCSD/chopper_frame_47_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Cyclists\\ frame\_9\end{tabular} & \parbox[c][14mm][c]{18.5mm}{\includegraphics[height=10.3mm]{figures/visual_results_UCSD/frame_9.jpg}} & \parbox[c][14mm][c]{18.5mm}{\includegraphics[height=10.3mm]{figures/visual_results_UCSD/cyclists_frame_9_GT.png}} & \parbox[c][14mm][c]{18.5mm}{\includegraphics[height=10.3mm]{figures/visual_results_UCSD/cyclists_frame_9_DECOLOR.png}} & \parbox[c][14mm][c]{18.5mm}{\includegraphics[height=10.3mm]{figures/visual_results_UCSD/cyclists_frame_9_SuBSENSE.png}} &  \parbox[c][14mm][c]{18.5mm}{\includegraphics[height=10.3mm]{figures/visual_results_UCSD/cyclists_frame_9_GraphMOD.png}} \\
\hline
\begin{tabular}[l]{@{}l@{}}Peds\\ frame\_25\end{tabular} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.2mm]{figures/visual_results_UCSD/frame_25.jpg}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.2mm]{figures/visual_results_UCSD/peds_frame_25_GT.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.2mm]{figures/visual_results_UCSD/peds_frame_25_DECOLOR.png}} & \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.2mm]{figures/visual_results_UCSD/peds_frame_25_SuBSENSE.png}} &  \parbox[c][16mm][c]{18.5mm}{\includegraphics[height=12.2mm]{figures/visual_results_UCSD/peds_frame_25_GraphMOD.png}} \\
\hline
\end{tabular}
}
}
\end{table*}

Table \ref{tbl:visual_results} shows some qualitative results of GraphMOD-Net compared with BSUV-Net \cite{tezcan2020bsuv}, and GraphBGS \cite{giraldo2020graphbgs} for CDNet2014. In the same way, Table \ref{tbl:visual_results_UCSD} shows visual results of GraphMOD-Net on UCSD compared with DECOLOR and SuBSENSE. For a fair comparison, the results in Table \ref{tbl:visual_results} for GraphBGS and GraphMOD-Net are computed with the same training percentage on each sequence. The visual results of GraphMOD-Net indicate that GCNNs are better than previous unsupervised and supervised algorithms used for MOD in some challenges. The improvement of GraphMOD-Net with respect to GraphBGS is because of the modeling capacity of GCNNs compared with previous semi-supervised learning methods, where a smooth prior assumption is required.

\section{Conclusions}
\label{sec:conclusions}

This paper introduces a method for MOD based on GCNN. The pipeline of the method involves a Cascade Mask R-CNN or a Mask R-CNN to get the object instances from the videos; a graph construction with k-nearest neighbors, where nodes are associated to object instances, and represented with feature vector encompassing optical flow, intensity and texture descriptors. We use a compact GCNN model trained in a semi-supervised mode.
Our algorithm outperforms previous state-of-the-art unsupervised, semi-supervised, and supervised learning algorithms for MOD in several challenges of the CDNet2014 and UCSD datasets.
In the future, we will further study the implications of the architecture of GCNNs in the problem of MOD. In the same way, we will analyze some theoretical aspects in the training of the GCNNs with focus in MOD. And finally, we will study a inductive framework \cite{hamilton2017inductive} for GraphMOD-Net in the context of MOD.

{\small
\bibliographystyle{ieee_fullname}