\section{Introduction}
Nuclear spins are promising candidates for quantum information applications due to their long coherence times \cite{muhonen2014,pla2013} that can even be observed up to room temperature \cite{saeedi2013}.
However, the small gyromagnetic ratio of nuclear spins that renders them well isolated from the environment and thus underlies their robust coherence also leads to long gate operation times compared to those reported for electron spins. Therefore, nuclear spins will most likely find their application as quantum memories \cite{maurer2012,steger2012}, either for pure storage or as buffers for quantum computation purposes. In many cases, it is beneficial to perform readout directly on the nuclear spin instead of coherently transferring the information to another system, e.g. an electron spin \cite{freer2017}, before this system is read out. Particularly with regard to scalable quantum computing devices, readout relying on electrical means is favourable over methods depending on ac-magnetic fields.

To a large extent driven by the microelectronics industry, the manufacturing of nanoscale semiconductor devices has matured during the last decades, and silicon-based devices have a particularly high potential for scaling. Moreover, isotopically purified $^{28}\mathrm{Si}$ material containing predominantly nuclear spin $0$ atoms can be produced and thus provides an excellent host material for spin qubits based on the electron spin or single impurity nuclear spins \cite{loss1998,kane1998,tyryshkin2012,zwanenburg2013,awschalom2013,veldhorst2014,morton2008,freer2017}.

Cavity quantum electrodynamics (cQED) has been successfully used for charge-photon  \cite{mi2017,stockklauser2017,bruhat2018} and spin-photon coupling \cite{kubo2010,laflamme2012,tosi2014,mi2018,benito2017,landig2018,samkharadze2018,cubaynes2019}, as well as the detection of photons \cite{schuster2007}.
Moreover, cQED and gate reflectometry lend themselves for qubit readout \cite{blais2004,didier2015,didier2015a,govia2017,mi2017,danjou2019,zheng2019,west2019,crippa2019}. Nevertheless, it is an open question whether or not single nuclear spins could be detected via a cavity or coupled to cavity photons.
Among the before mentioned achievements in cQED it is particularly noteworthy that the strong coupling regime is accessible for the spin of a single electron in a $\mathrm{Si}$ double quantum dot (DQD) subject to a magnetic field gradient where spin photon coupling emerges due to
\begin{figure}[H]
	\includegraphics[width=\columnwidth]{Figure1}
	\caption{
	QD-donor system coupled to a single-mode microwave cavity.
	The cavity transmission $A_c=\langle a_{\mathrm{out}}\rangle/\langle a_{\mathrm{in}}\rangle$ reveals the nuclear spin state. The QD-donor energy levels are detuned by an amount $\epsilon$ and hybridized by tunnel coupling $t_c$. The spin of the confined electron is subject to a homogeneous magnetic field $B_z$ and a gradient field $b_x$ perpendicular to $B_z$, while the electron charge is coupled to a single mode ($\omega_c$) of a microwave cavity with electric-dipole coupling strength $g_c$. The electron interacts with the nuclear spin of the implanted donor via the hyperfine interaction $A$.}
	\label{fig:model}
\end{figure}
\noindent an effective spin orbit coupling caused by the the combination of the electric dipole interaction and spin-charge hybridization \cite{mi2018,benito2017}. The same mechanism can also be used to realize a flopping-mode spin qubit with full electrical spin control via electric dipole spin resonance  (EDSR) \cite{benito2019,croot2020}. In this system, a longitudinal magnetic field gradient leads to a shift of the phase and amplitude response of the cavity transmission depending on the strength of the field gradient \cite{croot2020}.
Motivated by this observation we consider a lateral architecture consisting of a quantum dot (QD) in a planar Si/SiGe structure and a single $^{31}\mathrm{P}$ donor implanted in the Si host material. While this system has been successfully operated in the multi electron regime \cite{harvey-collard2017}, we consider the single electron regime to form a flopping-mode electron spin qubit (Fig.~\ref{fig:model}).

As a consequence, if the electron is confined to the donor, it
couples to the donor nuclear spin via the hyperfine interaction. In this configuration, we expect the donor to generate a nuclear spin state dependent Overhauser field. This field constitutes a longitudinal magnetic field gradient that leads to a nuclear spin state and detuning dependent shift of the electron spin transition frequency. Therefore, the cavity response essentially probing the EDSR frequency is expected to shift accordingly.
Our detailed discussion of the expected characteristics in the cavity transmission indicates that the observable signature of the strong electron spin photon coupling \cite{benito2017,mi2018} is indeed significantly altered by the state of the nuclear spin and could therefore be used for nuclear spin state readout. This prediction is verified by calculating the cavity transmission using input-output theory.

Moreover, we investigate the effective excitation conserving nuclear spin photon coupling and find that our suggested method for nuclear spin readout does not require strong nuclear spin photon coupling.

This article is organized as follows:  The following Sec.~\ref{sec:model} contains a discussion of the model of the QD-donor system coupled to a cavity mode. In Sec.~\ref{sec:nuclearspinreadout} we predict the impact of the donor nuclear spin state on the cavity transmission, and verify our expectation by calculating the cavity transmission using input-output theory. Section~\ref{sec:nuclear_spin-photon_coupling} contains the derivation of an effective Hamiltonian describing the nuclear spin dynamics followed by a discussion of the emerging effective nuclear spin photon coupling. Finally, we summarize our results in Sec.~\ref{sec:conclusion}.

\section{Theoretical Model \label{sec:model}}
We consider a lateral QD-donor system fabricated in isotopically enriched $^{28}$Si. QD and donor are aligned along the $z$-axis, such that a single electron can either be localized in the QD on the left or the  donor on the right by adjusting the  QD-donor energy level detuning $\epsilon$. QD-donor tunnel coupling $t_c$ results in charge hybridization near $\epsilon = 0$. The proposed experimental setup, including the various interactions, is sketched in Fig.~\ref{fig:model}.
The detuning $\epsilon$ determined by the energy difference between QD and donor can be controlled by applying an electric field in $z$-direction and by tuning the gates defining the QD confinement potential.
In the presence of a homogeneous magnetic field $B_z$ and a magnetic field gradient $b_x$ perpendicular to $B_z$, the
QD-donor system in the single electron configuration can be modelled by the Hamiltonian
\begin{eqnarray}
\widetilde{H}_0=\tfrac{1}{2}\left(\epsilon \widetilde{\tau}_z +2 t_c \widetilde{\tau}_x+B_z \sigma_z + b_x \sigma_x \widetilde{\tau}_z\right),
\label{eq:H0tilde}
\end{eqnarray}
with $\widetilde{\tau}_i$ and $\sigma_i$ the Pauli operators in position and electron spin space, respectively. The interaction between the nuclear spin and the magnetic field is neglected because it is roughly three orders of magnitude smaller than all other relevant energy scales \cite{tosi2017}. The magnetic fields $B_z$ and $b_x$ are given in units of energy and energy units are chosen such that $\hbar=1$. The donor ground and first excited state are energetically separated by $\gtrsim 2.5\,\mathrm{meV}$ taking into account strain effects due to the Si/SiGe interface \cite{usman2015}. On the other hand, a low-lying excited state, the excited valley state, is present in Si QDs. However, valley splittings $\gtrsim50\,\upmu\mathrm{eV}$ observed in recent devices \cite{mi2017b,borjans2021} together with the possibility to operate the QD-donor system at temperatures $\lesssim 50\,\mathrm{mK}$ allow for a negligible population of the excited valley state. Hence, the valley degree of freedom can be neglected in our model.

Electric dipole interactions allow to couple the electron in the
QD-donor system to microwave resonator photons, described by the coupling Hamiltonian
\begin{eqnarray}
\widetilde{H}_I=g_c\widetilde{\tau}_z(a+a^{\dagger}),
\end{eqnarray}
where $a$ and $a^{\dagger}$ are the bosonic cavity photon annihilation and creation operators of the relevant cavity mode, respectively.
The charge-photon coupling strength for a DQD has been found to be on the order of $g_c/2\pi \approx 30$ to $40\,\mathrm{MHz}$ \cite{mi2018,petersson2012a}. In the QD-donor scenario we expect it to be $\approx 1/3$ of the DQD value, as discussed in Appendix~\ref{appendix:QD-donor system}.
The Hamiltonian for the cavity mode with frequency $\omega_c$ is given by $\widetilde{H}_{\mathrm{cav}}=\omega_c a^{\dagger}a$.
If the electron is confined to the donor, electron spin and $^{31}\mathrm{P}$ donor nuclear spin couple via the hyperfine interaction. The hyperfine interaction strength $A=117 \,\mathrm{MHz}$ \cite{steger2011,feher1959} present in bulk Si is significantly reduced to $A\approx 25\,\mathrm{MHz}$ in the Si quantum well of a Si/Si$_{0.7}$Ge$_{0.3}$ heterostructure due to strain effects caused by the Si- and SiGe-lattice mismatch \cite{huebl2006,usman2015}. On the other hand, the donor is ionized and the electron does not interact with the donor nuclear spin if it occupies the left QD. Therefore, we can represent the electron spin nuclear spin interaction as
\begin{eqnarray}
\widetilde{H}_{e-n}=\frac{A}{8}\vec{\sigma}\cdot \vec{\nu}\cdot (1-\widetilde{\tau}_z),
\label{eq:He-n}
\end{eqnarray}
with $\vec{\nu}=(\nu_x,\nu_y,\nu_z)^T$ and $\nu_i$ the nuclear spin Pauli operators. The factor $(1-\widetilde{\tau}_z)/2$ is a projection on the subspace with the electron bound to the donor.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\columnwidth]{energyspectrum_system}
	\caption{Energy spectrum of the QD-donor Hamiltonian $\widetilde{H}_{0}$ as a function of the
	QD-donor detuning parameter $\epsilon$ for $B_z=14.5\,\upmu\mathrm{eV}$. The dashed blue lines show the energy levels for $t_c=b_x=0$ with the corresponding eigenstates denoted by the electronic position in the
	QD-donor system, $L$ and $R$, and its spin projection $(\downarrow/\uparrow)$ along the $z-$axis. Switching on the tunnel coupling $t_c$ (here $t_c=9\,\upmu\mathrm{eV}$) mixes the $L$ and $R$ states with the same spin resulting in the bonding $(-)$ and antibonding $(+)$ molecular orbital states, whose energies are given by the dashed black lines. Finally, the dotted red energy levels include the hyperfine coupling to the nuclear spin (eigenenergies $E_0,\ldots,E_7$ of $\widetilde{H}_{\rm sys}$). For this plot, we have used exaggerated values $b_x=5\,\upmu\mathrm{eV}$ and $A=1\,\mathrm{GHz}$ to emphasize the effects of the magnetic field gradient and the hyperfine interaction.}
	\label{fig:energylevels}
\end{figure}

Signatures of the electron spin-photon coupling can be observed in the cavity transmission \cite{mi2018,benito2017}. We now investigate whether these signatures will be altered in the presence of a nuclear spin interacting with the electron spin via the hyperfine interaction and whether the combined spin-photon and hyperfine interactions have a potential application for nuclear spin readout.

To calculate the cavity response, we first transform the total Hamiltonian to the eigenbasis $|\pm\rangle$
of $\epsilon \widetilde{\tau}_z/2 +t_c \widetilde{\tau}_x$,
with the electron position expressed in terms of antibonding ($+$) and bonding ($-$) molecular orbital states because these basis states are a good approximation for the eigenstates $|n\rangle$ with corresponding energies $E_n$ of $\widetilde{H}_{\rm sys}=\widetilde{H}_0+\widetilde{H}_{\mathrm{e-n}}$ as illustrated in Fig.~\ref{fig:energylevels}. Then, the Hamiltonian $\widetilde{H}=\widetilde{H}_{\rm sys}+\widetilde{H}_{I}+\widetilde{H}_{\mathrm{cav}}$ can be written as the sum of a diagonal part $H_0$ and an off-diagonal perturbation $V$ as $H=H_0+V$, with
\begin{align}
H_0=&\tfrac{1}{2}\left(\Omega \tau_z + B_z \sigma_z  \right)+
\tfrac{1}{8} A \sigma_z \nu_z \left(1-\sin\theta\, \tau_z\right) + \omega_c a^{\dagger} a,
\label{eq:H0}\\
V=&\frac{b_x}{2}\left(\sin\theta\,\tau_z -\cos\theta\, \tau_x \right)\sigma_x + \frac{A}{8}\cos\theta(\sigma_z \nu_z) \,\tau_x \nonumber\\
&+\frac{A}{8}(\sigma_x \nu_x + \sigma_y \nu_y) \left(1-
\sin\theta\,\tau_z + \cos\theta\,\tau_x\right)\nonumber\\
&+ g_c (\sin\theta\,\tau_z-\cos\theta\,\tau_x)(a + a^{\dagger}),
\label{eq:V}
\end{align}
where $\tau_i$ are Pauli operators acting on the space of bonding ($+$) and antibonding ($-$) orbitals, i.e. $\tau_z |\pm\rangle=\pm |\pm\rangle$.
Moreover, we introduce the orbital energy $\Omega=\sqrt{\epsilon^2+
	4t_c^2}$ and the orbital mixing angle $\theta=\arctan(\epsilon /2 t_c)$.
$H_0$ is diagonal with respect to the basis $\{|\pm,\downarrow (\uparrow), \Downarrow (\Uparrow),n\rangle\}$ indicating the orbital state of the electron $(\pm)$, the electron spin state ($\downarrow,\uparrow$), the nuclear spin state ($\Downarrow,\Uparrow$) and the number of photons in the cavity mode $(n)$, respectively, while $V$ is purely off-diagonal in this basis.
In order to predict the impact of the nuclear spin on the cavity transmission, we derive an effective Hamiltonian for the lower orbital subspace defined by the projection operator
$P_0=(1-\tau_z)/2$,
that projects on the subspace spanned by the states
${|-,\downarrow,\Uparrow,n\rangle},{|-,\downarrow,\Downarrow,n\rangle},{|-,\uparrow,\Uparrow,n\rangle},{|-,\uparrow,\Downarrow,n\rangle}$ with $n=0,1,2,\ldots $.

 As a next step, we apply a Schrieffer-Wolff transformation to decouple the subspaces defined by the projection operators $P_0$ and $Q_0=1-P_0$ \cite{bravyi2011},
to find the effective Hamiltonian $H_{\mathrm{eff}}=e^SHe^{-S}$,
and follow the perturbative method presented in \cite{bravyi2011} to determine the block off-diagonal and antihermitian generator $S$ defining the unitary transformation $e^S$.
If one chooses the ansatz $S=\sum_{n=1}^{\infty}S_n$ with $S_n\sim V^n$, the first contribution $(S_1)$ must obey the relation \cite{bravyi2011}
$[H_0,S_1]= P_0VQ_0+Q_0VP_0$.
This relation together with the commutation relations of the Pauli operators and the bosonic photon operators allows us to determine $S_1$.
The knowledge of $S_1$ is in turn sufficient to compute the effective Hamiltonian for the subspace defined by $P_0$ up to second order in the perturbation $V$ \cite{bravyi2011},
\begin{align}
H_{e}=&P_0H_0P_0+P_0VP_0\nonumber\\
&+\tfrac{1}{2}P_0[S_1,P_0VQ_0+Q_0VP_0]P_0.
\label{eq:Heffeformula}
\end{align}
The explicit form of $H_{e}$ is presented in Appendix \ref{appendix:effective_Hamiltonian}. However, for the following discussion it is essential to determine transition frequencies as precisely as possible. To this end, we transform $H_{e}$ to a basis accounting for the electron spin mixing due to the magnetic field gradient with the basis states
\begin{align}
|\tilde{\downarrow},\Uparrow(\Downarrow)\rangle=&\cos \frac{\phi}{2}|-,\downarrow,\Uparrow(\Downarrow)\rangle
-\sin\frac{\phi}{2}|-,\uparrow,\Uparrow(\Downarrow)\rangle,\\
|\tilde{\uparrow},\Uparrow(\Downarrow)\rangle=&\sin\frac{\phi}{2}|-,\downarrow,\Uparrow(\Downarrow)\rangle
+\cos\frac{\phi}{2}|-,\uparrow,\Uparrow(\Downarrow)\rangle,
\end{align}
defined by the electron spin mixing angle $\phi$ via
\begin{align}
\tan\phi=\frac{-b_x \sin \theta}{B_z\left( 1-\cos^2\theta\left(\tfrac{A^2}{8}+b_x^2\right)\tfrac{1}{\Omega^{2}-B_z^{2}}\right)}.
\end{align}
Since, here, $b_x\ll B_z$ the electron spin mixing angle is small and therefore the states $|\tilde{\downarrow}(\tilde{\uparrow})\rangle$ are predominantly the electron spin states $|\downarrow(\uparrow)\rangle$ up to small contributions of the opposite electron spin state. Hence, in the following we refer to $|\tilde{\downarrow}(\tilde{\uparrow})\rangle$ as the electron spin states.
The diagonal part of the transformed Hamiltonian reads
\begin{align}
H_{e,0}=& \tfrac{1}{2}\left(E_{\tilde{\sigma}}+\delta E_{\tilde{\sigma}}\nu_z\right)\tilde{\sigma}_z+ \tfrac{1}{2} E_{\nu}\nu_z+\widetilde{\omega}_c a^\dagger a,
\label{eq:Heffdiagonalmain}
\end{align}
with the Pauli operators $\widetilde{\sigma}_i$ operating on the $|\tilde{\downarrow}(\tilde{\uparrow})\rangle$ states and
\begin{align}
    &E_{\tilde{\sigma}}=\sqrt{{B_z^2}\left(1-\frac{\left(A^2+8b_x^2\right)\cos^2\theta}{8(\Omega^{2}-B_z^{2})}\right)^2+\left(b_x\sin\theta\right)^2},\\
    %
    &\delta E_{\tilde{\sigma}}=\left(\frac{A}{4}(1\!+\!\sin\theta)
    +\frac{\Omega}{B_z}E_\nu\right)\cos\phi
    \nonumber\\
    &\quad\quad\quad -\frac{Ab_x\cos\theta B_z}{8(\Omega^{2}-B_z^{2})}\sin\phi,\\
    %
    &E_{\nu}=\frac{A^2\cos^2\theta\,{B_z}}{{16(\Omega^{2}}-B_z^{2})},\\
    %
    &\widetilde{\omega}_c=\omega_c-2\frac{g_c^2\cos^2\theta\,B_z}{\Omega^{2}-\omega_c^{2}},
\end{align}
as derived in Appendix \ref{appendix:effective_Hamiltonian}.
Since the signatures of the electron spin-photon coupling that we expect to change  due to the nuclear spin are observed close to resonance between the electron spin transition and the resonator \cite{benito2017,mi2018}, it is justified to assume $E_{\tilde{\sigma}}\approx\widetilde{\omega}_c$. Under this assumption we can apply the rotating wave approximation (RWA) retaining terms rotating with frequencies $\ll E_{\tilde{\sigma}}\approx\widetilde{\omega}_c$ and find that the nondiagonal part of the transformed Hamiltonian comprises interactions between the electron spin and the nuclear spin of the  QD-donor system and the cavity mode
\begin{align}
H_{e,\mathrm{cav-int}}=
&- g_{\tilde{\sigma}\nu}\sin^2\frac{\phi}{2}\left(\tilde{\sigma}^+ \nu^+ a +\tilde{\sigma}^- \nu^- a^{\dagger}\right)\nonumber\\
&+g_{\tilde{\sigma}\nu}\cos^2\frac{\phi}{2}\left(\tilde{\sigma}^+ \nu^- a +\tilde{\sigma}^- \nu^+ a^{\dagger}\right)\label{eq:HeffeintRWA}\\
&+\left(g_{\tilde{\sigma}}\cos\phi+\delta g_{\tilde{\sigma}}\sin\phi\,\nu_z\right)\left(\tilde{\sigma}^+a+\tilde{\sigma}^- a^{\dagger}\right),\nonumber
\end{align}
with the explicit forms of the spin-photon couplings $g_{\tilde{\sigma}\nu}, g_{\tilde{\sigma}}, \delta g_{\tilde{\sigma}}$
given in Appendix \ref{appendix:effective_Hamiltonian}.
The interaction terms in \eqref{eq:HeffeintRWA} are of particular interest since one can expect to see signatures of these interactions in the transmission. However, the terms in the first line are negligible for $\phi\ll 1$.

The terms in the second line of \eqref{eq:HeffeintRWA} incorporate a flip of both the nuclear spin and the electron spin if the two are antialigned  with the concomitant creation or annihilation of a cavity photon. This coupling emerges due to the combined effect of the dipole operator coupling the states $|-,\uparrow,\Downarrow(\Uparrow)\rangle$ and $|+,\uparrow,\Downarrow(\Uparrow)\rangle$, and the hyperfine interaction between the states $|+,\uparrow,\Downarrow\rangle$ and $|-,\downarrow,\Uparrow\rangle$. Thus, the interaction persists in the absence of the magnetic field gradient and has already been observed and analyzed in setups without such a gradient. The interaction can be used to control the flip-flop qubit and to construct gates between two such qubits \cite{tosi2017}, while the combination with an oscillating magnetic field allows for controlling the nuclear spin qubit and implementing a nuclear spin two-qubit gate \cite{tosi2018}.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\columnwidth]{energyspectrum_Heffdiag}
	\caption{Hyperfine-split electronic energy levels (\ref{eq:energyexpectationvaluesdown}) and (\ref{eq:energyexpectationvaluesup}) corresponding to the four states $|\tilde{\downarrow},\Uparrow\rangle$, $|\tilde{\downarrow},\Downarrow\rangle$, $|\tilde{\uparrow},\Uparrow\rangle$ and $|\tilde{\uparrow},\Downarrow\rangle$ as a function of the QD-donor detuning parameter $\epsilon$. The highlighted transition frequencies determine the location of the observable signatures in Fig.~\ref{fig:cavityTransmission}. The  parameters used for this plot are $B_z=\omega_c/2\pi=3.5\,\mathrm{GHz}$, $g_c/2\pi=13\,\mathrm{MHz}$, $t_c=9\,\upmu\mathrm{eV}$, $b_x=1.62\,\upmu\mathrm{eV}$, and $A=25\,\mathrm{MHz}$.}
	\label{fig:energyspectrumheffdiag}
\end{figure}

On the other hand, the combined effect of the magnetic field gradient, giving rise to the coupling between the states $|+,\uparrow,\Downarrow(\Uparrow)\rangle$ and $|-,\downarrow,\Downarrow(\Uparrow)\rangle$, and the dipole operator leads to the terms in the third line that describe a flip of the electron spin  accompanied by the annihilation or creation of a cavity photon, while the state of the nuclear spin remains unchanged. These two different types of interaction cause a hybridization of the QD-donor system and the cavity mode when the transition in the QD-donor system is close to resonance with the cavity mode. Since the resulting hybrid states have a significant impact on the cavity transmission we inspect the energy expectation values of the QD-donor system states involved in the respective transitions.
The energy expectation values of the four basis states defining the lower orbital subspace can be easily read off from \eqref{eq:Heffdiagonalmain}:
\begin{align}
&E_{|\tilde{\downarrow},\Uparrow(\Downarrow)\rangle}=-\frac{E_{\tilde{\sigma}}}{2}\varmp\frac{\delta E_{\tilde{\sigma}}}{2}\varpm\frac{E_{\nu}}{2},\label{eq:energyexpectationvaluesdown}\\
&E_{|\tilde{\uparrow},\Uparrow(\Downarrow)\rangle}= \frac{E_{\tilde{\sigma}}}{2}\varpm\frac{\delta E_{\tilde{\sigma}}}{2}\varpm\frac{E_{\nu}}{2},
\label{eq:energyexpectationvaluesup}
\end{align}
and we immediately find the transition frequencies for electron spin flips with a fixed nuclear spin state
\begin{align}
E_{|\tilde{\uparrow},\Uparrow(\Downarrow)\rangle}-E_{|\tilde{\downarrow},\Uparrow(\Downarrow)\rangle}=E_{\tilde{\sigma}}\varpm\delta E_{\tilde{\sigma}},
\label{eq:electrontransitionfrequency}
\end{align}
as well as the transition frequency for the electron spin-nuclear spin flip-flop
\begin{align}
E_{|\tilde{\uparrow},\Downarrow\rangle}-E_{|\tilde{\downarrow},\Uparrow\rangle}=E_{\tilde{\sigma}}-E_{\nu}.
\label{eq:ekectronnuclearflipflop}
\end{align}
\begin{figure*}[ht]
	\includegraphics[width=\textwidth]{Transmission}
	\caption{Cavity transmission $|A_c|$ on resonance, i.e., for $\omega=\omega_c$ with $\omega_c/2\pi=3.5\,\mathrm{GHz}= 14.5\,\upmu\mathrm{eV}$, as a function of the magnetic field $B_z$ and the QD-donor detuning $\epsilon$ for different populations of the energy levels with (a) both nuclear spin states  equally populated, (b)  the nuclear spin up state  populated, (c)  the nuclear spin down state  populated. The  dashed lines indicate the points in the $(\epsilon,B_z)$ parameter plane at which the transitions indicated in Fig.~\ref{fig:energyspectrumheffdiag} fulfill one of the resonance conditions \eqref{eq:resonance_spin_flip} or \eqref{eq:resonance_flipflop}. The remaining system parameters are	$t_c=9\,\upmu\mathrm{eV}$, $b_x=1.62\,\upmu\mathrm{eV}$, $g_c/2\pi=13\,\mathrm{MHz}$, ${2\kappa_1/2\pi=2\kappa_2/2\pi=\kappa/2\pi=1.3\,\mathrm{MHz}}$, and $A=25\,\mathrm{MHz}\approx 0.1\,\upmu\mathrm{eV}$. For the phonon induced decoherence entering the decoherence superoperator $\gamma$ we assume the parameters $d=37\,\mathrm{nm}$, $\omega_0/2\pi=30\,\mathrm{GHz}$, $c_b=4000\,\mathrm{m/s}$ and $J_0$ determined by $J(2t_c=5.4\,\mathrm{GHz},d=120\,\mathrm{nm})=35\,\mathrm{MHz}$. The strength of the quasistatic charge noise affecting the detuning paramter $\epsilon$ is chosen to be $\sigma_{\epsilon}=1\,\upmu\mathrm{eV}$.}
	\label{fig:cavityTransmission}
\end{figure*}
\begin{figure}[ht]
\includegraphics[width=\columnwidth]{goodreadoutpoints}
    \caption{Difference of the cavity transmission $|A_c|$ between the nuclear spin down case  $|A_c|_{\Downarrow}$, and  the nuclear spin up case $|A_c|_{\Uparrow}$. The dot-dashed lines in (a) indicate the positions of the linecuts shown in (b). Parameters and dashed lines are as in Fig.~\ref{fig:cavityTransmission}. }
    \label{fig:readoutdifference}
\end{figure}
\begin{figure}[ht]
    \centering
    \includegraphics[width=\columnwidth]{deltaomegaEpsilonFive}
    \caption{Cavity transmission $|A_c|_{\Uparrow}$, $|A_c|_{\Downarrow}$ and readout contrast, ${|A_c|_{\Downarrow}-|A_c|_{\Uparrow}}$, for $\epsilon=10\,\upmu\mathrm{eV}$ and $B_z=14.57\,\upmu\mathrm{eV}$, i.e. the point where the red and the second dashed orange line from the left in Fig.~\ref{fig:readoutdifference} intersect, as a function of the probe pulse detuning $\delta\omega$. The remaining parameters are as in Fig.~\ref{fig:cavityTransmission}.}
    \label{fig:deltaomega}
\end{figure}
The energy expectation values (\ref{eq:electrontransitionfrequency}) as a function of the QD-donor detuning $\epsilon$ and the various transition frequencies are presented in Fig.~\ref{fig:energyspectrumheffdiag}.
Both Figure~\ref{fig:energyspectrumheffdiag} and Eq.~\eqref{eq:electrontransitionfrequency} show that the electron spin flip transition frequency depends on the state of the nuclear spin. More precisely, for a small electron spin mixing angle $\phi \ll 1$ the transition frequency with the nuclear spin in the states $\Uparrow$ and $\Downarrow$ differs by
\begin{align}
\Delta=2\delta E_{\tilde{\sigma}}
\approx\frac{A}{2}\left(1+\sin\theta\right)+\frac{A^2\cos^2\theta\,\Omega}{8(\Omega^{2}-B_z^{2})}.
\label{eq:transitionfrequencydifference}
\end{align}
Hence, in the limits of large positive, zero, and large negative DQD detuning $\epsilon$, the shift in the resonance frequency $\Delta$ takes the values (note that $t_c>0$),
\begin{align}
\epsilon\gg 2t_c: & & \lim_{\theta\rightarrow \pi/2} \Delta\approx A, \nonumber\\
\epsilon=0:& & \Delta\approx \frac{A}{2}+\frac{A^2\Omega}{8(\Omega^{2}-B_z^{2})}, \\
\epsilon\ll - 2t_c: & & \lim_{\theta\rightarrow -\pi/2} \Delta\approx 0. \nonumber
\end{align}
The increasing impact of the nuclear spin on $\Delta$ with increasing QD-donor detuning is intuitively easy to understand: For $\epsilon\ll -2 t_c$ the electron is  localized in the left QD and therefore decoupled from the nuclear spin, at $\epsilon=0$ it is completely delocalized between the left QD and the donor, while it is trapped in the donor with a high probability for $\epsilon\gg 2 t_c$ such that the coupling to the nuclear spin is maximized.

We note that in a DQD architecture with the second QD overlapping with an isoelectric $^{29}$Si nuclear spin, readout of the nuclear spin state has been realized for the maximal coupling scenario  $\epsilon\gg 2 t_c$ by probing the electron spin resonance frequency with frequency-selective ac magnetic field pulses \cite{hensen2020}. Even though the hyperfine interaction is as low as a few hundred kHz in such a device, we expect that alternatively our suggested readout method can be used as discussed in detail in Appendix \ref{appendix:hyperfineresolution}.

\section{Nuclear spin readout via the electron spin \label{sec:nuclearspinreadout}}

We now describe how the nuclear-spin dependent shift $\Delta$ of the electron-spin resonance frequency Eq.~\eqref{eq:transitionfrequencydifference} allows for a read-out of the nuclear spin.
The last term in \eqref{eq:Heffdiagonalmain} identifies the cavity resonance frequency including shifts of the empty cavity frequency $\omega_c$ due to the interaction with the QD-donor system. Thus, the cavity mode is resonant with the electron spin flip transition for a fixed nuclear spin state if
\begin{align}
E_{|\tilde{\uparrow},\Uparrow(\Downarrow)\rangle}-E_{|\tilde{\downarrow},\Uparrow(\Downarrow)\rangle}=\widetilde{\omega}_c,
\label{eq:resonance_spin_flip}
\end{align} and resonant with the electron spin-nuclear spin flip-flop transition if
\begin{align}
E_{|\tilde{\uparrow},\Downarrow\rangle}-E_{|\tilde{\downarrow},\Uparrow\rangle}=\widetilde{\omega}_c.
\label{eq:resonance_flipflop}
\end{align}
 We expect a signature of the respective coupling in the cavity transmission in the vicinity of system parameters $\epsilon$, $t_c$, $B_z$, $b_x$, and $\omega_c$ for which one of these relations is fulfilled.

In order to verify our prediction we calculate the cavity transmission $A_c$ using input-output theory (Appendix~\ref{sec:InputOutput}) and compare the system parameters for which characteristic features emerge with those satisfying the resonance conditions derived above. The calculation of $A_c$ takes charge relaxation processes due to the phonon environment and quasi-static charge noise affecting the detuning parameter $\epsilon$ into account (see Appendix~\ref{appendix:DecoherenceModel} for details).
Figure~\ref{fig:cavityTransmission} shows the absolute value of the cavity transmission $|A_c|$ for three different populations of the hyperfine levels where (a) the two lowest energy levels are equally populated approximating the thermal equilibrium state for $T\gtrsim 30\,\mathrm{mK}$, i.e., the
QD-donor system is with equal probability in the states $|0\rangle$ and $|1\rangle$ which, up to small corrections, correspond to the nuclear spin up and down states
$|\tilde{\downarrow},\Uparrow\rangle$ and $|\tilde{\downarrow},\Downarrow\rangle$, respectively; (b) only the ground state $\approx |\tilde{\downarrow},\Uparrow\rangle$ is populated; (c) only the excited state $\approx |\tilde{\downarrow},\Downarrow\rangle$ is populated. We point out that a single measurement will always be represented by the Figs.~\ref{fig:cavityTransmission}$(b)$ or $(c)$, while Fig.~\ref{fig:cavityTransmission}$(a)$ corresponds to the average over many measurements if the system is initialized with equal probability in the states $|0\rangle\approx |\tilde{\downarrow},\Uparrow\rangle$ and $|1\rangle \approx |\tilde{\downarrow},\Downarrow\rangle$ before the measurement. We find that the emerging characteristic features, given by a significantly reduced transmission due to the interaction of the cavity mode with the QD-donor system appear in the immediate vicinity of the parameters fulfilling the resonance conditions Eqns. \eqref{eq:resonance_spin_flip} and \eqref{eq:resonance_flipflop}, as indicated by the dashed lines in Fig.~\ref{fig:cavityTransmission}.
One also observes that the signatures are less pronounced for $|\epsilon| \gg 2 t_c$. The last line of Eq.~\eqref{eq:V} shows that the electric dipole moment of the $|+\rangle\leftrightarrow|-\rangle$ transition is proportional to $\cos\theta$ and therefore decreases with increasing $|\epsilon/2t_c|$, which, in turn leads to the weakening of the effective couplings responsible for the observed signatures.

For the experimental realization of nuclear spin state readout it is essential to obtain a strong contrast between the signal for nuclear spin $\Uparrow$ and $\Downarrow$. In order to identify suitable readout points, we calculate the difference of the cavity transmission $|A_c|$ obtained for the excited state populated, $|A_c|_{\Downarrow}$, and the one with only the ground state populated, $|A_c|_{\Uparrow}$, i.e., Fig.~\ref{fig:cavityTransmission}$(b)$ is subtracted from Fig.~\ref{fig:cavityTransmission}$(c)$. The result presented in Fig.~\ref{fig:readoutdifference}$(a)$ unveils extended regions providing a high signal contrast for nuclear spin readout in the vicinity of the three resonance conditions, the two resonances \eqref{eq:resonance_spin_flip} and the resonance \eqref{eq:resonance_flipflop}, and weak QD-donor detuning in the range between $\epsilon=-10\,\upmu\mathrm{eV}$ and $\epsilon=15\,\upmu\mathrm{eV}$. The linecuts in Fig.~\ref{fig:readoutdifference}$(b)$ show that, within this range of QD-donor detuning, maximal contrast is achieved for points in the immediate vicinity of the resonance for nuclear spin $\Uparrow$. The amplitude difference of the readout contrast between the resonances for $\Uparrow$ and $\Downarrow$ can be attributed to a shift of the cavity resonance frequency caused by the interaction with the QD-donor system (see Appendix~\ref{appendix:hyperfineresolution} for more details).

Moreover, we can check the sensitivity of the readout contrast with respect to the cavity detuning from the probe field
\begin{align}
    \delta\omega=\omega-\omega_c,
\end{align}
for good readout points. To do so, we calculate
$|A_c|_{\Uparrow}$, $|A_c|_{\Downarrow}$ and the readout contrast, $|A_c|_{\Downarrow}-|A_c|_{\Uparrow}$, for the point in Fig.~\ref{fig:readoutdifference} where the red and the second dashed orange line from the left intersect, as a function of the detuning $\delta\omega$. The result is presented in Fig.~\ref{fig:deltaomega} and shows a readout contrast larger than $0.2$ for $|\delta\omega|<1.5\,\mathrm{MHz}$. In addition, the figure allows one to identify the origin of reduced transmission in Fig.~\ref{fig:cavityTransmission}$(b)$ and the  resulting good readout contrast:
At the chosen readout point, the electron spin flip transition for nuclear spin $\Uparrow$ is close to resonance with the cavity, while the electron spin flip transition for nuclear spin  $\Downarrow$ is off-resonant. Due to the strong electron spin photon-coupling one observes Rabi splitting for nuclear spin $\Uparrow$ ($|A_c|_{\Uparrow}$ in Fig.~\ref{fig:deltaomega}), whereas $|A_c|_{\Downarrow}$ shows a single resonance located between the Rabi split modes of $|A_c|_{\Uparrow}$.

To further characterize the nuclear spin measurement, we go beyond the input-output theory and  inspect Eqs.~\eqref{eq:Heffdiagonalmain} and \eqref{eq:Heintfull} describing the effective electronic Hamiltonian $H_{e,0}+H_{e,\mathrm{int}}$ in order to assess the expected measurement back-action. We note that since $[H_{e,0},\nu_z]=0$, the main part
of the hyperfine coupling leads to a nuclear spin readout in the form of a quantum non-demolition (QND) measurement \cite{braginsky1992}.  In general, $[H_{e,\mathrm{int}},\nu_z]\neq 0$ leading to small corrections to the QND behavior. However, for an adiabatic transfer of the electron from the left QD to the delocalized configuration between the QD and the donor and back under continuous transmission of a microwave field at constant frequency, we expect a recovery of the QND readout because the pure nuclear spin states are adiabatically transferred to eigenstates of $H_{e,0}+H_{e,\mathrm{int}}$.
Away from the resonance, the analogous argument holds with the non-resonant Hamiltonian \eqref{eq:effectiveHamiltonianTransmission}.

For the experimental verification of the suggested method for nuclear spin readout, we envision the following protocol: The cavity transmission is measured at one of the suitable readout points. Then, a nuclear spin resonance $\pi$-pulse is performed before the cavity transmission is probed again. Following the above discussion, successful nuclear spin readout is achieved if there is a significant difference in the absolute value of the transmission, and, depending on this value for the respective measurement, the state of the nuclear spin at the time of each measurement can be assigned.

\section{Nuclear spin photon coupling \label{sec:nuclear_spin-photon_coupling}}

 It has been shown that the nuclear spin of a QD-donor system can be controlled with a classical electric field \cite{boross2018}. However, this does not allow coherent information transfer between the nuclear spin and photons.
In order to assess the potential of the system for coherent coupling of the nuclear spin to cavity photons, we derive a Hamiltonian describing the effective dynamics of the nuclear spin interacting with the resonator mode while the remaining parts of the system are near the ground state.
More precisely, we investigate the dynamics of the subspace determined by the projection operator
\begin{align}
P_0=\frac{1-\tau_z}{2}\,\frac{1-\sigma_z}{2},
\end{align}
that defines the subspace spanned by the states ${|-,\downarrow,\Downarrow,n\rangle}$,
${|-,\downarrow,\Uparrow,n\rangle}$, with $n=0,1,2,...$ .
 To do this, we apply a Schrieffer-Wolff transformation to decouple the subspaces defined by the projection operators $P_0$ and $Q_0=1-P_0$ \cite{bravyi2011}. Following the  procedure sketched in Sec. \ref{sec:nuclearspinreadout}, we determine $S_1$ and $S_2$, where $S_2$ is defined by
$[H_0,S_2]=  -[P_0VP_0+Q_0VQ_0,S_1]$,
to obtain the effective Hamiltonian for the subspace defined by $P_0$ up to third order in the perturbation $V$ \cite{bravyi2011},
\begin{align}
H_{n}=&P_0H_0P_0+P_0VP_0+\tfrac{1}{2}P_0[S_1,P_0VQ_0+Q_0VP_0]P_0\nonumber\\
&+\tfrac{1}{2}P_0[S_2,P_0VQ_0+Q_0VP_0]P_0.
\label{eq:Heffformulanu}
\end{align}
In particular, the diagonal part of the effective Hamiltonian reads
\begin{align}
    H_{n,0}=&\frac{E_{\Bar{\nu}}}{2}\nu_z+\Bar{\omega}_ca^\dagger a+\frac{\delta E_{\Bar{\nu}}}{2}\nu_za^\dagger a,
\end{align}
with the expressions for ${E_{\Bar{\nu}},\,\Bar{\omega}_c}$ and $\delta E_{\Bar{\nu}}$ presented in Appendix \ref{appendix:EffectiveHnuclearspin}. We find that ${|\delta E_{\Bar{\nu}}|\ll |E_{\Bar{\nu}}|\ll \Bar{\omega}_c}$ if the electron is not entirely confined to the left QD. Thus, the microwave resonator and the donor nuclear spin flip transition cannot be tuned to resonance.
The coherent excitation exchange between these two subsystems is described by the term,
\begin{align}

    g_{\nu}\left(\nu^{+}a^{\dagger}+\nu^{-}a\right),
    \label{eq:nuplusadager}
\end{align}
within $H_n$ given in \eqref{eq:Heff}. We note that ${E_{|\Downarrow,n\rangle}>E_{|\Uparrow,n\rangle}}$ because ${E_{\Bar{\nu}}\approx-\tfrac{A}{4}(1+\sin(\theta))+\mathcal{O}(V^2)}$ such that Eq.~\eqref{eq:nuplusadager} is an excitation conserving interaction term.
The explicit form of the coupling constant $g_{\nu}$ in terms of the system parameters is given in Appendix~\ref{appendix:EffectiveHnuclearspin} and we find that nuclear spin to photon coupling strengths of  ${g_\nu\approx 0.5}$ MHz can be achieved.
Given realistic values for the nuclear-spin
and cavity loss rates, $\gamma\ll \kappa \approx 1$ MHz,
we note that the strong coupling regime for nuclear spin cavity QED ($g_\nu\gg \kappa, \gamma$) should be within reach. However, the coherent excitation exchange between these two subsystems is suppressed by the large detuning from resonance.

The raising or lowering of the nuclear spin state along with the creation or annihilation of a cavity photon results from the combined effect of the hyperfine interaction, the magnetic field gradient and the electric dipole interaction.
The fundamental problem preventing resonant coupling is that, in principle, the energy splitting between the orbital states ($+$ and $-$) and the energy splitting between the electron spin states ($\uparrow$ and $\downarrow$) can simultaneously be tuned close to resonance with the microwave resonator, while, at the same time, the energy splitting between the nuclear spin states ($\Uparrow$ and $\Downarrow$) is far off-resonant because the nuclear gyromagnetic ratio is $\approx 1000$ times smaller than the one of the electron spin.

\section{Conclusion \label{sec:conclusion}}
In conclusion, we have investigated a system composed of a donor nuclear spin coupled to the spin of a single electron in a QD-donor architecture via the hyperfine interaction. The electron is subject to a homogeneous magnetic field and a magnetic field gradient perpendicular to the homogeneous component, while it is also dipole coupled to a microwave resonator.
We demonstrate that the effective excitation-conserving nuclear spin-photon interaction resulting from the combined effect of the hyperfine interaction, the electric dipole interaction, and the magnetic field gradient cannot directly be tuned to resonance.

Nevertheless, we show that the signature of the strong electron spin-photon coupling \cite{benito2017} in the cavity transmission is altered due to the hyperfine interaction. We find well separated signatures for the electron spin-photon coupling with the nuclear spin in the states $\Uparrow$ and $\Downarrow$, whereby the splitting of the two signatures is determined by the hyperfine interaction strength $A$. For a $^{31}\mathrm{P}$ donor in the strained Si quantum well with $A\approx 25\,\mathrm{MHz}$ we expect that recent experimental setups are able to resolve the split signatures individually. Moreover, we identified good readout points at which one finds a high contrast between the measurement signal for the two opposing nuclear spin polarizations. Therefore, the cavity transmission allows for a readout of the nuclear spin state and for the measurement of the hyperfine interaction strength.

\section*{Acknowledgments}
We thank M\'{o}nica Benito and N. Tobias Jacobson for helpful discussions. This work has been supported by ARO grant number W911NF-15-1-0149.


\section{Introduction}
\label{sec:intro}

The fifth (5G) and sixth generations (6G) of cellular networks will undoubtedly accelerate the transition from inflexible and monolithic networks to agile, disaggregated architectures based on \emph{softwarization} and \emph{virtualization}, as well as on openness and re-programmability of network components~\cite{bonati2020open}.
These novel architectures are expected to become enablers of new functionalities, including the ability to: (i)~Provide on-demand virtual network slices that, albeit sharing the same physical infrastructure, are tailored to different mobile virtual network operators, network services and run-time traffic requirements; (ii)~split network functions across multiple software and hardware components, possibly provided by multiple vendors; (iii)~capture and expose \glspl{kpi} and network analytics through open interfaces that are not available in old architectures, and (iv)~control the entire network physical infrastructure in real time via third party software applications and open interfaces.

\subsection{Disaggregation and Programmability in O-RAN}

The \emph{O-RAN Alliance}---a consortium of industry and academic institutions---is working toward realizing the vision of Next Generation (NextG) cellular networks, where telecom operators use standardized interfaces to control multi-vendor infrastructures and deliver high performance services to their subscribers~\cite{oran-arch-spec}.
To achieve this goal, the Alliance proposes an architectural innovation based on two core principles. First, O-RAN embraces and promotes the \gls{3gpp} functional split, where \gls{bs} functionalities are virtualized as network functions and divided across multiple network nodes, i.e., \gls{cu}, \gls{du} and \gls{ru}~\cite{bonati2020open}.
This facilitates the instantiation and execution of diverse networking processes at different points of the network. Specifically, \glspl{cu} implement functionalities at the higher layers of the protocol stack operating over larger timescales, while \glspl{du} handle time-critical operations at the lower layers. Finally, the \glspl{ru} manage \gls{rf} components and lower \gls{phy} layer parts.

The second core innovation---which is likely to be even more impactful---is the \gls{ric}, a new architectural component that provides a centralized abstraction of the network, allowing operators to implement and deploy custom control plane functions.
In both its \textit{non} and \textit{near real-time} versions, the \gls{ric} facilitates \gls{ran} optimization through closed-control loops, i.e., autonomous action and feedback loops between \gls{ran} components and their controllers.
O-RAN envisions different
loops operating at timescales that range from $1\:\mathrm{ms}$ (e.g., for real-time control of transmission strategies)
to
thousands of milliseconds (e.g., for network slicing and traffic forecasting).
For instance, the non real-time \gls{ric} performs operations with a time granularity higher than one second, such as training of \gls{ai} and \gls{ml} models.
The near real-time \gls{ric} instead handles procedures at timescales above $10\:\mathrm{ms}$, hosts third party applications (\textit{xApps}) that communicate with the \gls{cu}/\gls{du} through standardized open interfaces, and implements intelligence in the \gls{ran} through data-driven control loops.

Figure~\ref{fig:deployment} illustrates one of the possible disaggregated deployments specified by O-RAN, where different network components are connected by open interfaces.

\begin{figure}[ht]
	\centering
	\includegraphics[width=\columnwidth]{figures/figure-1-new_single.pdf}
	\caption{O-RAN: An example of disaggregated deployment.}
	\label{fig:deployment}
\end{figure}

In this deployment (``Scenario~B'', deemed the most common~\cite{bonati2020open}), the \glspl{ric} are deployed in the cloud. They interact with each other via the A1 and O1 interfaces, and control specific parameters of the \gls{ran} defined through the so-called \glspl{sm}.
The \gls{cu} and \gls{du} are deployed at the network edge, interconnected through the F1 interface and controlled by the near real-time \gls{ric} via the E2 interface~\cite{oran-e2}. The \gls{ru} is located at the operator cell site, and controlled by the \gls{du} through the Open Fronthaul interface. Finally, \gls{cu}, \gls{du} and \gls{ru} are connected to the non real-time \gls{ric} through the O1 interface for periodic reporting.
Other deployment options allow to instantiate \glspl{ric} and \glspl{cu} in the edge or regional cloud (Scenarios~A and C-F, respectively); the \gls{du} in the edge cloud (A-D); and the \gls{ru} at the operator cell site (A-D) or in the cloud cell site, possibly co-located with the \gls{du} (E, F)~\cite{bonati2020open}.

\vspace{-0.4cm}
\subsection{Contributions}

While the O-RAN architectural vision is gaining momentum among researchers,
the challenges of implementing it for data-driven, open, programmable and virtualized NextG networks are still largely to be dealt with.
Important architectural questions are yet to be answered, including (i)~the exact functionalities and parameters to be controlled by each network component; (ii)~where to place network intelligence; (iii)~how to validate and train data-driven control loop solutions, and (iv)~how \gls{ai} agents can access data and analytics from the \gls{ran} while minimizing the overhead of moving them from the \gls{ran} to the storage and inference locations.
To answer these questions, we provide the following contributions.

\noindent
$\bullet$ We discuss how data-driven, closed-control loop solutions can be implemented in NextG \glspl{ran}. We focus on the \textit{opportunities} offered by the O-RAN architecture, including functional split and open interfaces, and on their role in advancing intelligent and programmable
networks.

\noindent
$\bullet$
Differently from prior work~\cite{niknam2020intelligent, bonati2020open},
we investigate the \textit{limitations} of the current O-RAN specifications and the \textit{challenges} associated with deploying data-driven policies at different nodes of the \gls{ran}.

\noindent
$\bullet$ We discuss how large-scale experimental testbeds will play a key role by providing researchers with heterogeneous and large datasets, critical to the success of data-driven solutions for cellular networks.
We focus on the three PAWR platforms, i.e., POWDER~\cite{breen2020powder}, COSMOS~\cite{raychaudhuri2020cosmos}, and AERPAW~\cite{sichitiu2020aerpaw}, and on Colosseum and Arena~\cite{bertizzolo2020arena}, which can all be used to generate massive datasets under a variety of network configurations and \gls{rf} conditions.

\noindent
$\bullet$ We provide the first demonstration of an O-RAN data-driven control loop in a \textit{large-scale experimental testbed using open-source, programmable} \gls{ran} and \gls{ric} components.
We deploy O-RAN on the Colosseum network emulator and use it to control multiple network slices instantiated on 4~\gls{sdr} \glspl{bs} serving 40~\gls{sdr} \glspl{ue}.

\noindent
$\bullet$ We develop a set of \gls{drl} agents as \gls{ric} xApps to optimize key performance metrics for different network slices through \textit{data-driven closed-control loops}.
Experimental results show that our \gls{drl} approach outperforms other control strategies improving
spectral efficiency by up to~20\% and reducing buffer occupancy by up to~37\%.
We released the \gls{drl} agents and the $7\:\mathrm{GB}$ dataset used to train them.\footnote{\url{https://github.com/wineslab/colosseum-oran-commag-dataset}}

\begin{figure*}
\setlength\belowcaptionskip{-0.5cm}
	\centering
	\includegraphics[width=\textwidth]{figures/figure-2-new.pdf}
	\caption{Learning-based closed-control loops in an O-RAN architecture.}
	\label{fig:loops}
\end{figure*}

The remainder of this article is organized as follows.
We first discuss how intelligent control schemes can be embedded in the O-RAN architecture.
We then present how experimental testbeds can foster the development of data-driven solutions.
Finally, we present our experimental evaluation and draw our conclusions.

\vspace{-0.3cm}
\section{Intelligent Wireless Architectures}
\label{sec:intelligent}

Openness, programmability, and disaggregation are key enablers of data-driven applications. However, they are only the first step toward the seamless integration of \gls{ai} and \gls{ml}-based control loops in cellular networks.
Typically, data-driven approaches involve several
steps, ranging from data collection and processing, to training, model deployment and closed-loop control and testing.

This section illustrates how O-RAN is steering~5G deployments to bring intelligence to the network, by defining a practical architecture for the swift execution of data-driven operations, and discusses extensions to control procedures not currently considered by O-RAN.

\textbf{Data Handling and Training Procedures}. The effectiveness of data-driven approaches heavily depends on how data is handled, starting from data collection and aggregation at the \gls{ran} (where data is generated) to the point where it is processed for model training and inference. However, collecting and moving large amounts of data might result in significant overhead and latency costs. Hence, data-driven architectures must cope with tradeoffs between centralized approaches---providing a comprehensive view of the state of the network at the cost of overhead and latency---and distributed ones---which operate at the edge only, gather data from a small number of sources while enjoying low latency~\cite{polese2018machine}.

In this context, the O-RAN \gls{ml} specifications introduce standardized interfaces (e.g., O1) to collect and distribute data across the entire infrastructure as well as operational guidelines for the deployment of \gls{ml} and \gls{ai} solutions in the network~\cite{oran-ml}. These include practical considerations on how, where and when models can be trained, tested and eventually deployed in the network. First, \gls{ai}/\gls{ml} models are made available to operators via a marketplace system similar to that of the well-established \Gls{nfv} \Gls{mano} architecture, where models are stored in a catalog together with details on their control objectives, required resources, and expected inputs and outputs. Second, data-driven solutions must be trained and validated offline to avoid causing inefficiencies---or even outages---to the \gls{ran}.
Indeed, since \gls{ai}/\gls{ml} techniques usually rely upon a randomized initialization, O-RAN requires all \gls{ml} models to be trained and validated offline before their deployment~\cite{oran-ml}.
As we will discuss next, albeit shielding the network from unwanted behavior, this requirement also limits the effectiveness of such approaches, especially the online ones.
Online \gls{ai}/\gls{ml} techniques could still be used in O-RAN compliant architectures by allowing models to be trained with offline data in the non real-time RIC, and then perform online learning in the near real-time \gls{ric}. The smaller time-scale of the control loops of the latter would in fact allow the online training pipeline to be fed with data collected in real time.

\textbf{Control Loops}. Figure~\ref{fig:loops} portraits
how intelligence can be embedded at different layers and entities of a disaggregated cellular network together with the challenges and limitations of doing so. Each closed-control loop optimizes \gls{ran} parameters and operations by running at different timescales, with different number of \glspl{ue}, and using different sources for the input data.
The O-RAN Alliance is also looking into how to standardize the data-driven workflows for these control loops. As of this writing, O-RAN only considers non and near real-time loops, while real-time loops are left for future studies.
Figure~\ref{fig:loops} also depicts the additional inference timescale below $1\:\mathrm{ms}$ to process raw I/Q samples and perform \gls{ai}-driven \gls{phy} layer tasks, currently not part of
O-RAN as it would require device- and/or \gls{ru}-level standardization.

To better highlight the potential and limitations of the approach proposed by O-RAN, in the following we analyze each control loop individually, highlighting the role of each network component.
Finally, we discuss how the current O-RAN architecture can be extended to realize the control loops and applications illustrated in Fig.~\ref{fig:loops}.

\vspace{-.3cm}
\subsection{Non Real-time Control Loop}

The O-RAN Alliance defines \textit{non real-time} any control loop that operates on a \textit{timescale of at least one second.} As shown in Fig.~\ref{fig:loops}, this involves the coordination between the \textit{non real-time} and \textit{near real-time} \gls{ric}
through the A1 interface. This control loop manages the orchestration of resources at the infrastructure level, making decisions and applying policies that impact thousands of devices. These actions can be performed using data-driven optimization algorithms processing data from multiple sources, and inference models deployed on the non real-time \gls{ric}~itself.

Practical examples of non real-time data-driven control include instantiating and orchestrating network slices, as well as selecting which pre-trained inference models in the catalog should be deployed to accomplish operator intents, and deciding in which near real-time \gls{ric} these models should be executed. Said decisions can be made according to a variety of factors, ranging from computational resources and data availability to minimum performance requirements to comply with \acrlongpl{sla}. Moreover, since the non real-time \gls{ric} is endowed with \acrlong{smo} capabilities, this control loop can also handle the association between the near real-time \gls{ric} and the \glspl{du}/\glspl{cu}. This is particularly useful in virtualized systems where \glspl{du} and \glspl{cu} are dynamically instantiated on-demand to match the requests and load of the \gls{ran}.
However, non real-time loops are challenging to actuate in practice because of the very many interactions among the non real-time \gls{ric} and the network elements, which require tight coordination, data collection and orchestration capabilities.

\vspace{-0.2cm}
\subsection{Near Real-time Control Loops}

\textit{Near real-time} control loops operate on a timescale \textit{between $10\:\mathrm{ms}$ and $1\:\mathrm{s}$}. As shown in Fig.~\ref{fig:loops}, they run between the near real-time \gls{ric} and two components of the \glspl{gnb}: The \gls{cu} and the \gls{du}. Because one near real-time \gls{ric} is associated to multiple \glspl{gnb}, these control loops can make decisions affecting up to thousands of \glspl{ue}, using user-session aggregated data and \gls{mac}/\gls{phy} layer \glspl{kpi}. \gls{ml}-based algorithms are implemented as external applications, i.e., \textit{xApps}, and are deployed on the near real-time \gls{ric} to deliver specific services such as inference, classification, and prediction pipelines to optimize the per-user quality of experience, controlling load balancing and handover processes, or the scheduling and beamforming design.
Challenges of near real-time control loops include the need to promptly make decisions in a matter of tens or hundreds of milliseconds for each of the several \glspl{cu} and \glspl{du} controlled by the \gls{ric}.

\vspace{-0.1cm}
\subsection{Real-time Control Loops}

A crucial component of the operations of a cellular network involves actions at a \textit{sub-$10\:\mathrm{ms}$---or even sub-ms---timescale.} In O-RAN, these operations are labeled as \textit{real-time control loops,} and mainly concern interactions between elements in the \gls{du}. Control loops at a similar timescale could also be envisioned to operate between the \gls{du} and the \gls{ru}, or at the \glspl{ue}.
However, as deploying \gls{ml} solutions at the \gls{du} is not currently supported, these loops are left for future extensions of the O-RAN specifications.

Finally, data-driven approaches at the lower layers of the protocol stack or at the device, i.e., involving sub-ms timescales, are extremely powerful and can be used for data-driven scheduling decisions~\cite{chinchali2018cellular} and for feedback-less detection of \gls{phy} layer parameters (e.g., modulation and coding scheme, and interference recognition)~\cite{oshea2017introduction}.
Overall, the fact that device-/\gls{ru}-level standardization is required for sub-$ms$ loops makes it very challenging to realize them in practice, thus limiting their applicability.

\begin{figure*}
\setlength\belowcaptionskip{-0.7cm}
	\centering
	\includegraphics[width=.95\textwidth]{figures/figure-3_cn.pdf}
	\caption{O-RAN integration in Colosseum.}
	\label{fig:colosseum}
\end{figure*}

\vspace{-0.2cm}
\section{Open Wireless Data Factories}
\label{sec:dataset}

Data-driven approaches aim at autonomously managing the network requiring little to no human intervention.
Training and testing algorithms and data-driven closed-control loop policies require large amounts of data gathered in diverse scenarios, with varying traffic patterns, requirements and user behaviors, so that the resulting policy is
effective when deployed in real networks.

Access to the massive amounts of data needed for training, however, is usually a privilege that only telecom operators enjoy. Owing to privacy and competition concerns, operators seldom share such data openly with the research community. As a consequence, researchers and practitioners are often constrained to rely on datasets collected in small laboratory setups, which seldom capture the variety and scale of real cellular deployments.
In the context of intelligent networking for NextG cellular systems, large-scale wireless testbeds are needed for developing, training and testing new data-driven solutions, serving as \textit{open wireless data factories} for the community.
Such open platforms would facilitate massive data collection in realistic and diverse wireless deployments~\cite{bonati2020open}.

The city-scale platforms of the U.S.\ National Science Foundation \acrshort{pawr} program promise to be a valuable tool to provide the community with the desired diversity of scenarios and scale.
The program is currently supporting three open testbeds representative of a variety of wireless use-cases, ranging from state-of-the-art \glspl{sdr} and massive \gls{mimo} communications (POWDER, in Salt Lake City, UT~\cite{breen2020powder}), to ultra-high capacity
and low-latency wireless networks (COSMOS, in New York City~\cite{raychaudhuri2020cosmos}), and to aerial wireless communications (AERPAW, in the Research Triangle of North Carolina~\cite{sichitiu2020aerpaw}).
All three platforms provide users with
data generation and analysis tools~\cite{bonati2020open}.
Arena is a \gls{sdr} ceiling testbed that allows to study \gls{mimo}, cellular and \gls{iot} applications with up to 64 antennas deployed in an $8\times8$ grid in an office space~\cite{bertizzolo2020arena}.

Another instrument for wireless research at scale
is Colosseum, the world's largest wireless network emulator with hardware-in-the-loop.
Colosseum includes 128 compute nodes, called \glspl{srn}, equipped with \acrshort{usrp}~X310 \glspl{sdr} that can be used to run generic protocol stacks. These are connected in a mesh topology through~128 additional \acrshortpl{usrp}~X310 of the \gls{mchem} for emulating realistic \gls{rf} scenarios.
The wireless channel between each pair of devices is modeled through complex-valued \acrlong{fir} filter taps. In this way, scenarios are able to capture effects such as path loss, multi-path and fading as if the \glspl{sdr} were operating in a real \gls{rf} environment.
Colosseum is also equipped with an edge datacenter, with $900\:\mathrm{TB}$ of storage and the capability of processing \gls{rf} data at a rate of $52\:\mathrm{TB/s}$, enabling massive data collection and testing of \gls{ml} algorithms on heterogeneous networks.

\section{Use Case: Scheduling Control in Sliced 5G Networks through the O-RAN RIC}
\label{sec:usecase}

This section showcases an example of a data-driven closed-loop control implemented using the O-RAN Software Community near real-time \gls{ric} and an open cellular stack on Colosseum (Fig.~\ref{fig:colosseum}).
We demonstrate the feasibility of a closed-control loop where \gls{drl} agents running in xApps on the near real-time \gls{ric} select the best-performing scheduling policy for each \gls{ran} slice.

\textbf{Experimental Scenario}. We have emulated a 5G network with 4 \glspl{bs} and 40 \glspl{ue} (Fig.~\ref{fig:colosseum}, left) in the dense urban scenario of Rome, Italy.
The locations of the \glspl{bs} have been extracted from OpenCelliD (a database of real-world cellular deployments) and cover an area of $0.11\:\mathrm{km^2}$.
Downlink and uplink frequencies have been set to $0.98$ and $1.02\:\mathrm{GHz}$, respectively; the channel bandwidth to $3\:\mathrm{MHz}$.
While these parameters might be atypical for 5G, their choice depends on the Colosseum environment. We note, however, that this does not affect our findings on how data-driven solutions improve the \gls{ran} performance.

We consider a multi-slice scenario in which \glspl{ue} are statically assigned to a slice of the network and request three different traffic types, i.e., high capacity \gls{embb}, \gls{urllc}, and \gls{mtc}.
This reflects the case, for instance, of telecom operators providing different levels of service to different devices (e.g., \gls{mtc} service to \gls{iot}-enabled devices, or \gls{urllc} to devices for time-critical applications).
The \glspl{bs} serve each slice with a dedicated---and possibly different---scheduling policy, selecting among Proportionally Fair (PF), Waterfilling (WF), and Round Robin (RR)~\cite{bonati2020cellos}.
We also consider the case where
the number of \glspl{prb} allocated to each slice varies over time~\cite{doroInfocom2019Slicing, doro2020sledge}.

We used srsLTE to implement our softwarized cellular network.
This open-source framework, which has recently been renamed ``srsRAN'' to reflect a new focus toward~5G NR, provides a full-stack implementation of \glspl{bs} and \glspl{ue}, as well as a lightweight core network.
Although this framework is not yet fully compliant with the NR specifications, we are confident that our \gls{drl}-based approach enabled by O-RAN can be easily extended to future NR-compliant versions of this (or of any other) software where \glspl{bs} expose control interfaces to the network.
For ease of prototyping, we co-located the core network on the same \gls{srn} that also runs the \gls{bs} application. For the purposes of our work, this setup is equivalent to deploying the core network on a dedicated \gls{srn} (see Fig.~\ref{fig:colosseum}).
We extended the \gls{bs} implementation to include network slicing capabilities and additional scheduling policies~\cite{bonati2020cellos}. The scenario we considered concerns pedestrian user mobility
with time-varying path-loss and channel conditions. Traffic among \glspl{bs} and \glspl{ue} is generated through the Colosseum \gls{tgen}, configured to send different traffic types to \glspl{ue} of different slices, i.e., \gls{embb} ($1\:\mathrm{Mbps}$ constant bitrate traffic), \gls{urllc} (Poisson traffic, with $10\:\mathrm{pkt/s}$ of $125\:\mathrm{bytes}$) and \gls{mtc} (Poisson traffic, with $30\:\mathrm{pkt/s}$ of $125\:\mathrm{bytes}$). For each \gls{bs}, the \gls{ue}-slice allocation is as follows: \gls{embb} and \gls{urllc} slices serve 3 \glspl{ue} each, while \gls{mtc} slices serve 4~\glspl{ue}.
We embedded the \gls{drl} agents into xApps running in the near real-time \gls{ric} (right of Fig.~\ref{fig:colosseum}), for a total of 12 \gls{drl} agents running in parallel
and making decisions with a time granularity of~$500\:\mathrm{ms}$.
Agents connect with the network \glspl{bs} through the O-RAN E2 interface. This interface is composed of two elements: The \acrlong{ap}, and the \gls{sm}~\cite{oran-e2}. The former defines the set of messages that the near real-time \gls{ric} and the \gls{ran} nodes can exchange, and the procedures for the \gls{ran} node subscription to the \gls{ric}. The \gls{sm}, instead, defines which parameters of the \gls{ran} nodes can be controlled by the \gls{ric} to achieve a given closed-loop control objective. Specifically, the E2 interface exposes analytics and the scheduler policy selection using a custom \gls{sm}.
As shown in Fig.~\ref{fig:colosseum}, xApps interface with the \glspl{bs}
through the O-RAN E2 manager, which ultimately connects with the \glspl{bs} via the E2 interface. Other components of the \gls{ric} include the \gls{ric} database, which keeps entries on the connected \glspl{bs}, the training engine, and the \gls{ml} model catalog, which deploys the \gls{drl} model chosen by the telecom operator on the near real-time \gls{ric}.
Finally, messages internal to the \gls{ric} are managed by the \acrlong{rmr}, a library which associates message types to destination endpoints.

\textbf{DRL Agent Training}. To train our \gls{drl} agents we generated some $7\:\mathrm{GB}$ of training data of various performance metrics (e.g., throughput and bit error rate), system state information (e.g., transmission queue size, signal-to-interference-plus-noise ratio, and channel quality information) and resource allocation strategies (e.g., slicing and scheduling policies) by running a total of 89 hours of experiments on Colosseum. Each \gls{drl} agent has been trained via the \gls{ppo} algorithm to manage a single slice for fine-grained and flexible control of the whole cellular network.
Agents have been trained under network configurations obtained by varying the distance between \glspl{bs} and \glspl{ue} and the mobility of the \glspl{ue}. Testing has been performed in the most challenging setup, which includes the random mobility of the \glspl{ue}.
Although the training is performed with the same topology configuration, we notice that our agents are topology-independent, as each of them controls a single slice for a given \gls{bs}.
Specifically, agents process the performance metrics received by the \gls{bs} they are controlling---which possibly expresses the performance of several \glspl{ue}---through an encoder. This allows them to cast the dimensionality of the input data to a fixed size and to process it regardless of the number of active \glspl{ue} of the slice.
As a consequence, the \gls{drl} agents do not need to be aware of the number of \glspl{ue} and \glspl{bs} in the network, which makes our approach general and scalable.
Through the \gls{ric} Indication messages sent via the O-RAN E2 interface (Fig.~\ref{fig:colosseum}), the agent is fed real-time performance measurements of the slice it controls. These messages generate an overhead of $72\:\mathrm{bytes/s}$ per \gls{ue}.
Data goes through an encoder for dimensionality reduction and
is then used by the agent to identify the state of the system.
The agent uses a fully connected neural network with 5~layers and 30~neurons each to determine the best scheduling policy for the corresponding slice.
This policy is then signaled to the corresponding \gls{bs} through \gls{ric} Control messages sent via the E2 interface.
The reward of the agents depends on the specific slice and the corresponding \gls{kpi} requirements. Specifically, \gls{embb} and \gls{mtc} agents have been trained to maximize the throughput of \glspl{ue}; the \gls{urllc} agent has been trained to minimize latency by allocating resources (i.e., \glspl{prb}) as quickly as possible.
To comply with O-RAN directives, we have trained the \gls{drl} agents offline
in the non real-time \gls{ric}, which also performs the initial data-collection and deploys the model in the near real-time \gls{ric}.
We have then tested them on the emulated Colosseum scenario.

\textbf{Experimental Results}.
Figure~\ref{fig:spectral} shows the \gls{cdf} of the downlink spectral efficiency of the \gls{embb} slice.
\vspace{-0.1cm}
\begin{figure}[t]
\setlength\belowcaptionskip{-1cm}
	\centering
	\includegraphics[width=\columnwidth]{figures/spectr_eff.pdf}
	\caption{Downlink spectral efficiency of the \acrshort{embb} slice for different scheduling policies and with DRL control.}
	\label{fig:spectral}
\end{figure}
We compare the performance of the network when \gls{drl} agents dynamically select the best scheduling strategy among RR, PF and WF against the case where scheduling strategies are fixed over time. Our results clearly indicate that data-driven optimization outperforms fixed policies by delivering gains in spectral efficiency that are up to 20\% higher than that of the best performing static policy. This is due to the fact that \gls{embb} traffic requires high data-rates and \gls{drl} agents are capable of dynamically adapting scheduling decisions to the current network state and traffic demand.

Figure~\ref{fig:buffer} shows the \gls{cdf}
of the downlink buffer size for the \gls{urllc} slice under different scheduling policies.
\begin{figure}[ht]
\setlength\belowcaptionskip{-0.2cm}
	\centering
	\includegraphics[width=0.85\columnwidth]{figures/buffer.pdf}
	\caption{Downlink buffer size of the URLLC slice for different scheduling policies and with DRL control.}
	\label{fig:buffer}
\end{figure}
Low buffer size indicates timely data delivery to requesting \glspl{ue}; higher buffer size results in higher latency due to packets waiting in the queue.
Results show that \gls{drl} agents serve the \glspl{ue} faster than the static policies, resulting in a lower latency.
Particularly, the average buffer size of the \gls{urllc} slice when using \gls{drl} control is $37\%$, $5\%$ and $17\%$ smaller than that of the RR, WF and PF scheduling policies, respectively. The \gls{drl} agent also significantly outperforms the WF policy between the $50$th and $90$th percentiles.

Figure~\ref{fig:actions} depicts how often \gls{drl} agents select specific scheduling policies as a function of the number of \glspl{prb} of each slice. The bigger the circle, the higher the probability of selecting a given policy. We observe that \gls{mtc} and \gls{embb} \gls{drl} agents select WF with $99\%$ probability.
\begin{figure}[t]
\setlength\belowcaptionskip{-0.7cm}
	\centering
	\includegraphics[width=0.9\columnwidth]{figures/bubbles_perc.pdf}
	\caption{DRL action selection distribution vs.\ number of slice PRBs. Values \mbox{$>99\%$} (big circles) or \mbox{$<0.5\%$} (small circles) are omitted.}
	\label{fig:actions}
\end{figure}
Also, \gls{embb} agents select RR with $4\%$ probability when only a few \glspl{prb} are allocated to the slice.
On the contrary, \gls{urllc} \gls{drl} agents are likely to select both PF and WF scheduling policies even when more \glspl{prb} are available. These results show that adapting control strategies to current network state and traffic requirements is essential to achieve remarkable performance improvements (Fig.~\ref{fig:spectral} and~\ref{fig:buffer}).
\gls{drl} agents dynamically select the best performing scheduling strategy based on available resources and current network state, providing performance gains simply unattainable with static scheduling policies.

\vspace{-0.2cm}
\section{Conclusions}
\label{sec:conclusions}

In this article we provide a path and a demonstration of the feasibility of integrating closed-control loops in cellular networks.
We first review key enablers, namely, virtualization, disaggregation, openness and reprogrammability of NextG cellular networks, using O-RAN as an exemplary technology. We then discuss which data-driven control loops can be implemented, their timescale, and whether the current O-RAN architecture supports them. We finally show how large-scale experimental testbeds can be used to develop and validate data-driven algorithms by deploying a DRL-based O-RAN \gls{ric} on Colosseum. Our results show that using closed-control loops can provide a strong foundation toward the full realization of future generation, data-driven, autonomous, and self-optimizing cellular networks.

\balance
\bibliographystyle{IEEEtran}