\section{Introduction}

In this paper we study the behavior as $\varepsilon\to 0$ of the monotone system
of Hamilton-Jacobi equations
\begin{equation}\label{HJi}
\left\{
  \begin{array}{ll}
\frac{\partial u^\varepsilon_i}{\partial t}+  H_i({x},\frac{x}{\varepsilon},u^\varepsilon, Du^\varepsilon_i )=0
   & (x,t)\in  \mathbb R^N\times (0,T],   \\[5pt]
  u^\varepsilon_{i}(x,0)=u_{0,i}(x)&x\in \mathbb R^N,\ i=1, \cdots M,
  \end{array}
\right.
\end{equation}
where   the Hamiltonians $H_i(x,y,r,p)$,
$i=1,\dots, M$, are periodic in $y,$ coercive in $p$ and
satisfy some uniform continuity properties, see \eqref{H1}.
The $u_{0,i}$'s are bounded uniformly continuous ($BUC$ in short). The
monotonicity condition, see  \eqref{A3},  we assume for
 the system is a standard assumption to obtain  a comparison principle for  \eqref{HJi} (see
\cite{el91}, \cite{ishii92}, \cite{ik91}, \cite{ik91b}).\par The main result
of the paper, see Theorem \ref{HT},  is the convergence of $u^\varepsilon$, as $\varepsilon\to 0$,  to a
$BUC$ function $u=(u_1,\dots,u_M)$ which
solves in viscosity sense the homogenized  system
\begin{eqnarray}\label{sys-homog}
\left\{
  \begin{array}{ll}
\frac{\partial u_i}{\partial t}+  \overline H_i(x,u, Du_i )=0
   & (x,t)\in  \mathbb R^N\times (0,T),\  \\[5pt]
  u_{i}(x,0)=u_{0,i}(x)&x\in \mathbb R^N\ i=1, \cdots M.
  \end{array}
\right.
\end{eqnarray}
The Hamiltonians $\overline H_i$ of the limit problem, the so-called
effective Hamiltonians, are characterized  by appropriate cell
problems. The comparison principle for \eqref{sys-homog} which provides
existence and uniqueness is not an imediate consequence of the comparison
principle for \eqref{HJi} since the regularity properties we could prove for
the effective Hamiltonians are weaker than those for the initial ones
(compare \eqref{H1} and \eqref{mod-hbarre}).\par
Homogenization of Hamilton-Jacobi equations in the framework of viscosity solution theory was firstly considered in the seminal paper by Lions, Papanicolau and Varadhan \cite{lpv86}.
The proof of the our homogenization result relies on an appropriate modification of the classical perturbed test function method. This
technique   was introduced  in the framework of the   viscosity solutions theory by Evans \cite{evans89} for the case of a   periodic equation.
Then it  has been  adapted to  many different homogenization problems,  see e.g. \cite{ab03}, \cite{ci01}, \cite{concordel96}, \cite{ls03}.
For a complete account of the homogenization theory in the periodic case we refer to \cite{ab03}.\par
Concerning the homogenization of  systems of Hamilton-Jacobi
equations we refer to \cite{evans89}, \cite{shimano06}. In these papers, homogenization of weakly coupled systems, i.e. systems with a linear coupling, was considered together with a penalization of the coupling term  of order $\varepsilon^{-1}$.
 Because of the penalization,  the limit problem is a single Hamilton-Jacobi equation and
 all the components of the solution of the perturbed system converge  to the unique solution of this equation.\par
We consider the more general class of monotone systems, which in particular includes the weakly coupled ones. Moreover,   since we do not penalize  the coupling term, the homogenized problem is still a   system
of Hamilton-Jacobi equations and  the perturbed test function method has to be adapted to this situation.\par
In Section \ref{Ex}, we  discuss in more details  the homogenization of   the  weakly coupled systems. In particular   we show  that the homogenized
system is not necessarily weakly coupled but only monotone. For a particular 1-dimensional weakly coupled eikonal system, we  give an explicit
formula for the effective Hamiltonians.\par
The plan of the paper is the
following.\par In Section \ref{prelim} we describe our assumptions and definitions. In Section \ref{system}
we study the system   \eqref{HJi} for $\varepsilon>0$. In Section
\ref{EH}, we define the effective Hamiltonians and we study their
properties. In Section \ref{Hom} we prove the homogenization
result. In Section \ref{Ex} we study some examples and in particular the
weakly coupled systems. Finally in the Appendix  we prove a
comparison theorem for \eqref{HJi}.\\[1mm]
\emph{Notation:}
We will use the following norm
\begin{eqnarray*}
    |f|_\infty=\mathop{\rm ess \,sup}_{x\in\mathbb R^N}|f(x)| \quad
\end{eqnarray*}
and $B_k(x,R)$ denotes the $k$-dimensional ball of center $x\in\mathbb R^N$ and radius $R>0.$

\section{Assumptions and preliminary results}\label{prelim}
We consider the monotone system of Hamilton-Jacobi equations
\begin{equation}\label{HJS}
\left\{
\begin{array}{ll}
\displaystyle{\frac{\partial u^\varepsilon_i}{\partial t}}+  H_i\left({x},\displaystyle{\frac{x}{\varepsilon}}, u^\varepsilon, Du^\varepsilon_i \right)=0
   & (x,t)\in  \mathbb R^N\times (0,T),\ \ i=1, \cdots M, \\[10pt]
  u^\varepsilon_{i}(x,0)=u_{0,i}(x)&x\in \mathbb R^N,
  \end{array}
\right.
\end{equation}
where $u^\varepsilon=(u^\varepsilon_1,\dots,u^\varepsilon_M)$ and $u^\varepsilon_i$ is a real valued function defined in $\mathbb R^N\times [0,T].$
We assume that the Hamiltonians $H_j:\mathbb R^N\times \mathbb R^N\times\mathbb R^M \times\mathbb R^N\to\mathbb R$,
$j=1,\dots,M$, are continuous and
satisfy the following assumptions:
\begin{eqnarray}
\label{H1}\left\{
\begin{array}{l}
\text{ i) $H_j(x,y,r,p)$ is $\mathbb Z^N$-periodic in $y$ for any $(x,r,p)$;}\\[6pt]
\text{ ii) $H_j(x,y,r,p)$ is coercive in $p$, i.e.}\\
\text{  \hspace*{0.5cm}
      $\displaystyle \lim_{|p|\to+\infty}H_j(x,y,r,p)=+\infty$  uniformly in $(x,y,r)$;}\\[8pt]
\text{ iii) For all $R>0,$
$H_j\in BUC(\mathbb R^N\times \mathbb R^N\times [-R,R]^M \times B_N(0,R))$;} \\[6pt]
\text{ iv) there exists a modulus of continuity
$\omega$ s.t}\\[3pt]
 \text{\hspace*{0.5cm}$|H_j(x_1, y_1, r,p)-H_j(x_2,y_2,r,p)|\le\omega((1+|p|)(|x_1-x_2|+|y_1-y_2|)),$ }\\[3pt]
 \text{\hspace*{0.5cm}for every $x_1$, $x_2$,  $y_1$, $y_2$, $p\in\mathbb R^N$ and $r\in \mathbb R^M$.}
\end{array}\right.
\end{eqnarray}
Unless otherwise specified  all the periodic functions we consider have period $\mathbb{T}^N=[0,1]^N.$
We also assume the following monotonicity condition
\begin{equation}\label{A3}
    \begin{array}{c}
   \text{If $r$, $s\in\mathbb R^M$ and $\displaystyle r_j-s_j=\max_{k=1,\dots,M}\{r_k-s_k\}\ge 0$, then }\\[3pt]
   \text{ for all $x$, $y$, $p\in\mathbb R^N$, \ \
   $H_j(x,y,r,p)-H_j(x,y,s,p)\ge 0.$}
    \end{array}
\end{equation}
Concerning the initial datum we assume
\begin{eqnarray}
\text{$u_{0,j}$ is bounded uniformly continuous in $\mathbb R^N$ for $j=1,\dots,M$}.\label{H2}
\end{eqnarray}
\begin{Example} \label{weak-exple} \ \\
1. Consider
\begin{eqnarray}\label{exemple-H}
H_j(x,y,r,p)= a_j(x,y)|p|+F_j(r).
\end{eqnarray}
where $a_j\in C(\mathbb R^N\times \mathbb R^N)$ and $F_j\in C(\mathbb R^M)$. If $a_j$ is
$\mathbb Z^N$-periodic in $y,$ then
\eqref{H1}i) holds. If there exists $\delta >0$ such that $a_j\geq \delta$ then
\eqref{H1}ii) is satisfied. If $a_j$ is bounded with respect to $x$ then
\eqref{H1}iii) holds (note that $a_j$ is bounded with respect to $y$ since
it is periodic). Finally, we have \eqref{H1}iv)
if, for instance, $a_j$ is Lipschitz continuous with respect to $(x,y).$
The assumption \eqref{A3}  is satisfied if $F_j$ is increasing in $r_j$, decreasing in $r_k$ for $k\neq j$.\\
2. A weakly coupled system is a system of the type
\begin{equation}\label{Ex1wk}
\frac{\partial u^\varepsilon_i}{\partial t}+ H_i\left(x,\frac{x}{\varepsilon},Du^\varepsilon_i\right)
+\sum_{j=1}^M c_{ji}\left(x,\frac{x}{\varepsilon} \right)u_j=0,\qquad i=1,\dots, M.
\end{equation}
 Some assumptions
on $c_{ij}$ to ensure \eqref{A3} are given in  Section \ref{Ex}.
Weakly coupled systems arise in optimal control theory of random evolution processes (see \cite{el91}). Moreover
they are associated to large deviation problems for small random perturbation of random evolution processes (see
\cite{cl07a}, \cite{ef90}).
We will study some specific case of weakly coupled systems in Section \ref{Ex}.
\end{Example}

For a function $u:E\to \mathbb R^M$, we say that $u=(u_1,..,u_M)$ is
upper-semicontinuous (u.s.c in short), respectively lower-semicontinuous (l.s.c. in short), in $E$
if all the components $u_i$, $i=1,\dots, M$, are   u.s.c., respectively l.s.c., in $E$.
We define in the same way bounded uniformly continuous ($BUC$) and Lipschitz
continuous functions $u:E\to \mathbb R^M.$
 If $u=(u_1,\dots,u_M)$, $v=(v_1,\dots,v_M)$, are two functions
defined in a set $E$ we write  $u\le v$ in $E$ if $u_i\le v_i$ in $E$ for all $i\in \{1,\dots,M\}$.

We recall the definition of viscosity solution for the  system \eqref{HJS}.
\begin{defn}
\item[i)] An u.s.c. function $u:  \mathbb R^N\times(0,T) \to\mathbb R^M$
is said a viscosity subsolution of \eqref{HJS} if $u_i(\cdot,0)\leq u_{0,i}$ in
$\mathbb R^N$ for all  $i\in \{1,\dots,M\}$ and if
whenever $\phi\in C^1$,
$i\in \{1,\dots,M\}$ and $u_i-\phi$ attains
a local maximum at $(x,t)$ with $t>0$, then
\[\frac{\partial \phi}{\partial t}(x,t)+H_i(x,\frac{x}{\varepsilon},u(x,t),D\phi(x,t)) \le 0.\]
\item[ii)]  A l.s.c. $v:  \mathbb R^N\times(0,T) \to\mathbb R^M$ is said a viscosity  supersolution of
\eqref{HJS} if $v_i(\cdot,0)\geq u_{0,i}$ in
$\mathbb R^N$ for all  $i\in \{1,\dots,M\}$ and if whenever $\phi\in C^1$,
$i\in \{1,\dots,M\}$ and $v_i-\phi$ attains
a local minimum at $(x,t)$ with $t>0$, then
\[\frac{\partial \phi}{\partial t}(x,t)+H_i(x,\frac{x}{\varepsilon},v(x,t),D\phi(x,t)) \ge 0.\]
\item[iii)] A continuous function $u$  is said a viscosity solution of  \eqref{HJS} if it is both
a viscosity sub- and supersolution of \eqref{HJS}.
\end{defn}

\section{The evolutive problem for $\varepsilon>0$}\label{system}
In this section we study the system \eqref{HJS}  for $\varepsilon>0$ fixed.
We first  prove a comparison theorem which applies to prove existence and uniqueness for \eqref{HJS}.
Without loss of generality, we can skip the $y$-dependence in the Hamiltonians
below and we prove a slightly more general result for $H_j=H_j(x,t,u,p)$ which depends also
on $t$ (and is continuous in $\mathbb R^N\times [0,T]\times\mathbb R^M \times\mathbb R^N$).
\begin{prop}\label{PrComp}
Let $u$ be a bounded u.s.c. subsolution and $v$  be a bounded l.s.c. supersolution of
\begin{equation}\label{HJSa}
\left\{
  \begin{array}{ll}
\frac{\partial u_i}{\partial t}+  H_i({x}, t,u, Du_i )=0
   & (x,t)\in  \mathbb R^N\times (0,T), \\[5pt]
  u_{i}(x,0)=u_{0,i}(x)&x\in \mathbb R^N,\ \ i=1, \cdots M,
  \end{array}
\right.
\end{equation}
where $H_i$ satisfies \eqref{H1}-\eqref{A3} and $u_{0}$ satisfies \eqref{H2}.
Then $u\leq v$ in $\mathbb R^N\times [0,T]$
and there exists a unique continuous viscosity solution $u$
of \eqref{HJSa}.
\end{prop}
\begin{proof}
See  the Appendix \ref{Appendix}.
\end{proof}
\begin{prop}\label{regBUC}
Under the assumptions of Proposition \ref{PrComp}, let $u$ be the
unique bounded continuous viscosity solution of \eqref{HJSa}.
Then $u\in BUC(\mathbb R^N\times [0,T]).$
\end{prop}
\begin{proof}
See  the Appendix \ref{Appendix}.
\end{proof}
In the following proposition, we prove
some a-priori bounds, independent of $\varepsilon$, which are used in the  homogenization theorem.
\begin{prop}\label{PrUC}
Assume \eqref{H1} and \eqref{A3} and \eqref{H2}.
For any $\varepsilon>0$ there exists a unique solution $u^\varepsilon\in BUC(\mathbb R^N\times
[0,T])$ of \eqref{HJS}. Moreover
\begin{itemize}
\item[i)] If $u_0$ is bounded Lipschitz continuous, then
$u^\varepsilon\in W^{1,\infty}(\mathbb R^N\times [0,T])$
and $|Du^\varepsilon|_\infty$ can be bounded  independently of $\varepsilon.$
\item[ii)] If $u_0$ is $BUC,$ then
\begin{eqnarray}
&& |u^\varepsilon|_\infty \leq L(H_i, |u_0|_\infty, T)\label{PrUC1},\\
&&  |u^\varepsilon(x,t)-u^\varepsilon(y,s)|\leq \omega_{\rm sp}(|x-y|)+\omega_{\rm tm}(|t-s|)
\quad x,y\in\mathbb R^N,\,  t,s\in [0,T]\label{PrUC2}
\end{eqnarray}
and $ L(H_i, |u_0|_\infty, T), \omega_{\rm sp}, \omega_{\rm tm}$ are
independent of $\varepsilon.$
\end{itemize}
\end{prop}
\begin{proof}
For fixed $\varepsilon >0,$
the existence and uniqueness of the solution $u^\varepsilon\in BUC(\mathbb R^N\times [0,T])$
to \eqref{HJS} follows immediately by Propositions \ref{PrComp} and
\ref{regBUC}. Note that the $L^\infty$ bound for $u^\varepsilon$ does not depend on
$\varepsilon.$ Indeed, replacing $C$ in \eqref{def1C} by
\begin{eqnarray*}
C:=\sup\left\{ \left|H_j\left(x,\frac{y}{\varepsilon},r,0\right)\right|: \, x\in\mathbb R^N,y\in\mathbb R^N,
|r|\le | u_0|_\infty,  \,1\leq j\leq M \right\}
\end{eqnarray*}
which is finite and independent of $\varepsilon$ by periodicity of $H_j$ in $y,$
we obtain \eqref{linfini}  and therefore \eqref{PrUC1}.\par
We now prove \eqref{PrUC2}.
Let $u^\varepsilon$ be a subsolution and $v^\varepsilon$ be a supersolution of \eqref{HJS}
which are $BUC$ (the modulus of continuity of the
solution may a priori depend of $\varepsilon$). Arguing as in Prop. \ref{PrComp}, from \eqref{blabla12}
we have, for all $j$ and $(x,t)\in\mathbb R^N\times [0,T],$
\begin{eqnarray*}
u_j^\varepsilon(x,t)-v_j^\varepsilon(x,t)
&\leq&
\eta t
+ (u_{\overline j}^\varepsilon(\overline x,0)-v_{\overline j}^\varepsilon (\overline y,0))^+ - \frac{|\overline x-\overline y|^2}{2\alpha}\\
&\leq&
\eta T +\mathop{\rm max}_{1\leq j\leq M}\mathop{\rm sup}_{\mathbb R^N}
(u_{j}^\varepsilon(\cdot,0)-v_{j}^\varepsilon(\cdot ,0))^+
+ (v_{\overline j}^\varepsilon (\overline x,0)- v_{\overline j}^\varepsilon (\overline y,0))^+ - \frac{|\overline x-\overline y|^2}{2\alpha}
\end{eqnarray*}
and
\begin{eqnarray*}
\mathop{\rm lim\,sup}_{\alpha\to 0} (v_{\overline j}^\varepsilon (\overline x,0)- v_{\overline j}^\varepsilon (\overline y,0))^+
- \frac{|\overline x-\overline y|^2}{2\alpha}
\leq 0
\end{eqnarray*}
since $v$ is uniformly continuous (see \eqref{unif-cont11} and \eqref{unif-cont22}).
Letting $\alpha\to 0$ and then $\eta \to 0,$ we obtain
\begin{eqnarray}\label{comp-max}
\mathop{\rm max}_{1\leq j\leq M}\mathop{\rm sup}_{\mathbb R^N\times [0,T]} u_j^\varepsilon-v_j^\varepsilon
\leq \mathop{\rm max}_{1\leq j\leq M}\mathop{\rm sup}_{\mathbb R^N}
\left(u_{j}^\varepsilon(\cdot,0)-v_{j}^\varepsilon(\cdot ,0)\right)^+.
\end{eqnarray}

We have to prove that the modulus of continuity of $u^\varepsilon$ do not
depend on $\varepsilon.$ We proceed by approximation showing first the
result for $u_0$ Lipschitz continuous. Replacing $C$ in \eqref{def1C} by
\begin{eqnarray*}
C:=\sup\left\{ \left|H_j\left(x,\frac{y}{\varepsilon},r,p\right)\right|: \, x\in\mathbb R^N,\,y\in\mathbb R^N,\,
|r|\le | u_0|_\infty,\,  |p|\leq |Du_0|_\infty, \,1\leq j\leq M \right\},
\end{eqnarray*}
we prove as at the beginning of the proof of Proposition \ref{regBUC}
that, if $v^\pm (x,t)=( u_{0,1}(x) \pm Ct,\cdots , u_{0,M}(x) \pm Ct),$
then $v^+$ is a supersolution and $v^-$ is a subsolution of \eqref{HJS}.
By Proposition \ref{PrComp}, it follows
\begin{eqnarray}\label{comp999}
v^-\leq u \leq v^+ \quad {\rm in} \ \mathbb R^N\times [0,T].
\end{eqnarray}
Let $0\leq h\leq T$ and
note that, since the $H_i$'s are independent of $t,$
 $u^\varepsilon (\cdot ,\cdot +h)$ is still a solution of \eqref{HJS}
with initial data $u^\varepsilon (\cdot ,h).$
By \eqref{comp-max} and \eqref{comp999}, we get that, for all $j,$ $(x,t)\in \mathbb R^N\times [0,T],$
\begin{eqnarray*}
 u_j^\varepsilon(x,t+h)-u_j^\varepsilon(x,t)
\leq \mathop{\rm max}_{1\leq j\leq M}\mathop{\rm sup}_{\mathbb R^N}
\left(u_{j}^\varepsilon(\cdot,h)-u_{0,j}\right)^+
\leq Ch
\end{eqnarray*}
and therefore $u_j^\varepsilon$ is Lipschitz with respect to $t$ for every $x$ with
\begin{eqnarray*}
\left|\frac{\partial u_j^\varepsilon}{\partial t}\right|_\infty \leq C.
\end{eqnarray*}
From   \eqref{HJS}, we obtain, in the viscosity sense
\begin{eqnarray*}
-C\leq H_j(x,\frac{x}{\varepsilon},u^\varepsilon, Du_j^\varepsilon ) \leq C \qquad(x,t)\in\mathbb R^N\times [0,T].
\end{eqnarray*}
By the coercivity of $H_i$ (uniformly with respect to the other variables, see
\eqref{H1}ii)),
there exists $L_j>0$ such that, for all $p\in\mathbb R^N,$
\begin{eqnarray*}
|p|\geq L_j \ \ \Longrightarrow \ \ {\rm for \ all} \ \varepsilon >0, x\in\mathbb R^N,  r\in\mathbb R^M, \
 H_j(x,\frac{x}{\varepsilon},r, p)>C.
\end{eqnarray*}
It follows that $u_j^\varepsilon$ is Lipschitz continuous in $x$ for every $t$ with
$|Du_j^\varepsilon|_\infty \leq L_j$ (with $L_j$ independent of $\varepsilon$).

Now if $u_0\in BUC(\mathbb R^N),$ then it is possible
to approach it by Lipschitz continuous functions: for all $\gamma>0,$ there
exists $u_0^\gamma$ such that $|u_0-u_0^\gamma|_\infty \leq \gamma.$
Let $u^\varepsilon$ (respectively $u^{\varepsilon,\gamma}$) be the unique $BUC$
(respectively Lipschitz continuous with constant $C_\gamma$) solution of \eqref{HJS} with initial data
$u_0$ (respectively $u_0^\gamma$). Note that $C_\gamma$ is independent of
$\varepsilon.$ By \eqref{comp-max}, we obtain
\begin{eqnarray*}
|u^\varepsilon - u^{\varepsilon,\gamma}|_\infty \leq |u_0-u_0^\gamma|_\infty \leq \gamma.
\end{eqnarray*}
It follows that, for all $1\leq j\leq M,$ $x,y\in \mathbb R^N,$ $t,s\in [0,T],$
\begin{eqnarray}\label{est634}\hspace*{1cm}
|u_j^\varepsilon (x,t) - u_j^{\varepsilon}(y,s)| \leq
|u_j^{\varepsilon,\gamma} (x,t) - u_j^{\varepsilon,\gamma}(y,s)|+2\gamma
\leq C_\gamma (|x-y|+|t-s|)+2\gamma.
\end{eqnarray}
Since \eqref{est634} holds for all $\gamma >0$ and $C_\gamma$ is independent
of $\varepsilon,$ we conclude that $u^\varepsilon$ is $BUC$ with
a modulus independent of $\varepsilon.$
\end{proof}.

\section{The cell problem}\label{EH}
In this section we prove the existence of the effective Hamiltonians, the
Hamiltonians for the limit system \eqref{sys-homog}.
Since at this level we work for a fixed index $i$, i.e.   there is no coupling,
we can follow the classical argument based on the ergodic approximation of the cell problem.
The only point is to prove that effective Hamiltonians we are going to define still verify some regularity and monotonicity properties
 so that the homogenized problem verifies a comparison principle.\\

\noindent\textbf{The Cell problem.}
For any $i=1,\dots,M$, given $(x,r,p)\in\mathbb R^N\times\mathbb R^M\times\mathbb R^N$, find $\lambda_i=\lambda_i(x,r,p)$ such that
the equation
\begin{equation}\label{CP}
    H_i(x,y,r,p+Dv(y))=\lambda_i\qquad y\in\mathbb{T}^N
\end{equation}
admits a   viscosity solution $v_i= v_{i,x,r,p}$.
\begin{prop}\label{PropCP}
Assume \eqref{H1}. For any $i=1,\dots,M$, there exists a unique $\lambda_i=\lambda_i(x,r,p)\in\mathbb R$
such that  the cell problem \eqref{CP} admits
a  periodic solution $v_i(y)=v_i(y;x,r,p)$ which is Lipschitz continuous.
More precisely, for all $R>0,$ there exists $L_R>0$ such that
 \begin{eqnarray*}
{\rm sup} \{ |D_y v_{i}(y;x,r,p)|_\infty \, : \, x\in\mathbb R^N, |r|+|p|\leq R \}\leq L_R.
\end{eqnarray*}
\end{prop}
\begin{proof}
We only give a sketch of the proof, see for instance \cite{concordel96} for details.
Fix $i\in\{1,\dots,M\},$ $R>0$ and $(x,r,p)\in \mathbb R^N\times\mathbb R^M\times\mathbb R^N$
such that $|r|+|p|\leq R.$
Consider the ergodic approximation of the cell problem
\begin{equation}\label{EA}
   \alpha w^\alpha_i(y)+H_i(x,y,r, p+Dw^\alpha_i(y))=0\qquad y\in \mathbb{T}^N.
\end{equation}
By \eqref{H1}, \eqref{EA} satisfies a comparison principle for any $\alpha>0$ and therefore it admits a unique
continuous viscosity solution $w^\alpha_i$ which is periodic. By \eqref{H1}iii),
 \begin{eqnarray*}
C_R:= {\rm sup} \{ |H_j(x,y,r,p)|\, :\,  x,y\in\mathbb R^N, |r|+|p|\leq R,
j=1,\cdots, M\}< +\infty
\end{eqnarray*}
and
$-C_R/\alpha$ is a subsolution  and  $C_R/\alpha$ is a supersolution of  \eqref{EA}.
It follows
 \begin{eqnarray*}
-C_R\leq \alpha w^\alpha_i\leq C_R.
\end{eqnarray*}
By the coercitivity of the Hamiltonian $H_i,$ there exists $L_R=L(R,H_i)$ such that
\begin{eqnarray*}
|p+Dw^\alpha_i|> L_R \ \ \Longrightarrow \ \ H_i(x,y,r,p+Dw^\alpha_i)>C_R.
\end{eqnarray*}
We then get the global gradient bounds for $w^\alpha_i$ independent of $\alpha:$
\begin{eqnarray*}
|Dw^\alpha_i|_\infty \leq L_R.
\end{eqnarray*}
It follows that, for a fixed $y_0\in\mathbb{T}^N$, there exists a sequence  $\alpha_n\to 0$ such that
\begin{align*}
&\lim_{n\to\infty} {\alpha_n} w^{\alpha_n}_i(y)=\lambda_i\qquad &\text{for any $y\in\mathbb{T}^N,$}\\
&\lim_{n\to\infty} w^{\alpha_n}_i(y)-w^{\alpha_n}_i(y_0)=v_i(y) &\text{uniformly in $\mathbb{T}^N$.}
\end{align*}
Moreover $v_i$ is Lipschitz continuous with constant $L_R$
and by standard stability result in viscosity solution theory $(\lambda_i, v_i)$ is a solution to \eqref{CP}.
Finally it is possible to prove that the number  $\lambda_i$ for which
\eqref{CP} admits a solution is univocally defined, while it is well known that
in general the viscosity solution of \eqref{CP} is not unique.
\end{proof}

\begin{defn}
For any $i=1,\dots,M$, the effective Hamiltonian $\overline H_i(x,r,p)$ associated to the Hamiltonian $H_i$ is defined
by setting
\[\overline H_i(x,r,p)=\lambda_i\]
where $\lambda_i$ is given by Proposition \ref{PropCP}.
\end{defn}
We now deduce some  properties of the Effective Hamiltonians
\begin{prop} \label{prop-hbarre}
Assume \eqref{H1}.
For any $i=1,\dots,M$, the effective Hamiltonian $\overline H_i$ satisfies
\begin{itemize}
\item[i)] $\overline H_i$ is  continuous  in $(x,r,p)$ and, for all $R>0,$
there exists a modulus of continuity $\omega_R$ such that,
for all $x,x'\in \mathbb R^N,$ $r, r'\in \mathbb R^M,$ $p,p'\in \mathbb R^N$ with $|r|+|r'|+|p|+|p'|\leq R,$ we have
\begin{eqnarray}\label{mod-hbarre}
|\overline H_i (x,r,p)-\overline H_i (x',r',p')|\leq \omega_R(|x-x'|) + \omega_R(|r-r'|+|p-p'|).
\end{eqnarray}
\item[ii)] $\overline H_i$ is coercive in $p$ (uniformly with respect to $(x,r)$).
\item[iii)] If $H_i$ is convex in $p$, then $\overline H_i$ is convex in $p$.
\item[iv)]  If $H_i$ satisfies \eqref{A3}, then
 $\overline H_i$ satisfies \eqref{A3}.
\end{itemize}
\end{prop}
\begin{proof}
We first prove i). Let $R>0$ and $(x,r,p), (\alpha,s,q)\in\mathbb R^N\times\mathbb R^M\times \mathbb R^N$
such that $|r|+|s|+|p|+|q|\leq R.$
Let $v$, $w$ two periodic functions such that
\begin{eqnarray}
    &H_i(x,y,r,p+Dv(y))=\overline H_i(x,r,p)\qquad y\in \mathbb{T}^N, \label{eq111}\\
     &H_i(x+\alpha,y,r+s,p+q+Dw(y))=\overline H_i(x+\alpha,r+s,p+q)\qquad y\in \mathbb{T}^N. \label{eq222}
\end{eqnarray}
By periodicity of $v$ and $w,$ for any $\varepsilon >0,$ the supremum
\begin{eqnarray*}
\mathop{\rm sup}_{z,y\in\mathbb R^N} \{v(z)-w(y)-\frac{|z-y|^2}{\varepsilon^2}\}
\end{eqnarray*}
is achieved at some point $(\bar{z},\bar{y})$
(which depends on $x,\alpha,p,q,r,s,\varepsilon$).
Moreover it is easy to see that, since $v,w$ are bounded Lipschitz continuous,
we have
\begin{eqnarray}\label{sup-lip}
\frac{|\bar{z}-\bar{y}|}{\varepsilon^2}\leq |Dv|_\infty, |Dw|_\infty\leq L_R
 \end{eqnarray}
where $L_R$ is given by Proposition \ref{PropCP}.
Since  $v$ is a viscosity subsolution of \eqref{eq111} and $w$ is a supersolution
of \eqref{eq222}, we obtain
\begin{eqnarray*}
&H_i(x,\bar{z},r,p+\bar{p})\leq \overline H_i(x,r,p), \\
&H_i(x+\alpha,\bar{y},r+s,p+q+\bar{p})\geq \overline H_i(x+\alpha,r+s,p+q).
\end{eqnarray*}
It follows, using \eqref{H1}, that
\begin{eqnarray*}
&& \overline H_i(x+\alpha,r+s,p+q) - \overline H_i(x,r,p)\\
&\leq& H_i(x+\alpha,\bar{y},r+s,p+q+\bar{p})-H_i(x,\bar{z},r,p+\bar{p}) \\
&=& H_i(x+\alpha,\bar{y},r+s,p+q+\bar{p})-  H_i(x,\bar{z},r+s,p+q+\bar{p})\\
&& + H_i(x,\bar{z},r+s,p+q+\bar{p}) - H_i(x,\bar{z},r,p+\bar{p})\\
&\leq & \omega \big((1+R+L_R) (|\alpha|+|\bar{z}-\bar{y}|)\big)
+ \tilde \omega_{R} (|s|+|q|),
\end{eqnarray*}
where $\omega$ is given by \eqref{H1}iv) and $\tilde \omega_{R}$ is a modulus of
continuity of the continuous function $H_i$
on the subset $\mathbb R^N\times \mathbb{T}^N\times [-R,R]^M\times B_N(0,R+L_R)$ given by \eqref{H1}iii).
Sending $\varepsilon$ to 0 and setting $\omega_R(l)={\rm max}\{ \omega
((1+R+L_R)l), \tilde \omega_R (l)\},$
we get
\begin{eqnarray*}
 \overline H_i(x+\alpha,r+s,p+q) - \overline H_i(x,r,p)\leq \omega_R(|\alpha|)+ \omega_{R} (|s|+|q|),
\end{eqnarray*}
 which ends the proof of i).
The proof of \textit{ii)} and \textit{iii)} are standard, see \cite{concordel96}.

We now prove that
$\overline H_i$, $i=1,\dots, M$, satisfies the monotonicity condition \eqref{A3}. We assume by contradiction that
there exist  $r$, $s\in\mathbb R^M$ such that
$\displaystyle r_j-s_j=\mathop{\max}_{k=1,\dots,M}\{r_k-s_k\}\ge 0$ and
\begin{eqnarray*}
\overline H_j(x, r,p)<\overline H_j(x ,s,p)
\end{eqnarray*}
for some $x$, $p\in\mathbb R^N$. Let $v$, $w$  be two periodic  functions such that
\begin{align*}
    &H_j (x,y,r ,p+Dv)=\overline H_j(x,r,p)\qquad y\in \mathbb{T}^N,\\
     &H_j(x,y,s,p+Dw)=\overline H_j(x,s,p)\qquad y\in \mathbb{T}^N.
\end{align*}
Since $v$, $w$ are bounded, by adding a constant we can assume  w.l.o.g. that $v>w$ in $\mathbb{T}^N$.
By \eqref{A3}
\begin{eqnarray*}
H_j(x,y,r,p+Dv)=\overline H_j(x,r ,p)<\overline H_j(x,s,p)
\le  H_j(x,y,s,p+Dw)\le H_j(x,y,r,p+Dw)
\end{eqnarray*}
and   for $\alpha$ sufficiently small
\[
\alpha v+H_j(x,y,r,p+Dv)>\alpha w+ H_j(x,y,r,p+Dw)\qquad y\in\mathbb{T}^N.
\]
This last inequality gives a  contradiction by the comparison principle for
\eqref{EA}.
\end{proof}

\section{The Homogenization theorem}\label{Hom}
In this section we prove the Homogenization theorem for the problem \eqref{HJi}.

\begin{prop} \label{comp-barre}
Assume \eqref{H1}, \eqref{A3} and \eqref{H2}.
Then there exists a unique solution $u\in BUC(\mathbb R^N\times[0,T])$
of
\begin{equation}\label{HJ0}
\left\{
  \begin{array}{ll}
  \displaystyle{\frac{\partial u_i }{\partial t}}+  \overline H_i(x,u , Du_i  )=0& (x,t)\in \mathbb R^N\times (0,\infty),
  \\[10pt]
  u_i (x,0)=u_{0,i} (x)&x\in \mathbb R^N, \quad i=1,\dots,M.
  \end{array}
\right.
\end{equation}
\end{prop}
\begin{proof}
The difficulty here is that the comparison principle for the limit system
\eqref{comp-barre} is not a straightforward consequence of Proposition
\ref{prop-hbarre}. Indeed, the regularity of  the Hamiltonians $\overline H_i$ is weaker
than \eqref{H1} (in particular compare \eqref{H1}iv) and \eqref{mod-hbarre}).
To prove the comparison principle we first prove comparison in the case
where either the subsolution or the supersolution is bounded Lipschitz continuous
and then we proceed by approximation.\par
Suppose that $u_0$ is bounded Lipschitz continuous, that $u$ is a bounded subsolution
and $v$ a bounded supersolution  of \eqref{HJ0} and that $u,$ for instance,
is Lipschitz continuous (with constant $L$).
Arguing as in Proposition \ref{PrComp} and looking carefully at the proof of
Proposition \ref{prop-hbarre} (see in particular \eqref{sup-lip}),
it follows that the second estimate in \eqref{mie21} could be replaced by
\begin{eqnarray*}
|\overline p|\leq L,
\end{eqnarray*}
and therefore, setting $R=|v|_\infty + 2M+L,$ from \eqref{mod-hbarre},
\eqref{T111} becomes
\begin{eqnarray*}
\mathcal{T}_1\leq \omega_R (L\alpha).
\end{eqnarray*}
The term $\mathcal{T}_2$ does not exist since $\overline H_{\overline j}$ does not depend
on $t.$ We deal with $\mathcal{T}_3$ using again \eqref{mod-hbarre} and
$\mathcal{T}_4\leq 0$ as in the proof of  Proposition \ref{prop-hbarre}.
The rest of the proof is the same (even easier since $u(\cdot ,0)$ is
Lipschitz continuous). It follows that we have comparison between Lispchitz
continuous sub and supersolutions. In particular, by Perron's method,
for any Lipschitz continuous $u_0,$ there exists a unique Lipschitz
continuous solution $u$ of \eqref{HJ0}. Moreover, repeating the beginning
of the proof of Proposition \ref{PrUC}, we obtain that \eqref{comp-max}
holds, i.e.
\begin{eqnarray}\label{val-ab}
\mathop{\rm max}_{1\leq j\leq M}\mathop{\rm sup}_{\mathbb R^N\times [0,T]} u_j-v_j
\leq \mathop{\rm max}_{1\leq j\leq M}\mathop{\rm sup}_{\mathbb R^N}
\left(u_{j}(\cdot,0)-v_{j}(\cdot ,0)\right)^+
\end{eqnarray}
if $u$ is a subsolution and $v$ a supersolution of \eqref{HJ0} and
either $u$ or $v$ is Lipschitz continuous.

Now, consider the case when $u_0$ is $BUC.$
Let $u$ (respectively $v$) be a $BUC$
subsolution (respectively supersolution) of \eqref{HJ0}.
For all $\gamma>0,$ there exists a Lipschitz continuous function $u_0^\gamma$
such that
\begin{eqnarray}\label{approx}
u_0^\gamma \leq u_0 \leq u_0^\gamma+\gamma \quad {\rm in} \ \mathbb R^N.
\end{eqnarray}
Let $u^\gamma$ (respectively $v^\gamma$) be the Lipschitz continuous solution to \eqref{HJ0} with
initial data $u_0^\gamma$ (respectively $u_0^\gamma+\gamma$).
By comparison in the Lipschitz case, $u^\gamma\leq v^\gamma.$
From \eqref{val-ab} and \eqref{approx}, it follows
\begin{eqnarray*}
u\leq u^\gamma +\gamma \leq v^\gamma +\gamma\leq v+2\gamma \quad {\rm in} \
\mathbb R^N\times [0,T].
\end{eqnarray*}
Since the previous inequality is true for all $\gamma>0,$
we obtain the desired comparison $u\leq v.$ Note that we obtain the existence
and the uniqueness of a $BUC$ solution as a
byproduct of this latter proof.
\end{proof}

\begin{thm}
\label{HT} Assume \eqref{H1},  \eqref{A3} and \eqref{H2}.
 The viscosity solution $u^\varepsilon$ of \eqref{HJi} converges locally  uniformly
on $\mathbb R^N\times [0,T]$ to the viscosity solution $u\in BUC(\mathbb R^N\times [0,T])$ of \eqref{HJ0}.
\end{thm}

\begin{proof}
By Proposition \ref{PrUC}  there exists a
continuous solution $u^\varepsilon$ of \eqref{HJi}
which is bounded independently of $\varepsilon.$ It follows that we can define the half-relaxed
limits
\begin{eqnarray*}
\overline u (x,t) = \mathop{\rm lim\,sup}_{\varepsilon\to 0, (x_\varepsilon, t_\varepsilon)\to(x,t)} u^\varepsilon(x_\varepsilon, t_\varepsilon)
\ \ \ {\rm and} \ \ \
\underline u (x,t) = \mathop{\rm lim\,inf}_{\varepsilon\to 0, (x_\varepsilon, t_\varepsilon)\to(x,t)} u^\varepsilon(x_\varepsilon, t_\varepsilon).
\end{eqnarray*}
Let us mention at this step that we could use Ascoli's theorem in view of the
equicontinuity property  of Proposition \ref{PrUC} ii) to obtain a limit for
$u^\varepsilon$ along a subsequence. We choose to use the
half-relaxed limits since it is not much more complicated and it does not require
uniform moduli of continuity for the $u^\varepsilon$'s.

We first show that $\overline u$ is a viscosity subsolution of the system \eqref{HJ0}.
We assume that there exist  $j\in\{1,\dots,M\}$ and $\phi\in C^1$ such that
$\overline{u}_j-\phi$ has a strict maximum point at some $(\overline x,\overline t)$ with
$\overline t>0$ and
$\overline u_j(\overline x,\overline t)=\phi(\overline x,\overline t).$
We assume w.l.o.g. that $j=1$. By Proposition \ref{PropCP}, there exists a
corrector $v$ for $(\overline x, \overline u(\overline x,\overline t), D\phi(\overline x,\overline t))$, i.e. a viscosity solution of
\begin{equation}\label{T2-new}
    H_1(\overline x,y,\overline u(\overline x,\overline t),D\phi(\overline x,\overline t)+Dv(y))=\overline H_1(\overline x,\overline u(\overline x,\overline t),D\phi(\overline x,\overline t))\qquad y\in\mathbb{T}^N.
\end{equation}
Define the ``perturbed test-function''
\begin{equation}\label{T3-new}
   \phi^{\varepsilon,\alpha}(x,y,t)=\phi(x,t)+\varepsilon v\left(\frac{y}{\varepsilon}\right)+\frac{|x-y|^2}{\alpha^2}.
\end{equation}
By classical results on viscosity solutions (see \cite[Lemma 4.3]{barles94} or
\cite{bcd97}), we have, since $\overline{u}_1-\phi$ has a strict maximum point at $(\overline x,\overline t),$
up to extract subsequences, there exist
$(x_{\varepsilon,\alpha}, y_{\varepsilon,\alpha}, t_{\varepsilon,\alpha})\in\mathbb R^N\times\mathbb R^N\times (0,T]$
and $(x_{\varepsilon}, t_{\varepsilon})\in\mathbb R^N\times (0,T]$ such that
$(x_{\varepsilon,\alpha}, y_{\varepsilon,\alpha}, t_{\varepsilon,\alpha})$ is a local maximum
of ${u}^\varepsilon_1(x,t)-\phi^{\varepsilon,\alpha}(x,y,t)$ and
\begin{eqnarray*}
&& (x_{\varepsilon,\alpha}, y_{\varepsilon,\alpha}, t_{\varepsilon,\alpha}) \to (x_{\varepsilon}, x_{\varepsilon},t_{\varepsilon})
\ \ \ {\rm as} \ \alpha\to 0, \\
&& (x_{\varepsilon}, t_{\varepsilon}) \to (\overline x,\overline t)\ \ \ {\rm as} \ \varepsilon\to 0, \\
&& \mathop{\rm lim}_{\varepsilon\to 0}\,  \mathop{\rm lim}_{\alpha\to 0}
{u}^\varepsilon_1(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})= u_1 (\overline x, \overline t).
\end{eqnarray*}
Since $u^\varepsilon_1(x,t)- \phi^{\varepsilon,\alpha}(x,y_{\varepsilon,\alpha},t)$ has a maximum point at
$(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})$ and $u^\varepsilon$ is a subsolution of \eqref{HJi},
setting $\displaystyle p_{\varepsilon,\alpha}=2\frac{x_{\varepsilon,\alpha}-y_{\varepsilon,\alpha}}{\alpha^2},$
we get
\begin{equation}\label{visc66bis}
\phi_t(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})
+H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon},{u}^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),
D\phi(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})+ p_{\varepsilon,\alpha})
\leq 0.
\end{equation}
Since  $v$ is a
supersolution of \eqref{T2-new} and $y\mapsto v(y)-\psi^{\varepsilon,\alpha}(\varepsilon y)$
 has a minimum point at $y_{\varepsilon,\alpha}/\varepsilon$ with
\begin{eqnarray*}
\psi^{\varepsilon,\alpha}(y)=-\frac{1}{\varepsilon}\left(\frac{|x_{\varepsilon,\alpha}-y|^2}{\alpha^2}+
\phi(x_{\varepsilon,\alpha}, t_{\varepsilon,\alpha})-u^\varepsilon_1(x_{\varepsilon,\alpha}, t_{\varepsilon,\alpha})\right),
\end{eqnarray*}
we get
\begin{equation}
H_1(\overline x,\frac{y_{\varepsilon,\alpha}}{\varepsilon},\overline u(\overline x,\overline t),D\phi(\overline x,\overline t)
+p_{\varepsilon,\alpha})
\geq  \overline H_1(\overline x,\overline u(\overline x,\overline t),D\phi(\overline x,\overline t)). \label{visc66}
\end{equation}
Note that the corrector $v$ is Lipschitz continuous by coercivity of $H_1$
(see Proposition \ref{PropCP}). Therefore,
from \eqref{sup-lip}, we have
\begin{eqnarray}\label{bornitude-p}
 |p_{\varepsilon,\alpha}|\leq |Dv|_\infty \leq L_R \ \ \ {\rm with} \ R=|D\phi(\overline x,\overline t)|+|\overline u|_\infty.
\end{eqnarray}

By \eqref{visc66bis} and \eqref{visc66}, we have
\begin{eqnarray} \label{eg876}
&&\phi_t (\overline x,\overline t)+\overline H_1(\overline x, \overline u(\overline x,\overline t),D\phi(\overline x,\overline t))\\
&\leq & \phi_t (\overline x,\overline t)-\phi_t (x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})\nonumber\\
& & + H_1(\overline x,\frac{y_{\varepsilon,\alpha}}{\varepsilon},\overline u(\overline x,\overline t),
D\phi(\overline x,\overline t)+p_{\varepsilon,\alpha})\nonumber\\
&& - H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon}, {u}^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),
D\phi(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})+ p_{\varepsilon,\alpha}).\nonumber
\end{eqnarray}

Since $(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})\to (\overline x,\overline t)$ and $\phi$ is smooth we get
\begin{equation}\label{esT1}
\mathop{\rm lim}_{\varepsilon\to 0}\,  \mathop{\rm lim}_{\alpha\to 0}
\phi_t (\overline x,\overline t)-\phi_t (x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})=0.
\end{equation}
To estimate the second term of the right-hand side we set
\begin{eqnarray*}
\mathcal{T}_1&=&H_1(\overline x,\frac{y_{\varepsilon,\alpha}}{\varepsilon},\overline u(\overline x,\overline t),D\phi(\overline x,\overline t)+ p_{\varepsilon,\alpha})
- H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon},\overline u(\overline x,\overline t),D\phi(\overline x,\overline t)+ p_{\varepsilon,\alpha}), \\
\mathcal{T}_2&=& H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon},\overline u(\overline x,\overline t),D\phi(\overline x,\overline t)+ p_{\varepsilon,\alpha})
-H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon}, {u}^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),
D\phi(\overline x ,\overline t)+ p_{\varepsilon,\alpha}),\\
\mathcal{T}_3&=&H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon}, {u}^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),
D\phi(\overline x ,\overline t)+ p_{\varepsilon,\alpha})\\
&& -H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon}, {u}^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),
D\phi(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})+ p_{\varepsilon,\alpha}).
 \end{eqnarray*}
From \eqref{H1}iv) and \eqref{bornitude-p},
\begin{eqnarray*}
\mathcal{T}_1 &\leq& \omega \big( (1+|D\phi(\overline x,\overline t)+ p_{\varepsilon,\alpha}|)
(|\overline x -x_{\varepsilon,\alpha}|+\frac{|x_{\varepsilon,\alpha}-y_{\varepsilon,\alpha}|}{\varepsilon})\big)\\
&\leq& \omega \big( (1+R+L_R)(|\overline x -x_{\varepsilon,\alpha}|+\frac{L_R\alpha^2}{\varepsilon})\big)
\end{eqnarray*}
and therefore
\begin{eqnarray*}
 \mathop{\rm lim}_{\varepsilon\to 0}\,  \mathop{\rm lim}_{\alpha\to 0}\mathcal{T}_1 =0.
\end{eqnarray*}

To deal with $\mathcal{T}_2,$ we use the monotonicity assumption \eqref{A3}.
Let $\delta >0.$
At first, up to extract some subsequences, by definition of $\overline u,$
we can assume that for $\alpha, \varepsilon$
small enough with $\alpha <\!\! < \varepsilon,$ we have
\begin{eqnarray*}
u_j^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha})- \overline u_j(\overline x,\overline t)\leq \frac{\delta}{2}
\ \ \ {\rm for} \ 2\leq j\leq M.
\end{eqnarray*}
It follows that
\begin{eqnarray*}
{\rm max}\big\{  \overline u_1(\overline x,\overline t)+\delta- \overline u_1(\overline x,\overline t),
u_2^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}) -\overline u_2(\overline x,\overline t),\cdots,
u_M^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}) -\overline u_M(\overline x,\overline t)
\big\}=\delta
\end{eqnarray*}
is achieved for the first component. Set $r_\delta=(\overline u_1(\overline x,\overline t)+\delta,
u_2^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),\cdots,
u_M^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}))$, then by \eqref{A3}
\begin{eqnarray*}
H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon},\overline u(\overline x,\overline t),
D\phi(\overline x,\overline t)+ p_{\varepsilon,\alpha})-H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon}, r_\delta,
D\phi(\overline x ,\overline t)+ p_{\varepsilon,\alpha})
\leq 0.
\end{eqnarray*}
Then
\begin{eqnarray*}
\mathcal{T}_2\leq
H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon},r_\delta,D\phi(\overline x,\overline t)+p_{\varepsilon,\alpha})
-H_1(x_{\varepsilon,\alpha},\frac{x_{\varepsilon,\alpha}}{\varepsilon}, {u}^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),
D\phi(\overline x ,\overline t)+ p_{\varepsilon,\alpha}):= \mathcal{T}_4.
\end{eqnarray*}

To prove that
\begin{eqnarray} \label{limlim}
 \mathop{\rm lim}_{\varepsilon\to 0}\,  \mathop{\rm lim}_{\alpha\to 0}\mathcal{T}_3 = 0
\ \ \ {\rm and} \ \ \
\mathop{\rm lim}_{\delta\to 0}\,\mathop{\rm lim}_{\varepsilon\to 0}\,  \mathop{\rm lim}_{\alpha\to 0}
 \mathcal{T}_4 = 0,
\end{eqnarray}
we use the uniform continuity of $H_1$ on compact subsets.
We have
\begin{eqnarray*}
\mathop{\rm lim}_{\varepsilon\to 0}\,  \mathop{\rm lim}_{\alpha\to 0}\,
(x_{\varepsilon,\alpha}, {u}^\varepsilon(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}),D\phi(x_{\varepsilon,\alpha},t_{\varepsilon,\alpha}))
= (\overline x, \overline u(\overline x,\overline t), D\phi(\overline x ,\overline t))
\end{eqnarray*}
and
\begin{eqnarray*}
\mathop{\rm lim}_{\delta\to 0}\,\mathop{\rm lim}_{\varepsilon\to 0}\,  \mathop{\rm lim}_{\alpha\to 0}\,
r_\delta = \overline u(\overline x,\overline t).
\end{eqnarray*}
Since $x_{\varepsilon,\alpha}, y_{\varepsilon,\alpha}\to \overline x$ we have
that $x_{\varepsilon,\alpha}, y_{\varepsilon,\alpha}$ stay in some ball $\overline B(\overline x,\overline R)$.
Hence choosing
$K=\overline B(\overline x,\overline R)\times \mathbb{T}^N\times
[-|\overline u|_\infty-1, |\overline u|_\infty+1]^M\times \overline B(0, |D\phi(\overline x,\overline t)|+L_R+1),$
by uniform continuity of $H_1$ on $K,$ \eqref{limlim} holds. Note that
the periodicity of $H_1$ allows
to deal with ${x_{\varepsilon,\alpha}}/{\varepsilon}$ which is not bounded.

Finally, sending $\alpha\to 0$ at first, then $\varepsilon\to 0$ and finally
$\delta\to 0,$ we conclude that the right-hand side of \eqref{eg876}
is nonpositive which proves that $\overline u$ is a subsolution of
\eqref{HJ0}.

We prove that $\underline u$ is a viscosity supersolution of \eqref{HJ0} in a
similar way. From Proposition \ref{PrComp}, we then obtain that $\overline u\leq
\underline u$ in $\mathbb R^N\times [0,T].$ It follows that $\overline u=\underline u:=u$
where $u$ is the (local) uniform limit of the $u^\varepsilon$'s.
\end{proof}

\begin{rem}\label{corr-non-lip} \ \\
1. As mentioned above, the coercivity of the Hamiltonians plays a crucial role: it ensures
the Lipschitz continuity of the correctors which allows us to deal with
$\mathcal{T}_3$ and $\mathcal{T}_4$ with weak regularity assumptions with respect to $(r,p)$
in \eqref{H1}.
When the Hamiltonians are not coercive anymore (and therefore the corrector
is not necessarily Lipschitz continuous), the proof is more delicate.
A way to solve this problem is to use the ideas of
Barles \cite{barles07} and his ''$F^k$-trick'' (see \cite[Lemma 2.1 and
Theorem  2.1]{barles07}). \\
2. In the Lipschitz case (when $u_0$ is Lipschitz continuous), the above proof
can be done in a simpler way using the uniform Lipschitz estimates on $u^\varepsilon$ given
by Proposition \ref{PrUC}.
\end{rem}

\section{Example}\label{Ex}
We first describes a class of   systems \eqref{HJS} which satisfy \eqref{A3}. We assume that the Hamiltonians $H_j$ satisfy the following
assumption
(see \cite{el91})
\begin{equation}\label{A1}
\begin{array}{c}
\text{There  exists $c_{ij}\in\mathbb R,$ $1\leq i,j\leq M,$ s.t. $\sum_{j=1}^M c_{ji}\ge  0$ and}\\[4pt]
\text{for any $(x,y,r,p)\in\mathbb R^N\times\mathbb R^N\times\mathbb R^M\times \mathbb R^N$, $\delta>0,$}\\[4pt]
 c_{ji} \delta\le H_i(x,y,r+\delta e_j,p)- H_i(x,y,r,p)\le 0\qquad \text{if $j\neq i$},\\[4pt]
  c_{ii} \delta \le H_i(x,y,r+\delta e_i,p)- H_i(x,y,r,p)
\end{array}
\end{equation}
where $(e_1,\cdots ,e_n)$ is the canonical basis of $\mathbb R^M.$
Note that necessarily $c_{ji}\leq 0$
for $i\not= j$ and $c_{ii}\geq  0$. \par

In the next proposition we prove that the  assumption \eqref{A1}  implies the      monotonicity
 condition \eqref{A3}
\begin{prop}
Condition \eqref{A1} implies  \eqref{A3}.
\end{prop}
\begin{proof}
Assume that $r_j-s_j=\max_{k=1,\dots,M}\{r_k-s_k\}\ge 0$. For simplicity, we
drop the dependence in $(x,y,p)$ in $H(x,y,r,p)$ in the proof of the proposition since
these variables do not play any role here.
We have
\begin{eqnarray*}
&& H_j(r)-H_j(s) \\
&=& H_j(r_1, r_2, \cdots , r_j, \cdots , r_M) -   H_j(s_1, s_2, \cdots , s_j, \cdots , s_M) \\
&=& H_j(r_1, r_2, \cdots , r_j, \cdots , r_M) -  H_j(s_1, r_2, \cdots , r_j, \cdots , r_M) \\
&& + H_j(s_1, r_2, \cdots , r_j, \cdots , r_M) -  H_j(s_1, s_2, r_3, \cdots ,r_j, \cdots , r_M) \\
&& + \cdots \\
&& + H_j (s_1,\cdots , s_{j-1}, r_j,\cdots r_M)- H_j (s_1,\cdots , s_{j}, r_{j+1},\cdots r_M) \\
&& + \cdots \\
&& + H_j (s_1,\cdots , s_{M-1}, r_M)- H_j (s_1, \cdots s_M).
\end{eqnarray*}
If $k\not= j,$
\begin{eqnarray*}
&& H_j(s_1, \cdots s_{k-1}, r_k,r_{k+1} \cdots , r_M) -  H_j(s_1, \cdots ,s_{k-1},s_k , r_{k+1}, \cdots , r_M)\\[5pt]
&\geq&
\left\{
\begin{array}{cc}
0 & {\rm if} \ r_k-s_k <0\\
c_{kj}(r_k-s_k) & {\rm if} \ r_k-s_k \geq 0
\end{array}\right.
\quad \geq  c_{kj}(r_j-s_j)
\end{eqnarray*}
since $c_{kj}\leq 0$ and $r_j-s_j=\max_{k=1,\dots,M}\{r_k-s_k\}\ge 0$. Moreover,
\begin{eqnarray*}
 H_j(s_1, \cdots s_{j-1}, r_j,r_{j+1} \cdots , r_M) -  H_j(s_1, \cdots s_{j-1},s_j , r_{j+1}, \cdots , r_M)
\geq c_{jj}(r_j-s_j).
\end{eqnarray*}
It follows
\begin{eqnarray*}
H_j(r)-H_j(s) \geq \sum_{k=1}^{M} c_{kj}(r_j-s_j)\geq 0
\end{eqnarray*}
as desired.
\end{proof}
\begin{rem}
Property \eqref{A3} is not equivalent to \eqref{A1}. More
precisely, if \eqref{A3} holds, the existence of $c_{ji}$ for $j\not= i$ is not always true
(the others assertions hold).
Indeed, for
$M=2,$ consider for instance $H_1 (r_1,r_2)= {\rm e}^{r_1-r_2}+2r_1 -r_2$
(and define $H_2$ symmetrically). Then
\begin{eqnarray*}
H_1( r_1+\delta , r_2+\mu)- H_1(r_1,r_2)=  {\rm e}^{r_1-r_2}({\rm  e}^{\delta-\mu} -1)+2\delta -\mu
\geq \delta
\end{eqnarray*}
when $\mu\leq \delta.$
This ensures \eqref{A3} with $\lambda_0=1.$ Nevertheless,
$H_1( r_1 , r_2+\mu)- H_1(r_1,r_2) \sim -(1+{\rm e}^{r_1-r_2}) \mu$ for small
$\mu$
and $1+{\rm e}^{r_1-r_2}$ is not bounded.
\end{rem}
A particular case of monotone systems are the    weakly coupled  systems \eqref{Ex1wk}.
For \eqref{Ex1wk}, assumption  \eqref{A1} is satisfied if
\begin{eqnarray}\label{cond-sur-c}
c_{ii}(x,y)\ge 0, \ \ c_{ji}(x,y)\le 0 \ \text{ for $j\neq i$}\ \ \text{ and}
\ \ \sum_{j=1}^M
c_{ji}(x,y)\geq 0
\end{eqnarray}
for any $x,y\in\mathbb R^N$, $i,j\in\{1,\dots,M\}$.
Let us consider a specific example of
weakly coupled system
for which it is possible to have an explicit
formula for the effective Hamiltonians. Consider the system
\[
\frac{\partial u_i}{\partial t}+ \left|\frac{\partial u_i}{\partial x}\right|
+\sum_{j=1}^M c_{ji}\left(\frac{x}{\varepsilon}\right)u_j=0\qquad (x,t)\in \mathbb R\times[0,\infty).
\]
where  the $c_{ji}$'s satisfy \eqref{cond-sur-c}. The associated cell problems are
\begin{equation}\label{Ex1CP}
|p+v'(y)|+\sum_{j=1}^M c_{ji}(y)r_j=\lambda\qquad y\in [0,1], \lambda\in\mathbb R,
\end{equation}
for $i=1,\dots,M$.  We rewrite \eqref{Ex1CP} as
\begin{equation}\label{Ex1CP1}
|p+v'(y)|=\lambda+f(y)\qquad y\in [0,1]
\end{equation}
where $\displaystyle f(y)=-\sum_{j=1}^Mc_{ji}(y)r_j$.
The effective Hamiltonian for \eqref{Ex1CP1} is given by (see \cite{concordel96})
\begin{equation}\label{Ex1EH}
    \overline H(p)=\max\{-\min_{[0,1]}f,\, |p|-\int_0^1 f(y)dy\}
\end{equation}
and therefore we get the effective Hamiltonian for \eqref{Ex1CP}
\begin{equation}\label{Ex1F}
\overline H_i(r,p)=\max\left\{\max_{[0,1]}\sum_{j=1}^M c_{ji}(y)r_j,\, |p|+\sum_{j=1}^M r_j\int_0^1 c_{ji}(y)dy\right\}
\end{equation}

A natural question is that if the problem \eqref{HJ0} which arises in the homogenized
limit of the weakly coupled system \eqref{Ex1wk} is still of weakly coupled type. Whereas the answer is
positive if the coefficients $c_{ij}$ are independent of $y$, in general it is
not necessarily true.

\begin{prop} Assume that \eqref{H1} holds and
that the coefficients $c_{ij}$ in \eqref{Ex1wk} are constant and
satisfy \eqref{cond-sur-c}. Then
\begin{equation}\label{Ex1wk1}
   \overline H_i(r,p)=\overline H_i(p)+  \sum_{j=1}^M c_{ji}\,r_j
\end{equation}
where $\overline H_i(p)$ is the effective Hamiltonian of $H_i(x,p)$, i.e. the unique $\lambda\in \mathbb R$ for which the equation
\begin{equation}\label{Ex1wk2}
    H_i(y,p+Dv(y))=\overline H_i(p)\qquad y\in\mathbb{T}^N
\end{equation}
admits a viscosity solution.
\end{prop}
\begin{proof}
By definition, there exists a  viscosity solution to
\[H_i(y,p+Dv(y))+\sum_{j=1}^M c_{ji}r_j=\overline H_i(r,p),\qquad y\in\mathbb{T}^N\]
or equivalently to
\[H_i(y,p+Dv(y))=\overline H_i(r,p)- \sum_{j=1}^M c_{ji}r_j\qquad y\in\mathbb{T}^N.\]
By \eqref{Ex1wk2} and the uniqueness of the effective Hamiltonian, the
constant in the right hand side of  the previous equation \ is given by $\overline H_i(p)$,
hence the formula \eqref{Ex1wk1}.
\end{proof}

The following example
shows that if the coupling coefficients $c_{ij}$ are not constants, the limit system is not
 necessarily weakly coupled. Consider the 1-dimensional case \eqref{Ex1CP}.  Take $i=1$ and $r=(r_1,0,\dots,0)$, then
 \[\overline H_1(r,p)=\max\left\{\max_{[0,1]} c_{11}(y)r_1,\, |p|+  r_1\int_0^1 c_{11}(y)dy\right\}.\]
If $\max_{[0,1]} c_{11} =\alpha$, $\min_{[0,1]} c_{11}=\beta$ and $\int_0^1 c_{11}(y)dy=\gamma$, with $\alpha>\gamma>\beta\ge 0$, then
for $p\neq 0$ fixed
\[ \overline H_1(r,p)=\left\{
\begin{array}{ll}
\beta r_1& {\rm if} \ r_1\le  |p|/(\beta-\gamma),\\
\gamma r_1+|p|& {\rm if} \ |p|/(\beta-\gamma)\le r_1\le |p|/(\alpha-\gamma),\\
\alpha r_1&  {\rm if} \ r_1\ge |p|/(\alpha-\gamma).
\end{array}
\right.\]
Then $H_1(r,p)$ is not a linear function of $r$ and therefore is not of weakly coupled type.\par
By the formula \eqref{Ex1F} it is possible to see another typical phenomenon in homogenization of Hamilton-Jacobi
equation, the presence of a flat part in the graph of effective Hamiltonian (see \cite{concordel96}, \cite{lpv86}).


\section{Introduction}

\label{sec:1}

Neutron stars are laboratories for dense matter physics, since they contain
matter in one of the densest forms found in the universe. It is expected
that the deconfinement phase transition may occur in the core of massive
neutron stars~\cite{Weber05}. The study of the hadron-quark phase transition
at high density is of great interest in both nuclear physics and
astrophysics. It has been pointed out by Glendenning~\cite{GL92} that the
hadron-quark phase transition in neutron stars may proceed through a mixed
phase of hadronic and quark matter over a finite range of pressures and
densities according to the Gibbs criteria for phase equilibrium. Such phase
transition has received much attention in neutron star physics~\cite%
{PR97,PR00,PRC99,NPA00,Menezes,Sharma,Lattimer,Burgio,HuangM}.
It is believed that hyperons may appear around twice normal nuclear matter
density through the weak interaction~\cite{PRC96}, which usually occur
earlier than the hadron-quark phase transition. The inclusion of hyperons
in the hadronic phase alters the threshold density and properties of the
mixed phase significantly. In general, the presence of new degrees of freedom,
such as hyperons and quarks, tends to soften the equation of state (EOS) at
high density and lower the maximum mass of neutron stars.

To study the hadron-quark phase transition, we need models to describe
hadronic matter and quark matter. Unfortunately, there is no single model
which can be used to describe both phases and the dynamic process of the
phase transition. We have to use different approaches for the description of
the two phases, then perform the Glendenning construction for the
charge-neutral mixed phase where both hadronic and quark phases coexist~\cite%
{GL92}. In this work, we adopt the relativistic mean field (RMF) theory to
describe the hadronic matter phase, while the Nambu-Jona-Lasinio (NJL) model
is used for the quark matter phase. The choice of the NJL model is motivated
by the fact that this model can successfully reproduce many aspects of
quantum chromodynamics such as the nonperturbative vacuum structure and
dynamical breaking of chiral symmetry~\cite{NJL1,NJL2,NJL3}. We adopt a
three-flavor version of the NJL model to describe the quark matter phase~%
\cite{NJL1}. With a definite EOS for quark matter based on the NJL model, we
examine the influence of the hadronic EOS on the hadron-quark phase
transition and neutron star properties.

We use the RMF theory to describe the hadronic matter phase. The RMF theory
has been successfully and widely used for the description of nuclear
matter and finite nuclei~\cite{Serot86,Ring90,Toki96,Ren02,Shen06}. It has
also been applied to provide the EOS of dense matter for the use in
supernovae and neutron stars~\cite{Shen98,Shen02}. In the RMF approach,
baryons interact through the exchange of scalar and vector mesons. The
meson-nucleon coupling constants are generally determined by fitting to some
nuclear matter properties or ground-state properties of finite nuclei.
However, there are large uncertainties in the meson-hyperon couplings due to
limited experimental data. One can use the coupling constants derived from
the quark model, or those values constrained by reasonable hyperon
potentials. The two additional strange mesons, $\sigma
^{\ast }$ and $\phi $, were originally introduced in order to obtain the
strong attractive hyperon-hyperon ($YY$) interaction deduced from the
earlier measurement~\cite{PRL93}. A recent observation of the double-$%
\Lambda $ hypernucleus $_{\Lambda \Lambda }^{6}\mathrm{He}$, called the
Nagara event~\cite{Nagara}, has had a significant impact on strangeness
nuclear physics. The Nagara event provides unambiguous identification of $%
_{\Lambda \Lambda }^{6}\mathrm{He}$ production with precise $\Lambda \Lambda
$ binding energy value $B_{\Lambda \Lambda }=7.25\pm 0.19_{-0.11}^{+0.18}\;%
\mathrm{MeV}$, which suggests that the effective $\Lambda \Lambda $
interaction should be considerably weaker ($\triangle B_{\Lambda \Lambda
}\sim 1\;\mathrm{MeV}$) than that deduced from the earlier measurement ($%
\triangle B_{\Lambda \Lambda }\sim 5\;\mathrm{MeV}$). The weak $YY$
interaction suggested by the Nagara event has been used to reinvestigate the
properties of multistrange systems, and it has been found that the change
of $YY$ interactions affects the properties of strange hadronic matter
dramatically~\cite{PRC03s,JPG04s,JPG05}. In order to examine the influence
of the hadronic EOS on the hadron-quark phase transition, we employ two
successful parameter sets of the RMF model, NL3~\cite{NL3} and TM1~\cite{TM1}%
, which have been widely used for the description of nuclear matter and
finite nuclei including unstable nuclei. For each parameter set of the
nucleonic sector, we consider two cases of hyperon-hyperon interactions, the
weak and strong $YY$ interactions. By comparing the results with
different parametrizations in the RMF model, we evaluate how sensitive the
hadron-quark phase transition is to the hadronic EOS used in the calculation.

For a comprehensive description of neutron stars, we need not only the EOS
at high density for the interior region but also the EOS for the inner and
outer crusts, where the density is low and heavy nuclei exist. For the
nonuniform matter at low density, we adopt a relativistic EOS based on the
RMF theory with a local density approximation~\cite{Shen98,Shen02}. The
nonuniform matter is modelled to be composed of a lattice of spherical
nuclei immersed in an electron gas with or without free neutrons dripping
out of nuclei. As the density increases, heavy nuclei dissolve and the
optimal state is a uniform matter consisting of neutrons, protons, and
leptons (electrons and muons) in $\beta $ equilibrium. The low density EOS
is therefore matched to an EOS of uniform nuclear matter at around $10^{14}$
g/cm$^{3}$~\cite{Shen02}. At higher densities, some additional degrees of
freedom such as hyperons and quarks may occur, and the hadron-quark phase
transition could proceed through a mixed phase of hadronic and quark
matter~\cite{PRC99,NPA00,Menezes,Sharma,Lattimer,Burgio,HuangM}.
Applying the EOS of neutron star matter over a wide density range,
we study the neutron star properties by solving the
Tolman-Oppenheimer-Volkoff equation, and examine whether or
not the deconfined quark phase can exist in the core of neutron stars.

This paper is arranged as follows. In Sec.~\ref{sec:2}, we discuss the EOS
for hadronic matter in the RMF theory, and the parameters used in the calculation.
In Sec.~\ref{sec:3}, the NJL model is used for the description of quark
matter. In Sec.~\ref{sec:4}, we investigate the hadron-quark phase
transition of neutron star matter, and examine the influence of the hadronic
EOS. We present in Sec.~\ref{sec:5} the properties of neutron stars.
Section~\ref{sec:6} is devoted to a summary.

\section{Hadronic Phase}

\label{sec:2}

To describe the hadronic matter phase, we adopt the relativistic mean field
(RMF) theory, in which baryons interact via the exchange of mesons. The
baryons considered in this work are nucleons ($p$ and $n$) and hyperons ($%
\Lambda $, $\Sigma $, and $\Xi $). The exchanged mesons include isoscalar
scalar and vector mesons ($\sigma $ and $\omega $), isovector vector meson ($%
\rho $), and two additional hidden-strangeness mesons ($\sigma ^{\ast }$ and
$\phi $). For neutron star matter consisting of a neutral mixture of baryons
and leptons in $\beta $ equilibrium, we start from the effective Lagrangian%
\begin{eqnarray}
\mathcal{L}_{RMF} &=&\sum_{B}\bar{\psi}_{B}\left[ i\gamma _{\mu }\partial
^{\mu }-m_{B}-g_{\sigma B}\sigma -g_{\sigma ^{\ast }B}\sigma ^{\ast
}-g_{\omega B}\gamma _{\mu }\omega ^{\mu }\right.  \nonumber \\
&&\left. -g_{\phi B}\gamma _{\mu }\phi ^{\mu }-g_{\rho B}\gamma _{\mu }\tau
_{i}\rho _{i}^{\mu }\right] \psi _{B}+\frac{1}{2}\partial _{\mu }\sigma
\partial ^{\mu }\sigma -\frac{1}{2}m_{\sigma }^{2}\sigma ^{2}  \nonumber \\
&&-\frac{1}{3}g_{2}\sigma ^{3}-\frac{1}{4}g_{3}\sigma ^{4}-\frac{1}{4}W_{\mu
\nu }W^{\mu \nu }+\frac{1}{2}m_{\omega }^{2}\omega _{\mu }\omega ^{\mu }
\nonumber \\
&&+\frac{1}{4}c_{3}\left( \omega _{\mu }\omega ^{\mu }\right) ^{2}-\frac{1}{4%
}R_{i\mu \nu }R_{i}^{\mu \nu }+\frac{1}{2}m_{\rho }^{2}\rho _{i\mu }\rho
_{i}^{\mu }  \nonumber \\
&&+\frac{1}{2}\partial _{\mu }\sigma ^{\ast }\partial ^{\mu }\sigma ^{\ast }-%
\frac{1}{2}m_{\sigma ^{\ast }}^{2}\sigma ^{\ast 2}-\frac{1}{4}S_{\mu \nu
}S^{\mu \nu }+\frac{1}{2}m_{\phi }^{2}\phi _{\mu }\phi ^{\mu }  \nonumber \\
&&+\sum_{l}\bar{\psi}_{l}\left[ i\gamma _{\mu }\partial ^{\mu }-m_{l}\right]
\psi _{l},  \label{eq:Lrmf}
\end{eqnarray}%
where the sum on $B$ runs over the baryon octet ($p$, $n$, $\Lambda $, $%
\Sigma ^{+}$, $\Sigma ^{0}$, $\Sigma ^{-}$, $\Xi ^{0}$, $\Xi ^{-}$),
and the sum on $l$ is over electrons and muons ($e^{-}$ and $\mu
^{-}$). In the RMF model, the meson fields are treated as classical
fields, and the field operators are replaced by their expectation
values. The meson field equations in uniform matter have the
following form:
\begin{eqnarray}
&&m_{\sigma }^{2}\sigma +g_{2}\sigma ^{2}+g_{3}\sigma ^{3}=-\sum_{B}\frac{%
g_{\sigma B}}{\pi ^{2}}\int_{0}^{k_{F}^{B}}\frac{m_{B}^{\ast }}{\sqrt{%
k^{2}+m_{B}^{\ast 2}}}k^{2}dk,  \label{eq:s} \\
&&m_{\omega }^{2}\omega +c_{3}\omega ^{3}=\sum_{B}\frac{g_{\omega B}\left(
k_{F}^{B}\right) ^{3}}{3\pi ^{2}},  \label{eq:w} \\
&&m_{\rho }^{2}\rho =\sum_{B}\frac{g_{\rho B}\tau _{3B}\left(
k_{F}^{B}\right) ^{3}}{3\pi ^{2}},  \label{eq:r} \\
&&m_{\sigma ^{\ast }}^{2}\sigma ^{\ast }=-\sum_{B}\frac{g_{\sigma ^{\ast }B}%
}{\pi ^{2}}\int_{0}^{k_{F}^{B}}\frac{m_{B}^{\ast }}{\sqrt{k^{2}+m_{B}^{\ast
2}}}k^{2}dk,  \label{eq:ss} \\
&&m_{\phi }^{2}\phi =\sum_{B}\frac{g_{\phi B}\left( k_{F}^{B}\right) ^{3}}{%
3\pi ^{2}},  \label{eq:ws}
\end{eqnarray}%
where $\sigma =\left\langle \sigma \right\rangle ,$ $\omega =\left\langle
\omega ^{0}\right\rangle ,$ $\rho =\left\langle \rho ^{30}\right\rangle ,$ $%
\sigma ^{\ast }=\left\langle \sigma ^{\ast }\right\rangle ,$ and $\phi
=\left\langle \phi ^{0}\right\rangle $ are the nonvanishing expectation
values of meson fields in neutron star matter. $m_{B}^{\ast
}=m_{B}+g_{\sigma B}\sigma +g_{\sigma ^{\ast }B}\sigma ^{\ast }$ is the
effective mass of the baryon species $B$, and $k_{F}^{B}$ is the Fermi
momentum.

In this work, we employ two successful parameter sets of the RMF model, NL3
and TM1, as listed in Table~\ref{tab:1}. These parameters have been
determined by fitting to some ground-state properties of finite nuclei, and
they can provide good description of nuclear matter and finite nuclei
including unstable nuclei~\cite{NL3,TM1}. As for the meson-hyperon
couplings, we take the naive quark model values for the vector coupling
constants,
\begin{eqnarray}
&&\frac{1}{3}g_{\omega N}=\frac{1}{2}g_{\omega \Lambda }=\frac{1}{2}%
g_{\omega \Sigma }=g_{\omega \Xi },  \nonumber \\
&&g_{\rho N}=\frac{1}{2}g_{\rho \Sigma }=g_{\rho \Xi },\ \ g_{\rho \Lambda
}=0, \nonumber \\
&&2g_{\phi \Lambda }=2g_{\phi \Sigma }=g_{\phi \Xi }=-\frac{2\sqrt{2}}{3}%
g_{\omega N},\ \ g_{\phi N}=0.
\end{eqnarray}%
The scalar coupling constants are chosen to give reasonable hyperon
potentials. The potential depth of the hyperon species $i$ in matter of the
baryon species $j$ is denoted by $U_{i}^{\left( j\right) }$, and we use
$U_{\Lambda }^{\left(N\right) }=-28$ MeV,
$U_{\Sigma  }^{\left(N\right) }=+30$ MeV, and
$U_{\Xi     }^{\left(N\right) }=-18$ MeV~\cite{PRC88,PRC00,PRC04}
to determine the scalar coupling constants
$g_{\sigma \Lambda }$, $g_{\sigma \Sigma }$, and $g_{\sigma \Xi }$,
respectively. The hyperon couplings to strange meson $\sigma^{\ast }$ are
restricted by the relation $U_{\Xi }^{\left(\Xi\right) }\simeq U_{\Lambda
}^{\left(\Xi\right) }\simeq 2U_{\Xi}^{\left(\Lambda\right) }\simeq
2U_{\Lambda }^{\left(\Lambda\right) }$ obtained in Ref.~\cite{ANN94}.
We consider two cases of hyperon-hyperon ($YY$) interactions.
The strong $YY$ interaction deduced from the earlier measurement~\cite{PRL93}
suggests $U_{\Lambda }^{(\Lambda )}\simeq -20\ \mathrm{MeV,}$ while the weak
$YY$ interaction implied by the Nagara event suggests
$U_{\Lambda}^{(\Lambda )}\simeq -5\ \mathrm{MeV}$~\cite{PRC03s,JPG04s,JPG05}.
In Table~ \ref{tab:2}, we present the meson-hyperon couplings
determined by these hyperon potentials. The hyperon masses are taken to be
$m_{\Lambda }=1115.7\ \text{MeV}$, $m_{\Sigma }=1193.1\ \text{MeV}$,
and $m_{\Xi }=1318.1\ \text{MeV}$, while the strange meson masses are
$m_{\sigma^{\ast }}=975\ \text{MeV}$ and $m_{\phi }=1020\ \text{MeV}$.

For neutron star matter consisting of a neutral mixture of baryons and
leptons, the $\beta $ equilibrium conditions without trapped neutrinos are
given by%
\begin{eqnarray}
&&\mu _{p}=\mu _{\Sigma ^{+}}=\mu _{n}-\mu _{e},  \label{eq:beta11} \\
&&\mu _{\Lambda }=\mu _{\Sigma ^{0}}=\mu _{\Xi ^{0}}=\mu _{n},
\label{eq:beta12} \\
&&\mu _{\Sigma ^{-}}=\mu _{\Xi ^{-}}=\mu _{n}+\mu _{e},  \label{eq:beta13} \\
&&\mu _{\mu }=\mu _{e},  \label{eq:beta14}
\end{eqnarray}%
where $\mu _{i}$ is the chemical potential of species $i$. At zero
temperature the chemical potentials of baryons and leptons are expressed by%
\begin{eqnarray}
&&\mu _{B}=\sqrt{{k_{F}^{B}}^{2}+m_{B}^{\ast 2}}+g_{\omega B}\omega +g_{\phi
B}\phi +g_{\rho B}\tau _{3B}\rho ,  \label{eq:mub} \\
&&\mu _{l}=\sqrt{{k_{F}^{l}}^{2}+m_{l}^{2}}.  \label{eq:mul}
\end{eqnarray}%
The charge neutrality condition is given by
\begin{eqnarray}
n_{p}+n_{\Sigma ^{+}}=n_{e}+n_{\mu }+n_{\Sigma ^{-}}+n_{\Xi ^{-}},
\end{eqnarray}
where $n_{i}=\left( k_{F}^{i}\right) ^{3}/(3\pi ^{2})$ is the number density
of species $i$. We can solve the coupled equations self-consistently at a
given baryon density $n_{B}=n_{p}+n_{n}+n_{\Lambda }+n_{\Sigma
^{+}}+n_{\Sigma ^{0}}+n_{\Sigma ^{-}}+n_{\Xi ^{0}}+n_{\Xi ^{-}}$. The total
energy density and pressure of neutron star matter are written by
\begin{eqnarray}
\varepsilon _{HP} &=&\sum_{B}\frac{1}{\pi ^{2}}\int_{0}^{k_{F}^{B}}\sqrt{%
k^{2}+m_{B}^{\ast 2}}\ k^{2}dk+\frac{1}{2}m_{\sigma }^{2}\sigma ^{2}+\frac{1%
}{3}g_{2}\sigma ^{3}+\frac{1}{4}g_{3}\sigma ^{4}  \nonumber \\
&&+\frac{1}{2}m_{\omega }^{2}\omega ^{2}+\frac{3}{4}c_{3}\omega ^{4}+\frac{1%
}{2}m_{\rho }^{2}\rho ^{2}+\frac{1}{2}m_{\sigma ^{\ast }}^{2}\sigma ^{\ast
2}+\frac{1}{2}m_{\phi }^{2}\phi ^{2}  \nonumber \\
&&+\sum_{l}\frac{1}{\pi ^{2}}\int_{0}^{k_{F}^{l}}\sqrt{k^{2}+m_{l}^{2}}\
k^{2}dk,  \label{eq:e1}
\end{eqnarray}%
\begin{eqnarray}
P_{HP} &=&\frac{1}{3}\sum_{B}\frac{1}{\pi ^{2}}\int_{0}^{k_{F}^{B}}\frac{%
k^{4}\ dk}{\sqrt{k^{2}+m_{B}^{\ast 2}}}-\frac{1}{2}m_{\sigma }^{2}\sigma
^{2}-\frac{1}{3}g_{2}\sigma ^{3}-\frac{1}{4}g_{3}\sigma ^{4}  \nonumber \\
&&+\frac{1}{2}m_{\omega }^{2}\omega ^{2}+\frac{1}{4}c_{3}\omega ^{4}+\frac{1%
}{2}m_{\rho }^{2}\rho ^{2}-\frac{1}{2}m_{\sigma ^{\ast }}^{2}\sigma ^{\ast
2}+\frac{1}{2}m_{\phi }^{2}\phi ^{2}  \nonumber \\
&&+\frac{1}{3}\sum_{l}\frac{1}{\pi ^{2}}\int_{0}^{k_{F}^{l}}\frac{k^{4}\ dk}{%
\sqrt{k^{2}+m_{l}^{2}}}.  \label{eq:p1}
\end{eqnarray}

\section{Quark Phase}

\label{sec:3}

In this section, we adopt a three-flavor version of the NJL model to
describe the deconfined quark phase. The Lagrangian is given by%
\begin{eqnarray}
\mathcal{L}_{NJL} &=&\bar{q}\left( i\gamma _{\mu }\partial ^{\mu
}-m^{0}\right) q+G\sum_{a=0}^{8}\left[ \left( \bar{q}\lambda _{a}q\right)
^{2}+\left( \bar{q}i\gamma _{5}\lambda _{a}q\right) ^{2}\right]  \nonumber \\
&&-K\left\{ \det \left[ \bar{q}\left( 1+\gamma _{5}\right) q\right] +\det %
\left[ \bar{q}\left( 1-\gamma _{5}\right) q\right] \right\} ,
\label{eq:Lnjl}
\end{eqnarray}%
where $q$ denotes a quark field with three flavors $\left( N_{f}=3\right) $
and three colors $\left( N_{c}=3\right) $. $m^{0}=$diag$\left(
m_{u}^{0},m_{d}^{0},m_{s}^{0}\right) $ is the current quark mass matrix, and
we assume isospin symmetry $m_{u}^{0}=m_{d}^{0}\equiv m_{q}^{0}$. The
coupling constants $G$ and $K$ have dimension energy$^{-2}$ and energy$^{-5}$%
, respectively. The model has five parameters, namely, the current quark
masses $m_{q}^{0}$ and $m_{s}^{0}$, the coupling constants $K$ and $G$, and
the momentum cutoff $\Lambda $. In the present calculation, we employ the
parameters given in Ref.~\cite{NJL96}, $m_{q}^{0}=5.5\ \text{MeV}$, $%
m_{s}^{0}=140.7\ \text{MeV}$, $\Lambda =602.3\ \text{MeV}$, $G\Lambda
^{2}=1.835$, and $K\Lambda ^{5}=12.36$. These parameters have been
determined by fitting $f_{\pi }$, $m_{\pi }$, $m_{K}$, and $m_{\eta
^{^{\prime }}}$ to their empirical values, while the mass of the $\eta $%
-meson is underestimated by about 6\%~\cite{NJL1}.

In the NJL model, the quark gets constituent quark mass by spontaneous
chiral symmetry breaking. The constituent quark mass in vacuum $m_{i}$ is
much larger than the current quark mass $m_{i}^{0}$. In the quark matter
at high density, the constituent quark mass $m_{i}^{\ast }$ becomes
approximately the same as $m_{i}^{0}$, which reflects the restoration of
chiral symmetry. Within the mean-field approximation, $m_{i}^{\ast }$ is
obtained by solving the gap equation%
\begin{equation}
m_{i}^{\ast }=m_{i}^{0}-4G\langle \bar{q}_{i}q_{i}\rangle +2K\langle \bar{q}%
_{j}q_{j}\rangle \langle \bar{q}_{k}q_{k}\rangle ,  \label{eq:gap}
\end{equation}%
with ($i,j,k$) being any permutation of ($u,d,s$). The quark condensate $%
C_{i}=\left\langle \bar{q}_{i}q_{i}\right\rangle $ is given by
\begin{equation}
C_{i}=-\frac{3}{{\pi }^{2}}{\int_{k_{F}^{i}}^{\Lambda }\frac{m_{i}^{\ast }}{%
\sqrt{k^{2}+m_{i}^{\ast 2}}}}k^{2}dk,
\end{equation}
where $k_{F}^{i}$ denotes the Fermi momentum of the quark flavor $i$,
which is connected with the number density $n_{i}$ and the
chemical potential $\mu _{i}$ via
\begin{eqnarray}
n_{i} &=&\frac{\left( k_{F}^{i}\right) ^{3}}{\pi ^{2}}, \\
\mu _{i} &=&\sqrt{{k_{F}^{i}}^{2}+{m_{i}^{\ast }}^{2}}.
\end{eqnarray}%
The energy density of the quark system is given by
\begin{eqnarray}
\varepsilon _{NJL} &=&\sum_{i=u,d,s}\left[ -\frac{3}{\pi ^{2}}%
\int_{k_{F}^{i}}^{\Lambda }\sqrt{k^{2}+m_{i}^{\ast 2}}\ k^{2}dk\right]
\nonumber \\
&&+2G\left( C_{u}^{2}+C_{d}^{2}+C_{s}^{2}\right) -4K{C_{u}}{C_{d}}{C_{s}}%
-\varepsilon _{0},  \label{eq:eNJL}
\end{eqnarray}%
where $\varepsilon _{0}$ is introduced to ensure $\varepsilon _{NJL}=0$ in
the vacuum.

For the quark matter consisting of a neutral mixture of quarks ($u$, $d$,\
and $s$) and leptons ($e$ and $\mu $) in $\beta $ equilibrium, the charge
neutrality condition is expressed as%
\begin{equation}
\frac{2}{3}n_{u}-\frac{1}{3}\left( n_{d}+n_{s}\right) -n_{e}-n_{\mu }=0,
\label{eq:charge2}
\end{equation}%
the $\beta $ equilibrium conditions are given by%
\begin{eqnarray}
\mu _{s} &=&\mu _{d}=\mu _{u}+\mu _{e},  \label{eq:beta21} \\
\mu _{\mu } &=&\mu _{e}.  \label{eq:beta22}
\end{eqnarray}%
The coupled equations can be solved self-consistently at a given baryon
density $n_{B}=\left( n_{u}+n_{d}+n_{s}\right) /3$. The total energy density
and pressure including the contributions from both quarks and leptons are
given by
\begin{eqnarray}
\varepsilon _{QP} &=&\varepsilon _{NJL}+\sum_{l=e,\mu }\frac{1}{\pi ^{2}}%
\int_{0}^{k_{F}^{l}}\sqrt{k^{2}+m_{l}^{2}}\ k^{2}dk,  \label{eq:e2} \\
P_{QP} &=&\sum_{i=u,d,s,e,\mu }n_{i}\mu _{i}-\varepsilon _{QP}.
\label{eq:p2}
\end{eqnarray}

\section{Hadron-Quark Phase Transition}

\label{sec:4}

In this section, we study the hadron-quark phase transition which may occur
in the core of massive neutron stars. It has been discussed extensively in
the literature that a mixed phase of hadronic and quark matter could exist
over a finite range of pressures and densities according to the Gibbs
criteria for phase equilibrium. In the mixed phase, the local charge
neutrality condition is replaced by a global one. This means that both
hadronic and quark matter are allowed to be separately charged. The
condition of global charge neutrality is expressed as
\begin{equation}
\chi n_{c}^{QP}+\left( 1-\chi \right) n_{c}^{HP}=0,  \label{eq:charge3}
\end{equation}%
where $\chi $ is the volume fraction occupied by quark matter in the mixed
phase, which increases from $\chi =0$ in the pure hadronic
phase to $\chi =1$ in the pure quark phase. $n_{c}^{HP}$ and $n_{c}^{QP}$
denote the charge densities of hadronic phase and quark phase, respectively.
Without the constraint of local charge neutrality, we impose that the two
phases are in weak equilibrium and described by two independent chemical
potentials ($\mu_{n}$, $\mu_{e}$). The Gibbs condition for phase
equilibrium at zero temperature is then given by
\begin{equation}
P_{HP}\left( \mu _{n},\mu _{e}\right) =P_{QP}\left( \mu _{n},\mu _{e}\right).
\label{eq:p3}
\end{equation}%
Using Eq.~(\ref{eq:p3}) we can calculate the equilibrium chemical potentials
of the mixed phase where $P_{HP}=P_{QP}=P_{MP}$ holds. The energy density
and the baryon density in the mixed phase are given by
\begin{equation}
\varepsilon _{MP}=\chi \varepsilon _{QP}+\left( 1-\chi \right) \varepsilon
_{HP},  \label{eq:e3}
\end{equation}
and
\begin{equation}
n_{B}^{MP}=\chi n_{B}^{QP}+\left( 1-\chi \right) n_{B}^{HP}.  \label{eq:r3}
\end{equation}

We show in Fig.~\ref{fig:1} the possible phase structure of neutron star
matter using the RMF model for the hadronic phase and the NJL model for the quark
phase. To examine the influence of the hadronic EOS on the hadron-quark
phase transition, we employ two successful parameter sets of the RMF model,
NL3~\cite{NL3} and TM1~\cite{TM1}, with both the weak and strong $YY$
interactions. The shaded regions correspond to the mixed phase. It is shown
that a pure hadronic phase is favored at low density. The mixed phase
appears at the critical density $n_{B}^{(1)}$ where the pressure of the pure
hadronic phase becomes to be lower than the pressure of the mixed phase. The
fraction of quark matter $\chi $ increases with increasing density in the
mixed phase. It turns to be a pure quark phase at the critical density $%
n_{B}^{(2)}$ where the pressure of the pure quark phase is above the
pressure of the mixed phase. The critical densities $n_{B}^{(1)}$ and $%
n_{B}^{(2)}$ are sensitive to the RMF parameters used, and we get $%
n_{B}^{(1)}\simeq 0.50\ \mathrm{{fm}^{-3}}$ ($0.71\ \mathrm{{fm}^{-3}}$) and
$n_{B}^{(2)}\simeq 0.75\ \mathrm{{fm}^{-3}}$ ($0.98\ \mathrm{{fm}^{-3}}$)
for the NL3 parameter set with the weak (strong) $YY$ interaction,
while $n_{B}^{(1)}\simeq 1.31\ \mathrm{{fm}^{-3}}$ ($1.75\ \mathrm{{fm}^{-3}}
$) and $n_{B}^{(2)}\simeq 2.03\ \mathrm{{fm}^{-3}}$ ($2.49\ \mathrm{{fm}^{-3}%
}$) are obtained for the TM1 cases. It is found that the hadron-quark phase
transition occurs at lower densities in the NL3 model than in the TM1 model,
and in general the weak $YY$ interaction leads to an earlier appearance
of the mixed phase. In order to estimate the influence of hadronic EOS on
the deconfinement phase transition, we plot in Fig.~\ref{fig:2} the hadronic
EOS in the different cases and the quark EOS in the NJL model with local charge
neutrality as a function of the neutron chemical potential $\mu _{n}$.
The crossing of the hadronic EOS with the quark EOS marks the transition
point between the charge neutral hadronic matter and quark matter.
It is seen that the NL3 model favors the phase transition at a lower $\mu_{n}$.
A more realistic treatment of the phase transition is to release the constraint
of local charge neutrality, which leads to the existence of the mixed phase of
charged hadronic and quark matter over a finite range of pressures and
densities as shown in Fig.~\ref{fig:1}. In general, a harder hadronic EOS
also favors an earlier appearance of the mixed phase.

In Fig.~\ref{fig:3} we plot the full EOS in the form $P=P(\varepsilon )$,
which consists of three parts: (a) the charge neutral hadronic matter phase
at low density (b) the mixed phase of charged hadronic and quark matter (c)
the charge neutral quark matter phase at high density. The mixed phase part
of the EOS is shaded gray, where the pressure varies continuously. It is
shown that the onset and width of the mixed phase depend on the RMF
parameters used in the calculation. The NL3 model leads to earlier
appearance of the mixed phase than the TM1 model, and the weak $YY$
interaction favors earlier onset of the mixed phase than the strong $YY$
interaction. This is mainly because that a harder hadronic EOS prefers an
earlier hadron-quark phase transition. By comparing the results of
different cases, we can see the influence of the hadronic EOS on the
hadron-quark phase transition.

We present in Figs.~\ref{fig:4} and \ref{fig:5} the particle fraction $%
Y_{i}=n_{i}/n_{B}$ as a function of the total baryon density $n_{B}$. At low
densities the fractions $Y_{p}$, $Y_{e}$, and$\ Y_{\mu }$ increase with
increasing density. When the Fermi energy of nucleons exceeds the rest mass
of hyperons, the conversion of nucleons to hyperons is energetically
favorable, and it can relieve the Fermi pressure of nucleons. The fraction
of hyperons increases with increasing density before the mixed phase occurs.
Quarks appear at the critical density $n_{B}^{(1)}$, then the fractions $%
Y_{u}$, $Y_{d}$, and$\ Y_{s}$ increase rapidly with increasing density. The
hadronic matter completely disappears at the critical density $n_{B}^{(2)}$
where the pure quark phase occurs. At extremely high density,
$Y_{u}\sim Y_{d}\sim Y_{s}\sim 1/3$ due to the restoration of
chiral symmetry. It is shown that the composition of neutron star matter
depends on the RMF parameters used in the calculation.

\section{Neutron stars properties}

\label{sec:5}

In this section, we investigate the properties of neutron stars by solving
the Tolman-Oppenheimer-Volkoff equation with the EOS over a wide density
range. For the nonuniform matter at low density, which exists in the inner
and outer crusts of neutron stars, we adopt a relativistic EOS based on the
RMF theory with a local density approximation~\cite{Shen98,Shen02}. The
nonuniform matter is modelled to be composed of a lattice of spherical
nuclei immersed in an electron gas with or without free neutrons dripping
out of nuclei. The low density EOS is matched to the EOS of uniform hadronic
matter at the density where they have equal pressures. The pure hadronic
phase ends at the critical density $n_{B}^{(1)}$, and the pure quark phase
starts at the critical density $n_{B}^{(2)}$. The values of these critical
densities depend on the RMF parameters used. The neutron star properties
are mainly determined by the EOS at high density. We calculate neutron star
profiles in order to examine whether or not the deconfined quark phase can
exist in the core of neutron stars.

In Fig.~\ref{fig:6} we present the mass-radius relation using the
EOS of the NL3 and TM1 models with both the weak and strong $YY$
interactions. It is shown that the results depend on the RMF
parameters used in the calculation. Since the pressure and density
inside neutron stars decrease from the center to the surface, the
most possible region where the deconfined quark phase can
exist is the center of the neutron star with maximum mass. We list in Table~%
\ref{tab:3} the properties of neutron stars with the maximum mass. It is found
that the central baryon density $n_{c}$ is between $n_{B}^{(1)}$ and $%
n_{B}^{(2)}$ for the NL3 model. This means that the neutron star can possess
a mixed phase core, but it is not dense enough to possess a pure quark core.
On the other hand, the values of $n_{c}$ in the TM1 model are smaller than
$n_{B}^{(1)}$, which means that the neutron star is only composed of hadronic
matter. As can be seen in Table~\ref{tab:3}, the neutron star properties
significantly depend on the RMF parameters used.

\section{Conclusions}

\label{sec:6}

We have studied the hadron-quark phase transition at high density,
which may occur in the core of massive neutron stars. In the present
work, we have adopted the RMF theory to describe the hadronic matter
phase, while the NJL model has been used for the quark matter phase.
With a definite EOS for the quark phase, we examine the influence of
the hadronic EOS on the deconfinement phase transition and neutron
star properties. In this paper, we employ two successful parameter
sets of the RMF model, NL3 and TM1, which have been widely used for
the description of nuclear matter and finite nuclei including unstable
nuclei. For each parameter set of the nucleonic sector,
we consider two cases of hyperon-hyperon interactions,
the weak and strong $YY$ interactions.
The hadron-quark phase transition can proceed through a mixed phase
of hadronic and quark matter over a finite range of pressures and
densities according to the Gibbs criteria for phase equilibrium. We
have found that the mixed phase starts at $n_{B}^{(1)}\simeq 0.50\
\mathrm{{fm}^{-3}}$ ($0.71\ \mathrm{{fm}^{-3}}$) and ends at
$n_{B}^{(2)}\simeq 0.75\ \mathrm{{fm}^{-3}}$ ($0.98\
\mathrm{{fm}^{-3}}$) for the NL3 parameter set with the weak
(strong) $YY$ interaction, while $n_{B}^{(1)}\simeq 1.31\
\mathrm{{fm}^{-3}}$ ($1.75\ \mathrm{{fm}^{-3}}$) and
$n_{B}^{(2)}\simeq 2.03\ \mathrm{{fm}^{-3}}$ ($2.49\
\mathrm{{fm}^{-3}}$) have been obtained for the TM1 cases. It is
shown that the hadron-quark phase transition occurs at lower
densities in the NL3 model than in the TM1 model, and in general the
weak $YY$ interaction leads to an earlier appearance of the mixed
phase. By comparing the results with different parametrizations in
the RMF model, we can see how sensitive the deconfinement phase
transition is to the hadronic EOS used in the calculation.

We have calculated the properties of neutron stars using the EOS over a wide
density range. The star properties such as their masses and radii are mainly
determined by the EOS at high density. We have found the maximum mass of neutron
stars falls in the range $1.62\sim 2.05\ M_{\odot }$ for the RMF parameters
used. With the NL3 model, the mixed phase can exist in the core of massive
neutron stars, but no pure quark phase can exist. For the TM1 model, the
neutron star is not dense enough to possess the mixed phase, and therefore
the hadron-quark phase transition could not occur inside neutron stars
in this approximation. We conclude that the maximum mass and
composition of neutron stars depend on the hadronic EOS adopted in the
calculation.

It is very interesting to compare our results with those previously published
in the literature~\cite{PRC99,NPA00,Menezes,Sharma}. In Ref.~\cite{PRC99},
the authors studied the possible hadron-quark phase transition utilizing the
same NJL model as used in the present work to describe the deconfined
quark phase, while the hadronic phase was described by several RMF models
(TM1, TM2, GL85, and GPS as listed in Table I of Ref.~\cite{PRC99}).
Comparing with our cases, different hyperon potentials were used to
constrain the hyperon couplings, but some of their values are not supported
by recent experimental observations. They found that the use of the GPS model
for the hadronic phase leads to the onset of the mixed phase at the energy
density $\epsilon \approx 7 \epsilon_0$ ($\epsilon_0=140\;\mathrm{MeV/fm^3}$
is the normal nuclear energy density), while the mixed phase does not appear
below $\epsilon \approx 10 \epsilon_0$ for the TM1, TM2, and GL85 models.
They concluded that within the model constructed in their calculation
the appearance of deconfined quark matter in the center of neutron stars
turns out to be very unlikely. This is consistent with our results although
the hyperon couplings are different between our study and theirs.
In Ref.~\cite{NPA00}, the authors employed an extended MIT bag model to describe
the quark phase and four RMF models (TM1, TM2, GL85, and GPS) to describe
the hadronic phase. They studied the influence of different hadronic EOS
and the influence of the model parameters of the quark phase on the properties
of the phase transition. Using the MIT bag model, they found the mixed phase
could appear below $\epsilon \approx 2 \epsilon_0$, which is much lower than
the threshold density obtained using the NJL model. This can be seen by
comparing Fig.~4 of Ref.~\cite{NPA00} with Fig.~9 of Ref.~\cite{PRC99}
and Fig.~3 of the present paper. In Ref.~\cite{Menezes}, the authors
investigated the deconfinement phase transition at zero and finite temperature
using both the MIT bag model and the NJL model for describing the quark phase
and a RMF model (GL model) for the hadronic phase. For the NJL model, however,
they adopted another parameter set which favors an earlier onset of the mixed
phase than the parameter set used in our calculation. For the hyperon couplings,
they used three choices and verified that for the NJL model the onset of
the mixed phase is very sensitive to the choice of the hyperon couplings.
Using the NJL model for the quark phase, they obtained two different behaviors.
One is the hyperons appear before the quarks and the mixed phase occurs at much
higher densities ($\sim 5 \rho_0$ with the normal nuclear matter density
$\rho_0=0.153\;\mathrm{{fm}^{-3}}$).
The other is the quarks appear at lower densities than the hyperons and
the mixed phase occurs at $\sim 2 \rho_0$. For almost all EOS used in their
study, the central energy density of the maximum-mass neutron star falls inside
the mixed phase, so the star can contain a core constituted by a mixed phase,
but it is not dense enough to possess a pure quark core.
In Ref.~\cite{Sharma}, the authors investigated the deconfinement phase
transition using an effective-field-theory-motivated RMF (E-RMF) model for the
hadronic phase and considering both unpaired quark matter (UQM) described by
the MIT bag model and paired quarks described by the color-flavor locked (CFL)
phase for the quark phase. They mentioned that they could not get any
mixed phase with the original G2 parameter set in the E-RMF model, so they
changed incompressibility $K$ from $215$ to $300$ MeV and the effective mass
$m_N^*/m_N$ from 0.664 to 0.7 to determine a modified parameter set G2$^*$
as given in Table I of Ref.~\cite{Sharma}. For the hyperon couplings,
they assumed that all the hyperons in the octet have the same couplings.
Using the G2$^*$ parameter set in the E-RMF model for the hadronic phase
and $B^{1/4}=170\;\mathrm{MeV}$ in the UQM model for the
quark phase, they obtained the onset of the mixed phase at $\sim 1.3 \rho_0$,
while the appearance of the mixed phase starts at $\sim 2.4 \rho_0$
using the CFL model with $B^{1/4}=188\;\mathrm{MeV}$ for the quark phase.
In our cases, the mixed phase occurs at $\sim 3.3 \rho_0 $ ($8.6 \rho_0$)
using the NL3 (TM1) model with the weak $YY$ interaction for the hadronic phase
and the NJL model for the quark phase. For simplicity, we have not considered
the color-flavor locked phase in the present work.
By comparing all these results, we conclude that the deconfinement phase
transition is very sensitive to both the hadronic EOS and the quark EOS,
whether or not the deconfined quark matter appears in the center
of neutron stars depends on the models adopted in the calculation.

\section*{Acknowledgments}

This work was supported in part by the National Natural Science Foundation
of China (No. 10675064) and the Specialized Research Fund for the Doctoral
Program of Higher Education (No. 20040055010).

\newpage