The forward-backward asymmetry induced CP asymmetry in ${{\overline{B}}^{0}\rightarrow K^{-}\pi^{+}\pi^{0}}$ in phase space around the resonances ${{\overline{K}}^{*}(892)^{0}}$ and ${{\overline{K}}^{*}_{0}(700)}$

CP violation (CPV) has played an important role in particle physics since its first discovery in the decay of $ K^{0}_{L} $ mesons in 1964 [1]. In addition to strange meson systems, CP violation has been observed in B, $ B_s $, and D meson systems [2−5], and in the $ \Lambda_b^0 $ decay process $ \Lambda_b^0\to p K^- \pi^+\pi^- $ [6], all of which are consistent with the description of the the standard model (SM) of particle physics, within which CPV is described by an irreducible complex phase in the Cabibbo-Kobayashi-Maskawa quark mixing matrix [7, 8]. According to Sakharov's criteria [9], CPV is also an important condition to explain the baryon asymmetry of the universe (BAU). However, it is widely accepted that CPV within the SM is insufficient to explain the current BAU [10−14]. Hence, a better understanding of CPV is of great importance both to particle physics and cosmology.

Owing to the increasing amount of data, CPV can now be investigated in some multi-body decays of bottom and charmed hadrons, in which CP asymmetries (CPAs) with great detailed structures were observed in different regions of the phase space [15−20]. Some of the regional CPAs in charged B meson multi-body decays can be as high as 80%, which represent the largest CPAs ever observed in laboratories. In contrast, the overall CPAs of these decays are considerably smaller.

The structures of regional CPAs in phase space in the three-body decay of charged B mesons indicates that the interference between different intermediate resonances play an important role in generating the large regional CPAs. Take the decay $ B^\pm\to\pi^\pm\pi^+\pi^- $ as an example. The results of the LHCb collaboration showed that the large regional CPA is located around the intermediate resonances $ \rho(770)^0 $. The behavior of the large regional CPA can be understood as a consequence of the interference effect between the $ \rho(770)^0 $ resonance and some s-wave structures [21, 22]. Since $ \rho(770)^0 $ has spin-1, meaning that the $ \pi^+\pi^- $ system originating from $ \rho(770)^0 $ with the residence pion are in p-wave, a forward-backward asymmetry (FBA) in the distribution of final particles is produced because of the interfering effects. Consequently, the CPV corresponding to this interfering effect, which is the dominant contribution to the large regional CPAs, can also result in CPAs corresponding to the FBA (FB-CPA) [23, 24].

In this study, we examine a similar interfering effect, which is the interference of amplitudes corresponding to $ \overline{K}^\ast(892)^0 $ and $ \overline{K}_0^\ast(700) $ in the decay channel of $ {\overline{B}}^{0}\rightarrow K^{-}\pi^{+}\pi^{0} $. This decay channel has been studied by the BaBar and Belle collaborations; no CPV is established, mainly because of the low statistics [25−28]. This study shows that the interference of $ \overline{K}^\ast(892)^0 $ and $ \overline{K}^\ast_0(700) $ may result in CPV, which has the potential to be detectable with more data through the measurements of observables such as FB-CPA. There is very strong evidence that the interference of $ \overline{K}^\ast(892)^0 $ and $ \overline{K}^\ast_0(700) $ in the baryon-anti-baryon-production decay channel $ \overline{B}^0\to p\bar{p} K^- \pi^+ $ can lead to large CPAs [29]. Moreover, the contribution of the interference of $ \overline{K}^\ast(892)^0 $ and $ \overline{K}^\ast_0(700) $ in the $ \Lambda_b^0 $ decay process $ \Lambda_b^0\to p K^- \pi^+\pi^- $ [6] may be related to the observed regional CPAs $ (5.3\pm1.3\pm0.2)\ $%, which deserves further investigation.

The remainder of this paper is organized as follows. In Sec. II, we present a brief introduction of FBA and FB-CPA. In Sec. III, we present the decay amplitudes of the decay channel $ {{\overline{B}}^{0}\rightarrow K^{-}\pi^{+}\pi^{0}} $, focusing on the phase space region where both the intermediate resonances $ \overline{K}^{*}(892)^{0} $ and $ \overline{K}^{*}_{0}(700) $ dominate. In Sec. IV, we present the numerical results and discussion. Finally, in Sec. V, we provide the summary and conclusions.

For a three-body decay process $ H\to M_1 M_2 M_3 $, with H being a heavy pseudo-scalar meson, and $ M_1,M_2,M_3 $ being three light pseudo-scalar mesons, we can parameterize the decay amplitude as

where $ P_{l} $ is the Legendre polynomial of order l, with l being the quantum number of the angular momentum between $ M_3 $ and the $ M_1M_2 $ system; θ is the helicity angle, which is defined as the angle between the momentum of $ M_1 $ and H (or $ M_3 $) in the center-of-mass (C.O.M.) frame of the $ M_1M_2 $ system, and $ {\cal{A}}_l $ is the amplitude of the l-th wave. Figure 1 presents an illustration of the helicity angle θ for the case $ B^0\to K^+\pi^-\pi^0 $, as this is the decay channel that we will focus on. Note that l is also the spin quantum number of the $ M_1M_2 $ system.

Figure 1.  Definition of θ in the $ B^{0}\rightarrow K^{+}\pi^{-}\pi^{0} $ decay channel.

The FBA in the decay channel $ H\to M_1 M_2 M_3 $ can be defined as the difference between the event yields for the $ M_3 $ flying forward ($ \cos\theta>0 $) and backward ($ \cos\theta<0 $) in the C.O.M. frame of the $ M_1M_2 $ systerm, which can be expressed as

where the notion ''$ \langle \cdots\rangle $'' represents the integration over phase space other than $ \cos\theta $. It can be shown that [24]

where $ { f_{lk} \equiv \int_0^1 P_{l}P_{k}d\cos\theta = \dfrac{( - )^{(l + k + 1)/2}l!k!}{2^{l + k - 1}(l - k)(l + k + 1)[(l/2)!]^2\{[(k - 1)/2]!\}^2}} $. From the above equation, one can see that FBA is caused by the real part of the interference between even- and odd-wave amplitudes $ \Re( \langle a_{l}a_{k}^{\ast}\rangle) $. In the simplest case, the amplitudes are dominated by the S- and P-wave amplitudes ($ l=0 $, and 1, respectively). Thus, the FBA can be expressed as

The FBA include CP asymmetry (FB-CPA) can be defined as ¹

where $ \overline{A^{\rm FB}} $ represents the FBA of the $\rm CP$ conjugate process.

When the invariant mass of the $ {K^-\pi^+} $ system is approximately the masses of $ \overline{K}^{*}_{0}(700) $ and $ \overline{K}^{*}(892)^{0} $, the decay will be dominated by the cascade decays $ \overline{B}^{0}\rightarrow \overline{K}^{*}_{0}(700) (\to K^{-}\pi^{+})\pi^{0} $ and $ \overline{B}^{0}\rightarrow \overline{K}^{*}(892)^{0} (\to K^{-}\pi^{+})\pi^{0} $. The decay amplitudes can then be expressed as

where $ {\cal{A}}_{K^{*}_{0}} $ and $ {\cal{A}}_{K^{\ast 0}} $ are the decay amplitudes for the two cascade processes, and δ is the relative phase between them. We can further isolate the Breit-Wigner factor so that the amplitudes $ {\cal{A}}_{{K}^{\ast}_{0}} $ and $ {\cal{A}}_{{K}^{\ast 0}} $ can be re-expressed as

for $ X={{K}^{\ast}_{0}} $ and $ {{K}^{\ast 0}} $, where s is the invariant mass squared of the $ K^-\pi^+ $ system, and

where $ F_{0}^{B\to K^{\ast 0}} $, $ A_{0}^{B\to K^\ast} $, $ F_{0}^{B\to K^\ast_0} $, $ F_0^{B\to\pi} $, and $ F_{1}^{B\to \pi} $ are the form factors for the corresponding transitions $ \overline{B}^0\to \overline{K}^\ast(892)^0 $, $ \overline{B}^0\to \overline{K}^\ast_0(700) $, and $ \overline{B}^0\to \pi^0 $;$ f_{K^{\ast 0}} $, $ f_{K^{*}_{0}} $, and$ f_\pi $ are the decay constants; $ s_{\pi^{0}\pi^{-},\text{max}} $ and $ s_{\pi^{0}\pi^{-},\text{min}} $ are the maximum and minimum values of $ s_{\pi^{0}\pi^{-}} $, respectively; and $ g_{K^{*}_{0} K\pi} $ and $ g_{K^{\ast 0} K\pi} $ are the strong coupling constants. All the $ a_{i} $'s are built up from the Wilson coefficients $ c_{i} $'s and take the form $ a_{i}=c_{i}+c_{i+1}/N_{c}^{\text{eff}} $ for odd i and $ a_{i}=c_{i}+c_{i-1}/N_{c}^{\text{eff}} $ for even i, where $ N_{c}^{\text{eff}} $ is the effective color number whose deviation from the color number $ N_c=3 $ measures the non-factorizable contribution. The strong coupling constants $ g_{K^{\ast 0} K\pi} $ and $ g_{K^{*}_{0} K\pi} $ are determined from the measured partial decay widths through the relations [30]:

where $ \Gamma_{K^{*}_{0} K\pi} $ and $ \Gamma_{K^{*0} K\pi} $ are the decay widths for $ K^{*}_{0}(700)\to K\pi $ and $ {K^{*}(892)^{0}\to K\pi} $, respectively, and $ p_c $ is the momentum of K or π in the rest frame of $ K^{*}_{0}(700) $ or $ K^{*}(892)_{0} $.

Figures 2, 3, and 4 present the AFBs of $ B^0\to K^+\pi^-\pi^0 $ and its $\rm CP$-conjugate process $ A^{\rm FB} $ and $ \overline{A^{\rm FB}} $, and the corresponding FB-CPAs $ A^{\rm FB}_{\rm CP} $ for $ N_c^{\text{eff}}=1 $, 2, and 3, respectively, for various values of the strong phase δ, where the inputs of various parameters are listed in Table 1. One can see from all these figures that $ A^{\rm FB} $ and $ \overline{A^{\rm FB}} $ tend to take large values when s is approximately the mass-squared of $ \overline{K}^\ast(892)^0 $ and $ \overline{K}^\ast_0(700) $, implying that the interference effect of $ \overline{K}^\ast(892)^0 $ and $ \overline{K}^\ast_0(700) $ induces large FBAs. Note that this behaviour is universal to different values of $ N_c^{\text{eff}} $. On the other hand, the CP violation observables FB-CPAs are much more sensitive to the values of $ N_c^{\text{eff}} $. By comparing these three figures, one can immediately see that non-zero FB-CPAs lie in the regions of $ \overline{K}^{\ast}(892) $ and $ \overline{K}^\ast_0(700) $ for $ N_c^{\text{eff}}=1 $ and 2, as can be seen from Figs. 2 and 3.The FB-CPAs are negligibly small for $ N_c^{\text{eff}}=3 $ from Fig. 4, indicating the important impact of non-factorizable contributions, which are characterized by $ N_c^{\text{eff}} $, to the FB-CPAs.

Figure 2.  s-Dependence of $ A^{\rm FB} $, $ \overline{A^{\rm FB}} $, $ A^{\rm FB}_{\rm CP} $, and $ A^{\rm FB}_{\text{ave}} $ of the decay channel $ B^{0}\rightarrow K^{+}\pi^{-}\pi^{0} $ for $ \delta=0 $, $ \pi/2 $, π, and $ 3\pi/2 $, respectively, and for $ N_{c}^{\text{eff}}=1 $. The range of s is taken from 0.4 GeV/$ c^{2} $ to 1.2 GeV/$ c^{2} $. The dotted, dashed, dash-dotted, and solid lines represent $ A^{\rm FB} $, $ \overline{A^{\rm FB}} $, $ A^{\rm FB}_{\text{ave}} $, and $ A^{\rm FB}_{\rm CP} $, respectively. The shadowed region indicate the location of the vector resonances $ K^{*}(892)^{0} $ ($ (m_{K^{*}(892)^{0}}- $$ \Gamma_{K^{*}(892)^{0}})^{2}<s<(m_{K^{*}(892)^{0}}+\Gamma_{K^{*}(892)^{0}})^{2} $).

Figure 3.  Same as Fig. 2 but for $ N_{c}^{\text{eff}}=2 $.

Figure 4.  Same as Figs. 2 and 3 but for $ N_{c}^{\text{eff}}=3 $.

Figures 2, 3, and 4 also show that the behavior of FB-CPA, as a function of s, strongly depends on the value δ. One important property is that for $ \sqrt{s} $ below and above the mass of the vector resonance $ \overline{K}^\ast(892)^0 $, when δ takes values of approximately $ 0 $ or π, the FB-CPA tends to take opposite values. When δ takes values of approximately $ \pi/2 $ or $ 3\pi/2 $, the FB-CPA does not change the sign. This behavior can be understood as follows.

For simplicity, we will denote $ {\overline{K}^\ast_0(700)} $ and $ \overline{K}^\ast(892)^0 $ as $ {S} $ and $ {P} $, respectively, to reflect that the angular momentum of the $ {\overline{K}^\ast_0(700)} \pi^0 $ and $ {\overline{K}^\ast(892)^0}\pi^0 $system are $ l=0 $ (S) and $ l=1 $ (P), respectively. By isolating the Breit-Wigner factors, the interference term in $ A^{\rm FB} $ can be expressed as

where

Note that $ \Gamma_S $ is considerably larger than $ \Gamma_P $. Meanwhile, since we focus mainly on the region when s is approximately the mass-squared of S and P, we have $ \Gamma_s \gg \sqrt{|s-m_S^2|} $ and $ \Gamma_s \gg \sqrt{|s-m_P^2|} $. This means that we can safely approximate Eq. (11) as

where we collectively denote the three small quantities, $ \Gamma_P/\Gamma_S $, $ \sqrt{\left|s-m_S^2\right|}/\Gamma_S $, and $ \sqrt{\left|s-m_P^2\right|}/\Gamma_S $, as $ \epsilon $.

Up to the order $ {\cal{O}}(\epsilon) $, $\Re( {\cal{A}}_{S}^\ast {\cal{A}}_{P} \, {\rm e}^{{\rm i}\delta})$ can be further split into two parts:

where

and

To explain the motivation for this split, notice that if δ is the only dominate strong phase, one will have $ \tilde{{\cal{A}}}_{S}^\ast\approx\tilde{{\cal{A}}}_{S}^{\rm CP} $ and $ \tilde{{\cal{A}}}_{P}^\ast\approx\tilde{{\cal{A}}}_{P}^{\rm CP} $, where $ \tilde{{\cal{A}}}_{S}^{\rm CP} $ and $ \tilde{{\cal{A}}}_{S}^{\rm CP} $ are the corresponding amplitudes for the $\rm CP$-conjugate processes. As a result, one has $ \Im (\tilde{{\cal{A}}}_{S}^\ast\tilde{{\cal{A}}}_{P})\approx-\Im (\tilde{{\cal{A}}}_{S}^{CP\ast}\tilde{{\cal{A}}}_{P}^{\rm CP}) $ and $ \Re (\tilde{{\cal{A}}}_{S}^\ast\tilde{{\cal{A}}}_{P})\approx \Re (\tilde{{\cal{A}}}_{S}^{CP\ast}\tilde{{\cal{A}}}_{P}^{\rm CP}) $. Consequently one has $ \Delta\approx-\Delta^{\rm CP} $ and $ \Sigma\approx\Sigma^{\rm CP} $, where $ \Delta^{\rm CP} $ ($ \Sigma^{\rm CP} $) is the same as Δ (Σ), except that the amplitudes are replaced by the CP-conjugate ones. This means that if δ is the only dominate strong phase, FB-CPA will be dominated by Δ such that

Now the different behaviours of FB-CPA in the four subfigures of Figs. 2, 3, and 4 can be understood. The two terms in Δ of Eq. (15) have quite different behaviour. The second term changes sign when s passing through $ m_P^2 $ because of the presence of the factor $ s-m_P^2 $, while the first term does not. When δ take values around $ 0 $ or π, the second term dominates, so that the FB-CPA tends to change sign when s passing through $ m_P^2 $; when δ take values around $ \pi/2 $ or $ 3\pi/2 $, the first term dominates, so that the FB-CPA tends to not change sign throughout the interference region ².

Note that in Figs. 2, 3, and 4, we also present the $\rm CP$-averaged FBA, which is defined as

It can be easily shown that the $\rm CP$-averaged FBA is dominated by Σ, $ A^{\rm FB}_{\text{ave}}\sim 2\Sigma $, if δ is the only dominate strong phase. As can be seen from Eq. (16) that $ A^{\rm FB}_{\text{ave}} $ tends to change sign for δ taking values around $ \pi/2 $ or $ 3\pi/2 $, while the sign of $ A^{\rm FB}_{\text{ave}} $ remains unchanged for δ taking values around $ 0 $ or π, in contrast to the situation of $ A^{\rm FB}_{\rm CP} $ ³.

The aforementioned behavior is also indicated in Fig. 5, in which the δ-dependence of FB-CPAs for $ N_c^{\text{eff}}=1 $ in different phase-space regions are $ m_P-\Gamma_P \lt s \lt m_P $, $ m_P<s<m_P+ \Gamma_P $, and $ m_P-\Gamma_P< s<m_P+\Gamma_P $, denoted as regions L, R, and $ L+R $, respectively. Figure 5 shows that when δ takes values of approximately 0 or π, the value of $ A^{\rm FB}_{\rm CP}(L) $ is quite different from that of $ A^{\rm FB}_{\rm CP}(R) $, and they tend to take opposite values. When δ takes values of approximately $ \pi/2 $ or $ 3\pi/2 $, the difference between the values of $ A^{\rm FB}_{\rm CP}(L) $ and $ A^{\rm FB}_{\rm CP}(R) $ is small.

Figure 5.  FB-CPAs of three different regions in phase space, L ($ m_P-\Gamma_P<s<m_P $), R ($ m_P<s<m_P+ \Gamma_P $), and $ L+R $ ($ m_P-\Gamma_P<s<m_P+\Gamma_P $), which are denoted as $ A^{\rm FB}_{\rm CP}(L) $, $ A^{\rm FB}_{\rm CP}(R) $, and $ A^{\rm FB}_{\rm CP}(L+R) $, respectively, in the maintext and are represented by dotted, dashed, and solid lines, respectively, as functions of the strong phase δ for $ N_c^{\text{eff}}=1 $. The shadowed area indicates the range of δ preferred by the data in Ref. [27].

A Dalitz analysis of the decay $ B^0 (\overline{B^0})\to K^\pm\pi^\mp \pi^0 $ has been performed by the BaBar collaboration [27]. The results of Fig. 7 in Ref. [27] indicate that the $\rm CP$-averaged FBA $ A^{\rm FB}_{\text{ave}} $ tends to take positive (negative) values when s is below (above) $ m_P^2 $. A closer comparison of the results of Fig. 7 in Ref. [27] with the $\rm CP$-averaged FBA show that the strong phase δ assumes values in the range $[3\pi/2, \; 5\pi/3]$. This means that the subfigures (d) in Figs. 2, 3, and 4 are more favorable by the data than the other three. As can be seen from Fig. 5, for δ taking values in the range $[\pi/2, \; 2\pi/3]$, the corresponding FB-CPA in the region $ L+R $ can take values as large as approximately 35% for $ N_c^{\text{eff}}=1 $, which are likely accessible for Bell and Bell-II.

In the above analysis of the interference effects of $ \overline{K}^\ast(892)^0 $ and $ \overline{K}_0^\ast(700) $ in $ \overline{B}^0\to K^-\pi^+ \pi^0 $, we used the condition that the strong phase δ is the only dominate strong phase. In other words, there are no large physical strong phases within the amplitudes for each of the cascade decays, which means that the direct CP asymmetries for the two-body decay processes $ B^0\to K^\ast(892)^0\pi^0 $ and $ B^0\to K_0^\ast(700)\pi^0 $ are negligibly small. This is consistent with BaBar's experimental results [27, 28]. This is true in the naive factorization approach for weak decay processes, which is what we have adopted in this study. Beyond the naive factorization, it is also possible that there are large strong phases within the amplitudes for the cascade decays. In this type of situation, non-negligible CPAs are expected in the two-body decays, and the above analysis will be contaminated. It should be noted that whether this is the case can be examined by comparing FB-CPA and the corresponding direct-CPA-subtracted FB-CPA. If there are obvious differences between them, it means that at least one of the the direct CPAs for the two two-body decays is relatively large. If we relaxed the assumption of the dominance of the strong phase δ, a combined analysis of the behavior of FBA, CP-averaged FBA, FB-CPA, direct-CPA-subtracted FB-CPA, and regional CPAs can also give us a more comprehensive understanding of the underlying dynamics of CP violation in the interference region.

Our analysis here is in fact quite general and can be applied to other three-body decay processes. Take the well studied $ B^\pm\to\pi^\pm\pi^+\pi^- $ channel as an example. It has been shown experimentally that the interference of $ \rho(770)^0 $ and $ f_0(500) $ has great impact on CPV in this decay channel. The regional CPAs change sign when the invariant mass squared of the $ \pi^+\pi^- $ system $ s_{\pi^+\pi^-} $ pass through the mass squared of $ \rho(770)^0 $ [18]. This behavior can be explained similar to that in Eq. (11), since this kind of term can also contribute to regional CPAs. We strongly suggest that our experimental colleagues perform the combined analysis of observables in three-body decays of heavy mesons in the interference region of intermediate resonances.

In this paper, we present an analysis of CPVs induced by the interference effect of the intermediate resonances $ \overline{K}^{*}(892)^{0} $ and $ \overline{K}^{*}_{0}(700) $ in three-body decay $ \overline{B}^{0}\rightarrow K^{-}\pi^{+}\pi^{0} $, based on the naive factorization approach for the weak two-body decay processes $ \overline{B}^{0}\rightarrow \overline{K}^{*}(892)^{0}\pi^{0} $ and $ \overline{B}^{0}\rightarrow \overline{K}^{*}_{0}(700)\pi^{0} $. The interference between $ \overline{K}^{*}(892)^{0} $ ($ J^P=1^- $) and $ \overline{K}^{*}_{0}(700) $ ($ J^P=0^+ $) can generate significant FBA. This FBA induces potentially measurable CP violations, with FB-CPA values reaching approximately 35% for phenomenologically reasonable parameters.

According to the analysis, the non-factorizabe contributions relative to the strong phase δ between the amplitudes corresponding to the cascade decays play an important role in the behavior of FBA and FB-CPA. We present a general analysis of the correlation between the non-factorizabe contributions, the values of the strong phase δ, and the behavior of FBAs and FB-CPAs. We discover a model-independent correlation between FBA and FB-CPA that persists across parameter variations.

The results of this study establish a framework for the inclusion of the $ \overline{K}^{*}(892)^{0}-\overline{K}^{*}_{0}(700) $ interference effect when studying CPV in multi-body decays, with both theoretical significance and immediate experimental relevance. This framework explains similar CPV observations in $ \Lambda_b\to p K^- \pi^+\pi^- $ and $ \overline{B}^0 \to p \overline{p} K^-\pi^+ $ decays. The analysis can also apply to other multi-body decays of bottom and charmed hadrons.