text:
# An X-ray Atlas of Groups of Galaxies

## 1 Introduction

Most galaxies in the local universe, including our own Milky Way, are members of poor galaxy groups (Geller & Huchra, 1983; Tully, 1987; Nolthenius & White, 1987). The realization that many groups are X-ray sources has led to considerable interest in these systems over the last decade (Mulchaey et al., 1993; Ponman & Bertram, 1993; David et al., 1994; Henry et al., 1995; Pildis, Bregman & Evrard, 1995; Doe et al., 1995; Saracco & Ciliegi, 1995; Mulchaey et al., 1996a; Ponman et al., 1996; Mulchaey & Zabludoff, 1998; Helsdon & Ponman, 2000a). The X-ray emission in groups is typically extended on scales of hundreds of kiloparsecs. X-ray spectroscopy suggests the emission mechanism is most likely a combination of thermal bremsstrahlung and line emission. This interpretation requires that the entire volume of groups be filled with a hot, low-density gas often referred to as the intragroup medium in analogy to the intracluster medium found in richer systems.

The existence of an intragroup medium is important for many reasons. The presence of a hot gas halo indicates that many groups are likely real, physical systems and not simply chance superpositions or large-scale filaments viewed edge-on. Assuming the intragroup gas is in hydrostatic equilibrium, the total mass of the group can be estimated (Mulchaey et al., 1993; Ponman & Bertram, 1993; Pildis, Bregman & Evrard, 1995; Mulchaey et al., 1996a). Based on ROSAT data, the typical group mass is approximately one-tenth the mass of a cluster like Virgo. However, because the number density of X-ray detected groups is considerably higher than the number density of clusters like Virgo, the contribution of X-ray detected groups to the total mass density of the universe is comparable to or greater than that of rich clusters. A comparison of the mass in galaxies and intragroup gas to the total group mass indicates that the known baryonic components typically account for only 10–20% of the total mass (Mulchaey et al., 1993; Ponman & Bertram, 1993; Henry et al., 1995; Pildis, Bregman & Evrard, 1995; Mulchaey et al., 1996a). This implies that groups are dominated by dark matter. Extrapolating the X-ray surface brightness profiles to the virial radius suggests that most of the baryonic mass in groups is in the intragroup medium. In fact, the intragroup medium may be the dominant baryonic component in the local universe (Fukugita, Hogan & Peebles, 1998).

The intragroup medium may also hold important clues into the formation and evolution of large-scale structure. Numerical simulations indicate that in the absence of non-gravitational heating, the density profiles of groups and clusters should be nearly identical (Navarro, Frenk & White, 1997). In this simple scenario, the relationships between X-ray luminosity (LX), X-ray temperature (T) and optical velocity dispersion ($`\sigma `$) are expected to be similar for groups and clusters. There is now considerable evidence for departures from such uniformity (Ponman, Cannon & Navarro, 1999; Lloyd-Davies, Ponman & Cannon, 2000; Mahdavi et al., 2000; Helsdon & Ponman, 2000a). The interpretation of these departures is currently a topic of much debate. One possibility is that preheating of the intragroup medium by an early generation of stars can explain the scaling relationships (Cavaliere, Menci & Tozzi, 1998; Ponman, Cannon & Navarro, 1999; Pierre, Bryan & Gastaud, 2000; Finoguenov, David & Ponman, 2000; Bialek, Evrard & Mohr, 2001; Borgani et al., 2001). Alternative explanations include heating by active galaxies (Valageas & Silk, 1999; Fujita & Takahara, 2000; Yamada & Fujita, 2001), internal heating mechanisms (Loewenstein, 2000) and the possibility that galaxy formation is more efficient in groups than in clusters (Bryan, 2000).

Most of our current understanding of the intragroup medium is based on observations of groups with ROSAT. The low internal background, large field of view, and good sensitivity to soft X-rays made the ROSAT PSPC detectors ideal instruments for studies of nearby groups. Unfortunately, large systematic surveys of well-selected groups were not performed during the limited lifetime of the PSPC detectors. However, a large number of groups were observed serendipitously. Although ROSAT results have been reported for many of these groups, a significant fraction of the groups in the ROSAT database have not been previously published. Furthermore, the derived X-ray properties of groups are somewhat sensitive to assumptions made during the data reduction and analysis. For this reason, different authors often derive vastly different results from the same ROSAT dataset. Therefore, constructing large samples from the literature can be problematic. In an attempt to produce a more uniform sample, we have reanalyzed the ROSAT PSPC data for all poor groups previously reported in the literature along with a large number of nearby groups that have not previously been published.

We describe the sample selection in §2 and the data reduction in §3. For those groups with detected diffuse emission, we have extracted an X-ray spectrum (§4). Fits to the surface brightness profiles of the groups are described in §5. In §6 we examine the morphological composition of the X-ray detected groups. Finally, a summary is provided in §7.

## 2 Sample

The ROSAT mission consisted of two main scientific phases, a six-month, all-sky survey with a mean exposure time of approximately 400 seconds and longer pointings of individual targets (the so-called ‘pointed mode’). While the all-sky survey data are deep enough to detect the most X-ray luminous groups (Ebeling, Voges & Böhringer, 1994; Henry et al., 1995; Mahdavi et al., 1997, 2000), it is not possible to carry-out detailed spatial and spectral studies with this dataset. As the ‘pointed mode’ observations allow more detailed studies, we restrict our analysis to groups observed during this phase of the ROSAT mission.

We select our groups from a number of different sources. We start by including all groups that were in previous ROSAT ‘pointed-mode’ studies. Many of the groups come from the compact group catalog of Hickson (1982) or from the loose group studies of Mulchaey et al. (1996a), Mulchaey & Zabludoff (1998) and Helsdon & Ponman (2000a). Additional targets were found by cross-correlating the ROSAT observation log with the positions of optically-selected groups in the catalogs of Huchra & Geller (1982), Geller & Huchra (1983), Maia, Da Costa & Latham (1989), Nolthenius (1993) and Garcia (1993). These catalogs also include richer galaxy systems (i.e. clusters). For the purpose of this paper, we only include “groups” with velocity dispersions less than 600 km/s or an intragroup medium temperature less than 2 keV. A few of the systems we have encountered satisfy one, but not the other criterion. We have opted to include these systems in our sample. In general, we only include groups whose centers fall within 20 of the center of the ROSAT field. The only exceptions are a few Hickson Compact Groups, which were previously included in Ponman et al. (1996) and are included here for comparison. We have not included HCG 73 in our sample, as it is right at the edge of the ROSAT field. We also eliminated any groups that are projected in front of or behind rich clusters.

The final sample of 109 groups is given in Table 1. The name of each group is given in the first column, followed by the Right Ascension and Declination (columns 2 and 3), the number of cataloged galaxies (column 4), redshift (column 5) and velocity dispersion (column 6). For groups with extended X-ray emission, the right ascensions and declinations are generally given for the peak of the X-ray emission, which was determined in each case from the ROSAT image smoothed with a Gaussian profile of width 30′′. For a few X-ray detected groups, no clear flux peak exists. In these cases, the coordinates refer to the approximate center of the emission morphology. For the non-detected groups, we adopt the coordinates given in the optical group catalogs. Column 7 gives the spiral fraction, defined as the fraction of cataloged members that are late-type galaxies (i.e. spirals and irregulars). We note that the quantities listed in Table 1 have not been calculated to a single minimum absolute magnitude. For most of the groups, the number of members, velocity dispersion and spiral fractions are based on the few brightest ($``$ L) galaxies in the original redshift surveys. However, a few groups have morphologies and membership data down to significantly lower luminosities (Zabludoff & Mulchaey, 1998; Mahdavi et al., 1999). Therefore, some caution should be used when comparing the optical properties of individual groups. Column 8 gives the references for the quantities in columns 2-7. The last column gives the cross references for the group in optical group catalogs.

## 3 Data Reduction and Analysis

To search for a hot intragroup medium, we follow the reduction method outlined in Mulchaey & Zabludoff (1998). First, to remove times of high particle background, we discard any data taken when the master veto rate is greater than 170 counts s-1. For each ROSAT observation, six images are created corresponding to the R2-R7 energy bands given in Snowden et al. (1994). Each of these images is corrected for vignetting using energy-dependent exposure maps. These individual images are combined to produce two final images; one corresponding to the “standard” energy range $``$ 0.45–2.05 keV used in our previous analysis (Mulchaey et al., 1996a; Mulchaey & Zabludoff, 1998) and one that includes the “softer” energy bands ($``$ 0.15-2.05 keV). We created the softer energy band images with the hope of detecting very cool intragroup gas not visible in the standard images. However, in all cases where emission was found in the softer images, it was also detected in the standard image. Given that the background level is much higher in the softer energy bands, all the analysis presented in this paper is based on the standard images.

Before searching for diffuse intragroup gas, we remove emission associated with point sources. Point sources are identified using the task “DETECT” in the Extended Object and X-ray Background Analysis software (Snowden et al., 1994). We note that in the present context, “point source” refers to a source that appears point-like at the resolution of the ROSAT PSPC (FWHM $``$ 30′′ on-axis and considerably worse off-axis). Emission from point sources is removed by excluding a circular region around each source with a radius 1.5 times the radius that encircles 90% of the source flux. This exclusion radius corresponds to 1.5 for sources on-axis (Hassinger et al., 1992). In a few cases, it was necessary to use a slightly larger aperture to remove the flux of more extended sources in the field (such as a very nearby galaxy or a background cluster of galaxies). In most cases where diffuse emission is detected, the emission is centered on a luminous elliptical or S0 galaxy. While the “DETECT” software will usually identify the peak of the extended emission as a “point source” we do not exclude the emission in these cases.

A small fraction of the groups were observed by ROSAT in multiple pointings. In these cases, we reduced the data for each pointing as described above and then combined the images before running the task “DETECT” .

Once the point source emission is removed, the unsmoothed images are examined to determine if a diffuse emission component is present. In addition to a visual inspection, we also create an azimuthally-averaged surface brightness profile for each group to determine if there is excess emission above the background level. Finally, we estimate the excess group counts by adding up the total counts within a radius of 200 h$`{}_{}{}^{1}{}_{100}{}^{}`$ kpc of the group center and subtracting the vignetting-corrected background counts estimated from an outer region of the field. In all cases where the visual inspection implied the presence of diffuse emission, both the surface brightness profile and excess count rate are consistent with the existence of intragroup gas. Furthermore, in all cases where the surface brightness profile suggested the presence of diffuse gas, the visual inspection also indicated a diffuse component.

The results of our diffuse emission search and the spectral properties of the diffuse gas are summarized for each group in Table 2. The second column in Table 2 gives the total useful ROSAT exposure time in seconds. The third column indicates whether diffuse emission was detected or not. Groups that clearly show evidence for diffuse emission are labeled “DE” (for diffuse emission), while the non-detected groups are labeled “UL” (for upper limit). A few of the groups that contain a diffuse component also contain a significant contribution from a central AGN. These hybrid objects are labeled “DE/AGN” in Table 2. For a few groups, emission is detected, but the nature of the emission is such that it might not be associated with the group or might not be truly diffuse. These cases are labeled “DE?” in column 3. In HCG 33, a very faint diffuse component is detected, but it is offset from the optical center of the group by nearly 18.5 (approximately 400 kpc). Thus, it is unclear whether this emission is associated with the group or a foreground/background object. There are several patches of diffuse emission in the field of HCG 22, but the brightest emission is centered on the background galaxy NGC 1192 and therefore most likely not associated with HCG 22. Several other groups (HCG 15, HCG 35, HCG 48, HCG 57) are clearly detected in the ROSAT image, but are so far off-axis that it is not possible to cleanly separate an intragroup gas component from emission associated with individual galaxies. Column 4 in Table 2 lists the Galactic neutral column density towards the line of site as given in Dickey & Lockman (1990), while column 5 gives the maximum radius of X-ray detection determined from examining the surface brightness profile of each group. Columns 6 and 7 give the temperature and metal abundance, respectively, derived from fits to the ROSAT spectra (see §4). The distance to each group (column 8) has been calculated from the recessional velocity with a correction for infall to Virgo and the Great Attractor (Fixsen et al., 1996). Finally, the bolometric luminosity is given in the last column in Table 2.

In Figure 1, we overlay contours of the X-ray emission on the STScI Digitized Sky Survey images for all groups that show evidence for diffuse emission. As has been noted in previous studies (Mulchaey et al., 1996a; Mulchaey & Zabludoff, 1998; Helsdon & Ponman, 2000a), the diffuse X-ray emission in groups is almost always peaked on an elliptical or S0 galaxy. The emission associated with the central galaxy has been included in Figure 1 except for the cases where an AGN component clearly dominates. All other emission due to point sources in the field has been removed.

Most of the groups in our survey have round, symmetrical morphologies. These morphologies are consistent with the groups being virialized. However, some groups have very irregular, clumpy morphologies that are not centered on any particular galaxy (e.g. HCG 16, HCG 37, SHK 202, NGC 7777). These groups are likely not virialized (Zabludoff & Mulchaey, 1998; Dos Santos & Mamon, 1999). Five groups have clearly defined bi-modal X-ray distributions (NGC 507, NGC 1407, NGC 3607, NGC 4065 and NGC 7619). In all these cases, the second X-ray peak is also centered on a luminous early-type galaxy. These objects could be groups in the process of merging. Several of the high redshift X-ray groups discovered in the deep Chandra pointing of the Hubble Deep Field North also show evidence for bi-modal X-ray morphologies (Bauer et al., 2002). Although the number of high redshift groups is too small to make any concrete statements, the HDF North data suggests that bi-modal groups may be more common at high redshift (bi-modal groups make up less than 10% of our X-ray detected groups). This idea should be testable in the near future as many more high redshift groups are discovered in deep Chandra and XMM-Newton pointings.

## 4 Spectral Analysis

For each group with a diffuse emission component, we have attempted to extract a spectrum. The spectra are extracted using a radius that corresponds to the maximum radius of detection determined from examining the surface brightness profile of each group (RX; see column 5 in Table 2). Point sources are excluded from the spectrum using the exclusion radii described above (§3). The vignetting-corrected background is estimated in most cases from an annulus with inner radius 36 and outer radius 42. In a few cases, a different background region is used to avoid the presence of a luminous cluster of galaxies.

To derive the spectral properties of the diffuse gas, we fit the spectra using the software package XSPEC. For those groups with multiple ROSAT pointings, we fit the individual spectra from each pointing simultaneously. We fit the spectra with an absorbed MEKAL plasma model (Mewe, Gronenschild & van den Oord, 1985; Kaastra & Mewe, 1993; Liedahl, Osterheld & Goldstein, 1995). We fix the absorbing column at the Galactic value given in Dickey & Lockman (1990). For most of the groups, we are unable to constrain the metallicity of the gas with the ROSAT data. In these cases, we fix the gas metallicity to 0.3 solar. For consistency with previous X-ray studies, we adopt the old photospheric value for the solar Fe abundance. To renormalize to the meteoritic value, our metallicity measurements should be increased by a factor of approximately $``$ 1.44. For those groups with sufficient data to fit the metallicity explicitly, we also perform spectral fits with the metallicity fixed at 0.3 solar.

The quoted errors on the temperature and metallicity measurements are at the 90% confidence level. In the UGC 1651 group, the temperature cannot be adequately constrained and no errors are given. The derived temperatures range from $``$ 0.3 keV to over 2 keV ($``$ 3.5 $`\times `$ 106K - 2.3 $`\times `$ 107 K), with the vast majority of groups centered around 1 keV ($``$ 1.2 $`\times `$ 107 K; see Figure 2). Groups cooler than $``$ 0.3 keV would be difficult to detect with ROSAT because of the high surface brightness of the X-ray background at these soft energies and heavy absorption by Galactic HI (Mulchaey et al., 1996b). Most systems above 2 keV would normally be considered clusters and were thus eliminated from our original sample. To examine how the derived temperatures depend on the assumed metallicity, we compare the derived temperatures for the groups where both a fixed metallicity and variable metallicity model were used (Figure 3). In general, the temperatures are not very sensitive to the assumed metallicity. An exception to this is the NGC 383 group. In this case, fixing the metallicity at a value of 0.3 solar leads to a temperature over twice that found in the variable abundance case.

In general, there is good agreement between the temperatures we derive and previous studies of the same groups with ROSAT. Figure 4 shows a comparison of the temperatures from our survey with the temperatures in Helsdon & Ponman (2000b). The agreement is good despite the fact that different apertures were used for the spectral extractions in many cases. In particular, the Helsdon & Ponman (2000b) temperatures for most of the Hickson Compact Groups were derived using a fixed physical size on the sky, while we use the maximum radius of detection for our extractions.

There is less agreement between our derived temperatures and those measured with the ASCA X-ray telescope. Figure 5 compares the temperatures derived from these two telescopes for all groups in our survey with published ASCA measurements. While the ROSAT and ASCA measurements are similar for cool groups, there are big differences for hotter systems. Above 1 keV, the ASCA temperatures tend to be higher than the ROSAT temperatures by as much as 50%. This trend has been previously noted by Hwang et al. (1999) and Horner (2001). Given the higher energy resolution and bandpass of ASCA, it seems likely the temperatures from this instrument are more reliable. Future observations with Chandra and XMM-Newton should allow more accurate measurements of the X-ray temperatures in hotter groups.

In Figure 6 we compare the metal abundances derived in our ROSAT survey with abundances derived from ASCA observations. While there is considerable scatter in the plot, both the ROSAT and ASCA data are consistent with sub-solar abundances in most cases. However, the reliability of metallicity measurements with these telescopes have been called into question (Bauer & Bregman, 1996; Buote, 1999, 2000).

The bolometric luminosity of each detected group was estimated from the best-fit MEKAL model assuming the distance to the group given in column 8 of Table 2. To estimate the errors on the luminosities, we varied the temperature and metallicity over their 90% confidence ranges. For those groups without a metallicity measurement, we assume a metallicity range from 0.0 to 1.0 solar. The luminosities are calculated out to the radius used in the spectral extractions. As this radius is usually a fraction of the virial radius of each group, the true bolometric luminosities for these groups could be substantially higher (Helsdon & Ponman, 2000a; Mulchaey, 2000; Horner, 2001). The bolometric luminosities for the X-ray detected groups in our sample span four orders of magnitude (from $``$ 1040 to 1044 h100-2 ergs s-1; see Figure 7). A comparison of our estimated bolometric luminosities with the common groups in Helsdon & Ponman (2000b) reveals good agreement (see Figure 8). We also compare our bolometric luminosities with luminosities derived from ASCA observations (Figure 9). In general, the ASCA luminosities are somewhat lower than the ROSAT-derived luminosities. This result is not surprising given that the ASCA luminosities are generally derived from smaller apertures.

Figure 10 plots the virial fraction versus the temperature of the system for the X-ray detected groups in our sample (groups with poorly constrained temperatures have been excluded). This plot indicates that the majority of groups are detected to less than half of the virial radius. There is also a trend for the hotter groups to be detected to a larger fraction of the virial radius (Mulchaey, 2000). This trend is important because it suggests that a larger fraction of the gas mass, and thus, X-ray luminosity is detected in the hotter systems. This trend continues to rich clusters of galaxies which are in general detected out to approximately the virial radius (Mulchaey, 2000).

For the groups that were not detected, we estimate upper limits on the X-ray luminosities using a method similar to that described in Mulchaey et al. (1996a). First, we measure the net counts (N) and rms error ($`\sigma `$) in a region 200 h-1 kpc in radius centered on each group (for a few nearby groups it was necessary to adopt a radius smaller than 200 h-1 kpc; see Table 2). We then calculate the flux a MEKAL plasma of temperature 1 keV and abundance 0.3 solar would need to produce N+3$`\sigma `$ (if N$`>`$0) or 3$`\sigma `$ (if N$`<`$ 0) counts.

## 5 Spatial Analysis

For each group with a detected diffuse component we have fit the two-dimensional surface brightness profile using a modified King function (the so-called ‘$`\beta `$-model’):

S(R)=So (1.0 + (R/Rcore)2)-3β+0.5.

The model is first convolved with the 1 keV ROSAT PSPC point spread function, and then fit to the data with So, Rcore, $`\beta `$ and the position of the center of the emission as free parameters. Following Helsdon & Ponman (2000a), we also fit both circular and elliptical fits. In the latter fits, the ellipticity and position angle are additional free parameters. The results of the elliptical fits are given in Table 3. During the fits, each free parameter was allowed to vary between a reasonable lower and upper bound. For a large number of groups, the best-fit core radius was at the lower bound value (0.1). This value is smaller than the resolution of the ROSAT PSPC. In these cases, the core radius is listed as ‘$`<`$0.1’ in Table 3. To determine what effect this lower bound might have on the other derived parameters, we fit a number of groups again without the lower bound. These fits suggest that removing the lower bound on Rcore has a minimal impact on the derived values of $`\beta `$ and ellipticity (see also the similar finding of Helsdon & Ponman (2000a)).

The goodness of each fit was estimated using $`\chi `$-squared statistics. Because the number of counts in each bin is small, we use the prescription of Gehrels (1986) to calculate the standard deviation. While the single-$`\beta `$ models fits provide an adequate description of the images in some cases, in general the fits are poor. Previous studies of X-ray luminous groups indicate that the surface brightness profiles of groups are complicated, often requiring multiple components to adequately describe the data (Mulchaey & Zabludoff, 1998; Helsdon & Ponman, 2000a). In particular, single King models tend to underestimate the flux at the very centers of luminous groups. Mulchaey & Zabludoff (1998) suggested that groups contain two components, one associated with the central galaxy (the ‘central’ component) and a second component associated with the group as a whole (the ‘extended’ component). For those groups in our sample with a sufficient number of counts ($``$ 1000), we fit the two-dimensional surface brightness profiles with a two-component $`\beta `$ model. Initially, we experimented with allowing both components to be elliptical in shape (i.e. let the ellipticity and position angle be free parameters). We found, however, that it was not possible to constrain these fits in most cases. Next, we adapted the proceedure used by Helsdon & Ponman (2000a) and allowed one of the components to be elliptical and required the other to be circular. While these models provided adequate fits to some groups, it was often necessary to restrict the number of free parameters even further in some cases. In particular, the $`\beta `$ values for the circular models often reached unphysically high values. When this occurred, we fit the surface brightness profiles again with the $`\beta `$ value of the core component fixed at a value of 1.0 (Helsdon & Ponman (2000a) encountered a similar problem in their fits; Stephen Helsdon, private communication). In those cases where a double $`\beta `$-model fit was performed, the results are given in Table 3. Note that for the double $`\beta `$-model fits, the elliptical component is always listed first (i.e. the elliptical component is identified as the ‘extended component’). In a few cases, this results in the core radius of the ‘extended component’ being smaller than the core radius of the ‘central component’.

Figure 11 shows the distribution of extended component $`\beta `$ values for our group sample (we adopt the two component fits where available). The mean value of $`\beta `$ for the sample is 0.47$`\pm 0.16`$. This value is in excellent agreement with the weighted mean that Helsdon & Ponman (2000a) find for their sample (0.46$`\pm 0.06`$) and is somewhat lower than the typical value found for rich clusters (Arnaud & Evrard, 1999; Mohr, Mathiesen & Evrard, 1999).

## 6 Detection Statistics

The existence of a hot intragroup medium in poor groups of galaxies has been firmly established over the past decade. However, the frequency of diffuse X-ray emission in groups is still rather uncertain. Most previous studies using ROSAT ‘pointed mode’ data have been restricted to rather small samples selected in a potentially biased way. For example, many of the targets in these surveys were a priori known to be bright X-ray sources based on previous detection (i.e. in the ROSAT All-Sky Survey). Furthermore, a significant fraction of the groups studied to date come from the Hickson Compact Group catalog, which may not be representative of groups in general (see, however Helsdon & Ponman (2000b)). Mahdavi et al. (2000) attempted to overcome these problems by looking for X-ray emission in the ROSAT All-Sky Survey data for a large sample of groups selected from the CfA redshift survey. Correcting for selection effects, Mahdavi et al. (2000) estimate that approximately 40% of groups are extended X-ray sources.

Our present survey represents the largest sample of groups studied to date with ROSAT PSPC ‘pointed mode’ data. Diffuse emission is detected in at least 61 of the 109 groups (56%) in our sample (not counting the 6 questionable detections) down to the detection limits of our survey. Unfortunately, our sample is plagued by the same biasing issues that effect the earlier pointed mode studies. A deep X-ray survey of a large, optically-selected group sample should be a priority for the current generation of X-ray telescopes.

Early studies of groups with ROSAT suggested that X-ray emission is largely restricted to systems with low spiral fractions (Ebeling, Voges & Böhringer, 1994; Pildis, Bregman & Evrard, 1995; Mulchaey et al., 1996a). However, Ponman et al. (1996) found that some spiral-rich groups do indeed contain X-ray emission. Figure 12 shows the distribution of spiral-fractions for our entire sample and for the X-ray detected groups. This figure is consistent with Ponman et al. (1996)’s suggestion that some spiral-rich groups are X-ray detected. Interestingly, however, all of the groups with a detectable intragroup medium contain at least one early-type galaxy. Therefore, the ROSAT data are consistent with no spiral-only groups containing a detectable hot intragroup medium.

Several explanations for the failure to detect spiral groups in X-rays have been proposed. One possibility is that spiral-only groups are not real, physical systems, but rather chance superpositions of galaxies along the sight. However, our own Local Group would be defined as “spiral-only” if it was placed at the distance typical of the groups in our survey (Zabludoff & Mulchaey, 1998). Therefore, this possibility seems unlikely. Another possibility is that the gas density is significantly lower in spiral-only groups. Lower gas density in spiral groups may in fact be consistent with expectations from recent preheating models (Ponman, Cannon & Navarro, 1999). The temperature of the gas in spiral-only groups may also be too cool to produce detectable X-ray emission (Mulchaey et al., 1996b). Based on their velocity dispersions, the virial temperatures of spiral-only groups do tend to be lower than those of their early-type dominated counterparts. A “warm” intragroup medium might produce detectable features in the far-ultraviolet or X-ray spectra of background quasars (Mulchaey et al., 1996b; Perna & Loeb, 1998; Hellsten, Gnedin & Miralda-Escude, 1998). High-ionization absorption line features consistent with cooler groups have recently been reported. For example, Fang et al. (2002) have reported the detection of an O VIII Ly$`\alpha `$ absorption line associated with a spiral group along the sightline towards PKS 2155-304. Similarly, some of the O VI systems found with HST and FUSE may be consistent with an origin in small galaxy groups (Savage et al., 2002). Further absorption studies should provide insight into the presence of warm intragroup gas in spiral-only groups.

## 7 Summary

We have searched for diffuse X-ray emission in 109 nearby galaxy groups using data taken with the ROSAT PSPC during its ‘pointed mode’ phase. We find evidence for a hot intragroup medium in approximately half of the groups in our sample. Although we detect many spiral-rich groups, none of the spiral-only groups in our survey show clear evidence for the presence of diffuse X-ray emission. Spiral-only groups may contain intragroup gas, but the temperature or gas density may be too low to produce appreciable X-ray emission.

In general, nearby groups are detected to only a fraction of the virial radius with the ROSAT PSPC. Therefore, a significant amount of the gas mass likely occurs beyond the current X-ray detection radius. Future observations with Chandra and XMM-Newton should provide further insight into the distribution of hot gas in these cosmologically-important systems.

The authors acknowledge useful conversations with Alex Athey, Steven Helsdon, Donald Horner, Trevor Ponman, Ann Zabludoff and Marc Zimer. We also acknowledge Mark Donikian for help with the figures. This project made extensive use of NASA’s HEASARC, NED and Skyview databases. Partial support for this project was provided by NASA grant NAG 5-3529.


format: text

text:
# Unitarity of the Higher Dimensional Standard Model

(February, 2003)

## Abstract

We study the unitarity of the standard model (SM) in higher dimensions. We show that the essential features of SM unitarity remain after compactification, and place bounds on the highest Kaluza-Klein (KK) level $`N_{\mathrm{KK}}`$ and the Higgs mass $`m_H^{}`$ in the effective four-dimensional (4d) low-energy theory. We demonstrate these general observations by explicitly analyzing the effective 4d KK theory of a compactified 5d SM on $`S^1/_2`$. The nontrivial energy cancellations in the scattering of longitudinal KK gluons or KK weak bosons, a consequence of the geometric Higgs mechanism, are verified. In the case of the electroweak gauge bosons, the longitudinal KK states also include a small mixture from the KK Higgs excitations. With the analyses before and after compactification, we derive the strongest bounds on $`N_{\mathrm{KK}}`$ from gauge KK scattering. Applying these bounds to higher-dimensional SUSY GUTs implies that only a small number of KK states can be used to accelerate gauge coupling unification. As a consequence, we show that the GUT scale in the 5d minimal SUSY GUT cannot be lower than about $`10^{14}`$ GeV.

preprint: hep-ph/0302263

1. Introduction

The conventional Higgs mechanism Higgs-64  provides the simplest way of perturbatively generating gauge boson masses while ensuring the unitarity of massive gauge boson scattering at high energies Smith-73 ; DM-73 ; Cornwall-74 ; LQT-77 ; Veltman-77 . Kaluza-Klein (KK) compactification KK  of extra spatial dimensions, on the contrary, can geometrically realize vector boson mass generation without invoking a scalar Higgs particle. In this geometric mechanism the longitudinal components of the massive vector bosons in the effective four-dimensional (4d) low-energy theory arise from the extra components of the higher-dimensional gauge field. The scattering of massive KK gauge bosons has recently been demonstrated to respect low-energy unitarity in generic 5d Yang-Mills theories 5DYM:2001a  up to energies inversely proportional to the square of the 5d coupling constant. In the present Letter, extending this study 5DYM:2001a , we analyze the unitarity of the standard model (SM) in $`D(=4+\delta )`$ dimensions, present a number of general observations, and derive their physical consequences.

2. Unitarity in $`D`$-Dimensions and Compactification

We begin with the study of unitarity in the uncompactified higher-dimensional SM, where the computation of the high energy scattering amplitudes is simpler than in the compactified theory. Yang-Mills theories in $`4+\delta `$ dimensions are not renormalizable. The gauge coupling constant $`\widehat{g}`$ has mass dimension $`\delta /2`$, and therefore we expect such a theory can only be an effective theory valid up to an ultraviolet (UV) cutoff $`\mathrm{\Lambda }`$ of order $`\widehat{g}^{2/\delta }`$. This effective description can only hold so long as the relevant high energy scattering amplitudes remain unitary, and we may estimate $`\mathrm{\Lambda }`$ by determining the scale at which the tree-level scattering amplitudes violate unitarity.

We start by considering the QCD sector of the $`D(=4+\delta )`$ dimensional SM, in which the gauge bosons and the Higgs doublet propogate in the extra $`\delta `$ dimensions. The analysis of $`D`$-dimensional QCD (D-QCD) is a direct application of the study of 5d Yang-Mills theory 5DYM:2001a . The Yang-Mills symmetry is $`SU(k)`$ with $`k=3`$, and the gauge Lagrangian is,

$$\begin{array}{cc}\widehat{}_{QCD}^D\hfill & =\frac{1}{2}\mathrm{Tr}(\widehat{G}_{MN}\widehat{G}^{MN}),\hfill \\ \widehat{G}_{MN}^a\hfill & =_M\widehat{G}_N^a_N\widehat{G}_M^a+\widehat{g}_s^{}C^{abc}\widehat{G}_M^b\widehat{G}_N^c,\hfill \end{array}$$
(1)
where, as noted above, the $`D`$-dimensional gauge coupling $`\widehat{g}_s^{}`$ has a negative mass dimension $`\delta /2`$. The gauge-fixing and Faddeev-Popov ghost terms $`\widehat{}_{\mathrm{gf}}+\widehat{}_{\mathrm{FP}}`$ can be constructed accordingly.

Consider gluon scattering, $`\widehat{G}_{j_1}^a\widehat{G}_{j_2}^b\widehat{G}_{j_3}^c\widehat{G}_{j_4}^d,`$ where the index $`j(1,2,\mathrm{},D2)`$ denotes the polarization states of gluon field $`\widehat{G}^{aM}`$. We expect that scattering amplitude will behave at high-energies as a constant of $`𝒪(\widehat{g}_s^2)`$. As in 5DYM:2001a , we expect a large elastic scattering amplitude in the spin-0 and gauge-singlet two-particle state,

$$|\mathrm{\Psi }_0=\frac{1}{\sqrt{(D2)(k^21)}}\underset{j=1}{\overset{D2}{}}\underset{a=1}{\overset{k^21}{}}|\widehat{G}_j^a\widehat{G}_j^a.$$
(2)
After a lengthy calculation 5DSM:2001 , we derive the scattering amplitude for this spin-0 and gauge-singlet channel,

$$\widehat{𝒯}_0[|\mathrm{\Psi }_0|\mathrm{\Psi }_0]=\frac{2k\widehat{g}_s^2}{D2}[\frac{\mathrm{\hspace{0.17em}4}(D2)}{\mathrm{sin}^2\theta }(D4)^2],$$
(3)
where $`\theta `$ is the scattering angle. The $`D`$-dimensional $`s`$-partial wave amplitude Soldate  is thus deduced as

$$\begin{array}{cc}\widehat{a}_{00}^{}\hfill & =\frac{(\sqrt{s})^\delta }{2(16\pi )^{1+\frac{\delta }{2}}\mathrm{\Gamma }\left(1+\frac{\delta }{2}\right)}_0^\pi 𝑑\theta (\mathrm{sin}\theta )^{1+\delta }\widehat{𝒯}_0\hfill \\ & =\frac{2k\sqrt{\pi }\left[4(2+\delta )\frac{\delta ^3}{1+\delta }\right]}{(16\pi )^{1+\delta /2}\delta (2+\delta )\mathrm{\Gamma }\left(\frac{1+\delta }{2}\right)}\left(\widehat{g}_s^2s_{}^{\frac{\delta }{2}}\right).\hfill \end{array}$$
(4)
With the unitarity condition $`|\mathrm{}𝔢\widehat{a}_{00}^{}|<2!/2`$ (where the factor $`2!`$ is due to the identical particles in the final state), we arrive at

$$\sqrt{s}<\left[\frac{(16\pi )^{1+\frac{\delta }{2}}\delta (2+\delta )\mathrm{\Gamma }\left({\displaystyle \frac{1+\delta }{2}}\right)}{2k\sqrt{\pi }\left|4(2+\delta ){\displaystyle \frac{\delta ^3}{1+\delta }}\right|}\frac{1}{\widehat{g}_s^2}\right]^{1/\delta },$$
(5)
which shows that tree-level unitarity violation in such a $`D`$-dimensional theory indeed occurs at an energy of order its intrinsic ultraviolet (UV) scale $`𝒪(\widehat{g}_s^{2/\delta })`$.

For the gluon scattering being described by this effective higher-dimensional gauge theory, the scattering energy $`\sqrt{s}`$ cannot exceed the cutoff $`\mathrm{\Lambda }`$ of the effective theory, $`\sqrt{s}<\mathrm{\Lambda }`$ . Therefore, for the highest possible scattering energy, the scale $`\mathrm{\Lambda }`$ is also bounded by the right hand side of Eq. (5), which results in a constraint on $`\widehat{g}_s^2`$ ,

$$\widehat{g}_s^2<\frac{(16\pi )^{1+\frac{\delta }{2}}\delta (2+\delta )\mathrm{\Gamma }\left({\displaystyle \frac{1+\delta }{2}}\right)}{2k\sqrt{\pi }\left|4(2+\delta ){\displaystyle \frac{\delta ^3}{1+\delta }}\right|}\frac{1}{\mathrm{\Lambda }^\delta }.$$
(6)
The uncompactified higher-dimensional SM by itself is, of course, of no direct phenomenological relevance. Rather, we would like to consider the situation when the $`\delta `$ extra spatial dimensions are compact. In this case, each extra dimension is manifested as Kaluza-Klein towers of the given fields, with mass spectra characterized by $`|n|/R`$, where $`1/R`$ is the compactification scale and $`n`$ the KK-level. The details of the spectra and interactions of these modes depend on the extra-dimensional boundary conditions imposed at the scale $`1/R`$. In the next section, we will explicitly investigate the unitarity of the 5d SM compactified on $`S^1/_2`$. However, we expect the essential properties of unitarity – which arise from the high-energy/short-distance behavior ($`E1/R`$) of the higher-dimensional gauge theory – should be insensitive to the details of the compactification. In particular, we expect that Eq. (6) should hold at least approximately after the extra dimensions are compactified.

In the following, we investigate the consequences of the bound Eq. (6) as applied to a toroidal compactification of the $`\delta `$ extra dimensions on $`T^\delta =(S^1/_2)^\delta `$, with a common radius $`R`$. (For $`\delta 2`$, the current analysis may be readily extended to the case of asymmetric compactification LN  or with nontrivial shape moduli shape , with which some of the constraints could be relaxed 5DSM:2001 .)

The $`D`$-dimensional gauge coupling $`\widehat{g}_s^2`$ is connected to the dimensionless coupling $`g_s^2`$ in the compactified effective 4d KK theory via, $`\widehat{g}_s^2=V_\delta g_s^2`$ with the $`D`$-dimensional volume $`V_\delta =(\pi R)^\delta `$. As described previously, we expect the coupling $`\widehat{g}_s^2`$ to be bounded for a given value of the cutoff $`\mathrm{\Lambda }`$. The effective 4d KK theory can only contain modes of a mass below the cutoff $`\mathrm{\Lambda }`$. Therefore, the highest KK level in this compactification is fixed by $`N_{\mathrm{KK}}=\mathrm{\Lambda }R`$ . We can thus convert Eq. (6) into a constraint on $`N_{\mathrm{KK}}`$,

$$N_{\mathrm{KK}}<\left[\frac{4^{2+\delta }\pi ^{\frac{1\delta }{2}}\delta (2+\delta )\mathrm{\Gamma }\left({\displaystyle \frac{1+\delta }{2}}\right)}{2k\left|4(2+\delta ){\displaystyle \frac{\delta ^3}{1+\delta }}\right|}\frac{1}{g_s^2}\right]^{1/\delta },$$
(7)
which, as written, applies to any compactified $`SU(k)`$ Yang-Mills theory.

Eq. (7) shows that for a given value of the four-dimensional gauge coupling, the highest KK level in the effective 4d theory is bounded from above. The emergence of this bound reflects the fact that, as shown in Ref. 5DYM:2001a , the bad high-energy behavior of the underlying higher-dimensional gauge theory manifests itself through the appearance of the myriad KK excitations which couple together to enhance the scattering amplitude and thus speed up unitarity saturation.

To illustrate these features, we first apply the bound (7) to the QCD sector. Taking a sample value of $`\alpha _s0.1`$ at $`𝒪(\mathrm{TeV})`$ scale, we derive the following bounds from Eq. (7),

$$N_{\mathrm{KK}}(2,\mathrm{\hspace{0.17em}3},\mathrm{\hspace{0.17em}3},\mathrm{\hspace{0.17em}4},\mathrm{\hspace{0.17em}4},\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}4}),$$
(8)
for $`\delta =D4=(1,\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}3},\mathrm{\hspace{0.17em}4},\mathrm{\hspace{0.17em}5},\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}7})`$, respectively. It is also straightforward to apply the condition (7) to the scattering of weak gauge bosons in the $`SU(2)_WU(1)_Y`$ electroweak (EW) gauge sector of the $`D`$-dimensional SM. Ignoring the small $`W_3`$-$`B`$ mixing and setting $`SU(k)=SU(2)_W`$, we deduce the following bounds on $`N_{\mathrm{KK}}`$, for $`\delta =(1,\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}3},\mathrm{\hspace{0.17em}4},\mathrm{\hspace{0.17em}5},\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}7})`$,

$$N_{\mathrm{KK}}(9,\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}5},\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}8},\mathrm{\hspace{0.17em}6}),$$
(9)
where we have replaced $`g_s^2`$ by $`g^2=(v/2m_w)^2`$ in (7), with $`v`$ the Higgs vacuum expectation value (VEV) and $`m_w`$ the mass of weak gauge bosons. For $`\delta 3`$, this is significantly weaker than (8) from the QCD sector. The non-monotonic behavior of $`N_{\mathrm{KK}}`$ bounds (8) and (9) as a function of $`\delta `$ is because in the right-hand side of Eq. (7) the denominator has a maximum at $`\delta 2.5`$ and flips sign at $`\delta 6.2`$.

Next, we analyze the scalar Higgs sector of the $`D`$-dimensional SM. The Higgs Lagrangian is

$$\widehat{}_h^D=D_M^{}\widehat{\mathrm{\Phi }}^{}D^M\widehat{\mathrm{\Phi }}\left[\mu ^2\widehat{\mathrm{\Phi }}^{}\widehat{\mathrm{\Phi }}+\widehat{\lambda }(\widehat{\mathrm{\Phi }}^{}\widehat{\mathrm{\Phi }})^2\right],$$
(10)
where $`D_M^{}\widehat{\mathrm{\Phi }}=(_M\frac{i}{2}\widehat{g}\tau ^a\widehat{W}_M^a\frac{i}{2}\widehat{g}^{}\widehat{B}_M)\widehat{\mathrm{\Phi }}`$, and the $`D`$-dimensional Higgs (gauge) coupling $`\widehat{\lambda }`$ $`(\widehat{g},\widehat{g}^{})`$ has a mass dimension of $`\delta `$ $`(\delta /2)`$. Due to $`\mu ^2<0`$, the Higgs doublet $`\widehat{\mathrm{\Phi }}`$ develops a VEV so that $`\widehat{\mathrm{\Phi }}=(i\widehat{\pi }^+,(\widehat{v}+\widehat{h}^0+i\widehat{\pi }^0)/\sqrt{2})^T`$, and the neutral Higgs boson $`\widehat{h}^0`$ acquires its mass at tree-level

$$\widehat{m}_H^{}=\sqrt{\mathrm{\hspace{0.17em}2}\widehat{\lambda }\widehat{v}^2}.$$
(11)
Consider the $`D`$-dimensional scalar-scalar scattering, $`|\widehat{\varphi }^a\widehat{\varphi }^b|\widehat{\varphi }^c\widehat{\varphi }^d`$, via the neutral channels $`|\widehat{\varphi }^a\widehat{\varphi }^b,|\widehat{\varphi }^c\widehat{\varphi }^d=\frac{1}{\sqrt{2}}|\widehat{h}^0\widehat{h}^0,\frac{1}{\sqrt{2}}|\widehat{\pi }^0\widehat{\pi }^0,|\widehat{\pi }^+\widehat{\pi }^{},|\widehat{h}^0\widehat{\pi }^0`$, in analogy with the customary analysis in the usual 4d SM Higgs sector LQT-77 . Their scattering amplitudes form a $`4\times 4`$ matrix and, for $`s\widehat{m}_H^2`$, approach a constant matrix which arises from the four-scalar contact interaction. (The channel $`|\widehat{h}^0\widehat{\pi }^0`$ actually decouples from the other three.) The eigenvalues of this constant matrix are readily worked out as $`\widehat{\lambda }(6,\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}2})`$ so that the maximal eigenvalue amplitude is $`\widehat{𝒯}_{\mathrm{max}}=6\widehat{\lambda }`$. Thus, using the first formula in Eq. (4), we derive the $`D`$-dimensional $`s`$-partial wave amplitude,

$$\begin{array}{c}\widehat{a}_0^{}=C_0(\sqrt{s}\widehat{\lambda }_{}^{\frac{1}{\delta }})^\delta ,C_0\frac{3\sqrt{\pi }}{(16\pi )^{1+\frac{\delta }{2}}\mathrm{\Gamma }\left(\frac{3+\delta }{2}\right)}.\hfill \end{array}$$
(12)
The relevant unitarity condition is $`|\mathrm{}𝔢\widehat{a}_0^{}|<\frac{1}{2}`$ , where the possible identical particle factors are included LQT-77  in the normalization of in/out state $`|\widehat{\varphi }^a\widehat{\varphi }^b`$ or $`|\widehat{\varphi }^c\widehat{\varphi }^d`$ mentioned above. So, we deduce,

$$\sqrt{s}<\left(2C_0\widehat{\lambda }\right)^{\frac{1}{\delta }},$$
(13)
which again shows that the unitarity violation in such a $`D`$-dimensional theory indeed occurs at the intrinsic UV scale, of $`𝒪(\widehat{\lambda }^{1/\delta })`$ in this case. Similar to Eq. (6), we may interpret Eq. (13) as an upper bound on the cutoff $`\mathrm{\Lambda }`$, and translate this into a condition on $`\widehat{\lambda }`$ ,

$$\widehat{\lambda }<\frac{1}{2C_0(\sqrt{s})_{\mathrm{max}}^\delta }=\frac{1}{2C_0\mathrm{\Lambda }^\delta }.$$
(14)
In parallel with our analysis of the gauge sector, we interpret (14) in the context of compactification on $`T^\delta =(S^1/_2)^\delta `$ and examine the consequences for the effective 4d KK-theory. The $`D`$-dimensional coupling $`\widehat{\lambda }`$ is related to the dimensionless coupling $`\lambda `$ in the 4d KK-theory via, $`\widehat{\lambda }=V_\delta \lambda =(\pi R)^\delta \lambda ,`$ and Eq. (14) can be rewritten as

$$\lambda <\frac{1}{2C_0V_\delta \mathrm{\Lambda }^\delta }\frac{1}{2C_0\pi ^\delta \left(N_{\mathrm{KK}}\right)^\delta },$$
(15)
where $`N_{\mathrm{KK}}\mathrm{\Lambda }R`$ represents the highest KK levels associated with each compactified extra dimension in the low-energy effective theory. Here, for simplicity, we have ignored the small $`𝒪(m_w^2R^2)`$ correction to the KK mass spectrum in the EW sector.

The 4d KK theory contains a zero-mode Higgs doublet $`\mathrm{\Phi }_0=(i\pi _0^+,(v+h_0^0+i\pi _0^0)/\sqrt{2})^T`$, and its KK excitations $`\mathrm{\Phi }_𝐧=(i\pi _𝐧^+,(v+H_𝐧^0+i\pi _𝐧^0)/\sqrt{2})^T`$, where the VEV of the 4d Higgs doublet $`\mathrm{\Phi }_0`$, $`v=(\sqrt{2}G_F)^{1/2}246`$ GeV, is related to that of $`\widehat{\mathrm{\Phi }}`$ via $`\widehat{v}=v/\sqrt{V_\delta }`$ . The corresponding mass of the zero-mode neutral Higgs boson $`h_0^0`$ is then given by

$$m_H^{}=\sqrt{\mathrm{\hspace{0.17em}2}\lambda v^2}=\sqrt{\mathrm{\hspace{0.17em}2}\widehat{\lambda }\widehat{v}^2}=\widehat{m}_H^{},$$
(16)
which is unchanged under the compactification. Hence, we can deduce, from (15), the unitarity bound on the physical mass of the 4d SM Higgs boson $`h_0^0`$ ,

$$m_H^{}<\frac{v}{\sqrt{C_0V_\delta \mathrm{\Lambda }_{}^\delta }}\frac{v}{\sqrt{C_0\pi ^\delta \left(N_{\mathrm{KK}}\right)^\delta }},$$
(17)
where for $`\delta =(1,2,3,4,5,6,7)`$,

$$1/\sqrt{C_0\pi ^\delta }(4.6,\mathrm{\hspace{0.17em}8.0},\mathrm{\hspace{0.17em}14.7},\mathrm{\hspace{0.17em}28.5},\mathrm{\hspace{0.17em}57.6},\mathrm{\hspace{0.17em}121},\mathrm{\hspace{0.17em}260}).$$

Alternatively, for a given $`m_H^{}`$,

$$N_{\mathrm{KK}}<\left(\frac{v^2}{C_0\pi ^\delta m_H^2}\right)^{\frac{1}{\delta }}.$$
(18)
A few comments are in order. We note that Eq. (18) shares similar features to the gauge KK bound (7), except that in the Higgs sector the coupling $`\lambda `$ (or mass $`m_H^{}`$) is not fixed by observation. If we impose the existing direct Higgs search limit at LEP-2, $`m_H^{}>114.3`$ GeV (95% C.L.), we can deduce, from (18),

$$N_{\mathrm{KK}}(98,\mathrm{\hspace{0.17em}17},\mathrm{\hspace{0.17em}10},\mathrm{\hspace{0.17em}7},\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}6},\mathrm{\hspace{0.17em}6}),$$
(19)
for $`\delta =(1,2,3,4,5,6,7)`$. The limits (19) are significantly weaker than the bounds (8)-(9) from the gauge KK sector, especially for $`\delta 3`$ . The bound (17) on the Higgs-boson mass is listed in Table I for various values of $`\delta `$ and $`N_{\mathrm{KK}}`$. The entries marked by $`_\times `$ are excluded by direct Higgs searches at LEP-2, cf. Eq. (19). We see that $`N_{\mathrm{KK}}`$ is more severely bounded by the unitarity of the gauge boson scattering, especially for gluons as long as they propagate in the bulk \[cf. (8)\]. Imposing the stronger bound (8), we can examine how Eq. (17) would constrain $`m_H^{}`$. The corresponding upper limits are displayed in the entries of Table I marked by  .

Finally, we stress that the strong bounds we have derived for either the coupling constants $`(g^2,\lambda )`$ or the Higgs mass $`m_H^{}`$ are due to the large extra dimensional volume $`V_\delta R^\delta `$ which appears in relating the $`D`$-dimensional and four-dimensional physics. Equivalently, following 5DYM:2001a , the strong bounds arise because of the appearance of many KK excitations in the compactified theory which couple together to enhance the scattering amplitude and speed up unitarity saturation. Therefore, it is clear that if any field is restricted on a brane, the corresponding unitarity bound on its couplings would reduce back to that of the customary 4d SM.

In the scenario with universal extra dimensions UED , all SM fields propagate in the extra dimensions and are thus subject to the unitarity limits discussed above. In particular, these results suggest that the “self-broken” standard model Arkani-Hamed:2000hv  would not be realized consistently. For a given $`N_{\mathrm{KK}}`$ and under the compactification $`(S^1/_2)^\delta `$, the total number of states consistent with the unitarity constraint is approximately given by

$$n(N_{\mathrm{KK}})\frac{\pi ^{\delta /2}}{\mathrm{\Gamma }\left(1+\frac{\delta }{2}\right)}\left(\frac{N_{\mathrm{KK}}}{2}\right)^\delta .$$
(20)
For $`D=6`$, our constraints in (8) yield a total number of KK modes $`n7`$, and for $`D=8`$ we find $`n79`$. Neither appears likely to yield a sufficiently light top quark mass at the observed value, for a wide range of compactification scales Arkani-Hamed:2000hv .

For theories with extra dimensional perturbative gauge unification powerlaw , all gauge fields live in the bulk and again these bounds apply. We will return to implications for GUTs in Sec. 4. Finally, we note that our analysis may be extended to bounds on gauge KK scattering and graviton KK scattering 5DSM:2001  in a warped 5d SM à la Randall-Sundrum (RS1) RS1 .

3. KK Theory: $`E`$-Cancellations and Unitarity Limits

We now turn to an explicit analysis of the effective 4d KK theory arising from compactifying a 5d SM on $`S^1/_2`$. The Lagrangian of this 4d KK theory will contain interactions involving purely physical fields 4DKK-LU ; Hill  and interactions involving additional geometric and ordinary would-be Goldstone fields 5DYM:2001a ; 5DRxi2 . In Ref. 5DYM:2001a , the scattering of the longitudinal components of the KK excitations of Yang-Mills fields was systematically computed. Here we will further compute all amplitudes involving transversely polarized gauge KK fields, and in the case of the EW gauge bosons, include the effect of EW symmetry breaking of the SM.

We begin by considering gluon KK scattering $`G_{j_1}^{a,n}G_{j_2}^{b,n}G_{j_3}^{c,\mathrm{}}G_{j_4}^{d,\mathrm{}}`$, where $`j(+,0,)`$ denotes the three helicity states and $`(n,\mathrm{})`$ the KK levels. We may define a spin-0, gauge-singlet state,

$$|\mathrm{\Psi }_0^n=\frac{1}{\sqrt{3(k^21)}}\underset{j=1}{\overset{+1}{}}\underset{a=1}{\overset{k^21}{}}|G_j^{a,n}G_j^{a,n},$$
(21)
where $`k=3`$ for QCD $`SU(3)_c`$. The corresponding $`|\mathrm{\Psi }_0^n|\mathrm{\Psi }_0^{\mathrm{}}`$ scattering channel consists of 9 helicity amplitudes, but only 4 of them are independent under the discrete (P, C, T) symmetries. We arrive at

$$\begin{array}{c}𝒯_0\left[|\mathrm{\Psi }_0^n|\mathrm{\Psi }_0^{\mathrm{}}\right]=\frac{1}{3(k^21)}\underset{j,j^{}=1}{\overset{+1}{}}\underset{a,c=1}{\overset{k^21}{}}\hfill \\ \left\{𝒯_{00,00}^{aa,cc}+2𝒯_{++,++}^{aa,cc}+2𝒯_{++,}^{aa,cc}+4𝒯_{00,++}^{aa,cc}\right\}.\hfill \end{array}$$
(22)
A systematic calculation shows that the amplitudes $`𝒯_{00,++}^{aa,cc}`$ and $`𝒯_{++,++}^{aa,cc}`$ vanish to $`𝒪(g_s^2E^0)`$, while the amplitudes $`𝒯_{00,00}^{aa,cc}`$ and $`𝒯_{++,}^{aa,cc}`$ both have nonzero $`𝒪(g_s^2E^0)`$ contributions. We have verified the nontrivial energy-cancellations at $`𝒪(E^{4,2})`$ \[$`𝒪(E^2)`$\] for the amplitudes $`𝒯_{00,00}^{ab,cd}`$ \[$`𝒯_{00,++}^{ab,cd}`$\] involving four \[two\] external longitudinal KK gluon states, and the consistency with the Kaluza-Klein Equivalence Theorem (KK-ET) 5DYM:2001a . We then compute the $`s`$-partial wave amplitude for (22) with $`n\mathrm{}`$,

$$a_{00}^{}=\frac{kg_s^2}{24\pi }\left[1+6\mathrm{ln}\frac{N_s^2}{|n^2\mathrm{}^2|}\right],$$
(23)
where $`N_s\sqrt{s}RN_{\mathrm{KK}}`$ . This explicitly shows that due to the exact $`E`$-cancellations, the partial wave amplitude indeed behaves as constant at the leading order.

To maximize the scattering amplitude, we define a normalized state consisting of KK-levels up to $`N_0`$,

$$|\mathrm{\Omega }=\frac{1}{\sqrt{N_0}}\underset{n=1}{\overset{N_0}{}}|\mathrm{\Psi }_0^n,$$
(24)
where the kinematics of $`22`$ scattering requires $`N_0<N_s/2`$. For $`(N_0)_{\mathrm{max}}=(N_s)_{\mathrm{max}}/2=N_{\mathrm{KK}}/2`$, we deduce the maximal $`s`$-wave amplitude for $`|\mathrm{\Omega }|\mathrm{\Omega }`$, to leading order in $`N_{\mathrm{KK}}`$,

$$a_{00}^{}[\mathrm{\Omega }]=\frac{kg_s^2}{8\pi }[\frac{N_{\mathrm{KK}}}{6}+\frac{4}{N_{\mathrm{KK}}}\underset{n\mathrm{}=1}{\overset{N_{\mathrm{KK}}/2}{}}\mathrm{ln}\frac{N_{\mathrm{KK}}^2}{|n^2\mathrm{}^2|}].$$
(25)
From the unitarity condition $`|a_{00}^{}|<\sqrt{2!}/2`$ (where the $`2!`$ arises from identical particles in the final state 5DSM:2001 ), we derive the following numerical bound for the QCD sector,

$$N_{\mathrm{KK}}4.$$
(26)
We have performed a similar analysis for KK scattering in the EW gauge sector, where a small mixing 5DRxi2  arises between the geometric KK Goldstone bosons $`W_n^{a5}`$ and the KK excitation modes $`\pi _n^a`$ of ordinary Goldstone bosons in the Higgs doublet. This mixing is described by a mixing angle $`\mathrm{sin}\theta _n=m_v/M_n1`$, and $`m_v(=m_{w,z})`$ is the mass of the zero-mode gauge bosons $`(W_0^\pm ,Z_0^0)`$. As a result, the “eaten” KK Goldstone field is $`\stackrel{~}{\pi }_n^a=\mathrm{cos}\theta _nV_n^{a,5}+\mathrm{sin}\theta _n\pi _n^a`$, ($`V_n^{a,5}=W_n^{\pm ,5},Z_n^{0,5}`$), and the gauge KK modes $`V_n^{a,\mu }(=W_n^{\pm ,\mu },Z_n^{0,\mu })`$ have mass $`M_n=\sqrt{(n/R)^2+m_v^2}n/R`$, ( $`m_vn/R`$). There are three types of physical KK Higgs states $`(H_n^\pm ,P_n^0)`$ which are just orthogonal to the “eaten” KK Goldstone fields $`(\stackrel{~}{\pi }_n^\pm ,\stackrel{~}{\pi }_n^0)`$ .

From direct calculation 5DSM:2001 , we find that this small mixing causes extra $`E^2`$ contributions to individual contribtuions to the scattering amplitude with four external longitudinal gauge KK states, but they exactly cancel to $`𝒪(E^0)`$ for each process. This is consistent with the $`E`$-counting for the corresponding KK Goldstone amplitude based on the KK-ET 5DYM:2001a  which, unlike the conventional ET for the 4d SM Cornwall-74 ; LQT-77 ; mike ; He-ET1 ; He-ET2 , involves the geometric Higgs mechanism from compactification. Analogous to our analysis in the QCD sector and ignoring the tiny constant terms suppressed by $`m_w^2R^2`$ or $`m_w^4R^4`$, we derive the unitarity bound on $`N_{\mathrm{KK}}`$ from a coupled channel analysis for the $`22`$ EW gauge KK scattering,

$$N_{\mathrm{KK}}11,$$
(27)
where we have replaced $`g_s^2`$ in Eq. (25) by the weak gauge coupling $`g^2=(2m_w/v)^2`$.

It is interesting to compare the bounds (26) and (27) with those estimated from the uncompactified $`D`$-dimensional scattering analysis for $`D=5`$. We see that (27) agrees with (9) quite well where the $`N_{\mathrm{KK}}`$ upper limits are about $`911`$, while (26) agrees with (8) up to a factor of $`2`$ where the $`N_{\mathrm{KK}}`$ is constrained to be no higher than the range of $`24`$. This is as expected, however, since for very low values of $`N_{\mathrm{KK}}`$ the kinematic effects due to the finite KK masses (which are absent in the uncompactified analysis) would become more important. Also, the subleading terms ignored in the 4d amplitude (25) are suppressed by a factor $`1/N_{\mathrm{KK}}`$ relative to the leading terms and imply a larger uncertainty for very low values of $`N_{\mathrm{KK}}`$. Hence, we see that the two independent analyses are consistent with each other, and they provide consistent estimates for the unitarity bounds.

Next, we perform a coupled channel analysis for the $`22`$ Higgs KK scattering and derive the $`m_H^{}`$ bounds in the effective 4d KK theory. There are four types of processes (and their crossing channels) which appear relevant to this coupled channel analysis: (i) $`\overline{H}_n\overline{H}_n\overline{H}_k\overline{H}_k`$, (ii) $`h_0^{}\overline{H}_{2k}\overline{H}_k\overline{H}_k`$, (iii) $`h_0^{}h_0^{}\overline{H}_k\overline{H}_k`$, (iv) $`V_{0L}^aV_{0L}^a\overline{H}_k\overline{H}_k`$, where $`\overline{H}_n(H_n^0,P_n^0,H_n^\pm )`$ represents three types of physical Higgs KK states and $`V_{0L}^a(W_{0L}^\pm ,Z_{0L}^0)`$ denotes the zero-modes of the longitudinal weak gauge bosons. As will be clear shortly, we find that the only important processes for our coupled channel analysis are type-(i) which involves only KK scalars (without zero-mode) for the in/out states of the $`22`$ scattering.

For the type-(i) channels, we will consider the scattering $`|\overline{H}_n\overline{H}_n|\overline{H}_k\overline{H}_k`$, via electrically neutral KK channels $`|\overline{H}_n\overline{H}_n,|\overline{H}_k\overline{H}_k=\frac{1}{\sqrt{2}}|H_{\mathrm{}}^0H_{\mathrm{}}^0,`$ $`\frac{1}{\sqrt{2}}|P_{\mathrm{}}^0P_{\mathrm{}}^0,|H_{\mathrm{}}^+H_{\mathrm{}}^{},|H_{\mathrm{}}^0P_{\mathrm{}}^0`$ ($`\mathrm{}=n,k`$) in analogy with the customary analysis of the 4d SM LQT-77 . Again, the channel $`|H_{\mathrm{}}^0P_{\mathrm{}}^0`$ decouples from the other three channels and their scattering amplitudes form a $`4\times 4`$ matrix which approaches constant for $`sM_{\overline{H}}^2`$. We then derive the eigenvalues of this matrix as, $`\lambda (6,\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}2})`$ for $`nk`$, and $`(3\lambda /2)(6,\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}2},\mathrm{\hspace{0.17em}2})`$ for $`n=k`$. Thus, the maximal eigenvalue amplitudes are

$$𝒯_{\mathrm{max}}[nn,kk]=6\lambda ,𝒯_{\mathrm{max}}[nn,nn]=9\lambda ,$$
(28)
where $`nk`$ . Defining a normalized state consisting of $`N_0`$ pairs of KK states,

$$|S=\frac{1}{\sqrt{N_0}}\underset{n=1}{\overset{N_0}{}}|nn,$$
(29)
we deduce the $`s`$-wave amplitude for $`|S|S`$, at the leading order of $`N_0`$ ,

$`a_0^{}[S]`$ $`=`$ $`{\displaystyle \frac{3N_0}{16\pi }}\left({\displaystyle \frac{m_H^{}}{v}}\right)^2,`$ (30)
where the inelastic channels $`nnkk`$ ($`nk`$) dominate while channels $`nnnn`$ are only of $`𝒪((N_0)^0)`$. From the unitarity condition $`|\mathrm{}𝔢a_0^{}|<1/2`$ and noting the kinematic requirement $`N_0N_{\mathrm{KK}}/2\mathrm{\Lambda }R/2`$, we deduce the $`N_{\mathrm{KK}}`$ limit,

$$N_{\mathrm{KK}}<\frac{\mathrm{\hspace{0.17em}16}\pi }{3}\left(\frac{v}{m_H^{}}\right)^2,$$
(31)
or, the bound on the Higgs mass,

$$m_H^{}<\left(\frac{\mathrm{\hspace{0.17em}16}\pi }{3}\right)^{\frac{1}{2}}\frac{v}{\sqrt{N_{\mathrm{KK}}}}.$$
(32)
With these we can constrain $`N_{\mathrm{KK}}`$ by imposing the LEP-2 Higgs search limit $`m_H^{}<114.3`$ GeV,

$`N_{\mathrm{KK}}`$ $``$ $`77.`$ (33)
Comparing our estimated bound (20) for $`D=5`$ with the above limit, we see that the difference is only about $`21\%`$. Using the condition (32), we further derive the Higgs mass limits,

$$m_H^{}<(581,\mathrm{\hspace{0.17em}503},\mathrm{\hspace{0.17em}450},\mathrm{\hspace{0.17em}411},\mathrm{\hspace{0.17em}356},\mathrm{\hspace{0.17em}318},\mathrm{\hspace{0.17em}291},\mathrm{\hspace{0.17em}260},\mathrm{\hspace{0.17em}225})\mathrm{GeV},$$
(34)
for the inputs $`N_{\mathrm{KK}}=(3,4,5,6,8,10,12,15,20)`$, respectively. Again, we notice that the estimated bounds in Table I ($`\delta =1`$) are in reasonable agreement.

Finally, we comment that for type-(ii), -(iii) and -(iv) processes we could define similar normalized in/out state to Eq. (29), but it is readily seen that the corresponding scattering amplitudes only have leading contributions at $`𝒪(1)`$, $`𝒪(\sqrt{N_0})`$ and $`𝒪(\sqrt{N_0})`$, respectively. The type-(i) channels with $`n=k`$ also have amplitudes of $`𝒪(1)`$. So, the bounds from these other channels are too weak to be useful in comparison with that of type-(i) with $`nk`$.

4. Gauge Unification and Unitarity Constraints

In higher dimensional Grand Unified Theories (GUTs), the KK states will contribute to the running gauge coupling constants. For sufficiently many KK states, these contributions mimic the power-law running expected in a higher dimensional theory and it has been proposed that this could substantially lower the GUT scale in these theories powerlaw . The unitarity bounds we derived from the gauge KK scattering in the previous sections can apply to any higher dimensional GUT (with/without supersymmetry) whose low energy theory contains the SM gauge bosons and their KK excitations as part of the spectrum. We will show that such unitarity constraints severely restrict the number of KK states which can consistently accelerate perturbative gauge-coupling unification. This bound prevents the unification scale in 5d minimal SUSY GUT from being lower than about $`10^{14}`$ GeV.

In the $`\overline{\mathrm{MS}}`$ scheme, the running gauge coupling may be expressed as powerlaw ; Hill-GUT

$$\alpha _j^1(\mu )=\alpha _{zj}^1\frac{b_j}{2\pi }\mathrm{ln}\frac{\mu }{m_z}\frac{\stackrel{~}{b}_j}{2\pi }F(\delta ,n(\mu ))+\kappa _j,$$
(35)
where $`\alpha _{zj}\alpha _j(m_z)`$, $`\kappa _j`$ represents corrections from the higher loop-levels and possible higher dimensional operators suppressed by GUT scale $`M_\mathrm{G}`$ Hill-GUT . The $`F(\delta ,n(\mu ))`$ term arises from the one-loop KK contributions,

$$F(\delta ,n(\mu ))=\underset{n=1}{\overset{n(\mu )}{}}𝒟_n\mathrm{ln}\frac{\mu }{M_n},$$
(36)
where $`M_n`$ is the mass of relevant KK excitations at level $`n`$, $`n(\mu )`$ is defined by $`M_{n(\mu )}<\mu <M_{n(\mu )+1}`$, and $`\mathrm{\hspace{0.17em}1}n(\mu )N_{\mathrm{KK}}`$. The $`𝒟_n`$ denotes the degeneracy at the KK level $`n`$.

For $`\delta =1`$ with compactification on $`S^1/_2`$, we have $`𝒟_n=1`$ and

$$F(1,N_{\mathrm{KK}})=N_{\mathrm{KK}}\mathrm{ln}N_{\mathrm{KK}}\mathrm{ln}(N_{\mathrm{KK}}!),$$
(37)
where we set $`N_{\mathrm{KK}}=RM_G`$. Unification at the GUT scale imposes the conditions $`\alpha _1(M_\mathrm{G})=\alpha _2(M_\mathrm{G})=\alpha _3(M_\mathrm{G})`$, where we have used the usual GUT normalization, $`\alpha _1=(5/3)g_{}^{}{}_{}{}^{2}/(4\pi )`$ . From this, we arrive at

$$M_\mathrm{G}=m_z\mathrm{exp}\{\frac{2\pi }{\mathrm{\Delta }b_{ij}}[\mathrm{\Delta }\alpha _{zij}^1\frac{\mathrm{\Delta }\stackrel{~}{b}_{ij}}{2\pi }F(\delta ,N_{\mathrm{KK}})+\mathrm{\Delta }_{ij}]\},$$
(38)
where $`\mathrm{\Delta }\alpha _{zij}^1=\alpha _{zi}^1\alpha _{zj}^1`$, $`\mathrm{\Delta }b_{ij}=b_ib_j`$, $`\mathrm{\Delta }\stackrel{~}{b}_{ij}=\stackrel{~}{b}_i\stackrel{~}{b}_j`$, $`\mathrm{\Delta }_{ij}=\kappa _i\kappa _j=𝒪(10^2)`$, and $`i<j=1,2,3`$. Eq. (38) contains three relations with $`(ij)=(12,23,13)`$, two of which are independent.

As in the case of four-dimensions, the $`D`$-dimensional extension of the SM without supersymmetry (SUSY) does not realize perturbative gauge unification, unless additional fields are added to the SM particle spectrum powerlaw . The simplest example of the perturbative gauge unification is the minimal supersymmetric extension of the SM (MSSM). Following Ant ; powerlaw ; Hill-GUT , we will consider a $`D`$-dimensional MSSM compactified on $`T^\delta =(S^1/_2)^\delta `$, with vector-supermultiplets and two Higgs supermultiplets propagating in the bulk, and chiral-supermultiplets for fermions sitting on the brane. For the simplest GUT group $`SU(5)`$SU5 , the SM fermions fill an entire $`SU(5)`$ representation, their presence in the bulk would not effect unification, but does change the value of the unified coupling. Since matter contributes positively to the beta functions and drives the theory to stronger coupling, allowing matter in the bulk will only enhance the effect of KK excitations and strengthen our unitarity bounds. For this $`D`$-dimensional MSSM, the coefficients of one-loop beta functions are, $`(b_1,b_2,b_3)=(33/5,\mathrm{\hspace{0.17em}1},3)`$, and $`(\stackrel{~}{b}_1,\stackrel{~}{b}_2,\stackrel{~}{b}_3)=(3/5,3,6)`$. Substituting these into (38), we derive the numerical relation

$$\begin{array}{cc}M_\mathrm{G}\hfill & 10^4\mathrm{exp}\left[28.3\left(1\frac{F(\delta ,N_{\mathrm{KK}})}{44.1}+0.040\mathrm{\Delta }_{12}\right)\right]\mathrm{GeV},\hfill \end{array}$$
(39)
where we have imposed the condition $`\alpha _1(M_\mathrm{G})=\alpha _2(M_\mathrm{G})`$ since the $`Z`$-pole values of $`\alpha _{1,2}`$ are much more precisely known than the value of $`\alpha _3`$.

Using (39), we plot the GUT scale $`M_\mathrm{G}`$ as a function of $`N_{\mathrm{KK}}`$ for the 5d MSSM in Fig. 1(a). In Hill-GUT , it was argued that the term $`\mathrm{\Delta }_{ij}`$ may receive contributions from the higher dimensional operators suppressed by the string scale which is of $`𝒪(10^2)`$ in the MSSM and could be as large as $`10\%`$ in its next-to-minimal extension (NMSSM). In Fig. 1(a), we have varied $`\mathrm{\Delta }_{12}`$ from $`+10\%`$ to $`10\%`$. Using the Eq. (35), we also plot in Fig. 1(b) the evolution of three gauge coupling constants with a typical high compactification scale $`R^1=10^{13}`$ GeV, where the higher order parameter $`\kappa _j`$ is varied within $`\pm 10\%`$. In this case, the unification is accelerated to a lower scale $`M_\mathrm{G}=10^{14}`$ GeV due to the presence of $`N_{\mathrm{KK}}=10`$ KK states.

For a compactification size $`R`$ and a GUT scale $`M_\mathrm{G}`$, perturbative unification can only occur if the KK modes of level $`N_{\mathrm{KK}}=RM_\mathrm{G}`$ satisfy the unitarity bounds in Sec. 3. We note that when $`M_\mathrm{G}`$ is close to the conventional GUT scale, the running coupling $`\alpha _2^{}=g^2/4\pi `$ in the range between $`R^1`$ and $`M_\mathrm{G}`$ has about the same size as (or slightly larger than) the weak scale value $`\alpha _2(m_z)`$ \[cf. Fig. 1(b)\]. Hence, we can apply the unitarity limit $`N_{\mathrm{KK}}11`$ in Eq. (27), and find that, because of $`N_{\mathrm{KK}}=M_\mathrm{G}R`$, the scales $`R^1`$ and $`M_\mathrm{G}`$ cannot be separated by more than one order of magnitude. This is a generic feature for any higher-dimensional GUT theory, when the bound (27) can be applied directly. Therefore, no substantial acceleration of four-dimensional perturbative gauge unification is possible from embedding the theory in higher dimensions. In particular, the GUT scale of the 5d minimal SUSY GUT has to be of order $`10^{14}`$ GeV or higher \[cf. Fig. 1(a)\]. An extension of our analysis to the 5d GUTs broken by orbifolds Hall  can be similarly performed 5DSM:2001 .

5. Conclusions

In this Letter we have extended the study of the unitarity of compactified 5d Yang-Mills theories, and investigated the unitarity of the standard model (SM) in higher dimensions. Analyzing gauge-boson scattering in the uncompactified $`D`$-dimensional theory, we derive an upper bound on the UV cutoff of the theory. Using this estimate in the compactified case yields bounds on the highest Kaluza-Klein (KK) level $`N_{\mathrm{KK}}`$ and the Higgs mass $`m_H^{}`$ allowed in the effective 4d low-energy theory. We demonstrated the validity of these general observations by explicitly analyzing the effective 4d Kaluza-Klein theory from a compactified five-dimensional SM on $`S^1/_2`$. With the analyses before and after compactification, we derive the strongest bounds on $`N_{\mathrm{KK}}`$ from the gauge KK scattering. Applying these bounds to higher-dimensional supersymmetric GUTs, we show that only a small number of KK states can be used to accelerate the perturbative gauge coupling unification. In particular, the unification scale $`M_\mathrm{G}`$ in the 5d minimal SUSY GUT and similar theories cannot be lower than about $`10^{14}`$ GeV.

As this work was being completed a related study xx  appeared, which considered bounds on $`m_H^{}`$ from the scattering of $`W_{0L}^+W_{0L}^{}`$ into the scalar KK states by using the equivalence theorem.

We thank N. Arkani-Hamed, W. A. Bardeen, J. Blum, B. A. Dobrescu, C. T. Hill, C. Macesanu, and W. W. Repko for discussions. This work was supported by the DOE under grants DE-FG02-91ER40676, DE-FG03-93ER40757, DE-FG03-98ER41076 and DE-FG02-01ER 45684.


text:
# Boundary of Quantum Evolution under Decoherence

## 1 Introduction

The control of quantum ensembles has many applications, ranging from coherent spectroscopy to quantum information processing. In practice, the quantum system of interest is not isolated but interacts with its environment. This leads to the phenomenon of relaxation, which results in signal loss and ultimately limits the range of applications. Relaxation is a major road block standing in the way of practical quantum computing. Manipulating quantum systems in a manner that minimizes relaxation losses is a fundamental challenge of utmost practical importance. What is the ultimate limit on how close an ensemble of quantum systems can be steered from an initial state to a desired target state in the presence of relaxation? Until now there existed no theory that answers this question. This situation is comparable to the time before the fundamental limits of a heat engine were known: More than hundred years after the invention of the steam engine, the physical limits for the maximum amount of work a steam engine could produce was unclear, in spite of decades of advances in its design. ”The theory of its operation is rudimentary and attempts to improve its performance are still made in an almost haphazard way” . Of course, the maximum efficiency of a heat engine is not given by the cleverness of the engineer who attempts to builds such a machine, but by the fundamental law of thermodynamics as captured in Carnot’s principle.

In this manuscript we derive fundamental limits on how close can an ensemble of nuclear spins be driven from its initial state to a desired target state in the presence of relaxation. In particular, we derive the maximum efficiency of polarization and coherence transfer between coupled nuclear spins. A premier example where such coherence transfer operations are important is nuclear magnetic resonance (NMR) spectroscopy . In structural biology, NMR spectroscopy is an important technique that allows to determine the structure of biological macro molecules, such as proteins, in aqueous solution. With increasing size of molecules or molecular complexes, the rotational tumbling of the molecules becomes slower and leads to increased relaxation losses. When these relaxation rates become comparable to the spin-spin couplings, the efficiency of coherence transfer is considerably reduced, leading to poor sensitivity and increased measurement times. Recent advances have made it possible to significantly extend the size limit of biological macro molecules amenable to study by liquid state NMR \[3-6\]. These techniques take advantage of the phenomenon of cross-correlated relaxation. Cross-correlated relaxation represents interference effects between two different ralaxation mechanisms . Until now it was not clear if further improvements can be made and what is the physical limit for the coherence transfer efficiency between coupled spins in the presence of cross-correlated relaxation.

In this letter, we give analytical expressions for this maximum achievable coherence transfer efficiency for two heteronuclear coupled spins under very general decoherence mechanisms that include cross-correlated relaxation. We describe the optimal pulse sequences that achieve this efficiency and experimental data that supports these results. We demonstrate that in the limit where the interference effects become comparable to the uncorrelated relaxation rates, complete coherence transfer is possible without any loss. In the general case of cross-correlated relaxation, we demonstrate substantial improvement over previously known sequences in NMR spectroscopy.

## 2 Theory

We consider an isolated heteronuclear spin system consisting of two coupled spins 1/2, denoted $`I`$ (e.g. 1H) and $`S`$ (e.g. 15N). To fix ideas, we first address the problem of selective population inversion of two energy levels (e.g. $`\alpha \beta `$ and $`\beta \beta `$) as shown in Fig. 1. This is a central step in high-resolution multi-dimensional NMR spectroscopy and corresponds to the transfer of an initial density operator $`I_z`$, representing polarization on spin $`I`$, to the target state $`2I_zS_z`$.

For large molecules in the so-called spin diffusion limit , where longitudinal relaxation rates are neglegible compared to transverse relaxation rates, both the initial term ($`I_z`$) and final term ($`2I_zS_z`$) of the density operator are long-lived. However, the transfer between these two states requires the creation of coherences which in general are subject to transverse relaxation. The two principle transverse relaxation mechanisms are dipole-dipole (DD) relaxation and relaxation due to the chemical shift anisotropy (CSA) of spins $`I`$ and $`S`$. The quantum mechanical equation of motion (Liouville-von Neumann equation) for the density operator $`\rho `$  is given by

$`\dot{\rho }`$ $`=`$ $`\pi J[i2I_zS_z,\rho ]+\pi k_{DD}[2I_zS_z,[2I_zS_z,\rho ]+\pi k_{CSA}^I[I_z,[I_z,\rho ]]+\pi k_{CSA}^S[S_z,[S_z,\rho ]]`$ (1)
$`+\pi k_{DD/CSA}^I[2I_zS_z,[I_z,\rho ]]+\pi k_{DD/CSA}^S[2I_zS_z,[S_z,\rho ]],`$
where $`J`$ is the heteronuclear coupling constant. The rates $`k_{DD}`$, $`k_{CSA}^I`$, $`k_{CSA}^S`$ represent auto-relaxation rates due to DD relaxation, CSA relaxation of spin $`I`$ and CSA relaxation of spin $`S`$, respectively. The rates $`k_{DD/CSA}^I`$ and $`k_{DD/CSA}^S`$ represent cross-correlation rates of spin $`I`$ and $`S`$ caused by interference effects between DD and CSA relaxation. These relaxation rates depend on various physical parameters, such as the gyromagnetic ratios of the spins, the internuclear distance, the CSA tensors, the strength of the magnetic field and the correlation time of the molecular tumbling . Let the initial density operator $`\rho (0)=A`$ and $`\rho (t)`$ denote the density operator at time $`t`$. The maximum efficiency of transfer between $`A`$ and target operator $`C`$ is defined as the largest possible value of $`trace(C^{}\rho (t))`$ for any time $`t`$  (by convention operators A and C are normalized).

The main result of this letter is as follows. The maximum efficiency of transfer between the operators $`I_z`$ and $`2I_zS_z`$ depends only on the scalar coupling constant $`J`$ and the net auto-correlated and cross-correlated relaxation rates of spin $`I`$, given by $`k_a=k_{DD}+k_{CSA}^I`$ and $`k_c=k_{DD/CSA}^I`$, respectively. This physical limit $`\eta `$ is given by

$$\eta =\sqrt{1+\zeta ^2}\zeta ,$$
(2)
where

$$\zeta =\sqrt{\frac{k_a^2k_c^2}{J^2+k_c^2}}.$$
(3)
The derivation of the maximum efficiency rests on the basic principles of optimal control theory and the development of a new class of control systems (see suppl. material for details). The optimal transfer scheme (CROP: cross-correlated relaxation optimized pulse) has two constants of motion. If $`l_1(t)`$ and $`l_2(t)`$ denote the two-dimensional vectors $`(I_x(t),I_y(t))`$ and $`(2I_xS_z(t),2I_yS_z(t))`$, respectively, then throughout the transfer process the ratio of the magnitudes of the vectors $`l_2`$ and $`l_1`$ is maintained constant at $`\eta `$. Furthermore, the angle $`\gamma ^{}`$ between $`l_1`$ and $`l_2`$ is constant throughout. The two constants of motion of the optimal transfer scheme determine the amplitude and phase of the rf field at each point in time and explicit expressions for the optimal pulse sequence can be derived (see suppl. material).

We now consider two important limiting cases of this problem:

(I) In the case when $`k_a>0`$ and $`k_c=0`$ (no cross-correlated relaxation), the optimal efficiency $`\eta `$ is equal to $`\sqrt{1+\frac{k_a^2}{J^2}}\frac{k_a}{J}<1`$ (see yellow curves in Figs. 2 and 3) and the optimal angle $`\gamma ^{}`$ is $`\pi /2`$ .

(II) In the limit where the cross-correlation coefficient $`k_c/k_a`$ approaches 1, the optimal transfer efficiency $`\eta `$ approaches 1 (see black curves in Figs. 2 and 3) and $`\gamma ^{}`$ approaches $`\pi `$. Surprisingly, in this case using optimal control it is possible to transfer coherence without any loss in the presence of relaxation. In the limit of large relaxation rates $`k_a`$, this relaxation-optimized transfer mechanism gains up to 100% compared to state of the art transfer schemes.

The optimal transfer scheme is best illustrated by decomposing the initial operator $`I_z`$ as a sum of the two operators $`I_zS_\alpha =\frac{I_z}{2}+I_zS_z`$ and $`I_zS_\beta =\frac{I_z}{2}I_zS_z`$. The transverse components $`I_xS_\alpha `$, $`I_yS_\alpha `$ and $`I_xS_\beta `$, $`I_yS_\beta `$ relax with rates $`k_a+k_c`$ and $`k_ak_c`$, respectively. When $`k_c/k_a`$ approaches 1, the transverse operators $`I_xS_\beta `$ and $`I_yS_\beta `$ do not relax. The optimal control in this case reduces to selectively inverting $`I_zS_\beta `$ to $`I_zS_\beta `$ by weak rf irradiation at the frequency ($`J/2`$) of the slowly relaxing multiplet component. Such selective inversions have been performed in the past in the absence of cross-correlated relaxation . However, since the component $`I_zS_\alpha `$, which we do not want to invert, has a large transverse relaxation rate given by $`k_a+k_c`$, it is possible to carry out the selective inversion process much more rapidly. In Fig. 2, optimal trajectories of the two multiplet components are shown for several cross-correlation coefficients $`k_c/k_a`$ and $`k_a=J`$.

In Fig. 4, the optimal rf amplitude and irradiation frequency of a CROP sequence is shown as a function of time for the case $`k_c/k_a=0.75`$ and $`k_a=J`$. Although the ideal sequence has a long duration, most of the transfer occurs in a relatively short time window, outside of which the rf amplitude is vanishingly small. The transfer efficiency $`\eta `$ is shown in Fig. 3 A for several ratios $`k_c/k_a`$ as a function of the auto-correlated relaxation rate $`k_a/J`$. For the case $`k_c/k_a=0.75`$, the physical limit of the transfer efficiency is compared in Fig. 3 B to the transfer efficiency of conventional transfer schemes.

The optimal control methods for the transfer from $`I_z`$ to $`2I_zS_z`$ in the presence of cross-correlated relaxation immediately extend to other routinely used transfer, such as inphase to inphase transfer ($`I_xS_x`$)  and single transition to single transition transfer ($`2I_xS^\alpha 2I^\alpha S_x`$) . Since the operators $`I_z`$, $`S_z`$ and $`2I_zS_z`$ do not decay, the optimal efficiency for the transfer $`I_x`$ to $`S_x`$ is achieved by first rotating $`I_x`$ to $`I_z`$ (which can be done rapidly with neglegible loss). Then $`I_z`$ is transferred optimally to $`2I_zS_z`$ with efficiency $`\eta `$ (Eq. 2), followed by the optimal transfer of $`2I_zS_z`$ to $`S_z`$, which is finally rotated rapidly to $`S_x`$. The optimal transfer $`2I_zS_zS_z`$ is analogous to the optimal transfer $`I_z2I_zS_z`$. The efficiency $`\eta ^{}`$ for this transfer is also given by Eq. (2), where the rates $`k_a`$ and $`k_c`$ are replaced by the corresponding rates $`k_a^{}=k_{DD}+k_{CSA}^S`$ and $`k_c^{}=k_{DD/CSA}^S`$ for spin $`S`$ and $`\zeta `$ is replaced by the corresponding $`\zeta ^{}`$. The maximum efficiency for the transfer $`I_xS_z`$ is the product of the efficiencies of the individual steps (see Table 1).

In the light of increasing use and superiority of TROSY (Transverse Relaxation-Optimized Spectroscopy) methods , the single transition to single transition transfer $`2I_xS^\alpha 2I^\alpha S_x`$ is important in NMR applications to structural biology. It is of both theoretical and practical interest to establish the physical limits for this transfer. This transfer can be achieved optimally as a sequence of the following steps. First the term $`2I_xS^\alpha `$ is rapidly rotated to $`2I_zS^\alpha =I_z+2I_zS_z`$. In a second step, $`2I_zS_z`$ is transferred via CROP to $`S_z`$, followed by the CROP transfer of $`I_z`$ to $`2I_zS_z`$. This completes the transfer from $`2I_xS^\alpha `$ to $`2I^\alpha S_z`$ which is finally rapidly rotated to $`2I^\alpha S_x`$. The maximum overall transfer efficiency is given by $`\sqrt{\eta ^2+\eta _{}^{}{}_{}{}^{2}}`$ (c.f. Table 1).

## 3 Experimental results

The performance of the analytically derived CROP sequences was tested experimentally using the coupled two-spin system of 13C-labeled sodium formate with a coupling of $`J=193.6`$ Hz between the 13C spin (denoted $`I`$) and the 1H spin (denoted $`S`$). In order to control the rotational correlation time, sodium formate was dissolved in a mixture of 96% D6-glycerol and 4% D2O. The viscosity of this solvent can be conveniently adjusted through a variation of temperature. The experiments were performed at a temperature of 256.5 K where $`k_a/J1`$ (see Fig. 5 A) and 260 K where $`k_a/J0.5`$ (see Fig. 5 B). At a magnetic field of 17.6 T, the experimentally determined ratio of cross and auto correlation rate was $`k_c/k_a0.75`$. In the preparation phase of the experiments, the thermal equilibrium 1H magnetization was dephased by applying a 90 proton pulse followed by a pulsed magnetic field gradient. The transfer efficiency of 13C polarization $`I_z`$ to $`2I_zS_z`$ was measured for the novel CROP sequence, as well as for INEPT , CRIPT  and CRINEPT  sequences. Finally, a hard 90$`{}_{}{}^{}{}_{y}{}^{}`$ proton pulse was applied to transform $`2I_zS_z`$ to $`2I_zS_x`$ and the amplitude of the resulting proton anti-phase signal was measured. The resulting experimental transfer amplitudes are shown in Fig. 5 as a function of the transfer time. CROP sequences were truncated symmetrically to acquire transfer amplitudes also for finite mixing times. Experimentally, the optimal transfer time of the CROP sequence was found to be 7.5 ms. This is a compromise between losses due to the truncation of the (very long) CROP sequence and losses due to the non-zero relaxation rates of the terms $`I_zS_z`$. The experimentally determined relaxation time of these terms was about 50 ms. In spite of these non-idealities of the model system, the CROP sequences are substantially more efficient than the conventional sequences. In Fig. 5 A and B, the experimental gains compared to CRINEPT are 34% and 22%, respectively. We found the although the optimal pulse sequences were designed for specific rates $`k_a`$ and $`k_c`$, they were robust to variations in these parameters.

## 4 Conclusion

Here, we derived for the first time upper achievable physical limits on the efficiency of coherence and polarization transfer for two coupled spins in the presence of very general decoherence mechanisms that include cross-correlated relaxation. In this letter, the focus was on the study of polarization and coherence transfer between an isolated pair of scalar coupled heteronuclear spins in the spin diffusion limit. For this example, new transfer schemes were found which yield substantial gains (of up to 100%) in transfer efficiency over conventional methods. With these physical limits established, it is expected that significant improvement can be achieved over state of the art experiments in protein NMR spectroscopy. Work is in progress to incorporate practical considerations like broadbandedness and robustness with respect to variations of relaxation rates and experimental imperfections. The methods presented here can be generalized for finding relaxation optimized pulse sequences in larger spin systems as commonly encountered e.g. in backbone and side chain assignments in protein NMR spectroscopy. Furthermore these methods directly extend to other routinely used experiments like excitation of multiple quantum coherence . The most surprising aspect of the presented results is that in spite of large relaxation rates, it is possible to exploit the structure of relaxation and have decoherence-free evolution by steering the system through a decoherence-free subspace. It is also expected that the methods presented here will be further developed to minimize decoherence losses in various proposed implementations of quantum information processing.

## 5 Supplementary Material: Derivation of Optimal Control of Coherence Transfer under Cross-Correlated Relaxation

The derivation of optimal control assumes that the two heteronuclear spins have well separated resonance frequencies, allowing for fast selective manipulation of each spin on a time-scale determined by the coupling $`J`$ and the relaxation rates $`k_a`$ and $`k_c`$. Therefore the Cartesian spin operator $`I_z`$ can be transformed to an operator of the form $`I_x\mathrm{cos}\beta _1+I_z\mathrm{sin}\beta _1`$ by the use of strong, spin-selective radio frequency (rf) pulses without relaxation losses (see Fig. 6). Let $`r_1(t)`$ represent the magnitude of polarization and in-phase coherence on spin $`I`$ at any given time $`t`$, i.e. $`r_1^2(t)=I_x^2+I_y^2+I_z^2`$ (where e.g. $`I_x=\mathrm{trace}\{\rho I_x\}`$ represents the expectation value of $`I_x`$). Let $`l_1(t)`$ be the magnitude of in-phase coherence on spin $`I`$, i.e. $`l_1^2(t)=I_x^2+I_y^2`$ and $`\beta _1=\mathrm{cos}^1\frac{l_1}{r_1}`$ (see Fig. 6). Using rf fields, we can exactly control the angle $`\beta _1`$. Hence we can think of $`\mathrm{cos}\beta _1`$ as a control parameter and denote it by $`u_1`$ (see Fig. 6). Observe that the operator $`I_z`$ is invariant under the evolution equation (1), whereas $`I_x`$ and $`I_y`$ evolve under the $`J`$ coupling and also relax. For example, $`I_x`$ evolves under the coupling to $`2I_yS_z`$ with rate $`J`$ and cross relaxes to $`2I_xS_z`$ with rate $`k_c`$. In the plane defined by the operators $`2I_xS_z`$ and $`2I_yS_z`$, the direction in which antiphase coherence begins to build up from an initial coherence $`I_x`$ forms an angle

$$\theta =\mathrm{tan}^1(\frac{J}{k_c})$$

with the axis $`2I_xS_z`$.

As the operators $`2I_xS_z`$ and $`2I_yS_z`$ are produced, they also relax. Let $`l_2(t)`$ measure the magnitude of the total antiphase coherence at time $`t`$, i.e. $`l_2^2(t)=2I_xS_z^2+2I_yS_z^2`$. By use of rf pulses it is possible to rotate the antiphase operators $`2I_xS_z`$ and $`2I_yS_z`$ to $`2I_zS_z`$, which is protected from relaxation. Let $`r_2`$ represent the total magnitude of the expectation values of these bilinear operators, i.e. $`r_2^2(t)=l_2^2+2I_zS_z^2`$ and $`\beta _2=\mathrm{cos}^1\frac{l_2}{r_2}`$ (see Fig. 6). We can control the angle $`\beta _2`$ and we define $`\mathrm{cos}\beta _2`$ as a second control parameter $`u_2`$ (see Fig. 6).

We superimpose the transverse planes defined by $`I_x`$ and $`I_y`$ with the plane defined by $`2I_xS_z`$ and $`2I_yS_z`$ such that $`I_x`$ is aligned with $`2I_xS_z`$ (see Fig. 7). If $`\gamma `$ represents the angle between $`l_1`$ and $`l_2`$, then by definition of $`\theta `$ we have

$`{\displaystyle \frac{d}{dt}}l_1(t)`$ $`=`$ $`\pi [k_al_1(t)\sqrt{k_c^2+J^2}\mathrm{cos}(\theta +\gamma )l_2(t)]`$
$`{\displaystyle \frac{d}{dt}}l_2(t)`$ $`=`$ $`\pi [k_al_2(t)\sqrt{k_c^2+J^2}\mathrm{cos}(\theta \gamma )l_1(t)].`$
This can be rewritten as

$$\frac{d}{dt}\left[\begin{array}{c}l_1(t)\\ l_2(t)\end{array}\right]=\pi J\left[\begin{array}{cc}\xi & \chi \mathrm{cos}(\theta +\gamma )\\ \chi \mathrm{cos}(\theta \gamma )& \xi \end{array}\right]\left[\begin{array}{c}l_1(t)\\ l_2(t)\end{array}\right],$$
(4)
where

$$\xi =k_a/J$$

and

$$\chi =\sqrt{1+\left(\frac{k_c}{J}\right)^2}.$$

For a given value of $`\beta _1`$ and $`\beta _2`$, we have $`l_1(t)=r_1(t)\mathrm{cos}(\beta _1)`$ and $`l_2(t)=r_2(t)\mathrm{cos}(\beta _2)`$. Since $`\frac{dI_z}{dt}=0`$ and $`\frac{d2I_zS_z}{dt}=0`$, we get $`\dot{r}_1(t)=\dot{l}_1(t)\mathrm{cos}(\beta _1)`$ and $`\dot{r}_2(t)=\dot{l}_2(t)\mathrm{cos}(\beta _2)`$. Substituting for $`\dot{l_1}`$ and $`\dot{l_2}`$, we then get

$$\frac{d}{dt}\left[\begin{array}{c}r_1(t)\\ r_2(t)\end{array}\right]=\pi J\left[\begin{array}{cc}\xi u_1^2& \chi u_1u_2\mathrm{cos}(\theta +\gamma )\\ \chi u_1u_2\mathrm{cos}(\theta \gamma )& \xi u_2^2\end{array}\right]\left[\begin{array}{c}r_1(t)\\ r_2(t)\end{array}\right].$$
(5)
Given the dynamical system in equation (5), we want to find the optimal values of $`u_1(t)`$, $`u_2(t)`$, and $`\gamma (t)`$, so that starting from $`r_1(0)=1`$ we achieve the largest value for $`r_2`$.

As the operator $`I_z`$ is transferred to $`2I_zS_z`$, the ratio $`\frac{r_2}{r_1}`$ increases from $`0`$ to $`\mathrm{}`$. The optimal choice of $`u_1`$ and $`u_2`$ must ensure that the ratio of gain $`dr_2`$ in $`r_2`$ to loss $`dr_1`$ in $`r_1`$ for incremental time steps $`dt`$ is maximized at each step. This ratio is

$$\frac{|\dot{r_2}|}{|\dot{r_1}|}=\frac{\xi u_2^2r_2+\chi u_1u_2r_1\mathrm{cos}(\theta \gamma )}{\xi u_1^2r_1\chi u_1u_2\mathrm{cos}(\theta +\gamma )r_2}.$$

Let $`\frac{u_2r_2}{u_1r_1}=g`$, then the above expression can be re-written as

$$\frac{|\dot{r_2}|}{|\dot{r_1}|}=\frac{r_1}{r_2}\frac{\xi g^2+\chi g\mathrm{cos}(\theta \gamma )}{\xi \chi gcos(\theta +\gamma )}.$$

This expression needs to be maximized with respect to choice of $`g`$ and $`\gamma `$. Let these optimal values be $`\eta `$ and $`\gamma ^{}`$ respectively. Then

$$\frac{d\frac{|\dot{r_2}|}{|\dot{r_1}|}}{dg}|_{g=\eta ,\gamma =\gamma ^{}}=0$$

yields

$$\frac{1}{\eta }\mathrm{cos}(\theta \gamma ^{})+\eta \mathrm{cos}(\theta +\gamma ^{})=\frac{2\xi }{\chi },$$
(6)
which yields $`\frac{|\dot{r_2}|}{|\dot{r_1}|}=\frac{r_1}{r_2}\eta ^2`$. Now the value of $`\gamma ^{}`$ in equation (6) is such that it maximizes $`\eta `$. Differentiating both sides of equation (6) with respect to $`\gamma ^{}`$ and substituting $`\frac{d\eta }{d\gamma ^{}}=0`$, we obtain that

$$\frac{1}{\eta }\mathrm{sin}(\theta \gamma ^{})\eta \mathrm{sin}(\theta +\gamma ^{})=0.$$
(7)
The optimal $`\eta `$ and $`\gamma ^{}`$ then satisfy equations (6, 7). The two equations can then be solved to give

$$\eta =\sqrt{\zeta ^2+1}\zeta ,$$
(8)
where $`\zeta =\sqrt{\frac{k_a^2k_c^2}{J^2+k_c^2}}`$ and optimal $`\gamma ^{}=\mathrm{tan}^1\frac{1\eta ^2}{(1+\eta ^2)\mathrm{cot}\theta }`$. By substituting the optimal control law $`\frac{u_2(t)}{u_1(t)}=\frac{\eta r_1(t)}{r_2(t)}`$, and integrating equation (5), we see that $`r_2`$ increases from $`0`$ to $`\sqrt{\zeta ^2+1}\zeta `$, which is then the maximum achievable transfer.

We see that throughout the optimal transfer, the angle $`\gamma `$ is maintained constant at $`\gamma ^{}`$. The optimal control ensures that the ratio of the transverse components $`l_2(t)`$ and $`l_1(t)`$ is always maintained constant at $`\eta `$. These two constraints can now be used to get explicit expressions for the magnitude and phase of the optimal rf-field.

Let $`\varphi `$ denote the phase of the rf-field relative to $`l_1`$$`\gamma \varphi `$ relative to $`l_2`$ ) and let $`A`$ be its amplitude. Let $`dl_1^{}`$ denote the change in transverse component perpendicular to the vector $`l_1`$ by application of the $`rf`$ field in small time $`dt`$. Then observe $`dl_1^{}=2\pi AI_z\mathrm{cos}(\varphi )dt`$. Similarly $`dl_2^{}=2\pi A2I_zS_z\mathrm{cos}(\gamma \varphi )dt`$. If $`\frac{l_2}{l_1}`$ is maintained at $`\eta `$ then the angle $`\gamma `$ does not change due to the evolution equation (4). Therefore we only need to consider the change in $`\gamma `$, due to the rf field. If $`\gamma `$ is maintained constant, then $`\frac{dl_1^{}}{l_1}=\frac{dl_2^{}}{l_2}`$ . This gives $`\mathrm{tan}(\beta _1)\mathrm{cos}(\varphi )=\mathrm{tan}(\beta _2)\mathrm{cos}(\gamma ^{}\varphi )`$ because $`\frac{I_z}{l_1}=\mathrm{tan}\beta _1`$ and $`\frac{2I_zS_z}{l_2}=\mathrm{tan}\beta _2`$. This then implies that

$$\varphi =\mathrm{tan}^1\left(\frac{\mathrm{tan}\beta _1}{\mathrm{tan}\beta _2\mathrm{sin}\gamma ^{}}\mathrm{cot}\gamma ^{}\right).$$

The amplitude can be determined from the condition that $`\frac{l_2(t)}{l_1(t)}`$ is maintained constant. This implies that $`\frac{dl_1}{l_1}=\frac{dl_2}{l_2}`$. Substituting $`dl_1=(\xi \pi Jl_1\chi \pi J\mathrm{cos}(\theta +\gamma )l_2+2\pi AI_z\mathrm{sin}(\varphi ))dt`$ and $`dl_2=(\xi \pi Jl_2\chi \pi J\mathrm{cos}(\theta \gamma )l_22\pi A2I_zS_z\mathrm{sin}(\gamma \varphi ))dt`$, we get

$$A=\frac{1}{2\pi }\frac{\left(\mathrm{cos}(\theta \gamma ^{})\eta ^2\mathrm{cos}(\theta +\gamma ^{})\right)\chi J}{\left(\mathrm{tan}\beta _1\mathrm{sin}\varphi +\mathrm{tan}\beta _2\mathrm{sin}(\gamma ^{}\varphi )\right)\eta }.$$

The expressions of $`A`$ and $`\varphi `$ are given in terms of the state of the system (angle $`\beta _1`$ and $`\beta _2`$). We can insert these in the equations for how $`\beta _1`$ and $`\beta _2`$ evolve as a function of $`A`$ and $`\varphi `$ to get explicit expressions of $`\beta _1`$, $`\beta _2`$, $`\varphi `$ and $`A`$ as a function of time. This gives us the amplitude and phase of the optimal rf pulse as a function of time.