detect horizon
Home Atmospheric Boundary Layer Dynamics Evolution of Refractive Index Formulas: From Edlén (1953) to Ciddor (1996)
Atmospheric Boundary Layer Dynamics

Evolution of Refractive Index Formulas: From Edlén (1953) to Ciddor (1996)

An analysis of the technical evolution from the 1953 Edl n formula to the 1996 Ciddor equation, exploring how atmospheric refractivity gradient mapping shapes modern geodesy.

Siobhan O'Malley
Siobhan O'Malley 3/19/2026
Evolution of Refractive Index Formulas: From Edlén (1953) to Ciddor (1996) All rights reserved to detecthorizon.com

Atmospheric refractivity gradient mapping is a technical discipline centered on the empirical quantification and predictive modeling of atmospheric optical phenomena. These phenomena emerge from localized variations in the refractive index of air, which are dictated by fluctuations in density, temperature, and humidity. By utilizing high-precision lidar systems and ground-based refractometers, researchers can meticulously map these gradients to identify distinct atmospheric layers, such as inversion layers and turbulent eddies. These features induce measurable deviations in the apparent position of celestial objects and terrestrial targets, particularly at low elevation angles.

The evolution of this field is intrinsically linked to the mathematical refinement of refractive index formulas. The transition from the 1953 Edl n formula to the 1996 Ciddor equation represents a significant advancement in precision, allowing for more accurate geodetic surveying and astronomical observation. These formulas provide the framework for processing interferometric data to resolve minute angular displacements, enabling the determination of the effective horizon line and the development of sophisticated optical propagation models for long-range atmospheric sensing and communication systems.

What changed

The progression from early mid-century standards to contemporary refractive index modeling involved several fundamental shifts in both methodology and atmospheric assumptions. The following points summarize the primary transitions in the field:

  • Precision Thresholds:The 1953 Edl n formula provided a precision of approximately 1  10⁻⁷, which was sufficient for mid-century optical measurements but became inadequate for modern laser-based distance measurements. The 1996 Ciddor equation improved this precision to roughly 1  10⁻⁸.
  • Carbon Dioxide Standardization:Bengt Edl n originally assumed a fixed CO2 concentration of 300 parts per million (ppm). The Ciddor model and subsequent updates allowed for variable CO2 concentrations, addressing the rise in global atmospheric CO2, which has surpassed 400 ppm.
  • Humidity Integration:Early models treated water vapor as a secondary correction factor. The Ciddor equation integrated the refractive effects of water vapor directly into the core dispersion formula, utilizing a more strong phase-refractivity approach.
  • Spectral Range Expansion:While the Edl n formula was optimized for the visible spectrum, the Ciddor equation extended high-precision modeling further into the infrared and ultraviolet regions, critical for satellite-to-ground optical links.
  • Geodetic Standards:The International Association of Geodesy (IAG) formally shifted its recommendations to favor the Ciddor equation for long-range distance measurements and atmospheric delay corrections in satellite laser ranging (SLR).

Background

The refractive index of air (n) is a dimensionless number that describes how light propagates through the atmosphere compared to a vacuum. Because the atmosphere is a heterogeneous medium, n is not a constant; it varies with altitude, geographic location, and local weather conditions. Atmospheric refractivity (N) is often expressed in ‘N-units’ using the relation N = (n - 1)  10⁶. Mapping the gradient of this value (∇N) is essential for correcting the bending of light paths, a phenomenon known as atmospheric refraction.

The physical basis for these calculations lies in the Gladstone-Dale relation, which suggests that the refractivity of a gas is proportional to its density. However, for a multi-component gas like air, the interactions between dry air molecules, water vapor, and carbon dioxide require more complex Sellmeier-type equations. These equations account for the dispersion of light, where the refractive index changes based on the wavelength of the light being used.

The Edl n Formula of 1953

Bengt Edl n’s 1953 paper, "The Refractive Index of Air," published in the Journal of the Optical Society of America, established the first widely accepted standard for atmospheric optics. Edl n’s work synthesized previous experimental data to create a dispersion formula for the standard atmosphere. Standard conditions were defined at the time as dry air at 15C, a pressure of 1013.25 hPa, and a CO2 concentration of 300 ppm.

Edl n’s formula was notable because it offered a concise mathematical expression for the refractive index as a function of wavelength. It utilized a Cauchy-type dispersion equation, which was later revised in 1966 to better account for humidity through an additive term. This revised Edl n formula remained the standard for several decades, serving as the basis for most geodetic and meteorological calculations during the first half of the Space Age.

The Ciddor Equation of 1996

By the 1990s, the limitations of the Edl n formula became apparent as laser technology and satellite geodesy demanded higher accuracy. In 1996, P.E. Ciddor published "Refractive Index of Air: New Equations for the Visible and Near Infrared" in Applied Optics. This work replaced the simpler Cauchy-type dispersion with a more detailed model based on the Lorentz-Lorenz equation.

The Ciddor equation is distinguished by its handling of air as a mixture of dry air and water vapor. Instead of applying a correction to dry air, it calculates the refractive index of each component separately and then combines them based on the partial pressures and temperatures of the specific environment. This approach is significantly more accurate in tropical or high-humidity environments where water vapor significantly alters the refractive profile of the lower atmosphere.

Mathematical Impact of Temperature and Pressure

The refractive index is highly sensitive to the thermodynamic state of the atmosphere. Temperature (T) and pressure (P) are the primary drivers of density variations. According to the Ideal Gas Law, density is directly proportional to pressure and inversely proportional to absolute temperature. Consequently, a decrease in temperature or an increase in pressure leads to an increase in the refractive index.

In atmospheric refractivity gradient mapping, vertical temperature gradients are of particular interest. Under standard conditions, the temperature decreases with altitude, leading to a gradual decrease in refractivity. However, in the case of a temperature inversion—where warm air sits above cold air—the refractivity gradient can become sharp enough to cause significant optical ducting or the appearance of mirages. High-precision algorithms must account for these non-linear temperature profiles to correct for the "apparent" vs. "true" zenith angle of observed objects.

The Role of CO2 Concentration Shifts

One of the most critical updates in the transition to the Ciddor standards was the treatment of carbon dioxide. In 1953, the global average for CO2 was approximately 310 ppm, and Edl n’s 300 ppm baseline was a reasonable approximation. However, by the late 20th century, anthropogenic emissions had shifted this baseline significantly. Because CO2 has a higher refractivity than oxygen or nitrogen, failing to account for its concentration introduces a systematic bias in measurements.

The IAG updated its standards to ensure that modern geodetic equipment can input the current CO2 levels as a variable. While the impact on a short distance might be negligible, over the thousands of kilometers involved in Satellite Laser Ranging (SLR) or Very Long Baseline Interferometry (VLBI), the difference can amount to several millimeters of path length error, which is unacceptable for modern geophysical research.

Applications in Advanced Sensing

The rigorous application of the Ciddor equation and gradient mapping is vital for several high-stakes industries. In astronomical observation, it allows for the sub-arcsecond positioning of stars by correcting for the "lifting" effect of the atmosphere at the horizon. In geodetic surveying, it enables the measurement of tectonic plate movement with millimeter-level precision across vast distances.

Furthermore, the development of long-range optical communication systems depends on these models to predict and mitigate the effects of atmospheric scintillation and beam wander. By mapping the refractivity gradients in real-time using lidar, these systems can employ adaptive optics to change the shape of the laser beam, compensating for the heterogeneous medium through which the light must travel. This fusion of historical refractive theory and modern sensing technology remains a cornerstone of atmospheric physics.

Tags: #Atmospheric refractivity # Ciddor equation # Edl n formula # refractive index of air # geodesy # lidar mapping # optical propagation
Share Article
Siobhan O'Malley

Siobhan O'Malley Contributor

Siobhan covers the practical applications of refractivity mapping in long-range optical communication systems. Her writing often bridges the gap between theoretical physics and next-generation signal stability.

detect horizon