Assessment of Satellite-Derived Surface Reflectances by NASA’s CAR Airborne Radiometer over Railroad Valley Playa

Abstract

CAR (Cloud Absorption Radiometer) is a multi-angular and multi-spectral airborne radiometer instrument, whose radiometric and geometric characteristics are well calibrated and adjusted before and after each flight campaign. CAR was built by NASA (National Aeronautics and Space Administration) in 1984. On 16 May 2008, a CAR flight campaign took place over the well-known calibration and validation site of Railroad Valley in Nevada, USA (38.504°N, 115.692°W). The campaign coincided with the overpasses of several key EO (Earth Observation) satellites such as Landsat-7, Envisat and Terra. Thus, there are nearly simultaneous measurements from these satellites and the CAR airborne sensor over the same calibration site. The CAR spectral bands are close to those of most EO satellites. CAR has the ability to cover the whole range of azimuth view angles and a variety of zenith angles depending on altitude and, as a consequence, the biases seen between satellite and CAR measurements due to both unmatched spectral bands and unmatched angles can be significantly reduced. A comparison is presented here between CAR’s land surface reflectance (BRF or Bidirectional Reflectance Factor) with those derived from Terra/MODIS (MOD09 and MAIAC), Terra/MISR, Envisat/MERIS and Landsat-7. In this study, we utilized CAR data from low altitude flights (approx. 180 m above the surface) in order to minimize the effects of the atmosphere on these measurements and then obtain a valuable ground-truth data set of surface reflectance. Furthermore, this study shows that differences between measurements caused by surface heterogeneity can be tolerated, thanks to the high homogeneity of the study site on the one hand, and on the other hand, to the spatial sampling and the large number of CAR samples. These results demonstrate that satellite BRF measurements over this site are in good agreement with CAR with variable biases across different spectral bands. This is most likely due to residual aerosol effects in the EO derived reflectances.

1. Introduction

Since the 1970s, EO (Earth-Observation) polar orbiter satellites for climate monitoring have increased in number and capabilities leading to a continuous improvement in terms of the accuracy and quality of measurements [1]. Consequently, as satellite data cover much wider areas compared to ground data, climate models of different ranges (short, medium) and areal coverage (global, land, ocean, atmosphere) are more and more dependent on EO satellite data [2]. These datasets have also greatly increased in size and complexity. However, advances in supercomputing and cloud computing in recent years in terms of memory storage, processing time, and accessibility has significantly improved the handling and the processing of such large datasets in the climate domain [3]. Nowadays, EO satellite data constitute the main input for most climate models [2].

Land surface reflectance products such as albedo, fAPAR (Fraction of Absorbed Photosynthetically Active Radiation), and LAI (Leaf Area Index) are part of the suite of essential climate variables (ECV) in the terrestrial domain. Moreover, several atmospheric ECVs also utilize these products, namely, surface BRF (Bidirectional Reflectance Factor), for minimizing the influence of the land surface reflectance on their results [4]. Therefore, the accuracy of any land ECV product derived from optical-domain satellite-borne sensors is related to the accuracy of surface reflectance or, more precisely, to the accuracy of the atmospheric correction that is applied to level 1 products of ToA (Top of Atmosphere) reflectance for generating land surface reflectance [5,6]. The importance of producing high quality surface reflectance products is clear, as well as being able to evaluate the uncertainty of these products in order to propagate this to the derived higher level products, such as fAPAR [5].

Land surface reflectance is the ratio of reflected to incoming energy for a given spectral solar band and under specific view and illumination (solar) angles. Thus, it is unitless and has values in the interval $[0,1]$. However, this value is often multiplied by $\pi$ in order to represent BRF, which is the ratio of the surface reflectance to a Lambertian surface under the same angular conditions and in the same spectral region. Thus, BRF values can exceed 1, especially over bright surfaces such as snow [7].

The assessment and validation of satellite-derived surface reflectance requires the collection of ground-truth data to be used as reference. Those data are often provided by in situ instrumentation (radiometers, spectrometers) of different types (hand held, ground based, tower based, airborne, etc.). Direct assessment of a value of satellite-derived surface reflectance requires temporally and spatially coincident ground measurement taken with the same viewing and illumination angles. Ideally, measurements should be taken under clear sky conditions to minimise the impact of the atmosphere (and any potential changes in atmospheric conditions). However, achieving such ideal conditions is hard to realise when utilising in situ instrumentation, especially when assessing satellites of low and moderate spatial resolution, which is the case for most satellites involved in climate monitoring [1], as ground measurements of surface reflectance have very small footprints compared to these of climate monitoring satellites. However, to minimise the effects of surface heterogeneity, assessments are often undertaken over homogeneous surfaces [8,9,10].

Some studies are based on ground measurements of albedo instead of surface reflectance, but these often rely upon the assumption that the surface is Lambertian in order to retrieve albedo from surface reflectance [11]. This has some advantages, as the in situ data can then be used to represent a wider area and differences in view angles can be tolerated.

Airborne instruments have also been used in surface reflectance assessment but with lower emphasis compared to what has been achieved with ground-based instruments. Many of these airborne instruments were developed to provide pre-flight data for specific satellite instruments like MASTER [12], which was developed for both MODIS (MODerate-resolution Imaging Spectroradiometer) and ASTER (Advanced Spaceborne Thermal Emission and Reflection) , and AirMISR (Airborne Multi-angle Imaging SpectroRadiometer) which was developed for MISR (Multi-angle Imaging SpectroRadiometer) [13]. However, some multiangular airborne instruments have been used in the validation of high level products such as BRDF (Bidirectional Reflectance Distribution Function) and albedo, and this is the case for POLDER (POLarization and Directionality of the Earth’s Reflectances), ASAS (Advanced Solid-state Array Spectroradiometer) and AirMISR [14,15,16].

This paper considers the use of the airborne CAR (Cloud Absorption Radiometer) instrument for the validation of satellite-borne sensors. CAR is a multispectral and multiangular instrument operational since 1984 [17]. In addition to its 14 narrow spectral bands dispersed between 340 nm and 2300 nm, CAR has the ability to cover a wide range of zenith and azimuth angles with an instantaneous field of view of 1° at a particular altitude. The lateral mobility of the aircraft in a circular flight track enables the CAR to take a nearly complete coverage of all azimuth angles for a fixed zenith angle of a full-hemisphere in a relatively short time of 2–3 min [17]. Furthermore, the aircraft on which CAR is mounted can fly at relatively low altitudes. At these low altitudes, it is assumed that the CAR data is less affected by the atmosphere, hence the atmospheric effects can be ignored in the measured reflectance [17]. All of these characteristics allow CAR to be one of the few instruments that can be used effectively for the assessment of satellite-derived surface reflectance. CAR data have been successfully used in the assessment of satellite products of BRDF and albedo over heterogeneous sites [18,19]. In this paper, we will demonstrate the usefulness of CAR data over the Railroad Valley site through comparison with several satellite datasets.

This paper is structured as follows: firstly, we will present the method including the study site and a description of the CAR and satellite data utilised. In the next section, we present the results of assessment that were obtained for particular satellite-derived products of surface reflectance. Finally, the two last sections are dedicated to a presentation and discussion of results and conclusions, respectively.

2. Method

2.1. Site

Data from the CAR campaign of 2008 over Railroad Valley Playa, NV, USA (38.504°N, 115.692°W) were selected to perform the assessment of satellite-derived surface reflectance. This choice can be justified by two main reasons. Firstly, the site is one of the key sites recommended by the CEOS (Committee on Earth Observation Satellites) for vicarious calibration [20]. It is chosen for its temporal stability, spatial homogeneity, flatness, the low levels of aerosol and high probability of clear skies. Together, these characteristics help to reduce differences between measurements, in particular where different sensor footprints are utilised or atmospheric correction is required. Secondly, the acquisition time (on 16 May 2008, from 18:00 to 19:30 UTC) over that site coincided with the overpasses of the three satellites used in this study (Landsat-7, Envisat, Terra), which enables comparisons between different sensors under similar illumination conditions. The background of the left image in Figure 1 shows a Railroad Valley true colour (red, green, blue) image that was captured by Landsat-7 within a few minutes to CAR’s data acquisition.

2.2. Data

2.2.1. CAR Data

CAR measures radiance with an $IFOV$ (Instantaneous Field Of View) of 1° over 14 spectral bands whose central wavelengths and widths are shown in

Table 1. Cloud Absorption Radiometer (CAR) band configuration during the campaign over Railroad Valley [17]. Bands that were deployed in this study are marked by (*).

CAR’s Spectral Bands [nm]
$340\pm 5$ *	$381\pm 3$ *	$472\pm 10$ *	$682\pm 11$ *	$870\pm 10$ *	$1036\pm 11$ *	$1219\pm 11$ *
$1273\pm 12$	$1556\pm 16$	$1656\pm 23$ *	$1737\pm 20$	$2103\pm 22$	$2205\pm 21$	$2302\pm 21$ *

. Note that this table represents the CAR band configuration between 1999 and 2011, as CAR band configurations had three slight amendments since its construction in 1984 [17]. Note that only eight of these 14 spectral bands can be deployed simultaneously.

CAR can scan a hemisphere (190°) perpendicular to the flight path with steps of 0.5° in the zenith direction, which leads to more than 380 measurements per scan. Thus, CAR can be configured either to focus on land only in such a way that it scans from horizon to nadir to horizon (VZA ∈ [−90°, 90°], where VZA denotes View Zenith Angles), or to focus on one side of both land and sky (right side or left side to the flight’s path) in such a way that it scans from top (zenith) to nadir (VZA ∈ [0°, 180°]). The latter scan pattern (top to nadir) was used over the target sites. However, as the scan is only allowed to cover the range of VZA [0°, 90°] for a given VAA (View Azimuth Angle), the CAR aircraft flies a circular path above the target sites, which results in covering the full VAA range [0°, 360°] (noting that the measurements are taken at slightly different times, between 2 and 3 min).

In Figure 1, the left image shows the aircraft path over the site at Railroad Valley Playa, distinguished by relative altitude, whilst the right image shows a sample of reflectance measurements. Nevertheless, sampling the site with closer (not identical) circular paths allows us to collect spatially scattered measurements that are necessary for representing each satellite’s pixel by a set of CAR’s samples, which help to smooth out surface spatial heterogeneity.

The surface reflectance data, or more precisely, the surface BRF data, for CAR at a spectral band, $\lambda$, are derived from radiance, solar irradiance, and solar zenith angle using the “Van-De-Hulst” formulation [21] as follows:

$$\rho =\frac{\pi L}{cos\left(SZA\right)E},$$

(1)

where L is the measured reflected or scattered intensity (radiance), E is the solar flux density (irradiance) incident at top of atmosphere (ToA), and SZA denotes the solar zenith angle. Note that both values of $\rho$ and L and E are calculated for a given spectral band (centred at wavelength $\lambda$). Furthermore, $\rho$ and L are calculated for a given angular conditions either for view angles (VZA and VAA), or for illumination angles (SZA and SAA). All data including radiance, irradiance, aircraft positions, times, geometries and other metadata are provided within the CAR D-product. For further information about CAR functionality and their products, we refer the reader to [17].

To minimize the effects of the atmosphere on the satellite-CAR comparison, we used only CAR data from low relative-altitude (relative to the surface) which was around 180 m for this study. As Figure 1 shows, most of the selected data are located in the southeast end of the Railroad Valley Playa site. The trade-off of using low altitude data is the high spatial resolution compared to most of the EO satellite data, which could impact the comparisons due to surface heterogeneity. However, as we will see in the results section, the impact of surface heterogeneity is limited over the study area.

Over a flat surface, the shape of the CAR footprint varies in accordance with VZA in such a way that the overall pattern takes an oval shape whose axes’ lengths increase with increasing VZA. That shape can be effectively approximated by an ellipsoid if VZA is not too high, and it takes a circular shape at nadir (VZA = 0). The lengths of the major axis and the minor axis of that ellipse reflect the spatial resolution (azimuthal and zenithal resolution) and they can be computed as follows:

$$\begin{array}{ccc}\hfill l_{major}& =& (tan(VZA+\frac{IFOV}{2})-tan(VZA-\frac{IFOV}{2}))h,\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill l_{minor}& =& \frac{2htan(\frac{IFOV}{2})}{cos(VZA)},\hfill \end{array}$$

(3)

where h denotes the aircraft relative altitude (distance to the surface) and $IFOV$ equals 1° (CAR’s IFOV). Thus, at a relative altitude of 180 m, CAR resolution at nadir (VZA = 0°) is around 3 m × 3 m and at VZA = 70° is around 27 m × 9 m. The right image in Figure 1 gives an idea of how CAR’s footprints change from viewing nadir to VZA = 70°, but with a relative altitude of 722 m over Railroad Valley. The image also shows that measurements with small VZA angles reveal more details, as they are characterized by their high spatial resolution. The colour scale in that image refers to the measured reflectance at a spectral band of 870 nm.

2.2.2. Satellite Data

Five products of surface reflectance have been evaluated in this study:

• MOD09: Surface reflectance derived from the top of atmosphere (ToA) data of Terra/MODIS, collection 6, at three spatial resolutions: 250 m (for two spectral bands only), 500 m and 1 km [22]. The atmospheric correction of MOD09 is performed through Second Simulation of a Satellite Signal in the Solar Spectrum (6S) radiative transfer models [23,24] that take as input ToA radiance data, water vapor, ozone, geopotential height, aerosol optical thickness, and digital elevation.
• MAIAC: MultiAngle Implementation of Atmospheric Correction [25] is another product of surface reflectance derived from Terra/MODIS ToA radiance data at two spatial resolutions: 500 m and 1 km. MAIAC computes Ross–Thick Li-Sparse (RTLS) BRDF parameters and then extracts BRF (Bidirectional Reflectance Factor) when the land surface is stable or changes slowly; otherwise, MAIAC follows the MODIS operational BRDF algorithm. MAIAC deploys a moving window of up to 16 days over MODIS ToA radiance data.
• MERIS: Medium Resolution Imaging Spectrometer, onboard the ENVISAT platform [26]. The atmospheric correction of MERIS is modeled using a simple single scattering model assuming that the atmospheric path radiance and absorption can be separated into two components, Rayleigh and aerosol in such a way that aerosol component is modeled by a simple angstrom exponent [27].
• Landsat-7: Surface reflectance product derived from ToA radiance data of Landsat-7 [28,29]. Like MOD09, the atmospheric correction of Landast7 is performed through Second Simulation of a Satellite Signal in the Solar Spectrum (6S) radiative transfer models [30].
• MISR: Multi-angle Imaging SpectroRadiometer is a multi-angular spaceborne instrument onboard the Terra platform. It deploys nine cameras at different view angles, over four spectral bands: $446\pm 21$ nm, $558\pm 15$ nm, $672\pm 11$ nm, and $867\pm 20$ nm. The MISR product that was used here is MISR_AM1_AS_LAND [31]. In this product ToA radiance data are first atmospherically corrected for generating the Hemispherical Directional Reflectance Factor (HDRF) and Bi-Hemispherical Reflectance (BHR) and then further atmospheric correction is applied on the two variables to remove all diffuse components and then generate BRF [13]. The accuracy of MISR’s atmospheric correction is strongly related to the accuracy of MISR’s aerosol product, which is considered accurate thanks to the nine view cameras, despite the fact that the aerosol product has a much lower spatial resolution (17.6 km × 17.6 km) [32].

2.3. Assessment Method

The flowchart in Figure 2 gives an overview of how satellite and CAR surface reflectance data have been processed and compared. The main outputs are the statistics of the intersected measurements (between CAR and satellites), and the differences between satellite’s measurements and CAR fitted data.

To evaluate the differences of satellite derived surface reflectance, we compared different satellite sensors with CAR under similar conditions and where the footprints of CAR and the satellite sensors overlapped or intersected. In this study, the angles are considered similar if the difference does not exceed 2°. However, in the case of the nadir view (VZA ≃ 0°), we remove the restriction on the VAA, as the effect of the view azimuth angles can be ignored at nadir [33].

The first stage in the comparison process is the reprojection of CAR data using DEMs (digital elevation models) followed by radiance converting to reflectance using the Equation (1). Then, CAR measurements were filtered to keep only those that intersect the region of interest (southeast of Railroad Valley Playa) (where low flight height (≃180 m) measurements were obtained). Once the data were collected, we computed and plotted the mean and standard deviation (stdev) for both CAR and the satellite sensor products, side by side.

Figure 3 shows an example of measurements (with their footprints) for both CAR (at band 870 nm) and MODIS (MOD09 collection 6, at band 860 nm) at nadir view (VZA ≃ 0°), in which the CAR relative altitude varies between 645 m and 3400 m. Note that both flight heights, 645 m and 3400 m, were not used in this study, but we show them here because the footprints at a flight height (≃180 m) is too small (3 m × 3 m at nadir view) and can not be displayed clearly above multiple satellite’s footprints. However, the principle is still the same because the shape of CAR’s footprint depends only on view angles (over flat surface), while footprint’s area varies with variation in flight altitude.

The comparisons shown in Figure 3 provide an idea of how satellite’s surface reflectance changes with respect to CAR surface reflectance measurements. However, to determine the actual difference between the ground truth data (CAR data in our case) and satellite observations at similar wavelength bands, we fitted CAR’s measurements with a polynomial function that maps CAR wavelengths to refelctance using the following equation, with the aim of accounting for the SRF (spectral response functions) differences between the instruments:

$$pol{y}_{car}\left(\lambda \right)=a_0+a_1\lambda +a_2{\lambda }^2+a_3{\lambda }^3+a_4{\lambda }^4+a_5{\lambda }^5.$$

(4)

Note that $pol{y}_{car}\left(\lambda \right)$ is calculated for each satellite sensor over its corresponding set of CAR data. Thus, each satellite dataset requires a different function of $pol{y}_{car}\left(\lambda \right)$. It should also be pointed out that only satellite data with the highest quality were considered in this study.

Then, we computed the difference between measured satellite’s reflectance to the CAR fitted reflectance at the corresponding satellite wavelengths:

$$\Delta \left(\lambda \right)=ref{l}_{sat}\left(\lambda \right)-pol{y}_{car}\left(\lambda \right).$$

(5)

The fit equations are related to small areas of our study site and under specific view angles. Thus, the derived functions cannot be assumed to apply to other conditions and they will not be provided in this paper. However, RMSE (Root Mean Square Error) is provided for all plotted polynomial functions. Furthermore, in the next section, we will provide all the numbers so that interested readers can calculate all polynomial functions if they wish.

3. Results

Surface reflectances from the three grids of MOD09 (file: MOD09.A2008137.1835. 006.2015174161557.hdf) were evaluated in this study. Figure 4 shows MOD09-1km, MOD09-500m, and MOD09-250m side by side with CAR-derived surface reflectance. The plot of $pol{y}_{car}(.)$ was computed from CAR data using five-degree polynomial regression whose RMSE (Root Mean Squared Error) is about 0.005 (annotated in that figure). As noted in that figure, the time gap between CAR and MOD09 (Terra overpass) does not exceed few minutes.

Similarly, Figure 5 shows the performance of MAIAC (file: MAIACTBRF.h03v08.20081371835.hdf) for their two spatial resolutions: 1 km and 500 m. Note that, even though MOD09 and MIAIC are based on the same input data (ToA reflectance of Terra/MODIS), MAIAC is more conservative in terms of cloud masking and quality assessment, which has resulted in fewer samples being compared to those of MOD09 (see

Table 2. Summary of statistics for satellite products : MOD09 (Surface reflectance product of MODerate-resolution Imaging Spectroradiometer) , MAIAC (MultiAngle Implementation of Atmospheric Correction), MERIS (MEdium Resolution Imaging Spectrometer), Landsat7, and their corresponding CAR (Cloud Absorption Radiometer) data. For each product the table provide the number of considered samples (num of samples), spatial resolution, solar zenith angles (SZA), solar azimuth angles (SAA), view zenith angles (VZA), view azimuth angles (VAA); and the mean and standard deviation (stdev) of surface reflectance at instrument bands.

Product	Num of Samples	Spatial Resolution	SZA	SAA	VZA	VAA	Reflectance: (band [nm], mean, stdev)
MERIS_1 km	2	1 km × 1 km	30.1°	121.9°	29.4°	101.2°	(412, 0.295, 0.025), (442, 0.322, 0.035), (490, 0.356, 0.045), (510, 0.37, 0.015), (560, 0.408, 0.006), (620, 0.433, 0.008), (665, 0.457, 0.019), (681, 0.464, 0.019), (709, 0.474, 0.017), (754, 0.494, 0.035), (779, 0.501, 0.031), (865, 0.507, 0.03), (885, 0.504, 0.034)
CAR	230	4 m × 4 m	22.3 ± 1.8°	149.0 ± 13.9°	29.8 ± 1.4°	101.3 ± 1.2°	(340, 0.116, 0.017), (380, 0.221, 0.015), (470, 0.339, 0.021), (680, 0.482, 0.026), (870, 0.496, 0.026), (1037, 0.498, 0.027), (1219, 0.473, 0.028), (1275, 0.47, 0.028), (1657, 0.469, 0.018), (2202, 0.424, 0.016)
MOD09_1 km	46	1 km × 1 km	23.1°	141.5°	0.4°	24.5°	(470, 0.237, 0.041), (555, 0.395, 0.047), (645, 0.449, 0.044), (860, 0.484, 0.041), (1240, 0.475, 0.035), (1640, 0.505, 0.032), (2130, 0.449, 0.034)
MOD09_500 m	112	500 m × 500 m	23.1°	141.5°	0.3°	28.0°	(470, 0.227, 0.041), (555, 0.384, 0.042), (645, 0.437, 0.04), (860, 0.472, 0.037), (1240, 0.463, 0.034), (1640, 0.494, 0.034), (2130, 0.441, 0.032)
MOD09_250 m	256	250 m × 250 m	23.1°	141.5°	0.4°	−0.4°	(645, 0.447, 0.048), (860, 0.48, 0.046)
CAR	5863	3 m × 3 m	23.0 ± 1.2°	143.3 ± 8.1°	0.8 ± 0.6°	83.3 ± 97.1°	(340, 0.096, 0.02), (380, 0.191, 0.038), (470, 0.307, 0.081), (680, 0.455, 0.045), (870, 0.478, 0.046), (1037, 0.478, 0.048), (1219, 0.463, 0.045), (1275, 0.465, 0.046), (1657, 0.492, 0.062), (2202, 0.395, 0.042), (2303, 0.295, 0.013)
MAIAC_1 km	28	1 km × 1 km	23.1°	141.5°	0.8°	101.5°	(470, 0.293, 0.03), (555, 0.392, 0.026), (645, 0.423, 0.023), (860, 0.462, 0.022), (1240, 0.464, 0.02), (1640, 0.492, 0.022), (2130, 0.443, 0.023)
MAIAC_500 m	88	500 m × 500 m	23.1°	141.5°	0.8°	105.6°	(470, 0.286, 0.032), (555, 0.385, 0.03), (645, 0.417, 0.027), (860, 0.457, 0.026), (1240, 0.458, 0.024), (1640, 0.49, 0.028), (2130, 0.441, 0.028)
CAR	4410	3m × 3m	23.0 ± 1.2°	143.3 ± 8.1°	1.0 ± 0.4°	83.3 ± 97.1°	(340, 0.096, 0.02), (380, 0.191, 0.038), (470, 0.307, 0.082), (680, 0.455, 0.045), (870, 0.478, 0.046), (1037, 0.478, 0.047), (1219, 0.463, 0.045), (1275, 0.465, 0.046), (1657, 0.492, 0.062), (2202, 0.395, 0.042), (2303, 0.302, 0.008)
Landsat7	1472	30 m × 30 m	27.3°	129.4°	0.0°	-	(485, 0.304, 0.037), (560, 0.383, 0.037), (660, 0.427, 0.037), (835, 0.465, 0.046), (1650, 0.479, 0.039), (2198, 0.407, 0.043)
CAR	7355	3 m × 3 m	23.0 ± 1.2°	143.3 ± 8.1°	1.0 ± 0.7°	83.3 ± 97.1°	(340, 0.096, 0.02), (380, 0.191, 0.038), (470, 0.307, 0.081), (680, 0.455, 0.045), (870, 0.478, 0.046), (1037, 0.478, 0.047), (1219, 0.463, 0.045), (1275, 0.465, 0.046), (1657, 0.492, 0.062), (2202, 0.395, 0.042), (2303, 0.297, 0.011)
MISR.Aa_1.1 km	5	1.1 km × 1.1 km	23.2°	141.5°	26.2°	15.8°	(443, 0.224, 0.008), (555, 0.342, 0.01), (670, 0.384, 0.008), (865, 0.416, 0.012)
CAR	58	4 m × 4 m	22.8 ± 1.4°	144.5 ± 8.9°	26.2 ± 0.2°	16.3 ± 1.2°	(340, 0.083, 0.025), (380, 0.154, 0.033), (470, 0.254, 0.03), (680, 0.406, 0.021), (870, 0.434, 0.024), (1037, 0.441, 0.024), (1219, 0.431, 0.021), (1275, 0.426, 0.019), (1657, 0.485, 0.015), (2202, 0.356, 0.02)
MISR.Af_1.1 km	3	1.1 km × 1.1 km	23.2°	141.2°	26.0°	197.6°	(443, 0.307, 0.008), (555, 0.421, 0.008), (670, 0.46, 0.011), (865, 0.482, 0.013)
CAR	66	4 m × 3 m	23.3 ± 0.6°	140.9 ± 2.4°	26.0 ± 0.4°	196.3 ± 1.3°	(340, 0.094, 0.015), (380, 0.191, 0.029), (470, 0.293, 0.037), (680, 0.436, 0.051), (870, 0.47, 0.043), (1037, 0.476, 0.043), (1219, 0.458, 0.042), (1275, 0.46, 0.043), (1657, 0.446, 0.042), (2202, 0.413, 0.034)
MISR.An_1.1 km	40	1.1 km × 1.1 km	23.1°	141.4°	0.8°	202.4°	(443, 0.264, 0.029), (555, 0.382, 0.026), (670, 0.424, 0.028), (865, 0.451, 0.028)
CAR	4233	3 m × 3 m	23.0 ± 1.2°	143.4 ± 8.3°	1.0 ± 0.4°	90.3 ± 92.9°	(340, 0.095, 0.02), (380, 0.191, 0.038), (470, 0.306, 0.083), (680, 0.454, 0.045), (870, 0.478, 0.046), (1037, 0.477, 0.047), (1219, 0.463, 0.045), (1275, 0.464, 0.046), (1657, 0.49, 0.062), (2202, 0.395, 0.042), (2303, 0.302, 0.008)
MISR.Ba_1.1 km	5	1.1 km × 1.1 km	23.2°	141.5°	45.7°	16.3°	(443, 0.213, 0.008), (555, 0.326, 0.011), (670, 0.367, 0.008), (865, 0.4, 0.013)
CAR	44	7 m × 5 m	22.8 ± 1.5°	144.8 ± 9.3°	45.8 ± 0.2°	16.3 ± 0.8°	(340, 0.081, 0.023), (380, 0.146, 0.027), (470, 0.242, 0.031), (680, 0.382, 0.04), (870, 0.409, 0.032), (1037, 0.426, 0.029), (1219, 0.416, 0.027), (1275, 0.407, 0.027), (1657, 0.453, 0.047), (2202, 0.352, 0.017)
MISR.Bf_1.1 km	3	1.1 km × 1.1 km	23.2°	141.2°	45.6°	196.5°	(443, 0.305, 0.009), (555, 0.424, 0.005), (670, 0.458, 0.005), (865, 0.49, 0.016)
CAR	42	6 m × 4 m	23.3 ± 0.6°	140.8 ± 2.4°	45.5 ± 0.4°	195.7 ± 0.8°	(340, 0.103, 0.013), (380, 0.201, 0.026), (470, 0.309, 0.03), (680, 0.454, 0.036), (870, 0.482, 0.034), (1037, 0.493, 0.034), (1219, 0.476, 0.035), (1275, 0.476, 0.035), (1657, 0.467, 0.034), (2202, 0.43, 0.027)
MISR.Ca_1.1 km	5	1.1 km × 1.1 km	23.2°	141.5°	60.2°	16.7°	(443, 0.203, 0.008), (555, 0.31, 0.012), (670, 0.349, 0.009), (865, 0.383, 0.014)
CAR	32	14 m × 7 m	22.5 ± 1.6°	146.5 ± 10.3°	60.2 ± 0.2°	16.7 ± 0.7°	(340, 0.091, 0.028), (380, 0.156, 0.028), (470, 0.244, 0.027), (680, 0.377, 0.021), (870, 0.404, 0.016), (1037, 0.42, 0.015), (1219, 0.407, 0.014), (1275, 0.399, 0.016), (1657, 0.452, 0.021), (2202, 0.342, 0.015)
MISR.Cf_1.1 km	3	1.1 km × 1.1 km	23.2°	141.2°	60.0°	195.9°	(443, 0.281, 0.004), (555, 0.41, 0.008), (670, 0.45, 0.006), (865, 0.481, 0.02)
CAR	30	12 m × 6 m	23.2 ± 0.6°	141.1 ± 2.4°	60.0 ± 0.4°	195.3 ± 0.5°	(340, 0.11, 0.018), (380, 0.212, 0.032), (470, 0.318, 0.042), (680, 0.456, 0.051), (870, 0.479, 0.046), (1037, 0.491, 0.045), (1219, 0.47, 0.043), (1275, 0.469, 0.043), (1657, 0.444, 0.016), (2202, 0.44, 0.024)
MISR.Da_1.1 km	5	1.1 km × 1.1 km	23.2°	141.5°	70.6°	16.9°	(443, 0.192, 0.01), (555, 0.294, 0.01), (670, 0.332, 0.011), (865, 0.366, 0.014)
CAR	28	32 m × 10 m	22.8 ± 1.6°	145.3 ± 10.5°	70.8 ± 0.2°	17.0 ± 0.6°	(340, 0.1, 0.031), (380, 0.163, 0.026), (470, 0.235, 0.02), (680, 0.361, 0.032), (870, 0.387, 0.027), (1037, 0.403, 0.026), (1219, 0.39, 0.031), (1275, 0.386, 0.034), (1657, 0.419, 0.033), (2202, 0.337, 0.027)
MISR.Df_1.1 km	2	1.1 km × 1.1 km	23.2°	141.2°	70.2°	195.4°	(443, 0.26, 0.001), (555, 0.408, 0.0), (670, 0.455, 0.002), (865, 0.488, 0.003)
CAR	16	26 m × 9 m	23.2 ± 0.6°	141.3 ± 2.5°	70.2 ± 0.2°	195.0 ± 0.4°	(340, 0.111, 0.022), (380, 0.21, 0.039), (470, 0.308, 0.043), (680, 0.452, 0.038), (870, 0.477, 0.034), (1037, 0.49, 0.034), (1219, 0.472, 0.029), (1275, 0.473, 0.027), (1657, 0.491, 0.014), (2202, 0.423, 0.032)

).

Figure 6 and Figure 7 show the performance of surface reflectance products of MERIS (file: MER_RR__2PRBCM20080516_175119_000000562068_00356_32474_0001.N1) and Landsat-7 (file: LE70400332008137-SC20160917102527.tar.gz), respectively. The time gap (to CAR) is about 40 min for MERIS and 20 min for Landsat-7. We note here that the number of considered CAR samples for MERIS is much lower than those of MOD09, MAIAC and Landast-7 because the view zenith angle of the three latter products are close to nadir view, which allowed us to remove the restriction from view azimuth angles of CAR data [33]. However, for MERIS, where the view zenith angle was 29.4°and view azimuth angle was 101.2°, all CAR samples must have an azimuth angle in [99.2°, 103.2°] (see the range of considered angles in figures’ annotations).

MISR (file: MISR_AM1_AS_LAND_P040_O044742_F07_0022.hdf) behaves like a set of nine spaceborne instruments, each with a different view angle but with nearly identical overpass time and spectral bands. Figure 8 gives an overview of the performance of these nine cameras across their four spectral bands side by side with their corresponding CAR data (for each camera). As MISR is also onboard Terra (same platform as MODIS), the time gap with CAR does not exceed a few minutes for all nine MISR cameras.

Table 2 summarizes the reflectance comparisons for the previous Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8). For both satellite products and CAR data, it provides: the number of samples, the spatial resolutions, VZA, VAA, SZA, SAA, wavelength, mean, and standard deviation of measured surface reflectance.

As we can see from Table 2, the spatial resolution of CAR was around 3 m × 3 m for MOD09 and MAIC, around 4 m × 4 m for MERIS, and varies (due to variation in view angles) from 3 m × 3 m to 30 m × 10 m for MISR.

To assess how the satellite surface reflectance data performs against CAR measurements for each satellite band, we computed the difference between the satellite data and the specific fit function (computed over CAR data for each satellite dataset) using the Equation 5. Figure 9 shows that difference varies with the spectral bands for MOD09_1 km, MAIAC_1 km, MERIS_1 km and Landsat-7. Similarly, Figure 10 shows the difference values over the nine cameras of MISR.

4. Discussion

The results presented in the previous section clearly show that the satellite-derived surface reflectance of MODIS, MISR, MERIS, and Landsat-7 are generally in agreement with the CAR data. The biases against CAR is in the range of $\pm 0.05$ for all measurements except for MOD09 at 469 nm (MODIS’s band 3); the other MODIS product (MIAIC) showed much smaller biases in this band. Furthermore, if we exclude MOD09 at 469 nm, and the two MISR cameras Af and Bf at two bands of 443 nm and 555 nm, the difference to the CAR data is in the range of $\pm 0.025$.

The results also revealed that there is a negative biases between the CAR data and Landsat-7 reflectances for all Landsat-7 bands except the last one (centered at 2198 nm). The negative bias is also revealed for two MISR cameras (An and Da), especially at 670 nm and 865 nm. In contrast, the values from the two MISR cameras (Af and Bf) seem to be have a positive bias compared to the CAR data, mainly at 443 nm and 555 nm.

The results also confirmed that the heterogeneity of Railroad Valley Playa does not appear to have a big impact on the comparison between CAR and satellite at different resolutions. This is mainly confirmed when MOD09 and MAIAC were at different spatial resolutions (250 m, 500 m, 1 km) were evaluated against the CAR measurements (at 3 m × 3 m and 4 m × 4 m), and the higher resolution grids does not seem to be more accurate.

Furthermore, the existence of collocated measurements at Railroad Valley Playa allowed us to collect several scattered CAR measurements per satellite pixel, especially for the medium resolution satellites (MERIS, MODIS, MISR), which allowed us to get a high degree of representativeness for each satellite pixel. For example, in the case of MERIS, 230 CAR samples (of 4 m × 4 m) have been used to evaluate two samples of MERIS (see Table 2). Another element that strongly contributed to the reduction of the impact of surface heterogeneity is the fact that we have limited our statistics to samples that have intersected footprints (between CAR and satellite).

Railroad Valley Playa cannot be considered isotropic because the overall CAR’s measurements vary significantly with variation in view angles (Figure 8), which is mainly due to the brightness and smoothness of the surface. However, this non-isotropic behaviour does not seem to affect our results in this study, especially for the nine MISR cameras, which do not show any significant behaviour pattern for any given geometrical angle. We can say that comparisons between measurements of CAR with those of satellites with the condition that they have similar measurements of geometrical angles, which strongly contributed to the reduction of the effect of surface anisotropicity.

5. Conclusions

This research paper has shown that the multi-angular and multi-spectral data of CAR can be employed as reliable ground-truth in situ data for assessing the accuracy of satellite-derived products of surface reflectance. The coverage of a very wide range of view and azimuth angles by CAR makes this comparison exercise with satellite surface reflectance more robust because only matching sample pairs (from CAR and satellite) having overlapping footprints and similar view angles were utilized. Moreover, CAR data from low relative altitude (height above the surface) allowed us to assume negligible atmospheric correction effects and avoided the associated issues that this correction would introduce. In addition, the good timing of the selected campaign that coincided with the overpasses of some key EO satellites (ENVISAT, Landsat-7, Terra) allowed us to make a comparison under the similar illumination conditions, thereby allowing the differences due to the solar angles and the long-term temporal instability to be ignored. Moreover, the place of the selected campaign (which is a calibration and validation site of COES) contributed significantly to the reduction of differences that can be caused by high levels of aerosol, especially due to surface heterogeneity.

This study has clearly demonstrated that the surface reflectance products that are derived from Envisat/MERIS, Terra/MODIS, Terra/MISR and Landsat-7 are generally in good agreement with the CAR measurements. Nevertheless, some declines in performance have been revealed by this study, such as in the case of MOD09, which appears to be less accurate than MAIAC at 469 nm. Furthermore, some underestimations and overestimations were revealed here for Landsat-7 and some MISR cameras.

Finally, we can say that the CAR data appear to be very suitable for the assessment of satellite-derived level-2 products of surface reflectance, especially for low and moderate resolution satellite data that ground-based instruments cannot assess properly. The results obtained during this study highlighted the need to conduct regular campaigns over calibration sites (either instrumented, such as Railroad Valley, or PICS (Pseudo Invariant Calibration Sites) to evaluate the interseasonal and long-term variability, either for the level-2 product of surface reflectance or for the derived level-3 and level-4 products, such as BRDF and albedo. The need for more narrow spectral bands is also identified to fill some gaps (i.e., around 550 nm); this will allow for the optimization of the fit for the polynomial functions that were used as reference for the accuracy assessment.