Validation of the replay performances – Representativeness and reproducibility of replay techniques

SUMMARY

Autonomous navigation is increasingly exploiting satellite geolocation technologies. Economic efficiency and technical requirements are challenges for the entire industry.

The validation of the geolocation function in a rich and complex terrestrial environment is inevitably accompanied by an increase in testing costs to cover a sufficient number of operational situations.

For this reason new metrology techniques are emerging. They exploit the digitization of GNSS signals to provide reliable and repeatable methods.

 

INTRODUCTION

The development of applications based on advanced geolocation functions requires significant testing for their development, their validation and their maintenance in the future.

However, performance under operational conditions is difficult to characterize. Field tests are complex, time-consuming and expensive with results fluctuating from one campaign to another. Simulations using constellation generators offer a popular alternative in the field of aviation. However, the representation of signal propagation in terrestrial environments is much more difficult, especially the simulation of phenomena such as obstacles, attenuation, diffraction and reflections undergone by satellite signals.

For several years now, the replay of real GNSS signals previously digitized in the field has opened a third avenue, described in this article. Commissioned by the CNES, the GUIDE test laboratory has conducted a scientific study aimed at demonstrating the ability of this technique to reliably reproduce real data for complex physical environments. The immediate aim is to determine measurement repeatability and reproducibility using these test methods – ref. ISO5725.

 

Four types of test areas - Peri-urban, Motorway, Urban and Urban Canyon

TEST FACILITIES

Context

GNSS signals used for road, rail, agricultural or waterway applications are altered by numerous phenomena which must be understood. These satellite signals can be masked by bridges, reflected by buildings, undergo diffraction and attenuation when passing through tree canopies. Such complex environments simultaneously combine a multitude of constraints. In addition, before even considering jamming, i.e. attacks in the form of intentional interference, urban areas already have abundant sources of industrial or domestic interference.

Additional technologies, such as real-time reception of correction messages (EGNOS , NRTK , PPP , …) and/or mergers with additional sensor measurements (inertia, odometry, …), are designed to improve the performance of GNSS terminals. Nevertheless, the design and development of these algorithms is difficult. The diversity of situations and dynamics studied sometimes reveals higher error rates if the hybridizations are poorly calibrated.

Checking the performance of a GNSS receiver localisation function requires controlled test engineering based on accurate and reliable evaluation methods.

Environment

As part of this study, the selected test areas cover a wide variety of situations:
 Succession of bridges
 Neighbourhoods with apartment blocks
 Long narrow streets oriented North-South
 Tree-lined boulevards
The test vehicle was compelled to obey speed limits with an average speed of 35km/h in the city and 80km/h on the highway (Toulouse ring road).

 

Photo of Instrumentation setup

Vehicles & Instruments

The test vehicle has precision instrumentation on board to determine ground truth (a reference trajectory) and to digitize GNSS signals with the highest accuracy. The receivers on board the vehicle all share the same antenna with the digitizer, with the exception of the one dedicated to the reference trajectory.

The reference instrumentation consists of a professional GNSS receiver (ASTERXU), a high-precision inertial navigation unit and an odometer. The reference trajectory is consolidated during post-processing with the measurements of a fixed GNSS receiver (PPK).
The digitizer is a NI PXI chassis equipped with a VST-5646R, synchronized with an OCXO clock unit and covering the upper L-band (1559MHz-1610MHz), also called L1/E1/G1, with 16 bit quantization . The input gain of the recordings is automatically controlled by the AGC function of the VST module. Configuration and backup are provided by the STELLA software, developed and marketed by M3 SYSTEMS.

The six test receivers are uBlox M8Ts using the manufacturer’s default settings. These receivers were divided into two groups each sharing the same antenna by means of a 4-way splitter, the 4th channel being reserved for the digitizer. The second antenna is dedicated the ground truth. The position of the one relative to the other is perfectly known.

 

System block diagram for GNSS signal digitization and position measurement recording

Implementation

One-hour trials were conducted during the spring and summer of 2018. In both cases, the tree foliage was already present, thus having comparable signal disturbance conditions.
In addition to the driver, a GNSS test engineer was on board the vehicle to monitor the ambient conditions and the proper operation of the instrumentation.

The run was filmed from beginning to end so that during the analysis, the external elements likely to impact the performance of the receivers could be taken into account.

 

Block diagram of installation for replaying the digitized signals to the batch of receivers used for field recordings

Test bench

In the GUIDE laboratory, the GNSS signals digitized during the trial were replayed by reusing the same instrumentation, i.e. the NI VST and the STELLA software.
The same receivers used during the trial were installed on the test bench in the same positions and with the same setup. The VST is thus connected in place of the antennas.

At the end of the replay sessions, GUIDE had two sets of the same position measurements, the first set being recorded on site and the second on the test bench.
The analysis then consisted in comparing the results of these two sets in order to determine to what extent the replayed signals were representative of the originals (the real world signals). The operation was repeated using other replay sessions in order to determine how reproducible the results were from one session to another.

Checking the performance of a GNSS receiver localisation function requires controlled test engineering based on accurate and reliable evaluation methods.

Replays with NRTK

The digitized GNSS signals have also been reused with NRTK messages to increase accuracy. In collaboration with EXAGONE, operator of the TERIA network, an AsteRx-U receiver was connected to their server delivering contextual correction messages. These messages, calculated in conjunction with the digitization campaigns, were saved in their database to be rebroadcast upon request. Thus, during the replay sessions, in response to its requests, the receiver received the correction messages from the local virtual positions as they were actually issued on the dates of the trials.

RESULTS

The differences in the time domain between the recorded measurements and the signals restored on the test bench are negligible. The curves modelling the dynamics of the signals (C/N0) are perfectly superposable. The additional replay sessions confirm the very good representativeness of the GNSS signals and the excellent instrumental reproducibility of the bench tests

 

Representativeness

This diagram evaluates the representativeness of the data generated by the replay technique by comparing it to the real-world data collected by means of an onboard vehicle collection campaign.
At first, a batch of 6 uBlox M8T receivers is used to calculate the position of the vehicle with the original GNSS signals. Next, the same batch of receivers is subjected to the same GNSS signals digitized during the campaign and synthesized on a test bench.
The two curves, showing the respective statistical distribution of the two test techniques, show very close agreement.

 

Reproductivity

This diagram evaluates the reproducibility of the replay technique by comparing 3 successive replay sessions on the same batch of 6 uBlox M8T receivers subject to the same signals previously digitized on-site. As for the previous diagram, the errors of the 6 receivers are consolidated and sampled.

The superposition of the 3 curves shows a perfect reproducibility for 60% of the results. Beyond this, the differences are due to the dispersions of receiver measurements within the defined zones.

 

NB. : The cumulative distribution function (CDF ) displays the cumulative probability of positional deviations between a receiver’s GNSS measurements and the ground truth. The resulting curve thus shows the percentage of occurrences over a given error interval.

Collecting campaigns to digitize signals require sophisticated technical means and a rigorous methodology. The replay sessions are, for their part, much more accessible and easier to implement. The degree of representation depends on the digitizing technology, the receiver chain (antenna, LNA , cables, splitter ), the instrument options (AGC , OCXO) and the selected settings (bandwidth, quantification). The quality of this technique is therefore determined by the initial collection operations.

 

DISCUSSION

Representativeness and reproducibility of GNSS signals opens up new perspectives for developing, evaluating and validating geolocation equipment. In particular, the operational results can be compared from one GNSS terminal to another, and for the same equipment, with different sets of parameters.

We have also seen that reconstructed GNSS signals are comparable from one SDR replayer to another. In fact, the position measurements from the same receiver obtained from two different sets of replay equipment are similar. The technique is independent, inexpensive, simple and quick to implement, as the digitized signals and associated metadata are freely available. In this context, standardization of file formats is already in progress.

It is primarily intended for companies operating within the ITS value chain. Equipment choices for industry applications will be based on more precise criteria and be more closely related to the final destination of the products. R & D teams will adjust the performance of their prototypes by concentrating on improved settings.

 

CONCLUSIONS

This study has revealed that replay techniques have many advantages. The EN16803 standard developed within CEN/CENELEC takes them into account to validate the performance of GNSS terminals dedicated to road transport. This technique has a promising future with multiple uses and evolutions.