Support

Introduction

This tutorial is intended to help understand how to interpret and use various quality information reported by the Swift GNSS receivers and software. It explains the concepts of variance and covariance, how these are communicated through Swift Binary Protocol (SBP) messages, and has some hints and caveats to help with integration. Refer to the Swift Binary Protocol documentation for detailed description of each message.

The following quality information is described in chapters below: variance, covariance, estimated horizontal/vertical errors and DOPs.

Variance

Variance is the degree to which observations deviate from the mean value. It measures the "spread" of data in a data set. 

For instance, if you had a list of 4 measurements of a meter stick, [1.0, 1.1, 0.9, 1.0], the mean of the values would be 1.0, and the variance would be the sum of the squares of the deviation from this mean divided by the number of samples ((0.0^2 + 0.1^2 + 0.1^2 + 0.0^2 ) / 4) or 0.005 meters^2. The variance tells you that the data is not spread very far from the mean. If you take the square root of the variance, you’ll get the standard deviation, a quantity in the familiar units of meters that tells you, in simple terms, that you can expect a typical sample taken from the set at random to have about 0.07 meters of error.

Covariance

Covariance is a measure of the strength of the correlation between two or more sets of variables.

Let’s imagine that instead of trying to measure a meter stick, we are measuring a floor tile that is exactly one meter by one meter. In this case we could create a matrix to describe how well measurements agree with themselves (variance) and how the errors of the width are correlated with errors of the length (co-variance). The matrix would look like this:

| Length Variance               Length to Width Covariance |

| Width to Length Covariance    Width Variance             |

Let's imagine the following 4 measurements of the length and width of the tile. Just looking at the measurements in the table below, you can understand that when the measurement was too short on the length, the person measuring was also likely to be too long measuring the width, and vice versa. This would mean that the measurements are inversely correlated, so we can expect the covariance to have a negative value. Similarly, there is a lot of deviation from the mean, so we expect the diagonals of the matrix to be high.

Measurement of dimensions of a 1 meter by 1 meter square floor tile

After doing the math, this data set has the following covariance matrix. 

[  0.0125   -0.0106 ]

[ -0.0106    0.0092 ]

You’ll notice a few things about the matrix.

1. It is symmetrical. We take advantage of this fact to only broadcast the upper triangular part of the matrix in the SBP messages
2. There is more likely to be error in the length (bigger variance), than the width as you noticed in the data table. You see this from comparing the term in the top left of the matrix to the bottom right of the matrix, showing the bigger variance of the length.
3. Length error tends to occur with width error (in the off-diagonal terms, the covariance is not 0).
4. The error in length in the positive direction is likely to correspond to error in width in the negative direction, and vice versa, which makes the off diagonal covariance a negative value.

Variance and Covariance Reported by Swift Navigation Output

With each navigation solution, the system has its internal estimate of the "variance" of individual variables in its state. The variance tells you how well we think we know the position of a vehicle in the "northing" direction (i.e. the latitude) and they show up on the diagonals of the matrix. The Covariance (off-diagonal) tells you how well the Northing Position agrees with the Easting Position estimate (i.e. longitude). From the covariance matrix and some calculations, you can create an ellipse (2D) or a spheroid (3D) in which the position estimate should fall. Internally, the covariance output is derived from the quality of the underlying measurements (pseudorange, acceleration, etc), and the degree to which the new measurements agree with the latest estimates.

Covariance in SBP Messages

Swift products (like Piksi Multi, Starling) report covariances as part of its standard SBP output. The following messages include the upper triangular part of the 3 by 3 covariance matrix according to the 3D direction of the corresponding vector in the message. These messages are recommended as the best estimate of quality and should be used by applications in real-time control, state estimation, machine learning, and mapping.

Estimated Accuracy

For some of the more common messages, rather than the full covariance matrix, SBP reports scalar “estimated accuracies” with the same units as the measurements reported in the message. These estimated accuracies are provided for test and evaluation where something easy to understand in the physical world is expected. They are derived in a clear mathematical way (the square root of the norm) of the covariance matrices reported in the “_COV” messages.

Estimated Horizontal Accuracy (h_accuracy in LLH messages) represents the standard deviation of the error in the worst-defined horizontal direction. Expressed another way, this value multiplied by 1.515, represents the 2 dimensional error magnitude under which 68% of the points should fall. 

Estimated Vertical Accuracy (v_accuracy in LLH messages) this value represents the standard deviation of the error in the vertical direction, which is to say that 68% of vertical errors will have a magnitude less than this.

Accuracy (accuracy in ECEF messages): this value represents the standard deviation of the error in the worst-defined direction (typically this will be close to the vertical direction). Expressed another way, this value multiplied by 1.878 represents a sphere about the true position in which 68% of points should fall.

The following set of messages report these “accuracy” values which condense the covariance matrix down into human-friendly values.

Note: The accuracy values in the SBP position and velocity messages have a maximum value of 65.355 meters for positions and 65.355 meters per second for velocity. This non-linearity and maximum value should be considered in any application. Starling may have an internal estimate with larger estimated accuracy than this max, especially in the case of very long periods of dead-reckoning or very poor sky view.

Covariance and Estimated Accuracy Usage

It is intended that the covariance and estimated accuracies are used in three ways. First, for some applications, very coarse estimates of position and velocity are not useful in the application. In this case, it is expected that in the integration the estimates are discarded or treated as invalid if the covariance or estimated accuracy exceeds a certain threshold. Swift products default to allowing out most position and velocity estimates.

Secondly, it may be necessary to trigger different operating conditions if the covariance estimates are outside of a certain threshold. For instance, a high covariance could illuminate a warning light on a operator’s display to indicate degraded position, or reduce the operating envelope or maximum speed of a ground robot until a more confident position, velocity, or attitude estimate is achieved.

Third, when using the data as part of a downstream estimation or control system, it is expected to use the actual values of the covariance matrix as an input to the downstream filter or control loop. This also applies if the outputs are used to position tag other data such as lidar returns or photos or video. It can be very useful to include confidence information when assembling this data into an overall map, survey, or aggregating form different vehicles and this information should not be discarded. Accuracy of 10 centimeters or below indicates a very high quality position, and this kind of position should not be treated the same as a position with only 10 meters of expected accuracy in an application.

DOP

DOP stands for Dilution Of Precision, an estimated error indicator as a mathematical effect of navigation satellites geometry on positional measurement precision.

The DOP was frequently used in the past to determine the estimated accuracy of GNSS position but it no longer applies to a RTK GNSS system with the inertial fusion.

Many historical GNSS systems were built for a sea-born navigation at a time when there were very few GNSS Satellites. In this case, one of the biggest things that determined overall accuracy was the geometry of the observations used. Some systems use the DOPS as a gauge of overall position accuracy and in fact sometimes vendors completely replace this field with something that has nothing to do with the geometry of the observations used in the navigation solution in an effort to be backwards compatible but still convey accuracy information. 

Swift discourages using the DOPs as an indication of quality and does not have any heuristics or workarounds in their output to facilitate such usage. Keep in mind that in an urban canyon or under a tree, you may have excellent satellite geometry, but terrible signal to noise ratio and poor GNSS position quality. The satellite geometry in the DOPs, while related, is not a gauge of the accuracy. Additionally, there is not really any satellite geometry anymore during dead reckoning periods so it is challenging to convey meaningful DOP in that case and Swift omits DOP output during dead-reckoning.