Georeferencing and Orthophoto Mosaicking of UAV Images
Current Challenges and Progress
Ben Wilkinson and Bon Dewitt
Geomatics Program, School of Forest Resources and Conservation
Introduction
The absence of a preflight camera calibration created the need for determination of the focal length and lens distortion parameters required for a bundle-adjustment solution. Although a self-calibrating bundle-adjustment may be used to solve for these missing parameters, the position of the camera at exposure is needed for this method. Unfortunately, the data from the navigation unit has low precision compared with typical mapping systems, has time gaps of up to 14 seconds in the data collected, and has a host of other issues. The flying height of the camera at exposure could be used to solve for the focal length, but since terrain around the NBR project area has dynamic relief (figure 5), the flying height above ground (HAG) cannot be readily determined from either the barometric altimeter or the GPS height above ellipsoid without a high resolution digital surface model. Finally, because the camera used to capture the NBR data is no longer functioning, a different camera of the same model (Canon Elura 20MC) was calibrated with results leading to the best estimate we have for the focal length of the mission camera. These problems may be circumvented by the high redundancy of data and the associated error propagation along the flight lines.
Navigation Data
GPS: Furuno GH-80(81) Series GPS Receiver Accuracy
Horizontal: 15m
Vertical: 22m
INS: Kestrel Autopilot v2.2 Attitude Estimation Error: Roll and Pitch
Level Flight: 5°
During Turns 10°
Altimeter: Kestrel Autopilot v2.2 Altitude Resolution:
Resolution: 0.116m, 0.249m (v2.22)
Figure 1. Navigation Unit Manufacturer’s Specs
In addition to the poor accuracy of the GPS altitude, the precision of the barometric altitude, although it has a high “resolution” is also poor, as illustrated in the following graphs of recorded altitude vs. time, figures 2-4.
The autopilot data file logs at varying intervals of about one second each necessitating interpolation of the data, although gaps greater than 4 seconds can be found in the data file including a 14 second gap in flight 2 of the NBR set, which is an equivalent loss of information for about 400 images at 30fps, 260 meters at an air velocity of 20m/s which equates to a distance of 1300 pixels or 2.7 frames in the flight direction assuming a focal length of 1000 pixels and a flying height of 200m. The graph below (figure 2) highlights the area of the 13 second gap.
Figure 2 Plot of GPS Altitude (green) and Barometric Altitude (blue) and their differences
Click figure to enlarge
Also noticeable in figure 2: there seems to be a shift in the recordation time of the GPS altitude and barometric altitude in the data file. Applying a shift in the GPS data of about 13 seconds results in the following graph (figure 3). This suggests there is a lag in the recordation of all of the GPS data including latitude and longitude in addition to height above the ellipsoid. The cause or an explanation of this shift/lag has not been found.
Figure 3 Plot of GPS Altitude (green) and Barometric Altitude (blue) with a 13 second shift applied
Click figure to enlarge
At first, a shift accounting for all data gaps was applied to the data which resulted in the graph below (figure 4), however, a comparison of these plots indicates that there is about a 13 second difference in the data written for GPS and barometric altitudes for all values in the entire flight, and not several shifts resulting from the data gaps. Even with the applied shift, the altitude measurements must be used in conjunction with an elevation model to find the height above ground value requisite for solving for f.
Figure 4 Plot of GPS Altitude (green) and Barometric Altitude (blue) with shifts accounting for all data gaps greater than 2 seconds applied
Click figure to enlarge
Camera Calibration
Since the Canon Elura 20MC used to collect the Bison Data broke after the mission, we had to use a different Elura 20MC for calibration flights over the Archer airfield in order to determine the focal length and lens distortion parameters. We found a significant amount of lens distortion in the tested camera, which suggests that the camera used for the NBR flights probably did too (this jibes with preliminary results from the actual NBR data). In addition to this test, another test was performed by recording objects of known size at different baselines. The focal length for the tested camera was found to be around 1000 pixels by both methods. Unfortunately, with off- the-shelf cameras, there is no way to know how different the focal length and distortion parameters compare from one unit to the next; we can only assume they are close. Also, we don’t know what zoom settings were used during the NBR flights. If the zoom settings were different in the NBR flights and the calibrations, the focal length could be very different from our assumption.
In addition to these calibration methods, the NBR data was compared to DOQQ data using a nominal HAG of 220m. This yielded a shorter focal length than in the calibration methods, about 900 pixels. This could be due to simply a difference in the cameras, the zoom settings, or an incorrect HAG due to the dynamic terrain of the NBR (figures 6 and 7) coupled with the low precision of the height data.
In short, if we knew one of either the HAG or focal length we could solve for the other. Currently, we are using an estimated focal length of 950 pixels for our model instead of flying height, assuming that it is the better of the two guesses. The true value remains, however, quite ambiguous.
Figure 5, Relationship of Focal Length and Height
Click figure to enlarge
Figure 6, The Four Flights at the NBR Project Area
Click figure to enlarge
Figure 7, Flight 2 of the NBR Project, Illustrating Relief (vertical scale x2)
Click figure to enlarge
Current Work
We are presently working on the determination of interior orientation parameters and relative orientation for “chunks” of photos in a strip using an arbitrary coordinate system, step 5 in the proposal-- reprinted here:
Using an arbitrary coordinate system defined by treating the initial estimation for the focal length as the flying height above ground (and therefore scaling found ground control coordinates to image scale, i.e. 1 pixel units), and estimating the photo bases by using the found coefficients from a projective coordinate transformation solution utilizing image tie-points, a relative orientation solution will be found for photos in a chunk. In this project, a chunk of photos will be defined as a group of contiguous photos where the first and last photos have a nominal number of tie-points in common. Since we can use a self-calibrating method for the relative orientation solution, we can solve for the interior orientation parameters, focal length and lens distortion. However this method will depend greatly on the estimations of the photo base distances in order to achieve the best solutions. The result will be the relative position and orientation of all photos in terms of the first photo in the chunk and 3D coordinates for all tie points at photo scale.
Initially a conformal coordinate transformation was used to estimate the photo bases (distances in the photo coordinate system between images, or exposure stations), however the solution indicated that the focal length was incorrect—too small for some chunks, too large in others. Therefore, a projective transformation was used with the assumption that it would better model the relationship between photos. However, the extra coefficients proved to be insignificant and the translation values derived to be used for photo base values were unreasonable. This is probably due to a combination of the projective method being too complex for the transformation and the relief of the project area (these transformations are assuming a flat terrain). Therefore, an affine method is currently being implemented and will most likely yield satisfactory results.
Figure 8 illustrates the importance of knowing the photo base when performing relative orientation. The diagram on the far left shows a stereo pair with height set to the same value as focal length and a photo base that fits with the values. The next two images are the results of lengthening the photo base and fixing the focal length (middle), allowing the height to change, and fixing the height (right), and allowing the focal length to change.
Figure 8. The Effect of Photo Base on the Height and Focal Length of a Stereo Pair of Images
Click figure to enlarge
The Next Steps
After the chunks are formed by relative orientation implementing affine-found photo bases, and attached using a 3d conformal transformation, cross lines will be used to strengthen the geometry of the bundle, increasing the precision of coordinates in the y-direction (orthogonal to the direction of flight) and breaking the correlation between attitude and photo coordinate measurements. Finally strips and cross lines will be attached to each other resulting in the parameters we need to find the location and attitude of the images allowing us to finally create some mosaicks.