Construction of a Point Cloud Data Set with Pix4D Software: Part 2 - With GCPs

Introduction

          In the previous activity, data collected by a Phantom 4 Pro at the Litchfield Mine in Eau Claire, WI was processed in Pix4D without GCPs. The generated output had grievous elevation and datum errors. This time, GCPs were added to the data to correct these errors where proper analysis of the data is possible. GCPs are highly recommended by Pix4D (and probably most surveyors) for processing digital images.

          GCP stands for Ground Control Point, and are points of known coordinates in an area of interest. These coordinates are collected using traditional surveying methods, in this case, using a Topcon Sokkia HiPER HR, or by LiDAR, older maps, or Web Map Service. GCPs increase the data's accuracy immensely, but they have to be placed homogeneously within the area of interest. A minimum of three GCPs are required, but five are recommended (Figure 1). Five to ten are usually sufficient and increasing the amount of GCPs will not increase the data's accuracy. If the area of interest is highly variable in elevation, then more GCPs are appropriate. Before collecting GCPs, the GCP coordinate system must be defined. In this case, the GCP coordinate system is UTM Zone 15N. GCP accuracy should also be considered - what dimensions should they be measured in and are they visible in the images. Lastly, decided how the GCPs will be collected - total station, GPS system, or or by other means.

Figure 1: Ideal distribution of GCPs.

          When processing data in Pix4D with GCPs, the generated image is able to be georeferenced, thus providing scale, position, and orientation to the image. These elements are essential for deriving any accurate measurements from the data.

Methods

          The methods for starting a project in Pix4D remain mostly identical from the previous activity. As a summary, start a new project in Pix4D and name it so that it indicates that GCPs were used. Add the images to the project. Return to the Camera Model Editor and change the Shutter Model to a Linear Rolling Shutter. Accept the Output Coordinate System defaults and select 3D Maps.

          In Map View, from the Project drop down, open the GCP/MTP Manger (Figure 2). From here, import the GCP points. Make sure the X and Y values are correct and aren't switched around. By slecting OK, the GCPs will appear in Map View within the flight plan. Uncheck Steps 2 and 3 and start the Initial Processing.

Figure 2: GCP/MTP Manager window.

          GCPs would normally be marked through the rayCloud Editor, but altitude problems occur in rayCloud with GCPs  regarding EXIF data from DJI Platforms. By selecting a GCP in the rayCloud, images that don't contain the GCP are brought up. Instead, open the Basic Editor from the GCP/MTP Manger window. Selecting GCP 1 will bring up all the images that contain GCP 1. Zoom in and locate the GCP. In at least two of the images, mark the center of the GCP in the Preview window. Do the same for eight to ten more GCPs. Click OK and from the Process drop down, select Reoptimize. After reoptimizing, Return to rayCloud Editor and select a GCP that hasn't been marked in two images yet. Images with the GCP will be brought up with a circle around the GCP (Figure 3). Place a mark in the center of the GCP in at least two of the images. Place at markers in at least two images foe the rest of the GCPs. Uncheck Step 1, check Steps 2 and 3, and finish processing.

Figure 3: Selection of images zoomed in on the same GCPs where at least two will be marked.

Results

          The data, now in rayCloud, can be viewed as a triangle mesh with the GCPs visible in the landscape (Figure 4). The forested areas aren't rendered in triangle mesh because of many of the canopy images weren't imported into Pix4D and trees are generally difficult data.

Figure 4: The data rendered as triangle mesh in Pix4D.

          Because the GCPs geometrically correct the data, accurate information can be derived from the data, like measuring the volume of the stockpiles. Pix4D allows users to make much measurements by digitizing the area of interest (Figure 5) and will calculate its area and volume (Figure 6).

Figure 5: Digitizing a stockpile to calculate volume in Pix4D.

Figure 6: The calculated volume of the stockpile in Pix4D.

          Pix4D also allows many other interesting operations, like creating an animation of a trajectory of the data (Video 1).

Video 1: Trajectory animation of the data in triangle mesh.

          It was stated earlier that GCPs are points of known coordinates. With these known coordinates, the data was able to be properly placed spatially. With the newly process DSM over laid on top of the DSM without GCPs, it's clear how the data with the GCPs were shifted southeast by several meters (Figure 7). Even the elevation range of the two separate data sets are vastly different, demonstrating how data without GCPs is very inaccurate.

Figure 7: DSM generated with GCPs overlayed with DSM rendered without GCPs

          Comparing the two Hillside DSMs, the edge of the pond lines up to the baseman's pond much better in the Hillside with GCPs (Figure 8). The shape of the stockpiles also seem to differ. In addition, Pix4D rendered the forested areas differently with the GCPs than without. The Hillshade without the GCPs created false elevation lines, trying to compensate for the difficult forested area.

Figure 8: Comparison of Hillshaded DSMs generated with and without GCPs.

          The orthomosaic looks very similar to the one from last week at first glance but the data is geometrically correct (Figure 9). With all the GCPs located directly in the mine, the images near the southern road appears warped in order to geometrically correct the data near the GCPs. Transparency was applied to the orthomosaic over the Hillshade to give the stockpiles some relief (Figure 10). 

Figure 9: Orthomosaic map of Litchfield Mine.

Figure 10: Orthomosaic with Hillshade effect of Litchfield Mine. 

Conclusion

          Processing the same imagery with and without GCPs in Pix4D denote how the software works and its capabilities, but ultimately the importance of GCPs. By comparing the two, it's clear that the data processed last week was inaccurate along the x, y, and z, plane. Any information derived from the data would have been inaccurate and useless. GCPs geometrically place imagery in the real world, making them a critical component to digital image processing and other geospatial analysis methods. 

Construction of a Point Cloud Data Set with Pix4D Software: Part 1 - Without GCPs

Introduction

          Back in November, the Geospatial Field Methods class went to the Litchfield Mine in Eau Claire, Wisconsin to collected GCPs and observe various UAS Platforms launch and take imagery of the mine. Now the data collected from that day will be processed in Pix4D, the most current and easy-to-use software for constructing point clouds. Basic operations are easy to learn in Pix4D, but the software offers much more advanced processing options. The Pix4D Manual gives the user the proper knowledge to prepare projects suitable for the software to render accurate output and how to process the data in the software. Certain questions were asked in order to better understand Pix4D and how it works.

o What is the overlap needed for Pix4D to process imagery?

          High overlap is recommended by Pix4D to process imagery - at least 75% frontal overlap and 60% side overlap (Figure 1). In addition, a grid pattern and uniform camera height above the terrain is recommended to ensure accuracy. The high percentage of overlap will allow the software to find thousands of matched key points, which are characteristic points that are found within two images that will generate a 3D point. Therefore, more matched key points will create a more accurate 3D image.

Figure 1: Ideal Image Acquisition Plan that will result in the recommended overlap percentages.

o What if the user is flying over sand/snow, or uniform fields?

          The general case previously mentioned won't generate accurate results in Pix4D if the terrain in sand, snow, or uniform in general. Such fields have little visual content, making the collection of key points difficult for the software. The frontal and the side overlap should be increased to 85% and 70% respectively. Adjusting the camera exposure settings to increase the contrast of the field will making mapping such difficult fields easier.

o What is Rapid Check?

          Rapid Check is a program in Pix4D that checks the requirements for images quickly, sacrificing accuracy by reducing the resolution of the image. It's used preferably in the field.

o Can Pix4D process multiple flights? What does the pilot need to maintain if so?

          Pix4D can process multiple flights, but the pilot needs to have a proper flight plan if the software is too process them correctly. The different flights need to have enough overlap between them along with the recommended overlap within the individual flight plans (Figure 2). The flight plans also need to be taken in relatively similar conditions. The flight height should be very similar too, as different flight heights will lead to different spatial resolutions.

       

Figure 2: Proper and improper multiple flight plans.

o Can Pix4D process oblique images? What type of data do you need if so?

        Pix4D can process oblique images, in fact, it's often used for building reconstruction (Figure 3). It's recommended to fly around a building at a 45 degree angle and then increase the platform's height while decreasing the angle of the camera for a second flight, taking images every 5-10 degrees.

Figure 3: Ideal flight plane for building reconstruction, utilizing oblique images.

o Are GCPs necessary for Pix4D? When are they highly recommended?

          GCPs are not necessary for Pix4D but are highly recommended as they will increase the accuracy of the project. They strongly recommended if a project has no geolocation for its images. The final results of a project that has no GCPs or geolocation will have no scale, orientation, or absolute position information and measurements, overlay, and comparisons will be impossible. They may produce an inverted 3D model in the rayCloud. The 3D reconstruction may not preserve the shape of the surveyed area. When using GCPs, 5 is the minimum recommended amount for a project (Figure 4).

Figure 4: Recommended distribution of GCP placement.

o What is the quality report?

          A quality report is automatically displayed after initial processing which checks the quality of the data. It's the user's responsibility to verify that the data is accurate and of good quality. 

Methods

            When starting a project in Pix4D, select New Project, which the software offers when it's opened (Figure 5).

Figure 5: The first page displayed in Pix4D where a user can start a new project.

          The user is then prompted to name the project (Figure 6). It's recommended to include information on the date, site, platform, sensor, and altitude in the project name. Save the project to an appropriate folder.

Figure 6: Name the project that represents it's specific characteristics.

          Next, the user needs to select the images that will by processed by Pix4D (Figure 7). The user can add pictures individually or by directory. If the flight images have been cleared of any useless photos, like images taken during takeoff, uploading images by directory is much easier.

Figure 7: Pix4D page where the images are uploaded.

          Once the images are added, the software reviews the exif file to read the image's metadata. It will note if the images are geotagged and the coordinate system, though most UAS platforms default to WGS 1984 in Decimal Degrees. The camera information it provided are often not accurate and need to be edited. Open the Camera Editor window and click edit under the Camera Model Name (Figure 8). Change the Shutter Model from a global shutter to a linear rolling shutter, which is what a Phantom 4 Pro has. 

Figure 8: Camera Model Editor window where the shutter model needs to be changed for Phantom 4 Pro.

        Next, leave the Output Coordinate System as its default (Figure 9). It can be reprojected, if need be, in GIS software. 

Figure 9: Pix4D page where the user can change the output coordinate system.

          In the Processing Options Template, chose 3D Maps, which will generate a DSM and an orthomosaic (Figure 10). The Processing Template come with preset processing parameters and outputs, but 3D Maps is the most commonly used template. 

Figure 10: Processing Potions Template page where the processing parameters and output is selected.

          Afterwards, the flight plan will be shown in map view. Uncheck steps 2 and 3 and then process the data. The Processing Options window will open where a Full, Rapid, or Custom Keypoint Image Scale can be selected. In the DSM, Orthomosaic, and Index tab, chose triangulation for a raster DSM method. Select start to officially start the initial processing. A quality report will display in the screen when Step 1 is finished (Figure 11). It's important to thoroughly review the quality report to assess the quality and accuracy of the data. Some of the problems can be addressed before processing the data completely. According to this the quality report in Figure 11, the median number of key points per image is acceptable, but only 197 out of 222 images were processed (88%). Most, if not, all of the images should be processed. This is probably do to complex data in the images from the trees. This can be prevented by increasing the overlap to above 80% and flying at a higher altitude. The percentage of difference between initial and optimized focal length is between 5% and 20%. It should be less than 5%. The median number of matched key points calibrated per image is acceptable. The report also notes that no GCPs were added to the data. 

Figure 11: First page in the quality report.

          The quality report provides an examination of many aspects of the data, including the initial image positions, which resembles the flight plan of the Phantom 4 Pro (Figure 12).

Figure 12: Figure given by the quality report of the Initial Image Positions. 

          Image overlap is critical to the accuracy of the data. The quality report generates a figure of the number of overlapping images per pixel for the orthomosaic (Figure 13). Five images per pixel is the recommended number. Pixels represented in red and yellow represent pixels with low overall and poor results will be generated for these pixels. The gap in the AOI is evident, indicated earlier when the report noted that only 88% of the images were processed. The area omitted covered trees, difficult terrain for a general flight plan in Pix4D. Poor overlap seems to be concentrated to the edges of the AOI, probably from not having images surrounding those pixels. The omitted area also is surrounded by poor overlap since a good portion of nearby images and their associated overlap where cut from the processing. 

Figure 13: The quality report figure showing the image overlap for the orthomosaic pixels.

          The quality report also provides a preview of the DSM and Orthomosaic that the final processing operations will generate (Figure 13).

Figure 13: A preview of the DSM and Orthomosaic that Pix4D will generate.

           After reviewing the quality report, uncheck the initial processing option and check Steps 2 and 3 to finish processing. All the steps will turn green when processing is complete. The data can be viewed in rayCloud or the camera can be turned off and the triangle mesh turned on to view the model. 

Results

          The orthomosaic of the data allows the viewer to identify certain features in the mine - different rock piles, location of machinery, and roads (Figure 14). The gap in the data in right in the center of a cluster of trees, which had a higher probability to cause issues in the processing. The Hillshade tool was used to give the rock piles more relief, giving an idea of their height relative to other features (Figure 15). However, since GCPs were not used, the elevation of features are highly inaccurate and the entire study area is potentially off by 30 meters, illustrated by the misalignment of the large water body in the DSM and Hillshade (Figure 16 and 17). Figure 16 and 17 also illustrated the processing errors in the forested areas south of the mine. Another issue with the data is that it was processed as if the images were taken using an ellipsoid height but it was really using a geoid height.

Figure 14: Orthomosaic map of Litchfield Mine.

Figure 15: Transparent orthomosaic with Hillshade effect.

Figure 16: DSM map of Litchfield Mine.

Figure 17: Hillshade of Litchfield Mine. 

Conclusion 

          Pix4D proved to be an easy to understand and easy to learn software for generating 3D images that can be used to locate features and measure distances, elevation, and volume. Concerning the mine, knowing the volume of their materials would be one of their main goals when processing this data (though, not THIS data, because it's bad data). The software made it clear that developing a proper flight plan for its purposes and knowing the parameters of the platform and sensor are critical steps for generating accurate data. Next time, GCPs will be brought into the equation and will hopefully fix the problems with this data.

Sandbox Survey 2: Visualizing and Refining your Terrain Survey

Introduction

          As a followup activity to the Sandbox Survey back in September, the class imported the data they collected using a surveying method for their sandbox terrain and then create a series of DEMs (Digital Elevation Models). Recall in the first activity, the class broke up into groups and created a unique terrain in a 115 cm x 115 cm sandbox. Selecting a sampling method - in this case, systematic point sampling - the groups decided on an arbitrary sea level and measured the terrain's elevation. Many groups utilized at grid system to accomplish accurate systematic measurements. This group constructed a grid with 10 cm x 10 cm cells and measured the elevation at the northwestern corner of the cells using the grid as sea level (Figure 1).

Figure 1: The grid stretched over the sandbox terrain.

           Before the collected data could be imported into ArcMap, it had to be recorded and normalized in an Excel sheet. This activity involved x and y values from the grid and z measurements that were the elevation at a point. Therefore, all the x values were in their own column labeled "x," all the y values in their own column labeled "y," and all the z measurements in their own column labeled "z." If read horizontally, the x and y values would indicate the point from the grid that had an elevation of the z value in the row (Figure 2). This is normalized data. Data normalization is (Wenzel, 2017)

Figure 2: A sample of the normalized data in an Excel sheet.

          Since the sampling scheme was a systematic point method, all the data points are distributed in an equal distance from each other when laid out spatially by it's coordinates. Interpolation procedures will essentially fill in information between the data points by various methods that will create a complete picture of the survey area. It's accuracy depends on the interpolation method, sampling method, and type of survey. Using a selection of interpolation methods, the sampling frame will be rendered in a 2-D and 3-D scene, visualizing the data in a DEM.

Methods

          The normalized data of the survey was imported into the geodatabase for the DEMs. From the imported table, a feature class was created, the x,y, and z values rendering a spatial extent of points.
The following five interpolation methods were used to visualize the data from the created feature class as DEMs:

• IDW

               IDW stands for Inverse Distance Weighted. IDW estimates cell values by averaging the values of the sample data points in the neighborhood of each processing cell. A sample points have more influence in the averaging process the closer they are to the center of the cell being averaged (ESRI, 2016).

• Natural Neighbors

               Natural Neighbors use an algorithm that finds the closest subset of input samples to a point and applies weights to them based on proportionate areas to interpolate values. Interpolated heights are within the range of the samples used. Therefore, peaks, pits, ridges, and valleys will not be rendered unless they are already represented by sampled points. Voronoi polygons are initially constructed around sample points. Then during interpolation, Moroni polygons are constructed around interpolation points. Therefore, the sampled point's polygon is weighted more in the interpolation process when it takes up the most area in the interpolation point's polygon (ESRI, 2016).

• Kriging

               Kriging is an advanced geostatistical procedure that generates an estimated surface from a scattered set of points with z values. This tool involves the investigation of the spatial behavior represented by the z values before selecting the best estimation method. Unlike other methods,
Kriging uses statistical models to predict surfaces and provide a measure of accuracy. It assumes distance and direction between sample points reflect a spatial correlation that can be used to explain variation in the surface  (ESRI, 2016).

• Spline

                Spline estimates values by a mathematical function that minimizes the overall surface curvature. The created surface passes smoothly through sample points. Spline is ideal for gradually varying surfaces, like elevation, water table heights, and pollution concentration  (ESRI, 2016).

• TIN

               TIN stand for Triangular Irregular Network. The sampled points in a TIN model are connected to each other through a series of triangles that represent a surface's morphology. TINs can represent ridges and streams lines readily. Tins can vary in their resolution based on the features it's meant to represent, increasing resolution for peaks and ridges and decreasing resolution for flat plains. The model's units are in feet or meters. TINs are typically used for modeling smaller areas with high-precision purposes. However, TIN models can be more expensive than raster models  (ESRI, 2016).

          Each Interpolation raster was visualized as a 3-D model in ArcScene and exported in a 2-D format and incorporated into map of the raster. If a person were to stand before the 3-D model as if they were facing the surveyed terrain, they would be facing north. This is the orientation depicted in the maps. A scale was edited into the maps to show the extent of sampling frame, which was a 115 cm x 115 cm sandbox.

Discussion

          The IDW interpolation method created numerous peaks and dips at the sample points that simply weren't present in the actual terrain (Figure 3). This is caused by how the interpolation methods works, so IDW doesn't seem like an ideal method to represent elevation collected systematically. Perhaps a stratified sampling method would avoid creating unnatural peaks and dips in a IDW DEM. A gully at the bottom left of the sandbox failed to appear in this DEM, and the ridges that do appear sport peaks that look like a mountain range which severely misrepresents the data.

Figure 3: Map of the IDW Interpolation method of the sandbox terrain.

          The numerous peaks and dips are gone with the Kriging method, but the sharpness of ridges and elevation changes aren't accurately represented (Figure 4). They seemed to have been buffered down too much. The ridge in the bottom left corner appear more subtle in the DEM than in reality. The rough texture generated by the Kriging method is reminiscent of the sloppy sculpting of sand, though this is due to the Kriging method, not a representation of a sandy terrain.

Figure 4: Map of the Natural Neighbor Interpolation method of the sandbox terrain.

          The Natural Neighbor interpolation output created a more seemly transition between elevation changes while generating the sharpness of the ridges and gullies (Figure 5). However, the representation is very angular and kind of disrupts the direction of some of the ridges and valleys. This might have been fixed by having more sample points along highly variable areas on the terrain.

Figure 5: Map of the Kriging Interpolation method of the sandbox terrain.

          Like it's model description, the Spline interpolation methods generated a smooth, continuous surface of the elevation (Figure 6). The pesky valley in the bottom left corner of the DEM is represented much better with Spline. The rides and slopes are nicely captured, though they might have been smooth down slightly too much. The Spline method appears to have best represented the terrain.

Figure 6: Map of the Spline Interpolation method of the sandbox terrain.

          While not the most pleasing to look at and not technically interpolation, the major land features are represented; though, they are rather over-simplified and don't reflect slopes that make up real terrain (Figure 7). The triangles make it difficult to make out anything in the 2-D map. The 3-D DEM is needed to be able to recognize the features of the terrain.

Figure 7: Map of the TIN Interpolation method of the sandbox terrain.

          It would have been interesting to see how a combination of a systematic point sampling and stratified point sampling method would represent the terrain. The systematic sampling method is ideal because it guarantees data points are dispersed equally across the sampling frame. A stratified sampling method would give highly variable land areas accurate representation and prevent the underrepresentation that tends to result from the systematic method. Since the study area is extremely small in this case, smaller cell sizes in the grid would be practical and beneficial for generating DEMs.

Conclusion

          This survey is similar to other elevation field survey, except on a much smaller scale. For larger land areas, a grid system for sampling wouldn't be practical. Depending on the terrain, a surveyor might have to choose a completely different sampling method, one that takes time, cost, measurements, and nature into consideration. In addition, while the Spline interpolation provided the most accurate DEM, that won't be the case for every survey. Besides elevation, interpolation can be utilized for data like microclimates, the pH or mineral content of the soil, or land use.

Sources

ESRI. (2016). How IDW Works. Retrieved December 3, 2017, from ArcGIS for Desktop: http://desktop.arcgis.com/en/arcmap/10.3/tools/spatial-analyst-toolbox/how-idw-works.htm

ESRI. (2016). How Kriging Works. Retrieved December 3, 2017, from ArcGIS for Desktop: http://desktop.arcgis.com/en/arcmap/10.3/tools/spatial-analyst-toolbox/how-kriging-works.htm

ESRI. (2016). How Natural Neighbor Works. Retrieved December 3, 2017, from ArcGIS for Desktop: http://desktop.arcgis.com/en/arcmap/10.3/tools/spatial-analyst-toolbox/how-natural-neighbor-works.htm

ESRI. (2016). How Spline Works. Retrieved December 3, 2017, from ArcGIS for Desktop: http://desktop.arcgis.com/en/arcmap/10.3/tools/spatial-analyst-toolbox/how-spline-works.htm

ESRI. (2016). How Spline Works. Retrieved December 3, 2017, from ArcGIS for Desktop: http://desktop.arcgis.com/en/arcmap/10.3/tools/spatial-analyst-toolbox/how-spline-works.htm

ESRI. (2016). What is a TIN Surface? Retrieved December 4, 2017, from ArcGIS For Desktop: http://desktop.arcgis.com/en/arcmap/10.3/manage-data/tin/fundamentals-of-tin-surfaces.htm

Wenzel, K. (2017). Database Normalization Explained in Simple English. Retrieved December 1, 2017, from EssentialSQL: https://www.essentialsql.com/get-ready-to-learn-sql-database-normalization-explained-in-simple-english/