Applying Drone Photogrammetry to 3D Point Cloud Mapping for Mining

Introduction

Photogrammetry is the process of getting information about an area of interest from photographic images. It has become an essential tool for mining companies looking to optimize their processes, reduce costs, and minimize environmental impact.  This involves a technique called point cloud mapping for mining. 

Drone photogrammetry has significantly improved surveying, mapping, and stockpile management by providing faster and more accurate data collection methods. Drones enable efficient exploration and prospecting, improving geological surveys and mineral deposit identification. They also enhance safety by inspecting hazardous areas and highwalls, reducing risks to workers. Furthermore, drones play a vital role in environmental monitoring and compliance, ensuring that mining operations adhere to environmental regulations and minimize ecological impact.  

Drones with high resolution cameras are commonly used to survey large areas to make measurements. This usually involves taking the images and stitching them together to create a 3D map or 3D point cloud. A 3D point cloud is a collection of data points in a three-dimensional coordinate system, representing the spatial positions of objects or surfaces in the real world. Each point in the point cloud is defined by its X, Y, and Z coordinates and can also carry additional information, such as color or intensity values. 

These 3D representations are then used to make volumetric measurements. By capturing detailed 3D models and aerial imagery, drones aid in mine design and layout optimization. They offer cost savings, especially in remote or inaccessible locations, and facilitate remote site management. The photogrammetric data collected by drones can be seamlessly integrated into mining software, empowering data-driven decision-making and improving overall operational efficiency. 

Figure 1. Sample mine where photogrammetry can be used to measure the volume of soil extracted from the previous day.

Problem

A large, recognized mining company used EyesOnIt to streamline their photogrammetry operations due to our extensive experience with computer vision and software development. The mining company previously worked with various drone pilots to capture images of their mines, make the 3D point clouds, manually “clean” (remove manmade objects from) the point clouds, and upload them to the company’s cloud environment. However, this process typically took a lengthy 14 days, and the quality of the uploaded point clouds varied significantly between different vendors.  

The mining company wanted to automate point cloud generation to produce consistent, high quality point clouds within 24 hours. In addition, they wanted point cloud cleaning to be automated.  These changes would result in an accurate digital twin of their mine.  

Applying our expertise in cloud computing, EyesOnIt streamlined the acquisition of drone imagery and quickly automated point cloud generation once drone imagery was made available from the pilots. 

Solution

EyesOnIt developed an easy-to-use portal for drone pilots to upload their images. We used a software package called Pix4D to process uploaded imagery and generate the point cloud. Pix4D is an extensive photogrammetry package; there are many settings that allow users to modify the processing such as adding a geological boundary around an area for the 3D point cloud, specifying the density of points in the 3D point cloud, and processing with different coordinate systems. 

Point clouds do a great job of representing the mine surface, but they also include artificial or man-made objects like trucks, shovels or other mining equipment. These artificial objects prohibit customers from making accurate measurements of volumetric changes in the mine surface. Artificial objects need to be removed from the point cloud as shown in Figure 2 before those point cloud mapping for mining can be used for volumetric calculations.

Figure 2. On the left, the original image of the mine; on the right the clean mine. (These images are 2D. The real point cloud is 3D).

Optimizing the Point Cloud Mapping for Mining Cleaning Process

Automated point cloud cleaning can be a difficult task since it is not always clear which objects should and should not be removed. Under normal conditions, a computer vision model would be trained to detect specific objects and would pick those out of the point cloud. The reality of a large outdoor mining operation is that there are very few controls regarding what kind of objects may end up there. As a result, there is not a discrete set of “known” objects that a model could be trained on. Point cloud cleaning is further complicated by the mining environment where dust and mud on artificial objects makes them difficult to distinguish from natural objects.

The EyesOnIt team trained a model using commonly known artificial and natural objects to ensure that the resulting model was generalizable enough that it could extrapolate to objects it had not seen before, such as a new vehicle, or new geological features. When testing this methodology, our process far outperformed leading third–party cleaning solutions.  

Previously, the cleaning was done manually by the drone vendors with varying degrees of thoroughness and precision. Our goal was to automate as much of the cleaning as possible to reduce labor hours and expedite final measurements. To do this, we first applied a cloth simulation algorithm which inverts the point cloud and simulates a cloth draped across its 3D surface, as demonstrated in Figure 3. Any point of the cloth which touches the surface is considered a “ground” point. The ground points are removed, leaving clusters of points behind which are then classified as natural or artificial. 

Figure 3. 2D representation of a point cloud showing ground, a house, an undulation in the ground, and a tree trunk. The red line represents the cloth from the cloth simulation algorithm.

EyesOnIt then leveraged our experience in advanced analytics and machine learning to convert the top-down view of the point clouds clusters into standard images suitable for computer vision. The customer provided information regarding the smallest and largest objects in their mines, such as area and perimeter. With this information, we were able to discard some objects because they were over/under the thresholds for the customer’s objects.  

Image Classification for Point Cloud Mapping

Next, a neural network was deployed to extract features such as lines and circles from the cluster images. We trained a classifier using the images obtained from the customer’s mines to determine if objects protruding in our cloth simulation should be classified as natural or artificial. We only used high confidence scores from our model to avoid removing objects that could not be accurately identified as artificial. This process is shown in Figure 4. 

Figure 4. The process for classifying point clusters as either natural or artificial.

Once a cluster was classified as artificial, we removed it from the point cloud leaving a flat surface in its place. Automated cleaning removed over 90% of artificial objects. While we expect that a mining analyst will need to perform minor cleanup of remaining artifacts, our software remembers each artifact that the analyst removes and automatically removes that artifact from future datasets. This approach, coupled with our automatic cleaning, reduces manual cleaning effort by up to 98%. 

We were able to successfully deliver the automatically cleaned point cloud, giving the customer time to do a final cleaning inspection within the 24-hour deadline. 

Conclusion

In response to the challenges faced by a prominent mining company in streamlining their photogrammetry operations, EyesOnIt, a recognized leader in computer vision and software development, was engaged to transform their processes by applying point cloud mapping for mining. The mining company had previously relied on multiple drone pilots to capture mine imagery, create 3D point clouds, manually clean these point clouds, and upload them to their cloud environment, resulting in a cumbersome and time-consuming 14-day process. Moreover, the quality of the point clouds exhibited significant variations across different vendors. 

Recognizing the need for consistent high-quality point cloud generation within 24 hours and the ability to remove “man-made” objects from these point clouds for an accurate digital representation of their mine, the mining company turned to EyesOnIt for a solution. Leveraging their expertise in cloud computing and management, EyesOnIt efficiently streamlined the acquisition of drone imagery and automated the point cloud generation process.

This innovative approach not only met the client’s stringent time and quality requirements but also offered the means to remove undesired elements, such as buildings or vehicles, thus delivering an enhanced digital twin of the mining site. The successful collaboration between EyesOnIt and the mining company exemplifies the transformative power of technology and expert knowledge in optimizing mining operations.