Having recently helped DJI’s M300 drone fly autonomously underground (through its Hovermap Autonomy Level 2 (AL2) solution) and signed an agreement with Deswik to provide surveyors and planners with more accurate data from inaccessible areas, Emesent has been on a roll of late. IM put some questions to CEO, Dr Stefan Hrabar, to find out more.

IM: First off, if no communications infrastructure is in place at an underground mine, how do Emesent’s drones stream a 3D map of the environment back to the operator’s tablet?

SH: Hovermap is smartly designed to operate beyond the communication range of the operator. The operator does not always need to see a live map since Hovermap is navigating by itself. The user can place a waypoint beyond the current limits of the map, and beyond line of sight and communication range. Hovermap self-navigates towards the waypoint, avoiding obstacles and building the map as it goes. Once it reaches the waypoint (or if the waypoint is impossible to reach), it automatically returns back to the operator. The map data is stored onboard Hovermap and when it returns back to within Wi-Fi range the new map data is uploaded to the tablet. The operator can then see the new areas that were mapped and place a new waypoint in or beyond that map, sending the drone back out again to explore further.

IM: What results have you so far received from using AL2 for Hovermap at mine sites? Were the results PYBAR got from trials at Dargues and Woodlawn in line with your expectations?

SH: Last year’s trials at Dargues and Woodlawn showcased some great outcomes for the PYBAR team, including the ability for Hovermap to capture valuable data using Autonomy Level 1 (AL1). The team saw great potential in the technology, leading to the purchase of two systems for their use. Earlier this year, AL2 flights were conducted at Dargues during the final pre-release testing phase. Even the first stope at Dargues that was mapped using AL2 highlighted the benefit of the system over traditional CMS (cavity monitoring systems). A large area of overbreak was identified in the Hovermap scan. The same stope had been mapped with a CMS, but this area was not visible from the CMS scan location so the overbreak was not identified.

A number of mines have been using AL2 to map their stopes and other areas beyond line-of-sight. With AL2, they can send Hovermap into places that previously would have been inaccessible, enabling them to obtain critical data in real time without risking the machine or personnel.

The AL2-based stope scans have been more detailed and complete (lack of shadowing) than ever before. A beyond line-of-sight flight down an ore pass was also conducted recently, with Hovermap guiding the drone down 120 m and returning safely to produce a very detailed scan.

The high level of autonomy provided by AL2 also allows remote operation of the drone. We recently completed a trans-continental demo, with a customer in South Africa operating a drone in Australia using our AL2 technology and standard remote collaboration tools. The remote operator in South Africa was able to use their laptop to experiment with the technology from the other side of the world, sending Hovermap exploring down a tunnel.

This is a taste of what’s to come, with drones underground being operated from the surface or from remote operation centres thousands of kilometers away. This will remove the need for skilled personnel on site, and reduce the time spent underground.

IM: What had been holding you back from achieving AL2 with drones/payloads? Is it the on-board computing power needed to that has been the issue?

SH: Flying underground where there is no GPS, the space is tight and there are hazards such as mesh, wires, dripping water and dust is very challenging. We overcame many of these with AL1, which makes it safe and easy for a pilot to operate the drone within line-of-sight (Hovermap provides collision avoidance, position hold and velocity control). AL1 has been deployed for 18 months with many customers around the world, clocking up thousands of hours of use. This helped to improve the robustness and reliability of the core flight capabilities.

AL2 builds on this mission-proved base capability to provide additional features. AL2 allows the system to fly beyond line-of-sight and beyond commination range. This means it’s on its own with no help from the operator and needs to deal with any situation it comes across. There are many edge cases that need to be considered, addressed and thoroughly tested. A significant amount of effort was put into these areas to ensure Hovermap with AL2 is extremely robust in these challenging environments. For example, the drone downwash can kick up dust, blinding the LiDAR sensor. We’ve implemented a way to deal with this, to bring the drone home safely. Other considerations are returning in a safe and efficient way when the battery is running low, or what to do if waypoints cannot be reached.

IM: How do you anticipate your partnership with Deswik impacting the mine planning and survey process? Do you see this reducing the amount of time needed to carry out this work, as well as potentially cutting the costs associated with it? Have you already carried out work at mine sites that has proven these benefits?

SH: Our commitment is to help mining companies increase safety and production while reducing costs and downtime. We do this by providing surveyors and planners with more accurate data from inaccessible areas, allowing them to derive new insights. Our partnership with Deswik means we’re able to provide a more comprehensive end-to-end solution to the industry.

We see this as a very natural partnership that will improve the overall customer experience. Hovermap excels at capturing rich 3D data in all parts of the mine (whether drone based, hand-held, lowered down a shaft on a cable or vehicle mounted). Once the data is captured and converted to 3D, customers need to visualise and interrogate the data to derive insights. This is where Deswik and other mining software vendors come into play. They have powerful software tools for planning, survey, drill and blast, geotechnical mapping and a host of other applications. We’re partnering with these vendors to ensure seamless integration between Hovermap data and their tools. We’re working with them to build automated workflows to import, geo-reference, clean and trim the data, and convert it into formats that are suitable for various tasks.

Surveyors at Evolution Mining’s Mungari operation have been using this new process in Deswik. Previously they needed a third software tool to perform part of the workflow manually before importing to Dewik.CAD. The intermediate steps have been eliminated and others have been automated, reducing the time from more than 30 minutes per scan to five minutes per scan.

IM: Since really starting to catch on in the mining sector in the last five years, drones have gone from carrying out simple open-pit surveys and surveillance to drill and blasting reconciliation platforms to reconnaissance solutions carrying out some of the riskiest tasks in underground mining. In the next decade, how do you see them further evolving? What new tasks could drones carry out to improve safety, cut costs or increase productivity?

SH: Emesent’s vision is to drive forward the development of ‘Sentient Digital Twins’ of industrial sites to future-proof the world’s major industries, from mining to energy and construction. These industries will be able to move to more automated decision-making using high-quality, autonomously collected data across their sites and tapping into thousands of data points to make split-second decisions about potential dangers, opportunities and efficiencies using a centralised decision-making platform.

We see our Hovermap technology being a key enabler for this future. Drones and other autonomous systems will become an integral part of the mine of the future. Drones will be permanently stationed underground and operated remotely, ready for routine data collection flights or to be deployed as needed after an incident.

Hovermap is already addressing some of the biggest challenges in mining — including safety and operational downtime. It improves critical safety to mines, keeping workers away from hazardous environments while providing better data to inform safety related decisions such as the level of ground support needed. This then feeds into better efficiency by helping mines to more accurately calculate risks and opportunities, aid decision making and predict situations.

Hovermap can significantly reduce downtime after an incident. For example, it was used to assess the level of damage in LKAB’s Kiruna mine after a seismic event. More than 30 scans were captured covering 1.2 km of underground drives that were not safe to access due to fall of ground. In another case, one of our customers saved around A$20 million ($14.6 million) after an incident, as they could use Hovermap to quickly capture the data necessary to make a critical decision.

IM: In terms of R&D, what future payload developments are you investing in currently that may have applications in mining?

SH: We’ll keep adapting our Hovermap design to suit new LiDAR improvements as they are released. More importantly, we’ll improve the autonomy capabilities so that even more challenging areas can be mapped with ease. We’re also adding additional sensors such as cameras, as these provide additional insights not visible in the LiDAR data. Our colourisation solution is an add-on module for Hovermap, which uses GoPro video to add colour to the LiDAR scans. This allows the identification of geological and other features.