It isn’t often that you get to talk with the top technology gurus at Caterpillar about the future of mining and the future of automation in mining. Paul Moore, IM Editorial Director had just such a sit-down meeting at MINExpo 2021 with Karl Weiss, Chief Technology Officer; Michael Murphy, Chief Engineer for Mining Technology; and Jim Hawkins, General Manager MineStar Solutions*

The amount of new technologies in mining, especially new machine capabilities, can seem overwhelming, not least at a show like MINExpo – but Caterpillar is keen to stress that everything has a purpose. “We aren’t developing technology for mining for technology’s sake – Caterpillar’s stated purpose is that our solutions help our customers build a better world and so we are really focussed on what the customer’s challenges are and solving them,” opened Karl Weiss, Cat CTO. He went on to emphasise the safety and security aspects of Cat’s MineStar Command for hauling with zero lost time injuries. BHP at Jimblebar, one of the pioneering Caterpillar Command for hauling sites, has reported a reduction in the frequency of safety incidents by 50% as an example.

Karl Weiss, Chief Technology Officer, Caterpillar

But Weiss added that because of the level of connected assets in these systems today that cybersecurity is an increasingly important factor – and stressed that this is now a core part of what Cat’s MineStar solutions bring to mining. And skilled labour remains a problem for customers – “obviously the autonomous haulage systems help with that because you don’t need as many operators in remote locations. But it is more than that – pre-COVID I met with a customer in Australia using our semi-autonomous dozer – the operator was beyond retirement age, when he transferred from working in the machine to a remote station in a trailer. He is now running four dozers instead of one.”

Playing for the ACE

On productivity Weiss said that its customers are consistently seeing a 20-30% increase in productivity and equivalent cost reduction. “So these are the types of problems we are trying to solve – but today we are also really focussed on what we call in the industry ACE – Autonomy, Connectivity and Electrification. Within our development teams, we are really starting to knit these elements together. And it has to be all three equally – we can’t reduce priority on any of them. When we do full electrification of a mining fleet, it is almost certainly going to need some sort of autonomy to help ensure that we don’t send a truck down into the pit that only has enough charge to get halfway back up. And where do you send it to get its next charge? All of these things are quite complex. But it has to be done – the customers that are seeing the benefits of autonomy will not accept going backwards with conventional electric fleets.”

Murphy added: “The customers are telling us every day that autonomy and electrification are joined at the hip for them.” And Weiss emphasised that connectivity is critical to making all that work – “even if you don’t have full autonomy and you want to get the benefits of improving your operation through our other MineStar solutions, you need to be connected. And that keeps evolving – 5G is coming and cloud technology is getting better all the time. How useful it is in terms of a deep pit depends on the coverage but suffice to say we continue to evolve to give mining customers the best connections to their fleets and their information.”

Autonomy for the little guys…

Weiss added that Caterpillar is also now focussed on taking the knowhow it has developed in mining and applying it in other resource-related areas. “Mining comes with its own economics, where productivity improvements pay for themselves very quickly. If you move then to quarrying, while of course it is akin to mining, it has a very different cost structure. So we are taking our mining MineStar knowhow and finding ways to simplify it for quarrying with lighter solutions then moving beyond that into construction plus waste & recycling. And often these have relevance to mining – we are working with a customer to automate solar farm construction – because it is a repeatable process using a skid steer or compact tracked loader. However, construction site autonomy is different, due to the complexity and volume of machines and movements. That is more about automating a certain machine doing a certain task currently.” Weiss added that a fall in the cost of autonomy-related hardware such as sensors had helped broaden its applicability.

Retrofits and sensor development

It was also fascinating to talk about big moves that Caterpillar has made in autonomy over the years, and Jim Hawkins said that the decision early on to go with autonomy as a retrofit option as well as a new truck capability was a major step: “We have also done a tremendous amount to facilitate the retrofitting of sensors and other hardware. And that was a big decision back in 2015 – initially Cat had envisioned that autonomy would only be applied to new trucks. But the growth of interest in autonomy coincided with an economic downturn that meant that the appetite for capex in new equipment was pretty small. So to get that safety and productivity benefit out there, we made a quick decision to make it retrofit available, and that was a huge decision at the time.”

Hawkins added that while the sensor companies are now releasing more industrial grade products, Cat has also done a lot of its own sensor evolution work – taking off the shelf sensors and ruggedising them for the mining environment. This includes a lot of hardening of electronic components. “This has made a huge difference in availability of the autonomous system.”

Murphy added: “What started out as something which took months down at the proving grounds has now come down to eight days using Caterpillar’s production system. Our production engineers just laid it out like it would be in the factory in the customer workshop – bring the truck in, get it fitted, run it at the customer test track area and get it into the fleet as quickly as possible. It is a great example of how we continue to refine as time goes on our autonomous mining processes.”

So what proportion of the Cat autonomous haulage fleet today is retrofit versus “new?” Hawkins said: “We can’t give you an exact figure but let’s clarify it this way – today I would say overall it is approximately 75% retrofit to 25% factory fitted. In the future of course this is going to flip as fleets age.”

Michael Murphy, Caterpillar Chief Engineer for Mining Technology

Murphy adds: “Another point is that when you look at where the industry is – the mining companies up to now have not been building very many greenfield mines – today we have three greenfield autonomy projects in process – QB2, Quellaveco and Gudai Darri.”

And are new autonomous trucks any more effective or productive than retrofits? Hawkins said that in theory no, they should be able to achieve the same results but that it would really come down to the base truck – if the base machine had a certain efficiency to begin with then the MineStar autonomy will enable it to maximise that. “Today’s retrofit speed is very important – you can’t have a multi-million dollar asset idled for a month to be retrofitted as that would be a lot of lost production but it is still eight days. At the same time, our factory fitted autonomous trucks while generally they don’t have all the sensors added yet because the trucks over 100 ton class don’t ship in one piece anyway and are assembled at site – but all the necessary wiring, bracketry etc plus links to hydraulics and brakes is done so you can literally plug and play. When we retrofit it is that piece that takes time.”

Biggest AHS fleet in the world, plus increased machine diversity

Collectively Cat now has over 460 autonomous trucks running – now the largest in the world – and has passed 3.5 billion tonnes hauled – this still trails Komatsu by 0.5 billion tonnes, but Komatsu was out of the blocks first, commercially anyway, with Codelco Gaby in 2005 then the first Pilbara site at West Angelas in 2008. But Caterpillar in recent years has been accelerating its autonomous reach. Plus the diversity of the truck types is fast increasing. It was initially dominated by the 793F – but everything from the 785 to the 797 is now available with autonomy, with QB2 and Quellaveco being a good case in point as these are 794F AC fleets. And this is also significant given that these recent autonomous fleets are mainly electric drive not mechanical drive.

Plus it isn’t all just about trucks – the Cat dozer fully autonomous capabilities continue to evolve – including to a broader application range; plus Command for drilling is stepping up from single row to multi-pass. Underground, autonomy is well established on LHDs and is now expanding to trucks, starting with the AD45. And back to surface, Greg Hepler, General Manager of Tracked Products – Resource Industries at Caterpillar confirmed to IM at MINExpo that the company has a very active nextgen large hydraulic mining shovel R&D program which includes looking at teleoperation and autonomy, with prototypes already having run for some time at the Tinaja Hills proving ground in Arizona.

Hawkins added: “Our work is focussed on understanding what it will take to automate the shovel plus we have made some acquisitions of IP that will help us. But it remains one of the last machines that mining customers are looking to automate.”

Weiss commented: “Each equipment area has its own automation challenges. Look at the 777G autonomous water truck we recently announced. The operator running a water truck knows how to water the haul road correctly – in the right places and how much. It took a lot of work to automate that.”

Murphy continued: “I think that is where you have to look at the process. Why would our customers want to automate a water truck ahead of a shovel? Safety and productivity, yes – getting the watering right can make a big operational and safety difference. Plus, with an AHS fleet what causes the most variability? It’s the manned machines that remain; and so the water truck is often slowing down the autonomous trucks – so it was the data and the analytics that made it the next logical choice. Now that they don’t slow down the autonomous trucks however if they overwater that can cause its own traction issues. Automating watering makes it more consistent and removes as much variability as possible.”

Hawkins also pointed out that it wasn’t simply putting a tank on an autonomous truck – it was integrating that watering system with the machine and autonomy layer to get that right amount of water in the right place at the right time. “The system knowing where you watered, how long it will take to evaporate, when do you need to go back.”

So is there an appetite for graders, wheeled dozers and other vehicles? Hawkins says all of this is on the Cat autonomous roadmap “but we are looking at this holistically in terms of surface management. But the machines you mention tend not to be utilised that much and that efficiently – autonomy should allow for much greater utilisation especially around loading and tipping areas.”

Battery electric transition and autonomy

Back to ACE – how are customer buying decisions relating to autonomy being affected by the electric mine transition? Hawkins replied: “Customers are at a tipping point today. You buy a new diesel truck now that’s autonomous; or use the assets you can for longer and wait for what’s coming with battery electric? We are working with these customers today on how we can provide their autonomy needs but through that electrification window.” Following the meeting, this was exemplified by Cat and Rio Tinto’s announcement that they are to jointly develop a future 220 t 793 zero-emissions autonomous haul truck; leading to operational deployment of approximately 35 new Caterpillar 793 zero-emissions autonomous haul trucks at Gudai-Darri. Caterpillar had also in August announced a partnership to develop and deploy zero-emissions mining trucks at BHP sites to reduce operational greenhouse gas (GHG) emissions; while following MINExpo it announced a strategic alliance with Newmont in November to achieve zero emissions mining.

And the group agreed it will be a staged introduction of battery trucks with co-existence of battery with diesel trucks plus autonomous and non-autonomous trucks. Plus there are likely to be both mobile and fixed charging developments. Murphy added: “Today we look at our truck efficiency in FMS terms mainly in terms of two variables – production in making sure you don’t have trucks queuing and blending making sure the trucks are in the right place at the right time based on ore grade. Electrification will bring in a third variable – where and how do you charge to keep the trucks running. So those algorithms are going to change very significantly as time goes on. It is going to be complicated due to the number of variables but on the plus side we have so much sensor data today plus on and off board computer power that we can do it.”

Hawkins continued: “That is why autonomy is going to be so critical. You have to eliminate human variability in reacting to truck assignment decisions. If we tell an operator you need to go and charge the truck now at a fixed charging station or pick up a charge on the haul road and for whatever reason they delay it or it doesn’t happen, you could easily end up with a dead battery and a stranded truck. Don’t get me wrong, we love operators, but we know it will happen as we see it in the conventional fleet environment all the time. The best trained operators can almost compete with autonomous fleets but people move on and training falls off.”

So is it likely that the future autonomous MineStar FMS screen for zero emissions fleets will show a lot of trucks with a battery symbol indicating relative remaining charge? Murphy said that to some extent Caterpillar has this today but showing fuel levels – “the difference being with the fuel we have a 24-hour tank so we have time to make decisions about when to bring the truck in and what is the optimum time. It is just going to get more time critical and more complex. Even today with an autonomous fleet some of our customers do operate an F1 type pitstop system so if they are over trucked at any one time, they can bring an AHS truck out of the circuit and park it then bring it back in when they have the need for it. You can do all this when you have a high level of predictability in the operation.”

A new zero emission strategic alliance was announced in November 2021: pictured are Rob Atkinson, Executive Vice President and Chief Operating Officer Newmont, Tom Palmer, President and CEO Newmont, Jim Umpleby, Chairman and CEO Caterpillar Inc., and Denise Johnson, Group President, Caterpillar Inc.

How would Cat summarise its approach to charging strategy with autonomy? Hawkins commented: “We are approaching from the point of view of what is going to be best to meet production requirements for that shift or day or week. And there is no doubt about it there are some big paradigm shifts that are going to take place. Today we maximise the rate of the trucks managing the assignments across the site and the number of hours you can do that in a given day. In this fast-approaching future world, it may be the right decision to meet the production targets to slow the truck down and take a charge and get the energy state of that truck up.”

Murphy added: “Today, customers have a plan they need to get done for that shift, and here are all the variables so what’s the best combination of permutations of trucks, loading tools, loading areas, ore grade but adding in battery life to make it all come together.”

Trucks as decision makers

With edge computing and more onboard computing power, the trucks are becoming more able to make decisions themselves as they acquire more AI such as recognising obstacles themselves and taking their own evasive action rather than stopping and awaiting instruction. Hawkins highlighted the fact that Caterpillar trucks today have tremendous computing power, for one thing indicated by the amount of cooling power visible on top of the truck in the cabinet, so they can already do a lot more independently including object detection and avoidance. “But it is the assignment decisions – what to do, not how to do it – that is done in the back office – and that’s where you need visibility of what’s going on across the whole site.”

Murphy continued: “Over the last 30 years I think Caterpillar has done a good job of balancing onboard and offboard processing. The advantage of onboard is obviously it works even if you don’t have radio comms but the difficultly is that these computers are working in a rough environment with a lot of vibration and heat. We can make them live and we do every day, but it’s a challenge. Overall I would say decision making that is not as time critical and relates to wider mine operations is still done in the office, and faster decisions regarding the environment close to and around the truck is done onboard. As technology evolves we have altered the balance accordingly and today, the truck makes its own decisions based on the sensors.”

Hawkins added: “The trend will be for more and more onboard processing and we have a lot of things underway to better utilise AI and machine learning to increase what I would call the decision range the machine is able to make. But also to reduce the level of reliance on the intensity of the radio network. And to reduce reliance on GPS for positioning, being able to use other technologies for that which are less infrastructure dependent.”

Weiss made reference to a strategic acquisition in this regard: “A year ago we acquired a small start-up in the Bay Area called Marble Robot; and the reason we did that is that they were developing technology to do last mile delivery of packages in the middle of the city. So their knowhow included two things – one, how to manage when you lose connectivity quickly in city ‘canyons’ of high rise buildings and second, managing all kinds of obstacles that are coming and going. So they are able to allow that small vehicle to continue to operate without a network signal and to keep replotting its course as it goes along. So we are leveraging that knowhow now in mining autonomy as we evolve to get to a point where AHS truck shutdowns become less and less.”

Does that mean the heavy push towards LTE and onto 5G will become less important? Weiss said he still believes that 5G will be critical to enable the industry to not have to put WiFi everywhere and that the developments are more about handling small periods of time where there is a loss of connection. Murphy comments: “We want more decision making on board and close to the truck. But at the same time there is going to be so much data flowing that we need the bigger bandwidth to get data to the machine and off the machine. There’s the real time decisions but then the stuff you have to do in the background. We also have to be able to look ahead – today our MineStar algorithms look about an hour ahead – we want to be able to do that in the background and when you need to make a real time decision you are able to give the machine the right input. Believe me I have been in this business over 25 years, and I have never seen any of the radio comms at our customer sites get less. Today they want more video and more data off their machines.”

Today you have data from GPS, GNSS, radar, LiDAR and other sources – is this constant and is it possible to use only the data that is required to get the right picture at that time and therefore save on bandwidth? Murphy said that MineStar doesn’t actually stream all of the data all of the time and this is where our experience comes into play – what do you want to stream, when do we need it, how does it differ to the last data set from that sensor and so on. He says MineStar already manages that very well.

Hawkins added that it has customers in South America with hundreds of trucks in one mine working in a deep pit and at the time the autonomous fleet was put in it wasn’t possible to have network coverage all of the time – “so we had to make the trucks be able to think for themselves for short periods.”

Murphy adds: “A good example is when a truck gets into the load area. It knows where the shovel is and makes its own calculations on how to get there and queue or spot. It scans the face with LiDAR as it comes into the area to make sure it is safe. It also scans to calculate how much the shovel has moved.”

Do all of the sensors still have a role to play in AHS? Murphy says: “They all have their part to play. LiDAR works great at high speed and long distances; radar works through dust; cameras provide enhanced visuals. So at the moment just like in automotive we need it all. Will that change in five or ten years time? That’s certainly possible but not yet.” Hawkins adds: “First they will get more compact and less expensive; I think cameras will take on a bigger role eventually but it is still early days and it depends how things evolve.”

Training for autonomy

Last but not least, what about the evolution of training for autonomy? A long term relationship with Simformation plus a recent tie-up with ThoroughTec has enhanced training capabilities for truck operators, and for shovel operators interacting with AHS fleets. But for the AHS system itself, Cat takes care of this in-house. Hawkins comments: “We developed our own offboard training for the controllers, builders and application specialists. With the pandemic we are utilising also a lot more virtual capabilities – at our Rapid City, South Dakota design centre we are collaborating to develop further virtual tools for things like maintenance of both autonomous and non-autonomous fleets. And our dealers are doing great work – WesTrac now has its dedicated training facility in Collie, which is completely focused on the technical skills required to operate autonomous equipment for use in the resources sector.”

The Rapid City centre is known as the Black Hills Engineering Design Center, which opened in 2010 to provide additional engineering support services for Caterpillar, complementing existing engineering design centres in Chennai, India and Champaign, Illinois. Murphy said Cat has also created a simulation school – an environment where you have a simulated mine similar to that used in the airline industry. “We built our own – and we now have one in Western Australia, one in Tucson and one in Santiago, Chile. These simulation capabilities mean we can put people through there, they can run the MineStar system and go through a full training course – it has been very successful.”

*Jim Hawkins has since retired from Caterpillar