Caterpillar has been using virtual-reality technology in machine design for 20 years, and recently commercialized the first application of augmented reality to heavy equipment service. What these technologies do today might seem esoteric and, perhaps, a little anti-climactic given the hoopla VR and AR devices are generating in the consumer electronics sphere.
But the path to a new world of machine operating and service start to take shape when you consider how these technologies could easily be combined and delivered to operators and technicians in the field.
Cat engineers load a machine’s CAD files into a virtual-reality program and step into a VR “cave” where three-dimensional images are projected on the walls to immerse themselves in the details of a machine. From sitting in the cab, to slicing through body panels, castings and components like waving a wand, they can access every nut and bolt on a machine in 1:1 scale without ever having to lift a wrench.
Cat partnered with computer video-card manufacturer Invidia and its VR software supplier to push the performance of the HTC Vive consumer VR headset by a factor of 20 to be able to run a very similar version on the head-mounted display.
“This is a full, 100% representation of our product – our CAD file, the same file we use to build a D6,” says Galen Faidley, senior engineering project team leader at Caterpillar. “And it is running at full frame rates on a single desktop computer with one graphics card. This is technology that’s only a few months old.”
The development will eventually immerse a lot more engineers in virtual machines because the personal computer and VR goggles are far less expensive than projection-based technologies, and far more portable. Bigger implications include adapting the CAD content to service and repair processes, too.
At the same time, Caterpillar announced the license of live-support video calling that uses augmented reality to show a technical expert in a remote location a machine problem that’s stumping a technician in the field. The tech calls the expert, points his phone or tablet camera at the problem area of the machine, and the expert on the other end can see what’s going on. The expert can annotate the image he’s seeing, and the notes show on the technician’s end, too. So communication is clearer and more productive.
This technology is being offered to Cat dealers today. Integrating the CAD information in exchanges with remote experts will likely be an important future step in helping technicians conquer the toughest troubleshooting challenges. The depth of the CAD content offers an incredibly detailed window into how machines come together and work.
Caterpillar recently demonstrated a third application for computer-enhanced reality that displays vital safety and performance information right in front of the machine operators’ eyes. Cat showed a proof of concept with a mostly empty, static cab and a pair of Microsoft Hololens augmented reality glasses. The display in the glasses replaces the buttons and switches in the cab with a visual electronic interface the operator can manipulate through the AR headset.
“If this was a car, you could say I left you the steering wheel, the gas pedal and the brake. All the electronics on the dashboard disappear. When you put on the AR headset, all that functionality comes back into view,” says Lonnie Johnson, augmented reality and virtual reality technology portfolio manager at Caterpillar. “The physical electronic components are gone from the cab, but the capability for you to control all those functions is in the augmented reality headset.”
The next step is to develop a proof-of-concept that replaces switch panels and displays with the goggles in a working machine. The benefits far-exceed “gee whiz.”
The headset becomes a programable key to the machine. If a walk-around is required before going to work, the headset can be programmed not to show the operator the machine’s start button until the inspection is completed. The operator can change the size and location of displays of views from safety cameras gauges, and indicators. The goggles can show the operator warnings and location indications when laborers who are wearing RFID tags are on the ground in their working area. Augmented-realty technology can overlay information from intelligent-compaction or grade-control systems in 1:1 scale on the operator’s view of the jobsite.
