Cameras and Radar Activate Safety on Site

Active safety systems combine technologies to help operators better understand a machine surroundings.

Within the cab, Hyundai’s AAVM system provides operators with a virtual, composite view that shows the machine in the center and actual images from the four cameras.
Within the cab, Hyundai’s AAVM system provides operators with a virtual, composite view that shows the machine in the center and actual images from the four cameras.

Safety continues to be an important element in equipment design, particularly for larger machines in which operator visibility is hindered due to the size of the vehicle. While mirrors and cameras aid in improving visibility, they can only do so much. Peter Evans, vice president of sales and marketing, Preco Electronics Inc., notes these components are passive because they only work if operators are looking at their mirrors or the camera monitor. “That’s why you need the active technology of radar,” he says.

Preco’s PreView Plus, for example, utilizes pulsed radar for proximity detection of objects around a vehicle along with a camera. “It’s a system that can actively give what we call a ‘gentle tap on the shoulder,’” says Evans. “We proactively engage the driver with audible and visual alerts. The audible is coming from the radar detection and the visual is coming from the camera monitor.”

Radar is typically the technology of choice for systems used in heavier duty applications due to its ability to withstand the rugged environmental conditions in which these types of machines work. According to Dale Hessing, Preco’s vice president and chief technical officer, radar provides better performance for both on- and off-road applications, as well as in darker environments and inclement weather. While ultrasonic or laser detection technologies have their place, he says they are better suited for short range or clean environment applications.

In July, Hyundai Construction Equipment Americas introduced an active safety system called All Around View Monitoring (AAVM), for its new HX Series hydraulic excavators. Four cameras placed on each side of the vehicle — a traditional backup camera, two side cameras and a front-mounted camera — are used to provide a 360° view around the vehicle. The operator sees a virtual, composite view that shows the machine in the center and actual images from the four cameras.

[VIDEO] Hyundai HX Excavators Offer Optional Camera Mode to Increase Safety

In addition, the system includes Hyundai’s Intelligent Moving Object Detection (IMOD), which is capable of detecting objects that enter into a 16 1/2-ft. radius of the machine. “If an object were to come in anywhere within that 360° radius, the operator hears an alarm and sees an indication on the monitor that there’s an object in that range,” explains Corey Rogers, marketing manager at Hyundai.

Developed by parent company Hyundai Heavy Industries (HHI), AAVM is an improvement upon the company’s previous system, which used two side cameras and a backup camera to provide three angles of view around a machine.

“There’s definitely a trend toward improved safety on construction equipment,” says Rogers. “The ultimate goal is to provide the safest working environment for the operator and crew around the machine.”

A More Informative System

Pairing radar technology with that of a camera into a single system enables it to better understand a machine’s surroundings.

Fred Andersky, director of Customer Solutions - Controls, Bendix Commercial Vehicle Systems LLC, explains that using the two technologies in tandem allows the system to gain more insight by combining visual information from the camera with object detection provided by the radar. “Now we can do more things more robustly than we were able to do with just the radar alone,” he comments.

Stationary vehicle braking is one capability now available through the combination of camera and radar technology in Bendix’s Wingman Fusion safety system. If the system detects a stationary vehicle in a truck’s path, it will first alert the operator. If no action is taken, the system will automatically engage the brakes to help prevent a collision, or at least attempt to reduce the severity if one occurs.

“When we had the radar by itself we would provide an alert,” says Andersky. “But now, because we’re able to do more confirmation and cross checking of what it is the radar and camera see, we have a higher level of confidence that the object in front of us is a vehicle and we do want to apply brakes to help mitigate a potential collision.”

Including a camera also helps with other features of the system, such as lane departure warning and traffic sign recognition for over-speed alerts. Using the camera, the system can read speed limit signs and give an alert if drivers are going 5 miles over the speed limit. At 10 miles over, the system will alert the driver, as well as momentarily cut the throttle to get the driver’s attention.

Alert prioritization is another key component, helping to avoid distractions that may be caused by having two alerts going off at the same time. “In the past, you could have a collision mitigation system by Bendix and you might have a lane departure warning system by somebody else,” says Andersky. “Now it’s all integrated together. So if the driver is in a situation where you could have both a collision warning and a lane departure warning at the same time, the system prioritizes the most important alert and provides that to the driver.”

When developing these types of systems, Hessing says Preco not only spends a lot of time ensuring their ruggedness but also understanding the types of applications in which they’ll be used. The company has to take the time to understand the types of objects that are important to detect and zones of view for the vehicle type in order to write the appropriate algorithms for the system. “[We work] with some of the OEMs to pick out certain objects or ignore other smaller objects that may not be of interest to the operator,” he notes. “There’s a lot of customization we spend time doing.”

Ensuring the reliability of the system and that all of its components work together properly is also key to development, says Andersky. For example, because the Bendix Wingman Fusion system has the capability to automatically engage the brakes when necessary, it’s crucial that the system judge the threat of collision as accurately as possible. Otherwise, engaging the brakes for no reason would not only be irritating for the driver but also a potential safety hazard.

“We have to make sure systems now have as minimal false alerts, false interventions, as possible, because now that we’re intervening on more things, it can create more issues,” says Andersky.

Growing Market Acceptance

Evans says there is a definite trend within the heavy equipment industry toward active safety systems. Though more widely accepted outside the U.S., the systems are gaining ground in the States. Education on the benefits of the technology has been aiding market acceptance, along with the marketing efforts of the auto industry.

“Think of someone who gets in their car to get to the work site. Their car has [an active] system in it, then they get into their heavy piece of equipment and it has nothing,” says Evans. “And so really the market is demanding the product. It’s just the acceptance of the OEMs and some of the aftermarket people to put that budget together and make that commitment to safety.”

Hessing adds that OEMs and fleets alike want to be safe, but need to be convinced of the return on investment, such as the potential for reduced insurance risks and thus costs. “We spend a lot of time helping them understand the safety equation.”

According to Evans, the transition to active systems is generally a three-phase approach beginning with discussions and investigations into potential safety systems to use. The intermediary phase is adding a camera and monitor for that extra visibility around the vehicle.

“What they quickly find, with a camera/monitor by itself, is it becomes irrelevant very quickly because of the dirt, dust, sunlight. It’s not robust, and as a passive technology, they quickly move over to the active radar technology,” Evans says. “It’s really a step that should be bypassed, but it’s a learning curve.” Eventually, he says, they come to the realization that object detection is also a necessary aspect of the safety system.

The lower cost of camera and radar technology has benefited market acceptance, as well. According to Andersky, systems using both technologies weren’t feasible 15 years ago because they were cost prohibitive. However, now they have become more affordable and provide improved processing power, which is particularly important given these systems need to make millions of decisions at a time in micro milliseconds.

“There’s a lot more processing power available to us in a more compact package than there was before,” Andersky indicates. “We not only have the ECU on the vehicle, but can also have small ECUs in the radar and camera that can also process information.”

Smarter Systems Yet to Come

As market acceptance of these technologies continues to grow, so will their capabilities. Hessing says a combination of technologies will most likely be the winner in the long run. In addition to radar and cameras, the incorporation of WiFi location, dispatch and logistics will help create a safer jobsite. He also notes the customization aspect of technologies, including radar, will need to continue to become more intelligent about what they’re detecting and not detecting.

“There are algorithms we’re working on now to help ignore things [operators are] not worried about, as well as better algorithms for what they’re viewing,” says Hessing. “The base technology is evolving. We are getting better at recognizing objects in terms of more granularity, more refinement, so that you can better manage your zone of interest.

“Where it’s going to go is you will have more capability in the base sensor technology, so you can define those areas better and have better algorithm development to be sure you’re detecting what really matters and ignoring what doesn’t,” he adds.

Andersky believes more sensors, such as ultrasonic or shorter range radar, will be implemented as camera and radar systems progress. Moving toward one safety system that includes various safety features, such as lane departure warning or collision mitigation, as opposed to multiple systems will also be the way of the future. These will all aid the further move toward intelligent transportation systems, or vehicle to vehicle communications, and eventually play a role in autonomous vehicle operation.

“Additional information coming into the system is going to be important,” he adds.

Bringing in the additional information could allow system designers to begin automating corrective actions should the system detect a problem, similar to Wingman Fusion’s stationary vehicle capability. “Right now, we warn the driver when he’s going out of the lane,” says Andersky. “If we add some steering control in the future, we would be able to maintain the lane for the driver until he’s ready to make whatever maneuver he wants.

“When we look at the pathway to getting to autonomous vehicles in the future, camera and radar technology are important components,” he continues. “An autonomous vehicle has to be able to see what’s going on around it, and the radar and camera provide different approaches on vision.”