John Deere's SmartDetect system combines cameras and machine learning to help operators improve situational awareness from inside the cab.
John Deere
It happens all too often to ignore. In June 2023, an earthworks employee showed up at a jobsite in Stoughton, Massachusetts, expecting fully to make it home in time for dinner that night.
Unfortunately, a track loader’s bucket was elevated on uneven ground, upending the vehicle and striking and pinning the worker between the ground and the bucket.
OSHA calls this a caught-in or -between accident. These occur “when a person or a part of their body is caught, squeezed, crushed, or otherwise compressed between two or more objects.”
Equipment, walls, stacked or stored materials, shoring, and other items are considered solid objects, with accidents resulting in broken bones, asphyxiation, or death.
OSHA standards require employers to protect workers from these hazards. Outfitting heavy machinery, equipment, and tools with enhanced object detection and visibility devices is one recommended remedy.
It is critical for contractors to not only develop and oversee, but also enforce a plan to protect their employees. As these employees inevitably make mistakes, the implementation of solutions to help prevent or correct these mistakes can help to mitigate potential outcomes. If not, the company and its employees are left exposed to the consequences of poor risk management.
Failing to protect workers cost businesses approximately $132.3 million in fines and penalties during 2023, when OSHA added 227 compliance officers to conduct an additional 2,000+ inspections on the year. Other consequences of unsafe operations include damaged equipment, project delays, and reputational damage, to name a few.
DEVELON's Transparent Bucket feeds video images captured by cameras outside the vehicle onto a display inside the cab. The images converge on screen to project a “ghosted,” real-time video of what’s going on beyond the bucket, in front of the machine.DEVELON
Applying Technology Makes Operating Heavy Equipment Safer
“There are many solutions available that provide obstacle detection capabilities for construction equipment,” said Seth Meisel, product manager for excavators and loading at Trimble, an industrial technologies company based in Colorado.
“These include camera systems that employ AI/ML technology for pedestrian detection, as well as radar systems that provide accurate distance estimation of obstacles.”
Cameras on heavy-duty equipment improve visibility around a machine. When paired with sensors and other detection devices, such as ground positioning radar (GPR, a technology that captures subsurface data), modern visibility solutions are able capture and provide consistent data to keep vehicles from colliding with people and objects and on a safe course.
DEVELON (owned by HD Hyundai), a heavy equipment manufacturer serving the construction, forestry, and agriculture industries, provides a unique solution meant to improve visibility on wheel loaders and other bucket-equipped machinery.
It’s called the Transparent Bucket, a breakthrough that feeds video images captured by cameras outside the vehicle onto a display inside the cab. The images converge on screen to project a “ghosted,” real-time video of what’s going on beyond the bucket, in front of the machine.
“Imagine you’re driving an 80,000-pound machine blindfolded, with the bucket obstructing your line-of-sight to anything directly in front of you,” said Bill Zak, Wheel Loader Product Manager at DEVELON. “Our Transparent Bucket technology lets you see virtually through the bucket at any objects in your path.”
DEVELON demonstrated the technology at AEM’s Celebration of Modern Agriculture on the National Mall two years ago. According to Zak, OSHA visitors wanted to know why the technology wasn’t standard on all heavy equipment.
“When you see it action, it’s an eye-opening experience,” said Zak. “It’s like the Klingon Cloaking Device on Star Trek. Now you see the bucket, now you don’t. It’s now a standard feature on every bucket-equipped machine we sell.”
The Technology Will Only Continue to Get Better
Cameras, sensors, and radar obstacle detection systems have evolved over the years in passenger vehicles.
Where rearview mirrors and horns were once basic awareness accessories, cameras and radar-infused systems have expanded the scope to deliver actionable information, either to an operator or to the machine’s control system itself.
With each new innovation, the heavy equipment sector has responded with great enthusiasm. Many of these systems now deliver data over the cloud to support back-office reporting and planning.
John Deere, a pioneer of intelligent, connected machines and applications serving the agriculture and construction industries, has developed a complete Smart Industrial strategy to address vision and object detection on the jobsite. The company's SmartDetect system was debuted in 2023 at CONEXPO-CON/AGG.
The system combines cameras and machine learning to help operators improve situational awareness from inside the cab. It signals operators when objects enter the machine’s pathway. Special detection alerts sound and flash when people enter the camera’s field of view. What’s more, an optional upgrade to SmartDetect with Assist can automatically stop a machine before it comes in contact with a person.
“One of our customers was testing the technology on a project near a college campus,” said Katie Voelliger, product marketing manager for John Deere.
“He said students would cut across the jobsite on their way to classes, not paying attention and usually looking at their phones as they walked. Within the first few hours on the job, the SmartDetect alarm went off over 20 times. It was such a game-changer for them, they concluded never again to work on a jobsite without SmartDetect.”
In loud environments, where operators cannot clearly hear backup alarms to react in time, blind spot monitoring adds visual alerts, including color-coded indicators to supplement audible alarms.
Other solutions incorporate pulsing alerts that physically warn the operator when an obstacle or person is in the way.
Increasing Control Leads to Safer Worksites
According to the Construction Safety Research Alliance’s (CSRA) High-Energy Controls Assessment (HECA), every construction company should prioritize the implementation of direct controls to mitigate the risk of high-energy hazards, especially when someone makes a mistake.
High-energy sources may include electrical, mechanical, hydraulic, pneumatic, chemical, thermal, and other sources that may unexpectedly release stored energy when workers interact with machines. HECA measures and monitors safety by computing high-energy risk against corresponding direct control. An on-site HECA assessment can be effective in identifying such hazards, illuminating areas where direct control can alleviate the risk.
“Trimble is supporting local distributors by putting them in touch with object detection equipment manufacturers who can meet their end customers’ needs,” said Meisel, who assists Colorado’s Department of Transportation on construction projects. “A visit with a knowledgeable dealer can answer a lot of questions and point the way to affordable, effective solutions regarding both new and existing heavy equipment.”
It Still Comes Down to People
Indeed, while back-up alarms, cameras, radar, ultrasonic sensors, mounted light assemblies, and radio frequency identification (RFID) systems all contribute significantly to safety, increasing behind-the-wheel training and operator knowledge play a large role in ensuring a safe worksite. Worker safety planning is therefore a recommended best practice across the industry. Every day should start with a site safety meeting.
Regular hazard assessments of the worksite, including traffic control plans accounting for high-risk areas where back-up maneuvers or tight operating spaces exist should be part of every project game plan.
With the assistance of technology solutions designed support their innate decision-making capabilities, operators can improve their responses to unsafe conditions and respond quickly to threats to their safety. It is this actionable data that offers the operator a more accurate, detailed view of their surroundings and the ability to pivot and react efficiently.
A Bird's-Eye View Improves Awareness
As well, having a digital operations center capability that collects real-time data on operators and equipment can help an organization learn faster and better plan for reduced jobsite risk.
“Our SmartDetect Digital functionality represents one way to remain consistently aware of what’s happening and inform decisions that fuel future improvements,” said John Deere’s Voelliger.
The system is programmed to push notifications of near misses to the operator and command center, supported by near-miss videos to supplement weekly reporting of technology usage and related events.
“A heat map of jobsite operations can help companies optimize operations on a daily basis, as well as analyze trends for future planning and ongoing improvement,” Voelliger added.
By incorporating such data into an ongoing risk-prevention strategy, companies can better understand their current state and plot a more deliberate journey toward improved situational awareness.
Jobsite Safety Is a Conscious Decision
Start by analyzing problem areas and focus on root causes. Are there areas of congestion that can be debottlenecked? Can operator training improve outcomes? What equipment needs upgrading, and are the onboard systems being properly used?
Assembling a safety team to analyze patterns and brainstorm solutions will go a long way toward getting the most out of object detection and visibility technology investments. Findings and conclusions should always be documented, to defend against “knowledge leak,” should key personnel leave the organization or retire.
Working with a proven safety technology solutions provider can also amplify results. Getting an experienced, objective outsider to weigh in on better ways to operate can delimit the possibilities for improvement and add a fresh perspective on creative ways to solve problems.
Technology Waits for No One
As the future unfolds, expect to see more innovations in the area of jobsite safety, not only in the areas of object detection and visibility, but also in areas such as fuel economy, extended battery life, hybrid fuel solutions, and the increased use of autonomous machinery.
Throughout the mining industry, for example, autonomous technologies are already on a path that allows unmanned machines to traverse dangerous environments via remote control. Perception technologies play a big part in the success of these remote-control deployments.
As cameras continue to get smaller and resolutions improve, remote robotics will become more practical, allowing remote machine operators to control construction activities with greater clarity and from greater distances away.
“Obstacle detection systems are already beginning to integrate with deeper levels of machine control, making it possible inhibit a machine from even moving,” said Meisel. “We’re already observing the paradigm shift, from passive visibility to active collision avoidance.”
Much to Gain
The benefits of combined object detection, visibility, and machine control are compelling.
As Voelliger put it, “Operators have the potential to get burned out when loaders are constantly shifting into reverse. Having to turn around in your seat and continuously swivel your head to look back creates fatigue, which makes for reduced productivity and increased risk of accidents.”
Zak notes two significant savings attached to advanced object detection and visibility in the cab.
“First, if you can keep your people safe and fresh throughout the day, you will experience less downtime, less rework, and get more work done. Second, by avoiding accidents, you protect your machinery equipment from damage and prolong the life of existing investments.”
Getting on board with situational awareness technology has never been easier. Most manufacturers are including it on their equipment already. Before you buy, be sure to inquire about your options with your dealer. And when you buy (or upgrade), make sure you know how to use it, train your people to use it, and hold your team accountable for making sure everyone makes it home in time for dinner.
3 Practices That Contribute to a Safe Worksite
1. Mark the site for safety using physical obstacles such as cones, danger tape, fencing, etc.
2. Monitor the site using dedicated monitoring alarms, proximity detection systems, spotters, etc.
3. Post visual reminders of high-energy hazards and other danger zones on equipment and around the site.
DEVELON's Transparent Bucket feeds video images captured by cameras outside the vehicle onto a display inside the cab. The images converge on screen to project a “ghosted,” real-time video of what’s going on beyond the bucket, in front of the machine.DEVELON
DEVELON's Transparent Bucket feeds video images captured by cameras outside the vehicle onto a display inside the cab. The images converge on screen to project a “ghosted,” real-time video of what’s going on beyond the bucket, in front of the machine.DEVELON
