Augmented Reality: Pending Revolution Already Changing Construction

Augmented and virtual reality are already changing the way contractors sell and manage projects even though much of the headset hardware capable of extending the technology to project sites is months away from retail stores.

The augmented-reality (AR) industry says businesses that depend heavily on field service could save, collectively, as much as $1 billion every year with Google-Glass-like abilities.

The advent of wearable computers that can take augmented and virtual reality mobile may not be quite as momentous as the first PCs or the internet, but it does combine those two breakthroughs in a way that opens the door to communicating with holograms (as in, “Help us Obi Wan. You’re our only hope”). Being able to generate and manipulate three-dimensional building information models (BIM) is already proving immensely valuable in architects’ studios and boardrooms, regulatory agencies and project sites.

Enough sales pitch. Why now?

Building information modeling is taking over design, engineering, planning, quality control and project delivery with its ability to illustrate construction projects in 3D. Few people are good at visualizing the result of 2D plans in 3D. Building information models show them, and the technology is helping slash construction rework costs, conservatively estimated at $15 billion per year in industrial construction.

“In a recent survey we conducted of over 2,000 construction professionals, we found that 65% of the companies are using building information modeling (BIM) software,” writes James Benham, CEO of construction-technology firm JBKnowledge Inc., in a recent blog post. “I’m highlighting this statistic because it reveals the biggest opportunity for wearables [computers] in the construction industry. What better way to use interactive, hands-free technology than to experience three-dimensional structures before, during and after they are being built? BIM allows for that on a flat screen, while wearables could go 10 steps past current BIM solutions by putting the user inside the model.”

Benham illustrates the power of BIM with a story, recalling a high-profile occasion when poor human visualization ate a contractor’s profits on a Las Vegas mega casino. 

In one vast concrete pour, the contractor formed the building’s foundation and swimming complex, then realized the pool cut between the materials staging area and the building foundation. The contractor spent millions building a temporary bridge over the pool to keep materials flowing to the tower.

“The GC paid a huge price because they didn’t simulate the schedule in four dimensions, they only looked at the schedule chart,” says Benham.

Building information modeling turbocharges people’s powers of visualization by adding a fourth dimension – time – to its illustrations. Scrolling through a BIM file along the timeline, you can watch how the project was intended to be built virtually, and stop to see what it is supposed to look like at any point in the process.

Augmented reality is a computer display technique that aligns data with what the AR user is looking at using the wearable device’s internet connection, built-in cameras, GPS and accelerometer. Augmented-reality glasses line up the images in a project’s files with their camera’s field of view and project it onto the lens-sized transparent screen the user is looking through. Three-dimensional images match the scale and perspective of the site from the location of the user’s eyes.

So when you scroll through a project timeline while looking at the actual site through AR glasses, you can stop the process simulation at the point in the job lifecycle when you’re actually looking at the site. Your view is overlayed with what engineers expect the project to look like today. Differences in degree of completion or execution stand out like sore thumbs, compared to looking down at plans and then looking up at the work as built.

Scroll forward in the model, and you can see if anything that’s being done today will interfere with the next steps in the process.

“In the AEC (architecture/engineering/construction) world, AR is part of the natural progression of extending BIM from the line office to the job trailer and the project space,” says Dace Campbell, customer success manager at BIM software provider Autodesk. “Augmented reality takes the BIM from the architect and puts it in the hands of people swinging hammers and making the final decisions on how a finished project is going to look. Building information modeling increases the effectiveness of communication, comprehension and confidence.”

Why Augmented Reality is Inevitable

Benham’s company gives away an augmented reality app for tablets that, using the tablet’s camera, recognizes the paper plans for a JBKnowledge client’s project and fetches a BIM model of that project stored on JBKnowledge servers. The screen displays a view of the model from the user’s perspective. SmartReality app users can move around and in the model to see in detail – from whatever point of view they choose – how the project is to be constructed.

The relatively basic AR application is already changing the way contractors sell projects. Benham recalls how SmartReality put new spin on a contractor’s pitch to a school board for a $68 million contract.

“The school board members each got an iPad and they pointed them at the plan file and the 3D model came out of the plans,” says Benham. “They were able to manipulate it with their hands and view it from their own perspective. And they approved it. One of the school board members said it was the first time they voted on a project they really felt they understood.”

The AR revolution really starts rolling with wearable computers that users interact with hands-free, or nearly so, to get project data on site. For all of the media coverage of AR-capable wearable computers, though, the feature list on those currently available on the market is short, and their performance is not yet up to the demands of interacting with 4D construction models.

“They all need faster hardware, better optics, faster processors, better graphics cards,” says Benham. “That’s what Hololens is working on with Microsoft. That’s what Daqri is working on with their Smart Helmet. That’s what Magic Leap is working on.

“The next leap that we’re all waiting for is the right viewing hardware.”

The who’s who of hardware companies scrambling to develop AR hardware suggests imminent breakthrough.

Facebook made waves with its purchase of the Oculus VR studio (for $2 billion) more than a year ago. Google has invested heavily in next-generation AR/VR studio Magic Leap whose technology is no less boggling than projecting digital information through your cornea onto your retina wall. Apple and Microsoft both have divisions dedicated to AR and VR, with Microsoft unveiling pre-production Hololens AR glasses, integrated with the company’s Windows 10 operating system released in July.

JBKnowledge has a version of SmartReality that runs on Samsung’s Gear VR, simple VR goggles powered by a Samsung Galaxy Note smart phone clipped into the headset. Epson’s Moverio BT-200 glasses are on the market too. And Sony has issued the software development kit for its SmartEyeglass.

ODG R-7 Smart Glasses

Today’s most-promising wearable tech is currently in prototype or beta stage. Designs generally start with adapting smartphone and tablet technology to wearable forms; glasses primarily. But the positioning capacity must be far more accurate than a phone. One of the most vexing hurdles, after solving the miracle of projecting and scaling the imagery on lens-sized screens, is souping up hardware and software to recognize gestures, but network connectivity and image quality are also pernicious.

Benham, as a software developer, has seen much of the wave of new devices expected to break in the coming 12 months, and he believes some have what it takes to bring real value on project sites. Once the hardware companies have issued software development kits, software developers will take on the task of writing programs that integrate data streams, network with the devices, and allow the user to select information they want to augment their world.

Gaming and entertainment markets are so vast, and the medium so undeveloped that they’re hard to forecast. Their perceived boundlessness is providing the greatest pull for developers. But enough notable innovators and relentless opportunists are customizing products for what will likely be more-profitable industrial and other enterprise AR and VR applications that it seems likely there will be choices of devices to satisfy the most ambitious business users before 2017. Expect beefed-up processors and more and better sensors and cameras to locate the device and sense user movements with greater accuracy.

“Microsoft’s bill of materials (for Hololens) is about $200. They’re going after the consumer market; we’ve not constrained ourselves in that way,” says Andy Lowry, president of Daqri, a company whose Smart Helmet is designed for industrial use. “Our price point is more in the range of $10,000 per helmet.”

Daqri’s Intellitrack software applies Smart Helmet’s 360-degree array of cameras and industrial inertial measurement unit can map the environment the Smart Helmet sees in 3D so it can overlay digital information precisely, and provide the data necessary to quickly model as-built surroundings. Communications capability allows a user to contact off-site experts, show them what they’re working on using the helmet’s cameras, and confer with them about problems and solutions.

“Think of everything it could replace: digital cameras, computers, two-way communication systems, an infrared camera, a 3D mapping tool,” says Lowry. “The price isn’t too far afield from that of the equipment that the helmet could replace.”

Fleet Management Value

Daqri has already begun working with select software developers to build Smart Helmet applications, and one of the first projects is developing capabilities that could improve equipment management. The AR app supports operating and maintaining a bioreactor; equipment that cooks up advanced medical proteins.

If the operator reaches for the wrong chemical to add to the reactor, the Smart Helmet recognizes the error and flashes a warning. The helmet networks wirelessly with the bioreactor’s on-board computer, so the operator can access condition data and fault-code history.

The Smart Helmet’s infrared camera mode shows users electrical circuits that are cooler than the others to help identify those where no current is flowing. And the technician can pull up schematics, repair procedures and parts manuals that offer step-by-step instructions for diagnosing a problem, finding necessary parts and making the repair.

A technician servicing construction equipment in the field could use those capabilities to access any machine’s history of fault codes, condition-monitoring data, repair history, service manual, parts manual, and even call in off-site experts to get troubleshooting and repair advice.

“Now users can have all of these systems coming together in a network,” says Lowry, “And they finally have a wearable people tool that makes sense to be a kind of hub for collecting data, and linking it to create solutions from Step A to Step B to Step C to Step D and so on.”

Somebody will make the business case to integrate the software and networks necessary to create an AR app for construction equipment. Lowry expects they’ll be able to get started with the basics for $20,000 to $25,000, and implementing all of the Smart Helmet’s capabilities could run up to “a few hundred thousand dollars.”

Consider the fact that Trimble is working with Microsoft to integrate BIM data – where the return on investment in AR is virtually guaranteed – with Microsoft’s Hololens AR glasses (dev kits coming in 2016). Another Trimble division works with Caterpillar on the Vision Link network that brings equipment owners detailed machine productivity and status data.

[VIDEO]Trimble and Microsoft Integrate BIM with HoloLens Glasses to Bring Models to Construction Sites

If venture-capital markets remain as flush with cash as they are today, more players will apply AR to a market as large and desperate for mobile data to augment the decision-making interface between technician and machine. Most AR devices claim to be capable of the core features expected from Smart Helmet, although less-expensive wearables are unlikely to be able to produce as refined a user experience.

The coming generation of AR glasses and other wearable visualization hardware has changed the important unknowns. There’s no longer a question if augmented reality will be applied to construction. The key question is how soon contractors will be able to find competitive advantage in available off-the-shelf applications.