October 2013, Vol. 68 No. 10
Features
‘Augmented Reality’ Peers Below The Surface: Technology Could Revolutionize Underground Engineering, Design, Construction
Imagine a construction project in a well-developed part of a city where a directional drilling crew is preparing to install a section of gas pipe in an easement that already contains multiple utilities.
The foreman holds a tablet computer, looking at the screen which displays a real-time view of the work site as it is at that instant – streets, sidewalks, landscaping, trees and buildings as seen by the tablet’s built-in camera.
But wait, look closer.
Not only does the image on the screen show the live scene, it clearly depicts what is beneath the ground with images of color-coded pipes and cables, ducts, manholes – and showing their precise locations in relation to the real-time picture of the surface.
At a time when most utility locates are made by handheld electromagnetic tools and marked on the surface with small colored flags and spray paint, the scene just described might be considered a preview of the future.
Actually, technologies are in place to do exactly what was described and indeed, they have been successfully used on several projects.
Clearly it will be a while before combined surface/subsurface images are commonplace, but there are those in the industry who believe ultimately these technologies will change the way underground utility projects are approached.
Mark Wallbom is among those certain this will happen, perhaps sooner than many expect.
“Combining real-time video images with previously-mapped subsurface features is accomplished with augmented reality, also known as enhanced reality,” said Wallbom, chief executive officer of Underground Imaging Technologies, a geophysical services company that specializes in three-dimensional imaging of the subsurface to accurately map underground utilities and other objects prior to excavation.
“Augmented reality has become a popular topic for discussion,” Wallbom continued. “But it’s actually not new and most of us have experienced it without realizing it. On televised football games, the yellow and blue lines representing the line of scrimmage and first down marker, are superimposed over the actual view of the football field seen through the TV camera and ‘augmented’ to the picture to add helpful reference points.”
Jobsite application
Wallbom explains how the technology works on a construction site.
“The tablet or smart phone provides a live image of the location,” he said. “Because tablets and smart phones employ a built-in GPS, compass and an inclinometer, the device knows its exact location and orientation, depending on what angle it is being held. A Wi-Fi enabled device can stream real-time images to the cloud and, in return, receive information back from the cloud.”
By utilizing a service provider with a database that contains the digital subsurface dataset of the work site that was previously mapped using various geophysical tools and survey grade positioning instruments, the returned three-dimensional images are incorporated into the smart device and the images are combined and displayed on its screen. To relate to the real-time view from the camera, the underground map must be in a digital format with survey grade control points imbedded in the database.
“Within the boundary of the survey,” said Wallbom, “it is possible to point the device at, for example, a fire hydrant, and see an augmented representation of the underground pipe that serves the hydrant.
“As the person holding the device moves about the site, so does the image of the subsurface. In addition to the hydrant’s pipe, the image also would show other pipes, cables and anything else that was captured in the 3D subsurface survey providing an enhanced view of reality.”
Being able to see the relationships between above ground structures and below ground infrastructure clearly offers multiple advantages to engineers and planners, contractors and property owners.
A recent project in Las Vegas illustrates the potential of combining various technologies to achieve augmented reality.
VTN, a full-service engineering, planning and surveying company, based in Las Vegas, completed a subsurface utility excavation (SUE) project for the city of Las Vegas that produced a set of 3D models of above ground structures and underground infrastructure of a 1.5-mile corridor of Main Street in front of the new City Hall in downtown Las Vegas. The models were designed to run with software that Las Vegas already was using.
A variety of techniques were used to capture all of the underground utilities and other underground infrastructure, including GIS, survey, design records, test holes, traditional pipe and cable locators and ground penetrating radar (GPR). Above-ground images were produced by combining GIS data with mobile laser scanning.
Example
Wallbom suggests viewing the VTN website to see firsthand a video made in Las Vegas.
The video depicts the surface features as well as valve boxes, elbows and other color coded underground objects.
“This level of detail is atypical,” Wallbom said. “Normally, what is seen would be polylines as on a CAD drawing or blueprint. There are software programs in use today that allow the operator to add sizing to these lines as well as different color coding, and there is an image library with different pipe fittings, valves, manholes, etc. In this case, the polyline 3D dataset is enhanced or augmented by adding elements to the view that creates a lifelike ‘feel’ to the data.”
When fully implemented, engineers will be able to call up an augmented reality “map” from the cloud that is coordinated with existing subsurface data, click on a culvert, and bring up all information about that culvert. Staff can then update or modify information in the field, essentially creating an as-built on the fly.
Recently, Michael Kennerly, P.E., director of the Office of Design, Iowa Department of Transportation, said that having a mobile device like an iPad that could be taken out into the field and pointed in any direction and see in real time the overlaying of the subsurface infrastructure in the correct prospective and orientation to structures above ground would be of “enormous assistance.” Kennerly went on to say that “. . . once these 3D visual platforms become available it will open up a seemingly endless number of possibilities.”
Such combined surface/subsurface images and the attending digital database from which they are extracted can be used both by the design engineer in the office to plan new installations in a fully virtual 3D CAD environment. Then, the installation contractor out in the field can use this information on his tablet or phone during the excavation to more safely avoid existing utilities.
What’s next?
Wallbom said many organizations are contributing to the development of augmented reality for construction and other applications.
“Companies like Trimble are way down the road looking for ways to leverage their suite of complementary technologies,” he explained. “Both smaller boutique software companies like JBKnowledge Technologies and large firms like Bentley and Autodesk, engineering firms like VTN, and service companies like Guardian Prostar are charging ahead as they individually envision different possibilities.”
Positioning and achieving the correct perspective of all features is a huge challenge for software designers. The international research and best practices organization, Fiatech, published an in-depth report in April of this year that took a critical look at augmented reality, its viability, its business case and barriers to wide scale adoption. Their findings support the assertion that augmented reality will create a paradigm shift in the engineering and construction industries but not until more robust software integration is developed.
Wallbom stressed that data fusion is not a trivial matter, but man has been to the moon and back and can land things on rocks out in the cosmos.
“I’m sure,” he said, “that as really smart people begin to apply their skills on all of the related issues such as object recognition, perspective issues and data capture and delivery methods and means are addressed, the faster this paradigm shift will take place.”
UIT’s role is providing underground infrastructure data. “We will be the people who ‘feed’ the beasts with one ‘beast’ being GIS programs that have an insatiable need for more and more data, and the other ‘beast’ being people who see this technology in action and say to themselves: ‘I have to have one of those,’ ” Wallbom said. “Once engineers and contractors get used to using smart devices to actually visualize the layout of a new project, the world as we know it will never be the same.”
Wallbom said he has no doubt that augmented reality and the second tier technologies that are supportive of it will create a paradigm on the order of what the PC was to mainframe computers.
“The boat has sailed,” he concluded. “The train has left the station and if you want a seat on either, it’s best to get on board quickly before it is out of sight.”
FOR MORE INFO:
Underground Imaging Technologies, (407) 271-8911, http://uit-systems.com
VTN, (702) 873-7550, www.vtnnv.com
Making A 3D Utility Map
How are various elements consolidated to provide a virtual 3D map showing buildings, streets, and underground infrastructure and objects?
Underground Imaging Technologies CEO Mark Wallbom explains. “Everything has to have a starting point followed by milestones along the way. In the case of surveying, geodetic survey markers are used to create other control points that are in turn used by survey crews to establish an exact physical location of an object at a given elevation.
“Above ground objects are surveyed from these control points using highly accurate survey equipment that employ laser technology or satellite-based positioning that is capable of making measurements that are accurate to within centimeters and elevations changes to within 1/100 of a foot. LIDAR is another widely-used technology that can be used to create near survey grade representations of above ground structures as if wire mesh were carefully molded over everything being measured.”
From this framework, other layers can be added (augmented and enhanced) to provide color and texture and shading to levels of detail that resemble photos. Because many laser scanners today have built-in cameras, draping actual site photos over the point cloud and resulting mesh to create a photorealistic 3D model is relatively easy.
“The data that is being measured comes from the point cloud and resulting site model that in turn can be used to measure to another point of interest or milestone,” Wallbom continued. “The same thing works for below ground because we capture the exact ‘X’ and ‘Y’ [latitude and longitude] location of every square inch of the area being surveyed. Using various geophysical instruments, we capture the ‘Z’ dimension [depth] that is tied to the X and Y locations, providing a 3D dataset that can be looked at from any angle. If I know that some point of reference such as a telephone pole or a building or fire hydrant is located so far from one control point and so far from another, then through triangulation I can know where everything else is in relationship to where I am.
“Two disparate datasets need to be combined: the first being the above ground dataset that came into being through a number of different sources such as Google Maps, photogrammetry technologies from companies like Earthmine and Pictometry or previously completed LIDAR scans done by others which are housed in a GIS database. The second dataset comes from a company like UIT who is capable of providing survey grade 3D digital models of the subsurface that incorporate the same control points as those used for the above ground dataset. Through the magic of the computer, these two separate datasets are merged or in technical terms fused together, so that images are being augmented on the screen change depending on how the operator approaches a given subject and where he/she points the camera lens on the smart device,” he concluded.
Comments