ar-based real time 3d geospatial data vizualization1

AR-Based Real Time 3D Geospatial Data Visualization:

Above and Below Mapping Programs

2

AR-Based Real Time 3D Geospatial Data Visualization:Above and Below Mapping Programs

2

AbstractThe use of Remote sensing platforms have been on the rise in the past few decades with the advent of

maturing geospatial technologies such as Lidar and Digital Imaging systems. However, the technology to capture the above ground real world environment is relatively easy in comparison to the challenges we face when mapping underground characteristics. In fact, no one subsurface mapping device can scan the surface of the earth and provide a recognizable image to interpret the underworld and no one software program can generate an image that can be interpreted into a viewable form on a mobile device. While mobile, terrestrial, and airborne Lidar systems have expedited data acquisition of the aboveground environment, underground data and mapping is still in its infancy. Subsurface Utility Mapping consultants use an array of technologies such as Ground Penetrating Radar, Infra Red technologies,

Electromagnetic Induction Systems, etc., all working in unison to create a two dimensional picture of the underground. In some instances, there are 3D data acquisition systems designed to accept the 3D data, merge it with a Lidar point cloud or AutoCAD file, and export into a single device for viewing. However, these are just designs, with no firm date of deployment. A number of organizations around the world have a vision to utilize commercially available technologies for the purpose of building a Multi-Sensor Platform to quickly capture data in the real world environment and display the data in a field-deployable Augmented Reality system. The intent is to enable decision makers, designers, and constructors to capture and visualize the real world in precise detail, above and below ground in a user-friendly manner, like what a Google Underground Mapping system would offer.

Figure 1 - Lidar-GPR System

Figure 2 - Fusion of Imagery & Lidar Point Cloud

IntroductionAs populations increase, demands on existing infrastructures increase, to the point of unsustainability. Infact, the infrastructure in many populated cities around the world is in decay. For example, the average gas main in New York City is 56 years old (Chart 1, below). A NYC City Planner even stated, “In some cases, the infrastructure in New York is so old we don’t even know where it is under the street ... there can be a water main break in lower Manhattan and our engineers won’t be able to find it.”1

Facilities built 50-100 years ago cannot handle new demands. The recent global economic hardship has only exacerbated the challenges as national, state and local governments, and municipalities worldwide slashed infrastructure funding, while existing infrastructure continues to decay and fail. Replacement facilities cannot be funded, constructed fast enough, or even in a non-intrusive manner. Even when properly funded and planned, updating gas pipeline systems in a large

Figure 3 - Boston Infrastructure

3


3

metropolitan area like New York City would take decades. Natural disasters and a changing climate highlights the fragility of our infrastructure, witness the infrastructure failures in New York and New Jersey in the aftermath of Hurricane Sandy.

They only illustrate how quickly a major metropolis can fail in less than 24 hours. According to the U.S. Dept. of Energy, it estimates that Sandy (and other weather events) cost $27-$52 billion, from power outages.3

Chart 1: “Average Age of New York City Infrastructure in Years”2

Most parts of the world today have been built up in unstructured, unplanned, and dynamic environments indicative of urban sprawl. To manage growth, government, state agencies, city planners, and decision makers need a plan on how to build new facilities while continuing to maintain the existing infrastructure, much of which is in use long after the end of its intended design life. Having factual, precise, detailed three dimensional data is the only way decision makers can diagnose problems, and make informed decisions on planning new systems and replacing inadequate ones. Transportation, water, waste water, electrical systems, and communication networks must adequately address the day-to-day demands for all customers.

However, today’s social and economic expectations are that the public will not tolerate service interruptions, even during infrastructure rebuilding, replacement, and updating. Precise planning and design of successful projects requires an intimate knowledge of existing systems. However, incomplete and/or inaccurate infrastructure records are often attributed as the leading cause of failure for project teams, and happens all to often. Moreover, poor project planning and execution results in project interruptions, accidents (Figure 4, left4) and cost overruns. Good for headlines, bad for service. To mitigate these failures, or avoid future ones, key decision makers have sought solutions to reduce uncertainty, and increase the probability of success (saving taxpayers countless millions, instead of costing them).

The past decade has seen increased focus on the lack of accurate real world data by lawmakers and government officials. Today, US MAP 21 is a result of the U.S. Congress demanding better geospatial data on all aspects of state and local government decision-making. In Australia, the government’s Information Technology Committee (ITC) is seeking geospatial and digital solutions for improved infrastructure project delivery methods and technological advancement in major works, which better serve the public. Additionally, in many infrastructure-related accidents, the inability of key site supervisors to access the project files on-site has contributed to the inability of key personnel

to maintain project safety for the workers or the general public. For example, a gas line explosion in Walnut

Figure 4 - New York Steam Pipe Explosion

4


4

Creek, Calif. in 2004 claimed the lives of five workers, injured several others, and cost the energy company nearly $100 million5. The California Fire Marshall attributed it to a “Utility Location Problem” that an accurate, deployable visualization device would have prevented. The Common Ground Alliance recently noted that 273,599 “occurrences of downtime, damages, and near misses” were reported in the U.S. in 2014, resulting in an estimated $800 BILLION in costs (see chart, right). They further note that 17% ($136 billion) of these are caused by “Locating Practices NOT Sufficient”.6

Tens of thousands of lives are also lost on construction projects around the world, mostly on-site workers, though nearby innocent bystanders are also at risk. The U.S. Occupational Safety & Health Administration (OSHA) reports that, “The fatality rate for excavation work is 112% higher than the rate for general construction.”8 Their “Incident rate and number of nonfatal occupational injuries by industry and ownership, 2013” table shows that there were 26,400 such incidents that year in heavy and civil engineering construction.9 It is not uncommon for a civil and/or criminal suit to be filed against the contractor in these instances. Notwithstanding, many engineers and contractors violate U.S. Federal Labor law10 out of ignorance, unaware of the responsibility to adequately protect workers from underground hazards and specifically utilities, until it is too late.

Chart 2: “Estimated number of total underground utility damages resulting from excavation (U.S)”7

BackgroundIn recent years the Geographic Information Systems (GIS) and CAD platforms used by all infrastructure professionals have followed the trend to find ways for project visualization of infrastructure using mobile devices and tablets on site. Civil and infrastructure professionals have been looking to break the umbilical cord from the desktop CAD systems and become empowered on job sites with all the plans and files for Civil and Architectural projects. However, due to the massive size of the data sets, and the memory and processing power required to handle said data, CAD and GIS does not easily lend itself to a mobile system. In addition, suitable laptops have certain flaws: they are up to 2x as expensive as desktops; CAD & GIS apps run $7k-$10k each - too much to invest into a fragile job site laptop; laptops aren’t environmentally suitable for job sites, where it’s either too hot, cold or wet (environmentally safe laptops are extremely rare - and expensive); they often overheat; and, power constraints mean that coffee shops need to be available (and time-consuming) to recharge batteries.

While mobile device apps have access to large digital file storage through smartphones and tablets, it is not a real interactive solution. CAD, pdf, and jpegs can only be viewed. Another issue is that geo-referenced files are exceedingly rare. On-site users do not get to walk around and have the files update in real time while walking through the site. Today, hard copy paper plans are still the norm on all building sites. While reliable, they are typically scaled - which is open to interpretation, introducing uncertainty in precise measuring of distances. Susceptibility to dirt, the elements, and being torn on the job site renders large-format plans nearly useless.

However, new computing capabilities have introduced Figure 5 - Using an iPad in the Field

5


5

three dimensional designs into mainstream civil design workflows. The Building Information Modeling (BIM) revolution has become the Holy Grail for architectural and civil designers and building new projects in 3, 4 and 5 dimensions is the desired delivery method for all smart projects (encompassing estimating, pricing, delivery, scheduling and managing the project). The United States, United Kingdom, India, China, and some European countries are developing new construction standards which address the uniformity of BIM specifications and built their respective industry Best Practices and Project Guide books to design, build, and maintain projects in 3, 4, and 5 dimensional BIM environments.

Existing Best Practices and Geospatial Industry ShortcomingsTablets appear to be the cutting edge of on-site data interpretation. However, as stated above, paper documents and large clumsy plans are the norm. This is outdated, and it is impractical. 2D plans, documents, and bible-sized text documents do not adequately properly communicate site details and a majority of on-site stakeholders never see them. In fact, most on-site workers lack a higher education and may even be either illiterate, cannot speak the prevalent language, or just articulate (or even understand) when potential catastrophes are imminent. A recent event, in which a construction manager instructed workers in a pit to get out, they didn’t understand, and a fatality ensued. “By the time the pit reached about 13 feet, two hours later, Mr. Prestia told the workers, in English, to get out. But the workers, who spoke mainly Spanish, did not. Not long afterward, the walls collapsed, crushing Mr. Moncayo.”11

In addition, paper documents do not communicate project safety concerns, important details, and other contextual information for a true project understanding. There are no real industry programs available to solve these issues and share all the relevant project details in 3D so all project stakeholders can allow them to make their own informed decisions. Again, all project sites such as buildings, hospitals, schools, universities, manufacturing plants, air and seaports, etc. have available is a handful of invisible decision makers, alone possessing the only full set of project key details. Vital information is not available to those who need to make split second decisions in the field - with information that can often save lives, improve productivity, minimize errors, assist with cost and time schedules, provide clash detection, and also improve quality is unavailable to those who most need it.

Figure 6 - Paper Plans Still Used

A Wearable Device for All Site ProfessionalsOver the past two decades, the role of a dedicated utility mapping professional has emerged as an instrumental team member dedicated to mapping underground site conditions. Land surveyors the world over have greatly benefited from the falling cost of Lidar and photogrammetry equipment. Combined with SUM professionals, there is an abundance of well-qualified real-world mappers. The new challenge is how to build a mobile system that can share three dimensional topside and subsurface maps and datasets with all office and field staff. Figure 7 - 3D Model for Design Planning and

Visualization

6


6

However, it is not realistic to have all site professionals equipped with laptops and CAD software, nor is it financially feasible to train all staff in the use of CAD and GIS. What is financially feasible is the ability to export the 3D datasets to a wearable device which can be streamed for just the area of the project works. Localized projects represents 99.9% of all civil and architectural projects and in all cases there is the ability to insert geocoding and georeferencing devices into the project environment. This will ensure the device is working in sync with the user and positioned correctly to optimize the system. In fact, limiting the data with geo-referenced boundaries is a very prudent way to manage a project.

What is further needed is the mobile device itself. It must first be robust enough to handle on site conditions: protected from dirt, rain and impact; offer sufficient processing, imaging and battery power; be able to operate within wide temperature ranges (provide sufficient cooling); and, offer easily interpreted images of above- and below-ground characteristics, that anyone on site can utilize (with minimal training - they have to know what they’re looking at). Utilizing 3D data acquisition systems, the data of which is merged with the requisite Lidar point cloud or AutoCAD file, which is then exported to a single device for viewing in bite-sized chunks, an alternative is brewing.

Utilizing a user-friendly field-deployable Augmented Reality system on the job site is coming. Soon, all site professionals will have data streamed to glass on a job site with critical safety and work parameters live and available with the tap of a function

key. Eventually, these devices will also be able to stream and record site conditions, monitor locations, and identify safety boundaries for safe work practices. Added benefits will be the ability to record the work to ensure compliance, record best work practices of trades on site, document the build- and fit-out of projects for BIM, improve quality assurance and quality control monitoring, and save thousands of lives on-site and those who unwittingly get in harms way. This wearable device will enhance the day-to-day operations of project owners, Civil Designers, architects, mechanical, electrical, and environmental professionals. In addition, it will also have applications in situational awareness-intensive environments experienced by fire and rescue, law enforcement, and military professionals.

Figure 8 - Augmented Reality HUD Concept

Figure 9 - Image to be Super-Imposed Onto Augmented Reality Visor

7


7

Conclusion

Contact Information

The use of Lidar and Digital Imaging systems have been in use for years. However, mapping underground characteristics, and presenting said characteristics in a manner advantageous to job site conditions, is elusive. What is needed is a field-deployable, untethered device that utilizes software to generate subsurface mapping (and surface imaging) in a viewable form. Utilizing commercially available technologies to build a Multi-Sensor Platform to quickly capture data in the real world environment and display the data in a field-deployable Augmented Reality system is coming. The goal of said system is to increase safety on the job site (especially during excavation), by providing those on-site with the ability to “see” where they are digging with much higher fidelity and confidence than what’s offered today. Join us in our march towards this reality.

Christopher [email protected]

Michael Twohig [email protected]

Rupert MeghnotAdvisor [email protected]

SymbioVR, Inc.3406 Bishop Park Drive, #423Winter Park, FL 32792

1 Caution Ahead - Overdue Investments for New York’s Aging Infrastructure, Center for an Urban Future, www.nycfuture.org, MARCH 2014

2 ibid

3 http://www.bloomberg.com/news/2014-02-11/con-edison-to-calculate-benefits-of-preparing-for-climate-change.html

4 http://www.dailymail.co.uk/news/article-469413/Explosion-tears-New-York-replay-September-11-panic.html

5 http://www.sfgate.com/bayarea/article/Energy-firm-convicted-in-Walnut-Creek-pipeline-2539356.php

6 DIRT - Damage Information Reporting Tool, Analysis & Recommendations, Volume 11, ©2015 Common Ground Alliance, https://www.cga-dirt.com/ annual/2014/2014_DIRT_final_8-10.pdf

7 ibid, pg 3

8 https://www.osha.gov/SLTC/etools/construction/trenching/mainpage.html

9 http://www.bls.gov/iif/oshwc/osh/os/ostb3966.pdf

10 29 Code of Federal Regulations, subpart B, 1926.651(b) (1), (2) & (3) - https://www.osha.gov/pls/oshaweb/owadisp.show_document? p_table=standards&p_id=10775

11 http://www.nytimes.com/2015/08/06/nyregion/construction-managers-to-face-manslaughter-charges-in-death-of-queens-worker.html

ar-based real time 3d geospatial data vizualization1

Documents