Thursday, October 29, 2020
Published: Jul 2020
Authors
  • Burns Foster
    • Burns Foster
    • Company: CARIS
  • Dan Krumiel
    • Dan Krumiel
Categories
  • Seafloor mapping and Cartography

Teledyne CARIS – A Voyage into AI

There is no doubt that sonar has revolutionized the way in which we are able to map the seafloor. This huge advancement, however came with one key drawback — the introduction of ‘noise’.

  • Figure 1.
  • Figure 2.
  • Figure 3.
  • Figure 4.
  • Figure 5.
  • Burns Foster
  • Dan Krumiel
  • Figure 1. A completed bathymetric survey assisted by AI to remove sonar noise. Image courtesy of Veris
  • Figure 2. Bathymetric data before (top) and after (bottom) filtering based on the sonar noise classifier. Image courtesy of Geoscience Australia
  • Figure 3. Bathymetric data before (left) and after (right) filtering based on the sonar noise classifier. Image courtesy of CSIRO
  • Figure 4. Bathymetric data before (top) and after (bottom) filtering based on the sonar noise classifier. Images courtesy of Veris
  • Figure 5. Land / water classification of LiDAR returns by human operator (top), by AI (middle) and a correlated image (bottom) for context.
  • Burns Foster Burns Foster is the R&D Projects Manager for New Product Initiatives with Teledyne CARIS, focusing on developing new products and services outside of CARIS’ core competencies. He previously spent five years as the Product Manager for HIPS and SIPS, CARIS’ flagship processing software suite.
  • Dan Krumiel Dan Krumiel has a vast range of international experience on hydrographic, land and aerial survey projects. Having worked over a number of disciplines for both manufacturers and consultants, he has gained exposure to a wide variety of technology and methodologies for generating geospatial data.

Why the noise?

There is no doubt that sonar has revolutionized the way in which we are able to map the seafloor. This huge advancement, however came with one key drawback — the introduction of ‘noise’.

Conventional sounding techniques involving a lead-line were quite simple and directly measured depth. With sonar, a sound pulse is propagated through the water column and the return is measured based on echo intensity. Distance is now calculated, with time being the actual unit of measurement. If the return echo is incorrectly identified, this will lead to a sounding being incorrectly computed. The common term for this in the hydrographic industry is ‘noise.’

Techniques of ‘noise cancelling’

Before surveys and final products can be exported, this noise needs to be removed to ensure only ‘real’ soundings remain. No matter how expensive the equipment being used is or how experienced the operator, some level of noise is inevitable.

Over the years most industry processing software has introduced many standard filters and processes to deal with this, including simple approaches like spike detection up to more complex algorithms like CUBE (Combined Uncertainty and Bathymetric Estimator). While useful, these tools have limitations and can’t be applied unilaterally, as they aren’t well suited across a broad range of noise patterns. Instead, a user must manually determine where and when each filter is appropriate to apply, which really hinders potential time savings in automating processing pipelines. For areas where these processes don’t work at all, the user must resort to manual editing — which is one of the most tedious tasks faced by a hydrographer.

Fortunately, recent advancements in Artificial Intelligence (AI) mean a generalized algorithm to identify and remove a broad range of noise patterns is now possible.

Training your AI

Development of the AI driven ‘Sonar Noise Classifier’ began in 2018 and Teledyne CARIS’ development team quickly discovered that training an AI to recognize and remove noise is a challenging problem. Within even the noisiest datasets, the number of noise points only makes up about 5% of the total dataset. In AI terminology this is an “imbalanced” dataset, and simply feeding massive quantities of this data into a learning algorithm (which is the standard approach) won’t really work with so few relative samples. The solution to this challenge involved afairly laborious process of hand-picking samples with a more balanced ratio of noise from a variety of public datasets and cleaning each one by hand.

With datasets prepped, the next step was to decide upon the best AI technique. After trialing both old and new methods, it was determined that a state-of-the-art approach using Convolutional Neural Networks would yield the best results. This specific type of architecture is inspired by the assembly of the visual cortex of the brain across most of the animal kingdom and has seen broad adoption in recent years to locate and identify items of interest in images and video. Sonar noise did introduce a particularly unique challenge in that contrary to the (2D) nature of images and video, soundings are (3D) in space. This has seen relatively little research compared to its 2D counterpart.

AI — turning the impossible to reality

Teledyne CARIS launched a new CARIS Mira AI platform and officially released the Sonar Noise Classifier at the end of January 2020. This marked the culmination of a long journey into AI for the product development team, part of which involved feedback from several beta testers.

One of these was Geoscience Australia, who have vast amounts of collected data, some of which is yet to be processed to a usable format. With much of this data not having uncertainty information, modern processing techniques such as CUBE are not available. Manually editing and cleaning this data would be a huge and cumbersome task. Fortunately, this is a mission that AI is well suited to.

To tackle this task, Geoscience Australia has started work on establishing an automated processing routine as a pathway towards a scalable solution for managing this backlog of data. In addition to helping with scalability, Teledyne CARIS’ sonar noise classifier also provides consistency for the processed datasets.

Using time wisely

Earlier this year, the Geophysical Survey and Mapping team at CSIRO Oceans & Atmosphere put the new tool to test on datasets collected with Kongsberg EM2040, EM710 and EM122 (and some older EM300) sonar systems in a variety of water depths, from very shallow (

In two independent cases, they were able to reduce manual editing time by 65%. One of these was for a transit line, which would have normally taken 25 minutes to edit with manual processing. This was reduced to 90 seconds of AI processing, plus 7 ½ minutes of checking and residual editing, resulting in a total time of 9 minutes. A shorter line was used for testing as it was feasible to spend time manually editing the whole dataset to get a baseline time. While this is a relatively short transit line, the 65% improvement shows promising time savings which will be magnified on larger lines or entire survey areas.

Bringing efficiency and cost savings to projects

Building on the trend of early adopters in Australia, another organization that has seen value in the sonar noise classifier tool is private survey company Veris. Having recently transitioned their multibeam workflow to HIPS and SIPS, Veris’ specialist hydrographic team are achieving even greater efficiency on their projects with the help of CARIS Mira AI.

The sonar noise classifier tool analyses every single sounding in a survey, assigning a percentage confidence as to whether the point is likely to be noise. While it isn’t feasible for a person to inspect every sounding, they can use this information to adjust filters and visualize the results, providing valuable insight into the decision-making process.

In addition to saving time spent on manual cleaning, Nathan Green from Veris also appreciates what is effectively a ‘second opinion’ on the data. The AI tool provides an independent check as to what a processor would have considered to be noise. For any questionable areas where data hasn’t been classified as noise, this captures the processor’s attention to look more thoroughly into that area and ultimately make a more diligent decision.

Continuing the voyage

The team at Teledyne CARIS have put great focus over the last few months in educating the hydrographic community on this groundbreaking new technology. Being international by nature and underpinned by standards, adoption of new processes and techniques can take time, especially in relation to surveys for nautical charting and safety of navigation.

Feedback received to date has been positive and the development team have already started looking at what’s next. Another feature currently in development is object detection with AI. Examining collected bathymetry and side scan imagery to locate features such as rocks and shipwrecks is critical and once again — an engineering task well suited to AI.

Collaboration with fellow Teledyne business units operating in other domains is providing further insight and learning into best practices for overall architecture on how to best leverage AI. A leading example for this can be seen in cross pollination with Teledyne Optech, a leading LiDAR manufacturer.

Teledyne Optech’s CZMIL sensor has a global user base and is being deployed to accurately map challenging environments where water interfaces with land. One of the first and foremost requirements for bathymetric LiDAR data processing is being able to differentiate between returns on land and in water. AI trials for this have vastly surpassed what a human operator capable of, with AI being able to analyze returns at a much finer level of detail than a person can interpret. This is demonstrated in the dataset shown in figure 5.

With this year’s theme for World Hydrography Day being ‘Hydrography enabling autonomous technologies,’ AI will play a pivotal role in realizing the level of autonomy that is required to efficiently map the Earth’s oceans.

Back to Special Reports and Whitepapers Library
Marine Technology ENews subscription Marine Technology ENews is the subsea industry's largest circulation and most authoritative ENews Service, delivered to your Email three times per week
Subscribe for MTR E-news
Figure 1
page 2

Using high resolution 3D meshes for improved shape reconstuction of marine survey data

When using dense, high-precision survey data, the method for management and visualization of the data can have a large impact on the final decision making process. This is an important factor when accurate shape reconstruction is required, as there are significant trade-offs with traditional approaches. For applications where it is critical to know exactly the shape and size of surveyed objects, a high-resolution 3D mesh is likely the best option.

Ultra-high-resolution, NORBIT iWBMSh-STX
page 16

A Surveyor’s Perspective

The National Society of Professional Surveyors has, as its first pledge of a professional land surveyor “To give the utmost of performance”. While this mantra provides a continued mentorship to the professional as they carry out their geo-scrutiny it also acts as a guiding principal for those of us behind the scenes that deliver the tools, training and support of technology.

;