We would like to invite you to attend the Ocean Networks Canada (ONC) & MERIDIAN workshop this fall (November 1st - 3rd, 2022)

Join Zoom Meeting

https://uvic.zoom.us/j/84340072841?pwd=ZlQzaGJ2NmRJZDhxLzdkODhmQ0Fkdz09

Meeting ID: 843 4007 2841

Password: 560772

One tap mobile

+17789072071,,84340072841#,,,,0#,,560772# Canada

+16475580588,,84340072841#,,,,0#,,560772# Canada

Dial by your location

+1 778 907 2071 Canada

+1 647 558 0588 Canada

Meeting ID: 843 4007 2841

Password: 560772

Find your local number: https://uvic.zoom.us/u/kdSUH0O5M1

Machine Learning Advances for Marine Acoustics and Imagery Data

The workshop will be held in webinar format with technical panel discussions over three days focusing on

1. Machine learning (ML) applications for underwater acoustic data,
2. ML applications for ocean imagery and aerial/satellite imagery data,
3. Collocated acoustic and imagery data for ocean science applications.

The workshop will investigate ML applications to ocean big data, in particular acoustics data and image/video data, explore areas of particular interest and relevance, and identify the critical next steps for implementing practical solutions and facilitating widespread application of ML techniques to ocean data.

Day one will address ML/deep learning for developing acoustic detection and classification models for underwater passive acoustic monitoring, day two will evaluate ML/deep learning for automating the analysis of underwater image and video data, as well as aerial and satellite imagery data. On day three we discuss collocated data sources, research problems that could be solved with ML tools, and collaboration opportunities among institutions.

Please contact the workshop planning team (rgehrmann@dal.ca) for any questions.

Workshop Schedule

Acoustic Analyses Session 1 - November 1st 8:30am-11:00am PDT

Detecting and Tracking whales with IR cameras and PAM

Harald Yurk (DFO)

Developing an autoencoder based binary classifier for orca calls

Val Viers (Orcasound)

A short review of signal preprocessing, autoencoder design and development as a feature detector for a convolutional neural net that predicts orca Call/No Call for real time hydrophone data.

Building an open source pipeline to separate Southern Resident Killer Whales (SRKWs) acoustics

Praful Mathur (Orcasound)

As we study SRKWs, we want to isolate their vocalizations from other background noises e.g. ships, water passing over hydrophone, etc. In our project, we developed a separator built on open source tech along with our own pre-processing sequence.

We're looking for help to use our separator to classify acoustics to the SRKW pod which can be fed into a separator such as BioCPPNet so we can extract individual tracks.

More than a whistle: Automating the detection of marine sources across diverse soundscapes with a convolutional neural network

Ellen White (University of Southampton)

The effective analysis of Passive Acoustic Monitoring (PAM) data has the potential to determine spatial and temporal variations in ecosystem health and species presence if automated detection and classification algorithms are capable of discrimination between marine species and the presence of anthropogenic and environmental noise. Extracting more than a single sound source or call type will enrich our understanding of the interaction between biological, anthropogenic and geophonic soundscape components in the marine environment. Advances in extracting ecologically valuable cues from the marine environment, embedded within the soundscape, are limited by the time required for manual analyses and the accuracy of existing algorithms when applied to large PAM datasets.

A deep learning model is trained for multi-class marine sound source detection to explore its utility for extracting sound sources for use in marine mammal conservation and ecosystem monitoring. Transfer learning is used to fine-tune an open-source state-of-the-art ‘small-scale’ convolutional neural network (CNN) to detect odontocete tonal and broadband call types and vessel noise (from 0 to 48 kHz). Confidence in model performance in diverse ambient noise conditions is critical of we are to rely on automated pipelines for information relevant to the management of marine protected areas and the conservation of vulnerable species. We evaluate the effect of ambient noise on detector performance, to understand the variability of the regional soundscape for which it will be deployed. We then extend this work by deploying the model on data collected in new soundscapes to understand the adaptability of broadband detectors across oceanic regions.

Adaptable, Open-source Deep Learning NARW Vocalization Detection Tool

Bruno Padovese (MERIDIAN); Fabio Frazao (MERIDIAN)

Detectors and classifiers based on deep neural networks and deep learning techniques have shown great potential in identifying marine mammal vocalizations from underwater acoustic recordings. While many studies have explored DL for marine mammal detection, few have provided the software tools necessary to reproduce and apply these techniques to different acoustic environments and datasets. We developed an open-source tool to facilitate the creation of deep learning-based acoustic detectors and classifiers for North Atlantic right whale (NARW) vocalizations. This tool include options to 1) create training and test datasets from raw acoustic data using different audio representations (i.e., spectrograms, cepstrogram, etc); 2) train a deep learning model to classify NARW vocalizations; 3) evaluate the model and report standard performance metrics; 4) run the model on a directory of audio files; and 5) adapt an existing model to a new acoustic environment through transfer learning.

Acoustic Analyses Session 2 - November 1st 11am-12:30pm PDT

Machine learning to find whales and submarines in a noisy ocean background: Challenges and opportunities

Dr. Carolyn M. Binder (DRDC)

Passive and active sonar are the primary modalities to detect, classify, localize, and track (DCLT) objects in the ocean. Defence Research and Development Canada (DRDC) is researching machine learning methods to automate DCLT of underwater acoustic problems; ranging from using passive sonar to note the presence of whales to reduce risk of harm during active sonar exercises, to using active sonar for detection and tracking of ships and submarines. In most scenarios, machine learning methods must be applied in such a way to minimize false positives resulting from the noisy ocean background. This talk will focus on identifying requirements for state-of-the-art acoustic DCLT methods, particularly topics best-suited to the application of machine learning. Challenges in applying machine learning to ocean acoustic DCLT will be identified with the intent to stimulate thought and discussion on how to address these challenges. There are many opportunities for machine learning to demonstrate innovation and development of creative solutions to complex ocean acoustic problems.

Letting Marine Mammal Vocalization Neural Networks Swim On Their Own

Mark Thomas (Dalhousie University); Dr. Bruce Martin (JASCO Applied Sciences); Nisarg Patel (JASCO Applied Sciences)

Neural networks (NNs) are known to perform very well as detectors for marine mammal (MM) calls on annotated datasets and transfer learning has been shown to allow a NN trained on one dataset to adapt to environmental noises in a different dataset. However, these techniques have not been widely employed in ‘production’ environments, whether for the analysis of large archived data sets or for use in autonomous real-time systems. Therefore, JASCO Applied Sciences, teamed with the Big Data Institute at Dalhousie, has begun a six-month project to operationalize our existing NN for detection of low-frequency cetaceans. The existing NN will first be re-factored so that it can run on large data sets and then it’s performance compared with JASCO’s existing contour based detectors on archived data from the Canada’s East Coast. Next, the NN will be re-trained to classify low-bandwidth contours rather than full spectrograms for use on autonomous systems. The new NN will then be ported to run on an embedded Tensor Processing Unit and the performance evaluated in terms of weight-space-and-power. Finally, we will design an active learning system to annotate noises and sounds from new environments to quickly adapt NN-based detectors to new platforms and noise environments.

From extensive data fusion to a knowledge-based system for vessel signature detection

Dr. Gabriel Spadon De Souza (Dalhousie University)

AIS data plays a significant role in tracking vessels around the globe, with applications ranging from national defense to regional traffic planning. While AIS messages are transmitted to satellite or terrestrial antennas, vessels propagate noise in the underwater environment that hydrophones can capture. The joining of these two different data can bring greater awareness about the ocean environment but also pose a challenge in the face of the size and complexity of dealing with heterogeneous data. Deep Learning techniques, named neural networks, have shown a remarkable capability of extracting meaningful performance over vasts amount of data applied to different applications. In this sense, this research contributes with a methodological framework alongside a neural network tool-set for joining such data to detect vessels based on their underwater signature, considering distance not only as a factor that impacts sound propagation but also as a feature for achieving increased performance during the neural network training process.

Fish-Focused Analyses - November 2nd 8:30am-11am PDT

Acoustic and Imagery Data at Ocean Networks Canada

Jasper Kanes (ONC)

Ocean Networks Canada (ONC) has been operating cabled underwater observatories collecting continuous oceanographic data and making them freely available for researchers to use for more than 10 years. With more than a petabyte of archived data and hundreds of instruments across their networks continuing to send data ashore in near-real-time, finding signals of interest in the archive has become a challenging task. ONC is helping researchers find data they’re interested in by offering a number of data display and access tools, producing curated data sets, and leveraging the interest of citizen scientists to help label an extensive archive of imagery data. But achieving the goal of making data searchable by labelling the entire archive will require machine learning.

ONC's Fish Acoustics Experiment

Dr. Fabio De Leo Cabrera (ONC)

Ocean Networks Canada (ONC) operates large seafloor cabled observatories in the Arctic and NE Pacific, with some of its long-term observations surpassing 15 years of archived data. The seafloor network of 850+ km of backbone cables connects > 50 instrumented sites (>400 oceanographic instruments, >5,000 sensors), in habitats ranging from temperate coastal fjords, ice-covered Arctic bays, to deep-sea canyons, abyssal plains and hydrothermal vents. We present results from in-situ experiments studying the benthic boundary layer in two contrasting ecosystems: at 150 m depth at the mouth of the Fraser River Delta, Strait of Georgia, and at 640 m depth in the upper slope offshore Vancouver Island, Pacific Canada. Broadly denominated ‘Fish Acoustics Experiment’ (FAE), instrument platforms combining video and acoustic imaging (dual-frequency identification sonars) and high-frequency hydrophones were deployed to better understand local ocean soundscapes, poorly understood fish vocalizations and overall temporal changes in benthic abundance and diversity. The second FAE iteration, deployed adjacent to Barkley Canyon, also included an autonomous bait-plume dispenser, allowing for the study of benthic successional patterns of deep-sea scavenger communities. Preliminary results from the Strait of Georgia showed a moderately diverse assemblage of fish and invertebrates with dominance of the spotted ratfish Hydrolagus colliei, rockfish Sebastes sp., and the shrimp Pandalus platycerus. Due to an extremely turbid environment, the acoustic camera proved to be the most efficient method for measuring total faunal abundance. In addition, light avoidance behaviour was detected in a large number of species while light attraction was verified for H. colliei. At the upper slope, in a 3.5-month experimental period with 21 independent bait releases, the Pacific hagfish Eptatretus stoutii and tanner crab Chionoecetes tanneri were the only species that truly responded to the carrion signal.

Coarse to fine detection: Using multiple methods of sablefish startle detection to efficiently search large underwater video datasets

Declan McIntosh

As institutions like Oceans Networks Canada, Monterey Bay Aquarium Institute, and others deploy increasing numbers of underwater cameras, more and more efficient large-scale data processing methods are required to use this data effectively. While we increase inference speed to allow for a wider net of data collection, we also want to ensure we create granular enough data that it is still useful. We present our work towards Sablefish startle detection from a coarse clip-wise end-to-end convolutional neural network detector to a more fine individual Sablefish track-wise startle detection. Our end-to-end detector processes video at more than 100 times real-time speed on a single low-end GPU node while maintaining strong recall. Meanwhile, our slower but more precise track-wise startle detector gives the exact location, time and individual Sablefish exhibiting the behaviour. We achieve faster inference while still generating low-level behaviour detection by first processing the expansive dataset with the fast end-to-end detector, then taking only positive detections and generating more fine-tuned predictions with the track-based method.

Collocated acoustic and video data to identifying fish sounds in the wild

Dr. Xavier Mouy (Integrated Statistics, Inc.; NOAA)

Some fish produce sounds for a variety of reasons, such as to find mates, defend their territory, or maintain cohesion within their group. These sounds could be used to non-intrusively detect the presence of fish and potentially to estimate fish density over large areas and long time periods. However, many fish sounds have not yet been associated to specific species, which limits the usefulness of this approach. In this presentation, Xavier will describe how collocated acoustic and video data can help address this knowledge gap and identify fish sounds in the wild. The presentation will introduce a dataset collected off Vancouver Island by three portable arrays composed of multiple hydrophones and video cameras. Xavier will describe the methodology currently in place to process these data and will discuss how machine learning could help make this analysis more efficient and help gain more insights on the acoustic behavior of fish.

Classification of Soniferous Fish Calls Using Ocean Acoustics

Dr. Farook Sattar (University of Victoria)

It is challenging to accurately classify soniferous fishes due to underwater noise interference. In this talk, we will present effective method(s) for the classification of soniferous fish calls. Interesting ideas and results will be shown for the proposed method(s) related to fish sounds monitoring using biological/bioacoustic signals.

Fish detection and classification using deep learning

Devi Ayyagari (Dalhousie University); Dr. Chris Whidden (Dalhousie University)

Deep learning models have shown great results at detecting and classifying objects in other domains like medical imaging and robotics, but deep learning is still underutilized in ocean applications such as analyzing underwater video. Working collaboratively with NRC and DFO, we aim to bridge this gap and develop deep learning methods and best practices designed for studying fish health and biodiversity. The goals of this research are twofold a) Detection and classification of different species of fish in the underwater video camera data gathered by DFO b) Quantification of the number of samples of different species required to train an efficient classification and detection model. In this talk I will present preliminary results from applying the widely used YOLOv4 object detection and classification model to six species of fish in the dataset. This work is a first step towards better understanding biodiversity and fish health and the impact of human activity such as seismic testing on fish.

Satellite Image Analyses - November 2nd 11am-12:30pm PDT

Space Whales and Arctic Marine Mammals: using satellite imagery for identifying marine mammals in the Arctic

Bryanna Sherbo (DFO)

Satellite imagery advances in resolution have allowed for direct detection and counting of animals. Commercially available satellites now have resolutions under 40 cm and can be used for monitoring marine mammals in remote locations. This study focuses on using satellite imagery to detect narwhal (Monodon monoceros), beluga whales (Delphinapterus leucas), and walrus (sp). In addition to manual identification of marine mammals, we have also explored methods such as autodetection and crowd counting. A crowd counting study was conducted in March 2021 with a total of 14,435 km2 of very high resolution imagery (30 cm) collected from 15 locations taken during the low or no ice season (July to September). A total of 11,282 features were detected and identified with the highest probability of being whales. Narwhal and beluga whale identification from satellite imagery comes with uncertainty regarding the depth of observed individuals, as such a ground truthing study where inanimate whales were deployed at discrete depths in a clear and murky waterbody to determine the depth they can be observed from satellite. There are certain factors that need to be considered when using satellite imagery for distribution and abundance of marine mammals including imagery acquisition, environmental variables (i.e. cloud cover, water clarity, wave action, sun glare, etc.), and reading large amounts of imagery can be time consuming. This presentation will discuss the use of satellite imagery for detecting marine mammals in the Canadian Arctic, including an overview of the benefits, drawbacks, and some preliminary data.

Identifying Algae Blooms in Near-Real Time Using High-Resolution Satellite Imagery

Catherine Evans (MERIDIAN)

Algae blooms are an increasing threat to fresh and saltwater ecosystems. Harmful algae blooms (HABs) have the potential to contaminate drinking water reservoirs and can threaten wildlife in affected areas through toxic emissions and development of an anoxic environment. HABs are predicted to increase in frequency and severity in future years due to a combination of deforestation, increased agricultural and residential development, and rising global temperatures.

Blooms that develop at or close to the water’s surface can be observed with optical satellite imagery. Cyanobacteria – or blue-green algae – can be observed through RGB satellite imagery and normalized band combinations that estimate surface chlorophyll content. By creating a time series of satellite images over water bodies of interest, our team aims to detect abnormal changes in surface chlorophyll content indicative of an algae bloom. We will present k-means clustering result to delineate water, algae and land on lakes in New Brunswick and Nova Scotia.

Arctic sea ice and open water classification from spaceborne fully polarimetric SAR

Yiru Lu (School of Marine Science, NUIST; Bedford Institute of Oceanography); Dr. Biao Zhang (School of Marine Science, NUIST; Bedford Institute of Oceanography); Dr. Yijun He (School of Marine Science, NUIST); Dr. Will Perrie (Bedford Institute of Oceanography)

We present an approach for automatic classification of Arctic sea ice and open water from spaceborne C-band RADARSAT-2 fully polarimetric (HH, HV, VH, VV) synthetic aperture radar (SAR) imagery, based on the random forest (RF) model. The HH- and HV-polarized radar backscatter, incidence angle and optimal polarimetric features are inputs of the RF model. First, we use a physically-based unsupervised method to generate well-annotated sea ice and open water samples. This method can alleviate labeling errors arising from visual interpretation or temporal-spatial mismatch between SAR data and operational ice chart. Subsequently, we estimate a set of polarimetric characteristic parameters and select the most representative features, using the recursive feature elimination method. Finally, the RF model is trained and validated using more than one million labelled samples. Statistical validation results show that the overall sea ice and water classification accuracy is 99.94% and the Kappa coefficient is 0.998. We find that ice–water discrimination accuracy can be improved by 5%–10%, when optimal polarimetric features are involved in the RF model input. Moreover, the polarization difference (PD, VV–HH) is found to be the foremost polarimetric parameter for distinguishing sea ice from open water. The proposed approach achieves satisfactory ice–water classification over the complicated marginal ice zone. High-resolution (200 m) ice–water classification maps clearly show that sea ice leads and their surroundings are also well separated.

Multimodal Data Analyses - November 3rd 8:30am-10:00am PDT

Towards Predicting Whale Movement

Dr. Michael Dowd (Dalhousie University); Dr. Ruth Joy (HALLO; SFU); Teng Wei Lim (HALLO)

Shipping traffic continues to grow in the Salish Sea and with this comes an increased risk of cetaceans being disturbed, harassed, and potentially colliding with ships. With the aim of reducing these risks to whales, particularly the endangered Southern Resident killer whale, we aim to build a forecast system for the movement of whales near coastal shipping lanes. We've built a sequential data assimilation framework that ingests near real-time citizen science visual detections of SRKW. I will present the methods we've developed for the movement forecast system, and show how it can be adapted to the ingestion of additional types of detection data including hydrophone (acoustic) data that streams in near real-time across various hydrophone systems in Canada and the US. This approach makes use of A.I. systems for multi-species detection and classification, as well as historical whale tracks from which to build further insights into whale preference and whale movement in the Salish Sea. The methods will provide ship pilots and captains a few hours lead time to adjust their speed or pathway to minimize whale-vessel interactions.

COVE’s Stella Maris

Riley Kennedy (COVE)

Stella Maris, a multi-sensor seabed platform at COVE, provides ocean technology companies with a low-cost and easily accessible way to conduct product development and verification aided by data sets from several reference corroborating sensors. Users are often in process of commercialization and thus able to get their products to market faster.

Uncrewed surface vehicles as a platform for marine mammal monitoring

Dr. Oliver Kirsebom (Open Ocean Robotics)

Uncrewed surface vehicles (USVs) are an innovative platform for marine mammal monitoring, offering advantages in safety, affordability, and sustainability compared to traditional methods. Open Ocean Robotics' (OOR) Data Xplorer vehicle carries optical and thermal cameras and can additionally be equipped with towed hydrophones. In this contribution, I will discuss how such multi-sensor data can be combined to achieve improved, real-time detection of marine mammals, paying particular attention to the role played by machine learning in automating such systems.

Cross-Discipline Panel - November 3rd 10:00am-11:00am PDT

Catherine Evans (MERIDIAN) - Satellite Images

Bryanna Sherbo (DFO) - Satellite Images

Dr. Oliver Kirsebom (Open Ocean Robotics) - Multimodal Machine Learning

Dr. Gabriel Spadon De Souza (Dalhousie University) - Vessel Acoustics

Dr. Valentina Staneva (University of Washington) - Open Science/Machine Learning

Dr. Scott Veirs (Beam Reach Marine Science) - Bioacoustics