Deeper Dive
One of the great misconceptions is that we understand our oceans. Despite being in the planet’s backyard, the ocean and its creatures are almost alien to us, simply because studying life beneath the surface is so difficult. Over 300,000 cetaceans die every year due to human activity, primarily ship collisions and fishing entanglements, with roughly 67% of all whale deaths attributed to human interactions. The North Atlantic right whale is critically endangered, numbering only around 350. To combat this, we proposed a real-time right whale detection network of low-cost, 3D printed buoys deployed across the Atlantic Ocean, designed to prevent human-caused deaths by monitoring whale locations and warning ships and fishermen away from collisions. Each buoy consists of an ESP32, solar panels and a hydrophone and costs significantly less than current buoys. It captures audio and sends it to a cloud server, where a two branch ensemble model consisting of a 2D Convolutional Neural Network feeding a custom attention module alongside a Bi LSTM classifies whether whale calls are present. Client applications then receive whales’ real-time locations through an online API. In testing, the design demonstrated reliability, scalability and accuracy, with promising results even when extended to classifying other cetacean species.
The project began in the summer of 2024 with an interest in convolutional neural networks and the discovery that audio could be visualized as an image through a Fourier transformation, essentially allowing machine learning to “see” sound. We experimented with this concept on animal noises, eventually landing on North Atlantic right whales through the Marinexplore and Cornell University Whale Detection Challenge dataset on Kaggle. While researching how to understand whale noises, we came across NOAA papers describing the decline in whale populations, which prompted the idea for this project.
The work was mostly done on weekends in small sprints of rapid prototyping and testing, and many adjustments to the initial plan were forced by unforeseen hurdles. Processing originally happened on the buoy itself, which couldn’t keep up with the incoming stream of data, so we moved Moby, our detection model, to the server side, uploading compressed audio in real time through an API. Our solar panels provide 5V at 60 mA each, too little for the Raspberry Pi 4 we started with, so we swapped the buoy’s processor for a low-power ESP32. Moby itself evolved from a pure CNN, the most common choice in the literature, to an attention-based CBAM model after the CNN proved inaccurate whenever a whale call landed at an unexpected timestamp within the 2-second sample.
The work was completed at home, with testing at a local pool. School laid the foundational skills, like coding, but the project itself felt more like wandering in a dark forest: We were working well outside our curriculum, often failing spectacularly and scouring the internet for adjacent research to use as guides.
Information about the ocean is a rarity. MobyGlobal’s initial infrastructure was designed specifically for North Atlantic right whales, but the work can be translated to other animals, improving our understanding of the ocean. Real-time data about oceanic sounds can provide the location of marine mammals loud enough for a hydrophone to hear. Most directly, that could help ships and fishermen steer clear of areas with marine mammal activity, addressing the human-caused population decline this research was aimed at. But it extends much further: Researchers can better understand migration patterns, mating habits and climate change’s effects on marine life, and even whale watching tourism could become more predictable.