Picture a hydrophone hanging in dark Arctic water, recording nothing you can see, only sound. A narwhal’s buzzing clicks. A bowhead whale’s low, descending moan. A bearded seal trill, eerie and trembling, a sound that belongs more to a ghost story than to a mammal.
I wanted to know whether a machine learning model, given nothing but 4,761 real recordings and no ecological knowledge at all, could tell these animals apart by voice alone. Then I wanted to know whether a more ambitious idea, a transformer model, the same family of architecture behind modern language AI, could learn anything useful when trained from scratch.
Both answers turned out to be more interesting than a simple yes or no.
Update, November 2026: I have written this project up as a preprint on EcoEvoRxiv (doi.org/10.32942/X2P68X) and added two pretrained AI models, BirdNET and Perch 2.0. They fell into the same trap. The new results are below.
The short version
- The models were given 4,761 real recordings of nine marine mammal species, six of them Arctic or Subarctic, from 148 separate recording tapes.
- Tested fairly, on tapes the model had never heard, a random forest identified baleen whales, toothed whales and seals with 91 percent accuracy, and the exact species 74 percent of the time.
- Tested the usual shortcut way, with clips from the same tape on both sides, the same model scored 97 percent. Much of that extra accuracy came from recognising the tape, not the animal.
- A small transformer trained from scratch matched the random forest on the three groups but fell behind on species.
- Two pretrained AI models, BirdNET and Perch 2.0, did best on unseen tapes, 66 and 69 percent balanced accuracy for species, but random splits still inflated them by 28 and 26 points.
Real recordings, not synthetic data
Every clip in this project comes from the Watkins Marine Mammal Sound Database, built at Woods Hole Oceanographic Institution from decades of field recordings. It is one of the most widely used open bioacoustic archives in marine science, holding roughly 2,000 recordings across more than 60 species.
This version uses every recording of nine species in the database, 4,761 clips from 148 recording tapes made between 1949 and 1997: six genuinely Arctic or Subarctic animals, narwhal, beluga, bowhead whale, ringed seal, bearded seal and walrus, alongside three wider ranging comparison species, killer whale, humpback whale and North Atlantic right whale. The first version of this project used only 225 of them.
The official Woods Hole site was down for maintenance, so every clip was pulled one file at a time from the public Internet Archive mirror of the database rather than as its full 6.2 gigabyte release.

What these six Arctic animals actually sound like
Before any of this becomes a modelling problem, it is worth knowing what the model is being asked to pull apart. The descriptions and frequency ranges below come from Discovery of Sound in the Sea, with the bearded seal figures from NOAA Fisheries.
Narwhal. Echolocation clicks fired anywhere from twice to more than five hundred times a second, reaching up to 160 kHz, far above anything a human ear registers. Between the clicks sit tonal pulsed calls and whistles roughly a second long.
Beluga. Whistles, squeals, chirps and clicks, varied and frequent enough to have earned the species the nickname sea canary. Most social calls sit between about 2 and 6 kHz, low enough to be heard through the hull of a boat.
Bowhead whale. Long moans, most of them between 25 and 900 Hz, at the very bottom of the range. Repeated in sequence, those moans become songs.
Ringed seal. Simple barks, yelps and clicks between roughly 0.4 and 16 kHz, made all year rather than only in the breeding season.
Bearded seal. The strangest voice in the Arctic. Males sing a long descending trill while spiralling slowly downward, releasing bubbles as they go. NOAA Fisheries reports that a single trill can last as long as three minutes and carry up to twelve miles, and the longer a male can hold it, the better his condition.
Walrus. Taps, knocks, pulses and bell like tones between 100 Hz and 10 kHz, delivered underwater by males during the breeding season.
Even written out in words, these are not subtle differences. The question is whether they survive being turned into numbers.
Why sound is a sequence problem, just like DNA
Ecology job postings increasingly ask for experience with transformer based models and sequence data, usually in the context of eDNA, the genetic material organisms shed into water or soil.
That request makes more sense once you notice that a short audio clip is a sequence in exactly the same mathematical sense as a strand of DNA or a chain of amino acids. A transformer does not care whether the tokens it is reading are genetic bases or slices of a spectrogram. It cares about patterns across a sequence.
That is the real reason this project uses audio as a stand in for the kind of sequence modelling ecology increasingly leans on, and why a transformer belongs in the toolkit here rather than as decoration.
What the model sees instead of hears
Before any modelling, every clip becomes a spectrogram, a visual map of frequency against time.

Even to the eye, these calls are already distinct. The question is whether that distinctness survives being reduced to numbers a model can learn from.
Clustering could not find the groups on its own
If you have ever run a PCA on your own field data and seen one grey cloud where you expected three clean groups, this is that moment, and it is completely normal.
The first test mirrored the same honest question asked of any new biodiversity dataset: does structure exist here without being told what to look for.
Each clip was reduced to a fixed length set of classical acoustic features, pooled MFCCs and spectral descriptors, then clustered using PCA and KMeans, with the number of clusters chosen by silhouette score.

The adjusted Rand index against species was slightly below zero, on a scale where 1 means the clusters match the real groups perfectly and 0 is what random guessing would give you. Against broad taxonomic group it was also essentially zero. Left alone, the raw feature space does not sort these calls into clean groups.
The trap: a model that learns the tape, not the animal
Here is the part I did not expect, and the most useful thing this project taught me.
Every Watkins recording comes from a tape, made on one expedition, with one hydrophone, in one patch of ocean. Clips from the same tape share background noise, equipment hiss and recording conditions. If some clips from a tape end up in training and others from the same tape end up in testing, a model can score well by recognising the tape rather than the animal.
So I tested every model twice. Once the common way, splitting clips at random. Once the strict way, holding out whole tapes so the model is always tested on recordings it has never heard anything from.

With random splits, the random forest looked nearly perfect, 97 percent for species and 97 percent for groups. Holding out whole tapes, species accuracy dropped to 74 percent and balanced accuracy, which gives every species equal weight, fell from 89 to 49 percent.
The species view shows exactly where the illusion came from.

Narwhal scored 0.91 with random splits and 0.02 on unseen tapes. Ringed seal went from 0.73 to zero. Both were recorded on only five or six tapes, so the model had memorised those tapes instead of learning the animals, and on new tapes it mostly guessed killer whale, the species it had heard most. Killer whale and bowhead whale, by contrast, held up well.
This is not a quirk of whale sounds. Any ecological dataset where samples cluster, by site, by camera trap, by sequencing run, by observer, can leak the same way, and random splits will hide it.

But a random forest found real signal
Tested the strict way, the random forest still found real signal. It separated baleen whales, toothed whales and seals with 91 percent accuracy on unseen tapes, against a 61 percent baseline from always guessing the most common group, and 71 percent balanced accuracy. For all nine species it reached 74 percent accuracy against a 56 percent baseline.
The features driving it were spectral bandwidth, centroid and rolloff, sensible acoustic correlates of the real frequency differences between a baleen whale’s low moan, a toothed whale’s clicks and whistles, and a pinniped’s call.
The transformer experiment: an honest result, not a triumphant one
Then came the harder question. Could a small transformer, trained entirely from scratch with no pretrained weights, learn the same tasks directly from spectrograms, tested on the same unseen tapes as the random forest.

For the three groups it matched the random forest on balanced accuracy, 71 percent each, though with lower overall accuracy, 82 against 91 percent. For the nine species it fell behind, 42 percent balanced accuracy against 49, and only 59 percent accuracy against a 56 percent baseline.
In the first, smaller version of this project the transformer looked slightly ahead on a single split of 68 clips. With five rounds of testing on unseen tapes, that edge disappears. That is the honest result, and it matches the current state of the field: real gains from transformers in bioacoustics come from large scale pretraining, not from training a fresh model on a few thousand clips. The transformer used no pretrained weights, which is exactly why the comparison shows what training from zero can and cannot do. The pretrained models below show the other side.
Even pretrained AI falls for the same trap
What if the model already knows a lot about sound before it hears a single whale?
BirdNET, from the Cornell Lab of Ornithology and Chemnitz University of Technology, and Perch 2.0, from Google DeepMind, are pretrained audio models. Researchers often use them as feature extractors: the model turns each clip into a list of numbers, and a simple classifier is trained on top. BirdNET learned mostly from birds. Perch 2.0 learned from a much wider range of animals.
I ran every clip through both models and tested them the same two ways as before.

On unseen tapes, the pretrained models beat everything else. Perch 2.0 reached 69 percent balanced accuracy for species and 83 percent for the three groups, BirdNET 66 and 81 percent. The random forest managed 49 and 71.
But random splits inflated both, to 94 and 96 percent for species. Pretraining shrinks the leak, it does not close it. Even Perch 2.0 still struggled with the rare species: its F1 score for narwhal was 0.22 on unseen tapes, while ringed seal reached 0.80.
This does not mean BirdNET can identify whales. It means the features it learned from birds describe sound well enough for a simple classifier to separate these calls.
What this says about the bigger picture
Two very different questions, asked of two completely different kinds of biodiversity data in this project and its companion seabird diet study, produced the same underlying pattern. Unsupervised methods, left to find structure on their own, came up close to empty both times. Supervised methods, asked a specific, well posed question, found real, ecologically sensible signal both times.
That is not a coincidence. It says something true about how biological signal tends to be arranged, present and learnable, but rarely sitting in the obvious, low dimensional view a first pass at the data hands you.
A hydrophone in Arctic water keeps recording whether or not anyone is listening. The open question is no longer whether a machine can hear the difference between these animals. It is how little it needs to be taught before it can.
The limits, stated plainly
Species are very unevenly represented, from 46 clips on 5 tapes for ringed seal to 2,647 clips on 65 tapes for killer whale, so scores for the rarest species rest on few tapes. Holding out tapes removes the most obvious leakage, but tapes from the same expedition may still share conditions. The transformer only hears the first second and a half of each clip. Labels come from the database’s own folders, not independently reverified, and no environmental context, location, season, depth or ice cover, was used, only the sound itself. BirdNET and Perch 2.0 were used only as fixed feature extractors with a linear classifier, not fine tuned.
See the full analysis
All code, figures and results are public on GitHub at github.com/elifrey/marine-mammal-acoustics-ml. Version 1.2.0, with the BirdNET and Perch 2.0 results, is archived on Zenodo under the DOI 10.5281/zenodo.24158120, and 10.5281/zenodo.22859088 always points to the latest version. The full study is published as a preprint: Frey, E. (2026). Recording session leakage inflates accuracy in marine mammal call classification. EcoEvoRxiv. doi.org/10.32942/X2P68X. My ORCID record is 0009-0002-7482-7986.
Questions people ask about this kind of project
How does a computer listen to a whale call?
It does not listen the way we do. The audio is converted into a spectrogram, a picture of frequency over time, and the model learns patterns in that picture rather than processing raw sound waves directly.
What is a transformer model, in plain terms?
A transformer is a type of model built to find patterns across a sequence, whether that sequence is words in a sentence, bases in a strand of DNA, or slices of time in a spectrogram. It is the same underlying idea behind most modern language AI, applied here to sound.
Is this kind of technology actually used in real ocean research?
Yes. Passive acoustic monitoring, listening to the ocean over long periods with fixed hydrophones, is a growing part of real marine mammal and Arctic biodiversity monitoring, and machine learning is increasingly how researchers process the enormous amount of audio that generates.
What does a narwhal sound like?
The narwhal (Monodon monoceros) clicks, buzzes and whistles. Its echolocation clicks come in trains of two to more than five hundred a second, reach up to 160 kHz, far above human hearing, and speed up into a buzz as it closes in on prey. Between the clicks it uses whistles and pulsed calls about a second long to keep in touch with other narwhals.
What does a beluga whale sound like?
The beluga (Delphinapterus leucas) is one of the most vocal whales, with whistles, squeals, chirps and clicks that earned it the nickname sea canary. Most of its social calls sit between about 2 and 6 kHz, well inside human hearing, which is why belugas can be heard through the hull of a boat.
Can BirdNET identify whales and seals?
Not on its own. BirdNET was trained mainly on birds. What it can do is turn any sound into useful numbers, and a simple classifier trained on those numbers identified these nine species with 66 percent balanced accuracy on unseen tapes. Perch 2.0, trained on wider data, did slightly better.
Where can I find real marine mammal recordings to practice with myself?
The Watkins Marine Mammal Sound Database is free to use and one of the best starting points, covering roughly 2,000 recordings of more than 60 species and about 15,000 annotated clips. The official Woods Hole site has been offline for maintenance, so the route that works today is the public Internet Archive mirror of the full cuts collection.

Erzsebet Frey (Eli Frey) is an ecologist and online entrepreneur with a Master of Science in Ecology from the University of Belgrade. Originally from Serbia, she has lived in Sri Lanka since 2017. Eli has worked internationally in countries like Oman, Brazil, Germany, and Sri Lanka. In 2018, she expanded into SEO and blogging, completing courses from UC Davis and Edinburgh. Eli has founded multiple websites focused on biology, ecology, environmental science, sustainable and simple living, and outdoor activities. She enjoys creating nature and simple living videos on YouTube and participates in speleology, diving, and hiking. ORCID: 0009-0002-7482-7986
From the shop
Gifts for people who love the wild
Wildlife themed t-shirts, notebooks, tote bags, bandanas and guides, designed by an ecologist.
Visit the shop
