Picture a hydrophone hanging in dark Arctic water, recording nothing you can see, only sound. A narwhal’s buzzing clicks. A bowhead whale’s low, descending moan. A bearded seal trill, eerie and trembling, a sound that belongs more to a ghost story than to a mammal.
I wanted to know whether a machine learning model, given nothing but 225 real recordings and no ecological knowledge at all, could tell these animals apart by voice alone. Then I wanted to know whether a more ambitious idea, a transformer model, the same family of architecture behind modern language AI, could learn anything at all from so little data.
Both answers turned out to be more interesting than a simple yes or no.
The short version
- The model was given 225 real recordings of nine marine mammal species, six of them Arctic or Subarctic, and no ecological knowledge whatsoever.
- Left to find structure on its own, it found almost none. Asked a specific question, a random forest separated baleen whales, toothed whales and seals with 81 percent accuracy against a 33 percent baseline.
- A small transformer trained from scratch reached 84 percent on the same split, competitive with the random forest but not proven better, because the held out set contains only about 68 clips.
Real recordings, not synthetic data
Every clip in this project comes from the Watkins Marine Mammal Sound Database, built at Woods Hole Oceanographic Institution from decades of field recordings. It is one of the most widely used open bioacoustic archives in marine science, holding roughly 2,000 recordings across more than 60 species.
The subset used here is nine species and 225 clips, chosen deliberately to include six genuinely Arctic or Subarctic animals, narwhal, beluga, bowhead whale, ringed seal, bearded seal and walrus, alongside two wider ranging comparison species, killer whale and humpback whale.
The official Woods Hole site was down for maintenance while this project ran, so every clip was pulled one file at a time from the public Internet Archive mirror of the database rather than as its full 6.2 gigabyte release, a small technical workaround that kept the whole project reproducible in a single day.
What these six Arctic animals actually sound like
Before any of this becomes a modelling problem, it is worth knowing what the model is being asked to pull apart. The descriptions and frequency ranges below come from Discovery of Sound in the Sea, with the bearded seal figures from NOAA Fisheries.
Narwhal. Echolocation clicks fired anywhere from twice to more than five hundred times a second, reaching up to 160 kHz, far above anything a human ear registers. Between the clicks sit tonal pulsed calls and whistles roughly a second long.
Beluga. Whistles, squeals, chirps and clicks, varied and frequent enough to have earned the species the nickname sea canary. Most social calls sit between about 2 and 6 kHz, low enough to be heard through the hull of a boat.
Bowhead whale. Long moans, most of them between 25 and 900 Hz, at the very bottom of the range. Repeated in sequence, those moans become songs.
Ringed seal. Simple barks, yelps and clicks between roughly 0.4 and 16 kHz, made all year rather than only in the breeding season.
Bearded seal. The strangest voice in the Arctic. Males sing a long descending trill while spiralling slowly downward, releasing bubbles as they go. NOAA Fisheries reports that a single trill can last as long as three minutes and carry up to twelve miles, and the longer a male can hold it, the better his condition.
Walrus. Taps, knocks, pulses and bell like tones between 100 Hz and 10 kHz, delivered underwater by males during the breeding season.
Even written out in words, these are not subtle differences. The question is whether they survive being turned into numbers.
Why sound is a sequence problem, just like DNA
Ecology job postings increasingly ask for experience with transformer based models and sequence data, usually in the context of eDNA, the genetic material organisms shed into water or soil.
That request makes more sense once you notice that a short audio clip is a sequence in exactly the same mathematical sense as a strand of DNA or a chain of amino acids. A transformer does not care whether the tokens it is reading are genetic bases or slices of a spectrogram. It cares about patterns across a sequence.
That is the real reason this project uses audio as a stand in for the kind of sequence modelling ecology increasingly leans on, and why a transformer belongs in the toolkit here rather than as decoration.
What the model sees instead of hears
Before any modelling, every clip becomes a spectrogram, a visual map of frequency against time.
Even to the eye, these calls are already distinct. The question is whether that distinctness survives being reduced to numbers a model can learn from.
Clustering could not find the groups on its own
If you have ever run a PCA on your own field data and seen one grey cloud where you expected three clean groups, this is that moment, and it is completely normal.
The first test mirrored the same honest question asked of any new biodiversity dataset: does structure exist here without being told what to look for.
Each clip was reduced to a fixed length set of classical acoustic features, pooled MFCCs and spectral descriptors, then clustered using PCA and KMeans, with the number of clusters chosen by silhouette score.
The adjusted Rand index against species was 0.044, on a scale where 1 means the clusters match the real groups perfectly and 0 is what random guessing would give you. Against broad taxonomic group, baleen whale, toothed whale or seal, it was 0.003, essentially nothing. Left alone, the raw feature space does not sort these calls into clean groups.
But a random forest found real signal
Asked a sharper, supervised question, the story changed completely. A random forest predicting the broad taxonomic group from acoustic features alone reached 81 percent held out accuracy against a 33 percent baseline, with 86 percent five fold cross validated accuracy.
The features driving that accuracy were spectral bandwidth and centroid, sensible acoustic correlates of the real frequency differences between a baleen whale’s moan, a toothed whale’s clicks and whistles, and a pinniped’s call. A species level version of the same model, distinguishing all nine species individually, reached 68 percent held out accuracy and 75 percent under five fold cross validation, again led by acoustically sensible features rather than anything arbitrary.
The transformer experiment: an honest result, not a triumphant one
Then came the harder question. Could a small transformer, trained entirely from scratch with no pretrained weights, learn the taxonomic group task directly from raw spectrograms, competing against the random forest on the exact same held out clips.
It reached 84 percent accuracy, technically ahead of the random forest’s 81 percent on that identical split. That is a genuinely interesting result, and it deserves an equally honest caveat.
With only about 68 held out clips, a handful of examples shift accuracy by several percentage points, and unlike the random forest this comparison was not cross validated. The fair conclusion is that a from scratch transformer is competitive here, not proven superior.
That caveat is not a weakness in the project. It is the actual, current state of transformer models applied to small ecological datasets, and pretending otherwise would be the less useful story to tell.
Real gains from transformers in bioacoustics and eDNA work come from large scale pretraining, not from training a fresh model on 225 clips. No pretrained audio or bioacoustic embedding model was available in the environment this project was built in, which is exactly why the comparison exists in the first place, as an honest look at what training from zero can and cannot do.
What this says about the bigger picture
Two very different questions, asked of two completely different kinds of biodiversity data in this project and its companion seabird diet study, produced the same underlying pattern. Unsupervised methods, left to find structure on their own, came up close to empty both times. Supervised methods, asked a specific, well posed question, found real, ecologically sensible signal both times.
That is not a coincidence. It says something true about how biological signal tends to be arranged, present and learnable, but rarely sitting in the obvious, low dimensional view a first pass at the data hands you.
A hydrophone in Arctic water keeps recording whether or not anyone is listening. The open question is no longer whether a machine can hear the difference between these animals. It is how little it needs to be taught before it can.
The limits, stated plainly
225 clips is small by deep learning standards, chosen deliberately to keep this reproducible in a day without downloading the full archive. Species and group labels come from the database’s own folder structure, not independently reverified here. The three taxonomic groups are a useful simplification, not the only valid way to group these species. No environmental context, location, season, depth or ice cover, was used, only the sound itself.
See the full analysis
The complete notebook, all figures, the trained model code and the acoustic feature pipeline are public on GitHub at github.com/elifrey/marine-mammal-acoustics-ml. Version 1.0.0 is archived on Zenodo with a permanent DOI, 10.5281/zenodo.22859088, and my ORCID record is 0009-0002-7482-7986.
Questions people ask about this kind of project
How does a computer listen to a whale call?
It does not listen the way we do. The audio is converted into a spectrogram, a picture of frequency over time, and the model learns patterns in that picture rather than processing raw sound waves directly.
What is a transformer model, in plain terms?
A transformer is a type of model built to find patterns across a sequence, whether that sequence is words in a sentence, bases in a strand of DNA, or slices of time in a spectrogram. It is the same underlying idea behind most modern language AI, applied here to sound.
Is this kind of technology actually used in real ocean research?
Yes. Passive acoustic monitoring, listening to the ocean over long periods with fixed hydrophones, is a growing part of real marine mammal and Arctic biodiversity monitoring, and machine learning is increasingly how researchers process the enormous amount of audio that generates.
Where can I find real marine mammal recordings to practice with myself?
The Watkins Marine Mammal Sound Database is free to use and one of the best starting points, covering roughly 2,000 recordings of more than 60 species and about 15,000 annotated clips. The official Woods Hole site has been offline for maintenance, so the route that works today is the public Internet Archive mirror of the full cuts collection.
Erzsebet Frey (Eli Frey) is an ecologist and online entrepreneur with a Master of Science in Ecology from the University of Belgrade. Originally from Serbia, she has lived in Sri Lanka since 2017. Eli has worked internationally in countries like Oman, Brazil, Germany, and Sri Lanka. In 2018, she expanded into SEO and blogging, completing courses from UC Davis and Edinburgh. Eli has founded multiple websites focused on biology, ecology, environmental science, sustainable and simple living, and outdoor activities. She enjoys creating nature and simple living videos on YouTube and participates in speleology, diving, and hiking.
🌿 Explore the Wild Side!
Discover eBooks, guides, templates and stylish wildlife-themed T-shirts, notebooks, scrunchies, bandanas, and tote bags. Perfect for nature lovers and wildlife enthusiasts!
Visit My Shop →