AI photo id

problem

North Atlantic right whales with bounding box annotations in preparation for machine learning in Flukebook. Permit #17355. Credit: NOAA Fisheries

There are less than 350 North Atlantic right whales alive today making them one of the most endangered animals on the planet. Individuals can be identified by the pattern on their head along with scars and other markings. Researchers take photographs from vessels and airplanes, and then compare those photographs to the online North Atlantic Right Whale Catalog curated by the New England Aquarium. Knowing the individual identity of a whale opens up many avenues of research and conservation management including demographics, social structure, reproductive biology, communication, and informed disentanglement operations. The process of matching a photograph to the catalog can be time-consuming, and finding a way to automate this process could free up valuable time and resources.

solution

Flukebook detection to machine learning pipeline for right whale matching.

We took an innovative approach by hosting a data science competition on Kaggle using a dataset of 7,000 aerial photographs of known right whales. The winning convolutional neural network pipeline by Deepsense.ai achieved 87% top-1 accuracy! Collaborating with Wild Me to move this solution into production, the Deepsense algorithms were retrained on the full dataset of images from the North Atlantic Right Whale Catalog and deployed on the Flukebook platform. The system was also expanded to allow for matching of the closely related Southern Right Whale with collaborations in Chile, Argentina, New Zealand, Australia, Brazil, and South Africa. The Wild Me team adapted the pose-invariant embeddings (PIE) algorithm to allow for the platform to also match photographs taken from a vessel. This project has been honored with the NOAA Bronze Medal Award in 2019 and the Gears of Government Award in 2020.

publications

Khan C, Blount D, Parham J et al. 2022. Artificial intelligence for right whale photo identification: from data science competition to worldwide collaboration. Mammalian Biology. https://doi.org/10.1007/s42991-022-00253-3

Bogucki R , Cygan M , Khan CB, Klimek M , Milczek JK and Mucha M. 2018. Applying deep learning to right whale photo identification. Conservation Biology. doi:10.1111/cobi.13226

links

press

awards

2020 Gears of Government Award through the Executive Office of the President. This Government-wide program is centered on recognizing outstanding contributions toward mission results, customer service, and accountable stewardship.

2019 NOAA Bronze Medal Award. The highest honor award granted by the Under Secretary of Commerce for Oceans and Atmosphere, Bronze Medals recognize federal employees for superior performance.

2017 Semifinalist in the Innovations in American Government Awards competition by the Ash Center for Democratic Governance and Innovation at the John F. Kennedy School of Government, Harvard University.