The Met, embedded: a visual semantic map of 235,000 artworks
Embedded The Metropolitan Museum of Art's public-domain collection with an open multimodal model and built an interactive map in Nomic Atlas, so the art can be searched by what it looks like.
Visit project ↗
The Met has released hundreds of thousands of artworks into the public domain, but you can only find them through the words a cataloguer wrote. Search for a quiet harbor at dusk and you get nothing unless someone typed those words into the record.
What I did
I ran the collection’s images through an open-source multimodal embedding model, so each artwork becomes a vector that captures what it depicts rather than what its record says. Then I loaded the embeddings into Nomic Atlas, which lays similar works out near each other on a two-dimensional map and supports natural-language search across the whole collection.
The result is a map of more than 235,000 works where landscapes cluster with landscapes and portraits with portraits, and where one query pulls up matches across centuries and media. It is featured on Nomic’s Discover page.
Images: Van Gogh, Turner, and Bruegel, from The Met’s Open Access collection.
