Same Energy – image similarity search with Deep Learning

Same Energy - Image similarity search with Deep Learning

In my previous post on Bing image search vs. Google, Yahoo & Tin Eye, I mentioned the “visual search” or “search by image” features of the major search engines, in which a reference image is the basis of the search rather than text.

Bing and Google will search first for additional copies of the same image, and when past those, will show “similar images” or “related images”. This can be an interesting way to discover new art and artists.

In comments on that post, a reader was kind enough to inform me of a new and fascinating entry into the field of image based search: Same Energy.

Same Energy is based on deep learning, a subset of machine learning, which is a form of computer process that is sometimes over generously called “artificial intelligence” or “AI”.

Although I’m sure Microsoft and Google are incorporating elements of machine learning into their image search algorithms, I believe they still rely heavily on associated text, tags and meta information. Same Energy appears to rely entirely on deep learning.

Bin an Google similar images search

I fed a source image of Monet’s On the Seine at Bennecourt to Bing, Google and Same Energy. The regular search engines returned many sources of copies of the same image, and then started to suggest “similar” images (images above).

Same Eneregy similar image search

Same Energy, while often finding a few copies of the same image, would much more readily start showing a range of similar images, often with more varied results than the big two (image above). I found its results were pretty good in terms of similarity, but often ranged wider and led to interesting discoveries.

When you upload an image (most easily by dragging one from your desktop, there is no provision for entering a URL for an image on the web), Same Energy immediately displays a tight grid, arranged mosaic style, of numerous images (you can modify the grid under “Settings”). The image being referred to is displayed at the upper left.

Clicking on one of the similar images places that image in the upper left, and shifts the response and field of images to similarity to that image. It also allows you to click on that new reference image and see more detail, including a link to the page from which the image was sourced (images above).

While Same Energy seemed to have a broader interpretation of “similar” than the major search engines, it was quite good at recognizing the nature of the image and the quality of the art. I deliberately tried to get it to mistake very realistic paintings for photographs, but it almost never did. It also did not readily mix in amateurish low quality paintings with high quality work by more accomplished painters. Both of these can be unfortunate tendencies in the big search engines’ image similarity searches.

The combination of the quick, densely displayed returns and the level of quality makes searching for similar images more engaging with Same Energy than with Bing or Google. I found myself fascinated, dragging images of widely varied styles into the interface, and often tracking down images that came up that were new to me, and bookmarking often. If you sign up for an account, the option is offered to save images and create Collections.

Searching for images with nudity, as in figure drawing models, returns blurred images. You must click through on an image and check your acceptance to see images that contain nudity, or have been tagged NSFW, etc. Be aware that the results (many sourced from Reddit) are then pretty much unfiltered.

I tried uploading an image from my own webcomic, Argon Zark!, and I found the returns actually gave me some insight into how others (or at least a deep learning algorithm) might perceive my style (image above).

You can also enter text searches, but I found this feature weak at this point in the system’s development. I tried the names of several artists, and many weren’t recognized as viable search terms. There is no provision yet for setting image size, or other refinements that might possibly be added in the future. Also, I noticed the same images coming up fairly often in my searches for a given genre, for instance landscape painting.

Same Energy is young, still in Beta, and likely has a much smaller database of indexed material at this point than the big players, who have been at it for years and with significantly greater resources.

Jacob Jackson, the developer, indicates that Same Energy is an ongoing project and will continue to be refined and improved, even while its reach increases.

I, for one, will be stopping back frequently to indulge.

The upshot is, if you’re looking for alternate sizes or additional information on a given image, use Bing or Google. If you are more interested in exploring, discovering and just having fun searching through art images (or any kind of images), give Same Energy a try.

[Timesink Warning!]


Bing image search vs. Google, Yahoo & Tin Eye

Bing image search interface

As you might imagine, in the course of writing Lines and Colors I do a fair bit of searching out art images on the web — whenever possible searching for the largest examples of images of artwork that I can find.

One of the ways I do this is to use the “image search” features of the major search engines. Unfortunately, Google Image Search, which used to be the standard, has been diminished in its usefulness, as Google, perhaps nervous about copyright issues, has gotten namby pamby about searching for large images and taken away the ability to search for images in extra large or custom, viewer chosen sizes.

I have of late switched the majority of my art image searching to another search engine.

Bing Image Search

OK, I hear you snickering (Bing, really? BING?). Yes, Bing, Microsoft’s seldom used (but actually decent) competitor to Google’s overwhelming dominance of the search arena.

While Google hamstrings its image search, Bing offers a full featured image search, that not only allows you to search for extra large images, but offers a few features Google’s version never did.

Unlike Google’s typically spare opening page, Bing Image Search is crowded with suggested images of pop stars, cute animals and a bunch of other pop culture garbage you’re sure to be fascinated by. (I think if you’re logged into a Microsoft account, it may remember your own recent searches.) The simple search field is at the top.

In the initial search term result (images above, with detail crop; I’ve searched for “Dutch landscape painting”), click on “Filter” to the right, and in the sub-navigation that drops down, click on “Image Size” at the left. You’ll have a choice for Small, Medium and Large, as with Google, but in addition you can choose “Extra Large”, or enter custom size parameters in the provided fields. I often search for 2000 x 2000 pixels. (The little icon in the right side of the search bar is for “search by image”.)

The filtered page will show large images with the size displayed over them. If you click on the hamburger menu at the upper right, you’ll have the option to display information from the page under the images.

Clicking on an image gives a close up. In the column to the right, the first two entries are ads, the third is the link to the originating age for the image, Under that are buttons for “Visit site”, “Pages” and Image sizes”, and below that similar images (related, but not the images in question) and related searches.

Clicking “Pages” produces a list of pages that display version of that image.

Clicking “Image sizes” organizes the image sources by the size of the image (largest is not always best as some may be watermarked or less accessible than others, you can also have multiple choices for the same size image).

The little icon in the search field at the top of the page (that I assume is supposed to be a camera) opens a Visual Search box. It offers you the option to upload an image, or enter a link to one, and search for other, hopefully larger, versions of that same image. It also allows you to search for a page with additional information about an image you’re trying to identify.

Google Image Search

Google image search interface

The Google Image Search initial returns on a search is similar to Bing’s. A link for “Tools” on the right drops down a sub-navigation from which you can choose “Image Size” on the left, with choices only for Small, Medium and Large as well as “All”.

The filtered returns show page location under them, image size is not available.

Google image search interface

Clicking on an image shows a preview in a right hand column, with the page name and link below it. In this case, the size is available by rollover. Under that are “Related images” (similar but not copies of the same image), and “Related Searches”.

Google image search interface

The Camera icon in the search bar is for visual image search. Upload or paste the URL of an image (the image itself, not a page containing an image), and returns an array of copies of the image with the source page underneath.

If you right click (or Control-click on Mac) on an image in Google Chrome, you will see a choice to “Search Googe for image”. There are plugins that provide the same functionality for other browsers.

Yahoo Image Search

Yahoo Image Search interface

Yahoo Image Search exists. Why, I’m not sure.

The initial search term results look much like Bing or Google, but there is no page or size information. Clicking on “Advanced” at right provides filters for color, size and image type. Sizes are S,M,L. (The others let you search by color as well, from the sub-menus.)

Clicking on an image returns a detail panel with the page and size info and the option to “Visit page” or View image”. There is no visual image search that I can find.

Tin Eye reverse Image Search

Tin Eye reverse Image Search interface

Tin Eye is a venerable visual image search engine that provided that service before the big guys, um… borrowed the idea. I mention it primarily out of respect for that. It still does a good job in its initial mission, but there is no provision for image size choices. Tin Eye offers plugins to put their reverse image search in browser menus. Tin Eye offers a service to track your own images and notify you if it finds they’re being used elsewhere, but the service is expensive, probably mostly of use to corporate intellectual property holders.

Tip for searching by site

All three of the above major search engines allow you to search for images (or other content) from a particular site. In the regular search bar, enter the search terms, followed by a space and then the word site, a colon (no space) and the URL of the site. For example: “dutch landscape paintings”. This will return a page with results for that topic only from the Sothebys auction site.

Other sources for high res art images

General search engines are just one avenue for searching out art images on the web. Another, often more fruitful way to find large art images is to do local searches on the sites of major museums, or on art image agglomeration sites, such as the Google Art Project, Wikimedia Commons of the Art Renewal Center. These should be the topic of another post.

Happy image searching!

(Oh yes, and Time Sink Warning!)


Eye Candy for Today: Johan Christian Dahl landscape

View From Stalheim, Johan Christian Dahl, oil on canvas
View From Stalheim, Johan Christian Dahl, oil on canvas

View From Stalheim, Johan Christian Dahl, oil on canvas, 75 x 97 inches (190 x 246 cm).

Link is to zoomable image on Google Art Project; (very) high resolution file on Wikimedia Commons, original is in the National Museum of Art and Design, Oslo.

19th century Norwegian painter Johan Christian Dahl’s large scale view of a mountainous landscape from a village in the western region of the country gives a grand vista, and resolves in closer views to a series of interesting details, obviously designed to please the viewer’s eye at several levels.

Be aware that the high-resolution downloadble file on Wikimedia Commons is quite large at 74MB.


Eye Candy for Today: Tarbell’s Preparing for the Matinee

Preparing for the Matinee, Edmund Charles Tarbell, oil on canvas
Preparing for the Matinee, Edmund Charles Tarbell, oil on canvas

Preparing for the Matinee, Edmund Charles Tarbell; oil on canvas, roughly 45 x 35″ (114 x 89 cm); link is to zoomable version on Google Art Project; downloadable (very) high resolution file on Wikimedia Commons; original is in the Indianapolis Museum of Art, which also has zoomable and downloadable versions.

Like other members of the early 20th century group of painters known as the Boston School, Edmund Charles Tarbell frequently took as his subjects well-to-do young women quietly engaged in everyday activities. Here, a young woman tends to her appearance in a gilt-edged mirror, barely seen to the left of the composition.

Like most of his fellow Boston School painters, Tarbell combined academic refinement with the loose painterly brush work and clear observation of a scene as light and color championed by Monet and the French Impressionists.

A major influence on the Boston School painters, and Tarbell in particular, was 17th century Dutch master Johannes Vermeer, a corner of whose painting Lady at the Virginal with a Gentleman (“The Music Lesson”) is suggested at the upper right, in much the same way Vermeer himself would reference parts of other paintings at the edges of his composition. (See images above, bottom, in which I’ve inset Vermeer’s painting next to Tarbell’s shadowed homage.)


Beautiful “rediscovered” Constable

Dedham Vale With The River Stour In Flood From The Grounds Of Old Hall, East Bergholt, John Constable, rediscovered Constable landscape
Dedham Vale With The River Stour In Flood From The Grounds Of Old Hall, East Bergholt, John Constable

We’re fortunate that so much of the world’s great art is currently in museums and public collections. Works in private collections can often go unseen by the public for decades, or even hundreds of years.

From time to time, works that have gone unseen for extended periods become available and enter the art market.

This is the case with a newly “rediscovered” painting by John Constable, and boy is it a beauty!

Of those Constable paintings that I have not seen in person, it is already one of my favorites.

The painting is currently set to be part of the December 2017 Old Masters Evening Sale at Sotheby’s, one of the premiere auction houses through which high valued art reaches the market (see my 2009 post on Sotheby’s). If you have an extra $3 or $4 million lying around, maybe you can pick it up for over your couch.

There is a reasonably large zoomable photo on the Sotheby’s auction listing, but a much higher resolution image accompanying this article on Art Market Monitor (even larger than my detail crops would indicate).

There is also an article on the Sotheby’s site that describes the find: “Important Constable Rediscovered After 50 Years“.


Treasure trove of high-res images from Nationalmuseum Stockholm

high-resolution painting images from Nationalmuseum Stockholm; Anders Zorn, Oscar Torna, Hanna Pauli, Alfred Thorne, Rembrandt van Rijn, John Bauer, Frits Thaulow, Pieter de Hooch, William Blair Bruce, Egron Lundgren, Carl Wilhelmson, Anthony van Dyck, Gustaf Rydberg, Bernardino Mei, Johan Christoffer Boklund, Laurits Andersen Ring, Jean Simeon Chardin

In a gesture to make up for the inaccessibility of much of the museum’s collections during a major renovation to the building, the Nationalmuseum Stockholm has just released 3000 high resolution public domain art images from its collection to Wikimedia Commons.

There is an article on the museum’s website here.

The images are arranged on the Wikimedia commons site in a special (hidden) category: Media contributed by Nationalmuseum Stockholm: 2016-10, that is arranged for browsing alphabetically (note the “previous page”/”next page” links at the bottom of each page of thumbnails).

I don’t see a way to search specifically within the category, but I suppose you can do a general search for an artist’s name plus “Nationalmuseum” in the Wikimedia search box. Should you want more information about any of the works or the artists, you can switch over to the Nationalmuseum’s collection search.

Most of the images are at least 3,000 to 4,000 pixels wide, certainly large enough to see paint texture and individual brushstrokes in many of the paintings.

Browsing tip: If you click on the image thumbnails on Wikimedia Commons, they will open in a kind of viewer; however, if you click on the text title, you’ll open the image detail page with options to view or download the image at various sizes.

If you want the largest image without the largest file size, note that the last images in the list of available image sizes are TIFF files that are large in file size. You will usually see the next-to last image in the list of sizes is a JPEG image that is the same dimensions as the TIFF, but much smaller in file size. Though JPEG is a “lossy” format (throwing away image data to achieve higher compression) the compression levels are low enough that you won’t see much, if any, difference.

Not only are there beautiful works in this lot from the museum’s deep collection of Swedish and Norwegian artists, like Anders Zorn, Carl Larsson, Carl Fredrik Hill, John Bauer and Frits Thaulow; there are works by greats from elsewhere in Europe, like Rembrandt, François Boucher, Francesco Guardi, Gustave Courbet, Jan Lievens, Pieter de Hooch, Auguste Renoir, Jean Siméon Chardin and many others.

What a great resource.

You may have to dig a bit to find the kind of works you’re most interested in, but if you’re inclined to browse and linger through high-res art images the way I am, I’ll issue my customary time-sink warning, so you don’t inadvertantly wake up with half a day gone.

The release of the images coincides with a new exhibition at the museum that promises to be terrific, featuring more than 160 works of Scandinavian 19th century painting from the collection. Turn-of-the-Century Gems will be on view at the Nationalmuseum Stockholm from 23 June to 24 August, 2017.

(Images above: Anders Zorn, Oscar Törnå, Hanna Pauli, Alfred Thörne, Rembrandt van Rijn, John Bauer, Frits Thaulow, Pieter de Hooch, William Blair Bruce, Egron Lundgren, Carl Wilhelmson, Anthony van Dyck, Gustaf Rydberg, Bernardino Mei, Johan Christoffer Boklund, Laurits Andersen Ring, Jean Siméon Chardin)