The major contributors of this repository include Jing Liao, Yuan Yao, Lu Yuan, Gang Hua and Sing Bing Kang at Microsoft Research.

Deep Image Analogy is a technique to find semantically-meaningful dense correspondences between two input images. It adapts the notion of image analogy with features extracted from a Deep Convolutional Neural Network.

Deep Image Analogy is initially described in a SIGGRAPH 2017 paper

This is an official C++ combined with CUDA implementation of Deep Image Analogy. It is worth noticing that:

• Our codes are based on Caffe.
• Our codes only have been tested on Windows 10 and Windows Server 2012 R2 with CUDA 8 or 7.5.
• Our codes only have been tested on several Nvidia GPU: Titan X, Titan Z, K40, GTX770.
• The size of input image is limited, mostly should not be large than 700x500 if you use 1.0 for parameter ratio.

© Microsoft, 2017. Licensed under a MIT license.

If you find Deep Image Analogy (include deep patchmatch) helpful for your research, please consider citing:

@article{Liao:2017:VAT:3072959.3073683,
 author = {Liao, Jing and Yao, Yuan and Yuan, Lu and Hua, Gang and Kang, Sing Bing},
 title = {Visual Attribute Transfer Through Deep Image Analogy},
 journal = {ACM Trans. Graph.},
 issue_date = {July 2017},
 volume = {36},
 number = {4},
 month = jul,
 year = {2017},
 issn = {0730-0301},
 pages = {120:1--120:15},
 articleno = {120},
 numpages = {15},
 url = {http://doi.acm.org/10.1145/3072959.3073683},
 doi = {10.1145/3072959.3073683},
 acmid = {3073683},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {deep matching, image analogy, transfer},
} 

One major application of our code is to transfer the style from a painting to a photo.

It can also swap the styles between two artworks.

The most challenging application is converting a sketch or a painting to a photo.

It can do color transfer between two photos, such as generating time lapse.

• Windows 7/8/10 (for linux or mac os x user, please check branch linux.)
• CUDA 8 or 7.5
• Visual Studio 2013

• Build Caffe at first. Just follow the tutorial here.
• Edit deep_image_analogy.vcxproj under windows/deep_image_analogy to make the CUDA version in it match yours .
• Open solution Caffe and add deep_image_analogy project.
• Build project deep_image_analogy.

You need to download models VGG-19 model before start to run a demo. Go to windows/deep_image_analogy/models/vgg19/ folder and download:

• http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_19_layers.caffemodel

Open main.cpp in windows/deep_image_analogy/source/ to see how to run a demo. You need to set several parameters which have been mentioned in the paper. To be more specific, you need to set

• path_model, where the VGG-19 model is.
• path_A, the input image A.
• path_BP, the input image BP.
• path_output, the output path.
• GPU Number, GPU ID you want to run this experiment.
• Ratio, the ratio to resize the inputs before sending them into the network.
• Blend Weight, the level of weights in blending process.
• Flag of WLS Filter, if you are trying to do photo style transfer, we recommend to switch this on to keep the structure of original photo.

We also provide a pre-built executable file in folder windows/deep_image_analogy/exe/, don't hesitate to try it.

To run this deep_image_analogy.exe, you need to write a command line as:

deep_image_analogy.exe ../models/ ../demo/content.png ../demo/style.png ../demo/output/ 0 0.5 2 0

which means

• path_model=../models/
• path_A=../demo/content.png
• path_BP=../demo/style.png
• path_output=../demo/output/
• GPU Number=0
• Ratio=0.5
• Blend Weight=2
• Flag of WLS Filter=0( 0: WLS filter disabled, 1: WLS filter enabled, only required for the case of photo to photo)

• We often test images of size 600x400 and 448x448.
• We set ratio to 1.0 by default. Specifically, for face (portrait) cases, we find ratio = 0.5 often make the results better.
• Blend weight controls the result appearance. If you want the result to be more like original content photo, please increase it; if you want the result more faithful to the style, please reduce it.
• For the four applications, our settings are mostly (but not definitely):
  • Photo to Style: blend weight=3, ratio=0.5 for face and ratio=1 for other cases.
  • Style to Style: blend weight=3, ratio=1.
  • Style to Photo: blend weight=2, ratio=0.5.
  • Photo to Photo: blend weight=3, ratio=1.

Our codes acknowledge Eigen, PatchMatch, CudaLBFGS and Caffe. We also acknowledge to the authors of our image and style examples but we do not own the copyrights of them.