research-article

Painting with CATS: Camera-Aided Texture Synthesis

Authors:

Ticha Sethapakdi,

James McCannAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 57, Pages 1 - 9

https://doi.org/10.1145/3290605.3300287

Published: 02 May 2019 Publication History

Abstract

We present CATS, a digital painting system that synthesizes textures from live video in real-time, short-cutting the typical brush- and texture- gathering workflow. Through the use of boundary-aware texture synthesis, CATS produces strokes that are non-repeating and blend smoothly with each other. This allows CATS to produce paintings that would be difficult to create with traditional art supplies or existing software. We evaluated the effectiveness of CATS by asking artists to integrate the tool into their creative practice for two weeks; their paintings and feedback demonstrate that CATS is an expressive tool which can be used to create richly textured paintings.

Supplementary Material

SRT File (paper057vc.srt)

Video figure captions

Download
3.76 KB

ZIP File (paper057.zip)

Full resolution paintings, painting bases for paint overs, and swatch files (.png) for the paintings in the paper, source code for hole filling (.cpp), plus some paintings by one of the authors. Images can be viewed with any standard image-viewing program. Please note that the hole filling code is for algorithm reference only and is not standalone.

Download
46.23 MB

MP4 File (paper057.mp4)

Supplemental video

Download
28.67 MB

MP4 File (paper057p.mp4)

Preview video

Download
5.58 MB

References

[1]

{n. d.}. Adobe Capture. https://www.adobe.com/products/capture.html.

[2]

Adobe. 2010. Content-Aware Fill. https://research.adobe.com/project/content-aware-fill/

[3]

Bill Atkinson. {n. d.}. MacPaint. https://archive.org/details/mac_Paint_ 2.

[4]

Connelly Barnes, Dan B. Goldman, Eli Shechtman, and Adam Finkelstein. 2011. The PatchMatch Randomized Matching Algorithm for Image Manipulation. Commun. ACM 54, 11 (Nov. 2011), 103--110.

Digital Library

[5]

Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. 2009. PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing. In ACM SIGGRAPH 2009 Papers (SIGGRAPH '09). ACM, New York, NY, USA, Article 24, 11 pages.

Digital Library

[6]

Erin A. Carroll and Celine Latulipe. 2009. The Creativity Support Index. In CHI '09 Extended Abstracts on Human Factors in Computing Systems (CHI EA '09). ACM, New York, NY, USA, 4009--4014.

Digital Library

[7]

Soheil Darabi, Eli Shechtman, Connelly Barnes, Dan B. Goldman, and Pradeep Sen. 2012. Image Melding: Combining Inconsistent Images Using Patch-based Synthesis. ACM Trans. Graph. 31, 4, Article 82 (July 2012), 10 pages.

Digital Library

[8]

Olga Diamanti, Connelly Barnes, Sylvain Paris, Eli Shechtman, and Olga Sorkine-Hornung. 2015. Synthesis of Complex Image Appearance from Limited Exemplars. ACM Transactions on Graphics 34, 2 (2015).

Digital Library

[9]

Charles Han, Eric Risser, Ravi Ramamoorthi, and Eitan Grinspun. 2008. Multiscale Texture Synthesis. In ACM SIGGRAPH 2008 Papers (SIGGRAPH '08). ACM, New York, NY, USA, Article 51, 8 pages.

Digital Library

[10]

Anuruddha Hettiarachchi, Suranga Nanayakkara, Kian Peen Yeo, Roy Shilkrot, and Pattie Maes. 2013. FingerDraw: More Than a Digital Paintbrush. In Proceedings of the 4th Augmented Human International Conference (AH'13). ACM, New York, NY, USA, 1--4.

Digital Library

[11]

Craig Hickman. {n. d.}. Kid Pix - The Early Years. http://red-green-blue.com/kid-pix-the-early-years/.

[12]

Mami Kosaka and Kaori Fujinami. 2016. UnicrePaint: Digital Painting Through Physical Objects for Unique Creative Experiences. In Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '16). ACM, New York, NY, USA, 475--481.

Digital Library

[13]

Sylvain Lefebvre and Hugues Hoppe. 2005. Parallel Controllable Texture Synthesis. In ACM SIGGRAPH 2005 Papers (SIGGRAPH '05). ACM, New York, NY, USA, 777--786.

Digital Library

[14]

Sylvain Lefebvre and Hugues Hoppe. 2006. Appearance-space Texture Synthesis. In ACM SIGGRAPH 2006 Papers (SIGGRAPH '06). ACM, New York, NY, USA, 541--548.

Digital Library

[15]

Jingwan Lu, Connelly Barnes, Stephen DiVerdi, and Adam Finkelstein. 2013. RealBrush: Painting with Examples of Physical Media. ACM Trans. Graph. 32, 4, Article 117 (July 2013), 12 pages.

Digital Library

[16]

M. Lukac, P. Asente, J. Lu, E. Shechtman, and D. Sýkora. 2015. Brushables: Example-based Edge-aware Directional Texture Painting. Comput. Graph. Forum 34, 7 (Oct. 2015), 257--267.

Digital Library

[17]

Michal Lukac, Jakub Fiser, Jean-Charles Bazin, Ondrej Jamriska, Alexander Sorkine-Hornung, and Daniel Sýkora. 2013. Painting by Feature: Texture Boundaries for Example-based Image Creation. ACM Trans. Graph. 32, 4, Article 116 (July 2013), 8 pages.

Digital Library

[18]

Lincoln Ritter, Wilmot Li, Brian Curless, Maneesh Agrawala, and David Salesin. 2006. Painting with Texture. In Proceedings of the 17th Eurographics Conference on Rendering Techniques (EGSR '06). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 371--376.

Digital Library

[19]

Kimiko Ryokai, Stefan Marti, and Hiroshi Ishii. 2004. I/O Brush: Drawing with Everyday Objects As Ink. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, New York, NY, USA, 303--310.

Digital Library

[20]

Richard Shoup. {n. d.}. SuperPaint. https://web.archive.org/web/20020110013554/ http://www.rgshoup.com/prof/SuperPaint/.

[21]

Yonatan Wexler, Eli Shechtman, and Michal Irani. 2007. Space-Time Completion of Video. IEEE Trans. Pattern Anal. Mach. Intell. 29, 3 (March 2007), 463--476.

Digital Library

Cited By

Sun XQin J(2024)AI for Supporting the Freedom of DrawingMachine Intelligence Research10.1007/s11633-023-1438-421:1(63-88)Online publication date: 15-Jan-2024
https://doi.org/10.1007/s11633-023-1438-4
Li JRawn ERitchie JTran O'Leary JFollmer S(2023)Beyond the Artifact: Power as a Lens for Creativity Support ToolsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606831(1-15)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606831
Chung JAdar E(2023)PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like InteractionsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606777(1-17)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606777
Show More Cited By

Index Terms

Painting with CATS: Camera-Aided Texture Synthesis

Recommendations

Painting with CATS: Camera-Aided Texture Synthesis
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

We present CATS, a digital painting system that synthesizes textures from live video in real-time, short-cutting the typical brush- and texture- gathering workflow. Through the use of boundary-aware texture synthesis, CATS produces strokes that are non-...
Painting with CATS: Camera-Aided Texture Synthesis
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

We present CATS, a digital painting system that synthesizes textures from live video in real-time, short-cutting the typical brush- and texture- gathering workflow. Through the use of boundary-aware texture synthesis, CATS produces strokes that are non-...
Playful palette: an interactive parametric color mixer for artists

We present Playful Palette, a color picker interface for digital paint programs that derives intuition from oil paint and watercolor palettes, but extends them with digital features. A Playful Palette is a set of blobs of color that blend together to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
582
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)6

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sun XQin J(2024)AI for Supporting the Freedom of DrawingMachine Intelligence Research10.1007/s11633-023-1438-421:1(63-88)Online publication date: 15-Jan-2024
https://doi.org/10.1007/s11633-023-1438-4
Li JRawn ERitchie JTran O'Leary JFollmer S(2023)Beyond the Artifact: Power as a Lens for Creativity Support ToolsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606831(1-15)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606831
Chung JAdar E(2023)PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like InteractionsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606777(1-17)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606777
Stemasov ELedo DFitzmaurice GAnderson F(2023)Immersive Sampling: Exploring Sampling for Future Creative Practices in Media-Rich, Immersive SpacesProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596131(212-229)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596131
Qin JSun XXu W(2023)A State-of-Art Review on Intelligent Systems for Drawing AssistingHuman Interface and the Management of Information10.1007/978-3-031-35132-7_44(583-605)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35132-7_44
Sethapakdi TLeake MRodriguez CCai MMueller S(2022)KineCAMProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/35336135:4(1-9)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3533613
Chung JHe SAdar E(2021)The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support ToolsProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462050(1817-1833)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1145/3461778.3462050
Fang XXia CSugiura Y(2021)FacialPen: Using Facial Detection to Augment Pen-Based InteractionProceedings of the Asian CHI Symposium 202110.1145/3429360.3467672(1-8)Online publication date: 8-May-2021
https://dl.acm.org/doi/10.1145/3429360.3467672
Rafner JHjorth ARisi SPhilipsen LDumas CBiskjær MNoy LTylén KBergenholtz CLynch JZana BSherson JMirza-Babaei PMcArthur VVanden Abeele VBirk M(2020)crea.blenderExtended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play10.1145/3383668.3419907(340-344)Online publication date: 2-Nov-2020
https://dl.acm.org/doi/10.1145/3383668.3419907
Lin YGuo JChen YYao CYing FBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)It Is Your Turn: Collaborative Ideation With a Co-Creative Robot through SketchProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376258(1-14)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376258
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents