research-article

Intermodulation: Improvisation and Collaborative Art Practice for HCI

Authors:

Laewoo (Leo) Kang,

Steven J. Jackson,

Phoebe SengersAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 160, Pages 1 - 13

https://doi.org/10.1145/3173574.3173734

Published: 19 April 2018 Publication History

Abstract

This paper integrates theory, ethnography, and collaborative artwork to explore improvisational activity as both topic and tool of multidisciplinary HCI inquiry. Building on theories of improvisation drawn from art, music, HCI and social science, and two ethnographic studies based on interviews, participant observation and collaborative art practice, we seek to elucidate the improvisational nature of practice in both art and ordinary action, including human-computer interaction. We identify five key features of improvisational action -- reflexivity, transgression, tension, listening, and interdependence -- and show how these can deepen and extend both linear and open-ended methodologies in HCI and design. We conclude by highlighting collaborative engagement based on 'intermodulation' as a tool of multidisciplinary inquiry for HCI research and design.

Supplementary Material

suppl.mov (pn2076-file5.mp4)

Supplemental video

Download
28.61 MB

MP4 File (pn2076.mp4)

Download
344.74 MB

References

[1]

Philip E Agre, and David Chapman. 1987. Pengi: lementation of a Theory of Activity. aaai.org.

Digital Library

[2]

Philip E Agre. 1997. Computation and Human Experience.

Digital Library

[3]

Jeffrey Bardzell, Shaowen Bardzell, Peter Dalsgaard, Shad Gross, and Kim Halskov. 2016. Documenting the Research Through Design Process. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems - DIS '16, 96--107.

Digital Library

[4]

Jeffrey Bardzell, Shaowen Bardzell, and Lone Koefoed Hansen. 2015. Immodest Proposals. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15, 2093--2102.

Digital Library

[5]

Estelle Barrett and Barbara Bolt. 2007. Practice as research: approaches to creative arts enquiry.

[6]

Steve Benford, Chris Greenhalgh, Andy Crabtree, Martin Flintham, Brendan Walker, Joe Marshall, Boriana Koleva, Stefan Rennick Egglestone, Gabriella Giannachi, Matt Adams, Nick Tandavanitj, and Ju Row Farr. 2013. Performance-Led Research in the Wild. ACM Transactions on Computer-Human Interaction 20, 3: 14:1--14:22.

Digital Library

[7]

Paul Berliner. 1994. Thinking in Jazz: The Infinite Art of Improvisation.

[8]

Wiebe E Bijker, Thomas P Hughes, Trevor Pinch, and Deborah G Douglas. 2012. The social construction of technological systems: New directions in the sociology and history of technology. MIT press.

[9]

J Bowers. 2002. Improvising machines: Ethnographically informed design for improvised electro-acoustic music. ARiADATexts (4).

[10]

John Bowers, Robyn Taylor, Jonathan Hook, Dustin Freeman, Charlie Bramley, and Christopher Newell. 2014. HCI: human-computer improvisation. In Proceedings of the 2014 companion publication on Designing interactive systems, 203--206.

Digital Library

[11]

1John Cage. 1952. 4'33''(Four thirty-three).

[12]

John Cage and David Tudor. 1959. Indeterminacy: New Aspect of Form in Instrumental and Electronic Music: Reading. Folkways Records.

[13]

Kathy Charmaz and Linda Liska Belgrave. 2007. Grounded theory. Wiley Online Library.

[14]

Taylan Cihan. 2013. Blister. https://vimeo.com/56305601

[15]

Peter Dalsgaard and Peter. 2016. Experimental Systems in Research through Design. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16, 4991--4996.

Digital Library

[16]

Paul Dourish, Annette Adler, and Brian Cantwell Smith. 1996. Organising User Interfaces Around Reflective Accounts. In Reflection'96, 235--244.

[17]

Ernest A. Edmonds, Alastair Weakley, Linda Candy, Mark Fell, Roger Knott, and Sandra Pauletto. 2005. The studio as laboratory: Combining creative practice and digital technology research. International Journal of Human Computer Studies 63, 4--5 SPEC. ISS.: 452--481.

Digital Library

[18]

Ernest Edmonds and Linda Candy. 2002. Creativity, art practice, and knowledge. Communications of the ACM 45, 10: 91--95.

Digital Library

[19]

Bill Evans. 1957. Improvisation in Jazz. Kind of Blue.

[20]

Bill Evans and Jim Hall. 1966. Intermodulation.

[21]

2SM Feisst. 2009. John Cage and improvisation: an unresolved relationship. Musical improvisation: Art, education, and society.

[22]

2Bill Gaver and John Bowers. 2012. Annotated portfolios. interactions 19, 4: 40.

Digital Library

[23]

William Gaver and William. 2012. What should we expect from research through design? In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12, 937.

Digital Library

[24]

Elizabeth Gerber and Elizabeth. 2007. Improvisation principles and techniques for design. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '07, 1069.

Digital Library

[25]

Elisa Giaccardi and Gerhard Fischer. 2008. Creativity and evolution: a metadesign perspective. Digital Creativity 19, March 2015: 19--32.

[26]

Hermann Gottschewski. 2007. The Provisum and the Improvisum in Musical Performance. Yonsei Music Research 14: 13--22.

[27]

Guy Hoffman and Gil Weinberg. 2010. Gesture-based human-robot jazz improvisation. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, 582--587.

[28]

Hrönn Brynjarsdóttir Holmer, Carl Disalvo, Phoebe Sengers, and Thomas Lodato. 2015. Constructing and constraining participation in participatory arts and HCI. International Journal of Human Computer Studies 74: 107--123.

Digital Library

[29]

Jonathan Hook, Guy Schofield, Robyn Taylor, Tom Bartindale, John McCarthy, and Peter Wright. 2012. Exploring HCI's relationship with liveness. In Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts - CHI EA '12, 2771.

Digital Library

[30]

Kristina Höök, Peter Dalsgaard, Stuart Reeves, Jeffrey Bardzell, Jonas Löwgren, Erik Stolterman, and Yvonne Rogers. 2015. Knowledge Production in Interaction Design. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15, 2429--2432.

Digital Library

[31]

3Steven J. Jackson and Laewoo Kang. 2014. Breakdown, obsolescence and reuse. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14, 449--458.

Digital Library

[32]

3Giulio Jacucci, Mira Wagner, Ina Wagner, Elisa Giaccardi, Mauro Annunziato, Nell Breyer, Jonas Hansen, Kazuhiro Jo, Stijn Ossevoort, Alessandro Perini, Natacha Roussel, and Susanne Schuricht. 2010. ParticipArt: Exploring participation in interactive art installations. In 9th IEEE International Symposium on Mixed and Augmented Reality 2010: Arts, Media, and Humanities, ISMAR-AMH 2010 - Proceedings, 3--10.

[33]

3P.a Jennings, E.b Giaccardi, and M.c Wesolkowska. 2006. About face interface: Creative engagement in the new media arts and HCI. Conference on Human Factors in Computing Systems - Proceedings: 1663--1666.

Digital Library

[34]

Laewoo (Leo) Kang. 2016. Breaking AndyWall: Transgressive and Playful Exploration on the Dynamic Role of Users in Art and Design. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 3855--3858.

Digital Library

[35]

Laewoo (Leo) Kang. 2018. Intermodulator: interactive audio-visual system for collaborative improvisation. In Proceedings of the 2018 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '18).

Digital Library

[36]

Laewoo (Leo) Kang. 2017. Echo (): Listening to the Reflection of Obsolete Technology. In Proceedings of the 2016 ACM Conference Companion Publication on Designing Interactive Systems, 305--308.

Digital Library

[37]

3Scott Barry Kaufman and Carolyn Gregoire. 2016. Wired to create: Unraveling the mysteries of the creative mind. Penguin.

[38]

3Nathaniel Klemp, Ray Mcdermott, and Jason Raley. 2008. Plans, Takes, and Mistakes. Critical Social Studies, 1: 4--21.

[39]

3Don Koberg. 1981. The all new universal traveler: A soft-systems guide to creativity, problem-solving, and the process of reaching goals. William Kaufmann Inc.

[40]

A. Baki Kocaballi and Yeliz Yorulmaz. 2016. Performative Photography as an Ideation Method. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems - DIS '16, 1083--1095.

Digital Library

[41]

4Patricia Leavy. 2010. Method meets art: Arts-based research practice. Counselling and Psychotherapy Research 10, 1: 76--77.

[42]

4George E. Lewis and Benjamin Piekut. 2016. The Oxford Handbook of Critical Improvisation Studies, Volume 1.

[43]

4John McCarthy and Peter Wright. 2004. Technology as experience. interactions 11, 5: 42.

Digital Library

[44]

4Timothy J. (Timothy James) McGee. 2003. Improvisation in the arts of the Middle Ages and Renaissance. Medieval Institute Publications, Western Michigan University.

[45]

4Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic Books (AZ).

Digital Library

[46]

4T Nunn. 1998. Wisdom of the Impulse On the Nature of Musical Free Improvisation Part 2. Pdf julkaisu International Improvised Music Archive.

[47]

4William Odom, Ron Wakkary, Youn-kyung Lim, Audrey Desjardins, Bart Hengeveld, and Richard Banks. 2016. From Research Prototype to Research Product. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16.

Digital Library

[48]

4Nam June Paik. 1965. Zen for Film.

[49]

4Nam June Paik. 1969. Participation TV.

[50]

J Pressing. 1988. Improvisation: methods and models. John A. Sloboda (Hg.): Generative processes in.

[51]

5Geber Ramalho and Jean-Gabriel Ganascia. 1994. Simulating creativity in jazz performance. In AAAI, 108--113.

Digital Library

[52]

5Matt Ratto. 2011. Critical Making: Conceptual and Material Studies in Technology and Social Life. The Information Society 27, 4: 252--260.

Digital Library

[53]

5Horst W J Rittel and Melvin M Webber. 1973. Dilemmas in a general theory of planning. Policy sciences 4, 2: 155--169.

[54]

5Francis. Rumsey and Tim. McCormick. 2006. Sound and recording: an introduction. Focal Press.

[55]

5R. K. Sawyer. 2006. Group creativity: musical performance and collaboration. Psychology of Music 34, 2: 148--165.

[56]

5R. Keith Sawyer. 2000. Improvisation and the Creative Process: Dewey, Collingwood, and the Aesthetics of Spontaneity. The Journal of Aesthetics and Art Criticism 58, 2: 149.

[57]

5R. Keith (Robert Keith) Sawyer. 2011. Structure and improvisation in creative teaching. Cambridge University Press.

[58]

5Lucy Suchman. 1987. Plans and Situated Actions. Cambridge University Press: 224.

Digital Library

[59]

5John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '07, Paper 41.

Digital Library

[60]

John Zimmerman, Erik Stolterman, and Jodi Forlizzi. 2010. An analysis and critique of Research through Design. In Proceedings of the 8th ACM Conference on Designing Interactive Systems - DIS '10, 310.

Digital Library

[61]

6Adversarial Design. Retrieved September 17, 2017 from http://dl.acm.org/citation.cfm?id=2341007

[62]

6Festival de Cannes: The Mystery of Picasso (In Competition). festival-cannes.com.

[63]

6Critical Studies in Improvisation. http://www.criticalimprov.com/

[64]

6The Electric Golem. http://levelgreen.com/electricgolem/

[65]

6Annie Lewandowski (Powerdove). http://www.annielewandowski.com/

Cited By

Dunn KShklovski IBjørn P(2024)What Research through Art can bring to CSCW: exploring ambiguous futures of worki-com10.1515/icom-2023-003823:1(33-55)Online publication date: 22-Apr-2024
https://doi.org/10.1515/icom-2023-0038
Barbareschi GZhou SRyoichi AKawaguchi MArmstrong MOgino MAoiki SOhta ETaguchi HKamiyama YMuto MYoshifuji KMinamizawa K(2024)Brain Body Jockey project: Transcending Bodily Limitations in Live Performance via Human AugmentationProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675621(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675621
Dong KZhang ZChang XChirarattananon PLC R(2024)Dances with Drones: Spatial Matching and Perceived Agency in Improvised Movements with Drone and Human PartnersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642345(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642345
Show More Cited By

Index Terms

Intermodulation: Improvisation and Collaborative Art Practice for HCI

Recommendations

Using empathy to research creativity: collaborative investigations into distributed digital textile art and design practice
C&C '07: Proceedings of the 6th ACM SIGCHI conference on Creativity & cognition

This paper describes the use of practice-based distributed collaborative investigations to examine ways in which digital technology can support creative visual art practice. The development of artworks through digital collaboration has enabled empathy ...
Heterogeneity in harmony: diverse practice in a multimedia arts collective
GROUP '05: Proceedings of the 2005 ACM International Conference on Supporting Group Work

HCI and CSCW researchers have begun to call for greater and more explicit support of creative endeavors. Current theories of creativity suggest that it is an inherently collaborative activity, situated and highly contextualized. This work argues that a ...
Digital tools in collaborative creative work
NordiCHI '18: Proceedings of the 10th Nordic Conference on Human-Computer Interaction

The aim of the workshop is to examine and discuss the role and functions of digital tools in collaborative creative work. While advancements in Human-Computer Interaction (HCI) research offer considerable potential for supporting creative work practices,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
678
Total Downloads

Downloads (Last 12 months)96
Downloads (Last 6 weeks)16

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dunn KShklovski IBjørn P(2024)What Research through Art can bring to CSCW: exploring ambiguous futures of worki-com10.1515/icom-2023-003823:1(33-55)Online publication date: 22-Apr-2024
https://doi.org/10.1515/icom-2023-0038
Barbareschi GZhou SRyoichi AKawaguchi MArmstrong MOgino MAoiki SOhta ETaguchi HKamiyama YMuto MYoshifuji KMinamizawa K(2024)Brain Body Jockey project: Transcending Bodily Limitations in Live Performance via Human AugmentationProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675621(1-14)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675621
Dong KZhang ZChang XChirarattananon PLC R(2024)Dances with Drones: Spatial Matching and Perceived Agency in Improvised Movements with Drone and Human PartnersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642345(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642345
Arawjo ISwoopes CVaithilingam PWattenberg MGlassman E(2024)ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis TestingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642016(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642016
Jiang JAhmadpour N(2023)POVR: Designing an Immersive Experience for Generating Moments of Self-reflection at HomeProceedings of the 35th Australian Computer-Human Interaction Conference10.1145/3638380.3638382(324-339)Online publication date: 2-Dec-2023
https://dl.acm.org/doi/10.1145/3638380.3638382
Batista NMonteiro I(2023)How HCI concepts are used in articles featuring interactive digital arts: a literature review.Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638116(1-12)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638116
Carrera DBhattacharjee GSoden R(2023)"We’re Not Decorators": Fostering Interdisciplinary Exchange in STEM–Artist CollaborationsProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595973(1398-1410)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595973
Subbaraman BShim SPeek N(2023)Forking a Sketch: How the OpenProcessing Community Uses Remixing to Collect, Annotate, Tune, and Extend Creative CodeProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595969(326-342)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595969
Han JChernyshov GSugawa MZheng DHynds DFurukawa TPadovani Macieira MMarky KMinamizawa KWard JKunze K(2023)Linking Audience Physiology to ChoreographyACM Transactions on Computer-Human Interaction10.1145/355788730:1(1-32)Online publication date: 7-Mar-2023
https://dl.acm.org/doi/10.1145/3557887
Bang TFdili Alaoui SSchwartz E(2023)Designing in Conversation With Dance PracticeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581543(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581543
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents