FIELD KIT · EMBODIED & POETIC INTERACTIONS

Tools for Designing With the Body

A practical reference for artists and designers, not engineers — what to buy, what to learn, and who's already doing this work. ~12 min read.

You don't need an engineering degree to start prototyping embodied and poetic interactions. Most of the tools below were built specifically for artists, and the learning curve for a first working prototype is closer to a weekend than a semester.

Physical Computing Basics

Physical computing means connecting sensors and actuators to a small computer so a piece of software can sense and respond to the physical world. This is the foundation almost everything else in this kit builds on.

ToolWhat It's ForGood First Project
Arduino (Uno / Nano)The classic beginner microcontroller — huge community, endless tutorials, cheap.A pressure sensor under a rug that triggers a sound when stepped on.
ESP32Like Arduino but with built-in Wi-Fi/Bluetooth — good once you want a prototype to talk to a phone or laptop wirelessly.A wearable that vibrates when a friend across the room does the same gesture.
Raspberry PiA full small computer, useful when your piece needs to run video, audio, or a local web interface.An installation that plays a different video loop depending on how close someone stands.
Adafruit / Bela / TeensySpecialized boards for audio-reactive or ultra-low-latency embodied work.An instrument where a physical gesture directly shapes a synthesized sound in real time.

Movement & Motion-Sensing Tools

These are the tools that let a project actually feel the body — not just a button press, but posture, gesture, gait, or presence in space.

  • Azure Kinect / Kinect v2 — full-body skeletal tracking without wearables; the standard entry point for movement-based interactive art.
  • MediaPipe (Google) — free, runs in a browser or on a laptop webcam, tracks hands, pose, and face without any special hardware.
  • IMU sensors (accelerometer/gyroscope) — small, cheap chips that detect tilt, shake, and orientation; the core of most wearable and alternative-controller projects.
  • Pressure and flex sensors — detect weight, grip, or bend; used in floor pieces, wearables, and soft interfaces.
  • Leap Motion / Ultraleap — precise hand tracking at close range, good for tabletop or desk-scale gestural pieces.

Tangible Interaction Materials

The physical "skin" of a project matters as much as its electronics. These materials are common in tangible and poetic interaction work because they carry meaning on their own, before any sensor is added.

  • Conductive fabric and thread — lets you sew a working switch or sensor directly into cloth, useful for wearables and soft objects.
  • Copper tape — the fastest way to prototype a touch sensor on paper, wood, or cardboard.
  • Shape-memory alloy (Nitinol) wire — contracts when heated with a small current, letting objects move without motors.
  • Sand, water, sound, and light — the oldest "materials" of poetic interaction; an hourglass or a wind chime is a fully realized interface with no electronics at all.
  • Clay, felt, and found objects — for fast physical prototyping before committing to a final material.

Software for Interactive Art

SoftwareBest For
TouchDesignerNode-based visual programming for real-time, sensor-reactive visuals and installations.
Max/MSP (or Pure Data)Patch-based audio and interaction programming, popular for embodied musical instruments.
Processing / p5.jsCode-first creative programming, great for artists learning to program for the first time.
openFrameworksC++ toolkit for advanced, performance-heavy interactive installations.
Unity / Unreal (with sensor plugins)When a project needs full 3D environments alongside body tracking.

Artists & Studios Doing This Work

Looking at existing practice is often more useful than any tutorial. A short, non-exhaustive list to start with:

  • Rafael Lozano-Hemmer — large-scale interactive installations using heartbeats, shadows, and crowds as raw material.
  • Random International — creators of Rain Room, where the installation tracks bodies to withhold rain from anyone standing beneath it.
  • teamLab — immersive, body-responsive digital environments blending movement, light, and collective presence.
  • Lauren McCarthy — critical, often unsettling explorations of surveillance and human-computer intimacy.
  • Camille Utterback — pioneering work in movement-driven generative visuals, bridging dance and computation.
  • Studio Roosegaarde — poetic public installations that respond to presence, weather, and social behavior at urban scale.

Where to Practice

If you're new to all of this, start with the Activities page — the exercises there are designed to be doable with nothing more than the materials and software listed here, before you invest in specialized hardware.

Menu