The Listening Koto
A playable koto · a machine player that listens

The Listening Koto

A thirteen-string koto you can play, with a machine player that listens to the room. When someone whisks tea, pours water or speaks, it goes quiet, then answers in the gap; when the kettle sings, it plays low and soft beneath it. A tea room already has a sound design: the kettle, water, a gong, the host's hands. The player is built to join that family, as one more sound of the room rather than a performer in it. Play it below, then read what koto music told its listeners across twelve centuries, and how the player listens and yields.

Tap a string to the right of its bridge to pluck it, or drag across strings to sweep them. Hold to the left of a bridge to press that string up a step (oshide), the way a player's left hand does. Strings are named 一 to 十, then 斗 為 巾, from the far side of the instrument to the near side.

MACHINEIdle. Let the machine play, or play the strings yourself.

The player

Jo-ha-kyū

How it breathes

Silence between phrases. Longer still in jo, and under the kettle.
Left-hand colour after the pluck: press, pull, shake. This is where variability lives.
Turn it down to hear the difference: at zero it plays through the room like a music box.

The room

These are synthesized stand-ins for what a room microphone would hear. The machine yields while they sound, then answers in the gap. The kettle is different: it is a drone the koto plays underneath.

The evening's score, read right to left like a koto score
machineyouroom・ ma
Notes, silences and room sounds appear here as columns of tablature: which string, and a small mark for what the left hand did.
釜The room

The tea room already has instruments

A formal tea gathering has no musical program. It does have sounds the host prepares for, and each one is tied to an action or to a threshold in the gathering. That is the family the koto should join.

釜
The kettle · matsukaze

When the water in the iron kettle comes up to the boil, its hiss is called matsukaze, wind in the pines. Guests listen for it. It is low and continuous, and it changes as the fire changes.

水琴窟
Water koto cave · suikinkutsu

An upturned pot buried by the stone basin where guests rinse their hands before tea. Water drips through a hole and rings inside like a koto, which is where the name comes from. Tea master Kobori Enshū is often credited with inventing it.

銅鑼
The gong · dora

At the end of the break in a formal gathering, the host strikes a gong to call guests back from the garden for thick tea. A sound that marks a threshold and then decays to nothing.

The design precedent is the suikinkutsu, not the concert stage.

It is already an automatic koto, played by the guests' own actions, placed at the threshold of the tea garden. Nobody performs it and nobody applauds it. The machine player above is built to take the same standing.

A proposed sound map for one gathering

A starting point for a tea practitioner to overrule. The roles match the machine's behaviours above.

待合1
Waiting room
Leads

A slow jo while guests gather. The only stretch where the koto may set the mood.

蹲踞2
Rinsing hands at the basin
Answers

Single notes in reply to water, like the water koto cave.

初座3
Charcoal and meal
Under

Rare, low and soft beneath the kettle. Silent while the host speaks.

中立4
Break in the garden
Leads

The room empties. The koto moves into ha, carried out to the garden.

銅鑼5
The gong
Marks

Stops before the gong and lets it decay. The gong is the threshold; the koto only clears the air for it.

濃茶6
Thick tea
Silent

The most concentrated moment of the gathering. Nothing.

薄茶7
Thin tea
Answers

Lighter, conversational. Tegoto behaviour: answers in the gaps, a little brighter.

退席8
Departure
Closes

A brief kyū and one long after-sound as the last guest leaves.

時代Periods

What koto music told its listeners

Across twelve centuries the koto told surprisingly few stories in the Western sense. What changed was where listeners stood and what the instrument was for. Each period ends with what it suggests for a machine player, and its button plays a gesture in that spirit through the koto above: gestures, not reconstructions of the pieces.

8th–12th c.Heian court雅楽 · 楽箏

The koto behind the blinds

In the court orchestra, gagaku, the koto is not the melody. It repeats short arpeggio patterns that mark time under the winds. In court literature such as The Tale of Genji, zithers are often heard before their players are seen: at night, through blinds and screens, a sign that someone refined is present.

How it was heard
Overheard, from outside a screen. Listening was a way of sensing a person.
For the player
Better overheard than watched. Marking time under other sounds is a respectable role, not a lesser one.
Late 16th c.Tsukushi-goto筑紫箏

Music as a path

Kenjun, a Buddhist priest in northern Kyushu, shaped a repertoire of songs with koto drawing on court music. Priests and gentry cultivated it as personal refinement, in the same decades that Sen no Rikyū was turning tea into a path of its own.

How it was heard
Small rooms, few listeners. The player's discipline mattered as much as the listener's pleasure.
For the player
The tea room's nearest musical cousin. A single line, long notes, nothing to prove.
1614–1685Yatsuhashi Kengyō八橋検校 · 段物

Form is the story

A blind musician, Yatsuhashi brought the koto to professional players, adapted the hirajōshi tuning from shamisen music, and wrote the instrumental pieces at the heart of the repertoire. Rokudan is six sections with no words and no program: a study of one melody, quickening as it goes. Midare, whose name means disturbance, deliberately has neither fixed stops nor a set number of measures. For about two centuries afterwards the professional tradition lived inside the guild of blind musicians, taught by ear and by touch.

A contested and lovely idea: musicologist Tatsuo Minagawa has argued that Rokudan descends from the Latin Credo that missionaries brought to Japan in the 16th century, and in 2012 the two were performed together in Rome. The most abstract koto piece may carry a prayer inside it.

How it was heard
Lessons and merchant households. Form, variation and pace were the content.
For the player
A fixed skeleton plus a licensed disturbance gives variety that still sounds like itself; raise iro and the danmono gesture breaks into a midare phrase. A machine player should work the way the kengyō played: by touch and by ear, not by sight.
c. 1690–1820Sarashi, Ikuta, Yamada晒 · 生田流 · 山田流 · 手事

The instrument speaks between the words

Sarashi, from Fukakusa Kengyō in the Genroku era, is about ordinary people bleaching cloth in a river, and its busy repeated figures sound like work. Ikuta Kengyō's school in Kyoto and Osaka joined the koto to shamisen songs, and that lineage grew tegoto: long instrumental passages between sung verses where the instruments talk among themselves. Later, in Edo, Yamada Kengyō's school leaned toward the voice and dramatic storytelling.

How it was heard
Parlours, lessons, the pleasure quarters. Love, seasons, work.
For the player
Tegoto is the player's interaction model. The host, the kettle and the guests have the words; the koto plays in the gaps they leave.
1855Yoshizawa Kengyō II吉沢検校 · 千鳥の曲

Looking back to the court

In Nagoya, Yoshizawa set poems from the tenth-century Kokinshū anthology and used a tuning adapted from gagaku, reaching back past two centuries of shamisen-era music. His Chidori no kyoku imitates the cries of plovers and the sound of waves through left-hand technique, such as a pressed string released into a short falling tone. Some musicians shunned him for it; the piece spread across Japan in the Meiji era and is still loved.

How it was heard
As nostalgia. A late-Edo audience hearing an imagined Heian.
For the player
Feeling arrives through shared references (a poem, a bird, a season) more reliably than through expressing a mood. The host chooses those references for every gathering, and a player should read them.
1920s–1950sMiyagi Michio宮城道雄 · 新日本音楽

The koto paints

Blind from early childhood, Miyagi started the New Japanese Music movement in 1920 with Yoshida Seifū, brought in Western forms like theme and variations, and invented the 17-string bass koto in 1921. Haru no umi, The Sea in Spring, for koto and shakuhachi, paints a seascape, and in a violin arrangement it reached Western concert halls.

How it was heard
Concert halls, radio, records. Music as a picture to look at.
For the player
Take the warmth and skip the picture. Depiction pulls attention toward the instrument, and in a tea room attention belongs to the host's hands.
1969–nowNosaka, Sawai, Masaoka二十絃 · 現代

Sound as sound

With composer Minoru Miki, Keiko Nosaka introduced a 20-string koto in 1969 and a 25-string one in 1991. Tadao and Kazue Sawai pushed the technique and the stage presence of the 13-string instrument. Composers now write for scraped strings, harmonics, clusters and silence, and improvisers carry the koto into jazz and electronics. Miya Masaoka's Laser Koto fitted the instrument with light sensors and a computer.

How it was heard
Everywhere. The instrument opens to other musics and to machines.
For the player
Interactivity has precedent. Contemporary technique also offers sounds a tea room might welcome: a single harmonic, a slow scrape, nothing at all.
間Attention

Engineer attention, not emotion

A machine built to engineer emotion needs a model of feelings and a controller that steers listeners toward them. This repertoire suggests a different target. Its feeling comes from three levers, and a machine can hold all three.

序破急

The shape of time

Jo-ha-kyū: an unhurried opening, a development, a quick close. Rokudan quickens section by section, and a gathering has its own arc over hours. The player keeps one time-shape and applies it at the scale of a phrase, a piece and an evening. The Auto setting above runs it.

余韻

The after-sound

The right hand starts a note. The left hand, pressing, pulling or shaking the string beyond the bridge, decides what the note becomes. Most of the koto's expressive range lives in that second gesture, which is why the iro slider changes the character so much more than the notes do.

見立

Shared reference

Poems, birds, seasons. The host already chooses a scroll, flowers, sweets and the poetic names of the utensils for each gathering. That set of choices is the player's brief, and it keeps the koto saying the same thing as everything else in the room.

What to avoid: mood targets on valence and arousal axes, builds to a climax, ambient wallpaper that never stops, and pictures in sound.

聴The player

How the player listens and yields

The machine above is a model of a player for a real tea room. It plays from rules rather than recordings, and it gives every other sound in the room the right of way. Here is how it decides when to play, and what a physical version would need.

Listen, yield, answer, breatheThe rules

Four rules run in real time. The player listens for the room. While whisking, pouring or conversation is sounding, it yields: the phrase it was playing stops where it is. When the room goes quiet it waits a breath, then answers in the gap with something shaped by what it heard: the whisk's own rhythm, fading; a short falling run after the water; two low notes after conversation, like a nod. Then it breathes, leaving a silence set by ma before the next phrase.

Two sounds are treated differently. The gong is a threshold, so the player stays silent until it has decayed, and in Auto it begins a new jo after. The kettle is a drone it plays under rather than around: while matsukaze sings, phrases move to the lower strings, soften, spread further apart and come home to 五.

Try itTurn Listening down. Around the middle the player still yields but no longer answers; near zero it plays straight through the room like a music box.

A slow mind, fast handsTwo clocks

Those rules are the fast hands, and they are simple on purpose, because they have to react within a fraction of a second. Above them sits a slow mind that reads the gathering's references and writes the skeleton for each phase ahead of time, as tablature a person can review. A language model is a good fit for the slow mind and a poor one for the hands: it is too slow for the note-by-note loop, and too eager to say something. On this page the skeletons come from hand-written phrase rules, one set per gesture.

TradeoffLess spontaneous than a model improvising live, and far more predictable in a room where one wrong sound costs a great deal.

Scores in tablatureScore language

Each event says which string (一 to 巾), which technique, how hard, and how much silence comes before it. Koto notation already works by string number, so the machine's score stays readable by a koto player, by guests on a screen, and by a language model drafting it. Silence is an event with a duration, not a gap in the data. The score under the instrument is written this way as the evening unfolds.

TradeoffIt gives up the convenience of MIDI tooling, and gains a score that the people in the room can read and correct.

Three layers of freedomVariability

The skeleton is fixed per phase, like a dan. Ornaments are chosen by chance within the idiom. Timing comes from the room. In a tea room each gathering would be seeded from its date and guests, played once and never replayed, so that ichi-go ichi-e (one time, one meeting) is a property of the system rather than a slogan. Here the seed comes from today's date, and the New gathering button above reseeds.

TradeoffA gathering you liked cannot be repeated, unless you keep the recording rather than the seed.

Ears for itself and for the roomListening in a real room

In the browser, the room buttons tell the player what it would have heard. In a tea room, two microphones would. A contact microphone on the body lets the koto hear itself and correct its pitch bends; strings and movable bridges drift with humidity, so a press that worked at setup will not land the same at midnight. A room microphone only needs to tell apart a handful of sounds: kettle, whisk, pouring, voices, gong, silence. A small classifier on loudness and spectrum, trained in the room itself during setup, is enough.

TradeoffA general audio model would cope with surprises, but it is heavier, slower to react, and harder to trust than six sounds learned in place.

Two hands, unequal budgetsMechanism

A physical player needs two hands, and they deserve unequal budgets. The right hand can be thirteen fixed pluckers, one per string, each a quiet servo swinging a tsume-shaped pick: reliable, cheap and accurate to a few milliseconds. The real engineering belongs in the left hand: two or three force-controlled pressers on a rail that push the string beyond its bridge. That is what makes oshide, ato-oshi and yuri-iro possible. Felt dampers complete it, because silence needs an actuator too.

TradeoffSolenoids are faster and cheaper, but they click, and a tea room is quiet enough that the mechanism's own noise becomes part of the music. That self-noise has to be heard in the real room before anything else is settled.

Borrow ma from a teacherLearning

The random numbers in the simulation are placeholders. Their honest source is a koto player playing to the same room through the same two microphones: gap lengths, phrase lengths and ornament rates fitted from that. The machine's silence should be a person's silence, measured.

TradeoffIt needs a willing player and an afternoon. It is also the most respectful way to bring a machine to the tradition.

From this page to a tea room, in six steps

  1. Play from a written score

    Tablature in, sound out. Proves the hands, the timing and the noise budget.

  2. Teach it one iro

    One left-hand technique, with ato-oshi a good first choice, checked by its own contact mic.

  3. Give it ears

    A room classifier for kettle, whisk, water, voices, gong and silence, trained in place.

  4. Yield and answer

    The tegoto rules from the simulation, running on real sensors and real actuators.

  5. Write from the scroll

    The slow mind: the gathering's references in, one phase of tablature out, reviewed by a person.

  6. Learn ma from a player

    Timing and ornament rates fitted from a koto player recorded in the same room.

Nearest prior art

I found no well-documented robot for the 13-string koto itself in a quick search, so this is fairly open ground. The closest relatives:

  • Robotic Taishogoto (NIME 2014): a robotic version of the keyed Japanese zither, built for performance, installations and teaching. Paper
  • Shimon: a marimba-playing robot that takes cues from musicians and improvises with them. Report
  • Kyoto University's thereminist robot: a humanoid that plays theremin alongside people, using a pitch model to stay in tune when bodies move nearby. Report
  • Laser Koto (Miya Masaoka): laser beams and light sensors on a koto, driving a laptop of recorded koto sounds. NPR
問Open

Open questions

  • Visible or hidden?

    A mechanical hand is theatre: guests will watch it, and attention leaves the host. Hidden mechanics make it a suikinkutsu. A visible but very still mechanism sits between the two.

  • Inside the gathering, or around it?

    The sound map above is a proposal. A practitioner may want the koto silent in the tea room entirely and heard only in the garden and the waiting room.

  • Calm a busy room, or mirror it?

    Outside the tea room, should the koto give a busy space a slower clock to fall into, or reflect its energy back at it?

  • Who tunes it?

    Setting the thirteen bridges by hand before each gathering could become the host's ritual, or the machine's own visible routine.

  • Whose tea?

    Which school and which practitioner set the rules will change what is appropriate. Their blessing is worth more than any feature.

  • Is there a koto player in the room?

    A player teaching the machine is the best data source there is, and the most respectful framing for the whole project.

典Sources

About the simulation: notes are synthesized in your browser with a plucked-string model, and pitch bends are applied after the pluck, as a left hand would. Tunings are simplified: hirajōshi on D, and a semitone-free tuning standing in for the court and Kokin tunings. Room sounds are rough stand-ins. Phrase rules are illustrative placeholders for what a real player would teach.