Evan Lenz

My MIDI-to-USB adapter arrived via UPS today, so I hooked it up to the Roland RD-600 that I’m borrowing from my wonderful neighbors. I’m pleased to see that it works right out of the box and I only needed to make one small modification to my code to make (what’s implemented of) the video game to work. (Whereas previously it was only expecting MIDI note off events to appear as “note on” with volume 0, now it will handle explicit “note off” events as well.)

(Please forgive the low recording quality. I was using one hand to hold the camera and one hand to play the keyboard.) As you can see, there’s a fairly significant delay between the sound (which is pretty immediate) of me striking a key and the visual (green or red) note response that appears on the screen. This is something I have to work on before the game will be close to release-worthy.

Now that I have a functioning MIDI keyboard, I’m really motivated to get some real music in there (instead of these hard-coded notes that rise up the screen).

Over Christmas break, I got inspired to work on the Klavarskribo game some more. The video below shows the color coding I mentioned in my last post. When a currently-played piano key overlaps with a supposed-to-be-played piano key, the note is green for the duration that it’s supposed to be played. If you play a wrong note or if you hold the note for longer than it’s supposed to be played, it will be red. Any supposed-to-be-played notes that you don’t play will simply be black, until you hit them and then they’ll turn green. So the object of the game, at least as currently realized, is to make things as green as possible.

You’ll notice that the notes rising on the screen (which include an ascending chromatic scale and some random chords) have nothing to do with the live MIDI input. It’s as if the user is ignoring the notes on the screen and instead playing a Chopin waltz. So you end up seeing mostly red notes with a couple of green notes where the two happen to line up. That’s because I don’t yet have an actual MIDI keyboard hooked up. I’m waiting for a MIDI-to-USB adapter to come in the mail on Monday…

I still don’t have a name for this game I’m working on, so I’m not sure how to refer to it yet. Also, I’m still on the inept side when it comes to video publishing, so this first offering is of pretty low quality.

What this shows is live MIDI input (from a keyboard, eventually). What it doesn’t show is the rising notes of a score, which I’m still working on. This is an early video, so things have already changed quite a bit from what you see here. But at least it conveys some idea of what it will look like (piano keyboard at the top of the screen, Klavarskribo groups of 3 and 2 vertical lines down the screen). I’ve since removed the stems from the live (currently-being-played) notes, but they will remain on the notes that rise up the screen. As with Dance Dance Revolution’s arrows, when a note hits the top of the screen, that’s when you’re supposed to play it. I’ve also added color-coding to the notes: red for wrong note, green for correct note, and black for missed note (not shown in the above video). The aim is to provide immediate visual feedback as to how closely you’re matching the prescribed notes as they hit the top of the screen.

Steven Wilson commented on my last blog post. I went to his site and found a remarkably similar thing that he’s working on, along with a video showing the system in action. It doesn’t use Klavarskribo notation; it looks more like Synthesia’s vertical bars (with length indicating note values). But unlike Synthesia (and like what I’m doing), the keyboard is at the top of the screen (as opposed to the bottom of the screen). The link to the video on Steven’s page appears to be disabled at the moment though.

In my almost-daily scouring of the Web for Klavarskribo-related information (in English, as I don’t yet speak Dutch), I came across a thread on the Cakewalk forums where Johannes Drinda was proposing a Klavar-like notation for MIDI editing. (I’m not a Cakewalk user, so I don’t really know the context.) Anyway, I admired this whistling philosopher’s persistence and enthusiasm, so I emailed him.

In his response, he pointed me to the Janko keyboard design. Quoting him (with his permission): “What a shame, that true revolutionary notions, such as Klavarskribo and Janko keyboard pattern have almost vanished. They are indeed invaluable tools to create a system, which offers/ enables any musically minded (hobby/ home) musician to play MIDI instruments the easiest way.” So he shares my passion to reduce the cognitive overhead that’s built into conventional notation (and/or instruments). He also has created instructions for a DIY Janko keyboard overlay project (PDF file), which I found quite impressive.

That led me to some YouTube videos about the Chromatone, which is based on the Janko keyboard. Pretty interesting stuff. If nothing else, it helps situate my understanding of the traditional piano keyboard. But I’m keeping my mind open to what role it might play in my investigations. I’m also keeping an eye on what Thumtronics is doing, and I subscribed to its founder’s blog, which has a bunch of fascinating reading.

If you have experience with Klavarskribo or are particularly interested in it, please contact me. I’m hoping to get feedback once I’m able to get an initial release of this game out, and more ideas and suggestions in the meantime.

This alternative musical keyboard layout is rockin’ my world. I will post more about it soon.

I’ve been steadily learning to read and play piano music in Klavarskribo notation. I still feel like a beginner but the pace of learning is good. New observations are coming fast and furious.

From the start, I wanted to have a computer game to teach me Klavarskribo interactively a la Dance Dance Revolution. There are some games out there that come close: Piano Wizard and Synthesia. I even crashed the Synthesia forums to see if Nicholas Piegdon would consider adding support for Klavarskribo notation. He was very gracious, but he’s also got his plate full with lots of feature requests, including support for displaying traditional notation.

I couldn’t wait, so I’ve started to cobble something together, working on it on and off over the last couple of months. It’s not close to being ready to release. It’s more of a test to help me visualize the particulars and assess its feasibility. So far, I’m encouraged that I’m on the right track.

Here’s a screenshot of an early version:

I will post a video once I figure out the best way to do that using WordPress and/or YouTube. Also, I plan to blog more about this throughout the rest of this first month of 2008.