Pure Data, Android audio, and random stuff

Noisepages: Websites for smart artists.

More pointlessness

by Peter Brinkmann

After I got my Arduino to talk to a Wiimote for no reason at all, I felt bothered by the fact that the communication only went one way, from the Arduino to the Wiimote. An old blog post shows that bidirectional communication is possible, at least one byte at a time, but sadly the sample code seems to have disappeared. Motivated by this post, I sat down and took another look at the communication between Wiimote and Arduino, and it turns out that it’s perfectly straightforward to make sense of the bytes that the Wiimote sends to the Arduino in response to writes to the Nunchuk register range on the Wiimote. The latest revision of my Wiimote-Arduino library includes support for bidirectional communication. Messages from the Arduino to the Wiimote are always six bytes; messages from the Wiimote to the Arduino can be longer but will be broken into chunks of at most seven bytes delivered at a time. I still have no idea what this might be good for.

1

When I started working on MIDI and Bluetooth, my first thought was to connect an Arduino to a MIDI device on one side and to a Wiimote on the other, using the Wiimote as a cheap and easily available Bluetooth module. That was a brain-charred idea, for many reasons, and I abandoned it soon after I thought of it, but the damage was done: I had already purchased the necessary pieces. Now they were sitting on my shelf, looking at me accusingly. I had to do something with them.

I expected this to be a trivial job because Wiimotes and Nunchuks came out four years ago and were reverse engineered soon after that. Surprisingly, though, getting my Arduino to talk to a Wiimote and a Nunchuk took a few hours of work and much head-scratching, and so I figure it’s worthwhile sharing my experiences even though I’m way late to the party.

In a nutshell, here’s what I found:

  • People have come up with various ways of plugging into the nonstandard connector of the Wiimote expansion port, such as using FireWire plugs, cannibalizing Nunchuks, and custom PCBs, but I haven’t found any reference to what I consider the best solution: Get a Nunchuk extension cable and cut it in half.
  • A number of posts on Arduino and Wiimotes/Nunchuks tell you to modify twi.h and rebuild the Arduino development environment. With recent versions of the Arduino software, this is no longer necessary. In fact, the Arduino won’t talk to the other pieces if you change twi.h.
  • There are a few Wiimote libraries for Arduino out there, but none of them really seem to work for me. Some don’t work at all, some seem incomplete, and in one case I had misgivings about timing issues. In the end, I put together my own Wiimote/Nunchuk API, available here. The code includes some documentation as well as instructions for wiring things up.

Now for the hard part: What is this good for?

3
YouTube Preview Image

Here’s a quick demo of the MIDI-Bluetooth interface, showing MIDI events flowing from my digital piano to my phone and vice versa. After the video was done, I started working on an enclosure for the interface, ruining two perfectly good boxes with my shiny new rotary tool before finally figuring it out. Then I went to work on mounting the interface in the box and permanently sealed the power switch in the off position with epoxy. I think I’ll call it a night.

17

Free Day at SparkFun!

by Peter Brinkmann

If you are tempted to build your own MIDI-Bluetooth interface but were deterred by the price tag of the BlueSMiRF Bluetooth modem, here’s your chance to get one at a much more reasonable cost: SparkFun Electronics is having another free day on Thursday, January 13.

In the meantime, I ordered another, much cheaper Bluetooth modem, a KC-21. It’s supposed to be a bit fiddly to set up and it’ll probably require extra circuitry, but I’ll see how far I get with it. Stay turned…

PS: I just took a look at the data sheet of the KC-21, and it looks like it doesn’t support the nonstandard MIDI bit rate. I guess I should have looked at the specs before ordering the thing. Then again, this gives me an excuse to dream up a new Bluetooth project.

0

The software to go with the MIDI-Bluetooth interface (I really need to think of a catchy name for this gadget…) is now available at Gitorious. It’s based on the BluetoothChat sample that comes with the Android SDK, and it consists of an activity for selecting a Bluetooth device to connect to and a service that manages the MIDI connection. I factored out a class for managing Bluetooth SPP connections that may be of independent interest. Client code sends MIDI messages by calling methods of the service, and it receives MIDI messages by implementing a MIDI receiver and registering it with the MIDI service.

In addition to the basic MIDI support, the repository also contains two sample projects that illustrate how to use the MIDI service. Moreover, Pd for Android now includes the btmidi repository as a submodule, and it comes with a utility class that automatically connects the MIDI functionality of libpd to the Bluetooth MIDI service. I put some effort into polishing and documenting the new classes. I’m pretty happy with them by now, but the code is only a few days old and should still be considered experimental. Like the code it is based on, the Bluetooth MIDI service is Apache licensed. Let the hacking begin!

1

MIDI over Bluetooth, new hardware

After I convinced myself that MIDI over Bluetooth is viable, the breadboard approach didn’t seem adequate anymore, and so I fired up ye olde soldering iron and made a more durable version of the MIDI-Bluetooth interface. It even has a power switch and a little power indicator LED. Now I need to find a suitable enclosure. That’s always the hard part. RadioShack sells an enclosure whose size is almost right, except that the dimensions of the circuit board (also from RadioShack) are the outer dimensions of the enclosure. What were they thinking?

4

A 2005 paper on MIDI over Bluetooth by Bartolomeu et al. presents some rather discouraging latency measurements. They report typical delays ranging from 25ms to 275ms, with the odd peak over 500ms, for events sent from the slave device to the master. For events sent from master to slave, the numbers are a little better but still bad, with typical delays between 15ms and 80ms. That’s very bad news, huge latency and terrible jitter, with a side of asymmetry. Their results seem sound, and their experimental setup indicates that they were really measuring Bluetooth delays and nothing else.

On the other hand, my initial experiences with my own MIDI over Bluetooth setup have been rather promising. I never noticed any major latency, but then again, I haven’t yet used it much, either. Time to investigate.

Since this project is strictly a hobby and I don’t intend to publish the results in any formal way, I figured that crude measurements with a certain systematic error would be acceptable. Here’s the basic setup: Take one MIDI cable and plug one end into the MIDI In port of the Bluetooth adapter and the other one into the MIDI Out port. Now send a MIDI event from the Android phone to the interface and measure the time until the phone gets the event back from the interface.

This approach has all sorts of problems, of course. It only measures round-trip times rather than one way, it relies on the timing capabilities of Java/Dalvik, and it is subject to the whim of the scheduler on the Android side. In other words, it measures a sequence of random delays, and the Bluetooth delays may only be a small part of it. Still, the results show that my initial impression was correct: I’m getting round-trip times between 30ms and 70ms, with a mean of 34ms and a standard deviation of 6ms.

In other words, the round-trip delays plus delays from other sources are much better than the one-way times that the 2005 paper reports. Those numbers are still not great; they probably won’t be good enough for highly sensitive musical applications. Still, they’re small enough to be useful in a lot of settings. I’m wondering where the improvement came from. My best guess is that technology has progressed since 2005; in particular, the BlueSMiRF modem that I’m using is a Bluetooth 2.1 device, while the authors of the paper were using Bluetooth 1.1. In any case, I’m glad I started hacking before I read their paper.

3

MIDI over Bluetooth

A little while ago, I added MIDI support to libpd because CoreMIDI is available in the latest version of iOS. I wasn’t too thrilled about this, not just because I’m no fan of MIDI but also because Android currently doesn’t support MIDI. So, I decided to add MIDI capabilities to Pd-for-Android.

Simply plugging in a USB MIDI keyboard was not an option because stock Android devices don’t support USB host mode without hacks and besides, tethering a phone to a big chunk of hardware seems antithetical to mobile devices. Bluetooth seems like the way to go, but a quick search turned up relatively little useful information, and no commercial interface for transmitting MIDI over Bluetooth. Maybe that means that Bluetooth is not a good medium for MIDI, but I figured I’d give it a try anyway.

The hardware is perfectly straightforward. There’s no shortage of schematics for MIDI interfaces on the web; I followed Lady Ada’s excellent instructions, with a few minor modifications:

  • Instead of an Xbee radio, I used a BlueSMiRF Bluetooth modem.
  • Instead of a 74HC04 hex inverter, I used a CD4049UBE hex inverter because I happened to have one lying around.
  • The BlueSMiRF seemed a bit unstable with four batteries, even though the specs say that it can take an operating voltage of up to 6V. I switched to three AAA batteries and haven’t had any trouble since.
  • I left the shield of the MIDI In socket unconnected in order to avoid ground loops.

One of the great features of the BlueSMiRF modem is that it is easy to set up and capable of handling the nonstandard MIDI bit rate. In order to configure it for MIDI, connect a Bluetooth terminal (such as BlueTerm for Android) to the modem and enter the following incantation:

$$$
SF,1
SZ,128
---

You need to do this within 60 second after switching the BlueSMiRF on. After a reboot the modem will be ready to transmit MIDI messages.

I haven’t yet had the chance to evaluate jitter and latency for this setup. It probably won’t be accurate enough for all musical applications, but I figure it’ll still be useful. At some point, I’ll probably use it for transmitting measurements from a Doepfer Pocket Electronic. For the time being, in any case, this rough proof of concept is working nicely.

12

I recently read You Are Not a Gadget by Jaron Lanier. It’s a strange book, half-baked and infuriating, yet hard to put down. It’s great fun to contemplate the myriad ways in which Lanier’s arguments fall flat. There’s only one major point that I agree with: MIDI is a plague upon humanity. (Note: In this context, I mean MIDI as a way of representing music in software; MIDI as a way of connecting various pieces of hardware is great.)

I had been resisting repeated requests for MIDI support in libpd, but when Apple added CoreMIDI support to iOS 4.2, I finally caved and added basic MIDI functionality to libpd. My excuse is that sooner or later it would have happened anyway, and besides the goal is to create a tool that’s useful to many people, not a vehicle for my worldview.

The new functionality mirrors the message passing mechanism that has been part of libpd from the beginning: There are functions for sending MIDI events to libpd, and if you want to receive MIDI events from libpd, you implement handler functions and assign them to the appropriate hooks in libpd. Exchanging MIDI events with other components such as CoreMIDI is the responsibility of the client code. The new functionality is documented in the wiki.

I hope that developers will not have to support MIDI in their software in the not too distant future. I just made a small contribution to the entrenchment of MIDI in audio software. I’m experiencing a bit of cognitive dissonance right now.

11

Good news — the HTC Legend bug I wrote about earlier has apparently been fixed in Android 2.2 – FroYo. This is a relief since I really had no idea how to straighten this out without an HTC Legend to experiment with. Fortunately, some problems go away if you just ignore them long enough.

1