I have long been expecting that the appearance of libpd would motivate a general refactoring of Pd that untangles the audio engine from other components, such as user interface and audio drivers. I consider such a refactoring a prerequisite for future maintenance and development, but I always assumed that it would have a new audio library at its core, and that this new library would eventually replace libpd.

Recent developments around libpd, however, have me wondering whether this refactoring of Pd might involve libpd itself. To wit, Dan Wilcox recently posted PdParty, a partial iOS port of the Pd GUI that’s based on libpd. PdParty follows in the footsteps of Chris McCormick’s PdDroidParty, which provides a partial Android port of the Pd GUI.

For the time being, neither PdDroidParty nor PdParty will let you create or edit patches. I suspect that libpd in its current form is not general enough to support the full functionality of the Pd GUI, but I don’t think it would take much work to retrofit libpd for this purpose.

So, given these developments, I see a new transition path to a general overhaul of Pd:

• Expand the libpd API to support the needs of the GUI.
• Build a new GUI on top of libpd.
• Retire the old GUI and revise the original core of Pd to remove everything that’s not immediately related to audio processing.

This seems like a realistic approach. It will allow for a smooth transition because the current version and the new version will peacefully coexist on top of the same core functionality. Every step will be a modification of components that already exist, and developers won’t have to learn a whole new API. Besides, it’s already happening.

I’m happy to announce my upcoming book on libpd! The title is “Making Musical Apps”, and it’ll be published by O’Reilly. The project is moving along nicely; it’s a few weeks ahead of schedule, and it looks like we’ll be able to release the book in February, as planned. You can pre-order a copy here.

I attended the Pure Data Convention in Weimar last week. (I presented a paper on libpd.) The convention was an awesome event, with lots of great talks, workshops, and performances. It was great to catch up with people I’ve known for years and to meet new people as well. Pure Data attracts an amazing community.

After my talk, Albert Gräf suggested an alternative C API for sending compound (i.e., list or typed) messages to Pd, and I’ve been thinking about that these past few days. Arguments of compound messages are arrays of type t_atom, a Pd-specific data type that can hold floats or symbols. The original approach of libpd was to provide a set of simple functions that take float or string parameters and implicitly assemble an array of type t_atom. Albert’s suggestion was straightforward enough, just add a couple of functions that allow developers to send compound messages to Pd using their own t_atom arrays, but at first glance it didn’t seem like a good fit because one of the goals of libpd is to protect developers from Pd-specific data types.

libpd largely succeeds in this goal, except for two callback functions that receive compound messages from Pd and take a parameter of type t_atom*. There’s no way to avoid custom data types here because the purpose of this parameter is to pass a heterogeneous list, but it still bothered me because extracting values from instances of t_atom requires some knowledge of the internals of Pd. Another aesthetic concern was the asymmetry between sending and receiving messages — sending messages involves calling functions in libpd, while receiving messages involves reaching into the guts of Pd to interpret t_atom.

That was the state of affairs until a few days ago, when I started thinking about Albert’s suggestion. The first step was to add two functions to the API:

int libpd_list(const char *recv, int argc, t_atom *argv);
int libpd_message(const char *recv, const char *msg, int argc, t_atom *argv);

The first one sends a list to a receive symbol in Pd, the other one sends a typed message. There’s no way around using t_atom arrays here: If you want to be able to manage your own arrays, you need to have access to t_atom, or else memory allocation and pointer arithmetic won’t work. The next job was to figure out how to populate those t_atom arrays. Pd comes with a couple of macros for writing to instances of t_atom, and I decided to hide those behind libpd-style functions:

void libpd_set_float(t_atom *v, float x);
void libpd_set_symbol(t_atom *v, const char *sym);

In other words, if you have an instance of t_atom, you can fill it with a float value or a string value without having to know how t_atom works. At that point I had a minor epiphany: The correct solution is to expose t_atom in the libpd API while treating it like an opaque type.

Now it was clear how to handle t_atom arrays received from Pd: Simply add a few functions (or macros) for evaluating instances of t_atom. With the new additions, reading lists received from Pd is a breeze:

void receive_list(const char *src, int argc, t_atom *argv) {
  int i;
  for (i = 0; i < argc; i++) {
    t_atom a = argv[i];
    if (libpd_is_float(a)) {  // New
      float x = libpd_get_float(a);  // New
      // do something with the float x
    } else if (libpd_is_symbol(a)) {  // New
      char *s = libpd_get_symbol(a);  // New
      // do something with the C string s
    } else {
      // handle unsupported atom type (e.g., pointers)
    }
  }
}

With this new approach, all the complexity of dealing with t_atom is gone. From the point of view of the programmer, t_atom is perfectly opaque, and you work with it through setters, getters, and type checks provided by libpd. At the same time, the compiler has all the information it needs to allocate t_atom arrays and perform pointer arithmetic on t_atom*.

All of this seems blindingly obvious in hindsight, but I didn’t see it before because I was trapped in a rigidly object-oriented mindset. (The thinking went sort of like this: Gotta hide the implementation of t_atom! Can’t hide it? Crud, it’s all exposed. Might as well give up.) Somehow it didn’t occur to me it’s okay to expose the implementation as long as there’s a reasonable API on top of it. I blame Java.

Anyway, I’m very happy with the libpd API as it is now. This minor addition takes care of the last bit that I was worried about. (By the way, in case you’re wondering, the latest revision of libpd is entirely backward compatible with previous versions; it just adds a few functions and macros.) I feel like libpd is truly done now.

New Bluetooth MIDI interface, built by Andrew Tergis

I’m thrilled to report that we’re getting closer to turning the Bluetooth MIDI interface into a kit. Andrew Tergis of Bug Labs designed a really nice printed circuit board, and it works, as this video demonstrates.

My favorite aspect of Andrew’s PCB is that offers several different ways of powering the board: battery or wall-wart, step-up converter or voltage limiter — you get to choose. It’s also designed to fit in a rather nice case, but we still have to figure out the details. Stay tuned!

I’ve been working on an Android app that plays MIDI files via Bluetooth. (Why would anyone want to do such a thing? I have my reasons, but for now let’s just say that I’m a glutton for punishment.) The first obstacle was that Android offers no API support for parsing MIDI files. While I generally feel that it is wise not to support MIDI, the decision to remove javax.sound.midi from the API is a bit of a mystery because the functionality is still there. After all, the media player handles MIDI files just fine, as does the Jet engine.

Fortunately, free (as in GPL) implementations of javax.sound.midi are available, and it was easy enough to paste them into an Android project. The next step was to get Android to launch my app when opening a MIDI file. No problem, just register an intent filter for the MIME type audio/midi and you’re done.

Well, almost. Catching launch intents for audio/midi worked nicely until my Droid X received an upgrade to Android 2.3 (Gingerbread). My first impressions of Gingerbread are quite favorable, but the one downside I noticed was that my MIDI file player stopped working. Whenever I tried to open a MIDI file using the Files app, it would just ignore my intent filter and launch the media player instead.

It turns out that the version of the Files app that comes with Gingerbread thinks that MIDI files are of MIME type audio/sp-midi. This makes no sense, but at least it’s easy to work around. (If you download a new MIDI file and open it by selecting its download notification, it’ll still show up with a MIME type of audio/midi, and so a MIDI file player app needs to register intent filters for both audio/midi and audio/sp-midi.)

Not a terribly compelling post, but I sort of feel like this MIME type issue needs to be documented, just in case someone else is crazy enough to write an app that handles MIDI files. Given the way I feel about MIDI, however, I sincerely hope that this post will be of no interest to anybody.

A former colleague once told me that he wouldn’t let his little daughter play with Legos because he doesn’t want her to see the world as a set of standardized bricks that fit together in predictable ways. That got me thinking. I grew up playing with Legos, and I pretty much see the world that way. I’ve been wondering whether that’s a good thing ever since.

Case in point, when I was putting together a low-cost Bluetooth-MIDI interface, I was thinking Legos. I had three bricks: The electrical specification of MIDI, running at 5V; a KC-21 Bluetooth module, running at 3.3V; and a logic level converter circuit that would turn a 3.3V signal into a 5V signal and back. I only had to stick those bricks together and presto, low-cost Bluetooth-MIDI interface.

I didn’t think much of it until I showed the device to Andrew Tergis of Bug Labs. Andrew pointed out that my solution was a bit of a Rube Goldberg construction and gave me some advice on how to revise the MIDI circuitry so that the entire gadget would operate at 3.3V, eliminating level shifting altogether. Ultimately, the solution was remarkably simple, just changing a few resistor values and replacing the TTL hex inverter with a PNP emitter follower. This approach reduces the number of parts by half, and it works like a charm. I just didn’t see it because I had Legos on the brain. Then again, without the Legos mindset, I probably wouldn’t have tackled the project in the first place.

In addition to simplifications, the new design brings one major improvement, a 3.3V step-up converter. This converter will take an input voltage between 1V and 3.3V and turn it into an output voltage of 3.3V. In particular, it can be powered with one or two batteries, AA or AAA, regular or rechargeable, and the output will remain constant even as the voltage provided by the batteries declines.

The resulting circuit is shown above, as a breadboard prototype, as well as below. I’m pretty happy with it now, and I don’t think there’s much room for further simplification. I documented the circuit in Fritzing and added it to the btmidi repository. The way I set it up, it’s extremely flexible. Right now I’m running it with two AAA NiMH rechargeable batteries and an RN-42 Bluetooth module, but you can easily plug in different batteries and a KC-21 Bluetooth module, or you can use XBee modules instead. In other words, the new 3.3V MIDI circuit is a Lego brick, ready to be combined with other bricks.

New MIDI-Bluetooth interface, based on a KC-21 Bluetooth module

Peter Kirn recently wrote a very kind post on Bluetooth and MIDI, and so I figure it’s time to report on my attempts to reduce the cost of the MIDI-Bluetooth setup that I’ve been experimenting with, even though the design will probably undergo another revision or two.

The original version used a BlueSMiRF module, which is great for prototyping because it’ll accept voltages between 3.3V and 6V (according to specs, anyway) so that you can plug it into any reasonable setup and it’ll work. What’s not so great is the price — basically, the BlueSMiRF consists of a $20 Bluetooth module and $2 worth of level-shifting circuitry, but SparkFun sells it for $64.95. Clearly, significant savings are possible if you’re willing to do your own voltage conversion.

So, I went ahead and ordered a KC-21 Bluetooth module for about $20, only to discover that it wouldn’t support the nonstandard MIDI baud rate. Fortunately, Chris at KC Wirefree was extremely helpful and prepared a new firmware version that can be configured for arbitrary baud rates (thanks, Chris!). Updating the firmware was a breeze, and adding components for shifting voltage levels was easy, too, using schematics found here. The resulting unit is pictured above. The KC-21 module is on top; the parts on the left operate at around 5V, and the voltage conversion to 3.3V happens on the right, with a couple of MOSFET transistors hidden underneath the circuit board.

The resulting gadget works, for a total of less than $25, and preliminary measurements of latency and jitter suggest that it performs about as well as the original version. I’ve since been told by a real electrical engineer that there’s a much simpler way to convert voltages for the purposes of MIDI; maybe we’ll come up with a simplified design at next week’s Handmade Music Open Lab.

I’ve been meaning to try the same setup with an RN-42 unit for comparison, but I have yet to find the time to do that (maybe at the lab next weekend…). In the meantime, I’ve finally managed to install the original interface in a proper project enclosure. It took many attempts, made a big mess, and required the replacement of a couple of parts that I ruined with runaway epoxy, but the reward is a vastly more generic-looking device. Yay!

The original Bluetooth-MIDI interface, now in a proper project enclosure

Peter Kirn and I will present Pure Data for Android at the March Android Developer Meetup at Pivotal Labs in Manhattan tonight (Wednesday, March 23, 7pm).

A recent thread on pros and cons of libpd at Pd Everywhere got me thinking about workflows and the implications of libpd.

I always knew that libpd provides a very smooth and natural workflow from the creation of a patch in Pd to the deployment of a patch to an embedded instance of libpd, but until this morning I only looked at this from my own point of view, i.e., that of a single developer creating both patches and client code. Then I realized that this has much larger implications.

First, libpd provides great separation of concerns — sound designers don’t have to know about programming, and programmers don’t have to know about sound design. All they have to do is agree on the number of input and output channels as well as the collection of send and receive symbols through which the client code will interact with the Pd patch. The sound designer can go ahead and build a patch, controlling it with the usual GUI elements. Building a patch for libpd is no different from building a patch for Pd itself. In order to deploy the patch, the sound designer only has to assign the appropriate send and receive symbols to the GUI elements. Now the programmer can simply load the patch and use it as a black box.

Second, libpd erases the distinction between prototypes and production code. Several people have hooked up Pd or Max/MSP to game engines as a prototyping tool for game audio, using OSC or UDP. While this is a powerful approach, I see two drawbacks: One drawback is that the sound designer needs to add network objects to the patch, in addition to the GUI elements that are needed for development and testing. This adds some friction to the patching process. The other drawback is that this approach is fragile; you have two separate pieces, the game and the audio engine, that have to be launched and networked correctly. You wouldn’t ship production code that way. The prototype that you build this way will always be a prototype.

With libpd, the prototype is the production code. The GUI scaffolding that you use when designing and testing a patch is also the conduit through which the deployed patch communicates with the client code, and the embedded copy of Pd simply becomes a part of the application. Add in the large number of artists who are already familiar with Pd (even larger if you count users of Max/MSP) as well as the fact the libpd comes with the extremely permissive BSD license, and you have some serious disruptive potential.

Rich Eakin recently figured out how to handle multiple copies of the same patch in libpd. The idea is to modify the file opening function of Pd so that it returns a pointer to the newly loaded patch. The pointer serves as an opaque handle to the patch; when the same file is opened a second time, we can tell the two copies apart because their handles are different. This allows us to close individual copies of a patch (which wasn’t possible before) and it allows for easy retrieval of the $0 tag of a patch (which was possible before but took some work).

The latest version of libpd includes Rich’s modification, as well as updated bindings for Java and Python that implement additional functionality on top of the new patch handling API. Specifically, the new file opening methods in Python and Java open a patch, look up the $0 tag, and update a dictionary that maps $0 tags to patch pointers. Like this, the $0 tag acts as a handle of the patch, and no raw pointers are exposed in the API.

The new version is available at Gitorious — check it out!