The Misadventures of Unity and PureData (libpd)

Over the last few days I’ve been bashing my head against the wall trying to get libpd working with Unity. This is going to be a super technical post relevant to very few people, but if you end up hitting these problems hopefully this will save you a few hours of brain smushing frustration.

For those that don’t know Puredata is an awesome visual programming language that allows you to perform audio synthesis from first principles. Libpd is the embedded version of pure data, which means that thanks to the work of many wonderful people it’s possible to use puredata to process your audio in Unity. Absolutely perfect for expanding the sound pallet in my game Cadence

Important links:

libpd on github
libpd4unity on github

Overview

It took me a long time to figure out how all the parts fit together. Libpd4unity is pretty great starting point, which uses the C# port of libpd and drops it into unity. But I ran into a host of hairy issues which took a lot of frustrated debugging to solve.

So puredata normally works as a stand alone application, and has everything it needs to speak to your soundcard in the way of drivers and timing and all that jazz. In order to make puredata work as an embeddable system it’s been stripped of all the platform dependant sound driver code. What this means is that by itself libpd has no concept of timing or audio playback. It’s simply a dsp black box. Samples (numbers) go in and samples come out. It’s up to you to do something with those samples to turn them into sound.

This was the first thing I had to get my head around. By using MonoBehaviour.OnFilterRead() we can then turn these samples into sound. Essentially it’s designed to let you insert a “custom” filter into the Unity DSP chain. This means effectively libpd runs as giant audio filter on Unity’s Audio Thread.

Compiling Libpd

Windows

First thing I tried to do was compile the libpdcsharp.dll on windows. Compilation from Visual Studio seems to be a no-go, although there is some interest adding support (in particular through the ofxlibpd project), but I couldn’t find any evidence of someone who got this working. So that left MinGW which provides a windows implementation of the GCC compiler that is very popular in UNIX toolchains. Inside the libpd folder there is bat file called mingw_build.bat that allows you to simply execute the make operation from Command Prompt.

Right away MinGW started complaining about not being able to find pthreads.h. This has something to with providing POSIX style threads on Windows. I managed to fix this by going back to the mingw installer and making sure all “threads” related .dlls were downloaded and installed. Which then lead to this wonderful error output: https://gist.github.com/anonymous/10274995.

Some digging through obscure corners of the internet revealed that it seems to be a MinGW problem. I solved it by downloading a legacy version from here MinGW Legacy Builds. These don’t come with an installer, but you can delete your MinGW folder and simply unzip to the same location. Finally, this got me the mythical libpdcsharp.dll!

OS X

This was much easier. GCC seems to built into OS X, so you can simply run make csharplib from terminal and it should compile without issues. (Will spit out loads of warnings but I don’t think they are a problem). However this gives you a .dylib which unity doesn’t gel with. As described in this post you can get around this by renaming it “libpdcsharp.bundle” and then Unity can speak to it. (As well as some unfortunate build gymnastics you have perform when you want to publish. That is mentioned in the article as well).

Using lip4unity

Checking out the github is a pretty good place to start. It includes the libpdcsharp lib precompiled for several platforms and example projects, as well as how to hook up libpd to OnFilterRead() in the file LibPDFilterRead.cs. This actually the meat of libpd4unity, the rest of the files are taken directly from the C# wrapper as found in the original libpd project. But again on windows I ran into a wonderfully mysterious error:


DllNotFoundException: libpdcsharp.dll
PluginInterface.Start () (at Assets/PluginInterface.cs:18)

Bit odd seeing as the dll was right there. Turns out first problem is the dll needs to be alongside the executing exe. So that means it needs to be inside the Unity application folder (e.g. C:\Program Files (x86)\Unity\Editor). Even after doing so and restarting the problem persisted. Further investigation reveals the error is a bit of a lie, and will also be thrown if there are any referenced dlls that are missing. By analysing libpdcsharp.dll in Dependency Walker, it revealed that there was a missing reference to “pthreadsGC2.dll”. By copying the file from MinGW into the unity application folder I got it to work.

Side Note: Theoretically I could have just used the .dll that came with libpd4unity, but i had the same error. Analysing their lippdcsharp.dll showed that libpthreads-2.dll was missing. This seems to be because their dll was compiled by a newer version of MinGW which replaced pthreadsGC2.dll as used by my legacy-I-DONT-CARE-IT-WORKS version. I don’t know how this ever worked for them, but I assume it’s because MinGW was in their system path. This actually hints at a really nice way to clean up the need for putting .dlls in the unity application folder (yuck!), by dynamically modifying the system environment PATH (as described in this unity forum post. I’ll be posting a github pull request with fixes soon).

Sweet, Sweet sound

This finally got my sound working. But as soon I put the editor into play mode for a second time Unity would crash and die. After another wild goose chase I managed to determine it was being caused by a bad pointer reference on the print hook callback that receives messages. I’m still working on a fix, but you can get around it by commenting out code that sets the print callback in the LibPdMessaging class (this will disable print messages from libpd). Don’t have time to go into further detail now, but hopefully I can shed more light on as my understanding increases.

  • zack hagan

    I dig the tutorial, but I can’t get it to work with unity 4.3 free! I get a bazillion errors about the libpdcsharp.bundle. Does this work with unity free? Any suggestions? ps – Candence looks awesome!

    • http://www.madewithmonsterlove.com Peter Cardwell-Gardner

      To the best of my knowledge Native plugins are a pro only feature. As far as I know only dlls with pure csharp code (aka not libpd) will work in the free version.

    • http://daandelange.com/ Daan

      You have to rename libpdcsharp.dylib to libpdcsharp.bundle then put it in your project root folder (not in the /Assets folder)

      Also remove or rename /Assets/Plugins/libpdcsharp.bundle to get rid of the License error message.

      Tested in Unity 4.5 for Mac