The Misadventures of Unity and PureData (libpd)

Over the last few days I’ve been bashing my head against the wall trying to get libpd working with Unity. This is going to be a super technical post relevant to very few people, but if you end up hitting these problems hopefully this will save you a few hours of brain smushing frustration.

For those that don’t know Puredata is an awesome visual programming language that allows you to perform audio synthesis from first principles. Libpd is the embedded version of pure data, which means that thanks to the work of many wonderful people it’s possible to use puredata to process your audio in Unity. Absolutely perfect for expanding the sound pallet in my game Cadence

Important links:

libpd on github
libpd4unity on github

Overview

It took me a long time to figure out how all the parts fit together. Libpd4unity is pretty great starting point, which uses the C# port of libpd and drops it into unity. But I ran into a host of hairy issues which took a lot of frustrated debugging to solve.

So puredata normally works as a stand alone application, and has everything it needs to speak to your soundcard in the way of drivers and timing and all that jazz. In order to make puredata work as an embeddable system it’s been stripped of all the platform dependant sound driver code. What this means is that by itself libpd has no concept of timing or audio playback. It’s simply a dsp black box. Samples (numbers) go in and samples come out. It’s up to you to do something with those samples to turn them into sound.

This was the first thing I had to get my head around. By using MonoBehaviour.OnFilterRead() we can then turn these samples into sound. Essentially it’s designed to let you insert a “custom” filter into the Unity DSP chain. This means effectively libpd runs as giant audio filter on Unity’s Audio Thread.

Compiling Libpd

Windows

First thing I tried to do was compile the libpdcsharp.dll on windows. Compilation from Visual Studio seems to be a no-go, although there is some interest adding support (in particular through the ofxlibpd project), but I couldn’t find any evidence of someone who got this working. So that left MinGW which provides a windows implementation of the GCC compiler that is very popular in UNIX toolchains. Inside the libpd folder there is bat file called mingw_build.bat that allows you to simply execute the make operation from Command Prompt.

Right away MinGW started complaining about not being able to find pthreads.h. This has something to with providing POSIX style threads on Windows. I managed to fix this by going back to the mingw installer and making sure all “threads” related .dlls were downloaded and installed. Which then lead to this wonderful error output: https://gist.github.com/anonymous/10274995.

Some digging through obscure corners of the internet revealed that it seems to be a MinGW problem. I solved it by downloading a legacy version from here MinGW Legacy Builds. These don’t come with an installer, but you can delete your MinGW folder and simply unzip to the same location. Finally, this got me the mythical libpdcsharp.dll!

OS X

This was much easier. GCC seems to built into OS X, so you can simply run make csharplib from terminal and it should compile without issues. (Will spit out loads of warnings but I don’t think they are a problem). However this gives you a .dylib which unity doesn’t gel with. As described in this post you can get around this by renaming it “libpdcsharp.bundle” and then Unity can speak to it. (As well as some unfortunate build gymnastics you have perform when you want to publish. That is mentioned in the article as well).

Using lip4unity

Checking out the github is a pretty good place to start. It includes the libpdcsharp lib precompiled for several platforms and example projects, as well as how to hook up libpd to OnFilterRead() in the file LibPDFilterRead.cs. This actually the meat of libpd4unity, the rest of the files are taken directly from the C# wrapper as found in the original libpd project. But again on windows I ran into a wonderfully mysterious error:


DllNotFoundException: libpdcsharp.dll
PluginInterface.Start () (at Assets/PluginInterface.cs:18)

Bit odd seeing as the dll was right there. Turns out first problem is the dll needs to be alongside the executing exe. So that means it needs to be inside the Unity application folder (e.g. C:\Program Files (x86)\Unity\Editor). Even after doing so and restarting the problem persisted. Further investigation reveals the error is a bit of a lie, and will also be thrown if there are any referenced dlls that are missing. By analysing libpdcsharp.dll in Dependency Walker, it revealed that there was a missing reference to “pthreadsGC2.dll”. By copying the file from MinGW into the unity application folder I got it to work.

Side Note: Theoretically I could have just used the .dll that came with libpd4unity, but i had the same error. Analysing their lippdcsharp.dll showed that libpthreads-2.dll was missing. This seems to be because their dll was compiled by a newer version of MinGW which replaced pthreadsGC2.dll as used by my legacy-I-DONT-CARE-IT-WORKS version. I don’t know how this ever worked for them, but I assume it’s because MinGW was in their system path. This actually hints at a really nice way to clean up the need for putting .dlls in the unity application folder (yuck!), by dynamically modifying the system environment PATH (as described in this unity forum post. I’ll be posting a github pull request with fixes soon).

Sweet, Sweet sound

This finally got my sound working. But as soon I put the editor into play mode for a second time Unity would crash and die. After another wild goose chase I managed to determine it was being caused by a bad pointer reference on the print hook callback that receives messages. I’m still working on a fix, but you can get around it by commenting out code that sets the print callback in the LibPdMessaging class (this will disable print messages from libpd). Don’t have time to go into further detail now, but hopefully I can shed more light on as my understanding increases.

Motivational Anti-Matter

Hi my name is Peter, and I have a problem. I’ve been suffering from a lack of motivation.

Almost a year and a half ago I quit my job to go indie. My circumstances were not unlike many in the same situation – I had six months in savings and was going to have a full go at it. If it didn’t work, hey no sweat I could always go find a job again afterwards. So how did that turn out?

Well 17 months later I can say I’ve done a lot of contract work, travelled to some pretty crazy places, experienced what I can only call a quarter life crisis, suffered the demise of my long term relationship, learnt more about myself than I ever thought I would and found a game I want to work on ( Cadence ). Despite having made some massive strides in my own personal development, I haven’t exactly leapt into indie career-mode. Rather I’d say I’ve desperately clung to the edge and dabbled my toes in the water a bit.

A big part of that is fear. Fear of failure. Fear that leads to crushing keep-you-awake-at-night anxiety. Anxiety, the ever present vanguard of fear. No one really tells you about the anxiety when you set out to pursue your dreams. Maybe they do, but in any case it’s not until you stagger around under it’s slow death grip that you realise just how debilitating it can be. How whenever you finally get a chance to work on your passion project you’ve been so hungry for, you just kind of end up staring at your screen and before long you’re on facebook and twitter. Or you know, that admin you needed to do for months is suddenly getting done whilst your game starts developing abandonment issues and stews in cold neglect. (Fantastic post from aeiowu elaborating on the motivational short fall of becoming a full time indie: http://aeiowu.tumblr.com/day/2013/06/17 )

I’m of course talking about procrastination, which we all suffer from to a degree. In this context though I firmly believe procrastination is not a laziness. It’s a defense system. A defense against the fear of failure. As far as your brain’s reward system is concerned, it’s impossible to feel the sting of failure if you never even try (oh, the irony). To understand what powers this fallacy, it also helps to understand that I am chronic perfectionist.

Perfectionism might be considered a positive trait, but in this light it definitely qualifies as another defense system. It’s based on the flawed belief that your inherent worth as a person is based on your performance. As if somehow imperfect work means you as a person are imperfect, you are unworthy! Whilst this relentless drive to prove yourself is an attribute of many high performers and highly driven people – it’s a bit of a disaster zone when you start pinning your self worth on activities you haven’t had the chance to master yet.

In my case years in the software industry may have taught me plenty about programming, but it’s taken a lot of self awareness to begrudgingly admit I’m by no means a master of game design (or programming for that matter). And really what rights do I have to believe I should be a master when I simply haven’t allowed myself the adequate space to get it wrong? Denying yourself exposure to failure puts a severe damper on the learning opportunities that arise from that failure.

Even in situations rigged for failure, such as game jams, I still end up performing only through a crushing sense of anxiety that I’m going to be judged and that I need to show the world how “brilliant” I am. This leads to an unhealthy paralysis where even though I might be able to deliver a well polished game, I’m definitely not exploring the full possibility space of my games. Whilst there is still some form of learning happening, I’m not really sure I’m learning as much as I could be.

This plays to the fickle motivation of passion projects and why I think so many people have many unfinished game projects. I myself have several projects that despite burning brightly somehow lost steam, and now languish in a state of motivational inertia. I think this mostly happens you when you encounter design challenges and you start to have doubts that your game is going to live up to your expectations: and so onto the next exciting project still alive with possibility and unburdened by the weight of expectation! (Hint: your life falling short of your expectations of who you thought you’d become also seems to be what a mid life crisis is all about)

I think perhaps this touches on one of the compulsions for the indie dream: The promise of recognition ( and consequently the implication that you are a worthy person because you’ve managed to create something of value in the world – regardless of the validity of this notion). This also makes me think about my twitter stream, and how it’s filled with “indie darlings” – successful developers who are already living the dream. Why do we engage in this form of hero worship? Is it not strange we dedicate so much of our attention to people who honestly have no reason to return it, and why does it feel so rewarding when one of them throws you a scrap of attention by replying your post or tweet?

Whilst there is no arguing that social media is a powerful tool for starving indies – I have to wonder what this constant reinforcement of “you are not successful yet” is really having on our mental health. So whilst this is still only intuition: I’m think there is a lot to be said be said for turning down the volume on your social media and being okay with not reading everything you find on the internet. If nothing else a remedial dose of the dunning kruger effect can only be a good thing for someone who’s fear and anxieties are preventing them from getting anything made…

PS: For anyone looking for some tools to help curb their social media usage, I recommend Anti-Social and Rescue Time

The Psychology of Why We Fail to Fail [Updated]

Update: The incredbile Amaze Festival in joburg gave me a chance to revisit this talk and throw all of my new discoveries since last time I presented at a local Make Games SA community gathering. It was fantastic to see a legitimate celebration of games happening on African soil!

This talk investigates the premise that failure is an essential ingredient in the journey to becoming successful – and yet our brains are loaded with a treacherous arsenal designed to sabotage this process in it’s desperate attempt to avoid failure at all costs.

The talk tries to arrive at an understanding of why traits like perfectionism and procrastination spiral out of control, especially when the stakes are highest and we are working on our passions and reaching for our dreams. It then suggests strategies we might use to overcome this creative paralysis.

The focus is naturally on creating games, but will be relevant to anyone who struggles with creative endeavours.

Some of the inspirations mentioned in the talk:

Ira Glass on Creativity:

John Cleese on Creativity:

James Murphy, on being a failure:

Dan Pink on Motivation:

Brene Brown on Shame

How Programming Changed My Life

I will never forget sitting in that bar, wondering if my future colleagues would smell the fact I’d only just endured a 15 hour cross country trek in my old VW golf without aircon. For them it was just an excuse to kick back with end-of-the-month friday drinks. For me it was the first drink of the rest of my life.

Several things about my world changed in the space of those few days: moving out of the house, starting fresh in a new city, saying goodbye to a girlfriend for the last time, starting a strange job in a new career. They say change is inevitable, but I had just dived head first into a storm of change. I also had no idea how different life would be 3 years later when I would be saying my goodbyes to my new co-workers and starting the next chapter in my life.

Up until then I was working a low paying job as a sound technician in a recording studio. Despite the pay, it managed to be pretty safe and comfortable: how many people get to work in the music business? Plus my hours were pretty relaxed and there was definitely no dress code. I was also dreadfully bored. Realising this though would take many years and the gift of hindsight.

My life was, essentially, in a rut. Perhaps it was a lack of life experience that kept me blind to it, but I was in a go nowhere job (as kind as the company was). And to be honest, career growth was so far off the table I didn’t even bother thinking about it. More disturbingly, the lack of engagement could be so bad I would sometimes catch myself nodding off while sitting mere centimetres from my producer at the same workstation! It’s an awful thought to think that could have been the next 40 years of my life, and I thank fate for sowing the seed that steered the rudder toward better waters.

The company’s websites were supplied by a brother of a co-worker. They looked good, but had some crazy rough edges. Despite many complaints and requests for improvement, the reason for this eventually came to light. The dude was illegally ripping templates (back in the day when flash was still cool), mangling in some content, and charging us full price for web development. The company was horrified! I was adamant that even I, despite knowing barely anything about flash, could do a better job customizing a legally purchased template. Well it seems the company believed me too.

I struggled along at first. Programming and its mysteries were completely opaque to me. Simple successes felt like monumentally epic victories. It was slow going, but I soldiered on. And slowly but surely things started coming into focus. The “aha” moments started becoming more frequent. On the surface I had discovered programming, but deeper down I had found my challenge, my purpose, my zen. I was hooked. Deciphering lines of code was exhilarating – giving birth to that which had previously only existed in my mind. It was genesis and I the binary god.

It wasn’t long before I had a cheap pc to keep me busy with web dev during the studio down time. I would long for these moments, so I could pour myself back into the challenge of programming. By then I even had my first programming book: a dense and dry reference manual on Actionscript 3, my language at the time. I read that book cover to cover, making notes along the way. I don’t think I even realised that I was basically studying the same way I used to for exams. Of course there was no exam, just a pressing thirst to know more about this new world. I was learning at an incredible rate. And it was fun.

The avalanche eventually culminated in me sitting in the bar on that friday, a thousand miles from home, nervously awaiting the first encounter with my new co-workers, the creative geniuses I had so admired from afar. The next monday I would be working alongside them at one of the countries hottest digital agencies as Flash Developer. Only the day previous I had packed all the belongings I could fit into my car, said my goodbyes and set off into the unknown.

What I couldn’t know is from then on the pace would only accelerate. Battle scars soon replaced the stars in my eyes – these were just people slugging it out the same way I was. But the learning didn’t cease – being out of my depth meant finding new ways to swim. This gave me the resolve to voice my opinion whenever I saw problems. And then people started seeking out my opinion, not only that, but respected it too. My confidence skyrocketed – it took seeing it reach new heights to realise it had even been lacking in the first place! This of course spilled into the other areas of my life – I felt like a big boy now, grown up, reading to deal with life’s challenges.

*****

Looking back it’s easy to see the difference in my professional life before and after. As Daniel Plink outlines in his book Drive, I had found the three keys: Mastery, Autonomy and Purpose. Although I’m not really sure I had found them, as much as they had found me.

The reason I write this is because I once again find those things lacking in my professional life. The more you level up, the more you learn, the more you achieve – the higher your standards become, and the more difficult it is to level up again. I think one has to be too careful that your standards are tuned right. I had done this to myself before: my standards for music were so high that instead of a actually creating music and learning how to get good at it – my frustrations would paralyse my development.

Also one has to be careful of setting yourself the right type of motivation. I wanted people to respect me for the music I made. I wanted to be successful. I wanted to make good music. While I was no stranger to toil and hard work, what i couldn’t realise is that my motivations were all wrong. No matter how many hours I put in, wasn’t enjoying music for myself first. Whenever I would create I would be so obsessed with what people might think of it, that I’m just get stuck in a loop, and never actually get past the blank page.

Now that I’m on my path to becoming an indie games developer, I need to remind myself to avoid the same trap. Whether or not other people like my work is of course a real concern, for if they don’t I’ll have learn about the nutritional value of old leather boots first hand. However it pays to remember risk aversion is why big companies never make anything interesting. Make games/music/art for yourself first and foremost. If your creative output makes you happy, chances are others will be grateful for giving them a chance to play in your universe too.

Toward the Light – 7dfps entry

A while ago I participated in the 7dfps challenge to build a first person shooter in 7 days. Starting a day late probably wasn’t the smartest idea, but I’m still very happy with the way things turned out. I’ve always been put off by the cheesy horror in slasher flicks, and believe the art of creating tension and suspense requires a much subtler hand. Exploring those nuances is really what my entry was all about.

Toward the Light screenshot

Toward the Light screenshot

You can grab the download here (built with unity of course):

http://www.thefuntastic.com/demo/toward-the-light/

– UPDATE –

Seems this the project has escaped onto the internet! Reaction vids have started popping up all over the youtube. This is one pure win [May contain strong language]:

Launching Corona from Sublime Text 2 [Windows]

So recently I embarked on a new adventure using Lua, Corona and Sublime Text 2 all for the first time. Out the bunch my favourite so far has to be Sublime Text 2, which is all the rage these days. I find it’s approach to simple no fuss extensibility especially awesome, even if I’m still not sold on full scale development in a text editor.

I admit I’m a bit of a work flow zealot, so much so that alt-tabbing to corona and hitting ctrl-R every time was starting to get on my nerves. Fortunately thanks to Sublime Text build systems there is a fairly easy way around this.

Tab switching script

The first (and really the crux of this method) is this super handy vb script by John Lindquist. You can find an in depth explanation over at his site, but basically it intelligently seeks the corona process and refreshes the simulator if its already running. And if not it simply launches a new instance of the corona simulator.

Set WshShell = WScript.CreateObject ("WScript.Shell")
Set colProcessList = GetObject("Winmgmts:").ExecQuery ("Select * from Win32_Process")
 
'Check if Corona is already running
For Each objProcess in colProcessList
If objProcess.name = "Corona Simulator.exe" then
vFound = True
End if
Next
If vFound = True then
'If Corona is already running, then switch to it and refresh
WshShell.AppActivate "C:\Program Files\Corona Labs\Corona SDK\Corona Simulator.exe"
WScript.Sleep 100
WshShell.SendKeys "{ESC}"
WScript.Sleep 100
'This has changed to be appened to the project name. So add hyphen to avoid focusing the output panel
WshShell.AppActivate " - Corona Simulator"
WScript.Sleep 200
WshShell.SendKeys "^r"
Else
'If Corona isn't running, launch it with your main.lua
WshShell.Run """C:\Program Files\Corona Labs\Corona SDK\Corona Simulator.exe """ & WScript.Arguments(0)
End If

Points to take note of:

  • I had to add “-” to the line in the original script which looks for the corona simulator. It seems corona now prefixes the project title to it’s process name e.g “Project Title – Corona Simulator”. Without this the script will instead focus the corona output window.
  • If Corona isn’t installed in the default location, you will need to modify this script to point the correct directory

Sublime Build System

The next part of the process is hooking this up in Sublime Text. This is pretty easy thanks to sublime’s json based build system configuration. The easiest way to do this is through Tools>Build System>New Build System… This will create a new empty build file. You can simply copy the supplied code over then save it with an appropriate name.

{
	"cmd": ["C:/dev/corona/CoronaLauncher.vbs", "main.lua"],
	"working_dir": "${project_path:${folder}}",
	"shell":true,
	"selector":"source.lua"
}

Notes:

  • The first line simply specifies the location of the vbscript and passes it main.lua as an argument. This should be where ever you saved the script above.
  • Unlike some of the examples posted online, I had to specify the working dir in order to get the main.lua argument above to evaluate properly
  • Setting shell to true tells sublime to launch through the native shell. This is pretty essential because it is not possible to invoke vbscript from the python exec task that sublime text uses.
  • the final line is a handy way of saying you want this build behaviour to be the default for all lua files when the build system is set to “Automatic”

That’s pretty much all there is to it. From now on you can simply launch Corona from Sublime Text by using Ctrl-B! Magic!

Addendum:

Try as I might, the one thing that still seems unpossible is capturing errors from Corona’s output panel. Although sublime provides you a handy reg-ex methods to parse error strings (and thus conveniently jump to the position in code where the errors were thrown), there is no way to actually send corona output to anywhere useful on windows. This is kinda a bummer as well for anyone hoping for a CI (continuous integration) set-up.

Simple Finite State Machine – Unity3d (C#)

UPDATE: The original state machine implementation has been radically reworked, and is now (in my opinion) one of the cleanest and most easy-to-use state machine in existence for unity. In a perfect world I’d find time to write a blog post detailing it’s benefits, but in the mean time you can get a comprehensive overview over on the project github project page. Please note that the information detailed below refers to the obsolete v1.0 of the state machine, so feel free to ignore it entirely.

Well, well – Seems it’s time again for my shining annual contribution to internet blog cruft! Do I have to post cross post this on google plus now too? Okay enough with the nonsense. The first time I encountered the State Machine design pattern, I enjoyed the concepts behind it, but I always found the actual implementation a little bit awkward and messy. I think that rather than place the blame on the pattern, this was more specifically a problem of trying to squash a square peg through a round hole. A trap that every design pattern neophyte is supposed to fall into at least once as they try to lay their shiny new patterns on top of every problem presented to them.

A while ago I was pondering the problems I encountered trying to get my very early games to “progress” through different scenarios. Scenarios such as “disable player input on Intro and show intro animations” changing to “okay, now disable animations, and enable input because the game has stared.” and then again to “Okay now disable the player input again and show the round over screen because the round is over“. What I didn’t realise at the time was that basically I was trying to manipulate state. Trying to do all of this in the same localised class led to monstrous classes hundreds of lines deep with spaghetti logic everywhere. What I didn’t know at the time was that this is a case where the State Machine pattern is perfect (also known as a FSM or Finite State Machine. I refuse to believe the references to the flying spaghetti monster (FSM) and spaghetti logic is a mere co-incidence).

Read More »