Sound and How It Affects Playback

Audio is a very important element of an animation like “2001.” We could have saved ourselves a lot of time by just removing the audio from the file, but it would not have been the same experience.

Our first major challenge was to get the animation to work on the phone—with sound. We were able to get it to work without sound relatively quickly, but when we published it with sound, we would get an out-of-memory error as soon as we started to play the animation.

Length Matters

To resolve this issue, we began by slicing up the animation and putting each scene or cut into a movie clip that we would then place on the main Timeline. Then we added the audio one cut at a time into the matching movie clip, not the main Timeline, and tested the result on the phone—it worked!

However, that didn’t resolve all of our problems. Sometimes, one audio piece would cover several cuts, like the music at the beginning, and there was no easy way to have it exactly timed to match each Timeline. So we just put the entire longer section into the first cut, and extended the Timeline with blank frames. Unfortunately, as soon as we published this to the device, the phone would run out of memory halfway through the animation.

We concluded that running audio in two clips simultaneously causes a problem. We removed the sound from the clip where it would overlay the first longer sound, and tried that—and it played just fine. Of course, now we were missing a piece, and we stuck it in the first clip—and, surprise, it played! We still don’t know the exact cause for this; all we know is that playing sounds in two clips at the same time is a bad idea.

We also never understood why we could not put the entire soundtrack on the main Timeline. Our best guess is that the Flash Lite player caches the sound for an entire Timeline all at once, and handling all the sound at once was just too much. However, it seems to handle memory for each movie clip separately, which is why this segmenting technique succeeded.

Even with this technique, we would still run out of memory at some point of adding sound to the animation. We solved this by optimizing the animation scene by scene. Sound and animation do share memory resources, so it only makes sense that a cleaner animation allows for more sound.

And Then There Is Volume

After we resolved these issues, we could look into some other issues related to sound. The first one was that the soundtrack played awfully loud on the phone, and we had to really lower it. You will have to experiment with that, but we found that we had to set the audio to about 50% volume of what sounded fine on the desktop.

The very beginning of the soundtrack was pretty much bass only. On the phone all you could hear was something crackling and we just had to remove that part. We concluded that the small speaker of the phone was just not capable of playing sounds of that frequency.

However, this also has its benefit – you can reduce the audio quality pretty drastically, and you still don’t hear a difference on the phone.

Sound playback shares processor resources with the display. As a result, you may notice a delay in the animation when a sound starts—at that point Flash is starting up the sound device, causes a delay in the display.

We had a particular problem in the scene where the robot is kicking the vending machine. It would just freeze for a second making him stick to the machine, and then the kick sound started playing. We solved this problem by playing a silent sound from the beginning of that scene. That triggers Flash to start the sound device before the scene, when the user doesn’t notice the delay as opposed to in the middle of the action.

Note: On the desktop, we sometimes use a two-frame looping clip with a silent streaming sound to keep the sound device active, and to avoid such delays. Unfortunately, this doesn’t work with Flash Lite. Firstly, it will slow down the playback. Secondly, it triggers the out-of-memory alert when running sound in two clips.

The sharing of processor cycles with the animation has another side effect: Heavy animation or a scene cut may cause hiccups for the sound, noticeable as small cracking sounds. To work around this issue, try to avoid the particular situation or make the transition easier on the CPU.

On the Nokia 6600 phones we have also noticed a cracking sound after a sound has finished playing, even with nothing else going on. We’re still looking for a solution to this problem. It seems to be an issue with the device, maybe it’s just our phone. Note: Not all devices can play all sounds! Flash Lite can embed sounds in different formats (MP3, ADPCM, MIDI, or raw) for better cross-platform distribution. It can even embed alternate sounds in the same file, and then choose the appropriate one.