One of the things I am most interested in mucking around with in terms of game programming is music that changes interactively rather than just loops over and over until an abrupt finish as the map/level/location changes. There are already pretty good examples of it in commercial games such as Meteos where each planet has its own set of sounds that fit together to make a song that matches what's going on in your puzzle screen. Most of the 3D Zeldas also have a little bit of dynamically changing music, especially Wind Waker which played along a sequence of notes as you got closer to finishing off enemies so you could tell about how many hits they needed by the sounds. There are also non-game examples of composers writing music to progress in sync with things other than time. The torch carrying music in the Athens Olympics opening ceremony for example was arranged (it is Shostakovich's "Pirogov") to be flexible so that it progressed as the carriers approached the torch to compensate for unpredictable running speeds.


As I see it, the problem with the state of dynamic music in games isn't a lack of interest but rather a lack of development. Most of the dynamic game music I've come across has been more like a programming/design experiment or an extra feature than a mature component. That is, music in recent games is so static compared to their visuals. Everything on screen is always flashy and animated and responding to player input, but I don't feel that the music always follows it to the extent it could.

Part of the problem is that (to my knowledge) software tools haven't really been put together that capture the different dimensions along which game's score that could be interactive and expose them to composers. Most programmers can't compose or have very limited music theory knowledge. At any rate, I certainly can't create music very well. So there's this kind of rift - only people who are both skilled programmers and composers could create a game with dynamically changing music as a serious component. To make interactive music as popular as interactive graphics, it's therefore necessary for the rest of us to talk about the problems involved and do experiments - programmers putting together the tools and then composers playing with them and seeing what works.

To start off with a more concrete example, I've been thinking about implementing a musical map editor for games with tile-based maps like RPGs and platformers. (I will never get time to do this but believe my lies for a moment.) The theory is that you start off with a map editor like in RPG maker or Game Maker whatever and then kind of paint sound loops over different parts of the map. The score would be divided into different layers like a photoshop image such that you could have one for percussion, one for strings etc. The game would then change which music was playing on each layer as the player moved through different tiles (but only at appropriate times such as at the end of loops/patterns). This would allow songs to finish properly as players near the exit of maps. It would also allow composers to give different areas musical scents so that players could kind of navigate by sound. Additionally, interactive objects in the game could change different layers of the score. For example as an enemy approaches the strings could die out but the percussion remains the same until it's in view. The tempo of music could rise as players approach death, and so on. Interactivity on this level could make a game feel more alive.

The problem with me thinking these things on my own is that I don't have the composition knowledge to tell if my ideas are shit. I suspect that there are also problems that need solving in audio programming which is something I'm very inexperienced at. Since GW has a lot of generally knowledgeable and intelligent members, many of whom design games, program, write, draw and compose music I was hoping that some of you might be able to contribute your own ideas about using music interactively in games or at least point me in the right direction.

There's an action rpg in PS that I heard has great music/action sync..hmm, now what was it's title

You mean something like this maybe?


The music changes based on what is happening

I think it's a great idea and a really good gimmick if you can make it properly

kinda like that awful DJ hero or whatever it's called but with chiptunes?

came in here to post Rez

Mince Wobley that's somewhat along the lines of what I meant, I guess. I'm more interested in ideas about design/composition/programming techniques people have for making that sort of thing less of a gimmick and more of a musical analogue to interactive graphics (i.e. by making it easier to implement and think about).

Just looked at a video of Rez. Yeah, stuff like that I guess! That looks really cool. If I could afford to buy video games at the moment I'd probably run out and try to find a copy actually. Not likely within the foreseeable future though. The connotations of the 'trance vibrator' peripheral as a sex toy is pretty funny/interesting also. Was it an intentional "game outside the game" thing or just something people assumed because "hehehe vibrator"?

But yeah, I kind of think that music that changes with gameplay has the potential to move outside of gimmicky programming stuff and or minimal art games and be a standard tool for game developers in general. Maybe I'm being dumb here but I'd like to talk GAMES DESIGN THEORY about it.

edit: oh and fuck dj hero. i hate rhythm games.

ive thought about this as well it is a confusing idea.

could do loops that come in based on waypoints in the level and maybe keep track of the beat so you can cue transitions in tempo?

idk this is way ponderous.  it would make the music very unmusical and badly arranged to some degree but i guess that doesnt matter.  idk

Rez is what I thought of too.  I can't think of any others right now but I know there are more!  I think it's an awesome addition to a game.

there is an excellent article on gamasutra about this here

I am extremely interested in adaptive music and being a composer and programmer i have the opportunity to mess about with it. Your idea about a 'musical map' is great and a good (basic) implementation of it.

It is very possible to put into a game with seperate composer/programmers, just you would have to sit down and work it out. The use of trackers would be essential, rather than mp3s or wavs or the like as it would be extremely difficult to time the tracks correctly, but trackers can be read at bars/beats rather than time and loop points can be changed to progress the music when the game dictates. I am only just starting to use trackers in Game Maker to see what I can do with them, and if they work as well as I think, I shall be putting adaptive music into my future games.

As for using it as a primary game mechanic rather than a gimmick, I would find it difficult to find uses other than in rhythm mini games. But thats just me. I'm sure there are many many great ideas for it that need to be found.

btw that game Brutal Legend uses music in the game for casting spells i think, so it is a game mechanic, but it is not adaptive as far as I am aware.

but yes I am very very interested in learning about and using adaptive music, and would love to collaborate with someone in making a game with it.

read up on iMUSE, this is how lucasarts achieved it in their excellent adventure games. maybe you could experiement with a similar system.

what's wrong with dj hero?

That gamasutra article and the iMUSE site are both pretty interesting. Thanks. I don't know why it didn't occur to me that iMUSE was for adaptive music (I like this term) before - I've been playing a lot of Lucasarts adventure games recently. The stuff about Beethoven's combinatorial dice music is fascinating, also.

I think I am going to start mucking about with this musical map thing after I've sat my exams and see where it ends up. I'll need to do some research about tracker style music in Python. If I manage to finish some kind of prototype I'll post it up for you guys to play with. (Probably I will get distracted though.)

I've thought about this before as well, and I always run up against a barrier: The game doesn't know in advance what the player is going to do, so it's hard to have music that "builds up". The player might decide to just hold off shooting the boss the one last time it would take to finish him off and win the game, but your music engine can't predict that. It's a tricky problem.

I've thought about this before as well, and I always run up against a barrier: The game doesn't know in advance what the player is going to do, so it's hard to have music that "builds up". The player might decide to just hold off shooting the boss the one last time it would take to finish him off and win the game, but your music engine can't predict that. It's a tricky problem.

yeah, it is difficult to find a balance between musicality / integrity of arrangement and generativity/player interaction.

But having it build up to just be let down at the last minute would technically reflect what was happening in game, so I'm pretty sure that if you just put it in how you normally would (the lower the boss/your HP the more tense the music) then you'll find that no matter what it would kinda work.