Digital Worlds - Interactive Media and Game Design

If the recent Digital Worlds posts on the topic of audio in computer games interested you, and you would like to learn more about working in this area, there are two sets of Skillset Standards that you may find useful: Skillset Standards relating to “Create Sound Effects For Interactive Media Products (IM27)” and Skillset Standards relating to “Create music for interactive media products (IM28)”.

Please not that the Digital Worlds blog only covered audio in the most rudimentary sense, and a full treatment would require one or more dedicated uncourse blogs on the subject to do it justice!

Here’s a brief summary of the standards for sound effects creation:

IM27 CREATE SOUND EFFECTS FOR INTERACTIVE MEDIA PRODUCTS
Example job titles: Sound Effects Designer, Audio Engineer

Overview
This unit is about your ability to create sound effects that work in an interactive context.

Knowledge and Understanding
This is what you must know
a. How to interpret and follow specifications or other briefs;
b. How, and to whom, to ask questions to clarify requirements or raise issues in response to the specification or brief;
c. Principles of sound design,sound effects and acoustics;
d. How to locate sources of audio material suitable for meeting the creative brief;
e. The types of audio effects that are available and their suitability for different products and contexts;
f. Ways in which sound effects can be used to enhance the user’s experience and/or give feedback on user interactions;
g. Appropriate file formats for saving sound effects;
h. The effect of audio sampling-rates and bit-depth on file-size and data-transfer rates;
i. When and why a sound effect might be cut-off prematurely,and how to minimise the risk of this adversely affecting the product.
j. The various types of data compression and their relative merits and demerits.
k. How to layer sounds to achieve a combined audio effect or to produce a complex replay of elements with logical replay rules
l. The various techniques for synchronising sounds to moving images
m. How to use and work within bespoke 3D geometry rules
n. The recording,editing and post production of dialogue

Awareness
This is what you must be aware of
i. Project parameters and constraints including target platforms and their capabilities, especially relating to audio playback;
ii. Any other audio, such as background music, that the sound effects you create will need to fit with;
iii. The events or user interactions that will trigger sound effects in the product;
iv. How each sound effect will be used in the product (for example, whether it will play once, loop several times or indefinitely etc.);
v. Compatibility issues between mono, stereo, multi-channel and surround sound;
vi. When permission is needed to use material created by others;
vii. The limits of what you may legally do with material created by others before permission is needed;
viii. Any naming conventions, standards, guidelines or specifications that you need to follow;
ix. The requirements and expectations of other team members who will use the sound effects you create.

Performance Statements
This is what you must be able to do
1. Generate original sound effects to meet a brief or specification;
2. Systematically assess the implementation of your work in iterative versions and specify changes in effects, volume, pitch and panning
3. Edit existing audio material to create sound effects to meet a brief or specification;
4. Save sound effects in an appropriate format for different target platforms;
5. Organise sound effects using appropriate filing and naming conventions so that they can be located easily by others;
6. Provide clear documentation and audio demonstration clips as necessary for others to incorporate your sound effects into the product;
7. Liaise with colleagues, such as designers and developers, to ensure your sound effects are appropriate and meet requirements;
8. Liaise with the relevant authority to obtain approval for your work.

Here’s a brief summary of the standards for music composition:

IM28 CREATE MUSIC FOR INTERACTIVE MEDIA PRODUCTS
Example job titles: Composer, Musician, Music Writer

Overview
This unit is about your ability to compose and record music for use in interactive products. It assumes you already know how to compose music generally and now need to apply this skill in an interactive media context.

You might need to save your compositions as:
• MIDI files
• AIFF sound files
• WAV sound files
• AC3 files

Knowledge and Understanding
a. How to interpret and follow specifications or other briefs;
b. Leading the process of assessing and specifying music requirements as necessary
c. How, and to whom, to ask questions to clarify requirements or raise issues in response to the specification or brief;
d. The different technologies used in a computer-based music studio, including samplers, sequencers, MIDI devices, ‘outboard’ recording studio hardware and mixing desks;
e. How to sample audio from legitimate sources and use sound samples in your composition;
f. How to use appropriate software to record, sequence and mix audio;
g. Different formats in which music can be output, and when it would be appropriate to use them;
h. The effect of audio sampling-rates and bit-depth on file-size and data-transfer rates;
i. How to address the challenges of scoring music for non-linear medium with scenes of indeterminate length by employing techniques like branching segments and the use of music layers mixed dynamically at run-time.
j. How to articulate designs for bespoke development tools to enable auditioning of your work out of context

Awareness
This is what you must be aware of
i. Project parameters and constraints including target platforms and their capabilities, especially relating to audio playback and data-transfer rates;
ii. How the music will be used in the product (for example, whether it will play once, loop several times or indefinitely, whether it needs to sync with specific parts of the product, etc.);
iii. How the music content will work in conjunction with sound effects and dialogue
iv. Any requirement for the music to change in response to events or user interactions (for example by changing key or tempo, or by segueing into another piece);
v. When permission is needed to sample or use material created by others;
vi. The limits of what you may legally do with material created by others before permission is needed;
vii. The overall purpose and mood of the product and its intended user experience;
viii. How music has been used to enhance comparable products including competitor products.

Performance Statements
This is what you must be able to do
1. Compose music that is appropriate for the purpose and mood of the product;
2. Record music in an appropriate format that can be reproduced within the capabilities of the target platforms;
3. Mix and edit music in an appropriate format that can be reproduced within the capabilities of the target platforms;
4. Create music that can respond to events and user interactions as required;
5. Organise your work using appropriate filing and naming conventions so that it can be located easily by others;
6. Provide clear documentation as necessary for others to incorporate your work into the product;
7. Liaise with colleagues, such as designers and developers, to ensure your work is appropriate and meets requirements;
8. Liaise with the relevant authority to obtain approval for your work.

You might also find the following Open University course of interest: TA212 The technology of music. Here’s a summary: This joint technology/arts course starts with an introduction to music theory and notation and the technological techniques needed in a study of music technology. The principles of sound and acoustics are studied and the course relates musical terms and fundamentals to their physical equivalents. You will study the operation and characteristics of various musical instruments, how music can be represented and stored, the fundamentals of recording, manipulation and transmission of sound, MIDI, current developments and some associated legal/commercial issues.

You can find sample material from the course in the following OpenLearn Units:

• Sound for music technology: an introduction - an introduction to the physics of sound;
• Creating musical sounds - how traditional musical instruments create sound;
• Revolutions in sound recording - a history of the early days of analogue sound recording.

“Skillset is the industry body which supports skills and training for people and businesses to ensure the UK audio visual and publishing industries maintain their world class position.”

If you root around their website, you should be able to find the National Occupational Standards for Interactive Media and Computer Games.

As I’m posting quite a bit about storylines and narrative at the moment, I thought it might be useful to mention the Skillset standards relating to “Creating Narrative Scripts For Interactive Media Products (IM23)”.

IM23 CREATE NARRATIVE SCRIPTS FOR INTERACTIVE MEDIA PRODUCTS
Example job titles: Scriptwriter, Storyline Author

Overview
This unit is about your ability to help create an immersive experience for users by creating narrative scripts for interactive media products. You might be creating stories for games, interactive television or other interactive media product. While many of the skills required are the same as for creative writing generally,you will need to anticipate and account for the way interactivity and non-linearity can affect the user’s experience of a story.

Knowledge and Understanding
This is what you must know
a. Relevant theories and principles of narrative structure;
b. The characteristics and appeal of different genres of story;
c. Relevant theories relating to the wider cultural impact of stories;
d. Different types of interactive narrative structure (for example,tree and web);
e. User perspectives and points of view in relation to narratives in interactive products;
f. Principles of interaction design;
g. How to create an immersive experience for the target audience;
h. The broad technical constraints applicable to combining narrative with interactivity;
i. How user interaction and non-linearity can affect the user’s experience of the narrative or indeed the
narrative itself;
j. How to obtain and analyse product information to determine relevant creative or technical parameters that will affect your writing.

Awareness
This is what you must be aware of
i. Project parameters and constraints including target platforms and their capabilities;
ii. The overall product purpose,mood and target audiences;
iii. The types of interaction that will be available to users and which might influence the structure or style of your narrative;
iv. Any requirement for the storyline,events, perspectives or other aspects of the narrative to change in response to events or user interactions;
v. The resource and time implications of using different types of interactive narrative structure;
vi. Who you need to liase with from other specialisms to ensure your narratives will work in the intended
interactive product.

Performance Statements
This is what you must be able to [do]
1. Create narratives that are engaging for the intended audience and appropriate for the target platform(s) or technologies being used;
2. Define story worlds and characters in sufficient detail;
3. Write narratives that are consistent and congruent with the story world and the characters’ backgrounds,agendas,personalities and abilities;
4. Organise the narrative flow so that it is congruent with the story;
5. Provide clear instructions for programmers and designers regarding how and when user interactions or other events should affect the narrative (for example by indicating when a user interaction should cause the narrative to jump to a particular scene);
6. Liaise with colleagues,such as designers and developers,as appropriate to ensure that the narrative is appropriate and fit for purpose.

If you are considering working in the games or interactive media industry, then using the Skillset standards as a way of organising your own personal portfolio might well be a good idea.

This being an uncourse, there isn’t a syllabus as such (though I do keep referring to my interactive mindmap overview of the course we originally had in mind…), but hopefully the Digital Worlds blog will at least introduce many, if not all, of the issues identified in the Skillset standards (I’ll post more of them as and when appropriate).

To this end, if anyone fancies using a collaborative online mindmapping tool (such as Mindmeister for example) to outline the Skillset standards and maybe relate individual Digital Worlds posts to the appropriate items, feel free to invite the rest of us to help via a comment… or blog about it and link back here…

See also: getting into the games industry [presentation], by truna aka j.turner.