We just released a Feb. 5 '89 prototype of DuckTales for the NES!
If you'd like to support our preservation efforts (and this wasn't cheap), please consider donating or supporting us on Patreon. Thank you!

Format:Krawall

From The Cutting Room Floor
Jump to navigation Jump to search

The Krawall sound engine is an XM/S3M module player/tracker for the Game Boy Advance. Written by Sebastian Kienzl, it was used in various licensed games, such as Lord of the Rings, Spiderman, and The Sims.[1] Since 2013 the source code for the engine has been available on GitHub.[2] Krawall can be used to play background music in a game, as well as to play tracked single-activation jingles. The data is stored in the ROM as a set of samples, an optional set of instruments (depending on whether it's using S3M or XM modules), and one or more modules, with up to 256 patterns per module.

Signature

Most ROMs with Krawall have a signature somewhere in the file that declares the presence of Krawall as well as the version. (Some retail ROMs have this stripped out. The Sims Bustin' Out is one example, though interestingly enough the Japanese release still has the signature.) The signature will likely look similar to the following in ASCII form:

$Id: Krawall $Id: version.h 8 2005-04-21 12:24:45Z seb $

Depending on the version, it may also be in a different format:

$Id: Krawall $Date: 2003/09/01 06:51:01 $

The first form of the string identifies the engine name (Krawall), the file the string came from, the build date of the version, and the creator ("seb" -> Sebastian Kienzl). The second form identifies just the engine name and the build date.

This signature is a surefire way to detect the presence of Krawall in a ROM. If it is not present, you may need to use a more heuristic scan on the ROM to check for modules/samples. You can also look for seb's name in the credits.

Versions

There have been various versions of Krawall since its initial release. The most recent version is 2005-04-21. The most important change that was made to the format was in version 2004-07-07, when the patterns were modified to allow more than 512 instruments and 255 rows. Make sure you know which version of Krawall is used by the ROM before ripping, as the two pattern formats are incompatible. Look for the signature in the ROM if present to find the version.

Known versions
  • 2003-09-01
  • 2004-07-07
  • 2004-09-17
  • 2004-11-14
  • 2005-04-21

Data Format

Krawall stores its data in multiple places. Samples for all modules are stored in a single bank, as well as a separate bank for instruments if present. Both banks are indexed using a central array of pointers. Modules are stored individually, with pointers to each pattern at the end. Most of the data stored in each structure is analogous to a property in the comparable XM or S3M structure, so this makes conversion between the two/three formats easy.

All properties are stored in little-endian format. Pointers are stored as they would be addressed on the GBA, with a base address of 08000000. When converting down to ROM offsets, make sure to bitwise AND any pointers with 01FFFFFF. When converting up from ROM offsets, bitwise OR the offset with 08000000.

Samples

Samples are stored as signed 8-bit PCM data, with an 18 byte header with a few properties relating to looping and frequency.

Sample structure format
Offset Size (bytes) Source name Description
00 4 loopLength The length of the loop region. The start of the loop can be determined with (size - loopLength), and the end of the loop is always the end of the sample (size).
04 4 end A pointer to the end of the data. The size of the data can be determined by subtracting the position of the start of the data from this pointer (AND 01ffffff).
08 4 c2freq The frequency for middle C (used for S3M), in Hertz.
0C 1 fineTune The finetune value for the sample (used for XM), as a signed integer.
0D 1 relativeNote The relative note for the sample (used for XM), as a signed integer.
0E 1 volDefault The default volume for the sample.
0F 1 panDefault The default pan position for the sample.
10 1 loop A boolean value determining whether the sample should loop. If set, once sample playback reaches the end it will loop back to (size - loopLength) samples.
11 1 hq A boolean value telling Krawall whether to play back the sample with interpolation. (0 = no interpolation, 1 = linear? interpolation)
12 (size) data[] The actual sample data, as signed 8-bit PCM.

Pointers to all samples for a ROM are stored in a central pointer array. The index of each sample in this array is used for instrument sample assignments (in XM) or instrument selection in patterns (in S3M).

You can easily find samples in a ROM by opening it as raw data in Audacity or other program with signed (or unsigned if unavailable) 8-bit mono PCM at 16384 Hz. Look for the first sample of a region that appears to be audio, and backtrack 18 samples before that. Then seek to the position in the file equal to the sample position that you just got. This should be the beginning of a sample structure. Verify that the size property is valid, and if so you've now got a sample structure.

Once you have the position of a sample structure, it should be trivial to find the master sample table. Just search the ROM for the offset of the start of the sample structure (bitwise OR'ed with 08000000) as a little-endian 32-bit integer, and you should be able to find the rest of the table.

Instruments

Instruments are used for modules converted from XM format. They can use multiple samples for different pitches to avoid distortion when playing a note that is much higher than the original sample. They also store volume and panning envelope information, as well as vibrato types. Instrument structures are always exactly 302 bytes in size. Some modules may not use instruments - check the flagInstrumentBased flag in each module to see whether instruments are required. Typically, modules converted from XM have instruments, while modules converted from S3M do not.

Instrument structure format
Offset Size (bytes) Source name Description
000 192 (2[96]) samples[96] An array of 96 unsigned 16-bit integers denoting which notes use which samples. Index 0 = C0, 1 = D0, ..., 95 = A#7. The value denotes the index of the sample in the master sample table.
0C0 52 (Envelope) envVol An Envelope structure containing the volume envelope. (See below for the format of the Envelope structure.)
0F4 52 (Envelope) envPan An Envelope structure containing the panning envelope. (See below for the format of the Envelope structure.)
128 2 volFade Volume fade out value.
12A 1 vibType The type of vibrato used for the instrument. This should be the same as the XM vibrato type field.
12B 1 vibSweep The number of ticks to reach full vibrato speed.
12C 1 vibDepth The depth of the vibrato in 1/64ths of a semitone.
12D 1 vibRate The speed to play the vibrato at.
Envelope structure format
Offset Size (bytes) Source name Description
00 48 (4[12]) nodes[12] The nodes in the envelope. This is packed as an array of 12 pairs of two unsigned 16-bit integers. The first integer (coord) stores the X and Y position of the node. The X position is in the 9 least significant bits, while the Y position is in the 7 most significant bits. The second integer (inc) is used internally as the amount to add each tick to reach the next node (essentially a slope). This is not necessary when ripping, but if you are modifying an instrument's envelope you'll want to recalculate the inc field accordingly.
30 1 max The index of the final node (NOT the number of nodes). Add one to get the total number of nodes.
31 1 sus The node where the instrument will sustain until fadeout.
32 1 loopStart The node to restart at if the envelope finishes.
33 1 flags A bitfield with some flags determining behavior of the envelope. 2: sustain the envelope at sus. 4: loop the envelope at the end, restarting from loopStart.

As with the sample structures, pointers to all instruments are kept in a central array to be referenced by the game/Krawall. The indices of the instruments in the array are used when playing back patterns.

Most of the data in the instrument structure is analogous to the XM format's instrument structure's fields. This is likely because instruments are only available when converting from the XM format, so it can stay the same for the most part. However, some semi-redundant data (like envelope loop end) was removed to save space.

Modules

Modules store the metadata for each song. They contain the same sort of information as would be found at the top of an XM/S3M module file. Since Krawall handles both XM and S3M modules, there are some fields that are only useful for one of the formats and not the other. These will be noted in the table below.

Module structure format
Offset Size (bytes) Source name XM/S3M Description
000 1 channels XM/S3M The number of audio channels in the module.
001 1 numOrders XM/S3M The number of patterns in the order table. This is not necessarily equal to the total number of patterns!
002 1 songRestart XM The position in the module to restart at.
003 256 (1[256]) order[256] XM/S3M The order of patterns in the module. Each value refers to the index of a pattern in patterns. A value of 254 tells Krawall to call a user-defined callback function.
103 32 (1[32]) channelPan[32] S3M Initial panning positions for each channel (up to 32), as signed 8-bit integers.
123 64 (1[64]) songIndex[64] N/A Holds the indices of start positions for modules with multiple songs.
163 1 volGlobal XM/S3M Default volume for the module.
164 1 initSpeed XM/S3M The default speed (ticks/beat) for the module.
165 1 initBPM XM/S3M The default tempo (beats/minute) for the module.
166 1 flagInstrumentBased N/A Determines whether the module uses instruments (1) or not (0). S3M modules always set 0, and XM modules usually set 1.
167 1 flagLinearSlides XM Determines whether the module uses linear frequency slides (1) or Amiga frequency slides (0).
168 1 flagVolSlides S3M Determines whether the module uses ScreamTracker 3.00 volume slides (1) or not (0). When set to 1, play volume slides on the first tick as well as the others; when cleared to 0, don't play slides on the first tick.
169 1 flagVolOpt S3M Determines whether the module uses volume optimization (1) or not (0). If set and the volume on a channel is 0 for more than two rows, the channel is stopped.
16A 1 flagAmigaLimits S3M Determines whether the module uses Amiga frequency limits (1) or not (0). If set, any note that goes beyond Amiga hardware limits is illegal.
16B 1 __padding N/A Unused. Pads pattern pointers to 4-byte offsets.
16C * patterns[] XM/S3M The pointers to each pattern used by the module. The size of this array is determined by the largest index in the order array: if the largest value was 7, there would be 8 entries in this array. Make sure to ignore 254, as noted above.

Modules are directly referenced by the code, and thus do not have any sort of central pointer list. To find the offsets, you will need to either search the code regions for pointers to the module, or you can look for a block of data that roughly matches the format of the structure. One key characteristic is that there are often large regions of zero bytes in the order and songIndex arrays. You can try to search for these zero regions to find possible modules.

Patterns

Patterns store the actual notes used in a module. They can be reused and rearranged in a module as needed. Patterns are split into a certain number of rows, which are played back at a set speed. There are two versions of the pattern format with some minor variations, so you'll need to make sure you're using the correct version before reading patterns. A pattern consists of some tracking indices, the number of rows, and packed pattern data.

Pattern structure format (>= 2004-07-07)
Offset Size (bytes) Source name Description
00 32 (2[16]) index[16] The offsets of the first 64 rows inside data. Only every fourth row is stored in here. For example, to find the offset of the 21st row (14), check index[5]. This is likely used for seeking in a pattern.
20 2 rows The number of rows in the pattern. As of version 2004-07-07, this field is 16 bits wide.
22 * data[] The packed pattern data.
Pattern structure format (< 2004-07-07)
Offset Size (bytes) Source name Description
00 32 (2[16]) index[16] The offsets of the first 64 rows inside data. Only every fourth row is stored in here. For example, to find the offset of the 21st row (14), check index[5]. This is likely used for seeking in a pattern.
20 1 rows The number of rows in the pattern. Before version 2004-07-07, this field was 8 bits wide.
21 * data[] The packed pattern data.

The format of the packed data is very similar to S3M's packed data format. Each row is encoded as a list of note entries, with an initial channel/follow byte, followed by the data being set on the channel. A row can consist of zero or more entries with the most significant three bits denoting what's being set, and the least significant 5 bits denoting the channel. If the follow byte is 0, the row is finished and the next row is starting.

To decode the packed data:

  1. Read a byte from the data
  2. If the byte is 0, increment the row and restart the loop (if the row == rows, we're done)
  3. If bit 5 is set (20): read two bytes for the note and instrument
    • If the version is >= 2004-07-07, the first byte is the note, and the second byte is the least significant byte of the instrument
      • If the high bit is set on the note, unset it and read in another byte for the most significant byte of the instrument
    • If the version is < 2004-07-07, the high 9 bits are the instrument, and the low 7 bits are the note
  4. If bit 6 is set (40): read 1 byte for the volume (in XM volume column format, 10-50 for volume maps to 00-40 actual volume)
  5. If bit 7 is set (80): read 2 bytes for the effect and effect parameter (listed below)
  6. Write the data to the channel at the low 5 bits of the first byte (follow & 0x1F)
  7. Repeat from the top

Effects

These are the effects available in Krawall, as well as mappings to XM and S3M effects.

Krawall effect IDs
ID XM equivalent S3M equivalent Description
0 - - None
1 Fxx Axx Speed
2 Fxx Txx Tempo
3 Fxx - Speed/Tempo
4 Bxx Bxx Pattern Jump
5 Dxx Cxx Pattern Break
6 Axy1 Dxy Volume Slide (S3M)
7 Axy Dxy1 Volume Slide (XM)
8 EBx DFx Fine Volume Slide
9 EAx DxF Extra Fine Volume Slide
10 2xx Exx1 Portamento Down (XM)
11 2xx1 Exx Portamento Down (S3M)
12 E2x EFx Fine Portamento Down
13 X2x ExF Extra Fine Portamento Down
14 1xx Fxx1 Portamento Up (XM)
15 1xx1 Fxx Portamento Up (S3M)
16 E1x FFx Fine Portamento Up
17 X1x FEx Extra Fine Portamento Up
18 Cxx - Note Volume
19 3xx Gxx Note Portamento
20 4xy Hxy Vibrato
21 Txy Ixy Tremor
22 0xx Jxx Arpeggio
23 6xy Kxy Volume Slide + Vibrato
24 5xy Lxy Volume Slide + Portamento
25 - Mxx2 Channel Volume
26 - Nxx2 Channel Volume Slide
27 9xx Oxx Sample Offset
28 Pxy Pxy2 Panning Slide
29 Rxy Qxy Retrigger
30 7xy Rxy Tremolo
31 - Uxx Fine Vibrato
32 Gxx Vxx Global Volume
33 Hxy Wxy Global Volume Slide
34 8xx Xxy Pan
35 Yxy2 Yxy2 Panbrello
36 - - Mark (calls user callback)
37 E3x S1x Glissando
38 E4x S3x Set Vibrato Wave
39 E7x S4x Set Tremolo Wave
40 X5x2 S5x2 Set Panbrello Wave
41 X6x2 S6x Fine Pattern Delay
42 E8x S8x Rough Pan (old)
43 E6x SBx Pattern Loop
44 ECx SCx Note Cut
45 EDx SDx Note Delay
46 EEx SEx Pattern Delay
47 Lxx - Set Envelope Position
48 - SAx2 Sample Offset (High)
49 6xx Kxx Volume Slide + Vibrato (XM)
50 5xx Lxx Volume Slide + Portamento (XM)

1Effect is slightly incompatible with the conversion, you may need to adjust its behavior.
2Effect is only available in ModPlug/OpenMPT, it will not work in other trackers.

Tools

To convert XM or S3M modules to Krawall format, the Krawerter program included in the Krawall sources should work well. This method was likely used by publishers to convert their music for games. To rip music from a Krawall ROM (or other compiled binary with Krawall data), UnkrawerterGBA by JackMacWindows can rip music into XM and S3M files.

Converting modules to WAV/FLAC/MP3

Since XM and S3M are module formats, to get playable audio files you will need to convert them using a tracker program. OpenMPT is recommended because it supports some special effects that are incompatible with the base specifications, but are implemented in Krawall and OpenMPT. The following instructions will detail how to convert a module using OpenMPT.

  1. Open the module file in OpenMPT.
  2. Go to File => Stream Export (WAV, FLAC, MP3, etc.)...
  3. Select your file type, sample rate, channel count, and bitrate under the Format box.
  4. Select the loop type under the Limit box.
  5. Optionally, add metadata under the Tags box.
  6. Make sure Export To: File is selected, and click OK.
  7. Browse to the destination, enter the file name, and click Save.
  8. Wait for the audio to be rendered.

Once complete, the audio file will be located where you saved it.

References