Delays are a fundamental tool used in audio applications Time alignment of speaker arrays (sound reinforcement, live sound, home theater) Lip sync processing (home theater) delay based effects (recording studio, live sound)
Delays are a fundamental tool used in audio applications Time alignment of speaker arrays (sound reinforcement, live sound, home theater) Lip sync processing (home theater) delay based effects (recording studio, live sound)
Delays are a fundamental tool used in audio applications Time alignment of speaker arrays (sound reinforcement, live sound, home theater) Lip sync processing (home theater) delay based effects (recording studio, live sound)
Delay Based Audio Effects Using the TMS320C6713 DSP Lee Minich President Lab X Technologies, LLC lee@labxtechnologies.com 2 TI DSP Developer Online Training Session Objectives Learn the basics of fixed and variable delays Learn basic technique for implementing sub- sample delays Learn how to efficiently implement fixed and variable delays using the TMS320C6713 DSP Investigate performance trade-offs implementing delays 3 TI DSP Developer Online Training Why Delays are Important? Delays are a fundamental tool used in audio applications Time alignment of speaker arrays (sound reinforcement, live sound, home theater) Lip sync processing (home theater) Delay based effects (recording studio, live sound) 4 TI DSP Developer Online Training Who uses Delays? Sound reinforcement/Live sound Music Industry (effects manufacturers) Professional Studio equipment Home Theater 5 TI DSP Developer Online Training What You will Learn in this Session Firm understanding of fixed and variable delays See examples of C code implementing fixed and variable delays A Basic technique for implementing sub-sample delays Understanding of performance trade-offs Exposure to various audio applications for delay 6 TI DSP Developer Online Training Agenda Introduction Fixed Digital Delay Variable Digital Delay Implementation Details Further Investigation 7 TI DSP Developer Online Training Introduction Various audio applications based on Digital Delay. Time alignment (Live sound, Sound Distribution, Home Theater) Audio effects (Recording, FOH, MI). Two classes of delays Fixed Delay Variable Delay 8 TI DSP Developer Online Training Anatomy of a Digital Delay Write pointer Head Read pointer Tail Delay buffer Regeneration Mix coefficients of wet and dry Delay Buffer + + Output y(n) Input x(n) Regen MixWet MixDry Write Pointer Head Read Pointer Tail 9 TI DSP Developer Online Training Circular Buffer Delay buffer actually implemented as a circular buffer Metaphor of a recording platter, advancing once per sample. Modulo pointer arithmetic used in the calculation of read and write pointer positions Write pointer eventually overwrites oldest samples 10 TI DSP Developer Online Training Agenda Introduction Fixed Digital Delay Variable Digital Delay Implementation Details Further Investigation 11 TI DSP Developer Online Training Fixed Digital Delay As name implies, constant or Fixed delay length Typical uses include: Time alignment Canyon echo simulation Play Along delay effect Delay Buffer + + Output y(n) Input x(n) Regen MixWet MixDry Write Pointer Head Read Pointer Tail 12 TI DSP Developer Online Training Home Theater Time Alignment Example Speed of sound: 1116 ft / second (at sea level) Approx 1 ms = 1 ft 7.1 Surround System, 7 discrete channels of delay to time align each speaker to optimal listening location Listener Delay_RS Delay_RF Delay_C Delay_LF Delay_LS Delay_LB Delay_RB 13 TI DSP Developer Online Training Audio Effect Examples Canyon Echo simulation Popularized by people at canyons Play along Delay Popularized by U2s The Edge 14 TI DSP Developer Online Training Agenda Introduction Fixed Digital Delay Variable Digital Delay Flanger Chorus Implementation Details Further Investigation 15 TI DSP Developer Online Training Variable Digital Delay As name implies, delay length varies over time. Primary use is in audio effects Read pointer is driven by a Low Frequency Oscillator (LFO) Reading faster or slower than input sample rate creating a respective increase / decrease in pitch This pitch shifting can be thought of as Doppler shift, implying motion 16 TI DSP Developer Online Training Anatomy of a Variable Delay Exactly the same as a Fixed Delay except read pointer varies via a Low Frequency Oscillator (LFO) Typically variable delay composed of two portions: Static delay Variable delay Delay Buffer + + Output y(n) Input x(n) Regen MixWet MixDry Write Pointer Head Read Pointer Tail Static Variable 17 TI DSP Developer Online Training Low Frequency Oscillator (LFO) Oscillator 1.0 0.0 Width Integer . Fraction Static Delay Read Address Read pointer is driven by a Low Frequency Oscillator (LFO) As its name implies, this oscillator is a very low frequency (0.1 Hz to 2 Hz) Output of the LFO is both integer and fractional samples Integer portion of the LFO determines the specific samples involved in interpolation Fractional portion of the LFO is used in the interpolation to create sub-sample delays 18 TI DSP Developer Online Training Sample Skipping The crudest form of variable delay is moving the read pointer on integer sample boundaries. Sometimes referred to as sample skipping since the pointer skips from one sample to the next instead of smooth motion Results in noticeable zipper noise as the delay buffer is varied 19 TI DSP Developer Online Training Sub-sample interpolation Delay pointer can be moved fractional samples Accomplished by interpolating between successive samples. The simplest implementation is linear interpolation between two successive samples 20 TI DSP Developer Online Training Agenda Introduction Fixed Digital Delay Variable Digital Delay Flanger Chorus Implementation Details Further Investigation 21 TI DSP Developer Online Training Flanger Effect Legend has it that a recording engineer bumped against the flange of the tape reel while double tracking a guitar track. This momentarily slowed the tape speed, thus changing the pitch slightly, and subsequently recovering to the appropriate tape speed and pitch. Mixed with the original track caused the characteristic sound, thus Flanging , in the analog sense, was born. 22 TI DSP Developer Online Training Flanger Effect Frequency response of a Flanger is of primary importance. Creates harmonically related notches at multiples of delayLength/2 When viewed on a logarithmic frequency scale, these notches appear as teeth in a comb , and are thus sometimes called comb filters. Varying delay length over time changes the number and frequencies of notches Ideally, the delay length goes to zero, which allows the frequency notches to sweep to infinity Regen path used to intensify the effect 23 TI DSP Developer Online Training Flanger Audio Example Dramatic Sweeping effect Most famous example is Barracuda by Heart 24 TI DSP Developer Online Training Agenda Introduction Fixed Digital Delay Variable Digital Delay Flanger Chorus Implementation Details Further Investigation 25 TI DSP Developer Online Training Chorus Effect Simulation of multiple instruments playing simultaneously, creating thicker sound due to minute differences in pitch and timing Similar effects occur with multiple vocalists singing the same notes simultaneously in a chorus, thus the name Chorus Primary importance is pitch shift, not frequency response To avoid frequency notching associated with flangers, a minimum delay is introduced called a static delay Typically no regen path 26 TI DSP Developer Online Training Chorus Audio Example Popularized by textural players such as Andy Summers of The Police 27 TI DSP Developer Online Training Agenda Introduction Fixed Digital Delay Variable Digital Delay Implementation Details Sample by Sample vs. Block Processing Block Processing on the TMS320C6713 DSP Starter Kit (DSK) Using Ping-Pong Buffers Code Excerpts inside Block Processing Further Investigation 28 TI DSP Developer Online Training Sample by Sample Processing An input sample is processed and an output sample generated within a single audio sample. PRO: Minimal transport delay. CON: High overhead accessing the relatively slow A/Ds, external memories, and D/As. 29 TI DSP Developer Online Training External Memory Audio samples rates ranging from 44.1 kHz (Compact Disc), 48 kHz, 96 kHz, and now 192 kHz at widths from 20 to 32 bits Delays of tens of milliseconds consume 4410 to 19200 samples, per audio channel. Insufficient on chip RAM to meet these delay requirements Larger external bulk delay RAM is needed In high performance DSP such as TI TMS320C6713 DSP, external memory poses a bottleneck. 30 TI DSP Developer Online Training Block Processing A block of samples are gathered, a block of samples are processed, and a block of samples are output. PRO: Increased efficiency by amortizing overhead costs over a block of samples. CON: Increased transport delay. 31 TI DSP Developer Online Training Block Processing TMS320C6713 DSP Enhanced Direct Memory Access (EDMA) transfers, in which several external memory accesses can be queued up. EDMA hardware orchestrates the manipulation of buffer pointers. Primary process is notified when the samples have been cached into internal memory. Processing is done on an entire block, within fast internal memory, and coefficients can be cached to further improve performance. 32 TI DSP Developer Online Training Block Processing on TMS320C6713 DSK McBsP1 RX Ping EDMA RX Pong TX Pong TX Ping Block Processing TMS320C6713 External Hardware ADC DAC Frame Clock Left Channel Right Channel Left Channel Right Channel Analog Out Left Analog Out Right Analog In Left Analog In Right 33 TI DSP Developer Online Training RX/TX Ping-Pong Buffers Two receive (RX) buffers Two transmit (TX) buffers EDMA hardware orchestrates data transfers Moves samples to/from appropriate RX/TX buffers Generates interrupt with buffers are full Handles pointer manipulation to Ping/Pong between buffers on successive block transfers 34 TI DSP Developer Online Training Block Processing use of Ping/Pong buffers Processing opposite buffer EDMA is accessing Provides an entire block length period of time to complete processing 35 TI DSP Developer Online Training Inside Block Processing Split Left Buffer Right Buffer Merge Algorithm 0 Algorithm 1 Algorithm N TX Ping/Pong RX Ping/Pong External Left Buffer External Right Buffer 36 TI DSP Developer Online Training Processing the Buffers void processBuffer(void) { Uint32 pingPong; Int16 *receiveBuffer; Int16 *transmitBuffer; /* Get contents of mailbox posted by edmaHwi */ pingPong = SWI_getmbox(); /* Copy data from transmit to receive, could process audio here */ if (pingPong == PING) { /* Toggle LED #3 as a visual cue */ DSK6713_LED_toggle(3); receiveBuffer = gBufferRcvPing; transmitBuffer = gBufferXmtPing; } else { /* Toggle LED #2 as a visual cue */ DSK6713_LED_toggle(2); receiveBuffer = gBufferRcvPong; transmitBuffer = gBufferXmtPong; } splitData(receiveBuffer, leftProcessingBuffer, rightProcessingBuffer, BUFFSIZE); Flanger_Process(leftProcessingBuffer, rightProcessingBuffer, BUFFSIZE/2, variableDelayType_LinearInterpolation); Delay_Process(leftProcessingBuffer, rightProcessingBuffer, BUFFSIZE/2, 48000*3.5); mergeData(transmitBuffer, leftProcessingBuffer, rightProcessingBuffer, BUFFSIZE/2); } 37 TI DSP Developer Online Training Split /* * 2 16 bit words are packed into single 32 bit transfer. The input buffer * is interleaved left/right/left/right/... and subsequently split into * separate left and right buffers. */ void splitData(Int16 *inbuf, Int16 *leftBuff, Int16 *rightBuff, Int16 length) { length = length / 2; while(length--) { *leftBuff++ = *inbuf++; *rightBuff++ = *inbuf++; } } 38 TI DSP Developer Online Training Merge /* * 2 separate buffers are merged to form single output buffer * interleaved as left/right/left/right/.... * * NOTE: Assumes left and right buffers are equal length, and * that output buffer will be 2 times the input length */ void mergeData(Int16 *outbuf, Int16 *leftBuff, Int16 *rightBuff, Int16 length) { while(length--) { *outbuf++ = *leftBuff++; *outbuf++ = *rightBuff++; } } 39 TI DSP Developer Online Training Fixed Delay static void delay(Int16 *audioBuffer, Int16 audioBufferLength, Int32 delayLength, DelayInfo* delayInfo) { Int16 i = 0; Int16 input, output; Int32 index = delayInfo->index; delayLength = DELAY_BUFFER_LENGTH - delayLength; for (i = 0; i < audioBufferLength; i++) { output = delayInfo->buffer[(index + delayLength) % DELAY_BUFFER_LENGTH]; // get tail from within buffer input = audioBuffer[i]; delayInfo->buffer[index] = input + (delayInfo->regen * output); // update head with new input audioBuffer[i] = (delayInfo->mixDry * input) + (delayInfo->mixWet * output); // get tail from within buffer index = (index + 1) % DELAY_BUFFER_LENGTH; } delayInfo->index = index; } Delay Buffer + + Output y(n) Input x(n) Regen MixWet MixDry Write Pointer Head Read Pointer Tail 40 TI DSP Developer Online Training Allocating Delay Buffers in SDRAM #pragma DATA_SECTION(delayBufferLeft, " SDRAM" ) Int16 delayBufferLeft[DELAY_BUFFER_LENGTH]; #pragma DATA_SECTION(delayBufferRight, " SDRAM" ) Int16 delayBufferRight[DELAY_BUFFER_LENGTH]; 41 TI DSP Developer Online Training Delay Instance Data typedef struct { Int32 index; Int16* buffer; float regen; float mixDry; float mixWet; } DelayInfo; DelayInfo leftDelayInfo = { 0, // index delayBufferLeft, // buffer 0.5, // regen 0.707, // mixDry 0.707 // mixWet }; DelayInfo rightDelayInfo = { 0, // index delayBufferRight, // buffer 0.5, // regen 0.707, // mixDry 0.707 // mixWet }; void Delay_Process(Int16 *audioBufferLeft, Int16 *audioBufferRight, Int16 audioBufferLength, Int32 delayLength) { delay(audioBufferLeft, audioBufferLength, delayLength, &leftDelayInfo); // Left delay delay(audioBufferRight, audioBufferLength, delayLength, &rightDelayInfo); // Right delay } 42 TI DSP Developer Online Training Sample Skipping Flanger static void flangerSampleSkip(Int16 *audioBuffer, Int16 audioBufferLength, VariableDelayInfo *delayInfo) { Int16 i = 0; Int32 delayLength; Int16 input, output; Int32 index=delayInfo->index; LFOInfo lfoInfo = *(delayInfo->lfoInfo); for (i = 0; i < audioBufferLength; i++) { updateLFO(&lfoInfo); delayLength =(lfoInfo.lfo*delayInfo->width*LFO_MAX) + delayInfo->staticDelay; // calculate tail pointer delayLength = DELAY_BUFFER_LENGTH - delayLength; output = delayInfo->buffer[(index + delayLength) % DELAY_BUFFER_LENGTH]; // get tail from within buffer input = audioBuffer[i]; // update head with new sample delayInfo->buffer[index] = input + (delayInfo->regen * output); // write to buffer audioBuffer[i] = (delayInfo->mixDry * input) + (delayInfo->mixWet * output); // mix wet and dry paths index = (index + 1) % DELAY_BUFFER_LENGTH; // update head pointer } *(delayInfo->lfoInfo) = lfoInfo; // update instance variables delayInfo->index = index; } 43 TI DSP Developer Online Training Flanger via Linear Interpolation static void flangerLinearInterpolation(Int16 *audioBuffer, Int16 audioBufferLength, VariableDelayInfo *delayInfo) { Int16 i = 0; Int32 integerDelayLength; float fractionalDelayLength; Int16 input, output, currentSample; LFOInfo lfoInfo = *(delayInfo->lfoInfo); Int32 index = delayInfo->index; Int16 previousSample = delayInfo->previousSample; for (i = 0; i < audioBufferLength; i++) { updateLFO(&lfoInfo); // calculate tail pointer fractionalDelayLength =(lfoInfo.lfo*delayInfo->width*LFO_MAX) + delayInfo->staticDelay; fractionalDelayLength = (float)DELAY_BUFFER_LENGTH - fractionalDelayLength; integerDelayLength = fractionalDelayLength; // implicit cast to Int32 does floor() fractionalDelayLength -= integerDelayLength; // subtract off integer portion to leave fractional portion of pointer // get tail from within buffer currentSample = delayInfo->buffer[(index + integerDelayLength) % DELAY_BUFFER_LENGTH]; // linear interpolation between current and previ ous samples output = (currentSample * fractionalDelayLength) + (previousSample * (1.0 - fractionalDelayLength)); input = audioBuffer[i]; // update head with new sample delayInfo->buffer[index] = input + (delayInfo->regen * output); // write input and regenerated output audioBuffer[i] = (delayInfo->mixDry * input) + (delayInfo->mixWet * output); // mix wet and dry paths previousSample = currentSample // update previous sample index = (index + 1) % DELAY_BUFFER_LENGTH; // update head pointer } *(delayInfo->lfoInfo) = lfoInfo; // update instance variables delayInfo->index = index; delayInfo->previousSample = previousSample; } 44 TI DSP Developer Online Training Triangle Wave LFO typedef struct{ float lfo; float lfoStep; int sign; }LFOInfo; static float lfoStep(float speed) { return speed*2.0/SAMPLE_RATE; } static void updateLFO(LFOInfo* lfoInfo) { float speculativeStep; // Update LFO speculativeStep = lfoInfo->lfo + lfoInfo->lfoStep; if (speculativeStep < 0 || speculativeStep > 1) lfoInfo->sign = -lfoInfo->sign; lfoInfo->lfo += (lfoInfo->sign * lfoInfo->lfoStep); } 45 TI DSP Developer Online Training Transport Delay vs. CPU Usage Block Size Transport Delay in mSec % CPU Usage 2 0.0625 100+ 4 0.125 67.75 8 0.25 47.85 16 0.5 38.21 32 1 33.21 64 2 30.78 128 4 29.62 256 8 28.94 46 TI DSP Developer Online Training Agenda Introduction Fixed Digital Delay Variable Digital Delay Implementation Details Further Investigation 47 TI DSP Developer Online Training Further Investigation Multirate signal processing Reverberation Other LFO waveforms 48 TI DSP Developer Online Training References Effect Design (part 2) , Jon Dattorro, J ournal of the AES, October 1997 Multirate Digital Signal Processing , Crochiere & Rabiner, Prentice-Hall 1983 DSP Applications Using C and the TMS320C6x DSK, Rulph Chassaing, Wiley 2002 49 TI DSP Developer Online Training Summary Fixed Digital Delay Variable Digital Delay LFO Flanger/Chorus Implementation Details Block processing on TMS320C6713 DSK Use of Ping/Pong buffers Circular buffers Locating delay buffers in external memory Linear Interpolation sub-sample delays Transport Delay vs. Block Size 50 TI DSP Developer Online Training Copyright by Texas Instruments Incorporated Copyright Notice 51 TI DSP Developer Online Training TIs eLearning solutions provider Visit T&OE www.ti-eservices.com Reach your customers with audio/video presentations. On-demand delivery to TIers or TI customers worldwide, anywhere, anytime. Deliver your training online! T&OE has a volume publishing agreement for using Presedias PPT2Flash services. Find more info at http:/ / www.ti-eServices.com Steve Enty (enty@ti.com): 972-917-5029 Clark Hise (hise@ti.com): 972-917-1239 T&OE Services T&OE Services Breeze Copyright by Macromedia