01076531 Multimedia Systems 01076531 Multimedia Systems

Transcript

1 Department of Computer Engineering, Faculty of Engineering King Mongkut’s Institute of Technology Ladkrabang 01076531 MULTIMEDIA SYSTEMS P k Pakorn Watanachaturaporn, W t h t Ph.D. Ph D [email protected], [email protected] Chapter 5 Digital Audio Processing 2 Tools for Digital Audio Processing … Digital Audio Work Environments 3 Tools for Digital Audio Processing … Audio Connectors XLR RCA Composite TRS 4 Tools for Digital Audio Processing … Audio Connectors 5 Tools for Digital Audio Processing … Audio Connectors 6 Tools for Digital Audio Processing … Audio Connectors 7 Tools for Digital Audio Processing … D Digital lA Audio d Processing P Software S f † † E.g., Audacity, Digidesign Pro, Adobe Audition, Apple Logic, Sony Sound Forge, Forge Nuendo, Nuendo Cakewalk Music Creator, Creator DAP, DAP MixViews, MixViews ReZound, Kwave, Brahms, Jazz++, BladeEnc, etc. Basic Features „ „ „ „ „ „ „ „ „ Import/save audio files Interfacing for recording and playing sound A waveform view Multitrack editor Audio restoration tools S i l effects Special ff t Control for equalizing Frequency filters Compression codecs 8 Tools for Digital Audio Processing … Destructive vs. Nondestructive Editing † Desctructive „ „ „ † An audio processing program create a temporary file on a hard disk and write any change to this temporary copy of the audio file If you never save the edited version version, the temp file goes away If you do save the edited audio, the previous version is overwritten with the temp file Nondestructive „ „ Be handled by means of additional instructions that are added to the audio di fil file Saving the audio file in a file format that accommodates these extra instructions 9 Tools for Digital Audio Processing … Digital Audio File Types † There are three major groups of audio file formats: „ „ „ Uncompressed audio formats, „ E.g., WAV, AIFF, AU or raw header-less PCM; f formats t with ith lossless l l compression, i „ E.g., FLAC, Monkey's Audio (APE), WavPack (WV), Shorten, TTA, ATRACAdvanced Lossless, Apple Lossless, MPEG-4 SLS, MPEG-4 ALS, MPEG-4 DST, Windows Media Audio Lossless (WMA Lossless). formats with lossy compression, compression „ E.g., MP3, Vorbis, Musepack, AAC, ATRAC and lossy Windows Media Audio (WMA). 10 Dynamics Processing … … The process of adjusting the dynamic range of an audio selection either to reduce (attenuation) or to increase (gain or boost) the difference between the loudest and softest passages The purpose of adjusting dynamic range is to improve the texture of balance of sound † Texture of music is the way the melodic, rhythmic, and harmonic materials are combined in a composition, thus determining the overall quality of sound of a piece 11 Dynamics Processing Original audio selection Compressed dynamic range Original audio selection Expanded dynamic range T Type off dynamic d i range compression i and d expansion i 12 Dynamics Processing … D Downward d compression i † … Upward compression † … Raises the amplitude of signals that are below a designated level, level without altering the amplitude of signals above the designated level. It reduces the dynamic range. Upward expansion † … Lowers the amplitude of signals that are above a designated level, without changing the amplitude of signals below the designated level. It reduces the dynamic range. Raises the amplitude of signals that are above a designated level, without changing the amplitude of signals below that level. It increases the d dynamic namic range range. Downward expansion † Lowers the amplitude p of signals g that are below a designated g level,, without changing the amplitude of signals above this level. It increases the dynamic range. 13 Dynamics Processing 14 Dynamics Processing … Audio limiting † Hard limiting – the recording system does not allow sound to be recorded above a given amplitude † Clipping – cuts amplitudes of samples to a given maximum and/or minimum level † Soft limiting – audio signals above the designated amplitude are recorded at lower amplitude † Both hard and soft limiting cause some distortion of the waveform 15 Dynamics Processing … Normalization † A process which raises the amplitude of audio signal values and thus the perceived loudness of an audio selection † The normalization algorithm proceeds as follows: „ „ „ Find the highest amplitude sample in the audio selection Determine the gain needed in the amplitude to raise the highest amplitude to maximum amplitude (0 dBFS by default, or some limit set by the user) Raise all samples in the selection by this amount 16 Audio Restoration … Three basic types of audio restoration Noise gating † Noise reduction † Click and pop removal † 17 Audio Restoration … N i gating Noise ti † Serves as a block to signals below a given p threshold amplitude † Attack – how quickly the gate to open † Release – how quickly the gate to close † Hold – the minimum amount of time that the gate must stay open † Hysteresis „ Handle cases where the audio signal hovers around the threshold and create a kind of chatter „ IIndicate di the h diff difference b between the h value l that h caused the gate to open and the value that will cause it to close again „ If the difference is large enough to contain the fluctuating signal, the noise gate will not cause chatter Interface to a noise gate (from Logic Pro) 18 Audio Restoration … N i reduction Noise d ti † Eliminate noise in a digital audio file after the file has been recorded † The first step is to get a profile of the background noise „ „ „ Selecting an area that should be silent, but that contains a hum or buzz The noise reduction tool does a spectral analysis of the selected area in order to determine the frequencies in the noise and their corresponding amplitude level Then, the entire signal is processed in section. If these section contain frequency components similar to the noise, these can be eliminated below certain amplitudes Interface to a noise reduction (from Audition) 19 Audio Restoration White Noise Pink vs White Noise Pink Noise 20 Audio Restoration … Cli k or pop eliminator Click li i t † Detect a sudden amplitude change g and eliminate this change g by interpolating the sound wave between the start and end point of the click or pop 21 Digital Audio Filters and Related Processing … Types of Digital Audio Filters A digital audio filter is a linear system that changes the amplitude or phase of one or more frequency components of an audio signal † Can C be b applied li d to either i h analog l or di digital i l audio di signals i l † 22 Digital Audio Filters and Related Processing … Types of Digital Audio Filters † FIR (finite-impulse response) filter 23 Digital Audio Filters and Related Processing … Types of Digital Audio Filters † IIR (infinite-impulse response) filter 24 Digital Audio Filters and Related Processing … Impulse and Frequency Response † Impulse response – FIR / IIR „ † A unit impulse (a.k.a. delta f ti ) – a vector function) t off value l th where the 0 value in the vector is 1 all the rest are 0 Frequency response „ A counterpart to the impulse response 25 Digital Audio Filters and Related Processing … Filters and Related Tools in Digital Audio Processing Software † Band filter „ † L Low-pass, hi h high-pass, b d bandpass, b d t filter bandstop filt IIR filter (also called scientific filters) „ „ Bessel,, Butterworth,, Chebyshev y 1,, Chebyshev y 2 Fine tune control over the frequency bands that are permitted to pass through and the way in which phase is affected † Comb filter – add delayed versions of a wave to itself, itself resulting in phase cancellations that can be perceived as echo † Convolution filter – can be used to add an acoustical environment to a sound file 26 Digital Audio Filters and Related Processing … Filters and Related Tools in Digital Audio Processing Software † Graphic filter – gives a graphical view that allows you to adjust the gain of frequencies within an array of bands † Parametric equalizer – similar to a graphic equalizer but with control over the width of frequency bands relative to their location † Crossover – splits an input signal into several output signals, each within a certain frequency range, so that each output signal can be directed to a speaker that handles that range of frequency best 27 Digital Audio Filters and Related Processing … Equalization 28 Digital Audio Filters and Related Processing … Equalization 29 Digital Audio Filters and Related Processing … Parametric EQ 30 Digital Audio Filters and Related Processing … Parametric EQ 31 Digital Audio Filters and Related Processing … Comb Filters, Delay, Reverb, and Convolution Filters 32 Digital Audio Filters and Related Processing … Comb Filters, Delay, Reverb, and Convolution Filters 33 Digital Audio Filters and Related Processing … Comb Filters, Delay, Reverb, and Convolution Filters 34 Digital Audio Filters and Related Processing … Comb Filters, Delay, Reverb, and Convolution Filters 35 Digital Audio Filters and Related Processing … The relationship between convolution and the Fourier transform 36 Digital Audio Compression … Time-based compression methods Reduce the amount of data in voice signals by quantizing highamplitude signals more coarsely than low-amplitude ones † DPCM – recording the difference between one sample and the next rather h than h recording di the h actuall sample l value l † These methods are not really compression at all. Rather, they are ways off reducing d i the th amountt off data d t att the th momentt an audio signal is digitized – often be considered conversion techniques rather than compression methods † 37 Digital Audio Compression … P Perceptual l Encoding E d † † Based upon an analysis of how the ear and brain perceive sound, called psychoacoustical modeling Psychoacoustical tests „ „ „ „ Shown a great deal of nonlinearity in human sound perception Human hears the best in the 100 – 5000 Hz One have to play the 100 Hz tone louder than the 1000 Hz tone to make them equally loud Human ability to distinguish between frequencies decreases nonlinearly from lower to higher frequencies „ At lower freq., q , we can distinguish g between sound waves that are onlyy a few Hz apart „ At higher freq., the frequencies must be a hundred or more Hz apart for us to hear the difference 38 Digital Audio Compression … Perceptual Encoding 39 Digital Audio Compression … MPEG Audio Compression † Overview „ „ Motion Picture Experts Group (MPEG) Standardized by ISO/IEC „ MPEG-1: MPEG 1 covers CD quality lit and d audio di suitable it bl for f video id games „ MPEG-2: covers multichannel surround sound „ MPEG-4: covers a wide range g of audio including g simple p voice „ MPEG-7: Multimedia Content Description 40 Digital Audio Compression … MPEG Audio Compression 41 Digital Audio Compression … MPEG-1 Audio Compression 42 Digital Audio Compression … MPEG-1 Audio Compression 43 Digital Audio Compression … MPEG-1 Audio Compression † † Divide the audio file into frames and analyze the psychoacoustical properties of each frame individually Apply a bank of filter, separate the signal into frequency b d bands 44 Digital Audio Compression … M G Audio MPEG-1 A d Compression C † † † † Perform a Fourier transform on the samples in each band in order to analyze the band band’ss frequency spectrum Analyze the influence of tonal and nontonal elements in each p sinusoidal components, p , such band. ((Tonal elements are simple as frequencies related to melodic and harmonic music. Nontonal elements are transients like the strike of a drum or the clapping of hands) Determine how much each band’s influence is likely to spread e g bo g frequency eque cy bands ba ds too neighboring Find the masking threshold and signal-to-mask (SMR) for each band, and determine the bit depth of each band accordingly 45 Digital Audio Compression … MPEG-1 Audio Compression 46 Digital Audio Compression … MPEG-1 Audio Compression Quantize the samples for the band with the appropriate number of bits, possibly following this with Huffman encoding † „ MPEG-1 Layers 1 and 2 use linear quantization, while MP3 uses nonlinear 47