초록 ▼

DTS Interactive provides low cost fully interactive immersive digital surround sound environment suitable for 3D gaming and other high fidelity audio applications, which can be configured to maintain compatibility with the existing infrastructure of Digital Surround Sound decoders. The component aud

DTS Interactive provides low cost fully interactive immersive digital surround sound environment suitable for 3D gaming and other high fidelity audio applications, which can be configured to maintain compatibility with the existing infrastructure of Digital Surround Sound decoders. The component audio is stored and mixed in a compressed and simplified format that reduces memory requirements and processor utilization and increases the number of components that can be mixed without degrading audio quality. Techniques are also provided for “looping” compressed audio, which is an important and standard feature in gaming applications that manipulate PCM audio. In addition, decoder sync is ensured by transmitting frames of “silence” whenever mixed audio is not present either due to processing latency or the gaming application.

대표청구항 ▼

1. A multi-channel interactive audio system, comprising:A memory for storing a plurality of audio components having positional coordinates as sequences of input data frames, each said input data frame including subband data and their scale factors that have been compressed and packed;A human input d

1. A multi-channel interactive audio system, comprising:A memory for storing a plurality of audio components having positional coordinates as sequences of input data frames, each said input data frame including subband data and their scale factors that have been compressed and packed;A human input device (HID) for receiving a positional input from a user;An application programming interface (API) that generates a list of audio components in response to the positional input; andAn audio renderer thatFor each audio component on the list, unpacks and decompresses the audio components' scale factors and, as needed, the subband data;Calculates scale factors for the subband data in accordance with the positional input relative to the positional coordinates;Mixes the audio components' subband data in the subband domain for each channel using the calculated scale factors for each channel;Compresses the mixed subband data and their scale factors for each channel;Packs and multiplexes the channels' compressed subband data and scale factors into an output frame; andPlaces the output frame into a queue for transmission to a decoder.2. The multi-channel interactive audio system of claim 1, wherein the audio renderer mixes only the subband data that is considered audible to the user.3. The multi-channel interactive audio system of claim 2, wherein the audio renderer determines which subbands are audible to the user by using the listed audio components' calculated scale factors to calculate the intra-subband masking effects and discard the inaudible audio components for each subband.4. The multi-channel interactive audio system of claim 3, wherein the audio renderer unpacks and decompresses the audio components' scale factors first, calculates the scale factors, determines the audible subbands, and then unpacks and decompresses only the subband data in the audible subbands.5. The multi-channel interactive audio system of claim 4, wherein the audio renderera. stores the unpacked and decompressed subband data in a left shifted format in the memory in which the sign bit of the N-bit subband data is aligned to the sign bit of the M-bit format and the M-N rightmost bits represent noise that is below a noise floor;b. for each subband, multiplies the audible subband data by their respective scale factor and adds them together into a sum;c. for each subband, multiplies the sum by the reciprocal of the maximum scale factor for the audible subband data to produce the mixed subband data;d. if the mixed subband data overflows the format, increments the maximum scale factor to the next largest value and repeats step c.6. The multi-channel interactive audio system of claim 1, wherein the input data frame further includes a header and a bit allocation table that are fixed from frame-to-frame so that only the content of the scale factors and subband data are allowed to vary but are otherwise of fixed size in the compressed stream.7. The multi-channel interactive audio system of claim 6, wherein the compressed subband data is coded with fixed length codes.8. The multi-channel interactive audio system of claim 7, wherein the audio renderer unpacks each piece of the N-bit subband data, where N varies across the subbands, as follows:a. Utilizes the FLCs and fixed bit allocation to calculate the position of the subband data in the input audio frame, extract the subband data and stores it in the memory as M-bit words where the leftmost bit is a sign bit; andb. Left shifts the subband data until its sign bit is aligned with the M-bit word's sign bit, the rightmost M-N bits remaining in said M-bit word as noise.9. The multi-channel interactive audio system of claim 8, wherein the audio renderer is hardcoded for the fixed header and bit allocation table so that the audio renderer only processes the scale factors and subband data to increase speed.10. The multi-channel interactive audio system of claim 1, wherein the audio render interfaces with an application that provides equalization of the audio components, said audio renderer equalizing each said audio component by modifying its scale factors.11. The multi-channel interactive audio system of claim 1, wherein the audio render interfaces with an application that provides for sideways localization of the audio components, said audio renderer sideways localizing the audio components by applying a phase positioning filter to the subband data.12. The multi-channel interactive audio system of claim 1, wherein the input and output frames also include a header and a bit allocation table, the audio renderer providing for the seamless generation of output frames to maintain decoder sync by,a. placing a null output template in the queue that includes the header, the bit allocation table and subband data and scale factors that represent an inaudible signal;b. if the next frame of mixed subband data and scale factors is ready, writing the mixed subband data and scale factors over the previous output frame and transmitting the output frame; andc. if the next frame is not ready, transmitting the null output template.13. The multi-channel interactive audio system of claim 1, wherein said decoder is a Digital Surround Sound decoder that is capable of decoding prerecorded multi-channel audio, said audio renderer transmitting a sequence of said output frames that provide real-time interactive multi-channel audio with the same format as the prerecorded multi-channel audio.14. The multi-channel interactive audio system of claim 13, further comprising a single bandlimited connector, said audio renderer transmitting, in real-time and in response to the user input, the output frames as a unified and compressed bitstream over the single bandlimited connector to the Digital Surround Sound decoder, which decodes the bitstream into the interactive multi-channel audio whose bandwidth exceeds that of the single bandlimited connector.15. The multi-channel interactive audio system of claim 1, further comprising a single bandlimited connector, said audio renderer transmitting, in real-time and in response to the user input, the output frames as a unified and compressed bitstream over the single bandlimited connector to the decoder, which decodes the bitstream into multi-channel audio whose bandwidth exceeds that of the single bandlimited connector.16. The multi-channel interactive audio system of claim 1, wherein one or more of the audio components comprise looped data having commencing input frames and closing input frames whose subband data has been preprocessed to ensure seamless concatenation with the commencing frame.17. A multi-channel interactive audio system, comprising:A memory for storing a plurality of audio components as sequences of input data frames in a bitstream that is coded with fixed length codes (FLCs), each said input data frame including a header, a bit allocation table, subband data and their scale factors that have been compressed and packed, said header and bit allocation table being fixed from component-to-component, channel-to-channel and frame-to-frame so that only the content of the scale factors and subband data are allowed to vary;A human input device (HID) for receiving input from a user;An application programming interface (API) that generates a list of audio components in response to the user input; andAn audio renderer, which is hardcoded for the fixed header and bit allocation table, thatFor each audio component on said list, unpacks and decompresses the audio components' scale factorsCalculates scale factors for the mixed subband data in accordance with the user input;Uses the scale factors to determine the audible subband data;Unpacks and decompresses only the audible subband dataMixes the audible subband data in the subband domain for each channel using the calculated scale factors;Compresses the mixed subband data and their scale factors for each channel;Packs and multiplexes the channels' compressed subband data and scale factors into an output frame; andPlaces the output frame into a queue for transmission to a decoder.18. The multi-channel interactive audio system of claim 17, wherein the audio renderer unpacks each piece of the N-bit audible subband data, where N varies across the subbands, as follows:a. Utilizes the FLCs and fixed bit allocation to calculate the position of the audible subband data in the input audio frame, extract the audible subband data and stores it in the memory as M-bit words where the leftmost bit is a sign bit; andb. Left shifts the audible subband data until its sign bit is aligned with the M-bit word's sign bit, the rightmost M-N bits remaining in said M-bit word as noise.19. The multi-channel interactive audio system of claim 17, wherein the decoder is a Digital Surround Sound decoder that is capable of decoding pre-recorded multi-channel audio.20. The multi-channel interactive audio system of claim 17, wherein the audio renderer generates a seamless sequence of output frames bya. placing a null output template in a queue for transmission to a decoder that includes the header, the bit allocation table and subband data and scale factors that represent an inaudible signal;b. if the next frame of mixed subband data and scale factors is ready, writing the mixed subband data and scale factors over the previous output frame and transmitting the output frame; andc. if the next frame is not ready, transmitting the null output template.21. A multi-channel interactive audio system, comprising:A memory for storing a plurality of audio components as sequences of input data frames, each said input data frame including a header, a bit allocation table, and audio data that has been compressed and packed;A human input device (HID) for receiving input from a user;An application programming interface (API) that generates a list of audio components in response to the user input; andAn audio renderer that generates a seamless sequence of output frames bya. placing a null output template in a queue for transmission to a decoder that includes the header, the bit allocation table and subband data and scale factors that represent an inaudible signal;b. concurrently unpacking and decompressing the audio components' data as needed, and for each channel, calculating scale factors for the mixed data in accordance with the user input, mixing the audio components' data for each channel, compressing the mixed data for each channel, and packing and multiplexing the channels' compressed data;c. if the next frame of mixed data is ready, writing the mixed data over the previous output frame and transmitting the output frame; andd. if the next frame is not ready, transmitting the null output template.22. The multi-channel interactive audio system of claim 21, wherein the decoder is a Digital Surround Sound decoder that is capable of decoding pre-recorded multi-channel audio.23. The multi-channel interactive audio system of claim 21, wherein the audio data comprises subband data and its scale factors, the audio renderer mixing only the subband data that is considered audible to the user.24. The multi-channel interactive audio system of claim 23, wherein the audio renderer determines which subbands are audible to the user by using the listed audio components' scale factors to calculate the intra-subband masking effects and discard the inaudible audio components for each subband.25. The multi-channel interactive audio system of claim 24, wherein the audio renderer unpacks and decompresses the audio components' scale factors first, determines the audible subbands, and then unpacks and decompresses only the subband data in the audible subbands.26. A multi-channel interactive audio system, comprising:A human input device (HID) for receiving a positional input from a user;A console, comprising:A memory for storing a plurality of audio components having positional coordinates?as sequences of input data frames, each said input data frame including subband data and their scale factors that have been compressed and packed;An application programming interface (API) that generates a list of audio components in response to the positional input; andAn audio renderer thatFor each audio component on the list, Unpacks and decompresses the audio components' scale factors and, as needed, the subband data;Calculates scale factors for the mixed subband data in accordance with the positional input relative to the positional coordinates;Mixes the audio components' subband data in the subband domain for each channel in accordance with the calculated scale factors;Compresses the mixed subband data and their scale factors for each channel;Packs and multiplexes the channels' compressed subband data and scale factors into an output frame; andPlaces the output frame into a queue where the compressed audio data is output as a seamless unified bitstream;A digital decoder that decodes the bitstream into a multi-channel audio signal; andA single bandlimited connector that delivers the bitstream to the decoder.27. A method of rendering multi-channel audio, comprising:a. Storing a plurality of audio components having positional coordinates as sequence of input data frames, each said input data frame including subband data and their scale factors that have been compressed and packed;b. In response to a user positional input for a user, generating a list of audio components;c. For each audio component on the list, Unpacking and decompressing the input data frames to extract the scale factors;d. Modifying the scale factors in accordance with the positional input relative to the positional coordinates;e. Further unpacking and decompressing the input data frames to extract the subband data;f. Mixing the subband data for each channel in accordance with the calculated scale factors;g. Compressing the mixed subband data and their scale factors;h. Packing and multiplexing the channels' compressed subband data and scale factors into an output frame; andi. Placing the output frame into a queue for transmission to a decoder.28. The method of claim 27, wherein unpacking and decompressing the input data frames comprises:unpacking and decompressing only the scale factors;using the modified scale factors to determine which subbands are audible;unpacking and decompressing only the audible subband data.29. The method of claim 28, further comprising sideways localizing the audio components by applying a phase positioning filter to the subband data for the first and second subbands that span the range from approximately 200 Hz to approximately 1200 Hz.30. The method of claim 27, further comprising:a. placing a null output template in a queue for transmission to a decoder that includes the header, the bit allocation table and subband data and scale factors that represent an inaudible signal;b. if the next frame of mixed subband data and scale factors is ready, writing the mixed subband data and scale factors over the previous output frame and transmitting the output frame; andc. if the next frame is not ready, transmitting the null output template.31. The multi-channel interactive audio system of claim 1, wherein said API generates the list in response to an action input received by the HID and tracks the positional coordinates of the audio components on the list.32. The multi-channel interactive audio system of claim 16, wherein the audio components are stored as PCM audio data in a file, said subband data being preprocessed by:a. Compacting or dilating the PCM audio data in time to fit the boundaries defined by a whole number of compressed audio frames to form a looped segment;b. Appending N frames of PCM audio data from the end of the file to the start of the looped segment;c. Encoding the looped segment into a bitstream, andd. Deleting N compressed frames from the beginning of the encoded bitstream to yield a compressed audio loop sequence in which the compressed audio data in the closing input frames of the loop sequence ensure seamless concatenation with the commencing input frames during looping.33. A multi-channel interactive audio system, comprising:A memory for storing a plurality of audio components having positional coordinates in a 3D sound field as sequences of input data frames, each said input data frame including subband data and their scale factors that have been compressed and packed;A human input device (HID) for receiving input for a user including a positional input and an action input;An application programming interface (API) that in response to the action input generates a list comprising a subset of said audio components and tracks their positional coordinates in the 3D sound field; andAn audio renderer that,For each audio component on the list, unpacks and decompresses the input data frames to extract the scale factors;Modifies the scale factors in response to the user positional input relative to the audio component's tracked positional coordinates;Modifies the scale factors according to assigned mapping coefficients for each channel in the 3D sound field;Further Unpacks and decompresses the input data frames to extract the subband data;Mixes the subband data in the subband domain in accordance with the modified scale factors for each channel;Compresses the mixed subband data and their scale factors for each channel;Packs and multiplexes the channels' compressed subband data and scale factors into an output frame; andPlaces the output frame into a queue for transmission to a decoder.34. The multi-channel interactive audio system of claim 33, wherein the audio renderer uses the modified scale factors to determine the inaudible audio components for each subband and then further unpacks and decompresses the input data frames to extract only the subband date in the audible subbands.35. The multi-channel interactive audio system of claim 33, wherein the input data frame further includes a header and a bit allocation table that are fixed from frame-to-frame so that only the content of the scale factors and subband data are allowed to vary but are otherwise of fixed size in the compressed stream.36. The multi-channel interactive audio system of claim 33, wherein the input and output frames also include a header and a bit allocation table, the audio renderer providing for the seamless generation of output frames to maintain decoder sync by,a. placing a null output template in the queue that includes the header, the bit allocation table and subband data and scale factors that represent an inaudible signal;b. if the next frame of mixed subband data and scale factors is ready, writing the mixed subband data and scale factors over the previous output frame and transmitting the output frame; andd. if the next frame is not ready, transmitting the null output template.37. The multi-channel interactive audio system of claim 33, wherein one or more of the audio components comprise looped data having commencing input frames and closing input frames whose subband data has been preprocessed to ensure seamless concatenation with the commencing frame.38. A multi-channel interactive audio system, comprising:A memory for storing a plurality of audio components as sequences of input data frames, each said input data frame including subband data and their scale factors that have been compressed and packed; andAn audio renderer that,Unpacks and decompresses the input data frames to extract the scale factors for audio components selected in response to a user input;Modifies the scale factors in response to the user input;Modifies the scale factors according to assigned mapping coefficients for each channel in a 3D environment;Further unpacks and decompresses the input data frames to extract the subband data;Mixes the subband data in the subband domain in accordance with the modified scale factors for each channel;Compresses the mixed subband data and their modified scale factors for each channel;Packs and multiplexes the channels' compressed subband data and modified scale factors into an output frame; andPlaces the output frame into a queue for transmission to a decoder;wherein the input and output frames also include a header and a bit allocation table, the audio renderer providing for the seamless generation of output frames to maintain decoder sync by,a. placing a null output template in the queue that includes the header, the bit allocation table and subband data and scale factors that represent an inaudible signal;b. if the next frame of mixed subband data and scale factors is ready, writing the mixed subband data and scale factors over the previous output frame and transmitting the output frame; ande. if the next frame is not ready, transmitting the null output template.