Got it

# AI lossless compression

Latest reply: Mar 7, 2022 05:30:17 517 24 9 0 0

Basic Information:

### Data Compression/Source Coding

Data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation in signal processing. Any compression method can be lossy or lossless. Lossless compression saves bits by detecting and removing statistical redundancy.

### Shanon Theory:

Shannon calculated the amount of information created by a source—for example, the amount in a message—using a formula similar to the one used to determine thermodynamic entropy in physics. Shannon's informational entropy is the number of binary digits necessary to encode a message in its most basic form.

### Lossless Compression:

Lossless compression is a type of compression that does not lose any data during the compression process. Lossless compression "packs" data into a smaller file size by employing internal shorthand to represent redundant data.

Fig: lossy and lossless compression

# AI lossless compression

Source coding, which is based on Shannon Information Theory, is used for data compression. The primary idea behind source coding is to code higher probability data with fewer bits while coding lower probability data with more bits.

Fig:Lossless compression of deep neural network

It is universally assumed that the predicted code-length for each data set is greater than the Shannon Entropy of the data distribution. Many coders, including Huffman Tree, Arithmetic Coding (AC), and asymmetric numeral systems, can obtain code lengths that are extremely close to the Shannon Entropy (ANS). As a result, as long as we know the exact data distribution, we can use existing coders to achieve optimal encoding. The real distribution P(x) is, however, unknown and can only be guessed.

Lets say,

• Real distribution is P(x)

• Estimated Distribution is P’(x)

• Real Code Length is L(x)

So L(x) is defined as,

L(x) = H(x) + KL(P||P’)

As a result, the important point for data compression is to estimate the distribution as closely as feasible to the true one. AI compression results from the development of generative models that can accurately forecast data distribution. AI compression enhances the compression ratio for many different data formats due to the strength of generative models. Existing AI compression works on images and texts increase the compression ratio by more than 60%.

## Conclusion:

Because of the powerful DGM, AI compression, including AI lossless compression, may considerably enhance compression ratio, particularly for structured data such as photographs and videos. This could have far-reaching implications for the future of data compression. Furthermore, AI compression has injected fresh paths into deep generative model research. More emphasis should be made to the creation of an explicit generative model. Potential research paths include the development of new DGMs, such as diffusion models, and their conjunction with lossless compression. Given the limited throughput of AI compression, designing tiny generative models is another possible study topic. Theoretically analyzing DGM's generalization ability is equally significant. Designing (dynamic) generating models to better adapt to OOD data compression

### Sources:

https://www.noahlab.com.hk/#/news/619c46c6415c7d501c2a3dbe

https://www.scientificamerican.com/article/claude-e-shannon-founder/#:~:text=Shannon defined the quantity of,required to encode a message.

• x
• convention：

Favorite(0) Share
 Anno7 Author Created Feb 24, 2022 05:23:49 .pcb{margin-right:0} .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } very nice.. View more x convention： MahMush Created Feb 25, 2022 04:35:46 (0) (0) thanks   x Anno7 Reply MahMush  Created Mar 1, 2022 04:56:31 (0) (0) pkease further define on it in future   x Saqibaz Created Mar 7, 2022 05:30:02 (0) (0) Thanks   x
 zaheernew MVE Author Created Feb 24, 2022 05:34:03 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } Useful info View more x convention： MahMush Created Feb 25, 2022 04:35:56 (0) (0) glad to know it   x
 Unicef MVE Created Feb 24, 2022 05:48:34 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } THANKS FOR SHARING View more x convention： MahMush Created Feb 25, 2022 04:36:59 (0) (0) thanks for your response   x
 Saqib123 Created Mar 1, 2022 17:02:41 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } Thanks for Sharing View more x convention： SaraZahid Created Mar 3, 2022 16:28:10 (0) (0)   x MahMush Created Mar 6, 2022 08:15:40 (0) (0) thanks for your response   x
 Herediano Created Mar 3, 2022 16:18:26 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } Excellent, thank you for sharing! View more x convention： shakeela Created Mar 3, 2022 16:29:14 (0) (0)   x MahMush Created Mar 6, 2022 08:16:28 (0) (0) thankyou for your time   x
 SaraZahid Created Mar 3, 2022 16:28:18 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } Excellent, thank you for sharing! View more x convention： MahMush Created Mar 6, 2022 08:15:53 (0) (0) nice to hear it   x
 shakeela Created Mar 3, 2022 16:28:57 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } Thank you for sharing! View more x convention： MahMush Created Mar 6, 2022 08:16:15 (0) (0) glad to know it   x
 faysalji Author Created Mar 6, 2022 13:14:32 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } Good to read about AI lossless compression, thanks for sharing. View more x convention： MahMush Created Mar 8, 2022 14:58:11 (0) (0) glad to know your feedback   x
 NTan33 Created Mar 7, 2022 01:43:18 .pcb table{table-layout:auto;} div#jdc_suspicious_threads{ background: #fcf5d8; text-align: center; font-size: 12px; } An interesting read indeed. View more x convention： MahMush Created Mar 8, 2022 14:58:25 (0) (0) thankyou for appreciation   x
12
Back to list

Comment

Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
• Politically sensitive content
• Content concerning pornography, gambling, and drug abuse
• Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

### My Followers

Login and enjoy all the member benefits

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder