Got it

AI lossless compression

Latest reply: Mar 7, 2022 05:30:17 517 24 9 0 0

Basic Information:

Data Compression/Source Coding

Data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation in signal processing. Any compression method can be lossy or lossless. Lossless compression saves bits by detecting and removing statistical redundancy.

Shanon Theory:

Shannon calculated the amount of information created by a source—for example, the amount in a message—using a formula similar to the one used to determine thermodynamic entropy in physics. Shannon's informational entropy is the number of binary digits necessary to encode a message in its most basic form.

Lossless Compression:

Lossless compression is a type of compression that does not lose any data during the compression process. Lossless compression "packs" data into a smaller file size by employing internal shorthand to represent redundant data.

loss

Fig: lossy and lossless compression


AI lossless compression

Source coding, which is based on Shannon Information Theory, is used for data compression. The primary idea behind source coding is to code higher probability data with fewer bits while coding lower probability data with more bits.

ai

Fig:Lossless compression of deep neural network

 It is universally assumed that the predicted code-length for each data set is greater than the Shannon Entropy of the data distribution. Many coders, including Huffman Tree, Arithmetic Coding (AC), and asymmetric numeral systems, can obtain code lengths that are extremely close to the Shannon Entropy (ANS). As a result, as long as we know the exact data distribution, we can use existing coders to achieve optimal encoding. The real distribution P(x) is, however, unknown and can only be guessed.

Lets say,

  • Real distribution is P(x)

  • Estimated Distribution is P’(x)

  • Real Code Length is L(x)

 

So L(x) is defined as,

L(x) = H(x) + KL(P||P’)

 

As a result, the important point for data compression is to estimate the distribution as closely as feasible to the true one. AI compression results from the development of generative models that can accurately forecast data distribution. AI compression enhances the compression ratio for many different data formats due to the strength of generative models. Existing AI compression works on images and texts increase the compression ratio by more than 60%.

Conclusion:

Because of the powerful DGM, AI compression, including AI lossless compression, may considerably enhance compression ratio, particularly for structured data such as photographs and videos. This could have far-reaching implications for the future of data compression. Furthermore, AI compression has injected fresh paths into deep generative model research. More emphasis should be made to the creation of an explicit generative model. Potential research paths include the development of new DGMs, such as diffusion models, and their conjunction with lossless compression. Given the limited throughput of AI compression, designing tiny generative models is another possible study topic. Theoretically analyzing DGM's generalization ability is equally significant. Designing (dynamic) generating models to better adapt to OOD data compression

 

Sources:

https://www.noahlab.com.hk/#/news/619c46c6415c7d501c2a3dbe

https://www.scientificamerican.com/article/claude-e-shannon-founder/#:~:text=Shannon defined the quantity of,required to encode a message.

https://www.facebook.com/MontrealAI/posts/lossless-compression-of-deep-neural-networksserra-et-al-httpsarxivorgabs20010021/1168671690144754/

 

 

 

 

 

 

 


  • x
  • convention:

Anno7
Author Created Feb 24, 2022 05:23:49

very nice..
View more
  • x
  • convention:

MahMush
MahMush Created Feb 25, 2022 04:35:46 (0) (0)
thanks  
Anno7
Anno7 Reply MahMush  Created Mar 1, 2022 04:56:31 (0) (0)
pkease further define on it in future  
Saqibaz
Saqibaz Created Mar 7, 2022 05:30:02 (0) (0)
Thanks  
zaheernew
MVE Author Created Feb 24, 2022 05:34:03

Useful info
View more
  • x
  • convention:

MahMush
MahMush Created Feb 25, 2022 04:35:56 (0) (0)
glad to know it  
Unicef
MVE Created Feb 24, 2022 05:48:34

THANKS FOR SHARING
View more
  • x
  • convention:

MahMush
MahMush Created Feb 25, 2022 04:36:59 (0) (0)
thanks for your response  
Saqib123
Created Mar 1, 2022 17:02:41

Thanks for Sharing
View more
  • x
  • convention:

SaraZahid
SaraZahid Created Mar 3, 2022 16:28:10 (0) (0)
 
MahMush
MahMush Created Mar 6, 2022 08:15:40 (0) (0)
thanks for your response  
Herediano
Created Mar 3, 2022 16:18:26

Excellent, thank you for sharing!
View more
  • x
  • convention:

shakeela
shakeela Created Mar 3, 2022 16:29:14 (0) (0)
 
MahMush
MahMush Created Mar 6, 2022 08:16:28 (0) (0)
thankyou for your time  
SaraZahid
Created Mar 3, 2022 16:28:18

Excellent, thank you for sharing!
View more
  • x
  • convention:

MahMush
MahMush Created Mar 6, 2022 08:15:53 (0) (0)
nice to hear it  
shakeela
Created Mar 3, 2022 16:28:57

Thank you for sharing!
View more
  • x
  • convention:

MahMush
MahMush Created Mar 6, 2022 08:16:15 (0) (0)
glad to know it  
faysalji
Author Created Mar 6, 2022 13:14:32

Good to read about AI lossless compression, thanks for sharing.
View more
  • x
  • convention:

MahMush
MahMush Created Mar 8, 2022 14:58:11 (0) (0)
glad to know your feedback  
NTan33
Created Mar 7, 2022 01:43:18

An interesting read indeed.
View more
  • x
  • convention:

MahMush
MahMush Created Mar 8, 2022 14:58:25 (0) (0)
thankyou for appreciation  
12
Back to list

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.