ビデオ	担当者：森川 博之・山口 英・村井 純 第02回 10/18/2000

Random Cipher Model (1)

• Number of symbols in language: q
  
  
• qN=2H0N possible messages of length N, where H0=log2q
  
  
• Most possible messages will not make any sense but some will make perfect language sense
  
  
• The difference between these two quantities defines a language’s redundancy
  
  
• ⇒ Number of meaningful messages: 2HN
  
  
  • H: Entropy of language
    
    
  • Entropy of a message H(X) is related to the number of bits needed to encode a message X
    
    
• ⇒ Number of meaningless messages: 2H0N -2HN