# 亂度與資訊量

Entropy and Information

1. I(p) >= 0 ( a real nonnegative measure )
2. I(p1p2) = I(p1) + I(p2) for independent events (additive)
3. I(p) is a continuous function of p
H(X) = -Σ(pi*log(pi)) - 資訊量的定義
Theorem 1 : -Σ(pi*log(pi)) <= -Σ(pi*log(qi))
Theorem 2 : 0<=H(p1, p2, .. ,pn) <= log(n) - 平均分佈具有最大資訊量
Theorem 3 : H(f(X)) <= H(X) - 資訊量經函數作用後遞減
Theorem 4 : H(X/Y) = -Σ( p(x/y)*log(p(x/y)) ) - 條件熵
Theorem 5 : I(X; Y) = H(X) - H(X/Y)
= Σ(P(x, y) * log( P(x,y)/P(x)P(y) )
= H(Y) - H(Y/X) - 互資訊

# 霍夫曼編碼法

Hoffman Tree and Code

Shannon Theorem

# 公開金匙密碼系統

RSA
step1 : choose large prime number p, q, let n = p*q
step2 : choose number d such that gcd(d, n) = 1
step3 : calculate inverse number e such that e*d mod n = 1
step4 : E(m) = m^e mod n = c
step5 : D(c) = c^d mod n
{"found_math": false, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0}