{"id":1260,"date":"2021-07-27T00:00:00","date_gmt":"2021-07-27T00:00:00","guid":{"rendered":"urn:uuid:06b61805-542c-4b91-82ab-e40159f109f3"},"modified":"2021-07-27T00:00:00","modified_gmt":"2021-07-27T00:00:00","slug":"kullbach-leibler-divergence","status":"publish","type":"post","link":"https:\/\/www.sekaiken.com\/?p=1260","title":{"rendered":"Kullbach-Leibler Divergence"},"content":{"rendered":"<p>Kullbach-Leibler Divergence\u306e\u52d5\u753b\u3092\u898b\u307e\u3057\u3087\u3046\u3002<br \/>\nhttps:\/\/www.youtube.com\/watch?v=ErfnhcEV1O8<\/p>\n<p>\u65e5\u672c\u3067\u306fKL\u60c5\u5831\u91cf\u3068\u7565\u3055\u308c\u307e\u3059\u3002\u73fe\u8c61\u306e\u8d77\u3053\u308b\u78ba\u7387\u3092\u8003\u616e\u3057\u305f\u60c5\u5831\u91cf\u3067\u3059\u3002\u3042\u308b\u78ba\u7387\u5206\u5e03\u3092\u6301\u3064\u73fe\u8c61\u3092\u60c5\u5831\u3068\u3057\u3066\u9001\u4fe1\u3059\u308b\u3068\u304d\u306b\u3001\u4f55\u30d3\u30c3\u30c8\u5fc5\u8981\u304b\u3092\u4e0e\u3048\u308b\u305f\u3081\u3001\u60c5\u5831\u5727\u7e2e\u306e\u4e0b\u9650\u5024\u306e\u898b\u7a4d\u3082\u308a\u306b\u4f7f\u3048\u308b\u91cd\u8981\u306a\u6982\u5ff5\u3067\u3059\u3002\u7d71\u8a08\u529b\u5b66\u306e\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u3068\u540c\u3058\u5f62\u3092\u3057\u3066\u3044\u308b\u306e\u3067\u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u3068\u547c\u3070\u308c\u308b\u91cf\u3068\u95a2\u9023\u3057\u3066\u3044\u3066\u9762\u767d\u3044\u3067\u3059\u3002<br \/>\n\u30aa\u30c3\u30ab\u30e0\u306e\u5243\u5200\uff08\u304b\u307f\u305d\u308a\uff09\u306e\u5b9a\u91cf\u5316\u306e\u4e00\u7a2e\u3068\u3057\u3066\u3082\u4f7f\u3048\u307e\u3059\u3002<\/p>\n<p>logarithm \u5bfe\u6570<br \/>\nBayesian inference \u30d9\u30a4\u30ba\u63a8\u5b9a<br \/>\nD_KL is often called the information gain \uff08\u60c5\u5831\u5229\u5f97\uff09\u3000achieved if P would be used instead of Q which is currently used.<br \/>\ndivergence \u300c\u5206\u6563\u300d\u3068\u8a33\u3059\u3053\u3068\u304c\u591a\u3044\u3067\u3059<br \/>\ntriangle inequality  \u4e09\u89d2\u4e0d\u7b49\u5f0f\u3000\u4e09\u89d2\u5f62\u306e\u4e00\u8fba\u306f\u4ed6\u306e\uff12\u8fba\u306e\u9577\u3055\u306e\u548c\u3088\u308a\u3082\u77ed\u3044<br \/>\npdf\uff1a\u3000 partial differential equation \u504f\u5fae\u5206\u65b9\u7a0b\u5f0f<\/p>\n<p>Occam&rsquo;s razor\uff08\u30aa\u30c3\u30ab\u30e0\u306e\u5243\u5200\uff09 or the principle of parsimony(\u3051\u3061\u306e\u539f\u7406) or law of parsimony is the problem-solving principle\uff08\u554f\u984c\u89e3\u6c7a\u306e\u539f\u5247\uff09 that &ldquo;entities should not multiplied beyond necessity&rdquo;.\u3000\u5fc5\u8981\u4ee5\u4e0a\u306b\u591a\u304f\u306e\u5b9f\u4f53\u3092\u7f6e\u304f\u3079\u304d\u3067\u306f\u306a\u3044\u3002<br \/>\nentity \u306f\u3001\u5b9f\u5728\u3001\u5b58\u5728\u3001\u5b9f\u4f53\u3000\u306a\u3069\u3068\u8a33\u3055\u308c\u307e\u3059\u304c\u3001\u30aa\u30c3\u30ab\u30e0\u306e\u5834\u5408\u306f\u300c\u4eee\u5b9a\u300d\u3068\u8a33\u3055\u308c\u308b\u3053\u3068\u304c\u591a\u3044\u3067\u3059\u3002<br \/>\nhttps:\/\/wa3.i-3-i.info\/word11594.html<br \/>\nJohn von Neumann:<br \/>\n  \u201cWith four parameters I can fit an elephant, and with five I can make him wiggle his trunk.\u201d<br \/>\n\u30d1\u30e9\u30e1\u30fc\u30bf\u304c\uff14\u3064\u3042\u308c\u3070\u8c61\u3067\u3082\u30d5\u30a3\u30c3\u30c6\u30a3\u30f3\u30b0\u3067\u304d\u308b\u3057\u3001\uff15\u3064\u4f7f\u3048\u3070\u8c61\u306e\u9f3b\u3092\u52d5\u304b\u3059\u3053\u3068\u3082\u3067\u304d\u308b\u3000\uff0d\u30d5\u30a9\u30f3\u30fb\u30ce\u30a4\u30de\u30f3<\/p>\n<blockquote class=\"wp-embedded-content\" data-secret=\"82STsTNPaI\"><p><a href=\"https:\/\/www.johndcook.com\/blog\/2011\/06\/21\/how-to-fit-an-elephant\/\">How to fit an elephant<\/a><\/p><\/blockquote>\n<p><iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; visibility: hidden;\" title=\"&#8220;How to fit an elephant&#8221; &#8212; John D. Cook\" src=\"https:\/\/www.johndcook.com\/blog\/2011\/06\/21\/how-to-fit-an-elephant\/embed\/#?secret=ibrS3kAZhL#?secret=82STsTNPaI\" data-secret=\"82STsTNPaI\" width=\"500\" height=\"282\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Kullbach-Leibler Divergence\u306e\u52d5\u753b\u3092\u898b\u307e\u3057\u3087\u3046\u3002 https:\/\/www.youtube.com\/watch?v=ErfnhcEV1O8 \u65e5\u672c\u3067\u306fKL\u60c5\u5831\u91cf\u3068\u7565\u3055\u308c\u307e\u3059\u3002\u73fe\u8c61\u306e\u8d77\u3053\u308b\u78ba\u7387\u3092\u8003\u616e\u3057\u305f\u60c5\u5831\u91cf\u3067\u3059\u3002\u3042\u308b\u78ba\u7387\u5206\u5e03\u3092\u6301\u3064\u73fe\u8c61\u3092\u60c5\u5831\u3068\u3057\u3066\u9001\u4fe1\u3059\u308b\u3068\u304d\u306b\u3001\u4f55\u30d3\u30c3\u30c8\u5fc5\u8981\u304b\u3092\u4e0e\u3048\u308b\u305f\u3081\u3001\u60c5\u5831\u5727\u7e2e\u306e\u4e0b\u9650\u5024\u306e\u898b\u7a4d\u3082\u308a\u306b\u4f7f\u3048\u308b\u91cd\u8981\u306a\u6982\u5ff5\u3067\u3059\u3002\u7d71\u8a08\u529b\u5b66\u306e\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u3068\u540c\u3058\u5f62\u3092\u3057\u3066\u3044\u308b\u306e\u3067\u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u3068\u547c\u3070\u308c\u308b\u91cf\u3068\u95a2\u9023\u3057\u3066\u3044\u3066\u9762\u767d\u3044\u3067\u3059\u3002 \u30aa\u30c3\u30ab\u30e0\u306e\u5243\u5200\uff08\u304b\u307f\u305d\u308a\uff09\u306e\u5b9a\u91cf\u5316\u306e\u4e00\u7a2e\u3068\u3057\u3066\u3082\u4f7f\u3048\u307e\u3059\u3002 logarithm \u5bfe\u6570 Bayesian inference \u30d9\u30a4\u30ba\u63a8\u5b9a D&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[46],"tags":[9],"class_list":["post-1260","post","type-post","status-publish","format-standard","hentry","category-bio","tag-bio"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=\/wp\/v2\/posts\/1260","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1260"}],"version-history":[{"count":0,"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=\/wp\/v2\/posts\/1260\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1260"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1260"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sekaiken.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1260"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}