A Comparison of Variance and Renyi's Entropy with Application to Machine Learning

by Adric M Peccarelli

Institution: Northern Illinois University
Year: 2017
Keywords: Statistics
Posted: 02/01/2018
Record ID: 2218709
Full text PDF: http://pqdtopen.proquest.com/#viewpdf?dispub=10603911


This research explores parametric and nonparametric similarities and disagreements between variance and the information theoretic measure of entropy, specifically Renyis entropy. A history and known relationships of the two different uncertainty measures is examined. Then, twenty discrete and continuous parametric families are tabulated with their respective variance and Renyi entropy functions ordered to understand the behavior of these two measures of uncertainty. Finally, an algorithm for variable selection using Renyis Quadratic Entropy and its kernel estimation is explored and compared to other popular selection methods using real data.