Search

Nonparametric learning is a fundamental concept in machine learning that aims to capture complex patterns and relationships in data without making strong assumptions about the underlying data distribution. Owing to simplicity and familiarity, one of the most well-known algorithms under this paradigm is the $k$-nearest neighbors ($k$-NN) algorithm. Driven by the usage of machine learning in safety-critical applications, in this work, we shed new light on the traditional nearest neighbors algorithm from the perspective of information theory and propose a robust and interpretable framework for tasks such as classification, regression, density estimation, and anomaly detection using a single model. We can determine data point weights as well as feature contributions by calculating the conditional entropy for adding a feature without the need for explicit model training. This allows us to compute feature contributions by providing detailed data point influence weights with perfect attribution and can be used to query counterfactuals. Instead of using a traditional distance measure which needs to be scaled and contextualized, we use a novel formulation of $\textit{surprisal}$ (amount of information required to explain the difference between the observed and expected result). Finally, our work showcases the architecture's versatility by achieving state-of-the-art results in classification and anomaly detection, while also attaining competitive results for regression across a statistically significant number of datasets.

Machine learning models have become more and more complex in order to better approximate complex functions. Although fruitful in many domains, the added complexity has come at the cost of model interpretability. The once popular k-nearest neighbors (kNN) approach, which finds and uses the most similar data for reasoning, has received much less attention in recent decades due to numerous problems when compared to other techniques. We show that many of these historical problems with kNN can be overcome, and our contribution has applications not only in machine learning but also in online learning, data synthesis, anomaly detection, model compression, and reinforcement learning, without sacrificing interpretability. We introduce a synthesis between kNN and information theory that we hope will provide a clear path towards models that are innately interpretable and auditable. Through this work we hope to gather interest in combining kNN with information theory as a promising path to fully auditable machine learning and artificial intelligence., Comment: 16 pages

This paper examines one aspect of school governance--the role of the school board--which historically has governed education at the local level. It discusses the basic and unique functions of school boards in American education, as well as their effectiveness in meeting the rigors of a changing society. The discussion is presented in four parts: (1) the significance and benefits of local school-board governance as currently established; (2) the major attributes of good school-board governance, including examples of school boards that are providing leadership to raise student achievement; (3) why local school boards are preferable to alternatives that have been suggested; and (4) how the state, as a governance partner, can strengthen the capacity of local school boards to govern and lead. Whereas the paper specifically deals with the school-board role, it recognizes that education governance operates in a broader context that embraces the state and federal levels. It also recognizes that school boards are not monolithic institutions. It details how a good local school board will set a vision, focus on student learning and achievement, provide a structure for success, advocate for education, and involve the community. (RJM)

Are private schools superior to public schools? This report presents the numbers, analyzes the arguments, and concludes that the answer is "no." All things being equal, a good school is a good school, whether it is public or private. The report provides information about school demography, school characteristics, student achievement, teachers and principals, and public opinion. Findings indicate that 11 percent of all school-age children attend private schools, and that the population of private school students is less diverse than that of public schools. Private and public schools have highly similar graduation requirements, although teachers at private schools believe that they have more influence over curriculum offerings than do public school teachers. Private schools as a whole have smaller class sizes than do public schools. Public school teachers earn higher salaries and are more likely to have an advanced degree than private school teachers. The data on academic achievement is mixed; however, parents' level of education and family income are strongly linked to student achievement, independent of school type. Private schools have lower dropout rates and higher college-attendance rates than public schools. This may be explained by the selective nature of private schools. Polling data show that the general public believes that private schools have higher academic standards, are safer, and are more likely to promote "honesty and responsibility." On the other hand, public schools provide a better education to special needs children and more diverse learning envUironments. Six tables and one figure are included. (Contains 30 references.) (LMI)