Why and how to construct an epistemic justification of machine learning?
Consider a set of shuffled observations drawn from a fixed probability distribution over some instance domain. What enables learning of inductive generalizations which proceed from such a set of observations? The scenario is worthwhile because it epistemically characterizes most of machine learning....
| Published in: | Synthese Vol. 204; no. 2; pp. 1 - 25 |
|---|---|
| Main Authors: | , |
| Format: | Article |
| Published: |
Springer Nature
Aug2024
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=178978170&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 178978170 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 00397857 4LI jtl: Synthese issn: 00397857 maglogo: N pubinfo: dt: Aug2024 vid: 204 iid: 2 pid: 237 pub: Springer Nature artinfo: ui: 178978170 10.1007/s11229-024-04702-z ppf: 1 ppct: 24 formats: fmt: – @attributes: type: T – @attributes: type: P size: 715KB tig: atl: Why and how to construct an epistemic justification of machine learning? aug: au: Spelda, Petr Stritecky, Vit affil: https://ror.org/024d6js02 Department of Security Studies, Institute of Political Studies, Faculty of Social Sciences, Charles University, U Kříže 8, 158 00, Praha 5, Czech Republic sug: keyword: Complexity regularization Empirical risk minimization Lottery ticket hypothesis Material theory of induction ab: Consider a set of shuffled observations drawn from a fixed probability distribution over some instance domain. What enables learning of inductive generalizations which proceed from such a set of observations? The scenario is worthwhile because it epistemically characterizes most of machine learning. This kind of learning from observations is also inverse and ill-posed. What reduces the non-uniqueness of its result and, thus, its problematic epistemic justification, which stems from a one-to-many relation between the observations and many learnable generalizations? The paper argues that this role belongs to any complexity regularization which satisfies Norton’s Material Theory of Induction (MTI) by localizing the inductive risk to facts in the given domain. A prime example of the localization is the Lottery Ticket Hypothesis (LTH) about overparameterized neural networks. The explanation of MTI’s role in complexity regularization of neural networks is provided by analyzing the stability of Empirical Risk Minimization (ERM), an inductive rule that controls the learning process and leads to an inductive generalization on the given set of observations. In cases where ERM might become asymptotically unstable, making the justification of the generalization by uniform convergence unavailable, LTH and MTI can be used to define a local stability. A priori, overparameterized neural networks are such cases and the combination of LTH and MTI can block ERM’s trivialization caused by equalizing the strengths of its inductive support for risk minimization. We bring closer the investigation of generalization in artificial neural networks and the study of inductive inference and show the division of labor between MTI and the optimality justifications (developed by Gerhard Schurz) in machine learning. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Synthese is a copyright of Springer, 2024. All Rights Reserved. item: Synthese holder: Springer Nature dt: @attributes: year: 2024 holdings: @attributes: islocal: N |
|---|