Högskolan i Skövde

his.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Accuracy on a Hold-out Set: The Red Herring of Data Mining
School of Business and Informatics, University of Borås, Sweden.
Högskolan i Skövde, Institutionen för kommunikation och information. Högskolan i Skövde, Forskningscentrum för Informationsteknologi. School of Business and Informatics, University of Borås, Sweden. (Skövde Cognition and Artificial Intelligence Lab)
Högskolan i Skövde, Institutionen för kommunikation och information. Högskolan i Skövde, Forskningscentrum för Informationsteknologi. (Skövde Cognition and Artificial Intelligence Lab)
2006 (Engelska)Ingår i: Proceedings of SAIS 2006: The 23rd Annunual Workshop of the Swedish Artificial Intelligence Society / [ed] Michael Minock; Patrik Eklund; Helena Lindgren, Umeå: Swedish Artificial Intelligence Society - SAIS, Umeå University , 2006, s. 137-146Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Abstract: When performing predictive modeling, the overall goal is to generate models likely to have high accuracy when applied to novel data. A technique commonly used to maximize generalization accuracy is to create ensembles of models, e.g., averaging the output from a number of individual models. Several, more or less sophisticated techniques, aimed at either directly creating ensembles or selecting ensemble members from a pool of available models, have been suggested. Many techniques utilize a part of the available data not used for the training of the models (a hold-out set) to rank and select either ensembles or ensemble members based on accuracy on that set. The obvious underlying assumption is that increased accuracy on the hold-out set is a good indicator of increased generalization capability on novel data. Or, put in another way, that there is high correlation between accuracy on the hold-out set and accuracy on yet novel data. The experiments in this study, however, show that this is generally not the case; i.e. there is little to gain from selecting ensembles using hold-out set accuracy. The experiments also show that this low correlation holds for individual neural networks as well; making the entire use of hold-out sets to compare predictive models questionable

Ort, förlag, år, upplaga, sidor
Umeå: Swedish Artificial Intelligence Society - SAIS, Umeå University , 2006. s. 137-146
Serie
Report / UMINF - Umeå University, Department of Computing Science, ISSN 0348-0542 ; 06.19
Nationell ämneskategori
Systemvetenskap, informationssystem och informatik Datavetenskap (datalogi)
Identifikatorer
URN: urn:nbn:se:his:diva-2020OAI: oai:DiVA.org:his-2020DiVA, id: diva2:32296
Konferens
The 23rd Annual Workshop of the Swedish Artificial Intelligence Society Workshop, SAIS 2006, Umeå, Sweden, May 10-12
Tillgänglig från: 2007-03-22 Skapad: 2007-03-22 Senast uppdaterad: 2021-06-28Bibliografiskt granskad

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

http://sais2006.cs.umu.se/

Person

Löfström, TuveNiklasson, Lars

Sök vidare i DiVA

Av författaren/redaktören
Löfström, TuveNiklasson, Lars
Av organisationen
Institutionen för kommunikation och informationForskningscentrum för Informationsteknologi
Systemvetenskap, informationssystem och informatikDatavetenskap (datalogi)

Sök vidare utanför DiVA

GoogleGoogle Scholar

urn-nbn

Altmetricpoäng

urn-nbn
Totalt: 654 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf