his.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Accuracy on a Hold-out Set: The Red Herring of Data Mining
Högskolan i Skövde, Institutionen för kommunikation och information. Högskolan i Skövde, Forskningscentrum för Informationsteknologi.
2006 (Engelska)Ingår i: 23rd Annual Workshop of the Swedish Artificial Intelligence Society, Swedish Artificial Intelligence Society - SAIS, Umeå universitet , 2006Konferensbidrag, Publicerat paper (Övrigt vetenskapligt)
Abstract [en]

Abstract: When performing predictive modeling, the overall goal is to generate models likely to have high accuracy when applied to novel data. A technique commonly used to maximize generalization accuracy is to create ensembles of models, e.g., averaging the output from a number of individual models. Several, more or less sophisticated techniques, aimed at either directly creating ensembles or selecting ensemble members from a pool of available models, have been suggested. Many techniques utilize a part of the available data not used for the training of the models (a hold-out set) to rank and select either ensembles or ensemble members based on accuracy on that set. The obvious underlying assumption is that increased accuracy on the hold-out set is a good indicator of increased generalization capability on novel data. Or, put in another way, that there is high correlation between accuracy on the hold-out set and accuracy on yet novel data. The experiments in this study, however, show that this is generally not the case; i.e. there is little to gain from selecting ensembles using hold-out set accuracy. The experiments also show that this low correlation holds for individual neural networks as well; making the entire use of hold-out sets to compare predictive models questionable

Ort, förlag, år, upplaga, sidor
Swedish Artificial Intelligence Society - SAIS, Umeå universitet , 2006.
Serie
UMINF, ISSN 0348-0542
Identifikatorer
URN: urn:nbn:se:his:diva-2020OAI: oai:DiVA.org:his-2020DiVA, id: diva2:32296
Konferens
The 23rd Annual Workshop of the Swedish Artificial Intelligence Society Workshop, Umeå, Sweden, May 10-12
Tillgänglig från: 2007-03-22 Skapad: 2007-03-22 Senast uppdaterad: 2017-11-27

Open Access i DiVA

Fulltext saknas i DiVA

Personposter BETA

Niklasson, Lars

Sök vidare i DiVA

Av författaren/redaktören
Niklasson, Lars
Av organisationen
Institutionen för kommunikation och informationForskningscentrum för Informationsteknologi

Sök vidare utanför DiVA

GoogleGoogle Scholar

urn-nbn

Altmetricpoäng

urn-nbn
Totalt: 540 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf