Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Regression methods in multidimensional prediction and estimation
Stockholm University, Faculty of Science, Department of Mathematics.
2007 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

In regression with near collinear explanatory variables, the least squares predictor has large variance. Ordinary least squares regression (OLSR) often leads to unrealistic regression coefficients. Several regularized regression methods have been proposed as alternatives. Well-known are principal components regression (PCR), ridge regression (RR) and continuum regression (CR). The latter two involve a continuous metaparameter, offering additional flexibility.

For a univariate response variable, CR incorporates OLSR, PLSR, and PCR as special cases, for special values of the metaparameter. CR is also closely related to RR. However, CR can in fact yield regressors that vary discontinuously with the metaparameter. Thus, the relation between CR and RR is not always one-to-one. We develop a new class of regression methods, LSRR, essentially the same as CR, but without discontinuities, and prove that any optimization principle will yield a regressor proportional to a RR, provided only that the principle implies maximizing some function of the regressor's sample correlation coefficient and its sample variance. For a multivariate response vector we demonstrate that a number of well-established regression methods are related, in that they are special cases of basically one general procedure. We try a more general method based on this procedure, with two meta-parameters. In a simulation study we compare this method to ridge regression, multivariate PLSR and repeated univariate PLSR. For most types of data studied, all methods do approximately equally well. There are cases where RR and LSRR yield larger errors than the other methods, and we conclude that one-factor methods are not adequate for situations where more than one latent variable are needed to describe the data. Among those based on latent variables, none of the methods tried is superior to the others in any obvious way.

Place, publisher, year, edition, pages
Stockholm: Matematiska institutionen , 2007. , 146 p.
Keyword [en]
regression, prediction, principal compnents regression, ridge regression, partial least squares
National Category
Probability Theory and Statistics
Research subject
Mathematical Statistics
Identifiers
URN: urn:nbn:se:su:diva-7025ISBN: 978-91-7155-486-4 (print)OAI: oai:DiVA.org:su-7025DiVA: diva2:197492
Public defence
2007-09-28, sal 14, hus 5, Kräftriket, Stockholm, 13:00
Opponent
Supervisors
Available from: 2007-09-06 Created: 2007-08-28Bibliographically approved
List of papers
1. Continuum regression is not always continuous
Open this publication in new window or tab >>Continuum regression is not always continuous
1996 In: J. R. Statist. Soc., Vol. B 58, no 4, 703-710 p.Article in journal (Refereed) Published
Identifiers
urn:nbn:se:su:diva-24415 (URN)
Note
Part of urn:nbn:se:su:diva-7025Available from: 2007-09-06 Created: 2007-08-28Bibliographically approved
2. A Generalized View on Continuum Regression
Open this publication in new window or tab >>A Generalized View on Continuum Regression
1999 In: Scand. J. Statist., ISSN 0303-6898, Vol. 26, no 1, 17-30 p.Article in journal (Refereed) Published
Identifiers
urn:nbn:se:su:diva-24416 (URN)
Note
Part of urn:nbn:se:su:diva-7025Available from: 2007-09-06 Created: 2007-08-28Bibliographically approved
3. Ridge Regression and inverse problems
Open this publication in new window or tab >>Ridge Regression and inverse problems
Manuscript (Other academic)
Identifiers
urn:nbn:se:su:diva-24417 (URN)
Note
Part of urn:nbn:se:su:diva-7025Available from: 2007-09-06 Created: 2007-08-28 Last updated: 2010-01-13Bibliographically approved
4. A two-parametric class of predictors in multivariate regression
Open this publication in new window or tab >>A two-parametric class of predictors in multivariate regression
2007 (English)In: Journal of Chemometrics, ISSN 0886-9383, E-ISSN 1099-128X, Vol. 21, no 5-6, 215-226 p.Article in journal (Refereed) Published
Abstract [en]

We demonstrate that a number of well-established multivariate regression methods for prediction are related in that they are special cases of basically one general procedure. We try a more general method based on this procedure with two metaparameters. In a simulation study, based on a latent structure model, we compare this method to ridge regression (RR), multivariate partial least squares regression (PLSR) and repeated univariate PLSR. For most types of data sets studied, all methods do approximately equally well. There are some cases where RR and least squares ridge regression (LSRR) yield larger errors than the other methods, and we conclude that one-factor methods are not adequate for situations where more than one latent variable are needed to describe the data. Among those based on latent variables, none of the methods tried is superior to the others in any obvious way.

Keyword
joint continuum regression, multivariate prediction, multivariate regression, PCR, PLSR, reduced rank regression, ridge regression, SIMPLS, total least squares
National Category
Mathematics
Identifiers
urn:nbn:se:su:diva-24418 (URN)10.1002/cem.1063 (DOI)000250098200006 ()
Note
Part of urn:nbn:se:su:diva-7025Available from: 2007-09-06 Created: 2007-08-28 Last updated: 2017-12-13Bibliographically approved
5. Krylov sequences as a tool for analysing iterated regression algorithms
Open this publication in new window or tab >>Krylov sequences as a tool for analysing iterated regression algorithms
Manuscript (Other academic)
Identifiers
urn:nbn:se:su:diva-24419 (URN)
Note
Part of urn:nbn:se:su:diva-7025Available from: 2007-09-06 Created: 2007-08-28 Last updated: 2010-01-13Bibliographically approved

Open Access in DiVA

fulltext(2272 kB)794 downloads
File information
File name FULLTEXT01.pdfFile size 2272 kBChecksum SHA-1
3453ebac48367c6c6acb45ab3a5d4e95cfd2870705f9190c12b4276452222cd26de10d3e
Type fulltextMimetype application/pdf

By organisation
Department of Mathematics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 794 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 2177 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf