H O M E / I N F O / P R O D U C T S / O R D E R / S U P P O R T TSP code / Berndt / Greene / Benchmarks / Manual / Advanced Examples

Benchmarks

Here are some standard benchmark datasets and models for testing the accuracy of TSP or other econometrics packages. Most of the models are just classic published results; others are designed to be numerically difficult (designated as [difficult] here).

Basic Statistics, Linear and Nonlinear Regression

• longley.tsp longley.out
  linear regression: Longley, JASA (1967) [difficult]
  longley.htm Some brief research (2/1997) on obtaining a Longley coefficient vector accurate to 11 digits.
• nasty.tsp nasty.out
  missing data, means, correlation, linear regression:
  Wilkinson's "Statistics Quiz" [difficult]
  wilk.txt This is the full original text of the "Statistics Quiz", including the correct answers. For a comparison of how several packages fared on this benchmark, see:
  Sawitzki, G. "Testing numerical reliability of data analysis systems," Computational Statistics and Data Analysis 18 (1994), pp.269-286.
• National Institutes of Standards and Technology (NIST) "Statistical Reference Datasets"
  NIST StRD web page Contains many test problems and certified results for univariate statistics, anova, linear regression, and nonlinear regression.
  • Univariate statistics [difficult]
    • uni.tsp Code to run 7 series
    • Table of results for Sun 4 TSP obtains at least 8 correct digits for mean, standard deviation, and first order autocorrelation for 7 data series.
  • ANOVA [difficult]
    • ano.tsp Code to run 11 models
    • Table of results for Sun 4 TSP obtains at least 7 correct digits for the F-statistic in 8 of the 11 models.
  • Ordinary Least Squares [difficult]
    • ols.tsp Code to run 11 models
    • Table of results for Sun 4 TSP obtains at least 7 correct digits for coefficients and standard errors in 10 of the 11 models.
  • Nonlinear Least Squares [difficult]
    • nls.tsp Code to run 27 models
    • nls.dat Data file for 27 models
    • Table of results for 486 and Sun 4 TSP obtains at least 6 correct digits for coefficients and standard errors in 26 of the 27 models.

Simultaneous Equations

• grunsur.tsp
  SUR (Seemingly Unrelated Regressions), iterated SUR, and (iterated) SUR with cross-equation constraints:
  Grunfeld investment data, following Theil (1971)
• klein3s.tsp klein3s.out
  2SLS and 3SLS: Klein I model
• kleinlml.tsp kleinlml.out
  LIML: Klein I
• kleinfml.tsp kleinfml.out
  FIML: Klein I
  kleinfm2.tsp
  FIML: Klein I (high precision benchmark)

Time Series

• AR(1) regression models (conditional and exact ML)
  Changes to TSP's AR1 command 8/1998 Note: the AR1 code used in the examples below does not take advantage of the 8/1998 changes. It uses the older grid search and iteration methods before they were combined on 8/1998.
  • pears.tsp
    AR(1) conditional ML via grid search (Hildreth-Lu),
    AR(1) conditional ML via full iteration (Cochran-Orcutt),
    AR(1) exact ML via full iteration (Beach-MacKinnon),
    Durbin-Watson statistic and its exact P-value:
    Bartlett pears data -- analyzed by Hildreth and Lu, and by Henshaw
  • longar1.tsp
    AR(1) conditional and exact ML:
    Longley data -- following Lovell and Selover [difficult]
  • ar1lag.tsp
    AR(1) with lagged dependent variable -- multiple optima and consistent standard errors,
    testing for autocorrelation in OLS with a lagged dependent variable -- Durbin-Watson, Durbin's h, Durbin's m:
    electric utility demand, Berndt (1990)
  • dufour.tsp
    AR(1) conditional and exact ML, with multiple optima:
    Dufour, Gaudry and Liem, also Lovell and Selover [difficult]
• unit root testing
  • unitgdp.tsp
    Dickey-Fuller and augmented Dickey-Fuller
    US GDP 70:1-91:4, Gujarati (1995)
  • see our regular examples page for the Nelson-Plosser dataset and reproducing various unit roots with it
• univariate ARIMA models (exact ML)
  • bja.tsp (includes general comments on ARIMA benchmarks)
    ARMA(1,1) with constant,
    MA(1) -- actually ARIMA(0,1,1):
    Box-Jenkins series A
  • bjc.tsp
    MA(2) -- actually ARIMA(0,2,2):
    Box-Jenkins series C, 2 subsets, following Osborn (1976)
  • bje.tsp
    AR(2) and AR(3) with constant: Box-Jenkins series E (sunspots)
  • bjg.tsp
    ARIMA(0,0, 1,1, 1,1) -- multiplicative seasonal MA:
    Box-Jenkins series G (monthly airline passengers)
  • bjnam.tsp
    ARIMA(0,0, 1,1, 1,1) -- multiplicative seasonal MA:
    Pankratz (1991) series 12 (log KWH), 23 (housing starts), 24 (housing sales) -- follows Newbold, Agiakloglou, and Miller (1994)
  • bjfpac.tsp
    Partial AutoCorrelation function -- compares
    Yule-Walker and AR(p) methods (exact ML and conditional ML)
    Box-Jenkins series F
• GARCH models
  • bg44.tsp dmbp.dat
    GARCH(1,1) with constant
    Bollerslev and Ghysels (1996) daily Deutschmark-British Pound exchange rate
    uses TSP 4.4 features as of 5/11/98: different init options for h(t) and e(t)**2, iteration with analytic second derivatives, and QMLE standard errors
  • bgfcp.tsp Same model as above, but gives a full 11-digit solution. Verified with software from Fiorentini, Calzolari, and Panattoni, as well as the independent TSP code in bg11 below.
  • bg11.tsp
    GARCH(1,1) with constant
    Bollerslev and Ghysels (1996) daily Deutschmark-British Pound exchange rate
    same as bg44.tsp, but can run on earlier versions of TSP (uses a lot of complicated code to evaluate the analytic second derivatives by hand).
    • uses the following input and data files:
    • dmbp.dat data
    • g11s.tsp LogL, 1st, 2nd deriv. code
    • mlnewt.tsp Newton iteration code
    • gradchec.tsp Checks 1st derivs
    • hesschec.tsp Checks 2nd derivs

Qualitative Dependent Variables

• greenelp.tsp
  binary Logit, Probit: following Greene (1993, 1997)
• count.tsp
  Poisson, Negative Binomial 1 and 2, Ordered Probit: following Cameron and Trivedi (1986, 1998)
• poisfe.tsp
  Poisson Fixed Effects (panel data) following Cameron and Trivedi (1998)

If you have any comments on these benchmarks please send some email to Clint Cummins: clint@leland.stanford.edu.

If you have any questions or comments about TSP please send an
email to info@tspintl.com. Comments or questions about this website 
should be sent to the webmaster. Lost? Please consult the site map.