Recent Changes to TSP

H O M E / I N F O / P R O D U C T S / O R D E R / S U P P O R T

Contents / Getting started / FAQ / Recent changes

Detailed changes to TSP 4.3 (1995-1996)

Following is a list of bugfixes and enhancements made to TSP 4.3 between March 1995 and November 1996

ENHANCEMENTS

11-05-96 ML PROC -- upgraded this new feature to allow the HITER=B and HCOV=B options. To use these options, have your PROC store a series called LOGL (conditional loglikelihood for each observation). ML will then use this series to compute numeric derivatives at each observation, and their covariance. Your PROC still has to store @LOGL (i.e. mat @logl = sum(logl); ). @LOGL is still useful for flagging numerical errors or violation of constraints.
10-25-96 P-values for t-statistics -- made more accurate for degrees of freedom greater than 500.
10-15-96 CNORMI() function -- made much more accurate (now 14 digits, was about 4 digits in worst case previously).
10-15-96 CNORM() function -- made much more accurate (now 14 digits, was about 7 digits). Ditto for DLCNORM().
09-30-96 LISTs -- can now be subscripted. LIST(PREFIX=chars) can be used to make non-numeric lists. Examples:
list x a b c;
print x(2); ? same as print b;
list x(3) = d; ? same as list x a b d; (replace 3rd list item)
list(prefix=e) eqs x; ? same as list eqs ea eb ed;
09-12-96 PDL -- now adjusts estimation sample to handle missing values, when they are due to initial observations for lags.
08-05-96 ML PROC -- upgraded this new feature to allow the Proc to call other iterative commands (such as SOLVE or even ML PROC).
08-05-96 SOLVE(SILENT,NOPRNSIM) -- reduced output to zero. This former small output was important when SOLVE is used in conjuction with ML PROC for recursive evaluation of equations.
07-31-96 ML PROC name list_of_params; New form of ML command allows you to evaluate a log-likelihood function in a PROC (vs. a single FRML). This enables ML estimation of several new classes of models: recursive time series (e.g. GARCH, ARIMA), state space/Kalman filter (hyperparameter estimation), simulation models such as multivariate normal integration, and even simple things like concentrated log likelihood functions. It is also easy to impose inequality constraints on the input parameters, or on functions of them, such as h(t) in GARCH models. Only numeric derivatives are available at present, but with the default HITER=F (BFGS), they seem to perform fairly well. We are assembling several examples of PROCs for the above models, in the TSP examples library.
06-27-96 BJFRCST(EXP): new option to compute exponentiated forecasts and bounds when original model was in logs. Store @FITSE (SE for fitted value). @FITSE is for log version of model when EXP option is used, since bounds are not symmetric around the forecast in this case.
06-07-96 Nonlinear/Monte Carlo: don't print any non-convergence warning messages when the SILENT option is on and all other output is being SUPRESed by the user (SUPRES LOGL NOB COEF;). The user should be checking the @IFCONV variable in such a situation.
05-31-96 Improved memory usage for 'C JSMPL' variable (removes need to use COMPRESS in long DO loops with many SMPLIFs). It is still useful to initialize length of 'C JSMPL' and @SMPL to as long as possible with commands like trend tmpt; smplif int(tmpt/2)*2 .ne. tmpt; (i.e. SMPL 1,1 3,3 5,5 ... ;). This prevents them from growing slowly and wasting memory.
05-31-96 Improved error message for errors like SIML(ENDOG(X,Y)) ... ; (should be ENDOG=(X,Y)) . Just gave message about "Non-scalar lag ==> X". Now also says "Complete lag expression: ENDOG(X,Y) ".
05-08-96 Exact or approximate P-value for Durbin-Watson statistic in OLSQ. The REGOPT(EXACTDW); command will print the exact P-value for subsequent OLSQ commands. This involves (roughly) forming the @NOB x @NOB matrix M*A and computing its eigenvalues, so it could be a heavy calculation for large @NOB. Alternatively, the REGOPT(PVPRINT) DW; or REGOPT(PVPRINT) ALL; commands will print an approximate P-value. This is a nonlinear regression approximation to the 5% dL table, using @NOB and @NCOEF, and yields a conservative (i.e. slightly large) P-value.
05-08-96 Support subscripted matrices in FRMLs for LSQ, FIML, ML, etc. I.e. treat like a scalar CONST.
04-19-96 AR1: store @IFCONV
04-19-96 BJEST(EXACTML): impose invertibility for NMA=1
04-19-96 DIFFER: DEPVARPR=prefix option. Example: FRML R E = Y - X*B; DIFFER(DEP=Q) R B; creates FRML DR1 Q1 = -X;
04-19-96 DIFFER: default unnormalized output frml if input frml is unnormalized. Example: FRML E Y - X*B; DIFFER E B; creates FRML DE1 -X;
04-19-96 EQSUB: now processes lagged dependent variables, such as FRML E U - RHO*U(-1); FRML U Y - X*B; EQSUB E U; Note that all the variables in the U FRML must be defined when EQSUB runs, so that EQSUB can figure out which ones are series (or equations), and apply the appropriate lag to them.
04-19-96 FIND: allow search for abbreviated command, like FIND OLS
04-19-96 FORM: RESIDUAL option for unnormalized Y - (A+BX) type eqn. Example: OLSQ Y C X; FORM(RESID,PREFIX=B) EQ1; creates FRML EQ1 Y - (B0 + B1*X);
04-19-96 Nonlinear (numeric gradients). If there is a numerical error in computing the numeric gradient, try a smaller EPS (up to 10 times).
03-14-96 BJEST(EXACTML,...) estimates using exact likelihood function, using Applied Statistics algorithm 197. Not integrated with most BJEST features yet.
03-14-96 When problems occur attempting to read an Excel 5 file, suggest that the file be saved in Excel 4 format.
02-29-96 Long programs, Procs and DO/DOT loops: approximately 25% faster execution, due to a more direct way of locating program line code.
02-29-96 BJEST: stores @ARSTAT and @MAINV flags, to indicate stationarity of AR polynomial(s) and strict invertibility of MA polynomial(s).
02-29-96 BJEST: if no series argument is given, BJEST will look for starting values in @START, and check them for stationarity and invertibility. This is normally done silently, with results stored in @ARSTAT and @MAINV, but the actual roots will be printed if the PRINT option is on.
02-29-96 FORCST after AR1: for (default) DYNAM option, look backwards for presample data, to calculate a presample residual. Use it with a power of RHO to help forecast the first and subsequent observations. Previously, the presample residual was treated as zero, and this results in a forecast that ignores RHO; i.e. yhat = X*B. This can still occur if a complete presample observation is not available.
02-29-96 DATE seconds; If you supply an argument to the DATE command, it will store the number of seconds since midnight. This is useful for timing the execution of a single command or series of commands. I.e. use DATE before and after, and take the difference to see the elapsed time.
01-29-96 BJIDENT: in autocorrelation plots, print number of lags and values of autocorrelations.
01-02-96 BJEST: store @QSTAT, %QSTAT; use missing instead of zero for first NAR+NMA P-values.
12-21-95 New nonlinear options: HITER=F (BFGS, better than DFP), GRADIENT=ANALYTIC/FORWARD/C2/etc. (type of gradient; default is GRAD=A, except for HITER=D, where it's GRAD=F, to retain upward compatibility), TOLS=value (convergence tolerance on squeezed parameter changes).
12-07-95 LSQ/SUR/3SLS/GMM: give warning when (#equations) < (#observations)
12-07-95 reading Excel v5 files: improve message when BOF record not found
11-30-95 DOS/Win, Mac: allow arbitrary missing values in hi-res PLOT (including SMPL gaps, some series longer than others, etc.)
11-30-95 ignore blank/comment lines when looking for OPTIONS MEMORY=n;
10-26-95 FORM(CONST) list of equations; converts all scalar variables (CONST, PARAM) to their current values. Useful for printing out an equation, or for storing one permanently in a databank, without having to worry about storing the PARAMs also.
10-26-95 DOS/Win: support Home and End keys for editing commands
10-10-95 IN/OUT: allow more than 16 diff. databanks in run
10-02-95 GMM(NOOPTCOV,COVOC=mat,INST=...) computes correct VCOV for "one-step" estimator, using Hansen Theorem 3.1, i.e. doesn't assume that the user-supplied COVOC is "optimal". Default: NOOPTCOV.
09-28-95 DOS/Win PLOT/GRAPH: new options: A4 = A4 paper size (LJ3 drivers only), HIRES = use patched DEV= in batch mode
09-28-95 DOS/Win PATCH program now supports A4, HIRES, and DRVDIR (network directory with driver files)
09-22-95 PROCs now waste less memory (memory from local vars recovered)
09-13-95 DOS/Win support for higher-resolution cards/monitors (SVGA, etc.)
06-02-95 parse ^ as ** or .NOT. in context
06-02-95 LAD(QUANTILE=value) (default QUANTILE=.5)
05-12-95 slightly improved memory alloc for double precision variables
05-12-95 better msg for Lotus .WK4 files (can only read .WKS, .WK3)
05-12-95 FREQ variablename; , store @FREQ .
05-12-95 OPTIONS APPEND; for NL .OUT file updating (close and reopen often)
04-14-95 arrow keys -- echo Enter
04-14-95 subscripted matrix -- better error messages when past bounds
04-14-95 LSQ/FIML error message for NOVAR >= NOB*NEQ
04-14-95 PANEL(ROBUST): implement for TOTAL, restore Hausman test for WITHIN (note: Hausman test uses the het-robust @VCOVW in this case)
04-14-95 COINT: Opt: column label for chosen number of lags, UNIT(NOCOINT) default
04-14-95 BJEST: use @START for starting values, print value for numerical errors
03-30-95 PANEL(ROBUST) heteroskedastic-robust SEs for WITHIN (see 04-14-95)
03-29-95 GMM(ITEROC) defaults for MASK=
03-29-95 COINT(EG) store @CIVEG (cointegrating vectors)
03-28-95 switch to Table 1.1* for COINT(JOH,NOTREND) P-values

MAJOR BUGFIXES

11-05-96 SOLVE -- Gauss-Seidel can (spuriously) fail to converge for the last simultaneous block, if it is followed by a very short recursive block. This can happen due to a memory allocation error, when there are some endogenous variables which do not appear on the right hand side of any equation. In this situation, the variables in the last simultaneous block can be overwritten by intermediate values.
10-02-96 LSQ avoided making a perfect fit to any equation, because this makes the objective function (log likelihood) undefined (log of zero). This has been corrected when LSQ is estimating a single equation. The problem is more complicated for multiple equations; a perfect fit in any one equation will dominate results in all other equations, so it's not clear how to extend the objective function in this case.
10-02-96 RANDOM(VCOV=vcv,VMEAN=vm) u1 u2 ... uk; If there are any exact zero elements in the mean vector, which are followed by nonzero elements, the means will not be added to the proper columns.
09-13-96 program listing in output file. The first few commands (or comments) in a batch program file might not be echoed to the output file, unless the very first line was an OPTIONS command.
08-06-96 COMPRESS: For copies of TSP created after 02-29-96 (see Enhancements above), used with long programs (those with more than 1000 program lines), the COMPRESS command attempts to store the locations of lines past 1000, and this overwrites information in the symbol table. Usually it causes the program to crash after 25 error messages like "unable to find program line".
06-24-96 IF/THEN/PROC/ELSE: prevent IF/THEN commands within a PROC from affecting the results of IF/THEN commands outside the PROC. In particular, an ELSE immediately after a PROC call used to often be affected by IF commands within the PROC.
06-10-96 wide comma-separated-values (.CSV) files: use the commas to break up the lines into pieces up to 80 bytes (like is done when blanks separate the values). Otherwise the warning message "at least 80 consecutive digits in line!" appears.
05-23-96 TOBIT. dP/dX needed to be divided by SIGMA.
05-15-96 COINT(MAXLAG=0): correctly implement the bugfix of 8-11-95 on the DOS/Win version.
05-15-96 EQSUB w/ lagged dependent variables: finished beta testing this new feature; various bugs fixed.
05-09-96 SIML(METHOD=GAUSSN): failed to converge or gave false convergence. It attempted to iterate using dY = (J'J)J'E instead of dY = (J'J)"J'E . The default METHOD=GAUSS was not affected.
04-19-96 INPUT: correctly remember extension .TSP if not supplied by user.
04-19-96 MAT: q"s*q" (q,s symm) did not expand q" before YQUAD, and gave incorrect results.
04-19-96 MAT: don't allow MAT C = ... ; (C would later always be a vector of ones).
04-19-96 FRESH: avoid crashing if certain files are not open
04-19-96 SYSTEM: strip blanks from command string. Prevented using the DOS EDIT command; gave warning for commands like DIR.
03-14-96 GMM(NOOPTCOV,...): in some cases, singularity of the @VCOV resulted in what appeared to be negative standard errors. These are now flagged as zeros in the usual way.
01-29-96 BJIDENT: in autocorrelation plots, use +/-1.96*SE, instead of +/-1*SE.
01-02-96 warning message for reserved variable/function names like ABS, LOG, LNRM, etc.
01-02-96 BJEST: implement SILENT option fully
11-20-95 ADD crashed TSP if the new line was > 80 characters
09-28-95 RESTORE didn't properly restore the SMPL
09-22-95 COMPRESS corrupted data due to improper sort
09-13-95 DOS/Win hardcopy graphics - no longer hangs or makes small plots
08-29-95 LAD standard errors (all but last were incorrect)
08-29-95 Excel 5 files (MULRK rows now are read)
08-11-95 improve memory allocation message in MFORM
08-11-95 COINT(JOH) when optimal lag or maxlag=0
05-25-95 batch loop -- failed on second batch file
05-22-95 BJEST(CONST,...) followed by BJFRCST .
05-12-95 SELECT/SMPLIF with series not defined for current SMPL endpoints
04-05-95 LMAR diagnostic
03-29-95 conversion of SMPL for FREQ change
03-29-95 PLOT(DEV=CHAR)
03-27-95 INPUT, OUTPUT, !/Recdel, COINT(JOH,MAXLAG=7)
03-23-95 COLLECT, SYSTEM, INDX.TMP, any non-command prompting
03-21-95 SELECT

OBSCURE BUGFIXES

10-10-96 DBCOMP '\data\myfile'; (on DOS) did not work, because the DOS command "RENAME \data\myfile.tlb \data\myfile.bak" does not work. The revised DOS command "RENAME \data\myfile.tlb myfile.bak" works properly.
09-12-96 PDL with CUSUM plots turned on -- made TSP crash.
09-12-96 regression output with extremely large coefficients (larger than 1.E+38 or so) -- made TSP crash.
09-12-96 KALMAN(XFIXED=x,SILENT) y ...; used to print the dependent variable name (now suppressed).
07-25-96 KALMAN(...) y; give a proper error message if there are no RHS variables, and the XFIXED option is not supplied.
07-17-96 PLOT/GRAPH(TITLE='string') ... ; on DOS/Win version, the title was ignored unless DEV=printer was supplied or patched in.
06-24-96 PROC/DOT: if the first DOT command of a run occured within a PROC, the second call to the PROC caused problems.
06-24-96 IF/THEN/DO/ENDDO/ELSE (on Alpha): used to result in a bogus error message "ELSE must follow IF/THEN".
06-24-96 BJEST(SILENT): suppress the stationarity/invertibility warning for SILENT when other output has been suppressed, for Monte Carlo loops (same situations where "non-convergence" warnings are suppressed).
06-24-96 BJEST(EXACTML,NAR=p,NMA=0): pure AR models did not run, claiming that "starting values cause numerical errors" (this error message was bogus).
06-03-96 VAR(NLAG=-2) Y1 Y2; -- don't allow NLAGS or NHORIZ to be negative (TSP crashes).
05-15-96 COINT(SILENT or TERSE): the NOB row in @TABEG was set to missing values.
05-15-96 COINT(PRINT,SILENT) - when the PRINT option is used with SILENT, to increase the items stored in @TABEG, etc., don't print the options list.
05-08-96 ANALYZ: match named and numeric lags properly. I.e. OLSQ Y C X(-1); FRML R X(L); SET L=-1; ANALYZ R; should understand that X(L) and X(-1) are the same.
05-08-96 Dotted FRML with named lags, for example FRML E. Y. = A*Y.(L); . The lag name (L) was being replaced by its value at the time the FRML command was executed (and if the lag name did not exist, an error message occurred). Now the lag name is replaced (temporarily) only while the FRML is being used.
04-30-96 EQSUB: the new feature changes of 04-19-96 introduced a bug when a macro equation does not appear in the input equation. You don't have to worry about this one unless you got a new disk between 4-19-96 and 4-30-96.
04-19-96 DOT/FRML/EQSUB: if DOT creates a duplicate argument, such as DOT 1; FRML EQ Y = A. + A1; ENDDOT; , then merge the duplicate arguments together. Otherwise EQSUB would later have just substituted for the first of the two.
04-19-96 Files: try both upper- and lowercase .tsp/.TSP extension for unix, when user does not supply this extension.
04-19-96 BJEST: if no series argument is given (stationarity check), some of the starting values may not have been used.
03-15-96 SHOW C; crashed TSP sometimes on PCs
03-14-96 PANEL without C in RHS variables -- numerator DF for F test of (Ai,B) = (A,B) was off by one (was N instead of N-1).
03-14-96 MSD with 2 observations -- caused a zero divide.
02-20-96 SMPL: round arguments to nearest integer, so that powers like 2**4 aren't truncated to 15 on some machines
02-20-96 MSD: compute @SKEW when NOB=3
02-20-96 REGOPT: compute T*RSQ correctly when BPLIST does not contain C
01-02-96 BJEST: don't count CONSTANT in DF for P-values of Q-statistics
12-07-95 binary Logit and Probit: change %correct label to "fraction correct"
10-27-95 LIST(LAST=n,...) -- check new args and expand if any are lists
10-10-95 SET w/ single-obs series -- don't ignore SMPL
10-10-95 MAT w/ D.P. series -- don't ignore SMPL
09-28-95 MODEL(FILE=causor) crashed on PC
08-11-95 nested DO and IF
08-11-95 REGOPT(BPLIST) overwritten
08-11-95 reading free format number with exactly 10 digits
08-11-95 MFORM crash on options error
08-11-95 OLSQ: @RECRES - more careful check of initial rank
08-11-95 ARCH overflow when ALPHA(i)=0
06-07-95 minor Lotus option fixes
06-02-95 correct title in repeated batch loop
06-02-95 trap syntax like T LOG(2)
05-12-95 hi-res PLOT w/ more than 32768 observations
04-19-95 OLSQ: TOL affected SSR via restoring X of large scale
04-14-95 LOCAL, MAKEID commands now work
03-29-95 PLOT(DEV=CHAR)
04-14-95 COINT: correct labels for PRINT,MINLAG>1, store @CIVEG for NOALLORD

If you have any questions or comments about TSP please send an email to info@tspintl.com.

Comments or questions about this website should be sent to news@tspintl.com.