Last updated: 1 April 2012


ANS Forth Test Programs


Introduction

Checking that an ANS Forth system is compliant with the ANS Forth standard requires a set of test programs. This download provides some test programs that go some way towards meeting this requirement. The tests are based on the widely known tester developed by John Hayes and include his original core tests. The tests provided in the download are:

  • Core words (by John Hayes) plus some additional tests
  • Core extension words (incomplete)
  • The Double Number word set and extensions
  • The Exception word set and extensions
  • The File-Access word set and extensions
  • The Memory-Allocation word set
  • The Programming-Tools word set and extensions (incomplete - a few words in each)
  • The Search-Order word set and extensions
  • The String word set

The download also includes this file as readme.html

The tests are not comprehensive, no claim is made about their being correct and no warranty is provided. However they have been run successfully with such widely used systems as GForth, VFX Forth, SwiftForth and Win32 Forth, any failing tests being due to the Forth system rather than the tests themselves.

Download

Version 0.9, 1 April 2012, anstests0.9.zip

See the file changes.txt for a summary of the changes to the tests.

Running the tests

  1. Unzip the files into a suitable directory
  2. Start the Forth system and set the working directory/file path to that directory
  3. Possibly edit the file paths in the file runtests.fth
  4. Select either tester.fr or ttester.fs by commenting out the unwanted file. Note that tester.fr is better for developing a new Forth system whereas ttester.fs is more suited for regression testing of an existing system.
  5. Include the file runtests.fth to run all the tests

There is a flag, called VERBOSE, in file tester.fr that can be set to obtain more output from the test programs. When a test fails an error message is displayed and the tests carry on. To stop on the first failure uncomment the line marked *** in tester.fr

Error Messages

There are two possible error messages when a test fails:

  1. "WRONG NUMBER OF RESULTS:" when the stack depth is wrong
  2. "INCORRECT RESULT:" when the stack contents are incorrect

Both are followed by the offending line of source code.

Sample Output

The output from running the tests under GForth 0.7.0 using tester.fr with the VERBOSE flag clear are:

Gforth 0.7.0, Copyright (C) 1995-2008 Free Software Foundation, Inc.
Gforth comes with ABSOLUTELY NO WARRANTY; for details type `license'
Type `bye' to exit
s" gwj\tests\runtests.fth" included
Running ANS Forth test programs, version 0.9

*********************YOU SHOULD SEE THE STANDARD GRAPHIC CHARACTERS:
 !"#$%&'()*+,-./0123456789:;<=>?@
ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`
abcdefghijklmnopqrstuvwxyz{|}~
YOU SHOULD SEE 0-9 SEPARATED BY A SPACE:
0 1 2 3 4 5 6 7 8 9
YOU SHOULD SEE 0-9 (WITH NO SPACES):
0123456789
YOU SHOULD SEE A-G SEPARATED BY A SPACE:
A B C D E F G
YOU SHOULD SEE 0-5 SEPARATED BY TWO SPACES:
0  1  2  3  4  5
YOU SHOULD SEE TWO SEPARATE LINES:
LINE 1
LINE 2
YOU SHOULD SEE THE NUMBER RANGES OF SIGNED AND UNSIGNED NUMBERS:
  SIGNED: -80000000 7FFFFFFF
UNSIGNED: 0 FFFFFFFF
*
PLEASE TYPE UP TO 80 CHARACTERS:
abc
RECEIVED: "abc"
*redefined GDX
End of Core word set tests
********
You should see 2345: 2345
End of additional Core tests
**************
You should see -9876: -9876
Repeated: -9876
*
End of Core Extension word tests
*****************
You should see lines duplicated:
     8970676912557384689
     8970676912557384689
        8970676912557384689
        8970676912557384689
     -8522862768232894102
     -8522862768232894102
          -8522862768232894102
          -8522862768232894102
*
End of Double-Number word tests
***
End of Exception word tests
***********
End of File-Access word tests
***INCORRECT RESULT: T{ addr1 @ -1 RESIZE 0= -> addr1 @ FALSE }T

End of Memory-Allocation word tests
*****
End of Programming Tools word tests
*********redefined W1 with w1  *
ONLY FORTH DEFINITIONS search order and compilation list
Forth Root     Forth
Plus another unnamed wordlist at the head of the search order
??? Forth Root     ???
End of Search Order word tests
********
End of String word tests

Forth tests completed

 ok

As can be seen GForth fails one test.

The output will differ for different Forth systems, e.g. warnings of redefinitions, error messages and display of the search order.

Additional tests

As indicated above some of the word set tests are incomplete, others are absent. If anyone wishes to contribute test programs to cover these gaps, please contact me via the email address below.

Contact

Gerry Jackson