▼How to Program Racket:   a Style Guide
►2 Testing
On this page:
2.1 Test Suites
2.2 Always Test!
  top  ← prev  up  next → 

2 Testing

2.1 Test Suites

Most of our collections come with test suites. These tests suites tend to live in collects/tests/ in the PLT repository, though due to historical reasons, a few collections come with their own local test suites. If you add a new collection, create a new test suite in the tests collection.

Run the test suites before you commit. To facilitate testing, we urge you to add a TESTME.txt file to your collections. Ideally, you may also wish to have a file in this directory that runs the basic tests. See the 2htdp, which is one of the collections with its own testing style. The file should describe where the tests are located, how to run these tests, and what to look for in terms of successes and failures. These files are necessary because different collections have different needs for testing, and testing evolved in many different ways in our history.

After you commit, watch for and read(!) DrDr’s emails. Do not ignore them. If you have tests that are known to fail and fixing this requires a lot of work, consider splitting your test directory into two parts: success and failure. The former is for tests that should succeed now, and the latter is for tests that are currently expected to fail. See the Typed Racket testing arrangement for an example. When you create such failure tests, you may wish to disable DrDr’s checking like this:

  git prop set drdr:command-line "" <file> ...

This is a Racket-specific git command.

2.2 Always Test!

When you debug an existing piece of code, formulate a test case first. Put it into the test suite for the component so the mistake will never be accidentally re-introduced and add a note that points to the problem report. Second, modify the code to fix the mistake. Do this second to be sure you didn’t introduce a mistake in your tests; it is all too easy to think you have fixed a mistake when in reality your new test just doesn’t properly reveal the old mistake. Third, re-run the test suite to ensure that the mistake is fixed and no existing tests fail.

If there is no test suite and you aren’t sure how to build one, then ask on the developer mailing list. Perhaps people will explain why there isn’t one or they will sketch how to create one. Please don’t ignore the problem. If you cannot build a test suite, you have a few options:

  1. Add functionality to the library to enable testing. Of course, adding functionality means adding external documentation. Robby and Matthew have done so for the GUI library, and there is now a large automated test suite for DrRacket. So even GUI programs can come with extended test suites.

  2. Add an end-to-end test that may have to be verified by a human. For example, it might be hard to test Slideshow, so you could create a slide set and describe what it should look like so future maintainers to verify when they make changes. Consider this the last and least desirable option, however.

The lack of tests for some collection will not disappear overnight. But if we all contribute a little bit, we will eventually expand the test suites to cover the entire code base, and future generations of maintainers will be grateful.

  top  ← prev  up  next →