🧪 Testing Code Automatically

As we write more and more code, especially bits that start to depend on each other in complicated ways, it’s easy for bugs and unexpected behavior to creep in. Or, as versions of various dependencies get updated from year to year, some code that used to behave one way might stop working.

To help solve these problems, we can write simple tests for our code. When we make changes to the code package, we can run the tests to make sure we haven’t broken anything and the code’s doing what we want.

How do we test code with pytest?¶

Here’s a quick overview of what running tests might look like:

1. Write a snippet of code that should work if your code is working and (ideally) raise an error if not.

2. Put that code in a function that has the string test in its name, stored in a .py file with test in its name inside the exoatlas/tests directory.

3. From the main repository directory, from the command line run pytest. This will search for all your test functions, run them all, and give you a report about what worked and what didn’t. (The Developer Installation should have automatically installed pytest for you.)

If one or more of your tests break, you either need to fix something in your code or in your test function. If all of your tests pass, you should celebrate and feel very pleased with yourself.

What are useful tests?¶

The most useful tests are the ones you’ve actually written! It’s OK if your automatic tests don’t comprehensively cover every single possible way that your code might goof up; often a code’s error might be a fundamental conceptual misunderstanding that can only be caught by a clever and cautious human noticing something amiss in a plot.

In approximate order from simpler to more sophisticated, tests might look like:

• Does this function run on reasonable inputs?

• Does it run, and give plausible outputs?

• Does it run, and give accurate outputs?

• Does it run, give accurate outputs, and respond appropriately to changing inputs?

• Does it run, give accurate outputs, respond appropriately, and raise informative errors when it should?

No matter how detailed and thoughtfully you write your tests, the Universe is complicated, so they might not capture every fascinating way your code might break. Don’t aim for perfection; just try to do something.