Intro¶

Archimedes is a collection of macros for writing tests. It’s geared towards nose, but other test runners that work in similar manner should work too.

Main goal for Archimedes is to make testing fun and as easy as possible.

pip install libarchimedes

Facts and checks¶

Basic building block is of course a test case. Archimedes follows nose convention, where test function name starts with “test_”, so they’re easy to collect and execute programmatically. To define a test case, fact macro is used:

(fact "this is a test case"
  (assert (= 1 1)))

This will define a function, which is equivalent to:

(defn test_this_is_a_test_case []
  "this is a test case"
  (assert (= 1 1)))

Nose (or any other test framework that follows the convention) can then programmatically find this and execute it.

Sometimes one might want to execute test case immediately. This can be useful when working in interactive mode, be it Hy repl or Jupyter notebook. For these situations, there is check macro. It defines test case just like fact macro does and then executes it:

(check "this is executed immediately"
  (assert (= 1 1)))

Both of these support specifying common setup code that can be shared between several test cases. background macro specifies setup code with a unique name and with-background takes one or more variables from that specification in use. Since this probably sounds a bit confusing, an example is in order:

(background some-numbers
  [a 3]
  [b 4]
  [c 5])

(fact "sum of two numbers"
  (with-background some-numbers [a b]
    (assert (= (+ a b) 7))))

(fact "product of three numbers"
  (with-background some-numbers [a b c]
    (assert (= (* a b c) 60))))

Background can contain arbitrarily many variable definitions and they can be more complex than simple values (calculations for example).

Hypothesis¶

Hypothesis is a Python library for property based testing, similar to what QuickCheck in Haskell. Archimedes provides few parameters for fact and check macros that are used to instruct Hypothesis to generate test data.

Three macros are provided for controlling Hypothesis: variants, sample and profile.

variants macro controls test data generation. It maps into given decorator in Hypothesis. Body of variants consists of two or more items. Every odd specifies variable name and element after that is strategy specifying what kind of data to generate.

sample maps to example decorator. It specifies concrete examples for variables to check. Other than that, it works just like variants macro (variable, value).

profile maps into settings decorator in Hypothesis. It is used to tweak behaviour of Hypothesis for a specific test case.

Below is an example test case that showcases usage of all these elements.

(require archimedes)
(import [hypothesis.strategies [integers]])

(fact "sum of two positive numbers is larger than either one of them"
   (variants :a (integers :min-size 1)
             :b (integers :min-size 1))
   (sample :a 0 :b 0)
   (profile :max-examples 500)
   (assert (> (+ a b) a))
   (assert (> (+ a b) b)))

This causes test case to be run at maximum of 500 times. There are two parameters a and b, which both are integers and have value of 1 or greater. There is also a specific test case for them being zero.

Errors¶

Sometimes it’s useful to verify that a certain exception is raised. This is achieved with assert-error or assert-macro-error macro. Both take two parameters: a string and piece of code. The code is executed and resulting exception is compared with the provided string. In case of assert-error this comparison is done by simply calling str for exception. For assert-macro-error message attribute is used. If no exception is raised, or raised exception doesn’t match the provided string, assertion fails.

(fact "errors can be asserted"
      (assert-error "error"
                    (raise (ValueError "error"))))

(fact "macro errors can be asserted"
      (assert-macro-error "cond branches need to be a list"
                          (cond (= 1 1) true)))

Hamcrest¶

(defmatcher is-zero? []
            :match? (= item 0)
            :match! "a zero"
            :no-match! (.format "was a value of {0}" item))

(assert-that 0 (is-zero?))

(attribute-matcher item-with-length?
                   len =
                   "an item with length {0}"
                   "was an item with length {0}")

(assert-that "foo" (is- (item-with-length? 3)))

Hymn¶

(assert-right (do-monad [status (advance-time-m society)]
                         status)
              (assert-that society
                           (has-less-resources-than? old-resources)))