Welcome to Glassbox’s documentation!

Glassbox is a small library for introspecting program state to detect novel program executions when running tests. It’s mostly based on American Fuzzy Lop’s branch detection algorithm.

Its main interesting feature is that it is extremely fast. Programs running under Glassbox should generally not see more than 10-30% of a slowdown when running on CPython (on pypy there is currently a significantly more substantial slow down because tracing can prevent the JIT from working well).

glassbox.begin()

Start collecting data until a matching call to collect occurs

glassbox.collect()

Stop collecting data and return a Record containing the program execution since the matching begin call

class glassbox.Record(labels)

A record is a structured representation of a program’s execution path.

A record has a set of labels, which may be accessed as record.labels and are a sorted array of unsigned 32-bit integers. Each one corresponds to some interesting observed behaviour.

contained_in(other)

Return True if every behaviour observed by this record is also observed in the other

class glassbox.NoveltyDetector

A NoveltyDetector is used to test when a Record exhibits behaviour not previously seen.

novel(record)

Return True if this record exhibits some behaviour that no previous record passed in to novel has shown

The intended usage pattern is something along the lines of:

def interesting_values(values, run_test):
    detector = NoveltyDetector()
    for value in my_values():
        begin()
        run_test(value)
        record = collect()
        if detector.novel(record):
            yield value

This takes a set of values and prunes it down to the subset which produced a behaviour not in the previously seen ones.

You could also do something like this:

def interesting_values(values, run_test):
    seen = {}
    for value in my_values():
        begin()
        run_test(value)
        record = collect()
        for label in record.labels:
            if label not in seen or better(value, seen[label]):
                seen[label] = value
    return seen

This maintains a current “best” value that exhibits each label.

Note: It would not be a problem if run_test itself used the glassbox API. As long as begin/collect calls are kept balanced, it is perfectly safe to nest them.

Warning: Glassbox is not currently thread safe. Your results will be very confusing if you try to use it in threaded code. In the long-term it will probably simply refuse to run on more than one thread, but right now it will just break weirdly.