model-based testing

Overview Software testing often feels like a game of whack-a-mole. You write a unit test for a specific scenario, but a bug slips through because of a specific sequence of actions you never considered. Model-based testing (or stateful testing) solves this by treating your application as a state machine. Instead of manual scripts, you define the possible actions and the rules that must always hold true. This technique allows a testing framework to explore thousands of execution paths, finding deep, logic-breaking bugs that standard unit tests frequently miss. Prerequisites To get the most out of this tutorial, you should have a solid grasp of Python and basic Object-Oriented Programming. Familiarity with Pytest is helpful since we will be using it as our test runner. You don't need prior experience with property-based testing, though understanding the concept of generating random inputs will give you a head start. Key Libraries & Tools * **Hypothesis**: A powerful library for property-based and stateful testing in Python. * **Pytest**: The standard testing framework used to execute our Hypothesis state machines. * **HypoFuzz**: An extension that provides fuzzing capabilities to find malformed input vulnerabilities. * **Crosshair**: A tool that analyzes Python code and type annotations to verify post-conditions. Code Walkthrough We start by defining a state machine class that inherits from `RuleBasedStateMachine`. We initialize our state—in this case, an `Order` and a list to track `LineItem` objects. ```python from hypothesis.stateful import RuleBasedStateMachine, rule, precondition from hypothesis import strategies as st class OrderTest(RuleBasedStateMachine): def __init__(self): super().__init__() self.order = Order(customer="John Doe") self.created_items = [] ``` Defining Rules and Actions Rules are the actions Hypothesis can perform. We use the `@rule` decorator to tell the framework how to generate inputs. For example, creating a line item involves generating random text and integers. ```python @rule(description=st.text(), price=st.integers(), quantity=st.integers()) def create_line_item(self, description, price, quantity): item = LineItem(description, price, quantity) self.created_items.append(item) ``` Constraints with Preconditions You cannot remove an item from an empty order. We use the `@precondition` decorator to ensure Hypothesis only attempts actions when they make logical sense. ```python @precondition(lambda self: len(self.order.line_items) > 0) @rule(data=st.data()) def remove_line_item(self, data): item = data.draw(st.sampled_from(self.order.line_items)) self.order.remove_line_item(item) ``` Verifying Invariants Invariants (or sanity checks) are properties that must remain true after every single action. We check that the cached total in our `Order` class matches the actual sum of the items. ```python @rule() def total_agrees(self): expected = sum(item.total for item in self.order.line_items) assert self.order.total == expected ``` Syntax Notes * **`st.data()`**: This strategy allows you to "draw" values inside the method body, which is vital when the input depends on the current state (like picking an item from an existing list). * **`st.sampled_from()`**: A strategy that picks an element from a specific collection, ensuring our test doesn't try to interact with objects that don't exist. * **`RuleBasedStateMachine`**: The core class from Hypothesis that manages the lifecycle of the stateful test. Practical Examples Model-based testing shines in systems with complex state transitions. Think of a banking API where users can deposit, withdraw, and transfer funds. A state machine can verify that no sequence of transfers results in a negative balance or "ghost" money. It is equally effective for testing file systems, database drivers, or UI workflows where the order of clicks matters. Tips & Gotchas * **Shrinking**: When Hypothesis finds a bug, it will try to find the smallest possible sequence of actions to reproduce it. This makes debugging significantly easier. * **Caching Traps**: Be wary of manual cache invalidation. As shown in our walkthrough, updating a total manually often leads to bugs when the same object is added twice. Recomputing state or using robust observers is often safer. * **Performance**: Stateful tests run many more iterations than unit tests. Run them continuously in CI, but keep them focused on high-risk logic to avoid slowing down your development loop.