All in One View

Content from There and Back Again


Last updated on 2026-09-25 | Edit this page

Estimated time: 15 minutes

A round-doored hillside dwelling at sunset opposite a large acorn resting in moss.

Overview

Questions

  • What does Rust add to a successful Python project?
  • When is a mixed-language project worth the added complexity?
  • Where is Rust already present in the Python ecosystem?

Objectives

  • Contrast the strengths and costs of Python and Rust.
  • Recognize common motivations for placing Rust behind a Python API.
  • Form a specific hypothesis before optimizing or migrating code.

Two languages, two jobs


Python optimizes for developer momentum: expressive code, a rich ecosystem, and a fast path from experiment to working software. Rust optimizes for control and confidence: predictable performance, memory safety without a garbage collector, and errors caught before deployment.

Ask where each language creates the most value in the project.

Python is often strongest at Rust is often strongest at
orchestration and application logic tight compute-heavy loops
exploration and rapid iteration predictable memory and latency
broad scientific and web ecosystems safe low-level or concurrent work
approachable, flexible APIs standalone native libraries

Reasons to cross the boundary


Common motivations include a measured performance bottleneck, memory pressure, parallel work constrained by Python’s runtime, reuse of an existing Rust crate, or a need for stronger guarantees in a small critical component.

Crossing the boundary has costs. Native builds complicate packaging, values must be converted between language runtimes, and maintainers need enough Rust knowledge to support the result. A benchmark and an explicit goal give the decision a foundation.

Turn a hunch into a decision


Before writing Rust, record three things. They keep a promising experiment from quietly becoming an open-ended rewrite.

Decision input Question to answer ACORN-shaped example
Baseline What happens now on representative data? Record median and slowest-case time for a realistic batch of DOI values.
Target What improvement would matter to users or operators? Reduce validation time enough to shorten an actual ingest job, not merely a microbenchmark.
Cost budget What added complexity is acceptable? Support the required wheels without making every Python maintainer debug Rust.

Measure through the interface users will call. A Rust function may be fast in isolation while conversion, repeated boundary crossings, or wheel startup dominates the installed package. The experiment may also succeed without a speedup: reusing a trusted crate or making memory use more predictable can be a valid result when that was the stated target.

Our route: acorn-py


Throughout the workshop we will build a representative slice of acorn-py. Its Python users work with familiar imports such as:

PYTHON

from acorn.schema.validate import is_doi

assert is_doi("10.11578/dc.20250604.1")

Behind that API, PyO3 exposes validation and schema behavior from the Rust crates acorn-lib and acorn-schema. This is a credible incremental boundary: identifier validation is self-contained, the Python contract is easy to test, and the Rust implementation already exists. The Python package exposes a selected subset of the Rust APIs.

The completed project also gives us production questions to examine: why the distribution is named acorn-py while the import is acorn, why it uses the CPython stable ABI, and how its wheels are tested on more than one Python version.

Challenge

Write the reason before choosing the tool

Think of a Python project you know. Complete this sentence:

Moving ______ to Rust may improve ______, which we will verify by ______.

Then name one reason that component should remain in Python.

A useful answer names a narrow component, a measurable outcome, and a test. For example: “Moving file-format parsing to Rust may reduce import time, which we will verify with a representative benchmark.” Keeping CLI orchestration in Python may preserve iteration speed and ecosystem integrations.

Rust may already be in your environment


Python packages can expose native code while presenting ordinary Python modules to their users. Projects such as Polars, Pydantic Core, Ruff, uv, and Tokenizers demonstrate different ways Rust can support Python-facing tools. They show that a carefully chosen boundary can work well, while each project still needs its own reason to adopt Rust.

Key Points
  • Python and Rust are complementary when each has a clearly defined role.
  • Start from a measured need before considering a rewrite.
  • The integration boundary and its maintenance cost are part of the design.

Content from Translating Python into Rust


Last updated on 2026-09-25 | Edit this page

Estimated time: 35 minutes

Overview

Questions

  • How do familiar Python constructs appear in Rust?
  • How do Rust expressions, semicolons, and return determine a value?
  • How do Python strings and lambdas compare with Rust strings and closures?
  • Which differences are syntax, and which change the way we design programs?
  • How are Rust attributes different from Python decorators?

Objectives

  • Read Rust variables, collections, functions, modules, and loops.
  • Explain when a Rust block returns its final expression and when it returns ().
  • Distinguish &str, String, char, and .chars().
  • Read a Rust closure and explain how it captures surrounding values.
  • Use Option and Result to represent missing values and failures.
  • Use pattern matching to handle explicit alternatives.
  • Distinguish compile-time Rust attributes from runtime Python decorators.

Build a first mental map


Rust will feel less foreign when we anchor new syntax to familiar Python ideas. The comparisons below are starting points; similar syntax can behave differently.

Python Rust Important difference
name = "Bilbo" let name = "Bilbo"; Rust bindings are immutable by default.
MAX_RETRIES = 3 const MAX_RETRIES: u8 = 3; Python uses a naming convention; Rust enforces a declared type and compile-time value.
list[int] Vec<i64> A Rust vector has one element type.
dict[str, int] HashMap<String, i64> Key and value types are explicit.
None Option::None Absence is represented in the type.
exception Result::Err Recoverable failure is commonly returned.
for item in items for item in items Ownership determines what the loop may consume.

Variables, types, and functions


PYTHON

from acorn.schema.validate import is_doi

def count_valid_dois(values: list[str]) -> int:
    return sum(is_doi(value) for value in values)

RUST

use acorn_schema::validation::rules;

fn count_valid_dois(values: &[String]) -> usize {
    values
        .iter()
        .filter(|value| rules::doi(value.as_str()).is_ok())
        .count()
}

The Rust function accepts a borrowed slice, so callers can provide a view of a vector without transferring ownership. Its return type records that a count is never negative. ACORN groups scalar validators under validation::rules; each rule returns a structured Result, and this iterator counts the successful results. The compiler verifies that each call respects this contract.

Expressions produce values


Most Rust constructs are expressions, including blocks, if, and match. The final expression in a block becomes that block’s value when it has no semicolon:

RUST

fn doubled(value: i64) -> i64 {
    value * 2
}

fn describe(valid: bool) -> &'static str {
    if valid {
        "valid"
    } else {
        "invalid"
    }
}

Adding a semicolon evaluates an expression and discards its value. The block then produces the unit value (), roughly Rust’s “no useful value” type. This version therefore fails to compile because the signature promises an i64:

RUST

fn doubled(value: i64) -> i64 {
    value * 2;
}

Use return to leave the current function early. Rust permits it for the final value too, but an unadorned tail expression is the usual style:

RUST

fn checked_double(value: i64) -> Result<i64, String> {
    if value < 0 {
        return Err("value must not be negative".to_string());
    }

    Ok(value * 2)
}

Python differs in two ways. An expression on the last line of a normal Python function is discarded, and a function that reaches the end returns None. Python needs return value to send a value to the caller. Python semicolons only separate statements; adding or removing one does not decide a function’s return value.

Form Rust Python
Final expression without ; Becomes the block’s value Evaluated and discarded in a normal function
Expression followed by ; Value is discarded; the statement produces () Semicolon is an optional statement separator
return value Exits the function explicitly, often for an early path Exits the function and supplies its value
Reaching the end Returns the tail expression, or () if none exists Returns None

Strings are UTF-8, but the types differ


Python’s single and double quotes create the same str type. Choose the form that follows the project’s style or avoids escapes:

PYTHON

single = 'There and Back Again'
double = "There and Back Again"

assert single == double
assert isinstance(single[0], str)

Rust uses double quotes for strings and single quotes for one char. Its two main string types express ownership. Single quotes also appear in lifetime names such as 'a; context distinguishes a lifetime from a character literal.

RUST

let borrowed: &str = "There and Back Again";
let owned: String = borrowed.to_string();
let letter: char = 'T';
let first: Option<char> = owned.chars().next();
Form Meaning
"text" A string literal, normally used as a borrowed &'static str
&str A borrowed UTF-8 string slice; it does not own or grow the text
String Owned, growable UTF-8 text
'T' One Unicode scalar value of type char, not a one-character string
text.chars() An iterator over Unicode scalar values
text.bytes() An iterator over the UTF-8 bytes

Rust does not allow text[0]: a UTF-8 character may occupy more than one byte, so a numeric index would be ambiguous. Use .chars() when Unicode scalar values are the intended unit and .bytes() when the encoding bytes are. A visible user-perceived character can contain several scalar values, so code that needs grapheme clusters should use a Unicode-segmentation library rather than assuming that one char equals one displayed character.

A function normally accepts &str when it only needs to read text and returns String when it creates owned text:

RUST

fn add_prefix(value: &str) -> String {
    format!("doi:{value}")
}

Collections and iteration


RUST

fn positive_squares(values: &[i64]) -> Vec<i64> {
    values
        .iter()
        .filter(|value| **value > 0)
        .map(|value| value * value)
        .collect()
}

Iterator chains may look like Python comprehensions, but they remain strongly typed and are compiled into efficient loops.

Anonymous functions are closures


Python calls its compact anonymous function a lambda. Rust calls the corresponding construct a closure and places parameters between vertical bars:

PYTHON

lengths = list(map(lambda value: len(value), values))

RUST

let lengths: Vec<usize> = values
    .iter()
    .map(|value| value.chars().count())
    .collect();
Callout

Why is it called a lambda?

The name predates Python by decades. In 1932, mathematician Alonzo Church used the Greek letter lambda in his lambda calculus, a compact formal notation for creating functions by abstraction and applying them to arguments. John McCarthy’s original Lisp paper used LAMBDA in 1960, helping carry the term from mathematical logic into programming-language vocabulary.

Python keeps lambda as the keyword for an anonymous function expression. Rust uses the term closure, which emphasizes that the callable value may capture part of its surrounding environment. A lambda or closure can still be assigned to a name; “anonymous” describes how it was created, not whether the program can refer to it later.

Python limits a lambda body to one expression. A Rust closure may use either one expression or a block, and parameter and return types are usually inferred from the call site. Both languages can capture surrounding values:

PYTHON

suffix = "!"
decorate = lambda value: f"{value}{suffix}"

RUST

let suffix = String::from("!");
let decorate = |value: &str| format!("{value}{suffix}");

The Python closure looks up the captured name when it is called. Rust decides whether a closure borrows, mutably borrows, or consumes each captured value from how the closure uses it. Adding move forces capture by value, which is common when a closure must outlive the current scope or move to another thread. That capture behavior determines whether the closure implements Fn, FnMut, or FnOnce.

Use a named function when the operation is reused or deserves its own test. Closures work well for small, local transformations passed to map, filter, thread spawners, and similar APIs.

Pattern matching and errors


RUST

fn parse_port(raw: &str) -> Result<u16, String> {
    match raw.parse::<u16>() {
        | Ok(port) if port > 0 => Ok(port),
        | Ok(_) => Err("port must be greater than zero".to_string()),
        | Err(error) => Err(format!("invalid port: {error}")),
    }
}

match makes the successful and unsuccessful paths visible. The ? operator can propagate an error when a function does not need to transform it. ACORN’s domain validators use the same shape with ValidationError, which preserves a stable error code separately from its human-readable message.

Callout

Use ? for deliberate propagation, not automatic error handling

Use ? when the current function deliberately delegates a failure to its caller. Avoid it when this layer has the context to recover, attach a domain error, or choose a different path; use match, map_err, or another explicit transformation instead.

The ? operator does not ignore an error and does not panic. On an Err, it returns early with a compatible Err; on an Ok, it unwraps the value. A function returning Result therefore still has an explicit, deterministic output. Panics come from operations such as unwrap(), expect(), and panic!(), not from ? itself.

Some functional-programming-oriented teams avoid ? when its early return hides a branch that matters to the design. They prefer match or combinators so the transformation remains visible and composable. That is a readability choice, not a requirement of functional programming: deliberate Result propagation with ? is still typed and non-panicking.

Attributes resemble decorators, but run at a different time


Python decorators and Rust attributes both place declarative-looking syntax above a function, class, or type. That visual similarity is useful for reading code, but their execution models differ.

PYTHON

from dataclasses import dataclass

@dataclass(frozen=True)
class Doi:
    value: str

RUST

#[derive(Clone, Debug, Eq, PartialEq)]
struct Doi {
    value: String,
}

Both examples ask tooling to supply common behavior. Python calls dataclass after the class body executes, usually during module import. Rust expands derive while compiling and generates implementations of the named traits; nothing runs merely because the program starts.

Python decorator Rust attribute
Syntax @decorator #[attribute]
When it acts When the decorated definition executes During parsing or compilation
What it receives A runtime object such as a function or class Source-level input understood by the compiler or a macro
Common jobs Wrap, register, or replace an object Generate implementations, select tests, configure compilation, or generate binding code

An outer attribute such as #[test] applies to the item that follows. An inner attribute such as #![allow(dead_code)] applies to the item that contains it, often a module or crate. Later, PyO3 attributes such as #[pyfunction] and #[pymodule] will generate Python binding code at compile time. They do not behave like stacked runtime wrappers, so decorator order is not a reliable mental model for attribute order.

Keep modules and tests close to the domain


ACORN organizes identifier implementations under pid and scalar rules under validation::rules. Tests sit beside those modules instead of in one distant integration-test file. A validator test uses small tables and names the value when an assertion fails:

RUST

use acorn_schema::validation::rules;

#[test]
fn test_is_doi() {
    let values = [
        "10.1000/182",
        "https://doi.org/10.11578/dc.20250604.1",
        "10.11578/dc.20250604.1",
    ];
    values.into_iter().for_each(|value|
        assert!(rules::doi(value).is_ok(), "{value} is NOT a valid DOI")
    );
}

In the crate itself, a domain module includes its adjacent tests with #[cfg(test)] mod tests;, so test-only code is absent from normal builds.

Challenge

Translate a small function

Translate this Python function into Rust. Decide how the return type should represent the case where no DOI is found.

PYTHON

from acorn.schema.validate import is_doi

def first_doi(values: list[str]) -> str | None:
    for value in values:
        if is_doi(value):
            return value
    return None

RUST

use acorn_schema::validation::rules;

fn first_doi(values: &[String]) -> Option<&str> {
    values
        .iter()
        .find(|value| rules::doi(value.as_str()).is_ok())
        .map(String::as_str)
}

Option<&str> records that the search may not find a value and borrows the matching text from the input collection. We will return to that reference when we discuss ownership and lifetimes.

Key Points
  • Familiar surface syntax can help us begin reading Rust.
  • A Rust block can return its final expression; a semicolon discards that expression’s value, while return exits explicitly.
  • Python quote style does not change its string type; Rust distinguishes borrowed &str, owned String, and scalar char values.
  • Rust closures use |arguments| expression and capture by borrow, mutable borrow, or value according to how they use their environment.
  • Rust makes mutability, data types, absence, and recoverable errors explicit.
  • The ? operator propagates a typed failure; it neither handles the failure locally nor causes a panic.
  • Rust attributes provide compile-time instructions; Python decorators operate on runtime objects as definitions execute.
  • A slice such as &[i64] lets a function inspect sequential data without taking ownership of it.

Content from Where the Analogy Ends


Last updated on 2026-09-25 | Edit this page

Estimated time: 25 minutes

Overview

Questions

  • What problems are ownership and borrowing designed to prevent?
  • How do traits and enums shape Rust APIs?
  • What does “fearless concurrency” mean in practice?
  • Do Rust’s guarantees leave room for exploratory code?

Objectives

  • Explain ownership, borrowing, and lifetimes without relying on Python analogies.
  • Distinguish Rust enums and traits from superficially similar Python features.
  • Connect compile-time checks to safe resource use and concurrency.
  • Separate prototyping shortcuts from production error and ownership policies.

Ownership is a resource model


Every Rust value has an owner. When that owner goes out of scope, the value is dropped. A value can move to a new owner, be borrowed immutably by many readers, or be borrowed mutably by one writer. These rules prevent use-after-free and data races before the program runs.

RUST

fn label_length(label: &str) -> usize {
    label.len()
}

fn main() {
    let label = String::from("second breakfast");
    let length = label_length(&label);
    println!("{label} has {length} bytes");
}

label_length borrows the string, so label remains available to its owner.

When the borrow checker objects, first ask what the callee needs to do. The answer usually selects the parameter shape:

Callee’s intent Typical parameter Consequence for the caller
Read for the duration of the call &T The caller keeps ownership; many immutable borrows may coexist.
Change the caller’s value &mut T The caller keeps ownership; only one mutable borrow may exist at a time.
Store or consume the value T Ownership moves unless the type implements Copy.
Work on an independent duplicate T from value.clone() The caller keeps the original and pays the clone’s explicit cost.

Start with the least authority the function needs. A validator that only reads text should accept &str; taking a String would force ownership transfer or an unnecessary clone.

Lifetimes describe relationships


Most lifetimes are inferred. When a function returns a reference, an explicit lifetime may be needed to state which input keeps that reference valid.

RUST

fn choose_longer<'a>(left: &'a str, right: &'a str) -> &'a str {
    if left.len() >= right.len() {
        left
    } else {
        right
    }
}

The annotation does not extend either value’s lifetime. It gives the compiler a relationship it can check.

Enums carry data; traits describe behavior


Rust enums model a closed set of alternatives, and each variant may carry different data. Pattern matching ensures that every case is considered. Traits define shared behavior and can be used for generics or dynamic dispatch.

acorn-schema makes both ideas concrete. A patent is one type with distinct granted, application, and publication variants. The acorn-py binding matches those variants and constructs one consistent Python-facing Patent object.

RUST

use acorn_schema::pid::patent::{CountryCode, KindCode};
use acorn_schema::pid::{Patent, PersistentIdentifier};

fn kind(patent: &Patent) -> &'static str {
    match patent {
        | Patent::Granted { .. } => "grant",
        | Patent::Application { .. } => "application",
        | Patent::Publication { .. } => "publication",
    }
}
fn main() {
    let patent = Patent::Granted {
        country_code: Some(CountryCode::US),
        kind_code: KindCode::B2,
        serial_number: "7654321".to_string(),
    };
    assert_eq!(kind(&patent), "grant");
    assert_eq!(patent.identifier(), "US 7654321 B2");
}

This is the schema crate’s real Patent enum rather than a workshop-only facsimile. Importing PersistentIdentifier brings its shared methods into scope; ACORN’s identifier types use that trait for behaviors such as identifier() and schema_uri(). The exhaustive match must be revisited if the schema adds another patent variant.

Compile-time guarantees and concurrency


The same ownership rules apply across threads. A value cannot be mutated from multiple places unless its type provides safe synchronization. Data races are far harder to express, although other concurrency bugs remain possible.

Scoped threads can safely borrow inputs because Rust proves that every worker finishes before the scope ends. Each worker below computes its own value, and the parent combines the results instead of sharing mutable state:

RUST

use std::thread;

fn count_nonempty(left: &[String], right: &[String]) -> Result<usize, &'static str> {
    thread::scope(|scope| {
        let left_worker =
            scope.spawn(|| left.iter().filter(|value| !value.is_empty()).count());
        let right_worker =
            scope.spawn(|| right.iter().filter(|value| !value.is_empty()).count());

        match (left_worker.join(), right_worker.join()) {
            | (Ok(left_count), Ok(right_count)) => Ok(left_count + right_count),
            | _ => Err("a counting worker panicked"),
        }
    })
}

The slices are borrowed, not copied. Returning partial counts also avoids a mutex. Ownership rules prevent either worker from outliving the borrowed data, while the explicit Result records that a worker may panic. Rust still cannot prevent deadlocks, starvation, or a logically incorrect division of work.

😢 Typical misbeliefs


Rust prototypes do not have to look like finished library code. These beliefs usually come from treating every early decision as permanent:

Misbelief A more useful working assumption
“Memory safety and prototyping just don’t go together.” Compiler feedback can be part of the experiment. It rules out invalid memory relationships while you test the idea.
“Ownership and borrowing take the fun out of prototyping.” They can interrupt a first draft, so begin with owned values and use a temporary clone when that keeps the experiment moving. Revisit the ownership once the shape is clear.
“You have to get all the details right from the beginning.” Type inference, concrete types, and todo!() let you postpone decisions without pretending the unfinished path works.
“Rust always requires you to handle errors.” Rust makes recoverable failure visible with Result, but a prototype can deliberately stop with unwrap(), todo!(), or unreachable!(). Production code still needs an intentional policy for user-triggerable failures.

The prototype’s job is to answer a question. Rust makes many shortcuts visible, which gives us a practical list to revisit before shipping.

Challenge

Read the borrow checker as a design review

Why should this function fail to compile?

RUST

fn first_word() -> &str {
    let message = String::from("hello from Rust");
    &message[..5]
}

How could its API be changed?

message is dropped when the function returns, so the reference would point to freed memory. Return an owned String instead:

RUST

fn first_word() -> String {
    String::from("hello")
}

Alternatively, accept an input string and return a slice borrowed from that input.

Key Points
  • Ownership determines who is responsible for a value and when it is released.
  • Borrowing provides temporary access without transferring ownership.
  • Lifetimes let the compiler verify relationships between references.
  • Enums, traits, and concurrency checks are central Rust design tools.
  • Prototypes may defer decisions, provided their shortcuts remain visible and are reviewed before production.

Content from Adding Rust to Python Incrementally


Last updated on 2026-09-25 | Edit this page

Estimated time: 25 minutes

Overview

Questions

  • Where should the Python-Rust boundary go?
  • How can we preserve an existing Python API?
  • How much work should cross the boundary in one call?
  • What should stay in Python?

Objectives

  • Evaluate a component as a candidate for a Rust implementation.
  • Design a narrow, stable, and testable language boundary.
  • Preserve a Python-facing API while changing its implementation.
  • Protect existing behavior with characterization tests before replacing it.

Choose a seam for Rust


A good first Rust component has a clear input and output, meaningful work per call, limited dependence on Python objects, and tests that already describe its behavior. Parsers, codecs, validators, algorithms, and self-contained data transformations are common candidates.

A poor first boundary crosses languages inside a tight Python loop, depends on many callbacks into Python, or translates a large object graph on every call.

Keep the public API Pythonic


Start from the import path and behavior callers should keep. For our worked example, the contract is ordinary Python even though its implementation is a native extension:

PYTHON

from acorn.schema.validate import is_doi

assert is_doi("10.11578/dc.20250604.1")
assert not is_doi("not an identifier")

acorn-py is the distribution name shown by package installers, while acorn is the import name and acorn.schema.validate is a nested module registered by the extension. Treat all three names as part of the packaging contract. Callers can keep using those public names. The internal crate names and pinned Git revision remain implementation details.

Our first seam is one validator: string in, boolean out. Later checkpoints add a Rust-backed identifier class and a fallible file-reading operation. That sequence keeps each boundary narrow enough to test before the next one is added.

Protect the behavior before replacing the code


A characterization test records what callers observe today, including awkward cases that a rewrite may be tempted to “fix.” Run the same test against the Python implementation, the development extension, and eventually the wheel:

PYTHON

import pytest

from acorn.schema.validate import is_doi

@pytest.mark.parametrize(
    ("value", "expected"),
    [
        ("10.1000/182", True),
        ("https://doi.org/10.11578/dc.20250604.1", True),
        ("not an identifier", False),
        ("", False),
    ],
)
def test_is_doi_contract(value: str, expected: bool):
    assert is_doi(value) is expected

This test intentionally imports the public path rather than a private helper. It protects the name, accepted Python value, return type, and domain behavior in one place. Add edge cases from production data before changing the implementation; invented examples rarely capture every compatibility promise.

Choose the call granularity


The narrowest function is not always the cheapest boundary. Consider how the real caller uses it:

Boundary shape Advantage Cost or risk
str -> bool Simple contract and easy error isolation A Python loop may make thousands of native calls.
list[str] -> list[bool] One crossing amortizes call overhead Every string still needs extraction, and the whole batch occupies memory.
Python callback from Rust Preserves flexible Python behavior Repeated callbacks couple the implementation to Python and may constrain parallel work.

For a batch API, the conceptual Rust shape remains simple and concrete:

RUST

use acorn_schema::validation::rules;

fn are_dois(values: &[String]) -> Vec<bool> {
    values
        .iter()
        .map(|value| rules::doi(value.as_str()).is_ok())
        .collect()
}

Do not add batching merely because it sounds faster. Benchmark the scalar and batch interfaces with realistic inputs, including the conversion performed by PyO3. Choose the smallest contract that meets the measured target.

A boundary checklist


  • Is the bottleneck measured with representative data?
  • Can inputs and outputs be expressed with simple, stable types?
  • Do characterization tests describe the public behavior before it changes?
  • Is each call substantial enough to justify conversion overhead?
  • Can the behavior be tested from both Python and Rust?
  • Will the project build wheels for every supported platform and Python version?
  • Does the expected benefit justify another language and toolchain?
Challenge

Find the seam

For a Python package that reads files, validates records, calculates summaries, and generates plots, choose one first candidate for Rust. Sketch the function signature at the Python boundary and list one component you would deliberately leave in Python.

Parsing or record validation may be a good boundary when it is measurable and self-contained. Plotting should generally remain in Python so the project keeps its mature visualization ecosystem and flexible user-facing options.

Key Points
  • Migrate one well-tested, high-value component at a time.
  • Minimize language crossings and data conversion.
  • Keep the public Python API separate from the native implementation.
  • Leaving a component in Python is a valid engineering outcome.

Content from Building a Python Extension with PyO3


Last updated on 2026-09-25 | Edit this page

Estimated time: 30 minutes

Overview

Questions

  • How does acorn-py expose Rust functions and nested modules to Python?
  • What do PyO3, Maturin, Cargo, and Pixi each provide?
  • What happens between maturin develop and import acorn?
  • Why do the distribution, crate, and import names differ?

Objectives

  • Identify the roles of Pixi, Cargo, PyO3, and Maturin in acorn-py.
  • Expose an ACORN validator through acorn.schema.validate.
  • Build and install the extension in the locked Python 3.13 environment.
  • Trace a development build from the task runner to Python’s import machinery.

The project layers


  • Pixi selects the locked Python and Rust toolchains and runs project tasks.
  • Cargo resolves and builds acorn-lib, acorn-schema, PyO3, and the local Rust crate.
  • acorn-schema separates persistent-identifier types in pid from structured scalar validators in validation::rules.
  • PyO3 defines the native module, functions, classes, and error mappings.
  • Maturin builds the Rust crate as an installable Python distribution and wheel.

The teaching checkpoints retain acorn-py’s real pyproject.toml, Cargo.toml, pixi.lock, and Cargo.lock. We reduce src/lib.rs to one validator, then add the remaining boundary pieces incrementally. The completed checkout from setup remains available for comparison.

Follow one development build


pixi run -e py313 develop is short, but it coordinates several contracts:

Phase Tool What to inspect when it fails
Select an environment and task Pixi The environment, locked dependencies, and task in pixi.toml
Read the Python build configuration Maturin The backend and [tool.maturin] settings in pyproject.toml
Compile the native library Cargo and rustc Features, imports, and the dependency revision
Install the development artifact Maturin The extension filename and active Python environment
Initialize the imported module Python and PyO3 The import name, #[pymodule] name, and nested-module registration

The development install points Python at the compiled extension. Saving src/lib.rs does not rebuild that extension, so run develop again after a Rust change. If an import appears to use the wrong build, verify the interpreter through the same environment:

BASH

pixi run -e py313 python -c "import sys; print(sys.executable)"

Read the package contract


Three names describe different layers of the same project:

Name Where it appears Meaning
acorn-py pyproject.toml and Cargo.toml Python distribution and Rust crate
acorn [tool.maturin] module-name Python import name
acorn.schema.validate PyO3 module registration Public validator namespace

The project requires Python 3.10 or newer and enables PyO3’s abi3-py310 feature. A single wheel built against that stable ABI can support multiple compatible CPython minor versions.

Cargo.toml pins acorn-lib and acorn-schema to one immutable public Git revision. No local ACORN checkout is needed. Changing that revision requires a dependency review and falls outside the workshop.

Expose the first validator


At the first binding checkpoint, src/lib.rs contains one Python function and the module structure needed to preserve the public import path:

RUST

use acorn_schema::validation::rules;
use pyo3::prelude::*;

#[pyfunction]
fn is_doi(value: &str) -> bool {
    rules::doi(value).is_ok()
}
#[pymodule]
#[pyo3(name = "acorn")]
fn acorn_py(module: &Bound<'_, PyModule>) -> PyResult<()> {
    let python = module.py();
    let schema = PyModule::new(python, "schema")?;
    let validate = PyModule::new(python, "validate")?;
    validate.add_function(wrap_pyfunction!(is_doi, &validate)?)?;
    schema.add_submodule(&validate)?;
    module.add_submodule(&schema)?;
    let sys = python.import("sys")?;
    let modules = sys.getattr("modules")?;
    modules.set_item("acorn.schema", schema)?;
    modules.set_item("acorn.schema.validate", validate)?;
    Ok(())
}

#[pyfunction] makes the Rust function callable by Python. wrap_pyfunction! adds it to the validate module, and the sys.modules entries make Python’s nested import machinery recognize the public paths.

Read the initializer as five operations: obtain the Python handle, create the nested modules, wrap and add functions, register the public paths, and return Ok(()). The ? operators deliberately propagate initialization failures as Python exceptions because there is no useful local recovery. Propagating an error is different from ignoring it.

Build the current checkpoint and call it from Python:

BASH

pixi run -e py313 develop
pixi run -e py313 python -c "from acorn.schema.validate import is_doi; print(is_doi('10.11578/dc.20250604.1'))"

The command should print True. Re-run develop after changing Rust code so the active development environment receives the rebuilt extension.

Diagnose the layer, not just the symptom


Symptom Likely layer First check
Rust compiler error Cargo or Rust Read the first diagnostic before the cascading errors
ImportError mentioning PyInit_acorn Module naming Compare module-name with #[pyo3(name = "acorn")]
ModuleNotFoundError for acorn.schema.validate Nested registration Check both sys.modules entries
Python still shows old behavior Development install Re-run develop in the interpreter’s environment
Development import passes but the wheel fails Packaging or ABI Inspect wheel tags and test a clean installation

Start with the earliest failing layer. Rewriting Rust cannot repair a wrong Python interpreter, and changing the Python test cannot repair a missing module initializer.

Add a class when identity and behavior belong together


A function is enough for a yes-or-no validation. A Rust-backed class is useful when Python should keep a validated value and ask it for related behavior later. This teaching-sized class validates once during construction:

RUST

use pyo3::exceptions::PyValueError;

#[pyclass(frozen, module = "acorn.schema.pid")]
struct Doi {
    value: String,
}

#[pymethods]
impl Doi {
    #[new]
    fn new(value: String) -> PyResult<Self> {
        match rules::doi(value.as_str()) {
            | Ok(_) => Ok(Self { value }),
            | Err(error) => Err(PyValueError::new_err(error.to_string())),
        }
    }

    #[getter]
    fn value(&self) -> &str {
        self.value.as_str()
    }

    fn __str__(&self) -> &str {
        self.value.as_str()
    }
}

Register it under a sibling pid module using the same parent-child and sys.modules pattern as validate. Create and attach pid before attaching schema to the top-level module:

RUST

let pid = PyModule::new(python, "pid")?;
pid.add_class::<Doi>()?;
schema.add_submodule(&validate)?;
schema.add_submodule(&pid)?;
module.add_submodule(&schema)?;

After obtaining sys.modules, register all three qualified names:

RUST

modules.set_item("acorn.schema", schema)?;
modules.set_item("acorn.schema.validate", validate)?;
modules.set_item("acorn.schema.pid", pid)?;

frozen prevents Python callers from replacing Rust-managed fields. The constructor turns invalid input into ValueError, so a successfully created Doi always satisfies its invariant. Prefer a function when no durable state or behavior justifies a class.

Challenge

Bind a second validator

Add an is_orcid(value: &str) -> bool function using ACORN’s canonical ORCID rule. Register it beside is_doi, rebuild, and verify these calls:

PYTHON

from acorn.schema.validate import is_orcid

assert is_orcid("https://orcid.org/0000-0002-2057-9115")
assert not is_orcid("abc-0000-0000-0000")

The binding follows the same shape as the first validator:

RUST

#[pyfunction]
fn is_orcid(value: &str) -> bool {
    rules::orcid(value).is_ok()
}

Register it inside acorn_py:

RUST

validate.add_function(wrap_pyfunction!(is_orcid, &validate)?)?;

Then run pixi run -e py313 develop before executing the Python assertions.

Key Points
  • Pixi makes the workshop toolchain reproducible; Cargo and Maturin build the native Python package.
  • PyO3 exposes selected ACORN behavior without exposing the whole Rust crate.
  • Distribution, import, and nested-module names are separate contracts.
  • A build crosses distinct layers; diagnose the earliest layer that fails.
  • Use a #[pyclass] when validated state and related behavior must persist across Python calls.
  • The first working boundary is deliberately small: one string in and one boolean out.

Content from From Python to Rust and Back Again


Last updated on 2026-09-25 | Edit this page

Estimated time: 30 minutes

Overview

Questions

  • What happens to strings, paths, objects, and errors at the language boundary?
  • How do we choose conversions, exceptions, and tests for a binding contract?
  • How can generated examples exercise invariants and round trips?
  • How does acorn-py test its public Python API?
  • What does the stable-ABI wheel smoke test prove?

Objectives

  • Select PyO3 argument and return types appropriate for an ACORN binding.
  • Map a fallible Rust operation to a meaningful Python exception.
  • Distinguish domain, binding-contract, and packaging tests.
  • Write a property test for a pure transformation.
  • Test Rust-backed behavior through the public Python import path.

Data crosses by borrowing, conversion, or extraction


PyO3 can convert many ordinary values between Python and Rust. The choice in a binding signature documents what happens at the boundary:

  • &str borrows text for the duration of a call such as is_doi;
  • String owns converted text that must outlive the call;
  • PathBuf accepts Python path-like input for file operations; and
  • a #[pyclass] stores Rust data behind a Python-visible object.

Use the least complicated type that expresses the call’s needs:

Rust signature Python-side input Boundary effect Good fit
&str str Borrow text for this call A validator that only reads its argument
String str Create owned Rust text A value retained or transformed after extraction
Vec<String> A sequence of strings Build a Rust vector and own its elements A batch operation with enough work to justify one conversion
PathBuf A path-like object Convert into an owned native path Reading or writing a document
#[pyclass] A Python-visible Rust object Keep Rust-owned state across calls A typed identifier with methods and invariants

acorn-py starts with coarse operations. One call validates a complete identifier or reads a complete research-activity document. It does not ask Python to cross the native boundary once per byte or once per validation rule.

Convenient conversion can still allocate or copy. Measure realistic calls before redesigning an interface around more complex borrowed-buffer APIs.

Turn expected failures into Python exceptions


An invalid identifier naturally returns false, but a formatter or file read can fail with information callers need. PyResult<T> lets the binding return a value or raise a Python exception:

RUST

use acorn_schema::validation;
use pyo3::exceptions::PyValueError;
use pyo3::prelude::*;

#[pyfunction]
fn format_phone_number(value: &str) -> PyResult<String> {
    validation::format_phone_number(value).map_err(|error|
      PyValueError::new_err(error.to_string())
    )
}

The public Python contract is now explicit:

PYTHON

import pytest
from acorn.schema.validate import format_phone_number

def test_invalid_phone_number():
    with pytest.raises(ValueError, match="Unable to format"):
        format_phone_number("not a phone number")

The schema function returns ACORN’s structured ValidationError; converting it to text uses its human-readable message while PyO3 supplies the Python exception type. Do not use panics to handle expected domain failures. Return Rust Result values and translate them deliberately at the boundary.

Choose exceptions by caller action


An exception is part of the Python API. Choose it according to what a caller can do next, rather than according to the Rust type that happened to fail:

Failure Python result Why
Invalid domain value ValueError or a domain-specific subclass The caller can change the value
Wrong Python argument type PyO3’s generated TypeError The caller violated the function signature
Missing or unreadable file An appropriate OSError subclass Existing Python code already understands file failures
Broken internal invariant An internal error to fix User input should not trigger a Rust panic

Keep the full cause for logs or exception chaining when it helps diagnosis, but make the public message stable enough for a person to act on it. Tests should usually assert the exception class and a meaningful fragment, not every word of an implementation detail.

Test three layers


No single test proves that the whole extension works:

Layer What it proves Example failure it localizes
Rust unit test Domain logic and invariants DOI validation accepts an invalid value
Python contract test Imports, conversions, values, and exceptions is_doi is registered under the wrong module
Clean-wheel smoke test Distribution metadata, ABI tags, and installed artifact The development build works but the wheel omits the extension

Keep the fast Rust tests close to the implementation, then use a smaller set of Python tests at the public path. Finish with at least one clean install because an editable or development environment can hide missing files and stale native artifacts.

Generate examples from invariants


Example-based tests preserve known behavior. Property-based tests describe a rule and ask a test runner to generate many inputs that might break it. When a failure is found, tools such as Hypothesis for Python and Proptest for Rust shrink the input toward a smaller counterexample.

Normalization, parsing, and serialization often have useful properties:

  • normalizing twice produces the same value as normalizing once;
  • parsing a serialized value reconstructs an equivalent value;
  • formatting a valid identifier produces another valid identifier; and
  • a batch operation agrees with applying the scalar operation to every item.

The same idempotence property can be expressed on either side of the boundary. In Python:

PYTHON

from hypothesis import given
from hypothesis import strategies as st

def normalize_identifier(raw: str) -> str:
    return raw.strip()

@given(st.text())
def test_normalization_is_idempotent(raw: str):
    once = normalize_identifier(raw)
    assert normalize_identifier(once) == once

In Rust:

RUST

use proptest::prelude::*;

fn normalize_identifier(raw: &str) -> String {
    raw.trim().to_string()
}

proptest! {
    #[test]
    fn normalization_is_idempotent(raw in ".*") {
        let once = normalize_identifier(&raw);
        prop_assert_eq!(normalize_identifier(&once), once);
    }
}

Keep the smallest failing example as a regular regression test. At a migration boundary, a differential property can send the same generated values to the old Python implementation and the new Rust-backed implementation. That checks compatibility over more cases than a hand-written table, but it still samples inputs; it is evidence, not a proof.

Test the public path and the build artifact


Python regression tests protect the names, conversions, return values, and exceptions users observe:

PYTHON

from acorn.schema.validate import is_doi

def test_is_doi():
    assert is_doi("10.11578/dc.20250604.1")
    assert not is_doi("totally invalid string")

Run the Python-facing suite in both supported environments:

BASH

pixi run -e py310 test
pixi run -e py313 test

Test the distributable artifact as well as the development install:

BASH

pixi run -e py313 wheel-smoke

The smoke task builds the locked release wheel, installs it into a clean environment, imports acorn, and verifies that the installed distribution is acorn-py at the expected version.

Challenge

Test a complete round trip

Add Python tests for the successful and failing paths of format_phone_number. Then add a validator test that imports is_orcid from acorn.schema.validate. Which assertions protect the Rust behavior, and which protect the cross-language contract?

The formatted value and validation booleans describe domain behavior. The import path, accepted Python argument types, ValueError class, and exception message describe the Python-Rust contract. Python-facing regression tests should exercise both: packaging and exception translation can change what callers observe even when the Rust function is correct.

Key Points
  • Binding signatures reveal whether values are borrowed, owned, converted, or stored in Rust-backed Python objects.
  • Expected Rust failures should become intentional Python exceptions.
  • Layered tests make domain, binding, and packaging failures easier to locate.
  • Property tests generate and shrink examples for invariants, round trips, and comparisons between the Python and Rust implementations.
  • Test the real acorn.schema import paths on every supported Python version.
  • A clean wheel smoke test catches packaging failures a development import can miss.

Content from Practical Guidance and Q&A


Last updated on 2026-09-25 | Edit this page

Estimated time: 20 minutes

Overview

Questions

  • How do we take a mixed-language prototype toward production?
  • How can we prototype in Rust without designing the whole system first?
  • When should we use property testing, Miri, model checking, or deductive verification?
  • Which maintenance and packaging questions should be answered early?
  • What is the smallest useful next step?

Objectives

  • Review a Python-Rust integration for maintainability and distribution risks.
  • Use explicit, temporary shortcuts to keep a Rust prototype moving.
  • Choose an assurance tool that matches the property and risk being checked.
  • Plan one evidence-driven next step for a real project.
  • Identify trustworthy routes for continued learning.

🤓 Prototyping tips


A prototype should answer one question quickly. It does not need polished error types, a deep module tree, or a generic framework. The useful distinction is between a shortcut that is easy to find later and one that silently becomes policy.

Keep momentum

  • Use todo!() for code you have not written yet. Use unreachable!() only when the prototype assumes a branch cannot occur. Both macros panic if execution reaches them.
  • Use .unwrap() liberally inside a throwaway spike where a crash is an acceptable answer. Before shipping, replace every user-triggerable unwrap with deliberate error handling.
  • Use .clone() to get past an initial ownership problem, then review the copies after the data flow settles. The Problem with Clones in Rust - Why Functional Rust is Slower Than You Think (And How to Fix It) demonstrates why unnecessary clones can matter.
  • Use println! and dbg! to inspect a running experiment. Remove noisy output or replace it with project logging before release.
  • Use assert! to record invariants as executable assumptions. For critical code, the assurance tools below can check stronger claims, but each has its own supported Rust subset and trust assumptions.

This intentionally rough function makes its deferred decisions searchable:

RUST

fn normalize_identifier(raw: Option<String>) -> String {
    let raw = raw.unwrap(); // PROTOTYPE: decide how missing input should fail.
    let candidate = raw.clone(); // PROTOTYPE: revisit ownership and copying.
    dbg!(&candidate);
    assert!(!candidate.is_empty(), "identifier must not be empty");
    match candidate.strip_prefix("doi:") {
        | Some(value) => value.to_string(),
        | None => todo!("support identifiers without a DOI prefix"),
    }
}

Keep the design plain

  • Prefer simple, concrete types and let Rust infer local types. Add annotations where they explain an API or resolve ambiguity.
  • Design through types, but avoid generic types and explicit lifetimes until the problem actually requires them.
  • Keep the hierarchy flat at first. A main.rs or lib.rs can sketch the eventual layout with declarations such as mod input; and mod output;.
  • Use Anyhow for convenient error context in an application prototype. Keep typed domain errors where callers, a library API, or the Python boundary need to distinguish failures.
  • Do not optimize the first draft. Establish correct behavior, then benchmark the installed release build before removing clones or adding complexity.

Shorten the feedback loop

  • Use an editor that exposes compiler feedback while you type. VS Code with rust-analyzer is a well-supported starting point.
  • Use Bacon to keep check, test, or clippy running in the background while files change.
  • Consider cargo-script eventually for small, single-file experiments. Cargo also has built-in single-file package support, but that feature remains unstable, so neither is a workshop prerequisite.

Match the assurance tool to the question


These tools are complementary, not a ladder in which every project must reach the last rung. Begin with a precise claim: “normalization is idempotent,” “this unsafe block has no undefined behavior,” or “the result always satisfies this postcondition.” Then choose the least costly tool that can answer it.

Technique or tool What it does Important limit
Unit and integration tests Run chosen examples through domain and boundary behavior Cover only the executions supplied by the tests
Property testing Generates examples for an invariant and shrinks failures Searches the input space; it does not prove the property
Miri Interprets executed Rust code and detects many forms of undefined behavior Checks only the explored executions and is not a formal verifier
Kani Model-checks proof harnesses with symbolic, bit-precise values Proofs are scoped to the harness, model, supported features, and bounds
Verus Uses Rust-like specifications, proof code, and SMT solving Supports a deliberate subset of Rust and requires proof annotations
Creusot Translates annotated safe Rust contracts into Why3 verification conditions Requires contracts and code within its supported subset
Prusti Checks Rust preconditions, postconditions, and invariants using Viper It is a prototype verifier with unsupported Rust and library features
Aeneas Translates a supported safe-Rust subset into pure functional code for proof assistants The proof is completed in Lean or another backend, outside ordinary Cargo tests

Property testing is often the easiest next step after example tests. It works especially well for parsers, round trips, normalization, and agreement between the Python and Rust implementations. Preserve generated counterexamples as ordinary regression tests.

Miri belongs earlier than formal proof when a crate contains unsafe code or low-level memory manipulation:

BASH

cargo +nightly miri test

A clean Miri run means the executed tests did not trigger the undefined behavior Miri detects. It does not establish that unexecuted paths are sound, and platform APIs or FFI may be unavailable under the interpreter. For a PyO3 project, run it on the testable Rust core and keep Python boundary tests as a separate layer.

Kani proof harnesses resemble property tests, but symbolic values let the model checker reason about every value represented by the harness. This small example checks all pairs of u8 values:

RUST

fn midpoint(left: u8, right: u8) -> u8 {
    let total = u16::from(left) + u16::from(right);
    (total / 2) as u8
}

#[cfg(kani)]
#[kani::proof]
fn midpoint_stays_between_inputs() {
    let left = kani::any::<u8>();
    let right = kani::any::<u8>();
    let result = midpoint(left, right);
    assert!(result >= left.min(right));
    assert!(result <= left.max(right));
}

Verus, Creusot, and Prusti express contracts and invariants close to Rust. Aeneas takes another route: it uses Rust’s ownership discipline to translate a supported safe subset into pure functional definitions for proof assistants, primarily Lean. These approaches make sense for a small, high-consequence core with a stable specification. They rarely justify verifying glue code, wheel metadata, or the Python import path.

Formal verification proves the stated property under the tool’s model and assumptions. A correct proof of the wrong specification is still the wrong program. Keep example tests for concrete requirements, property tests for broad input exploration, and Python-facing tests for the cross-language contract.

Before shipping


  • Review every todo!(), unreachable!(), unwrap(), temporary clone, and debugging print. Remove it or document why it is still valid.
  • Benchmark the installed acorn-py wheel against a realistic validation or schema workload.
  • Keep a Python-level regression suite for the public API.
  • Add property tests for important invariants and round trips; consider Miri or a verifier when unsafe code, critical invariants, or the cost of failure justifies it.
  • Build wheels in continuous integration for supported operating systems and architectures; use the abi3-py310 contract across compatible CPython versions.
  • Decide whether a source build is supported and document the Python 3.10 and Rust 1.96 minimums.
  • Keep Cargo.lock, pixi.lock, and the immutable ACORN dependency revision under review.
  • Check binary size, import time, error messages, and failure behavior.
  • Make ownership of both the Python and Rust code explicit within the team.

Before proposing a binding or packaging change to the reference project, run:

BASH

make check
make test
make wheel-smoke

A sensible route forward


  1. Profile a production-shaped workload.
  2. Choose one narrow component and write characterization tests around it.
  3. Build a Rust prototype without changing the public Python API.
  4. Compare correctness, speed, memory, build complexity, and maintenance cost.
  5. Keep, revise, or remove the prototype based on that evidence.

Compared with Python: advantages, not guarantees


Rust raises the floor for some kinds of correctness, but the compiler cannot prove that the program meets its users’ needs.

Tempting claim What we can responsibly expect
“Rust does not need extensive testing to transition to production.” Static checks replace some tests for type shape, ownership, and memory safety. Behavior, integration, packaging, and platform support still need tests.
“Rust prototypes already have good performance.” A straightforward Rust implementation often starts from a useful performance baseline. Measure a release build with production-shaped input before claiming an improvement.
“Rust can be refactored almost risk free.” Compiler-guided refactoring catches many broken dependencies. It cannot catch every semantic or policy change, so characterization and public-API tests still matter.
“Rust is easier to maintain.” Explicit types and exhaustive matching can reduce surprises. Maintenance is easier only when the team can also debug, build, package, and release the Rust component.

Questions to bring home


  • Is conversion at the boundary dominating the work?
  • Can callers tell that the implementation changed?
  • What happens on platforms without a pre-built wheel?
  • Can another maintainer debug and release both halves of the project?
  • Would better Python, NumPy, Cython, Numba, or a service boundary solve the problem more simply?
Challenge

Write your next step

Name one component you might evaluate, one metric that matters, and one reason to stop the experiment. Share it with a partner or record it in your project notes.

Workshop reference


Use the lesson reference for the concept map, commands, and primary documentation links introduced during the workshop.

Key Points
  • Production readiness includes packaging, observability, support, and team skills.
  • Prototype shortcuts are useful when they are explicit and reviewed before release.
  • Compiler checks reduce some testing burden; they do not replace behavioral and integration tests.
  • Property testing explores broad input spaces; Miri and formal verification answer narrower questions with different guarantees and constraints.
  • Use the integrated package for benchmarks.
  • Incremental adoption should remain reversible until evidence supports it.
  • The goal is a better Python project.