All in One View
Content from There and Back Again
Last updated on 2026-09-25 | Edit this page
Estimated time: 15 minutes

Overview
Questions
- What does Rust add to a successful Python project?
- When is a mixed-language project worth the added complexity?
- Where is Rust already present in the Python ecosystem?
Objectives
- Contrast the strengths and costs of Python and Rust.
- Recognize common motivations for placing Rust behind a Python API.
- Form a specific hypothesis before optimizing or migrating code.
Two languages, two jobs
Python optimizes for developer momentum: expressive code, a rich ecosystem, and a fast path from experiment to working software. Rust optimizes for control and confidence: predictable performance, memory safety without a garbage collector, and errors caught before deployment.
Ask where each language creates the most value in the project.
| Python is often strongest at | Rust is often strongest at |
|---|---|
| orchestration and application logic | tight compute-heavy loops |
| exploration and rapid iteration | predictable memory and latency |
| broad scientific and web ecosystems | safe low-level or concurrent work |
| approachable, flexible APIs | standalone native libraries |
Reasons to cross the boundary
Common motivations include a measured performance bottleneck, memory pressure, parallel work constrained by Python’s runtime, reuse of an existing Rust crate, or a need for stronger guarantees in a small critical component.
Crossing the boundary has costs. Native builds complicate packaging, values must be converted between language runtimes, and maintainers need enough Rust knowledge to support the result. A benchmark and an explicit goal give the decision a foundation.
Turn a hunch into a decision
Before writing Rust, record three things. They keep a promising experiment from quietly becoming an open-ended rewrite.
| Decision input | Question to answer | ACORN-shaped example |
|---|---|---|
| Baseline | What happens now on representative data? | Record median and slowest-case time for a realistic batch of DOI values. |
| Target | What improvement would matter to users or operators? | Reduce validation time enough to shorten an actual ingest job, not merely a microbenchmark. |
| Cost budget | What added complexity is acceptable? | Support the required wheels without making every Python maintainer debug Rust. |
Measure through the interface users will call. A Rust function may be fast in isolation while conversion, repeated boundary crossings, or wheel startup dominates the installed package. The experiment may also succeed without a speedup: reusing a trusted crate or making memory use more predictable can be a valid result when that was the stated target.
Our route: acorn-py
Throughout the workshop we will build a representative slice of acorn-py.
Its Python users work with familiar imports such as:
Behind that API, PyO3 exposes validation and schema behavior from the
Rust crates acorn-lib and acorn-schema. This
is a credible incremental boundary: identifier validation is
self-contained, the Python contract is easy to test, and the Rust
implementation already exists. The Python package exposes a selected
subset of the Rust APIs.
The completed project also gives us production questions to examine:
why the distribution is named acorn-py while the import is
acorn, why it uses the CPython stable ABI, and how its
wheels are tested on more than one Python version.
Write the reason before choosing the tool
Think of a Python project you know. Complete this sentence:
Moving ______ to Rust may improve ______, which we will verify by ______.
Then name one reason that component should remain in Python.
A useful answer names a narrow component, a measurable outcome, and a test. For example: “Moving file-format parsing to Rust may reduce import time, which we will verify with a representative benchmark.” Keeping CLI orchestration in Python may preserve iteration speed and ecosystem integrations.
Rust may already be in your environment
Python packages can expose native code while presenting ordinary Python modules to their users. Projects such as Polars, Pydantic Core, Ruff, uv, and Tokenizers demonstrate different ways Rust can support Python-facing tools. They show that a carefully chosen boundary can work well, while each project still needs its own reason to adopt Rust.
- Python and Rust are complementary when each has a clearly defined role.
- Start from a measured need before considering a rewrite.
- The integration boundary and its maintenance cost are part of the design.
Content from Translating Python into Rust
Last updated on 2026-09-25 | Edit this page
Estimated time: 35 minutes
Overview
Questions
- How do familiar Python constructs appear in Rust?
- How do Rust expressions, semicolons, and
returndetermine a value? - How do Python strings and lambdas compare with Rust strings and closures?
- Which differences are syntax, and which change the way we design programs?
- How are Rust attributes different from Python decorators?
Objectives
- Read Rust variables, collections, functions, modules, and loops.
- Explain when a Rust block returns its final expression and when it
returns
(). - Distinguish
&str,String,char, and.chars(). - Read a Rust closure and explain how it captures surrounding values.
- Use
OptionandResultto represent missing values and failures. - Use pattern matching to handle explicit alternatives.
- Distinguish compile-time Rust attributes from runtime Python decorators.
Build a first mental map
Rust will feel less foreign when we anchor new syntax to familiar Python ideas. The comparisons below are starting points; similar syntax can behave differently.
| Python | Rust | Important difference |
|---|---|---|
name = "Bilbo" |
let name = "Bilbo"; |
Rust bindings are immutable by default. |
MAX_RETRIES = 3 |
const MAX_RETRIES: u8 = 3; |
Python uses a naming convention; Rust enforces a declared type and compile-time value. |
list[int] |
Vec<i64> |
A Rust vector has one element type. |
dict[str, int] |
HashMap<String, i64> |
Key and value types are explicit. |
None |
Option::None |
Absence is represented in the type. |
| exception | Result::Err |
Recoverable failure is commonly returned. |
for item in items |
for item in items |
Ownership determines what the loop may consume. |
Variables, types, and functions
PYTHON
from acorn.schema.validate import is_doi
def count_valid_dois(values: list[str]) -> int:
return sum(is_doi(value) for value in values)
RUST
use acorn_schema::validation::rules;
fn count_valid_dois(values: &[String]) -> usize {
values
.iter()
.filter(|value| rules::doi(value.as_str()).is_ok())
.count()
}
The Rust function accepts a borrowed slice, so callers can provide a
view of a vector without transferring ownership. Its return type records
that a count is never negative. ACORN groups scalar validators under
validation::rules; each rule returns a structured
Result, and this iterator counts the successful results.
The compiler verifies that each call respects this contract.
Expressions produce values
Most Rust constructs are expressions, including blocks,
if, and match. The final expression in a block
becomes that block’s value when it has no semicolon:
RUST
fn doubled(value: i64) -> i64 {
value * 2
}
fn describe(valid: bool) -> &'static str {
if valid {
"valid"
} else {
"invalid"
}
}
Adding a semicolon evaluates an expression and discards its value.
The block then produces the unit value (), roughly Rust’s
“no useful value” type. This version therefore fails to compile because
the signature promises an i64:
Use return to leave the current function early. Rust
permits it for the final value too, but an unadorned tail expression is
the usual style:
RUST
fn checked_double(value: i64) -> Result<i64, String> {
if value < 0 {
return Err("value must not be negative".to_string());
}
Ok(value * 2)
}
Python differs in two ways. An expression on the last line of a
normal Python function is discarded, and a function that reaches the end
returns None. Python needs return value to
send a value to the caller. Python semicolons only separate statements;
adding or removing one does not decide a function’s return value.
| Form | Rust | Python |
|---|---|---|
Final expression without ;
|
Becomes the block’s value | Evaluated and discarded in a normal function |
Expression followed by ;
|
Value is discarded; the statement produces
()
|
Semicolon is an optional statement separator |
return value |
Exits the function explicitly, often for an early path | Exits the function and supplies its value |
| Reaching the end | Returns the tail expression, or () if none
exists |
Returns None
|
Strings are UTF-8, but the types differ
Python’s single and double quotes create the same str
type. Choose the form that follows the project’s style or avoids
escapes:
PYTHON
single = 'There and Back Again'
double = "There and Back Again"
assert single == double
assert isinstance(single[0], str)
Rust uses double quotes for strings and single quotes for one
char. Its two main string types express ownership. Single
quotes also appear in lifetime names such as 'a; context
distinguishes a lifetime from a character literal.
RUST
let borrowed: &str = "There and Back Again";
let owned: String = borrowed.to_string();
let letter: char = 'T';
let first: Option<char> = owned.chars().next();
| Form | Meaning |
|---|---|
"text" |
A string literal, normally used as a borrowed
&'static str
|
&str |
A borrowed UTF-8 string slice; it does not own or grow the text |
String |
Owned, growable UTF-8 text |
'T' |
One Unicode scalar value of type char, not
a one-character string |
text.chars() |
An iterator over Unicode scalar values |
text.bytes() |
An iterator over the UTF-8 bytes |
Rust does not allow text[0]: a UTF-8 character may
occupy more than one byte, so a numeric index would be ambiguous. Use
.chars() when Unicode scalar values are the intended unit
and .bytes() when the encoding bytes are. A visible
user-perceived character can contain several scalar values, so code that
needs grapheme clusters should use a Unicode-segmentation library rather
than assuming that one char equals one displayed
character.
A function normally accepts &str when it only needs
to read text and returns String when it creates owned
text:
Collections and iteration
RUST
fn positive_squares(values: &[i64]) -> Vec<i64> {
values
.iter()
.filter(|value| **value > 0)
.map(|value| value * value)
.collect()
}
Iterator chains may look like Python comprehensions, but they remain strongly typed and are compiled into efficient loops.
Anonymous functions are closures
Python calls its compact anonymous function a lambda.
Rust calls the corresponding construct a closure and places parameters
between vertical bars:
Why is it called a lambda?
The name predates Python by decades. In 1932, mathematician Alonzo
Church used the Greek letter lambda in his lambda
calculus, a compact formal notation for creating functions by
abstraction and applying them to arguments. John McCarthy’s original Lisp
paper used LAMBDA in 1960, helping carry the term from
mathematical logic into programming-language vocabulary.
Python keeps lambda as the keyword for an anonymous
function expression. Rust uses the term closure, which
emphasizes that the callable value may capture part of its surrounding
environment. A lambda or closure can still be assigned to a name;
“anonymous” describes how it was created, not whether the program can
refer to it later.
Python limits a lambda body to one expression. A Rust closure may use either one expression or a block, and parameter and return types are usually inferred from the call site. Both languages can capture surrounding values:
The Python closure looks up the captured name when it is called. Rust
decides whether a closure borrows, mutably borrows, or consumes each
captured value from how the closure uses it. Adding move
forces capture by value, which is common when a closure must outlive the
current scope or move to another thread. That capture behavior
determines whether the closure implements Fn,
FnMut, or FnOnce.
Use a named function when the operation is reused or deserves its own
test. Closures work well for small, local transformations passed to
map, filter, thread spawners, and similar
APIs.
Pattern matching and errors
RUST
fn parse_port(raw: &str) -> Result<u16, String> {
match raw.parse::<u16>() {
| Ok(port) if port > 0 => Ok(port),
| Ok(_) => Err("port must be greater than zero".to_string()),
| Err(error) => Err(format!("invalid port: {error}")),
}
}
match makes the successful and unsuccessful paths
visible. The ? operator can propagate an error when a
function does not need to transform it. ACORN’s domain validators use
the same shape with ValidationError, which preserves a
stable error code separately from its human-readable message.
Use ? for deliberate propagation,
not automatic error handling
Use ? when the current function deliberately delegates a
failure to its caller. Avoid it when this layer has the context to
recover, attach a domain error, or choose a different path; use
match, map_err, or another explicit
transformation instead.
The ? operator does not ignore an error
and does not panic. On an Err, it returns
early with a compatible Err; on an Ok, it
unwraps the value. A function returning Result therefore
still has an explicit, deterministic output. Panics come from operations
such as unwrap(), expect(), and
panic!(), not from ? itself.
Some functional-programming-oriented teams avoid ? when
its early return hides a branch that matters to the design. They prefer
match or combinators so the transformation remains visible
and composable. That is a readability choice, not a requirement of
functional programming: deliberate Result propagation with
? is still typed and non-panicking.
Attributes resemble decorators, but run at a different time
Python decorators and Rust attributes both place declarative-looking syntax above a function, class, or type. That visual similarity is useful for reading code, but their execution models differ.
Both examples ask tooling to supply common behavior. Python calls
dataclass after the class body executes, usually during
module import. Rust expands derive while compiling and
generates implementations of the named traits; nothing runs merely
because the program starts.
| Python decorator | Rust attribute | |
|---|---|---|
| Syntax | @decorator |
#[attribute] |
| When it acts | When the decorated definition executes | During parsing or compilation |
| What it receives | A runtime object such as a function or class | Source-level input understood by the compiler or a macro |
| Common jobs | Wrap, register, or replace an object | Generate implementations, select tests, configure compilation, or generate binding code |
An outer attribute such as #[test] applies to the item
that follows. An inner attribute such as
#![allow(dead_code)] applies to the item that contains it,
often a module or crate. Later, PyO3 attributes such as
#[pyfunction] and #[pymodule] will generate
Python binding code at compile time. They do not behave like stacked
runtime wrappers, so decorator order is not a reliable mental model for
attribute order.
Keep modules and tests close to the domain
ACORN organizes identifier implementations under pid and
scalar rules under validation::rules. Tests sit beside
those modules instead of in one distant integration-test file. A
validator test uses small tables and names the value when an assertion
fails:
RUST
use acorn_schema::validation::rules;
#[test]
fn test_is_doi() {
let values = [
"10.1000/182",
"https://doi.org/10.11578/dc.20250604.1",
"10.11578/dc.20250604.1",
];
values.into_iter().for_each(|value|
assert!(rules::doi(value).is_ok(), "{value} is NOT a valid DOI")
);
}
In the crate itself, a domain module includes its adjacent tests with
#[cfg(test)] mod tests;, so test-only code is absent from
normal builds.
Translate a small function
RUST
use acorn_schema::validation::rules;
fn first_doi(values: &[String]) -> Option<&str> {
values
.iter()
.find(|value| rules::doi(value.as_str()).is_ok())
.map(String::as_str)
}
Option<&str> records that the search may not
find a value and borrows the matching text from the input collection. We
will return to that reference when we discuss ownership and
lifetimes.
- Familiar surface syntax can help us begin reading Rust.
- A Rust block can return its final expression; a semicolon discards
that expression’s value, while
returnexits explicitly. - Python quote style does not change its string type; Rust
distinguishes borrowed
&str, ownedString, and scalarcharvalues. - Rust closures use
|arguments| expressionand capture by borrow, mutable borrow, or value according to how they use their environment. - Rust makes mutability, data types, absence, and recoverable errors explicit.
- The
?operator propagates a typed failure; it neither handles the failure locally nor causes a panic. - Rust attributes provide compile-time instructions; Python decorators operate on runtime objects as definitions execute.
- A slice such as
&[i64]lets a function inspect sequential data without taking ownership of it.
Content from Where the Analogy Ends
Last updated on 2026-09-25 | Edit this page
Estimated time: 25 minutes
Overview
Questions
- What problems are ownership and borrowing designed to prevent?
- How do traits and enums shape Rust APIs?
- What does “fearless concurrency” mean in practice?
- Do Rust’s guarantees leave room for exploratory code?
Objectives
- Explain ownership, borrowing, and lifetimes without relying on Python analogies.
- Distinguish Rust enums and traits from superficially similar Python features.
- Connect compile-time checks to safe resource use and concurrency.
- Separate prototyping shortcuts from production error and ownership policies.
Ownership is a resource model
Every Rust value has an owner. When that owner goes out of scope, the value is dropped. A value can move to a new owner, be borrowed immutably by many readers, or be borrowed mutably by one writer. These rules prevent use-after-free and data races before the program runs.
RUST
fn label_length(label: &str) -> usize {
label.len()
}
fn main() {
let label = String::from("second breakfast");
let length = label_length(&label);
println!("{label} has {length} bytes");
}
label_length borrows the string, so label
remains available to its owner.
When the borrow checker objects, first ask what the callee needs to do. The answer usually selects the parameter shape:
| Callee’s intent | Typical parameter | Consequence for the caller |
|---|---|---|
| Read for the duration of the call | &T |
The caller keeps ownership; many immutable borrows may coexist. |
| Change the caller’s value | &mut T |
The caller keeps ownership; only one mutable borrow may exist at a time. |
| Store or consume the value | T |
Ownership moves unless the type implements
Copy. |
| Work on an independent duplicate |
T from value.clone()
|
The caller keeps the original and pays the clone’s explicit cost. |
Start with the least authority the function needs. A validator that
only reads text should accept &str; taking a
String would force ownership transfer or an unnecessary
clone.
Lifetimes describe relationships
Most lifetimes are inferred. When a function returns a reference, an explicit lifetime may be needed to state which input keeps that reference valid.
RUST
fn choose_longer<'a>(left: &'a str, right: &'a str) -> &'a str {
if left.len() >= right.len() {
left
} else {
right
}
}
The annotation does not extend either value’s lifetime. It gives the compiler a relationship it can check.
Enums carry data; traits describe behavior
Rust enums model a closed set of alternatives, and each variant may carry different data. Pattern matching ensures that every case is considered. Traits define shared behavior and can be used for generics or dynamic dispatch.
acorn-schema makes both ideas concrete. A patent is one
type with distinct granted, application, and publication variants. The
acorn-py binding matches those variants and constructs one
consistent Python-facing Patent object.
RUST
use acorn_schema::pid::patent::{CountryCode, KindCode};
use acorn_schema::pid::{Patent, PersistentIdentifier};
fn kind(patent: &Patent) -> &'static str {
match patent {
| Patent::Granted { .. } => "grant",
| Patent::Application { .. } => "application",
| Patent::Publication { .. } => "publication",
}
}
fn main() {
let patent = Patent::Granted {
country_code: Some(CountryCode::US),
kind_code: KindCode::B2,
serial_number: "7654321".to_string(),
};
assert_eq!(kind(&patent), "grant");
assert_eq!(patent.identifier(), "US 7654321 B2");
}
This is the schema crate’s real Patent enum rather than
a workshop-only facsimile. Importing PersistentIdentifier
brings its shared methods into scope; ACORN’s identifier types use that
trait for behaviors such as identifier() and
schema_uri(). The exhaustive match must be
revisited if the schema adds another patent variant.
Compile-time guarantees and concurrency
The same ownership rules apply across threads. A value cannot be mutated from multiple places unless its type provides safe synchronization. Data races are far harder to express, although other concurrency bugs remain possible.
Scoped threads can safely borrow inputs because Rust proves that every worker finishes before the scope ends. Each worker below computes its own value, and the parent combines the results instead of sharing mutable state:
RUST
use std::thread;
fn count_nonempty(left: &[String], right: &[String]) -> Result<usize, &'static str> {
thread::scope(|scope| {
let left_worker =
scope.spawn(|| left.iter().filter(|value| !value.is_empty()).count());
let right_worker =
scope.spawn(|| right.iter().filter(|value| !value.is_empty()).count());
match (left_worker.join(), right_worker.join()) {
| (Ok(left_count), Ok(right_count)) => Ok(left_count + right_count),
| _ => Err("a counting worker panicked"),
}
})
}
The slices are borrowed, not copied. Returning partial counts also
avoids a mutex. Ownership rules prevent either worker from outliving the
borrowed data, while the explicit Result records that a
worker may panic. Rust still cannot prevent deadlocks, starvation, or a
logically incorrect division of work.
😢 Typical misbeliefs
Rust prototypes do not have to look like finished library code. These beliefs usually come from treating every early decision as permanent:
| Misbelief | A more useful working assumption |
|---|---|
| “Memory safety and prototyping just don’t go together.” | Compiler feedback can be part of the experiment. It rules out invalid memory relationships while you test the idea. |
| “Ownership and borrowing take the fun out of prototyping.” | They can interrupt a first draft, so begin with owned values and use a temporary clone when that keeps the experiment moving. Revisit the ownership once the shape is clear. |
| “You have to get all the details right from the beginning.” | Type inference, concrete types, and
todo!() let you postpone decisions without pretending the
unfinished path works. |
| “Rust always requires you to handle errors.” | Rust makes recoverable failure visible with
Result, but a prototype can deliberately stop with
unwrap(), todo!(), or
unreachable!(). Production code still needs an intentional
policy for user-triggerable failures. |
The prototype’s job is to answer a question. Rust makes many shortcuts visible, which gives us a practical list to revisit before shipping.
- Ownership determines who is responsible for a value and when it is released.
- Borrowing provides temporary access without transferring ownership.
- Lifetimes let the compiler verify relationships between references.
- Enums, traits, and concurrency checks are central Rust design tools.
- Prototypes may defer decisions, provided their shortcuts remain visible and are reviewed before production.
Content from Adding Rust to Python Incrementally
Last updated on 2026-09-25 | Edit this page
Estimated time: 25 minutes
Overview
Questions
- Where should the Python-Rust boundary go?
- How can we preserve an existing Python API?
- How much work should cross the boundary in one call?
- What should stay in Python?
Objectives
- Evaluate a component as a candidate for a Rust implementation.
- Design a narrow, stable, and testable language boundary.
- Preserve a Python-facing API while changing its implementation.
- Protect existing behavior with characterization tests before replacing it.
Choose a seam for Rust
A good first Rust component has a clear input and output, meaningful work per call, limited dependence on Python objects, and tests that already describe its behavior. Parsers, codecs, validators, algorithms, and self-contained data transformations are common candidates.
A poor first boundary crosses languages inside a tight Python loop, depends on many callbacks into Python, or translates a large object graph on every call.
Keep the public API Pythonic
Start from the import path and behavior callers should keep. For our worked example, the contract is ordinary Python even though its implementation is a native extension:
PYTHON
from acorn.schema.validate import is_doi
assert is_doi("10.11578/dc.20250604.1")
assert not is_doi("not an identifier")
acorn-py is the distribution name shown by package
installers, while acorn is the import name and
acorn.schema.validate is a nested module registered by the
extension. Treat all three names as part of the packaging contract.
Callers can keep using those public names. The internal crate names and
pinned Git revision remain implementation details.
Our first seam is one validator: string in, boolean out. Later checkpoints add a Rust-backed identifier class and a fallible file-reading operation. That sequence keeps each boundary narrow enough to test before the next one is added.
Protect the behavior before replacing the code
A characterization test records what callers observe today, including awkward cases that a rewrite may be tempted to “fix.” Run the same test against the Python implementation, the development extension, and eventually the wheel:
PYTHON
import pytest
from acorn.schema.validate import is_doi
@pytest.mark.parametrize(
("value", "expected"),
[
("10.1000/182", True),
("https://doi.org/10.11578/dc.20250604.1", True),
("not an identifier", False),
("", False),
],
)
def test_is_doi_contract(value: str, expected: bool):
assert is_doi(value) is expected
This test intentionally imports the public path rather than a private helper. It protects the name, accepted Python value, return type, and domain behavior in one place. Add edge cases from production data before changing the implementation; invented examples rarely capture every compatibility promise.
Choose the call granularity
The narrowest function is not always the cheapest boundary. Consider how the real caller uses it:
| Boundary shape | Advantage | Cost or risk |
|---|---|---|
str -> bool |
Simple contract and easy error isolation | A Python loop may make thousands of native calls. |
list[str] -> list[bool] |
One crossing amortizes call overhead | Every string still needs extraction, and the whole batch occupies memory. |
| Python callback from Rust | Preserves flexible Python behavior | Repeated callbacks couple the implementation to Python and may constrain parallel work. |
For a batch API, the conceptual Rust shape remains simple and concrete:
RUST
use acorn_schema::validation::rules;
fn are_dois(values: &[String]) -> Vec<bool> {
values
.iter()
.map(|value| rules::doi(value.as_str()).is_ok())
.collect()
}
Do not add batching merely because it sounds faster. Benchmark the scalar and batch interfaces with realistic inputs, including the conversion performed by PyO3. Choose the smallest contract that meets the measured target.
A boundary checklist
- Is the bottleneck measured with representative data?
- Can inputs and outputs be expressed with simple, stable types?
- Do characterization tests describe the public behavior before it changes?
- Is each call substantial enough to justify conversion overhead?
- Can the behavior be tested from both Python and Rust?
- Will the project build wheels for every supported platform and Python version?
- Does the expected benefit justify another language and toolchain?
Find the seam
For a Python package that reads files, validates records, calculates summaries, and generates plots, choose one first candidate for Rust. Sketch the function signature at the Python boundary and list one component you would deliberately leave in Python.
Parsing or record validation may be a good boundary when it is measurable and self-contained. Plotting should generally remain in Python so the project keeps its mature visualization ecosystem and flexible user-facing options.
- Migrate one well-tested, high-value component at a time.
- Minimize language crossings and data conversion.
- Keep the public Python API separate from the native implementation.
- Leaving a component in Python is a valid engineering outcome.
Content from Building a Python Extension with PyO3
Last updated on 2026-09-25 | Edit this page
Estimated time: 30 minutes
Overview
Questions
- How does
acorn-pyexpose Rust functions and nested modules to Python? - What do PyO3, Maturin, Cargo, and Pixi each provide?
- What happens between
maturin developandimport acorn? - Why do the distribution, crate, and import names differ?
Objectives
- Identify the roles of Pixi, Cargo, PyO3, and Maturin in
acorn-py. - Expose an ACORN validator through
acorn.schema.validate. - Build and install the extension in the locked Python 3.13 environment.
- Trace a development build from the task runner to Python’s import machinery.
The project layers
-
Pixiselects the locked Python and Rust toolchains and runs project tasks. -
Cargoresolves and buildsacorn-lib,acorn-schema, PyO3, and the local Rust crate. -
acorn-schemaseparates persistent-identifier types inpidfrom structured scalar validators invalidation::rules. -
PyO3defines the native module, functions, classes, and error mappings. -
Maturinbuilds the Rust crate as an installable Python distribution and wheel.
The teaching checkpoints retain acorn-py’s real
pyproject.toml, Cargo.toml,
pixi.lock, and Cargo.lock. We reduce
src/lib.rs to one validator, then add the remaining
boundary pieces incrementally. The completed checkout from setup remains
available for comparison.
Follow one development build
pixi run -e py313 develop is short, but it coordinates
several contracts:
| Phase | Tool | What to inspect when it fails |
|---|---|---|
| Select an environment and task | Pixi | The environment, locked dependencies, and task in
pixi.toml
|
| Read the Python build configuration | Maturin | The backend and [tool.maturin] settings in
pyproject.toml
|
| Compile the native library | Cargo and rustc
|
Features, imports, and the dependency revision |
| Install the development artifact | Maturin | The extension filename and active Python environment |
| Initialize the imported module | Python and PyO3 | The import name, #[pymodule] name, and
nested-module registration |
The development install points Python at the compiled extension.
Saving src/lib.rs does not rebuild that extension, so run
develop again after a Rust change. If an import appears to
use the wrong build, verify the interpreter through the same
environment:
Read the package contract
Three names describe different layers of the same project:
| Name | Where it appears | Meaning |
|---|---|---|
acorn-py |
pyproject.toml and
Cargo.toml
|
Python distribution and Rust crate |
acorn |
[tool.maturin] module-name |
Python import name |
acorn.schema.validate |
PyO3 module registration | Public validator namespace |
The project requires Python 3.10 or newer and enables PyO3’s
abi3-py310 feature. A single wheel built against that
stable ABI can support multiple compatible CPython minor versions.
Cargo.toml pins acorn-lib and
acorn-schema to one immutable public Git revision. No local
ACORN checkout is needed. Changing that revision requires a dependency
review and falls outside the workshop.
Expose the first validator
At the first binding checkpoint, src/lib.rs contains one
Python function and the module structure needed to preserve the public
import path:
RUST
use acorn_schema::validation::rules;
use pyo3::prelude::*;
#[pyfunction]
fn is_doi(value: &str) -> bool {
rules::doi(value).is_ok()
}
#[pymodule]
#[pyo3(name = "acorn")]
fn acorn_py(module: &Bound<'_, PyModule>) -> PyResult<()> {
let python = module.py();
let schema = PyModule::new(python, "schema")?;
let validate = PyModule::new(python, "validate")?;
validate.add_function(wrap_pyfunction!(is_doi, &validate)?)?;
schema.add_submodule(&validate)?;
module.add_submodule(&schema)?;
let sys = python.import("sys")?;
let modules = sys.getattr("modules")?;
modules.set_item("acorn.schema", schema)?;
modules.set_item("acorn.schema.validate", validate)?;
Ok(())
}
#[pyfunction] makes the Rust function callable by
Python. wrap_pyfunction! adds it to the
validate module, and the sys.modules entries
make Python’s nested import machinery recognize the public paths.
Read the initializer as five operations: obtain the Python handle,
create the nested modules, wrap and add functions, register the public
paths, and return Ok(()). The ? operators
deliberately propagate initialization failures as Python exceptions
because there is no useful local recovery. Propagating an error is
different from ignoring it.
Build the current checkpoint and call it from Python:
BASH
pixi run -e py313 develop
pixi run -e py313 python -c "from acorn.schema.validate import is_doi; print(is_doi('10.11578/dc.20250604.1'))"
The command should print True. Re-run
develop after changing Rust code so the active development
environment receives the rebuilt extension.
Diagnose the layer, not just the symptom
| Symptom | Likely layer | First check |
|---|---|---|
| Rust compiler error | Cargo or Rust | Read the first diagnostic before the cascading errors |
ImportError mentioning
PyInit_acorn
|
Module naming | Compare module-name with
#[pyo3(name = "acorn")]
|
ModuleNotFoundError for
acorn.schema.validate
|
Nested registration | Check both sys.modules entries |
| Python still shows old behavior | Development install | Re-run develop in the interpreter’s
environment |
| Development import passes but the wheel fails | Packaging or ABI | Inspect wheel tags and test a clean installation |
Start with the earliest failing layer. Rewriting Rust cannot repair a wrong Python interpreter, and changing the Python test cannot repair a missing module initializer.
Add a class when identity and behavior belong together
A function is enough for a yes-or-no validation. A Rust-backed class is useful when Python should keep a validated value and ask it for related behavior later. This teaching-sized class validates once during construction:
RUST
use pyo3::exceptions::PyValueError;
#[pyclass(frozen, module = "acorn.schema.pid")]
struct Doi {
value: String,
}
#[pymethods]
impl Doi {
#[new]
fn new(value: String) -> PyResult<Self> {
match rules::doi(value.as_str()) {
| Ok(_) => Ok(Self { value }),
| Err(error) => Err(PyValueError::new_err(error.to_string())),
}
}
#[getter]
fn value(&self) -> &str {
self.value.as_str()
}
fn __str__(&self) -> &str {
self.value.as_str()
}
}
Register it under a sibling pid module using the same
parent-child and sys.modules pattern as
validate. Create and attach pid before
attaching schema to the top-level module:
RUST
let pid = PyModule::new(python, "pid")?;
pid.add_class::<Doi>()?;
schema.add_submodule(&validate)?;
schema.add_submodule(&pid)?;
module.add_submodule(&schema)?;
After obtaining sys.modules, register all three
qualified names:
RUST
modules.set_item("acorn.schema", schema)?;
modules.set_item("acorn.schema.validate", validate)?;
modules.set_item("acorn.schema.pid", pid)?;
frozen prevents Python callers from replacing
Rust-managed fields. The constructor turns invalid input into
ValueError, so a successfully created Doi
always satisfies its invariant. Prefer a function when no durable state
or behavior justifies a class.
Bind a second validator
The binding follows the same shape as the first validator:
Register it inside acorn_py:
Then run pixi run -e py313 develop before executing the
Python assertions.
- Pixi makes the workshop toolchain reproducible; Cargo and Maturin build the native Python package.
- PyO3 exposes selected ACORN behavior without exposing the whole Rust crate.
- Distribution, import, and nested-module names are separate contracts.
- A build crosses distinct layers; diagnose the earliest layer that fails.
- Use a
#[pyclass]when validated state and related behavior must persist across Python calls. - The first working boundary is deliberately small: one string in and one boolean out.
Content from From Python to Rust and Back Again
Last updated on 2026-09-25 | Edit this page
Estimated time: 30 minutes
Overview
Questions
- What happens to strings, paths, objects, and errors at the language boundary?
- How do we choose conversions, exceptions, and tests for a binding contract?
- How can generated examples exercise invariants and round trips?
- How does
acorn-pytest its public Python API? - What does the stable-ABI wheel smoke test prove?
Objectives
- Select PyO3 argument and return types appropriate for an ACORN binding.
- Map a fallible Rust operation to a meaningful Python exception.
- Distinguish domain, binding-contract, and packaging tests.
- Write a property test for a pure transformation.
- Test Rust-backed behavior through the public Python import path.
Data crosses by borrowing, conversion, or extraction
PyO3 can convert many ordinary values between Python and Rust. The choice in a binding signature documents what happens at the boundary:
-
&strborrows text for the duration of a call such asis_doi; -
Stringowns converted text that must outlive the call; -
PathBufaccepts Python path-like input for file operations; and - a
#[pyclass]stores Rust data behind a Python-visible object.
Use the least complicated type that expresses the call’s needs:
| Rust signature | Python-side input | Boundary effect | Good fit |
|---|---|---|---|
&str |
str |
Borrow text for this call | A validator that only reads its argument |
String |
str |
Create owned Rust text | A value retained or transformed after extraction |
Vec<String> |
A sequence of strings | Build a Rust vector and own its elements | A batch operation with enough work to justify one conversion |
PathBuf |
A path-like object | Convert into an owned native path | Reading or writing a document |
#[pyclass] |
A Python-visible Rust object | Keep Rust-owned state across calls | A typed identifier with methods and invariants |
acorn-py starts with coarse operations. One call
validates a complete identifier or reads a complete research-activity
document. It does not ask Python to cross the native boundary once per
byte or once per validation rule.
Convenient conversion can still allocate or copy. Measure realistic calls before redesigning an interface around more complex borrowed-buffer APIs.
Turn expected failures into Python exceptions
An invalid identifier naturally returns false, but a
formatter or file read can fail with information callers need.
PyResult<T> lets the binding return a value or raise
a Python exception:
RUST
use acorn_schema::validation;
use pyo3::exceptions::PyValueError;
use pyo3::prelude::*;
#[pyfunction]
fn format_phone_number(value: &str) -> PyResult<String> {
validation::format_phone_number(value).map_err(|error|
PyValueError::new_err(error.to_string())
)
}
The public Python contract is now explicit:
PYTHON
import pytest
from acorn.schema.validate import format_phone_number
def test_invalid_phone_number():
with pytest.raises(ValueError, match="Unable to format"):
format_phone_number("not a phone number")
The schema function returns ACORN’s structured
ValidationError; converting it to text uses its
human-readable message while PyO3 supplies the Python exception type. Do
not use panics to handle expected domain failures. Return Rust
Result values and translate them deliberately at the
boundary.
Choose exceptions by caller action
An exception is part of the Python API. Choose it according to what a caller can do next, rather than according to the Rust type that happened to fail:
| Failure | Python result | Why |
|---|---|---|
| Invalid domain value |
ValueError or a domain-specific
subclass |
The caller can change the value |
| Wrong Python argument type | PyO3’s generated TypeError
|
The caller violated the function signature |
| Missing or unreadable file | An appropriate OSError subclass |
Existing Python code already understands file failures |
| Broken internal invariant | An internal error to fix | User input should not trigger a Rust panic |
Keep the full cause for logs or exception chaining when it helps diagnosis, but make the public message stable enough for a person to act on it. Tests should usually assert the exception class and a meaningful fragment, not every word of an implementation detail.
Test three layers
No single test proves that the whole extension works:
| Layer | What it proves | Example failure it localizes |
|---|---|---|
| Rust unit test | Domain logic and invariants | DOI validation accepts an invalid value |
| Python contract test | Imports, conversions, values, and exceptions |
is_doi is registered under the wrong
module |
| Clean-wheel smoke test | Distribution metadata, ABI tags, and installed artifact | The development build works but the wheel omits the extension |
Keep the fast Rust tests close to the implementation, then use a smaller set of Python tests at the public path. Finish with at least one clean install because an editable or development environment can hide missing files and stale native artifacts.
Generate examples from invariants
Example-based tests preserve known behavior. Property-based tests describe a rule and ask a test runner to generate many inputs that might break it. When a failure is found, tools such as Hypothesis for Python and Proptest for Rust shrink the input toward a smaller counterexample.
Normalization, parsing, and serialization often have useful properties:
- normalizing twice produces the same value as normalizing once;
- parsing a serialized value reconstructs an equivalent value;
- formatting a valid identifier produces another valid identifier; and
- a batch operation agrees with applying the scalar operation to every item.
The same idempotence property can be expressed on either side of the boundary. In Python:
PYTHON
from hypothesis import given
from hypothesis import strategies as st
def normalize_identifier(raw: str) -> str:
return raw.strip()
@given(st.text())
def test_normalization_is_idempotent(raw: str):
once = normalize_identifier(raw)
assert normalize_identifier(once) == once
In Rust:
RUST
use proptest::prelude::*;
fn normalize_identifier(raw: &str) -> String {
raw.trim().to_string()
}
proptest! {
#[test]
fn normalization_is_idempotent(raw in ".*") {
let once = normalize_identifier(&raw);
prop_assert_eq!(normalize_identifier(&once), once);
}
}
Keep the smallest failing example as a regular regression test. At a migration boundary, a differential property can send the same generated values to the old Python implementation and the new Rust-backed implementation. That checks compatibility over more cases than a hand-written table, but it still samples inputs; it is evidence, not a proof.
Test the public path and the build artifact
Python regression tests protect the names, conversions, return values, and exceptions users observe:
PYTHON
from acorn.schema.validate import is_doi
def test_is_doi():
assert is_doi("10.11578/dc.20250604.1")
assert not is_doi("totally invalid string")
Run the Python-facing suite in both supported environments:
Test the distributable artifact as well as the development install:
The smoke task builds the locked release wheel, installs it into a
clean environment, imports acorn, and verifies that the
installed distribution is acorn-py at the expected
version.
Test a complete round trip
Add Python tests for the successful and failing paths of
format_phone_number. Then add a validator test that imports
is_orcid from acorn.schema.validate. Which
assertions protect the Rust behavior, and which protect the
cross-language contract?
The formatted value and validation booleans describe domain behavior.
The import path, accepted Python argument types, ValueError
class, and exception message describe the Python-Rust contract.
Python-facing regression tests should exercise both: packaging and
exception translation can change what callers observe even when the Rust
function is correct.
- Binding signatures reveal whether values are borrowed, owned, converted, or stored in Rust-backed Python objects.
- Expected Rust failures should become intentional Python exceptions.
- Layered tests make domain, binding, and packaging failures easier to locate.
- Property tests generate and shrink examples for invariants, round trips, and comparisons between the Python and Rust implementations.
- Test the real
acorn.schemaimport paths on every supported Python version. - A clean wheel smoke test catches packaging failures a development import can miss.
Content from Practical Guidance and Q&A
Last updated on 2026-09-25 | Edit this page
Estimated time: 20 minutes
Overview
Questions
- How do we take a mixed-language prototype toward production?
- How can we prototype in Rust without designing the whole system first?
- When should we use property testing, Miri, model checking, or deductive verification?
- Which maintenance and packaging questions should be answered early?
- What is the smallest useful next step?
Objectives
- Review a Python-Rust integration for maintainability and distribution risks.
- Use explicit, temporary shortcuts to keep a Rust prototype moving.
- Choose an assurance tool that matches the property and risk being checked.
- Plan one evidence-driven next step for a real project.
- Identify trustworthy routes for continued learning.
🤓 Prototyping tips
A prototype should answer one question quickly. It does not need polished error types, a deep module tree, or a generic framework. The useful distinction is between a shortcut that is easy to find later and one that silently becomes policy.
Keep momentum
- Use
todo!()for code you have not written yet. Useunreachable!()only when the prototype assumes a branch cannot occur. Both macros panic if execution reaches them. - Use
.unwrap()liberally inside a throwaway spike where a crash is an acceptable answer. Before shipping, replace every user-triggerable unwrap with deliberate error handling. - Use
.clone()to get past an initial ownership problem, then review the copies after the data flow settles. The Problem with Clones in Rust - Why Functional Rust is Slower Than You Think (And How to Fix It) demonstrates why unnecessary clones can matter. - Use
println!anddbg!to inspect a running experiment. Remove noisy output or replace it with project logging before release. - Use
assert!to record invariants as executable assumptions. For critical code, the assurance tools below can check stronger claims, but each has its own supported Rust subset and trust assumptions.
This intentionally rough function makes its deferred decisions searchable:
RUST
fn normalize_identifier(raw: Option<String>) -> String {
let raw = raw.unwrap(); // PROTOTYPE: decide how missing input should fail.
let candidate = raw.clone(); // PROTOTYPE: revisit ownership and copying.
dbg!(&candidate);
assert!(!candidate.is_empty(), "identifier must not be empty");
match candidate.strip_prefix("doi:") {
| Some(value) => value.to_string(),
| None => todo!("support identifiers without a DOI prefix"),
}
}
Keep the design plain
- Prefer simple, concrete types and let Rust infer local types. Add annotations where they explain an API or resolve ambiguity.
- Design through types, but avoid generic types and explicit lifetimes until the problem actually requires them.
- Keep the hierarchy flat at first. A
main.rsorlib.rscan sketch the eventual layout with declarations such asmod input;andmod output;. - Use Anyhow for convenient error context in an application prototype. Keep typed domain errors where callers, a library API, or the Python boundary need to distinguish failures.
- Do not optimize the first draft. Establish correct behavior, then benchmark the installed release build before removing clones or adding complexity.
Shorten the feedback loop
- Use an editor that exposes compiler feedback while you type. VS Code with rust-analyzer is a well-supported starting point.
- Use Bacon to keep
check,test, orclippyrunning in the background while files change. - Consider cargo-script eventually for small, single-file experiments. Cargo also has built-in single-file package support, but that feature remains unstable, so neither is a workshop prerequisite.
Match the assurance tool to the question
These tools are complementary, not a ladder in which every project must reach the last rung. Begin with a precise claim: “normalization is idempotent,” “this unsafe block has no undefined behavior,” or “the result always satisfies this postcondition.” Then choose the least costly tool that can answer it.
| Technique or tool | What it does | Important limit |
|---|---|---|
| Unit and integration tests | Run chosen examples through domain and boundary behavior | Cover only the executions supplied by the tests |
| Property testing | Generates examples for an invariant and shrinks failures | Searches the input space; it does not prove the property |
| Miri | Interprets executed Rust code and detects many forms of undefined behavior | Checks only the explored executions and is not a formal verifier |
| Kani | Model-checks proof harnesses with symbolic, bit-precise values | Proofs are scoped to the harness, model, supported features, and bounds |
| Verus | Uses Rust-like specifications, proof code, and SMT solving | Supports a deliberate subset of Rust and requires proof annotations |
| Creusot | Translates annotated safe Rust contracts into Why3 verification conditions | Requires contracts and code within its supported subset |
| Prusti | Checks Rust preconditions, postconditions, and invariants using Viper | It is a prototype verifier with unsupported Rust and library features |
| Aeneas | Translates a supported safe-Rust subset into pure functional code for proof assistants | The proof is completed in Lean or another backend, outside ordinary Cargo tests |
Property testing is often the easiest next step after example tests. It works especially well for parsers, round trips, normalization, and agreement between the Python and Rust implementations. Preserve generated counterexamples as ordinary regression tests.
Miri belongs earlier than formal proof when a crate contains
unsafe code or low-level memory manipulation:
A clean Miri run means the executed tests did not trigger the undefined behavior Miri detects. It does not establish that unexecuted paths are sound, and platform APIs or FFI may be unavailable under the interpreter. For a PyO3 project, run it on the testable Rust core and keep Python boundary tests as a separate layer.
Kani proof harnesses resemble property tests, but symbolic values let
the model checker reason about every value represented by the harness.
This small example checks all pairs of u8 values:
RUST
fn midpoint(left: u8, right: u8) -> u8 {
let total = u16::from(left) + u16::from(right);
(total / 2) as u8
}
#[cfg(kani)]
#[kani::proof]
fn midpoint_stays_between_inputs() {
let left = kani::any::<u8>();
let right = kani::any::<u8>();
let result = midpoint(left, right);
assert!(result >= left.min(right));
assert!(result <= left.max(right));
}
Verus, Creusot, and Prusti express contracts and invariants close to Rust. Aeneas takes another route: it uses Rust’s ownership discipline to translate a supported safe subset into pure functional definitions for proof assistants, primarily Lean. These approaches make sense for a small, high-consequence core with a stable specification. They rarely justify verifying glue code, wheel metadata, or the Python import path.
Formal verification proves the stated property under the tool’s model and assumptions. A correct proof of the wrong specification is still the wrong program. Keep example tests for concrete requirements, property tests for broad input exploration, and Python-facing tests for the cross-language contract.
Before shipping
- Review every
todo!(),unreachable!(),unwrap(), temporary clone, and debugging print. Remove it or document why it is still valid. - Benchmark the installed
acorn-pywheel against a realistic validation or schema workload. - Keep a Python-level regression suite for the public API.
- Add property tests for important invariants and round trips; consider Miri or a verifier when unsafe code, critical invariants, or the cost of failure justifies it.
- Build wheels in continuous integration for supported operating
systems and architectures; use the
abi3-py310contract across compatible CPython versions. - Decide whether a source build is supported and document the Python 3.10 and Rust 1.96 minimums.
- Keep
Cargo.lock,pixi.lock, and the immutable ACORN dependency revision under review. - Check binary size, import time, error messages, and failure behavior.
- Make ownership of both the Python and Rust code explicit within the team.
Before proposing a binding or packaging change to the reference project, run:
A sensible route forward
- Profile a production-shaped workload.
- Choose one narrow component and write characterization tests around it.
- Build a Rust prototype without changing the public Python API.
- Compare correctness, speed, memory, build complexity, and maintenance cost.
- Keep, revise, or remove the prototype based on that evidence.
Compared with Python: advantages, not guarantees
Rust raises the floor for some kinds of correctness, but the compiler cannot prove that the program meets its users’ needs.
| Tempting claim | What we can responsibly expect |
|---|---|
| “Rust does not need extensive testing to transition to production.” | Static checks replace some tests for type shape, ownership, and memory safety. Behavior, integration, packaging, and platform support still need tests. |
| “Rust prototypes already have good performance.” | A straightforward Rust implementation often starts from a useful performance baseline. Measure a release build with production-shaped input before claiming an improvement. |
| “Rust can be refactored almost risk free.” | Compiler-guided refactoring catches many broken dependencies. It cannot catch every semantic or policy change, so characterization and public-API tests still matter. |
| “Rust is easier to maintain.” | Explicit types and exhaustive matching can reduce surprises. Maintenance is easier only when the team can also debug, build, package, and release the Rust component. |
Questions to bring home
- Is conversion at the boundary dominating the work?
- Can callers tell that the implementation changed?
- What happens on platforms without a pre-built wheel?
- Can another maintainer debug and release both halves of the project?
- Would better Python, NumPy, Cython, Numba, or a service boundary solve the problem more simply?
Write your next step
Name one component you might evaluate, one metric that matters, and one reason to stop the experiment. Share it with a partner or record it in your project notes.
Workshop reference
Use the lesson reference for the concept map, commands, and primary documentation links introduced during the workshop.
- Production readiness includes packaging, observability, support, and team skills.
- Prototype shortcuts are useful when they are explicit and reviewed before release.
- Compiler checks reduce some testing burden; they do not replace behavioral and integration tests.
- Property testing explores broad input spaces; Miri and formal verification answer narrower questions with different guarantees and constraints.
- Use the integrated package for benchmarks.
- Incremental adoption should remain reversible until evidence supports it.
- The goal is a better Python project.
