Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,15 @@ assert_eq!(item.list, vec![7]);
```

#### Case 3 - Extend a struct from a crate
Deriving `Substrate` on a struct exposes the field information so that other crates can access it in a `build.rs`.
Deriving `Substrate` on a struct exposes the field information so that other crates can access it.
Deriving `Catalyst` reads the field information of a `Substrate` and generates a new complex struct.
In the other words, the catalyst is a struct with extra fields that the developer writes down in the downstream crate. The complex is a generated struct that combines the substrate's fields with the catalyst's extra fields. The overall behavior is like [chemical catalysts](https://en.wikipedia.org/wiki/Enzyme_catalysis): a catalyst **binds** onto a substrate to form a complex struct, which has all fields from both.
A complex can also **decouple** without cloning, returning the original catalyst and substrate. Check the [complex-example](./complex-example/catalyst/src/lib.rs).
With the `unsafe` feature, `bind` and `decouple` use `ManuallyDrop` + `ptr::read` to avoid memory moves, and `__substrate_new` uses `MaybeUninit` + `ptr::write` while `__substrate_unpack` uses `ManuallyDrop` + `ptr::read`, such that the copy will be less.

In terms of crate dependencies, the crate using `Substrate` is **upstream** (a dependency), and the crate using `Catalyst` is **downstream** (it depends on the substrate crate). The downstream crate calls `Substrate::expose()` in its `build.rs` to read the field information at compile time, then uses `#[catalyst(bind = ...)]` to generate the complex struct.
In terms of crate dependencies, the crate using `Substrate` is **upstream** (a dependency), and the crate using `Catalyst` is **downstream** (it depends on the substrate crate). There are two ways for the downstream crate to read the substrate's field layout:

**Option A: via `expose()` in `build.rs`**: call `Substrate::expose()` in the downstream crate's `build.rs`. This sets a `cargo:rustc-env` variable that the `Catalyst` macro reads at compile time.

```rust
/// In the substrate crate (src/lib.rs)
Expand Down Expand Up @@ -163,6 +165,30 @@ struct Amyloid {
// }
```

**Option B: via source code with `src` attribute**: point the `Catalyst` macro directly at the substrate crate's source file using `#[catalyst(src = "crate_name:/path/to/file.rs")]`. The macro uses `cargo_metadata` to locate the package and `syn` to parse the file, so no `build.rs` or `expose()` call is required. Check the [catalyst-src example](./complex-example/catalyst-src/src/lib.rs).

```rust
/// In the substrate crate (src/lib.rs)
use struct_patch::Substrate;

#[derive(Substrate)]
pub struct Base {
pub field_bool: bool,
pub field_string: String,
}

/// In the catalyst crate (src/lib.rs)
use struct_patch::Catalyst;

#[derive(Catalyst)]
#[catalyst(bind = Base, src = "substrate:/src/lib.rs")]
struct Amyloid {
pub extra_bool: bool,
pub extra_option: Option<usize>,
}
// AmyloidComplex is generated identically to Option A
```

## Attributes

You can customize the generated structs by defining `#[patch(...)]`, `#[filler(...)]`, `#[complex(...)]` (catalyst feature), or `#[catalyst(...)]` (catalyst feature) attributes on the original struct or its fields.
Expand All @@ -176,7 +202,8 @@ Two attribute namespaces are provided for the catalyst feature because we need t
- `#[patch(default_log(fn_path))]`: call `fn_path` with each patched field name on every `apply` call. Has no effect on `apply_with_log`. The function must accept `&str`.
- `#[filler(attribute(...))]`: add attributes to the generated filler struct.
- `#[filler(default_log(fn_path))]`: call `fn_path` with each filled field name on every `apply` call. Has no effect on `apply_with_log`. The function must accept `&str`.
- `#[catalyst(bind = "...")]`: specify the base (substrate) structure. (catalyst feature)
- `#[catalyst(bind = ...)]`: specify the base (substrate) structure. Need substrate expose() in build (catalyst feature)
- `#[catalyst(bind = ..., src = "crate_name:/path/to/file")]`: specify the base (substrate) structure. No need substrate expose() and based on source code. Avoide syn protocol change (catalyst feature)
- `#[catalyst(keep_field_attribute)]`: pass all field attributes from a substrate or catalyst through to the complex, unless an override is explicitly specified for that field. (catalyst feature)
- `#[catalyst(exclude_field_attributes = ["..."])]`: when `keep_field_attribute` is used, specifies attribute names to exclude from being passed through to the complex struct fields. For example, `exclude_field_attributes = ["serde"]` strips all `#[serde(...)]` field attributes from the substrate before they reach the complex. (catalyst feature)
- `#[complex(override_field_attribute("$substrate_field_name", ...))]`: override a complex field attribute, for example `serde(default = "default_str")`. (catalyst feature)
Expand Down
1 change: 1 addition & 0 deletions complex-example/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ resolver = "2"
members = [
"substrate",
"catalyst",
"catalyst-src",
]
[workspace.dependencies]
struct-patch = { path = "../lib", features = ["catalyst"] }
Expand Down
21 changes: 21 additions & 0 deletions complex-example/catalyst-src/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[package]
name = "catalyst-src"
authors.workspace = true
version.workspace = true
edition.workspace = true
categories.workspace = true
keywords.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
description.workspace = true
rust-version.workspace = true

[dependencies]
struct-patch.workspace = true
substrate = { path = "../substrate" }
serde = { version = "1", features = ["derive"] }
toml = "1.1.2"

[features]
unsafe = ["struct-patch/unsafe"]
73 changes: 73 additions & 0 deletions complex-example/catalyst-src/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
use serde::{Deserialize, Serialize};
use struct_patch::Catalyst;
use substrate::{Base, PhoneNumber};

#[derive(Default, Catalyst)]
#[catalyst(bind = Base, src = "substrate:/src/lib.rs")]
#[catalyst(keep_field_attribute)]
#[complex(attribute(derive(Debug, Deserialize, Serialize)))]
#[complex(override_field_attribute("filed_numbers", serde(default)))]
#[allow(dead_code)]
struct Amyloid {
pub extra_bool: bool,
#[complex(attribute(serde(default = "default_str")))]
pub extra_string: String,
pub extra_option: Option<usize>,
#[complex(attribute(serde(default = "default_extra_private_number")))]
extra_private_number: u8,
}

fn default_str() -> String {
"default".to_string()
}

fn default_extra_private_number() -> u8 {
7
}

#[allow(dead_code)]
impl AmyloidComplex {
fn private_number_sum(&self) -> u8 {
self.private_number + self.extra_private_number
}
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn complex_from_binding_src_works() {
let substrate = Base::__substrate_new(true, String::new(), None, 1u8, PhoneNumber::default());
let amyloid = Amyloid::default();
let complex = amyloid.bind(substrate);
assert_eq!(complex.field_bool, true);
assert_eq!(complex.private_number_sum(), 1);

let toml_str = toml::to_string_pretty(&complex).unwrap();
assert_eq!(
toml_str,
r#"field_bool = true
field_string = ""
private_number = 1
extra_bool = false
extra_string = ""
extra_private_number = 0

[filed_numbers]
country_code = 0
local_numbers = ""
"#
);

let toml_str = r#"field_bool = true
field_string = ""
private_number = 1
extra_bool = true
"#;
let complex: AmyloidComplex = toml::from_str(toml_str).unwrap();
assert_eq!(complex.extra_string, "default");
// extra_private_number defaults to 7 via default_extra_private_number
assert_eq!(complex.private_number_sum(), 8);
}
}
2 changes: 1 addition & 1 deletion complex-example/catalyst/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ rust-version.workspace = true
struct-patch.workspace = true
substrate = { path = "../substrate" }
serde = { version = "1", features = ["derive"] }
toml = "1.1.2+spec-1.1.0"
toml = "1.1.2"

[features]
unsafe = ["struct-patch/unsafe"]
Expand Down
3 changes: 2 additions & 1 deletion derive/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,14 @@ proc-macro2 = "1.0"
quote = "1.0"
syn = { version = "2.0", features = ["parsing"] }
syn-serde = { version = "0.3.2", features = ["json"], optional = true }
cargo_metadata = { version = "0.18", optional = true }

[features]
status = []
op = []
merge = []
nesting = []
catalyst = [ "syn-serde" ]
catalyst = [ "syn-serde", "dep:cargo_metadata", "syn/full" ]
unsafe = []
box = []

Expand Down
66 changes: 63 additions & 3 deletions derive/src/catalyst.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ pub(crate) struct Catalyst {
attributes: Vec<TokenStream>,
fields: syn::Fields,
bind: String,
src: Option<String>,
keep_field_attribute: bool,
override_field_attributes: HashMap<String, Vec<TokenStream>>, // TODO handle no-std
exclude_field_attributes: Vec<String>,
Expand All @@ -30,6 +31,7 @@ struct Field {
const CATALYST: &str = "catalyst";
const COMPLEX: &str = "complex";
const BIND: &str = "bind";
const SRC: &str = "src";
const NAME: &str = "name";
const ATTRIBUTE: &str = "attribute";
const OVERRIDE: &str = "override_field_attribute";
Expand All @@ -47,6 +49,7 @@ impl Catalyst {
attributes,
fields,
bind,
src,
keep_field_attribute,
override_field_attributes,
exclude_field_attributes,
Expand All @@ -62,9 +65,13 @@ impl Catalyst {
let mut substrate_fields: Vec<Field> = Vec::new();
let mut catalyst_fields: Vec<Field> = Vec::new();

let substrate_str = std::env::var(bind)
.expect("field information of substrate is absent, please expose it in build.rs");
let raw_substrate_fields: syn::Fields = syn_serde::json::from_str(&substrate_str).unwrap();
let raw_substrate_fields: syn::Fields = if let Some(src) = src {
find_fields_from_src(src, bind)
} else {
let substrate_str = std::env::var(bind)
.expect("field information of substrate is absent, please expose it in build.rs");
syn_serde::json::from_str(&substrate_str).unwrap()
};

for field in raw_substrate_fields.into_iter() {
raw_complex_fields.push(Field::from_ast(field.clone()));
Expand Down Expand Up @@ -249,6 +256,7 @@ impl Catalyst {
let mut name = None;
let mut attributes = vec![];
let mut bind = String::new();
let mut src: Option<String> = None;
let mut keep_field_attribute = false;
let mut override_field_attributes = HashMap::<String, Vec<TokenStream>>::new();
let mut exclude_field_attributes = Vec::<String>::new();
Expand Down Expand Up @@ -306,6 +314,12 @@ impl Catalyst {
}
}
}
SRC if attr_str == CATALYST => {
// #[catalyst(src = "crate_name:/path/to/file.rs")]
if let Some(lit) = crate::get_lit_str(path, &meta)? {
src = Some(lit.value());
}
}
KEEP_FIELD_ATTRIBUTE if attr_str == CATALYST => {
// #[catalyst(keep_field_attribute)]
keep_field_attribute = true;
Expand Down Expand Up @@ -354,13 +368,59 @@ impl Catalyst {
attributes,
fields,
bind,
src,
keep_field_attribute,
override_field_attributes,
exclude_field_attributes,
})
}
}

fn find_fields_from_src(src: &str, struct_name: &str) -> syn::Fields {
let (package_name, rel_path) = src
.split_once(':')
.expect("src attribute must be 'crate_name:/path/to/file.rs'");

let manifest_dir = std::env::var("CARGO_MANIFEST_DIR")
.expect("CARGO_MANIFEST_DIR is not set");

let meta = cargo_metadata::MetadataCommand::new()
.manifest_path(std::path::Path::new(&manifest_dir).join("Cargo.toml"))
.exec()
.expect("cargo metadata failed");

let package = meta
.packages
.iter()
.find(|p| p.name == package_name)
.unwrap_or_else(|| panic!("package `{}` not found in dependency graph", package_name));

let pkg_dir = package
.manifest_path
.parent()
.expect("manifest_path has no parent directory");

let src_path = std::path::Path::new(pkg_dir.as_str())
.join(rel_path.trim_start_matches('/'));

let content = std::fs::read_to_string(&src_path)
.unwrap_or_else(|e| panic!("cannot read `{}`: {}", src_path.display(), e));

let ast: syn::File = syn::parse_str(&content).expect("failed to parse source file");

ast.items
.iter()
.find_map(|item| {
if let syn::Item::Struct(s) = item {
if s.ident == struct_name {
return Some(s.fields.clone());
}
}
None
})
.unwrap_or_else(|| panic!("struct `{}` not found in source file", struct_name))
}

impl Field {
/// Generate the token stream for the Complex struct fields
pub fn to_token_stream(
Expand Down
8 changes: 5 additions & 3 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,13 @@
set -ex
cd $(git rev-parse --show-toplevel 2>/dev/null)
cd complex-example
cargo test -p substrate
cargo test -p catalyst
cargo test --quiet -p substrate
cargo test --quiet -p catalyst
cargo test --quiet -p catalyst-src

echo "Run catatyst test with unsafe features"
cargo test -p catalyst --features unsafe
cargo test --quiet -p catalyst --features unsafe
cargo test --quiet -p catalyst-src --features unsafe
'';
in
with pkgs;
Expand Down
2 changes: 1 addition & 1 deletion lib/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ struct-patch-derive = { version = "=0.13.2", path = "../derive" }
serde_json = "1.0"
serde = { version = "1", features = ["derive"] }
serde_with = "3.9.0"
toml = "1.1"
toml = "1.1.2"
humantime-serde = "1.1.1"
clap = { version = "4.4.7", features = ["derive"] }

Expand Down