diff --git a/.agents/skills/polyxml-codegen-workflow/SKILL.md b/.agents/skills/polyxml-codegen-workflow/SKILL.md index a522d15..dbc41d6 100644 --- a/.agents/skills/polyxml-codegen-workflow/SKILL.md +++ b/.agents/skills/polyxml-codegen-workflow/SKILL.md @@ -8,6 +8,42 @@ description: >- # PolyXML Polyglot Codegen Development & Verification Playbook +## XSD attribute values + +Use separate `CARGO_TARGET_DIR` paths for revision-specific CLI/native-binding +verification. A shared debug target can leave another checkout's same-named +top-level binary or cdylib in place even when Cargo reports a fresh build. Probe +a discriminating fixture before claiming a baseline comparison, record the +executable/binding hashes, and retain rejected preflights separately. The runtime +comparison runners already isolate baseline and candidate consumer targets. + +Normalize schema attributes with quick-xml's `normalized_value` before placing +them in the IR. This resolves built-in/numeric references once and normalizes +literal XML attribute whitespace while preserving referenced whitespace. Use the +same decoded values for namespace declarations, QName validation, UPA, facets, +enumerations and default/fixed constraints. Propagate malformed attributes and +reference errors; do not turn them into absent values. Validate attributes even +inside otherwise skipped annotations. Regression fixtures are in +`research/fixtures/schema_attribute_entities.xsd` and +`tests/test_schema_attribute_entities.rs` in the core crate. + +Decoded enum values need target-language string literal escaping at every output +site: metadata attributes, constants, conversion tables and parse matches. Rust +debug string formatting is suitable for Rust, but its `\u{...}` escapes are not +portable to other targets. The shared JSON-compatible literal helper also +escapes U+0085/U+2028/U+2029 because C# treats them as source line terminators; +C++ uses fixed-width octal escapes for ASCII controls. Execute +`test_enum_literal_codegen` with all seven toolchains to verify exact runtime +values and Rust Serde round trips, not just whether generated files exist. + +Schema defaults and fixed values need the same literal escaping as enums. +Python metadata must not use Rust Debug escapes, and C++ defaults need C++ +control-character escapes. C# defaults parsed through an internal XML element +must encode carriage returns as ` ` after escaping XML markup; otherwise XML +line-ending normalization changes the lexical value. Execute generated consumers +against independent UTF-8 expected data, including backslash-u text, referenced +whitespace, Unicode line separators, empty-element defaults and fixed rejection. + ## C# runtime availability for smoke checks The C# execution fixtures target `net8.0`. An installed .NET 10 SDK/runtime @@ -934,3 +970,11 @@ owned/borrowed XML and JSON round trips, use the declared root name explicitly when it differs from its type name, and distinguish C++/TypeScript model-only generation from their runtime binding schemas. Strict docs builds catch broken anchors and misplaced code fences, including guides appended after old fences. + +When Python generated-model tests use an isolated absolute CARGO_TARGET_DIR, +resolve the CLI under that target, not the checkout's default target directory +or an unrelated global executable. Relative target values resolve from the +checkout because the helper builds with that cwd. The generated-model test +helper now honors this override and builds the matching CLI if needed. Keep +workspace targets owned by one worktree; switching checkouts in a shared target +can leave stale unchanged executables even after a core compile message. diff --git a/.agents/skills/polyxml-core-engine/SKILL.md b/.agents/skills/polyxml-core-engine/SKILL.md index 86dd4b3..e291ef3 100644 --- a/.agents/skills/polyxml-core-engine/SKILL.md +++ b/.agents/skills/polyxml-core-engine/SKILL.md @@ -272,5 +272,20 @@ lexical-list obligations before traversing repeated values. Ordinary nested records validate their own fields. Lexical lists append tokens to one string; retain whitespace-token rejection, escaping and empty-list behavior. +An explicit null in the ordered mixed item stream is a present nil element, +including for nested branches. Serialize it with a local instance namespace +binding; avoid shadowing the element QName's prefix. Keep empty strings distinct +from nulls and preserve item order. Mixed branch schemas do not retain nillable +constraints, and the dynamic reader remains permissive; this round-trip behavior +does not establish nillability validation or ordinary optional-field nil output. + For measured changes, use [polyxml-runtime-investigation](../polyxml-runtime-investigation/SKILL.md). + +Namespace declarations are XML attribute values too: unescape predefined and +numeric references exactly once before storing their URI in NamespaceScope. +Propagate failures through document entry, nested Start/Empty handling and +XmlItemStream inheritance. Compare decoded URIs for strict-root and xsi:type +matching; escaped source text is not the namespace name. Preserve inherited +already-decoded bindings and test a literal amp-entity spelling against double +decoding. diff --git a/.agents/skills/polyxml-runtime-investigation/SKILL.md b/.agents/skills/polyxml-runtime-investigation/SKILL.md index b43445f..64b086c 100644 --- a/.agents/skills/polyxml-runtime-investigation/SKILL.md +++ b/.agents/skills/polyxml-runtime-investigation/SKILL.md @@ -30,6 +30,11 @@ separate from the schema-driven `PolyValue` runtime. also matches deserialization. Remove stale results only in the runner's owned scratch directory before measurement, not from retained raw evidence. - Check rich enums, patterns, lists and mixed branches alongside plain records. + The `mixed.rs` consumer checks text/scalar/enum/pattern/nested ordered items. + Text/GeneralRef/CData boundaries may coalesce when written: compare adjacent + text's concatenated value, rather than treating token segmentation as XML + semantics. Keep nil correctness tests separate from baseline timing consumers + when the baseline cannot serialize nil. Warm pattern-cache gains do not establish cold-start, churn or concurrent throughput. Entry/key bounds do not constitute a compiled-regex byte budget. - Preserve validation, errors, split Text/CData/GeneralRef handling, nil reads, @@ -42,3 +47,40 @@ separate from the schema-driven `PolyValue` runtime. - Publish positive and negative experiments with exact source revisions, raw evidence and remaining regressions. Run the full quality gate before pushing the experiment branch; a branch request does not authorize a merge to main. + +For mixed schemas with many possible child tags, measure the linear branch scan +with `mixed_branches.rs`, including sparse one-item controls. A temporary index +can borrow kind keys and branch references for large repeated payloads without +persisting stale mutable metadata. Preserve the original first-match behavior +for duplicate kind names; test metadata edits, tagged records, nil and unknown +kinds. Keep small tables on a linear path and measure the chosen crossover. + +Check the indexing cutoff with short documents too. A 256-branch table built for +64 mixed items was 30% slower, despite a 54% gain at 1,000 items. Requiring at +least 64 items and half as many items as branches avoids that observed crossover +regression. Retain 32/64/128-item controls, and keep the cutoff heuristic separate +from semantic behavior: duplicate first-match ordering and metadata edits still +need independent correctness tests. + +Also measure concentrated tag reuse and text-heavy content. Even above the +size cutoff, repeated first-branch hits were 11–20% slower with a hash table. +A bounded, evenly spaced sample can select linear lookup when the observed +branch-search work is low. Treat sampling as a performance heuristic only; +normal writer validation must still examine every item and preserve errors. + +Offset samples within their bins and include a power-of-two item count: a +strictly evenly spaced sample can repeatedly hit the first tag when the choice +cycle divides its stride, overlooking a distributed payload's lookup work. + +For large investigations, retain Criterion process directories in per-experiment +archives rather than thousands of loose JSON files in a PR. Verify the original +manifest, archive every file with its relative path, read back and compare each +member's checksum before removing loose copies, and retain member/top-level +manifests plus the packaging script. Extract into scratch to rerun summaries. + +For an integration branch containing merge commits, use `git pull --rebase=merges +origin main` before pushing so synchronization preserves the reviewed branch +history. Ordinary rebase flattens merges even when main has not advanced. If the +repository has no committer identity configured, pass the maintainer identity via +per-command `git -c user.name=... -c user.email=...`; do not change global config. +Stop a publish sequence if pull fails, and check the tested source tree afterward. diff --git a/benchmarks/rust-runtime-investigation/README.md b/benchmarks/rust-runtime-investigation/README.md index 8c2e4f4..136e647 100644 --- a/benchmarks/rust-runtime-investigation/README.md +++ b/benchmarks/rust-runtime-investigation/README.md @@ -113,3 +113,35 @@ For the generated sensor control, run the XML/Serde regression runner and use its output directory. This computes each process's median of seven timing samples, then the median across processes; it retains the process deltas. The statistic differs from the Criterion process means above. + +## Ordered mixed content + +Use `--harness benchmarks/rust-runtime-investigation/mixed.rs` for 1,000 +text, scalar, enum, warm-pattern or nested mixed items. Every decoded payload +is checked before timing. Adjacent text events can be coalesced on output, so +the text case compares the concatenated text value after a round trip; element +cases compare the complete ordered value. These cases contain no nil items and +can compare revisions whose nil writer is broken. Cover nil output separately +with `test_mixed_nil`; a timing harness must not turn a known correctness +failure into a performance result. + +`mixed_branches.rs` varies the child-tag table across 1, 16, 64 and 256 entries +and the payload across 1, 32, 64, 128, 1,000 and 1,024 items. It checks every kind against its +expected wire name and every integer before timing, then checks the full +round trip. Use `^branch_write/.*/1000$` to screen repeated writes; retain sparse +and read controls before promoting a branch-lookup optimization. A temporary +index trades a per-container allocation for faster repeated lookup. It must +preserve first-match behavior and rebuild after mutable metadata edits. + +The mixed branch consumer also includes 32, 64 and 128 items to check index +construction cost. The trigger requires at least 64 branches, at least 64 items, +and items numbering at least half the branch count. This is a measured heuristic, +not a universal break-even guarantee; tag distribution and text-only content can +affect how much lookup work is saved. + +The final selector samples at most 16 items spread across the payload, requiring at least +four tagged samples and an average linear-search depth of 16 before indexing. +This avoids the observed repeated-first-tag regression and skips a table for +text-only content. It is a heuristic, so retain concentrated/distributed cases +and the actual sampled payload along with timing evidence. All items still pass +through the normal writer validation. diff --git a/benchmarks/rust-runtime-investigation/mixed.rs b/benchmarks/rust-runtime-investigation/mixed.rs new file mode 100644 index 0000000..7d0413e --- /dev/null +++ b/benchmarks/rust-runtime-investigation/mixed.rs @@ -0,0 +1,120 @@ +use criterion::{ + criterion_group, criterion_main, BenchmarkId, Criterion, SamplingMode, Throughput, +}; +use polyxml::schema::ModelSchema; +use polyxml::schema_parser::XsdParser; +use polyxml::{deserialize, serialize, PolyValue}; +use std::{hint::black_box, sync::Arc}; + +fn fixture(kind: &str, count: usize) -> (Arc, Vec) { + let xsd = r#" + + + + + + + + + + "#; + let ir = XsdParser::new().parse_str(xsd).unwrap(); + let schema = ModelSchema::from_ir(&ir, Some("Root")).unwrap(); + let mut xml = String::from(""); + for index in 0..count { + let part = match kind { + "text" => format!("text-{index}&"), + "scalar" => format!("item-{index}&"), + "enum" => "Ready".into(), + "pattern" => format!("ABC{:03}", index % 1000), + "nested" => format!("item-{index}"), + _ => unreachable!(), + }; + xml.push_str(&part); + } + xml.push_str(""); + let value = deserialize(xml.as_bytes(), Arc::clone(&schema)).unwrap(); + let items_name = &schema.fields[schema.mixed_content.as_ref().unwrap().items_index].name; + let items = value.get(items_name).and_then(PolyValue::as_list).unwrap(); + if kind == "text" { + // Text, GeneralRef and CData become separate ordered items. + assert_eq!(items.len(), count * 3); + for (index, items) in items.chunks_exact(3).enumerate() { + for (item, expected) in + items + .iter() + .zip([format!("text-{index}"), "&".into(), "more".into()]) + { + assert_eq!(item.get("kind").and_then(PolyValue::as_str), Some("#text")); + assert_eq!( + item.get("value").and_then(PolyValue::as_str), + Some(expected.as_str()) + ); + } + } + } else { + assert_eq!(items.len(), count); + for (index, item) in items.iter().enumerate() { + let value = item.get("value").unwrap(); + match kind { + "scalar" => assert_eq!(value.as_str(), Some(format!("item-{index}&more").as_str())), + "enum" => assert_eq!(value.as_str(), Some("Ready")), + "pattern" => assert_eq!( + value.as_str(), + Some(format!("ABC{:03}", index % 1000).as_str()) + ), + "nested" => assert_eq!( + value.get("value").and_then(PolyValue::as_str), + Some(format!("item-{index}").as_str()) + ), + _ => unreachable!(), + } + } + } + let output = serialize("Root", &value, &schema, None).unwrap(); + let reparsed = deserialize(&output, Arc::clone(&schema)).unwrap(); + if kind == "text" { + // XML writers coalesce adjacent text segments. Compare their text value + // rather than treating the reader's segmentation as XML semantics. + let text = |value: &PolyValue| -> String { + value + .get(items_name) + .unwrap() + .as_list() + .unwrap() + .iter() + .map(|item| item.get("value").unwrap().as_str().unwrap()) + .collect() + }; + assert_eq!(text(&value), text(&reparsed)); + } else { + assert_eq!(value, reparsed); + } + (schema, xml.into_bytes()) +} + +fn benchmarks(c: &mut Criterion) { + for operation in ["mixed_read", "mixed_write"] { + let mut group = c.benchmark_group(operation); + group.sampling_mode(SamplingMode::Flat); + for kind in ["text", "scalar", "enum", "pattern", "nested"] { + let (schema, xml) = fixture(kind, 1000); + let value = deserialize(&xml, Arc::clone(&schema)).unwrap(); + group.throughput(Throughput::Bytes(xml.len() as u64)); + group.bench_function(BenchmarkId::new(kind, 1000), |b| { + if operation == "mixed_read" { + b.iter(|| { + black_box(deserialize(black_box(&xml), Arc::clone(&schema)).unwrap()) + }); + } else { + b.iter(|| { + black_box(serialize("Root", black_box(&value), &schema, None).unwrap()) + }); + } + }); + } + group.finish(); + } +} +criterion_group!(benches, benchmarks); +criterion_main!(benches); diff --git a/benchmarks/rust-runtime-investigation/mixed_branches.rs b/benchmarks/rust-runtime-investigation/mixed_branches.rs new file mode 100644 index 0000000..e635bfe --- /dev/null +++ b/benchmarks/rust-runtime-investigation/mixed_branches.rs @@ -0,0 +1,74 @@ +use criterion::{ + criterion_group, criterion_main, BenchmarkId, Criterion, SamplingMode, Throughput, +}; +use polyxml::schema::ModelSchema; +use polyxml::schema_parser::XsdParser; +use polyxml::{deserialize, serialize, PolyValue}; +use std::{hint::black_box, sync::Arc}; + +fn fixture(branches: usize, count: usize) -> (Arc, Vec) { + let declarations = (0..branches) + .map(|i| format!(r#""#)) + .collect::(); + let xsd = format!( + r#"{declarations}"# + ); + let ir = XsdParser::new().parse_str(&xsd).unwrap(); + let schema = ModelSchema::from_ir(&ir, Some("Root")).unwrap(); + let body = (0..count) + .map(|i| format!("{i}", i % branches, i % branches)) + .collect::(); + let xml = format!("{body}").into_bytes(); + let value = deserialize(&xml, Arc::clone(&schema)).unwrap(); + let mixed = schema.mixed_content.as_ref().unwrap(); + let field = &schema.fields[mixed.items_index].name; + let items = value.get(field).and_then(PolyValue::as_list).unwrap(); + assert_eq!(items.len(), count); + for (i, item) in items.iter().enumerate() { + assert_eq!(item.get("value"), Some(&PolyValue::Int(i as i64))); + let kind = item.get("kind").and_then(PolyValue::as_str).unwrap(); + let branch = mixed + .branches + .iter() + .find(|b| b.variant_name == kind) + .unwrap(); + assert_eq!(branch.xml_name, format!("B{}", i % branches).as_bytes()); + } + let output = serialize("Root", &value, &schema, None).unwrap(); + assert_eq!(value, deserialize(&output, Arc::clone(&schema)).unwrap()); + (schema, xml) +} + +fn benchmarks(c: &mut Criterion) { + for operation in ["branch_read", "branch_write"] { + let mut group = c.benchmark_group(operation); + group.sampling_mode(SamplingMode::Flat); + for branches in [1, 16, 64, 256] { + for count in [1, 32, 64, 128, 1000, 1024] { + let (schema, xml) = fixture(branches, count); + let value = deserialize(&xml, Arc::clone(&schema)).unwrap(); + group.throughput(Throughput::Bytes(xml.len() as u64)); + group.bench_with_input( + BenchmarkId::new(format!("branches_{branches}"), count), + &count, + |b, _| { + b.iter(|| { + if operation == "branch_read" { + black_box( + deserialize(black_box(&xml), Arc::clone(&schema)).unwrap(), + ); + } else { + black_box( + serialize("Root", black_box(&value), &schema, None).unwrap(), + ); + } + }); + }, + ); + } + } + group.finish(); + } +} +criterion_group!(benches, benchmarks); +criterion_main!(benches); diff --git a/crates/polyxml-core/src/codegen/cpp/mod.rs b/crates/polyxml-core/src/codegen/cpp/mod.rs index 7ed29cc..da235e8 100644 --- a/crates/polyxml-core/src/codegen/cpp/mod.rs +++ b/crates/polyxml-core/src/codegen/cpp/mod.rs @@ -553,7 +553,14 @@ endif() } else { "" }; - writeln!(out, " {:?}, T::{}{}", v.value, var_name, comma).unwrap(); + writeln!( + out, + " {}, T::{}{}", + super::cpp_string_literal(&v.value), + var_name, + comma + ) + .unwrap(); } writeln!(out, " );").unwrap(); } @@ -790,8 +797,10 @@ inline int polyxml_integer_compare(std::string_view a,std::string_view b) noexce for (variant, var_name) in enum_def.variants.iter().zip(&variant_names) { writeln!( out, - " case {}::{}: return \"{}\";", - enum_name, var_name, variant.value + " case {}::{}: return {};", + enum_name, + var_name, + super::cpp_string_literal(&variant.value) ) .unwrap(); } @@ -818,8 +827,10 @@ inline int polyxml_integer_compare(std::string_view a,std::string_view b) noexce for (variant, var_name) in enum_def.variants.iter().zip(&variant_names) { writeln!( out, - " if (s == \"{}\") return {}::{};", - variant.value, enum_name, var_name + " if (s == {}) return {}::{};", + super::cpp_string_literal(&variant.value), + enum_name, + var_name ) .unwrap(); } @@ -998,7 +1009,9 @@ inline int polyxml_integer_compare(std::string_view a,std::string_view b) noexce // Value field let init = match &f.default_value { Some(v) => match f.type_ref { - TypeRef::Primitive(p) if p.is_unbounded_integer() => format!(" = {v:?}"), + TypeRef::Primitive(p) if p.is_unbounded_integer() => { + format!(" = {}", super::cpp_string_literal(v)) + } TypeRef::Primitive(PrimitiveType::Boolean) => { format!(" = {}", v.to_lowercase()) } @@ -1008,7 +1021,7 @@ inline int polyxml_integer_compare(std::string_view a,std::string_view b) noexce TypeRef::Primitive(PrimitiveType::String) | TypeRef::Primitive(PrimitiveType::Token) | TypeRef::Primitive(PrimitiveType::NormalizedString) => { - format!(" = \"{}\"", v.replace('\\', "\\\\").replace('"', "\\\"")) + format!(" = {}", super::cpp_string_literal(v)) } TypeRef::Primitive(_) => format!(" = {}", v), TypeRef::Named(_) => format!(" = {}", v), diff --git a/crates/polyxml-core/src/codegen/csharp/mod.rs b/crates/polyxml-core/src/codegen/csharp/mod.rs index f15ae04..05153f4 100644 --- a/crates/polyxml-core/src/codegen/csharp/mod.rs +++ b/crates/polyxml-core/src/codegen/csharp/mod.rs @@ -618,7 +618,13 @@ impl CSharpCodegen { self.emit_docstring(out, doc, &format!("{} ", indent)); } if self.options.emit_xml_attributes { - writeln!(out, "{} [XmlEnum(\"{}\")]", indent, variant.value).unwrap(); + writeln!( + out, + "{} [XmlEnum({})]", + indent, + super::string_literal(&variant.value) + ) + .unwrap(); } writeln!(out, "{} {},", indent, variant_name).unwrap(); } @@ -659,8 +665,11 @@ impl CSharpCodegen { for (variant, variant_name) in e.variants.iter().zip(&variant_names) { writeln!( out, - "{} {}.{} => \"{}\",", - indent, enum_name, variant_name, variant.value + "{} {}.{} => {},", + indent, + enum_name, + variant_name, + super::string_literal(&variant.value) ) .unwrap(); } @@ -1295,10 +1304,10 @@ impl CSharpCodegen { if let Some(kind) = integer_kind { return format!("PolyxmlIntegerLexical.Validate({text},{kind})"); } - format!("({base})new XmlSerializer(typeof({base}), new XmlRootAttribute(\"value\")).Deserialize(new System.IO.StringReader(\"\" + System.Security.SecurityElement.Escape({text}) + \"\"))!") + format!("({base})new XmlSerializer(typeof({base}), new XmlRootAttribute(\"value\")).Deserialize(new System.IO.StringReader(\"\" + System.Security.SecurityElement.Escape({text}).Replace(\"\\r\", \" \") + \"\"))!") }; if let Some(fixed) = &field.fixed_value { - let expected = parse(&format!("{fixed:?}")); + let expected = parse(&super::string_literal(fixed)); let initial = if field.cardinality.is_optional() && field.kind == FieldKind::Element { "default!".to_string() } else { @@ -1318,7 +1327,7 @@ impl CSharpCodegen { {indent} [XmlIgnore] public {ty} {name} {{ get => _{name}; set {{ if ({present}!({equal})) throw new System.ComponentModel.DataAnnotations.ValidationException(\"Fixed value constraint violated for {name}\"); _{name} = value; }} }}").unwrap(); } else { let initial = if field.kind == FieldKind::Attribute { - parse(&format!("{default:?}")) + parse(&super::string_literal(default)) } else { "default!".to_string() }; @@ -1352,7 +1361,10 @@ impl CSharpCodegen { let text = if field.kind == FieldKind::Attribute { "value".into() } else { - format!("value.Length == 0 ? {default:?} : value") + format!( + "value.Length == 0 ? {} : value", + super::string_literal(default) + ) }; let parsed = parse("text"); writeln!(out, "{indent} set {{ if (value is null) {{ {name} = default!; return; }} var text = {text}; {name} = {parsed}; }} diff --git a/crates/polyxml-core/src/codegen/go/mod.rs b/crates/polyxml-core/src/codegen/go/mod.rs index 321d4b4..825d3b0 100644 --- a/crates/polyxml-core/src/codegen/go/mod.rs +++ b/crates/polyxml-core/src/codegen/go/mod.rs @@ -577,7 +577,7 @@ func (v {type_name}) MarshalText() ([]byte,error) {{ tokens:=make([]string,0,len let variants = enumeration .variants .iter() - .map(|v| format!("{:?}", v.value)) + .map(|v| super::string_literal(&v.value)) .collect::>() .join(","); writeln!(out,"switch string(value) {{case {variants}: default:return fmt.Errorf(\"invalid enum list item\")}}").unwrap(); @@ -670,8 +670,10 @@ func (v {type_name}) MarshalText() ([]byte,error) {{ tokens:=make([]string,0,len } writeln!( out, - " {} {} = \"{}\"", - const_name, enum_name, variant.value + " {} {} = {}", + const_name, + enum_name, + super::string_literal(&variant.value) ) .unwrap(); } @@ -925,7 +927,7 @@ func (v {type_name}) MarshalText() ([]byte,error) {{ tokens:=make([]string,0,len let comparisons = def .variants .iter() - .map(|v| format!("value == {:?}", v.value)) + .map(|v| format!("value == {}", super::string_literal(&v.value))) .collect::>() .join(" || "); let _ = writeln!( diff --git a/crates/polyxml-core/src/codegen/java/mod.rs b/crates/polyxml-core/src/codegen/java/mod.rs index 5ab5ee3..aa22279 100644 --- a/crates/polyxml-core/src/codegen/java/mod.rs +++ b/crates/polyxml-core/src/codegen/java/mod.rs @@ -597,8 +597,11 @@ impl JavaCodegen { let semi_or_comma = if i + 1 == e.variants.len() { ";" } else { "," }; let _ = writeln!( out, - "{} {}({:?}){}", - indent, const_name, v.value, semi_or_comma + "{} {}({}){}", + indent, + const_name, + super::string_literal(&v.value), + semi_or_comma ); } diff --git a/crates/polyxml-core/src/codegen/mod.rs b/crates/polyxml-core/src/codegen/mod.rs index 9e35479..6ae0135 100644 --- a/crates/polyxml-core/src/codegen/mod.rs +++ b/crates/polyxml-core/src/codegen/mod.rs @@ -24,6 +24,40 @@ pub use python::{ pub use rust::{RustCodegen, RustOptions}; pub use typescript::{TypeScriptBackend, TypeScriptCodegen, TypeScriptOptions}; +/// JSON string escapes are also valid in Python, Go, Java, C# and TypeScript +/// string literals. Rust and C++ need their own escapes for control characters. +pub(crate) fn string_literal(value: &str) -> String { + serde_json::to_string(value) + .expect("serializing a string cannot fail") + // C# treats these Unicode characters as source line terminators even + // inside quoted strings; JSON permits their literal forms. + .replace('\u{85}', "\\u0085") + .replace('\u{2028}', "\\u2028") + .replace('\u{2029}', "\\u2029") +} + +pub(crate) fn cpp_string_literal(value: &str) -> String { + use std::fmt::Write; + let mut literal = String::from("\""); + for character in value.chars() { + match character { + '\\' => literal.push_str("\\\\"), + '"' => literal.push_str("\\\""), + '\n' => literal.push_str("\\n"), + '\r' => literal.push_str("\\r"), + '\t' => literal.push_str("\\t"), + c if c.is_ascii_control() => { + // Three octal digits prevent a following digit from extending + // the escape; universal character names cannot encode these. + write!(literal, "\\{:03o}", c as u32).unwrap(); + } + c => literal.push(c), + } + } + literal.push('"'); + literal +} + /// Prefix every physical XSD documentation line so embedded newlines cannot /// escape a generated source comment. pub(crate) fn write_documentation_lines(out: &mut String, prefix: &str, doc: &str) { diff --git a/crates/polyxml-core/src/codegen/python/mod.rs b/crates/polyxml-core/src/codegen/python/mod.rs index 4c182fb..11b6d27 100644 --- a/crates/polyxml-core/src/codegen/python/mod.rs +++ b/crates/polyxml-core/src/codegen/python/mod.rs @@ -521,9 +521,9 @@ impl PythonCodegen { let _ = writeln!( out, - " {} = \"{}\"", + " {} = {}", var_id, - variant.value.replace('"', "\\\"") + super::string_literal(&variant.value) ); } } @@ -1012,10 +1012,10 @@ impl PythonCodegen { } if let Some(ref fixed) = field.fixed_value { - parts.push(format!("\"fixed\": {fixed:?}")); + parts.push(format!("\"fixed\": {}", super::string_literal(fixed))); } if let Some(ref default) = field.default_value { - parts.push(format!("\"default\": {default:?}")); + parts.push(format!("\"default\": {}", super::string_literal(default))); } if matches!(super::primitive_base(&field.type_ref, ir), TypeRef::List(_)) { parts.push("\"tokens\": True".into()); @@ -1045,7 +1045,7 @@ impl PythonCodegen { fn format_default_value(&self, val: &str, type_ref: &TypeRef, ir: &SchemaIR) -> String { if matches!(super::primitive_base(type_ref, ir), TypeRef::Primitive(p) if p.is_unbounded_integer()) { - return format!("int({val:?})"); + return format!("int({})", super::string_literal(val)); } let base = super::primitive_base(type_ref, ir); @@ -1073,20 +1073,20 @@ impl PythonCodegen { if val.trim().parse::().is_ok() { val.trim().to_string() } else { - format!("\"{}\"", val.replace('"', "\\\"")) + super::string_literal(val) } } TypeRef::Primitive(PrimitiveType::Float | PrimitiveType::Double) => { if val.trim().parse::().is_ok() { val.trim().to_string() } else { - format!("\"{}\"", val.replace('"', "\\\"")) + super::string_literal(val) } } TypeRef::Primitive(PrimitiveType::Decimal) => { - format!("Decimal(\"{}\")", val.replace('"', "\\\"")) + format!("Decimal({})", super::string_literal(val)) } - _ => format!("\"{}\"", val.replace('"', "\\\"")), + _ => super::string_literal(val), } } diff --git a/crates/polyxml-core/src/codegen/rust/mod.rs b/crates/polyxml-core/src/codegen/rust/mod.rs index 6a13123..f4d1722 100644 --- a/crates/polyxml-core/src/codegen/rust/mod.rs +++ b/crates/polyxml-core/src/codegen/rust/mod.rs @@ -724,7 +724,7 @@ impl RustCodegen { write_documentation_lines(out, " /// ", doc); } if self.options.derive_serde { - let _ = writeln!(out, " #[serde(rename = \"{}\")]", variant.value); + let _ = writeln!(out, " #[serde(rename = {:?})]", variant.value); } if self.options.derive_default && idx == 0 { let _ = writeln!(out, " #[default]"); @@ -740,7 +740,7 @@ impl RustCodegen { out.push_str(" pub fn as_str(&self) -> &'static str {\n"); out.push_str(" match self {\n"); for (var_id, val) in &variant_map { - let _ = writeln!(out, " Self::{} => \"{}\",", var_id, val); + let _ = writeln!(out, " Self::{} => {:?},", var_id, val); } if variant_map.is_empty() { out.push_str(" _ => \"\",\n"); @@ -755,7 +755,7 @@ impl RustCodegen { out.push_str(" fn from_str(s: &str) -> std::result::Result {\n"); out.push_str(" match s {\n"); for (var_id, val) in &variant_map { - let _ = writeln!(out, " \"{}\" => Ok(Self::{}),", val, var_id); + let _ = writeln!(out, " {:?} => Ok(Self::{}),", val, var_id); } let _ = writeln!( out, @@ -844,7 +844,7 @@ impl RustCodegen { write_documentation_lines(out, " /// ", doc); } if self.options.derive_serde && !u.is_lexical() { - let _ = writeln!(out, " #[serde(rename = \"{}\")]", branch.xml_name); + let _ = writeln!(out, " #[serde(rename = {:?})]", branch.xml_name); } if self.options.emit_polyxml_attrs && !u.is_lexical() { let _ = writeln!(out, " #[polyxml(element = \"{}\")]", branch.xml_name); diff --git a/crates/polyxml-core/src/codegen/typescript/mod.rs b/crates/polyxml-core/src/codegen/typescript/mod.rs index b01b75a..04812ed 100644 --- a/crates/polyxml-core/src/codegen/typescript/mod.rs +++ b/crates/polyxml-core/src/codegen/typescript/mod.rs @@ -422,7 +422,12 @@ impl TypeScriptCodegen { let mut seen = HashSet::new(); for v in &e.variants { let variant_key = self.unique_variant_name(&v.name, &mut seen); - let _ = writeln!(out, " {}: {:?},", variant_key, v.value); + let _ = writeln!( + out, + " {}: {},", + variant_key, + super::string_literal(&v.value) + ); } out.push_str("} as const;\n\n"); @@ -437,7 +442,7 @@ impl TypeScriptCodegen { let literals = e .variants .iter() - .map(|v| format!("{:?}", v.value)) + .map(|v| super::string_literal(&v.value)) .collect::>() .join(", "); @@ -461,7 +466,7 @@ impl TypeScriptCodegen { let tb_literals = e .variants .iter() - .map(|v| format!("Type.Literal({:?})", v.value)) + .map(|v| format!("Type.Literal({})", super::string_literal(&v.value))) .collect::>() .join(", "); let _ = writeln!( diff --git a/crates/polyxml-core/src/parser.rs b/crates/polyxml-core/src/parser.rs index afc027e..e6fe150 100644 --- a/crates/polyxml-core/src/parser.rs +++ b/crates/polyxml-core/src/parser.rs @@ -171,21 +171,23 @@ fn is_nil_element(e: &BytesStart) -> bool { /// Extend inherited namespace bindings for this element. Elements without /// declarations share the existing map. -fn namespace_scope(parent: &NamespaceScope, e: &BytesStart) -> NamespaceScope { +fn namespace_scope(parent: &NamespaceScope, e: &BytesStart) -> Result { let mut scope = None; for attr in e.attributes().flatten() { let key = attr.key.as_ref(); if key == "xmlns" { - scope - .get_or_insert_with(|| (**parent).clone()) - .insert(String::new(), attr.value.as_ref().to_string()); + scope.get_or_insert_with(|| (**parent).clone()).insert( + String::new(), + quick_xml::escape::unescape(attr.value.as_ref())?.into_owned(), + ); } else if let Some(prefix) = key.strip_prefix("xmlns:") { - scope - .get_or_insert_with(|| (**parent).clone()) - .insert(prefix.to_string(), attr.value.as_ref().to_string()); + scope.get_or_insert_with(|| (**parent).clone()).insert( + prefix.to_string(), + quick_xml::escape::unescape(attr.value.as_ref())?.into_owned(), + ); } } - scope.map(Arc::new).unwrap_or_else(|| Arc::clone(parent)) + Ok(scope.map(Arc::new).unwrap_or_else(|| Arc::clone(parent))) } /// Raw unescaped value of the type attribute in the XML Schema Instance @@ -342,7 +344,7 @@ pub const DEFAULT_MAX_DEPTH: usize = 256; impl XmlDeserializer { fn check_root(start: &BytesStart, schema: &ModelSchema) -> Result<()> { if schema.strict_root { - let scope = namespace_scope(&Arc::new(HashMap::new()), start); + let scope = namespace_scope(&Arc::new(HashMap::new()), start)?; let (ns, local) = resolve_element_qname(start, &scope); if local.as_bytes() != schema.xml_name || ns.as_deref().filter(|s| !s.is_empty()) @@ -384,7 +386,7 @@ impl XmlDeserializer { } Ok(Event::Empty(ref e)) => { Self::check_root(e, &root_schema)?; - let scope = namespace_scope(&Arc::new(HashMap::new()), e); + let scope = namespace_scope(&Arc::new(HashMap::new()), e)?; let schema = resolve_record_schema(&root_schema, e, &scope)?; let mut frame = StackFrame::new(schema); Self::parse_attributes(e, &mut frame)?; @@ -415,7 +417,7 @@ impl XmlDeserializer { inherited_scope: &NamespaceScope, ) -> Result { let mut stack: Vec = Vec::with_capacity(16); - let mut namespace_stack = vec![namespace_scope(inherited_scope, root_start)]; + let mut namespace_stack = vec![namespace_scope(inherited_scope, root_start)?]; let root_schema = resolve_record_schema(&root_schema, root_start, namespace_stack.last().unwrap())?; let mut root_frame = StackFrame::new(root_schema); @@ -431,7 +433,7 @@ impl XmlDeserializer { loop { match reader.read_event_into(&mut buf) { Ok(Event::Start(ref e)) => { - namespace_stack.push(namespace_scope(namespace_stack.last().unwrap(), e)); + namespace_stack.push(namespace_scope(namespace_stack.last().unwrap(), e)?); if unknown_depth > 0 { unknown_depth += 1; continue; @@ -626,11 +628,11 @@ impl XmlDeserializer { continue; } if any_stack.is_empty() { - let scope = namespace_scope(namespace_stack.last().unwrap(), e); + let scope = namespace_scope(namespace_stack.last().unwrap(), e)?; record_content_token(stack.last_mut().unwrap(), e, &scope); } if !any_stack.is_empty() { - let scope = namespace_scope(namespace_stack.last().unwrap(), e); + let scope = namespace_scope(namespace_stack.last().unwrap(), e)?; let (ns, local) = resolve_element_qname(e, &scope); let qname = if let Some(uri) = ns { format!("{{{}}}{}", uri, local) @@ -651,7 +653,7 @@ impl XmlDeserializer { continue; } let local_name = e.local_name(); - let scope = namespace_scope(namespace_stack.last().unwrap(), e); + let scope = namespace_scope(namespace_stack.last().unwrap(), e)?; let is_nil = is_nil_element(e); let current_schema = Arc::clone(&stack.last().unwrap().schema); @@ -1085,7 +1087,7 @@ impl XmlItemStream { .last() .cloned() .unwrap_or_else(|| Arc::new(HashMap::new())); - self.namespace_stack.push(namespace_scope(&inherited, e)); + self.namespace_stack.push(namespace_scope(&inherited, e)?); } Ok(Event::Empty(ref e)) => { let local = e.local_name(); @@ -1097,7 +1099,7 @@ impl XmlItemStream { .last() .cloned() .unwrap_or_else(|| Arc::new(HashMap::new())); - let scope = namespace_scope(&inherited, e); + let scope = namespace_scope(&inherited, e)?; let schema = resolve_record_schema(&self.schema, e, &scope)?; let mut frame = StackFrame::new(schema); XmlDeserializer::parse_attributes(e, &mut frame)?; diff --git a/crates/polyxml-core/src/schema_parser/mod.rs b/crates/polyxml-core/src/schema_parser/mod.rs index 1c7e870..42b0b7b 100644 --- a/crates/polyxml-core/src/schema_parser/mod.rs +++ b/crates/polyxml-core/src/schema_parser/mod.rs @@ -233,9 +233,11 @@ impl XsdParser { let name = e.name().into_inner(); let local = strip_prefix(name); if local == "schema" { - for attr in e.attributes().flatten() { + for attr in e.attributes() { + let attr = + attr.map_err(|error| SchemaError::Malformed(error.to_string()))?; let key = attr.key.as_ref(); - let val = attr.value.as_ref(); + let val = attr.normalized_value(quick_xml::XmlVersion::Implicit1_0)?; if key == "targetNamespace" { target_namespace = Some(val.to_string()); @@ -267,7 +269,7 @@ impl XsdParser { match local { "include" | "redefine" => { - if let Some(schema_location) = get_attr_value(e, "schemaLocation") { + if let Some(schema_location) = get_attr_value(e, "schemaLocation")? { if let Some(dir) = base_dir { let inc_path = dir.join(&schema_location); if inc_path.exists() { @@ -299,7 +301,7 @@ impl XsdParser { } } "import" => { - if let Some(schema_location) = get_attr_value(e, "schemaLocation") { + if let Some(schema_location) = get_attr_value(e, "schemaLocation")? { if let Some(dir) = base_dir { let imp_path = dir.join(&schema_location); if imp_path.exists() { @@ -341,7 +343,7 @@ impl XsdParser { &prefixes, &mut ir, )? { - if get_attr_value(e, "abstract") + if get_attr_value(e, "abstract")? .is_some_and(|value| value == "true" || value == "1") { ir.abstract_elements.insert(elem_def.qname.clone()); @@ -360,7 +362,7 @@ impl XsdParser { } "group" => { // Named model group definition. - if let Some(gname) = get_attr_value(e, "name") { + if let Some(gname) = get_attr_value(e, "name")? { let gq = QName::new(target_namespace.as_deref(), gname); let def = self.parse_group_body( &mut reader, @@ -381,7 +383,7 @@ impl XsdParser { match local { "include" | "redefine" => { - if let Some(schema_location) = get_attr_value(e, "schemaLocation") { + if let Some(schema_location) = get_attr_value(e, "schemaLocation")? { if let Some(dir) = base_dir { let inc_path = dir.join(&schema_location); if inc_path.exists() { @@ -413,7 +415,7 @@ impl XsdParser { } } "import" => { - if let Some(schema_location) = get_attr_value(e, "schemaLocation") { + if let Some(schema_location) = get_attr_value(e, "schemaLocation")? { if let Some(dir) = base_dir { let imp_path = dir.join(&schema_location); if imp_path.exists() { @@ -428,8 +430,8 @@ impl XsdParser { e, target_namespace.as_deref(), &prefixes, - ) { - if get_attr_value(e, "abstract") + )? { + if get_attr_value(e, "abstract")? .is_some_and(|value| value == "true" || value == "1") { ir.abstract_elements.insert(elem_def.qname.clone()); @@ -438,8 +440,8 @@ impl XsdParser { } } "attribute" => { - if let Some(name) = get_attr_value(e, "name") { - let type_ref = get_attr_value(e, "type") + if let Some(name) = get_attr_value(e, "name")? { + let type_ref = get_attr_value(e, "type")? .map(|ty| { resolve_type_ref( &ty, @@ -455,14 +457,14 @@ impl XsdParser { } "group" => { // Empty named model group (no particles). - if let Some(gname) = get_attr_value(e, "name") { + if let Some(gname) = get_attr_value(e, "name")? { let gq = QName::new(target_namespace.as_deref(), gname); self.groups.insert(gq, GroupDef::default()); } } "complexType" => { - if let Some(name) = get_attr_value(e, "name") { - let is_abstract = get_attr_value(e, "abstract") + if let Some(name) = get_attr_value(e, "name")? { + let is_abstract = get_attr_value(e, "abstract")? .map(|v| v == "true" || v == "1") .unwrap_or(false); let qname = QName::new(target_namespace.as_deref(), name); @@ -470,7 +472,7 @@ impl XsdParser { qname, base_type: None, is_abstract, - is_mixed: get_attr_value(e, "mixed") + is_mixed: get_attr_value(e, "mixed")? .is_some_and(|value| value == "true" || value == "1"), fields: Vec::new(), documentation: None, @@ -478,7 +480,7 @@ impl XsdParser { } } "simpleType" => { - if let Some(name) = get_attr_value(e, "name") { + if let Some(name) = get_attr_value(e, "name")? { let qname = QName::new(target_namespace.as_deref(), name); ir.add_type(TypeDef::Simple(Box::new(SimpleTypeDef { qname, @@ -512,16 +514,16 @@ impl XsdParser { name_override: Option, ir: &mut SchemaIR, ) -> Result, SchemaError> { - let name = match get_attr_value(start, "name").or(name_override) { + let name = match get_attr_value(start, "name")?.or(name_override) { Some(n) => n, None => return Ok(None), // Anonymous type handled in place }; - let is_abstract = get_attr_value(start, "abstract") + let is_abstract = get_attr_value(start, "abstract")? .map(|v| v == "true" || v == "1") .unwrap_or(false); let is_mixed = - get_attr_value(start, "mixed").is_some_and(|value| value == "true" || value == "1"); + get_attr_value(start, "mixed")?.is_some_and(|value| value == "true" || value == "1"); let qname = QName::new(target_ns, name.clone()); if !is_mixed { @@ -564,7 +566,7 @@ impl XsdParser { in_simple_content = true; } "restriction" if in_simple_content => { - if let Some(base) = get_attr_value(e, "base") { + if let Some(base) = get_attr_value(e, "base")? { let resolved = resolve_qname(&base, target_ns, prefixes); if resolved != qname { base_type = Some(resolved); @@ -576,7 +578,7 @@ impl XsdParser { } } "extension" => { - if let Some(base) = get_attr_value(e, "base") { + if let Some(base) = get_attr_value(e, "base")? { let resolved = resolve_qname(&base, target_ns, prefixes); if resolved != qname { base_type = Some(resolved); @@ -591,7 +593,7 @@ impl XsdParser { } "group" => { // Named model group particle; expanded post-parse. - if let Some(r) = get_attr_value(e, "ref") { + if let Some(r) = get_attr_value(e, "ref")? { group_refs .push((fields.len(), resolve_qname(&r, target_ns, prefixes))); } @@ -599,13 +601,13 @@ impl XsdParser { depth -= 1; } "sequence" | "all" => { - let is_unbounded = get_attr_value(e, "maxOccurs") + let is_unbounded = get_attr_value(e, "maxOccurs")? .map(|v| { v == "unbounded" || v.parse::().map(|n| n > 1).unwrap_or(false) }) .unwrap_or(false); - let min_occurs = get_attr_value(e, "minOccurs") + let min_occurs = get_attr_value(e, "minOccurs")? .and_then(|v| v.parse::().ok()) .unwrap_or(1); compositor_stack.push(CompositorFrame { @@ -622,13 +624,13 @@ impl XsdParser { }); } "choice" => { - let is_unbounded = get_attr_value(e, "maxOccurs") + let is_unbounded = get_attr_value(e, "maxOccurs")? .map(|v| { v == "unbounded" || v.parse::().map(|n| n > 1).unwrap_or(false) }) .unwrap_or(false); - let min_occurs = get_attr_value(e, "minOccurs") + let min_occurs = get_attr_value(e, "minOccurs")? .and_then(|v| v.parse::().ok()) .unwrap_or(1); compositor_stack.push(CompositorFrame { @@ -655,7 +657,7 @@ impl XsdParser { prefixes, in_any_choice, in_unbounded, - ) { + )? { // Consume inline type definitions so nested // fields cannot leak into the parent struct; // extracted types are registered in `ir`. @@ -680,7 +682,7 @@ impl XsdParser { } } "attribute" => { - if let Some(field) = parse_attribute_field(e, target_ns, prefixes) { + if let Some(field) = parse_attribute_field(e, target_ns, prefixes)? { fields.push(field); } } @@ -690,7 +692,7 @@ impl XsdParser { .last() .map(|c| c.kind == CompositorKind::Choice) .unwrap_or(false); - let field = parse_any_field(e, in_choice, in_unbounded); + let field = parse_any_field(e, in_choice, in_unbounded)?; if in_choice { if let Some(frame) = compositor_stack.last_mut() { if frame.kind == CompositorKind::Choice { @@ -721,7 +723,7 @@ impl XsdParser { "extension" | "restriction" if local == "extension" || in_simple_content => { - if let Some(base) = get_attr_value(e, "base") { + if let Some(base) = get_attr_value(e, "base")? { let resolved = resolve_qname(&base, target_ns, prefixes); if resolved != qname { base_type = Some(resolved); @@ -735,7 +737,7 @@ impl XsdParser { } "group" => { // Self-closing group reference; expanded post-parse. - if let Some(r) = get_attr_value(e, "ref") { + if let Some(r) = get_attr_value(e, "ref")? { group_refs .push((fields.len(), resolve_qname(&r, target_ns, prefixes))); } @@ -755,7 +757,7 @@ impl XsdParser { prefixes, in_any_choice, in_unbounded, - ) { + )? { if in_choice { if let Some(frame) = compositor_stack.last_mut() { if frame.kind == CompositorKind::Choice { @@ -773,7 +775,7 @@ impl XsdParser { } } "attribute" => { - if let Some(field) = parse_attribute_field(e, target_ns, prefixes) { + if let Some(field) = parse_attribute_field(e, target_ns, prefixes)? { fields.push(field); } } @@ -783,7 +785,7 @@ impl XsdParser { .last() .map(|c| c.kind == CompositorKind::Choice) .unwrap_or(false); - let field = parse_any_field(e, in_choice, in_unbounded); + let field = parse_any_field(e, in_choice, in_unbounded)?; if in_choice { if let Some(frame) = compositor_stack.last_mut() { if frame.kind == CompositorKind::Choice { @@ -1023,16 +1025,16 @@ impl XsdParser { field: &mut FieldDef, ir: &mut SchemaIR, ) -> Result<(), SchemaError> { - if get_attr_value(element_start, "type").is_some() - || get_attr_value(element_start, "ref").is_some() + if get_attr_value(element_start, "type")?.is_some() + || get_attr_value(element_start, "ref")?.is_some() { // Type already specified — discard (illegal) inline content. skip_subtree(reader)?; return Ok(()); } - let elem_local = get_attr_value(element_start, "name") - .or_else(|| get_attr_value(element_start, "ref").map(|r| strip_prefix(&r).to_string())) + let elem_local = get_attr_value(element_start, "name")? + .or(get_attr_value(element_start, "ref")?.map(|r| strip_prefix(&r).to_string())) .unwrap_or_else(|| field.name.clone()); let mut extracted = false; @@ -1122,7 +1124,7 @@ impl XsdParser { if local == "choice" { in_choice = true; } - let is_unbounded = get_attr_value(e, "maxOccurs") + let is_unbounded = get_attr_value(e, "maxOccurs")? .map(|v| { v == "unbounded" || v.parse::().map(|n| n > 1).unwrap_or(false) @@ -1132,9 +1134,13 @@ impl XsdParser { } "element" => { let in_unbounded = compositor_stack.iter().any(|&b| b); - if let Some(mut field) = - parse_element_field(e, target_ns, prefixes, in_choice, in_unbounded) - { + if let Some(mut field) = parse_element_field( + e, + target_ns, + prefixes, + in_choice, + in_unbounded, + )? { self.consume_inline_element_type( reader, e, target_ns, prefixes, "", &mut field, ir, )?; @@ -1143,7 +1149,7 @@ impl XsdParser { } } "group" => { - if let Some(r) = get_attr_value(e, "ref") { + if let Some(r) = get_attr_value(e, "ref")? { def.group_refs.push(( def.fields.len(), resolve_qname(&r, target_ns, prefixes), @@ -1154,7 +1160,8 @@ impl XsdParser { } "any" => { let in_unbounded = compositor_stack.iter().any(|&b| b); - def.fields.push(parse_any_field(e, in_choice, in_unbounded)); + def.fields + .push(parse_any_field(e, in_choice, in_unbounded)?); } _ => {} } @@ -1164,14 +1171,18 @@ impl XsdParser { match local { "element" => { let in_unbounded = compositor_stack.iter().any(|&b| b); - if let Some(field) = - parse_element_field(e, target_ns, prefixes, in_choice, in_unbounded) - { + if let Some(field) = parse_element_field( + e, + target_ns, + prefixes, + in_choice, + in_unbounded, + )? { def.fields.push(field); } } "group" => { - if let Some(r) = get_attr_value(e, "ref") { + if let Some(r) = get_attr_value(e, "ref")? { def.group_refs.push(( def.fields.len(), resolve_qname(&r, target_ns, prefixes), @@ -1180,7 +1191,8 @@ impl XsdParser { } "any" => { let in_unbounded = compositor_stack.iter().any(|&b| b); - def.fields.push(parse_any_field(e, in_choice, in_unbounded)); + def.fields + .push(parse_any_field(e, in_choice, in_unbounded)?); } _ => {} } @@ -1209,7 +1221,7 @@ impl XsdParser { name_override: Option, ir: &mut SchemaIR, ) -> Result, SchemaError> { - let name = match get_attr_value(start, "name").or(name_override) { + let name = match get_attr_value(start, "name")?.or(name_override) { Some(n) => n, None => return Ok(None), }; @@ -1237,12 +1249,12 @@ impl XsdParser { depth -= 1; } "restriction" => { - if let Some(base) = get_attr_value(e, "base") { + if let Some(base) = get_attr_value(e, "base")? { base_type = resolve_type_ref(&base, target_ns, prefixes); } } "list" => { - if let Some(item) = get_attr_value(e, "itemType") { + if let Some(item) = get_attr_value(e, "itemType")? { base_type = TypeRef::List(Box::new(resolve_type_ref( &item, target_ns, prefixes, ))); @@ -1251,7 +1263,7 @@ impl XsdParser { } } "union" => { - union_branches = Some(parse_union_members(e, target_ns, prefixes)); + union_branches = Some(parse_union_members(e, target_ns, prefixes)?); } "simpleType" if list_requires_inline => { let item_name = @@ -1299,12 +1311,12 @@ impl XsdParser { depth -= 1; } "pattern" => { - if let Some(val) = get_attr_value(e, "value") { + if let Some(val) = get_attr_value(e, "value")? { facets.patterns.push(val); } } "enumeration" => { - if let Some(val) = get_attr_value(e, "value") { + if let Some(val) = get_attr_value(e, "value")? { enum_values.push(EnumValue { name: sanitize_variant_name(&val), value: val.clone(), @@ -1321,12 +1333,12 @@ impl XsdParser { match local { "restriction" => { - if let Some(base) = get_attr_value(e, "base") { + if let Some(base) = get_attr_value(e, "base")? { base_type = resolve_type_ref(&base, target_ns, prefixes); } } "list" => { - if let Some(item) = get_attr_value(e, "itemType") { + if let Some(item) = get_attr_value(e, "itemType")? { base_type = TypeRef::List(Box::new(resolve_type_ref( &item, target_ns, prefixes, ))); @@ -1337,10 +1349,10 @@ impl XsdParser { } } "union" => { - union_branches = Some(parse_union_members(e, target_ns, prefixes)); + union_branches = Some(parse_union_members(e, target_ns, prefixes)?); } "enumeration" => { - if let Some(val) = get_attr_value(e, "value") { + if let Some(val) = get_attr_value(e, "value")? { enum_values.push(EnumValue { name: sanitize_variant_name(&val), value: val.clone(), @@ -1350,43 +1362,44 @@ impl XsdParser { } } "pattern" => { - if let Some(val) = get_attr_value(e, "value") { + if let Some(val) = get_attr_value(e, "value")? { facets.patterns.push(val); } } "minInclusive" => { - facets.min_inclusive = get_attr_value(e, "value"); + facets.min_inclusive = get_attr_value(e, "value")?; } "maxInclusive" => { - facets.max_inclusive = get_attr_value(e, "value"); + facets.max_inclusive = get_attr_value(e, "value")?; } "minExclusive" => { - facets.min_exclusive = get_attr_value(e, "value"); + facets.min_exclusive = get_attr_value(e, "value")?; } "maxExclusive" => { - facets.max_exclusive = get_attr_value(e, "value"); + facets.max_exclusive = get_attr_value(e, "value")?; } "minLength" => { facets.min_length = - get_attr_value(e, "value").and_then(|v| v.parse().ok()); + get_attr_value(e, "value")?.and_then(|v| v.parse().ok()); } "maxLength" => { facets.max_length = - get_attr_value(e, "value").and_then(|v| v.parse().ok()); + get_attr_value(e, "value")?.and_then(|v| v.parse().ok()); } "length" => { - facets.length = get_attr_value(e, "value").and_then(|v| v.parse().ok()); + facets.length = + get_attr_value(e, "value")?.and_then(|v| v.parse().ok()); } "totalDigits" => { facets.total_digits = - get_attr_value(e, "value").and_then(|v| v.parse().ok()); + get_attr_value(e, "value")?.and_then(|v| v.parse().ok()); } "fractionDigits" => { facets.fraction_digits = - get_attr_value(e, "value").and_then(|v| v.parse().ok()); + get_attr_value(e, "value")?.and_then(|v| v.parse().ok()); } "whiteSpace" => { - facets.white_space = get_attr_value(e, "value"); + facets.white_space = get_attr_value(e, "value")?; } _ => {} } @@ -1489,10 +1502,10 @@ impl XsdParser { prefixes: &HashMap, ir: &mut SchemaIR, ) -> Result<(), SchemaError> { - let Some(name) = get_attr_value(start, "name") else { + let Some(name) = get_attr_value(start, "name")? else { return skip_subtree(reader); }; - let mut type_ref = get_attr_value(start, "type") + let mut type_ref = get_attr_value(start, "type")? .map(|ty| resolve_type_ref(&ty, target_ns, prefixes)) .unwrap_or_else(TypeRef::string); loop { @@ -1529,19 +1542,19 @@ impl XsdParser { prefixes: &HashMap, ir: &mut SchemaIR, ) -> Result, SchemaError> { - let name = match get_attr_value(start, "name") { + let name = match get_attr_value(start, "name")? { Some(n) => n, None => return Ok(None), }; let qname = QName::new(target_ns, name.clone()); - let substitution_group = get_attr_value(start, "substitutionGroup") + let substitution_group = get_attr_value(start, "substitutionGroup")? .map(|s| resolve_qname(&s, target_ns, prefixes)); - let nillable = get_attr_value(start, "nillable") + let nillable = get_attr_value(start, "nillable")? .map(|v| v == "true" || v == "1") .unwrap_or(false); - let mut type_ref = get_attr_value(start, "type") + let mut type_ref = get_attr_value(start, "type")? .map(|t| resolve_type_ref(&t, target_ns, prefixes)) .unwrap_or(TypeRef::Primitive(PrimitiveType::AnyType)); @@ -1879,9 +1892,9 @@ fn parse_union_members( union: &BytesStart, target_ns: Option<&str>, prefixes: &HashMap, -) -> Vec { +) -> Result, SchemaError> { let mut names = HashMap::::new(); - get_attr_value(union, "memberTypes") + Ok(get_attr_value(union, "memberTypes")? .unwrap_or_default() .split_whitespace() .map(|member| { @@ -1905,7 +1918,7 @@ fn parse_union_members( documentation: None, } }) - .collect() + .collect()) } /// Replace element-reference placeholders with their global element's declared type. @@ -2053,14 +2066,18 @@ fn strip_prefix(s: &str) -> &str { s.split_once(':').map(|(_, local)| local).unwrap_or(s) } -fn get_attr_value(e: &BytesStart, name: &str) -> Option { - for attr in e.attributes().flatten() { +fn get_attr_value(e: &BytesStart, name: &str) -> Result, SchemaError> { + for attr in e.attributes() { + let attr = attr.map_err(|error| SchemaError::Malformed(error.to_string()))?; let key = attr.key.as_ref(); if key == name || strip_prefix(key) == name { - return Some(attr.value.as_ref().to_string()); + return Ok(Some( + attr.normalized_value(quick_xml::XmlVersion::Implicit1_0)? + .into_owned(), + )); } } - None + Ok(None) } fn resolve_qname(name: &str, target_ns: Option<&str>, prefixes: &HashMap) -> QName { @@ -2098,9 +2115,9 @@ fn parse_element_field( prefixes: &HashMap, in_choice: bool, in_unbounded_compositor: bool, -) -> Option { - let ref_attr = get_attr_value(e, "ref"); - let name_attr = get_attr_value(e, "name"); +) -> Result, SchemaError> { + let ref_attr = get_attr_value(e, "ref")?; + let name_attr = get_attr_value(e, "name")?; let (name, xml_name, namespace) = if let Some(ref r) = ref_attr { let qname = resolve_qname(r, target_ns, prefixes); @@ -2109,13 +2126,15 @@ fn parse_element_field( let namespace = qname.namespace; (field_name, xml_name, namespace) } else { - let n = name_attr?; + let Some(n) = name_attr else { + return Ok(None); + }; let xml_name = n.clone(); let namespace = target_ns.map(Into::into); (n, xml_name, namespace) }; - let type_ref = get_attr_value(e, "type") + let type_ref = get_attr_value(e, "type")? .map(|t| resolve_type_ref(&t, target_ns, prefixes)) .or_else(|| { ref_attr @@ -2127,7 +2146,7 @@ fn parse_element_field( let min_occurs = if in_choice { 0 } else { - get_attr_value(e, "minOccurs") + get_attr_value(e, "minOccurs")? .and_then(|v| v.parse().ok()) .unwrap_or(1) }; @@ -2135,21 +2154,21 @@ fn parse_element_field( let max_occurs = if in_unbounded_compositor { OccursLimit::Unbounded } else { - match get_attr_value(e, "maxOccurs").as_deref() { + match get_attr_value(e, "maxOccurs")?.as_deref() { Some("unbounded") => OccursLimit::Unbounded, Some(v) => OccursLimit::Count(v.parse().unwrap_or(1)), None => OccursLimit::Count(1), } }; - let nillable = get_attr_value(e, "nillable") + let nillable = get_attr_value(e, "nillable")? .map(|v| v == "true" || v == "1") .unwrap_or(false); - let default_value = get_attr_value(e, "default"); - let fixed_value = get_attr_value(e, "fixed"); + let default_value = get_attr_value(e, "default")?; + let fixed_value = get_attr_value(e, "fixed")?; - Some(FieldDef { + Ok(Some(FieldDef { name: sanitize_field_name(&name), xml_name, namespace, @@ -2165,16 +2184,16 @@ fn parse_element_field( documentation: None, facets: None, is_cycle_cut: false, - }) + })) } fn parse_attribute_field( e: &BytesStart, target_ns: Option<&str>, prefixes: &HashMap, -) -> Option { - let ref_attr = get_attr_value(e, "ref"); - let name_attr = get_attr_value(e, "name"); +) -> Result, SchemaError> { + let ref_attr = get_attr_value(e, "ref")?; + let name_attr = get_attr_value(e, "name")?; let (name, xml_name, namespace) = if let Some(ref r) = ref_attr { let qname = resolve_qname(r, target_ns, prefixes); @@ -2183,12 +2202,14 @@ fn parse_attribute_field( let namespace = qname.namespace; (field_name, xml_name, namespace) } else { - let n = name_attr?; + let Some(n) = name_attr else { + return Ok(None); + }; let xml_name = n.clone(); (n, xml_name, None) }; - let type_ref = get_attr_value(e, "type") + let type_ref = get_attr_value(e, "type")? .map(|t| resolve_type_ref(&t, target_ns, prefixes)) .or_else(|| { ref_attr @@ -2197,7 +2218,7 @@ fn parse_attribute_field( }) .unwrap_or(TypeRef::Primitive(PrimitiveType::String)); - let is_required = get_attr_value(e, "use") + let is_required = get_attr_value(e, "use")? .map(|u| u == "required") .unwrap_or(false); @@ -2207,10 +2228,10 @@ fn parse_attribute_field( Cardinality::optional_one() }; - let default_value = get_attr_value(e, "default"); - let fixed_value = get_attr_value(e, "fixed"); + let default_value = get_attr_value(e, "default")?; + let fixed_value = get_attr_value(e, "fixed")?; - Some(FieldDef { + Ok(Some(FieldDef { name: sanitize_field_name(&name), xml_name, namespace, @@ -2223,28 +2244,32 @@ fn parse_attribute_field( documentation: None, facets: None, is_cycle_cut: false, - }) + })) } -fn parse_any_field(e: &BytesStart, in_choice: bool, in_unbounded_compositor: bool) -> FieldDef { +fn parse_any_field( + e: &BytesStart, + in_choice: bool, + in_unbounded_compositor: bool, +) -> Result { let min_occurs = if in_choice { 0 } else { - get_attr_value(e, "minOccurs") + get_attr_value(e, "minOccurs")? .and_then(|v| v.parse::().ok()) .unwrap_or(1) }; let max_occurs = if in_unbounded_compositor { OccursLimit::Unbounded } else { - match get_attr_value(e, "maxOccurs").as_deref() { + match get_attr_value(e, "maxOccurs")?.as_deref() { Some("unbounded") => OccursLimit::Unbounded, Some(v) => OccursLimit::Count(v.parse::().unwrap_or(1)), None => OccursLimit::Count(1), } }; - let namespace = get_attr_value(e, "namespace"); - FieldDef { + let namespace = get_attr_value(e, "namespace")?; + Ok(FieldDef { name: "any".to_string(), xml_name: "*".to_string(), namespace, @@ -2260,7 +2285,7 @@ fn parse_any_field(e: &BytesStart, in_choice: bool, in_unbounded_compositor: boo documentation: None, facets: None, is_cycle_cut: false, - } + }) } fn parse_any_attribute_field(_e: &BytesStart) -> FieldDef { @@ -2284,26 +2309,28 @@ fn parse_empty_global_element( e: &BytesStart, target_ns: Option<&str>, prefixes: &HashMap, -) -> Option { - let name = get_attr_value(e, "name")?; +) -> Result, SchemaError> { + let Some(name) = get_attr_value(e, "name")? else { + return Ok(None); + }; let qname = QName::new(target_ns, name); let substitution_group = - get_attr_value(e, "substitutionGroup").map(|s| resolve_qname(&s, target_ns, prefixes)); - let nillable = get_attr_value(e, "nillable") + get_attr_value(e, "substitutionGroup")?.map(|s| resolve_qname(&s, target_ns, prefixes)); + let nillable = get_attr_value(e, "nillable")? .map(|v| v == "true" || v == "1") .unwrap_or(false); - let type_ref = get_attr_value(e, "type") + let type_ref = get_attr_value(e, "type")? .map(|t| resolve_type_ref(&t, target_ns, prefixes)) .unwrap_or(TypeRef::Primitive(PrimitiveType::AnyType)); - Some(ElementDef { + Ok(Some(ElementDef { qname, type_ref, substitution_group, nillable, documentation: None, - }) + })) } fn sanitize_field_name(name: &str) -> String { @@ -2668,34 +2695,34 @@ fn capture_content_model( } else { Particle::Sequence(items) }; - result.push(repeat(model, &e)); + result.push(repeat(model, &e)?); } else if local == "element" { if let Some(field) = - parse_element_field(&e, target_ns, prefixes, false, false) + parse_element_field(&e, target_ns, prefixes, false, false)? { result.push(repeat( - if get_attr_value(&e, "ref").is_some() { + if get_attr_value(&e, "ref")?.is_some() { Particle::Reference(QName::new(field.namespace, field.xml_name)) } else { Particle::Element(QName::new(field.namespace, field.xml_name)) }, &e, - )); + )?); } reader.read_to_end(e.name())?; } } Event::Empty(e) if strip_prefix(e.name().into_inner()) == "element" => { - if let Some(field) = parse_element_field(&e, target_ns, prefixes, false, false) + if let Some(field) = parse_element_field(&e, target_ns, prefixes, false, false)? { result.push(repeat( - if get_attr_value(&e, "ref").is_some() { + if get_attr_value(&e, "ref")?.is_some() { Particle::Reference(QName::new(field.namespace, field.xml_name)) } else { Particle::Element(QName::new(field.namespace, field.xml_name)) }, &e, - )); + )?); } } Event::End(e) if strip_prefix(e.name().into_inner()) == end => { @@ -2706,16 +2733,16 @@ fn capture_content_model( } } } - fn repeat(model: Particle, e: &BytesStart) -> Particle { - let min = get_attr_value(e, "minOccurs") + fn repeat(model: Particle, e: &BytesStart) -> Result { + let min = get_attr_value(e, "minOccurs")? .and_then(|s| s.parse().ok()) .unwrap_or(1); - let max = match get_attr_value(e, "maxOccurs").as_deref() { + let max = match get_attr_value(e, "maxOccurs")?.as_deref() { Some("unbounded") => None, Some(value) => value.parse().ok(), None => Some(1), }; - if min == 1 && max == Some(1) { + Ok(if min == 1 && max == Some(1) { model } else { Particle::Repeat { @@ -2723,7 +2750,7 @@ fn capture_content_model( min, max, } - } + }) } let mut preview = Reader::from_reader(*reader.get_ref()); preview.config_mut().allow_unmatched_ends = true; @@ -2803,16 +2830,18 @@ fn validate_schema_grammar(xml: &str) -> Result<(), SchemaError> { let mut prefixes = stack.last().map(|(_, p)| p.clone()).unwrap_or_else(|| { HashMap::from([("xml".into(), "http://www.w3.org/XML/1998/namespace".into())]) }); - for attr in element.attributes().flatten() { + for attr in element.attributes() { + let attr = attr.map_err(|error| SchemaError::Malformed(error.to_string()))?; + let value = attr.normalized_value(quick_xml::XmlVersion::Implicit1_0)?; if let Some(prefix) = attr.key.as_ref().strip_prefix("xmlns:") { - prefixes.insert(prefix.into(), attr.value.to_string()); + prefixes.insert(prefix.into(), value.to_string()); } else if attr.key.as_ref() == "xmlns" { - prefixes.insert(String::new(), attr.value.to_string()); + prefixes.insert(String::new(), value.to_string()); } } let local = strip_prefix(element.name().into_inner()); if stack.is_empty() && local == "schema" { - schema_namespace = get_attr_value(&element, "targetNamespace"); + schema_namespace = get_attr_value(&element, "targetNamespace")?; } let position = reader.buffer_position() as usize; line += xml[line_position..position] @@ -2849,14 +2878,16 @@ fn validate_schema_grammar(xml: &str) -> Result<(), SchemaError> { "Illegal child {local} under simpleType at line {line}" ))); } - for attr in element.attributes().flatten() { + for attr in element.attributes() { + let attr = attr.map_err(|error| SchemaError::Malformed(error.to_string()))?; + let value = attr.normalized_value(quick_xml::XmlVersion::Implicit1_0)?; if !annotation && matches!( attr.key.as_ref(), "type" | "base" | "itemType" | "memberTypes" | "ref" ) { - for value in attr.value.split_whitespace() { + for value in value.split_whitespace() { if let Some((prefix, local)) = value.split_once(':') { let Some(namespace) = prefixes.get(prefix) else { return Err(SchemaError::Resolution(format!( diff --git a/crates/polyxml-core/src/serializer.rs b/crates/polyxml-core/src/serializer.rs index f68ba99..f6de2f4 100644 --- a/crates/polyxml-core/src/serializer.rs +++ b/crates/polyxml-core/src/serializer.rs @@ -523,93 +523,7 @@ impl XmlSerializer { if let Some(PolyValue::List(items)) = get_field(mixed.items_index, &schema.fields[mixed.items_index].name) { - for item in items { - let tagged_field = |name: &str| -> Option<&PolyValue> { - match item { - PolyValue::Object(tagged) => tagged.get(name), - PolyValue::Record { schema, values } => schema - .fields - .iter() - .position(|field| field.name == name) - .and_then(|idx| values.get(idx)) - .and_then(Option::as_ref), - _ => None, - } - }; - let Some(kind) = tagged_field("kind").and_then(PolyValue::as_str) else { - return Err(PolyXmlError::SerializationError( - "Mixed content item has no kind".into(), - )); - }; - let Some(content) = tagged_field("value") else { - return Err(PolyXmlError::SerializationError( - "Mixed content item has no value".into(), - )); - }; - if kind == "#text" { - let Some(text) = content.as_str() else { - return Err(PolyXmlError::SerializationError( - "Mixed text item must be a string".into(), - )); - }; - writer - .write_event(Event::Text(BytesText::new(text))) - .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; - continue; - } - let Some(branch) = mixed - .branches - .iter() - .find(|branch| branch.variant_name == kind) - else { - return Err(PolyXmlError::SerializationError(format!( - "Unknown mixed content kind: {kind}" - ))); - }; - let local_name = std::str::from_utf8(&branch.xml_name)?; - let qualified = if let Some(ctx) = ns_ctx { - ctx.qualify_element( - local_name, - branch.namespace.as_deref().or(schema.namespace.as_deref()), - ) - } else { - Cow::Borrowed(local_name) - }; - match &branch.val_type { - ValueType::Scalar(_) => { - let mut buf = [0u8; lexical_core::BUFFER_SIZE]; - let Some(text) = Self::format_scalar_to(content, &mut buf) else { - return Err(PolyXmlError::SerializationError(format!( - "Invalid mixed content value for {kind}" - ))); - }; - Self::validate_scalar(&branch.val_type, &text, kind)?; - writer - .write_event(Event::Start(BytesStart::new(qualified.as_ref()))) - .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; - writer - .write_event(Event::Text(BytesText::new(&text))) - .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; - writer - .write_event(Event::End(BytesEnd::new(qualified.as_ref()))) - .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; - } - ValueType::Nested(nested) => Self::write_model( - writer, - &branch.xml_name, - content, - nested, - ns_ctx, - false, - branch.namespace.as_deref().or(nested.namespace.as_deref()), - )?, - ValueType::List(_) => { - return Err(PolyXmlError::SerializationError( - "Nested lists are invalid mixed content branches".into(), - )) - } - } - } + Self::write_mixed_items(writer, items, schema, mixed, ns_ctx)?; } } else { for (idx, field) in schema.fields.iter().enumerate() { @@ -759,6 +673,198 @@ impl XmlSerializer { Ok(()) } + fn write_mixed_items( + writer: &mut Writer, + items: &[PolyValue], + schema: &ModelSchema, + mixed: &crate::schema::MixedContentSchema, + ns_ctx: Option<&NamespaceContext>, + ) -> Result<()> { + // Amortize the temporary index only for sufficiently large schemas and + // repeated payloads. References avoid cloning keys or branch metadata. + if mixed.branches.len() >= 64 + && items.len() >= 64 + && items.len() >= mixed.branches.len().div_ceil(2) + && Self::mixed_lookup_sample_favors_index(items, mixed) + { + let mut index = std::collections::HashMap::with_capacity(mixed.branches.len()); + for branch in &mixed.branches { + // Preserve the linear lookup's first-match behavior for public + // metadata containing duplicate variant names. + index.entry(branch.variant_name.as_str()).or_insert(branch); + } + Self::write_mixed_items_using(writer, items, schema, ns_ctx, |kind| { + index.get(kind).copied() + }) + } else { + Self::write_mixed_items_using(writer, items, schema, ns_ctx, |kind| { + mixed + .branches + .iter() + .find(|branch| branch.variant_name == kind) + }) + } + } + + fn mixed_lookup_sample_favors_index( + items: &[PolyValue], + mixed: &crate::schema::MixedContentSchema, + ) -> bool { + // Spread a bounded sample across the payload. Early-branch reuse and + // text-only items otherwise pay for an index that saves little work. + let stride = items.len() / 16; + let mut tagged = 0; + let mut comparisons = 0; + for sample in 0..16 { + // Offset within each bin to avoid sampling the same tag when a + // repeated choice cycle divides the stride exactly. + let offset = (sample * 17 + 7) % stride; + let Some(kind) = items[sample * stride + offset] + .get("kind") + .and_then(PolyValue::as_str) + else { + continue; + }; + if kind == "#text" { + continue; + } + if let Some(position) = mixed + .branches + .iter() + .position(|branch| branch.variant_name == kind) + { + tagged += 1; + comparisons += position + 1; + } + } + tagged >= 4 && comparisons >= tagged * 16 + } + + #[inline(never)] + fn write_mixed_items_using<'a, W: std::io::Write, F>( + writer: &mut Writer, + items: &[PolyValue], + schema: &ModelSchema, + ns_ctx: Option<&NamespaceContext>, + lookup: F, + ) -> Result<()> + where + F: Fn(&str) -> Option<&'a crate::schema::MixedBranchSchema>, + { + for item in items { + let tagged_field = |name: &str| -> Option<&PolyValue> { + match item { + PolyValue::Object(tagged) => tagged.get(name), + PolyValue::Record { schema, values } => schema + .fields + .iter() + .position(|field| field.name == name) + .and_then(|idx| values.get(idx)) + .and_then(Option::as_ref), + _ => None, + } + }; + let Some(kind) = tagged_field("kind").and_then(PolyValue::as_str) else { + return Err(PolyXmlError::SerializationError( + "Mixed content item has no kind".into(), + )); + }; + let Some(content) = tagged_field("value") else { + return Err(PolyXmlError::SerializationError( + "Mixed content item has no value".into(), + )); + }; + if kind == "#text" { + let Some(text) = content.as_str() else { + return Err(PolyXmlError::SerializationError( + "Mixed text item must be a string".into(), + )); + }; + writer + .write_event(Event::Text(BytesText::new(text))) + .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; + continue; + } + let Some(branch) = lookup(kind) else { + return Err(PolyXmlError::SerializationError(format!( + "Unknown mixed content kind: {kind}" + ))); + }; + let local_name = std::str::from_utf8(&branch.xml_name)?; + let qualified = if let Some(ctx) = ns_ctx { + ctx.qualify_element( + local_name, + branch.namespace.as_deref().or(schema.namespace.as_deref()), + ) + } else { + Cow::Borrowed(local_name) + }; + match &branch.val_type { + ValueType::Scalar(_) => { + let mut buf = [0u8; lexical_core::BUFFER_SIZE]; + let Some(text) = Self::format_scalar_to(content, &mut buf) else { + if content.is_null() { + Self::write_mixed_nil(writer, qualified.as_ref())?; + continue; + } + return Err(PolyXmlError::SerializationError(format!( + "Invalid mixed content value for {kind}" + ))); + }; + Self::validate_scalar(&branch.val_type, &text, kind)?; + writer + .write_event(Event::Start(BytesStart::new(qualified.as_ref()))) + .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; + writer + .write_event(Event::Text(BytesText::new(&text))) + .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; + writer + .write_event(Event::End(BytesEnd::new(qualified.as_ref()))) + .map_err(|e| PolyXmlError::SerializationError(e.to_string()))?; + } + ValueType::Nested(nested) => { + if content.is_null() { + Self::write_mixed_nil(writer, qualified.as_ref())?; + } else { + Self::write_model( + writer, + &branch.xml_name, + content, + nested, + ns_ctx, + false, + branch.namespace.as_deref().or(nested.namespace.as_deref()), + )?; + } + } + ValueType::List(_) => { + return Err(PolyXmlError::SerializationError( + "Nested lists are invalid mixed content branches".into(), + )) + } + } + } + Ok(()) + } + + // A tagged null is a present nil element, unlike an absent ordinary field. + // Bind the instance prefix locally without shadowing the element's QName. + // Scalar callers reach this through their existing failed-format path. + #[cold] + fn write_mixed_nil(writer: &mut Writer, qualified: &str) -> Result<()> { + let (declaration, attribute) = if qualified.starts_with("xsi:") { + ("xmlns:xsi1", "xsi1:nil") + } else { + ("xmlns:xsi", "xsi:nil") + }; + let mut element = BytesStart::new(qualified); + element.push_attribute((declaration, XSI_NS)); + element.push_attribute((attribute, "true")); + writer + .write_event(Event::Empty(element)) + .map_err(|error| PolyXmlError::SerializationError(error.to_string())) + } + fn write_any_wildcard_element( writer: &mut quick_xml::Writer, val: &PolyValue, diff --git a/crates/polyxml-core/tests/test_default_literal_codegen.rs b/crates/polyxml-core/tests/test_default_literal_codegen.rs new file mode 100644 index 0000000..fabef09 --- /dev/null +++ b/crates/polyxml-core/tests/test_default_literal_codegen.rs @@ -0,0 +1,150 @@ +//! Execute consumers: escaping must preserve values as well as valid source. +use polyxml::codegen::*; +use polyxml::schema_parser::XsdParser; +use std::{fs, path::Path, process::Command}; + +fn run(directory: &Path, program: &str, args: &[&str]) { + let output = Command::new(program) + .args(args) + .current_dir(directory) + .output() + .unwrap(); + assert!( + output.status.success(), + "{program} {args:?}:\n{}\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); +} + +#[test] +fn generated_string_defaults_and_fixed_values_preserve_xml_characters() { + let ir = XsdParser::new() + .parse_str(include_str!( + "../../../research/fixtures/default_literal_escaping.xsd" + )) + .unwrap(); + let expected = "\"quoted\"\\path\\u000a\t\n\r\u{85}\u{2028}café 😀"; + let root = tempfile::tempdir().unwrap(); + let consumer = |name: &str| { + let path = root.path().join(name); + fs::create_dir(&path).unwrap(); + fs::write(path.join("expected.txt"), expected).unwrap(); + path + }; + let python = consumer("python"); + fs::write( + python.join("models.py"), + PythonCodegen::new(PythonOptions::default()).generate_module(&ir), + ) + .unwrap(); + fs::write( + python.join("check.py"), + r#"import dataclasses +from models import RootType +with open('expected.txt', encoding='utf-8', newline='') as f: + expected = f.read() +model = RootType(fixed=expected) +assert model.value == expected, repr(model.value) +assert model.label == expected, repr(model.label) +fields = {f.name: f for f in dataclasses.fields(RootType)} +assert fields['value'].metadata['default'] == expected +assert fields['label'].metadata['default'] == expected +assert fields['fixed'].metadata['fixed'] == expected +"#, + ) + .unwrap(); + run(&python, "python3", &["check.py"]); + let pydantic = consumer("pydantic"); + fs::write( + pydantic.join("models.py"), + PythonCodegen::new(PythonOptions { + backend: PythonBackend::Pydantic, + ..Default::default() + }) + .generate_module(&ir), + ) + .unwrap(); + fs::write( + pydantic.join("check.py"), + r#"from models import RootType +with open('expected.txt', encoding='utf-8', newline='') as f: + expected = f.read() +model = RootType(fixed=expected) +assert model.value == expected +assert model.label == expected +assert RootType.model_fields['value'].json_schema_extra['default'] == expected +assert RootType.model_fields['fixed'].json_schema_extra['fixed'] == expected +"#, + ) + .unwrap(); + run(&pydantic, "python3", &["check.py"]); + let cpp = consumer("cpp"); + fs::write( + cpp.join("models.hpp"), + CppCodegen::new(CppOptions::default()).generate_header(&ir), + ) + .unwrap(); + fs::write( + cpp.join("main.cpp"), + r#"#include "models.hpp" +#include +#include +#include +int main() { + std::ifstream input("expected.txt", std::ios::binary); + std::string expected((std::istreambuf_iterator(input)), std::istreambuf_iterator()); + polyxml::generated::RootType model; + if(model.value != expected) throw std::runtime_error("default mismatch"); +} +"#, + ) + .unwrap(); + run( + &cpp, + "g++", + &[ + "-std=c++20", + "-Wall", + "-Wextra", + "-Werror", + "main.cpp", + "-o", + "check", + ], + ); + run(&cpp, "./check", &[]); + let csharp = consumer("csharp"); + fs::write(csharp.join("Check.csproj"), "Exenet8.0enableenabletrue").unwrap(); + fs::write( + csharp.join("Models.cs"), + CSharpCodegen::new(CSharpOptions { + namespace: "Models".into(), + ..Default::default() + }) + .generate_module(&ir), + ) + .unwrap(); + fs::write( + csharp.join("Program.cs"), + r#"using Models; +using System.ComponentModel.DataAnnotations; +var expected = File.ReadAllText("expected.txt"); +var model = new RootType(); +if(model.Label != expected || model.Fixed != expected) throw new Exception("initializer mismatch"); +model.ValueXml = ""; +if(model.Value != expected) throw new Exception("element default mismatch"); +model.Fixed = expected; +try { model.Fixed = expected + "wrong"; throw new Exception("fixed value accepted"); } +catch(ValidationException) {} +model.LabelXml = expected; +if(model.Label != expected) throw new Exception("attribute proxy mismatch"); +"#, + ) + .unwrap(); + run( + &csharp, + "dotnet", + &["run", "--project", "Check.csproj", "--verbosity", "quiet"], + ); +} diff --git a/crates/polyxml-core/tests/test_enum_literal_codegen.rs b/crates/polyxml-core/tests/test_enum_literal_codegen.rs new file mode 100644 index 0000000..384b65d --- /dev/null +++ b/crates/polyxml-core/tests/test_enum_literal_codegen.rs @@ -0,0 +1,233 @@ +//! Compile generated enum consumers and compare runtime values with independent +//! data files. Source substring checks cannot catch escaping that changes values. +use std::{fs, path::Path, process::Command}; + +use polyxml::codegen::*; +use polyxml::ir::{QName, TypeDef}; +use polyxml::schema_parser::XsdParser; + +fn run(directory: &Path, program: &str, args: &[&str]) { + let output = Command::new(program) + .args(args) + .current_dir(directory) + .env("CARGO_BUILD_JOBS", "1") + .output() + .unwrap_or_else(|error| panic!("running {program}: {error}")); + assert!( + output.status.success(), + "{program} {args:?}:\n{}\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); +} + +#[test] +fn enum_literals_compile_and_preserve_values_in_all_seven_languages() { + let ir = XsdParser::new() + .parse_str(include_str!( + "../../../research/fixtures/enum_literal_escaping.xsd" + )) + .unwrap(); + let TypeDef::Enum(status) = &ir.types[&QName::local("Status")] else { + panic!("expected Status enum"); + }; + let expected = [ + "R&D", + "\"quoted\" 'value' ", + "café 😀", + "literal &", + "slash\\path\\u000a", + "tab\tline\ncarriage\rtail", + "\u{85}next\u{2028}line", + ]; + assert_eq!( + status + .variants + .iter() + .map(|v| v.value.as_str()) + .collect::>(), + expected + ); + let root = tempfile::tempdir().unwrap(); + let json = serde_json::to_vec(&expected).unwrap(); + let mut binary = Vec::new(); + for value in expected { + binary.extend_from_slice(&(value.len() as u32).to_le_bytes()); + binary.extend_from_slice(value.as_bytes()); + } + let consumer = |name: &str| { + let path = root.path().join(name); + fs::create_dir(&path).unwrap(); + fs::write(path.join("expected.json"), &json).unwrap(); + fs::write(path.join("expected.bin"), &binary).unwrap(); + path + }; + + let rust = consumer("rust"); + fs::create_dir(rust.join("src")).unwrap(); + let core = Path::new(env!("CARGO_MANIFEST_DIR")); + fs::write(rust.join("Cargo.toml"), format!("[package]\nname=\"enum-literal-test\"\nversion=\"0.0.0\"\nedition=\"2021\"\n[dependencies]\npolyxml={{path={core:?}}}\nquick-xml=\"0.42\"\nserde={{version=\"1\",features=[\"derive\"]}}\nserde_json=\"1\"\n")).unwrap(); + fs::write( + rust.join("src/models.rs"), + RustCodegen::new(RustOptions::default()).generate_module(&ir), + ) + .unwrap(); + fs::write(rust.join("src/main.rs"), r#"mod models; +fn main() { + let expected: Vec = serde_json::from_slice(&std::fs::read("expected.json").unwrap()).unwrap(); + for text in expected { + let value: models::Status = text.parse().unwrap(); + assert_eq!(value.as_str(), text); + let json = serde_json::to_string(&value).unwrap(); + assert_eq!(serde_json::from_str::(&json).unwrap(), text); + assert_eq!(serde_json::from_str::(&json).unwrap(), value); + } +}"#).unwrap(); + let output = Command::new("cargo") + .args(["run", "--offline", "--quiet"]) + .env( + "CARGO_TARGET_DIR", + core.join("../../target/enum-literal-tests"), + ) + .env("CARGO_BUILD_JOBS", "1") + .current_dir(&rust) + .output() + .unwrap(); + assert!( + output.status.success(), + "Rust: {}", + String::from_utf8_lossy(&output.stderr) + ); + + let python = consumer("python"); + fs::write( + python.join("models.py"), + PythonCodegen::new(PythonOptions::default()).generate_module(&ir), + ) + .unwrap(); + fs::write(python.join("check.py"), "import json\nfrom models import Status\nwith open('expected.json', encoding='utf-8') as f:\n expected = json.load(f)\nassert [v.value for v in Status] == expected\nfor value in expected:\n assert Status(value).value == value\n").unwrap(); + run(&python, "python3", &["check.py"]); + + let go = consumer("go"); + fs::write(go.join("go.mod"), "module literals\n\ngo 1.22\n").unwrap(); + fs::write( + go.join("models.go"), + GoCodegen::new(GoOptions::default()).generate_module(&ir), + ) + .unwrap(); + fs::write(go.join("models_test.go"), r#"package models +import("encoding/json";"os";"testing") +func TestValues(t *testing.T) { + data,err:=os.ReadFile("expected.json");if err!=nil{t.Fatal(err)};var expected []string + if err=json.Unmarshal(data,&expected);err!=nil{t.Fatal(err)} + for _,text:=range expected{value:=Status(text);if !value.IsValid(){t.Fatal(text)};encoded,err:=json.Marshal(value);if err!=nil{t.Fatal(err)};var actual string;if err=json.Unmarshal(encoded,&actual);err!=nil{t.Fatal(err)};if actual!=text{t.Fatal(actual,text)}} +}"#).unwrap(); + run(&go, "go", &["test", "./..."]); + + let cpp = consumer("cpp"); + fs::write( + cpp.join("models.hpp"), + CppCodegen::new(CppOptions::default()).generate_header(&ir), + ) + .unwrap(); + fs::write( + cpp.join("main.cpp"), + r#"#include "models.hpp" +#include +#include +int main(){std::ifstream input("expected.bin",std::ios::binary);unsigned char size[4]; + while(input.read(reinterpret_cast(size),4)){ + unsigned count=size[0]|(unsigned(size[1])<<8)|(unsigned(size[2])<<16)|(unsigned(size[3])<<24); + std::string expected(count,'\0');input.read(expected.data(),count); + auto value=polyxml::generated::status_from_string(expected); + if(!value||polyxml::generated::to_string(*value)!=expected)throw std::runtime_error(expected); + }}"#, + ) + .unwrap(); + run( + &cpp, + "c++", + &[ + "-std=c++20", + "-Wall", + "-Wextra", + "-Werror", + "main.cpp", + "-o", + "check", + ], + ); + run(&cpp, "./check", &[]); + + let java = consumer("java"); + fs::write( + java.join("Models.java"), + JavaCodegen::new(JavaOptions { + package_name: String::new(), + ..Default::default() + }) + .generate_module(&ir, "Models"), + ) + .unwrap(); + fs::write(java.join("Check.java"), r#"import java.nio.*;import java.nio.file.*;import java.nio.charset.*; +public class Check {public static void main(String[] args)throws Exception{ + ByteBuffer input=ByteBuffer.wrap(Files.readAllBytes(Path.of("expected.bin"))).order(ByteOrder.LITTLE_ENDIAN); + while(input.hasRemaining()){byte[] bytes=new byte[input.getInt()];input.get(bytes);String text=new String(bytes,StandardCharsets.UTF_8); + if(!Models.Status.fromValue(text).getValue().equals(text))throw new AssertionError(text); + }}} +"#).unwrap(); + run(&java, "javac", &["Models.java", "Check.java"]); + run(&java, "java", &["Check"]); + + let csharp = consumer("csharp"); + fs::write(csharp.join("Check.csproj"), "Exenet8.0enableenable").unwrap(); + fs::write( + csharp.join("Models.cs"), + CSharpCodegen::new(CSharpOptions { + namespace: "Models".into(), + ..Default::default() + }) + .generate_module(&ir), + ) + .unwrap(); + fs::write( + csharp.join("Program.cs"), + r#"using Models;using System.Text;using System.Xml.Serialization;using System.Reflection; +using var reader=new BinaryReader(File.OpenRead("expected.bin")); +foreach(var value in Enum.GetValues()){ + string expected=Encoding.UTF8.GetString(reader.ReadBytes(reader.ReadInt32())); + if(value.ToXmlValue()!=expected)throw new Exception(expected); + var attribute=typeof(Status).GetField(value.ToString())!.GetCustomAttribute()!; + if(attribute.Name!=expected)throw new Exception(attribute.Name); +} +if(reader.BaseStream.Position!=reader.BaseStream.Length)throw new Exception("missing variants"); +"#, + ) + .unwrap(); + run( + &csharp, + "dotnet", + &["run", "--project", "Check.csproj", "--verbosity", "quiet"], + ); + + let typescript = consumer("typescript"); + fs::write( + typescript.join("models.ts"), + TypeScriptCodegen::new(TypeScriptOptions::default()).generate_module(&ir), + ) + .unwrap(); + fs::write(typescript.join("check.cjs"), "const fs=require('fs');const {Status}=require('./models.js');const assert=require('assert');assert.deepStrictEqual(Object.values(Status),JSON.parse(fs.readFileSync('expected.json','utf8')));\n").unwrap(); + run( + &typescript, + "tsc", + &[ + "--target", + "es2020", + "--module", + "commonjs", + "--strict", + "models.ts", + ], + ); + run(&typescript, "node", &["check.cjs"]); +} diff --git a/crates/polyxml-core/tests/test_mixed_branch_lookup.rs b/crates/polyxml-core/tests/test_mixed_branch_lookup.rs new file mode 100644 index 0000000..52996dc --- /dev/null +++ b/crates/polyxml-core/tests/test_mixed_branch_lookup.rs @@ -0,0 +1,122 @@ +use polyxml::schema::{FieldKind, FieldSchema, ModelSchema, ScalarType, ValueType}; +use polyxml::schema_parser::XsdParser; +use polyxml::{deserialize, serialize, PolyValue}; +use std::{collections::HashMap, sync::Arc}; + +fn tagged(kind: &str, value: PolyValue) -> PolyValue { + PolyValue::Object(HashMap::from([ + ("kind".into(), PolyValue::String(kind.into())), + ("value".into(), value), + ])) +} + +#[test] +fn large_mixed_tables_keep_first_match_metadata_edits_records_and_nil() { + let declarations = (0..64) + .map(|i| format!(r#""#)) + .collect::(); + let xsd = format!( + r#"{declarations}"# + ); + let ir = XsdParser::new().parse_str(&xsd).unwrap(); + let mut schema = ModelSchema::from_ir(&ir, Some("Root")).unwrap(); + let mixed = schema.mixed_content.as_ref().unwrap(); + let items_name = schema.fields[mixed.items_index].name.clone(); + let kinds = (0..64) + .map(|i| { + mixed + .branches + .iter() + .find(|b| b.xml_name == format!("B{i}").as_bytes()) + .unwrap() + .variant_name + .clone() + }) + .collect::>(); + let first_index = mixed + .branches + .iter() + .position(|b| b.xml_name == b"B0") + .unwrap(); + let old_kind = kinds[0].clone(); + let record_schema = ModelSchema::builder("Tagged") + .field(FieldSchema::new( + "value", + b"value", + FieldKind::Element, + ValueType::Scalar(ScalarType::Int), + )) + .field(FieldSchema::new( + "kind", + b"kind", + FieldKind::Element, + ValueType::Scalar(ScalarType::String), + )) + .build(); + let mut items = (0..256) + .map(|i| tagged(&kinds[i % 64], PolyValue::Int(i as i64))) + .collect::>(); + items[0] = PolyValue::Record { + schema: record_schema, + values: vec![ + Some(PolyValue::Int(0)), + Some(PolyValue::String(old_kind.clone())), + ] + .into_boxed_slice(), + }; + items.push(tagged(&kinds[63], PolyValue::Null)); + let mut value = PolyValue::Object(HashMap::from([( + items_name.clone(), + PolyValue::List(items), + )])); + { + let mixed = Arc::get_mut(&mut schema) + .unwrap() + .mixed_content + .as_mut() + .unwrap(); + let mut duplicate = mixed.branches[first_index].clone(); + duplicate.xml_name = b"Duplicate".to_vec(); + mixed.branches.push(duplicate); + } + let output = serialize("Root", &value, &schema, None).unwrap(); + assert!(std::str::from_utf8(&output).unwrap().contains("0")); + assert!(!std::str::from_utf8(&output).unwrap().contains(" { + fields.insert("kind".into(), PolyValue::String("changed_kind".into())); + } + PolyValue::Record { values, .. } => { + values[1] = Some(PolyValue::String("changed_kind".into())) + } + _ => unreachable!(), + } + } + } + let output = serialize("Root", &value, &schema, None).unwrap(); + assert_eq!(value, deserialize(&output, Arc::clone(&schema)).unwrap()); + let PolyValue::Object(root) = &mut value else { + unreachable!() + }; + let PolyValue::List(items) = root.get_mut(&items_name).unwrap() else { + unreachable!() + }; + items[0] = tagged("unregistered", PolyValue::Int(0)); + assert!(serialize("Root", &value, &schema, None).is_err()); +} diff --git a/crates/polyxml-core/tests/test_mixed_nil.rs b/crates/polyxml-core/tests/test_mixed_nil.rs new file mode 100644 index 0000000..95cc765 --- /dev/null +++ b/crates/polyxml-core/tests/test_mixed_nil.rs @@ -0,0 +1,93 @@ +use std::collections::HashMap; +use std::sync::Arc; + +use polyxml::schema::ModelSchema; +use polyxml::schema_parser::XsdParser; +use polyxml::serializer::XmlSerializer; +use polyxml::{deserialize, serialize, PolyValue}; + +fn schema(namespace: Option<&str>) -> Arc { + let declaration = namespace + .map(|ns| format!("targetNamespace='{ns}' xmlns:t='{ns}' elementFormDefault='qualified'")) + .unwrap_or_default(); + let xsd = format!(" + + + + + + + "); + let ir = XsdParser::new().parse_str(&xsd).unwrap(); + ModelSchema::from_ir(&ir, Some("Root")).unwrap() +} + +#[test] +fn nil_scalar_and_nested_items_keep_order_and_differ_from_empty_strings() { + let schema = schema(None); + for nil in [ + "", + "", + "", + "", + ] { + let xml = format!("before{nil}nested{nil}after"); + let value = deserialize(xml.as_bytes(), Arc::clone(&schema)).unwrap(); + let items_field = &schema.fields[schema.mixed_content.as_ref().unwrap().items_index].name; + let items = value.get(items_field).and_then(PolyValue::as_list).unwrap(); + assert!(items[1].get("value").unwrap().is_null()); + assert_eq!(items[2].get("value").and_then(PolyValue::as_str), Some("")); + assert!(items[4].get("value").unwrap().is_null()); + let output = serialize("Root", &value, &schema, None).unwrap(); + assert_eq!(value, deserialize(&output, Arc::clone(&schema)).unwrap()); + } +} + +#[test] +fn nil_instance_prefix_does_not_rebind_the_element_namespace() { + let schema = schema(Some("urn:document")); + let xml = b"after"; + let value = deserialize(xml, Arc::clone(&schema)).unwrap(); + for prefix in ["", "doc", "xsi", "xsi1"] { + let ns_map = HashMap::from([(prefix.to_owned(), "urn:document".to_owned())]); + let output = XmlSerializer::serialize_with_options( + "Root", + &value, + &schema, + None, + Some(true), + Some(&ns_map), + ) + .unwrap(); + let mut reader = quick_xml::NsReader::from_reader(output.as_slice()); + let mut nils = 0; + loop { + let (namespace, event) = reader.read_resolved_event().unwrap(); + match event { + quick_xml::events::Event::Empty(element) => { + assert!( + matches!(namespace, quick_xml::name::ResolveResult::Bound(uri) if uri.as_ref() == "urn:document") + ); + for attribute in element.attributes() { + let attribute = attribute.unwrap(); + if attribute.key.local_name().as_ref() == "nil" { + let (namespace, _) = reader.resolver().resolve_attribute(attribute.key); + assert!( + matches!(namespace, quick_xml::name::ResolveResult::Bound(uri) if uri.as_ref() == "http://www.w3.org/2001/XMLSchema-instance") + ); + nils += 1; + } + } + } + quick_xml::events::Event::Eof => break, + _ => {} + } + } + assert_eq!(nils, 3); + assert_eq!(value, deserialize(&output, Arc::clone(&schema)).unwrap()); + } + let output = + XmlSerializer::serialize_with_options("Root", &value, &schema, None, Some(false), None) + .unwrap(); + assert_eq!(value, deserialize(&output, schema).unwrap()); +} diff --git a/crates/polyxml-core/tests/test_namespace_entities.rs b/crates/polyxml-core/tests/test_namespace_entities.rs new file mode 100644 index 0000000..9d92f2a --- /dev/null +++ b/crates/polyxml-core/tests/test_namespace_entities.rs @@ -0,0 +1,90 @@ +use polyxml::schema::{FieldKind, FieldSchema, ModelSchema, ScalarType, ValueType}; +use polyxml::{deserialize, serialize, PolyValue, XmlItemStream}; +use std::{io::Cursor, sync::Arc}; + +const NS: &str = "urn:polyxml:A&B"; +fn value_field() -> FieldSchema { + FieldSchema::new( + "value", + b"Value", + FieldKind::Element, + ValueType::Scalar(ScalarType::Int), + ) +} + +#[test] +fn escaped_namespace_bindings_match_strict_root_and_round_trip() { + let schema = ModelSchema::builder("Root") + .namespace(NS) + .strict_root(true) + .field(value_field()) + .build(); + for xml in [ + "7", + "7", + "7", + ] { + let value = deserialize(xml.as_bytes(), Arc::clone(&schema)).unwrap(); + assert_eq!(value.get("value"), Some(&PolyValue::Int(7))); + let output = serialize("Root", &value, &schema, None).unwrap(); + assert_eq!(value, deserialize(&output, Arc::clone(&schema)).unwrap()); + } +} + +#[test] +fn namespace_entity_values_are_decoded_once() { + let schema = ModelSchema::builder("Root") + .namespace("urn:polyxml:literal&") + .strict_root(true) + .build(); + let xml = b""; + assert!(deserialize(xml, schema).is_ok()); +} + +fn abstract_schema() -> Arc { + let base = ModelSchema::builder("Root").is_abstract(true).build(); + let concrete = ModelSchema::builder("Derived") + .namespace(NS) + .field(value_field()) + .build(); + base.set_variants(vec![concrete]); + base +} + +#[test] +fn entity_decoding_applies_to_inherited_and_local_xsi_type_bindings() { + let base = abstract_schema(); + for xml in [ + "9", + "", + ] { + let value=deserialize(xml.as_bytes(),Arc::clone(&base)).unwrap(); + let PolyValue::Record{schema,..}=value else{panic!("expected concrete record")}; + assert_eq!(schema.name,"Derived"); + } + let xml="9"; + let mut stream = XmlItemStream::new(Cursor::new(xml.as_bytes()), base, b"Root"); + for _ in 0..2 { + let value = stream.next_item().unwrap().unwrap(); + let PolyValue::Record { schema, .. } = value else { + panic!("expected concrete record") + }; + assert_eq!(schema.name, "Derived"); + } + assert!(stream.next_item().unwrap().is_none()); +} + +#[test] +fn undefined_entities_cannot_form_namespace_bindings() { + let schema = ModelSchema::builder("Root").build(); + for xml in [ + "", + "", + "", + ] { + assert!( + deserialize(xml.as_bytes(), Arc::clone(&schema)).is_err(), + "{xml}" + ); + } +} diff --git a/crates/polyxml-core/tests/test_runtime_fast_paths.rs b/crates/polyxml-core/tests/test_runtime_fast_paths.rs index c0f13f1..07aa439 100644 --- a/crates/polyxml-core/tests/test_runtime_fast_paths.rs +++ b/crates/polyxml-core/tests/test_runtime_fast_paths.rs @@ -239,6 +239,8 @@ fn unified_scalar_state_keeps_mixed_branch_order_nil_and_nested_frames() { assert_eq!(value,deserialize(&output,Arc::clone(&schema)).unwrap()); } let nil_document = b"Ready"; - assert!(deserialize(nil_document, Arc::clone(&schema)).is_ok()); + let value = deserialize(nil_document, Arc::clone(&schema)).unwrap(); + let output = serialize("Root", &value, &schema, None).unwrap(); + assert_eq!(value, deserialize(&output, Arc::clone(&schema)).unwrap()); assert!(deserialize(b"invalid", schema).is_err()); } diff --git a/crates/polyxml-core/tests/test_schema_attribute_entities.rs b/crates/polyxml-core/tests/test_schema_attribute_entities.rs new file mode 100644 index 0000000..a31540b --- /dev/null +++ b/crates/polyxml-core/tests/test_schema_attribute_entities.rs @@ -0,0 +1,114 @@ +use std::sync::Arc; + +use polyxml::ir::{QName, TypeDef, TypeRef}; +use polyxml::schema::ModelSchema; +use polyxml::schema_parser::XsdParser; +use polyxml::{deserialize, serialize, PolyValue}; + +const FIXTURE: &str = include_str!("../../../research/fixtures/schema_attribute_entities.xsd"); + +#[test] +fn attribute_entities_reach_enum_facets_and_defaults_in_value_space() { + let ir = XsdParser::new().parse_str(FIXTURE).unwrap(); + let TypeDef::Enum(status) = &ir.types[&QName::local("Status")] else { + panic!("Status must be an enum"); + }; + assert_eq!( + status + .variants + .iter() + .map(|v| v.value.as_str()) + .collect::>(), + [ + "R&D", + "\"quoted\" 'value' ", + "café 😀", + "literal &" + ] + ); + let TypeDef::Simple(code) = &ir.types[&QName::local("Code")] else { + panic!("Code must retain its pattern"); + }; + assert_eq!(code.facets.patterns, ["[A-Z]&[A-Z]"]); + let schema = ModelSchema::from_ir(&ir, Some("Root")).unwrap(); + for status in [ + "R&D", + ""quoted" 'value' <tag>", + "café 😀", + "literal &amp;", + ] { + let xml = + format!("{status}A&B"); + let value = deserialize(xml.as_bytes(), Arc::clone(&schema)).unwrap(); + for (name, expected) in [ + ("fixed", "R&D"), + ("default", "café"), + ("label", "