Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .agents/skills/polyxml-core-engine/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ This skill documents the high-performance design patterns and strict constraints
- Resolve refs leniently like `parser.rs::append_general_ref`: `is_char_ref()`/`resolve_char_ref()` for `&#NN;`/`&#xNN;`, `escape::resolve_xml_entity()` for `amp`/`lt`/`gt`/`quot`/`apos`, else push the raw name (never error — a document the core accepts must not fail in generated code).
- Never enable `trim_text` on a reader that assembles split text: per-segment trimming loses spaces around refs (`x & y` → `x&y`). Trim the assembled buffer once at element end (the transcoder `End` arm).
- Attribute values arrive raw too — always `escape::unescape` them (`parse_attributes`, the transcoder's Start/Empty attr arms).
5. **Document Transcoding Is Buffered**:
- `transcoder::xml_to_json` and `json_to_xml` accept complete byte slices and return complete byte vectors. The schema-free path constructs a `serde_json::Value` tree; the CLI reads all stdin or file input before calling it. Pipes do not make this API incremental or zero-copy. Keep CLI help and docs distinct from `XmlItemStream` and the Wasm record-stream API.

## 2. Testing & Verification

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,9 +96,9 @@ json_bytes = customer.to_json(indent=2) # native JSON on the same model

---

## 🔄 Dual-Format XML ↔ JSON Streaming Transcoder (`polyxml transcode`)
## 🔄 Dual-Format XML ↔ JSON Transcoding (`polyxml transcode`)

Bridge legacy enterprise XML (ISO 20022 banking, HL7 healthcare, FIXM aviation) and modern JSON microservices with an ultra-fast, zero-copy streaming transcoder — via stdin/stdout CLI pipes or the `polyxml.xml_to_json` / `polyxml.json_to_xml` Python APIs. **[Full reference →](docs/guides/compiler.md)**
Bridge legacy enterprise XML (ISO 20022 banking, HL7 healthcare, FIXM aviation) and modern JSON microservices through stdin/stdout CLI pipes or the `polyxml.xml_to_json` / `polyxml.json_to_xml` Python APIs. Each call reads and converts a complete document in memory. **[Full reference →](docs/guides/compiler.md)**

---

Expand Down
2 changes: 1 addition & 1 deletion crates/polyxml-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ pub enum Commands {
/// Validate XML schema syntax and structural invariants without generating code
Validate(ValidateArgs),

/// Bidirectionally transcode XML ↔ JSON with zero-copy streaming
/// Convert complete XML or JSON documents; stdin/stdout pipes are supported
Transcode(TranscodeArgs),

/// Print a shell completion script with target-aware option suggestions
Expand Down
6 changes: 4 additions & 2 deletions crates/polyxml-core/src/transcoder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ use crate::serializer::XmlSerializer;
///
/// If a `ModelSchema` is provided, typed data-binding is used, respecting numeric, boolean,
/// and collection types as well as field aliases.
/// If `schema` is None, zero-copy dynamic streaming transcoding is used.
/// If `schema` is None, XML events are assembled into a JSON value tree.
/// The complete input and output are held in memory.
pub fn xml_to_json(
xml: &[u8],
schema: Option<Arc<ModelSchema>>,
Expand All @@ -35,7 +36,8 @@ pub fn xml_to_json(
///
/// If a `ModelSchema` is provided, typed data-binding is used, mapping JSON properties to XML
/// elements, attributes, and namespaces.
/// If `schema` is None, zero-copy dynamic streaming transcoding is used.
/// If `schema` is None, the complete JSON input is parsed into a value tree
/// before XML output is written into a byte buffer.
pub fn json_to_xml(
json: &[u8],
schema: Option<Arc<ModelSchema>>,
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/compiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,13 +210,13 @@ Validates:

### 4. `polyxml transcode`

Bidirectionally transcode between XML and JSON using zero-copy streaming, with optional schema guidance:
Bidirectionally convert complete XML and JSON documents, with optional schema guidance. The command accepts stdin and stdout pipes, but reads the full input and buffers the full output before writing it:

```bash
# 1. Transcode XML to JSON with W3C XSD schema typing
polyxml transcode --schema order.xsd --pretty order.xml --out order.json

# 2. Stream directly through stdin / stdout pipes
# 2. Connect stdin / stdout pipes
cat order.xml | polyxml transcode --schema order.xsd > order.json

# 3. Transcode JSON back to XML with specified root element
Expand Down
3 changes: 1 addition & 2 deletions docs/guides/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,7 +457,7 @@ user = parser.parse("data.json", User)

## 8. High-Performance XML ↔ JSON Transcoding (`polyxml.xml_to_json` & `polyxml.json_to_xml`)

PolyXML provides zero-copy streaming functions to transcode between XML and JSON directly in Rust/C without building intermediate DOM trees or incurring Python loop overhead.
PolyXML provides Rust-backed functions to convert complete XML and JSON documents without a Python-level parsing loop. Each call holds the input and output in memory; schema-free conversion also builds an intermediate JSON value tree.

### Schema-Directed Transcoding

Expand Down Expand Up @@ -536,4 +536,3 @@ with open("order.xml", "rb") as f:
```



31 changes: 14 additions & 17 deletions docs/guides/rust.md
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,7 @@ To achieve the maximum throughput from `polyxml-core`:

---

## 8. Native JSON Codecs & Streaming Transcoder
## 8. Native JSON Codecs & Document Transcoder

### Inherent JSON Methods on Generated Models

Expand All @@ -325,34 +325,33 @@ let restored = Order::from_json_str(&json_str)?;
let from_bytes = Order::from_json_slice(&json_vec)?;
```

### Direct Streaming Transcoder (`polyxml::transcoder`)
### Whole-Document Transcoder (`polyxml::transcoder`)

For high-speed transcoding without compiling Rust structs, use `polyxml::transcoder`:
To convert a complete document without compiling Rust structs, use
`polyxml::transcoder`. It accepts byte slices and returns newly allocated
output bytes; schema-free conversion builds an intermediate JSON value tree:

```rust
use polyxml::transcoder::{xml_to_json, json_to_xml, TranscodeOptions};
use polyxml::transcoder::{xml_to_json, json_to_xml};

let xml_input = br#"<Product id="42"><name>Sensor</name><price>19.99</price></Product>"#;

// Transcode XML to JSON with pretty formatting
let json_bytes = xml_to_json(
xml_input,
None, // Optional Arc<ModelSchema> or SchemaIR
TranscodeOptions {
pretty: true,
..Default::default()
},
None, // Optional Arc<ModelSchema>
Some(2), // Indentation width
true, // Use schema field aliases
)?;

// Transcode JSON back to XML with specified root element
// Transcode JSON back to XML, inferring the root from its single top-level key
let restored_xml = json_to_xml(
&json_bytes,
None,
TranscodeOptions {
root_tag: Some("Product".to_string()),
pretty: true,
..Default::default()
},
None,
Some(2),
None, // Namespace handling
None, // Namespace map
)?;
```

Expand Down Expand Up @@ -392,5 +391,3 @@ let archived = rkyv::access::<ArchivedPacket, Error>(&bytes)?;
assert_eq!(archived.id, 42);
```



2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Legacy XML Toolchains (JAXB, CodeSynthesis, xsdata, xgen)
The PolyXML Way
✅ Unified Rust Tool: Generates idiomatic, type-safe code (like protoc) across 7 languages.
✅ Zero-Allocation Streaming: Direct-to-struct parsing with quick-xml & lexical-core (10x–24x faster).
✅ Dual-Format XML ↔ JSON: Zero-copy streaming transcoder (polyxml transcode) & dual-annotated models.
✅ Dual-Format XML ↔ JSON: Whole-document transcoding (`polyxml transcode`) and dual-annotated models.
✅ Modern Language Idioms: Immutable Java 21+ records, C++20 value types, Python 3.12 PEP 695 dataclasses.
✅ Secure by Design: Pure-Rust streaming parser structurally immune to XXE (CWE-611) & SSRF.
✅ 100% Permissive MIT: Zero commercial licensing fees, zero GPL infection risk.
Expand Down
6 changes: 3 additions & 3 deletions docs/why-polyxml.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,15 @@ Historical C++ tools like CodeSynthesis XSD and gSOAP enforce strict **GPL v2 /

PolyXML is **100% permissively licensed under the MIT License**, with zero runtime licensing fees, zero commercial paywalls, and zero legal restrictions on proprietary distribution.

### 5. Dual-Format Polyglot Architecture & Zero-Copy Streaming Transcoder (`polyxml transcode`)
### 5. Dual-Format Polyglot Architecture & XML/JSON Transcoding (`polyxml transcode`)
Enterprise engineering rarely lives in an XML-only silo. Interbank rails (ISO 20022), aviation telemetry (FIXM), and healthcare networks (HL7) mandate strict XML Schema contracts, but modern cloud services, microservices, and frontends operate on JSON.

Historically, bridging this divide forced engineering teams into painful trade-offs:
- **Fragile Untyped Parsers**: Running `xmltodict` or ad-hoc scripts drops XML attribute metadata (`@attr`), mangles repeated elements, and runs up to 38x slower.
- **Duplicate Schema Maintenance**: Manually writing and synchronizing separate XSD and OpenAPI/JSON schemas across teams inevitably leads to silent drift and catastrophic production outages.

PolyXML breaks this dichotomy through a **natively dual-format architecture**:
- **Zero-Copy Streaming Transcoder (`polyxml transcode`)**: A Rust-powered CLI and runtime transcoder that converts XML ↔ JSON bidirectionally via streaming events without building DOM trees.
- **Whole-Document Transcoder (`polyxml transcode`)**: A Rust-powered CLI and runtime converter for XML ↔ JSON. It accepts stdin/stdout pipes, but buffers each complete input and output document; schema-free conversion builds an intermediate JSON value tree.
- **Schema-Directed Precision**: Use `--schema schema.xsd` to ensure numeric types, booleans, and arrays in JSON match the exact XSD type definitions rather than ambiguous strings.
- **Dynamic Schema-Less Fallback**: Automatically preserves XML attributes (`@attr`) and text content (`#text`) in pure JSON when no schema is present.
- **Natively Dual-Annotated Generated Models**:
Expand Down Expand Up @@ -179,7 +179,7 @@ Across more than 600 official test groups from Sun Microsystems, Microsoft, and
| **`xsd-parser` in Rust** | A battle-tested compiler that doesn't panic on complex schemas, with automatic Tarjan `Box<T>` cycle breaks, inherent streaming XML codecs, and native `.to_json_string()` codecs. |
| **`xgen` in Go** | Dual `xml:"..."` and `json:"..."` struct tags on every model, true `xs:choice` mutual exclusivity validation, pointer cycle breaks, and canonical Go initialism normalization. |
| **`xsd.exe` in .NET** | Modern C# 12 records with primary constructors, dual `XmlSerializer` and `System.Text.Json` attributes (`[JsonPropertyName]`, `[JsonConverter]`), and standard `IValidatableObject` integration. |
| **Ad-hoc XML ↔ JSON Scripts** | Zero-copy streaming CLI (`polyxml transcode`) with schema-directed precision or dynamic `@attr` preservation, executing in microseconds. |
| **Ad-hoc XML ↔ JSON Scripts** | CLI document conversion (`polyxml transcode`) with schema-directed precision or dynamic `@attr` preservation. |

**Ready to modernize your XML infrastructure?**
👉 **[Get Started with the 5-Minute Quickstart →](quickstart.md)**
Expand Down
Loading