Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -286,9 +286,11 @@ jobs:
working-directory: ./rust
run: cargo test --doc --verbose

- name: Run example
- name: Run examples
working-directory: ./rust
run: cargo run --example basic_usage
run: |
cargo run --example basic_usage
cargo run --example append_only_log

# === BUILD ===
# Build package - only runs if lint and test pass
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -172,3 +172,6 @@ htmlcov/
.dmypy.json
dmypy.json
.ruff_cache/

# Cargo build output of the scratch crates under experiments/
experiments/**/target/
3 changes: 2 additions & 1 deletion .gitkeep
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# .gitkeep file auto-generated at 2026-05-10T19:22:27.543Z for PR creation at branch issue-35-03946ff48852 for issue https://github.com/link-foundation/lino-objects-codec/issues/35
# Updated: 2026-08-20T05:25:16.696Z
# Updated: 2026-08-20T06:10:07.182Z
# Updated: 2026-08-20T07:45:09.136Z
# Updated: 2026-08-20T07:45:09.136Z
# Updated: 2026-08-27T11:41:42.189Z
55 changes: 54 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ All implementations share the same design philosophy and provide feature parity.
- **C#**: `null`, `bool`, `int`, `long`, `float`, `double`, `string`, `List<object?>`, `Dictionary<string, object?>`
- Special float/number values: `NaN`, `Infinity`, `-Infinity`
- **Readable by Default**: In every language `encode()` writes indented, plain-text Links Notation; the previous single-line base64 form stays available as `encode_compact()` (alias `encode_obfuscated()`)
- **One Record per Line**: `encode_line()` writes the same readable document on one line and `decode_line()` reads it back exactly, so an append-only log stays greppable, tailable and countable by `wc -l`
- **Object Identity**: Shared references and circular references are preserved by the compact format via object ids; the readable format is a plain tree and raises a circular-reference error instead
- **Full Unicode**: Strings are written as text; only a value that cannot be written as text (one holding control characters) is base64-encoded, and it is marked individually as `(base64 "…")`
- **Opt-in Tracing**: Set `LINO_CODEC_DEBUG=1` to trace encoding and decoding, the same way in every language
Expand Down Expand Up @@ -115,8 +116,37 @@ assert_eq!(decoded, data);
)
```

For an append-only log, `encode_line()` writes the same document on one line:

```lino
(o: (name "Alice") (age 30) (active true))
```

```python
from link_notation_objects_codec import encode_line, decode_line

decode_line(encode_line(data)) == data
```

```javascript
import { encodeLine, decodeLine } from "lino-objects-codec";

decodeLine({ notation: encodeLine({ obj: data }) });
```

```rust
use lino_objects_codec::{decode_line, encode_line};

assert_eq!(decode_line(&encode_line(&data)).unwrap(), data);
```

```csharp
var line = Codec.EncodeLine(data);
var record = Codec.DecodeLine(line);
```

The single-line base64 form is still available as `encode_compact()` (alias
`encode_obfuscated()`) in every language, and `decode()` accepts both forms.
`encode_obfuscated()`) in every language, and `decode()` accepts all three forms.

### C#

Expand Down Expand Up @@ -395,6 +425,29 @@ object, bare-value lines make an array:
- The four languages produce byte-identical output, checked by the shared
fixtures in [`fixtures/readable-format/cases.json`](fixtures/readable-format/cases.json)

### Single-line format (`encode_line`)

The same readable document written on one line, so an append-only log holds one
record per line — appending is one write, compaction cuts at a newline, and
`grep`, `tail -f` and `wc -l` all treat a line as one event:

```lino
(o: (bytes 2827) (complete true) (server (o: (host "127.0.0.1") (port 18878))))
```

- An object is `(o: (key value) …)` and an empty object is `(o:)`
- An array is `(value …)` and an empty array is `()`
- Scalars and strings are written exactly as in the indented form, so a string
keeps its own characters and a number keeps its type
- The `o` marker is what removes the ambiguity a flat layout otherwise has:
without it `((key value))` reads both as a one-pair object and as an array
holding a two-element array. With it a bare `( )` on one line is always an
array, so a *hand-written* `(a 1)` is the two-element array — on one line,
objects say so
- `decode()` reads this form too, so a log reader needs no flag saying which form
a file holds; `decode_line()` is the exact inverse of `encode_line()` and
rejects input spanning more than one line

### Compact format (`encode_compact`)

The previous single-line form, kept for compatibility and for the object graphs
Expand Down
23 changes: 23 additions & 0 deletions csharp/.changeset/20260827_090000_issue_43_single_line_format.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
'Lino.Objects.Codec': minor
---

Add `Codec.EncodeLine` and `Codec.DecodeLine` (and the matching `ObjectCodec`
methods): the readable format written on one line, so an append-only log holds
one record per line. Appending is one write, compaction cuts at a newline, and
`grep`, `tail -f` and `wc -l` treat a line as one event. The output is valid
Links Notation, keeps numbers, booleans and `null` bare so types survive the
round trip, and `Decode(EncodeLine(v))` equals `Decode(Encode(v))`.

`Readable.ObjectMarker` (`o`) tells an object from an array on one line:
`(o: (bytes 2827) (complete true))` is a record, `("a" 1)` is a two-element
array, `(o:)` is the empty object and `()` the empty array. Because the marker
is part of the notation, the empty key round-trips as `(o: ("" 2))`. The
single-line spelling of every shared fixture is pinned in
`fixtures/readable-format/cases.json`, so all four languages write the same
bytes.

Also fixes `IsCompactNotation`, which used to claim a readable single-line
document such as `(null 1)`; the document `(null)` stays the compact null so
older documents keep decoding. See
[issue #43](https://github.com/link-foundation/lino-objects-codec/issues/43).
49 changes: 49 additions & 0 deletions csharp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ A C# library for working with Links Notation format. This library provides unive
- Collections: `List<object?>`, `Dictionary<string, object?>`
- Special float values: `NaN`, `Infinity`, `-Infinity`
- **Readable by Default**: `Codec.Encode()` writes plain, indented text that can be read and reviewed
- **One Record per Line**: `Codec.EncodeLine()` writes the same document on one line and `Codec.DecodeLine()` reads it back exactly, so an append-only log stays greppable, tailable and countable by `wc -l`
- **Object Identity**: Shared references and circular references are preserved by the compact format (`Codec.EncodeCompact`) via object ids
- **Full Unicode**: Strings are written as text; only a value that cannot be written as text (one holding control characters) is base64-encoded, and it is marked individually as `(base64 "…")`
- **Opt-in Tracing**: Set `LINO_CODEC_DEBUG=1` to trace encoding and decoding, the same way in every language
Expand Down Expand Up @@ -153,6 +154,7 @@ decoded = Codec.Decode(Codec.Encode(complexData));
| --- | --- |
| `Codec.Encode(obj)` | Readable, indented Links Notation (the default) |
| `Codec.Encode(obj, "\t")` | Same, with a custom indentation string |
| `Codec.EncodeLine(obj)` | The same readable document on one line, for append-only logs |
| `Codec.EncodeCompact(obj)` | The previous single-line, base64 form |
| `Codec.EncodeObfuscated(obj)` | Alias of `Codec.EncodeCompact` |

Expand Down Expand Up @@ -217,6 +219,25 @@ bare-value lines make a list:
base64-encoded on its own and marked as `(base64 "bGluZTEKbGluZTI=")`;
everything around it stays readable

### Single-line format (`Codec.EncodeLine`)

The same readable document on one line, so an append-only log holds one record
per line -- appending is one write, compaction cuts at a newline, and `grep`,
`tail -f` and `wc -l` all treat a line as one event:

```lino
(o: (bytes 2827) (complete true) (server (o: (host "127.0.0.1") (port 18878))))
```

- A dictionary is `(o: (key value) …)` and an empty dictionary is `(o:)`
- A list is `(value …)` and an empty list is `()`
- Scalars and strings are written exactly as in the indented form
- The `o` marker removes the ambiguity a flat layout otherwise has: a bare `( )`
on one line is always a list, so a *hand-written* `(a 1)` is the two-element
list, not the one-pair dictionary
- `Codec.Decode` reads this form too; `Codec.DecodeLine` is its exact inverse and
rejects input spanning more than one line

### Compact format (`Codec.EncodeCompact`)

The previous single-line form, kept for compatibility and for the object graphs
Expand Down Expand Up @@ -282,6 +303,34 @@ Decode Links Notation format to a C# object.
**Throws:**
- `InvalidOperationException` - If the type marker is unknown

#### `Codec.EncodeLine(object? obj)`

Encode a C# object into the readable format on one line.

**Parameters:**
- `obj` - The C# object to encode (can be null)

**Returns:**
- String representation in readable Links Notation format, holding no newline

```csharp
Codec.EncodeLine(new Dictionary<string, object?> { ["age"] = 30 }); // (o: (age 30))
```

#### `Codec.DecodeLine(string notation)`

Decode one line of a readable Links Notation log. The exact inverse of
`Codec.EncodeLine`.

**Parameters:**
- `notation` - One line written by `Codec.EncodeLine`

**Returns:**
- Reconstructed C# object (or null)

**Throws:**
- `FormatException` - If the input spans more than one line or is malformed

### ObjectCodec Class

The main codec class that performs encoding and decoding. The static `Codec` class creates a new instance for each operation to ensure thread safety.
Expand Down
35 changes: 35 additions & 0 deletions csharp/examples/BasicUsage.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

using System;
using System.Collections.Generic;
using System.Linq;
using Lino.Objects.Codec;

Console.WriteLine("=== lino-objects-codec C# Basic Usage Example ===\n");
Expand Down Expand Up @@ -146,4 +147,38 @@

Console.WriteLine();

// An append-only log wants one record per line: appending is one write, a
// compactor can cut the file at any newline, and `grep`, `tail -f` and `wc -l`
// all treat one line as one event.
Console.WriteLine("Append-only log, one record per line:");

Dictionary<string, object?> Record(string phase, int bytes, bool complete) => new()
{
{ "phase", phase },
{ "bytes", bytes },
{ "complete", complete }
};

var entries = new[]
{
Record("stream_start", 0, false),
Record("stream_chunk", 1024, false),
Record("stream_end", 2827, true)
};
var log = string.Concat(entries.Select(entry => Codec.EncodeLine(entry) + "\n"));
Console.Write(log);

var lines = log.TrimEnd('\n').Split('\n');
Console.WriteLine($" records: {lines.Length}");

// Reading: a line reader hands over one record at a time.
var lastRecord = Codec.DecodeLine(lines[^1]) as Dictionary<string, object?>;
Console.WriteLine($" last record phase: {lastRecord?["phase"]}");

// Filtering: the text stays readable, so plain string tools still work.
var finished = lines.Count(line => line.Contains("(complete true)", StringComparison.Ordinal));
Console.WriteLine($" finished records: {finished}");

Console.WriteLine();

Console.WriteLine("=== Example completed successfully! ===");
84 changes: 72 additions & 12 deletions csharp/src/Lino.Objects.Codec/ObjectCodec.cs
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,26 @@ private void FindObjectsNeedingIds(object? obj, Dictionary<object, bool>? seen =
/// <returns>String representation in readable Links Notation format</returns>
public string Encode(object? obj, string indent) => Readable.Encode(obj, indent);

/// <summary>
/// Encode a C# object into the readable format on one line.
/// </summary>
/// <remarks>
/// The result holds no newline, so an append-only log keeps one record per
/// line and stays greppable, tailable and countable by <c>wc -l</c>. See
/// <see cref="Readable"/> for the shape.
/// </remarks>
/// <param name="obj">The C# object to encode</param>
/// <returns>String representation in readable Links Notation format, on one line</returns>
public string EncodeLine(object? obj) => Readable.EncodeLine(obj);

/// <summary>
/// Decode one line of a readable Links Notation log back into a C# object.
/// </summary>
/// <remarks>The exact inverse of <see cref="EncodeLine"/>.</remarks>
/// <param name="notation">One line written by <see cref="EncodeLine"/></param>
/// <returns>Reconstructed C# object</returns>
public object? DecodeLine(string notation) => Readable.DecodeLine(notation);

/// <summary>
/// Encode a C# object to the compact Links Notation format.
/// </summary>
Expand Down Expand Up @@ -654,6 +674,18 @@ private Link<string> EncodeValue(object? obj, HashSet<object>? visited = null, i
"array", TypeList, "object", TypeDict,
};

/// <summary>
/// Markers a compact document writes without a payload.
/// </summary>
/// <remarks>
/// <c>(null)</c> is a compact null, while <c>(null 1)</c> is a readable line
/// holding two values, so the marker alone does not decide the format.
/// </remarks>
private static readonly HashSet<string> EmptyBodyMarkers = new(StringComparer.Ordinal)
{
TypeNull, "None",
};

/// <summary>
/// Whether a document is in the compact format.
/// </summary>
Expand All @@ -677,27 +709,41 @@ public static bool IsCompactNotation(string notation)
return false;
}

var tokens = firstLine[1..]
.Split(new[] { ' ', '\t', '\r', '(', ')' }, StringSplitOptions.RemoveEmptyEntries);

if (tokens.Length == 0)
{
return false;
}

var marker = tokens[0];
var (marker, rest) = SplitToken(firstLine[1..].TrimStart());

// Skip the `obj_N:` definition id, if present.
if (marker.EndsWith(':'))
{
if (!marker[..^1].StartsWith("obj_", StringComparison.Ordinal) || tokens.Length < 2)
if (!marker.StartsWith("obj_", StringComparison.Ordinal))
{
return false;
}
marker = tokens[1];
(marker, rest) = SplitToken(rest.TrimStart());
}

if (!CompactTypeMarkers.Contains(marker))
{
return false;
}

return CompactTypeMarkers.Contains(marker);
// A compact `null` carries no payload, so `(null)` is a compact document
// while `(null 1)` is a readable line holding two values.
if (EmptyBodyMarkers.Contains(marker))
{
return rest.TrimStart().StartsWith(')');
}

return true;
}

/// <summary>
/// Split off the first token of a line: everything up to the next whitespace
/// or parenthesis, plus what follows it.
/// </summary>
private static (string Token, string Remainder) SplitToken(string input)
{
var end = input.IndexOfAny(new[] { ' ', '\t', '\r', '\n', '(', ')' });
return end < 0 ? (input, string.Empty) : (input[..end], input[end..]);
}
}

Expand Down Expand Up @@ -754,6 +800,20 @@ public static class Codec
/// <returns>Reconstructed C# object</returns>
public static object? Decode(string notation) => new ObjectCodec().Decode(notation);

/// <summary>
/// Encode an object into the readable format on one line.
/// </summary>
/// <param name="obj">The C# object to encode</param>
/// <returns>String representation in readable Links Notation format, on one line</returns>
public static string EncodeLine(object? obj) => new ObjectCodec().EncodeLine(obj);

/// <summary>
/// Decode one line of a readable Links Notation log.
/// </summary>
/// <param name="notation">One line written by <see cref="EncodeLine"/></param>
/// <returns>Reconstructed C# object</returns>
public static object? DecodeLine(string notation) => new ObjectCodec().DecodeLine(notation);

/// <summary>
/// Decode the compact Links Notation format to a C# object.
/// </summary>
Expand Down
Loading
Loading