Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
refers to, as for manifests with a scan position list. Locations that share one file each report its full count.
- The Snappy decoder now keeps a window of up to 8 MiB per stream (the declared uncompressed length, capped), instead of
a fixed 1 MiB, so streams with copies further back decode within bounded memory.
- Scaled-integer Cartesian coordinates of E57 scans are decoded (raw value times scale plus offset, with the binary width
from the range of the field).

## [0.1.0] - 2026-11-15

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The library targets .NET 8 and .NET 10. It is managed code with no dependencies

| Format | Extensions | Support level |
| --- | --- | --- |
| ASTM E57 | `.e57` | Points decoded for a defined subset: uncompressed bit-packed streams with single or double precision Cartesian coordinates, 8-bit colour and 8-bit intensity. Pose translation and rotation are applied. Scaled-integer coordinates, spherical coordinates and other codecs are not supported: a scan that uses them lists normally and fails when its points are read. Per-point validity flags and images are not read, so points flagged as invalid are returned as stored, and colour or intensity fields of any other type or range are ignored. Page checksums are verified. |
| ASTM E57 | `.e57` | Points decoded for a defined subset: uncompressed bit-packed streams with single or double precision Cartesian coordinates or scaled-integer coordinates (raw value times scale plus offset), 8-bit colour and 8-bit intensity. Pose translation and rotation are applied. Spherical coordinates and other codecs are not supported: a scan that uses them lists normally and fails when its points are read. Per-point validity flags and images are not read, so points flagged as invalid are returned as stored, and colour or intensity fields of any other type or range are ignored. Page checksums are verified. |
| ReCap scan | `.rcs` | Experimental. The header is read; point decoding covers the node layouts that are described in the code, which is the one of major version 3. A file of another version is listed with its header, but reading its points fails with a message that names the version. A further layout is described only when the files of that version may be used for the tests, so that its offsets can be verified. The channels of the points follow from the size of the payload of the nodes: colour and intensity when the records are long enough, and normals when the node holds the plane of them. |
| ReCap project | `.rcp` | Lists the scans of the project manifest and opens the `.rcs` files it finds next to the project. The transform of a scan node places the scan in the project and is applied to its points unless `ApplyTransform` is false; when the manifest states no transform, the transform of the scan file is used. |
| Leica scan container | `.lgsx`, `.lgs` | Experimental. Points are decoded from the compressed point blocks: position, and colour, intensity and normals when the block has them. When the container assigns its points to its setups, there is one scan per setup, named after the setup, with the pose and the bounds of the setup when its record holds them; the pose is applied unless `ApplyTransform` is false. When the container does not assign its points, all points of the container form one scan and a warning says so. The point count is taken from the container when it states one; otherwise it is found by decoding the data, which is slow on large files. |
Expand Down
119 changes: 108 additions & 11 deletions src/PclIO/Formats/E57/E57Document.cs
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ namespace PclIO.Formats.E57;
/// An ASTM E57 3D Imaging Data File (.e57).
/// Supported: the file header, the checksummed page layout, the XML section, scan names, ids, Cartesian bounds and
/// poses (translation and rotation quaternion), and points stored as uncompressed bit-packed streams where
/// Cartesian X/Y/Z are single or double precision floats and colour or intensity are Integer fields in the range 0..255.
/// Not supported (reported with <see cref="NotSupportedException"/> when points are read): scaled-integer or integer
/// coordinates, other codecs, and packets whose streams hold different record counts.
/// Cartesian X/Y/Z are single or double precision floats or scaled integers (raw value times scale plus offset),
/// and colour or intensity are Integer fields in the range 0..255.
/// Not supported (reported with <see cref="NotSupportedException"/> when points are read): integer coordinates,
/// other codecs, and packets whose streams hold different record counts.
/// Spherical coordinates, per-point invalid-state flags, images and 2D data are not interpreted.
/// </summary>
public sealed class E57Document : PclDocument
Expand Down Expand Up @@ -277,9 +278,21 @@ internal sealed class E57PointLayout
/// <summary>Stream index of Cartesian X, Y and Z.</summary>
public int[] Coordinate { get; } = [-1, -1, -1];

/// <summary>Bytes per record (4 single, 8 double) of the Cartesian streams.</summary>
/// <summary>Bytes per record of the Cartesian streams: 4 or 8 for floats, 1, 2, 4 or 8 for scaled integers.</summary>
public int[] CoordinateSize { get; } = new int[3];

/// <summary>True for the axes stored as scaled integers (raw value times scale plus offset).</summary>
public bool[] CoordinateScaled { get; } = new bool[3];

/// <summary>True for the scaled axes whose range is not negative, stored as unsigned integers.</summary>
public bool[] CoordinateUnsigned { get; } = new bool[3];

/// <summary>Scale of the scaled axes; 1 for float axes.</summary>
public double[] CoordinateScale { get; } = [1, 1, 1];

/// <summary>Offset of the scaled axes; 0 for float axes.</summary>
public double[] CoordinateOffset { get; } = new double[3];

/// <summary>Stream index of red, green and blue, or all -1 when colour is not decoded.</summary>
public int[] Color { get; } = [-1, -1, -1];

Expand Down Expand Up @@ -328,14 +341,33 @@ internal sealed class E57PointLayout

if (axis >= 0)
{
if (layout.Coordinate[axis] >= 0)
{
unsupportedReason = "a cartesian field is duplicated";
return null;
}

int size = FloatSize(field);
if (size == 0 || layout.Coordinate[axis] >= 0)
if (size != 0)
{
layout.Coordinate[axis] = i;
layout.CoordinateSize[axis] = size;
continue;
}

int scaledSize = ScaledIntegerSize(field, out double scale, out double offset, out bool unsigned);
if (scaledSize == 0)
{
unsupportedReason = "a cartesian field is duplicated or is not a single or double precision float";
unsupportedReason = "a cartesian field is neither a single or double precision float nor a scaled integer with a sized range";
return null;
}

layout.Coordinate[axis] = i;
layout.CoordinateSize[axis] = size;
layout.CoordinateSize[axis] = scaledSize;
layout.CoordinateScaled[axis] = true;
layout.CoordinateUnsigned[axis] = unsigned;
layout.CoordinateScale[axis] = scale;
layout.CoordinateOffset[axis] = offset;
}
else if (IsByteRange(field))
{
Expand Down Expand Up @@ -374,6 +406,46 @@ private static int FloatSize(XElement field)
};
}

// The binary size of a ScaledInteger field follows from its range, as the standard sizes it: the smallest of
// 1, 2, 4 or 8 bytes that holds every value from minimum to maximum, unsigned when the minimum is not negative
// and signed otherwise. 0 when the field is no ScaledInteger. Scale defaults to 1 and offset to 0.
private static int ScaledIntegerSize(XElement field, out double scale, out double offset, out bool unsigned)
{
scale = 1;
offset = 0;
unsigned = true;
if ((string?)field.Attribute("type") != "ScaledInteger") return 0;

if (!long.TryParse((string?)field.Attribute("minimum"), NumberStyles.Integer, CultureInfo.InvariantCulture, out long min) ||
!long.TryParse((string?)field.Attribute("maximum"), NumberStyles.Integer, CultureInfo.InvariantCulture, out long max))
throw FormatError.Corrupted("E57 scaled integer minimum or maximum is not a valid integer.");
if (min > max) throw FormatError.Corrupted("E57 scaled integer minimum exceeds its maximum.");

string? scaleText = (string?)field.Attribute("scale");
if (!string.IsNullOrWhiteSpace(scaleText) &&
(!double.TryParse(scaleText, NumberStyles.Float, CultureInfo.InvariantCulture, out scale) || !double.IsFinite(scale)))
throw FormatError.Corrupted("E57 scaled integer scale is not a finite number.");

string? offsetText = (string?)field.Attribute("offset");
if (!string.IsNullOrWhiteSpace(offsetText) &&
(!double.TryParse(offsetText, NumberStyles.Float, CultureInfo.InvariantCulture, out offset) || !double.IsFinite(offset)))
throw FormatError.Corrupted("E57 scaled integer offset is not a finite number.");

if (min >= 0)
{
if ((ulong)max <= 0xFF) return 1;
if ((ulong)max <= 0xFFFF) return 2;
if ((ulong)max <= 0xFFFFFFFF) return 4;
return 8;
}

unsigned = false;
if (min >= -0x80 && max <= 0x7F) return 1;
if (min >= -0x8000 && max <= 0x7FFF) return 2;
if (min >= -0x80000000L && max <= 0x7FFFFFFF) return 4;
return 8;
}

// An Integer in 0..255 is bit-packed into exactly one byte per record.
private static bool IsByteRange(XElement field) =>
(string?)field.Attribute("type") == "Integer" &&
Expand Down Expand Up @@ -502,10 +574,7 @@ public override IEnumerable<PclPoint> ReadPoints(PclReadOptions? options = null)
{
if ((consumed + i) % step != 0) continue;

var coords = new PclVector3(
ReadCoordinate(pkt, starts[layout.Coordinate[0]], layout.CoordinateSize[0], i),
ReadCoordinate(pkt, starts[layout.Coordinate[1]], layout.CoordinateSize[1], i),
ReadCoordinate(pkt, starts[layout.Coordinate[2]], layout.CoordinateSize[2], i));
var coords = new PclVector3(ReadAxis(layout, pkt, starts, 0, i), ReadAxis(layout, pkt, starts, 1, i), ReadAxis(layout, pkt, starts, 2, i));

PclColor? color = null;
if (layout.HasColor)
Expand Down Expand Up @@ -559,8 +628,36 @@ private static void Agree(ref int records, int count)
records = count;
}

private static double ReadAxis(E57PointLayout layout, byte[] packet, int[] starts, int axis, int index)
{
int stream = layout.Coordinate[axis];
int size = layout.CoordinateSize[axis];
if (!layout.CoordinateScaled[axis])
return ReadCoordinate(packet, starts[stream], size, index);

return ReadScaledInteger(packet, starts[stream], size, index,
layout.CoordinateUnsigned[axis], layout.CoordinateScale[axis], layout.CoordinateOffset[axis]);
}

private static double ReadCoordinate(byte[] packet, int start, int size, int index) =>
size == 4
? Bytes.ReadSingleLE(packet.AsSpan(start + index * 4, 4))
: Bytes.ReadDoubleLE(packet.AsSpan(start + index * 8, 8));

// A raw value of the sized width, signed or unsigned as the range requires; the value is raw times scale plus offset.
private static double ReadScaledInteger(byte[] packet, int start, int size, int index, bool unsigned, double scale, double offset)
{
double raw = (size, unsigned) switch
{
(1, true) => packet[start + index],
(1, false) => (sbyte)packet[start + index],
(2, true) => Bytes.ReadUInt16LE(packet.AsSpan(start + index * 2, 2)),
(2, false) => Bytes.ReadInt16LE(packet.AsSpan(start + index * 2, 2)),
(4, true) => Bytes.ReadUInt32LE(packet.AsSpan(start + index * 4, 4)),
(4, false) => Bytes.ReadInt32LE(packet.AsSpan(start + index * 4, 4)),
(8, true) => Bytes.ReadUInt64LE(packet.AsSpan(start + index * 8, 8)),
_ => Bytes.ReadInt64LE(packet.AsSpan(start + index * 8, 8)),
};
return raw * scale + offset;
}
}
2 changes: 2 additions & 0 deletions src/PclIO/Internal/Bytes.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ internal static class Bytes

public static int ReadInt32LE(ReadOnlySpan<byte> span) => BinaryPrimitives.ReadInt32LittleEndian(span);

public static long ReadInt64LE(ReadOnlySpan<byte> span) => BinaryPrimitives.ReadInt64LittleEndian(span);

public static float ReadSingleLE(ReadOnlySpan<byte> span) => BinaryPrimitives.ReadSingleLittleEndian(span);

public static double ReadDoubleLE(ReadOnlySpan<byte> span) => BinaryPrimitives.ReadDoubleLittleEndian(span);
Expand Down
90 changes: 86 additions & 4 deletions tests/PclIO.Tests/FormatTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -495,20 +495,102 @@ public void E57_Without_Colour_Or_Intensity_Reports_Them_As_Absent()
}

[Fact]
public void E57_With_Scaled_Integer_Coordinates_Opens_But_Is_Not_Decoded()
public void E57_With_Scaled_Integer_Coordinates_Is_Decoded()
{
const string prototype = """
<prototype type="Structure">
<cartesianX type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
<cartesianY type="ScaledInteger" minimum="-100000" maximum="100000" scale="0.01" offset="5"/>
<cartesianZ type="Float" precision="single"/>
</prototype>
""";
byte[] xs = [.. BitConverter.GetBytes((ushort)0), .. BitConverter.GetBytes((ushort)1000)];
byte[] ys = [.. BitConverter.GetBytes(-50000), .. BitConverter.GetBytes(25000)];
byte[] zs = [.. BitConverter.GetBytes(1.5f), .. BitConverter.GetBytes(-2.5f)];
var spec = new E57Spec { PrototypeXml = prototype, Points = [], RecordCount = 2, RawPackets = E57Spec.DataPacket([xs, ys, zs]) };
using var dir = new TempDirectory();
using var doc = E57Document.Open(dir.Write("scaled.e57", spec.Build()));

var scan = Assert.Single(doc.Scans);
var points = scan.ReadPoints(NoTransform).ToList();

Assert.Equal(2, points.Count);
AssertNear(new PclVector3(0, -495, 1.5), points[0].Position);
AssertNear(new PclVector3(1, 255, -2.5), points[1].Position);
}

[Fact]
public void E57_With_Scaled_Integer_Coordinates_Of_Several_Widths_Is_Decoded()
{
const string prototype = """
<prototype type="Structure">
<cartesianX type="ScaledInteger" minimum="-10" maximum="10" scale="0.5"/>
<cartesianY type="ScaledInteger" minimum="0" maximum="200000"/>
<cartesianZ type="ScaledInteger" minimum="-1099511627776" maximum="1099511627776"/>
</prototype>
""";
byte[] xs = [0xF6, 0x0A];
byte[] ys = [.. BitConverter.GetBytes(0u), .. BitConverter.GetBytes(200000u)];
byte[] zs = [.. BitConverter.GetBytes(5_000_000_000L), .. BitConverter.GetBytes(-7_000_000_000L)];
var spec = new E57Spec { PrototypeXml = prototype, Points = [], RecordCount = 2, RawPackets = E57Spec.DataPacket([xs, ys, zs]) };
using var dir = new TempDirectory();
using var doc = E57Document.Open(dir.Write("widths.e57", spec.Build()));

var points = doc.Scans[0].ReadPoints(NoTransform).ToList();

Assert.Equal(2, points.Count);
AssertNear(new PclVector3(-5, 0, 5_000_000_000), points[0].Position);
AssertNear(new PclVector3(5, 200000, -7_000_000_000), points[1].Position);
}

[Fact]
public void E57_With_A_Scaled_Integer_Range_That_Is_Empty_Is_Rejected()
{
const string prototype = """
<prototype type="Structure">
<cartesianX type="ScaledInteger" minimum="5" maximum="4" scale="0.001"/>
<cartesianY type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
<cartesianZ type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
</prototype>
""";
using var dir = new TempDirectory();
using var doc = E57Document.Open(dir.Write("scaled.e57", new E57Spec { PrototypeXml = prototype }.Build()));

Assert.Single(doc.Scans);
Assert.Throws<NotSupportedException>(() => doc.Scans[0].ReadPoints().ToList());
Assert.Throws<InvalidDataException>(() => E57Document.Open(dir.Write("scaled.e57", new E57Spec { PrototypeXml = prototype }.Build())));
}

[Fact]
public void E57_With_A_Scaled_Integer_Scale_That_Is_Not_Finite_Is_Rejected()
{
const string prototype = """
<prototype type="Structure">
<cartesianX type="ScaledInteger" minimum="0" maximum="1000" scale="NaN"/>
<cartesianY type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
<cartesianZ type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
</prototype>
""";
using var dir = new TempDirectory();

Assert.Throws<InvalidDataException>(() => E57Document.Open(dir.Write("scaled.e57", new E57Spec { PrototypeXml = prototype }.Build())));
}

[Fact]
public void E57_With_A_Partial_Scaled_Integer_Record_Is_Rejected()
{
const string prototype = """
<prototype type="Structure">
<cartesianX type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
<cartesianY type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
<cartesianZ type="ScaledInteger" minimum="0" maximum="1000" scale="0.001"/>
</prototype>
""";
byte[] xs = [0x00, 0x00, 0xE8];
byte[] ys = [.. BitConverter.GetBytes((ushort)0), .. BitConverter.GetBytes((ushort)1000)];
byte[] zs = [.. BitConverter.GetBytes((ushort)0), .. BitConverter.GetBytes((ushort)1000)];
var spec = new E57Spec { PrototypeXml = prototype, Points = [], RecordCount = 2, RawPackets = E57Spec.DataPacket([xs, ys, zs]) };
using var dir = new TempDirectory();
using var doc = E57Document.Open(dir.Write("partial.e57", spec.Build()));

Assert.Throws<InvalidDataException>(() => doc.Scans[0].ReadPoints().ToList());
}

// ---- Leica -----------------------------------------------------------------------------------------------
Expand Down
Loading