Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- The Leica reader counts the data it cannot use (the points that are dropped as padding, the compressed streams that are
skipped and the point data that ends at an unknown block) and reports it in the warnings of the document. The new
`PclReadOptions.Strict` makes such a read fail with `InvalidDataException` instead.
- The point count of a FARO scan is now the number of valid returns, which is what a full read returns. The size of the
scan grid is available as `FaroScan.GridSize`.

## [0.1.0] - 2026-11-15

Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The library targets .NET 8 and .NET 10. It is managed code with no dependencies
| ReCap scan | `.rcs` | Experimental. The header is read; point decoding covers only the node layout of major version 3, and any other layout is rejected. Colour and intensity are decoded, and normals when a node has them. `HasColors`, `HasIntensities` and `HasNormals` are fixed for this layout and are not read from the file. |
| ReCap project | `.rcp` | Lists the scans of the project manifest and opens the `.rcs` files it finds next to the project. Node transforms are exposed as data and are not applied to points. |
| Leica scan container | `.lgsx`, `.lgs` | Experimental. Points are decoded from the compressed point blocks: position, and colour, intensity and normals when the block has them. When the container assigns its points to its setups, there is one scan per setup, named after the setup, with the pose and the bounds of the setup when its record holds them; the pose is applied unless `ApplyTransform` is false. When the container does not assign its points, all points of the container form one scan and a warning says so. The point count is taken from the container when it states one; otherwise it is found by decoding the data, which is slow on large files. |
| FARO scan | `.fls`, or a FARO scan folder | Experimental. The scan grid and angles are read from the stored parameters and points are projected from the range grid; a file without them fails to open. The point count is the grid size, not a count of valid returns. Folder scans are read with an assumed record layout and intensity scale (a warning says so). |
| FARO scan | `.fls`, or a FARO scan folder | Experimental. The scan grid and angles are read from the stored parameters and points are projected from the range grid; a file without them fails to open. The point count is the number of valid returns, which is counted on the first access; `FaroScan.GridSize` is the size of the scan grid. Folder scans are read with an assumed record layout and intensity scale (a warning says so). |
| PinPoint project | `.srf`, `.smf`, or a folder with `ScanProject.json` or `.smf`/`.srf` files | Experimental. The manifest (scan names, ids, poses, counts) is read. Points come from the referenced `.bin` files, in either of two layouts: a table of named channels, or numbered sections with a property table and Snappy-compressed channels, which PclIO decodes itself. Colour, intensity and normals are reported only when the file has that channel; in the second layout only positions, colours and intensities are read, and the per-point flags and grid indices stored beside them are not interpreted, so every stored point that passes the plausibility rule below is returned. A manifest with a scan position list takes the point count from the manifest, else from the data file; a manifest with a data object list always reports 0 because that count is not known. |
| EdgeWise | `.c3db`, `.c3ptx` | Experimental. A `.c3db` point database lists the scan clusters and a `.c3ptx` is one cluster file; the cluster files give positions only. The scale of the cluster positions is not stored in the files, so it is an assumption of the reader and positions are only as exact as it. |

Expand Down Expand Up @@ -83,7 +83,8 @@ Where the readers look for the files a project refers to:
These readers drop data without an error or a warning, so a scan can yield fewer points than `PointCount`.

- Every exporter, and the `points` command with `--format xyz` or `pts`, skips a point with a NaN or infinite coordinate; the count is reported (see Exporters).
- FARO: a grid cell without a valid return, or with a distance outside 0.1 m to 350 m, is not read.
- FARO: a grid cell without a valid return, or with a distance outside 0.1 m to 350 m, is not read. The point count of the
scan is the number of the cells that are read, so it matches what a full read returns.
- PinPoint: a point with a NaN or infinite coordinate, or a coordinate whose magnitude exceeds 5000 in the file's own unit, is not
returned. This is a plausibility rule of the reader, so coordinates in a projected system with large offsets are dropped. A
data file in neither layout, or in the sectioned layout with an element type or header value the reader does not support, cannot
Expand Down
2 changes: 1 addition & 1 deletion docs/public-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ drop points that fail a plausibility rule (see the README), so a scan can yield
| ReCap scan | Declared in the header. It is not checked against the node table. |
| ReCap project | Declared by the project manifest; 0 when the manifest states none. |
| Leica | The number of points the container states, when it states one; otherwise it is computed. The computed value needs the first access to decode all of the point data and caches the result, which can be slow on large files; the access throws `InvalidDataException` for corrupt data. A container that assigns its points to its setups has one scan per setup, each with the id and the name of its setup; otherwise the container is one scan with the id `unified-cloud`. The `Transform` and the `Bounds` of a setup are read from its record and are empty when the record does not hold them. |
| FARO | Approximate: the size of the scan grid (rows times columns), not the number of valid returns. For a folder scan it is limited by the size of the data stream. |
| FARO | The number of valid returns, which is counted on the first access by reading the distance data of the scan; `FaroScan.GridSize` is the size of the scan grid (rows times columns). For a folder scan the grid size is limited by the size of the data stream. |
| PinPoint | A manifest with a scan position list: declared by the manifest, else derived from the data file, else 0 (not known). A manifest with a data object list: always 0 (not known), so `TotalPointCount` is 0 for such a project. |
| EdgeWise | A `.c3db` database node: the count the database declares. A `.c3ptx` file: computed by walking the cluster headers when the file is opened. |

Expand Down
112 changes: 93 additions & 19 deletions src/PclIO/Formats/Faro/FaroFlsDocument.cs
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ private FaroFlsDocument(
_scans.Add(new FaroScan(
id: ScannerSerialNumber,
name: Path.GetFileNameWithoutExtension(path),
pointCount: pointCount,
gridSize: pointCount,
rows: g.Rows,
cols: g.Cols,
rowStart: g.RowStart,
Expand Down Expand Up @@ -272,14 +272,23 @@ public sealed class FaroScan : PclScan
private readonly bool _hasColors;
private readonly bool _hasIntensities;

private readonly Lazy<long> _validReturns;

/// <inheritdoc />
public override string Id { get; }

/// <inheritdoc />
public override string Name { get; }

/// <summary>Declared grid size (limited by the available data for folder scans), not the count of valid returns.</summary>
public override long PointCount { get; }
/// <summary>
/// Number of points that a full read returns: the cells of the grid with a valid return. The count is made on
/// the first access, which reads the distance data of the scan.
/// </summary>
/// <exception cref="InvalidDataException">The data of the scan is missing or damaged.</exception>
public override long PointCount => _validReturns.Value;

/// <summary>Number of cells of the scan grid, also the number of samples the file holds.</summary>
public long GridSize { get; }

/// <inheritdoc />
public override bool HasIntensities => _hasIntensities;
Expand All @@ -290,7 +299,7 @@ public sealed class FaroScan : PclScan
internal FaroScan(
string id,
string name,
long pointCount,
long gridSize,
int rows,
int cols,
double rowStart,
Expand All @@ -307,7 +316,7 @@ internal FaroScan(

Id = id;
Name = name;
PointCount = pointCount;
GridSize = gridSize;
_rows = rows;
_cols = cols;
_rowStart = rowStart;
Expand All @@ -318,6 +327,10 @@ internal FaroScan(
_ole2Path = ole2Path;
_hasColors = hasColors;
_hasIntensities = hasIntensities;

// The count is made once even when several threads reach it at the same time, and the value is published
// whole, so a concurrent first access cannot observe a partial count.
_validReturns = new Lazy<long>(CountValidReturns);
}

/// <summary>Streams the valid returns of the scan.</summary>
Expand All @@ -333,6 +346,71 @@ public override IEnumerable<PclPoint> ReadPoints(PclReadOptions? options = null)
return _isOle2 ? ReadContainerPoints(options) : ReadFolderPoints(options);
}

// Counts the cells of the grid with a valid return; these are the points that a full read returns.
private long CountValidReturns()
{
long count = _isOle2 ? CountContainerReturns() : CountFolderReturns();
return count;
}

private long CountContainerReturns()
{
string path = _ole2Path ?? throw new InvalidOperationException("Container path is not set.");
long fileLength = new FileInfo(path).Length;

using var storage = Ole2Storage.Open(path);
using var dist = Plane.Open(storage, "Dist0", 2, GridSize, fileLength)
?? throw FormatError.Corrupted("distance stream not found");

long count = 0;
byte[] cell = new byte[2];
for (long idx = 0; idx < GridSize; idx++)
{
// A distance plane shorter than the grid ends the scan early.
if (!dist.TryRead(idx, cell)) break;
if (IsValidDistance(cell)) count++;
}

return count;
}

private long CountFolderReturns()
{
string streamPath = _streamPath ?? throw FormatError.Corrupted("scan data stream not found");

using var fs = File.OpenRead(streamPath);
byte[] buffer = new byte[RecordSize * 8192];

long count = 0;
long baseIdx = 0;
while (baseIdx < GridSize)
{
int read = fs.ReadAtLeast(buffer, buffer.Length, throwOnEndOfStream: false);
int records = read / RecordSize;
if (records == 0) break;

for (int i = 0; i < records && baseIdx + i < GridSize; i++)
{
int off = i * RecordSize;
if (IsValidDistance(buffer.AsSpan(off, 2))) count++;
}

baseIdx += records;
}

return count;
}

// A distance is usable when the cell holds a return and the range is inside the accepted one.
private static bool IsValidDistance(ReadOnlySpan<byte> cell)
{
ushort distMm = Bytes.ReadUInt16LE(cell);
if (distMm == 0 || distMm == 0xFFFF) return false;

double range = distMm * 0.001;
return range >= MinRange && range <= MaxRange;
}

private IEnumerable<PclPoint> ReadContainerPoints(PclReadOptions options)
{
string path = _ole2Path ?? throw new InvalidOperationException("Container path is not set.");
Expand All @@ -347,7 +425,7 @@ private IEnumerable<PclPoint> ReadContainerPoints(PclReadOptions options)
using var color = Plane.Open(storage, "Color0", 3, cells, fileLength);
using var grey = Plane.Open(storage, "Grey0", 1, cells, fileLength);

long target = options.MaxPoints ?? PointCount;
long target = options.MaxPoints ?? long.MaxValue;
int step = Math.Max(1, options.SubsampleStep);
long yielded = 0;

Expand All @@ -356,16 +434,14 @@ private IEnumerable<PclPoint> ReadContainerPoints(PclReadOptions options)
byte[] greyCell = new byte[1];

// Samples beyond the declared grid cannot be placed, so they are not read.
for (long idx = 0; idx < PointCount && yielded < target; idx += step)
for (long idx = 0; idx < GridSize && yielded < target; idx += step)
{
// A distance plane shorter than the grid ends the scan early.
if (!dist.TryRead(idx, distCell)) yield break;

ushort distMm = Bytes.ReadUInt16LE(distCell);
if (distMm == 0 || distMm == 0xFFFF) continue;
if (!IsValidDistance(distCell)) continue;

double range = distMm * 0.001;
if (range < MinRange || range > MaxRange) continue;
double range = Bytes.ReadUInt16LE(distCell) * 0.001;

// A colour or grey plane shorter than the grid leaves the remaining points without that channel.
PclColor? c = color != null && color.TryRead(idx, colorCell)
Expand All @@ -387,7 +463,7 @@ private IEnumerable<PclPoint> ReadFolderPoints(PclReadOptions options)
{
string streamPath = _streamPath ?? throw FormatError.Corrupted("scan data stream not found");

long target = options.MaxPoints ?? PointCount;
long target = options.MaxPoints ?? long.MaxValue;
int step = Math.Max(1, options.SubsampleStep);
long yielded = 0;
long baseIdx = 0;
Expand All @@ -396,24 +472,22 @@ private IEnumerable<PclPoint> ReadFolderPoints(PclReadOptions options)
byte[] buffer = new byte[RecordSize * 8192];

// Every pass consumes at least one record or ends the loop, so scanning always terminates.
while (yielded < target && baseIdx < PointCount)
while (yielded < target && baseIdx < GridSize)
{
int read = fs.ReadAtLeast(buffer, buffer.Length, throwOnEndOfStream: false);
int count = read / RecordSize;
if (count == 0) break;

for (int i = 0; i < count && baseIdx + i < PointCount; i++)
for (int i = 0; i < count && baseIdx + i < GridSize; i++)
{
long idx = baseIdx + i;
if (idx % step != 0) continue;

int off = i * RecordSize;
ushort distRaw = Bytes.ReadUInt16LE(buffer.AsSpan(off, 2));
ushort intensityRaw = Bytes.ReadUInt16LE(buffer.AsSpan(off + 2, 2));
if (distRaw == 0 || distRaw == 0xFFFF) continue;
if (!IsValidDistance(buffer.AsSpan(off, 2))) continue;

double range = distRaw * 0.001;
if (range < MinRange || range > MaxRange) continue;
ushort intensityRaw = Bytes.ReadUInt16LE(buffer.AsSpan(off + 2, 2));
double range = Bytes.ReadUInt16LE(buffer.AsSpan(off, 2)) * 0.001;

var pt = new PclPoint(Project(idx, range), Intensity: intensityRaw / 2048.0f, Index: (ulong)yielded);
if (options.ApplyTransform)
Expand Down
25 changes: 25 additions & 0 deletions tests/PclIO.Tests/FormatTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -929,6 +929,31 @@ public void Faro_Container_Without_A_Grey_Stream_Has_No_Intensity()
Assert.All(doc.Scans[0].ReadPoints(), p => Assert.Null(p.Intensity));
}

[Fact]
public void A_Faro_Scan_Reports_The_Number_Of_Points_That_A_Full_Read_Returns()
{
// Six of the 32 cells of the grid hold no return.
using var dir = new TempDirectory();
using var doc = PclFile.Open(dir.Write("scan.fls", SyntheticFixtures.CreateSyntheticFlsContainer(invalidCells: 6)));

var scan = Assert.IsType<FaroScan>(Assert.Single(doc.Scans));
Assert.Equal(32, scan.GridSize);
Assert.Equal(26, scan.PointCount);
Assert.Equal(scan.PointCount, scan.ReadPoints().LongCount());
}

[Fact]
public void A_Faro_Scan_Folder_Reports_The_Number_Of_Points_That_A_Full_Read_Returns()
{
using var dir = new TempDirectory();
using var doc = FaroFlsDocument.Open(SyntheticFixtures.WriteFaroFolder(dir, invalidCells: 6));

var scan = Assert.IsType<FaroScan>(Assert.Single(doc.Scans));
Assert.Equal(32, scan.GridSize);
Assert.Equal(26, scan.PointCount);
Assert.Equal(scan.PointCount, scan.ReadPoints().LongCount());
}

[Fact]
public void Faro_Without_Stored_Model_And_Serial_Reports_Them_As_Unknown_With_A_Warning()
{
Expand Down
26 changes: 20 additions & 6 deletions tests/PclIO.Tests/SyntheticFixtures.cs
Original file line number Diff line number Diff line change
Expand Up @@ -528,35 +528,49 @@ public static string FaroParams(string extra = "") => $"""
""";

/// <summary>Little-endian uint16 distances in millimetres for the whole grid.</summary>
public static byte[] FaroDistances(ushort millimetres)
/// <param name="millimetres">The distance of every cell that holds a return.</param>
/// <param name="invalidCells">
/// Number of cells at the start of the grid that hold no return: the first half of them hold the value 0 and
/// the second half the value 0xFFFF.
/// </param>
public static byte[] FaroDistances(ushort millimetres, int invalidCells = 0)
{
byte[] data = new byte[FaroRows * FaroCols * 2];
for (int i = 0; i < FaroRows * FaroCols; i++) BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(i * 2, 2), millimetres);
for (int i = 0; i < FaroRows * FaroCols; i++)
{
ushort value = millimetres;
if (i < invalidCells) value = i < (invalidCells / 2) ? (ushort)0 : (ushort)0xFFFF;
BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(i * 2, 2), value);
}

return data;
}

/// <summary>A synthetic scan folder with a parameter file and one data stream.</summary>
public static string WriteFaroFolder(TempDirectory dir, string? parameters = null, ushort millimetres = 2000)
public static string WriteFaroFolder(TempDirectory dir, string? parameters = null, ushort millimetres = 2000, int invalidCells = 0)
{
dir.WriteText(Path.Combine("scan", "Scans", "1", "Params"), parameters ?? FaroParams());
byte[] records = new byte[FaroRows * FaroCols * 8];
byte[] distances = FaroDistances(millimetres, invalidCells);
for (int i = 0; i < FaroRows * FaroCols; i++)
{
BinaryPrimitives.WriteUInt16LittleEndian(records.AsSpan(i * 8, 2), millimetres);
Array.Copy(distances, i * 2, records, i * 8, 2);
BinaryPrimitives.WriteUInt16LittleEndian(records.AsSpan((i * 8) + 2, 2), 1024);
}

dir.Write(Path.Combine("scan", "Scans", "1", "ScanDataStream0"), records);
return dir.PathOf("scan");
}

/// <summary>A single-file container (.fls) with the parameter, distance and grey streams of one scan.</summary>
public static byte[] CreateSyntheticFlsContainer(string? parameters = null, ushort millimetres = 2000, bool withGrey = true)
public static byte[] CreateSyntheticFlsContainer(string? parameters = null, ushort millimetres = 2000, bool withGrey = true,
int invalidCells = 0)
{
var builder = new Ole2Builder();
int scans = builder.AddStorage(0, "Scans");
int one = builder.AddStorage(scans, "1");
builder.AddStream(one, "Params", Encoding.ASCII.GetBytes(parameters ?? FaroParams()));
builder.AddStream(one, "Dist0", FaroDistances(millimetres));
builder.AddStream(one, "Dist0", FaroDistances(millimetres, invalidCells));
if (withGrey) builder.AddStream(one, "Grey0", Enumerable.Range(0, FaroRows * FaroCols).Select(i => (byte)(i * 8)).ToArray());
return builder.Build();
}
Expand Down
Loading