Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`PclReadOptions.Strict` makes such a read fail with `InvalidDataException` instead.
- The point count of a FARO scan is now the number of valid returns, which is what a full read returns. The size of the
scan grid is available as `FaroScan.GridSize`.
- The layout of the samples of a FARO scan folder is derived from the size of its data stream and the size of the grid, and
the scale of the intensity from the values that are stored. The warning about an assumed layout is only made when the
sizes do not fit.

## [0.1.0] - 2026-11-15

Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The library targets .NET 8 and .NET 10. It is managed code with no dependencies
| ReCap scan | `.rcs` | Experimental. The header is read; point decoding covers only the node layout of major version 3, and any other layout is rejected. Colour and intensity are decoded, and normals when a node has them. `HasColors`, `HasIntensities` and `HasNormals` are fixed for this layout and are not read from the file. |
| ReCap project | `.rcp` | Lists the scans of the project manifest and opens the `.rcs` files it finds next to the project. Node transforms are exposed as data and are not applied to points. |
| Leica scan container | `.lgsx`, `.lgs` | Experimental. Points are decoded from the compressed point blocks: position, and colour, intensity and normals when the block has them. When the container assigns its points to its setups, there is one scan per setup, named after the setup, with the pose and the bounds of the setup when its record holds them; the pose is applied unless `ApplyTransform` is false. When the container does not assign its points, all points of the container form one scan and a warning says so. The point count is taken from the container when it states one; otherwise it is found by decoding the data, which is slow on large files. |
| FARO scan | `.fls`, or a FARO scan folder | Experimental. The scan grid and angles are read from the stored parameters and points are projected from the range grid; a file without them fails to open. The point count is the number of valid returns, which is counted on the first access; `FaroScan.GridSize` is the size of the scan grid. Folder scans are read with an assumed record layout and intensity scale (a warning says so). |
| FARO scan | `.fls`, or a FARO scan folder | Experimental. The scan grid and angles are read from the stored parameters and points are projected from the range grid; a file without them fails to open. The point count is the number of valid returns, which is counted on the first access; `FaroScan.GridSize` is the size of the scan grid. For a scan folder the layout of the samples is derived from the size of the data stream and the size of the grid, and the scale of the intensity from the values that are stored; when the sizes do not fit, the layout is assumed and a warning says so. |
| PinPoint project | `.srf`, `.smf`, or a folder with `ScanProject.json` or `.smf`/`.srf` files | Experimental. The manifest (scan names, ids, poses, counts) is read. Points come from the referenced `.bin` files, in either of two layouts: a table of named channels, or numbered sections with a property table and Snappy-compressed channels, which PclIO decodes itself. Colour, intensity and normals are reported only when the file has that channel; in the second layout only positions, colours and intensities are read, and the per-point flags and grid indices stored beside them are not interpreted, so every stored point that passes the plausibility rule below is returned. A manifest with a scan position list takes the point count from the manifest, else from the data file; a manifest with a data object list always reports 0 because that count is not known. |
| EdgeWise | `.c3db`, `.c3ptx` | Experimental. A `.c3db` point database lists the scan clusters and a `.c3ptx` is one cluster file; the cluster files give positions only. The scale of the cluster positions is not stored in the files, so it is an assumption of the reader and positions are only as exact as it. |

Expand Down Expand Up @@ -84,7 +84,9 @@ These readers drop data without an error or a warning, so a scan can yield fewer

- Every exporter, and the `points` command with `--format xyz` or `pts`, skips a point with a NaN or infinite coordinate; the count is reported (see Exporters).
- FARO: a grid cell without a valid return, or with a distance outside 0.1 m to 350 m, is not read. The point count of the
scan is the number of the cells that are read, so it matches what a full read returns.
scan is the number of the cells that are read, so it matches what a full read returns. A scan folder whose data stream does
not hold a whole number of samples for the grid is read with the assumed layout of eight bytes per sample, and a warning
says so.
- PinPoint: a point with a NaN or infinite coordinate, or a coordinate whose magnitude exceeds 5000 in the file's own unit, is not
returned. This is a plausibility rule of the reader, so coordinates in a projected system with large offsets are dropped. A
data file in neither layout, or in the sectioned layout with an element type or header value the reader does not support, cannot
Expand Down
2 changes: 1 addition & 1 deletion docs/public-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ drop points that fail a plausibility rule (see the README), so a scan can yield
| ReCap scan | Declared in the header. It is not checked against the node table. |
| ReCap project | Declared by the project manifest; 0 when the manifest states none. |
| Leica | The number of points the container states, when it states one; otherwise it is computed. The computed value needs the first access to decode all of the point data and caches the result, which can be slow on large files; the access throws `InvalidDataException` for corrupt data. A container that assigns its points to its setups has one scan per setup, each with the id and the name of its setup; otherwise the container is one scan with the id `unified-cloud`. The `Transform` and the `Bounds` of a setup are read from its record and are empty when the record does not hold them. |
| FARO | The number of valid returns, which is counted on the first access by reading the distance data of the scan; `FaroScan.GridSize` is the size of the scan grid (rows times columns). For a folder scan the grid size is limited by the size of the data stream. |
| FARO | The number of valid returns, which is counted on the first access by reading the distance data of the scan; `FaroScan.GridSize` is the size of the scan grid (rows times columns). For a folder scan the grid size is limited by the size of the data stream, and `FaroScan.RecordSize` is the size of one sample in bytes. |
| PinPoint | A manifest with a scan position list: declared by the manifest, else derived from the data file, else 0 (not known). A manifest with a data object list: always 0 (not known), so `TotalPointCount` is 0 for such a project. |
| EdgeWise | A `.c3db` database node: the count the database declares. A `.c3ptx` file: computed by walking the cluster headers when the file is opened. |

Expand Down
104 changes: 88 additions & 16 deletions src/PclIO/Formats/Faro/FaroFlsDocument.cs
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@ private FaroFlsDocument(
string? dataStreamPath,
bool isOle2,
bool hasColors,
bool hasIntensities)
bool hasIntensities,
int recordSize = FaroScan.DefaultRecordSize)
: base(path, fileSize)
{
ScannerModel = g.Model ?? Unknown;
Expand Down Expand Up @@ -100,7 +101,8 @@ private FaroFlsDocument(
isOle2: isOle2,
ole2Path: isOle2 ? path : null,
hasColors: hasColors,
hasIntensities: hasIntensities));
hasIntensities: hasIntensities,
recordSize: recordSize));
}

/// <summary>Opens a FARO scan container file or scan folder and reads its scan parameters.</summary>
Expand Down Expand Up @@ -128,13 +130,29 @@ public static FaroFlsDocument Open(string path, PclReadOptions? options = null)
string? streamPath = FindFile(rootDir, "ScanDataStream0");
long size = streamPath != null ? new FileInfo(streamPath).Length : 0;

// Never claim more samples than the stream can hold.
// The layout of the samples is derived from the sizes of the files when they fit.
long cells = (long)g.Rows * g.Cols;
long pointCount = streamPath != null ? Math.Min(cells, size / FaroScan.RecordSize) : cells;
int recordSize = FaroScan.DetermineRecordSize(size, cells, out bool layoutKnown);

// Never claim more samples than the stream can hold.
long pointCount = streamPath != null ? Math.Min(cells, size / recordSize) : cells;

var document = new FaroFlsDocument(path, size, g, pointCount, streamPath, isOle2: false, hasColors: false, hasIntensities: true);
var document = new FaroFlsDocument(path, size, g, pointCount, streamPath, isOle2: false, hasColors: false, hasIntensities: true,
recordSize: recordSize);
if (streamPath != null)
document._warnings.Add("Folder scan samples are read with an assumed record layout and intensity scale.");
{
if (layoutKnown)
{
document._metadata["RecordSize"] = recordSize.ToString(CultureInfo.InvariantCulture);
document._warnings.Add(
$"The record layout of the folder scan was derived from the size of its data stream: {recordSize} bytes per sample.");
}
else
{
document._warnings.Add("Folder scan samples are read with an assumed record layout and intensity scale.");
}
}

return document;
}

Expand Down Expand Up @@ -251,8 +269,17 @@ private static double RequireAngle(string text, string key, string label)
/// <summary>Scan of a FARO container file or scan folder, read as a regular spherical grid.</summary>
public sealed class FaroScan : PclScan
{
// Assumed record size of folder scan streams; the layout is not declared by the files.
internal const int RecordSize = 8;
// Record size of folder scan streams that the sizes of the files do not reveal.
internal const int DefaultRecordSize = 8;

// A folder scan stream holds one record per cell of the grid; a record is never longer than this.
private const int MaxRecordSize = 64;

// The intensity of a folder scan is stored in a 16-bit field. Values that fit in 12 bits are the 12-bit
// intensities of the scanner, which the format scales by 2048; wider values are 16-bit intensities.
private const ushort MaxTwelveBitIntensity = 4095;
private const double TwelveBitScale = 2048.0;
private const double SixteenBitScale = 65535.0;

// Container streams may start with a short text header ending in an opening brace; look for it near the start only.
private const int HeaderSearchWindow = 256;
Expand All @@ -271,8 +298,11 @@ public sealed class FaroScan : PclScan
private readonly string? _ole2Path;
private readonly bool _hasColors;
private readonly bool _hasIntensities;
private readonly int _recordSize;

private readonly Lazy<long> _validReturns;
private double _intensityScale = TwelveBitScale;
private bool _folderInspected;

/// <inheritdoc />
public override string Id { get; }
Expand All @@ -290,6 +320,12 @@ public sealed class FaroScan : PclScan
/// <summary>Number of cells of the scan grid, also the number of samples the file holds.</summary>
public long GridSize { get; }

/// <summary>
/// Bytes that one sample of a folder scan takes. The files do not declare the layout; it is derived from the
/// size of the data stream and the size of the grid.
/// </summary>
public int RecordSize => _recordSize;

/// <inheritdoc />
public override bool HasIntensities => _hasIntensities;

Expand All @@ -309,7 +345,8 @@ internal FaroScan(
bool isOle2 = false,
string? ole2Path = null,
bool hasColors = false,
bool hasIntensities = false)
bool hasIntensities = false,
int recordSize = DefaultRecordSize)
{
if (rows <= 0 || cols <= 0)
throw new ArgumentOutOfRangeException(nameof(rows), "Scan grid dimensions must be positive.");
Expand All @@ -327,12 +364,37 @@ internal FaroScan(
_ole2Path = ole2Path;
_hasColors = hasColors;
_hasIntensities = hasIntensities;
_recordSize = recordSize;

// The count is made once even when several threads reach it at the same time, and the value is published
// whole, so a concurrent first access cannot observe a partial count.
_validReturns = new Lazy<long>(CountValidReturns);
}

/// <summary>
/// The record layout of a folder scan is not declared by the files. It is derived from the size of the data
/// stream and the size of the grid: the stream holds one record per cell of the grid, so its size must be a
/// multiple of it. The assumed layout is used when the sizes do not fit.
/// </summary>
/// <param name="streamLength">Length of the data stream in bytes.</param>
/// <param name="cells">Number of cells of the scan grid.</param>
/// <param name="determined">True when the layout was derived from the sizes.</param>
/// <returns>Bytes that one sample takes.</returns>
internal static int DetermineRecordSize(long streamLength, long cells, out bool determined)
{
determined = false;
if (cells <= 0 || streamLength <= 0 || streamLength % cells != 0) return DefaultRecordSize;

long size = streamLength / cells;

// A record holds a 16-bit distance and a 16-bit intensity; a smaller derived size cannot hold both, and
// reading the intensity of its last record would run past the buffer. Use the assumed layout instead.
if (size < 4 || size > MaxRecordSize) return DefaultRecordSize;

determined = true;
return (int)size;
}

/// <summary>Streams the valid returns of the scan.</summary>
/// <remarks>
/// Container files supply distance, optional colour and optional grey (intensity 0..1) planes. Folder scans are
Expand Down Expand Up @@ -379,25 +441,32 @@ private long CountFolderReturns()
string streamPath = _streamPath ?? throw FormatError.Corrupted("scan data stream not found");

using var fs = File.OpenRead(streamPath);
byte[] buffer = new byte[RecordSize * 8192];
byte[] buffer = new byte[_recordSize * 8192];

long count = 0;
long baseIdx = 0;
ushort maxIntensity = 0;
while (baseIdx < GridSize)
{
int read = fs.ReadAtLeast(buffer, buffer.Length, throwOnEndOfStream: false);
int records = read / RecordSize;
int records = read / _recordSize;
if (records == 0) break;

for (int i = 0; i < records && baseIdx + i < GridSize; i++)
{
int off = i * RecordSize;
int off = i * _recordSize;
ushort intensity = Bytes.ReadUInt16LE(buffer.AsSpan(off + 2, 2));
if (intensity > maxIntensity) maxIntensity = intensity;
if (IsValidDistance(buffer.AsSpan(off, 2))) count++;
}

baseIdx += records;
}

// The scale of the intensity follows from the values that are stored.
_intensityScale = maxIntensity > MaxTwelveBitIntensity ? SixteenBitScale : TwelveBitScale;
_folderInspected = true;

return count;
}

Expand Down Expand Up @@ -463,33 +532,36 @@ private IEnumerable<PclPoint> ReadFolderPoints(PclReadOptions options)
{
string streamPath = _streamPath ?? throw FormatError.Corrupted("scan data stream not found");

// The scale of the intensity is known after the data has been looked at once.
if (!_folderInspected) _ = CountFolderReturns();

long target = options.MaxPoints ?? long.MaxValue;
int step = Math.Max(1, options.SubsampleStep);
long yielded = 0;
long baseIdx = 0;

using var fs = File.OpenRead(streamPath);
byte[] buffer = new byte[RecordSize * 8192];
byte[] buffer = new byte[_recordSize * 8192];

// Every pass consumes at least one record or ends the loop, so scanning always terminates.
while (yielded < target && baseIdx < GridSize)
{
int read = fs.ReadAtLeast(buffer, buffer.Length, throwOnEndOfStream: false);
int count = read / RecordSize;
int count = read / _recordSize;
if (count == 0) break;

for (int i = 0; i < count && baseIdx + i < GridSize; i++)
{
long idx = baseIdx + i;
if (idx % step != 0) continue;

int off = i * RecordSize;
int off = i * _recordSize;
if (!IsValidDistance(buffer.AsSpan(off, 2))) continue;

ushort intensityRaw = Bytes.ReadUInt16LE(buffer.AsSpan(off + 2, 2));
double range = Bytes.ReadUInt16LE(buffer.AsSpan(off, 2)) * 0.001;

var pt = new PclPoint(Project(idx, range), Intensity: intensityRaw / 2048.0f, Index: (ulong)yielded);
var pt = new PclPoint(Project(idx, range), Intensity: (float)(intensityRaw / _intensityScale), Index: (ulong)yielded);
if (options.ApplyTransform)
pt = pt with { Position = Transform.TransformPoint(pt.Position) };

Expand Down
Loading
Loading