diff --git a/CHANGELOG.md b/CHANGELOG.md
index c26b080..aa1891d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -20,6 +20,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`PclReadOptions.Strict` makes such a read fail with `InvalidDataException` instead.
- The point count of a FARO scan is now the number of valid returns, which is what a full read returns. The size of the
scan grid is available as `FaroScan.GridSize`.
+- The layout of the samples of a FARO scan folder is derived from the size of its data stream and the size of the grid, and
+ the scale of the intensity from the values that are stored. The warning about an assumed layout is only made when the
+ sizes do not fit.
## [0.1.0] - 2026-11-15
diff --git a/README.md b/README.md
index e9f6c0c..a59d4c7 100644
--- a/README.md
+++ b/README.md
@@ -39,7 +39,7 @@ The library targets .NET 8 and .NET 10. It is managed code with no dependencies
| ReCap scan | `.rcs` | Experimental. The header is read; point decoding covers only the node layout of major version 3, and any other layout is rejected. Colour and intensity are decoded, and normals when a node has them. `HasColors`, `HasIntensities` and `HasNormals` are fixed for this layout and are not read from the file. |
| ReCap project | `.rcp` | Lists the scans of the project manifest and opens the `.rcs` files it finds next to the project. Node transforms are exposed as data and are not applied to points. |
| Leica scan container | `.lgsx`, `.lgs` | Experimental. Points are decoded from the compressed point blocks: position, and colour, intensity and normals when the block has them. When the container assigns its points to its setups, there is one scan per setup, named after the setup, with the pose and the bounds of the setup when its record holds them; the pose is applied unless `ApplyTransform` is false. When the container does not assign its points, all points of the container form one scan and a warning says so. The point count is taken from the container when it states one; otherwise it is found by decoding the data, which is slow on large files. |
-| FARO scan | `.fls`, or a FARO scan folder | Experimental. The scan grid and angles are read from the stored parameters and points are projected from the range grid; a file without them fails to open. The point count is the number of valid returns, which is counted on the first access; `FaroScan.GridSize` is the size of the scan grid. Folder scans are read with an assumed record layout and intensity scale (a warning says so). |
+| FARO scan | `.fls`, or a FARO scan folder | Experimental. The scan grid and angles are read from the stored parameters and points are projected from the range grid; a file without them fails to open. The point count is the number of valid returns, which is counted on the first access; `FaroScan.GridSize` is the size of the scan grid. For a scan folder the layout of the samples is derived from the size of the data stream and the size of the grid, and the scale of the intensity from the values that are stored; when the sizes do not fit, the layout is assumed and a warning says so. |
| PinPoint project | `.srf`, `.smf`, or a folder with `ScanProject.json` or `.smf`/`.srf` files | Experimental. The manifest (scan names, ids, poses, counts) is read. Points come from the referenced `.bin` files, in either of two layouts: a table of named channels, or numbered sections with a property table and Snappy-compressed channels, which PclIO decodes itself. Colour, intensity and normals are reported only when the file has that channel; in the second layout only positions, colours and intensities are read, and the per-point flags and grid indices stored beside them are not interpreted, so every stored point that passes the plausibility rule below is returned. A manifest with a scan position list takes the point count from the manifest, else from the data file; a manifest with a data object list always reports 0 because that count is not known. |
| EdgeWise | `.c3db`, `.c3ptx` | Experimental. A `.c3db` point database lists the scan clusters and a `.c3ptx` is one cluster file; the cluster files give positions only. The scale of the cluster positions is not stored in the files, so it is an assumption of the reader and positions are only as exact as it. |
@@ -84,7 +84,9 @@ These readers drop data without an error or a warning, so a scan can yield fewer
- Every exporter, and the `points` command with `--format xyz` or `pts`, skips a point with a NaN or infinite coordinate; the count is reported (see Exporters).
- FARO: a grid cell without a valid return, or with a distance outside 0.1 m to 350 m, is not read. The point count of the
- scan is the number of the cells that are read, so it matches what a full read returns.
+ scan is the number of the cells that are read, so it matches what a full read returns. A scan folder whose data stream does
+ not hold a whole number of samples for the grid is read with the assumed layout of eight bytes per sample, and a warning
+ says so.
- PinPoint: a point with a NaN or infinite coordinate, or a coordinate whose magnitude exceeds 5000 in the file's own unit, is not
returned. This is a plausibility rule of the reader, so coordinates in a projected system with large offsets are dropped. A
data file in neither layout, or in the sectioned layout with an element type or header value the reader does not support, cannot
diff --git a/docs/public-api.md b/docs/public-api.md
index e5b2acd..6aa00df 100644
--- a/docs/public-api.md
+++ b/docs/public-api.md
@@ -63,7 +63,7 @@ drop points that fail a plausibility rule (see the README), so a scan can yield
| ReCap scan | Declared in the header. It is not checked against the node table. |
| ReCap project | Declared by the project manifest; 0 when the manifest states none. |
| Leica | The number of points the container states, when it states one; otherwise it is computed. The computed value needs the first access to decode all of the point data and caches the result, which can be slow on large files; the access throws `InvalidDataException` for corrupt data. A container that assigns its points to its setups has one scan per setup, each with the id and the name of its setup; otherwise the container is one scan with the id `unified-cloud`. The `Transform` and the `Bounds` of a setup are read from its record and are empty when the record does not hold them. |
-| FARO | The number of valid returns, which is counted on the first access by reading the distance data of the scan; `FaroScan.GridSize` is the size of the scan grid (rows times columns). For a folder scan the grid size is limited by the size of the data stream. |
+| FARO | The number of valid returns, which is counted on the first access by reading the distance data of the scan; `FaroScan.GridSize` is the size of the scan grid (rows times columns). For a folder scan the grid size is limited by the size of the data stream, and `FaroScan.RecordSize` is the size of one sample in bytes. |
| PinPoint | A manifest with a scan position list: declared by the manifest, else derived from the data file, else 0 (not known). A manifest with a data object list: always 0 (not known), so `TotalPointCount` is 0 for such a project. |
| EdgeWise | A `.c3db` database node: the count the database declares. A `.c3ptx` file: computed by walking the cluster headers when the file is opened. |
diff --git a/src/PclIO/Formats/Faro/FaroFlsDocument.cs b/src/PclIO/Formats/Faro/FaroFlsDocument.cs
index 016197e..becb23f 100644
--- a/src/PclIO/Formats/Faro/FaroFlsDocument.cs
+++ b/src/PclIO/Formats/Faro/FaroFlsDocument.cs
@@ -67,7 +67,8 @@ private FaroFlsDocument(
string? dataStreamPath,
bool isOle2,
bool hasColors,
- bool hasIntensities)
+ bool hasIntensities,
+ int recordSize = FaroScan.DefaultRecordSize)
: base(path, fileSize)
{
ScannerModel = g.Model ?? Unknown;
@@ -100,7 +101,8 @@ private FaroFlsDocument(
isOle2: isOle2,
ole2Path: isOle2 ? path : null,
hasColors: hasColors,
- hasIntensities: hasIntensities));
+ hasIntensities: hasIntensities,
+ recordSize: recordSize));
}
/// Opens a FARO scan container file or scan folder and reads its scan parameters.
@@ -128,13 +130,29 @@ public static FaroFlsDocument Open(string path, PclReadOptions? options = null)
string? streamPath = FindFile(rootDir, "ScanDataStream0");
long size = streamPath != null ? new FileInfo(streamPath).Length : 0;
- // Never claim more samples than the stream can hold.
+ // The layout of the samples is derived from the sizes of the files when they fit.
long cells = (long)g.Rows * g.Cols;
- long pointCount = streamPath != null ? Math.Min(cells, size / FaroScan.RecordSize) : cells;
+ int recordSize = FaroScan.DetermineRecordSize(size, cells, out bool layoutKnown);
+
+ // Never claim more samples than the stream can hold.
+ long pointCount = streamPath != null ? Math.Min(cells, size / recordSize) : cells;
- var document = new FaroFlsDocument(path, size, g, pointCount, streamPath, isOle2: false, hasColors: false, hasIntensities: true);
+ var document = new FaroFlsDocument(path, size, g, pointCount, streamPath, isOle2: false, hasColors: false, hasIntensities: true,
+ recordSize: recordSize);
if (streamPath != null)
- document._warnings.Add("Folder scan samples are read with an assumed record layout and intensity scale.");
+ {
+ if (layoutKnown)
+ {
+ document._metadata["RecordSize"] = recordSize.ToString(CultureInfo.InvariantCulture);
+ document._warnings.Add(
+ $"The record layout of the folder scan was derived from the size of its data stream: {recordSize} bytes per sample.");
+ }
+ else
+ {
+ document._warnings.Add("Folder scan samples are read with an assumed record layout and intensity scale.");
+ }
+ }
+
return document;
}
@@ -251,8 +269,17 @@ private static double RequireAngle(string text, string key, string label)
/// Scan of a FARO container file or scan folder, read as a regular spherical grid.
public sealed class FaroScan : PclScan
{
- // Assumed record size of folder scan streams; the layout is not declared by the files.
- internal const int RecordSize = 8;
+ // Record size of folder scan streams that the sizes of the files do not reveal.
+ internal const int DefaultRecordSize = 8;
+
+ // A folder scan stream holds one record per cell of the grid; a record is never longer than this.
+ private const int MaxRecordSize = 64;
+
+ // The intensity of a folder scan is stored in a 16-bit field. Values that fit in 12 bits are the 12-bit
+ // intensities of the scanner, which the format scales by 2048; wider values are 16-bit intensities.
+ private const ushort MaxTwelveBitIntensity = 4095;
+ private const double TwelveBitScale = 2048.0;
+ private const double SixteenBitScale = 65535.0;
// Container streams may start with a short text header ending in an opening brace; look for it near the start only.
private const int HeaderSearchWindow = 256;
@@ -271,8 +298,11 @@ public sealed class FaroScan : PclScan
private readonly string? _ole2Path;
private readonly bool _hasColors;
private readonly bool _hasIntensities;
+ private readonly int _recordSize;
private readonly Lazy _validReturns;
+ private double _intensityScale = TwelveBitScale;
+ private bool _folderInspected;
///
public override string Id { get; }
@@ -290,6 +320,12 @@ public sealed class FaroScan : PclScan
/// Number of cells of the scan grid, also the number of samples the file holds.
public long GridSize { get; }
+ ///
+ /// Bytes that one sample of a folder scan takes. The files do not declare the layout; it is derived from the
+ /// size of the data stream and the size of the grid.
+ ///
+ public int RecordSize => _recordSize;
+
///
public override bool HasIntensities => _hasIntensities;
@@ -309,7 +345,8 @@ internal FaroScan(
bool isOle2 = false,
string? ole2Path = null,
bool hasColors = false,
- bool hasIntensities = false)
+ bool hasIntensities = false,
+ int recordSize = DefaultRecordSize)
{
if (rows <= 0 || cols <= 0)
throw new ArgumentOutOfRangeException(nameof(rows), "Scan grid dimensions must be positive.");
@@ -327,12 +364,37 @@ internal FaroScan(
_ole2Path = ole2Path;
_hasColors = hasColors;
_hasIntensities = hasIntensities;
+ _recordSize = recordSize;
// The count is made once even when several threads reach it at the same time, and the value is published
// whole, so a concurrent first access cannot observe a partial count.
_validReturns = new Lazy(CountValidReturns);
}
+ ///
+ /// The record layout of a folder scan is not declared by the files. It is derived from the size of the data
+ /// stream and the size of the grid: the stream holds one record per cell of the grid, so its size must be a
+ /// multiple of it. The assumed layout is used when the sizes do not fit.
+ ///
+ /// Length of the data stream in bytes.
+ /// Number of cells of the scan grid.
+ /// True when the layout was derived from the sizes.
+ /// Bytes that one sample takes.
+ internal static int DetermineRecordSize(long streamLength, long cells, out bool determined)
+ {
+ determined = false;
+ if (cells <= 0 || streamLength <= 0 || streamLength % cells != 0) return DefaultRecordSize;
+
+ long size = streamLength / cells;
+
+ // A record holds a 16-bit distance and a 16-bit intensity; a smaller derived size cannot hold both, and
+ // reading the intensity of its last record would run past the buffer. Use the assumed layout instead.
+ if (size < 4 || size > MaxRecordSize) return DefaultRecordSize;
+
+ determined = true;
+ return (int)size;
+ }
+
/// Streams the valid returns of the scan.
///
/// Container files supply distance, optional colour and optional grey (intensity 0..1) planes. Folder scans are
@@ -379,25 +441,32 @@ private long CountFolderReturns()
string streamPath = _streamPath ?? throw FormatError.Corrupted("scan data stream not found");
using var fs = File.OpenRead(streamPath);
- byte[] buffer = new byte[RecordSize * 8192];
+ byte[] buffer = new byte[_recordSize * 8192];
long count = 0;
long baseIdx = 0;
+ ushort maxIntensity = 0;
while (baseIdx < GridSize)
{
int read = fs.ReadAtLeast(buffer, buffer.Length, throwOnEndOfStream: false);
- int records = read / RecordSize;
+ int records = read / _recordSize;
if (records == 0) break;
for (int i = 0; i < records && baseIdx + i < GridSize; i++)
{
- int off = i * RecordSize;
+ int off = i * _recordSize;
+ ushort intensity = Bytes.ReadUInt16LE(buffer.AsSpan(off + 2, 2));
+ if (intensity > maxIntensity) maxIntensity = intensity;
if (IsValidDistance(buffer.AsSpan(off, 2))) count++;
}
baseIdx += records;
}
+ // The scale of the intensity follows from the values that are stored.
+ _intensityScale = maxIntensity > MaxTwelveBitIntensity ? SixteenBitScale : TwelveBitScale;
+ _folderInspected = true;
+
return count;
}
@@ -463,19 +532,22 @@ private IEnumerable ReadFolderPoints(PclReadOptions options)
{
string streamPath = _streamPath ?? throw FormatError.Corrupted("scan data stream not found");
+ // The scale of the intensity is known after the data has been looked at once.
+ if (!_folderInspected) _ = CountFolderReturns();
+
long target = options.MaxPoints ?? long.MaxValue;
int step = Math.Max(1, options.SubsampleStep);
long yielded = 0;
long baseIdx = 0;
using var fs = File.OpenRead(streamPath);
- byte[] buffer = new byte[RecordSize * 8192];
+ byte[] buffer = new byte[_recordSize * 8192];
// Every pass consumes at least one record or ends the loop, so scanning always terminates.
while (yielded < target && baseIdx < GridSize)
{
int read = fs.ReadAtLeast(buffer, buffer.Length, throwOnEndOfStream: false);
- int count = read / RecordSize;
+ int count = read / _recordSize;
if (count == 0) break;
for (int i = 0; i < count && baseIdx + i < GridSize; i++)
@@ -483,13 +555,13 @@ private IEnumerable ReadFolderPoints(PclReadOptions options)
long idx = baseIdx + i;
if (idx % step != 0) continue;
- int off = i * RecordSize;
+ int off = i * _recordSize;
if (!IsValidDistance(buffer.AsSpan(off, 2))) continue;
ushort intensityRaw = Bytes.ReadUInt16LE(buffer.AsSpan(off + 2, 2));
double range = Bytes.ReadUInt16LE(buffer.AsSpan(off, 2)) * 0.001;
- var pt = new PclPoint(Project(idx, range), Intensity: intensityRaw / 2048.0f, Index: (ulong)yielded);
+ var pt = new PclPoint(Project(idx, range), Intensity: (float)(intensityRaw / _intensityScale), Index: (ulong)yielded);
if (options.ApplyTransform)
pt = pt with { Position = Transform.TransformPoint(pt.Position) };
diff --git a/tests/PclIO.Tests/FormatTests.cs b/tests/PclIO.Tests/FormatTests.cs
index 4f929d9..5453c1d 100644
--- a/tests/PclIO.Tests/FormatTests.cs
+++ b/tests/PclIO.Tests/FormatTests.cs
@@ -888,8 +888,8 @@ public void Read_Synthetic_Faro_Scan_Folder()
Assert.Equal("TestModel", doc.ScannerModel);
Assert.Equal("SN-0001", doc.ScannerSerialNumber);
- // The record layout of a folder scan is assumed, and the document says so.
- Assert.Contains("assumed", Assert.Single(doc.Warnings), StringComparison.Ordinal);
+ // The record layout of a folder scan is derived from the size of its data stream, and the document says so.
+ Assert.Contains("derived", Assert.Single(doc.Warnings), StringComparison.Ordinal);
var scan = Assert.Single(doc.Scans);
Assert.Equal(SyntheticFixtures.FaroRows * SyntheticFixtures.FaroCols, scan.PointCount);
@@ -901,6 +901,90 @@ public void Read_Synthetic_Faro_Scan_Folder()
Assert.Equal(16, scan.ReadPoints(new PclReadOptions { SubsampleStep = 2 }).Count());
}
+ [Fact]
+ public void A_Faro_Scan_Folder_With_A_Larger_Sample_Reads_It_With_The_Layout_Of_Its_Files()
+ {
+ using var dir = new TempDirectory();
+ using var doc = FaroFlsDocument.Open(SyntheticFixtures.WriteFaroFolder(dir, recordBytes: 16));
+
+ var scan = Assert.IsType(Assert.Single(doc.Scans));
+ Assert.Equal(16, scan.RecordSize);
+ Assert.Equal("16", doc.Metadata["RecordSize"]);
+
+ var points = scan.ReadPoints().ToList();
+ Assert.Equal(32, points.Count);
+ AssertFaroPoints(points);
+ }
+
+ [Theory]
+ [InlineData(2)]
+ [InlineData(3)]
+ public void A_Faro_Scan_Folder_With_Samples_Too_Small_For_Distance_And_Intensity_Keeps_The_Assumed_Layout(int recordBytes)
+ {
+ // A derived record of two or three bytes cannot hold both the distance and the intensity; reading the
+ // intensity of the last record of a full buffer would run past it. The assumed layout is used instead.
+ using var dir = new TempDirectory();
+ using var doc = FaroFlsDocument.Open(SyntheticFixtures.WriteFaroFolder(dir, recordBytes: recordBytes));
+
+ var scan = Assert.IsType(Assert.Single(doc.Scans));
+ Assert.Equal(FaroScan.DefaultRecordSize, scan.RecordSize);
+ Assert.Contains("assumed", Assert.Single(doc.Warnings), StringComparison.Ordinal);
+
+ // The count and the read stay usable; they do not crash on the small records.
+ Assert.True(scan.PointCount >= 0);
+ Assert.Equal(scan.PointCount, scan.ReadPoints().Count());
+ }
+
+ [Theory]
+ [InlineData(2, false)]
+ [InlineData(3, false)]
+ [InlineData(4, true)]
+ [InlineData(16, true)]
+ [InlineData(64, true)]
+ [InlineData(65, false)]
+ public void The_Derived_Faro_Record_Size_Needs_Room_For_Distance_And_Intensity(int size, bool determined)
+ {
+ long cells = 8192;
+ int recordSize = FaroScan.DetermineRecordSize(cells * size, cells, out bool wasDetermined);
+ Assert.Equal(determined, wasDetermined);
+ Assert.Equal(determined ? size : FaroScan.DefaultRecordSize, recordSize);
+ }
+
+ [Fact]
+ public void A_Faro_Scan_Folder_With_Sixteen_Bit_Intensity_Uses_That_Scale()
+ {
+ using var dir = new TempDirectory();
+ using var doc = FaroFlsDocument.Open(SyntheticFixtures.WriteFaroFolder(dir, intensity: 60000));
+
+ var points = doc.Scans[0].ReadPoints().ToList();
+ Assert.Equal(32, points.Count);
+ Assert.All(points, p => Assert.Equal(60000 / 65535.0f, p.Intensity!.Value, 1e-6));
+ }
+
+ [Fact]
+ public void A_Faro_Scan_Folder_Whose_Sizes_Do_Not_Fit_The_Grid_Keeps_The_Assumed_Layout()
+ {
+ // Three bytes after the samples: the size of the stream is not a multiple of the size of the grid.
+ using var dir = new TempDirectory();
+ using var doc = FaroFlsDocument.Open(SyntheticFixtures.WriteFaroFolder(dir, extraBytes: 3));
+
+ Assert.Contains("assumed", Assert.Single(doc.Warnings), StringComparison.Ordinal);
+ Assert.Equal(32, doc.Scans[0].ReadPoints().Count());
+ }
+
+ [Fact]
+ public void A_Faro_Scan_Folder_With_A_Truncated_Stream_Yields_The_Samples_It_Holds()
+ {
+ // Only ten of the 32 cells of the grid are stored.
+ using var dir = new TempDirectory();
+ using var doc = FaroFlsDocument.Open(SyntheticFixtures.WriteFaroFolder(dir, samples: 10));
+
+ var scan = Assert.IsType(Assert.Single(doc.Scans));
+ Assert.Equal(10, scan.GridSize);
+ Assert.Equal(10, scan.PointCount);
+ Assert.Equal(10, scan.ReadPoints().Count());
+ }
+
[Fact]
public void Read_Synthetic_Faro_Container_File()
{
diff --git a/tests/PclIO.Tests/SyntheticFixtures.cs b/tests/PclIO.Tests/SyntheticFixtures.cs
index 6b7c88f..e3231a4 100644
--- a/tests/PclIO.Tests/SyntheticFixtures.cs
+++ b/tests/PclIO.Tests/SyntheticFixtures.cs
@@ -547,15 +547,27 @@ public static byte[] FaroDistances(ushort millimetres, int invalidCells = 0)
}
/// A synthetic scan folder with a parameter file and one data stream.
- public static string WriteFaroFolder(TempDirectory dir, string? parameters = null, ushort millimetres = 2000, int invalidCells = 0)
+ /// Folder to write into.
+ /// Content of the parameter file.
+ /// The distance of the cells that hold a return.
+ /// Number of cells at the start of the grid that hold no return.
+ /// The stored intensity of every cell.
+ /// Bytes that one sample takes.
+ /// Bytes after the samples, so that the sizes do not fit the grid.
+ /// Number of samples that are written; fewer than the grid leaves the stream short.
+ public static string WriteFaroFolder(TempDirectory dir, string? parameters = null, ushort millimetres = 2000, int invalidCells = 0,
+ ushort intensity = 1024, int recordBytes = 8, int extraBytes = 0, int samples = FaroRows * FaroCols)
{
dir.WriteText(Path.Combine("scan", "Scans", "1", "Params"), parameters ?? FaroParams());
- byte[] records = new byte[FaroRows * FaroCols * 8];
+ byte[] records = new byte[(samples * recordBytes) + extraBytes];
byte[] distances = FaroDistances(millimetres, invalidCells);
- for (int i = 0; i < FaroRows * FaroCols; i++)
+ for (int i = 0; i < samples; i++)
{
- Array.Copy(distances, i * 2, records, i * 8, 2);
- BinaryPrimitives.WriteUInt16LittleEndian(records.AsSpan((i * 8) + 2, 2), 1024);
+ int off = i * recordBytes;
+ Array.Copy(distances, i * 2, records, off, 2);
+ // A record smaller than four bytes holds the distance only; there is no room for the intensity.
+ if (recordBytes >= 4)
+ BinaryPrimitives.WriteUInt16LittleEndian(records.AsSpan(off + 2, 2), intensity);
}
dir.Write(Path.Combine("scan", "Scans", "1", "ScanDataStream0"), records);