diff --git a/CHANGELOG.md b/CHANGELOG.md index fabf711..aa6ce71 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 1.2 with point format 2 stays the default and is byte-identical to before. A projected `EPSG:NNNN` coordinate system of the source is written as a GeoTIFF record for LAS 1.2 when asked; LAS 1.4 point formats 6 and above require WKT, which the exporter does not produce, so a 1.4 file carries no coordinate system record. +- LAS files are read (`PclFileKind.Las`): versions 1.0 to 1.4 with point formats 0 to 10, the legacy and the extended + 1.4 point counts, scale and offset, and the coordinate system records when present. GPS time, return numbers, scan + angles, point source ids, near infrared and waveform packet descriptors are not interpreted. - Leica scans read a block range with opt-in parallel decoding: the new `PclReadOptions.Blocks` selects the blocks of the scan in file order (per scan) and the new `PclReadOptions.MaxDegreeOfParallelism` (default 1, sequential streaming) decodes the selected blocks in parallel, keeping file order, consecutive `Index` values and `MaxPoints`/`SubsampleStep` diff --git a/README.md b/README.md index f0c6317..45fafc1 100644 --- a/README.md +++ b/README.md @@ -42,6 +42,7 @@ The library targets .NET 8 and .NET 10. It is managed code with no dependencies | FARO scan | `.fls`, or a FARO scan folder | Experimental. The scan grid and angles are read from the stored parameters and points are projected from the range grid; a file without them fails to open. The point count is the number of valid returns, which is counted on the first access; `FaroScan.GridSize` is the size of the scan grid. For a scan folder the layout of the samples is derived from the size of the data stream and the size of the grid, and the scale of the intensity from the values that are stored; when the sizes do not fit, the layout is assumed and a warning says so. | | PinPoint project | `.srf`, `.smf`, or a folder with `ScanProject.json` or `.smf`/`.srf` files | Experimental. The manifest (scan names, ids, poses, counts) is read. Points come from the referenced `.bin` files, in either of two layouts: a table of named channels, or numbered sections with a property table and Snappy-compressed channels, which PclIO decodes itself. Colour, intensity and normals are reported only when the file has that channel; in the second layout positions, colours, intensities, normals (`Normals`, three float32 components per point) and the per-point flags (`FilterBits`, one byte per point) are read, while the grid indices stored beside them are not interpreted, so by default every stored point that passes the plausibility rule below is returned. With `PclReadOptions.ExcludeFilteredPoints` the points with a non-zero flag are not returned. A manifest with a scan position list takes the point count from the manifest, else from the data file; a manifest with a data object list takes it from the data file the object refers to, and every location that shares one file reports its full count, because the shares are not recorded. | | EdgeWise | `.c3db`, `.c3ptx` | Experimental. A `.c3db` point database lists the scan clusters and a `.c3ptx` is one cluster file; the cluster files give positions only, as three 16-bit offsets from the cluster centre per point. The scale of the offsets is not stored in the files, so the reader assumes 32768 units = 0.128 m unless the caller states it with `PclReadOptions.EdgeWiseCoordinateScale`. | +| LAS | `.las` | Points decoded from versions 1.0 to 1.4 with point formats 0 to 10: XYZ with scale and offset, intensity, classification and RGB for the colour formats (2, 3, 5, 7, 8, 10). GPS time, return numbers, scan angles, point source ids, near infrared and waveform packet descriptors are not interpreted; extra bytes past a format's base size are skipped. The extended point counts of 1.4 are used when stated. Coordinate system records name an `EPSG:` code from a GeoTIFF key directory when it holds one, else the OGC WKT when one is stored. | ### Format detection @@ -51,6 +52,7 @@ regard to case. - E57, ReCap scan and Leica files are recognised by their content and accepted with any extension; for Leica, `.lgs` selects the older container and any other extension `.lgsx`. EdgeWise files are also recognised by content when the extension is none of its own. + A LAS file is recognised by the `LASF` signature with any extension except `.laz`, which shares the signature and stays unknown. - A FARO file must be an OLE2 compound file with the extension `.fls`. A ReCap project must be a ZIP archive with the extension `.rcp`. A PinPoint file must be a JSON text file with the extension `.smf` or `.srf`. - The extensions `.c3db` and `.c3ptx` select EdgeWise without a content check. A `.c3ptx` file is read as one cluster file and any @@ -120,6 +122,7 @@ outside the file is fetched. | FARO | Parameters 1 MiB; scan grid of at most 2^27 cells (134,217,728); a distance stream too small for the declared grid is refused; each compressed data stream is held in memory (see above) and is inflated in chunks, and must not inflate beyond its grid size plus 64 KiB. Compound file: 65,536 directory entries, depth 16. | | PinPoint | Manifest 64 MiB and the default nesting depth of `System.Text.Json` (64); 100,000 scans. The parsed manifest needs memory of about 15 times its size, so a manifest at the limit needs about 1 GB. Sectioned data file: at most 16 sections, a property table of at most 1 MiB nested at most 8 levels, copies in a compressed stream at most 8 MiB back with a decoder window of at most 8 MiB per stream; the sections must be contiguous and fill the file, and the `Points`, `RGB8`, `Intensities8`, `Normals` and `FilterBits` streams must decode to exactly 12, 3, 1, 12 and 1 bytes per stored point. | | EdgeWise | 1,048,576 characters per database line; 1,000,000 nodes. Patterns that parse a line have a time limit. | +| LAS | Header of at least 227 bytes (375 for 1.4); at most 4096 variable length records of at most 1 MiB each; point records at most 1 MiB. | ## Exporters diff --git a/docs/getting-started.md b/docs/getting-started.md index dae8bac..6114759 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -87,9 +87,11 @@ reading), `NotSupportedException` (unrecognised format or unsupported feature) o | FARO scan | `.fls`, or a FARO scan folder | Experimental; the scan grid comes from the stored parameters. A folder scan is read with an assumed layout, and a warning says so. | | PinPoint project | `.srf`, `.smf`, or a folder with `ScanProject.json` or `.smf`/`.srf` files | Experimental; manifest plus point data files in one of two layouts (named channels, or sections with Snappy-compressed channels); a data file in neither layout is not read. | | EdgeWise | `.c3db`, `.c3ptx` | Experimental; positions only, with a coordinate scale that is an assumption of the reader. | +| LAS | `.las` | Versions 1.0 to 1.4 with point formats 0 to 10; positions, intensity, classification and RGB for the colour formats. | A file is detected from its content, its extension or, for a folder, its layout. E57, ReCap scan and Leica files are recognised -by content, `.fls` needs an OLE2 compound file, `.rcp` a ZIP archive, `.srf` and `.smf` a JSON text file, and `.c3db` and `.c3ptx` +by content, `.fls` needs an OLE2 compound file, `.rcp` a ZIP archive, `.srf` and `.smf` a JSON text file, `.las` the `LASF` +signature (a `.laz` file shares it and stays unknown), and `.c3db` and `.c3ptx` are decided by extension alone, without a content check. Anything else is `PclFileKind.Unknown`. The rules are in [Public API](public-api.md#format-detection). Anything a decoder does not support fails instead of being guessed, apart from the assumptions named in the README (the FARO folder diff --git a/docs/public-api.md b/docs/public-api.md index 25c33d2..71c0249 100644 --- a/docs/public-api.md +++ b/docs/public-api.md @@ -14,7 +14,7 @@ format-specific members can change when the support for that format changes. | Type | Use | |---|---| | `PclFile` | Static entry point. `DetectKind(path)` and `DetectKind(stream, extension)` return a `PclFileKind`; `Open(path, options)` detects the format and returns a `PclDocument`. | -| `PclFileKind` | `Unknown`, `E57`, `RecapProject`, `RecapScan`, `LeicaLgsx`, `LeicaLgs`, `FaroFls`, `PinPoint` and `EdgeWise`. | +| `PclFileKind` | `Unknown`, `E57`, `RecapProject`, `RecapScan`, `LeicaLgsx`, `LeicaLgs`, `FaroFls`, `PinPoint`, `EdgeWise` and `Las`. | | `PclReadOptions` | `MaxPoints` (limit of one read call, null for all, 0 for none), `MaxCoordinateMagnitude` (default 5000, null for no limit; NaN and infinite coordinates are never returned; currently only PinPoint applies it), `EdgeWiseCoordinateScale` (default 0.128 / 32768 metres per unit; the files do not declare it), `SubsampleStep` (1 keeps every point, N keeps about one in N), `Blocks` (range of the scan's blocks in file order, null for all; currently only Leica honours it), `MaxDegreeOfParallelism` (default 1: sequential; above 1 decodes the selected Leica blocks in parallel, keeping file order), `ApplyTransform` (default true), `Strict` (default false: a reader that finds incomplete data fails with `InvalidDataException` instead of adding a warning) and `ExcludeFilteredPoints` (default false: points the file marks as filtered are excluded; currently the `FilterBits` channel of the PinPoint sectioned layout). Negative `MaxPoints`, a negative magnitude limit, a negative block start or count, a non-positive or non-finite EdgeWise scale and a step or parallelism below 1 throw `ArgumentOutOfRangeException`. `PclReadOptions.Default` has no limit. | | `BlockRange` | `StartBlock` (zero-based index of the first block to read) and `BlockCount` (how many blocks at most, null for all from the start on; a range past the last block yields no points, 0 yields none). | @@ -32,6 +32,7 @@ is the argument, which may also be a file name. - `E57`, `RecapScan` and the Leica kinds are recognised by content and accepted with any extension; `LeicaLgs` is chosen for the extension `.lgs` and `LeicaLgsx` for any other. `EdgeWise` is also recognised by content in a file with another extension. + `Las` is recognised by the `LASF` signature with any extension except `.laz`, which shares the signature and stays unknown. - `FaroFls` needs an OLE2 compound file with the extension `.fls`, `RecapProject` a ZIP archive with the extension `.rcp`, and `PinPoint` a JSON text file with the extension `.smf` or `.srf`. - The extensions `.c3db` and `.c3ptx` select `EdgeWise` without a content check, so a damaged file with one of these extensions is @@ -67,6 +68,7 @@ drop points that fail a plausibility rule (see the README), so a scan can yield | FARO | The number of valid returns, which is counted on the first access by reading the distance data of the scan; `FaroScan.GridSize` is the size of the scan grid (rows times columns). For a folder scan the grid size is limited by the size of the data stream, and `FaroScan.RecordSize` is the size of one sample in bytes. | | PinPoint | A manifest with a scan position list: declared by the manifest, else derived from the data file, else 0 (not known). A manifest with a data object list: derived from the data file the object refers to, else 0 (not known); locations that share one file each report its full count. | | EdgeWise | A `.c3db` database node: the count the database declares. A `.c3ptx` file: computed by walking the cluster headers when the file is opened. | +| LAS | The header count (the extended 1.4 count when it states one). It is not checked against the point data until the points are read; a truncated read fails then. | `HasColors`, `HasIntensities` and `HasNormals` follow what the file contains, except for the scans of a ReCap project, where all three are true when the `.rcs` file was found and false otherwise, without the file being opened. A ReCap scan reads the channels @@ -76,7 +78,7 @@ no colour. ## Format documents `E57Document`/`E57Scan`, `RcsScanDocument`/`RcsScan`, `RcpProjectDocument`/`RcpScan`, `LgsxDocument`/`LgsxScan`/`LgsxEntry`/`LgsxSetup`, -`FaroFlsDocument`/`FaroScan`, `PinPointDocument`/`PinPointScan` and `EdgeWiseDocument`/`EdgeWiseScan` are the readers behind +`FaroFlsDocument`/`FaroScan`, `PinPointDocument`/`PinPointScan`, `EdgeWiseDocument`/`EdgeWiseScan` and `LasDocument`/`LasScan` are the readers behind `PclFile.Open`. Each has a static `Open` method and format-specific properties (for example the E57 version or the FARO scanner model). Their XML documentation states what each reader supports. Prefer `PclFile.Open`, which detects the format and chooses the reader. diff --git a/src/PclIO.Cli/Program.cs b/src/PclIO.Cli/Program.cs index 596a510..5bb0f86 100644 --- a/src/PclIO.Cli/Program.cs +++ b/src/PclIO.Cli/Program.cs @@ -81,6 +81,7 @@ ReCap project a ZIP archive with the extension .rcp PinPoint a JSON text file with the extension .smf or .srf, or a folder that holds ScanProject.json, a .smf file or a .srf file EdgeWise the extension .c3db or .c3ptx, or a text or tree signature in another file + LAS by signature (.las) Output formats (--format, or the output extension; any other extension needs --format): las ASPRS LAS 1.2, point data record format 2 (XYZ, intensity, RGB) diff --git a/src/PclIO/Formats/Las/LasDocument.cs b/src/PclIO/Formats/Las/LasDocument.cs new file mode 100644 index 0000000..ab302ce --- /dev/null +++ b/src/PclIO/Formats/Las/LasDocument.cs @@ -0,0 +1,426 @@ +using PclIO.Internal; + +namespace PclIO.Formats.Las; + +/// Represents an ASPRS LAS point cloud file (.las). +/// +/// +/// Supported: versions 1.0 to 1.4 with point formats 0 to 10, the legacy and the extended (1.4) point counts, +/// scale and offset, and the coordinate system records when present. Positions, intensity, classification and, +/// for the colour formats (2, 3, 5, 7, 8, 10), 16-bit RGB downscaled to 8 bits are read. GPS time, return numbers, +/// scan angles, point source ids, near infrared and waveform packet descriptors are not interpreted; extra bytes +/// past a format's base size are skipped. Extended variable length records (after the point data) are not read. +/// +/// +/// Not supported (reported with when the file is opened): versions outside 1.0 +/// to 1.4, point formats above 10, and the colour or waveform formats 6 to 10 in a file that is not 1.4. +/// +/// +public sealed class LasDocument : PclDocument +{ + // Bounds for data taken from an untrusted file. + private const int MinHeaderSize = 227; + private const int Las14HeaderSize = 375; + private const int MaxVlrs = 4096; + private const int MaxVlrDataBytes = 1 << 20; + private const long MaxVlrTotalBytes = 16L << 20; + private const int MaxPointRecordLength = 1 << 20; + + /// + public override PclFileKind Kind => PclFileKind.Las; + + /// LAS version of the file, for example "1.2". + public string Version { get; } + + /// Point data format of the file (0 to 10). + public int PointFormat { get; } + + /// + public override IReadOnlyList Scans => _scans; + private readonly List _scans = []; + + private LasDocument(string path, long fileSize, string version, int pointFormat) + : base(path, fileSize) + { + Version = version; + PointFormat = pointFormat; + _metadata["Format"] = "ASPRS LAS"; + _metadata["Version"] = version; + _metadata["PointFormat"] = pointFormat.ToString(System.Globalization.CultureInfo.InvariantCulture); + } + + /// Opens an ASPRS LAS file and reads its header and variable length records. + /// Path of the .las file. + /// Currently unused; accepted for symmetry with the other readers. + /// The file does not exist. + /// The file is damaged or truncated, or declares sizes outside the file. + /// The version or the point format is not supported. + public static LasDocument Open(string path, PclReadOptions? options = null) + { + var fi = new FileInfo(path); + if (!fi.Exists) throw new FileNotFoundException("LAS file not found.", path); + + using var fs = File.OpenRead(path); + long fileLength = fs.Length; + + if (fileLength < 4) throw FormatError.Truncated("LAS header"); + byte[] signature = new byte[4]; + fs.ReadExactly(signature); + if (!signature.AsSpan().SequenceEqual("LASF"u8)) + throw FormatError.Corrupted("not a LAS file, signature mismatch."); + + if (fileLength < MinHeaderSize) throw FormatError.Truncated("LAS header"); + fs.Position = 0; + byte[] header = new byte[MinHeaderSize]; + fs.ReadExactly(header); + + int major = header[24]; + int minor = header[25]; + if (major != 1 || minor > 4) + throw FormatError.Unsupported($"LAS version {major}.{minor} is not supported; versions 1.0 to 1.4 are read."); + + int headerSize = Bytes.ReadUInt16LE(header.AsSpan(94, 2)); + if (headerSize < MinHeaderSize) throw FormatError.Corrupted($"LAS header size {headerSize} is smaller than the minimum {MinHeaderSize}."); + if (minor == 4 && headerSize < Las14HeaderSize) + throw FormatError.Corrupted($"LAS 1.4 header size {headerSize} is smaller than {Las14HeaderSize}."); + if (headerSize > fileLength) throw FormatError.Truncated("LAS header"); + + // The full header is needed for the 1.4 extended counts. + if (headerSize > header.Length) + { + fs.Position = 0; + header = new byte[headerSize]; + fs.ReadExactly(header); + } + + uint pointDataOffset = Bytes.ReadUInt32LE(header.AsSpan(96, 4)); + if (pointDataOffset < headerSize || pointDataOffset > fileLength) + throw FormatError.Corrupted("LAS offset to point data is outside the file."); + + uint vlrCount = Bytes.ReadUInt32LE(header.AsSpan(100, 4)); + if (vlrCount > MaxVlrs) + throw FormatError.Corrupted($"LAS file declares {vlrCount} variable length records; at most {MaxVlrs} are accepted."); + + // The low six bits hold the point format; bit 6 flags waveform data packets and bit 7 is reserved. The + // waveform tail is skipped as extra bytes, so a file that sets bit 6 still reads with its base format. + int pointFormat = header[104] & 0x3F; + if (pointFormat > 10) + throw FormatError.Unsupported($"LAS point format {pointFormat} is not supported; point formats 0 to 10 are read."); + if (pointFormat >= 6 && minor != 4) + throw FormatError.Unsupported($"LAS point format {pointFormat} needs version 1.4."); + + int recordLength = Bytes.ReadUInt16LE(header.AsSpan(105, 2)); + int baseSize = BaseRecordSize(pointFormat); + if (recordLength < baseSize) + throw FormatError.Corrupted($"LAS point record length {recordLength} is smaller than the {baseSize} bytes point format {pointFormat} needs."); + if (recordLength > MaxPointRecordLength) + throw FormatError.Corrupted($"LAS point record length {recordLength} is larger than the accepted size."); + + uint legacyCount = Bytes.ReadUInt32LE(header.AsSpan(107, 4)); + long pointCount = legacyCount; + if (minor == 4) + { + ulong extendedCount = Bytes.ReadUInt64LE(header.AsSpan(247, 8)); + if (extendedCount != 0) pointCount = extendedCount > long.MaxValue ? long.MaxValue : (long)extendedCount; + } + + double scaleX = Bytes.ReadDoubleLE(header.AsSpan(131, 8)); + double scaleY = Bytes.ReadDoubleLE(header.AsSpan(139, 8)); + double scaleZ = Bytes.ReadDoubleLE(header.AsSpan(147, 8)); + if (!double.IsFinite(scaleX) || scaleX == 0 || !double.IsFinite(scaleY) || scaleY == 0 || !double.IsFinite(scaleZ) || scaleZ == 0) + throw FormatError.Corrupted("LAS scale is not usable."); + + double offsetX = Bytes.ReadDoubleLE(header.AsSpan(155, 8)); + double offsetY = Bytes.ReadDoubleLE(header.AsSpan(163, 8)); + double offsetZ = Bytes.ReadDoubleLE(header.AsSpan(171, 8)); + if (!double.IsFinite(offsetX) || !double.IsFinite(offsetY) || !double.IsFinite(offsetZ)) + throw FormatError.Corrupted("LAS offset is not usable."); + + var bounds = ReadBounds(header); + + var vlrs = ReadVlrs(fs, headerSize, pointDataOffset, vlrCount); + string? coordinateSystem = FindCoordinateSystem(vlrs); + + var document = new LasDocument(path, fileLength, $"1.{minor}", pointFormat); + if (coordinateSystem != null) + { + document.CoordinateSystem = coordinateSystem; + document._metadata["CoordinateSystem"] = coordinateSystem; + } + + string systemId = ReadFixedAscii(header.AsSpan(26, 32)); + if (systemId.Length > 0) document._metadata["SystemIdentifier"] = systemId; + string software = ReadFixedAscii(header.AsSpan(58, 32)); + if (software.Length > 0) document._metadata["GeneratingSoftware"] = software; + + string name = Path.GetFileNameWithoutExtension(path); + document._scans.Add(new LasScan("0", name, pointCount, bounds, path, pointDataOffset, + pointFormat, recordLength, new PclVector3(scaleX, scaleY, scaleZ), new PclVector3(offsetX, offsetY, offsetZ))); + return document; + } + + // Minimum record size of a point format; waveform descriptors are part of the tail and are not interpreted. + private static int BaseRecordSize(int pointFormat) => pointFormat switch + { + 0 => 20, + 1 => 28, + 2 => 26, + 3 => 34, + 4 => 57, + 5 => 63, + 6 => 30, + 7 => 36, + 8 => 38, + 9 => 59, + _ => 67, + }; + + private static PclBounds ReadBounds(byte[] header) + { + double maxX = Bytes.ReadDoubleLE(header.AsSpan(179, 8)); + double minX = Bytes.ReadDoubleLE(header.AsSpan(187, 8)); + double maxY = Bytes.ReadDoubleLE(header.AsSpan(195, 8)); + double minY = Bytes.ReadDoubleLE(header.AsSpan(203, 8)); + double maxZ = Bytes.ReadDoubleLE(header.AsSpan(211, 8)); + double minZ = Bytes.ReadDoubleLE(header.AsSpan(219, 8)); + + if (!double.IsFinite(maxX) || !double.IsFinite(minX) || !double.IsFinite(maxY) || !double.IsFinite(minY) || + !double.IsFinite(maxZ) || !double.IsFinite(minZ)) + return PclBounds.Empty; + + var bounds = new PclBounds(new PclVector3(minX, minY, minZ), new PclVector3(maxX, maxY, maxZ)); + return bounds.IsEmpty ? PclBounds.Empty : bounds; + } + + private static string ReadFixedAscii(ReadOnlySpan field) + { + int nullIdx = field.IndexOf((byte)0); + if (nullIdx >= 0) field = field[..nullIdx]; + return System.Text.Encoding.ASCII.GetString(field).Trim(); + } + + private sealed record Vlr(string UserId, int RecordId, byte[] Data); + + // Reads the variable length records between the header and the point data. Every declared size is checked + // against the file before anything is allocated for it. Only the coordinate system records are kept; the rest + // are walked past without holding their payload, so a file with many large records does not exhaust memory. + private static List ReadVlrs(FileStream fs, int headerSize, uint pointDataOffset, uint vlrCount) + { + var vlrs = new List((int)Math.Min(vlrCount, 16)); + long position = headerSize; + long keptBytes = 0; + byte[] header = new byte[54]; + + for (uint i = 0; i < vlrCount; i++) + { + if (position + 54 > pointDataOffset) throw FormatError.Corrupted("LAS variable length record reaches past the point data."); + fs.Position = position; + fs.ReadExactly(header); + position += 54; + + string userId = ReadFixedAscii(header.AsSpan(2, 16)); + int recordId = Bytes.ReadUInt16LE(header.AsSpan(18, 2)); + uint dataLength = Bytes.ReadUInt32LE(header.AsSpan(20, 4)); + if (dataLength > MaxVlrDataBytes) + throw FormatError.Corrupted($"LAS variable length record holds {dataLength} bytes; at most {MaxVlrDataBytes} are accepted."); + if (position + dataLength > pointDataOffset) throw FormatError.Corrupted("LAS variable length record reaches past the point data."); + + // A compressed LAZ file shares the LAS signature and header; its point data is not LAS records, so it + // is rejected here rather than read as points. + if (userId.Equals("laszip encoded", StringComparison.Ordinal) && recordId == 22205) + throw FormatError.Unsupported("LAS file is compressed (LAZ); compressed point data is not read."); + + bool isCoordinateSystem = userId.Equals("LASF_Projection", StringComparison.Ordinal) && (recordId == 34735 || recordId == 2112); + if (isCoordinateSystem) + { + keptBytes += dataLength; + if (keptBytes > MaxVlrTotalBytes) + throw FormatError.Corrupted("LAS coordinate system records hold more bytes than are accepted."); + byte[] data = new byte[dataLength]; + if (dataLength > 0) fs.ReadExactly(data); + vlrs.Add(new Vlr(userId, recordId, data)); + } + + position += dataLength; + } + + return vlrs; + } + + // The coordinate system of the file: an EPSG code from a GeoTIFF key directory when it names one, else the + // OGC WKT when a WKT record holds one. Anything else leaves the coordinate system unknown. + private static string? FindCoordinateSystem(List vlrs) + { + foreach (var vlr in vlrs) + { + if (vlr.RecordId == 34735 && vlr.UserId.Equals("LASF_Projection", StringComparison.Ordinal) && TryReadEpsg(vlr.Data, out ushort code)) + return $"EPSG:{code}"; + } + + foreach (var vlr in vlrs) + { + // The OGC coordinate system WKT lives in record 2112 under LASF_Projection; record 2111 is a math + // transform, not a coordinate system, and is not read as one. + if (vlr.RecordId == 2112 && vlr.UserId.Equals("LASF_Projection", StringComparison.Ordinal)) + { + string wkt = System.Text.Encoding.UTF8.GetString(vlr.Data).TrimEnd('\0', ' ', '\r', '\n'); + if (wkt.Length > 0) return wkt; + } + } + + return null; + } + + // A GeoTIFF key directory names an EPSG code through the projected (3072) or the geographic (2048) type key. + private static bool TryReadEpsg(byte[] data, out ushort code) + { + code = 0; + if (data.Length < 8) return false; + int keyCount = Bytes.ReadUInt16LE(data.AsSpan(6, 2)); + if (keyCount <= 0 || keyCount > 1024) return false; + if (8 + (keyCount * 8) > data.Length) return false; + + // The projected reference wins over the geographic one. + for (int pass = 0; pass < 2; pass++) + { + int wanted = pass == 0 ? 3072 : 2048; + for (int k = 0; k < keyCount; k++) + { + var entry = data.AsSpan(8 + (k * 8), 8); + if (Bytes.ReadUInt16LE(entry[..2]) != wanted) continue; + int value = Bytes.ReadUInt16LE(entry.Slice(6, 2)); + // 32767 is the user-defined sentinel, not an assigned EPSG code; the definition lives elsewhere. + if (value is >= 1 and <= ushort.MaxValue && value != 32767) + { + code = (ushort)value; + return true; + } + } + } + + return false; + } +} + +/// The single scan of an ASPRS LAS file. +public sealed class LasScan : PclScan +{ + // At most this many point records are held at once; memory use does not depend on the number of points. + private const int MaxChunkBytes = 64 * 1024; + + /// + public override string Id { get; } + + /// + public override string Name { get; } + + /// Number of points the header declares (the extended count for 1.4 when it states one). + public override long PointCount { get; } + + /// Path of the LAS file. + public string FilePath { get; } + + /// + public override bool HasColors { get; } + + /// + public override bool HasIntensities => true; + + /// + public override bool HasNormals => false; + + private readonly long _pointDataOffset; + private readonly int _pointFormat; + private readonly int _recordLength; + private readonly PclVector3 _scale; + private readonly PclVector3 _offset; + + internal LasScan(string id, string name, long pointCount, PclBounds bounds, string filePath, + uint pointDataOffset, int pointFormat, int recordLength, PclVector3 scale, PclVector3 offset) + { + Id = id; + Name = name; + PointCount = pointCount; + Bounds = bounds; + FilePath = filePath; + _pointDataOffset = pointDataOffset; + _pointFormat = pointFormat; + _recordLength = recordLength; + _scale = scale; + _offset = offset; + HasColors = pointFormat is 2 or 3 or 5 or 7 or 8 or 10; + } + + /// Streams the points of the file. + /// + /// Records are read in chunks, so memory use does not depend on the number of points. + /// Points whose stored coordinates do not form finite positions are not returned. + /// and are ignored: + /// the point stream has no independent blocks to select or decode in parallel. + /// + /// The point data is truncated. + public override IEnumerable ReadPoints(PclReadOptions? options = null) + { + options ??= PclReadOptions.Default; + long targetCount = options.MaxPoints ?? PointCount; + if (targetCount <= 0) yield break; + targetCount = Math.Min(targetCount, PointCount); + int step = Math.Max(1, options.SubsampleStep); + bool legacy = _pointFormat <= 5; + bool colored = HasColors; + + using var fs = File.OpenRead(FilePath); + long fileLength = fs.Length; + + int recordsPerChunk = Math.Max(1, MaxChunkBytes / _recordLength); + byte[] chunk = new byte[recordsPerChunk * _recordLength]; + long yielded = 0; + + for (long start = 0; start < targetCount; start += recordsPerChunk) + { + int want = (int)Math.Min(recordsPerChunk, targetCount - start); + long byteOffset = _pointDataOffset + (start * (long)_recordLength); + if (byteOffset < 0 || want > (fileLength - byteOffset) / _recordLength) + throw FormatError.Truncated("point records"); + + fs.Position = byteOffset; + fs.ReadExactly(chunk, 0, want * _recordLength); + + for (int i = 0; i < want; i++) + { + if ((start + i) % step != 0) continue; + var record = chunk.AsSpan(i * _recordLength, _recordLength); + + double x = _offset.X + (Bytes.ReadInt32LE(record[..4]) * _scale.X); + double y = _offset.Y + (Bytes.ReadInt32LE(record.Slice(4, 4)) * _scale.Y); + double z = _offset.Z + (Bytes.ReadInt32LE(record.Slice(8, 4)) * _scale.Z); + if (!double.IsFinite(x) || !double.IsFinite(y) || !double.IsFinite(z)) continue; + + float intensity = Bytes.ReadUInt16LE(record.Slice(12, 2)) / 65535.0f; + byte classification = legacy + ? (byte)(record[15] & 0x1F) + : record[16]; + + PclColor? color = null; + if (colored) + { + int at = legacy ? 20 : 30; + color = new PclColor( + (byte)(Bytes.ReadUInt16LE(record.Slice(at, 2)) / 257), + (byte)(Bytes.ReadUInt16LE(record.Slice(at + 2, 2)) / 257), + (byte)(Bytes.ReadUInt16LE(record.Slice(at + 4, 2)) / 257)); + } + + var pt = new PclPoint(new PclVector3(x, y, z), Color: color, Intensity: intensity, + Classification: classification, Index: (ulong)yielded); + + if (options.ApplyTransform) + pt = pt with { Position = Transform.TransformPoint(pt.Position) }; + + yield return pt; + yielded++; + if (yielded >= targetCount) yield break; + } + } + } +} diff --git a/src/PclIO/Internal/KindDetector.cs b/src/PclIO/Internal/KindDetector.cs index f997662..cad536a 100644 --- a/src/PclIO/Internal/KindDetector.cs +++ b/src/PclIO/Internal/KindDetector.cs @@ -14,6 +14,9 @@ internal static class KindDetector // Leading bytes of the EdgeWise text database. private static readonly byte[] EdgeWiseTextMagic = "2\r\n _Tr"u8.ToArray(); private static readonly byte[] EdgeWiseTreeMagic = "_Tree"u8.ToArray(); + // First bytes of an ASPRS LAS file. LAZ files share the signature but need their own reader, so the .laz + // extension below stays unknown until that reader exists. + private static readonly byte[] LasMagic = "LASF"u8.ToArray(); public static PclFileKind Detect(string path) { @@ -90,6 +93,10 @@ public static PclFileKind Detect(Stream stream, string? extension = null) if (head.StartsWith(EdgeWiseTextMagic) || head.StartsWith(EdgeWiseTreeMagic)) return PclFileKind.EdgeWise; + // LAS files are recognised by content; a compressed LAZ file shares the signature and stays unknown. + if (head.StartsWith(LasMagic)) + return extension == ".laz" ? PclFileKind.Unknown : PclFileKind.Las; + return PclFileKind.Unknown; } } diff --git a/src/PclIO/PclFile.cs b/src/PclIO/PclFile.cs index c228c51..0f2caa6 100644 --- a/src/PclIO/PclFile.cs +++ b/src/PclIO/PclFile.cs @@ -1,6 +1,7 @@ using PclIO.Formats.E57; using PclIO.Formats.EdgeWise; using PclIO.Formats.Faro; +using PclIO.Formats.Las; using PclIO.Formats.Leica; using PclIO.Formats.PinPoint; using PclIO.Formats.Recap; @@ -48,6 +49,7 @@ public static PclDocument Open(string path, PclReadOptions? options = null) PclFileKind.FaroFls => FaroFlsDocument.Open(path, options), PclFileKind.PinPoint => PinPointDocument.Open(path, options), PclFileKind.EdgeWise => EdgeWiseDocument.Open(path, options), + PclFileKind.Las => LasDocument.Open(path, options), _ => throw new NotSupportedException($"Unsupported or unrecognized point cloud file: '{path}'"), }; } diff --git a/src/PclIO/PclFileKind.cs b/src/PclIO/PclFileKind.cs index bb13f36..5e1d745 100644 --- a/src/PclIO/PclFileKind.cs +++ b/src/PclIO/PclFileKind.cs @@ -29,4 +29,7 @@ public enum PclFileKind /// ClearEdge3D EdgeWise point database and scan clusters (.c3db, .c3ptx). EdgeWise = 8, + + /// ASPRS LAS point cloud file (.las). + Las = 9, } diff --git a/tests/PclIO.Tests/FormatTests.cs b/tests/PclIO.Tests/FormatTests.cs index 7d52cb3..26faa70 100644 --- a/tests/PclIO.Tests/FormatTests.cs +++ b/tests/PclIO.Tests/FormatTests.cs @@ -1,6 +1,8 @@ +using PclIO.Export; using PclIO.Formats.E57; using PclIO.Formats.EdgeWise; using PclIO.Formats.Faro; +using PclIO.Formats.Las; using PclIO.Formats.Leica; using PclIO.Formats.PinPoint; using PclIO.Formats.Recap; @@ -2151,4 +2153,385 @@ public void Faro_With_A_Missing_Grid_Parameter_Is_Rejected_At_Open(string key) } internal static bool IsFormatError(Exception? exception) => exception is InvalidDataException or NotSupportedException; + + // ---- LAS ---------------------------------------------------------------------------------------------------- + + private static readonly IReadOnlyList LasPoints = + [ + new(1, 2, 3, 0.25f, new PclColor(10, 20, 30), 5), + new(-4.5, 5.25, 6.125, 0.5f, null, 2), + new(7, 8, 9, 1.0f, new PclColor(255, 255, 255), 0), + ]; + + [Fact] + public void PclFile_Opens_Las_With_Count_Channels_Bounds_And_Metadata() + { + using var dir = new TempDirectory(); + using var doc = PclFile.Open(dir.Write("sample.las", SyntheticFixtures.CreateSyntheticLas(LasPoints))); + + Assert.Equal(PclFileKind.Las, doc.Kind); + var scan = Assert.Single(doc.Scans); + Assert.Equal("0", scan.Id); + Assert.Equal("sample", scan.Name); + Assert.Equal(3, scan.PointCount); + Assert.Equal(3, doc.TotalPointCount); + Assert.True(scan.HasColors); + Assert.True(scan.HasIntensities); + Assert.False(scan.HasNormals); + Assert.Equal(new PclBounds(new PclVector3(-4.5, 2, 3), new PclVector3(7, 8, 9)), scan.Bounds); + Assert.Equal(new PclBounds(new PclVector3(-4.5, 2, 3), new PclVector3(7, 8, 9)), doc.OverallBounds); + Assert.Equal("1.2", ((LasDocument)doc).Version); + Assert.Equal(2, ((LasDocument)doc).PointFormat); + Assert.Equal("ASPRS LAS", doc.Metadata["Format"]); + Assert.Equal("1.2", doc.Metadata["Version"]); + Assert.Null(doc.CoordinateSystem); + Assert.Empty(doc.Warnings); + } + + [Fact] + public void Las_Round_Trip_Positions_Intensity_Colour_And_Classification() + { + using var dir = new TempDirectory(); + using var doc = LasDocument.Open(dir.Write("sample.las", SyntheticFixtures.CreateSyntheticLas(LasPoints))); + + var points = doc.Scans[0].ReadPoints(NoTransform).ToList(); + + Assert.Equal(3, points.Count); + Assert.Equal([0UL, 1UL, 2UL], points.Select(p => p.Index)); + AssertNear(new PclVector3(1, 2, 3), points[0].Position); + AssertNear(new PclVector3(-4.5, 5.25, 6.125), points[1].Position); + AssertNear(new PclVector3(7, 8, 9), points[2].Position); + Assert.Equal(0.25, (double)points[0].Intensity!.Value, 4); + Assert.Equal(0.5, (double)points[1].Intensity!.Value, 4); + Assert.Equal(1.0, (double)points[2].Intensity!.Value, 4); + Assert.Equal(new PclColor(10, 20, 30), points[0].Color); + Assert.Equal(new PclColor(255, 255, 255), points[1].Color); + Assert.Equal(new PclColor(255, 255, 255), points[2].Color); + Assert.Equal((byte)5, points[0].Classification); + Assert.Equal((byte)2, points[1].Classification); + Assert.All(points, p => Assert.Null(p.Normal)); + } + + [Fact] + public void Las14_Reads_Extended_Counts_Scale_Offset_And_Epsg() + { + var points = new[] + { + new LasSamplePoint(1, 2, 3, 0.25f, new PclColor(10, 20, 30), 5), + new LasSamplePoint(-4.5, 5.25, 6.12, 0.5f, null, 2), + }; + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(points, versionMinor: 4, pointFormat: 7, + scale: 0.01, offset: (10, 20, 30), + vlrs: [SyntheticFixtures.LasEpsgVlr(3857)], + legacyCountOverride: 0, extendedCountOverride: (ulong)points.Length); + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + // The legacy count is zero, so the extended count decides. + var scan = Assert.Single(doc.Scans); + Assert.Equal(2, scan.PointCount); + Assert.Equal("EPSG:3857", doc.CoordinateSystem); + + var read = scan.ReadPoints(NoTransform).ToList(); + Assert.Equal(2, read.Count); + AssertNear(new PclVector3(1, 2, 3), read[0].Position); + AssertNear(new PclVector3(-4.5, 5.25, 6.12), read[1].Position); + Assert.Equal(new PclColor(10, 20, 30), read[0].Color); + } + + [Theory] + [InlineData(0, 2, false)] + [InlineData(1, 2, false)] + [InlineData(2, 2, true)] + [InlineData(3, 2, true)] + [InlineData(4, 2, false)] + [InlineData(5, 2, true)] + [InlineData(6, 4, false)] + [InlineData(7, 4, true)] + [InlineData(8, 4, true)] + [InlineData(9, 4, false)] + [InlineData(10, 4, true)] + public void Las_Point_Formats_Zero_To_Ten_Decode_Positions(int pointFormat, int versionMinor, bool colored) + { + var points = new[] { new LasSamplePoint(1, 2, 3, 0.5f, new PclColor(10, 20, 30), 7) }; + using var dir = new TempDirectory(); + using var doc = LasDocument.Open(dir.Write("sample.las", + SyntheticFixtures.CreateSyntheticLas(points, versionMinor: versionMinor, pointFormat: pointFormat))); + + var scan = Assert.Single(doc.Scans); + Assert.Equal(colored, scan.HasColors); + Assert.True(scan.HasIntensities); + Assert.False(scan.HasNormals); + + var point = Assert.Single(scan.ReadPoints(NoTransform)); + AssertNear(new PclVector3(1, 2, 3), point.Position); + Assert.Equal(0.5, (double)point.Intensity!.Value, 4); + Assert.Equal((byte)7, point.Classification); + Assert.Equal(colored ? new PclColor(10, 20, 30) : null, point.Color); + } + + [Fact] + public void Las_Honours_Limit_And_Step_And_Ignores_Blocks_And_Parallelism() + { + var points = Enumerable.Range(0, 8).Select(i => new LasSamplePoint(i, 0, 0)).ToList(); + using var dir = new TempDirectory(); + using var doc = LasDocument.Open(dir.Write("sample.las", SyntheticFixtures.CreateSyntheticLas(points))); + var scan = doc.Scans[0]; + + Assert.Equal(3, scan.ReadPoints(new PclReadOptions { MaxPoints = 3 }).Count()); + Assert.Equal(4, scan.ReadPoints(new PclReadOptions { SubsampleStep = 2 }).Count()); + Assert.Empty(scan.ReadPoints(new PclReadOptions { MaxPoints = 0 })); + + // The point stream has no independent blocks: the range and the parallelism are ignored. + var ranged = scan.ReadPoints(new PclReadOptions + { + Blocks = new BlockRange { StartBlock = 1, BlockCount = 1 }, + MaxDegreeOfParallelism = 4, + }).ToList(); + Assert.Equal(8, ranged.Count); + Assert.Equal([0UL, 1UL, 2UL, 3UL, 4UL, 5UL, 6UL, 7UL], ranged.Select(p => p.Index)); + } + + [Fact] + public void Las_Of_Exporter_Output_Reads_Back() + { + var source = new[] + { + new PclPoint(new PclVector3(1, 2, 3), new PclColor(10, 20, 30), 0.25f, null, 5), + new PclPoint(new PclVector3(-4.5, 5.25, 6.125), null, 0.5f), + }; + using var exported = FakeDocument.Of(source); + byte[] bytes; + using (var ms = new MemoryStream()) + { + PclLasExporter.Export(exported, ms); + bytes = ms.ToArray(); + } + + using var dir = new TempDirectory(); + using var doc = PclFile.Open(dir.Write("exported.las", bytes)); + + Assert.Equal(PclFileKind.Las, doc.Kind); + var points = doc.Scans[0].ReadPoints(NoTransform).ToList(); + Assert.Equal(2, points.Count); + AssertNear(new PclVector3(1, 2, 3), points[0].Position); + AssertNear(new PclVector3(-4.5, 5.25, 6.125), points[1].Position); + Assert.Equal(new PclColor(10, 20, 30), points[0].Color); + Assert.Equal(new PclColor(255, 255, 255), points[1].Color); + Assert.Equal(0.25, (double)points[0].Intensity!.Value, 4); + Assert.Equal((byte)5, points[0].Classification); + } + + [Fact] + public void Las_Coordinate_System_Comes_From_Wkt_When_No_Geotiff_Names_One() + { + const string wkt = "PROJCS[\"WGS 84 / Test\",GEOGCS[\"WGS 84\"]]"; + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, + vlrs: [SyntheticFixtures.LasWktVlr(wkt), ("LASF_Something", 42, new byte[] { 1, 2, 3 })]); + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + Assert.Equal(wkt, doc.CoordinateSystem); + } + + [Theory] + [InlineData(0, 0)] + [InlineData(1, 1)] + [InlineData(3, 3)] + public void Las_Versions_One_Zero_To_One_Three_Are_Read(int pointFormat, int versionMinor) + { + var points = new[] { new LasSamplePoint(1, 2, 3, 0.5f, null, 7) }; + using var dir = new TempDirectory(); + using var doc = LasDocument.Open(dir.Write("sample.las", + SyntheticFixtures.CreateSyntheticLas(points, versionMinor: versionMinor, pointFormat: pointFormat))); + + Assert.Equal($"1.{versionMinor}", doc.Version); + var point = Assert.Single(doc.Scans[0].ReadPoints(NoTransform)); + AssertNear(new PclVector3(1, 2, 3), point.Position); + } + + [Fact] + public void Las_With_The_Waveform_Bit_Set_Reads_With_Its_Base_Format() + { + // Bit 6 of the format byte flags waveform data packets; the format is the low six bits. + var points = new[] { new LasSamplePoint(1, 2, 3, 0.5f, null, 7) }; + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(points, versionMinor: 4, pointFormat: 4); + bytes[104] |= 0x40; + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + var scan = Assert.Single(doc.Scans); + Assert.Equal(4, doc.PointFormat); + AssertNear(new PclVector3(1, 2, 3), Assert.Single(scan.ReadPoints(NoTransform)).Position); + } + + [Fact] + public void Las_With_A_Laszip_Record_Is_Rejected_As_Compressed() + { + // A LAZ renamed to .las keeps the LAS signature and header; the laszip record marks it compressed. + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, + vlrs: [("laszip encoded", 22205, new byte[] { 1, 2, 3, 4 })]); + + Assert.Throws(() => LasDocument.Open(dir.Write("sample.las", bytes))); + Assert.Throws(() => PclFile.Open(dir.Write("renamed.las", bytes))); + } + + [Theory] + [InlineData("LASF_Spec", 2112)] + [InlineData("LASF_Projection", 2111)] + public void Las_Wkt_In_The_Wrong_Record_Identity_Is_Not_A_Crs(string userId, int recordId) + { + const string wkt = "PROJCS[\"WGS 84 / Test\",GEOGCS[\"WGS 84\"]]"; + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, + vlrs: [(userId, (ushort)recordId, System.Text.Encoding.UTF8.GetBytes(wkt + "\0"))]); + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + Assert.Null(doc.CoordinateSystem); + } + + [Fact] + public void Las_User_Defined_Epsg_Sentinel_Is_Not_A_Crs() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, + vlrs: [SyntheticFixtures.LasEpsgVlr(32767)]); + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + Assert.Null(doc.CoordinateSystem); + } + + [Fact] + public void Las_With_A_Huge_Declared_Count_Fails_Cleanly_When_Read() + { + // The count is not checked against the file at open; a read of the missing records fails with a truncation + // error rather than allocating for the declared count. + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, legacyCountOverride: uint.MaxValue); + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + Assert.Equal(uint.MaxValue, doc.Scans[0].PointCount); + Assert.Throws(() => doc.Scans[0].ReadPoints().ToList()); + } + + [Fact] + public void Las_Without_A_Recognised_Crs_Leaves_It_Unknown() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, + vlrs: [("LASF_Something", 42, new byte[] { 1, 2, 3 })]); + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + Assert.Null(doc.CoordinateSystem); + Assert.Equal(LasPoints.Count, doc.Scans[0].ReadPoints().Count()); + } + + [Fact] + public void Las_Wrong_Signature_Is_Rejected() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints); + bytes[0] = (byte)'X'; + + Assert.Throws(() => LasDocument.Open(dir.Write("sample.las", bytes))); + Assert.Equal(PclFileKind.Unknown, PclFile.DetectKind(dir.Write("other.las", bytes))); + Assert.Throws(() => PclFile.Open(dir.PathOf("other.las"))); + } + + [Fact] + public void Las_Unsupported_Version_Is_Rejected() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, versionMinor: 2); + bytes[25] = 5; + + Assert.Throws(() => LasDocument.Open(dir.Write("future.las", bytes))); + + byte[] major = SyntheticFixtures.CreateSyntheticLas(LasPoints); + major[24] = 2; + Assert.Throws(() => LasDocument.Open(dir.Write("major.las", major))); + } + + [Theory] + [InlineData(11, 2)] + [InlineData(6, 2)] + public void Las_Unsupported_Point_Format_Is_Rejected(int pointFormat, int versionMinor) + { + using var dir = new TempDirectory(); + byte[] bytes = pointFormat > 10 + ? SyntheticFixtures.CreateSyntheticLas([], versionMinor: versionMinor, pointFormat: pointFormat) + : SyntheticFixtures.CreateSyntheticLas(LasPoints, versionMinor: versionMinor, pointFormat: pointFormat); + + Assert.Throws(() => LasDocument.Open(dir.Write("sample.las", bytes))); + } + + [Fact] + public void Las_Truncated_Points_Are_Rejected_When_Read() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints); + byte[] cut = bytes[..(bytes.Length - 10)]; + using var doc = LasDocument.Open(dir.Write("cut.las", cut)); + + Assert.Equal(3, doc.Scans[0].PointCount); + Assert.Throws(() => doc.Scans[0].ReadPoints().ToList()); + } + + [Fact] + public void Las_Record_Length_Smaller_Than_The_Format_Needs_Is_Rejected() + { + using var dir = new TempDirectory(); + int @base = SyntheticFixtures.LasBaseRecordSize(2); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, pointFormat: 2, recordLengthOverride: @base - 1); + + Assert.Throws(() => LasDocument.Open(dir.Write("sample.las", bytes))); + } + + [Fact] + public void Las_Oversized_Vlr_Is_Rejected() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, + vlrs: [("LASF_Projection", 34735, new byte[8])]); + // The data length of the first VLR lives at header size + 20. + System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(bytes.AsSpan(227 + 20, 4), uint.MaxValue); + + Assert.Throws(() => LasDocument.Open(dir.Write("sample.las", bytes))); + } + + [Fact] + public void Las_Vlr_Overrunning_Point_Data_Is_Rejected() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, + vlrs: [("LASF_Projection", 34735, new byte[8])]); + // The point data is declared to start right after the header, inside the VLR. + System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(bytes.AsSpan(96, 4), 227); + + Assert.Throws(() => LasDocument.Open(dir.Write("sample.las", bytes))); + } + + [Fact] + public void Las_Too_Many_Vlrs_Are_Rejected() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints); + System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(bytes.AsSpan(100, 4), 5000); + + Assert.Throws(() => LasDocument.Open(dir.Write("sample.las", bytes))); + } + + [Fact] + public void Las_Extra_Bytes_Per_Record_Are_Skipped() + { + using var dir = new TempDirectory(); + byte[] bytes = SyntheticFixtures.CreateSyntheticLas(LasPoints, pointFormat: 2, extraBytesPerRecord: 4); + using var doc = LasDocument.Open(dir.Write("sample.las", bytes)); + + var points = doc.Scans[0].ReadPoints(NoTransform).ToList(); + Assert.Equal(3, points.Count); + AssertNear(new PclVector3(1, 2, 3), points[0].Position); + Assert.Equal(new PclColor(10, 20, 30), points[0].Color); + } } diff --git a/tests/PclIO.Tests/KindDetectionTests.cs b/tests/PclIO.Tests/KindDetectionTests.cs index 213c814..dcbb70c 100644 --- a/tests/PclIO.Tests/KindDetectionTests.cs +++ b/tests/PclIO.Tests/KindDetectionTests.cs @@ -45,7 +45,20 @@ public void Detects_Faro_Fls_With_Container_Signature_And_Extension() Assert.Equal(PclFileKind.FaroFls, Detect(bytes, "scanner.fls")); } + [Theory] + [InlineData("sample.las")] + [InlineData("SAMPLE.LAS")] + [InlineData("sample.bin")] + public void Detects_Las_By_Signature(string name) => + Assert.Equal(PclFileKind.Las, Detect(SyntheticFixtures.CreateSyntheticLas(), name)); + + [Fact] + public void Las_Needs_The_Signature_Not_Just_The_Extension() => + Assert.Equal(PclFileKind.Unknown, Detect(new byte[100], "sample.las")); + [Fact] + public void A_Laz_File_Is_Not_Detected_As_Las() => + Assert.Equal(PclFileKind.Unknown, Detect(SyntheticFixtures.CreateSyntheticLas(), "sample.laz")); [Fact] public void Faro_Needs_The_Container_Signature_Not_Just_The_Extension() => Assert.Equal(PclFileKind.Unknown, Detect(new byte[100], "scanner.fls")); diff --git a/tests/PclIO.Tests/RealFileTests.cs b/tests/PclIO.Tests/RealFileTests.cs index 10861a8..85afbd4 100644 --- a/tests/PclIO.Tests/RealFileTests.cs +++ b/tests/PclIO.Tests/RealFileTests.cs @@ -72,6 +72,9 @@ private static void AssertReadsPoints(string extension, PclFileKind kind) [SkippableFact] public void Read_Real_EdgeWise_Database_Points() => AssertReadsPoints(".c3db", PclFileKind.EdgeWise); + [SkippableFact] + public void Read_Real_Las_Points() => AssertReadsPoints(".las", PclFileKind.Las); + [SkippableFact] public void Export_Real_File_To_Las_Matches_The_Requested_Count() { diff --git a/tests/PclIO.Tests/SyntheticFixtures.cs b/tests/PclIO.Tests/SyntheticFixtures.cs index 8774428..f58a0c8 100644 --- a/tests/PclIO.Tests/SyntheticFixtures.cs +++ b/tests/PclIO.Tests/SyntheticFixtures.cs @@ -14,6 +14,9 @@ namespace PclIO.Tests; /// One cluster of a synthetic EdgeWise cluster file: a centre and 16-bit offsets from it. public readonly record struct C3Cluster(float Cx, float Cy, float Cz, (short Dx, short Dy, short Dz)[] Points); +/// One point of a synthetic LAS file: position, normalised intensity, colour and class code. +public readonly record struct LasSamplePoint(double X, double Y, double Z, float Intensity = 1.0f, PclColor? Color = null, byte Classification = 0); + /// Generates small synthetic scan files for the tests. Every layout is written from what the readers implement; /// no byte of any of them comes from a real scan. internal static class SyntheticFixtures @@ -606,4 +609,176 @@ public static byte[] CreateSyntheticFlsContainer(string? parameters = null, usho if (withGrey) builder.AddStream(one, "Grey0", Enumerable.Range(0, FaroRows * FaroCols).Select(i => (byte)(i * 8)).ToArray()); return builder.Build(); } + + // ---- LAS (.las) --------------------------------------------------------------------------------------------------- + + /// Base record size of a LAS point format; waveform tails are not interpreted. + public static int LasBaseRecordSize(int pointFormat) => pointFormat switch + { + 0 => 20, + 1 => 28, + 2 => 26, + 3 => 34, + 4 => 57, + 5 => 63, + 6 => 30, + 7 => 36, + 8 => 38, + 9 => 59, + _ => 67, + }; + + /// A GeoTIFF variable length record naming an EPSG code, as the LAS exporter writes it. + public static (string UserId, ushort RecordId, byte[] Data) LasEpsgVlr(ushort code) + { + byte[] data = new byte[24]; + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(0, 2), 1); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(2, 2), 1); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(4, 2), 0); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(6, 2), 2); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(8, 2), 1024); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(10, 2), 0); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(12, 2), 1); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(14, 2), 1); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(16, 2), 3072); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(18, 2), 0); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(20, 2), 1); + BinaryPrimitives.WriteUInt16LittleEndian(data.AsSpan(22, 2), code); + return ("LASF_Projection", 34735, data); + } + + /// An OGC WKT coordinate system record (LASF_Projection, record 2112, null-terminated UTF-8). + public static (string UserId, ushort RecordId, byte[] Data) LasWktVlr(string wkt) => + ("LASF_Projection", 2112, Encoding.UTF8.GetBytes(wkt + "\0")); + + /// A synthetic LAS file with one scan. + /// Points to store; empty writes a header with no records. + /// Minor version: 0 to 4 for 1.0 to 1.4. + /// Point data format; 0 to 10 carry points, anything else needs empty points. + /// Metres per stored unit, on every axis. + /// Coordinate offset, or null for the minimum of the points. + /// Variable length records to store. + /// Zero bytes appended to every record past the format's base size. + /// Record length to declare instead of the written one. + /// Legacy point count to declare instead of the written one. + /// Extended 1.4 point count to declare instead of the written one. + public static byte[] CreateSyntheticLas(IReadOnlyList? points = null, int versionMinor = 2, + int pointFormat = 2, double scale = 0.001, (double X, double Y, double Z)? offset = null, + IReadOnlyList<(string UserId, ushort RecordId, byte[] Data)>? vlrs = null, int extraBytesPerRecord = 0, + int? recordLengthOverride = null, uint? legacyCountOverride = null, ulong? extendedCountOverride = null) + { + points ??= [new(1, 2, 3, 0.25f, new PclColor(10, 20, 30), 5), new(-4.5, 5.25, 6.125, 0.5f, null, 2)]; + if ((pointFormat < 0 || pointFormat > 10) && points.Count > 0) + throw new ArgumentException("Only point formats 0 to 10 can hold points.", nameof(pointFormat)); + + bool is14 = versionMinor == 4; + int headerSize = is14 ? 375 : 227; + int baseSize = pointFormat is >= 0 and <= 10 ? LasBaseRecordSize(pointFormat) : 0; + int recordLength = recordLengthOverride ?? (baseSize + extraBytesPerRecord); + + var vlrBytes = new List(); + foreach (var (userId, recordId, data) in vlrs ?? []) + { + byte[] header = new byte[54]; + byte[] userBytes = Encoding.ASCII.GetBytes(userId); + Array.Copy(userBytes, 0, header, 2, Math.Min(userBytes.Length, 16)); + BinaryPrimitives.WriteUInt16LittleEndian(header.AsSpan(18, 2), recordId); + BinaryPrimitives.WriteUInt32LittleEndian(header.AsSpan(20, 4), (uint)data.Length); + Encoding.ASCII.GetBytes("PclIO test").CopyTo(header, 24); + vlrBytes.Add([.. header, .. data]); + } + + uint pointDataOffset = (uint)(headerSize + vlrBytes.Sum(b => b.Length)); + double offsetX = offset?.X ?? (points.Count > 0 ? points.Min(p => p.X) : 0); + double offsetY = offset?.Y ?? (points.Count > 0 ? points.Min(p => p.Y) : 0); + double offsetZ = offset?.Z ?? (points.Count > 0 ? points.Min(p => p.Z) : 0); + + byte[] headerBuf = new byte[headerSize]; + "LASF"u8.CopyTo(headerBuf); + headerBuf[24] = 1; + headerBuf[25] = (byte)versionMinor; + Encoding.ASCII.GetBytes("PclIO synthetic").CopyTo(headerBuf, 26); + Encoding.ASCII.GetBytes("PclIO tests").CopyTo(headerBuf, 58); + BinaryPrimitives.WriteUInt16LittleEndian(headerBuf.AsSpan(94, 2), (ushort)headerSize); + BinaryPrimitives.WriteUInt32LittleEndian(headerBuf.AsSpan(96, 4), pointDataOffset); + BinaryPrimitives.WriteUInt32LittleEndian(headerBuf.AsSpan(100, 4), (uint)(vlrs?.Count ?? 0)); + headerBuf[104] = (byte)pointFormat; + BinaryPrimitives.WriteUInt16LittleEndian(headerBuf.AsSpan(105, 2), (ushort)recordLength); + + uint legacyCount = legacyCountOverride ?? (uint)points.Count; + BinaryPrimitives.WriteUInt32LittleEndian(headerBuf.AsSpan(107, 4), legacyCount); + BinaryPrimitives.WriteUInt32LittleEndian(headerBuf.AsSpan(111, 4), legacyCount); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(131, 8), scale); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(139, 8), scale); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(147, 8), scale); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(155, 8), offsetX); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(163, 8), offsetY); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(171, 8), offsetZ); + + double minX = points.Count > 0 ? points.Min(p => p.X) : 0; + double minY = points.Count > 0 ? points.Min(p => p.Y) : 0; + double minZ = points.Count > 0 ? points.Min(p => p.Z) : 0; + double maxX = points.Count > 0 ? points.Max(p => p.X) : 0; + double maxY = points.Count > 0 ? points.Max(p => p.Y) : 0; + double maxZ = points.Count > 0 ? points.Max(p => p.Z) : 0; + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(179, 8), maxX); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(187, 8), minX); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(195, 8), maxY); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(203, 8), minY); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(211, 8), maxZ); + BinaryPrimitives.WriteDoubleLittleEndian(headerBuf.AsSpan(219, 8), minZ); + + if (is14) + { + ulong extended = extendedCountOverride ?? (ulong)points.Count; + BinaryPrimitives.WriteUInt64LittleEndian(headerBuf.AsSpan(247, 8), extended); + BinaryPrimitives.WriteUInt64LittleEndian(headerBuf.AsSpan(255, 8), extended); + } + + using var ms = new MemoryStream(); + ms.Write(headerBuf); + foreach (byte[] vlr in vlrBytes) ms.Write(vlr); + + bool legacy = pointFormat is >= 0 and <= 5; + // A declared length below the base size cannot hold the format; records are still written whole so that + // the file is otherwise valid and the open path rejects the declared length. + int writtenLength = Math.Max(recordLength, baseSize); + foreach (var p in points) + { + byte[] record = new byte[writtenLength]; + BinaryPrimitives.WriteInt32LittleEndian(record.AsSpan(0, 4), (int)Math.Round((p.X - offsetX) / scale)); + BinaryPrimitives.WriteInt32LittleEndian(record.AsSpan(4, 4), (int)Math.Round((p.Y - offsetY) / scale)); + BinaryPrimitives.WriteInt32LittleEndian(record.AsSpan(8, 4), (int)Math.Round((p.Z - offsetZ) / scale)); + BinaryPrimitives.WriteUInt16LittleEndian(record.AsSpan(12, 2), + (ushort)Math.Clamp(MathF.Round(p.Intensity * 65535f), 0f, 65535f)); + + if (legacy) + { + record[14] = 0x09; + record[15] = (byte)(p.Classification & 0x1F); + } + else if (pointFormat is >= 6 and <= 10) + { + record[14] = 0x11; + record[16] = p.Classification; + } + + if (pointFormat is 2 or 3 or 5) + { + BinaryPrimitives.WriteUInt16LittleEndian(record.AsSpan(20, 2), (ushort)((p.Color?.R ?? 255) * 257)); + BinaryPrimitives.WriteUInt16LittleEndian(record.AsSpan(22, 2), (ushort)((p.Color?.G ?? 255) * 257)); + BinaryPrimitives.WriteUInt16LittleEndian(record.AsSpan(24, 2), (ushort)((p.Color?.B ?? 255) * 257)); + } + else if (pointFormat is 7 or 8 or 10) + { + BinaryPrimitives.WriteUInt16LittleEndian(record.AsSpan(30, 2), (ushort)((p.Color?.R ?? 255) * 257)); + BinaryPrimitives.WriteUInt16LittleEndian(record.AsSpan(32, 2), (ushort)((p.Color?.G ?? 255) * 257)); + BinaryPrimitives.WriteUInt16LittleEndian(record.AsSpan(34, 2), (ushort)((p.Color?.B ?? 255) * 257)); + } + + ms.Write(record); + } + + return ms.ToArray(); + } }