Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ All notable changes to the `NwcIO` library, the CLI and the `NwcIO.Scene` adapte
### Added
- Cancellation and progress overloads for `NwcDocument.Open`, `NwcAssembly.LoadFromDirectory` (also with several patterns and recursion) and `NwcAssembly.OpenNwf` and `NwcAssembly.OpenManifest`, plus `OpenAsync`, `LoadFromDirectoryAsync`, `OpenNwfAsync` and `SaveAsync` (`.nwc`) wrappers on `Task.Run`. `NwcProgress` reports `read-container`, `read-geometry`, `open` and per-model `load-models` counts (`OpenNwf` counts every reference, so `Completed` reaches `Total`). The token is checked between phases and models; a running container read is not interrupted, and `SaveAsync` checks it only before it starts.
- `meshes-skipped` reader diagnostic: `NwcDocument.Diagnostics` reports geometry records that the lenient reader path skipped (vertex count out of range, truncated or malformed record). `nwcio --report` prints `NwcDocument.Diagnostics` and `NwcAssembly.Diagnostics` for a single file, a folder and an `.nwf`.
- Reader diagnostics also cover property-scanner failures (`property-scan-failed`, Warning: the first malformed string a property scan met, with its stream index and byte offset; not raised on the strict DbCache path) and are written to document and assembly JSON as a `diagnostics` array. `NwcReaderOptions.Strict` throws `InvalidDataException` when an error-level diagnostic is recorded.
- `HierarchyParser` keeps linked Revit placements instead of throwing: a repeated element under another parent is marked `NwcNode.IsLinkPlacement` and points at the first placement. `NwcSceneReader` writes each placement as its own `SceneModel`, so its geometry and `SourceId` are preserved without duplicate ids inside one model.
- `NwcIO.Scene` converts an `NwcAssembly` into one scene with one `SceneModel` per component document, unique keys from file names, and scene-wide element, geometry and material ids. The `.nwf` document itself is skipped when components exist.
- `NwcIO.Scene` writes Revit level elevations in metres from the level's `LcRevitData_Parameter` length property; a missing, non-finite or unknown-unit value stays null.
Expand Down
4 changes: 2 additions & 2 deletions docs/public-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ The supported surface is the `NwcIO` and `NwcIO.Model.*` namespaces listed here
| `NwcProgress` | Progress payload for the cancellation-aware overloads: a phase name, completed units, and total units (0 when unknown). |
| `NwcDocumentBuilder` | Fluent builder API for creating Navisworks documents and geometry programmatically from code. |
| `NwcWriter` | Direct `#LcUStream-V1.1` binary container serializer and continuous zlib chunk writer. Throws `ArgumentException` before writing for input it cannot store faithfully: an unrecognised header, scaled transforms, line meshes, triangle indices outside the vertex buffer, out-of-range vertex counts or coordinates, records or nodes without a name/class, and viewpoint names the reader would rename. Mesh ids, names and triangle indices are stored; normals and materials are not. `Materials`, `XRefs`, `DbCache`, `AppearanceProfiler`, `ClashTests` and `GridSystem` are not written; the output is read back by NwcIO but has not been verified in Autodesk Navisworks. |
| `NwcReaderOptions` | Per-open options: which streams to read (property records, GUIDs, transforms, meshes, viewpoints, selection sets, TimeLiner), whether to read materials (`ReadMaterials`: decoded from the file on the strict path, named in property records elsewhere, with a neutral inferred colour unless a `revit_Color` record gives one), whether to build the scene tree (`BuildSceneTree`), catalog retention. |
| `NwcReaderOptions` | Per-open options: which streams to read (property records, GUIDs, transforms, meshes, viewpoints, selection sets, TimeLiner), whether to read materials (`ReadMaterials`: decoded from the file on the strict path, named in property records elsewhere, with a neutral inferred colour unless a `revit_Color` record gives one), whether to build the scene tree (`BuildSceneTree`), catalog retention, and `Strict` (throw `InvalidDataException` when an error-level diagnostic is recorded). |
| `NwcAssembly` | Federated multi-model assembly: `Create`, `LoadFromDirectory` (several patterns, recursion), `OpenManifest` (`.nwf.xml`), `OpenNwf` (with `UnresolvedXRefs`/`SkippedXRefs` reporting), combined meshes/materials/bounds, per-assembly exports and clash runs. |
| `NwcDiagnostic` / `NwcDiagnosticSeverity` | Reader warning for skipped or substituted data (`Severity`, stable `Code`, `Message`, optional `StreamIndex` and `ByteOffset`), listed in `NwcDocument.Diagnostics` and `NwcAssembly.Diagnostics`. Codes: `meshes-skipped` (geometry records dropped because their vertex count is out of range or they are truncated or malformed; not raised on the strict DbCache path), `ContainerTrailingGap` (bytes after the last container stream, including a truncated zlib stream), `NwfUnresolvedXref` and `NwfSkippedXref` (from `NwcAssembly.OpenNwf`). `nwcio --report` prints them. |
| `NwcDiagnostic` / `NwcDiagnosticSeverity` | Reader warning for skipped or substituted data (`Severity`, stable `Code`, `Message`, optional `StreamIndex` and `ByteOffset`), listed in `NwcDocument.Diagnostics` and `NwcAssembly.Diagnostics`. Codes: `meshes-skipped` (geometry records dropped because their vertex count is out of range or they are truncated or malformed; not raised on the strict DbCache path), `property-scan-failed` (the first malformed string met by the lenient property scan, with its stream index and byte offset; not raised on the strict DbCache path), `ContainerTrailingGap` (bytes after the last container stream, including a truncated zlib stream), `NwfUnresolvedXref` and `NwfSkippedXref` (from `NwcAssembly.OpenNwf`). They are included in JSON export (`diagnostics`) and printed by `nwcio --report`. |
| `NwcNode` | Scene graph tree node representing layers, categories, composite elements, and geometry instances with optional transformations and properties. For Navisworks model files read through their DbCache directory (the strict path), `Meshes` lists every mesh placed on a leaf and `Mesh` the first, ids are depth-first preorder indices, `Transform` is null; `IsElement` marks Revit element nodes, `ElementGuid` is their export GUID from `LcOpGuidStore`, `IfcGlobalId` is the GlobalId the Revit IFC export gives the element: its valid `IFC_GUID` parameter, else its export GUID in the IFC base-64 form (the rule of `RvtIO.Scene`), and `Element` points every node inside an element (and the second node of a sibling pair that repeats one element) to its element node; properties include the tabs stored in `LcOaPartitionProps`; other files leave `Meshes` empty and `Mesh` null. |
| `NwcPropertyRecord` | Property record (`Class`, `Name`, `Value`, `ValueType`, `Tag`, `FirstInt`; typed `IntValue`, `DoubleValue`, `ReferenceId`, `ReferenceTag`, `ReferenceClass`). On node properties of the strict path, `Class` is the property tab name (category, for example `Element`), `Name` the property name and `FirstInt` the key object id. Records scanned from property streams (`NwcDocument.PropertyRecords`, and node properties of other files) keep their internal class names. `TabInternalName` and `InternalName` are the internal names of the tab and the key on strict-path node properties, null on scanned records. |
| `NwcPropertyTab` | One property tab of a strict-path node (`NwcNode.PropertyTabs`, in stored slot order): display `Name`, `InternalName`, `Properties`. A tab object shared by several nodes is one instance. |
Expand Down
22 changes: 22 additions & 0 deletions src/NwcIO/Internal/Json/JsonExporter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,28 @@ public static void Write(NwcDocument document, Stream output, NwcJsonExportOptio
writer.WriteEndArray();
}

if (document.Diagnostics.Count > 0)
{
writer.WriteStartArray("diagnostics");
foreach (var diagnostic in document.Diagnostics)
{
writer.WriteStartObject();
writer.WriteString("severity", diagnostic.Severity.ToString());
writer.WriteString("code", diagnostic.Code);
writer.WriteString("message", diagnostic.Message);
if (diagnostic.StreamIndex is int streamIndex)
{
writer.WriteNumber("streamIndex", streamIndex);
}
if (diagnostic.ByteOffset is long byteOffset)
{
writer.WriteNumber("byteOffset", byteOffset);
}
writer.WriteEndObject();
}
writer.WriteEndArray();
}

if (options.IncludeStreamCatalog)
{
WriteStreamCatalog(writer, document);
Expand Down
18 changes: 17 additions & 1 deletion src/NwcIO/Internal/Parsing/PropertyRecordScanner.cs
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,16 @@ internal sealed class PropertyRecordScanner

public string? FailureReason { get; private set; }

/// <summary>Byte offset of the first malformed probe; null when no probe failed.</summary>
public int? FailureOffset { get; private set; }

public List<NwcPropertyRecord> Scan()
{
var records = new List<NwcPropertyRecord>();
_pos = 0;
StoppedAt = 0;
FailureReason = null;
FailureOffset = null;

while (_pos + 8 <= _data.Length)
{
Expand Down Expand Up @@ -207,12 +211,24 @@ private bool TryReadLengthPrefixedString(ref int pos, out string? value)
value = null;
if (pos + 4 > _data.Length)
{
FailureReason ??= $"A length-prefixed string at byte {pos} runs past the end of the stream.";
FailureOffset ??= pos;
return false;
}

int len = ReadInt32At(pos);
if (len is < 0 or > 65536 || pos + 4 + len > _data.Length)
if (len == -1)
{
// The format uses -1 for a null string; it is valid in both strict and lenient readers.
value = null;
pos += 4;
return true;
}

if (len < 0 || len > 65536 || pos + 4 + len > _data.Length)
{
FailureReason ??= $"A length-prefixed string at byte {pos} has invalid length {len}.";
FailureOffset ??= pos;
return false;
}

Expand Down
22 changes: 22 additions & 0 deletions src/NwcIO/Model/NwcAssembly.cs
Original file line number Diff line number Diff line change
Expand Up @@ -930,6 +930,28 @@ public void SaveJson(Stream stream, NwcJsonExportOptions? options = null)
writer.WriteString("size", BoundingBox.Size.ToString());
writer.WriteEndObject();

if (Diagnostics.Count > 0)
{
writer.WriteStartArray("diagnostics");
foreach (var diagnostic in Diagnostics)
{
writer.WriteStartObject();
writer.WriteString("severity", diagnostic.Severity.ToString());
writer.WriteString("code", diagnostic.Code);
writer.WriteString("message", diagnostic.Message);
if (diagnostic.StreamIndex is int streamIndex)
{
writer.WriteNumber("streamIndex", streamIndex);
}
if (diagnostic.ByteOffset is long byteOffset)
{
writer.WriteNumber("byteOffset", byteOffset);
}
writer.WriteEndObject();
}
writer.WriteEndArray();
}

writer.WriteStartArray("models");
foreach (var m in Models)
{
Expand Down
25 changes: 22 additions & 3 deletions src/NwcIO/NwcDocument.cs
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ public static NwcDocument Open(

string rootName = Path.GetFileNameWithoutExtension(fileInfo.SourcePath ?? fallbackName ?? "Model");

var diagnostics = new List<NwcDiagnostic>();
var records = new List<NwcPropertyRecord>();
cancellationToken.ThrowIfCancellationRequested();
if (options.ReadPropertyRecords)
Expand All @@ -192,6 +193,16 @@ public static NwcDocument Open(
{
var scanner = new PropertyRecordScanner(s.Data);
records.AddRange(scanner.Scan());
if (segments is null && scanner.FailureReason is not null)
{
// The lenient scanner searches forward; a rejected probe is not a lost record.
diagnostics.Add(new NwcDiagnostic(
NwcDiagnosticSeverity.Warning,
"property-scan-failed",
scanner.FailureReason,
s.Index,
scanner.FailureOffset));
}
}
}

Expand Down Expand Up @@ -311,9 +322,6 @@ public static NwcDocument Open(
}

cancellationToken.ThrowIfCancellationRequested();
progress?.Report(new NwcProgress("open", 1, 1));
var document = new NwcDocument(fileInfo, records, guids, transforms, meshes, materials, rootNode, xRefs, dbCache, savedViewpoints, selectionSets, timeLiner, container);
var diagnostics = new List<NwcDiagnostic>();
if (skippedMeshRecords > 0)
{
diagnostics.Add(new NwcDiagnostic(
Expand All @@ -332,6 +340,17 @@ public static NwcDocument Open(
ByteOffset: container.FileLength - container.TrailingGap));
}

if (options.Strict)
{
var errors = diagnostics.Where(d => d.Severity == NwcDiagnosticSeverity.Error).ToList();
if (errors.Count > 0)
{
throw new InvalidDataException(string.Join(" ", errors.Select(d => d.Message)));
}
}

progress?.Report(new NwcProgress("open", 1, 1));
var document = new NwcDocument(fileInfo, records, guids, transforms, meshes, materials, rootNode, xRefs, dbCache, savedViewpoints, selectionSets, timeLiner, container);
document.Diagnostics = diagnostics;

return document;
Expand Down
6 changes: 6 additions & 0 deletions src/NwcIO/NwcReaderOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,10 @@ public sealed class NwcReaderOptions
/// Model files read through their DbCache directory use the strict geometry parser and its own bounds.
/// </summary>
public int MaxMeshVertices { get; set; } = Internal.Parsing.MeshParser.MaxPlausibleVertices;

/// <summary>
/// Throw <see cref="InvalidDataException"/> when the reader records an error-level diagnostic
/// (default: false, diagnostics are only recorded).
/// </summary>
public bool Strict { get; set; }
}
105 changes: 105 additions & 0 deletions tests/NwcIO.Tests/DiagnosticsExportTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
using System.Buffers.Binary;
using System.Text;
using NwcIO.Internal.Container;
using NwcIO.Model;
using Xunit;

namespace NwcIO.Tests;

public sealed class DiagnosticsExportTests
{
[Fact]
public void Open_MalformedPropertyRecord_ReportsPropertyScanFailedWarning()
{
using var stream = new MemoryStream(MalformedSceneTreeFile());

var document = NwcDocument.Open(stream);

var diagnostic = Assert.Single(document.Diagnostics, d => d.Code == "property-scan-failed");
Assert.Equal(NwcDiagnosticSeverity.Warning, diagnostic.Severity);
Assert.Contains("invalid length", diagnostic.Message);
Assert.Equal(0, diagnostic.StreamIndex);
Assert.Equal(16L, diagnostic.ByteOffset);
}

[Fact]
public void Open_StrictWithMalformedPropertyRecord_DoesNotThrow()
{
using var stream = new MemoryStream(MalformedSceneTreeFile());

var document = NwcDocument.Open(stream, new NwcReaderOptions { Strict = true });

Assert.DoesNotContain(document.Diagnostics, d => d.Severity == NwcDiagnosticSeverity.Error);
}

[Fact]
public void JsonExport_IncludesDocumentAndAssemblyDiagnostics()
{
using var stream = new MemoryStream(MalformedSceneTreeFile());
var document = NwcDocument.Open(stream);

string json = document.ToJson();
Assert.Contains("\"diagnostics\"", json);
Assert.Contains("property-scan-failed", json);

var assembly = NwcAssembly.Create("Fed", document);
using var assemblyStream = new MemoryStream();
assembly.SaveJson(assemblyStream);
string assemblyJson = Encoding.UTF8.GetString(assemblyStream.ToArray());
Assert.Contains("\"diagnostics\"", assemblyJson);
Assert.Contains("property-scan-failed", assemblyJson);
}

[Fact]
public void Open_NullPropertyString_IsValidAndStrictDoesNotThrow()
{
var data = new byte[20];
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(0), 100);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(4), 86);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(8), 0);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(12), 53);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(16), -1);
byte[] file = NwcContainerWriter.Write([data]);

var document = NwcDocument.Open(new MemoryStream(file), new NwcReaderOptions { Strict = true });

Assert.DoesNotContain(document.Diagnostics, d => d.Code == "property-scan-failed");
}

[Fact]
public void Scan_NullValueString_KeepsRecordAndAdvancesPastPrefix()
{
// tag 52 record: name "a", class "b", valueType 4 (string) with a null (-1) value, then tag 53 node "n".
var ms = new MemoryStream();
using (var w = new BinaryWriter(ms, Encoding.UTF8, leaveOpen: true))
{
w.Write(1); w.Write(52);
w.Write(1); w.Write(new byte[] { (byte)'a', 0, 0, 0 });
w.Write(1); w.Write(new byte[] { (byte)'b', 0, 0, 0 });
w.Write(4); w.Write(-1);
w.Write(2); w.Write(53);
w.Write(1); w.Write(new byte[] { (byte)'n', 0, 0, 0 });
}

var scanner = new NwcIO.Internal.Parsing.PropertyRecordScanner(ms.ToArray());
var records = scanner.Scan();

Assert.Equal(2, records.Count);
Assert.Null(records[0].Value);
Assert.Equal("n", records[1].Name);
Assert.Null(scanner.FailureReason);
}

private static byte[] MalformedSceneTreeFile()
{
// The first two ints classify the stream as SceneTree; tag 53 at byte 12 has an invalid -1 length.
var data = new byte[24];
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(0), 100);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(4), 86);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(8), 0);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(12), 53);
BinaryPrimitives.WriteInt32LittleEndian(data.AsSpan(16), -2);
return NwcContainerWriter.Write([data]);
}
}

Loading