Skip to content

Trim the README to what a consumer needs to get started #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ww-mw merged 1 commit into from
Sep 16, 2026
Merged

Trim the README to what a consumer needs to get started #31

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 84 additions & 113 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,87 +1,64 @@
# data-explorer-core

Core data model for Data Explorer: the parser, data model, and node schema for
Simulink data-dictionary (`.sldd`), model (`.slx`, `.mdl`), MAT-file (`.mat`),
and project (`.prj`) files.
Parser, data model, and node schema for Simulink data dictionaries (`.sldd`),
models (`.slx`, `.mdl`), MAT-files (`.mat`), and projects (`.prj`).
Presentation-independent and embeddable in-process.

Presentation-independent and embeddable in-process. Used by the Simulink Data
Explorer VS Code extension, and shared across other Data Explorer front-ends.
This is the host layer of the [Simulink Data Explorer VS Code
extension](https://github.com/mathworks/data-explorer-vscode), its main
consumer: host and data-model changes land here first, then the extension moves
its pin to a tag.

## Install
## Consuming it

```bash
npm install data-explorer-core
```json
"dependencies": {
"data-explorer-core": "github:mathworks/data-explorer-core#v1.21.0"
}
```

## Usage
Releases are tags, and `dist/` is committed, so installing needs no build step.
The package barrel is the whole supported surface — import from
`data-explorer-core`, never a deep path into `src/`.

```js
import { createSession } from 'data-explorer-core';
## Opening a file

const session = createSession();
const src = session.addDataSource('params.sldd', slddJson);
session.getDataSourceIds(); // ['params.sldd']
session.editProperty(id, 'Value', 5);
session.undo();
```

Each `createSession()` returns an independent instance with its own data
sources, node index, undo stacks, and event bus, so multiple sessions never
share state. Stateless parsers (`parseSlx`, `parseMdl`, `parseModel`, `parseMat`,
`parseProject`, `parseBinarySldd`) are also exported for consumers that only need
to parse a buffer.

Both model containers open to the same tree. A `.slx` is a zip of parts; a `.mdl`
is either the modern OPC *text* package carrying that same part set, or the
pre-R2012 nested-brace text that a model which was never migrated still has.
`parseModel` takes either one and picks the reader from the bytes.

Neither container has one fixed part layout, and the reader covers both eras of
each. A `.slx` written by R2026b or later stores its block diagram, config set
index and graphical interface as JSON; every earlier release wrote XML, kept its
blocks inside the block diagram before R2020a, and its model workspace in a plain
MAT-file part before R2019b. All of those read back to the same tree, down to the
values in the model workspace. Two things a pre-R2020a file simply does not record
are reported as absent rather than guessed: the model UUID (new in R2020a) and, for
a pre-R2014a file, a linked data dictionary. See
`test/parity/matlab/README.md` for the full layout matrix and what the parity suite
holds each era to.

### Universal ingest

Instead of choosing an `addXSource` by type, hand core the content plus a
filename and it sniffs (extension + magic bytes) and dispatches:
`DataModel` is a session: parse the bytes, hand the result over, read the tree.

```js
import { createSession, ingest, toDTO } from 'data-explorer-core';
const session = createSession();
const src = ingest(session, bytesOrTextOrObject, { filename: 'params.sldd' });
const snapshot = toDTO(src, { depth: 2 }); // JSON-safe; for RPC boundaries
import { DataModel, parseModel, readSlddContent } from 'data-explorer-core';

DataModel.addDataSource('params.sldd', readSlddContent(dictBytes), { path: 'params.sldd' });
DataModel.addModelSourceParsed('a.slx', parseModel(slxBytes, 'a.slx'));
DataModel.getDataSourceIds(); // ['params.sldd', 'a.slx']
DataModel.editProperty(id, 'Value', 5);
DataModel.undo();
```

`ingest` and `toDTO` are universal — no `fs` — so they run identically in the
browser (uploaded `ArrayBuffer`) and in Node. `toDTO` projects a live node into
a flat, serializable snapshot (child *ids*, not object references) for consumers
that cross a process or network boundary; in-process consumers hold live nodes
directly.
`srcId` is the caller's own key for a file and is never parsed; the filename is
what decides the kind. `createSession()` returns an independent session with the
same surface — its own sources, node index, undo stacks and event bus — for a
host that needs more than one.

To load from a filesystem path, import the Node-only subpath (the only part of
the package that touches `fs`; fenced out of browser bundles by `exports`):
`parseModel` takes either model container and picks the reader from the bytes.
The stateless parsers are exported for callers that only want a buffer read:
`parseSlx`, `parseMdl`, `parseMat`, `parseProject`, `parseBinarySldd`, plus
`scanSldd` / `scanMat` / `scanModelStructure` for the cheap pass that reads a
file's names and references without building a tree.

Filesystem loading lives in the Node-only subpath, the only part that touches
`fs` (`exports` fences it out of browser bundles):

```js
import { createSession, loadFromPath, loadDirectory } from 'data-explorer-core/node';
const s = createSession();
loadFromPath(s, 'params.sldd');
loadDirectory(s, 'some_dir/'); // all .sldd/.slx/.mdl/.mat/.prj into one session
```

### Usage across a folder
## Usage across a folder

`session.findUsages(nodeId)` answers "what refers to this?" for sources the
session holds, off their node trees. For a whole folder that is the wrong shape —
asking about a hundred models would mean holding a hundred trees — so the same
question over a set of *read but unopened* files is answered from file summaries
instead:
`session.findUsages(nodeId)` answers "what refers to this?" off the trees a
session holds. For a whole folder that is the wrong shape — asking about a
hundred models would mean holding a hundred trees — so the same question over
*read but unopened* files is answered from per-file summaries:

```js
import { buildUsageIndex } from 'data-explorer-core';
Expand All @@ -90,57 +67,51 @@ const index = buildUsageIndex([
{ srcId: 'a.slx', filename: 'a.slx', bytes: modelBytes },
{ srcId: 'params.sldd', filename: 'params.sldd', bytes: dictBytes },
]);
index.usagesOf('params.sldd', 'Kp'); // the blocks that read Kp, with link targets
index.paramsOf('a.slx', '15'); // each parameter of that block, and where it resolved
index.usagesOf('params.sldd', 'Kp'); // the blocks that read Kp, with link targets
index.paramsOf('a.slx', '15'); // each parameter of that block, and where it resolved
```

A block is addressed by its **SID** — `'15'` above — because a block name is unique
within its own system only: a model may hold four blocks named `Gain`, each with its own
gain. A row projected by the data model publishes it as `_blockKey`, and a usage's
`linkTarget` is `` `${sid}@${srcId}` ``. What a cell *shows* is the block's name, or
`<SID: 15>` for a block whose label the file leaves blank.

Because those four `Gain` blocks are now four rows reading the same word, each also
carries **where it is**: the model-relative path through the subsystems that hold it,
as `_systemPath` (the enclosing systems) and `_blockPath` (those plus the block) on a
row, and as `blockPath` on a `NodeUsage` from either resolver. A `/` inside a name is
doubled, as `getfullname` writes it. The three rules are exported for a host that
indexes parse results itself — `blockKey(name, sid)`, `blockLabel(name, sid)` and
`joinBlockPath(parentPath, label)`.

Not every block parameter is reported: only those that *can* name data. A value that
is a number, a non-finite, `on`/`off` or has no identifier in it at all is not a
reference, and neither is a parameter whose value space is a fixed option list — a Math
block's `Operator = square` reads like a variable and is one of fifteen menu choices
Simulink enforces. That table is keyed on the `(BlockType, parameter)` pair and measured
from MATLAB rather than hand-written, because the same name can be a menu on one block
and an expression on another; the few free-text parameters that measurably never name
data (a Bus Selector's `OutputSignals`, a Model block's file name) are excluded the same
way. A block type the tables do not know is judged on its value alone, so a toolbox
block is never silently reduced to fewer rows than it has.

A reported parameter can still resolve to nothing — `linkTarget: ''` with nulls
throughout — and that is an answer, not an absence: the name may be defined in a file the
caller did not hand over, or in the MATLAB base workspace, which is a live session and not
a file. Render the value, without a link.

A `srcId` is the caller's own key for a file and is never parsed; the `filename`
is what decides the kind. Resolution follows MATLAB: the mask workspaces the block
sits inside shadow the model workspace, which shadows a linked dictionary, which
shadows a linked MAT-file, dictionary references are followed transitively, and a
reference is matched without regard to case, as the file systems these live on do.
Answers are `NodeUsage`, the shape `findUsages` returns, so a consumer renders one
cell whichever resolver produced it. An unreadable file in the set contributes
nothing and does not fail the rest.

The innermost of those scopes is the only one that is not a file. A masked
subsystem's parameters are visible to the blocks inside it, so `Gain = g1` there
may name a mask parameter rather than anything in the workspace — and the mask
parameter's own value (`g1 = g1_param`) is an expression evaluated *outside* the
mask, which is what makes `g1_param` used by the masked block itself. `paramsOf`
reports that as `kind: 'mask'` with the masked block in `maskBlock`, and its
`linkTarget` is that block's `` `${sid}@${srcId}` `` rather than a name — so a host
routes it to wherever it shows blocks, not to a workspace row.
Its halves are exported for a host that caches: summarize each file
(`summarizeFiles`, or `summarizeParsedModel` / `summarizeSlddScan` /
`summarizeMatScan` off a parse or scan you already have),
`mergeFileSummaries` in workspace order, then `buildUsageIndexFromSummaries`.

Four things a consumer cannot guess:

- A block is addressed by its **SID** — `'15'` above — because a block name is
unique only within its own system, and one model may hold four blocks named
`Gain`. Rows publish it as `_blockKey`; a `linkTarget` is
`` `${sid}@${srcId}` ``. Each row also carries where it is, as `_systemPath` /
`_blockPath`.
- Resolution follows MATLAB's shadowing: mask workspaces, then the model
workspace, then a linked dictionary, then a linked MAT-file, with dictionary
references followed transitively and names matched case-insensitively.
- `linkTarget: ''` is an answer, not an absence — the name may be defined in a
file the caller never handed over, or in the base workspace, which is a live
session and not a file. Render the value, without a link.
- An unreadable file in the set contributes nothing and does not fail the rest.

## Detail that lives elsewhere, deliberately

Neither model container has one fixed part layout, and the readers cover both
eras of each — JSON parts and XML, blocks inside the block diagram, a MAT-file
model workspace part, and the two things a pre-R2020a file simply does not
record and so reports as absent rather than guessing.
`test/parity/matlab/README.md` holds the layout matrix and what the parity suite
pins each era to; it is the reference for this, not the file you are reading.

Same for which block parameters can name data: that table is keyed on the
`(BlockType, parameter)` pair and measured from MATLAB rather than
hand-written, and it is documented beside itself.

## Development

```bash
npm install
npm run typecheck
npm test
npm run build # writes dist/, which is committed
```

## License

Expand Down