[RFC]: finish the dimensional ndarray constructors (tensor3d typed packages, tensor4d, tensor5d)

[0PrashantYadav0]

Description

This RFC proposes the remaining work to provide dedicated dimensional ndarray constructors up to five dimensions. @stdlib/ndarray/vector/* and @stdlib/ndarray/matrix/* cover one and two dimensions. @stdlib/ndarray/tensor3d/ctor covers the three-dimensional case and is in review (see Related Issues). What is left is the typed three-dimensional constructors, the tensor3d namespace package, and the same three groups again for four and five dimensions.

Dimensional constructors through 5D cover the large majority of end-user ndarray dimensionality. Arrays of six or more dimensions are uncommon and the general-purpose @stdlib/ndarray/ctor and @stdlib/ndarray/array APIs already handle them.

What is already done

@stdlib/ndarray/tensor3d/ctor is written and under review.

What remains

Forty-one packages, grouped by dimensionality.

Group	Packages	Count	Approximate size
tensor3d typed constructors	bool, complex64, complex128, float32, float64, int8, int16, int32, uint8, uint8c, uint16, uint32	12	38,600 lines
ndarray/tensor3d namespace package	re-exports tensor3d and the twelve typed constructors	1	4,500 lines
tensor4d	ctor plus the same twelve typed constructors plus the namespace package	14	54,000 lines
tensor5d	ctor plus the same twelve typed constructors plus the namespace package	14	54,000 lines

The sizes come from measuring the equivalent vector and matrix packages and the finished tensor3d/ctor, which is 10,928 lines. Call it 150,000 lines in total. That number matters, and I return to it below.

Data types and constructor names

The typed set matches vector, which ships twelve. matrix currently ships five, and separate work is under way to bring it up to the full set.

Data type	Constructor	Description used in package.json
float64	Float64Tensor3D	double-precision floating-point
float32	Float32Tensor3D	single-precision floating-point
complex128	Complex128Tensor3D	double-precision complex floating-point
complex64	Complex64Tensor3D	single-precision complex floating-point
int32	Int32Tensor3D	signed 32-bit integer
int16	Int16Tensor3D	signed 16-bit integer
int8	Int8Tensor3D	signed 8-bit integer
uint32	Uint32Tensor3D	unsigned 32-bit integer
uint16	Uint16Tensor3D	unsigned 16-bit integer
uint8	Uint8Tensor3D	unsigned 8-bit integer
uint8c	Uint8ClampedTensor3D	clamped unsigned 8-bit integer
bool	BooleanTensor3D	boolean

Two names do not follow a literal transliteration of the data type, and both follow vector: bool exports BooleanTensor3D, and uint8c exports Uint8ClampedTensor3D. The 4D and 5D packages should use the same names with the dimensionality changed.

One wording note. Integer descriptions in this repository come in two forms, and the counts are lopsided: 39 packages write "signed 32-bit integer" and 3 write "32-bit signed integer". matrix/int32 is one of the three. New packages should use the majority form.

Prerequisites, all already present

Nothing needs to be built first.

• @stdlib/array/base/flatten2d through flatten5d all exist. There is no flatten6d, so the dependency chain stops exactly where this plan stops.
• @stdlib/types/ndarray declares Shape0D through Shape10D, so Shape4D and Shape5D are ready for the TypeScript declarations.
• Each typed package is a thin wrapper over require( '@stdlib/ndarray/<group>/ctor' ).factory( dtype ). The work in a typed package is documentation, TypeScript declarations, tests, and benchmarks, not logic.

Porting notes for whoever writes tensor4d and tensor5d

Adding a dimension shifts every ArrayBuffer signature form up one position, so the argument-count dispatch is not a find-and-replace. Going from 2D to 3D, seven places needed real thought. The same seven will need it again at 4D and 5D.

1. Maximum arity grows by one. lib/main.js gains a branch and the terminal fall-through handles one more argument.
2. A form disappears. matrix( M, N ) has no 3D counterpart, so two nonnegative integers stopped being a valid shape. At 4D, three integers stop being valid. This branch gets deleted rather than shifted, and it is the easiest change to miss.
3. The lowest shape-specifying arity gains the new full form and loses the ( dims..., dtype ) and ( dims..., options ) forms, which move up one.
4. The default ArrayBuffer shape gains a leading 1. At 3D it is [ 1, 1, N ]. At 4D it is [ 1, 1, 1, N ].
5. The collection and iterable paths need sh.length < N and slice( sh, 0, N ) updated, and the flattenNd import changed.
6. Shape length assertions move from x.length === 2 to the new dimensionality at four call sites.
7. Complex data type handling needs no change. The interleaved real and imaginary sniff adjusts the trailing dimension with sh[ sh.length-1 ] /= 2, which does not depend on dimensionality. Leave it alone.

Two defects got through the 3D port and were only caught later. Both are worth watching for.

Two TypeScript overloads kept the old dimension count. The signatures reading ( buffer, byteOffset, M, N, options ) needed a K inserted, and did not get one, while the JSDoc above them already listed K, M, N. Runtime tests cannot catch this because .d.ts files never execute. Only the repository's TypeScript lint found it. The same two overloads exist in every typed package, so the mistake multiplied to 26 signatures before anyone noticed.

Shape literals in final argument position lost the space before the closing paren. tensor3d( [ 1, 2, 5 ]) instead of ] ). The test file ESLint configuration does not enforce space-in-parens, so the pre-commit hook passed it. 249 occurrences.

Both argue for the same thing: run the full lint set, including the TypeScript declarations lint, before assuming a mechanical port is finished.

Namespace registration

• Each group needs an ndarray/<group> namespace package re-exporting the constructor and the typed constructors, matching ndarray/vector and ndarray/matrix.
• Each group needs an alias in @stdlib/namespace, added to lib/namespace/<letter>.js. The pkg2alias, alias2pkg, pkg2related, pkg2standalone, and standalone2pkg data files are generated from that source by each subpackage's scripts/build.js, so the entry and the regenerated data need to land together.
• There is an existing inconsistency to settle. ndarray/lib/index.js registers vector in the top-level @stdlib/ndarray namespace but does not register matrix. Whichever way that gets resolved, tensor3d, tensor4d, and tensor5d should follow the same rule.

On writing the dispatch four times

I recommended deferring this question when tensor3d/ctor started, on the grounds that the argument-count dispatch resists clean parameterization and that departing from the vector and matrix precedent would cost reviewers more than it saved. Having now finished the 3D port, I want to revise that.

The case for extracting a shared generator into @stdlib/ndarray/base is stronger than I expected:

• The remaining work is roughly 150,000 lines, most of it near-identical. Two typed packages differ by about 3.6% of their lines once you normalize the data type name, the constructor name, and the description.
• Mechanical copying propagates defects. The matrix/ctor factory bug described below survived into a released package precisely because nobody re-derived that branch. The two defects listed above came from the same kind of copying.
• The seven adaptation points are the same seven at every dimensionality. That is a pattern, not a series of one-off judgments.

The case against has not changed: error messages are positional ("Third argument must be a nonnegative integer"), so a parameterized version has to generate them; and the JSDoc annotations and TypeScript overloads still need writing by hand per package regardless, which is where most of the line count actually sits.

I lean toward extracting the generator before tensor4d begins, and I would rather settle it now than after another 54,000 lines exist. But the line count sits mostly in documentation and tests, which a generator does not remove, so this is a judgment call for maintainers rather than something the evidence decides outright.

Related Issues

Related issues #14959

Questions

1. Should the shared dispatch generator be extracted before tensor4d starts, or should 4D and 5D follow the same copy-and-adapt path as 3D?
2. Should matrix be registered in the top-level @stdlib/ndarray namespace alongside vector, so the three tensor groups have a consistent rule to follow?
3. Should the twelve typed tensor3d constructors land as one pull request or as several smaller ones? They are independent of each other and each is about 3,200 lines.

Other

Duplication of from_arraybuffer.js

lib/from_arraybuffer.js is dimension-agnostic and gets copied verbatim into each ctor package. It carries a TODO noting it should move to @stdlib/ndarray/from-arraybuffer. There are three copies now. Following this plan without acting on the TODO would make five. Separate work on @stdlib/ndarray/from-arraybuffer is already under way, which would reclaim all of them.

A defect in matrix/ctor, found and fixed while porting

The ( M, N, options ) branch of matrix/ctor's factory method passed the raw user options object to the main entry point instead of merging it into the validated defaults. Every other branch merged correctly, and so did vector/ctor, which made matrix the only package with the problem:

var f = factory( 'float64', { 'readonly': true } );

isReadOnly( f( 2, 2 ) );      // true
isReadOnly( f( 2, 2, {} ) );  // false, the default was discarded

Passing any options to that one signature silently threw away every default set when the factory was created, covering readonly, order, mode, and submode. The fix and a regression test are ready. The three-dimensional implementation follows the corrected behavior.

Worth checking the same class of problem elsewhere. Only three packages use this defaults-merging factory pattern: vector/ctor, matrix/ctor, and tensor3d/ctor. Exercising all 43 signature forms across the three with factory-level defaults set confirms nothing else discards them. The other factories that validate options (array/to-fancy, ndarray/to-fancy, random/base/beta, random/base/gamma, random/base/negative-binomial) return functions that take no options at all, so they cannot hit this.

Checklist

• I have read and understood the Code of Conduct.
• Searched for existing issues and pull requests.
• The issue name begins with RFC:.

[0PrashantYadav0]

@kgryte I need few answer mentioned above, after that i will create sub issues for this. Also I mention few bugs above and will open PR for them as well.