Add directed hypergraph layers and Cora node classification example by shonalidixit · Pull Request #15 · CoReACTER/HyperGraphNeuralNetworks.jl

shonalidixit · 2026-08-19T14:50:03Z

This PR adds several additional directed hypergraph neural network layers, together with their associated tests and documentation.

It also adds an end-to-end Cora node classification proof of concept under examples/cora_node_classification.jl. The example represents the citation network using separate tail and head incidence matrices and uses DirectedHypergraphLayer for directed message passing.

The Cora example includes:

data loading and preprocessing
construction of tail/head incidence representations
stratified train/validation/test splitting
directed hypergraph neural network setup
Enzyme-based automatic differentiation
Adam optimisation and model training
validation and test evaluation
confusion matrix and per-class performance reporting

With a 64-dimensional hidden representation and 100 training epochs, the current run achieved:

Training accuracy: 98.64%
Validation accuracy: 85.71%
Test accuracy: 85.22%

For the current proof of concept, the incidence matrices are passed directly to DirectedHypergraphLayer. Integration between the layers and HGNNDiHypergraph can be addressed separately.

Extends DirectedHypergraphAttentionLayer with user defined num_heads. preserves existing single-head behaviour with num_heads = 1 adds independent parameters for each attention head concatenates head outputs and projects them back to hidden_dim updates documentation expands tests for multi-head forward passes, attention normalisation, parameter shapes, and validation Full test suite passes.

Adds additional directed hypergraph layers with tests and documentation along with an end to end Cora node classification proof of concept using Enzyme for automatic differentiation

espottesmith

Some areas where more documentation would be helpful. Lots of code duplication. Seems like the example is unnecessarily long and probably should be broken up into multiple files. But damn, this is a huge body of work! I so greatly appreciate this contribution.

espottesmith · 2026-08-19T16:24:12Z

 uuid = "1aebdcae-dc77-44ee-b348-f058d326ae63"
-authors = ["Evan Walter Clark Spotte-Smith", "Punna Amornvivat"]
 version = "0.0.1-DEV"
+authors = ["Evan Walter Clark Spotte-Smith", "Punna Amornvivat"]


At this point, you should absolutely list yourself as an author.

espottesmith · 2026-08-19T16:27:36Z

+
+This allows the two roles in a directed hyperedge to be modelled differently
+throughout the complete message-passing step.
+


Follow the naming convention for new files.

You should explain how this is different from the existing message passing layer (or at least put enough detail in this docstring such that an average user can read both and see the difference).

Oh, also, how does this treat cases where a vertex v is in both the tail and the head of a dihyperedge?

espottesmith · 2026-08-19T16:30:08Z

+function _asymmetric_initialise_weight(
+    initializer,
+    rng::AbstractRNG,
+    input_dimension::Int,
+    output_dimension::Int,
+)
+    return permutedims(
+        initializer(
+            rng,
+            output_dimension,
+            input_dimension,
+        ),
+    )
+end
+


This looks identical to _initialize_weight. Am I missing something?

Also, does this function belong here? Is it likely to only be used by layers in this file, or does it belong in more of a utility file elsewhere? Should it be exported?

espottesmith · 2026-08-19T16:30:48Z

+function _asymmetric_initialise_bias(
+    initializer,
+    rng::AbstractRNG,
+    output_dimension::Int,
+)
+    return permutedims(
+        initializer(
+            rng,
+            output_dimension,
+            1,
+        ),
+    )
+end


See comment above. Is this actually providing anything new?

espottesmith · 2026-08-19T16:33:36Z

+function _asymmetric_unpack_input(
+    ::AsymmetricDirectedHypergraphLayer,
+    input::Tuple,
+)
+    throw(
+        ArgumentError(
+            "The layer expects either " *
+            "(X_vertex, source_matrix, target_matrix) or " *
+            "(X_vertex, X_hyperedge, source_matrix, target_matrix).",
+        ),
+    )
+end


What is the user provides something that isn't a Tuple?

espottesmith · 2026-08-20T09:21:37Z

+A learnable gate controls the balance between the information received
+through hypergraph message passing and the transformed representation of the
+vertex itself. Each hidden feature has its own gate value between zero and
+one.


Where does this come from? Has this been reported in the literature before? This goes for all of the layers you've made. I know you said you haven't copied from other sources, but even so, if these ideas have been published, we should give credit and references so readers/users can get more information

espottesmith · 2026-08-20T09:22:08Z

+function _gated_initialise_weight(
+    initializer,
+    rng::AbstractRNG,
+    input_dimension::Int,
+    output_dimension::Int,
+)
+    return permutedims(
+        initializer(
+            rng,
+            output_dimension,
+            input_dimension,
+        ),
+    )
+end
+
+
+function _gated_initialise_bias(
+    initializer,
+    rng::AbstractRNG,
+    output_dimension::Int,
+)
+    return permutedims(
+        initializer(
+            rng,
+            output_dimension,
+            1,
+        ),
+    )
+end


Code duplication

espottesmith · 2026-08-20T09:22:28Z

+function Lux.initialparameters(
+    rng::AbstractRNG,
+    layer::GatedDirectedHypergraphLayer,
+)


Some code duplication in these initialization functions, as well.

espottesmith · 2026-08-20T09:24:44Z

+using Lux
+using Random


Just as a heads up: you shouldn't need to do imports in each file. Anything you globally import in the package file HyperGraphNeuralNetworks.jl should also be available to all modules within the package.

espottesmith · 2026-08-20T09:25:28Z

+The layer preserves the source and target roles of vertices while adding a
+residual connection to the vertex update. Directed information is first


What does "residual connection" mean here? What does "preserv[ing] the [...] roles" mean? Kind of a vague description.

espottesmith · 2026-08-20T09:41:28Z

Oh, just a heads up: I skipped the tests for now.

- Add initial inspection of the glucose pyrolysis dataset - Parse reactant and product SMILES into individual molecular species - Inspect reactant and product counts for directed hypergraph construction - Remove atom-map labels as an initial molecular identity preprocessing step

Remove the initial glucose preprocessing scripts from the general package examples, keeping the CRN proof-of-concept work separate from the main HyperGraphNeuralNetworks.jl codebase.

…tion networks Glucose CRN modelling in Julia using molecular fingerprints, directed message passing, hyperedge regression, learning curves, and message-passing depth experiments

espottesmith · 2026-09-07T16:52:56Z

+        ntuple(
+            layer_index ->
+                DirectedHypergraphLayer(
+                    layer_index == 1 ?
+                        input_dim :
+                        hidden_dim,
+                    0,
+                    hidden_dim;
+                    activation = tanh,
+                    normalize = true,
+                ),
+            num_layers,
+        )


This is a nice one-liner, but is this the most understandable way to write this? Feels like making a vector and then converting to a tuple could be cleaner?

Since this will mostly be used for our internal testing, such points of style aren't so important. Still, worth questioning.

espottesmith · 2026-09-07T16:54:37Z

+function Lux.parameterlength(
+    model::GlucoseHyperedgeRegressor,
+)


As always, document your code. Took me a second to understand the output_parameters line (though I think I got there).

espottesmith · 2026-09-07T16:55:29Z

+function random_regression_split(
+    n::Int;
+    train_fraction = TRAIN_FRACTION,
+    validation_fraction = VALIDATION_FRACTION,
+    seed = RANDOM_SEED,
+)


Can you not using existing split utilities?

espottesmith · 2026-09-07T16:57:39Z

+    return mean(
+        abs2,
+        residuals,
+    )


Why did you choose mean squared (not signed) error for loss? Not opposed, just curious.

espottesmith · 2026-09-07T16:58:05Z

+"""
+    regression_metrics(predictions, targets, indices)
+
+Calculate MAE, RMSE and R¬¨‚â§ for reaction-property predictions.


R^2 looks seriously messed up. I'd also include mean signed error.

espottesmith · 2026-09-07T16:58:52Z

+copy_parameter_tree(
+    x::AbstractArray
+) = copy(x)
+
+
+copy_parameter_tree(
+    x::NamedTuple
+) =
+    NamedTuple{keys(x)}(
+        map(
+            copy_parameter_tree,
+            values(x),
+        )
+    )
+
+
+copy_parameter_tree(
+    x::Tuple
+) =
+    map(
+        copy_parameter_tree,
+        x,
+    )


Feels like this could be a useful utility outside of this example, no?

espottesmith · 2026-09-07T16:59:38Z

+    Enzyme.autodiff(
+        Enzyme.set_runtime_activity(
+            Enzyme.Reverse
+        ),
+        regression_objective,
+        Enzyme.Const(model),
+        Enzyme.Duplicated(
+            ps,
+            gradient_parameters,
+        ),
+        Enzyme.Const(st),
+        Enzyme.Const(X),
+        Enzyme.Const(
+            incidence_tail
+        ),
+        Enzyme.Const(
+            incidence_head
+        ),
+        Enzyme.Const(targets),
+        Enzyme.Const(
+            train_indices
+        ),
+        Enzyme.Const(
+            weight_decay
+        ),
+    )


Bless Enzyme.

espottesmith · 2026-09-07T17:05:22Z

+            run_regression_experiment(
+                X,
+                incidence_tail,
+                incidence_head,
+                targets,
+                selected_training,
+                split.validation,
+                split.test;
+                seed = seed,
+            )


Yeah, so you're either going to need to give up on using the training MSE/MAE/RMSE/etc., or else you're going to need to rewrite run_regression_experiment to take in the training data and, I guess, the full training set to be used for post hoc evaluation. Or, I guess, you just need to evaluate the model on the full training set and run the regression analysis separately. Different options.

espottesmith · 2026-09-07T17:05:48Z

+            grid = true,
+            xticks = percentages,
+            xrotation = 45,
+        )


Make sure you have units.

espottesmith · 2026-09-07T17:06:38Z

+end
+
+
+results = main()


Could you save the models so that, if we wanted to, we could run additional analysis after the initial training procedure? I guess it doesn't matter so much, since the training is so quick, but it could save some time and frustration.

shonalidixit added 3 commits August 9, 2026 16:57

Add directed hypergraph layers and Cora example

deedcb1

Adds additional directed hypergraph layers with tests and documentation along with an end to end Cora node classification proof of concept using Enzyme for automatic differentiation

Complete Cora experiments and directed layer implementations

5a62a24

espottesmith requested changes Aug 20, 2026

View reviewed changes

shonalidixit added 3 commits August 30, 2026 23:31

Remove glucose-specific preprocessing examples

e6b1495

Remove the initial glucose preprocessing scripts from the general package examples, keeping the CRN proof-of-concept work separate from the main HyperGraphNeuralNetworks.jl codebase.

Directed hypergraph neural network proof-of-concept for chemical reac…

f1c5fdc

…tion networks Glucose CRN modelling in Julia using molecular fingerprints, directed message passing, hyperedge regression, learning curves, and message-passing depth experiments

espottesmith reviewed Sep 7, 2026

View reviewed changes


		This allows the two roles in a directed hyperedge to be modelled differently
		throughout the complete message-passing step.

		The layer preserves the source and target roles of vertices while adding a
		residual connection to the vertex update. Directed information is first

Uh oh!

Conversation

shonalidixit commented Aug 19, 2026

Uh oh!

espottesmith left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

espottesmith commented Aug 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants