diff --git a/pages/advanced-algorithms.mdx b/pages/advanced-algorithms.mdx index 34abb89c3..d755ec38e 100644 --- a/pages/advanced-algorithms.mdx +++ b/pages/advanced-algorithms.mdx @@ -30,7 +30,7 @@ Dynamic graph algorithms are a part of [Memgraph Enterprise license](/database-management/enabling-memgraph-enterprise). For more algorithms, check the **Memgraph Advanced Graph Extensions** (**MAGE**) -library. It is part of the [Memgraph GitHub repository](https://github.com/memgraph/memgraph/tree/master/mage) +library. It is part of the [Memgraph GitHub repository](https://github.com/memgraph/memgraph/tree/master/src/mage) that contains [graph algorithms](/advanced-algorithms/available-algorithms) written by the team behind Memgraph and its users in the form of query modules. The project aims to give everyone the tools they need to tackle the most diff --git a/pages/advanced-algorithms/available-algorithms.mdx b/pages/advanced-algorithms/available-algorithms.mdx index 749aa1ea2..c648e57db 100644 --- a/pages/advanced-algorithms/available-algorithms.mdx +++ b/pages/advanced-algorithms/available-algorithms.mdx @@ -112,6 +112,7 @@ If you want to know more and learn how this affects you, read our [announcement] | [periodic](/advanced-algorithms/available-algorithms/periodic) | C++ | A module containing procedures for periodically running difficult and/or memory/time consuming queries. | | [refactor](/advanced-algorithms/available-algorithms/refactor) | C++ | The refactor module provides utilities for changing nodes and relationships. | | [rust_example](https://github.com/memgraph/memgraph/tree/master/mage/rust/rsmgp-example) | Rust | Example of a basic module with input parameters forwarding, made in Rust. | +| [search](/advanced-algorithms/available-algorithms/search) | C++ | A module for finding nodes by comparing one or more of their properties against a value with a comparison operator, without writing the equivalent `MATCH` and `WHERE` clauses. | | [set_property](/advanced-algorithms/available-algorithms/set_property) | C++ | A module for dynamical access and editing of node and relationship properties. | | [temporal](/advanced-algorithms/available-algorithms/temporal) | Python | A module that provides functions to handle temporal (time-related) operations and offers extended capabilities compared to the date module. | | [text](/advanced-algorithms/available-algorithms/text) | C++ | The `text` module offers a toolkit for manipulating strings. | @@ -173,9 +174,23 @@ Running `SHOW QUERY CALLABLE MAPPINGS` requires the `CONFIG` privilege. | apoc.coll.toSet | Converts the input list to a set | [collections.to_set()](/advanced-algorithms/available-algorithms/collections#to_set) | | apoc.coll.sum | Calculates the sum of listed elements | [collections.sum()](/advanced-algorithms/available-algorithms/collections#sum) | | apoc.coll.partition | Partitions the input list into sub-lists of the specified size | [collections.partition()](/advanced-algorithms/available-algorithms/collections#partition) | +| apoc.coll.sort | Sorts the elements of an input list of the same data type | [collections.sort()](/advanced-algorithms/available-algorithms/collections#sort) | +| apoc.coll.containsSorted | Verifies the presence of an element in a sorted list | [collections.contains_sorted()](/advanced-algorithms/available-algorithms/collections#contains_sorted) | +| apoc.coll.containsAll | Checks if a list contains all the values from another list | [collections.contains_all()](/advanced-algorithms/available-algorithms/collections#contains_all) | +| apoc.coll.intersection | Returns the unique intersection of two lists | [collections.intersection()](/advanced-algorithms/available-algorithms/collections#intersection) | +| apoc.coll.disjunction | Returns the symmetric difference of two lists (elements in exactly one) | [collections.disjunction()](/advanced-algorithms/available-algorithms/collections#disjunction) | +| apoc.coll.subtract | Returns the first list with the elements of the second removed, deduplicated | [collections.subtract()](/advanced-algorithms/available-algorithms/collections#subtract) | +| apoc.coll.duplicates | Returns the values that appear more than once in a list | [collections.duplicates()](/advanced-algorithms/available-algorithms/collections#duplicates) | +| apoc.coll.sumLongs | Calculates the sum of list elements cast to integers | [collections.sum_longs()](/advanced-algorithms/available-algorithms/collections#sum_longs) | +| apoc.coll.avg | Calculates the average of listed elements | [collections.avg()](/advanced-algorithms/available-algorithms/collections#avg) | +| apoc.coll.max | Returns the maximum-value element of the input list | [collections.max()](/advanced-algorithms/available-algorithms/collections#max) | +| apoc.coll.min | Returns the minimum-value element of the input list | [collections.min()](/advanced-algorithms/available-algorithms/collections#min) | +| apoc.coll.split | Splits the provided list based on a specified delimiter | [collections.split()](/advanced-algorithms/available-algorithms/collections#split) | | apoc.convert.toTree | Converts values into tree structures | [convert_c.to_tree()](/advanced-algorithms/available-algorithms/convert_c#to_tree) | -| apoc.convert.fromJsonList | Converts a JSON string representation of a list into an actual list object | [json_util.from_json_list()](/advanced-algorithms/available-algorithms/json_util#from_json_list) | -| apoc.convert.toJson | Converts any value to its JSON string representation | [json_util.to_json()](/advanced-algorithms/available-algorithms/json_util#to_json) | +| apoc.convert.fromJsonList | Converts a JSON string representation of a list into an actual list object | [convert.from_json_list()](/advanced-algorithms/available-algorithms/convert#from_json_list) | +| apoc.convert.fromJsonMap | Converts a JSON string representation of a map into an actual map object | [convert.from_json_map()](/advanced-algorithms/available-algorithms/convert#from_json_map) | +| apoc.convert.toJson | Converts any value to its JSON string representation | [convert.to_json()](/advanced-algorithms/available-algorithms/convert#to_json) | +| apoc.convert.toMap | Converts a value into a map | [convert.to_map()](/advanced-algorithms/available-algorithms/convert#to_map) | | apoc.create.node | Creates a single node with specified labels and properties | [create.node()](/advanced-algorithms/available-algorithms/create#node) | | apoc.create.nodes | Creates multiple nodes with specified labels and properties | [create.nodes()](/advanced-algorithms/available-algorithms/create#nodes) | | apoc.create.removeProperties | Removes properties from nodes | [create.remove_properties()](/advanced-algorithms/available-algorithms/create#remove_properties) | @@ -202,12 +217,15 @@ Running `SHOW QUERY CALLABLE MAPPINGS` requires the `CONFIG` privilege. | apoc.refactor.renameType | Changes the relationship type | [refactor.rename_type()](/advanced-algorithms/available-algorithms/refactor#rename_type) | | apoc.refactor.rename.typeProperty | Renames the property of a relationship | [refactor.rename_type_property()](/advanced-algorithms/available-algorithms/refactor#rename_type_property) | | apoc.refactor.mergeNodes | Merges properties, labels and relationships for source nodes to target node | [refactor.mergeNodes()](/advanced-algorithms/available-algorithms/refactor#mergenodes) | +| apoc.search.node | Finds nodes by a label-property map and comparison operator, returning each matching node once | [search.node()](/advanced-algorithms/available-algorithms/search#node) | +| apoc.search.nodeAll | Finds nodes by a label-property map and comparison operator, returning one row per matching property | [search.node_all()](/advanced-algorithms/available-algorithms/search#node_all) | | apoc.text.join | Joins all strings into a single one with given delimiter | [text.join()](/advanced-algorithms/available-algorithms/text#join) | | apoc.text.indexOf | Finds the index of first occurrence of a substring within a string | [text.indexOf()](/advanced-algorithms/available-algorithms/text#indexof) | | apoc.text.regexGroups | Returns all matched subexpressions of regex on provided text | [text.regexGroups()](/advanced-algorithms/available-algorithms/text#regexgroups) | | apoc.text.format | Formats strings using the C++ fmt library | [text.format()](/advanced-algorithms/available-algorithms/text#format) | | apoc.text.replace | Replaces substrings matching regex with replacement | [text.replace()](/advanced-algorithms/available-algorithms/text#replace) | | apoc.text.regReplace | Replaces substrings matching regex with replacement | [text.regReplace()](/advanced-algorithms/available-algorithms/text#regreplace) | +| apoc.text.compareCleaned | Compares two strings for equality after normalization | [text.compare_cleaned()](/advanced-algorithms/available-algorithms/text#compare_cleaned) | | apoc.util.md5 | Gets MD5 hash of concatenated string representations | [util_module.md5()](/advanced-algorithms/available-algorithms/util_module#md5) | | apoc.util.validatePredicate | Raises exception if predicate yields true with parameter interpolation | [mgps.validate_predicate()](/advanced-algorithms/available-algorithms/mgps#validate_predicate) | | db.awaitIndexes | No-op compatibility shim for clients that wait for index creation (e.g. the Neo4j Spark connector) | [mgps.await_indexes()](/advanced-algorithms/available-algorithms/mgps#await_indexes) | diff --git a/pages/advanced-algorithms/available-algorithms/_meta.ts b/pages/advanced-algorithms/available-algorithms/_meta.ts index 885416252..d81ceb7e4 100644 --- a/pages/advanced-algorithms/available-algorithms/_meta.ts +++ b/pages/advanced-algorithms/available-algorithms/_meta.ts @@ -59,6 +59,7 @@ export default { "path": "path", "periodic": "periodic", "refactor": "refactor", + "search": "search", "set_cover": "set_cover", "set_property": "set_property", "temporal": "temporal", diff --git a/pages/advanced-algorithms/available-algorithms/algo.mdx b/pages/advanced-algorithms/available-algorithms/algo.mdx index 16d6a13ec..c3d418b04 100644 --- a/pages/advanced-algorithms/available-algorithms/algo.mdx +++ b/pages/advanced-algorithms/available-algorithms/algo.mdx @@ -16,7 +16,7 @@ enabling users to perform complex graph-based operations and computations, such } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/algo_module/algo_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/algo_module/algo_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/betweenness_centrality.mdx b/pages/advanced-algorithms/available-algorithms/betweenness_centrality.mdx index 43f92557a..7383b0062 100644 --- a/pages/advanced-algorithms/available-algorithms/betweenness_centrality.mdx +++ b/pages/advanced-algorithms/available-algorithms/betweenness_centrality.mdx @@ -28,7 +28,7 @@ Centrality"](http://www.uvm.edu/pdodds/research/papers/others/2001/brandes2001a. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/betweenness_centrality_module/betweenness_centrality_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/betweenness_centrality_module/betweenness_centrality_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/betweenness_centrality_online.mdx b/pages/advanced-algorithms/available-algorithms/betweenness_centrality_online.mdx index 3d7ed9507..09f0b3654 100644 --- a/pages/advanced-algorithms/available-algorithms/betweenness_centrality_online.mdx +++ b/pages/advanced-algorithms/available-algorithms/betweenness_centrality_online.mdx @@ -48,7 +48,7 @@ reflective of real-time changes. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/betweenness_centrality_module/betweenness_centrality_online_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/betweenness_centrality_module/betweenness_centrality_online_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/biconnected_components.mdx b/pages/advanced-algorithms/available-algorithms/biconnected_components.mdx index a50fd6915..2144fd3a6 100644 --- a/pages/advanced-algorithms/available-algorithms/biconnected_components.mdx +++ b/pages/advanced-algorithms/available-algorithms/biconnected_components.mdx @@ -21,7 +21,7 @@ The algorithm works by finding articulation points, and then traversing from the } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/biconnected_components_module/biconnected_components_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/biconnected_components_module/biconnected_components_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/bipartite_matching.mdx b/pages/advanced-algorithms/available-algorithms/bipartite_matching.mdx index 9532fee59..84ed70d80 100644 --- a/pages/advanced-algorithms/available-algorithms/bipartite_matching.mdx +++ b/pages/advanced-algorithms/available-algorithms/bipartite_matching.mdx @@ -23,7 +23,7 @@ set of edges (relationships). } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/bipartite_matching_module/bipartite_matching_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/bipartite_matching_module/bipartite_matching_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/bridges.mdx b/pages/advanced-algorithms/available-algorithms/bridges.mdx index 1bf17867e..459ab30fb 100644 --- a/pages/advanced-algorithms/available-algorithms/bridges.mdx +++ b/pages/advanced-algorithms/available-algorithms/bridges.mdx @@ -20,7 +20,7 @@ valuable to detect it on time. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/bridges_module/bridges_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/bridges_module/bridges_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/collections.mdx b/pages/advanced-algorithms/available-algorithms/collections.mdx index 339874be6..bf80aa9d0 100644 --- a/pages/advanced-algorithms/available-algorithms/collections.mdx +++ b/pages/advanced-algorithms/available-algorithms/collections.mdx @@ -20,7 +20,7 @@ using the `CALL` subclause. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/collections_module/collections_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/collections_module/collections_module.cpp" /> @@ -39,11 +39,53 @@ uncomparable.
Only `Numeric` data types can be used in `sum()` and `avg()` functions, so only `Int` and `Double` data types are allowed. +### Handling `NULL` values + +Passing `NULL` as a whole list (or scalar) argument no longer raises an +argument-validation error. Each function returns a defined result: + +| Function | Result when a list argument is `NULL` | +| --- | --- | +| `sum()`, `sum_longs()`, `avg()`, `min()`, `max()`, `to_set()`, `pairs()` | `null` | +| `union()`, `union_all()`, `disjunction()` | the other list; `null` when both are `NULL` | +| `remove_all()`, `subtract()` | `null` when the first list is `NULL`; the first list when the second is `NULL` | +| `intersection()`, `sort()`, `flatten()`, `duplicates()` | empty list `[]` | +| `contains()`, `contains_all()`, `contains_sorted()` | `false` | +| `frequencies_as_map()` | empty map `{}` | +| `split()`, `partition()` | no rows | + +`NULL` elements inside a list are handled per function: + +- `min()` and `max()` skip `NULL` elements; a list containing only `NULL` values returns `null`. +- `contains()` and `contains_all()` never match a `NULL` search value (`NULL = NULL` is not true), so searching for `NULL` returns `false`. +- `split()` treats a `NULL` delimiter as matching nothing and returns the whole list as a single part; `NULL` list elements are kept. +- `frequencies_as_map()` counts `NULL` elements under the `"NO_VALUE"` key. +- `flatten()`, `pairs()`, `to_set()`, `union()`, `union_all()`, `remove_all()`, `intersection()`, `disjunction()`, `subtract()` and `duplicates()` keep `NULL` elements as ordinary values. +- `sum()`, `sum_longs()`, `avg()`, `sort()` and `contains_sorted()` still reject a list that contains a `NULL` element; `contains_sorted()` also rejects a `NULL` search value, and `partition()` still rejects a `NULL` size. + +For example, a `NULL` argument returns the defined result instead of erroring: + +```cypher +RETURN collections.sum(null) AS sum, collections.sort(null) AS sorted; +``` + +```plaintext ++----------------------------+ +| sum | sorted | ++----------------------------+ +| null | [] | ++----------------------------+ +``` + ### `sort()` Sorts the elements of an input list if they are of the same data type. For the input list to be sorted, its elements must be comparable and of the same type. + +This function is equivalent to **apoc.coll.sort**. + + {

Input:

} - `coll: List[Any]` ➡ List of elements that need to be sorted. @@ -74,6 +116,10 @@ Verifies the presence of a certain element in a sorted list. If an unsorted list is passed, there is no guarantee that the result will be correct. For the input list to be sorted, its elements must be comparable and of the same type. + +This function is equivalent to **apoc.coll.containsSorted**. + + {

Input:

} - `coll: List[Any]` ➡ The target list where the element is searched for. @@ -239,6 +285,10 @@ RETURN collections.contains([1,2,3], "e") AS output; Checks if a list contains all the values from another list. + +This function is equivalent to **apoc.coll.containsAll**. + + {

Input:

} - `coll: List[Any]` ➡ The target list used for searching values. @@ -270,6 +320,10 @@ RETURN collections.contains_all([1, 2, 3, "pero"], [1, 1, 1, 1, 2, 3]) AS contai Returns the unique intersection of two lists. + +This function is equivalent to **apoc.coll.intersection**. + + {

Input:

} - `first: List[Any]` ➡ The first list. @@ -295,6 +349,107 @@ RETURN collections.intersection([1, 1, 2, 3, 4, 5], [1, 1, 3, 5, 7, 9]) AS inter +---------------------------------------------------------+ ``` +### `disjunction()` + +Returns the disjunction (symmetric difference) of two lists: the unique elements +present in exactly one of the lists. The order of the result is not guaranteed. + + +This function is equivalent to **apoc.coll.disjunction**. + + +{

Input:

} + +- `list1: List[Any]` ➡ The first list. +- `list2: List[Any]` ➡ The second list. + +{

Output:

} + +- `List[Any]` ➡ The unique elements found in only one of the two lists. + +{

Usage:

} + +The following query will return the elements present in only one of the lists: + +```cypher +RETURN collections.disjunction([1, 2, 3, 4, 5], [3, 4, 5]) AS disjunction; +``` + +```plaintext ++---------------------------------------------------------+ +| disjunction | ++---------------------------------------------------------+ +| [1, 2] | ++---------------------------------------------------------+ +``` + +### `subtract()` + +Returns the first list as a set with all elements of the second list removed. The +result is deduplicated and its order is not guaranteed. + + +This function is equivalent to **apoc.coll.subtract**. + + +{

Input:

} + +- `list1: List[Any]` ➡ The list to subtract from. +- `list2: List[Any]` ➡ The list of elements to remove. + +{

Output:

} + +- `List[Any]` ➡ The unique elements of the first list that are not present in the second list. + +{

Usage:

} + +The following query will remove the elements of the second list from the first: + +```cypher +RETURN collections.subtract([1, 2, 3, 4, 5, 6, 6], [3, 4, 5]) AS subtracted; +``` + +```plaintext ++---------------------------------------------------------+ +| subtracted | ++---------------------------------------------------------+ +| [1, 2, 6] | ++---------------------------------------------------------+ +``` + +### `duplicates()` + +Returns the values that appear more than once in a list, each reported a single +time, in the order in which the duplicate is first observed. + + +This function is equivalent to **apoc.coll.duplicates**. + + +{

Input:

} + +- `coll: List[Any]` ➡ The input list. + +{

Output:

} + +- `List[Any]` ➡ The values that occur more than once in the input list. + +{

Usage:

} + +The following query will return the values that appear more than once: + +```cypher +RETURN collections.duplicates([1, 1, 2, 3, 3, 3]) AS duplicates; +``` + +```plaintext ++---------------------------------------------------------+ +| duplicates | ++---------------------------------------------------------+ +| [1, 3] | ++---------------------------------------------------------+ +``` + ### `flatten()` Returns flattened list of inputs provided. @@ -468,6 +623,10 @@ RETURN collections.sum([1, 2.3, -4, a.id]) AS sum; Calculates the sum of list elements casted to integers. The initial list elements have to be `Numeric` data type, or an exception is thrown. + +This function is equivalent to **apoc.coll.sumLongs**. + + {

Input:

} - `numbers: List[Any]` ➡ The list of numbers. @@ -498,6 +657,10 @@ Calculates the average of listed elements if they are of the same type and can be summed (the elements need to be numerics). Listing elements of different data types, or data types that are impossible to sum, will throw an exception. + +This function is equivalent to **apoc.coll.avg**. + + {

Input:

} - `numbers: List[Any]` ➡ The list of numbers. @@ -526,6 +689,10 @@ RETURN collections.avg([5, 5, 6, 7, -5]) AS average; The procedure returns the element of the maximum value from the input list. + +This function is equivalent to **apoc.coll.max**. + + {

Input:

} - `values: List[Any]` ➡ The input list where an element of the maximum value must be found. @@ -556,6 +723,10 @@ Finds the element of the minimum value in an input list. Listing elements of different data types, or data types that are impossible to compare, will throw an exception. + +This function is equivalent to **apoc.coll.min**. + + {

Input:

} - `values: List[Any]` ➡ The input list where an element of the minimum value must be found. @@ -588,6 +759,10 @@ Splits the provided list based on a specified delimiter. Returns a series of sublists generated by breaking the original list wherever the delimiter is encountered. The delimiter itself is not included in the resulting sublists. + +This procedure is equivalent to **apoc.coll.split**. + + {

Input:

} - `subgraph: Graph` (**OPTIONAL**) ➡ A specific subgraph, which is an [object of type Graph](/advanced-algorithms/run-algorithms#run-procedures-on-subgraph) returned by the `project()` function, on which the algorithm is run. @@ -660,4 +835,4 @@ RETURN result; +---------------------------------------------------------+ | [5,6] | +---------------------------------------------------------+ -``` \ No newline at end of file +``` diff --git a/pages/advanced-algorithms/available-algorithms/community_detection.mdx b/pages/advanced-algorithms/available-algorithms/community_detection.mdx index 51144ea3b..ed6cdc5c2 100644 --- a/pages/advanced-algorithms/available-algorithms/community_detection.mdx +++ b/pages/advanced-algorithms/available-algorithms/community_detection.mdx @@ -32,7 +32,7 @@ preserving communities. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/community_detection_module/community_detection_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/community_detection_module/community_detection_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/convert.mdx b/pages/advanced-algorithms/available-algorithms/convert.mdx index e4890a260..cc784d6d6 100644 --- a/pages/advanced-algorithms/available-algorithms/convert.mdx +++ b/pages/advanced-algorithms/available-algorithms/convert.mdx @@ -54,12 +54,205 @@ Use the following query to convert a JSON string to an object: RETURN convert.str2object('{"name": "Alice", "age": 30, "city": "New York"}') AS result; ``` -The output shows the parsed object: - -```json -{ - "name": "Alice", - "age": 30, - "city": "New York" -} +The output shows the parsed map: + +```plaintext +{name: "Alice", age: 30, city: "New York"} +``` + +### `from_json_map()` + +Parses a JSON-object string into a map. An optional `path` selects a nested part +of the document before conversion; the selected value must be a JSON object. + + +This function is equivalent to **apoc.convert.fromJsonMap**. + + +{

Input:

} + +- `map: String` ➡ The JSON string to parse. A `null` value returns `null`. +- `path: String` (default `""`) ➡ An optional selector for a nested part of the + document (see [Path option](#path-option)). + +{

Output:

} + +- `Map` ➡ The parsed map. Returns `null` when the input is `null`, when the path + does not resolve, or when the selected value is JSON `null`. An error is raised + when the input (or selected value) is not a JSON object. + +{

Usage:

} + +```cypher +RETURN convert.from_json_map('{"name": "GDS"}') AS result; +``` + +```plaintext +{name: "GDS"} +``` + +Select a nested object with `path`: + +```cypher +RETURN convert.from_json_map('{"a": 1, "b": {"c": 2, "d": [10, 20]}}', '$.b') AS result; +``` + +```plaintext +{c: 2, d: [10, 20]} +``` + +To read a single value out of the parsed map, index it with Cypher instead of +using `path`: + +```cypher +RETURN convert.from_json_map('{"mode": "fast"}')['mode'] AS result; +``` + +```text +"fast" +``` + +### `from_json_list()` + +Parses a JSON-array string into a list. An optional `path` selects a nested part +of the document before conversion; the selected value must be a JSON array. + + +This function is equivalent to **apoc.convert.fromJsonList**. + + +{

Input:

} + +- `list: String` ➡ The JSON string to parse. A `null` value returns `null`. +- `path: String` (default `""`) ➡ An optional selector for a nested part of the + document (see [Path option](#path-option)). + +{

Output:

} + +- `List` ➡ The parsed list. Returns `null` when the input is `null`, when the + path does not resolve, or when the selected value is JSON `null`. An error is + raised when the input (or selected value) is not a JSON array. + +{

Usage:

} + +```cypher +RETURN convert.from_json_list('[1, 2, 3]') AS result; +``` + +```plaintext +[1, 2, 3] +``` + +Select a nested array with `path`: + +```cypher +RETURN convert.from_json_list('{"a": [1, 2, 3]}', '$.a') AS result; +``` + +```plaintext +[1, 2, 3] +``` + +### Path option + +`from_json_map()` and `from_json_list()` accept an optional `path` that selects a +nested part of the JSON document before conversion. + +| Syntax | Meaning | Example (on `{"a": 1, "b": {"c": 2, "e": [10, 20]}}`) | +| ------ | ------- | ----------------------------------------------------- | +| `$` / empty / `null` | The whole document | `$` selects the whole object | +| `.key` | Object key step | `$.b` selects `{"c": 2, "e": [10, 20]}` | +| `['key']` or `["key"]` | Quoted key step, for keys with dots, spaces or special characters | `$['b']` selects `{"c": 2, "e": [10, 20]}` | +| `[index]` | Array element, 0-based | `$.b.e[1]` selects `20` | + +Steps chain left to right (`$.b.e[0]` selects `10`), and a leading `$` is +optional (`a.b` is equivalent to `$.a.b`). Wildcards (`$.e[*]`), recursive +descent (`$..x`), filter expressions and array slices are not supported and raise +an error. A path that does not resolve, or that resolves to JSON `null`, returns +`null`. + +### `to_map()` + +Returns a map unchanged, or a node or relationship as its property map. Any +other value — such as an integer, string or list — returns `null`. + + +This function is equivalent to **apoc.convert.toMap**. + + +{

Input:

} + +- `map: Any` ➡ The value to convert. + +{

Output:

} + +- `Map` ➡ A map is returned unchanged, a node or relationship is returned as its + property map, and `null` or any other value returns `null`. + +{

Usage:

} + +```cypher +CREATE (n:Person {id: 4, name: 'z'}) +RETURN convert.to_map(n) AS result; +``` + +```plaintext +{id: 4, name: "z"} +``` + +### `to_json()` + +Serializes a value into a JSON string. + + +This function is equivalent to **apoc.convert.toJson**. + + +{

Input:

} + +- `value: Any` ➡ The value to serialize. + +{

Output:

} + +- `String` ➡ The JSON string representation of the value. + +Scalars, lists and maps are serialized directly. Graph and spatial-temporal +values use a structured form: + +- **Node** ➡ `{id, type: "node", labels, properties}`; `labels` and `properties` + are omitted when empty. +- **Relationship** ➡ `{id, type: "relationship", label, start, end, properties}`, + where `start` and `end` are full node objects; `properties` is omitted when empty. +- **Path** ➡ a flat array `[node, relationship, node, ...]`. +- **Point** ➡ `{crs, x, y, z}` for cartesian points, `{crs, longitude, latitude, + height}` for geographic points. +- **Temporal** ➡ the canonical string form (for example `"2020-01-02"` for a + date, an ISO-8601 period with a fixed six-digit fractional-second field for a + duration, e.g. `"P1DT2H3M4.500000S"`). + +Object keys are serialized in alphabetical order, and the order of the `labels` +array follows the node's internal label IDs, so it is not guaranteed. + +Null, boolean, numeric, string, list, map, node, relationship, path, point and +temporal values are all supported. Serializing an enum value raises an error. + +{

Usage:

} + +```cypher +RETURN convert.to_json({a: 1, b: 'x', c: [1, 2], d: null}) AS result; +``` + +```text +{"a":1,"b":"x","c":[1,2],"d":null} +``` + +Serialize a node: + +```cypher +CREATE (n:Person:Human {name: 'Ana', age: 30}) +RETURN convert.to_json(n) AS result; +``` + +```text +{"id":"0","labels":["Person","Human"],"properties":{"age":30,"name":"Ana"},"type":"node"} ``` diff --git a/pages/advanced-algorithms/available-algorithms/create.mdx b/pages/advanced-algorithms/available-algorithms/create.mdx index cebc0f5aa..31f4f4bef 100644 --- a/pages/advanced-algorithms/available-algorithms/create.mdx +++ b/pages/advanced-algorithms/available-algorithms/create.mdx @@ -17,7 +17,7 @@ model, manipulate, and query complex graph data. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/create_module/create_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/create_module/create_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/cross_database.mdx b/pages/advanced-algorithms/available-algorithms/cross_database.mdx index 1856d6704..00f462f3a 100644 --- a/pages/advanced-algorithms/available-algorithms/cross_database.mdx +++ b/pages/advanced-algorithms/available-algorithms/cross_database.mdx @@ -24,7 +24,7 @@ procedure has been replaced by the more general [`cross_database.bolt()`](#bolt) } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/cross_database.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/cross_database.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/csv_utils.mdx b/pages/advanced-algorithms/available-algorithms/csv_utils.mdx index 1c1e91f0c..03afe001c 100644 --- a/pages/advanced-algorithms/available-algorithms/csv_utils.mdx +++ b/pages/advanced-algorithms/available-algorithms/csv_utils.mdx @@ -16,7 +16,7 @@ It allows users to create and delete CSV files directly from the database enviro } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/csv_utils_module/csv_utils_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/csv_utils_module/csv_utils_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/cugraph.mdx b/pages/advanced-algorithms/available-algorithms/cugraph.mdx index c63ba24ac..05a324041 100644 --- a/pages/advanced-algorithms/available-algorithms/cugraph.mdx +++ b/pages/advanced-algorithms/available-algorithms/cugraph.mdx @@ -23,7 +23,7 @@ wrappers for most of the algorithms present in the } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/cugraph_module" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/cugraph_module" /> diff --git a/pages/advanced-algorithms/available-algorithms/cycles.mdx b/pages/advanced-algorithms/available-algorithms/cycles.mdx index 6da8bbe66..b853583dc 100644 --- a/pages/advanced-algorithms/available-algorithms/cycles.mdx +++ b/pages/advanced-algorithms/available-algorithms/cycles.mdx @@ -26,7 +26,7 @@ set of nodes (vertices) of the given graph. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/cycles_module/cycles_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/cycles_module/cycles_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/degree_centrality.mdx b/pages/advanced-algorithms/available-algorithms/degree_centrality.mdx index 308b0252d..bbc62f4ff 100644 --- a/pages/advanced-algorithms/available-algorithms/degree_centrality.mdx +++ b/pages/advanced-algorithms/available-algorithms/degree_centrality.mdx @@ -25,7 +25,7 @@ a_{i,k}$ or in matrix form: $y = A 1$. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/degree_centrality_module/degree_centrality_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/degree_centrality_module/degree_centrality_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/distance_calculator.mdx b/pages/advanced-algorithms/available-algorithms/distance_calculator.mdx index 3e086203c..22ee28cd6 100644 --- a/pages/advanced-algorithms/available-algorithms/distance_calculator.mdx +++ b/pages/advanced-algorithms/available-algorithms/distance_calculator.mdx @@ -24,7 +24,7 @@ this: } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/distance_calculator/distance_calculator.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/distance_calculator/distance_calculator.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/do.mdx b/pages/advanced-algorithms/available-algorithms/do.mdx index e3ec392e8..cad9f2925 100644 --- a/pages/advanced-algorithms/available-algorithms/do.mdx +++ b/pages/advanced-algorithms/available-algorithms/do.mdx @@ -18,7 +18,7 @@ that will control query execution. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/do_module/do_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/do_module/do_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/elasticsearch_synchronization.mdx b/pages/advanced-algorithms/available-algorithms/elasticsearch_synchronization.mdx index f1d6b68a4..a72fd2927 100644 --- a/pages/advanced-algorithms/available-algorithms/elasticsearch_synchronization.mdx +++ b/pages/advanced-algorithms/available-algorithms/elasticsearch_synchronization.mdx @@ -49,7 +49,7 @@ create new ones with custom schema**. Indexing can be performed in two ways: } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/elastic_search_serialization.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/elastic_search_serialization.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/embeddings.mdx b/pages/advanced-algorithms/available-algorithms/embeddings.mdx index c7360ad43..b1034c9c7 100644 --- a/pages/advanced-algorithms/available-algorithms/embeddings.mdx +++ b/pages/advanced-algorithms/available-algorithms/embeddings.mdx @@ -27,7 +27,7 @@ credentials. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/embeddings.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/embeddings.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/export_util.mdx b/pages/advanced-algorithms/available-algorithms/export_util.mdx index 9b9feee57..961be5b4d 100644 --- a/pages/advanced-algorithms/available-algorithms/export_util.mdx +++ b/pages/advanced-algorithms/available-algorithms/export_util.mdx @@ -24,7 +24,7 @@ Currently, this module supports: } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/export_util.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/export_util.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/gnn.mdx b/pages/advanced-algorithms/available-algorithms/gnn.mdx index e1341f698..5278db274 100644 --- a/pages/advanced-algorithms/available-algorithms/gnn.mdx +++ b/pages/advanced-algorithms/available-algorithms/gnn.mdx @@ -29,7 +29,7 @@ Typical workflow: } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/gnn.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/gnn.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/gnn_link_prediction.mdx b/pages/advanced-algorithms/available-algorithms/gnn_link_prediction.mdx index 07c29dcb5..dc40f9907 100644 --- a/pages/advanced-algorithms/available-algorithms/gnn_link_prediction.mdx +++ b/pages/advanced-algorithms/available-algorithms/gnn_link_prediction.mdx @@ -32,7 +32,7 @@ representations by aggregating the representations of node neighbors and their representation from the previous iteration. Such properties make **graph neural networks** a great tool for various problems we in Memgraph encounter. If your graph is evolving in time, check [TGN -model](https://github.com/memgraph/memgraph/blob/master/mage/python/tgn.py) that Memgraph +model](https://github.com/memgraph/memgraph/blob/master/src/mage/python/tgn.py) that Memgraph engineers have already developed. In this includes the following features: @@ -92,7 +92,7 @@ For the underlying **GNN** training Memgraph uses the [DGL library](https://gith } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/link_prediction.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/link_prediction.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/gnn_node_classification.mdx b/pages/advanced-algorithms/available-algorithms/gnn_node_classification.mdx index f1e7fcb8d..9af741596 100644 --- a/pages/advanced-algorithms/available-algorithms/gnn_node_classification.mdx +++ b/pages/advanced-algorithms/available-algorithms/gnn_node_classification.mdx @@ -45,7 +45,7 @@ useful. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/node_classification.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/node_classification.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/graph_coloring.mdx b/pages/advanced-algorithms/available-algorithms/graph_coloring.mdx index 0888f2c51..faff998ef 100644 --- a/pages/advanced-algorithms/available-algorithms/graph_coloring.mdx +++ b/pages/advanced-algorithms/available-algorithms/graph_coloring.mdx @@ -40,7 +40,7 @@ converging to local minimums too early. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/graph_coloring.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/graph_coloring.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/graph_util.mdx b/pages/advanced-algorithms/available-algorithms/graph_util.mdx index 43cc207a4..5f12ffddb 100644 --- a/pages/advanced-algorithms/available-algorithms/graph_util.mdx +++ b/pages/advanced-algorithms/available-algorithms/graph_util.mdx @@ -18,7 +18,7 @@ manipulation tools to accelerate development. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/graph_util_module/graph_util_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/graph_util_module/graph_util_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/igraphalg.mdx b/pages/advanced-algorithms/available-algorithms/igraphalg.mdx index 47b62a7ec..7a5d734fa 100644 --- a/pages/advanced-algorithms/available-algorithms/igraphalg.mdx +++ b/pages/advanced-algorithms/available-algorithms/igraphalg.mdx @@ -18,7 +18,7 @@ stream the native database graph directly, significantly lowering memory usage. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/igraphalg.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/igraphalg.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/import_util.mdx b/pages/advanced-algorithms/available-algorithms/import_util.mdx index 04bcc81d5..b6f16e1a4 100644 --- a/pages/advanced-algorithms/available-algorithms/import_util.mdx +++ b/pages/advanced-algorithms/available-algorithms/import_util.mdx @@ -18,7 +18,7 @@ supports the import of JSON and graphML file formats. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/import_util.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/import_util.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/json_util.mdx b/pages/advanced-algorithms/available-algorithms/json_util.mdx index 4a01a1600..0f2dae62d 100644 --- a/pages/advanced-algorithms/available-algorithms/json_util.mdx +++ b/pages/advanced-algorithms/available-algorithms/json_util.mdx @@ -18,7 +18,7 @@ and if it is a map, the module loads it as a single value. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/json_util.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/json_util.py" /> @@ -35,7 +35,9 @@ and if it is a map, the module loads it as a single value. Converts a JSON string representation of a list into an actual list object. -This function is equivalent to **apoc.convert.fromJsonList**. +Prefer the C++ [`convert.from_json_list`](/advanced-algorithms/available-algorithms/convert#from_json_list), +which is what `apoc.convert.fromJsonList` maps to. This Python version can +produce slightly different output. {

Input:

} @@ -69,7 +71,9 @@ Results: Converts any value to its JSON string representation. -This function is equivalent to **apoc.convert.toJson**. +Prefer the C++ [`convert.to_json`](/advanced-algorithms/available-algorithms/convert#to_json), +which is what `apoc.convert.toJson` maps to. This Python version can produce +slightly different output. {

Input:

} diff --git a/pages/advanced-algorithms/available-algorithms/katz_centrality.mdx b/pages/advanced-algorithms/available-algorithms/katz_centrality.mdx index fc9174b61..c336b2804 100644 --- a/pages/advanced-algorithms/available-algorithms/katz_centrality.mdx +++ b/pages/advanced-algorithms/available-algorithms/katz_centrality.mdx @@ -38,7 +38,7 @@ resulting centralities will be correct. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/katz_centrality_module/katz_centrality_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/katz_centrality_module/katz_centrality_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/kmeans_clustering.mdx b/pages/advanced-algorithms/available-algorithms/kmeans_clustering.mdx index 25b7abd4e..ca50792d7 100644 --- a/pages/advanced-algorithms/available-algorithms/kmeans_clustering.mdx +++ b/pages/advanced-algorithms/available-algorithms/kmeans_clustering.mdx @@ -18,7 +18,7 @@ sum-of-squares. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/kmeans.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/kmeans.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/knn.mdx b/pages/advanced-algorithms/available-algorithms/knn.mdx index b71dde9a8..9c228f1a4 100644 --- a/pages/advanced-algorithms/available-algorithms/knn.mdx +++ b/pages/advanced-algorithms/available-algorithms/knn.mdx @@ -23,7 +23,7 @@ finding nodes with similar embeddings, features, or other vector-based propertie } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/knn_module/knn_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/knn_module/knn_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/label.mdx b/pages/advanced-algorithms/available-algorithms/label.mdx index 8da9d0b8a..73327f563 100644 --- a/pages/advanced-algorithms/available-algorithms/label.mdx +++ b/pages/advanced-algorithms/available-algorithms/label.mdx @@ -17,7 +17,7 @@ to check the existence of a label within the node. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/label_module/label_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/label_module/label_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/leiden_community_detection.mdx b/pages/advanced-algorithms/available-algorithms/leiden_community_detection.mdx index 3fbd31a39..bc6655d3d 100644 --- a/pages/advanced-algorithms/available-algorithms/leiden_community_detection.mdx +++ b/pages/advanced-algorithms/available-algorithms/leiden_community_detection.mdx @@ -32,7 +32,7 @@ space complexity if $\mathcal{O}(VE)$ for $V$ nodes and $E$ edges. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/leiden_community_detection_module/leiden_community_detection_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/leiden_community_detection_module/leiden_community_detection_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/llm_util.mdx b/pages/advanced-algorithms/available-algorithms/llm_util.mdx index fb3add1bc..5fdf93d5e 100644 --- a/pages/advanced-algorithms/available-algorithms/llm_util.mdx +++ b/pages/advanced-algorithms/available-algorithms/llm_util.mdx @@ -26,7 +26,7 @@ developing applications powered by language models. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/llm_util.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/llm_util.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/map.mdx b/pages/advanced-algorithms/available-algorithms/map.mdx index 320171d59..e3cfc5995 100644 --- a/pages/advanced-algorithms/available-algorithms/map.mdx +++ b/pages/advanced-algorithms/available-algorithms/map.mdx @@ -17,7 +17,7 @@ context. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/map_module/map_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/map_module/map_module.cpp" /> @@ -40,7 +40,7 @@ from any inner maps that are part of the input map. {

Input:

} -- `map: Map` ➡ The map from which the key will be removed. +- `map: Map` ➡ The map from which the key will be removed (a node or relationship may be passed; its properties are used). - `key: string` ➡ The key to be removed from the map. - `config: Map default = {recursive: false}` ➡ The config map which supports the `recursive` option. The option `recursive` is `false` by default, and should be @@ -94,7 +94,7 @@ This function is equivalent to **apoc.map.removeKeys**. {

Input:

} -- `map: Map[Any]` ➡ The input map. +- `map: Map[Any]` ➡ The input map (a node or relationship may be passed; its properties are used). - `keys: List[string]` ➡ A list of keys that will be removed. - `config: Map default = {recursive: false}` ➡ A config map which supports the `recursive` option. The `recursive` option is `false` by default, and should be @@ -181,8 +181,8 @@ This function is equivalent to **apoc.map.merge**. {

Input:

} -- `first: mgp.Nullable[Map]` ➡ A map containing key-value pairs that need to be merged with another map. -- `second: mgp.Nullable[Map]` ➡ The second map containing key-value pairs that need to be merged with the key-values from the first map. +- `map1: mgp.Nullable[Map]` ➡ The first map to merge (a node or relationship may be passed; its properties are used). +- `map2: mgp.Nullable[Map]` ➡ The second map to merge. On a key conflict, its value takes precedence. {

Output:

} @@ -204,13 +204,47 @@ RETURN map.merge({a: "b", c: "d"}, {e: "f", g: "h"}) AS merged; +----------------------------------------+ ``` +### `merge_list()` + +Merges a list of maps into a single map. Keys are merged left to right, so when +the same key appears in more than one map, the value from the last map wins. An +empty list yields an empty map. + + +This function is equivalent to **apoc.map.mergeList**. + + +{

Input:

} + +- `maps: List[Map]` ➡ The maps to merge (each element may be a node or relationship, whose properties are used). + +{

Output:

} + +- `Map` ➡ The merged map. + +{

Usage:

} + +The following query merges a list of maps: + +```cypher +RETURN map.merge_list([{a: 1}, {a: 2, b: 3}]) AS merged; +``` + +```plaintext ++----------------------------------------+ +| merged | ++----------------------------------------+ +| {a: 2, b: 3} | ++----------------------------------------+ +``` + ### `flatten()` The procedure flattens nested items in the input map. {

Input:

} -- `map: Map[Any]` ➡ The input map that needs to be modified. +- `map: Map[Any]` ➡ The input map that needs to be modified (a node or relationship may be passed; its properties are used). - `delimiter: string (default = ".")` ➡ The delimiter used for flattening. {

Output:

} @@ -270,8 +304,12 @@ RETURN map.from_lists(["key","key2"],[1,2]) AS result; ### `from_values()` Returns a map from the given list of values. The list has the format: `[key1, -value1, key2, value2]`. If the key is not convertible to a string, the function -throws `ValueException`. +value1, key2, value2]`. Keys are converted to strings; a pair whose key is +`null` is skipped (its value is ignored). + + +This function is equivalent to **apoc.map.fromValues**. + {

Input:

} @@ -300,13 +338,18 @@ RETURN map.from_values(["day", "sunny", 5, 6]) AS map; ### `set_key()` Updates the value at the position `key` in a map. If the key doesn't exist, -the function will insert it. +the function will insert it. A `null` map is treated as empty and a `null` key +is a no-op (the map is returned unchanged). + + +This function is equivalent to **apoc.map.setKey**. + {

Input:

} -- `map: Map` ➡ The map that will be modified. -- `key: string` ➡ The key of a certain key-value pair that needs to have a new value. -- `value: any` ➡ The new value of a certain key-value pair. +- `map: mgp.Nullable[Map]` ➡ The map that will be modified (a node or relationship may be passed; its properties are used). +- `key: mgp.Nullable[string]` ➡ The key to add or update; a `null` key leaves the map unchanged. +- `value: mgp.Nullable[Any]` ➡ The new value of the key-value pair. {

Output:

} @@ -328,6 +371,44 @@ RETURN map.set_key({name:"Ivan",country:"Croatia"}, "name", "Matija") AS map; +-------------------------------------------+ ``` +### `get()` + +Returns the value stored under `key` in the map. If the key is absent, the +function returns `value` when it is non-null; otherwise it throws when `fail` is +`true` (the default) or returns `null` when `fail` is `false`. An existing key +always wins, even when its stored value is `null`. + + +This function is equivalent to **apoc.map.get**. + + +{

Input:

} + +- `map: Map` ➡ The map to look up (a node or relationship may be passed; its properties are used). +- `key: string` ➡ The key to look up. +- `value: any (default = null)` ➡ The value returned when the key is absent. +- `fail: boolean (default = true)` ➡ When `true`, throws if the key is absent and `value` is null; when `false`, returns `null` instead. + +{

Output:

} + +- `any` ➡ The value at `key`, the fallback `value`, or `null`. + +{

Usage:

} + +The following query returns the fallback value because the key is absent: + +```cypher +RETURN map.get({name: "Ivan"}, "country", "unknown", false) AS value; +``` + +```plaintext ++----------------------------------------+ +| value | ++----------------------------------------+ +| "unknown" | ++----------------------------------------+ +``` + ## Procedures ### `from_nodes()` diff --git a/pages/advanced-algorithms/available-algorithms/math.mdx b/pages/advanced-algorithms/available-algorithms/math.mdx index 85e0ff78b..2ac659a11 100644 --- a/pages/advanced-algorithms/available-algorithms/math.mdx +++ b/pages/advanced-algorithms/available-algorithms/math.mdx @@ -15,7 +15,7 @@ The `math` module provides essential mathematical operations for precise numeric } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/math_module/math_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/math_module/math_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/max_flow.mdx b/pages/advanced-algorithms/available-algorithms/max_flow.mdx index e3595fbd9..26870030d 100644 --- a/pages/advanced-algorithms/available-algorithms/max_flow.mdx +++ b/pages/advanced-algorithms/available-algorithms/max_flow.mdx @@ -37,7 +37,7 @@ returning max flow value is 0. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/max_flow.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/max_flow.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/merge.mdx b/pages/advanced-algorithms/available-algorithms/merge.mdx index 7abdc1554..df5cad88b 100644 --- a/pages/advanced-algorithms/available-algorithms/merge.mdx +++ b/pages/advanced-algorithms/available-algorithms/merge.mdx @@ -16,7 +16,7 @@ It ensures precision and coherence in managing interconnected data structures. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/merge_module/merge_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/merge_module/merge_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/meta.mdx b/pages/advanced-algorithms/available-algorithms/meta.mdx index 743832583..003535ed0 100644 --- a/pages/advanced-algorithms/available-algorithms/meta.mdx +++ b/pages/advanced-algorithms/available-algorithms/meta.mdx @@ -21,7 +21,7 @@ The **meta** module provides a set of procedures for generating metadata about t } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/meta_module/meta_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/meta_module/meta_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/meta_util.mdx b/pages/advanced-algorithms/available-algorithms/meta_util.mdx index 69574b053..bb1cc714a 100644 --- a/pages/advanced-algorithms/available-algorithms/meta_util.mdx +++ b/pages/advanced-algorithms/available-algorithms/meta_util.mdx @@ -16,7 +16,7 @@ A module that contains procedures describing graphs on a meta-level. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/meta_util.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/meta_util.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/neighbors.mdx b/pages/advanced-algorithms/available-algorithms/neighbors.mdx index ce0cd70a0..585b059a5 100644 --- a/pages/advanced-algorithms/available-algorithms/neighbors.mdx +++ b/pages/advanced-algorithms/available-algorithms/neighbors.mdx @@ -18,7 +18,7 @@ network structure and connectivity. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/neighbors_module/neighbors_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/neighbors_module/neighbors_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/node.mdx b/pages/advanced-algorithms/available-algorithms/node.mdx index 5ed38df04..ba71b7e57 100644 --- a/pages/advanced-algorithms/available-algorithms/node.mdx +++ b/pages/advanced-algorithms/available-algorithms/node.mdx @@ -15,7 +15,7 @@ The `node` module provides a comprehensive toolkit for managing graph nodes, ena } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/node_module/node_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/node_module/node_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/node2vec.mdx b/pages/advanced-algorithms/available-algorithms/node2vec.mdx index 01b6a6e24..a58a9c685 100644 --- a/pages/advanced-algorithms/available-algorithms/node2vec.mdx +++ b/pages/advanced-algorithms/available-algorithms/node2vec.mdx @@ -66,7 +66,7 @@ are correct in approximately 75% cases). } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/node2vec.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/node2vec.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/node_similarity.mdx b/pages/advanced-algorithms/available-algorithms/node_similarity.mdx index b5696e96e..57589271d 100644 --- a/pages/advanced-algorithms/available-algorithms/node_similarity.mdx +++ b/pages/advanced-algorithms/available-algorithms/node_similarity.mdx @@ -52,7 +52,7 @@ takes into account pairwise similarities between two set of nodes. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/node_similarity_module/node_similarity_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/node_similarity_module/node_similarity_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/nodes.mdx b/pages/advanced-algorithms/available-algorithms/nodes.mdx index cfd3c5103..22cdff1ef 100644 --- a/pages/advanced-algorithms/available-algorithms/nodes.mdx +++ b/pages/advanced-algorithms/available-algorithms/nodes.mdx @@ -15,7 +15,7 @@ The `nodes` module provides a comprehensive toolkit for managing multiple graph } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/nodes_module/nodes_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/nodes_module/nodes_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/pagerank.mdx b/pages/advanced-algorithms/available-algorithms/pagerank.mdx index 045b6509a..8ceabb59d 100644 --- a/pages/advanced-algorithms/available-algorithms/pagerank.mdx +++ b/pages/advanced-algorithms/available-algorithms/pagerank.mdx @@ -42,7 +42,7 @@ PageRank implementation. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/pagerank_module/pagerank_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/pagerank_module/pagerank_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/path.mdx b/pages/advanced-algorithms/available-algorithms/path.mdx index 6ae75ce9c..a34654d22 100644 --- a/pages/advanced-algorithms/available-algorithms/path.mdx +++ b/pages/advanced-algorithms/available-algorithms/path.mdx @@ -19,7 +19,7 @@ various other path-oriented operations. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/path_module/path_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/path_module/path_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/periodic.mdx b/pages/advanced-algorithms/available-algorithms/periodic.mdx index e480283c6..87f241e75 100644 --- a/pages/advanced-algorithms/available-algorithms/periodic.mdx +++ b/pages/advanced-algorithms/available-algorithms/periodic.mdx @@ -44,7 +44,7 @@ procedure, the already committed batches cannot be rolled back. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/periodic_module/periodic.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/periodic_module/periodic.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/refactor.mdx b/pages/advanced-algorithms/available-algorithms/refactor.mdx index 987f09a10..36d5c054f 100644 --- a/pages/advanced-algorithms/available-algorithms/refactor.mdx +++ b/pages/advanced-algorithms/available-algorithms/refactor.mdx @@ -15,7 +15,7 @@ The `refactor` module provides utilities for changing nodes and relationships. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/refactor_module/refactor_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/refactor_module/refactor_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/search.mdx b/pages/advanced-algorithms/available-algorithms/search.mdx new file mode 100644 index 000000000..18d3d6bcb --- /dev/null +++ b/pages/advanced-algorithms/available-algorithms/search.mdx @@ -0,0 +1,196 @@ +--- +title: search +description: Look up nodes by a label and property value in Memgraph using a comparison operator, without writing the equivalent MATCH and WHERE clauses. +--- + +import { Callout } from 'nextra/components' +import { Cards } from 'nextra/components' +import GitHub from '/components/icons/GitHub' + +# search + +The `search` module finds nodes by comparing one or more of their properties +against a value. You describe which properties to search with a `{label: +property}` map, pick a comparison operator, and provide the value to compare +against, instead of writing the equivalent `MATCH` and `WHERE` clauses yourself. +When a matching label-property index exists, it is used automatically. + + + } + title="Source code" + href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/search_module/search_module.cpp" + /> + + +| Trait | Value | +| ------------------ | ---------- | +| **Module type** | module | +| **Implementation** | C++ | +| **Parallelism** | sequential | + +## Procedures + +Both procedures take the same arguments and differ only in how they handle a +node that matches more than once: `search.node` returns each matching node once, +while `search.node_all` returns one row per property that matches. + +### Arguments + +The `label_property_map` argument names the labels and properties to search. It +is a map from a label to a single property or to a list of properties, for +example `{Person: "name"}` or `{Person: ["name", "email"]}`. When a label maps +to a list of properties, a node matches if **any** of those properties matches +the value. The map may also be given as a JSON string, for example +`'{"Person": "name"}'` or `'{"Person": ["name", "email"]}'`. + +The `operator` argument selects the comparison to apply and is case-insensitive: + +| Operator | Meaning | +| ------------- | ---------------------------------------------------- | +| `=` / `exact` | Equal to the value. | +| `<>` | Not equal to the value. | +| `<` | Less than the value. | +| `<=` | Less than or equal to the value. | +| `>` | Greater than the value. | +| `>=` | Greater than or equal to the value. | +| `starts with` | String starts with the value. | +| `ends with` | String ends with the value. | +| `contains` | String contains the value. | +| `=~` | String matches the value as a regular expression. | + +The `starts with`, `ends with` and `contains` operators only match string +properties; against a non-string property they safely skip the node. The `=~` +regular-expression operator is not guarded this way and should only be used +against string properties. String comparisons are performed by codepoint, so +uppercase letters sort before lowercase ones. + +### `node()` + +Returns each node that matches the search criteria once, even if it matches on +more than one label or property. + + +This procedure is equivalent to **apoc.search.node**. + + +{

Input:

} + +- `label_property_map: Any` ➡ A map (or JSON string) from a label to the property or list of properties to search. +- `operator: string` ➡ The comparison operator to apply. Case-insensitive. +- `value: string` ➡ The value to compare each property against. If `null`, no nodes are returned. + +{

Output:

} + +- `node: Node` ➡ A node matching the search criteria. Each matching node is returned once. + +{

Usage:

} + +Given the following graph: + +```cypher +CREATE (:Person {name: 'Alice'}); +CREATE (:Person {name: 'Bob'}); +CREATE (:Person {name: 'Bobby'}); +CREATE (:Person {name: 'Carol'}); +``` + +The following query returns every `Person` whose `name` is greater than or equal +to `'Bob'`: + +```cypher +CALL search.node({Person: 'name'}, '>=', 'Bob') YIELD node +RETURN node.name AS name ORDER BY name; +``` + +```plaintext ++----------------------------+ +| name | ++----------------------------+ +| "Bob" | +| "Bobby" | +| "Carol" | ++----------------------------+ +``` + +The `label_property_map` can also be given as a JSON string: + +```cypher +CALL search.node('{"Person": "name"}', 'exact', 'Bob') YIELD node +RETURN node.name AS name; +``` + +```plaintext ++----------------------------+ +| name | ++----------------------------+ +| "Bob" | ++----------------------------+ +``` + +When a node carries several of the searched labels, `search.node` still returns +it only once: + +```cypher +CREATE (:P {name: 'x'}); +CREATE (:M {title: 'x'}); +CREATE (:P:M {name: 'x', title: 'x'}); +``` + +```cypher +CALL search.node({P: 'name', M: 'title'}, 'exact', 'x') YIELD node +RETURN count(node) AS c; +``` + +```plaintext ++----------------------------+ +| c | ++----------------------------+ +| 3 | ++----------------------------+ +``` + +### `node_all()` + +Returns a node once for every property that matches the search criteria, so a +node that matches on several properties or labels appears in more than one row. + + +This procedure is equivalent to **apoc.search.nodeAll**. + + +{

Input:

} + +- `label_property_map: Any` ➡ A map (or JSON string) from a label to the property or list of properties to search. +- `operator: string` ➡ The comparison operator to apply. Case-insensitive. +- `value: string` ➡ The value to compare each property against. If `null`, no nodes are returned. + +{

Output:

} + +- `node: Node` ➡ A node matching the search criteria, returned once per matching property. + +{

Usage:

} + +Given the following graph, where one movie matches on `title` and the other on +`tagline`: + +```cypher +CREATE (:Movie {title: 'Matrix', tagline: 'Neo'}); +CREATE (:Movie {title: 'Heat', tagline: 'Matrix'}); +``` + +Searching both properties returns both movies: + +```cypher +CALL search.node_all('{"Movie": ["title", "tagline"]}', 'exact', 'Matrix') YIELD node +RETURN node.title AS title ORDER BY title; +``` + +```plaintext ++----------------------------+ +| title | ++----------------------------+ +| "Heat" | +| "Matrix" | ++----------------------------+ +``` diff --git a/pages/advanced-algorithms/available-algorithms/set_cover.mdx b/pages/advanced-algorithms/available-algorithms/set_cover.mdx index d8dd6023e..1d9d7c45a 100644 --- a/pages/advanced-algorithms/available-algorithms/set_cover.mdx +++ b/pages/advanced-algorithms/available-algorithms/set_cover.mdx @@ -22,7 +22,7 @@ a constraint programming solver. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/set_cover.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/set_cover.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/set_property.mdx b/pages/advanced-algorithms/available-algorithms/set_property.mdx index 6c991ef9b..6786c7d2c 100644 --- a/pages/advanced-algorithms/available-algorithms/set_property.mdx +++ b/pages/advanced-algorithms/available-algorithms/set_property.mdx @@ -16,7 +16,7 @@ procedures included in it involve copying properties from one entity to another. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/set_property_module/set_property_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/set_property_module/set_property_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/temporal.mdx b/pages/advanced-algorithms/available-algorithms/temporal.mdx index 0a7b62a02..fb26ff16f 100644 --- a/pages/advanced-algorithms/available-algorithms/temporal.mdx +++ b/pages/advanced-algorithms/available-algorithms/temporal.mdx @@ -17,7 +17,7 @@ functions. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/temporal.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/temporal.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/text.mdx b/pages/advanced-algorithms/available-algorithms/text.mdx index 2c738184f..131ca5439 100644 --- a/pages/advanced-algorithms/available-algorithms/text.mdx +++ b/pages/advanced-algorithms/available-algorithms/text.mdx @@ -20,7 +20,7 @@ The `text` module offers a toolkit for manipulating strings. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/text_module/text_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/text_module/text_module.cpp" /> @@ -297,3 +297,46 @@ Result: | 1 | +--------+ ``` + +### `compare_cleaned()` + +Compares two strings for equality after normalizing each one: keeping only ASCII +letters and digits, converting them to lowercase, and dropping everything else +(accents, punctuation, whitespace, and non-ASCII characters). + + +This function is equivalent to **apoc.text.compareCleaned**. + + + +Normalization is limited to ASCII and performs no Unicode folding, so accented +and non-ASCII letters are dropped rather than reduced to a base letter. For +example, `café` cleans to `caf` and is therefore not equal to `cafe`. + + +{

Input:

} + +- `text1: string` ➡ The first string to normalize and compare. A `null` value results in `false`. +- `text2: string` ➡ The second string to normalize and compare. A `null` value results in `false`. + +{

Output:

} + +- `boolean` ➡ `true` if the two normalized strings are equal, and `false` otherwise. + +{

Usage:

} + +Use the following query to compare two strings while ignoring case and punctuation: + +```cypher +RETURN text.compare_cleaned("Hello, World!", "hello world") AS result; +``` + +Result: + +```plaintext ++--------+ +| result | ++--------+ +| true | ++--------+ +``` diff --git a/pages/advanced-algorithms/available-algorithms/tgn.mdx b/pages/advanced-algorithms/available-algorithms/tgn.mdx index 244bcc986..3574d2e1e 100644 --- a/pages/advanced-algorithms/available-algorithms/tgn.mdx +++ b/pages/advanced-algorithms/available-algorithms/tgn.mdx @@ -138,7 +138,7 @@ new edges are forwarded to the **TGN** and so on. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/tgn.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/tgn.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/tsp.mdx b/pages/advanced-algorithms/available-algorithms/tsp.mdx index 8b8cc9595..ac791253a 100644 --- a/pages/advanced-algorithms/available-algorithms/tsp.mdx +++ b/pages/advanced-algorithms/available-algorithms/tsp.mdx @@ -29,7 +29,7 @@ needs to have its *lat* and *lng* property. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/tsp.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/tsp.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/union_find.mdx b/pages/advanced-algorithms/available-algorithms/union_find.mdx index bf4a421f6..ab46a3c34 100644 --- a/pages/advanced-algorithms/available-algorithms/union_find.mdx +++ b/pages/advanced-algorithms/available-algorithms/union_find.mdx @@ -27,7 +27,7 @@ Algorithms](https://dl.acm.org/doi/10.1145/62.2160)" and presented with } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/union_find.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/union_find.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/util_module.mdx b/pages/advanced-algorithms/available-algorithms/util_module.mdx index 4349dc27c..c75b605ca 100644 --- a/pages/advanced-algorithms/available-algorithms/util_module.mdx +++ b/pages/advanced-algorithms/available-algorithms/util_module.mdx @@ -17,7 +17,7 @@ for streamlining a variety of tasks related to database operations. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/util_module/util_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/util_module/util_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/uuid_generator.mdx b/pages/advanced-algorithms/available-algorithms/uuid_generator.mdx index 464d1b3ed..f4d378305 100644 --- a/pages/advanced-algorithms/available-algorithms/uuid_generator.mdx +++ b/pages/advanced-algorithms/available-algorithms/uuid_generator.mdx @@ -19,7 +19,7 @@ installed by running `sudo apt-get install uuid-dev`. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/uuid_module/uuid_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/uuid_module/uuid_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/vrp.mdx b/pages/advanced-algorithms/available-algorithms/vrp.mdx index 419b41e14..22791a88b 100644 --- a/pages/advanced-algorithms/available-algorithms/vrp.mdx +++ b/pages/advanced-algorithms/available-algorithms/vrp.mdx @@ -31,7 +31,7 @@ have its *lat* and *lng* property. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/vrp.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/vrp.py" /> diff --git a/pages/advanced-algorithms/available-algorithms/weakly_connected_components.mdx b/pages/advanced-algorithms/available-algorithms/weakly_connected_components.mdx index bbf987914..d2e949d80 100644 --- a/pages/advanced-algorithms/available-algorithms/weakly_connected_components.mdx +++ b/pages/advanced-algorithms/available-algorithms/weakly_connected_components.mdx @@ -20,7 +20,7 @@ there is no edge that connects nodes from separate components. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/cpp/connectivity_module/connectivity_module.cpp" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/cpp/connectivity_module/connectivity_module.cpp" /> diff --git a/pages/advanced-algorithms/available-algorithms/xml_module.mdx b/pages/advanced-algorithms/available-algorithms/xml_module.mdx index bf681e96d..16edf36e6 100644 --- a/pages/advanced-algorithms/available-algorithms/xml_module.mdx +++ b/pages/advanced-algorithms/available-algorithms/xml_module.mdx @@ -16,7 +16,7 @@ loading and parsing XML data. } title="Source code" - href="https://github.com/memgraph/memgraph/blob/master/mage/python/xml_module.py" + href="https://github.com/memgraph/memgraph/blob/master/src/mage/python/xml_module.py" /> diff --git a/pages/advanced-algorithms/install-mage.mdx b/pages/advanced-algorithms/install-mage.mdx index feb131926..81cab9145 100644 --- a/pages/advanced-algorithms/install-mage.mdx +++ b/pages/advanced-algorithms/install-mage.mdx @@ -124,49 +124,25 @@ Memgraph package](/getting-started/install-memgraph). Algorithms and query modules will be loaded into a Memgraph instance on startup once you install MAGE, so make sure your instances are not running. -{

Install dependencies

} - -To build from source, you will need: -- Python3 -- Make -- CMake -- Clang -- UUID -- [Rust and Cargo](https://doc.rust-lang.org/cargo/getting-started/installation.html) - -{

Set up the machine

} +{

Download the Memgraph source code

} -Run the following commands: +MAGE is developed and built as part of the Memgraph repository. Clone the +[Memgraph source code](https://github.com/memgraph/memgraph) from GitHub +(install `git` first if you don't have it — `sudo apt-get install -y git`): -```bash -sudo apt-get update && sudo apt-get install -y \ - libcurl4 \ - libpython3.12 \ - libssl-dev \ - openssl \ - build-essential \ - cmake \ - curl \ - g++ \ - python3 \ - python3-pip \ - python3-setuptools \ - python3-dev \ - clang \ - git \ - pkg-config \ - uuid-dev \ - xmlsec1 \ - ninja-build \ - --no-install-recommends +``` +git clone https://github.com/memgraph/memgraph.git && cd memgraph/ ``` -{

Download the MAGE source code

} +{

Install dependencies

} -Clone the [Memgraph source code](https://github.com/memgraph/memgraph) from GitHub: +The repository ships scripts that install everything the toolchain and the +build need — `build.sh` checks for both sets and stops if anything is +missing: -``` -git clone https://github.com/memgraph/memgraph.git && cd memgraph/ +```bash +sudo ./environment/os/install_deps.sh install TOOLCHAIN_RUN_DEPS +sudo ./environment/os/install_deps.sh install MEMGRAPH_BUILD_DEPS ``` {

Set up the toolchain

} @@ -177,23 +153,18 @@ curl -L https://s3-eu-west-1.amazonaws.com/deps.memgraph.io/toolchain-v7/toolcha sudo tar xzvfm toolchain.tar.gz -C /opt ``` -Install runtime dependencies for the toolchain: -```bash -sudo ./environment/os/install_deps.sh install TOOLCHAIN_RUN_DEPS -``` - {

Install Rust and Python dependencies

} -Run the following command to install Rust and Python dependencies: +Run the following commands from the root of the repository to install Rust +and the Python packages the MAGE query modules use at runtime: ```shell -cd mage -curl https://sh.rustup.rs -sSf | sh -s -- -y -export PATH="/root/.cargo/bin:${PATH}" -python3 -m pip install -r python/requirements.txt -python3 -m pip install -r ../src/auth/reference_modules/requirements.txt -python3 -m pip install torch-sparse torch-cluster torch-spline-conv torch-geometric torch-scatter -f https://data.pyg.org/whl/torch-2.6.0+cpu.html -python3 -m pip install dgl -f https://data.dgl.ai/wheels/torch-2.6/repo.html +source environment/util.sh +install_rust 1.89 +python3 -m pip install -r src/mage/python/requirements.txt +python3 -m pip install -r src/auth/reference_modules/requirements.txt +python3 -m pip install torch-sparse torch-cluster torch-spline-conv torch-geometric torch-scatter -f https://data.pyg.org/whl/torch-2.8.0+cpu.html +python3 -m pip install dgl -f https://data.dgl.ai/wheels/torch-2.8/repo.html ``` @@ -201,46 +172,50 @@ python3 -m pip install dgl -f https://data.dgl.ai/wheels/torch-2.6/repo.html To install the dependencies for GPU-accelerated algorithms, you need to use the GPU-specific requirements file: ```shell -python3 -m pip install -r python/requirements-gpu.txt +python3 -m pip install -r src/mage/python/requirements-gpu.txt ``` -{

Run the `setup` script

} +{

Build and install MAGE

} -Run the following command: +MAGE is built with the same build system as Memgraph. Run the following +commands from the root of the repository: ```shell source /opt/toolchain-v7/activate -python3 setup build -sudo cp -r dist/* /usr/lib/memgraph/query_modules +./build.sh --mage only +sudo cmake --install build --component mage --prefix /usr ``` - +`./build.sh --mage only` builds just the MAGE query modules (C++, Python and +Rust) without Memgraph itself — the script sets up everything else it needs +(a Python virtual environment, the Conan package manager and the project's +dependencies) on first run. The built modules land in `build/mage/dist`. -If you don't need all of the algorithms you can build only some of them based on the laguauge. +The `cmake --install` command then installs the modules to +`/usr/lib/memgraph/query_modules`, the directory Memgraph loads query modules +from, together with the runtime libraries they need. -To build C++ based algorithms run: + + +If you don't need all of the algorithms, you can build a subset by passing +specific targets: ```shell -python3 setup build --lang cpp -``` +# Only the Python modules (a copy step - fast) +./build.sh --mage only --target mage_python_modules -To build Python based algorithms run: +# Only the Rust modules +./build.sh --mage only --target mage_rust_modules -```shell -python3 setup build --lang python +# Individual C++ modules by name +./build.sh --mage only --target map text ``` -The script will generate a `dist` directory with all the needed files. - -It will also copy the contents of the newly created `dist` directory to -`/usr/lib/memgraph/query_modules`. Memgraph loads algorithms and modules from -this directory. - -If something isn't installed properly, the `setup` script will stop the -installation process. If you have any questions, contact us on +If something isn't set up properly, the build will stop with an error. If you +have any questions, contact us on **[Discord](https://discord.gg/memgraph).** {

Start a Memgraph instance

} diff --git a/pages/clustering/high-availability.mdx b/pages/clustering/high-availability.mdx index be6a01e31..f999f27a9 100644 --- a/pages/clustering/high-availability.mdx +++ b/pages/clustering/high-availability.mdx @@ -49,6 +49,16 @@ recommended configuration patterns. Recommended practices for running a robust, reliable, and well-observed HA deployment. +### [Coordinator authentication](/clustering/high-availability/coordinator-authentication) + +Secure the cluster's control plane with single sign-on, Raft-replicated +coordinator roles, and the `COORDINATOR_READ` / `COORDINATOR_WRITE` privileges. + +### [Bulk import in analytical mode](/clustering/high-availability/analytical-import) + +Import data into a cluster using the in-memory analytical storage mode, then +switch back to transactional mode and register the replicas again. + ### [HA commands reference guide](/clustering/high-availability/ha-commands-reference) A complete reference of all commands for managing coordinators, registering diff --git a/pages/clustering/high-availability/_meta.ts b/pages/clustering/high-availability/_meta.ts index f09ece7ce..671508ae1 100644 --- a/pages/clustering/high-availability/_meta.ts +++ b/pages/clustering/high-availability/_meta.ts @@ -1,10 +1,12 @@ export default { "how-high-availability-works": "Under the hood", "querying-the-cluster-in-high-availability": "Querying the cluster in HA", + "coordinator-authentication": "Coordinator authentication", "setup-ha-cluster-docker": "Set up HA cluster with Docker", "setup-ha-cluster-docker-compose": "Set up HA cluster with Docker Compose", "setup-ha-cluster-k8s": "Set up HA cluster with K8s", "best-practices": "Best practices", + "analytical-import": "Bulk import in analytical mode", "ha-commands-reference": "Reference commands", "ha-reference-architectures": "Reference architectures", "migrating-to-v3-9-ha": "Migrating to v3.9 HA", diff --git a/pages/clustering/high-availability/analytical-import.mdx b/pages/clustering/high-availability/analytical-import.mdx new file mode 100644 index 000000000..6602f7f86 --- /dev/null +++ b/pages/clustering/high-availability/analytical-import.mdx @@ -0,0 +1,222 @@ +--- +title: Bulk import in analytical mode +description: Import data into a high availability cluster using the in-memory analytical storage mode, then switch back to transactional and register the replicas again. +--- + +import { Callout } from 'nextra/components' +import { Steps } from 'nextra/components' +import {CommunityLinks} from '/components/social-card/CommunityLinks' + +# Bulk import in analytical mode Enterprise + +The [in-memory analytical storage +mode](/fundamentals/storage-memory-usage#in-memory-analytical-storage-mode) +imports data up to 6 times faster and with significantly lower memory usage than +the transactional mode, because it does not create `Delta` objects. As of +Memgraph v3.13, a data instance in a high availability cluster can use that mode +for a bulk import: switch to `IN_MEMORY_ANALYTICAL`, import, switch back to +`IN_MEMORY_TRANSACTIONAL` and register the replicas again. + +Analytical writes are **never written to the WAL**, so they cannot be +replicated. The whole workflow is therefore built around a single rule: while +any database on the instance is analytical, the instance must have no replicas +attached to it. + + + +Before continuing, read the guides on [how high availability +works](/clustering/high-availability/how-high-availability-works) and on +[storage modes](/fundamentals/storage-memory-usage#storage-modes). + + + +## Requirements + +A data instance can enter the analytical storage mode only when: + +- it holds the **MAIN** role — a REPLICA receives replicated writes, which + analytical mode cannot apply, and +- it has **zero registered replicas**. + +Both conditions are instance-wide and all-or-nothing: + +- entering analytical mode on **any** database requires that no replica is + registered at all, and +- while **any** database is in analytical mode, registering and unregistering + replicas is rejected. + +Because [`UNREGISTER +INSTANCE`](/clustering/high-availability/ha-commands-reference#unregister-instance) +refuses to remove the current MAIN, the instance you import into is the one data +instance that stays in the cluster. + + + +While the import runs, the cluster consists of a single data instance and has +**no redundancy**: there is no replica to fail over to. Plan the import for a +maintenance window and keep it as short as possible. + + + +Starting a data instance with +[`--storage-mode=IN_MEMORY_ANALYTICAL`](/database-management/configuration) +remains forbidden. An instance started in analytical mode can never produce WAL +files, not even after switching to transactional mode, which would permanently +break replication. Analytical mode is reachable only through the runtime +`STORAGE MODE` query from a transactional start. + +## Import workflow + + + +### Unregister every replica + +On the **coordinator**, remove all data instances except the MAIN: + +```cypher +UNREGISTER INSTANCE instance_2; +``` + +The unregistered instance keeps running and keeps its data — unregistering +removes it from the cluster, it does not wipe it. You do **not** need to clear +its data directory before registering it back. + +### Switch the MAIN to analytical mode + +On the **MAIN data instance**: + +```cypher +STORAGE MODE IN_MEMORY_ANALYTICAL; +``` + +If a replica is still registered, the query fails and the storage mode is +unchanged. + +### Import the data + +Run the import as usual, for example with [`LOAD +CSV`](/data-migration/csv) or plain Cypher: + +```cypher +LOAD CSV FROM "/import/nodes.csv" WITH HEADER AS row +CREATE (:Node {id: row.id}); +``` + +### Switch back to transactional mode + +```cypher +STORAGE MODE IN_MEMORY_TRANSACTIONAL; +``` + +This writes a snapshot of the imported data synchronously, before the mode +change completes. Do **not** run `CREATE SNAPSHOT` while still in analytical +mode — see [Durability of the switch back](#durability-of-the-switch-back). + +### Register the replicas back + +On the **coordinator**: + +```cypher +REGISTER INSTANCE instance_2 WITH CONFIG { + "bolt_server": "localhost:7688", + "management_server": "localhost:10012", + "replication_server": "localhost:10002" +}; +``` + +The replica recovers from the snapshot written in the previous step and ends up +with exactly the data the MAIN holds. Verify with `SHOW INSTANCES` on the +coordinator and by counting nodes on the replica's own Bolt endpoint. + + + +## Durability of the switch back + +The `IN_MEMORY_ANALYTICAL` → `IN_MEMORY_TRANSACTIONAL` switch is the point at +which the imported data becomes durable, so it does more than flip a flag: + +- **It writes a snapshot** stamped with a timestamp that covers the import, and + publishes that timestamp as the instance's last durable timestamp. This is + what a re-registered replica recovers from. +- **If the snapshot cannot be written, the switch is aborted** and the query + throws. The database stays in analytical mode with its data and all durability + files untouched, so you can fix the cause (most often disk space) and retry. +- **Superseded snapshots and WAL files are archived**. The switch-back snapshot + is a new durability base rather than an increment on the old one, so older + snapshots and all WAL files are moved to the `.old` directory — or deleted + when [`--storage-backup-dir-enabled`](/database-management/configuration) is + set to `false` — and WAL sequence numbering restarts from 0. +- **The WAL is finalized when analytical mode is entered**, which is what lets + replica recovery detect that the imported data exists in no WAL file and must + be shipped as a snapshot. + + + +Do not run `CREATE SNAPSHOT` between the import and the switch back. A snapshot +taken while the instance is in analytical mode carries the **pre-import** +durable timestamp, because analytical writes never advance it. It is redundant +at best — the switch back writes a correctly stamped snapshot on its own. + + + +## Re-registering a replica that holds old data + +A replica that was unregistered before the import keeps whatever it had at that +moment, and it may be strictly behind the MAIN. When you register it back, +Memgraph compares the newest snapshot's timestamp against the ranges of the WAL +files. The analytical episode leaves a gap that no WAL covers, so recovery +detects that the WAL chain cannot reproduce the imported data and sends the +snapshot instead of the WAL files. + +The consequences for you as an operator: + +- There is **no need to wipe the replica's data directory** before registering + it back. +- The detection is derived from the durability files, not from in-memory state, + so it also works if the MAIN is **restarted** between the import and the + re-registration. + +## Cluster changes while the MAIN is still analytical + +`REGISTER INSTANCE` and `UNREGISTER INSTANCE` are committed to the Raft log +first and only then applied on the MAIN over RPC, so both queries still report +success while the MAIN is analytical, even though the MAIN rejects the RPC and +logs the reason: + +- **`REGISTER INSTANCE`** — the instance is part of the cluster state, but no + replication client is created for it, so it receives nothing. +- **`UNREGISTER INSTANCE`** — the instance is removed from the cluster state, + but the MAIN keeps its replication client. + +In both cases the [reconciliation +loop](/clustering/high-availability/how-high-availability-works#how-the-reconciliation-loop-works) +resolves the difference on its own once every database is back in transactional +mode. Still, a query reported as successful does not mean the replica is +attached or detached yet: change the cluster composition only while the cluster +is in transactional mode, and confirm the state with `SHOW INSTANCES` on the +coordinator. + +## Errors + +| Query | Error | Cause | +|-------|-------|-------| +| `STORAGE MODE IN_MEMORY_ANALYTICAL` | `Only the MAIN data instance can use analytical mode.` | The instance holds the REPLICA role. | +| `STORAGE MODE IN_MEMORY_ANALYTICAL` | `Cannot switch to analytical mode while replicas are registered (...)` | At least one replica is registered. The message lists the names. | +| `STORAGE MODE IN_MEMORY_TRANSACTIONAL` | `Failed to create the snapshot required to leave IN_MEMORY_ANALYTICAL.` | The switch-back snapshot could not be written. The database stays analytical and unchanged. | +| `REGISTER REPLICA` | `Couldn't register replica ... because a database is in analytical storage mode.` | Some database on the instance is analytical. | +| `DROP REPLICA` | `Couldn't unregister replica ... because a database is in analytical storage mode.` | Some database on the instance is analytical. | + +The same gates apply to the RPCs the coordinator sends behind `REGISTER +INSTANCE` and `UNREGISTER INSTANCE`; there the rejection is visible in the data +instance's log rather than in the query result. + +## Plain replication clusters + +The gates are not specific to high availability. In a [replication +cluster](/clustering/replication) without coordinators, a MAIN with registered +replicas is likewise refused the switch to analytical mode, and `REGISTER +REPLICA` / `DROP REPLICA` are refused while any database is analytical. Use the +same workflow with `DROP REPLICA` and `REGISTER REPLICA` in place of the +coordinator queries. + + diff --git a/pages/clustering/high-availability/best-practices.mdx b/pages/clustering/high-availability/best-practices.mdx index 80fb03e94..39af74b7d 100644 --- a/pages/clustering/high-availability/best-practices.mdx +++ b/pages/clustering/high-availability/best-practices.mdx @@ -189,6 +189,20 @@ the command line argument. +## Storage mode + +Data instances run in the `IN_MEMORY_TRANSACTIONAL` storage mode. Analytical +writes are not written to the WAL and therefore cannot be replicated, so a data +instance can enter `IN_MEMORY_ANALYTICAL` only when it holds the MAIN role and +has no registered replicas, and replicas can be registered or unregistered only +while every database is transactional. + +For a fast bulk import, unregister the replicas, switch the MAIN to analytical +mode, import, switch back to transactional mode and register the replicas again. +The full recipe, together with the durability guarantees of the switch back, is +in [Bulk import in analytical +mode](/clustering/high-availability/analytical-import). + ## Observability Monitoring cluster health is essential. Key metrics include: diff --git a/pages/clustering/high-availability/coordinator-authentication.mdx b/pages/clustering/high-availability/coordinator-authentication.mdx new file mode 100644 index 000000000..6ec9b1223 --- /dev/null +++ b/pages/clustering/high-availability/coordinator-authentication.mdx @@ -0,0 +1,592 @@ +--- +title: Coordinator authentication and authorization +description: Learn how to secure Memgraph high availability coordinators with single sign-on, Raft-replicated roles and the COORDINATOR_READ and COORDINATOR_WRITE privileges. +--- + +import { Callout } from 'nextra/components' +import { Steps } from 'nextra/components' +import {CommunityLinks} from '/components/social-card/CommunityLinks' + +# Coordinator authentication and authorization Enterprise + +Coordinators are the control plane of a [high availability +cluster](/clustering/high-availability): they register and unregister data +instances, promote and demote MAIN, hand out the routing table, and change +cluster-wide settings. From Memgraph 3.13, that control plane can be protected +with [single sign-on (SSO)](/database-management/authentication-and-authorization/auth-system-integrations#single-sign-on) +against your corporate identity provider (IdP), backed by a small set of +Raft-replicated roles. + + +SSO authentication on coordinators, coordinator role management, privilege +grants and privilege enforcement are **Memgraph Enterprise** features and +require a valid license. Without a valid license, coordinators keep the +pre-3.13 behavior: username/password connections are accepted as a passthrough +with full access. + + +Before continuing, read [how high availability +works](/clustering/high-availability/how-high-availability-works) and [querying +the cluster in high +availability](/clustering/high-availability/querying-the-cluster-in-high-availability). + +## How coordinator auth differs from data instances + +Coordinators do **not** store users. There is no auth key-value store on a +coordinator, no `CREATE USER`, no passwords, and no fine-grained access control. +The only authorization state a coordinator keeps is a list of **roles**, each +carrying a coordinator privilege mask. + +| | Data instance | Coordinator | +|---|---|---| +| Users | Stored in the auth store | Not supported | +| Roles | Stored in the auth store | Stored in the **Raft-replicated cluster state** | +| Privileges | Full privilege set + fine-grained access control | Exactly two: `COORDINATOR_READ`, `COORDINATOR_WRITE` | +| Basic auth | Username and password are validated | Passthrough — credentials are ignored (see [below](#basic-authentication-passthrough)) | +| SSO | Supported (OIDC, SAML, Kerberos) | Supported (OIDC, SAML, Kerberos) | + +Because roles live in the Raft log, they survive coordinator restarts, are +replicated to every coordinator, are reconstructed by a follower that rejoins +after being down, and survive leader failover. + + +**Breaking change in Memgraph 3.13:** the `COORDINATOR` privilege has been +removed. It never gated any operation. It is no longer accepted in `GRANT` / +`DENY` / `REVOKE` statements and is no longer reported by `SHOW PRIVILEGES`. A +stale `COORDINATOR` grant in an existing deployment is simply never reported and +never checked — no migration is required. Coordinator access is now controlled +by the new `COORDINATOR_READ` and `COORDINATOR_WRITE` privileges described +below, which are granted **on the coordinators themselves**, not on data +instances. + + +## Coordinator privileges + +Coordinators enforce exactly two privileges: + +| Privilege | Grants | +|---|---| +| `COORDINATOR_READ` | Reading the routing table and all read-only introspection queries. | +| `COORDINATOR_WRITE` | Every query runnable on a coordinator. `COORDINATOR_WRITE` is a **superset** of `COORDINATOR_READ`. | + +These names were chosen because bare `READ` and `WRITE` already exist as +data-instance fine-grained privileges. They are meaningful **only on +coordinators** and are deliberately excluded from the data-instance privilege +set, so `GRANT ALL PRIVILEGES` on a data instance does not grant them. + +A session's effective privilege is the **union** of the masks of all its roles. +A role with no grant confers nothing. + +### Which privilege each query requires + +Read-only introspection requires `COORDINATOR_READ`; every mutating or +administrative query requires `COORDINATOR_WRITE`. A `COORDINATOR_WRITE` grant +satisfies a `COORDINATOR_READ` requirement, but not the other way round. + +| Query | Required privilege | +|---|---| +| Routing table (Bolt `ROUTE` message) | `COORDINATOR_READ` | +| [`SHOW INSTANCE`](/clustering/high-availability/ha-commands-reference#show-instance) | `COORDINATOR_READ` | +| [`SHOW INSTANCES`](/clustering/high-availability/ha-commands-reference#show-instances) | `COORDINATOR_READ` | +| [`SHOW COORDINATOR SETTINGS`](/clustering/high-availability/ha-commands-reference#coordinator-runtime-settings) | `COORDINATOR_READ` | +| [`SHOW REPLICATION LAG`](/clustering/high-availability/ha-commands-reference#show-replication-lag) | `COORDINATOR_READ` | +| `SHOW ROLES` | `COORDINATOR_READ` | +| `SHOW PRIVILEGES FOR ROLE ` | `COORDINATOR_READ` | +| `SHOW CONFIG`, `SHOW SETTING`, system info queries | `COORDINATOR_READ` | +| [`SHOW VERSION`](/database-management/server-stats#instance-version) | `COORDINATOR_READ` | +| `SHOW CURRENT USER`, `SHOW CURRENT ROLE` | **None** (self-service, see [below](#self-service-identity-queries)) | +| [`REGISTER INSTANCE`](/clustering/high-availability/ha-commands-reference#register-instance) / [`UNREGISTER INSTANCE`](/clustering/high-availability/ha-commands-reference#unregister-instance) | `COORDINATOR_WRITE` | +| [`SET INSTANCE ... TO MAIN`](/clustering/high-availability/ha-commands-reference#set-instance--to-main) / [`DEMOTE INSTANCE`](/clustering/high-availability/ha-commands-reference#demote-instance) | `COORDINATOR_WRITE` | +| [`ADD COORDINATOR`](/clustering/high-availability/ha-commands-reference#add-coordinator) / [`REMOVE COORDINATOR`](/clustering/high-availability/ha-commands-reference#remove-coordinator) | `COORDINATOR_WRITE` | +| [`UPDATE CONFIG`](/clustering/high-availability/ha-commands-reference#update-config) | `COORDINATOR_WRITE` | +| [`YIELD LEADERSHIP`](/clustering/high-availability/ha-commands-reference#yield-leadership) | `COORDINATOR_WRITE` | +| [`SET COORDINATOR SETTING`](/clustering/high-availability/ha-commands-reference#coordinator-runtime-settings) | `COORDINATOR_WRITE` | +| [`FORCE RESET CLUSTER STATE`](/clustering/high-availability/ha-commands-reference#force-reset-cluster-state) | `COORDINATOR_WRITE` | +| `SET SETTING` | `COORDINATOR_WRITE` | +| [`RELOAD BOLT_SERVER TLS` / `RELOAD INTRA_CLUSTER TLS`](/database-management/ssl-encryption#reload-ssl-certificates-at-runtime) | `COORDINATOR_WRITE` | +| `CREATE ROLE`, `DROP ROLE`, `GRANT`, `REVOKE` | `COORDINATOR_WRITE` | + +Anything not recognized fails closed and requires `COORDINATOR_WRITE`. + +## Authentication modes + +A coordinator accepts exactly two kinds of Bolt connection: **basic/none** and +an **SSO scheme listed in `--auth-module-mappings`**. Any other scheme is +rejected with: + +``` +The "" authentication scheme isn't supported on this coordinator; +connect with basic auth or an SSO scheme listed in the auth-module-mappings flag. +``` + +### Basic authentication passthrough + +When SSO is not in effect, connecting with a username and password (or with no +auth at all) **succeeds and the credentials are ignored**. The session gets full +`COORDINATOR_WRITE` access. This is the pre-3.13 behavior, it requires no +license, and it keeps existing admin tooling working unchanged. + +Basic/none authentication is **denied** only when **all three** of the following +hold: + +1. SSO is configured — `--auth-module-mappings` is non-empty. +2. The enterprise license is valid. +3. The committed role set contains at least one role holding + `COORDINATOR_WRITE`. + +In that case the connection is rejected with: + +``` +Basic authentication is disabled on this coordinator because SSO is configured; +connect with an SSO scheme listed in the auth-module-mappings flag. +``` + + +**Anybody can log in until a `COORDINATOR_WRITE` role exists.** Condition 3 is +what makes coordinator SSO self-bootstrapping. On a coordinator that starts with +`--auth-module-mappings` set but an empty role set, SSO cannot yet grant a +privileged session to anybody — so basic auth stays open, and you use it to +create the first role and grant it `COORDINATOR_WRITE`. The moment that grant +commits to Raft, basic auth closes on the next login attempt and SSO takes over. +**No coordinator restart is required.** + +The same rule applies in reverse: if you drop or revoke the last +`COORDINATOR_WRITE` role on a live cluster, basic auth reopens rather than +leaving the cluster unadministrable. + + + +**Break-glass on license loss.** If the enterprise license is missing, expired +or invalid, SSO rejects every login. Condition 2 above means basic auth falls +back to the passthrough in exactly that case, so a license transition can never +lock every Bolt session out of a coordinator. Use that session to re-install the +license over Bolt: + +```cypher +SET DATABASE SETTING 'enterprise.license' TO 'License'; +SET DATABASE SETTING 'organization.name' TO 'Organization'; +``` + +The license check for this decision is the full, non-cached check — a license +that expires by date takes effect immediately rather than at the next cache +refresh. + + +### A follower without quorum cannot be logged into + +Both the basic-auth decision and the SSO role check need the **leader's** +committed role set, and both are **fail-closed**: when the leader cannot be +reached, the login is rejected rather than validated against possibly-stale +local replicated state, which could still list a dropped role or an +already-revoked privilege mask. + +So on a coordinator that has lost quorum (it is a follower, no leader is +elected, or the leader is unreachable) and has SSO configured with a valid +license: + +- **SSO logins are rejected** with: + + ``` + SSO authentication failed: the coordinator leader is unreachable, so roles + can't be validated. Retry once a leader is elected. + ``` + +- **Basic/none logins are also rejected.** An unknown role set is not treated as + "no writable role", so the break-glass path does not open during a transient + leader outage. + +This is intentional and temporary: SSO is unavailable in that window anyway, and +access returns as soon as a leader is elected. If SSO is **not** configured, the +basic-auth passthrough never contacts the leader, so it keeps working on a +partitioned follower. + +### SSO authentication + +For a scheme present in `--auth-module-mappings`, the coordinator runs the +corresponding auth module and genuinely authenticates the connection. The same +[built-in and custom auth +modules](/database-management/authentication-and-authorization/auth-system-integrations) +used on data instances work here, and `MEMGRAPH_SSO_*` environment variables are +inherited by the module subprocess exactly as they are on data instances — there +is no separate coordinator-side SSO configuration. + +The login is accepted only when **all** of the following hold: + +1. The module authenticates the identity (valid, unexpired IdP token). +2. The module returns at least one role. +3. **Every** role the module returns exists in the coordinator's committed role + set. A multi-role response succeeds only when all of its roles exist. +4. The union of those roles' masks grants at least `COORDINATOR_READ`. + + +**A role without privileges cannot be used to log in.** If the roles all exist +but none of them has been granted `COORDINATOR_READ` or `COORDINATOR_WRITE`, the +connection is **rejected at login** rather than admitted as a session that would +be denied every query — including the routing table. Grant a privilege to the +role before mapping identities onto it. + + +Each rejection reason has its own message, so an operator rolling SSO out can +tell a bad token apart from a misconfigured role mapping: + +| Situation | Error returned to the client | +|---|---| +| Invalid/expired token, module failure, or missing license | `SSO authentication failed: the identity provider token was rejected, the auth module failed, or the enterprise license is missing.` | +| Module returned no roles | `SSO authentication failed: the identity provider returned no roles for this identity. Map the identity's group to a coordinator role.` | +| A returned role does not exist on the coordinator | `SSO authentication failed: the identity provider returned a role that doesn't exist on this coordinator. Create it with CREATE ROLE, or fix the identity provider mapping.` | +| Roles exist but carry no privilege | `SSO authentication failed: this identity's role(s) exist but carry no coordinator privilege. Grant COORDINATOR_READ or COORDINATOR_WRITE to one of them.` | +| Leader unreachable | `SSO authentication failed: the coordinator leader is unreachable, so roles can't be validated. Retry once a leader is elected.` | + +The offending role name is written to the coordinator log, **not** returned to +the client, so a rejected login cannot be used to enumerate the coordinator's +role set. + +The username the module reports is recorded as the session principal and is used +for [audit logging](/database-management/logs) and `SHOW CURRENT USER`. A module +that omits the username still logs in — the coordinator authorizes by role — but +its queries cannot be attributed to a person, and a warning is logged once at +login. + +## Managing coordinator roles and privileges + +All role and privilege queries are documented in the [HA reference +commands](/clustering/high-availability/ha-commands-reference#coordinator-role-and-privilege-management) +guide. In short: + +```cypher +CREATE ROLE ops; +CREATE ROLE IF NOT EXISTS ops; +DROP ROLE ops; +SHOW ROLES; + +GRANT COORDINATOR_READ TO ops; +GRANT COORDINATOR_WRITE TO ops; +GRANT ALL PRIVILEGES TO ops; -- grants both coordinator privileges +REVOKE COORDINATOR_WRITE FROM ops; +REVOKE ALL PRIVILEGES FROM ops; -- removes both + +SHOW PRIVILEGES FOR ROLE ops; +``` + +All of these can be run on **any** coordinator: on a follower they are +transparently forwarded to the leader. Writes are committed through the Raft +log; `SHOW ROLES` and `SHOW PRIVILEGES FOR ROLE` are strong reads served by the +leader. + +### Self-service identity queries + +`SHOW CURRENT USER` and `SHOW CURRENT ROLE` are exempt from the privilege check +and from the license gate — they only reveal the session's own identity, so even +a session whose roles were revoked mid-flight can still inspect who it is. + +- `SHOW CURRENT USER` returns the principal the identity provider authenticated. + It is purely session-local and works even when the leader is unreachable. A + basic-auth passthrough session authenticated no principal and returns `null`. +- `SHOW CURRENT ROLE` returns the session's roles **filtered against the + leader's committed role set**, so it stops naming a role that `DROP ROLE` + already removed. A basic-auth passthrough session has no roles and returns + `null`. + +### Auth queries rejected on coordinators + +Everything outside the small surface above is rejected with: + +``` +Coordinator can run only coordinator queries! +``` + +That includes: + +- User management: `CREATE USER`, `SET PASSWORD`, `SHOW USERS`, `SET ROLE`, + `GRANT ROLE`. +- `DENY` in **any** form. +- `GRANT` / `REVOKE` targeting a `USER` (`GRANT COORDINATOR_READ TO USER foo`). +- Privilege lists containing any non-coordinator privilege + (`GRANT MATCH TO ops`). +- [Fine-grained access + control](/database-management/authentication-and-authorization/role-based-access-control#fine-grained-access-control) + — `GRANT ... ON NODES ...` / `ON EDGES ...` — coordinators have no graph. +- Property permissions. +- [Multi-tenancy](/database-management/multi-tenancy) database access: + `GRANT DATABASE`, `REVOKE DATABASE`, `SET MAIN DATABASE`. +- `SHOW PRIVILEGES FOR USER `. +- `SHOW PRIVILEGES FOR ROLE ` with a trailing `ON MAIN`, `ON CURRENT` or + `ON DATABASE ` clause — coordinators have no databases. + +## Privileges are re-checked on every query + +An SSO session does **not** cache the privilege mask it was given at login. On +**every** query — and on every routing-table request — the coordinator +re-derives the session's effective mask from the **leader's** committed role set: +if this coordinator is the ready leader it reads locally, otherwise it sends a +`GetRolesRpc` to the leader. + +This has three consequences worth planning for: + +- **`REVOKE` and `DROP ROLE` take effect immediately**, without the client + reconnecting. Long-lived connections — driver routing pools, open admin shells + — are downgraded on their very next query. A session whose roles were all + dropped keeps its connection but is denied every privileged query. +- **A leader outage denies queries on SSO sessions.** With no readable role set + the effective mask is empty (fail-closed), so queries fail until a leader is + elected. Retry once `SHOW INSTANCES` reports a leader again. +- **Basic-auth passthrough sessions are unaffected.** They carry no roles, so + they keep their login-time full `COORDINATOR_WRITE` mask and never contact the + leader for a privilege check. + +Denied queries fail with: + +``` +You don't have the required privilege to run this query on the coordinator! +``` + +and a denied routing-table request fails with: + +``` +You don't have permission to read the routing table on the coordinator! +``` + +The routing-table denial is reported as a non-retryable client error, so drivers +do not retry it as if it were a transient failure. + +## Bolt+routing with SSO + + +**A `neo4j://` routing connection works only if the same roles exist on both the +coordinators and the data instances.** + + +[Bolt+routing](/clustering/high-availability/querying-the-cluster-in-high-availability) +is entirely client-side, and the driver reuses **one set of credentials for both +legs** of the connection. With SSO in the picture, a single `neo4j://` session +therefore performs two independent authentications: + + + +{

The driver authenticates against a coordinator

} + +It sends the SSO scheme and IdP token to a coordinator and issues a `ROUTE` +message. The coordinator runs the auth module, requires every returned role to +exist **in the Raft-replicated coordinator role set**, and requires the union of +their masks to grant at least `COORDINATOR_READ` — otherwise the routing table +request is denied. + +{

The driver authenticates against a data instance

} + +Using the routing table, the driver opens a connection to MAIN or a REPLICA with +the **same scheme and the same token**. The data instance runs its own auth +module and requires every returned role to exist **in its auth store**, with +whatever data privileges (`MATCH`, `CREATE`, …) the query needs. + + + +For that to work end to end: + +| Requirement | Why | +|---|---| +| The **same SSO scheme** is listed in `--auth-module-mappings` on coordinators *and* on every data instance | The driver sends one scheme to both; a scheme absent from a node's mappings is rejected there. | +| The **same role names** the IdP returns exist on coordinators *and* on data instances | Each side validates the returned roles against its own role store, and any missing role rejects the whole login on that side. | +| Those roles hold `COORDINATOR_READ` (or `COORDINATOR_WRITE`) **on the coordinators** | Otherwise the `ROUTE` message is denied and the driver never obtains a routing table. | +| Those roles hold the needed **data privileges on the data instances** | The coordinator privileges are not visible to, and mean nothing on, a data instance. | + +The privilege *values* are necessarily different on the two sides — coordinators +know only `COORDINATOR_READ` / `COORDINATOR_WRITE`, data instances know only the +data privilege set — but the **role names must match**. A typical setup mirrors +each IdP group into a role of the same name on both sides: + +```cypher +-- On any coordinator +CREATE ROLE analyst; +GRANT COORDINATOR_READ TO analyst; + +CREATE ROLE dba; +GRANT ALL PRIVILEGES TO dba; -- COORDINATOR_READ + COORDINATOR_WRITE +``` + +```cypher +-- On the MAIN data instance (replicated to REPLICAs) +CREATE ROLE analyst; +GRANT MATCH TO analyst; + +CREATE ROLE dba; +GRANT ALL PRIVILEGES TO dba; -- the full data-instance privilege set +``` + + +A role that exists on the data instances but not on the coordinators produces a +confusing failure mode: direct `bolt://` connections to MAIN work, but every +`neo4j://` routing connection fails at the coordinator. Check `SHOW ROLES` on a +coordinator and `SHOW ROLES` on MAIN and reconcile the two lists. + + +## Rolling SSO out + +The bootstrap order differs between coordinators and data instances, and getting +it wrong is the most common way to lock yourself out. + +### On coordinators — no restart needed + +Coordinators can be started with `--auth-module-mappings` from the very +beginning. Because basic auth stays open until a `COORDINATOR_WRITE` role +exists, you can create the roles over Bolt on the running cluster. + + + +{

Start the coordinators with the SSO mapping

} + +``` +--auth-module-mappings=oidc-entra-id +``` + +Set the `MEMGRAPH_SSO_*` environment variables the module needs, exactly as you +would on a data instance. + +{

Connect with basic auth

} + +No `COORDINATOR_WRITE` role exists yet, so username/password (or no credentials +at all) is still accepted with full access: + +``` +mgconsole --host --port 7687 +``` + +{

Create the roles and grant privileges

} + +```cypher +CREATE ROLE dba; +GRANT ALL PRIVILEGES TO dba; + +CREATE ROLE analyst; +GRANT COORDINATOR_READ TO analyst; +``` + +The role names must match what your IdP returns — see [role +mapping](/database-management/authentication-and-authorization/auth-system-integrations#role-mapping). + +{

SSO is now enforced

} + +As soon as `GRANT ALL PRIVILEGES TO dba` commits to Raft, all three deny +conditions hold and the **next** basic-auth login attempt is rejected. Existing +basic-auth sessions keep their full mask until they disconnect. Verify with: + +```cypher +SHOW ROLES; +SHOW PRIVILEGES FOR ROLE dba; +``` + + + +### On data instances — restart required + + +This only concerns data instances where SSO is **not** set up yet. If your data +instances already authenticate through SSO, nothing changes for them — enabling +SSO on the coordinators does not require any modification on the data-instance +side, beyond using the [same role names](#boltrouting-with-sso) on both sides. + + +Data instances have no equivalent escape hatch. Their SSO path validates the +IdP's roles against the auth store, and a role that does not exist means the +login fails. If you enable the module before the roles exist, SSO users cannot +log in and — when the mapped scheme is `basic` (LDAP) — username/password +authentication is delegated to the module too, so you may have no way in at all +to create them. + +The safe order is therefore: + + + +{

Start the data instance *without* `--auth-module-mappings`

} + +{

Log in and create the roles

} + +```cypher +CREATE ROLE dba; +GRANT ALL PRIVILEGES TO dba; + +CREATE ROLE analyst; +GRANT MATCH TO analyst; +``` + +{

Restart the data instance with `--auth-module-mappings` set

} + +SSO logins now find their roles and succeed. + + + + +Do the coordinator side and the data-instance side with the **same role names**, +or [Bolt+routing](#boltrouting-with-sso) connections will fail even though +direct connections to each node work. + + +## Rolling upgrades + +The new role and privilege queries are carried by new coordinator-to-coordinator +RPCs. There is no RPC version negotiation. During a rolling upgrade, if a +new follower forwards a role or privilege query to a leader that has not been +upgraded yet, the query **fails with an error** rather than crashing the +coordinator: + +``` +Query forwarded to the leader but it failed to process the request! +Check the logs on the leader to find out what happened. +``` + +Complete the upgrade of all coordinators before managing roles, or run the role +queries directly against an upgraded leader. + +The Raft cluster state itself is version-safe in both directions: an older +coordinator ignores the unknown roles key, and a newer coordinator reading an +older log or snapshot sees an empty role set. No log store version bump is +involved. + +## Timeouts + +Role and privilege queries — like all forwarded coordinator queries — run on the +caller's Bolt session thread, so they carry explicit RPC timeouts to prevent a +session blocking forever against a reachable-but-stuck leader. See [RPC +timeouts](/clustering/high-availability/how-high-availability-works#rpc-timeouts) +for the full table. The values relevant to authentication are: + +| RPC | Timeout | Used by | +|---|---|---| +| `GetRolesReq` | 10s | SSO login role validation, the per-query privilege re-check, `SHOW ROLES`, `SHOW CURRENT ROLE` | +| `GetRolePrivilegesReq` | 10s | `SHOW PRIVILEGES FOR ROLE` | +| `CreateRoleReq` / `DropRoleReq` | 10s | `CREATE ROLE`, `DROP ROLE` | +| `GrantPrivilegeReq` / `RevokePrivilegeReq` | 10s | `GRANT`, `REVOKE` | + +Each of these budgets covers a Raft commit, which is itself capped at 3 seconds, +plus headroom. Hitting the timeout surfaces as a query error; retry the +operation. + +Independently of the RPC timeout, the auth module subprocess is bounded by +[`--auth-module-timeout-ms`](/database-management/authentication-and-authorization/auth-system-integrations#configuration-flags), +the same flag used on data instances. A module that exceeds it fails the login. + +## Limitations + +- **No users on coordinators.** Only the role set and each role's privilege mask + are persisted. There is no way to create a coordinator-local account. +- **No SSO auto-provisioning of roles.** Unlike some data-instance setups, an + SSO identity does not implicitly create `readonly` / `readwrite` / `admin` + roles on a coordinator. Roles must be created explicitly with `CREATE ROLE`. +- **Only two privileges.** The fine-grained data-instance privilege set is not + modeled on coordinators. +- **Role names** must match the + [`--auth-user-or-role-name-regex`](/database-management/configuration) pattern, + the same as on data instances. + +## Troubleshooting + +| Symptom | Likely cause | +|---|---| +| Basic auth suddenly rejected after a `GRANT` | Expected — a `COORDINATOR_WRITE` role now exists, so SSO is enforced. Connect with an SSO scheme. | +| Basic auth unexpectedly accepted while SSO is configured | Either the license is invalid, or no role holds `COORDINATOR_WRITE`. Check `SHOW PRIVILEGES FOR ROLE` and the license status. | +| Every login rejected on one coordinator | That coordinator cannot reach a leader. Check `SHOW INSTANCES` from another coordinator and wait for the election to finish. | +| SSO login rejected with "role that doesn't exist" | `CREATE ROLE` on the coordinator, or fix the IdP group mapping. Check `SHOW ROLES`. | +| SSO login rejected with "carry no coordinator privilege" | `GRANT COORDINATOR_READ` or `GRANT COORDINATOR_WRITE` to the role. | +| `neo4j://` fails but `bolt://` to MAIN works | The role exists on the data instances but not on the coordinators, or it lacks `COORDINATOR_READ`. | +| A session was working and now every query is denied | Its roles were revoked or dropped — privileges are re-checked on every query. Reconnect after fixing the grant. | + + diff --git a/pages/clustering/high-availability/ha-commands-reference.mdx b/pages/clustering/high-availability/ha-commands-reference.mdx index 40afbf51d..434aae715 100644 --- a/pages/clustering/high-availability/ha-commands-reference.mdx +++ b/pages/clustering/high-availability/ha-commands-reference.mdx @@ -20,12 +20,32 @@ setup, the choice no longer matters. -All queries can be run on any coordinator. If currently the coordinator is not a -leader, the query will be automatically forwarded to the current leader and -executed there. This is because the Raft protocol specifies that only the -leader should accept changes in the cluster. The only exception is [`YIELD -LEADERSHIP`](#yield-leadership), which must be run directly on the current -leader. +All queries can be run on any coordinator. If the coordinator you are connected +to is not the leader, the query is automatically forwarded to the current leader +and executed there. This is because the Raft protocol specifies that only the +leader should accept changes in the cluster, and because only the leader holds an +up-to-date view of the cluster. + +This also holds for the read-only cluster queries — [`SHOW +INSTANCES`](#show-instances), [`SHOW COORDINATOR +SETTINGS`](#coordinator-runtime-settings) and [`SHOW REPLICATION +LAG`](#show-replication-lag) — which are always answered by the leader, and for +[`YIELD LEADERSHIP`](#yield-leadership). Followers never answer them from their +own local state, so you never see a stale or partial picture of the cluster. If +the leader cannot be reached, these queries return **no rows** together with a +warning notification instead of a degraded result — see [Error +handling](#error-handling). + + + +From Memgraph 3.13, coordinators can enforce privileges on these queries. Every +query on this page requires either `COORDINATOR_READ` (read-only introspection) +or `COORDINATOR_WRITE` (everything mutating). A session that connected with +basic auth carries full `COORDINATOR_WRITE`, so nothing changes unless you +enable [SSO on coordinators](/clustering/high-availability/coordinator-authentication). +See the [privilege +reference](/clustering/high-availability/coordinator-authentication#which-privilege-each-query-requires) +for the per-query mapping. ### `ADD COORDINATOR` @@ -88,8 +108,8 @@ REMOVE COORDINATOR coordinatorId; - Leader coordinator **cannot** remove itself. To remove the leader, first trigger a leadership change with [`YIELD - LEADERSHIP`](#yield-leadership) and then run `REMOVE COORDINATOR` against the - new leader. + LEADERSHIP`](#yield-leadership) and then run `REMOVE COORDINATOR` once a new + leader has been elected. {

Example

} @@ -175,6 +195,13 @@ REGISTER INSTANCE instanceName ( AS ASYNC | AS STRICT_SYNC ) ? WITH CONFIG { - In Kubernetes, use service DNS names (e.g. `memgraph-data-1.default.svc.cluster.local`). - Local development uses `localhost`. +- Register instances only while every database on the MAIN is in the + `IN_MEMORY_TRANSACTIONAL` storage mode. If the MAIN is in analytical mode, the + query still returns success (the Raft commit is the success criterion) but the + replica is not attached until the MAIN switches back, at which point the + reconciliation loop attaches it. The data instance logs the reason for the + rejection. See [Bulk import in analytical + mode](/clustering/high-availability/analytical-import). {

Example

} @@ -205,6 +232,16 @@ UNREGISTER INSTANCE instanceName; the replica from MAIN fails, the [reconciliation loop](/clustering/high-availability/how-high-availability-works#how-the-reconciliation-loop-works) automatically retries the operation. +- The unregistered instance keeps running and keeps all of its data. It is + removed from the cluster, not wiped, so you do not need to clear its data + directory before registering it back. +- The MAIN refuses the unregister RPC while any of its databases is in the + `IN_MEMORY_ANALYTICAL` storage mode, and logs the reason. The instance is + still removed from the Raft state, but the MAIN keeps its replication client + until the reconciliation loop retries once every database is transactional + again. Unregister instances only while the cluster is in transactional mode — + see [Bulk import in analytical + mode](/clustering/high-availability/analytical-import). {

Example

} @@ -296,16 +333,17 @@ informational notification that the request was submitted. {

Behavior

} -- Must be run **on the coordinator that is currently the leader**. Unlike most - other cluster management queries, this query is **not forwarded** to the - leader — running it on a follower fails with: - - > Only the current leader can yield the leadership! - +- Can be run on **any coordinator**. If the coordinator is a follower, the query + is forwarded to the current leader, which yields its leadership. You no longer + have to find the leader first. - Running it on a data instance fails with: > Only coordinator can run YIELD LEADERSHIP query. +- A coordinator that Raft already elected as leader but that has not yet + finished taking over the cluster still yields its leadership. This makes the + query usable as an escape hatch exactly when it is needed most — when a + freshly elected leader is stuck and you want another coordinator to take over. - The request is handed over to Raft and processed **asynchronously**. The query returns as soon as the request is submitted, not when the new leader is elected. @@ -317,22 +355,33 @@ informational notification that the request was submitted. checks toward all data instances. If no MAIN is found at that point, it performs a failover. +{

Failure modes

} + +| Error message | Meaning | +| ------------- | ------- | +| `Yielding leadership failed since the instance is not leader anymore!` | The request reached a coordinator that is no longer the leader — leadership changed in the meantime. Retry. | +| `Tried to forward the request to the current leader but the leader couldn't be found!` | There is currently no known leader to forward the request to (for example, an election is in progress). Retry once a leader is elected. | +| `Request forwarded to the leader but leader failed with request processing! Check logs on the leader to find out what happened!` | The leader was reached but failed to process the request. Inspect the leader's logs. | + {

Implications

} - This changes only the **coordinator** leadership. Data instances keep their MAIN and REPLICA roles — this is not a data failover, and client queries against MAIN and REPLICAs are unaffected. - During the short election window, cluster management queries (e.g. `SHOW - INSTANCES`, registration queries) may temporarily fail or report instances as - `down` because there is no leader to serve them. Retry once the new leader is - elected. + INSTANCES`, registration queries) may temporarily fail because there is no + leader to serve them. Read queries such as `SHOW INSTANCES` return no rows and + a warning notification rather than a stale picture of the cluster. Retry once + the new leader is elected. - At least one other healthy coordinator must be able to take over. In a single-coordinator cluster, or when the other coordinators are down, the same coordinator remains (or becomes again) the leader. - Failover of data instances is not triggered, but the new leader recomputes cluster state, so a cluster that was already missing a MAIN can fail over as part of the leadership change. -- No privilege is required to run this query on the coordinators. +- Requires `COORDINATOR_WRITE`. A basic-auth session carries it implicitly; an + [SSO session](/clustering/high-availability/coordinator-authentication) needs + a role that has been granted it. {

Typical use cases

} @@ -369,14 +418,42 @@ SHOW INSTANCES; {

Output includes

} 1. Network endpoints (bolt, coordinator, management) -2. Health state -3. Role: MAIN, REPLICA, LEADER, FOLLOWER, or UNKNOWN +2. Health state (`up` or `down`) +3. Role: MAIN, REPLICA, LEADER or FOLLOWER 4. Time since last health ping -{

Behavior on followers

} +A data instance that is currently `down` keeps the role recorded in the Raft log +(`main` or `replica`) instead of being reported with an unknown role, so you can +still tell which instance the cluster considers MAIN while it is unreachable. -1. Follower attempts to query the leader for accurate state. -2. If leader unavailable, follower reports all servers as `"down"`. +{

Behavior

} + +The query is **strongly consistent**: the result always comes from the leader +coordinator, which is the only coordinator with an up-to-date view of the +cluster. + +1. If you are connected to the leader, it answers directly. +2. If you are connected to a follower, the follower forwards the request to the + leader and returns the leader's result. +3. If the leader cannot be reached, the query returns an **empty result set** + together with a `LeaderNotReachable` warning notification: + + > Couldn't reach the leader coordinator, so the state of the cluster is + > unknown. Please retry the query. + + This happens when no leader is currently elected, when the connection to the + leader is broken, or when the coordinator you are connected to was just + elected leader but has not finished taking over the cluster yet. + + +**Behavior change in Memgraph 3.13:** previously, a follower that could not +reach the leader fell back to reporting the cluster from its own local Raft +state, with health reported as `unknown`. It now returns no rows and a warning +notification instead, so a partial or stale cluster picture can never be +mistaken for the real one. If you have tooling or health checks that parse +`SHOW INSTANCES`, treat an empty result as "cluster state unknown, retry" rather +than as "no instances registered". + ### `SHOW INSTANCE` @@ -406,12 +483,83 @@ Shows replication lag (in committed transactions) for all instances. SHOW REPLICATION LAG; ``` +{

Behavior

} + +The lag data is collected by the leader coordinator from the current MAIN, so the +query is answered by the leader — a follower forwards the request and returns the +leader's result. + +Whenever the lag cannot be determined, the query returns **no rows** and a +warning notification explaining why, so you know whether retrying will help: + +| Notification code | Message | Meaning | +| ----------------- | ------- | ------- | +| `LeaderNotReachable` | Couldn't reach the leader coordinator, so the replication lag is unknown. Please retry the query. | No leader could be contacted (for example, an election is in progress). | +| `ReplicationLagUnavailable` | The leader coordinator hasn't finished taking over the cluster, so the replication lag is unknown. Please retry the query. | A new leader was elected but has not finished reconciling the cluster. | +| `ReplicationLagUnavailable` | No instance is currently main, so there is no replication lag to report. | The cluster has no MAIN — promote one with `SET INSTANCE ... TO MAIN`, or wait for failover. | +| `ReplicationLagUnavailable` | The current main didn't respond, so the replication lag is unknown. Check whether the main is up. | The MAIN did not answer the leader's request. | +| `ReplicationLagUnavailable` | The instance the leader considers main reports that it is a replica, so the replication lag is unknown. Please retry the query once the cluster state is reconciled. | The leader's view is stale; the [reconciliation loop](/clustering/high-availability/how-high-availability-works#how-the-reconciliation-loop-works) will fix it. | + {

Implications

} - Lag values survive restarts (stored in snapshots + WAL). - Useful during manual failover to evaluate risk of data loss. +### `SHOW ROUTING TABLE` + +Shows the routing table that a coordinator hands out to +[`bolt+routing`](/clustering/high-availability/querying-the-cluster-in-high-availability) +clients for the default database. + +```cypher +SHOW ROUTING TABLE; +``` + +{

Output includes

} + +Each row contains a `role` and the list of Bolt `servers` serving that role: + +| `role` | `servers` | +| ------- | ------------------------------------------------------------------ | +| `WRITE` | Bolt endpoint of the current MAIN. | +| `READ` | Bolt endpoints of all REPLICAs, plus MAIN if [`enabled_reads_on_main`](#enabled_reads_on_main) is set to `true`. | +| `ROUTE` | Bolt endpoints of all coordinators. | + +Example output on a cluster with three coordinators, one MAIN and two REPLICAs: + +```plaintext ++---------+------------------------------------------------------------------+ +| role | servers | ++---------+------------------------------------------------------------------+ +| "WRITE" | ["localhost:7687"] | +| "READ" | ["localhost:7688", "localhost:7689"] | +| "ROUTE" | ["localhost:7690", "localhost:7691", "localhost:7692"] | ++---------+------------------------------------------------------------------+ +``` + +{

Behavior

} + +- The query can only be run on a coordinator. Running it on a data instance + fails with `Only coordinator can run SHOW ROUTING TABLE query.` +- The query is always answered from the leader's state, so every coordinator + returns the same routing table. If the leader cannot be contacted, an empty + routing table is returned. +- Roles with no servers are omitted. For example, if no data instance is + registered yet, only the `ROUTE` row is returned. +- The routing table is reported for the default database, which is the same + database `bolt+routing` clients are routed to. + +{

Implications

} + +- Useful for verifying which instance clients will send writes to, and which + instances they can read from, without inspecting driver internals. +- Because drivers cache the routing table for up to 5 minutes, the output of + this query can differ from what a connected client is currently using. See + [routing table TTL and refresh + behavior](/clustering/high-availability/querying-the-cluster-in-high-availability#routing-table-ttl-and-refresh-behavior). + + ## Coordinator runtime settings Coordinator runtime settings are Raft-replicated and can be changed on a live @@ -419,6 +567,14 @@ cluster without downtime. Use `SET COORDINATOR SETTING` to modify a value and `SHOW COORDINATOR SETTINGS` to inspect all current values. Changes propagate automatically to every coordinator in the cluster. +Both queries can be run on any coordinator and are served by the leader — a +follower forwards the request and returns the leader's answer. If the leader +cannot be reached, `SHOW COORDINATOR SETTINGS` returns **no rows** together with a +`LeaderNotReachable` warning notification: + +> Couldn't reach the leader coordinator, so the coordinator settings are unknown. +> Please retry the query. + ### `instance_health_check_frequency_sec` How often the coordinator pings data instances, in seconds. @@ -553,7 +709,11 @@ survives coordinator restarts and leader re-elections, and is honored across failovers: a newly promoted MAIN comes up read-only when the cluster is in read-only mode, instead of silently accepting writes. -No privilege is required to run this query on the coordinators. +`SET COORDINATOR SETTING` requires `COORDINATOR_WRITE` and `SHOW COORDINATOR +SETTINGS` requires `COORDINATOR_READ`. A basic-auth session carries both +implicitly; an [SSO +session](/clustering/high-availability/coordinator-authentication) needs a role +that has been granted them. Enabling read-only mode blocks **all write sources** on the MAIN — user Cypher @@ -575,13 +735,202 @@ cluster self-heals to the requested state. +## Coordinator role and privilege management + + +These queries are **Memgraph Enterprise** features and require a valid license. +They exist so that [SSO +identities](/clustering/high-availability/coordinator-authentication) have +something to map onto. Coordinators have no users — only roles. + + +Coordinator roles are stored in the **Raft-replicated cluster state**, not in +the auth store, so they survive restarts, follower catch-up and leader failover. +Like the cluster registration queries, all of these can be run on any +coordinator: writes are transparently forwarded to the leader and committed +through the Raft log, and `SHOW ROLES` / `SHOW PRIVILEGES FOR ROLE` are strong +reads served by the leader. + +### `CREATE ROLE` + +Creates a coordinator role. New roles start with **no** privileges. + +```cypher +CREATE ROLE ifNotExists? roleName; +``` + +{

Behavior & implications

} + +- Errors if the role already exists, unless `IF NOT EXISTS` is given. +- The role name must match the `--auth-user-or-role-name-regex` pattern, + otherwise the query fails with `Invalid role name ''.` +- Requires `COORDINATOR_WRITE`. + +{

Example

} + +```cypher +CREATE ROLE dba; +CREATE ROLE IF NOT EXISTS analyst; +``` + +### `DROP ROLE` + +Removes a coordinator role. + +```cypher +DROP ROLE roleName; +``` + +{

Behavior & implications

} + +- Errors with `Role '' doesn't exist.` if the role is not present. +- Takes effect on **already-connected sessions immediately** — privileges are + re-derived from the committed role set on every query, so a session that + authenticated with the dropped role is denied its next privileged query + without needing to reconnect. +- Requires `COORDINATOR_WRITE`. + +{

Example

} + +```cypher +DROP ROLE analyst; +``` + +### `SHOW ROLES` + +Lists the coordinator roles. Returns one `role` column, name only. + +```cypher +SHOW ROLES; +``` + +{

Behavior & implications

} + +- Strong read served by the leader. If no leader can be reached, the query fails + rather than returning possibly-stale local state. +- Requires `COORDINATOR_READ`. + +### `GRANT` / `REVOKE` coordinator privileges + +Grants or revokes a coordinator privilege on a role. Coordinators support +exactly two privileges: `COORDINATOR_READ` and `COORDINATOR_WRITE`, where +`COORDINATOR_WRITE` is a superset of `COORDINATOR_READ`. + +```cypher +GRANT ( ALL PRIVILEGES | COORDINATOR_READ | COORDINATOR_WRITE [, ...] ) TO ROLE? roleName; +REVOKE ( ALL PRIVILEGES | COORDINATOR_READ | COORDINATOR_WRITE [, ...] ) FROM ROLE? roleName; +``` + +{

Behavior & implications

} + +- `GRANT ALL PRIVILEGES` grants **both** coordinator privileges; + `REVOKE ALL PRIVILEGES` removes both. +- Errors with `Role '' doesn't exist.` if the role is not present. +- Only `COORDINATOR_READ` and `COORDINATOR_WRITE` may appear in the privilege + list. Any other privilege, `DENY` in any form, a `USER` target, fine-grained + access control (`ON NODES` / `ON EDGES`), property permissions and + `GRANT DATABASE` are all rejected on a coordinator. +- Like `DROP ROLE`, a `REVOKE` applies to already-connected sessions on their + next query. +- Requires `COORDINATOR_WRITE`. + +{

Example

} + +```cypher +GRANT ALL PRIVILEGES TO dba; +GRANT COORDINATOR_READ TO analyst; +REVOKE COORDINATOR_WRITE FROM analyst; +``` + +### `SHOW PRIVILEGES FOR ROLE` + +Reports the privileges granted to a coordinator role, one per row. + +```cypher +SHOW PRIVILEGES FOR ROLE? roleName; +``` + +{

Behavior & implications

} + +- A role with no grants returns no rows. +- The trailing `ON MAIN | CURRENT | DATABASE ` clause is **rejected** — + coordinators have no databases. +- `SHOW PRIVILEGES FOR USER ` is rejected — coordinators have no users. +- Strong read served by the leader. +- Requires `COORDINATOR_READ`. + +{

Example

} + +```cypher +SHOW PRIVILEGES FOR ROLE dba; +``` + +```plaintext ++---------------------+ +| privilege | ++---------------------+ +| COORDINATOR_READ | +| COORDINATOR_WRITE | ++---------------------+ +``` + +### `SHOW CURRENT USER` and `SHOW CURRENT ROLE` + +Report the identity of the current session. + +```cypher +SHOW CURRENT USER; +SHOW CURRENT ROLE; +``` + +{

Behavior & implications

} + +- **No privilege and no license are required** — these are self-service queries + that reveal only the session's own identity. +- `SHOW CURRENT USER` returns the principal the SSO module reported. It is + session-local and works even when the leader is unreachable. A basic-auth + passthrough session returns `null`. +- `SHOW CURRENT ROLE` returns the session's roles filtered against the leader's + committed role set, so a dropped role stops being reported. A basic-auth + passthrough session has no roles and returns `null`. + ## Error handling If a Raft log commit fails for any cluster operation (register, unregister, -promote, demote, add coordinator), the error message will indicate: +promote, demote, add coordinator, role or privilege change), the error message +will indicate: > Writing to Raft log failed. Please retry the operation. +### When there is no leader to serve the query + +Because every cluster query is served by the leader, queries fail (or return +nothing) while the cluster has no usable leader — most commonly during a leader +election, or right after one, while the new leader is still taking over the +cluster. + +State-changing queries (`ADD COORDINATOR`, `REMOVE COORDINATOR`, `UPDATE +CONFIG`, `REGISTER INSTANCE`, `UNREGISTER INSTANCE`, `SET INSTANCE ... TO MAIN`, +`DEMOTE INSTANCE`, `FORCE RESET CLUSTER STATE`) fail with an explicit error: + +> Couldn't <operation> since coordinator is not a leader! Try contacting +> other coordinators as there might be leader election happening or other +> coordinators are down. + +When the leader is known but the request still could not be executed there, the +message instead names the current leader's id and Bolt address so you can +connect to it directly. If the request could not be forwarded at all, the error +is: + +> Tried to forward the request to the current leader but the leader couldn't be +> found! + +Read queries (`SHOW INSTANCES`, `SHOW COORDINATOR SETTINGS`, `SHOW REPLICATION +LAG`) do not fail — they return an **empty result set** with a warning +notification (`LeaderNotReachable`, or `ReplicationLagUnavailable` for `SHOW +REPLICATION LAG`). In both cases the operation is safe to retry once a leader is +available. + ## Troubleshooting commands diff --git a/pages/clustering/high-availability/how-high-availability-works.mdx b/pages/clustering/high-availability/how-high-availability-works.mdx index 0773a4442..822a5539b 100644 --- a/pages/clustering/high-availability/how-high-availability-works.mdx +++ b/pages/clustering/high-availability/how-high-availability-works.mdx @@ -116,9 +116,9 @@ The COORDINATOR instance is a **very restricted instance**, and it will not respond to any queries that are not related to management of the cluster. That means, you cannot run any data queries on the coordinator directly (we will talk more about routing data queries in the next sections). However, -system information queries such as `SHOW CONFIG`, `SHOW LICENSE INFO`, -`SHOW BUILD INFO` and `SHOW STORAGE INFO` are supported on coordinators, as -well as `SET DATABASE SETTING`, `RELOAD BOLT_SERVER TLS` and +system information queries such as `SHOW CONFIG`, `SHOW VERSION`, +`SHOW LICENSE INFO`, `SHOW BUILD INFO` and `SHOW STORAGE INFO` are supported on +coordinators, as well as `SET DATABASE SETTING`, `RELOAD BOLT_SERVER TLS` and `RELOAD INTRA_CLUSTER TLS`. @@ -161,6 +161,8 @@ Below is a cleaned-up categorization. | RPC | Purpose | Description | | ------------------------ | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `ShowInstancesRpc` | Follower requests cluster state from leader. | Sent by a follower coordinator to the leader coordinator when a user executes `SHOW INSTANCES` through the follower. | +| `ShowCoordSettingsRpc` | Follower requests coordinator settings. | Sent by a follower coordinator to the leader coordinator when a user executes `SHOW COORDINATOR SETTINGS` through the follower. | +| `YieldLeadershipRpc` | Follower asks the leader to step down. | Sent by a follower coordinator to the leader coordinator when a user executes `YIELD LEADERSHIP` through the follower. | | `AddCoordinatorRpc` | Follower requests adding coordinator. | Sent by a follower coordinator to the leader coordinator when a user executes `ADD COORDINATOR` through the follower. | | `RemoveCoordinatorRpc` | Follower requests removing coordinator. | Sent by a follower coordinator to the leader coordinator when a user executes `REMOVE COORDINATOR` through the follower. | | `RegisterInstanceRpc` | Follower requests registering an instance. | Sent by a follower coordinator to the leader coordinator when a user executes `REGISTER INSTANCE` through the follower. | @@ -169,7 +171,7 @@ Below is a cleaned-up categorization. | `DemoteInstanceRpc` | Follower requests demoting an instance. | Sent by a follower coordinator to the leader coordinator when a user executes `DEMOTE INSTANCE` through the follower. | | `UpdateConfigRpc` | Follower requests updating config. | Sent by a follower coordinator to the leader coordinator when a user executes `UPDATE CONFIG` through the follower. | | `ForceResetRpc` | Follower requests resetting cluster state. | Sent by a follower coordinator to the leader coordinator when a user executes `FORCE RESET` through the follower. | -| `GetRoutingTableRpc` | Follower requests a routing table. | Sent by a follower coordinator to the leader coordinator when a user connects using `bolt+routing` through the follower. | +| `GetRoutingTableRpc` | Follower requests a routing table. | Sent by a follower coordinator to the leader coordinator when a user connects using `bolt+routing` or executes `SHOW ROUTING TABLE` through the follower. | | `CoordReplicationLagRpc` | Follower requests replication lag info. | Sent by a follower coordinator to the leader coordinator when a user executes `SHOW REPLICATION LAG` through the follower. | @@ -255,24 +257,109 @@ in the cluster to ensure high availability, with timeouts. | RPC message request | source | target | timeout | |--------------------------|-------------|----------------| -----------------| -| `ShowInstancesReq` | Coordinator | Coordinator | | -| `DemoteMainToReplicaReq` | Coordinator | Data instance | | -| `PromoteToMainReq` | Coordinator | Data instance | | -| `RegisterReplicaOnMainReq` | Coordinator | Data instance | | -| `UnregisterReplicaReq` | Coordinator | Data instance | | -| `EnableWritingOnMainReq` | Coordinator | Data instance | deprecated | -| `GetDatabaseHistoriesReq` | Coordinator | Data instance | | +| `ShowInstancesReq` | Coordinator | Coordinator | 10s | +| `DemoteMainToReplicaReq` | Coordinator | Data instance | 10s | +| `PromoteToMainReq` | Coordinator | Data instance | 10s | +| `RegisterReplicaOnMainReq` | Coordinator | Data instance | 10s | +| `UnregisterReplicaReq` | Coordinator | Data instance | 10s | +| `ReplicationLagReq` | Coordinator | Data instance | 5s | +| `GetDatabaseHistoriesReq` | Coordinator | Data instance | 10s | | `StateCheckReq` | Coordinator | Data instance | 5s | -| `SwapMainUUIDReq` | Coordinator | Data instance | | +| `SwapMainUUIDReq` | Coordinator | Data instance | 10s | +| `UpdateDataInstanceConfigReq` | Coordinator | Data instance | 10s | | `FrequentHeartbeatReq` | Main | Replica | 5s | -| `HeartbeatReq` | Main | Replica | | -| `SystemRecoveryReq` | Main | Replica | 5s | +| `HeartbeatReq` | Main | Replica | 10s | +| `SystemRecoveryReq` | Main | Replica | 30s | | `PrepareCommitRpc` | Main | Replica | proportional | | `FinalizeCommitReq` | Main | Replica | 10s | | `SnapshotRpc` | Main | Replica | proportional | | `WalFilesRpc` | Main | Replica | proportional | | `CurrentWalRpc` | Main | Replica | proportional | + +`EnableWritingOnMainReq` was **removed in Memgraph 3.13**. It was never sent — +writing on a newly promoted MAIN is enabled through the `writing_enabled` flag +carried inside `PromoteToMainRpc`. Its Prometheus counters were removed along +with it. + + +{
Follower-to-leader forwarding timeouts
} + +Cluster management queries can be run on any coordinator; a follower forwards +them to the leader over RPC. From Memgraph 3.13, each of these RPCs has an +explicit timeout. They run on the caller's **Bolt session thread**, so without +one, a session would block forever against a leader that is reachable but stuck. + +The four instance operations must outlast the work they trigger on the leader, +or a follower would report failure for an operation the leader has already +committed to Raft. Their budgets are the sum of that work plus headroom, where a +Raft commit is capped at 3 seconds and each RPC from the leader to a data +instance is capped by its entry in the table above. + +| RPC message request | source | target | timeout | Budget breakdown | +|--------------------------|-------------|-------------|---------|------------------| +| `RegisterInstanceReq` | Coordinator | Coordinator | 30s | Raft commit + demote the new replica + register it | +| `UnregisterInstanceReq` | Coordinator | Coordinator | 20s | Raft commit + one RPC to the current MAIN | +| `DemoteInstanceReq` | Coordinator | Coordinator | 20s | Raft commit + one RPC to the current MAIN | +| `SetInstanceToMainReq` | Coordinator | Coordinator | 60s | Raft commit + one `SwapMainUUID` per other instance + promote the new MAIN | +| `AddCoordinatorReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `RemoveCoordinatorReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `UpdateConfigReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `ForceResetReq` | Coordinator | Coordinator | 60s | Unbounded leader-side work — see the note below | +| `SetCoordinatorSettingReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `GetRoutingTableReq` | Coordinator | Coordinator | 10s | Read served by the leader | +| `CoordReplLagReq` | Coordinator | Coordinator | 10s | Read served by the leader | +| `CreateRoleReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `DropRoleReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `GrantPrivilegeReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `RevokePrivilegeReq` | Coordinator | Coordinator | 10s | Raft commit only | +| `GetRolesReq` | Coordinator | Coordinator | 10s | Read served by the leader | +| `GetRolePrivilegesReq` | Coordinator | Coordinator | 10s | Read served by the leader | + + +`SetInstanceToMainReq` sends one `SwapMainUUID` per other instance, so its cost +grows with the number of data instances. The 60s budget comfortably covers five +instances. Beyond that, a follower can time out before the leader answers — no +fixed value bounds it. Run `SET INSTANCE ... TO MAIN` directly on the leader in +very large clusters. + +`ForceResetReq` triggers a reconciliation that retries under a 1s–60s backoff +for as long as the coordinator stays leader, so the leader-side work has no +upper bound at all. The 60s budget only keeps a genuinely wedged leader from +blocking the session — hitting it does **not** mean the reset failed, and +[`FORCE RESET CLUSTER +STATE`](/clustering/high-availability/ha-commands-reference#force-reset-cluster-state) +is safe to re-run. + + +The role and privilege RPCs are used by [coordinator +authentication](/clustering/high-availability/coordinator-authentication): +`GetRolesReq` in particular is sent on **every query of an SSO session**, because +coordinator privileges are re-derived from the leader's committed role set +rather than cached at login. + +{
System transaction timeouts
} + +MAIN-to-REPLICA system-delta RPCs are sent while committing a system +transaction, so they must not block indefinitely either. From Memgraph 3.13 each +carries a **10 second** timeout; a timeout marks the REPLICA as `BEHIND` and +defers to system recovery. + +| RPC message request | source | target | timeout | +|---------------------|--------|---------|---------| +| `UpdateAuthDataReq` | Main | Replica | 10s | +| `DropAuthDataReq` | Main | Replica | 10s | +| `FinalizeSystemTxReq` | Main | Replica | 10s | +| `CreateDatabaseReq` | Main | Replica | 10s | +| `DropDatabaseReq` | Main | Replica | 10s | +| `SuspendDatabaseReq` | Main | Replica | 10s | +| `ResumeDatabaseReq` | Main | Replica | 10s | +| `RenameDatabaseReq` | Main | Replica | 10s | +| `TenantProfileReq` | Main | Replica | 10s | +| `SetParameterReq` | Main | Replica | 10s | +| `UnsetParameterReq` | Main | Replica | 10s | +| `DeleteAllParametersReq`| Main | Replica | 10s | + ## Intra-cluster TLS By default, the communication between instances in a high-availability cluster @@ -510,7 +597,7 @@ it undergoes a controlled recovery process: is reused on subsequent recoveries, meaning **only one backup copy is kept at a time**. -Use the `--storage-enable-backup-dir` flag to control this behavior: +Use the `--storage-backup-dir-enabled` flag to control this behavior: - `true` (default) - Old durability files are moved to `.old` directories - `false` - Old durability files are deleted immediately @@ -546,21 +633,43 @@ modes](/clustering/replication/how-replication-works#replication-modes). ## Actions on follower coordinators -Follower coordinators operate in a restricted mode. -They can **only execute** the `SHOW INSTANCES` command. - -All state-changing operations are disabled on followers, including: - -- Registering data instances -- Unregistering data instances -- Demoting an instance -- Promoting an instance to MAIN -- Forcing a cluster state reset -- Yielding leadership - -These operations are permitted **only on the leader coordinator**. Note that -[`YIELD LEADERSHIP`](/clustering/high-availability/ha-commands-reference#yield-leadership) -is not forwarded to the leader — it fails on a follower instead. +Follower coordinators never execute cluster operations themselves. Instead, they +act as a transparent entry point: every cluster query you run on a follower is +**forwarded to the current leader**, executed there, and the leader's answer is +returned to you. This holds both for state-changing operations (registering and +unregistering data instances, promoting and demoting instances, adding and +removing coordinators, updating configuration, forcing a cluster state reset, +[yielding +leadership](/clustering/high-availability/ha-commands-reference#yield-leadership)) +and for read-only ones (`SHOW INSTANCES`, `SHOW COORDINATOR SETTINGS`, `SHOW +REPLICATION LAG`, and `bolt+routing` routing table requests). + +As a result you can point your tooling at any coordinator without first having to +discover which one is the leader. + +### Why followers never answer from local state + +A follower's own Raft state is not enough to describe the cluster: health of the +data instances is only known to the leader, which is the coordinator that pings +them. For this reason followers do not fall back to a local, partial answer when +the leader cannot be reached. Instead: + +- Read queries return an **empty result set** together with a warning + notification (`LeaderNotReachable`, or `ReplicationLagUnavailable` for `SHOW + REPLICATION LAG`). +- State-changing queries fail with an error telling you that the coordinator is + not the leader, or that the leader could not be found. + +Both cases mean "cluster state unknown — retry", and both are expected during the +brief window of a leader election or while a newly elected leader is still taking +over the cluster. See [Error +handling](/clustering/high-availability/ha-commands-reference#when-there-is-no-leader-to-serve-the-query) +for the exact messages. + +The one exception is the `bolt+routing` routing table: a coordinator that Raft +elected as leader answers routing requests from its own Raft state even before it +has finished taking over the cluster, so that clients can keep routing queries +during the leadership transition. ## Raft-first operations and the reconciliation loop diff --git a/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx b/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx index 74eaa3b51..7a91fb7ab 100644 --- a/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx +++ b/pages/clustering/high-availability/querying-the-cluster-in-high-availability.mdx @@ -64,6 +64,8 @@ The routing protocol works as follows: receives the latest routing table. - If it contacts a **follower**, the follower forwards the request to the leader and returns the leader’s result. +- If the leader cannot be reached at all, an **empty routing table** is returned + and the driver retries against another coordinator. Because leader state is synchronized via Raft, routing information is always accurate. @@ -75,6 +77,22 @@ This ensures: - **Transparency:** Clients work seamlessly whether they connect to leaders or followers. +### Inspecting the routing table + +The routing table a coordinator hands out can be inspected manually with the +[`SHOW ROUTING TABLE`](/clustering/high-availability/ha-commands-reference#show-routing-table) +query: + +```cypher +SHOW ROUTING TABLE; +``` + +It returns one row per role (`WRITE`, `READ`, `ROUTE`) with the Bolt endpoints +serving that role, for the default database. The query can only be run on a +coordinator, and it is always answered from the leader's state, so every +coordinator returns the same result. If the leader cannot be contacted, an empty +routing table is returned. + ### Routing table TTL and refresh behavior Because routing is entirely client-side, the driver caches the routing table and @@ -172,21 +190,51 @@ When using the **bolt+routing protocol**, provide credentials for users that exist on the data instances. The authentication flow works as follows: 1. The client authenticates and connects to a **coordinator**. -2. The coordinator returns a routing table - no authentication occurs here. +2. The coordinator returns a routing table. 3. The client connects to the appropriate **data instance** using the **same credentials**. 4. The data instance performs authentication and executes the query. -This preserves a clean separation: coordinators route traffic, while data -instances manage users. - -1. You may connect to a coordinator via plain Bolt **without authentication**. -2. When using Bolt+routing, you **must** provide credentials - authentication is +1. By default, you may connect to a coordinator via plain Bolt **without + authentication** — username and password are accepted as a passthrough and + ignored. +2. When using Bolt+routing, you **must** provide credentials — authentication is performed on the data instances. +### Authenticating against coordinators + +From Memgraph 3.13, coordinators are no longer unconditionally open. They can +enforce [single sign-on and coordinator +privileges](/clustering/high-availability/coordinator-authentication), which +changes the flow above in two ways: + +- **Step 1 becomes a real authentication.** Once SSO is configured on the + coordinators, basic auth is refused and the client must present a valid IdP + token for a scheme listed in `--auth-module-mappings`. +- **Step 2 requires a privilege.** Serving the routing table requires + `COORDINATOR_READ`. A session whose roles carry no coordinator privilege is + denied the routing table with `You don't have permission to read the routing + table on the coordinator!` — reported as a non-retryable client error so + drivers do not retry it. + + + +Because the driver reuses **one set of credentials for both legs** of a routing +connection, a `neo4j://` session works only when the **same role names exist on +both the coordinators and the data instances**, and the same SSO scheme is +configured on both. The privileges attached to those roles differ by design — +coordinators know only `COORDINATOR_READ` / `COORDINATOR_WRITE`, data instances +know only the data privilege set — but the names must match. + +See [Bolt+routing with +SSO](/clustering/high-availability/coordinator-authentication#boltrouting-with-sso) +for the full setup. + + + diff --git a/pages/clustering/high-availability/setup-ha-cluster-docker.mdx b/pages/clustering/high-availability/setup-ha-cluster-docker.mdx index 0394b8239..5c0c83e2e 100644 --- a/pages/clustering/high-availability/setup-ha-cluster-docker.mdx +++ b/pages/clustering/high-availability/setup-ha-cluster-docker.mdx @@ -202,7 +202,8 @@ SET INSTANCE instance_3 TO MAIN; ## Check cluster state -Connect to the leader coordinator and check cluster state with `SHOW INSTANCES`; +Connect to any coordinator and check cluster state with `SHOW INSTANCES`. The +query is always answered by the leader, so followers report the same state: | name | bolt_server | coordinator_server | management_server | health | role | last_succ_resp_ms | | ------------- | -------------- | ------------------ | ----------------- | ------ | -------- | ---------------- | diff --git a/pages/clustering/replication/best-practices.mdx b/pages/clustering/replication/best-practices.mdx index 5e4f307bc..cd671628e 100644 --- a/pages/clustering/replication/best-practices.mdx +++ b/pages/clustering/replication/best-practices.mdx @@ -53,12 +53,25 @@ Invalid: Replication works **only** in the [in-memory transactional storage mode](/fundamentals/storage-memory-usage#in-memory-transactional-storage-mode-default). - -If you imported data using **in-memory analytical mode**, you must: - -1. Import the data -2. Switch the instance to **in-memory transactional mode** -3. Then configure replication +Writes performed in the [in-memory analytical storage +mode](/fundamentals/storage-memory-usage#in-memory-analytical-storage-mode) +never reach the WAL, so they cannot be replicated. + +If you want to import data using **in-memory analytical mode**, you must: + +1. `DROP REPLICA` every registered replica +2. Switch the MAIN to **in-memory analytical mode** +3. Import the data +4. Switch the MAIN back to **in-memory transactional mode** +5. `REGISTER REPLICA` the replicas again + +Step 4 writes a snapshot of the imported data and archives the superseded +durability files, so there is no need to run `CREATE SNAPSHOT` manually — and no +need to wipe a replica's data directory before registering it back, because +recovery detects that the imported data exists in no WAL file and ships the +snapshot instead. The same workflow for a high availability cluster is described +in [Bulk import in analytical +mode](/clustering/high-availability/analytical-import). ## Hardware requirements diff --git a/pages/clustering/replication/replication-commands-reference.mdx b/pages/clustering/replication/replication-commands-reference.mdx index 53ff3616e..e6e71297d 100644 --- a/pages/clustering/replication/replication-commands-reference.mdx +++ b/pages/clustering/replication/replication-commands-reference.mdx @@ -94,6 +94,18 @@ It should give you enough information to decide on which instance you can perfor ## Replica registration commands + + +`REGISTER REPLICA` and `DROP REPLICA` are refused while any database on the +instance is in the [`IN_MEMORY_ANALYTICAL` storage +mode](/fundamentals/storage-memory-usage#in-memory-analytical-storage-mode), +because analytical writes are not replicated. Switch every database back to +`IN_MEMORY_TRANSACTIONAL` and retry. For the analytical bulk import workflow, +see [storage mode +requirements](/clustering/replication/best-practices#storage-mode-requirements). + + + ### REGISTER REPLICA (SYNC) Registers a REPLICA instance with synchronous replication mode. diff --git a/pages/custom-query-modules.mdx b/pages/custom-query-modules.mdx index 2812a598c..8ac636c1a 100644 --- a/pages/custom-query-modules.mdx +++ b/pages/custom-query-modules.mdx @@ -89,14 +89,14 @@ Then, select a language you want to develop in. {

Develop a query

} - When developing with Mage, take a look at the basis of developed [algorithms and utility procedures](https://github.com/memgraph/memgraph/tree/master/mage) and extend based on that. + When developing with Mage, take a look at the basis of developed [algorithms and utility procedures](https://github.com/memgraph/memgraph/tree/master/src/mage) and extend based on that. If you are not extending the existing query modules, you can start from scratch by following these guides or using the API docs and knowledge to develop your query modules. - [Python](/custom-query-modules/python), - [C](/custom-query-modules/c), - [C++](/custom-query-modules/cpp), - - [Rust](https://github.com/memgraph/memgraph/tree/master/mage/rust/rsmgp-example). + - [Rust](https://github.com/memgraph/memgraph/tree/master/src/mage/rust/rsmgp-example). {

Start the MAGE container

} @@ -210,14 +210,14 @@ Then, select a language you want to develop in. {

Develop modules

} - When developing with Mage, take a look at the basis of developed [algorithms and utility procedures](https://github.com/memgraph/memgraph/tree/master/mage) and extend based on that. + When developing with Mage, take a look at the basis of developed [algorithms and utility procedures](https://github.com/memgraph/memgraph/tree/master/src/mage) and extend based on that. If you are not extending the existing query modules, you can start from scratch by following these guides or using the API docs and knowledge to develop your query modules. - [Python](/custom-query-modules/python), - [C](/custom-query-modules/c), - [C++](/custom-query-modules/cpp), - - [Rust](https://github.com/memgraph/memgraph/tree/master/mage/rust/rsmgp-example). + - [Rust](https://github.com/memgraph/memgraph/tree/master/src/mage/rust/rsmgp-example). {

Create the `dev` image

} @@ -279,106 +279,54 @@ Then, select a language you want to develop in. - {

Install dependencies

} - - To build from source, you will need: - - Python3 - - Make - - CMake - - Clang - - UUID - - [Rust and Cargo](https://doc.rust-lang.org/cargo/getting-started/installation.html) - - {

Set up the machine

} - - Run the following commands: - - ```bash - PY_VERSION="$(python3 --version | cut -d' ' -f2 | cut -d'.' -f1,2)" - sudo apt update - sudo apt install -y \ - libcurl4 \ - libpython${PY_VERSION} \ - libssl-dev \ - openssl \ - build-essential \ - cmake \ - curl \ - g++ \ - python3 \ - python3-pip \ - python3-setuptools \ - python3-dev \ - clang \ - git \ - libboost-dev \ - ninja-build \ - --no-install-recommends - ``` - - {

Download the Memgraph source code

} - - Clone the [Memgraph source code](https://github.com/memgraph/memgraph) from GitHub: + {

Set up the build environment

} - ``` - git clone https://github.com/memgraph/memgraph.git && cd memgraph/mage - ``` + Follow the [Build from source + guide](/advanced-algorithms/install-mage#build-from-source-linux) to clone + the Memgraph repository, install the build dependencies, set up the + toolchain and install the Rust and Python dependencies. - {

Run the `setup` script

} + {

Build MAGE

} - Run the following command: + Run the following commands from the root of the repository: ```shell - python3 setup build -p /usr/lib/memgraph/query_modules + source /opt/toolchain-v7/activate + ./build.sh --mage only ``` - The script will generate a `dist` directory with all the necessary files. - - It will also copy the contents of the newly created `dist` directory to - `/usr/lib/memgraph/query_modules`. Memgraph loads algorithms and modules from - this directory. - - If something isn't installed properly, the `setup` script will stop the - installation process. If you have any questions, contact us on - **[Discord](https://discord.gg/memgraph).** - - {
Set a different `query_modules` directory
} + The built query modules land in `build/mage/dist`. - The `setup` script can set your local `mage/dist` directory or any other - directory as the default one by changing the value of the - `--query-modules-directory` flag in the `/etc/memgraph/memgraph.conf`, - Memgraph's configuration file. + {

Install the query modules

} - By setting the `/mage/dist` as the default directory you don't need to copy - `*.so` and `*.py` files from the `mage/dist` directory - to`/usr/lib/memgraph/query_modules` every time you run `build`: + Install the modules to `/usr/lib/memgraph/query_modules`, the directory + Memgraph loads query modules from: - ``` - python3 setup modules_storage + ```shell + sudo cmake --install build --component mage --prefix /usr ``` - By setting `` as the default one, Memgraph will be looking for - query modules inside ``, instead of `/usr/lib/memgraph/query_modules`: + - ``` - python3 setup modules_storage -p - ``` + While developing, you can skip the install step on every rebuild by + pointing Memgraph directly at the build output instead: set + `--query-modules-directory=/build/mage/dist` in + `/etc/memgraph/memgraph.conf` (or on the command line). - Don't forget to copy the aforementioned files from `mage/dist` to - ``. + {

Develop modules

} - When developing with Mage, take a look at the basis of developed [algorithms and utility procedures](https://github.com/memgraph/memgraph/tree/master/mage) and extend based on that. + When developing with Mage, take a look at the basis of developed [algorithms and utility procedures](https://github.com/memgraph/memgraph/tree/master/src/mage) and extend based on that. If you are not extending the existing query modules, you can start from scratch by following these guides or using the API docs and knowledge to develop your query modules. - [Python](/custom-query-modules/python), - [C](/custom-query-modules/c), - [C++](/custom-query-modules/cpp), - - [Rust](https://github.com/memgraph/memgraph/tree/master/mage/rust/rsmgp-example). + - [Rust](https://github.com/memgraph/memgraph/tree/master/src/mage/rust/rsmgp-example). - {

Start Memgraph

} + {

Rebuild and load your changes

} Make sure your Memgraph instance is running: @@ -386,12 +334,18 @@ Then, select a language you want to develop in. sudo systemctl status memgraph.service ``` - {

Copy the query module

} - - Copy your developed query module to `/usr/lib/memgraph/query_modules` or your directory if you changed the default location of query modules: + After changing a module, rebuild and reinstall (the install step isn't + needed if you pointed `--query-modules-directory` at `build/mage/dist`): ```shell - python3 setup build -p /usr/lib/memgraph/query_modules + ./build.sh --mage only --dev + sudo cmake --install build --component mage --prefix /usr + ``` + + Then reload the query modules in a running instance: + + ```cypher + CALL mg.load_all(); ``` diff --git a/pages/custom-query-modules/contributing.mdx b/pages/custom-query-modules/contributing.mdx index 9e8d4fedb..0561f6705 100644 --- a/pages/custom-query-modules/contributing.mdx +++ b/pages/custom-query-modules/contributing.mdx @@ -18,7 +18,7 @@ Here are links to Memgraph and MAGE, which are both opened and ready to receive and your contribution: - [**Memgraph**](https://github.com/memgraph/memgraph) -- [**MAGE**](https://github.com/memgraph/memgraph/tree/master/mage) (NOTE: MAGE now lives in the Memgraph repository) +- [**MAGE**](https://github.com/memgraph/memgraph/tree/master/src/mage) (NOTE: MAGE now lives in the Memgraph repository) Feel free to create an issue or open a pull request on our Github repo to speed up the development. diff --git a/pages/custom-query-modules/cpp/cpp-example.mdx b/pages/custom-query-modules/cpp/cpp-example.mdx index 8604315c4..a3bed2132 100644 --- a/pages/custom-query-modules/cpp/cpp-example.mdx +++ b/pages/custom-query-modules/cpp/cpp-example.mdx @@ -1101,7 +1101,7 @@ cpp To make sure the module is linked with the rest of MAGE code, we need to add a `CMakeLists.txt` script in the new directory and register our module in the `cpp/CMakelists.txt` script as well. Refer to the existing scripts in MAGE’s -[query modules](https://github.com/memgraph/memgraph/tree/master/mage/cpp). +[query modules](https://github.com/memgraph/memgraph/tree/master/src/mage/cpp). diff --git a/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx b/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx index db2a1eb4c..252430ec0 100644 --- a/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx +++ b/pages/database-management/authentication-and-authorization/auth-system-integrations.mdx @@ -154,6 +154,29 @@ This approach ensures that all roles are created before the external authentication module is activated, allowing users to log in seamlessly across all supported authentication methods. +This workflow only applies to standalone Memgraph instances. On [data instances +running in HA +mode](/clustering/high-availability/how-high-availability-works#data-instance-implementation) +(i.e. when `--management-port` is set), `--init-file` is **not supported** and +the instance will fail to start if the flag is provided. There you must use the +manual procedure instead: start the data instance **without** +`--auth-module-mappings`, log in, create the roles and grant their privileges, +then restart with the flag set. + + +From Memgraph 3.13, [high availability +coordinators](/clustering/high-availability/coordinator-authentication) also +support SSO, using these same modules, schemes and `MEMGRAPH_SSO_*` environment +variables. Coordinators **do not** need this stop-and-restart cycle: basic +authentication stays open until a role holding `COORDINATOR_WRITE` exists, so +you can start them with `--auth-module-mappings` from the beginning and create +the roles over Bolt on the running cluster. + +Note that a `neo4j://` routing connection authenticates against a coordinator +*and* a data instance with the same credentials, so the **same role names must +exist on both**. + + ## Auth module architecture ### Communication protocol diff --git a/pages/database-management/authentication-and-authorization/query-privileges.mdx b/pages/database-management/authentication-and-authorization/query-privileges.mdx index 7fb7ab614..b2fcdba03 100644 --- a/pages/database-management/authentication-and-authorization/query-privileges.mdx +++ b/pages/database-management/authentication-and-authorization/query-privileges.mdx @@ -143,9 +143,10 @@ Memgraph's privilege system controls access to various database operations throu | `SHOW SNAPSHOTS` | `DURABILITY` | `SHOW SNAPSHOTS` | | `SHOW NEXT SNAPSHOT` | `DURABILITY` | `SHOW NEXT SNAPSHOT` | | `SET SETTING` | `CONFIG` | `SET SETTING ...` | -| `SHOW VERSION` | `STATS` | `SHOW VERSION` | +| `SHOW VERSION` | `STATS` | `SHOW VERSION` — on coordinators it requires `COORDINATOR_READ` instead, see [Coordinator operations](#coordinator-operations). | | `SHOW TRANSACTIONS` | `TRANSACTION_MANAGEMENT` | `SHOW TRANSACTIONS` | | `TERMINATE TRANSACTIONS` | `TRANSACTION_MANAGEMENT` | `TERMINATE TRANSACTIONS 'transaction_id'` | +| `TERMINATE TRANSACTIONS "*"` | `TRANSACTION_MANAGEMENT` | `TERMINATE TRANSACTIONS "*"` terminates every transaction the user may terminate. Without the privilege, only the user's own transactions are terminated. | | `RELOAD BOLT_SERVER TLS` | `RELOAD_TLS` | `RELOAD BOLT_SERVER TLS` | | `RELOAD INTRA_CLUSTER TLS` | `RELOAD_TLS` | `RELOAD INTRA_CLUSTER TLS` | @@ -156,7 +157,7 @@ Memgraph's privilege system controls access to various database operations throu | `REPLICATION` operations | `REPLICATION` | Various replication commands. | | `SHOW REPLICATION ROLE` | `REPLICATION` | `SHOW REPLICATION ROLE` | | `SHOW REPLICAS` | `REPLICATION` | `SHOW REPLICAS` | -| `SHOW REPLICATION LAG` | `COORDINATOR` | `SHOW REPLICATION LAG` | +| `SHOW REPLICATION LAG` | `COORDINATOR_READ` | Coordinators only — see [Coordinator operations](#coordinator-operations). | ## Multi-database operations @@ -191,10 +192,39 @@ Memgraph's privilege system controls access to various database operations throu ## Coordinator operations +These privileges apply **only on [high availability +coordinators](/clustering/high-availability/coordinator-authentication)**. They +are granted to roles that live in the coordinators' Raft-replicated cluster +state, not in the data-instance auth store, and are not part of the +data-instance privilege set — `GRANT ALL PRIVILEGES` on a data instance does not +grant them. + +`COORDINATOR_WRITE` is a superset of `COORDINATOR_READ`: a `COORDINATOR_WRITE` +grant satisfies a `COORDINATOR_READ` requirement. A session that connected to a +coordinator with basic auth carries both implicitly. + | Query Type | Required Privileges | Example | |------------|-------------------|---------| +| Routing table (Bolt `ROUTE` message) | `COORDINATOR_READ` | Any `neo4j://` connection. | +| `SHOW INSTANCE` / `SHOW INSTANCES` | `COORDINATOR_READ` | `SHOW INSTANCES` | +| `SHOW COORDINATOR SETTINGS` | `COORDINATOR_READ` | `SHOW COORDINATOR SETTINGS` | +| `SHOW REPLICATION LAG` | `COORDINATOR_READ` | `SHOW REPLICATION LAG` | +| `SHOW ROLES` / `SHOW PRIVILEGES FOR ROLE` | `COORDINATOR_READ` | `SHOW PRIVILEGES FOR ROLE dba` | +| `SHOW CONFIG`, `SHOW SETTING`, system info queries | `COORDINATOR_READ` | `SHOW CONFIG` | +| `SHOW VERSION` | `COORDINATOR_READ` | `SHOW VERSION` | +| `SHOW CURRENT USER` / `SHOW CURRENT ROLE` | **None** | Self-service identity queries. | +| Cluster management operations | `COORDINATOR_WRITE` | `REGISTER INSTANCE`, `SET INSTANCE ... TO MAIN`, `ADD COORDINATOR`, `YIELD LEADERSHIP`, `FORCE RESET CLUSTER STATE`, … | +| `SET COORDINATOR SETTING` / `SET SETTING` | `COORDINATOR_WRITE` | `SET COORDINATOR SETTING 'sync_failover_only' TO 'true'` | +| `CREATE ROLE` / `DROP ROLE` / `GRANT` / `REVOKE` | `COORDINATOR_WRITE` | `GRANT COORDINATOR_READ TO analyst` | | `COORDINATOR` operations | `COORDINATOR` | Various coordinator commands. | | `SHOW COORDINATOR SETTINGS` | `COORDINATOR` | `SHOW COORDINATOR SETTINGS` | +| `SHOW ROUTING TABLE` | `COORDINATOR` | `SHOW ROUTING TABLE` | + + +**Breaking change in Memgraph 3.13:** the `COORDINATOR` privilege was removed +and replaced by `COORDINATOR_READ` and `COORDINATOR_WRITE`. It never gated any +operation and is no longer accepted in `GRANT` / `DENY` / `REVOKE` statements. + ## Schema information diff --git a/pages/database-management/authentication-and-authorization/role-based-access-control.mdx b/pages/database-management/authentication-and-authorization/role-based-access-control.mdx index cc2259042..d5ddaf21a 100644 --- a/pages/database-management/authentication-and-authorization/role-based-access-control.mdx +++ b/pages/database-management/authentication-and-authorization/role-based-access-control.mdx @@ -243,7 +243,6 @@ of the following commands: | Privilege to change [storage mode](/fundamentals/storage-memory-usage#storage-modes). | `STORAGE_MODE` | | Privilege to manage [multi-tenant databases](/database-management/multi-tenancy). | `MULTI_DATABASE_EDIT` | | Privilege to use a database within the multi-tenant architecture. | `MULTI_DATABASE_USE` | -| Privilege to configure [high-availability](/clustering/high-availability) coordinators. | `COORDINATOR` | | Privilege to [impersonate other users](/database-management/authentication-and-authorization/impersonate-user). | `IMPERSONATE_USER` | | Privilege to use [parallel execution](/querying/parallel-execution). | `PARALLEL_EXECUTION` | | Privilege to set limits and monitor resource usage per user (via [user profiles](/database-management/authentication-and-authorization/user-profiles)) or per database (via [tenant profiles](/database-management/tenant-profiles)). | `PROFILE_RESTRICTION` | @@ -257,6 +256,22 @@ of the following commands: For a comprehensive reference of which privileges are required for specific queries and operations, see the [Query privileges reference](/database-management/authentication-and-authorization/query-privileges) documentation. + +**Breaking change in Memgraph 3.13:** the `COORDINATOR` privilege was removed — +it never gated any operation. It is no longer accepted in `GRANT` / `DENY` / +`REVOKE` statements and is no longer reported by `SHOW PRIVILEGES`. A stale +`COORDINATOR` grant in an existing deployment is never reported and never +checked, so no migration is needed. + +Access to [high availability](/clustering/high-availability) coordinators is now +controlled by the `COORDINATOR_READ` and `COORDINATOR_WRITE` privileges. These +are **coordinator-only**: they are granted to roles that live on the +coordinators themselves, they are not part of the data-instance privilege set, +and `GRANT ALL PRIVILEGES` on a data instance does not grant them. See +[coordinator +authentication](/clustering/high-availability/coordinator-authentication). + + ### First user privileges When you create the first user in Memgraph, that user automatically becomes a diff --git a/pages/database-management/backup-and-restore.mdx b/pages/database-management/backup-and-restore.mdx index 07114a5f0..1787e3cc3 100644 --- a/pages/database-management/backup-and-restore.mdx +++ b/pages/database-management/backup-and-restore.mdx @@ -357,7 +357,7 @@ his protects your data if the newly loaded snapshot turns out to be corrupted. Configuration: -Use the `--storage-enable-backup-dir` flag to control this behavior: +Use the `--storage-backup-dir-enabled` flag to control this behavior: - `true` (default) - Old durability files are moved to `.old` directories - `false` - Old durability files are deleted immediately diff --git a/pages/database-management/configuration.mdx b/pages/database-management/configuration.mdx index 8166eeb3d..0dcb3b760 100644 --- a/pages/database-management/configuration.mdx +++ b/pages/database-management/configuration.mdx @@ -486,10 +486,11 @@ in Memgraph. | `--storage-snapshot-retention-count=3` | The number of snapshots that should always be kept. | `[uint64]` | | `--storage-parallel-snapshot-creation=false` | Controls whether the snapshot creation can be done in a multi-threaded fashion. | `[bool]` | | `--storage-snapshot-thread-count` | The number of threads used to create snapshots. Defaults to using system's maximum thread count. | `[uint64]` | +| `--storage-snapshot-writeback-window-mib` | How large a chunk of a snapshot is written to disk at a time, in MiB. Defaults to 32. Set to 0 to disable. See [limiting the impact of snapshots on queries](/fundamentals/data-durability#limiting-the-impact-of-snapshots-on-queries). | `[uint64]` | | `--storage-wal-enabled=true` | Controls whether the storage uses write-ahead-logging. To enable WAL, periodic snapshots must be enabled. | `[bool]` | | `--storage-wal-file-flush-every-n-tx=100000` | Issue a 'fsync' call after this amount of transactions are written to the WAL file. Set to 1 for fully synchronous operation. | `[uint64]` | | `--storage-wal-file-size-kib=20480` | Minimum file size of each WAL file. | `[uint64]` | -| `--storage-mode=IN_MEMORY_TRANSACTIONAL` | The storage mode Memgraph will run on startup. Can be IN_MEMORY_TRANSACTIONAL, IN_MEMORY_ANALYTICAL or ON_DISK_TRANSACTIONAL. | `[string]` | +| `--storage-mode=IN_MEMORY_TRANSACTIONAL` | The storage mode Memgraph will run on startup. Can be IN_MEMORY_TRANSACTIONAL, IN_MEMORY_ANALYTICAL or ON_DISK_TRANSACTIONAL. A data instance in a high availability cluster cannot start in IN_MEMORY_ANALYTICAL; it can only [switch to it at runtime](/clustering/high-availability/analytical-import). | `[string]` | | `--storage-enable-schema-metadata=false` | Facilitates the utilization of a specialized cache designed to store specific metadata related to the database. | `[bool]` | | `--storage-enable-edges-metadata=false` | Utilizes additional memory to store metadata related to edges. This metadata is used to speed up id based lookups on edges. | `[bool]` | | `--storage-light-edge=false` | Stores edges as lightweight objects to reduce memory footprint (saves ~24B per edge by removing the dedicated edge container). Implies `--storage-properties-on-edges=true`. Direct edge lookups by ID become slower; pair with `--storage-enable-edges-metadata=true` for such workloads. In-memory storage modes only. | `[bool]` | @@ -499,7 +500,7 @@ in Memgraph. | `--storage-property-store-compression-level=mid` | Controls property store compression level. Allowed values: low, mid, high | `[string]` | | `--storage-floating-point-resolution-bits=64` | Max bits for floating-point property storage. Allowed values: 16, 32, 64. Lower values save memory but reduce precision. | `[uint64]` | | `--storage-access-timeout-sec=1` | Storage access timeout in seconds. Used to fine-tune the responsiveness and guard against queries indefinitely waiting. Can also be changed at runtime via `SET DATABASE SETTING 'storage.access_timeout_sec' TO 'value'`. Valid range: [1, 1000000]. | `[uint64]` | -| `--storage-enable-backup-dir=true` | Controls whether `.old` directory will be used to store backup. | `[bool]` | +| `--storage-backup-dir-enabled=true` | Controls whether `.old` directory will be used to store backup. | `[bool]` | ### Streams diff --git a/pages/database-management/enabling-memgraph-enterprise.mdx b/pages/database-management/enabling-memgraph-enterprise.mdx index 0c33f655a..5dc2ac744 100644 --- a/pages/database-management/enabling-memgraph-enterprise.mdx +++ b/pages/database-management/enabling-memgraph-enterprise.mdx @@ -245,6 +245,12 @@ ideal solution for those who need a worry-free, highly available system with failover ensures your system remains operational with minimal downtime and manual intervention. +The cluster's control plane can also be secured: [coordinator +authentication](/clustering/high-availability/coordinator-authentication) lets +coordinators authenticate OIDC, SAML and Kerberos connections against your +identity provider and enforce the `COORDINATOR_READ` and `COORDINATOR_WRITE` +privileges on Raft-replicated roles. + ### Multi-tenancy [Multi-tenant support](/database-management/multi-tenancy) enables you to manage diff --git a/pages/database-management/monitoring.mdx b/pages/database-management/monitoring.mdx index c596a0d3d..e967f5d25 100644 --- a/pages/database-management/monitoring.mdx +++ b/pages/database-management/monitoring.mdx @@ -264,6 +264,7 @@ JSON endpoint and `SHOW METRICS INFO` use different names — see | memgraph\_active\_edge\_type\_indices | Gauge | Number of active edge-type indexes. | | memgraph\_active\_edge\_type\_property\_indices | Gauge | Number of active edge-type-property indexes. | | memgraph\_active\_edge\_property\_indices | Gauge | Number of active edge-property indexes. | + | memgraph\_active\_vertex\_property\_indices | Gauge | Number of active vertex-property indexes. | | memgraph\_active\_point\_indices | Gauge | Number of active point indexes. | | memgraph\_active\_text\_indices | Gauge | Number of active text indexes on vertices. | | memgraph\_active\_text\_edge\_indices | Gauge | Number of active text indexes on edges. | @@ -321,6 +322,7 @@ used. | memgraph\_scan\_all\_by\_edge\_property\_value\_operator\_total | Counter | | memgraph\_scan\_all\_by\_edge\_property\_range\_operator\_total | Counter | | memgraph\_scan\_all\_by\_edge\_id\_operator\_total | Counter | + | memgraph\_scan\_all\_by\_vertex\_property\_operator\_total | Counter | | memgraph\_scan\_all\_by\_point\_distance\_operator\_total | Counter | | memgraph\_scan\_all\_by\_point\_withinbbox\_operator\_total | Counter | | memgraph\_expand\_operator\_total | Counter | diff --git a/pages/database-management/server-side-descriptions.mdx b/pages/database-management/server-side-descriptions.mdx index 1cd56770e..4fafdf48b 100644 --- a/pages/database-management/server-side-descriptions.mdx +++ b/pages/database-management/server-side-descriptions.mdx @@ -1,6 +1,6 @@ --- title: Server-side descriptions -description: Annotate labels, edge types, properties and databases with human-readable descriptions that are persisted by Memgraph and surfaced in SHOW SCHEMA INFO. +description: Annotate labels, edge types, properties, property values and databases with human-readable descriptions that are persisted by Memgraph and surfaced in SHOW SCHEMA INFO and the description() function. --- # Server-side descriptions @@ -9,6 +9,10 @@ Server-side descriptions are human-readable strings attached to schema elements - labels, edge types, properties and databases - that Memgraph stores durably and surfaces alongside the schema. +You can also describe individual property *values* - for example, decoding an +enum or lookup code such as `"1"` into `"Male"` - and read them back at query +time with the [`description()`](#resolve-a-value-with-description) function. + They are useful for documenting the meaning of nodes, edges and properties directly inside the database, so tools that consume `SHOW SCHEMA INFO` (such as LLM-based clients, GraphChat, MCP, text2cypher, or your own tooling) can pick @@ -33,11 +37,17 @@ You can attach a description to any of the following targets: | Edge type property | `EDGE TYPE PROPERTY :KNOWS(since)` | | Edge type pattern property | `EDGE TYPE PROPERTY (:Person)-[:KNOWS]->(:Person)(since)` | | Property (global) | `PROPERTY age` | +| Property value | `PROPERTY gender VALUE "1"` | | Database | `DATABASE memgraph` | Multi-label combinations are matched exactly; setting a description on `:Person:Student` does not affect nodes that only carry `:Person`. +A property-value description is keyed on the exact value: `PROPERTY gender VALUE "1"` +describes only the value `"1"` of `gender`, independent of any global +`PROPERTY gender` description. The value can be any literal (a string, number or +boolean). + ## Set a description Use `SET DESCRIPTION ON ""`: @@ -54,6 +64,8 @@ SET DESCRIPTION ON LABEL PROPERTY :Person(name) "Full name"; SET DESCRIPTION ON EDGE TYPE PROPERTY :KNOWS(since) "Year they met"; SET DESCRIPTION ON EDGE TYPE PROPERTY (:Person)-[:KNOWS]->(:Person)(since) "Year they met (pattern)"; SET DESCRIPTION ON PROPERTY age "Age in years"; +SET DESCRIPTION ON PROPERTY gender VALUE "1" "Male"; +SET DESCRIPTION ON PROPERTY gender VALUE "2" "Female"; SET DESCRIPTION ON DATABASE memgraph "Main graph database"; ``` @@ -70,6 +82,7 @@ DELETE DESCRIPTION ON LABEL :Person; DELETE DESCRIPTION ON EDGE TYPE (:Person)-[:KNOWS]->(:Person); DELETE DESCRIPTION ON LABEL PROPERTY :Person(name); DELETE DESCRIPTION ON PROPERTY age; +DELETE DESCRIPTION ON PROPERTY gender VALUE "1"; DELETE DESCRIPTION ON DATABASE memgraph; ``` @@ -84,18 +97,56 @@ SHOW DESCRIPTIONS; Result columns: - `type` - kind of target. One of `"label"`, `"edge type"`, `"label property"`, - `"edge type property"`, `"property"`, or `"database"`. Edge-type-pattern - targets share the `"edge type"` / `"edge type property"` value with their - global counterparts and are distinguished by the populated + `"edge type property"`, `"property"`, `"property value"`, or `"database"`. + Edge-type-pattern targets share the `"edge type"` / `"edge type property"` + value with their global counterparts and are distinguished by the populated `start_node_labels` and `end_node_labels` columns. - `label` - label or label combination, when applicable. - `start_node_labels` - source labels, for edge type patterns. - `end_node_labels` - destination labels, for edge type patterns. - `property` - property key, when applicable. +- `value` - the described value, for `"property value"` rows. - `description` - the stored text. Columns that don't apply to a given row are returned as `Null`. +## Resolve a value with `description()` + +Property-value descriptions are read back at query time with the `description()` +function, which maps a value to the description set for it: + +```opencypher +description(property_name, value) +``` + +- `property_name` - the property key, as a string. +- `value` - the value to look up, typically a stored property. + +It returns the description for that property/value pair, or `Null` if none is +set (or if `value` is `Null`). + +For example, after: + +```opencypher +SET DESCRIPTION ON PROPERTY gender VALUE "1" "Male"; +SET DESCRIPTION ON PROPERTY gender VALUE "2" "Female"; +``` + +stored codes can be decoded into labels: + +```opencypher +MATCH (p:Person) +RETURN p.name, description("gender", p.gender) AS gender; +``` + +| p.name | gender | +|---------|------------| +| "Alice" | "Male" | +| "Bob" | "Female" | +| "Carol" | `Null` | + +`Carol`'s `gender` has no matching description, so it resolves to `Null`. + ## Descriptions in `SHOW SCHEMA INFO` When [run-time schema tracking](/querying/schema) is enabled, `SHOW SCHEMA INFO` @@ -178,3 +229,17 @@ SET DESCRIPTION ON LABEL PROPERTY :Sensor(temperature) "Reading in degrees Celsi SET DESCRIPTION ON LABEL PROPERTY :Order(amount) "Total in cents, in the order's currency"; SET DESCRIPTION ON EDGE TYPE :PAID_WITH "Links an order to the payment method actually charged"; ``` + +### Decoding enum or lookup values + +Property-value descriptions turn opaque codes into readable labels without a +join or a separate lookup table. Describe each code once, then resolve it at +query time with `description()`: + +```opencypher +SET DESCRIPTION ON PROPERTY status VALUE "A" "Active"; +SET DESCRIPTION ON PROPERTY status VALUE "C" "Closed"; + +MATCH (a:Account) +RETURN a.id, description("status", a.status) AS status; +``` diff --git a/pages/database-management/server-stats.mdx b/pages/database-management/server-stats.mdx index 86bbec23e..c3ffc364f 100644 --- a/pages/database-management/server-stats.mdx +++ b/pages/database-management/server-stats.mdx @@ -18,6 +18,13 @@ To get the version of the instance being queried, run the following query: SHOW VERSION; ``` +The query can be run on data instances and on +[coordinators](/clustering/high-availability), so you can check the version of +every instance in a high-availability cluster without connecting through the +MAIN. On data instances it requires the `STATS` privilege; on coordinators it is +a read-only query that requires +[`COORDINATOR_READ`](/clustering/high-availability/coordinator-authentication#which-privilege-each-query-requires). + ## Storage information `SHOW STORAGE INFO` comes in two flavors: @@ -230,6 +237,7 @@ are reported in microseconds. | "ActiveEdgeTypePropertyIndices" | "Index" | "Gauge" | 0 | | "ActiveLabelIndices" | "Index" | "Gauge" | 6 | | "ActiveLabelPropertyIndices" | "Index" | "Gauge" | 18 | +| "ActiveVertexPropertyIndices" | "Index" | "Gauge" | 0 | | "ActivePointIndices" | "Index" | "Gauge" | 0 | | "ActiveTextIndices" | "Index" | "Gauge" | 0 | | "ActiveTextEdgeIndices" | "Index" | "Gauge" | 0 | @@ -293,6 +301,7 @@ are reported in microseconds. | "ScanAllByIdOperator" | "Operator" | "Counter" | 0 | | "ScanAllByLabelOperator" | "Operator" | "Counter" | 0 | | "ScanAllByLabelPropertiesOperator" | "Operator" | "Counter" | 0 | +| "ScanAllByVertexPropertyOperator" | "Operator" | "Counter" | 0 | | "ScanAllByPointDistanceOperator" | "Operator" | "Counter" | 0 | | "ScanAllByPointWithinbboxOperator" | "Operator" | "Counter" | 0 | | "ScanAllOperator" | "Operator" | "Counter" | 0 | diff --git a/pages/fundamentals/data-durability.mdx b/pages/fundamentals/data-durability.mdx index 704d09793..46a8410dc 100644 --- a/pages/fundamentals/data-durability.mdx +++ b/pages/fundamentals/data-durability.mdx @@ -103,6 +103,11 @@ Per-transaction WAL checksums were introduced in Memgraph v3.12. WAL files written by older versions do not contain checksums and are recovered without integrity verification. Checksum verification applies only to WAL files written by Memgraph v3.12 or newer. + +Before Memgraph v3.13, a checksum mismatch anywhere in the WAL chain recovered +the readable prefix of that file and then continued with the files after it, +which could bring the database up with an incomplete dataset without reporting +it. Since v3.13, damage in a completed WAL file fails recovery instead. To guard against silent on-disk corruption, Memgraph protects WAL files with @@ -113,16 +118,37 @@ To guard against silent on-disk corruption, Memgraph protects WAL files with - **Each transaction** is protected by a 4-byte checksum covering the transaction's bytes (transaction start, deltas and transaction end). -Checksums are verified automatically during recovery. If a transaction's stored -checksum does not match the recomputed value, the WAL is considered corrupted at -that point and recovery stops, so corrupted data is never applied to the -database. A mismatch in the WAL header causes recovery from that file to fail. +Checksums are verified automatically during recovery, so corrupted data is never +applied to the database. A mismatch in the WAL header causes recovery from that +file to fail. What happens on a transaction checksum mismatch depends on whether +the file was completed: + +- A **completed (finalized) WAL file** — one Memgraph finished writing and + rotated away from — was flushed to disk before it was closed, so every + transaction in it is durable and was already acknowledged. A mismatch there + means the bytes on disk rotted, and recovery fails rather than continuing with + part of the file. Recovering only a prefix would be unsound, because the WAL + files that follow were written on top of the transactions that went missing. +- A **WAL file that was still being written** when the process stopped may + legitimately have a torn tail. Recovery applies its transactions up to the + last whole one and stops there, which is how an interrupted write degrades + gracefully. + +Files that the newest snapshot already covers are skipped without being read, so +corruption in those is harmless. If a WAL file written after the newest snapshot +is damaged, the database fails to recover; use +[`--storage-allow-recovery-failure`](/database-management/configuration) together +with `RECOVER SNAPSHOT` (see [recovery failure +handling](#recovery-failure-handling)), or restore from a backup. The same checksums protect WAL files that are buffered on disk on a replica before being applied, so corruption introduced between the main and the replica -is detected before the data is committed. Deltas streamed during the commit -(`PrepareCommitRpc`) are not checksummed because the TCP transport already -provides integrity guarantees. +is detected before the data is committed. When a replica hits damage in a +finalized file it reports the failure and stops applying the rest of the batch +instead of skipping ahead; the in-flight transaction is aborted, main does not +advance its view of the replica, and the transfer is retried later. Deltas +streamed during the commit (`PrepareCommitRpc`) are not checksummed because the +TCP transport already provides integrity guarantees. Snapshots are not yet protected by checksums. @@ -295,6 +321,24 @@ test, remove the file. You can also monitor real-time disk utilization during snapshot creation using tools like `iostat`, `iotop`, or `dstat`. +### Limiting the impact of snapshots on queries + +A snapshot writes the whole dataset to disk, and on larger datasets that can +slow down the queries running at the same time. Memgraph therefore writes +snapshots out in small chunks, so a snapshot in progress doesn't take memory +away from your workload. + +The chunk size is set with `--storage-snapshot-writeback-window-mib` and +defaults to 32 MiB. Each snapshot thread uses its own chunk, so with +`--storage-parallel-snapshot-creation=true` the total is the chunk size times +`--storage-snapshot-thread-count`. + +The default suits most deployments. Tuning it trades off snapshot duration +against the performance of queries running at the same time, but the balance +depends on your hardware, your operating system and your workload, so treat any +change as something to measure on your own deployment. Setting it to `0` turns +the behavior off and lets the operating system decide. + ## Recovery failure handling By default, if a database fails durability recovery on startup — because of a @@ -401,7 +445,7 @@ RECOVER SNAPSHOT "/path/to/good.snapshot"; On success, Memgraph clears the broken flag and the database resumes normal operation and background durability. The existing `RECOVER SNAPSHOT` behavior moves all prior/corrupt snapshots and WAL files to the `.old` directory (or -deletes them when [`--storage-enable-backup-dir`](/database-management/configuration) +deletes them when [`--storage-backup-dir-enabled`](/database-management/configuration) is off), leaving a clean single-snapshot directory. As a result, the database **recovers cleanly on the next restart and does not re-enter the broken state**. diff --git a/pages/fundamentals/indexes.mdx b/pages/fundamentals/indexes.mdx index 63e5c9d58..ab20bfbf6 100644 --- a/pages/fundamentals/indexes.mdx +++ b/pages/fundamentals/indexes.mdx @@ -477,6 +477,53 @@ not yet supported. +### Global vertex property index + + + +This index supports non-blocking creation: reads continue without interruption, +while writes are briefly paused. For more information, see the [concurrent index +creation](#concurrent-index-creation). + + + +A label-property index requires you to know the label up front. But sometimes +you just want to find a node by a property, say `uuid`, regardless of what +labels the node has. Global vertex-property indices allow you to do exactly +this. For example, to create an index on all nodes with a `uuid` property: + +```cypher +CREATE GLOBAL INDEX ON :(uuid); +``` + +Once created, queries that search by that property can use the index +automatically: + +```cypher +MATCH (n) WHERE n.uuid = "abc-123" RETURN n; +MATCH (n {uuid: "abc-123"}) RETURN n; +``` + +The planner also considers this index as a fallback when a query specifies a +label but no matching label-property index exists. For example, if there is no +index on `:Person(uuid)` but a global index on `uuid` exists, +`MATCH (n:Person {uuid: "abc-123"}) RETURN n` can use the global index and +filter by label afterwards. + + + +When both a label-property index and a global vertex property index could serve +a query, the planner picks whichever has the lower estimated cardinality. In +practice this usually means the label-property index wins, since it's scoped to +a single label. + + + +To drop the index: +```cypher +DROP GLOBAL INDEX ON :(uuid); +``` + ### Edge-type index @@ -696,7 +743,8 @@ memgraph> MATCH (n:Person) WHERE n.name =~ ".*an$" RETURN n.name; ## Show created indexes To see all the information on the label, label-property, edge-type, edge-type -property, point indexes and vector indexes, run the following query: +property, global vertex property, global edge property, point indexes and vector +indexes, run the following query: ```cypher SHOW INDEX INFO; @@ -739,6 +787,10 @@ DROP INDEX ON :Label(property); DROP INDEX ON :Label(property1, property2); ``` +```cypher +DROP GLOBAL INDEX ON :(property_name); +``` + ```cypher DROP EDGE INDEX ON :EDGE_TYPE; ``` @@ -760,8 +812,8 @@ DROP POINT INDEX ON :Label(property); The `DROP ALL INDEXES` clause allows you to delete all indices in your database in a single operation. This includes all types of indices: label indices, label-property indices, edge type indices, edge type-property indices, global -edge indices, point indices, text indices, vector indices, and vector edge -indices. +vertex property indices, global edge property indices, point indices, text +indices, vector indices, and vector edge indices. ```cypher DROP ALL INDEXES; @@ -956,6 +1008,10 @@ USING INDEX :Label(property) ...; USING INDEX :Label(property1, property2) ...; ``` +```cypher +USING INDEX :(property) ...; +``` + It is also possible to specify multiple hints separated with comma. In that case, the planner will apply the first hint that is applicable for a given match. @@ -1008,8 +1064,8 @@ b_1$, or $a_1 = b_1$ and $a_2 < b_2$. Memgraph supports **(almost) fully concurrent index creation** for all skiplist-based indices, including label, label-property, composite, edge-type, -edge-type property, and global edge property, with minimal impact on -performance. +edge-type property, global edge property, and global vertex property, with +minimal impact on performance. The three-phase implementation begins with a brief **registration phase** that requires `READ ONLY` access. This ensures that all pending write transactions diff --git a/pages/fundamentals/storage-memory-usage.mdx b/pages/fundamentals/storage-memory-usage.mdx index 826fe6436..cf0c4b0c4 100644 --- a/pages/fundamentals/storage-memory-usage.mdx +++ b/pages/fundamentals/storage-memory-usage.mdx @@ -62,6 +62,11 @@ If some other transactions are running in your system, you will receive a warning message, so be sure to [set the log level at least to `WARNING`](/database-management/logs). +Switching to `IN_MEMORY_ANALYTICAL` is refused when replicas are registered, +because analytical writes are not replicated. See [In-memory analytical storage +mode](#in-memory-analytical-storage-mode) and [Bulk import in analytical +mode](/clustering/high-availability/analytical-import). + Switching from the in-memory storage mode to the on-disk storage mode is allowed when there is only one active session and the database is empty. As Memgraph Lab uses multiple sessions to run queries in parallel, it is currently impossible to @@ -223,23 +228,41 @@ SNAPSHOT;` Cypher query. In the analytical storage mode, WAL files and periodic snapshots are **not created**. -Before switching back to the in-memory transactional storage mode create a -snapshot manually. In the in-memory analytical storage mode, Memgraph guarantees -that creating a snapshot is **the only** transaction present in the system, and -all the other transactions will wait until the snapshot is created to ensure its -validity. Once Memgraph switches to the in-memory transactional mode, it will -restore data from the snapshot file and create a WAL for all new updates, if not -otherwise instructed by the [config +Switching back to the in-memory transactional storage mode writes a snapshot on +its own, synchronously and before the mode change completes, so there is no need +to create one manually first. If that snapshot cannot be written, the switch is +aborted and the query throws: the database stays in analytical mode with its +data and durability files unchanged, so you can fix the cause and retry. When +the snapshot succeeds, the superseded snapshots and WAL files are moved to the +`.old` directory — or deleted when +[`--storage-backup-dir-enabled`](/database-management/configuration) is set to +`false` — and WAL sequence numbering restarts from 0, because the new snapshot is +a new durability base rather than an increment on the old one. New updates are +written to a WAL again, if not otherwise instructed by the [config file](/configuration/configuration-settings#storage). +In the in-memory analytical storage mode, Memgraph guarantees that creating a +snapshot is **the only** transaction present in the system, and all the other +transactions will wait until the snapshot is created to ensure its validity. +Avoid taking a manual snapshot in the middle of an import you intend to finish +with a switch back to transactional mode, though: analytical writes do not +advance the durable timestamp, so such a snapshot is stamped with the +**pre-import** timestamp. These are some of the implications of not having ACID properties. But if you do not have write-heavy workload, and you want to run analytical queries that will not change the data, you can take advantage of the low memory costs of the analytical mode. -At the moment, the in-memory analytical storage mode **doesn't support -replication and high availability**. +Analytical writes are never appended to the WAL, so they **cannot be +replicated**. An instance can enter the analytical storage mode only when it is +the MAIN and has no registered replicas, and replicas can be registered or +unregistered only while every database is in transactional mode. Within those +rules, analytical mode can be used for a bulk import on a +[replication](/clustering/replication/best-practices#storage-mode-requirements) +or [high availability](/clustering/high-availability/analytical-import) cluster: +unregister the replicas, import, switch back to transactional mode and register +them again. ### On-disk transactional storage mode diff --git a/pages/fundamentals/transactions.mdx b/pages/fundamentals/transactions.mdx index 188764e57..537c0f434 100644 --- a/pages/fundamentals/transactions.mdx +++ b/pages/fundamentals/transactions.mdx @@ -204,10 +204,11 @@ synthetic `gc` row may appear at once, distinguished by `trigger`. -Synthetic rows cannot be terminated. Passing `"snapshot"` or `"gc"` to -`TERMINATE TRANSACTIONS` has no effect — background snapshot creation and -garbage collection run outside the normal transaction lifecycle and cannot be -interrupted via Cypher. +Synthetic rows cannot be terminated. Background snapshot creation and garbage +collection run outside the normal transaction lifecycle and cannot be +interrupted via Cypher. Passing `"snapshot"` or `"gc"` to `TERMINATE +TRANSACTIONS` raises an error because they are not valid transaction ids, and +`TERMINATE TRANSACTIONS "*"` skips them. Because snapshot and garbage-collection rows always have `status` `"running"`, @@ -285,6 +286,77 @@ The `TERMINATE TRANSACTIONS` query signalizes to the thread executing the transaction that it should stop the execution. No violent interruption will happen, and the whole system will stay in a consistent state. +The result has two columns, `transaction_id` and `killed`. A transaction +reports `killed: true` only if it was actually found, the caller was allowed to +terminate it, and it was still running. Ids that do not match a running +transaction, and matches the caller is not authorized to terminate, both report +`killed: false` — an unauthorized match is indistinguishable from a missing id, +so the query never reveals that somebody else's transaction exists. + + +**Breaking change in Memgraph 3.13**: transaction ids must now parse as a whole +number. Previously an id with trailing characters (for example +`TERMINATE TRANSACTIONS "9223372036854794885abc"`) silently terminated the +transaction matching the numeric prefix, and a completely unparseable id was +reported back as a termination attempt on `18446744073709551615`. Both cases +now raise an error instead. + +In the same release, naming a transaction you are not authorized to terminate +reports `killed: false` instead of `true`. Previously such a query claimed a +kill that never happened. + + +#### Terminate all transactions + +Instead of copying ids out of `SHOW TRANSACTIONS` one at a time, you can +terminate everything at once with the `"*"` wildcard: + +```cypher +TERMINATE TRANSACTIONS "*"; +``` + +This terminates every transaction the caller is authorized to terminate, +following the same rules as the id list form: + +- **Scope**: all transactions visible through `SHOW TRANSACTIONS`, across all + databases — not just the caller's current database. +- **Authorization**: a transaction is terminated if the caller owns it, or holds + the **TRANSACTION_MANAGEMENT** privilege on the database that transaction is + running on. A user without the privilege terminates only its own + transactions; the query does not fail, it simply returns fewer rows. + Transactions the caller may not terminate are absent from the output rather + than reported as `killed: false`, so no transaction id is leaked. +- **The caller's own transaction is skipped**, so the session issuing the sweep + survives it and can read back the result. +- **Output**: the usual `transaction_id` and `killed` columns, one row per + terminated transaction ordered by ascending transaction id (oldest first), + and zero rows if nothing matched. +- Transactions that are already committing or aborting are skipped, the same as + with the id list form. + +The wildcard must be the only argument. Mixing it with ids, such as +`TERMINATE TRANSACTIONS "*", "9223372036854794885"`, raises an error. + +A parameterized id is treated exactly like a literal one, so running +`TERMINATE TRANSACTIONS $id` with `$id = "*"` also terminates everything. + + +System transactions (for example an in-flight `CREATE DATABASE` or `GRANT`) are +reported as terminated but run to completion — they are not abortable. This +applies to the id list form as well. + + +```copy=false +memgraph> TERMINATE TRANSACTIONS "*"; ++-----------------------+-----------------------+ +| transaction_id | killed | ++-----------------------+-----------------------+ +| "9223372036854794885" | true | +| "9223372036854794891" | true | ++-----------------------+-----------------------+ +2 rows in set (round trip in 0.001 sec) +``` + ### Terminate custom procedures diff --git a/pages/getting-started/packaging-memgraph.mdx b/pages/getting-started/packaging-memgraph.mdx index faa373d84..2c0e37aef 100644 --- a/pages/getting-started/packaging-memgraph.mdx +++ b/pages/getting-started/packaging-memgraph.mdx @@ -9,10 +9,11 @@ import { Steps } from 'nextra/components' # Package Memgraph This guide will show you how to package Memgraph for one of out supportedLinux distributions. -There are two main ways to package Memgraph: +There are three main ways to package Memgraph: - [Using the `mgbuild.sh` script](#using-the-mgbuildsh-script), which builds Memgraph in a Docker container. (Recommended) -- [Using CPack](#using-cpack), which builds Memgraph using CMake. +- [Using the `package.sh` script](#using-the-packagesh-script), which packages a local build directory. +- [Using CPack](#using-cpack), which invokes CPack manually. The distributions of Linux currently supported by Memgraph are and their associated `OS` environment variable: @@ -114,12 +115,41 @@ sudo apt install ./output/memgraph__.deb +## Using the `package.sh` script + +After following the build instructions using +[`build.sh`](/getting-started/build-memgraph-from-source#use-buildsh-script), or +[`conan` and `cmake`](/getting-started/build-memgraph-from-source#use-conan-and-cmake), the +`package.sh` script in the repository root packages the build directory directly on the host. +It takes a target and a package format: + +```bash +source /opt/toolchain-v7/activate +./package.sh memgraph deb +``` + +- `TARGET` is one of `memgraph` (the `memgraph` + `memgraph-debuginfo` packages), `mage` + (the `memgraph-mage` package, plus `memgraph-mage-debuginfo` when the build used + `--split-debug`), or `all` (everything the build was configured for). +- `FORMAT` is `deb` or `rpm`. + +For example, to package the MAGE query modules as an RPM from a +`./build.sh --mage only` build: + +```bash +./package.sh mage rpm +``` + +The script checks that the build directory is actually configured for the requested target +(e.g. packaging `mage` requires a build with `--mage on` or `--mage only`), selects the right +CPack components, and verifies the expected number of packages was produced. The packages are +written to `build/output/`. Use `--build-dir DIR` to package from a non-default build +directory. + ## Using CPack -After following the build instructions using -[`build.sh`](/getting-started/build-memgraph-from-source#use-buildsh-script), or -[`conan` and `cmake`](/getting-started/build-memgraph-from-source#use-conan-and-cmake), one can use -CPackto build a Memgraph package for a Linux distribution. +`package.sh` is a thin wrapper around CPack — if you need full control, you can also invoke +CPack manually. Prior to packaging, create the output directory: diff --git a/pages/querying/differences-in-cypher-implementations.mdx b/pages/querying/differences-in-cypher-implementations.mdx index 687f1a700..48df4b6c1 100644 --- a/pages/querying/differences-in-cypher-implementations.mdx +++ b/pages/querying/differences-in-cypher-implementations.mdx @@ -34,6 +34,15 @@ CREATE INDEX FOR (n:Person) ON (n.age, n.country); CREATE INDEX FOR ()-[r:KNOWS]-() ON (r.since); ``` +The optional `RANGE` keyword is also accepted in the `FOR ... ON` syntax, for both nodes and relationships: + +```cypher +CREATE RANGE INDEX FOR (n:Person) ON (n.surname); +CREATE RANGE INDEX FOR ()-[r:KNOWS]-() ON (r.since); +``` + +Neo4j's `RANGE` index is its general-purpose ordered property index, used for equality, range and prefix lookups — the same role Memgraph's label-property and edge-type property indexes already fill. `RANGE` is therefore accepted as a synonym: it maps onto the existing index with no new index type and no change in behavior. + The native Memgraph syntax remains supported as well: ```cypher diff --git a/pages/querying/text-search.mdx b/pages/querying/text-search.mdx index bad2c236b..a74861ad7 100644 --- a/pages/querying/text-search.mdx +++ b/pages/querying/text-search.mdx @@ -616,6 +616,71 @@ Result: +-------------------------------+ ``` +## Access control Enterprise + +When [fine-grained access control](/database-management/authentication-and-authorization/role-based-access-control#fine-grained-access-control) +is active for the user running the query, text search respects their read +permissions: a match is returned only if the user could read it with a normal +query. Content the user is not allowed to read is never revealed through the +search results or their relevance scores. + +A node or relationship appears in the results only when **both** of these hold: + +- **The node or relationship is readable.** The user has `READ` on its + [labels](/database-management/authentication-and-authorization/role-based-access-control#label-based-access-control) + — or, for a relationship, on its + [type](/database-management/authentication-and-authorization/role-based-access-control#relationship-permissions) + and on both of its endpoint nodes. Permissions are evaluated against the + entity's *actual* labels, so a `DENY` on any of them removes it from the + results. +- **The matched property is readable.** Which properties must be readable + depends on the procedure: + - `text_search.search` and `text_search.search_edges` search a property named + in the query (for example `data.name:Alice`), so only that + [property](/database-management/authentication-and-authorization/role-based-access-control#property-based-access-control) + must be readable. The match surfaces even if the entity has other properties + the user cannot read. + - `text_search.search_all`, `text_search.regex_search` and their `_edges` + variants search across every indexed property, so **all** indexed text + properties the entity has must be readable — if even one is denied, the + entity is dropped, because the match could have come from it. + +A hit whose match depends on a property the user cannot read is **silently +dropped**: the results are simply empty, just as a +[denied property](/database-management/authentication-and-authorization/role-based-access-control#how-denied-properties-behave) +reads back as `Null` in a normal query. No error is raised. + +For example, with a text index on `:Employee` and an analyst who can read +employees and their properties, except `ssn`: + +```cypher +GRANT READ ON NODES CONTAINING LABELS :Employee TO analyst; +GRANT READ {*} ON NODES CONTAINING LABELS :Employee TO analyst; +DENY READ {ssn} ON NODES CONTAINING LABELS :Employee TO analyst; +``` + +- `CALL text_search.search('employees', 'data.name:Alice')` returns matching + employees — `name` is readable. +- `CALL text_search.search('employees', 'data.ssn:123456789')` returns nothing — + the queried property is denied. +- `CALL text_search.search_all('employees', 'Alice')` returns nothing for an + employee whose indexed `ssn` is set, because the match could have come from the + denied property. + + +`text_search.aggregate` and `text_search.aggregate_edges` run inside the search +engine and cannot filter individual results, so they cannot apply per-row +permissions. When the caller has a fine-grained restriction on the index's +properties, the aggregation returns an error message instead of a result — use +the search procedures above when you need permission-aware results. + + + +These checks apply only when fine-grained access control is active for the user. +An unrestricted user (for example an administrator) and Memgraph Community +edition see all matches. + + ## Drop text index Text indices are dropped with the `DROP TEXT INDEX` command. You need to give the name of the index to be deleted. diff --git a/pages/querying/vector-search.mdx b/pages/querying/vector-search.mdx index 8d0e525d0..66c4e0e5c 100644 --- a/pages/querying/vector-search.mdx +++ b/pages/querying/vector-search.mdx @@ -307,6 +307,52 @@ Alternative options, such as `f16` for lower memory usage, allow you to fine-tun | `i16` | 16-bit signed integer. | | `i8` | 8-bit signed integer. | +## Access control Enterprise + +When [fine-grained access control](/database-management/authentication-and-authorization/role-based-access-control#fine-grained-access-control) +is active for the user running the query, vector search respects their read +permissions: `vector_search.search` and `vector_search.search_edges` return a +node or relationship only if the user could read it with a normal query. + +A result is included only when **both** of these hold: + +- **The node or relationship is readable.** The user has `READ` on its + [labels](/database-management/authentication-and-authorization/role-based-access-control#label-based-access-control) + — or, for a relationship, on its + [type](/database-management/authentication-and-authorization/role-based-access-control#relationship-permissions) + and on both endpoint nodes. Permissions are evaluated against the entity's + *actual* labels, so a `DENY` on any of them removes it. +- **The indexed property is readable.** The user must be able to read the + [property](/database-management/authentication-and-authorization/role-based-access-control#property-based-access-control) + the index is built on (the embedding). If it is denied, the result is dropped — + otherwise the match and its similarity score would reveal the vector. + +Results the user is not allowed to see are **silently dropped** (no error is +raised), so a search may return fewer than the requested number of results. For +an index that spans several labels or edge types — including a +[wildcard index](/querying/vector-search#index-on-multiple-labels-or-edge-types) — +this check runs per result, so one index returns only the entities the user is +permitted to read. + +For example, with a wildcard vector index `docs` on `(embedding)` and a reader +who may see public documents but not confidential ones: + +```cypher +GRANT READ ON NODES CONTAINING LABELS :Public TO reader; +DENY READ ON NODES CONTAINING LABELS :Confidential TO reader; +GRANT READ {*} ON NODES CONTAINING LABELS * TO reader; +``` + +`CALL vector_search.search('docs', 5, [0.1, 0.2])` returns only the `:Public` +documents among the nearest neighbours; `:Confidential` ones are dropped even if +they are closer. + + +These checks apply only when fine-grained access control is active for the user. +An unrestricted user (for example an administrator) and Memgraph Community +edition see all matches. + + ## Monitor vector index memory Memgraph tracks vector index memory separately from the rest of the graph data. You can inspect both from `SHOW STORAGE INFO`: diff --git a/pages/release-notes.mdx b/pages/release-notes.mdx index 0cf175471..43dacaa9b 100644 --- a/pages/release-notes.mdx +++ b/pages/release-notes.mdx @@ -46,6 +46,241 @@ guide. ## 🚀 Latest release +### Memgraph v3.13.0 - September 9th, 2026 + +{

⚠️ Breaking changes

} + +- The APOC compatibility names `apoc.convert.fromJsonList` and + `apoc.convert.toJson` now resolve to the C++ `convert` module + (`convert.from_json_list` / `convert.to_json`) instead of the Python + `json_util` implementations. Output can differ — especially for nodes, + relationships, paths, points, and temporals. Call `json_util.*` explicitly if + you need the previous Python behavior, or update consumers for the new + structured / ISO-8601 JSON forms. + [#4443](https://github.com/memgraph/memgraph/pull/4443) +- The `COORDINATOR` privilege was removed — it never gated any operation. It is + no longer accepted in `GRANT` / `DENY` / `REVOKE` statements, which now fail + with a syntax error, and it is no longer reported by `SHOW PRIVILEGES` or + included in `GRANT ALL PRIVILEGES`. Remove any `GRANT COORDINATOR` / + `DENY COORDINATOR` / `REVOKE COORDINATOR` statements from provisioning scripts + and update tooling that compares against the full privilege list. No data + migration is required: a stale grant in an existing deployment is never + reported and never checked. The new `COORDINATOR_READ` and `COORDINATOR_WRITE` + privileges are not a replacement — they are coordinator-only and are rejected + on data instances. + [#4399](https://github.com/memgraph/memgraph/pull/4399) +- Connecting to a coordinator with a username and password is no longer always + accepted. Once `--auth-module-mappings` is set, the enterprise license is + valid, **and** a role holding `COORDINATOR_WRITE` exists, basic authentication + is denied in favor of SSO. Until all three hold, the previous passthrough + behavior is unchanged. + [#4399](https://github.com/memgraph/memgraph/pull/4399) +- `SHOW INSTANCES` is now strongly consistent and always answered by the leader + coordinator. A follower that cannot reach the leader no longer falls back to + reporting the cluster from its own local Raft state with `unknown` health — + it returns an empty result set with a `LeaderNotReachable` warning + notification instead. Tooling that parses `SHOW INSTANCES` should treat an + empty result as "cluster state unknown, retry" rather than "no instances + registered". A data instance that is down now reports the role recorded in the + Raft log (`main` / `replica`) instead of an unknown role. + [#4492](https://github.com/memgraph/memgraph/pull/4492) +- `TERMINATE TRANSACTIONS` now requires transaction ids to parse in full. Ids + with trailing characters previously terminated the transaction matching the + numeric prefix, and unparseable ids were reported back as an attempt on + `18446744073709551615`; both now raise an error. + [#4534](https://github.com/memgraph/memgraph/pull/4534) + +{

🐞 Bug fixes

} + +- Text and vector search now respect fine-grained label and property + permissions. Hits the caller cannot read are filtered out, and + `text_search.aggregate` / `text_search.aggregate_edges` return an error unless + every indexed property is readable. + [#4316](https://github.com/memgraph/memgraph/pull/4316) +- Fixed unbounded memory growth from the query AST cache when many + structurally unique queries were run (for example streams of inlined + embeddings). The cache is now a bounded LRU controlled by + `--query-ast-cache-max-size` (default `1000`; `0` disables it). + [#4334](https://github.com/memgraph/memgraph/pull/4334) +- The `collections.*` functions and procedures now accept `NULL` list and scalar + arguments instead of failing argument validation. Each function returns a + defined result (for example `collections.sum(null)` → `null`, + `collections.sort(null)` → `[]`, `collections.contains(null, x)` → `false`), + matching the compatibility-layer behavior. + [#4416](https://github.com/memgraph/memgraph/pull/4416) +- Fixed stale coordinator entries in the HA routing table and a possible + coordinator crash on `REMOVE COORDINATOR`. Coordinators now deduplicate + entries when applying Raft logs and snapshots. + [#4444](https://github.com/memgraph/memgraph/pull/4444) +- Fixed `path.subgraph_all` and `path.subgraph_nodes` so their config options + work as expected. + [#4447](https://github.com/memgraph/memgraph/pull/4447) +- Under heavy concurrent load, operations that need exclusive access to the + database could hang indefinitely while shared readers kept arriving. Those + exclusive waiters now take priority so the hang no longer happens. + [#4450](https://github.com/memgraph/memgraph/pull/4450) +- Fixed a crash when `LOAD PARQUET` hit invalid values (for example a time + outside Memgraph’s supported range). The query now fails with an error + instead of taking down the server. + [#4454](https://github.com/memgraph/memgraph/pull/4454) +- Fixed `LOAD PARQUET` silently storing wrong timestamps or durations when + converting large temporal values from Parquet. Those cases now fail with a + clear query error instead of corrupting data. + [#4455](https://github.com/memgraph/memgraph/pull/4455) +- Fixed a rare crash in garbage collection under heavy concurrent writes when + indexes or unique constraints are defined. + [#4475](https://github.com/memgraph/memgraph/pull/4475) +- Fixed a crash when multiple sessions printed query plans at the same time + (for example concurrent `EXPLAIN` / `PROFILE`, or query-plan logging). + [#4489](https://github.com/memgraph/memgraph/pull/4489) +- Fixed silent data loss when a WAL file other than the last one in the chain + was damaged. Recovery used to apply the readable prefix of that file and then + apply the following files in full, coming up with an incomplete dataset + without reporting anything; on a replica the divergence never healed because + the replica reported the later timestamp and main considered it caught up. + Damage in a completed (finalized) WAL file is now fatal: the database fails + recovery instead of starting stale, and a replica stops applying the rest of + the chain. A WAL file that was still being written when the process crashed + is unaffected and is still truncated to its last whole transaction. Recover + such an instance with + [`--storage-allow-recovery-failure`](/database-management/configuration) plus + `RECOVER SNAPSHOT`, or from a backup. + [#4528](https://github.com/memgraph/memgraph/pull/4528) +- `TERMINATE TRANSACTIONS` no longer reports `killed: true` for a transaction + the caller is not authorized to terminate. Such a match now reports + `killed: false`, indistinguishable from an id that does not exist. + [#4534](https://github.com/memgraph/memgraph/pull/4534) + +{

🛠️ Improvements

} + +- MAGE is now built and packaged from the same unified CMake / Conan tree as + Memgraph. Build it with `build.sh --mage on` or `--mage only`; the old + `mage/setup` Python script is removed. Only affects people building MAGE from + source — prebuilt packages are unchanged. + [#4348](https://github.com/memgraph/memgraph/pull/4348) +- `build.sh --target` now accepts several targets in one invocation (for + example `--target memgraph memgraph__unit`), so you can build multiple + targets with a single configure step instead of one run per target. + [#4404](https://github.com/memgraph/memgraph/pull/4404) +- Reduced memory used by the query abstract syntax tree and plan caches by + storing each LRU cache key once instead of twice per entry. + [#4445](https://github.com/memgraph/memgraph/pull/4445) +- Queries that use query-module functions or `CALL` procedures are now + plan-cached instead of being re-parsed and re-planned on every run. Module + reloads stay safe; only module-dependent queries are invalidated, and the + next execution picks up updated code. + [#4461](https://github.com/memgraph/memgraph/pull/4461) + [#4482](https://github.com/memgraph/memgraph/pull/4482) +- In in-memory transactional mode, `CREATE INDEX` / `CREATE CONSTRAINT` no + longer stall behind garbage collection (and GC no longer waits on those DDL + statements). Under index-heavy workloads this removes GC-correlated latency + spikes on index and constraint creation. + [#4468](https://github.com/memgraph/memgraph/pull/4468) +- Recovery from WAL files is roughly 35% faster. Each WAL file now records its + timestamp range and transaction count in its header, so deciding which files + to recover from no longer requires parsing every file end to end, and + replaying a file parses it once instead of twice. This also speeds up replica + recovery and the WAL cleanup that runs after every snapshot. + [#4528](https://github.com/memgraph/memgraph/pull/4528) +- Cluster management queries forwarded from a follower coordinator to the leader + now have [explicit RPC + timeouts](/clustering/high-availability/how-high-availability-works#rpc-timeouts), + so a Bolt session can no longer block indefinitely against a leader that is + reachable but stuck. The same applies to MAIN-to-REPLICA system transaction + messages (auth, multi-tenancy, parameters), which now time out after 10s and + defer to system recovery. The unused `EnableWritingOnMainRpc` and its + Prometheus counters were removed. + [#4399](https://github.com/memgraph/memgraph/pull/4399) +- `YIELD LEADERSHIP` and `SHOW COORDINATOR SETTINGS` can now be run on any + coordinator — followers forward them to the leader instead of failing or + answering from local state. When no leader can serve a cluster query, + Memgraph now says so explicitly: read queries (`SHOW INSTANCES`, `SHOW + COORDINATOR SETTINGS`, `SHOW REPLICATION LAG`) return no rows with a + `LeaderNotReachable` warning, `SHOW REPLICATION LAG` additionally reports why + the lag is unavailable via `ReplicationLagUnavailable` (no current main, main + unresponsive, leader still taking over, stale leader view), and + `ADD COORDINATOR`, `REMOVE COORDINATOR` and `UPDATE CONFIG` fail with a clear + "coordinator is not a leader" error instead of a misleading one. + [#4492](https://github.com/memgraph/memgraph/pull/4492) + +{

✨ New features

} + +- Added `collections.disjunction`, `collections.subtract`, and + `collections.duplicates` to the MAGE `collections` module for set-style list + difference, subtraction, and finding repeated values. + [#4415](https://github.com/memgraph/memgraph/pull/4415) +- Added `map.get` and `map.merge_list` to the MAGE `map` module. Existing + helpers such as `map.merge` and `map.set_key` now treat a `null` map as empty + (and a `null` key as a no-op) instead of failing, and accept nodes or + relationships by using their property maps. + [#4417](https://github.com/memgraph/memgraph/pull/4417) +- Added `text.compare_cleaned(text1, text2)` to the MAGE `text` module. It + compares two strings for equality after keeping only ASCII letters and digits + (lowercased) and dropping everything else; `NULL` arguments return `false`. + [#4435](https://github.com/memgraph/memgraph/pull/4435) +- Added `convert.from_json_map`, `convert.from_json_list`, `convert.to_map`, and + `convert.to_json` for parsing and serializing JSON, with an optional `path` + selector on the from_* helpers and structured output for graph and temporal + types from `to_json`. + [#4443](https://github.com/memgraph/memgraph/pull/4443) +- Added `search.node` and `search.node_all` procedures that find nodes by a + `{label: property}` map, a comparison operator, and a value. A property list + per label is OR-ed; `search.node` de-duplicates by node while + `search.node_all` keeps duplicates. An existing label-property index is used + automatically when available. + [#4460](https://github.com/memgraph/memgraph/pull/4460) +- `--coordinator-id=0` is now a valid coordinator id (previously rejected). + [#4483](https://github.com/memgraph/memgraph/pull/4483) +- `CREATE RANGE INDEX FOR ... ON ...` now works for nodes and relationships and + creates Memgraph’s usual property index. + [#4486](https://github.com/memgraph/memgraph/pull/4486) +- [SSO authentication on high availability + coordinators](/clustering/high-availability/coordinator-authentication) + (Enterprise). Coordinators authenticate OIDC, SAML and Kerberos connections + against a Raft-replicated set of roles and enforce two privileges, + `COORDINATOR_READ` and `COORDINATOR_WRITE`. Manage them with `CREATE ROLE`, + `DROP ROLE`, `SHOW ROLES`, `GRANT` / `REVOKE COORDINATOR_READ | + COORDINATOR_WRITE | ALL PRIVILEGES`, and `SHOW PRIVILEGES FOR ROLE` on any + coordinator — follower queries are forwarded to the leader. Basic + authentication stays open until a `COORDINATOR_WRITE` role exists, so no + coordinator restart is needed to roll SSO out, and it reopens if the license + becomes invalid so a coordinator can never lock every session out. + [#4399](https://github.com/memgraph/memgraph/pull/4399) +- `SHOW VERSION` can now be run on coordinators, alongside the other read-only + introspection queries such as `SHOW CONFIG` and `SHOW BUILD INFO`. It + previously failed with `Coordinator can run only coordinator queries!`. On + coordinators it is a `COORDINATOR_READ` query, so an SSO role with only + `COORDINATOR_READ` granted can run it. + [#4535](https://github.com/memgraph/memgraph/pull/4535) +- Added the `SHOW ROUTING TABLE` query, which shows the routing table a + coordinator hands out to `bolt+routing` clients as `WRITE`, `READ` and `ROUTE` + rows. It can only be run on a coordinator, is always answered from the + leader's state, and returns an empty table if the leader cannot be contacted. + [#4502](https://github.com/memgraph/memgraph/pull/4502) +- Added `TERMINATE TRANSACTIONS "*"`, which terminates every transaction the + caller is authorized to terminate across all databases, instead of naming ids + one by one. The caller's own transaction is skipped, transactions it may not + terminate are omitted from the result, and rows come back ordered by + ascending transaction id. The wildcard cannot be combined with ids. + [#4534](https://github.com/memgraph/memgraph/pull/4534) +- A data instance in a high availability cluster can now use the + `IN_MEMORY_ANALYTICAL` storage mode for a bulk import: unregister the + replicas, switch to analytical, import, switch back to + `IN_MEMORY_TRANSACTIONAL` and register the replicas again. The switch back + writes a snapshot of the imported data, archives the superseded durability + files and restarts WAL numbering, and replica recovery detects that the + imported data exists in no WAL file, so a re-registered replica catches up + from that snapshot without wiping its data directory. Starting a data instance + with `--storage-mode=IN_MEMORY_ANALYTICAL` remains forbidden. See [bulk import + in analytical mode](/clustering/high-availability/analytical-import). + [#4510](https://github.com/memgraph/memgraph/pull/4510) + +### Lab v3.13.0 - September 9th, 2026 + + + +## Previous releases + ### Memgraph v3.12.0 - July 15th, 2026 {

⚠️ Breaking changes

} @@ -196,8 +431,6 @@ guide. -## Previous releases - ### Memgraph v3.11.0 - June 17th, 2026 {

⚠️ Breaking changes

} @@ -1102,7 +1335,7 @@ the `username()` and `roles()` built-in functions. These functions allow users to programmatically retrieve current authentication details, simplifying auditing and the development of dynamic, role-based logic within queries. [#3563](https://github.com/memgraph/memgraph/pull/3563) -- Added `--storage-enable-backup-dir` flag to control whether the `.old` +- Added `--storage-backup-dir-enabled` flag to control whether the `.old` directory is used for storing old durability files during potential data loss scenarios. When disabled, old durability files are deleted instead of backed up. [#3631](https://github.com/memgraph/memgraph/pull/3631)