Summary
Substrait-explain should support three output formats:
- A
verbose output format that shows every last detail of the protobuf. Function signatures and and anchors always shown in the Plan section; emit always shown.
- A
full output mode that elides redundant information, but that includes all the information in the protobuf. A valid substrait plan should be convertible to the 'full' substrait-explain output format (text), and then back to a protobuf, and be exactly the same (possibly with the exception of set / unset defaults in Protobuf). In this case, emit would still have to be shown; function signatures and anchors would only need to be shown in the Plan section if not unique in the Extensions section.
- A
simple output format, that shows all the semantics of the plan, but not necessarily every last detail of its encoding. Function signatures and anchors would be shown as in the full mode, but emit would not - emits can be shown by the order of expression, rather than their numerics (so DIRECT may be indistinguishable from the 0..N ordering, which are semantically equivalent); etc.
Motivation
verbose is useful as a 'canonical' representation- Both
full and simple are useful for inspection: simple should be easier to understand the logic of the plan, for debugging planners / execution; full can be useful for understanding the Substrait itself and what was sent over the wire.
Summary
Substrait-explain should support three output formats:
verboseoutput format that shows every last detail of the protobuf. Function signatures and and anchors always shown in thePlansection;emitalways shown.fulloutput mode that elides redundant information, but that includes all the information in the protobuf. A valid substrait plan should be convertible to the 'full' substrait-explain output format (text), and then back to a protobuf, and be exactly the same (possibly with the exception of set / unset defaults in Protobuf). In this case,emitwould still have to be shown; function signatures and anchors would only need to be shown in thePlansection if not unique in theExtensionssection.simpleoutput format, that shows all the semantics of the plan, but not necessarily every last detail of its encoding. Function signatures and anchors would be shown as in thefullmode, butemitwould not - emits can be shown by the order of expression, rather than their numerics (so DIRECT may be indistinguishable from the 0..N ordering, which are semantically equivalent); etc.Motivation
verboseis useful as a 'canonical' representationfullandsimpleare useful for inspection:simpleshould be easier to understand the logic of the plan, for debugging planners / execution;fullcan be useful for understanding the Substrait itself and what was sent over the wire.