diff --git a/docs/development/usability-plan.md b/docs/development/usability-plan.md index c9854f32..e14a0481 100644 --- a/docs/development/usability-plan.md +++ b/docs/development/usability-plan.md @@ -223,26 +223,26 @@ visible. No modal dialog opens for a successful action. ### Phase 1: Shared layer roles -- [ ] Add layer roles to the data manager: `geology`, `geology_unit_field`, +- [x] Add layer roles to the data manager: `geology`, `geology_unit_field`, `fault_traces`, `structure`, `basal_contacts`, `dem`. Save them with the application state. -- [ ] Add a signal or observer event when a role changes. -- [ ] Make the "Load Data" widgets write to the roles. -- [ ] Make each map2loop dialog read the roles as its default values. The user +- [x] Add a signal or observer event when a role changes. +- [x] Make the "Load Data" widgets write to the roles. +- [x] Make each map2loop dialog read the roles as its default values. The user can still change the value in the dialog. -- [ ] When Basal Contacts makes a new layer, set it as the `basal_contacts` +- [x] When Basal Contacts makes a new layer, set it as the `basal_contacts` role. -- [ ] The stratigraphic column "geology layer" pickers use the `geology` role. -- [ ] Add the contacts source setting: "Calculate from geology polygons" or +- [x] The stratigraphic column "geology layer" pickers use the `geology` role. +- [x] Add the contacts source setting: "Calculate from geology polygons" or "Use a contacts layer". Save it with the application state. -- [ ] Add a derived-data record to the data manager. For each result (basal +- [x] Add a derived-data record to the data manager. For each result (basal contacts, calculated thickness, styled fields), keep a hash of its inputs and a status: current or out of date. -- [ ] Compare the hash when the column, a layer role or a tool setting changes. +- [x] Compare the hash when the column, a layer role or a tool setting changes. Send an event when a status changes. -- [ ] Record the source of each unit thickness: "typed" or "calculated". A +- [x] Record the source of each unit thickness: "typed" or "calculated". A thickness that the user types in the column list is "typed". -- [ ] In step 2 (until phase 3, in the Stratigraphic Column tab), show the +- [x] In step 2 (until phase 3, in the Stratigraphic Column tab), show the out-of-date results and an **Update** button. Files: `main/data_manager.py`, diff --git a/docs/usage/interface.md b/docs/usage/interface.md index e9c1011d..53642bd1 100644 --- a/docs/usage/interface.md +++ b/docs/usage/interface.md @@ -29,6 +29,15 @@ Two layers can be used to constrain the stratigraphy of the model: ![Stratigraphic Layer](../static/stratigraphic_layer.png) +### Shared layers +You select each layer one time. The plugin keeps the geology layer, the unit name field, the fault traces, the structure layer, the basal contacts and the DEM as shared layers. The map2loop tools (Basal Contacts, Thickness Calculator, Sorter, Sampler and Paint Stratigraphic Order) show these layers as their default values. You can select another layer in a tool for one run. + +The geology layer is the layer that you select in the Stratigraphic Column tab. If you select a layer in one of the tools and no geology layer is set, the plugin uses that layer for the other tools. The plugin saves the shared layers with the application state. + +The **Source** setting in the basal contacts group selects where the basal contacts come from: +- **Calculate from geology polygons** (default). The Basal Contacts tool extracts the contacts from the geology layer and the stratigraphic column. When it finishes, the new layer becomes the basal contacts layer of the model. +- **Use a contacts layer**. Your own layer is an input. The plugin does not change it, and the Thickness Calculator uses it. + ## Stratigraphic Column The stratigraphic column defines the order of the contacts and any unconformable relationships between them. The column is defined by a list of units - these units are ordered from oldest at the bottom to youngest at the top. Unconformities can be inserted between units to define an unconformable relationship. The thicknesses define the true thickness of each unit and are used to parameterise the interpolation. The unit names should match the names of the contacts in the basal contacts layer. Units without basal contacts can be included in the stratigraphic column but will not be constrained by any data. @@ -37,6 +46,11 @@ The stratigraphic column can be initialised from the basal contacts layer by cli ![Stratigraphic Column](../static/stratigraphic_column_04.png) +### Out-of-date results +The basal contacts, the calculated thicknesses and the `strat_order` field on a map layer depend on the stratigraphic column and on the layers and settings of the tools that made them. When one of these inputs changes, the Stratigraphic Column tab shows a message, for example "Basal contacts are out of date (the order of the units changed)", with an **Update** button. The button runs the tool again with the settings of the project. The plugin does not calculate again after each change, so you can move many units first. If you move a unit and then move it back, the results stay current. + +The plugin records if the thickness of a unit was typed or calculated. The Thickness Calculator does not replace a thickness that you typed. To use a calculated value, set the thickness of the unit to 0 first. + ## Fault topology relationships diff --git a/loopstructural/gui/map2loop_tools/basal_contacts_widget.py b/loopstructural/gui/map2loop_tools/basal_contacts_widget.py index 91a4fbb1..28fc3225 100644 --- a/loopstructural/gui/map2loop_tools/basal_contacts_widget.py +++ b/loopstructural/gui/map2loop_tools/basal_contacts_widget.py @@ -7,12 +7,14 @@ from qgis.PyQt.QtCore import pyqtSignal from qgis.PyQt.QtWidgets import QMessageBox, QWidget +from ...main import derived_data, layer_roles from ...main.helpers import ColumnMatcher, get_layer_names from ...main.m2l_api import extract_basal_contacts from ...main.vectorLayerWrapper import addGeoDataFrameToproject from ..background_task import finish_background_task, start_background_task from ..compatibility import configure_layer_combo from ..messages import push_success, push_warning +from .layer_defaults import adopt_geology_role, apply_layer_role_defaults class BasalContactsWidget(QWidget): @@ -63,6 +65,15 @@ def __init__(self, parent=None, data_manager=None, debug_manager=None): # Set up field combo boxes self._setup_field_combo_boxes() self._restore_selection() + # The layers of the shared roles are the defaults of the tool + apply_layer_role_defaults( + self.data_manager, + { + layer_roles.GEOLOGY: self.geologyLayerComboBox, + layer_roles.FAULT_TRACES: self.faultsLayerComboBox, + }, + unit_field_combo=self.unitNameFieldComboBox, + ) def set_debug_manager(self, debug_manager): """Attach a debug manager instance.""" @@ -164,6 +175,8 @@ def _restore_selection(self): self.unitNameFieldComboBox.setField(field) if units := settings.get('basal_override_units'): self.basalOverrideUnitsLineEdit.setText(', '.join(units)) + if units := settings.get('ignore_units'): + self.ignoreUnitsLineEdit.setText(', '.join(units)) def _persist_selection(self): """Persist current selections into data manager.""" @@ -182,6 +195,7 @@ def _persist_selection(self): ), 'unit_name_field': self.unitNameFieldComboBox.currentField(), 'basal_override_units': self._units_from_line_edit(self.basalOverrideUnitsLineEdit), + 'ignore_units': self._units_from_line_edit(self.ignoreUnitsLineEdit), } self.data_manager.set_widget_settings('basal_contacts_widget', settings) @@ -229,6 +243,23 @@ def _run_extractor(self): target = self._make_extract_contacts_target() + geology = self.geologyLayerComboBox.currentLayer() + unit_name_field = self.unitNameFieldComboBox.currentField() + adopt_geology_role(self.data_manager, geology=geology, unit_field=unit_name_field) + # The inputs of this run, read before the run starts. A change during + # the run makes the result out of date. + self._run_inputs = ( + self.data_manager.basal_contacts_inputs( + geology=geology, + unit_field=unit_name_field, + faults=self.faultsLayerComboBox.currentLayer(), + ignore_units=self._units_from_line_edit(self.ignoreUnitsLineEdit), + override_units=self._units_from_line_edit(self.basalOverrideUnitsLineEdit), + ) + if self.data_manager + else None + ) + self.setEnabled(False) self._extractor_thread, self._extractor_worker, self._extractor_progress = ( start_background_task( @@ -268,6 +299,7 @@ def _on_extractor_finished(self, payload): self.data_manager.apply_stratigraphic_colours_to_layer( basal_layer, 'basal_unit' ) + self._use_new_contacts(basal_layer) else: contact_type = None if contact_type is None: @@ -290,6 +322,16 @@ def _on_extractor_finished(self, payload): else: self.task_failed.emit() + def _use_new_contacts(self, basal_layer): + """Make the new layer the basal contacts of the model, and record the run. + + With the contacts source "Use a contacts layer", the contacts layer + of the user is an input and does not change. + """ + if self.data_manager.layer_roles.contacts_source == layer_roles.CONTACTS_FROM_GEOLOGY: + self.data_manager.set_basal_contacts(basal_layer, unitname_field='basal_unit') + self.data_manager.derived.record(derived_data.BASAL_CONTACTS, inputs=self._run_inputs) + def _on_extractor_error(self, traceback_text): finish_background_task( self._extractor_thread, self._extractor_worker, self._extractor_progress diff --git a/loopstructural/gui/map2loop_tools/layer_defaults.py b/loopstructural/gui/map2loop_tools/layer_defaults.py new file mode 100644 index 00000000..eed4a85b --- /dev/null +++ b/loopstructural/gui/map2loop_tools/layer_defaults.py @@ -0,0 +1,57 @@ +"""Set the default layers of a map2loop tool from the shared layer roles. + +The user selects a layer for a role one time. Each tool shows that layer as +its default. The user can select another layer in the tool for one run. +""" + +from loopstructural.main import layer_roles + + +def apply_layer_role_defaults(data_manager, combos, *, unit_field_combo=None): + """Select the layer of each role in the layer combo boxes. + + Parameters + ---------- + data_manager : ModellingDataManager or None + Data manager that has the layer roles. + combos : dict + Role name -> `QgsMapLayerComboBox`. A role with no layer does not + change its combo box. + unit_field_combo : QgsFieldComboBox, optional + Combo box of the unit name field of the geology layer. It shows the + `geology_unit_field` role when the field is in the geology layer. + """ + if data_manager is None: + return + for role, combo in combos.items(): + layer = data_manager.layer_roles.get(role) + if layer is not None: + combo.setLayer(layer) + if unit_field_combo is not None: + geology = data_manager.layer_roles.get(layer_roles.GEOLOGY) + field = data_manager.layer_roles.get(layer_roles.GEOLOGY_UNIT_FIELD) + if geology is not None and field and geology.fields().indexFromName(field) >= 0: + if unit_field_combo.layer() is None or unit_field_combo.layer().id() != geology.id(): + unit_field_combo.setLayer(geology) + unit_field_combo.setField(field) + + +def adopt_geology_role(data_manager, *, geology=None, unit_field=None): + """Give the geology role the layer that a tool used, if it has no layer. + + The other roles follow the Load Data widgets, so a tool does not set + them. A role that has a layer does not change. The unit field is set only + when the geology layer of the role is the layer of the tool. + """ + if data_manager is None: + return + data_manager.adopt_layer_roles(geology=geology) + current = data_manager.layer_roles.get(layer_roles.GEOLOGY) + if ( + unit_field + and geology is not None + and current is not None + and current.id() == geology.id() + and data_manager.layer_roles.get(layer_roles.GEOLOGY_UNIT_FIELD) is None + ): + data_manager.layer_roles.set(layer_roles.GEOLOGY_UNIT_FIELD, unit_field) diff --git a/loopstructural/gui/map2loop_tools/paint_stratigraphic_order_widget.py b/loopstructural/gui/map2loop_tools/paint_stratigraphic_order_widget.py index 6b29e575..46de3fcd 100644 --- a/loopstructural/gui/map2loop_tools/paint_stratigraphic_order_widget.py +++ b/loopstructural/gui/map2loop_tools/paint_stratigraphic_order_widget.py @@ -6,8 +6,10 @@ from qgis.PyQt import uic from qgis.PyQt.QtWidgets import QMessageBox, QWidget +from ...main import derived_data, layer_roles from ...main.m2l_api import paint_stratigraphic_order from ..compatibility import configure_layer_combo +from .layer_defaults import adopt_geology_role, apply_layer_role_defaults class PaintStratigraphicOrderWidget(QWidget): @@ -69,6 +71,12 @@ def __init__(self, parent=None, data_manager=None, debug_manager=None): # Set up field combo boxes self._setup_field_combo_boxes() + # The geology layer of the shared role is the default of the tool + apply_layer_role_defaults( + self.data_manager, + {layer_roles.GEOLOGY: self.geologyLayerComboBox}, + unit_field_combo=self.unitNameFieldComboBox, + ) def set_debug_manager(self, debug_manager): """Attach a debug manager instance.""" @@ -159,6 +167,15 @@ def _run_painter(self): paint_stratigraphic_order( geology_layer, stratigraphic_order, unit_name_field, debug_manager=self._debug ) + if self.data_manager: + adopt_geology_role( + self.data_manager, geology=geology_layer, unit_field=unit_name_field + ) + self.data_manager.derived.record( + derived_data.STYLED_FIELDS, + inputs=self.data_manager.styled_fields_inputs(), + detail={'layer': geology_layer.name(), 'field': unit_name_field}, + ) # If requested, duplicate layer and apply style using selected colour ramp try: diff --git a/loopstructural/gui/map2loop_tools/sampler_widget.py b/loopstructural/gui/map2loop_tools/sampler_widget.py index 0f1035a9..db0d73df 100644 --- a/loopstructural/gui/map2loop_tools/sampler_widget.py +++ b/loopstructural/gui/map2loop_tools/sampler_widget.py @@ -9,8 +9,11 @@ from loopstructural.gui.background_task import finish_background_task, start_background_task from loopstructural.gui.compatibility import configure_layer_combo +from loopstructural.main import layer_roles from loopstructural.toolbelt.preferences import PlgOptionsManager +from .layer_defaults import apply_layer_role_defaults + class SamplerWidget(QWidget): """Widget for configuring and running the sampler. @@ -55,6 +58,14 @@ def __init__(self, parent=None, data_manager=None, debug_manager=None): QgsMapLayerProxyModel.Filter.LineLayer | QgsMapLayerProxyModel.Filter.PointLayer, ) # spatialData can be any type, leave default + # The layers of the shared roles are the defaults of the tool + apply_layer_role_defaults( + self.data_manager, + { + layer_roles.DEM: self.dtmLayerComboBox, + layer_roles.GEOLOGY: self.geologyLayerComboBox, + }, + ) # Initialize sampler types self.sampler_types = ["Decimator", "Spacing"] diff --git a/loopstructural/gui/map2loop_tools/sorter_widget.py b/loopstructural/gui/map2loop_tools/sorter_widget.py index d97fa9a8..694bc9a9 100644 --- a/loopstructural/gui/map2loop_tools/sorter_widget.py +++ b/loopstructural/gui/map2loop_tools/sorter_widget.py @@ -9,6 +9,7 @@ from loopstructural.gui.background_task import finish_background_task, start_background_task from loopstructural.gui.compatibility import configure_layer_combo +from loopstructural.main import layer_roles from loopstructural.main.helpers import get_layer_names from loopstructural.main.m2l_api import ( PARAMETERS_DICTIONARY, @@ -17,6 +18,8 @@ ) from loopstructural.toolbelt.preferences import PlgOptionsManager +from .layer_defaults import adopt_geology_role, apply_layer_role_defaults + class SorterWidget(QWidget): """Widget for configuring and running the automatic stratigraphic sorter. @@ -80,6 +83,16 @@ def __init__(self, parent=None, data_manager=None, debug_manager=None): # Set up field combo boxes self._setup_field_combo_boxes() self._restore_selection() + # The layers of the shared roles are the defaults of the tool + apply_layer_role_defaults( + self.data_manager, + { + layer_roles.GEOLOGY: self.geologyLayerComboBox, + layer_roles.STRUCTURE: self.structureLayerComboBox, + layer_roles.DEM: self.dtmLayerComboBox, + }, + unit_field_combo=self.unitNameFieldComboBox, + ) # Initial state update self._on_algorithm_changed() @@ -365,6 +378,11 @@ def _run_sorter(self): if not self.geologyLayerComboBox.currentLayer(): QMessageBox.warning(self, "Missing Input", "Please select a geology layer.") return False + adopt_geology_role( + self.data_manager, + geology=self.geologyLayerComboBox.currentLayer(), + unit_field=self.unitNameFieldComboBox.currentField(), + ) algorithm_index = self.sortingAlgorithmComboBox.currentIndex() algorithm_name = self.sorting_algorithms[algorithm_index] diff --git a/loopstructural/gui/map2loop_tools/thickness_calculator_widget.py b/loopstructural/gui/map2loop_tools/thickness_calculator_widget.py index b77013f5..51b6a112 100644 --- a/loopstructural/gui/map2loop_tools/thickness_calculator_widget.py +++ b/loopstructural/gui/map2loop_tools/thickness_calculator_widget.py @@ -13,8 +13,10 @@ from loopstructural.gui.messages import push_success from loopstructural.toolbelt.preferences import PlgOptionsManager +from ...main import derived_data, layer_roles from ...main.helpers import ColumnMatcher, get_layer_names from ...main.vectorLayerWrapper import addGeoDataFrameToproject +from .layer_defaults import adopt_geology_role, apply_layer_role_defaults class ThicknessCalculatorWidget(QWidget): @@ -77,6 +79,7 @@ def __init__(self, parent=None, data_manager=None, debug_manager=None): # Set up field combo boxes self._setup_field_combo_boxes() self._restore_selection() + self._apply_role_defaults() # Initial state update self._on_calculator_type_changed() @@ -85,6 +88,33 @@ def set_debug_manager(self, debug_manager): """Attach a debug manager instance.""" self._debug = debug_manager + def _apply_role_defaults(self): + """Show the layers of the shared roles as the defaults of the tool.""" + if not self.data_manager: + return + apply_layer_role_defaults( + self.data_manager, + { + layer_roles.GEOLOGY: self.geologyLayerComboBox, + layer_roles.STRUCTURE: self.structureLayerComboBox, + layer_roles.DEM: self.dtmLayerComboBox, + }, + unit_field_combo=self.unitNameFieldComboBox, + ) + if self.data_manager.layer_roles.contacts_source == layer_roles.CONTACTS_FROM_LAYER: + contacts = self.data_manager.layer_roles.get(layer_roles.BASAL_CONTACTS) + if contacts is not None: + self.basalContactsComboBox.setLayer(contacts) + config = self.data_manager.get_basal_contacts() or {} + field = config.get('unitname_field') + if field: + self.basalUnitNameFieldComboBox.setField(field) + else: + # The contacts are calculated from the geology layer and the + # column when no layer is selected. A user can select a layer + # for one run. + self.basalContactsComboBox.setLayer(None) + def _export_layer_for_debug(self, layer, name_prefix: str): # Prefer using DebugManager.export_layer if available try: @@ -305,6 +335,7 @@ def _persist_selection(self): if self.crossSectionLayerComboBox.currentLayer() else None ), + 'calculator_type': self.calculatorTypeComboBox.currentText(), 'calculator_type_index': self.calculatorTypeComboBox.currentIndex(), 'orientation_type_index': self.orientationTypeComboBox.currentIndex(), 'max_line_length': self.maxLineLengthSpinBox.value(), @@ -371,6 +402,26 @@ def _run_calculator(self): # Prepare parameters params = self.get_parameters() + adopt_geology_role( + self.data_manager, + geology=self.geologyLayerComboBox.currentLayer(), + unit_field=self.unitNameFieldComboBox.currentField(), + ) + # The inputs of this run, read before the run starts. A change during + # the run makes the result out of date. + self._run_inputs = ( + self.data_manager.thickness_inputs( + geology=self.geologyLayerComboBox.currentLayer(), + unit_field=self.unitNameFieldComboBox.currentField(), + contacts_layer=self.basalContactsComboBox.currentLayer(), + calculator_type=calculator_type, + structure=self.structureLayerComboBox.currentLayer(), + cross_sections=self.crossSectionLayerComboBox.currentLayer(), + ) + if self.data_manager + else None + ) + def target(progress_callback): return calculate_thickness(updater=progress_callback, **params) @@ -410,69 +461,25 @@ def _on_calculator_finished(self, result): thicknesses = result.get('thicknesses') lines = result.get('lines') location_tracking = result.get('location_tracking') - # If thicknesses were calculated, update the stratigraphic column units - try: - if thicknesses is not None and getattr(self, 'data_manager', None): - # Prefer median thickness if available, fallback to mean - thickness_col = ( - 'ThicknessMedian' - if 'ThicknessMedian' in getattr(thicknesses, 'columns', []) - else ( - 'ThicknessMean' - if 'ThicknessMean' in getattr(thicknesses, 'columns', []) - else None - ) - ) - if thickness_col is not None: - for _, row in thicknesses.iterrows(): - unit_name = row.get('name') or row.get('UNITNAME') - if not unit_name: - continue - try: - value = row.get(thickness_col) - except Exception: - value = None - # Skip invalid values (e.g. -1 means not calculated) - try: - is_invalid = pd.isna(value) or float(value) == -1 - except Exception: - is_invalid = value is None - if is_invalid: - continue - # Find unit in stratigraphic column and update thickness - try: - strat_col = self.data_manager.get_stratigraphic_column() - unit = strat_col.get_unit_by_name(unit_name) - if unit is not None: - unit.thickness = float(value) - except Exception as err: - # Log but don't fail the widget - try: - if getattr(self, '_debug', None): - self._debug.plugin.log( - message=f"Failed to update stratigraphic unit thickness for {unit_name}: {err}", - log_level=2, - ) - except Exception: - pass - # Notify any stratigraphic column callbacks - try: - if getattr(self.data_manager, 'stratigraphic_column_callback', None): - self.data_manager.stratigraphic_column_callback() - except Exception: - pass - except Exception: - pass + skipped = [] + # If thicknesses were calculated, update the stratigraphic column units. + # A thickness that the user typed does not change. + if thicknesses is not None and self.data_manager: + skipped = self._apply_thicknesses(thicknesses) + if self.data_manager and getattr(self, '_run_inputs', None) is not None: + self.data_manager.derived.record(derived_data.THICKNESS, inputs=self._run_inputs) # if thicknesses is not None: # addGeoDataFrameToproject(thicknesses, "Thickness Results") if lines is not None: addGeoDataFrameToproject(lines, "Thickness Lines") if location_tracking is not None: addGeoDataFrameToproject(location_tracking, "Thickness Locations") - push_success( - "Thickness Calculator", - "Thickness calculation completed successfully and added to project.", - ) + message = "Thickness calculation completed successfully and added to project." + if skipped: + message += ( + "\n\nThese units keep the thickness that you typed: " + ", ".join(skipped) + "." + ) + push_success("Thickness Calculator", message) self.task_succeeded.emit() return @@ -488,6 +495,35 @@ def _on_calculator_finished(self, result): push_success("Thickness Calculator", f"Thickness calculation completed: {result}") self.task_succeeded.emit() + def _apply_thicknesses(self, thicknesses): + """Set the calculated thicknesses on the units of the column. + + Returns the names of the units that keep a typed thickness. + """ + # Prefer median thickness if available, fallback to mean + columns = getattr(thicknesses, 'columns', []) + if 'ThicknessMedian' in columns: + thickness_col = 'ThicknessMedian' + elif 'ThicknessMean' in columns: + thickness_col = 'ThicknessMean' + else: + return [] + values = {} + for _, row in thicknesses.iterrows(): + unit_name = row.get('name') or row.get('UNITNAME') + if not unit_name: + continue + value = row.get(thickness_col) + # Skip invalid values (e.g. -1 means not calculated) + try: + if pd.isna(value) or float(value) == -1: + continue + except (TypeError, ValueError): + continue + values[unit_name] = float(value) + _, skipped = self.data_manager.apply_calculated_thicknesses(values) + return skipped + def _on_calculator_error(self, traceback_text): finish_background_task( self._calculator_thread, self._calculator_worker, self._calculator_progress diff --git a/loopstructural/gui/modelling/model_definition/stratigraphic_layers.py b/loopstructural/gui/modelling/model_definition/stratigraphic_layers.py index a6c4c135..87980594 100644 --- a/loopstructural/gui/modelling/model_definition/stratigraphic_layers.py +++ b/loopstructural/gui/modelling/model_definition/stratigraphic_layers.py @@ -3,9 +3,10 @@ from qgis.core import QgsMapLayerProxyModel, QgsWkbTypes from qgis.PyQt import uic from qgis.PyQt.QtCore import Qt -from qgis.PyQt.QtWidgets import QWidget +from qgis.PyQt.QtWidgets import QComboBox, QWidget from ...compatibility import configure_layer_combo +from ....main import layer_roles from ....main.helpers import ColumnMatcher, get_layer_names @@ -24,6 +25,7 @@ def __init__(self, parent=None, data_manager=None): ) # Structural data can only be points configure_layer_combo(self.structuralDataLayer, QgsMapLayerProxyModel.Filter.PointLayer) + self._add_contacts_source_combo() self.basalContactsLayer.layerChanged.connect(self.onBasalContactsChanged) self.structuralDataLayer.layerChanged.connect(self.onStructuralDataLayerChanged) self.unitNameField.fieldChanged.connect(self.onUnitFieldChanged) @@ -56,6 +58,39 @@ def __init__(self, parent=None, data_manager=None): self._guess_layers_and_fields() self._restore_selection() + def _add_contacts_source_combo(self): + """Add the choice of where the basal contacts come from.""" + self.contactsSourceComboBox = QComboBox(self) + self.contactsSourceComboBox.addItem( + "Calculate from geology polygons", layer_roles.CONTACTS_FROM_GEOLOGY + ) + self.contactsSourceComboBox.addItem("Use a contacts layer", layer_roles.CONTACTS_FROM_LAYER) + self.contactsSourceComboBox.setToolTip( + "Calculate from geology polygons: the plugin extracts the basal contacts from " + "the geology layer and the stratigraphic column. The layer that it adds to the " + "project is for display.\n" + "Use a contacts layer: your own layer is an input. The plugin does not change it." + ) + self.formLayout_basalContacts.insertRow(0, "Source", self.contactsSourceComboBox) + self._show_contacts_source(self.data_manager.layer_roles.contacts_source) + self.contactsSourceComboBox.currentIndexChanged.connect(self._on_contacts_source_selected) + self.data_manager.layer_roles.attach(self._on_layer_role_changed) + + def _show_contacts_source(self, source): + index = self.contactsSourceComboBox.findData(source) + if index >= 0 and index != self.contactsSourceComboBox.currentIndex(): + self.contactsSourceComboBox.blockSignals(True) + self.contactsSourceComboBox.setCurrentIndex(index) + self.contactsSourceComboBox.blockSignals(False) + + def _on_contacts_source_selected(self, index): + self.data_manager.layer_roles.contacts_source = self.contactsSourceComboBox.itemData(index) + + def _on_layer_role_changed(self, role, value): + """Show a contacts source that was set by a loaded state.""" + if role == 'contacts_source': + self._show_contacts_source(value) + def enableBasalContactsZCheckBox(self, enable): self.useBasalContactsZCoordinatesCheckBox.setEnabled(enable) if enable: diff --git a/loopstructural/gui/modelling/stratigraphic_column/stratigraphic_column.py b/loopstructural/gui/modelling/stratigraphic_column/stratigraphic_column.py index 48faf685..a4a073cd 100644 --- a/loopstructural/gui/modelling/stratigraphic_column/stratigraphic_column.py +++ b/loopstructural/gui/modelling/stratigraphic_column/stratigraphic_column.py @@ -12,6 +12,7 @@ QListWidget, QListWidgetItem, QMessageBox, + QPushButton, QToolButton, QVBoxLayout, QWidget, @@ -21,6 +22,7 @@ from loopstructural.gui.compatibility import configure_layer_combo from loopstructural.gui.messages import push_success from loopstructural.gui.modelling.stratigraphic_column.unconformity import UnconformityWidget +from loopstructural.main import derived_data, layer_roles from loopstructural.main.helpers import ColumnMatcher, get_layer_names from .init_from_field_dialog import InitFromLayerFieldDialog @@ -68,6 +70,12 @@ def __init__(self, parent=None, data_manager=None): # layer/field is selected, so the name-match warning is skipped. self._known_unit_names = None + # True while the pickers follow a change of the shared geology role, + # so that the change is not written back + self._syncing_roles = False + # The hidden tool widget that an Update button runs + self._update_widget = None + # Main list widget self.unitList = QListWidget() self.unitList.setDragDropMode(QAbstractItemView.DragDropMode.InternalMove) @@ -173,9 +181,13 @@ def __init__(self, parent=None, data_manager=None): ageRow.addWidget(applyAgeButton) layout.addLayout(ageRow) + self._add_derived_data_panel(layout) + self._guess_units_layer() self._restore_units_layer_selection() + self._sync_units_layer_from_roles() self._known_unit_names = self._get_known_unit_names() + self.data_manager.layer_roles.attach(self._on_layer_role_changed) # Update display from data manager self.update_display() @@ -195,6 +207,152 @@ def _on_fault_topology_changed(self, observable, event, *args, **kwargs): # Widget was deleted pass + # Words for the inputs that changed, in the message of an out-of-date result + _INPUT_WORDS = { + 'unit_order': 'the order of the units', + 'geology': 'the geology layer', + 'unit_field': 'the unit name field', + 'faults': 'the faults layer', + 'ignore_units': 'the ignored units', + 'override_units': 'the basal override units', + 'contacts': 'the basal contacts settings', + 'calculator_type': 'the calculator type', + 'structure': 'the structure layer', + 'cross_sections': 'the cross-sections layer', + 'thicknesses': 'a unit thickness', + } + + def _add_derived_data_panel(self, layout): + """Add the list of the derived results that are out of date. + + Each result has a line of text and an Update button. The panel is + hidden when all results are current. The column does not calculate + again after each change, because the extraction is slow and the user + often moves many rows. + """ + self.derivedDataPanel = QWidget(self) + panel_layout = QVBoxLayout(self.derivedDataPanel) + panel_layout.setContentsMargins(0, 0, 0, 0) + self._derived_rows = {} + for name in ( + derived_data.BASAL_CONTACTS, + derived_data.THICKNESS, + derived_data.STYLED_FIELDS, + ): + row = QWidget(self.derivedDataPanel) + row_layout = QHBoxLayout(row) + row_layout.setContentsMargins(0, 0, 0, 0) + label = QLabel(row) + label.setWordWrap(True) + button = QPushButton("Update", row) + button.setIcon(QgsApplication.getThemeIcon("mActionRefresh.svg")) + button.clicked.connect(lambda _checked=False, n=name: self._update_derived(n)) + row_layout.addWidget(label, 1) + row_layout.addWidget(button) + panel_layout.addWidget(row) + self._derived_rows[name] = (row, label, button) + layout.addWidget(self.derivedDataPanel) + self.data_manager.derived.attach(self._on_derived_status_changed) + self._refresh_derived_panel() + + def _on_derived_status_changed(self, name, status): + try: + self._refresh_derived_panel() + except RuntimeError: + # the widget was deleted + pass + + def _refresh_derived_panel(self): + """Show a line for each result that is out of date.""" + derived = self.data_manager.derived + any_visible = False + for name, (row, label, _button) in self._derived_rows.items(): + visible = derived.is_out_of_date(name) + # With "Use a contacts layer", the contacts are an input of the + # user and not a result of the plugin. + if ( + name == derived_data.BASAL_CONTACTS + and self.data_manager.layer_roles.contacts_source == layer_roles.CONTACTS_FROM_LAYER + ): + visible = False + row.setVisible(visible) + if visible: + any_visible = True + words = [ + self._INPUT_WORDS.get(key, key.replace('_', ' ')) + for key in derived.changed_inputs(name) + ] + reason = f" ({', '.join(words)} changed)" if words else "" + label.setText(f"{derived_data.DESCRIPTIONS[name]} are out of date{reason}.") + self.derivedDataPanel.setVisible(any_visible) + + def _update_derived(self, name): + """Calculate an out-of-date result again with the settings of the project.""" + if self._update_widget is not None: + # a calculation is running + return + if name == derived_data.STYLED_FIELDS: + self._update_styled_fields() + return + if name == derived_data.BASAL_CONTACTS: + from loopstructural.gui.map2loop_tools.basal_contacts_widget import ( + BasalContactsWidget as tool_class, + ) + + run_method = '_run_extractor' + else: + from loopstructural.gui.map2loop_tools.thickness_calculator_widget import ( + ThicknessCalculatorWidget as tool_class, + ) + + run_method = '_run_calculator' + # The tool widget takes its settings from the layer roles and from the + # last run. It is not shown. Its progress dialog is shown. + widget = tool_class( + self, data_manager=self.data_manager, debug_manager=self.data_manager.debug_manager + ) + widget.hide() + widget.task_succeeded.connect(self._finish_update) + widget.task_failed.connect(self._finish_update) + self._update_widget = widget + self._set_update_buttons_enabled(False) + if getattr(widget, run_method)() is False: + # The tool did not start, for example no geology layer is selected. + # It showed the reason. + self._finish_update() + + def _update_styled_fields(self): + """Write the order of the column again to the layer that was styled.""" + detail = self.data_manager.derived.detail(derived_data.STYLED_FIELDS) + layer = self.data_manager.find_layer_by_name(detail.get('layer')) + field = detail.get('field') or self.data_manager.layer_roles.get( + layer_roles.GEOLOGY_UNIT_FIELD + ) + if layer is None or not field: + QMessageBox.warning( + self, + "Update Map Layer Fields", + "The layer that has the stratigraphic order fields is not in the project. " + "Use 'Apply Stratigraphic Age to Map Layer' to write them again.", + ) + return + if not self.data_manager.refresh_stratigraphic_order_field(layer, field): + QMessageBox.warning( + self, + "Update Map Layer Fields", + f"Could not write the stratigraphic order to layer '{layer.name()}'.", + ) + + def _finish_update(self): + widget, self._update_widget = self._update_widget, None + if widget is not None: + widget.deleteLater() + self._set_update_buttons_enabled(True) + + def _set_update_buttons_enabled(self, enabled): + for _row, _label, button in self._derived_rows.values(): + button.setEnabled(enabled) + def _make_tool_button(self, theme_icon_name: str, tooltip: str) -> QToolButton: """Build a small icon-only tool button using a QGIS theme icon, with the given tooltip standing in for the label text it no longer shows. @@ -413,13 +571,44 @@ def _persist_units_layer_selection(self): if not self.data_manager: return layer = self.unitsLayerComboBox.currentLayer() + field = self.unitsLayerFieldComboBox.currentField() self.data_manager.set_widget_settings( 'stratigraphic_column_widget', { 'units_layer': layer.name() if layer else None, - 'units_layer_field': self.unitsLayerFieldComboBox.currentField(), + 'units_layer_field': field, }, ) + # The geology layer is shared with the map2loop tools + if not self._syncing_roles: + self.data_manager.layer_roles.set(layer_roles.GEOLOGY, layer) + self.data_manager.layer_roles.set(layer_roles.GEOLOGY_UNIT_FIELD, field or None) + + def _sync_units_layer_from_roles(self): + """Show the geology layer and the unit name field of the shared roles.""" + roles = self.data_manager.layer_roles + layer = roles.get(layer_roles.GEOLOGY) + field = roles.get(layer_roles.GEOLOGY_UNIT_FIELD) + if layer is None: + return + self._syncing_roles = True + try: + if self.unitsLayerComboBox.currentLayer() != layer: + self.unitsLayerComboBox.setLayer(layer) + if field and layer.fields().indexFromName(field) >= 0: + self.unitsLayerFieldComboBox.setField(field) + finally: + self._syncing_roles = False + self._revalidate_unit_names() + + def _on_layer_role_changed(self, role, value): + """Follow a change of the geology role that did not come from this widget.""" + if role in (layer_roles.GEOLOGY, layer_roles.GEOLOGY_UNIT_FIELD): + try: + self._sync_units_layer_from_roles() + except RuntimeError: + # the widget was deleted + pass def _on_units_field_changed(self, _field_name): """Persist and re-validate when the unit-name field selection changes.""" @@ -763,6 +952,10 @@ def update_element(self, unit_widget): ) else: self.data_manager.clear_fault_boundary(unit_widget.uuid) + if not isinstance(unit_widget, UnconformityWidget): + # a thickness that the user typed must not be replaced by a + # calculated thickness + self.data_manager.note_thickness_edit(unit_data) self.data_manager._stratigraphic_column.update_element(unit_data) # Trigger callback to notify all listeners of the change if self.data_manager.stratigraphic_column_callback: diff --git a/loopstructural/main/data_manager.py b/loopstructural/main/data_manager.py index b2f2e3d6..64ae50c3 100644 --- a/loopstructural/main/data_manager.py +++ b/loopstructural/main/data_manager.py @@ -12,6 +12,7 @@ QgsCoordinateReferenceSystem, QgsCoordinateTransform, QgsGraduatedSymbolRenderer, + QgsMapLayer, QgsPointXY, QgsProject, QgsRectangle, @@ -26,7 +27,10 @@ from LoopStructural import FaultTopology, StratigraphicColumn +from . import derived_data, layer_roles from .data_types import BasalContactsConfig, FaultTracesConfig, StructuralOrientationsConfig +from .derived_data import DerivedData, ThicknessSources +from .layer_roles import LayerRoles from .m2l_api import paint_stratigraphic_order from .vectorLayerWrapper import qgsLayerToGeoDataFrame @@ -49,6 +53,22 @@ def _colour_to_qcolor(colour): __title__ = "LoopStructural" +# Marks an argument that the caller did not give, because None is a value +# for some of them (for example, no faults layer). +_UNSET = object() + + +def _layer_name(layer): + """Return the name of a layer, the same string, or None.""" + if layer is None or isinstance(layer, str): + return layer + try: + return layer.name() + except RuntimeError: + # the C++ layer was deleted + return None + + # number of cells in the bounding box grid (used for isosurfaces and # evaluation, not for the interpolation) DEFAULT_BOUNDING_BOX_NELEMENTS = 100_000 @@ -131,6 +151,19 @@ def __init__(self, *, project=None, mapCanvas=None, logger=None): # data was last read from them self._changed_layer_ids = set() self._layer_data_changed_callbacks = [] + # The layer that each role has (geology, fault traces, ...). The + # tools read these as their default layers. + self.layer_roles = LayerRoles(layer_resolver=self._find_any_layer) + self.layer_roles.attach(self._on_layer_role_changed) + # For each derived result, the inputs of the last run. See + # `derived_data` for how an out-of-date result is found. + self.derived = DerivedData() + self.derived.register(derived_data.BASAL_CONTACTS, self.basal_contacts_inputs) + self.derived.register(derived_data.THICKNESS, self.thickness_inputs) + self.derived.register(derived_data.STYLED_FIELDS, self.styled_fields_inputs) + # Records if the thickness of each unit was typed or calculated + self.thickness_sources = ThicknessSources() + self.debug_manager = None def onSaveProject(self): """Save project data.""" @@ -245,11 +278,230 @@ def set_stratigraphic_column_callback(self, callback): @property def stratigraphic_column_callback(self): def call_all(): + # Find the derived results that the change made out of date + # first, so that the widgets show the new status. + self.derived.refresh() for cb in self._stratigraphic_column_callbacks: cb() return call_all + def set_debug_manager(self, debug_manager): + """Set the debug manager for the tools that the data manager can start.""" + self.debug_manager = debug_manager + + def _on_layer_role_changed(self, role, value): + """A layer role changed: the derived results can be out of date.""" + self.derived.refresh() + + def get_layer_role(self, role): + """Get the layer (or the field name) that a role has, or None.""" + return self.layer_roles.get(role) + + def adopt_layer_roles(self, **roles): + """Set the roles that have no value. A role with a value does not change. + + A tool calls this when it runs, so that the layer that the user + selected in the tool becomes the default of the other tools. + """ + for role, value in roles.items(): + if value is not None and self.layer_roles.get(role) is None: + self.layer_roles.set(role, value) + + # -- derived data -------------------------------------------------- + + def _settings_layer_name(self, settings, key, role): + """Return the layer name that a tool used last, or the role value.""" + if key in settings: + return settings[key] + return _layer_name(self.layer_roles.get(role)) + + def basal_contacts_inputs( + self, + *, + geology=_UNSET, + unit_field=_UNSET, + faults=_UNSET, + ignore_units=_UNSET, + override_units=_UNSET, + ): + """Return the inputs of the basal contacts calculation. + + A value that is not given comes from the layer roles and from the + last settings of the Basal Contacts tool. A tool that runs gives its + own values, so the record is the inputs of the run. + """ + settings = self.get_widget_settings('basal_contacts_widget', {}) or {} + if geology is _UNSET: + geology = self.layer_roles.get(layer_roles.GEOLOGY) + if unit_field is _UNSET: + unit_field = self.layer_roles.get(layer_roles.GEOLOGY_UNIT_FIELD) + if faults is _UNSET: + faults = self._settings_layer_name(settings, 'faults_layer', layer_roles.FAULT_TRACES) + if ignore_units is _UNSET: + ignore_units = settings.get('ignore_units', []) + if override_units is _UNSET: + override_units = settings.get('basal_override_units', []) + return { + 'unit_order': self.get_stratigraphic_unit_names(), + 'geology': _layer_name(geology), + 'unit_field': unit_field, + 'faults': _layer_name(faults), + 'ignore_units': sorted(ignore_units or []), + 'override_units': sorted(override_units or []), + } + + def thickness_inputs( + self, + *, + geology=_UNSET, + unit_field=_UNSET, + contacts_layer=_UNSET, + calculator_type=_UNSET, + structure=_UNSET, + cross_sections=_UNSET, + ): + """Return the inputs of the thickness calculation. + + Without a contacts layer, the calculation makes the contacts from + the geology layer, so the inputs of the contacts are inputs of the + thickness. + """ + settings = self.get_widget_settings('thickness_calculator_widget', {}) or {} + if geology is _UNSET: + geology = self.layer_roles.get(layer_roles.GEOLOGY) + if unit_field is _UNSET: + unit_field = self.layer_roles.get(layer_roles.GEOLOGY_UNIT_FIELD) + if contacts_layer is _UNSET: + if self.layer_roles.contacts_source == layer_roles.CONTACTS_FROM_LAYER: + contacts_layer = self.layer_roles.get(layer_roles.BASAL_CONTACTS) + else: + contacts_layer = settings.get('basal_contacts_layer') + if calculator_type is _UNSET: + calculator_type = settings.get('calculator_type') + if structure is _UNSET: + structure = self._settings_layer_name( + settings, 'structure_layer', layer_roles.STRUCTURE + ) + if cross_sections is _UNSET: + cross_sections = settings.get('cross_sections_layer') + contacts_layer = _layer_name(contacts_layer) + if contacts_layer is not None: + contacts = {'layer': contacts_layer} + else: + contacts = { + 'calculated': self.basal_contacts_inputs(geology=geology, unit_field=unit_field) + } + return { + 'unit_order': self.get_stratigraphic_unit_names(), + 'geology': _layer_name(geology), + 'unit_field': unit_field, + 'contacts': contacts, + 'calculator_type': calculator_type, + 'structure': _layer_name(structure), + 'cross_sections': _layer_name(cross_sections), + } + + def styled_fields_inputs(self): + """Return the inputs of the `strat_order` / `strat_thickness` fields.""" + thicknesses = {} + for unit in self._stratigraphic_column.order: + if unit.element_type == StratigraphicColumnElementType.UNIT: + thickness = unit.thickness + thicknesses[unit.name] = ( + round(float(thickness), 6) if derived_data.thickness_is_set(thickness) else None + ) + return {'unit_order': self.get_stratigraphic_unit_names(), 'thicknesses': thicknesses} + + def refresh_stratigraphic_order_field(self, layer, field_name): + """Write the order of the column to the `strat_order` field of a layer again. + + The style of the layer does not change. + + Returns + ------- + bool + True if the field was written. + """ + unit_names = self.get_stratigraphic_unit_names() + if layer is None or not field_name or not unit_names: + return False + try: + paint_stratigraphic_order(layer, unit_names, field_name) + except Exception as err: + self.logger(message=f"Failed to write stratigraphic order onto layer: {err}") + return False + self.derived.record( + derived_data.STYLED_FIELDS, + inputs=self.styled_fields_inputs(), + detail={'layer': layer.name(), 'field': field_name}, + ) + return True + + def get_derived_status(self, name): + """Return `current`, `out_of_date` or `not_run` for a derived result.""" + return self.derived.status(name) + + # -- thickness ----------------------------------------------------- + + def get_thickness_source(self, unit_uuid): + """Return `typed`, `calculated` or None for the thickness of a unit.""" + return self.thickness_sources.get(unit_uuid) + + def note_thickness_edit(self, unit_data): + """Record that the user typed the thickness of a unit. + + The unit rows also send their value back when the data manager + sets it, so only a value that is not the same as the value in the + column is a user edit. The row shows two decimals. + """ + element = self._stratigraphic_column.get_element_by_uuid(unit_data.get('uuid')) + if element is None or element.element_type != StratigraphicColumnElementType.UNIT: + return + new = unit_data.get('thickness') + if not derived_data.thickness_is_set(new): + # the user cleared the value + self.thickness_sources.discard(element.uuid) + return + old = element.thickness + if derived_data.thickness_is_set(old) and abs(float(old) - float(new)) <= 0.005 + 1e-9: + return + self.thickness_sources.set(element.uuid, derived_data.TYPED) + + def apply_calculated_thicknesses(self, thicknesses): + """Set the thickness of units from a calculation. + + A thickness that the user typed does not change. + + Parameters + ---------- + thicknesses : dict + Unit name -> thickness. A value that is not a number above zero + (map2loop uses -1 for a unit with no result) is ignored. + + Returns + ------- + tuple of list + The names of the units that changed, and the names of the units + that kept their typed thickness. + """ + applied, skipped = [], [] + for name, value in thicknesses.items(): + if not derived_data.thickness_is_set(value): + continue + unit = self._stratigraphic_column.get_unit_by_name(name=name) + if unit is None: + continue + if not self.thickness_sources.can_overwrite(unit.uuid, unit.thickness): + skipped.append(name) + continue + unit.thickness = float(value) + self.thickness_sources.set(unit.uuid, derived_data.CALCULATED) + applied.append(name) + if applied: + self.stratigraphic_column_callback() + return applied, skipped + def set_dem_callback(self, callback): """Set the callback for when the DEM layer is updated.""" self.dem_callback = callback @@ -417,6 +669,7 @@ def set_elevation(self, elevation): def set_dem_layer(self, dem_layer): """Set the DEM layer to sample elevation from when `use_dem` is True.""" self.dem_layer = dem_layer + self.layer_roles.set(layer_roles.DEM, dem_layer) if dem_layer is None: self.logger( message="DEM layer is None, using 0.0 for elevation. Choose a valid layer or specify a constant value", @@ -439,6 +692,7 @@ def set_basal_contacts(self, basal_contacts, unitname_field=None, use_z_coordina 'unitname_field': unitname_field, 'use_z_coordinate': use_z_coordinate, } + self.layer_roles.set(layer_roles.BASAL_CONTACTS, basal_contacts) # self._unitname_field = unitname_field self.calculate_unique_basal_units() # if stratigraphic column is not empty, update contacts @@ -640,6 +894,11 @@ def apply_stratigraphic_age_to_layer(self, layer, field_name, ramp_name=None): message=f"Applied stratigraphic age field '{age_field_name}' and graduated " f"styling to layer '{layer.name()}'." ) + self.derived.record( + derived_data.STYLED_FIELDS, + inputs=self.styled_fields_inputs(), + detail={'layer': layer.name(), 'field': field_name}, + ) return True def get_stratigraphic_unit_names(self): @@ -1084,6 +1343,7 @@ def set_fault_trace_layer( 'fault_displacement_field': fault_displacement_field, 'use_z_coordinate': use_z_coordinate, } + self.layer_roles.set(layer_roles.FAULT_TRACES, fault_trace_layer) self.update_faults() if self.fault_traces_callback: self.fault_traces_callback(**self._fault_traces) @@ -1110,6 +1370,7 @@ def set_structural_orientations( self._structural_orientations['unitname_field'] = unitname_field self._structural_orientations['orientation_type'] = orientation_type self._structural_orientations['use_z_coordinate'] = use_z_coordinate + self.layer_roles.set(layer_roles.STRUCTURE, structural_orientations) if self.structural_orientations_callback: self.structural_orientations_callback(**self._structural_orientations) self.update_stratigraphy() @@ -1319,6 +1580,9 @@ def reset(self): self.fault_stratigraphy_adjacency = None self.feature_data = defaultdict(dict) self.widget_settings = {} + self.layer_roles.clear() + self.thickness_sources.clear() + self.derived.clear() self.set_dem_layer(None) self.use_dem = True @@ -1472,8 +1736,29 @@ def to_dict(self): 'widget_settings': self.widget_settings, 'model_crs': self._get_model_crs_authid(), 'use_project_crs': self._use_project_crs, + 'layer_roles': self.layer_roles.to_dict(), + 'derived_data': self.derived.to_dict(), + 'thickness_sources': self.thickness_sources.to_dict(), } + def _restore_derived_state(self, data): + """Restore the layer roles, derived-data records and thickness sources. + + A state file from an older version has none of these keys. The roles + for the layers that the data manager already restored are the same + as the layers of these settings. The other roles and the records + get their default values. + """ + roles = data.get('layer_roles') + if roles: + self.layer_roles.from_dict(roles) + else: + self.layer_roles.set(layer_roles.GEOLOGY, None) + self.layer_roles.set(layer_roles.GEOLOGY_UNIT_FIELD, None) + self.layer_roles.contacts_source = layer_roles.CONTACTS_FROM_GEOLOGY + self.thickness_sources.from_dict(data.get('thickness_sources')) + self.derived.from_dict(data.get('derived_data')) + def from_dict(self, data): """Load data from a dictionary.""" if 'bounding_box' in data: @@ -1518,6 +1803,7 @@ def from_dict(self, data): self._flipped_fault_boundaries.update(data['flipped_fault_boundaries']) if 'widget_settings' in data: self.widget_settings = data['widget_settings'] + self._restore_derived_state(data) # Load model CRS settings if 'use_project_crs' in data: @@ -1639,6 +1925,8 @@ def update_from_dict(self, data): else: self.widget_settings = {} + self._restore_derived_state(data) + if self.stratigraphic_column_callback: self.stratigraphic_column_callback() @@ -1670,6 +1958,10 @@ def find_layer_by_name(self, layer_name, layer_type=QgsVectorLayer): self.logger(message=f"Layer '{layer_name}' is not a vector layer.", log_level=2) return None + def _find_any_layer(self, layer_name): + """Find a layer of any type (vector or raster) by name.""" + return self.find_layer_by_name(layer_name, layer_type=QgsMapLayer) + def update_feature_data(self, feature_name: str, feature_data: dict): """Update the feature data in the data manager.""" if not isinstance(feature_data, dict): @@ -1679,8 +1971,13 @@ def update_feature_data(self, feature_name: str, feature_data: dict): self.refresh_layer_watchers() def set_widget_settings(self, widget_name: str, settings: dict): - """Store widget settings for persistence.""" + """Store widget settings for persistence. + + The settings of a tool are inputs of the derived results, so the + status of these results can change. + """ self.widget_settings[widget_name] = settings + self.derived.refresh() def get_widget_settings(self, widget_name: str, default=None): """Retrieve persisted widget settings.""" diff --git a/loopstructural/main/derived_data.py b/loopstructural/main/derived_data.py new file mode 100644 index 00000000..d2766e42 --- /dev/null +++ b/loopstructural/main/derived_data.py @@ -0,0 +1,242 @@ +"""Records of the inputs of derived data. + +Some data is calculated from other inputs. For example, the basal contacts +come from the geology layer and the order of the stratigraphic column. When an +input changes, the result is out of date. + +For each result, this module keeps a hash of the inputs of the last run. The +status is "current" when the hash of the inputs now is the same as the +recorded hash. A change and its undo (for example, two reorders) give the same +hash, so the result stays current. + +This module does not import QGIS, so the unit tests can run it. +""" + +import hashlib +import json +import math +from typing import Callable, Optional + +BASAL_CONTACTS = 'basal_contacts' +THICKNESS = 'thickness' +STYLED_FIELDS = 'styled_fields' + +NOT_RUN = 'not_run' +CURRENT = 'current' +OUT_OF_DATE = 'out_of_date' + +TYPED = 'typed' +CALCULATED = 'calculated' + +# Short names for the user interface. +DESCRIPTIONS = { + BASAL_CONTACTS: 'Basal contacts', + THICKNESS: 'Calculated thicknesses', + STYLED_FIELDS: 'Styled map layer fields', +} + + +def normalise_inputs(inputs): + """Return the inputs as plain JSON data (the same data as after a save and load).""" + return json.loads(json.dumps(inputs, sort_keys=True, default=str)) + + +def hash_inputs(inputs) -> str: + """Return a stable hash of the inputs of a calculation. + + ``inputs`` can have dictionaries, lists, strings, numbers and None. The + order of the keys of a dictionary does not change the hash. The order of a + list does change it. + """ + text = json.dumps(inputs, sort_keys=True, default=str, separators=(',', ':')) + return hashlib.sha256(text.encode('utf-8')).hexdigest() + + +class DerivedData: + """Keep the inputs of each derived result and tell when it is out of date. + + A result has a name and an input provider. The provider is a function + that gives the inputs now. `record` stores the hash of the inputs of a + run. `status` compares the stored hash with the hash of the inputs now. + """ + + def __init__(self): + self._providers = {} + self._records = {} + self._last_status = {} + self._callbacks = [] + + def register(self, name: str, provider: Callable[[], object]): + """Register the function that gives the current inputs of a result.""" + self._providers[name] = provider + self._last_status[name] = self.status(name) + + @property + def names(self): + return list(self._providers) + + def attach(self, callback: Callable[[str, str], None]): + """Call ``callback(name, status)`` each time a status changes.""" + if callback not in self._callbacks: + self._callbacks.append(callback) + + def detach(self, callback): + if callback in self._callbacks: + self._callbacks.remove(callback) + + def current_hash(self, name: str) -> Optional[str]: + """Return the hash of the inputs now, or None if no provider exists.""" + provider = self._providers.get(name) + if provider is None: + return None + return hash_inputs(provider()) + + def record(self, name: str, inputs=None, detail=None): + """Record that the result was calculated from ``inputs``. + + Parameters + ---------- + name : str + Name of the derived result. + inputs : optional + The inputs of the run. Use the inputs that the run read, so that a + change during the run marks the result as out of date. If this is + None, the inputs now are used. + detail : dict, optional + Extra information for the user interface, for example the name of + the layer that was styled. It must be safe for JSON. + """ + if inputs is None: + provider = self._providers.get(name) + inputs = provider() if provider is not None else None + # The hash decides if the result is out of date. The inputs are + # kept only to tell the user what changed. + self._records[name] = { + 'hash': hash_inputs(inputs), + 'inputs': normalise_inputs(inputs), + 'detail': dict(detail or {}), + } + self.refresh() + + def forget(self, name: str): + """Remove the record of a result. Its status is then "not run".""" + if self._records.pop(name, None) is not None: + self.refresh() + + def detail(self, name: str) -> dict: + record = self._records.get(name) + return dict(record['detail']) if record else {} + + def changed_inputs(self, name: str): + """Return the keys of the inputs that are not the same as in the last run. + + The list is empty when the result is current, was never calculated, + or has no kept inputs (for example, from an older state file). + """ + record = self._records.get(name) + provider = self._providers.get(name) + if record is None or provider is None or not isinstance(record.get('inputs'), dict): + return [] + old = record['inputs'] + new = normalise_inputs(provider()) + if not isinstance(new, dict): + return [] + return sorted(key for key in set(old) | set(new) if old.get(key) != new.get(key)) + + def status(self, name: str) -> str: + record = self._records.get(name) + if record is None: + return NOT_RUN + if record['hash'] == self.current_hash(name): + return CURRENT + return OUT_OF_DATE + + def is_out_of_date(self, name: str) -> bool: + return self.status(name) == OUT_OF_DATE + + def out_of_date(self): + """Return the names of all results that are out of date.""" + return [name for name in self._providers if self.is_out_of_date(name)] + + def refresh(self): + """Compare the hashes again and send an event for each status change.""" + for name in self._providers: + status = self.status(name) + if status != self._last_status.get(name): + self._last_status[name] = status + for callback in list(self._callbacks): + callback(name, status) + + def clear(self): + """Remove all records.""" + self._records.clear() + self.refresh() + + def to_dict(self): + return {name: dict(record) for name, record in self._records.items()} + + def from_dict(self, data): + """Restore the records. Do not send events for the restored records.""" + self._records = {} + for name, record in (data or {}).items(): + if isinstance(record, dict) and isinstance(record.get('hash'), str): + self._records[name] = { + 'hash': record['hash'], + 'inputs': record.get('inputs'), + 'detail': dict(record.get('detail') or {}), + } + self.refresh() + + +def thickness_is_set(value) -> bool: + """Return True if a unit thickness is a usable number above zero.""" + try: + value = float(value) + except (TypeError, ValueError): + return False + return math.isfinite(value) and value > 0 + + +class ThicknessSources: + """Record if the thickness of each unit was typed or calculated. + + A calculated thickness must not replace a typed one. The key is the uuid + of the unit. + """ + + def __init__(self): + self._sources = {} + + def get(self, uuid) -> Optional[str]: + return self._sources.get(uuid) + + def set(self, uuid, source: str): + if source not in (TYPED, CALCULATED): + raise ValueError(f"Unknown thickness source '{source}'.") + self._sources[uuid] = source + + def discard(self, uuid): + self._sources.pop(uuid, None) + + def clear(self): + self._sources.clear() + + def can_overwrite(self, uuid, current_thickness) -> bool: + """Return True if a calculated thickness can replace the thickness. + + A thickness with no value can always be replaced. A calculated one can + be replaced. A typed one cannot. A value with no source, for example + from a state file of an older version, is treated as typed, because + the plugin cannot know who made it. + """ + if not thickness_is_set(current_thickness): + return True + return self._sources.get(uuid) == CALCULATED + + def to_dict(self): + return dict(self._sources) + + def from_dict(self, data): + self._sources = { + uuid: source for uuid, source in (data or {}).items() if source in (TYPED, CALCULATED) + } diff --git a/loopstructural/main/layer_roles.py b/loopstructural/main/layer_roles.py new file mode 100644 index 00000000..7516a471 --- /dev/null +++ b/loopstructural/main/layer_roles.py @@ -0,0 +1,170 @@ +"""Shared layer roles. + +A layer role is a name for the job that a layer has in the model, for example +the geology polygons. The user selects a layer for a role one time. Each tool +reads the role as its default value. + +This module does not import QGIS. A layer is any object with a ``name()`` +method, so the code can run in the unit tests. +""" + +from typing import Callable, Optional + +GEOLOGY = 'geology' +GEOLOGY_UNIT_FIELD = 'geology_unit_field' +FAULT_TRACES = 'fault_traces' +STRUCTURE = 'structure' +BASAL_CONTACTS = 'basal_contacts' +DEM = 'dem' + +# Roles that hold a layer. +LAYER_ROLES = (GEOLOGY, FAULT_TRACES, STRUCTURE, BASAL_CONTACTS, DEM) +# Roles that hold a field name. +FIELD_ROLES = (GEOLOGY_UNIT_FIELD,) +ALL_ROLES = LAYER_ROLES + FIELD_ROLES + +# Where the basal contacts come from. +CONTACTS_FROM_GEOLOGY = 'geology' +CONTACTS_FROM_LAYER = 'layer' +CONTACTS_SOURCES = (CONTACTS_FROM_GEOLOGY, CONTACTS_FROM_LAYER) + + +def _layer_name(layer): + """Return the name of a layer, or None if the layer is gone.""" + if layer is None: + return None + try: + return layer.name() + except RuntimeError: + # the C++ layer was deleted + return None + + +class LayerRoles: + """Store the layer and the field that each role has. + + Parameters + ---------- + layer_resolver : callable, optional + Function that gives a layer for a layer name. `from_dict` uses it. + """ + + def __init__(self, layer_resolver: Optional[Callable] = None): + self._layer_resolver = layer_resolver + self._values = {} + self._callbacks = [] + self._contacts_source = CONTACTS_FROM_GEOLOGY + + def attach(self, callback: Callable): + """Call ``callback(role, value)`` each time a role changes.""" + if callback not in self._callbacks: + self._callbacks.append(callback) + + def detach(self, callback: Callable): + if callback in self._callbacks: + self._callbacks.remove(callback) + + def _notify(self, role, value): + for callback in list(self._callbacks): + callback(role, value) + + @staticmethod + def _check_role(role): + if role not in ALL_ROLES: + raise ValueError(f"Unknown layer role '{role}'. Use one of {ALL_ROLES}.") + + @staticmethod + def _same(old, new): + """True if two role values are the same layer or the same field.""" + if old is new: + return True + if old is None or new is None: + return False + if isinstance(old, str) or isinstance(new, str): + return old == new + # two wrappers can point to one layer: compare the layer ids + try: + return old.id() == new.id() + except (AttributeError, RuntimeError): + return False + + def get(self, role): + """Return the value of a role, or None if it is not set. + + A layer that was removed from the project gives None. + """ + self._check_role(role) + value = self._values.get(role) + if value is not None and role in LAYER_ROLES and _layer_name(value) is None: + return None + return value + + def set(self, role, value): + """Set the value of a role. + + Returns True if the value changed. The callbacks run only for a + change, so a widget can write the role that it just read. + """ + self._check_role(role) + old = self._values.get(role) + if self._same(old, value): + return False + if value is None: + self._values.pop(role, None) + else: + self._values[role] = value + self._notify(role, value) + return True + + def clear(self): + """Remove all roles and set the contacts source to its default.""" + for role in list(self._values): + self.set(role, None) + self.contacts_source = CONTACTS_FROM_GEOLOGY + + @property + def contacts_source(self): + return self._contacts_source + + @contacts_source.setter + def contacts_source(self, source): + if source not in CONTACTS_SOURCES: + raise ValueError(f"Unknown contacts source '{source}'. Use one of {CONTACTS_SOURCES}.") + if source != self._contacts_source: + self._contacts_source = source + self._notify('contacts_source', source) + + def to_dict(self): + """Return the roles as a dictionary that is safe for JSON. + + A layer is saved by its name, like the other layers in the state. + """ + data = {'contacts_source': self._contacts_source} + for role in LAYER_ROLES: + data[role] = _layer_name(self._values.get(role)) + for role in FIELD_ROLES: + data[role] = self._values.get(role) + return data + + def from_dict(self, data): + """Set the roles from the output of `to_dict`. + + A key that is not in ``data`` does not change its role, so a state + file from an older version of the plugin loads without errors. A + layer that is not in the project gives an empty role. + """ + data = data or {} + if 'contacts_source' in data: + source = data['contacts_source'] + self.contacts_source = source if source in CONTACTS_SOURCES else CONTACTS_FROM_GEOLOGY + for role in LAYER_ROLES: + if role not in data: + continue + name = data[role] + layer = None + if name is not None and self._layer_resolver is not None: + layer = self._layer_resolver(name) + self.set(role, layer) + for role in FIELD_ROLES: + if role in data: + self.set(role, data[role]) diff --git a/loopstructural/plugin_main.py b/loopstructural/plugin_main.py index 9f3c48b3..8c818788 100644 --- a/loopstructural/plugin_main.py +++ b/loopstructural/plugin_main.py @@ -82,6 +82,7 @@ def __init__(self, iface: QgisInterface): ) self.model_manager = GeologicalModelManager(debug_manager=self.debug_manager) self.data_manager.set_model_manager(self.model_manager) + self.data_manager.set_debug_manager(self.debug_manager) def injectLogHandler(self): """Install LoopStructural logging handler that forwards logs to the plugin logger. diff --git a/tests/qgis/test_derived_data_state.py b/tests/qgis/test_derived_data_state.py new file mode 100644 index 00000000..4f4ca425 --- /dev/null +++ b/tests/qgis/test_derived_data_state.py @@ -0,0 +1,269 @@ +"""Pytest tests for the shared layer roles and the derived-data records in +`ModellingDataManager`. + +The basal contacts and the thicknesses are calculated from the column and +the layers. When one of these inputs changes, the result is out of date. +""" + +import json +from unittest.mock import Mock + +import pytest +from qgis.core import QgsProject, QgsVectorLayer + +from loopstructural.main import derived_data, layer_roles +from loopstructural.main.data_manager import ModellingDataManager +from loopstructural.main.model_manager import GeologicalModelManager + + +class _DebugManager: + def log(self, *args, **kwargs): + pass + + +def _layer(name, geometry='Polygon', fields='field=UNITNAME:string'): + layer = QgsVectorLayer(f'{geometry}?crs=EPSG:32755&{fields}', name, 'memory') + assert layer.isValid() + QgsProject.instance().addMapLayer(layer) + return layer + + +@pytest.fixture +def project(): + yield QgsProject.instance() + QgsProject.instance().removeAllMapLayers() + + +@pytest.fixture +def data_manager(project): + data_manager = ModellingDataManager(project=project, mapCanvas=Mock(), logger=Mock()) + data_manager.set_model_manager(GeologicalModelManager(debug_manager=_DebugManager())) + column = data_manager.get_stratigraphic_column() + column.clear(basement=False) + for name in ('oldest', 'middle', 'youngest'): + column.add_unit(name=name, colour='#ff0000', where='top') + return data_manager + + +@pytest.fixture +def geology(project): + return _layer('geology') + + +def _unit(data_manager, name): + return data_manager.get_stratigraphic_column().get_unit_by_name(name=name) + + +def _reorder(data_manager, names_oldest_first): + column = data_manager.get_stratigraphic_column() + uuids = [column.get_unit_by_name(name=name).uuid for name in names_oldest_first] + data_manager.update_stratigraphic_column_order(uuids) + + +class TestLayerRoles: + def test_the_load_data_widgets_write_the_roles(self, data_manager, project): + faults = _layer('faults', 'LineString') + structure = _layer('structure', 'Point') + contacts = _layer('contacts', 'LineString') + data_manager.set_fault_trace_layer(faults, fault_name_field='UNITNAME') + data_manager.set_structural_orientations(structure) + data_manager.set_basal_contacts(contacts, unitname_field='UNITNAME') + roles = data_manager.layer_roles + assert roles.get(layer_roles.FAULT_TRACES) is faults + assert roles.get(layer_roles.STRUCTURE) is structure + assert roles.get(layer_roles.BASAL_CONTACTS) is contacts + + def test_a_role_change_sends_an_event(self, data_manager, geology): + events = [] + data_manager.layer_roles.attach(lambda role, value: events.append(role)) + data_manager.layer_roles.set(layer_roles.GEOLOGY, geology) + assert events == [layer_roles.GEOLOGY] + + def test_the_roles_are_saved_and_loaded(self, data_manager, geology, project): + data_manager.layer_roles.set(layer_roles.GEOLOGY, geology) + data_manager.layer_roles.set(layer_roles.GEOLOGY_UNIT_FIELD, 'UNITNAME') + data_manager.layer_roles.contacts_source = layer_roles.CONTACTS_FROM_LAYER + state = json.loads(json.dumps(data_manager.to_dict())) + + other = ModellingDataManager(project=project, mapCanvas=Mock(), logger=Mock()) + other.set_model_manager(GeologicalModelManager(debug_manager=_DebugManager())) + other.update_from_dict(state) + assert other.layer_roles.get(layer_roles.GEOLOGY) is geology + assert other.layer_roles.get(layer_roles.GEOLOGY_UNIT_FIELD) == 'UNITNAME' + assert other.layer_roles.contacts_source == layer_roles.CONTACTS_FROM_LAYER + + def test_an_old_state_file_loads_with_default_roles(self, data_manager, geology): + data_manager.layer_roles.set(layer_roles.GEOLOGY, geology) + state = data_manager.to_dict() + for key in ('layer_roles', 'derived_data', 'thickness_sources'): + state.pop(key) + data_manager.update_from_dict(state) + assert data_manager.layer_roles.get(layer_roles.GEOLOGY) is None + assert data_manager.layer_roles.contacts_source == layer_roles.CONTACTS_FROM_GEOLOGY + assert data_manager.derived.out_of_date() == [] + + def test_a_tool_gives_the_geology_role_a_layer_that_has_none(self, data_manager, geology): + other = _layer('other') + data_manager.adopt_layer_roles(geology=geology) + data_manager.adopt_layer_roles(geology=other) + assert data_manager.layer_roles.get(layer_roles.GEOLOGY) is geology + + def test_reset_clears_the_roles(self, data_manager, geology): + data_manager.layer_roles.set(layer_roles.GEOLOGY, geology) + data_manager.reset() + assert data_manager.layer_roles.get(layer_roles.GEOLOGY) is None + + +class TestDerivedData: + @pytest.fixture + def with_contacts(self, data_manager, geology): + data_manager.layer_roles.set(layer_roles.GEOLOGY, geology) + data_manager.layer_roles.set(layer_roles.GEOLOGY_UNIT_FIELD, 'UNITNAME') + data_manager.derived.record(derived_data.BASAL_CONTACTS) + data_manager.derived.record(derived_data.THICKNESS) + return data_manager + + def test_a_result_that_did_not_run_is_not_out_of_date(self, data_manager): + assert data_manager.get_derived_status(derived_data.BASAL_CONTACTS) == derived_data.NOT_RUN + assert data_manager.derived.out_of_date() == [] + + def test_a_reorder_marks_the_contacts_and_thicknesses_out_of_date(self, with_contacts): + _reorder(with_contacts, ['middle', 'oldest', 'youngest']) + assert with_contacts.derived.out_of_date() == [ + derived_data.BASAL_CONTACTS, + derived_data.THICKNESS, + ] + assert with_contacts.derived.changed_inputs(derived_data.BASAL_CONTACTS) == ['unit_order'] + + def test_a_move_and_the_move_back_keep_the_contacts_current(self, with_contacts): + _reorder(with_contacts, ['middle', 'oldest', 'youngest']) + assert with_contacts.derived.is_out_of_date(derived_data.BASAL_CONTACTS) + _reorder(with_contacts, ['oldest', 'middle', 'youngest']) + assert with_contacts.derived.out_of_date() == [] + + def test_the_status_event_is_sent_through_the_column_callback(self, with_contacts): + events = [] + with_contacts.derived.attach(lambda name, status: events.append((name, status))) + _reorder(with_contacts, ['middle', 'oldest', 'youngest']) + assert (derived_data.BASAL_CONTACTS, derived_data.OUT_OF_DATE) in events + + def test_a_unit_colour_does_not_mark_the_contacts_out_of_date(self, with_contacts): + unit = _unit(with_contacts, 'middle') + with_contacts.get_stratigraphic_column().update_element( + {'uuid': unit.uuid, 'name': 'middle', 'colour': '#00ff00'} + ) + with_contacts.stratigraphic_column_callback() + assert with_contacts.derived.out_of_date() == [] + + def test_a_new_unconformity_does_not_mark_the_contacts_out_of_date(self, with_contacts): + with_contacts.add_to_stratigraphic_column({'type': 'unconformity', 'name': 'u'}) + assert with_contacts.derived.out_of_date() == [] + + def test_a_change_of_the_geology_role_marks_the_contacts_out_of_date(self, with_contacts): + with_contacts.layer_roles.set(layer_roles.GEOLOGY, _layer('geology 2')) + assert with_contacts.derived.is_out_of_date(derived_data.BASAL_CONTACTS) + assert with_contacts.derived.is_out_of_date(derived_data.THICKNESS) + + def test_a_change_of_the_unit_field_marks_the_contacts_out_of_date(self, with_contacts): + with_contacts.layer_roles.set(layer_roles.GEOLOGY_UNIT_FIELD, 'other') + assert with_contacts.derived.is_out_of_date(derived_data.BASAL_CONTACTS) + + def test_a_change_of_a_tool_setting_marks_the_contacts_out_of_date(self, with_contacts): + with_contacts.set_widget_settings( + 'basal_contacts_widget', {'ignore_units': ['middle'], 'basal_override_units': []} + ) + assert with_contacts.derived.is_out_of_date(derived_data.BASAL_CONTACTS) + # the thickness uses the contacts, so it is out of date too + assert with_contacts.derived.is_out_of_date(derived_data.THICKNESS) + + def test_a_change_of_the_thickness_settings_marks_only_the_thickness(self, with_contacts): + with_contacts.set_widget_settings( + 'thickness_calculator_widget', {'calculator_type': 'StructuralPoint'} + ) + assert with_contacts.derived.out_of_date() == [derived_data.THICKNESS] + + def test_the_inputs_of_the_run_are_recorded(self, data_manager, geology): + data_manager.layer_roles.set(layer_roles.GEOLOGY, geology) + # a tool gives its own inputs for the run + inputs = data_manager.basal_contacts_inputs(geology=geology, unit_field='UNITNAME') + data_manager.layer_roles.set(layer_roles.GEOLOGY_UNIT_FIELD, 'UNITNAME') + data_manager.derived.record(derived_data.BASAL_CONTACTS, inputs=inputs) + assert data_manager.derived.status(derived_data.BASAL_CONTACTS) == derived_data.CURRENT + + def test_styled_fields_follow_the_order_and_the_thickness(self, data_manager): + data_manager.derived.record(derived_data.STYLED_FIELDS) + _unit(data_manager, 'middle').thickness = 12.0 + data_manager.stratigraphic_column_callback() + assert data_manager.derived.is_out_of_date(derived_data.STYLED_FIELDS) + + def test_the_records_are_saved_and_loaded(self, with_contacts, project): + state = json.loads(json.dumps(with_contacts.to_dict())) + _reorder(with_contacts, ['middle', 'oldest', 'youngest']) + with_contacts.update_from_dict(state) + # the saved column has the first order again + assert with_contacts.derived.out_of_date() == [] + assert with_contacts.derived.status(derived_data.BASAL_CONTACTS) == derived_data.CURRENT + + def test_the_contacts_source_changes_the_inputs_of_the_thickness(self, with_contacts): + contacts = _layer('contacts', 'LineString') + with_contacts.set_basal_contacts(contacts, unitname_field='UNITNAME') + with_contacts.derived.record(derived_data.THICKNESS) + with_contacts.layer_roles.contacts_source = layer_roles.CONTACTS_FROM_LAYER + assert with_contacts.derived.is_out_of_date(derived_data.THICKNESS) + + +class TestThicknessSource: + def test_a_typed_thickness_is_not_overwritten(self, data_manager): + unit = _unit(data_manager, 'middle') + data_manager.note_thickness_edit({'uuid': unit.uuid, 'thickness': 40.0}) + unit.thickness = 40.0 + assert data_manager.get_thickness_source(unit.uuid) == derived_data.TYPED + applied, skipped = data_manager.apply_calculated_thicknesses({'middle': 99.0}) + assert applied == [] + assert skipped == ['middle'] + assert unit.thickness == 40.0 + + def test_a_calculated_thickness_is_set_and_can_be_calculated_again(self, data_manager): + unit = _unit(data_manager, 'middle') + applied, skipped = data_manager.apply_calculated_thicknesses({'middle': 25.0}) + assert applied == ['middle'] + assert unit.thickness == 25.0 + assert data_manager.get_thickness_source(unit.uuid) == derived_data.CALCULATED + data_manager.apply_calculated_thicknesses({'middle': 30.0}) + assert unit.thickness == 30.0 + + def test_a_value_that_is_not_calculated_is_ignored(self, data_manager): + applied, skipped = data_manager.apply_calculated_thicknesses( + {'middle': -1, 'oldest': float('nan'), 'unknown unit': 5.0} + ) + assert applied == [] and skipped == [] + + def test_a_row_that_sends_the_calculated_value_back_is_not_a_typed_value(self, data_manager): + unit = _unit(data_manager, 'middle') + data_manager.apply_calculated_thicknesses({'middle': 25.4567}) + # the row shows two decimals and sends that value back + data_manager.note_thickness_edit({'uuid': unit.uuid, 'thickness': 25.46}) + assert data_manager.get_thickness_source(unit.uuid) == derived_data.CALCULATED + + def test_a_user_edit_of_a_calculated_thickness_makes_it_typed(self, data_manager): + unit = _unit(data_manager, 'middle') + data_manager.apply_calculated_thicknesses({'middle': 25.0}) + data_manager.note_thickness_edit({'uuid': unit.uuid, 'thickness': 60.0}) + assert data_manager.get_thickness_source(unit.uuid) == derived_data.TYPED + + def test_clearing_a_typed_thickness_allows_a_calculated_one(self, data_manager): + unit = _unit(data_manager, 'middle') + data_manager.note_thickness_edit({'uuid': unit.uuid, 'thickness': 40.0}) + unit.thickness = 40.0 + data_manager.note_thickness_edit({'uuid': unit.uuid, 'thickness': 0.0}) + unit.thickness = 0.0 + applied, _ = data_manager.apply_calculated_thicknesses({'middle': 25.0}) + assert applied == ['middle'] + + def test_the_sources_are_saved_and_loaded(self, data_manager): + unit = _unit(data_manager, 'middle') + data_manager.apply_calculated_thicknesses({'middle': 25.0}) + state = json.loads(json.dumps(data_manager.to_dict())) + data_manager.thickness_sources.clear() + data_manager.update_from_dict(state) + assert data_manager.get_thickness_source(unit.uuid) == derived_data.CALCULATED diff --git a/tests/qgis/test_gui_paint_stratigraphic_order_widget.py b/tests/qgis/test_gui_paint_stratigraphic_order_widget.py index b3be23b2..f2420288 100644 --- a/tests/qgis/test_gui_paint_stratigraphic_order_widget.py +++ b/tests/qgis/test_gui_paint_stratigraphic_order_widget.py @@ -3,6 +3,8 @@ from qgis.testing import start_app from loopstructural.gui.compatibility import QVariantCompat +from loopstructural.main.derived_data import DerivedData +from loopstructural.main.layer_roles import LayerRoles # Monkeypatch uic.loadUi to avoid needing the .ui file and to provide minimal widgets @@ -104,6 +106,16 @@ class DummyDataManager: def __init__(self, names): self._names = names self.stratigraphic_order = None + self.layer_roles = LayerRoles() + self.derived = DerivedData() + + def adopt_layer_roles(self, **roles): + for role, value in roles.items(): + if value is not None and self.layer_roles.get(role) is None: + self.layer_roles.set(role, value) + + def styled_fields_inputs(self): + return {'unit_order': self._names} def get_stratigraphic_unit_names(self): return self._names diff --git a/tests/unit/test_derived_data.py b/tests/unit/test_derived_data.py new file mode 100644 index 00000000..98357a06 --- /dev/null +++ b/tests/unit/test_derived_data.py @@ -0,0 +1,190 @@ +"""Pytest tests for the records of the inputs of derived data. + +The module does not import QGIS, so these tests run in the fast tests/unit/ job. +The inputs here have the same shape as the inputs that the data manager gives +(see tests/qgis/test_derived_data_state.py for the tests with the data manager). +""" + +import json + +import pytest + +from loopstructural.main.derived_data import ( + BASAL_CONTACTS, + CALCULATED, + CURRENT, + NOT_RUN, + OUT_OF_DATE, + TYPED, + DerivedData, + ThicknessSources, + hash_inputs, +) + + +class _Column: + """A small stand-in for the inputs of the basal contacts.""" + + def __init__(self): + self.order = ['youngest', 'middle', 'oldest'] + self.colours = {'youngest': 'red', 'middle': 'green', 'oldest': 'blue'} + self.geology = 'geology' + + def inputs(self): + # the colours are not an input of the basal contacts + return {'unit_order': list(self.order), 'geology': self.geology} + + +@pytest.fixture +def column(): + return _Column() + + +@pytest.fixture +def derived(column): + derived = DerivedData() + derived.register(BASAL_CONTACTS, column.inputs) + return derived + + +class TestHash: + def test_the_order_of_the_keys_does_not_change_the_hash(self): + assert hash_inputs({'a': 1, 'b': 2}) == hash_inputs({'b': 2, 'a': 1}) + + def test_the_order_of_a_list_changes_the_hash(self): + assert hash_inputs({'order': ['a', 'b']}) != hash_inputs({'order': ['b', 'a']}) + + +class TestStatus: + def test_a_result_that_did_not_run_is_not_run(self, derived): + assert derived.status(BASAL_CONTACTS) == NOT_RUN + assert derived.out_of_date() == [] + + def test_a_result_is_current_after_a_run(self, derived): + derived.record(BASAL_CONTACTS) + assert derived.status(BASAL_CONTACTS) == CURRENT + + def test_a_reorder_marks_the_contacts_out_of_date(self, derived, column): + derived.record(BASAL_CONTACTS) + column.order = ['middle', 'youngest', 'oldest'] + assert derived.status(BASAL_CONTACTS) == OUT_OF_DATE + assert derived.out_of_date() == [BASAL_CONTACTS] + + def test_a_change_of_a_unit_colour_does_not(self, derived, column): + derived.record(BASAL_CONTACTS) + column.colours['youngest'] = 'purple' + assert derived.status(BASAL_CONTACTS) == CURRENT + + def test_a_reorder_and_its_undo_give_the_same_hash(self, derived, column): + derived.record(BASAL_CONTACTS) + hash_before = derived.current_hash(BASAL_CONTACTS) + column.order = ['middle', 'youngest', 'oldest'] + assert derived.status(BASAL_CONTACTS) == OUT_OF_DATE + column.order = ['youngest', 'middle', 'oldest'] + assert derived.current_hash(BASAL_CONTACTS) == hash_before + assert derived.status(BASAL_CONTACTS) == CURRENT + + def test_a_new_run_makes_the_result_current_again(self, derived, column): + derived.record(BASAL_CONTACTS) + column.order.reverse() + derived.record(BASAL_CONTACTS) + assert derived.status(BASAL_CONTACTS) == CURRENT + + def test_the_inputs_of_the_run_are_recorded(self, derived, column): + # the user changes the column while the run is active + run_inputs = column.inputs() + column.order.reverse() + derived.record(BASAL_CONTACTS, inputs=run_inputs) + assert derived.status(BASAL_CONTACTS) == OUT_OF_DATE + + def test_forget(self, derived): + derived.record(BASAL_CONTACTS) + derived.forget(BASAL_CONTACTS) + assert derived.status(BASAL_CONTACTS) == NOT_RUN + + def test_changed_inputs_name_the_inputs_that_changed(self, derived, column): + derived.record(BASAL_CONTACTS) + assert derived.changed_inputs(BASAL_CONTACTS) == [] + column.order.reverse() + column.geology = 'other' + assert derived.changed_inputs(BASAL_CONTACTS) == ['geology', 'unit_order'] + + +class TestEvents: + def test_refresh_sends_an_event_when_a_status_changes(self, derived, column): + events = [] + derived.attach(lambda name, status: events.append((name, status))) + derived.record(BASAL_CONTACTS) + column.order.reverse() + derived.refresh() + derived.refresh() + assert events == [(BASAL_CONTACTS, CURRENT), (BASAL_CONTACTS, OUT_OF_DATE)] + + def test_a_reorder_and_its_undo_send_two_events(self, derived, column): + derived.record(BASAL_CONTACTS) + events = [] + derived.attach(lambda name, status: events.append(status)) + column.order.reverse() + derived.refresh() + column.order.reverse() + derived.refresh() + assert events == [OUT_OF_DATE, CURRENT] + + +class TestSaveAndLoad: + def test_round_trip(self, derived, column): + derived.record(BASAL_CONTACTS, detail={'layer': 'geology'}) + data = json.loads(json.dumps(derived.to_dict())) + + loaded = DerivedData() + loaded.register(BASAL_CONTACTS, column.inputs) + loaded.from_dict(data) + assert loaded.status(BASAL_CONTACTS) == CURRENT + assert loaded.detail(BASAL_CONTACTS) == {'layer': 'geology'} + column.order.reverse() + assert loaded.status(BASAL_CONTACTS) == OUT_OF_DATE + assert loaded.changed_inputs(BASAL_CONTACTS) == ['unit_order'] + + def test_an_old_state_file_has_no_records(self, derived): + derived.from_dict(None) + assert derived.status(BASAL_CONTACTS) == NOT_RUN + + def test_a_bad_record_is_ignored(self, derived): + derived.from_dict({BASAL_CONTACTS: 'bad', 'other': {'hash': 3}}) + assert derived.status(BASAL_CONTACTS) == NOT_RUN + + +class TestThicknessSources: + def test_a_calculated_thickness_does_not_replace_a_typed_one(self): + sources = ThicknessSources() + sources.set('unit', TYPED) + assert sources.can_overwrite('unit', 25.0) is False + + def test_a_calculated_thickness_replaces_a_calculated_one(self): + sources = ThicknessSources() + sources.set('unit', CALCULATED) + assert sources.can_overwrite('unit', 25.0) is True + + def test_a_thickness_with_no_value_can_be_replaced(self): + sources = ThicknessSources() + sources.set('unit', TYPED) + for empty in (None, 0.0, float('nan'), float('inf'), -1): + assert sources.can_overwrite('unit', empty) is True + + def test_a_thickness_with_no_source_is_treated_as_typed(self): + assert ThicknessSources().can_overwrite('unit', 25.0) is False + + def test_an_unknown_source_is_an_error(self): + with pytest.raises(ValueError): + ThicknessSources().set('unit', 'guessed') + + def test_round_trip(self): + sources = ThicknessSources() + sources.set('a', TYPED) + sources.set('b', CALCULATED) + loaded = ThicknessSources() + loaded.from_dict(json.loads(json.dumps(sources.to_dict()))) + assert loaded.get('a') == TYPED + assert loaded.get('b') == CALCULATED + loaded.from_dict({'a': 'bad'}) + assert loaded.get('a') is None diff --git a/tests/unit/test_layer_roles.py b/tests/unit/test_layer_roles.py new file mode 100644 index 00000000..349e8a6e --- /dev/null +++ b/tests/unit/test_layer_roles.py @@ -0,0 +1,170 @@ +"""Pytest tests for the shared layer roles. + +The layer roles do not import QGIS, so these tests use a fake layer and run +in the fast tests/unit/ job. +""" + +import pytest + +from loopstructural.main.layer_roles import ( + BASAL_CONTACTS, + CONTACTS_FROM_GEOLOGY, + CONTACTS_FROM_LAYER, + GEOLOGY, + GEOLOGY_UNIT_FIELD, + LayerRoles, +) + + +class FakeLayer: + def __init__(self, name, layer_id=None): + self._name = name + self._id = layer_id or f'{name}_id' + self.deleted = False + + def name(self): + if self.deleted: + raise RuntimeError('wrapped C/C++ object has been deleted') + return self._name + + def id(self): + return self._id + + +@pytest.fixture +def project(): + return { + 'geology': FakeLayer('geology'), + 'contacts': FakeLayer('contacts'), + } + + +@pytest.fixture +def roles(project): + return LayerRoles(layer_resolver=project.get) + + +class TestStorage: + def test_a_role_has_no_value_at_the_start(self, roles): + assert roles.get(GEOLOGY) is None + assert roles.get(GEOLOGY_UNIT_FIELD) is None + + def test_set_and_get_a_layer(self, roles, project): + roles.set(GEOLOGY, project['geology']) + assert roles.get(GEOLOGY) is project['geology'] + + def test_set_and_get_a_field(self, roles): + roles.set(GEOLOGY_UNIT_FIELD, 'UNITNAME') + assert roles.get(GEOLOGY_UNIT_FIELD) == 'UNITNAME' + + def test_an_unknown_role_is_an_error(self, roles): + with pytest.raises(ValueError): + roles.set('not_a_role', None) + with pytest.raises(ValueError): + roles.get('not_a_role') + + def test_setting_none_clears_a_role(self, roles, project): + roles.set(GEOLOGY, project['geology']) + roles.set(GEOLOGY, None) + assert roles.get(GEOLOGY) is None + + def test_a_deleted_layer_gives_none(self, roles, project): + roles.set(GEOLOGY, project['geology']) + project['geology'].deleted = True + assert roles.get(GEOLOGY) is None + + def test_the_contacts_source_starts_with_the_geology(self, roles): + assert roles.contacts_source == CONTACTS_FROM_GEOLOGY + + def test_an_unknown_contacts_source_is_an_error(self, roles): + with pytest.raises(ValueError): + roles.contacts_source = 'somewhere' + + +class TestChangeEvent: + def test_a_change_calls_the_callback(self, roles, project): + events = [] + roles.attach(lambda role, value: events.append((role, value))) + roles.set(GEOLOGY, project['geology']) + assert events == [(GEOLOGY, project['geology'])] + + def test_the_same_value_gives_no_event(self, roles, project): + events = [] + roles.set(GEOLOGY, project['geology']) + roles.attach(lambda role, value: events.append(role)) + assert roles.set(GEOLOGY, project['geology']) is False + roles.set(GEOLOGY_UNIT_FIELD, 'UNITNAME') + roles.set(GEOLOGY_UNIT_FIELD, 'UNITNAME') + assert events == [GEOLOGY_UNIT_FIELD] + + def test_two_wrappers_of_one_layer_are_the_same_value(self, roles): + events = [] + roles.set(GEOLOGY, FakeLayer('geology', layer_id='same')) + roles.attach(lambda role, value: events.append(role)) + assert roles.set(GEOLOGY, FakeLayer('geology', layer_id='same')) is False + assert events == [] + + def test_a_change_of_the_contacts_source_gives_an_event(self, roles): + events = [] + roles.attach(lambda role, value: events.append((role, value))) + roles.contacts_source = CONTACTS_FROM_LAYER + roles.contacts_source = CONTACTS_FROM_LAYER + assert events == [('contacts_source', CONTACTS_FROM_LAYER)] + + def test_a_detached_callback_is_not_called(self, roles, project): + events = [] + + def callback(role, value): + events.append(role) + + roles.attach(callback) + roles.detach(callback) + roles.set(GEOLOGY, project['geology']) + assert events == [] + + +class TestSaveAndLoad: + def test_round_trip(self, roles, project): + roles.set(GEOLOGY, project['geology']) + roles.set(GEOLOGY_UNIT_FIELD, 'UNITNAME') + roles.set(BASAL_CONTACTS, project['contacts']) + roles.contacts_source = CONTACTS_FROM_LAYER + data = roles.to_dict() + assert data['geology'] == 'geology' + assert data['basal_contacts'] == 'contacts' + + loaded = LayerRoles(layer_resolver=project.get) + loaded.from_dict(data) + assert loaded.get(GEOLOGY) is project['geology'] + assert loaded.get(GEOLOGY_UNIT_FIELD) == 'UNITNAME' + assert loaded.get(BASAL_CONTACTS) is project['contacts'] + assert loaded.contacts_source == CONTACTS_FROM_LAYER + + def test_the_saved_data_is_json(self, roles, project): + import json + + roles.set(GEOLOGY, project['geology']) + assert json.loads(json.dumps(roles.to_dict())) == roles.to_dict() + + def test_a_layer_that_is_not_in_the_project_gives_an_empty_role(self, roles): + roles.from_dict({'geology': 'removed'}) + assert roles.get(GEOLOGY) is None + + def test_an_old_state_file_does_not_change_the_roles(self, roles, project): + roles.set(GEOLOGY, project['geology']) + roles.from_dict({}) + roles.from_dict(None) + assert roles.get(GEOLOGY) is project['geology'] + assert roles.contacts_source == CONTACTS_FROM_GEOLOGY + + def test_a_bad_contacts_source_gives_the_default(self, roles): + roles.contacts_source = CONTACTS_FROM_LAYER + roles.from_dict({'contacts_source': 'somewhere'}) + assert roles.contacts_source == CONTACTS_FROM_GEOLOGY + + def test_clear(self, roles, project): + roles.set(GEOLOGY, project['geology']) + roles.contacts_source = CONTACTS_FROM_LAYER + roles.clear() + assert roles.get(GEOLOGY) is None + assert roles.contacts_source == CONTACTS_FROM_GEOLOGY