Coupled generators, generator-state LP, support bounds, and a generative-model driver

CHANGELOG.md

@@ -5,6 +5,15 @@
 A second correctness review: every fix below was reproduced against a
 brute-force reference or a closed-form value and has a regression test.
 
+### Added
+
+- `generative_model` / `GenerativeModelResult`: bracket search for generative
+  complexity with support, flow, and heuristic coverage lower bounds;
+  enumeration of exact-common-information optima with `generator_kernel`;
+  optional `bilevel_search`; coupled and memory-lift uppers; gap flags
+  (`cg_exceeds_g`, `n_g_optima_generating`). `BidirectionalEpsilonMachine.generative_model()`.
+- `coverage_bound` helpers for partial-sum (coverage) lower bounds on full supports.
+
 ### Breaking changes
 
 - `minimal_generative_model`, `wyner_generative_model`,

docs/generators/generative_models.rst

@@ -141,6 +141,89 @@ only, so ``gap`` can be positive even when some non-lifted generator attains
 :math:`\op{G}{S^+ : S^-}` (the interleaved process above). The gap vanishes for
 first-order Markov processes.
 
+Coupled generators
+==================
+
+:mod:`sofic.generators.coupled_generators` drops the lifting restriction. A
+generator with :math:`k` states is described, jointly with the causal states,
+by a state law :math:`\pi`, beliefs :math:`Q_w(s^+)` and :math:`R_w(s^-)`, and a
+free kernel :math:`K(x, w' \mid w)`; the product
+:math:`\pi_w Q_w(s^+_0) K(x_0, w_1 \mid w_0) R_{w_1}(s^-_1)` must reproduce the
+bidirectional edge law :cite:`Ellison2009`, be stationary on the past side and
+consistent on the future side. :func:`~sofic.generators.coupled_generators.coupled_search`
+minimizes :math:`\H{\pi}` subject to these equations (an augmented Lagrangian
+with analytic gradients) from the ε-machines, the bidirectional machine, extra
+seeds and random starts; :func:`~sofic.generators.coupled_generators.coupled_generative_model`
+returns the best candidate that reproduces the source process. For the
+interleaved process above it reaches :math:`2 h(\tfrac49) = 1.98215`, below the
+best lifted generator (1.99108); for the :math:`(2, \infty)` process it finds
+the ε-machine (1.5).
+
+Two narrower tools share the same equations:
+
+* :func:`~sofic.generators.coupled_generators.memory_lift` builds the
+  *memory-1 lift* of a channel :math:`\nu(v \mid s^+, s^-)`: draw
+  :math:`V_t \sim \nu(\cdot \mid B_t)` independently at each time and take
+  :math:`W_t = (V_t, V_{t+1})`, so consecutive states share randomness. The
+  interleaved process's minimal generator is the memory-1 lift of a single
+  flag, and :func:`~sofic.generators.coupled_generators.memory_lift_search`
+  finds it.
+* :func:`~sofic.generators.coupled_generators.generator_kernel` decides whether
+  a given channel's variable is the state of *any* generator: for a fixed channel
+  the equations are linear in :math:`K`, and a linear program returns the
+  smallest violation and the kernel. The exact-common-information optimum of the
+  :math:`(2, \infty)` process has violation 0.0625, so no generator has it as
+  its state. :func:`~sofic.generators.coupled_generators.bilevel_search`
+  (experimental) searches channels against this violation.
+
+All three searches are local and return upper bounds on :math:`C_g` for the
+given number of states.
+
+Searching for a generator
+=========================
+
+:func:`~sofic.generators.generative_search.generative_model` combines these
+tools into one pipeline and returns the best generator it can verify together
+with a rigorous bracket on :math:`C_g`:
+
+* lower bounds: the excess entropy, support-type and flow bounds, and (on large
+  supports) an optional heuristic coverage bound;
+* the persistent-crypticity criteria; when both vanish every separating variable
+  is a generator state and :math:`C_g = \op{G}{S^+ : S^-}`;
+* the exact-common-information optima (dit, several starts, since the optimum
+  need not be unique and only some optima generate), each polished to exact
+  separation with :func:`~sofic.generators.coupled_generators.polish_separating_channel`
+  and tested with :func:`~sofic.generators.coupled_generators.generator_kernel`;
+  when none generate, :func:`~sofic.generators.coupled_generators.bilevel_search`
+  may propose feasible channels;
+* upper bounds over nested classes, cheapest first: the ε-machines and
+  bidirectional machine, memory-1 lifts, deterministic and lifted channels (for
+  small machines), and the coupled search seeded with everything found so far.
+
+.. code-block:: python
+
+   result = bidir.generative_model()
+   result.lower, result.upper, result.certified   # bracket on C_g
+   result.cg_exceeds_g                            # True when no G optimum generates
+   result.separating_variables_generate           # some G optimum passed kernel LP
+   result.model                                   # the generator, verified against the source
+
+The searches are local; ``certified`` reports when the bracket closes (for
+example the golden mean, where the support bound meets :math:`C_\mu`).
+
+Lower bounds
+============
+
+:mod:`sofic.generators.generator_bounds` bounds :math:`\op{G}{S^+ : S^-}`, and
+hence :math:`C_g` (Löhr & Ay :cite:`LohrAy2009`), from below using only the
+support of the causal-state joint distribution. Each value of a separating
+variable has a rectangular support; typing values by maximal rectangles gives a
+polytope of type masses on which entropy is minimized at a vertex
+(:func:`~sofic.generators.generator_bounds.support_type_bound`). The bound is
+exact for the :math:`(2, \infty)` process (:math:`2 - \tfrac38 \log_2 3`) and
+for the golden mean process (:math:`C_\mu`, proving :math:`C_g = C_\mu` there);
+it is uninformative when the support is the full product.
+
 API
 ===
 
@@ -152,6 +235,24 @@ Constructors
 .. autofunction:: functional_generative_model
 .. autofunction:: gacs_korner_generative_model
 
+Coupled generators and bounds
+-----------------------------
+
+.. autofunction:: sofic.generators.generative_search.generative_model
+.. autoclass:: sofic.generators.generative_search.GenerativeModelResult
+   :members: certified, gap, g_optima_generate
+.. autofunction:: sofic.generators.coverage_bound.coverage_entropy_lower
+.. autofunction:: sofic.generators.coverage_bound.mu1_search
+.. autofunction:: sofic.generators.coupled_generators.coupled_generative_model
+.. autofunction:: sofic.generators.coupled_generators.polish_separating_channel
+.. autofunction:: sofic.generators.coupled_generators.coupled_search
+.. autofunction:: sofic.generators.coupled_generators.memory_lift
+.. autofunction:: sofic.generators.coupled_generators.memory_lift_search
+.. autofunction:: sofic.generators.coupled_generators.generator_kernel
+.. autofunction:: sofic.generators.coupled_generators.bilevel_search
+.. autofunction:: sofic.generators.generator_bounds.support_type_bound
+.. autofunction:: sofic.generators.generator_bounds.generator_support_bound
+
 Measures
 --------
 

sofic/generators/__init__.py

@@ -11,6 +11,7 @@
 )
 from sofic.generators.complexity_dimension import StatisticalComplexityDimension, statistical_complexity_dimension
 from sofic.generators.correlations import autocorrelation, mutual_information_function, power_spectrum
+from sofic.generators.coverage_bound import coverage_entropy_lower, mu1_brute_2x2, mu1_search
 from sofic.generators.directional_flow import (
     directed_information,
     independent_pair_generator,
@@ -21,6 +22,7 @@
 )
 from sofic.generators.epsilon_machine import EpsilonMachine
 from sofic.generators.epsilon_transducer import EpsilonTransducer
+from sofic.generators.generative_search import GenerativeModelResult, generative_model
 from sofic.generators.lumping import LumpabilityError, is_lumpable, lump, normalize_partition
 from sofic.generators.markov import MarkovChain
 from sofic.generators.mealy import MealyHMM
@@ -74,6 +76,11 @@
     "BlockEntropyEstimates",
     "BlockConvergenceDiagram",
     "BlockConvergenceEstimates",
+    "GenerativeModelResult",
+    "coverage_entropy_lower",
+    "generative_model",
+    "mu1_brute_2x2",
+    "mu1_search",
     "EpsilonMachine",
     "EpsilonTransducer",
     "FiniteStateChannel",

sofic/generators/bidirectional_epsilon_machine.py

@@ -608,6 +608,15 @@ def generative_complexity(self, **kwargs: Any) -> float:
         """H[G] of the minimal generative model, an upper bound on C_g."""
         return self.minimal_generative_model(**kwargs).generative_complexity()
 
+    def generative_model(self, **kwargs: Any) -> Any:
+        """Search for a verified minimal generator and a bracket on :math:`C_g`.
+
+        See :func:`~sofic.generators.generative_search.generative_model`.
+        """
+        from sofic.generators.generative_search import generative_model
+
+        return generative_model(self, **kwargs)
+
     def wyner_common_information(self, **kwargs: Any) -> float:
         """Wyner common information C[S+ : S-] = C[past : future] (no generator requirement)."""
         from sofic.generators.minimal_generative_model import wyner_common_information