Skip to content

fix(task) #32 #33 #34 #35 #36 entity_manager option, readers re-executed for each input, table optional with sql, null input errors, lazy hydration #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
njoubert-cleverage merged 1 commit into from
Oct 2, 2026
Merged

fix(task) #32 #33 #34 #35 #36 entity_manager option, readers re-executed for each input, table optional with sql, null input errors, lazy hydration #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
Latest
------

### Fixes
* [#32](https://github.com/cleverage/doctrine-process-bundle/issues/32) Fix EntityManager tasks: use the entity manager given by the `entity_manager` option (it was ignored by every task except ClearEntityManagerTask), the one managing the entity class otherwise. Update documentation, add tests.
* [#33](https://github.com/cleverage/doctrine-process-bundle/issues/33) Fix DatabaseReaderTask and DoctrineReaderTask: execute the query again for each input (the input following a complete iteration was skipped). Update documentation, add tests.
* [#34](https://github.com/cleverage/doctrine-process-bundle/issues/34) Fix DatabaseReaderTask: the `table` option is only required when `sql` is not set. Update documentation, add tests.
* [#35](https://github.com/cleverage/doctrine-process-bundle/issues/35) Fix DoctrineDetacherTask error message on a null input (it named DoctrineWriterTask), and throw an explicit `\RuntimeException` on a null input in DoctrineRemoverTask (a `\TypeError` was triggered). Update documentation, add tests.
* [#36](https://github.com/cleverage/doctrine-process-bundle/issues/36) Fix DoctrineReaderTask: hydrate the entities one at a time while iterating (every entity was hydrated before the first output). Update documentation, add tests.

v3.1
------

Expand Down
5 changes: 2 additions & 3 deletions docs/cookbooks/database_to_csv_export.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ clever_age_process:
read_books:
service: '@CleverAge\DoctrineProcessBundle\Task\Database\DatabaseReaderTask'
options:
table: 'book' # Required, even if a custom sql query is used
sql: >
SELECT b.id, b.title, a.firstname, a.lastname
FROM book b
Expand Down Expand Up @@ -78,5 +77,5 @@ How it works:

To export entities instead of raw rows, replace the first task by a
[DoctrineReaderTask](../reference/tasks/doctrine_reader_task.md) and read the values with property paths
(e.g. `code: 'author.lastname'`). Note that the DoctrineReaderTask loads all the matching entities in memory: for
big volumes, prefer the DatabaseReaderTask.
(e.g. `code: 'author.lastname'`). Note that the hydrated entities stay managed by the entity manager: for big
volumes, clear it regularly or prefer the DatabaseReaderTask.
8 changes: 4 additions & 4 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,10 @@ doctrine:
[DatabaseUpdaterTask](reference/tasks/database_updater_task.md)) run raw SQL queries through Doctrine DBAL. Their
`connection` option takes the name of a connection (a key under `doctrine.dbal.connections`); the default connection
is used if it is not set.
* **EntityManager** tasks work with Doctrine ORM entities. Except for the
[ClearEntityManagerTask](reference/tasks/doctrine_clear_task.md), which takes the name of an entity manager (a key
under `doctrine.orm.entity_managers`) in its `entity_manager` option, they use the entity manager that manages the
class of the handled entity.
* **EntityManager** tasks work with Doctrine ORM entities. Their `entity_manager` option takes the name of an entity
manager (a key under `doctrine.orm.entity_managers`); if it is not set, they use the entity manager that manages the
class of the handled entity (the default entity manager for the
[ClearEntityManagerTask](reference/tasks/doctrine_clear_task.md)).

See the [DoctrineBundle documentation](https://symfony.com/bundles/DoctrineBundle/current/configuration.html) for the
configuration of multiple connections and entity managers.
Expand Down
8 changes: 3 additions & 5 deletions docs/reference/tasks/database_reader_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Options

| Code | Type | Required | Default | Description |
|-------------------|---------------|:--------:|-----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `table` | `string` | **X** | | Table to read from when `sql` is not set: the query is `SELECT tbl.* FROM <table> tbl`.<br/>Required even when `sql` is set (its value is then ignored) |
| `table` | `string` | | | Table to read from: the query is `SELECT tbl.* FROM <table> tbl`.<br/>Required when `sql` is not set (ignored otherwise) |
| `connection` | `string\|null` | | `null` | Name of the Doctrine DBAL connection (as defined in `doctrine.dbal.connections`). If `null`, the default connection is used |
| `sql` | `string\|null` | | `null` | Custom SQL query to execute, with optional named (`:name`) or positional (`?`) parameters |
| `limit` | `int\|null` | | `null` | Maximum number of rows. Only used when `sql` is not set |
Expand Down Expand Up @@ -66,7 +66,6 @@ read_books:
read_books:
service: '@CleverAge\DoctrineProcessBundle\Task\Database\DatabaseReaderTask'
options:
table: 'book' # Required but not used
sql: >
SELECT b.id, b.title, a.lastname AS author
FROM book b INNER JOIN author a ON a.id = b.author_id
Expand All @@ -89,7 +88,6 @@ get_params:
read_books:
service: '@CleverAge\DoctrineProcessBundle\Task\Database\DatabaseReaderTask'
options:
table: 'book'
sql: 'SELECT * FROM book WHERE id >= :min_id'
input_as_params: true
types:
Expand All @@ -108,5 +106,5 @@ Notes
process is finalized.
* Array parameters (e.g. for an `IN (:ids)` clause) require an `ArrayParameterType` in `types`, for instance
`ids: !php/enum Doctrine\DBAL\ArrayParameterType::INTEGER` with Doctrine DBAL 4.
* The task is designed to be executed once per process run (typically as the entry point): if it receives a new input
after having iterated over all the rows, that input only resets the task, which is skipped.
* The query is executed again for each input received by the task (e.g. after an iterable task): with
`input_as_params`, each input gives its own parameters.
8 changes: 4 additions & 4 deletions docs/reference/tasks/doctrine_batchwriter_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ input is only buffered, and on flush if there is no remaining entity.
Options
-------

| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|--------------------------------------------------------------------------------------------------------------|
| `batch_count` | `int` | | `10` | Number of entities to buffer before writing them to the database |
| `entity_manager` | `string\|null` | | `null` | Inherited from the base Doctrine task but not used: the entity manager is the one managing each entity class |
| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|-----------------------------------------------------------------------------------------------------------------------------------------|
| `batch_count` | `int` | | `10` | Number of entities to buffer before writing them to the database |
| `entity_manager` | `string\|null` | | `null` | Name of the entity manager (as defined in `doctrine.orm.entity_managers`). If `null`, the one managing the class of each entity is used |

Examples
--------
Expand Down
8 changes: 4 additions & 4 deletions docs/reference/tasks/doctrine_cleaner_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ DoctrineCleanerTask

Clears the entity manager that manages the class of the entity received as input: **all** the entities of this
entity manager are detached (not only the input entity). Useful when the entity manager is not the default one, as it
is guessed from the input.
is guessed from the input (unless the `entity_manager` option is set).

Task reference
--------------
Expand All @@ -24,9 +24,9 @@ No output is set.
Options
-------

| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|--------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Inherited from the base Doctrine task but not used: the entity manager is the one managing the input's class |
| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|----------------------------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Name of the entity manager (as defined in `doctrine.orm.entity_managers`). If `null`, the one managing the input's class is used |

Examples
--------
Expand Down
6 changes: 3 additions & 3 deletions docs/reference/tasks/doctrine_detacher_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ No output is set.
Options
-------

| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|--------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Inherited from the base Doctrine task but not used: the entity manager is the one managing the input's class |
| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|----------------------------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Name of the entity manager (as defined in `doctrine.orm.entity_managers`). If `null`, the one managing the input's class is used |

Examples
--------
Expand Down
13 changes: 7 additions & 6 deletions docs/reference/tasks/doctrine_reader_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Options
| `limit` | `int\|null` | | `null` | Maximum number of entities |
| `offset` | `int\|null` | | `null` | Index of the first entity |
| `empty_log_level` | `string` | | `warning` | PSR log level (`Psr\Log\LogLevel` values) used to log an empty result set |
| `entity_manager` | `string\|null` | | `null` | Inherited from the base Doctrine task but not used: the entity manager is the one managing `class_name` |
| `entity_manager` | `string\|null` | | `null` | Name of the entity manager (as defined in `doctrine.orm.entity_managers`). If `null`, the one managing the `class_name` is used |

Examples
--------
Expand Down Expand Up @@ -74,13 +74,14 @@ read_books:
Notes
-----

* The query is executed on the first execution of the task and **all** the matching entities are loaded in memory
before being output one by one. For big volumes, use `limit`/`offset`, clear the entity manager downstream (see
[ClearEntityManagerTask](doctrine_clear_task.md)) or read raw rows with the
* The query is executed on the first execution of the task, then the entities are hydrated one at a time while the
process iterates (`Query::toIterable()`). They stay managed by the entity manager: for big volumes, clear it
downstream (see [ClearEntityManagerTask](doctrine_clear_task.md)) or detach the entities (see
[DoctrineDetacherTask](doctrine_detacher_task.md)) to keep the memory usage low, or read raw rows with the
[DatabaseReaderTask](database_reader_task.md).
* Entities stay managed by the entity manager: they can be modified then saved with the
[DoctrineWriterTask](doctrine_writer_task.md).
* The task is designed to be executed once per process run (typically as the entry point): if it receives a new input
after having iterated over all the entities, that input only resets the task, which is skipped.
* The query is executed again for each input received by the task (e.g. after an iterable task); the input itself
is not used.
* For more complex queries, extend `CleverAge\DoctrineProcessBundle\Task\EntityManager\AbstractDoctrineQueryTask`
(which provides the options above and a `getQueryBuilder()` method) or this task.
6 changes: 3 additions & 3 deletions docs/reference/tasks/doctrine_refresher_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ Possible outputs
Options
-------

| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|--------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Inherited from the base Doctrine task but not used: the entity manager is the one managing the input's class |
| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|----------------------------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Name of the entity manager (as defined in `doctrine.orm.entity_managers`). If `null`, the one managing the input's class is used |

Examples
--------
Expand Down
11 changes: 5 additions & 6 deletions docs/reference/tasks/doctrine_remover_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ Task reference
Accepted inputs
---------------

`object`: a Doctrine managed entity. An object whose class is not managed by any entity manager throws an
`\UnexpectedValueException`.
`object`: a Doctrine managed entity. A `null` input throws a `\RuntimeException`, and an object whose class is not
managed by any entity manager throws an `\UnexpectedValueException`.

Possible outputs
----------------
Expand All @@ -22,9 +22,9 @@ No output is set.
Options
-------

| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|--------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Inherited from the base Doctrine task but not used: the entity manager is the one managing the input's class |
| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|----------------------------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Name of the entity manager (as defined in `doctrine.orm.entity_managers`). If `null`, the one managing the input's class is used |

Examples
--------
Expand All @@ -50,4 +50,3 @@ Notes
* `flush()` writes **all** the pending changes of the entity manager, not only the removal.
* Cascade and `orphanRemoval` rules of the entity mapping apply. To delete many rows at once, a single `DELETE`
statement with the [DatabaseUpdaterTask](database_updater_task.md) is much faster.
* A `null` input is not supported (it throws a `\TypeError`).
6 changes: 3 additions & 3 deletions docs/reference/tasks/doctrine_writer_task.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ Possible outputs
Options
-------

| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|--------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Inherited from the base Doctrine task but not used: the entity manager is the one managing the input's class |
| Code | Type | Required | Default | Description |
|------------------|----------------|:--------:|---------|----------------------------------------------------------------------------------------------------------------------------------|
| `entity_manager` | `string\|null` | | `null` | Name of the entity manager (as defined in `doctrine.orm.entity_managers`). If `null`, the one managing the input's class is used |

Examples
--------
Expand Down
Loading
Loading