From 24744744a27f200a468d56987c12b52185475d31 Mon Sep 17 00:00:00 2001 From: Nicolas Joubert Date: Wed, 30 Sep 2026 09:46:26 +0200 Subject: [PATCH] chore(doc) #19 Add missing documentations: complete Adapter, GetTask & SetTask reference pages, configuration and custom cache tasks guides, cookbooks. Harmonize and fix existing documentation. Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 1 + docs/cookbooks/cache_warmup.md | 118 +++++++++++++++++ docs/cookbooks/share_data_between_branches.md | 115 ++++++++++++++++ docs/index.md | 63 ++++++++- docs/reference/adapter.md | 83 +++++++++--- docs/reference/tasks/_template.md | 33 ++--- docs/reference/tasks/get_task.md | 78 ++++++++--- docs/reference/tasks/set_task.md | 125 +++++++++--------- 8 files changed, 508 insertions(+), 108 deletions(-) create mode 100644 docs/cookbooks/cache_warmup.md create mode 100644 docs/cookbooks/share_data_between_branches.md diff --git a/CHANGELOG.md b/CHANGELOG.md index be752fc..a17f035 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ Latest ### Changes * [#17](https://github.com/cleverage/cache-process-bundle/issues/17) Update quality stack: use Rector `withComposerBased()` sets (removed `SYMFONY_64` / `PHPUNIT_100` sets), declare used Symfony packages and PHPUnit range in composer.json, apply quality tools fixes +* [#19](https://github.com/cleverage/cache-process-bundle/issues/19) Add missing documentations: complete Adapter, GetTask & SetTask reference pages, configuration and custom cache tasks guides, cookbooks. Harmonize and fix existing documentation. v2.0 ------ diff --git a/docs/cookbooks/cache_warmup.md b/docs/cookbooks/cache_warmup.md new file mode 100644 index 0000000..851c36c --- /dev/null +++ b/docs/cookbooks/cache_warmup.md @@ -0,0 +1,118 @@ +Warm up a persistent cache from a file +====================================== + +This recipe loads a reference CSV file into a persistent cache, so that other processes can read its lines by key +without parsing the file again. + +First, declare a FrameworkBundle cache pool and wrap it in a cache [adapter](../reference/adapter.md) with the +`catalog` code: + +```yaml +# config/packages/cache.yaml +framework: + cache: + pools: + app.cache.catalog: + adapter: cache.adapter.filesystem + default_lifetime: 86400 # One day + +# config/services.yaml +services: + app.cleverage_cache_process.adapter.catalog: + class: CleverAge\CacheProcessBundle\Adapter\Adapter + arguments: ['@app.cache.catalog', 'catalog'] + tags: + - { name: cleverage.cache.adapter } +``` + +Then define a process storing each line of the file, and another one reading a line by key: + +```yaml +clever_age_process: + configurations: + app.catalog_cache_warmup: + description: 'Store each line of the catalog CSV file in the catalog cache, indexed by sku' + tasks: + read_source: + service: '@CleverAge\ProcessBundle\Task\File\Csv\CsvReaderTask' + options: + file_path: '%kernel.project_dir%/var/data/catalog.csv' + delimiter: ';' + outputs: [filter_valid] + + filter_valid: + service: '@CleverAge\ProcessBundle\Task\FilterTask' + options: + not_empty: + '[sku]': ~ + outputs: [format] + + format: + service: '@CleverAge\ProcessBundle\Task\TransformerTask' + options: + transformers: + mapping: + mapping: + key: + code: '[sku]' + value: + code: '.' # The whole line + outputs: [store, count_rows] + + store: + service: '@CleverAge\CacheProcessBundle\Task\SetTask' + error_strategy: skip # A sku which is not a valid cache key is logged and skipped + options: + adapter: 'catalog' + key: '' # Overridden by the input + value: ~ # Overridden by the input + + count_rows: + service: '@CleverAge\ProcessBundle\Task\Reporting\StatCounterTask' + + app.catalog_cache_read: + description: 'Read a catalog line from the catalog cache' + help: "bin/console cleverage:process:execute app.catalog_cache_read -c sku:\"'ABC-001'\"" + tasks: + read: + service: '@CleverAge\CacheProcessBundle\Task\GetTask' + options: + adapter: 'catalog' + key: '{{ sku }}' + outputs: [skip_missing] + + skip_missing: + service: '@CleverAge\ProcessBundle\Task\SkipEmptyTask' + outputs: [log] + + log: + service: '@CleverAge\ProcessBundle\Task\Reporting\LoggerTask' + options: + level: info + message: 'Catalog line found in cache' +``` + +How it works: +- [CsvReaderTask](https://github.com/cleverage/process-bundle/blob/main/docs/reference/tasks/csv_reader_task.md) is + iterable: each line goes through the following tasks before the next one is read. +- [FilterTask](https://github.com/cleverage/process-bundle/blob/main/docs/reference/tasks/filter_task.md) skips the + lines without `sku`, which cannot be used as cache key. +- The [TransformerTask](https://github.com/cleverage/process-bundle/blob/main/docs/reference/tasks/transformer_task.md) + builds a `key` / `value` array with the + [mapping](https://github.com/cleverage/process-bundle/blob/main/docs/reference/transformers/mapping_transformer.md) + transformer (`code: '.'` maps the whole line). +- [SetTask](../reference/tasks/set_task.md) merges this array over its options: `key` and `value` placeholders are + replaced by the values of the current line, which is stored in the `catalog` adapter. With `error_strategy: skip`, + a `sku` containing a PSR-6 reserved character (`{}()/\@:`) is logged and the next line is processed. +- [StatCounterTask](https://github.com/cleverage/process-bundle/blob/main/docs/reference/tasks/stat_counter_task.md) + logs the number of stored lines at the end of the process. +- In the second process, [GetTask](../reference/tasks/get_task.md) reads the key given by the `sku` context value + (see [contextual values](https://github.com/cleverage/process-bundle/blob/main/docs/01-quick_start.md#contextual-values)). + Since the pool is persistent, the lines stored by the first process are available until they expire. +- A missing key outputs `null`: + [SkipEmptyTask](https://github.com/cleverage/process-bundle/blob/main/docs/reference/tasks/skip_empty_task.md) stops + the branch, so the [LoggerTask](https://github.com/cleverage/process-bundle/blob/main/docs/reference/tasks/logger_task.md) + only logs found lines. + +Note that the items expire after the `default_lifetime` of the pool: schedule the warm up process more often than +this lifetime if the other processes must always find the data. diff --git a/docs/cookbooks/share_data_between_branches.md b/docs/cookbooks/share_data_between_branches.md new file mode 100644 index 0000000..97ecb38 --- /dev/null +++ b/docs/cookbooks/share_data_between_branches.md @@ -0,0 +1,115 @@ +Share data between process branches +=================================== + +This recipe uses an in-memory cache to store data in a first branch of a process, then read it in another branch of +the same execution. Nothing is persisted: the values only live during the current PHP process. + +First, declare an in-memory cache [adapter](../reference/adapter.md) with the `memory` code: + +```php + ['all' => true], ``` +## Configuration + +The bundle has no semantic configuration. To use the cache tasks, declare at least one cache +[adapter](reference/adapter.md): a service implementing `CleverAge\CacheProcessBundle\Adapter\AdapterInterface` +(usually `CleverAge\CacheProcessBundle\Adapter\Adapter`, wrapping any Symfony cache pool), tagged +`cleverage.cache.adapter`. Its code is then used in the `adapter` option of the tasks. + +```yaml +# config/services.yaml +services: + app.cleverage_cache_process.adapter.app: + class: CleverAge\CacheProcessBundle\Adapter\Adapter + arguments: ['@cache.app', 'app'] + tags: + - { name: cleverage.cache.adapter } +``` + +## Custom cache tasks + +`CleverAge\CacheProcessBundle\Task\AbstractCacheTask` can be extended to implement other cache operations. It extends +[AbstractConfigurableTask](https://github.com/cleverage/process-bundle/blob/main/docs/03-custom_tasks.md), requires +the `cleverage_cache_process.registry.adapter` service (`AdapterRegistry`) as constructor argument, defines the +required `adapter` and `key` string options, and provides `getMergedOptions()` (options merged with the array input) +and `$this->registry->getAdapter($code)`. + +```php +getMergedOptions($state); + + $this->registry->getAdapter($options['adapter'])->deleteItem($options['key']); + } +} +``` + +```yaml +# config/services.yaml +services: + App\Task\DeleteTask: + public: true + shared: false + arguments: ['@cleverage_cache_process.registry.adapter'] +``` + ## Reference - [Adapter](reference/adapter.md) - Tasks - [GetTask](reference/tasks/get_task.md) - [SetTask](reference/tasks/set_task.md) + +## Cookbooks + +- [Share data between process branches](cookbooks/share_data_between_branches.md) +- [Warm up a persistent cache from a file](cookbooks/cache_warmup.md) diff --git a/docs/reference/adapter.md b/docs/reference/adapter.md index a06400c..7ee7463 100644 --- a/docs/reference/adapter.md +++ b/docs/reference/adapter.md @@ -1,24 +1,36 @@ Adapter -=============== +======= -Create cache adapter. +A cache adapter makes a [Symfony Cache](https://symfony.com/doc/current/components/cache.html) pool available to the +cache tasks ([GetTask](tasks/get_task.md), [SetTask](tasks/set_task.md)) under a short **code**, referenced by the +`adapter` option of these tasks. -Reference --------------- +Adapter reference +----------------- -* **Adapter Service Interface**: `CleverAge\CacheProcessBundle\Adapter\AdapterInterface` +* **Interface**: `CleverAge\CacheProcessBundle\Adapter\AdapterInterface`, extends + `Symfony\Component\Cache\Adapter\AdapterInterface` and adds `getCode(): string` (the code used in the `adapter` + option of the tasks) +* **Base class**: `CleverAge\CacheProcessBundle\Adapter\Adapter`, decorates any + `Symfony\Component\Cache\Adapter\AdapterInterface` (every PSR-6 method is forwarded to it) +* **Service tag**: `cleverage.cache.adapter`, every tagged service is registered in the + `cleverage_cache_process.registry.adapter` registry (`CleverAge\CacheProcessBundle\Registry\AdapterRegistry`) -Options -------- +Constructor arguments +--------------------- -| Code | Type | Required | Default | Description | -|-----------|----------|:--------:|---------|------------------------------------------------------------| -| `code` | `string` | **X** | | Service identifier, used by Task adapter option | -| `adapter` | `string` | **X** | | `Symfony\Component\Cache\Adapter\AdapterInterface` service | +Arguments of the `CleverAge\CacheProcessBundle\Adapter\Adapter` base class: + +| Code | Type | Required | Default | Description | +|-----------|----------------------------------------------------|:--------:|---------|---------------------------------------------------------------------| +| `adapter` | `Symfony\Component\Cache\Adapter\AdapterInterface` | **X** | | The decorated cache pool, where items are actually read and stored | +| `code` | `string` | **X** | | Unique adapter code, referenced by the `adapter` option of the tasks | Examples -------- +* In-memory adapter, as a PHP class (values only live during the current PHP process) + ```php is already defined`) when the registry is instantiated, i.e. the first time a cache task is used. +* Using a code that is not registered throws a `CleverAge\CacheProcessBundle\Exception\MissingAdapterException` + (`Adapter is missing`) when the task is executed. +* The cache tasks do not handle any expiration: the lifetime of the items is the default lifetime of the decorated + pool (`default_lifetime` of a FrameworkBundle pool, `$defaultLifetime` constructor argument of Symfony adapters). +* Cache keys must follow the PSR-6 rules: an empty key, or a key containing one of the reserved characters + `{}()/\@:`, throws a `Psr\Cache\InvalidArgumentException`. +* Only the PSR-6 methods are forwarded by the base class: tag-aware features of the decorated pool are not exposed. diff --git a/docs/reference/tasks/_template.md b/docs/reference/tasks/_template.md index ed1d4a5..919390a 100644 --- a/docs/reference/tasks/_template.md +++ b/docs/reference/tasks/_template.md @@ -1,44 +1,47 @@ TaskName ======== -_Describe main goal an use cases of the task_ +_Describe the main goal and use cases of the task._ Task reference -------------- -* **Service**: `ClassName` +* **Service**: `Fully\Qualified\ClassName` +* **Iterable task** _(only if it implements `IterableTaskInterface`)_ +* **Blocking task** _(only if it implements `BlockingTaskInterface`)_ +* **Flushable task** _(only if it implements `FlushableTaskInterface`)_ Accepted inputs --------------- -_Description of allowed types_ +_Description of allowed types, or "Input is ignored"._ Possible outputs ---------------- -_Description of possible types_ +_Description of possible types._ Options ------- -| Code | Type | Required | Default | Description | -| ---- | ---- | :------: | ------- | ----------- | -| `code` | `type` | **X** _or nothing_ | `default value` _if available_ | _description_ | +| Code | Type | Required | Default | Description | +|--------|--------|:--------:|-----------------|---------------| +| `code` | `type` | **X** | `default value` | _description_ | + +_If the task has no option, replace the table with "This task has no option."._ Examples -------- -_YAML samples and explanations_ - * Example 1 - details - - details - + ```yaml # Task configuration level code: - service: '@service_ref' - options: - a: 1 - b: 2 + service: '@Fully\Qualified\ClassName' + options: + a: 1 + b: 2 + outputs: [next_task] ``` diff --git a/docs/reference/tasks/get_task.md b/docs/reference/tasks/get_task.md index 464c7d0..ae79b2d 100644 --- a/docs/reference/tasks/get_task.md +++ b/docs/reference/tasks/get_task.md @@ -1,44 +1,92 @@ GetTask -=============== +======= -Get data from cache adapter. - -Adapter reference --------------- - -* [Documentation](../adapter.md) +Reads an item from a cache [adapter](../adapter.md) and outputs its value. Useful to reuse a value computed by a +previous branch of the process, or stored by another process when the adapter is persistent. Task reference -------------- -* **Task Service**: `CleverAge\CacheProcessBundle\Task\GetTask` +* **Service**: `CleverAge\CacheProcessBundle\Task\GetTask` Accepted inputs --------------- -`array`: inputs are merged with task defined options. +`array` or empty (`null`): if the input is not empty, it is merged over the resolved options (`array_merge`), so +input keys `adapter` and `key` override the options of the same name. Other input keys are ignored. + +Any other non-empty input (e.g. a `string`) triggers a `\TypeError`. Possible outputs ---------------- -`mixed` or `null`: the content of the cache key. +`mixed`: the value of the cache item, or `null` if the key is missing from the cache. Options ------- -| Code | Type | Required | Default | Description | -|-----------|----------|:--------:|----------|----------------------------------------------------------------------------| -| `adapter` | `string` | **X** | | `CleverAge\CacheProcessBundle\Adapter\AdapterInterface` service identifier | -| `key` | `string` | **X** | | index where retrieving data | +| Code | Type | Required | Default | Description | +|-----------|----------|:--------:|---------|-------------------------------------------------------------------------------------------| +| `adapter` | `string` | **X** | | Code of the [adapter](../adapter.md) to read from (see `AdapterInterface::getCode()`) | +| `key` | `string` | **X** | | Key of the cache item to read, must be a valid PSR-6 key (can be overridden by the input) | Examples -------- +* Read a fixed key + ```yaml # Task configuration level -code: +get: service: '@CleverAge\CacheProcessBundle\Task\GetTask' options: adapter: 'memory' key: 'key2' + outputs: [debug] +``` + +* Read a key given by the process context (`-c sku:"'ABC-001'"`) and stop the branch if it is missing + +```yaml +# Task configuration level +get: + service: '@CleverAge\CacheProcessBundle\Task\GetTask' + options: + adapter: 'catalog' + key: '{{ sku }}' + outputs: [skip_missing] +skip_missing: + service: '@CleverAge\ProcessBundle\Task\SkipEmptyTask' + outputs: [debug] ``` + +* Read a key computed from the input + +```yaml +# Task configuration level +build_key: + service: '@CleverAge\ProcessBundle\Task\TransformerTask' + options: + transformers: + mapping: + mapping: + key: + code: '[sku]' + outputs: [get] +get: + service: '@CleverAge\CacheProcessBundle\Task\GetTask' + options: + adapter: 'catalog' + key: '' # Overridden by the input +``` + +Notes +----- + +* `adapter` and `key` are required at configuration level, even when they are always given by the input: set them to + a placeholder value (e.g. `key: ''`). +* The values coming from the input are not validated by the options resolver. +* A missing key and an item stored with a `null` value both output `null`. Chain a + [SkipEmptyTask](https://github.com/cleverage/process-bundle/blob/main/docs/reference/tasks/skip_empty_task.md) to + stop the branch when nothing is found. +* The input is replaced by the cached value: the rest of the input is not transmitted to the next tasks. diff --git a/docs/reference/tasks/set_task.md b/docs/reference/tasks/set_task.md index 5799a4f..5180d7e 100644 --- a/docs/reference/tasks/set_task.md +++ b/docs/reference/tasks/set_task.md @@ -1,90 +1,95 @@ SetTask -=============== +======= -Set data on cache adapter. - -Adapter reference --------------- - -* [Documentation](../adapter.md) +Stores a value in a cache [adapter](../adapter.md). The key and the value are usually given by the input, so the task +can store every item of an iterable flow (e.g. each line of a file). Task reference -------------- -* **Task Service**: `CleverAge\CacheProcessBundle\Task\SetTask` +* **Service**: `CleverAge\CacheProcessBundle\Task\SetTask` Accepted inputs --------------- -`array`: inputs are merged with task defined options. +`array` or empty (`null`): if the input is not empty, it is merged over the resolved options (`array_merge`), so +input keys `adapter`, `key` and `value` override the options of the same name. Other input keys are ignored. + +Any other non-empty input (e.g. a `string`) triggers a `\TypeError`. Possible outputs ---------------- -none +`null`: the task does not set any output, the next tasks (if any) receive `null`. Options ------- -| Code | Type | Required | Default | Description | -|-----------|----------|:--------:|----------|----------------------------------------------------------------------------| -| `adapter` | `string` | **X** | | `CleverAge\CacheProcessBundle\Adapter\AdapterInterface` service identifier | -| `key` | `string` | **X** | | index where storing data | -| `value` | `mixed` | **X** | | data to store | +| Code | Type | Required | Default | Description | +|-----------|----------|:--------:|---------|--------------------------------------------------------------------------------------------| +| `adapter` | `string` | **X** | | Code of the [adapter](../adapter.md) to write to (see `AdapterInterface::getCode()`) | +| `key` | `string` | **X** | | Key of the cache item to store, must be a valid PSR-6 key (can be overridden by the input) | +| `value` | `mixed` | **X** | | Value to store, must be serializable by the adapter (can be overridden by the input) | Examples -------- +* Store a constant value + ```yaml # Task configuration level -code: - set: - service: '@CleverAge\CacheProcessBundle\Task\SetTask' - options: - adapter: 'memory' - key: 'key1' - value: - - column1: value1-1 - column2: value2-1 - column3: value3-1 +set: + service: '@CleverAge\CacheProcessBundle\Task\SetTask' + options: + adapter: 'memory' + key: 'key1' + value: + - column1: value1-1 + column2: value2-1 + column3: value3-1 ``` +* Store each item of a flow, using its `key` column as cache key and the whole item as value + ```yaml # Task configuration level -code: - data: - service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask' - outputs: [ format ] - options: - output: - - key: 'key1' - column1: value1-1 - column2: value2-1 - column3: value3-1 - - key: 'key2' - column1: value1-2 - column2: value2-2 - column3: value3-2 - - key: 'key3' - column1: '' - column2: null - column3: value3-3 - format: - service: '@CleverAge\ProcessBundle\Task\TransformerTask' - options: - transformers: +data: + service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask' + options: + output: + - key: 'key1' + column1: value1-1 + column2: value2-1 + - key: 'key2' + column1: value1-2 + column2: value2-2 + outputs: [format] +format: + service: '@CleverAge\ProcessBundle\Task\TransformerTask' + options: + transformers: + mapping: mapping: - mapping: - key: - code: '[key]' - value: - code: '.' - outputs: [ set ] - - set: - service: '@CleverAge\CacheProcessBundle\Task\SetTask' - options: - adapter: 'memory' - key: '' # overrided by input' - value: '' # overrided by input + key: + code: '[key]' + value: + code: '.' + outputs: [set] +set: + service: '@CleverAge\CacheProcessBundle\Task\SetTask' + options: + adapter: 'memory' + key: '' # Overridden by the input + value: ~ # Overridden by the input ``` + +Notes +----- + +* `adapter`, `key` and `value` are required at configuration level, even when they are always given by the input: + set them to a placeholder value (e.g. `key: ''`, `value: ~`). If the input does not override the placeholder key, + the empty key throws a `Psr\Cache\InvalidArgumentException`. +* The values coming from the input are not validated by the options resolver. +* No expiration is set on the item: its lifetime is the default lifetime of the adapter (see + [Adapter](../adapter.md#notes)). +* The item is saved immediately (`save()`, not `saveDeferred()`), an existing item with the same key is overwritten.