Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,15 @@ Since `formatjs_extract` is a custom rule, you can attach Bazel aspects to it fo
- Aggregating messages across multiple targets
- Custom reporting and analysis

### Automatic extraction from runtime dependencies

Use [`formatjs_message_collector_aspect`](docs/automatic-extraction.md) to extract sources
when their configured dependencies enable FormatJS, then collect catalogs through
transitive depsets. Supply runtime labels or a runtime predicate; no
language-rule dependency is required. Custom rules/aspects can reuse
`formatjs_extract_action` instead of duplicating CLI wiring. Select extraction roots
with `collect_at` for rule adapters or `formatjs_root` for CLI use.

### Message Aggregation Aspect

The `formatjs_aggregate_aspect` collects and merges extracted messages from a target and all its dependencies into a single JSON file using `jq`.
Expand Down
142 changes: 142 additions & 0 deletions docs/automatic-extraction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Dependency-triggered extraction

`formatjs_message_collector_aspect` follows configured `deps` and `srcs`, detects
Intl usage, and carries source plans and explicit catalogs in depsets. Extraction
runs only at selected roots. Libraries do not flatten transitive source sets.

## CLI use

Define `build/i18n.bzl`:

```starlark
load("@rules_formatjs//formatjs:defs.bzl", "formatjs_message_collector_aspect")

rust_messages = formatjs_message_collector_aspect(
name = "rust_messages",
runtime_labels = [Label("@my_crates//:formatjs_intl")],
source_extensions = ["rs"],
id_interpolation_pattern = "[sha512:contenthash:base64:10]",
extract_source_location = True,
flatten = True,
)
```

Select the extraction root explicitly:

```sh
bazel build //app:main \
--aspects=//build:i18n.bzl%rust_messages \
--aspects_parameters=formatjs_root=//app:main \
--output_groups=all_messages
```

`formatjs_root` must be the actual target's absolute canonical label, not an alias
or relative label. It scopes materialization to that consumer. Other nodes only
propagate source plans and existing catalogs.

Runtime labels must be caller-created `Label` objects. This preserves the caller's
repository mappings. Aliases, including configurable aliases, are resolved in the
target configuration through private resolver dependencies. They are not traversed
for Intl detection, so configuring a resolver does not opt unrelated graphs into
extraction. Their files are not extraction inputs and are not built for detection.
Resolution can configure runtime dependencies in otherwise unrelated graphs; those
configured targets are shared within each configuration.

Alternatively, supply `is_runtime(target, ctx)` to inspect native language providers
or a `FormatjsRuntimeInfo` marker. Predicate-only adapters add no resolver dependency
on the runtime. No language build rules are imported by this ruleset.

## Rule adapters

Rule-propagated aspects use a `collect_at(target, ctx)` predicate instead of the CLI
parameter. Choose application or catalog-consumer roots, not every library:

```starlark
def _at_application(_target, ctx):
return ctx.label == Label("//app:main")

rust_messages = formatjs_message_collector_aspect(
name = "rust_messages",
runtime_labels = [Label("@my_crates//:formatjs_intl")],
source_extensions = ["rs"],
collect_at = _at_application,
extract_source_location = True,
)
```

This configuration also works from the CLI without `--aspects_parameters`.
Consumers access `FormatjsMessagesInfo.catalogs` on a selected root:

```starlark
load("@rules_formatjs//formatjs:defs.bzl", "FormatjsMessagesInfo")

def _catalogs_impl(ctx):
return [DefaultInfo(files = ctx.attr.target[FormatjsMessagesInfo].catalogs)]

source_catalogs = rule(
implementation = _catalogs_impl,
attrs = {"target": attr.label(aspects = [rust_messages])},
)
```

The public provider has two fields: `catalogs` and `uses_runtime`. At intermediate
nodes, `catalogs` contains only explicit catalogs. A private provider carries source
plans and explicit coverage; consumers do not depend on that internal state.

## Catalogs and execution

Roots flatten candidate and covered sources once. Ownership, extension and explicit
coverage filters run before extraction actions are registered. Excluded generated
artifacts are not action inputs, so their producers do not run for catalog builds.
Partial explicit metadata excludes only its declared sources.

Without source locations, uncovered sources use one batch extraction per root.
With `extract_source_location = True`, each file gets its own catalog: the native
CLI keeps only one location for duplicate IDs within a batch. Per-file catalogs
preserve declarations across files and allow unchanged files to reuse cached
extraction actions. Duplicate occurrences within one file retain the native CLI's
behavior. Catalog order is deterministic; no intermediate merge actions are added.
Consumers choose their duplicate-ID and source-location merge policies. Output-name
hash collisions receive deterministic suffixes instead of failing analysis.

Existing `FormatjsExtractInfo` catalogs are collected unchanged. Automatic outputs
are exposed through `FormatjsMessagesInfo` and `all_messages`; they do not replace
or add a singular `FormatjsExtractInfo` on the underlying target.

Automatic mode requires only the native FormatJS CLI toolchain. No host shell,
Python or filtering toolchain is used. Without runtime labels or a predicate, the
factory only collects existing catalogs and requires no CLI toolchain. The existing
`message_collector_aspect` entrypoint keeps that collection-only behavior.

## Scope and options

By default, only targets in the main repository contribute new source candidates.
External dependencies can enable extraction and provide explicit catalogs without
having their implementation scanned. Override `should_extract(target, ctx)` to
change that target boundary. Rejected sources stay excluded when ancestors forward
them through source wrappers. Independent allowed declarations of the same source
remain eligible. Source plans retain configured identity, so allowing a generator
in one configuration cannot enable its rejected output in another. Plans stay in
depsets until the extraction root; each configured node is visited once there.
Only files owned by a contributing target's repository are eligible, including
when source labels point into another repository.

`source_extensions` defaults to `ts`, `tsx`, `js`, `jsx`, `mjs`, `cjs`, `rs`, and `py`.
`dependency_attributes` controls detection/collection edges; `source_attributes`
controls source inputs and is also traversed for extraction metadata. Both label
and label-list attributes work. Aliases and `select()` are handled after resolution.

Other keyword arguments are options accepted by `formatjs_extract_action`.
Use one configured aspect per collection pass. Mixed-language adapters can combine
runtime recognition and source extensions in that aspect. Give separate instances
unique `name` values to distinguish output files.

Archive ownership, translation projection, catalog merging and runtime installation
remain consumer decisions.

## Custom extraction actions

`formatjs_extract_action(ctx, srcs, out, **options)` registers the native action and
returns `FormatjsExtractInfo`. `srcs` accepts a list or depset of source Files; the
caller declares the output and CLI toolchain. The existing `formatjs_extract` rule
uses the same helper, including its options and response-file handling.
1 change: 1 addition & 0 deletions examples/simple/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ local_path_override(

# Test utilities (not required by rules_formatjs users)
bazel_dep(name = "aspect_bazel_lib", version = "2.22.5")
bazel_dep(name = "jq.bzl", version = "0.7.1")
37 changes: 37 additions & 0 deletions examples/simple/auto_extract/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
load("@jq.bzl//jq:jq.bzl", "jq_test")
load("@rules_formatjs//tests/auto_extract:fixtures.bzl", "catalogs", "graph")
load(":extract.bzl", "extract")

graph(
name = "consumer",
srcs = [
"messages.py",
"@rules_formatjs//tests/auto_extract:foreign.py",
],
deps = ["@rules_formatjs//tests/auto_extract:runtime_with_sources"],
)

catalogs(
name = "messages",
target = ":consumer",
)

jq_test(
name = "main_repository_only_test",
file1 = ":messages",
file2 = "expected.json",
filter1 = "[.[] | (.defaultMessage // .message)]",
)

extract(
name = "custom_rule",
srcs = ["messages.py"],
out = "custom_rule.json",
)

jq_test(
name = "consumer_action_test",
file1 = ":custom_rule",
file2 = "expected.json",
filter1 = "[.[] | (.defaultMessage // .message)]",
)
1 change: 1 addition & 0 deletions examples/simple/auto_extract/expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["Consumer message"]
16 changes: 16 additions & 0 deletions examples/simple/auto_extract/extract.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""Consumer-defined rule using the public action helper."""

load("@rules_formatjs//formatjs:defs.bzl", "formatjs_extract_action")

def _extract_impl(ctx):
info = formatjs_extract_action(ctx, ctx.files.srcs, ctx.outputs.out)
return [DefaultInfo(files = depset([info.messages])), info]

extract = rule(
implementation = _extract_impl,
attrs = {
"srcs": attr.label_list(allow_files = [".py"]),
"out": attr.output(mandatory = True),
},
toolchains = ["@rules_formatjs//formatjs_cli:toolchain_type"],
)
3 changes: 3 additions & 0 deletions examples/simple/auto_extract/messages.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from intl import define_message

GREETING = define_message(id="consumer.message", default_message="Consumer message")
1 change: 1 addition & 0 deletions formatjs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

exports_files([
"defs.bzl",
"collector.bzl",
"extract.bzl",
"compile.bzl",
"aggregate.bzl",
Expand Down
32 changes: 3 additions & 29 deletions formatjs/aspects.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
This module demonstrates how to create aspects that work with formatjs_extract.
"""

load(":collector.bzl", "formatjs_message_collector_aspect")
load(":extract.bzl", "FormatjsExtractInfo")

def _message_stats_aspect_impl(target, ctx):
Expand Down Expand Up @@ -113,32 +114,5 @@ message_validator_aspect = aspect(
""",
)

def _message_collector_aspect_impl(target, ctx):
direct = [target[FormatjsExtractInfo].messages] if FormatjsExtractInfo in target else []
transitive = []
for attr_name in ["deps", "srcs"]:
for dep in getattr(ctx.rule.attr, attr_name, []):
if OutputGroupInfo in dep and hasattr(dep[OutputGroupInfo], "all_messages"):
transitive.append(dep[OutputGroupInfo].all_messages)

return [OutputGroupInfo(all_messages = depset(direct, transitive = transitive))]

message_collector_aspect = aspect(
implementation = _message_collector_aspect_impl,
attr_aspects = ["deps", "srcs"],
doc = """Collect original message catalogs without registering merge actions.

Follows both deps and srcs, including libraries that carry extraction
metadata through a source target. Transitive depsets retain shared catalogs
only once without flattening the dependency graph at each target. No jq
toolchain is required. Empty graphs expose an empty all_messages group.

Consumers can merge the original catalogs with their own duplicate-ID and
source-location policies. This aspect does not extract messages from raw
sources: extraction targets must provide FormatjsExtractInfo.

Usage:
bazel build //path/to:target --aspects=//formatjs:aspects.bzl%message_collector_aspect \\
--output_groups=all_messages
""",
)
# Preserve the collection-only entrypoint without requiring a CLI toolchain.
message_collector_aspect = formatjs_message_collector_aspect()
Loading
Loading