From 888917e1e9e6af5b0f6c79bf7e502ab5532278ef Mon Sep 17 00:00:00 2001 From: Oliver Kriska Date: Wed, 10 Jun 2026 13:33:40 +0200 Subject: [PATCH] Parallelize locale merging in mix gettext.merge mix gettext.merge processes locales sequentially, but each locale's merge is fully independent: every locale task writes only to its own //LC_MESSAGES/*.po files, the POT files at the root of pot_dir are only read, and Gettext.Merger is stateless (pure functions over Expo structs). Switch merge_all_locale_dirs/3 to Task.async_stream over locales, mirroring the existing Task.async_stream over POT files within a locale in merge_dirs/5. The return value change (list -> :ok via Stream.run/1) is safe: the only caller is merge_messages_dir/3, whose result is discarded in run/1. The --locale single-locale path is untouched. A crashed locale task raises out of Stream.run/1, matching the sequential version's failure behavior. Mix.shell() output is GenServer-backed, so concurrent lines serialize correctly; the only observable change is that progress lines from different locales can interleave, as already happens in Gettext.Compiler's parallel paths. Benchmark on a 16-locale / 11-domain Phoenix app (160 PO files, ~4,700 msgids, Elixir 1.20.1 / OTP 29, Apple Silicon): mix gettext.merge priv/gettext --no-fuzzy before: 9.4s wall (55% CPU) after: 1.9-2.4s wall (341-381% CPU) ~4.9x speedup with byte-identical output files (clean git status over committed PO state after both runs). The win scales with locale count. --- CHANGELOG.md | 4 ++++ lib/mix/tasks/gettext.merge.ex | 14 +++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f7d3808..6cd9c7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## Unreleased + + * Parallelize per-locale merging in `mix gettext.merge`. + ## v1.0.2 * Only skip manifest removal on Elixir v1.19.3+ diff --git a/lib/mix/tasks/gettext.merge.ex b/lib/mix/tasks/gettext.merge.ex index 49efebc..3371bba 100644 --- a/lib/mix/tasks/gettext.merge.ex +++ b/lib/mix/tasks/gettext.merge.ex @@ -191,9 +191,17 @@ defmodule Mix.Tasks.Gettext.Merge do end defp merge_all_locale_dirs(pot_dir, opts, gettext_config) do - for locale <- File.ls!(pot_dir), File.dir?(Path.join(pot_dir, locale)) do - merge_dirs(locale_dir(pot_dir, locale), pot_dir, locale, opts, gettext_config) - end + pot_dir + |> File.ls!() + |> Enum.filter(&File.dir?(Path.join(pot_dir, &1))) + |> Task.async_stream( + fn locale -> + merge_dirs(locale_dir(pot_dir, locale), pot_dir, locale, opts, gettext_config) + end, + ordered: false, + timeout: :infinity + ) + |> Stream.run() end def locale_dir(pot_dir, locale) do