⚡ Bolt: Optimize RequestMetrics.to_dict() serialization#7036
⚡ Bolt: Optimize RequestMetrics.to_dict() serialization#7036
Conversation
Co-authored-by: ZeyuChen <1371212+ZeyuChen@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
Thanks for your contribution! |
|
|
There was a problem hiding this comment.
Pull request overview
该 PR 旨在优化引擎高频路径中 RequestMetrics.to_dict() 的序列化开销，避免 dataclasses.asdict() 递归触发 copy.deepcopy() 带来的性能损耗，从而降低每次输出序列化的 CPU 成本。
Changes:
- 将
RequestMetrics.to_dict()从asdict(self)替换为手写遍历__dataclass_fields__的序列化逻辑，并对 primitives / list / dict / nested dataclass 做分支处理。 - 新增
.jules/bolt.md记录本次性能优化的学习与行动项。
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| fastdeploy/engine/request.py | 优化 RequestMetrics.to_dict() 的序列化实现以提升热路径性能 |
| .jules/bolt.md | 记录此次优化的经验总结与后续行动建议 |
| import dataclasses | ||
| import copy | ||
|
|
There was a problem hiding this comment.
RequestMetrics.to_dict() 看起来是高频热路径，但这里在函数内部执行 import dataclasses / import copy（每次调用都会走一次 import 语句，虽然有缓存但仍有额外开销），与“优化序列化开销”的目标相冲突。建议把依赖提升到模块级（文件顶部）或至少缓存 is_dataclass/asdict/deepcopy 的局部引用，避免在每次序列化时触发 import 逻辑。
| result = {} | ||
| for k in self.__dataclass_fields__: | ||
| v = getattr(self, k) | ||
| if type(v) in (int, float, str, bool, type(None)): | ||
| result[k] = v | ||
| elif dataclasses.is_dataclass(v): | ||
| if hasattr(v, "to_dict"): | ||
| result[k] = v.to_dict() | ||
| else: | ||
| result[k] = dataclasses.asdict(v) | ||
| elif isinstance(v, list): | ||
| # NOTE: this assumes lists do not contain nested dataclasses | ||
| result[k] = list(v) | ||
| elif isinstance(v, dict): | ||
| # NOTE: this assumes dicts do not contain nested dataclasses | ||
| result[k] = dict(v) | ||
| else: | ||
| result[k] = copy.deepcopy(v) | ||
| return result |
There was a problem hiding this comment.
当前仓库 tests 中没有覆盖 RequestMetrics.to_dict()（以及其嵌套 dataclass / list / dict 分支）的单测：例如 tests/engine/test_request.py 只验证了 Request.to_dict()，但它并不调用这里的新实现。建议补充一个单测，至少断言：1）基础字段序列化结果正确；2）speculate_metrics 等嵌套 dataclass 能被正确转成 dict；3）list/dict 分支的浅拷贝行为符合预期，避免后续字段形态变化时静默引入序列化回归。
Motivation
dataclasses.asdictrecursively usescopy.deepcopy(), which introduces significant overhead for high-frequency serialization paths.RequestMetrics.to_dict()is called on every request output in the LLM serving engine.Modifications
dataclasses.asdict(self)inRequestMetrics.to_dict()with a manual iteration over__dataclass_fields__.listanddictobjects (assuming they don't contain nested dataclasses, which fits the current usage)..to_dict()orasdict()for nested dataclass objects.Usage or Command
No new usage.
Accuracy Tests
All existing request tests pass (
pytest tests/engine/test_request.py).Checklist
.jules/bolt.md.PR created automatically by Jules for task 9575716354012606787 started by @ZeyuChen