Vitis Unified Backend - #1376
Conversation
…i wrapper for vitisUnified partial backend and build the skeleton code for other generation section
… the vitis_writer.py since I worked with DFX4ML project, so I revert them because it is out of scope of this PR.
vloncar
left a comment
There was a problem hiding this comment.
Review of hls4ml PR #1376 "Vitis Unified Backend" against the VivadoAccelerator history
Date: 2026-09-09. PR head: 41b483764 (branch VitisUnifiedClean, 139 commits, based on main e99874595).
1. What was examined and how
- Issues: 77 issue threads found by keyword search (VivadoAccelerator, accelerator backend, pynq, axi_stream, axi master, zcu102, alveo, bitfile, VitisAccelerator, VitisUnified). 39 are about accelerator/board flows.
- Pull requests: 29 threads that touch accelerator backends (
#349 #420 #508 #509 #552 #588 #597 #626 #642 #646 #653 #683 #694 #724 #752 #755 #812 #851 #986 #991 #1037 #1134 #1160 #1198 #1347 #1448 #1451 #1483 #1495), including their review comments. - Git history: all 64 commits on
mainthat touchedhls4ml/backends/vivado_accelerator/,hls4ml/writer/vivado_accelerator_writer.py,hls4ml/templates/vivado_accelerator/,docs/backend/accelerator.rst(2021-06 to 2026-08), each diff read and classified as accelerator-only, shared refactor, or merge. - VivadoAccelerator code on
mainand all files of PR #1376, read in full. - Experiments: 32 conversions run one after the other with the PR checkout in an isolated venv. Each case: convert -> write ->
compile()(g++ only) ->predict()-> compare with the plain Vitis backend on the same model.
Line numbers below refer to the PR head.
2. What the VivadoAccelerator history shows (the problems a successor must not repeat)
Grouped by pattern; each one is backed by issues, PR discussion and commits.
| # | Pattern | Evidence |
|---|---|---|
| H1 | Writer rewrites files the parent writer already generated, by matching strings in generated C++/Tcl and renaming a temp file over the original (modify_build_script, write_wrapper_test, full rewrite of project.tcl, delete-and-rebuild of the tar). Every parent change then needs a mirrored change; several broke silently. |
vivado_accelerator_writer.py:236-359,361-398,407-411; commits ffbffd47d (inherited from pynq_writer), 6c300c053, 8c2ec95e9, 8c7a6b0b2, d1bdd3781, 4b7e12de8; issue #1157 (missing maximum_size in the duplicated project.tcl), #1194 (drifted build_lib.sh copy, fixed by copying again in #1198); stale rule 'unsigned short' in line left after c0f882c43. vloncar in #683: "I would strongly discourage this style of code in the writer. What's the point of templates if they are blank and all the logic is written by the writer". |
| H2 | String replacement on user-controlled names. Test bench rewrite replaces every occurrence of the input/output variable name and type name on any line that contains them. A layer named input broke the project (#442); the myproject_cosim match only worked when the project was literally named myproject (eb255871a, fixed in #626). |
vivado_accelerator_writer.py:296-310; #442, #626, #1495 (hard-coded type name inside a format string). |
| H3 | Constructor bypasses the parent's __init__ (super(VivadoBackend, self).__init__), so every parent registration must be copied by hand; a missing copy broke every RNN model (#722, fixed by #724 adding one line). |
vivado_accelerator_backend.py:10-12; #722, #724, 78ce8aadf. |
| H4 | Accelerator configuration object is built inside the writer, so board/interface/type/I-O-count validation happens at write time, and the writer mutates the user's config (Part silently replaced by the board's part, with a warning printed on every default run). | vivado_accelerator_writer.py:420, vivado_accelerator_config.py:20-26,55-58; commits c28512e84, 3433c10f0; #489, #491, #516. |
| H5 | Options accepted but not implemented: interface='axi_master', 'axi_lite', driver='c' are documented and stored, then crash at write time (depth=a emitted for axi_master). create_initial_config has no **kwargs, so Vivado writer options (namespace, write_tar) raise TypeError. |
vivado_accelerator_backend.py:98-147, vivado_accelerator_config.py:138-140. |
| H6 | Type hard-coding and type bugs in the wrapper: ap_fixed wrapper type emitted as fixed<W,I,TRN,WRAP,0> (cannot compile); input/output bit widths written swapped into project.tcl; width rounded to a multiple of 8 while keeping the integer bits; Alveo RTL fixed at 32 bits; #pragma HLS DATA_PACK (Vivado HLS only). |
vivado_accelerator_writer.py:37-92,171,396-397, vivado_accelerator_config.py:85-98, krnl_rtl_int.sv:34. |
| H7 | Single input, single output, single sample: asserts in the config, [0] everywhere, BRAM-weight ports of the core (BramFactor) not forwarded so the wrapper does not compile (#1406). |
vivado_accelerator_config.py:55-62, vivado_accelerator_writer.py:130; #1406. |
| H8 | Per-board copy-paste: pynq-z2 and zcu102 drivers byte-identical and patched in parallel (455e7345d then 1ebd1c1df seven weeks later, #420); every board request arrives as a copied driver plus a copied block-design Tcl (#752, #755, #812) and stalls; Tcl scripts hard-code part, board_part version, IP version, 100 MHz clock; drivers depend on literal block-design names (hier_0.axi_dma_0). vloncar in #755: adding a board must not require code changes. |
templates/vivado_accelerator/*/; #752, #755, #812, #933 (board files not installed), #616 (Vivado 2020.1-only packaging). |
| H9 | Build step: os.chdir + os.system inside try/except that can never fire, no exit-code check, bitfile=True needs export=True too; a failed Vivado run still returns a report. |
vivado_accelerator_backend.py:46-96; #508 review (vloncar asked for subprocess), #1347 review. |
| H10 | FIFO depth optimization pass calls model.write() and model.build() from inside an optimizer pass; reviewers asked for pre/post passes with the writer scheduled in the flow (#642, jmitrevs). The pass was copied per backend (#509 review), and later copied again into Vitis (#1037) and Coyote (#1347). |
backends/vivado_accelerator/passes/fifo_depth_optimization.py; #509, #642, #1037. |
| H11 | Accelerator knowledge leaked into shared files: _axi top name and in_local/out_local FIFO names in the shared Vivado build_prj.tcl, nnet_helpers.h (copy_data_axi), report parser. |
templates/vivado/build_prj.tcl:62, nnet_helpers.h; commit 3559053b7. |
| H12 | Silent hardware failures and no validation path: all-zero or hung boards with no maintainer answer (#282, #776, #913, #1026, #1048, #1339); driver ignores its constructor arguments and returns its internal buffer (#420 diff); no timeout on DMA waits; Alveo driver frees the overlay after the first profiled call; allocate_mem has no return (#771). |
driver files; #282 #771 #776 #913 #1026 #1048 #1339. |
| H13 | No tests, weak docs: no pytest references VivadoAccelerator at all; accelerator.rst example calls a non-existent hls4ml.build(); CLI hls4ml build does not know the backend (#401, #922); board JSON missing from the package (#515). |
test/pytest, docs/backend/accelerator.rst:52; #401, #515, #922. |
Lessons stated by maintainers in those threads that a successor should satisfy: configuration validated at conversion time, own templates instead of rewriting parent output, subprocess with exit codes, a board added as data (no code change), a driver whose API is stable across boards, a software validation path that uses the same data conversion as the driver, and a test in CI.
3. Findings on the VitisUnified backend (PR #1376)
Severity: blocker = wrong result, crash or unusable output for a configuration the backend accepts; major = plausible use silently does not work, or the design repeats a pattern that caused the VivadoAccelerator problems; minor = code quality. "Hn" marks the VivadoAccelerator pattern it repeats. "Verified" = reproduced in the experiments. Each finding ends with a Fix: proposal.
3.1 Wrong or silently wrong behaviour
| # | Sev | Finding and suggested fix | Evidence |
|---|---|---|---|
| U1 | blocker | predict() returns all zeros, without any message, when the numpy dtype does not match input_type. The bridge has both <project>_float and <project>_double entry points (as ModelGraph._get_top_function requires), but the writer emits a body only for the one whose dtype equals input_type; the other function is empty, so the outputs stay zero. Verified three times (float types + float64 input, double types + float32 input, both AXI modes). This is exactly the "board returns zeros with no explanation" class of #1026/#1048, now reproduced in software. Fix: emit both bodies and convert at the boundary, the way VivadoWriter.write_bridge does for every dtype with nnet::convert_data: copy the incoming array into a local buffer of the wrapper's type, call the wrapper, copy back. If a conversion is deliberately unsupported, the generated body must abort with a message instead of being empty. |
hls4ml/writer/vitis_unified_writer.py:340 (if dtype == ...get_input_type(): with no else); templates/vitis_unified/myproject_bridge.cpp:55-69. |
| U2 | blocker | input_type='double' with axi_mode='axi_stream' does not compile: the backend's AXI helpers hard-code hls::stream<hls::axis<float,0,0,0>> while the wrapper's packet type is hls::axis<double,...>. Verified: no matching function for call to convert_data_axis<double,double,N_IN>(...). H6 (type hard-coding) again. Fix: template the helpers on the packet type (template <class pack_T, class src_T, size_t SIZE> void convert_data_axis(src_T *src, hls::stream<pack_T> &dst)) and instantiate them with the dma_data_packet typedef the writer already generates; the writer knows the typedef name at every call site. |
templates/vitis_unified/nnet_utils/nnet_helpers_axi.h:14,23,37; vitis_unified_writer.py:359,365,688. |
| U3 | blocker | BRAM weights (Strategy: Resource with a small BramFactor) do not compile in either AXI mode: the core top function gains weight arguments, the wrapper calls myproject(stream_in, stream_out) without them. The writer even collects model_brams for the test bench includes, so the case was known. Verified: too few arguments to function 'void myproject(...)'. Same defect as VivadoAccelerator #1406 (H7). Fix: either forward the ports (append model_brams to the wrapper signature and inner call, load them in the test bench and bridge, and give them m_axi/s_axilite interface pragmas so weights become run-time loadable — the feature #113 asked for in 2018), or reject the configuration at conversion with "BramFactor weights are not supported by VitisUnified". Silent acceptance followed by a C++ error is the one wrong option. |
vitis_unified_writer.py:548-555,310,658; templates/vitis_unified/myproject_axi_master.cpp:41-45, myproject_axi_stream.cpp:38-43. |
| U4 | major | create_initial_config swallows every Vivado/Vitis writer option (namespace, write_tar, write_weights_txt, write_emulation_constants, tb_output_stream) through **_ and does not forward them to super().create_initial_config(part, clock_period, clock_uncertainty, io_type). Verified: namespace='nsone' and write_tar=True give Namespace: null, WriteTar: false and no tar file, with no warning. VivadoAccelerator raised TypeError (H5); silent is worse. The writer does contain namespace code paths that can never be reached this way. Fix: name the backend-specific parameters explicitly, collect the rest as **kwargs, and pass them through: super().create_initial_config(part=part, clock_period=clock_period, clock_uncertainty=clock_uncertainty, io_type=io_type, **kwargs). Then any option the parent grows is inherited automatically. |
vitis_unified_backend.py:119-145; vitis_writer.py:58-70. |
| U5 | major | build() returns None: no report is parsed; the CI helper run_synthesis_test then fails its assert data and expected_keys.issubset(...) check on the None return (save_report(None) itself writes null without raising), and synthesis_helpers.py has neither a build_args nor an EXPECTED_REPORT_KEYS entry for this backend, so a generic synthesis test calls build() with all flags False, runs nothing, and fails on the empty result. The public build() signature also differs from every other backend: validation, export removed; reset, vsynth accepted but ignored; synth=True always also runs package; cosim=True without synth=True runs cosim on a project that was never synthesized. Fix: keep the parent's keyword set and raise or warn on the unsupported ones instead of dropping them from the signature; implement reset (delete vitis_unified_project and the linker work dir); make cosim imply synth the way fifo_opt already implies cosim; parse vitis_unified_project/reports/* (csynth and cosim reports) into the dict shape parse_vivado_report produces and return it; add a build_args['VitisUnified'] entry to synthesis_helpers.py. |
vitis_unified_backend.py:17-101; test/pytest/synthesis_helpers.py:137-165. |
3.2 Architecture: the same anti-patterns, in a new place
| # | Sev | Finding and suggested fix | Evidence |
|---|---|---|---|
| U6 | major | Constructor bypasses two parents: super(VivadoBackend, self).__init__ skips VitisBackend.__init__ and VivadoBackend.__init__ and re-does _register_layer_attributes() and _register_flows() by hand. This is H3, the pattern that produced #722; any future registration added to VitisBackend.__init__ is silently missed here. (nghielme raised the same point on #1134.) Fix: the parent __init__ methods only set the name and call the two _register_* methods, and _register_flows is already overridden in this class — so plain super().__init__(name='VitisUnified') cannot be used only because the parents hard-code their names. Either give the backend constructors a name parameter upstream (small, benefits every derived backend), or keep the bypass but add a comment listing exactly what is being re-done, plus a test that fails when VitisBackend.__init__ gains a step (compare the set of registered flows/attributes against a fresh VitisBackend). |
vitis_unified_backend.py:11-15. |
| U7 | major | Config object created inside the writer (_set_unified_config in write_hls), so board, axi_mode, driver, type equality and input/output-count checks all run at write time. Verified: unknown board, axi_stream with 2 inputs, and input_type != output_type all pass convert_from_keras_model and fail only in compile(); the last one with an assert whose message reads "must be the same type different". create_initial_config accepts any board and silently substitutes the zcu102 part. H4 again (without the Part override, which is an improvement). Fix: validate everything that does not need the graph (board, axi_mode, driver, types) in create_initial_config, and everything that needs the graph (input/output counts vs axi_mode) in a ModelOptimizerPass registered in the backend's default flow, so convert_from_keras_model fails immediately; replace the asserts with exceptions (asserts vanish under python -O) and fix the message. The writer then only consumes an already validated config. |
vitis_unified_writer.py:63-66,787-799; vitis_unified_config.py:11-20,61,67,70-81; vitis_unified_backend.py:134-143. |
| U8 | major | Parent writer output is left in the project as dead or misleading files. VitisWriter.write_hls still copies build_prj.tcl and build_opt.tcl (the Tcl flow this backend does not use) and VivadoWriter.write_test_bench still writes <project>_test.cpp next to the backend's own myproject_test.cpp; the two test benches call different top functions (with the default project name the file names coincide and the backend's version happens to overwrite the parent's). project.tcl is not written at all, which is why write_board_script_override had to be turned into a no-op. write_tar runs up to three times. Verified in the generated directories. This is the mirror image of H1: instead of rewriting parent files, the subclass leaves them in place and adds its own beside them. Fix: override write_build_prj_override and write_build_opts as documented no-ops (the mechanism already used for write_board_script_override), and override write_test_bench so the backend's wrapper test bench is the test bench — written once, under <project>_test.cpp (which also removes the myproject_test hard-coding, U12). A user opening the project should not have to guess which files the flow actually uses. |
vitis_unified_writer.py:22-23,35-38,140-147,649-653,787-799; vitis_writer.py:59-76; vivado_writer.py:1134-1148. |
| U9 | major | Build step rewrites generated files and derives configs by substring editing. prepare_sim_config_file copies hls_kernel_config_{csim,cosim}.cfg over hls_kernel_config.cfg at build time and string-replaces a placeholder; the csim variant is produced by commenting out any template line containing enable_fifo_sizing or -DRTL_SIM. vitis-comp.json points at hls_kernel_config.cfg, which does not exist until build() runs, so the advertised "open the workspace in the Vitis IDE" does not work on a freshly written project. Fix: resolve both cfg files completely at write time (the FIFO-sizing flag is a config value the writer already has, not something only build() knows) and pass the wanted file directly to v++ --config / vitis-run --config; point vitis-comp.json at the csim cfg. Then build() never edits generated files and the IDE works from the moment the project is written. |
vitis_unified_backend.py:103-117; vitis_unified_writer.py:167-196,209-218. |
| U10 | major | FIFO depth pass calls model.write() and model.build() from inside transform() (H10; copied from the Vitis pass, itself copied from VivadoAccelerator). The cus_hls_prj_path default builds a path (<out>/<project>/_prj/solution1) that does not exist for this backend. Fix: apply the #642 proposal where it is inherited from — split the Vitis pass into a pre-pass (enlarge FIFOs), the normal write+build scheduled by the flow, and a post-pass (read the measured depths, set them) — and let this backend inherit the split instead of adding a fourth copy of the pattern. |
backends/vitis_unified/passes/fifo_depth_optimization.py:26-27,61-72,104-110. |
| U11 | major | Absolute output paths baked into generated files: syn.file={OUTDIR}/..., tb.file={OUTDIR}/... in both cfg files and configFiles in vitis-comp.json. The project cannot be moved, tarred, or built on another machine (a common user workflow in the issue history). VivadoAccelerator's Tcl used relative paths. Fix: emit paths relative to the directory the tool is started from (the backend controls cwd for every command it launches) or relative to the cfg file, and compute them with os.path.relpath. A generated project must build after mv. |
templates/vitis_unified/hls_kernel_config.cfg:7-18; vitis_unified_writer.py:182-183,216-217. |
| U12 | major | Hard-coded names and tool/platform versions (H8): test bench file name myproject_test regardless of project name (verified: myproject left in the cfg files of a project named mynet); XILINX_VITIS default /opt/Xilinx/Vitis/2023.2; link_system.sh hard-codes _x/link/vivado/vpl/prj/prj.gen/sources_1/bd/vitis_design/hw_handoff/vitis_design.hwh and needs xclbinutil and vivado on PATH without checking; package.ip.version=1.0.0 fixed and the driver binds to xilinx.com:hls:<top>:1.0, so the Version config that #851 added is ignored; flow_target=vivado fixed; clock_uncertainty default 12.5% (Vivado) although the Vitis backend uses 27%. Fix: make _get_sim_file_name return <project>_test (folds into U8); derive XILINX_VITIS from the environment and fail with a clear message when unset; glob for the .hwh under _x/link instead of a fixed path; substitute the Version config into package.ip.version and the driver's bindto; inherit the clock-uncertainty default from the Vitis backend; check for xclbinutil/vivado in build() next to the existing v++/vitis-run checks. |
vitis_unified_writer.py:81-82, vitis_unified_config.py:105-108, hls_kernel_config.cfg:19-21, link_system.sh:7-12, vitis_unified_backend.py:124. |
| U13 | major | Adding a board still requires editing package files: boards live in supported_boards.json inside the package plus a per-board directory of drivers and Tcl; there is no way to pass a custom .xpfm/.xsa path (the PR description still advertises xpfmPath, the code no longer has it). This is the exact situation of #752/#755/#812 (H8) that vloncar asked to solve with an extension point. Fix: accept a platform argument in create_initial_config holding a path to an .xpfm or .xsa; when given, skip the board lookup for the platform (the board entry then only supplies the part and driver choice, both of which can also be arguments). Since v++ links against a platform file, this makes an unlisted board usable with zero code changes — the main structural advantage this backend has over VivadoAccelerator, currently not exposed. |
vitis_unified_config.py:5-58,86-109; PR description "configuration" section. |
| U14 | major | Drivers are per-board copies again (H8): zcu102/python_drivers/* and kv260/python_drivers/* are byte-identical (verified with diff). The AXI-master driver computes the register map in Python (0x10 base, 12-byte stride per pointer, batch-size register after the pointers) instead of reading it from the generated .hwh/HLS driver header; the AXI-stream driver hard-codes axi_dma_0 and axi_dma_0/s2mm_introut; both call PL.reset() on every predict, use one dtype for input and output, and keep a long docstring about ap_fixed encode/decode although only float/double are allowed. c_drivers are empty placeholders and driver is asserted to be python at write time (H5). Fix: keep one driver template per AXI mode under templates/vitis_unified/drivers/ and drop the per-board directories (they are already identical); on the board, take the register offsets from pynq's ip.register_map (pynq parses the .hwh, so the offsets are always right) instead of the Python-side arithmetic; move PL.reset() to __init__ or make it an argument; separate input and output dtype; delete the ap_fixed docstring or implement encode/decode; remove the c_drivers placeholders until a C driver exists. |
templates/vitis_unified/zcu102/python_drivers/axi_master_driver.py.hls4ml:53-110,116-137; axi_stream_driver.py.hls4ml:62-66; vitis_unified_writer.py:616-621; vitis_unified_config.py:61. |
3.3 Minor findings
| # | Sev | Finding and suggested fix | Evidence |
|---|---|---|---|
| U15 | minor | String-matching template fill with fragile ordering: elif 'myproject' in line before PROJECT_FILE_NAME; 'STREAM_BUF_IN_SZ' in line: line.replace('VAL', ...); markers consumed differently in .cpp (replaced) and .h (appended). Works for the shipped templates, breaks the moment a template line contains two placeholders. The AXI-master template also contains a stray label mem_rd: before a declaration. Fix: one substitution style: unique {PLACEHOLDER} tokens applied with unconditional line.replace on every line (no elif chains keyed on substrings), insertion markers only for multi-line blocks; remove the stray labels. |
vitis_unified_writer.py:302-320,410-451,482-569,575-584; myproject_axi_master.cpp:11,27. |
| U16 | minor | Copy-paste and dead code: exec_dir = self.get_vitis_hls_dir(model) (should be get_vitis_hls_exec_dir), _, _, _, _ = ...get_corrected_types(), write_tar override that only calls super(), _get_kernel_declaration always uses the axi_stream name even in axi_master mode (same length today, wrong by construction), get_XPFMPath/get_platform_path aliases, getattr defaults for attributes that are always set. Fix: delete each; make _get_kernel_declaration take the mode from _is_axi_master(). |
vitis_unified_writer.py:22-23,84-87,201,655; vitis_unified_config.py:163-176. |
| U17 | minor | The 64-character kernel-name guard is a real improvement, but the message "Project name must not exceed 18 characters" hard-codes the arithmetic of the current suffixes. Fix: compute the allowed length from the actual suffix lengths and put both numbers in the message. | vitis_unified_writer.py:40-47. |
| U18 | minor | The AXI-stream framing contract is implicit. The last-beat condition (q == batch_size-1) && ((chunk_idx+1)*(elem_idx+1) == N_OUT) is correct only because equality is reachable solely at both loop maxima — nothing says so; TLAST is raised once per batch, not per sample; incoming TLAST is ignored and exactly batch_size*N_IN beats are consumed (a short transfer hangs, the failure mode of #776/#913/#1339). Fix: write the condition as (chunk_idx == N_OUT/OUTPUT_LAYER_TYPE::size - 1) && (elem_idx == OUTPUT_LAYER_TYPE::size - 1), and state the contract (beats expected per start, TLAST behaviour in both directions) in the docs and in the driver docstring. |
templates/vitis_unified/myproject_axi_stream.cpp:24-35,3-18. |
| U19 | minor | The m_axi pragmas set depth=<one sample> while the kernel reads batch_size samples per start; depth only affects simulation, so cosim with a batch larger than 1 is either misleading or warns. Fix: set depth from a documented cosim batch (or max_batch_size config value) and add a comment that depth is simulation-only. |
vitis_unified_writer.py:485-498. |
| U20 | minor | TKEEP/TSTRB side channels are carried but handled asymmetrically. The AXI-stream packet type is the full hls::axis<T,0,0,0>: the output path sets keep = -1 on every beat (the Vitis-example idiom, functionally fine for full-width float/double payloads), but TSTRB is never written anywhere (undefined on the wire), the simulation helpers create packets without setting keep at all (simulation and hardware traffic disagree), and the input path ignores TKEEP, so a sparse beat from a master would be consumed as if full, silently. Since this backend only ever transfers whole 4- or 8-byte elements, TKEEP carries no information here. Fix: use the signal selection the bundled ap_axi_sdata.h provides — hls::axis_data<T, AXIS_ENABLE_LAST> — so the interface has only TDATA/TVALID/TREADY/TLAST: the keep = -1 assignment disappears and no side channel can be left undriven or inconsistent (absent TKEEP means "all bytes valid" per the AXI-Stream specification, and IPI ties off the DMA's TKEEP input; VivadoAccelerator's own data+last struct relied on the same behaviour; one board test should confirm the v++ stream_connect path accepts it). If TKEEP is kept for platform compatibility, use `AXIS_ENABLE_LAST |
AXIS_ENABLE_KEEP, set keep` in every place a packet is created (wrapper and both simulation helpers), and state in the U18 contract that input TKEEP is not honored mid-packet — which matches what AXI DMA MM2S produces for aligned buffers. |
3.4 Tests and documentation (required before merge)
| # | Sev | Finding and suggested fix | Evidence |
|---|---|---|---|
| U21 | major | Test coverage is far too narrow for a backend that aims to be the default. The suite covers one Conv2D U-Net and one 2-in/2-out CNN, io_stream, Latency, float, clock_period=10 hard-coded; bitstream tests are skipped by default; nothing asserts on generated code except the driver port counts. None of U1-U4 would have been caught. Fix: a small matrix that covers many axes with few builds, all in the g++-only software path so it runs in normal CI: (a) parametrize the existing predict-parity test over axi_mode x input_type in {float,double} (4 cases — catches U2 and any future type regression); (b) one test that calls predict with the wrong numpy dtype and asserts a correct result or a raised error, not zeros (catches U1); (c) one Resource + BramFactor conversion asserting either a working compile or a clean conversion-time rejection (catches U3); (d) one conversion with project_name='custom', namespace, write_tar=True asserting the tar exists, the namespace appears in the generated header, and no myproject literal remains outside nnet_utils (catches U4, U12); (e) one invalid-configuration test asserting that unknown board / mismatched types / multi-input axi_stream fail at convert_from_keras_model, not at compile() (locks in U7 once fixed). That is roughly seven cheap tests; the existing synthesis/cosim/bitstream tests stay opt-in as they are. |
test/pytest/test_vitis_unified.py. |
| U22 | major | Documentation gaps. docs/backend/vitis_unified.rst says an unknown board falls back to the zcu102 part, but the config object rejects unknown boards at write time (contradiction with the code); nothing says how this backend relates to VivadoAccelerator (default for AMD SoC boards? deprecation? coexistence?) and accelerator.rst still presents VivadoAccelerator unchanged; the PR description advertises xpfmPath, which no longer exists; the build() flags and their differences from other backends are undocumented; the AXI-stream framing contract (U18) and the exact meaning of in_stream_buf_size/out_stream_buf_size in each mode are not stated; there is no "how to add a board / bring your own platform" section although that is the request that dominated the VivadoAccelerator backlog (#752/#755/#812). Fix: correct the unknown-board paragraph to match the code (or the code to match it, per U7/U13); add a short positioning paragraph to accelerator.rst naming VitisUnified as the recommended flow for AMD SoC boards and what remains VivadoAccelerator-only; document build()'s arguments and outputs; document the stream contract and buffer sizes; add the bring-your-own-platform section once U13 lands (they belong in the same change). |
docs/backend/vitis_unified.rst:70-74; docs/backend/accelerator.rst; PR description. |
3.5 What the PR does better than VivadoAccelerator
Own templates for wrapper, bridge and test bench (no rewriting of parent-generated files, no os.rename); subprocess.Popen with return-code checks and per-step logs instead of os.chdir/os.system; set -e in the link script; multiple inputs and outputs in axi_master mode (verified in software: 2-in/2-out and 1-in/3-out give results identical to the Vitis backend); whole-batch execution per kernel start with a batch_size register; interrupt-driven driver; v++ platform linking (a platform is data, not a hand-maintained block-design script per board, at least for zcu102/axi_master); a current tool (Vitis HLS, .cfg flow) instead of the discontinued Vivado HLS and DATA_PACK; backend-local simulation headers copied by the backend's own writer step (ap_axi_sdata.h taken from AMD's Apache-2.0 Xilinx/hls-utilities, with the source recorded in the file); a kernel-name-length guard; a pytest file; a documentation page; Trace works through the wrapper (verified in both modes).
4. Adaptability beyond the shipped examples
Results of the 32 sequential cases (software path only: write, g++ compile, predict, compared with the Vitis backend on the same model). "identical" = max abs difference 0.0.
| Situation | Result |
|---|---|
Dense MLP, axi_master / axi_stream |
works, identical |
MLP with sizes 9 -> 11 -> 7 (axi_stream) |
works, identical |
| Conv1D + Flatten + Dense, both modes | works, identical |
| Conv2D U-Net (the PR's example), both modes | works, identical |
LSTM + Dense (axi_master) |
works, identical |
2 inputs / 2 outputs, axi_master |
works, identical |
1 input / 3 outputs, axi_master |
works, identical |
2 inputs / 2 outputs, axi_stream |
rejected, but only at compile() (write time), not at conversion |
io_type='io_parallel' |
rejected at conversion ("io_type must be io_stream"). VivadoAccelerator supported io_parallel; small Dense models lose the option recommended in #425 |
input_type=output_type='double', axi_master |
works, identical |
input_type=output_type='double', axi_stream |
compile error (U2) |
types double, predict with float32 array |
all zeros, no error (U1) |
types float, predict with float64 array, either mode |
all zeros, no error (U1) |
input_type='float', output_type='double' |
rejected at write time by an assert with a garbled message (U7) |
input_type='ap_fixed<16,6>' |
rejected at conversion. VivadoAccelerator accepted it (with bugs, H6); the new backend is float/double only |
Trace=True + trace() , both modes |
works, all layer outputs returned |
Resource strategy with BramFactor=0 (BRAM weight ports), both modes |
compile error (U3) |
project_name='mynet', both modes |
works; but the test bench is still named myproject_test.cpp and the cfg files reference it (U12) |
namespace='nsone' |
silently ignored (U4) |
write_tar=True |
silently ignored (U4) |
IOType placed in hls_config instead of the kwarg |
not seen by the backend at all (kwarg default used); generic hls4ml behaviour |
board pynq-z2 (not in the JSON) |
rejected at write time, after conversion succeeded (U7); no way to supply a platform path (U13) |
board kv260, axi_stream |
works |
no axi_mode given |
defaults to axi_master, works |
Not executed, judged from the code: PyTorch/ONNX frontends go through the same ModelGraph and should behave like Keras (the backend never looks at the frontend); multigraph is rejected explicitly; a board other than zcu102/kv260 needs a new package directory with Tcl and drivers, exactly as before; Alveo/PCIe cards are out of scope (the #991 VitisAccelerator effort was to be merged into this PR according to #991, but nothing of it is here); a C/C++ host driver (#1339) is absent.
Overall: the software path adapts well across layer types and input/output counts (in axi_master mode), which is a real step beyond VivadoAccelerator. It does not adapt across io types (io_stream only, by design), interface types (float/double only, with double broken for axi_stream and dtype mismatches silently returning zeros), weight storage (BRAM ports), writer options (silently dropped), or boards/platforms (package data only).
5. What should change, in order
Every numbered finding (U1-U22) appears in at least one item below, so working through the list covers the whole review, not only the items marked as required. All of these are proposals: the authors should push back on any item where they consider the finding not relevant or the proposed fix the wrong approach — a reasoned disagreement recorded on the PR is a valid way to close an item.
Required before merge:
- Fix the silent-zero bridge: both dtype entry points convert instead of one being empty (U1).
- Template the AXI helpers on the packet type so
doubleworks inaxi_streammode (U2). - Forward BRAM weight ports or reject
BramFactorat conversion (U3). - Expand the tests with the small wide-coverage matrix of U21 (about seven g++-only cases: axi_mode x float/double parity, wrong-dtype behaviour,
BramFactor, custom name + namespace + tar, invalid configurations rejected at conversion). - Fix the documentation (U22): unknown-board paragraph matching the code, positioning against VivadoAccelerator in
accelerator.rst,build()arguments and outputs, the AXI-stream contract and buffer sizes, and removal of stale options (xpfmPath) from the description.
Needed for it to be a credible default:
- Validate at conversion time and forward parent options: config checks in
create_initial_configplus a graph-level pass, exceptions instead of asserts,**kwargspass-through, parent constructors called or their bypass tested (U4, U6, U7). - Make the build honest and compatible: parent-compatible
build()keywords, a returned report, cfg files resolved at write time, relative paths so the project survives a move (U5, U9, U11). - Boards and platforms as data: a
platformpath argument, one shared driver template with the register map read from the hardware description, no per-board copies, no hard-coded tool paths (U12, U13, U14). - Take a position on the capability gaps against VivadoAccelerator listed in section 4 (
io_parallel,ap_fixedinterface types): either plan them, or document them as deliberate non-goals as part of the U22 documentation work.
Cleanups and longer term:
- Neutralize the unused parent writer steps and unify the test bench under the project name (U8, U12).
- Split the FIFO pass in the Vitis backend (pre-pass / flow-scheduled build / post-pass, the #642 design) and inherit it here (U10).
- The minor items: one substitution style in the writer, dead-code removal, clearer guard message, explicit stream contract in the wrapper source, simulation-depth comment, side-channel selection instead of per-beat
keep = -1(U15-U20).
6. Verdict
The PR is a real improvement in tool generation (Vitis HLS, v++ platform linking), in the build step (subprocess, logs, error propagation), and in interface generality (multiple inputs and outputs, batch execution), and it touches no file outside its own backend and template directories. It does not yet meet the bar of "improvement in all areas": two silent-failure modes in the software path (dtype mismatch returns zeros — U1; writer options dropped without a message — U4), two accepted configurations that do not compile (double + axi_stream — U2; BramFactor weights — U3), test coverage too narrow to have caught any of these (U21), documentation that contradicts the code in places and does not position the backend against VivadoAccelerator (U22), capability narrower than VivadoAccelerator in three places (io_stream only, float/double only, no BRAM weights), and five of the VivadoAccelerator patterns that caused most of that backend's issue history (constructor bypass, write-time validation, hard-coded names and tool versions, per-board driver copies, FIFO pass building the project from inside a pass). Items 1-5 of section 5 are required before merge; items 6-9 are what would make it a credible default.
…type - add test_predict_any_numpy_dtype (review U1)
…ks in axi_stream - add axi_stream double to test_predict_any_numpy_dtype (review U2)
…e wrapper build - add vitisunified:ip default flow with validate_bram_weights, test and doc note (review U3)
…ame and invalid configs - cover items (d) and (e) of review U21; they flip to passing when U4, U12 and U7 land
…nd limitations - unknown boards are rejected, buffer size unit, positioning against VivadoAccelerator (review U22)
…nd support namespace in the wrapper - namespace, write_tar and the other writer options are no longer dropped (review U4)
…lement reset - add parse_vitis_unified_report reusing the Vivado csynth and cosim parsers, CI entries and a test (review U5)
…eter - VitisUnified calls super().__init__ instead of bypassing two parents (review U6)
…te time - checks in create_initial_config and a validate_config pass, asserts replaced by exceptions (review U7)
…r edits generated files - three cfg variants, vitis-comp.json points at the csim one, test added (review U9)
…lative paths - the generated project builds after a move; the test moves it and checks every path (review U11)
…gument, no hard-coded names or tool paths - one driver per AXI mode, project-named test bench, IP version from the config, XILINX_VITIS from the environment (review U12, U13, U14)
… scheduled by the flow - VitisUnified inherits the split and only overrides the HLS project path; test with a faked co-simulation (review U10)
…tcl, and write the tar once - documented no-op overrides, the tar is made at the end of write_hls (review U8)
…moved, explicit stream contract - guard message computes the allowed length, TKEEP set on every packet, sim-only depth comment (review U15 to U20)
…ions - fixtures, conversion, write, compile, Vitis HLS, bitstream; bodies unchanged
- their AXI DMA is 32 bits wide; use float, or a user platform with a 64-bit DMA - drop test_fifo_depth_passes, covered by test_fifo_depth, and comment the config tests
- the AXI DMA never completes a transfer without TKEEP, so the review's drop-TKEEP option is not possible
…e saved prediction - verified on kv260 for axi_stream, axi_master and a 2-in/2-out axi_master model
- one entry holds the last dimension of the tensor, so a sample takes N / channels entries
|
Thanks for the detailed review. All 22 findings are addressed in the 20 commits after the reviewed head Required before merge
Needed for it to be a credible default
Cleanups and longer term
Changes outside the VitisUnified backend
No behaviour change for the Vivado, Vitis, VivadoAccelerator or Coyote backends. Test results
KV260 (PYNQ 3.0.1), generated drivers, batch of 10, compared with the software prediction saved by the test:
Script: |
vloncar
left a comment
There was a problem hiding this comment.
Thanks for the detailed response, this is exactly what I was hoping to see.
I checked the update independently at the current head. I went through all the new commits against your table, ran the new test suite (same as your numbers), and reran the 32 configurations from my original review. I didn't run the hardware tests myself, but I trust your numbers. All 22 findings are addressed. The input array with the wrong dtype now predicts correctly instead of silently returning zeros, and double on axi_stream and BramFactor weights are rejected at conversion with a clear message instead of failing later in the C++ build. Every invalid configuration I tried now stops at conversion and tells the user what to do, and all 24 valid ones give results identical to the Vitis backend. The platform and part arguments together with the board-independent driver solve the board support problem properly, a board that is not in the list needs no code changes at all.
The places where you went a different way than I (in truth, Claude) suggested (like keeping TKEEP since the DMA needs it, the loop labels, the simulation-only depth) are all fine with me.
One small non-blocking follow-up: docs/backend/accelerator.rst still doesn't mention this backend, so someone reading the VivadoAccelerator page won't learn about its successor. A one-line cross-reference would do, and it can be a separate PR along with other documentation fixes we have planned.
Looks good. Approving.
|
The test failures are unrelated to this PR. Merging. Huge thanks to the team for making this. |
Description
VitisUnified backend
Motivation
The Vitis backend stops at the HLS project. This backend takes an hls4ml model to a file set that is ready for a PYNQ board: bitstream, hardware handoff and Python driver, built with the Vitis Unified flow (
v++compile and package,v++link against a platform). It is meant as the flow for AMD SoC boards with Vitis 2023.2 or newer; models withio_parallelorap_fixedinterfaces stay with VivadoAccelerator.Features
axi_mode:axi_master(kernel accesses DDR itself, multiple inputs and outputs) oraxi_stream(AXI DMA in the platform, one input and one output).One batch per kernel start, interrupt driven; one PYNQ driver per AXI mode with register offsets from the hardware handoff.
build()with the parent keywords:synth,csim,cosim,fifo_opt,vitis_fifo_sizing,bitfile; returns the Vitis report dictionary.Boards as data:
supported_boards.json(zcu102, kv260) or your own platform viaplatform=andpart=.Configuration validated at conversion;
predict()accepts any numpy dtype.Documentation page
docs/backend/vitis_unified.rst.Limitations
io_streamonly;float/doubleinterfaces only, both the same; noBramFactorweights (rejected at conversion);axi_streamwith one input and one output; no multigraph; Python driver only;doubleonaxi_streamneeds a platform with a 64-bit DMA (the shipped ones are 32 bits).Type of change
Tests
test/pytest/test_vitis_unified.py, 41 tests in six sections ordered by flow depth:version, driver contents, own platform, name-length guardaxi_modexfloat/doublex numpy dtype, writer options forwardedreset(fake reports, no Vitis needed)RUN_SYNTHESIS=true): csim, cosim, FIFO depth optimization, both AXI modesRUN_VITIS_UNIFIED_BITSTREAM=true): kv260,axi_stream,axi_master, 2-input/2-outputaxi_masterResults: without Vitis 32 passed, 9 skipped. With Vitis 2023.2 all 41 pass.
Hardware, KV260 with PYNQ 3.0.1, generated drivers, batch of 10, compared with the software prediction saved by the test (
test/board/vitis_unified_hw_test.py):axi_streamaxi_masteraxi_masterTest reproduce
build(log_to_stdout=False)writes<step>_stdout.logand<step>_stderr.login the output directory instead of printing.Checklist
pre-commiton the files I edited or added.AI assistance disclosure
Claude Code (Claude Fable 5.1) was used in this PR; commits from March 2026 onward may contain code, tests or documentation written with it, including all fixes after the first review. Every change was reviewed, and all numbers above come from runs on my machine and my KV260.
Implementation detail
Three stages produce the files to ship:
v++ -c --mode hlsandvitis-run --packageproduce the.xolink_system.shrunsv++ -lagainst the platform, then extractssystem.bitandsystem.hwhfrom the.xclbinTemplate structure
hls4ml/templates/vitis_unified:Output structure
All paths in the generated files are relative, so the output directory can be moved or copied to another machine.
Configuration
Keyword arguments of the converter:
version(default1.0.0) setspackage.ip.versionand the driver'sbindto.Notes
vitis_workspace/<project_name>in the Vitis Unified IDE to debug the kernel; the component points at the csim config.m_axidepthin the AXI-master wrapper is for simulation only, one sample; the test bench sends one sample per kernel start.vitis_workspace/system_link/_x/link/vivado/vpl/prj.Generated warnings
From the U-Net model used in the tests, not from the backend:
Kernel synthesis prints the usual Vitis HLS warnings (unused parameter, deprecated pragma).