Can an XC7Z020 run a Transformer? A Smart Zynq SP2 board did, according to the project's published measurements: its six-layer model generated one character every 129.5 ms on average. This was a character-level model trained on Shakespeare text with a 65-character vocabulary. Its characters per second are therefore not comparable to the word-piece token rates quoted for general-purpose language models.
How it fit on the device #
The design reused one compute core for projection, attention and the feed-forward network instead of giving each operation separate hardware. That saved logic, though the operations then had to run in sequence. Previously calculated key and value data sat in DDR memory. For each new character, the board processed the new step without recalculating the entire history.
The post-route figures show the tradeoff: about 51.9% of LUTs, 114 of 220 DSP blocks and 77.9% of BRAM were in use. On-chip memory was closer to its limit than logic. A longer context or larger model would require a fresh cache, DDR traffic and timing budget; spare LUTs alone would not settle the question.
What the speed figure is good for #
The team compared board output with a fixed-point Python reference and reported no mismatches across 200 generated characters. That is a useful check on the implementation, not a measure of the model's language ability. The records do not include matched board-power measurements either, so they cannot establish an energy advantage over another platform.
For a class project or a narrowly defined prototype, the approach is worth studying: size the model and its memory path first, then decide how much hardware to share. A product with a larger model, longer context or guaranteed throughput needs measurements on its own board and workload.
The exact ordering code matters as much as the headline device name. The project's Vivado target was xc7z020clg484-2. The listed XC7Z020-1CLG400I and XC7Z020-2CLG400I use CLG400 packages, so they are not same-package matches for that board. Check the approved BOM, speed grade and temperature grade before asking for a lot or delivery quote.
