FPGA Neural Networks
Article FPGA Neural Networks The inference of neural networks on FPGA devices Introduction The ever-increasing connectivity in the world is generating ever-increasing levels of data.
REFERENCE DESIGN
MAU Accelerator for AI Financial Trading Models
Ultra-low Latency, High Throughput Machine Learning Inference
Well suited to a range of applications in financial services, with deployment scenarios ranging from co-location to offline, the Myrtle.ai MAU Accelerator is provided as IP to run on the latest FPGAs.
Market Data Prediction
Quantitative Trading
Algorithmic Trading
What is the MAU Accelerator IP?
Designed to be integrated into your existing software stack, the IP supports various bit depths for Floating Point, Block Floating Point, Brain Floating Point and Integer formats. Existing models developed in popular frameworks can be imported using the ONNX OSI format.
Benefits
- Deterministic single inference latency ~9us
- High accuracy FP16 precision
- Multiple models can be hosted on a single platform
- Low power (< 100W) for integration in co-location servers
System Examples
Tap to zoom
Performance Examples
- Acceleration of a trading algorithm, using an LSTM-based neural network model with ~10k parameters, achieves a latency of <1µs per timestep, a throughput of over 700k timesteps per second and a capacity of 150 models per accelerator card.
- Acceleration of a small, 64 node, stacked LSTM model, with 2 LSTM layers and ~3M parameters and 20 timesteps achieves an inference latency of 9µs.
About Myrtle.ai
Myrtle.ai has considerable experience in efficient hardware-acceleration of ML models, such as RNN and LSTM networks, using FPGA accelerator cards. These are designed to achieve the highest throughput and lowest cost for inference workloads with very tight latency constraints.
Deliverables
Open-source reference model and export scripts in PyTorch
Example application code for inference
C and Python bindings for MAU Accelerator inference API
FPGA bitstream and source code (conditions apply)
Designed for BittWare Hardware
The MAU Accelerator reference design can be run on a range of BittWare products featuring Intel and Xilinx FPGAs. For deployment, we recommend the ultra-high density TeraBox 1401B, with four cards and an AMD EPYC CPU.
Get more details on MAU performance!
Request a meeting to get in depth on how the MAU Accelerator IP can work for your organization!
Preview of materials we'll share:
