CAPI stands for "Coherent Accelerator Processor Interface" which enables FPGA to access Host memory by virtual address. You can find more introduction about this interface on https://developer.ibm.com/linuxonpower/capi/. It is an important feature to develop hardware accelerators in heterogeneous computing. In this document, the "hardware accelerators" are built on FPGA.

A complete accelerator has software part (APP, or Applications) running on CPU Processor and the hardware part (AFU, Acceleration Function Unit) running on FPGA chip. APP and AFU are sharing host memory, that means, they both can read and write the 2^64 range of virtual memory address. To make it happen, CAPI technology has a CAPP (Coherent Acceleration Processor Proxy) logic unit in Processor chip, and also needs a PSL (Processor Service Layer) logic unit in FPGA chip. For CAPI1.0 and CAPI2.0, the interconnection between processor and FPGA is using PCIe physical links and PCIe form factor. CAPI1.0 uses PCIe Gen3x8. CAPI2.0 uses PCIe Gen4x8 or Gen3x16. OpenCAPI is not covered in this document. Visit https://opencapi.org for more information.

This document only applies to the cards using Xilinx FPGA chips. A customer FPGA card needs to have a PSL module (Processor Service Interface) to become a "CAPI-enabled" card. This PSL module is provided by OpenPower Foundation. For CAPI1.0, PSL module and the surrounding board specific modules are provided in the form of a routed dcp file (Xilinx Vivado design checkpoint). It's usually called b_route_design.dcp. For CAPI2.0, PSL is an IP package with encrypted source code. It's named like ibm.com_CAPI_PSL9_WRAP_2.00.zip. They can be downloaded at https://www.ibm.com/systems/power/openpower. From the menu, select "CAPI","Coherent Accelerator Processor Interface (CAPI)" or directly click the "CAPI" icon to go to the CAPI section. Then download the appropriate files depending on your target system being POWER8 (CAPI 1.0) or POWER9 (CAPI 2.0). You need to register an IBM ID to download them. Users can develop CAPI accelerators in two modes: HDK and SNAP. HDK is the abbreviation of Hardware Development Kit. As shown in the diagram below, on the FPGA side, the Xilinx Vivado project includes two parts: BSP (Board Supporting Package, containing PSL module) and AFU (Acceleration Function Unit). How to generate BSP will be introduced in Chapter

AFU is where to implement user-defined functions. The developer working on AFU needs to understand the protocol between AFU and BSP, which is called PSL/AFU interface specification. Please refer to CAPI1.0 PSL Spec and CAPI2.0 PSL Spec or search "PSL/AFU interface" in your web browser. When developing an acceleration, PSLSE (PSL Simulation Engine) is also needed to make a software-hardware co-simulation which guarantees the correctness of accelerator design. When deploying the acceleration to OpenPower servers, it requires user library libcxl and kernel module cxl to run the application. In all, HDK mode will provide the maximum control, utilization of resources and shortest latency. However, SNAP mode simplifies and standardizes the application development significantly and is more recommended.

SNAP is the abbreviation of Storage, Networking and Analytics Programming. It is an open-source acceleration development framework https://github.com/open-power/snap. The benefits are: On the FPGA side, SNAP framework adds a bridge to provide AXI interface to developers. So the developer can focus on acceleration function logic design, and doesn't need to study the details of PSL interface specification. AXI is the defacto industry standard for on-chip bus interconnections and is part of AMBA (Advanced Microcontroller Bus Architecture). It also provides DDR SDRAM controller and an optional NVMe controller. The developer can use the card memory or storage directly. SNAP supports using HLS (High Level Synthesis) to develop the acceleration functional unit ("Hardware Action" in yellow box). Developers can write C++ functions and Vivado HLS will compile/convert them to Verilog or VHDL design automatically. A new layer of user library "libsnap" provides more convenient APIs. SNAP is an integrated developing environment that the developer can configure, create Vivado project, run co-simulation or build bitstream with simple commands. Many action examples help new developers to get started.

Equipping the new FPGA card with SNAP framework needs a few additional steps and is introduced in Chapter This document focuses on CAPI2.0. For CAPI1.0 enablement, please contact capi-snap-doc@mailinglist.openpowerfoundation.org for more information. It is assumed the reader knows how to work on Vivado Project and SNAP already. Many materials on how to develop an accelerator with SNAP can be found in "docs" folder of snap github or other webpages so they are not discussed in this document.