Packet Forwarding within the ACI Fabric

The ACI fabric appears as a single switch to the outside world, capable of bridging and routing. As traffic enters the fabric, ACI encapsulates and applies policy to it, forwards it as needed across the fabric through a spine switch (maximum two hops), and de- encapsulates it upon exiting the fabric. All traffic in the ACI fabric is normalized as VXLAN packets. At ingress, ACI encapsulates external VLAN, VXLAN, and NVGRE packets in a VXLAN packet.

Figure 8-25 shows ACI encapsulation normalization.

Figure 8-25 ACI Encapsulation Normalization

Within the fabric, ACI uses IS-IS and COOP for all forwarding of endpoint-to-endpoint communications. The forwarding across switch nodes is performed based on the Tunnel Endpoint (TEP) IP in the VXLAN encapsulation. IS-IS provides IP reachability among TEP addresses. In case the ingress leaf is not aware of the destination endpoint location (TEP), ACI has a distributed database called the Council of Oracles Protocol (COOP) on each spine that knows all the mapping of endpoints and TEPs. For propagating routing information between software-defined networks within the fabric and routers external to the fabric, ACI uses the Multiprotocol Border Gateway Protocol (MP-BGP). All ACI links in the fabric are active, equal-cost multipath (ECMP) forwarding, and fast-reconverging.

Summary

This chapter discussed Cisco ACI building blocks, deployment models, hardware used in a Cisco ACI solution, Cisco ACI fabric startup discovery, the Cisco ACI policy model, and packet forwarding within the Cisco ACI fabric, including the following points:

• ACI is a spine/leaf network of Nexus 9k switches using the ACI operating system with a management platform called APIC, which provides a single place from which the network can be managed.
• Cisco ACI has many benefits over a traditional network, such as simple spine/leaf architecture, Layer 2 loop avoidance using ECMP, better security using an allow list model, and REST API automation advantage.
• Starting from Cisco ACI 4.1, Cisco ACI supports both two-tier and multitier topologies.
• Leaf switches are the switches to which all endpoints (servers, storage, service nodes, and so on) connect, and they provide the VXLAN tunnel endpoint (VTEP) function. Leaf switches are often categorized by function, such as border leaf, compute leaf, and so on.
• Spine switches interconnect leaf switches and provide the backbone of the ACI fabric.
• The ACI solution supports multiple deployment/architectural models such as Cisco ACI MultiPod, Cisco Nexus Dashboard Orchestrator, Cisco ACI vPod, and so on.
• APICs can be deployed either as physical or virtual appliances.
• The Cisco Nexus 9000 Series switches operate in one of two modes: Cisco Application-Centric Infrastructure (Cisco ACI) or Cisco NX-OS. The Cisco Nexus 9000 Series switches in ACI mode provide the spine and leaf switches that build the fabric.
• During startup discovery, the ACI fabric is brought up in a cascading manner, starting with the leaf nodes that are directly attached to the APIC.
• The ACI policy model enables the specification of application requirements policies. The policy model manages the entire fabric, including the infrastructure, authentication, security, services, applications, and diagnostics.
• A tenant is a container for policies that enable an administrator to exercise domain-based access control. There are four types of tenants: user tenant, common tenant, infrastructure tenant, and management tenant.
• Access policies govern the operation of switch access ports that provide connectivity to resources such as storage, compute, Layer 2 and Layer 3 (bridged and routed) connectivity, virtual machine hypervisors, Layer 4 to Layer 7 devices, and so on.
• Fabric policies govern the operation of the switch fabric ports, including functions such as Network Time Protocol (NTP) server synchronization, Intermediate System to Intermediate System Protocol (IS-IS), Border Gateway Protocol (BGP) route reflectors, Domain Name System (DNS), and so on.
• A tenant is a logical container for application policies that enable an administrator to exercise domain-based access control.
• Virtual routing and forwarding (VRF) is a tenant network and defines a Layer 3 address domain.
• An application profile defines the policies, services, and relationships between endpoint groups (EPGs). An EPG is a managed object that is a named logical entity that contains a collection of endpoints. Endpoints are devices that are connected to the network directly or indirectly.
• A bridge domain represents a Layer 2 forwarding construct within the fabric. While a VRF defines a unique IP address space, that address space can consist of multiple subnets. Those subnets are defined in one or more BDs that reference the corresponding VRF.
• An attachable entity profile (AEP) represents a group of external entities with similar infrastructure policy requirements.
• EPGs can only communicate with other EPGs according to contract rules. Label, filter, and subject managed objects enable mixing and matching among EPGs and contracts so as to satisfy various applications or service delivery requirements.
• All traffic in the ACI fabric is normalized as VXLAN packets. As traffic enters the fabric, ACI encapsulates it in VXLAN packets and applies policy to it, forwards it as needed across the fabric through a spine switch, and de-encapsulates it upon exiting the fabric.