Cisco ACI Layer 3 Out (L3Out)
The Layer 3 Out (L3Out) in Cisco ACI is the set of configurations that define connectivity outside of ACI via routing. The ACI fabric is formed from multiple components, including bridge domains (BDs) and endpoint groups (EPGs) to provide L2 connectivity or default gateway functions for a group of endpoints. Another one is the Layer 3 Out (L3Out), which is to provide L3 connectivity between servers connected to ACI and other network domains outside of the ACI fabric through routing protocol or static route.
Cisco ACI was originally built to be a stub network in a data center to manage endpoints. The ACI L3Out was initially designed only as a border between the stub network formed by ACI and the rest of the network, such as intranet, Internet, WAN, and so on, not as a transit network, as shown in Figure 9-1.
Figure 9-1 ACI Fabric as a Stub Network
Due to this stub nature, traffic traversing from one L3Out to another through the ACI network was originally not supported. Beginning with APIC Release 1.1, however, Cisco ACI introduced the Transit Routing feature, which allows the ACI fabric to be a transit network so that traffic can traverse from one L3Out to another L3Out, as shown in Figure 9-2.
Figure 9-2 ACI Fabric as a Transit Network
The border leafs (BLEAFs) are ACI leaves that provide Layer 3 connections to outside networks. Any ACI leaf can be a border leaf. In addition to supporting routing protocols to exchange routes with external routers, the border leaf also applies and enforces policy for traffic between internal and external endpoints.
Three different types of interfaces are supported on a border leaf switch to connect to an external router:
- Layer 3 interface: With a physical interface dedicated to a VRF.
- Sub-interface with 802.1Q tagging: With sub-interface, the same physical interface can be used to provide multiple outside connections for multiple tenants or VRFs.
- Switched Virtual Interface (SVI): With an SVI, the same physical interface that supports Layer 2 and Layer 3 and the same physical interface can be used for a Layer 2 outside connection as well as a Layer 3 outside connection. In addition to supporting routing protocols to exchange routes with external routers, the border leaf also applies and enforces policy for traffic between internal and external endpoints.
Within the ACI fabric, Multiprotocol BGP (MP-BGP) is implemented between leaf and spine switches to propagate external routes within the ACI fabric. The BGP route reflector technology is deployed in order to support a large number of leaf switches within a single fabric. All of the leaf and spine switches are in one single BGP autonomous system (AS). Once the border leaf learns the external routes, it can then redistribute the external routes of a given VRF to an MP-BGP address family VPN version 4 (or VPN version 6 when IPv6 routing is configured). With address family VPN version 4, MP-BGP maintains a separate BGP routing table for each VRF. Within MP-BGP, the border leaf advertises routes to a spine switch, which is a BGP route reflector. The routes are then propagated to all the leafs where the VRFs are instantiated.
The L3Out provides the necessary configuration objects for the following five key functions, which are also displayed in Figure 9-3:
Figure 9-3 The Five Key Components of L3Out
Learn external routes via routing protocols (or static routes).
Distribute learned external routes (or static routes) to other leaf switches.
Advertise ACI internal routes (BD subnets) to outside ACI.
Advertise learned external routes to other L3Outs (Transit Routing).
Allow traffic to arrive from or be sent to external networks via L3Out by using a contract.