Juniper Networks EVPN Implementation for Next-Generation Data Center Architectures

Transcription

1 EVPN VPN 1

2 VXLAN EVPN...3 VXLAN EVPN...5 EVPN...6 EVPN MAC... 7 EVPN... 7 EVPN...10 EVPN BUM EVPN BUM EVPN BUM EVPN EVPN MAC...16 EVPN...17 EVPN VXLAN EVPN VXLAN EVPN MAC/IP EVPN EVPN 1 ESI EVPN 2 ESI LS2 LS3 LS1 H LS2 LS3 LS1 H LS2 LS3 LS1 H MAC EVPN MAC EVPN EVPN EVPN EVPN ESI EVPN EVPN MAC EVPN EVPN L3 Clos L3 Clos ASN L3 Clos ASN EVPN/VXLAN

3 Spanning Tree Protocol STP MC-LAG Transparent Interconnection of Lots of Links TRILL 2 Virtual Extensible LAN VXLAN 3 IP VXLAN-EVPN L2 L2 1 IT 3 1 VXLAN VXLAN 1, VXLAN EVPN L2 VLAN VLAN レイヤー 2 レイヤー 2 レイヤー 2 VM VM VM VM VM 1 レイヤー 2 ー 1 3

4 VLAN IEEE 802.1ad VLAN 4,096 VLAN 1 1 4,096 VLAN VLAN VM VXLAN VXLAN IETF RFC7348 VXLAN L3 L2 VXLAN 24 ID VNID 1,600 VXLAN VXLAN Juniper Networks QFX5100 QFX10000 EX9200 MX 3D VXLAN Spanning Tree Protocol STP レイヤー 3 IP ファブリック VXLAN オーバーレイ ポッド ポッド ポッド アプリ 1 アプリ 2 アプリ 3 アプリ 4 アプリ 5 2 ー ー ポッド アプリ ー VXLAN L2 2 VXLAN L3 Clos VXLAN VXLAN 2 L2 2 VTEP 2 MAC MAC MAC 4

5 BGP MP-BGP MP-BGP L2 MAC L3 IP MAC IP BGP L2 MAC L3 IP VPN EVPN EVPN IP VXLAN MPLS EVPN VXLAN EVPN 3 Software Defined Networking SDN EVPN EVPN LAN VPLS L2VPN MPLS IP VXLAN VXLAN EVPN MPLS EVPN MPLS EVPN EVPN MAC MAC IP ARP VXLAN / L3 VMTO VM BGP L3VPN L2VPN DCI BGP EVPN 3 EVPN RFC 4364 BGP/MPLS IP Virtual Private Networks (VPNs) RFC 4761 Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling RFC 7432 BGP MPLS-Based Ethernet VPN 5

6 EVPN スパインスイッチプロバイダコア (P) ホストカスタマーエッジ (CE) リーフスイッチプロバイダエッジ (PE) VTEP VXLAN トンネルエンドポイント VXLAN トンネル MP-IBGP セッション EVPN インスタンス (EVI) = 仮想スイッチ ブリッジドメイン (BD) = VLAN 3 EVPN 3 L3 Clos 2 2 N IP / EVI = EVPN EVPN PE MAC-VRF PE MAC MAC-VRF RD ES = EVPN 1 PE 1 ES ESI = ES ESI PE EVPN ESI CE MP-BGP EVPN EVPN 5 1 AD - EVI ESI CE CE ESI 2 MAC/IP - EVPN EVPN NLRI IP MAC MAC 3 - BUM PE PD VLAN ESI 4 - ESI 2 PE CE / / PE ES 5 IP - IP 6

7 EVPN MAC EVPN MP-BGP 2 EVPN EVPN 2 スパインスイッチ S1 スパインスイッチ S2 ホスト H1 リーフスイッチ L /32 リーフスイッチ L /32 ホスト H /24 1:1:1:1:1:1 MP-IBGP /24 5:5:5:5:1 NLRI ルートタイプルート識別子 (RD) イーサネットセグメント識別子 (ESI) イーサネットタグ ID MAC アドレス IP アドレス MPLS ラベル 1 + MPLS ラベル 2 MAC/IP アドバタイズメントルート ( タイプ 2) リーフスイッチ L2 上の赤い EVI の RD 0( シングルホームドホスト ) ブリッジドメインのグローバル VXLAN VNID ホスト H2 の MAC アドレス (5:5:5:5:5:1) ホスト H2 の IP アドレス ( )** VNID ネクストホップ 拡張コミュニティ 他の属性 ( 発信元 AS-Path Local-Pref など ) L2 のループバック IP アドレス ( ) ルートターゲット ( 赤 )... 4 リ ート (MAC/IP アドバタイズメント EVPN タイプ 2 ルート ) 4 L2 L2 H2 MAC L2 DHCP ARP IP MAC H2 H1 H1 H2 BUM ARP L1 H2 MAC L1 H2 MAC H1 H2 ES EVPN L2 H2 MAC 2 VXLAN VNID MP-BGP EVPN 1 EVPN MC-LAG MC-LAG 2 PE EVPN PE L3 Clos EVPN ESI EVPN 1 7

8 イーサネット自動検知 ( タイプ 1) ルートイーサネットセグメント (ES) 単位 : マルチパスと迅速なコンバージェンス ルートタイプ ルート識別子 (RD) イーサネット自動検知ルート ( タイプ 1) リーフスイッチ LS2 上の EVI の RD (LS2 の IP を含む ) NLRI イーサネットセグメント ID(ESI) 0:1:1:1:1:1:1:1:1:1 イーサネットタグ ID MPLS ラベル拡張コミュニティネクストホップ MAX-ET 0 ESI ラベル拡張コミュニティ : シングルアクティブフラグ = false (0) ESI ラベル = null LS2 のループバック IP アドレス 他の属性 ( 発信元 AS-Path Local-Pref など ) MP-IBGP H1 リーフスイッチ LS2 H2 LS1 LS3 MP-IBGP ESI 0:1:1:1:1:1:1:1:1:1 イーサネット自動検知 ( タイプ 1) ルート 5:EVPN タイプ 1 アドバタイ メント ESI 5 H2 LAG L2 LS2 LS3 LS2 LS3 LS1 1 L2 ESI 1 ESI MAC MAC 2 LS2 LS3 H2 MAC MAC/IP アドバタイズメント ( タイプ 2) ルート再検討 ( マルチホームドホストの場合 ) ルートタイプ ルート識別子 (RD) MAC/IP アドバタイズメントルート ( タイプ 2)... イーサネットセグメント識別子 (ESI) 0:1:1:1:1:1:1:1:1:1 NLRI イーサネットタグ ID MAC アドレス IP アドレス MPLS ラベル 1 + MPLS ラベル 2 VNID ホスト H2 の MAC アドレス (5:5:5:5:5:1) ホスト H2 の IP アドレス ( ) VNID ネクストホップ他の属性 ( 発信元 AS-Path Local-Pref など ) LS3 のループバック... MP-IBGP H1 リーフスイッチ LS2 H2 LS3 LS1 MP-IBGP LAG /24 ESI 5:5:5:5:5:1 0:1:1:1:1:1:1:1:1:1 ネクストホップ LS3 を使用する MAC/IP アドバタイズメント ( タイプ 2) ルート 6:EVPN タイプ 2 アドバタイ メントと ESI 8

9 6 LS1 H2 MAC 2 LS3 ESI 0:1:1:1:1:1:1:1:1:1 LS1 H2 2 ESI LS2 LS1 H2 各 VXLAN トンネルはスパインスイッチ上でマルチパス化される SS1 LS2 H1 LS1 H2 SS2 LS3 LAG /24 5:5:5:5:5:1 ブリッジドメイン転送テーブル宛先 MAC ネクストホップ ECMP:VNID 1 以上 5:5:5:5:5:1... VTEP1... VTEP2 7:LS2 LS3 LS1 H2 マルチパス 7 VXLAN H2 LS1 LS2 LS3 LS2/LS3 H2 MAC 8 EVPN 9 1. ホスト H2 は 1 つの LAG メンバー上のすべてのトラフィックをリーフスイッチ LS2 宛て (LS3 宛てではない ) に送信する 2. リーフスイッチ LS3 は H2 の MAC を学習しない 3. リーフスイッチ LS3 は H2 の MAC ルートをアドバタイズしない 4. リーフスイッチ LS1 は H2 へのトラフィックを負荷分散せず トラフィックを LS2 に送信するだけになってしまう ホスト H2 の MAC/IP アドバタイズメント ( タイプ 2) ルート MP-IBGP H1 LS1 LS2 1 H2 LS3 4 LAG MP-IBGP 2 ホスト H2 の MAC/IP アドバタイズメント ( タイプ 2) ルート 8: イリ ン LS2 LS3 LS1 H2 マルチパス 3 9

10 9 1. LS1 は LS2 と LS3 の両方からイーサネット自動検知 ( タイプ 1) ルートを受信する 2. LS1 は LS2 からのみ MAC/IP アドバタイズメント ( タイプ 2) ルートを受信する LS1 は ESI ホスト H2 の場所を認識している LS1 は LS2 および LS3 を経由して H2 が存在する ESI に到達可能なことを認識している 3. LS1 は LS2 への VTP および LS3 への VTEP 経由の両方で ECMP トラフィックをホスト H2 に送信できる ESI 0:1:1:1:1:1:1:1:1:1 のイーサネット自動検知 ( タイプ 1) ルート ESI 0:1:1:1:1:1:1:1:1:1 上のホスト H2 の MAC/IP アドバタイズメント ( タイプ 2) ルート 1 2 MP-IBGP H1 LS1 LS2 H2 LS3 3 MP-IBGP LAG ESI 0:1:1:1:1:1:1:1:1:1 ESI 0:1:1:1:1:1:1:1:1:1 のイーサネット自動検知 ( タイプ 1) ルート 1 9 イ ア ン を し LS2 と LS3 を経由し LS1 から H2 への ル ス H1 LS2 2 ESI 0:1:1:1:1:1:1:1:1:1 H2 MAC LS3 ESI 1 H2 MAC LS3 EVPN L2 MAC 障害後の再コンバージェンスには時間がかかる : 1. リーフスイッチ LS3 とスイッチ S 間の接続がダウンする 2. イーグレスリーフスイッチ LS3 が スイッチ S の背後にあるすべてのホスト (H2~H100) のルートを取り消す 3. イングレスリーフスイッチ LS1 が 取り消されたすべてのルートを転送テーブルから削除する MP-IBGP H2 H1 LS1 LS2 ESI S H3 3 LS3... H99 MP-IBGP 1 H100 ホスト H2 の MAC/IP アドバタイズメント ( タイプ 2) ルートを取り消す ホスト H3 の MAC/IP アドバタイズメント ( タイプ 2) ルートを取り消す 2... 仮想化サーバー ホスト H100 の MAC/IP アドバタイズメント ( タイプ 1) ルートを取り消す 10 の MAC アドバタイズメントによる な ンバー ンス 10

11 10 LS2 LS3 H2 H100 L2 S1 S1 / LS3 S1 LS1 100 MAC LS1 100 LS3 EVPN リーフスイッチ LS3 とスイッチ S 間の接続がダウンする 2. イーグレスリーフスイッチ LS3 が ホストのルートを取り消す前に ESI のルートを取り消す 3. LS1 は ESI が取り消されたことを認識すると その ESI のすべてのルートを FIB から削除する MP-IBGP H2 H1 LS1 LS2 ESI S H3 3 LS3... H99 MP-IBGP 1 H100 ESI のイーサネット自動検知 ( タイプ 1) ルートを取り消す ホスト H2 の MAC/IP アドバタイズメント ( タイプ 2) ルートを取り消す 2 仮想化サーバー ホスト H3 の MAC/IP アドバタイズメント ( タイプ 2) ルートを取り消す... ホスト H100 の MAC/IP アドバタイズメント ( タイプ 2) ルートを取り消す 11 EVPN イリア ングを た の MAC アドバタイズメント EVPN LS3 1 ESI H1 1 ESI LS3 MAC EVPN BUM EVPN BUM EVPN MAC BGP EVPN PE MAC PE EVPN BUM

12 2 12 イングレスレプリケーション 送信元 アンダーレイレプリケーションマルチキャスト宛先ツリーへの VXLAN は 複数の VNI 間で共有可能 送信元 12 イングレスレプリケーション アンダーレイレプリケーションの EVPN BUM BUM BUM VNI VNI 2 EVPN BUM 3 13 NLRI ルートタイプルート識別子 (RD) イーサネットタグ ID 発信元 IP アドレス インクルーシブマルチキャストイーサネットタグルート ( タイプ 3) フラグ 0( リーフ情報は不要 ) プロバイダマルチキャストサービスインターフェイス (PMSI) トンネル 拡張コミュニティ トンネルタイプイングレスレプリケーション PIM-SSM PIM-SM BIDIR-PIM... MPLS ラベル 0( 未使用 ) マルチキャストグループ IP アドレストンネル識別子送信者 IIP アドレス ルートターゲット 他の属性 ( 発信元 ネクストホップ AS-Path Local-Pref など ) EVPN イプ 3 ルート EVPN BUM EVPN PE PE VXLAN 12

13 EVPN CE 1 PE EVPN PE CE BUM VLAN CE VLAN PE CE PE PE BUM VLAN CE PE EVPN 14 送信元 ローカル CE から : 同じ VLAN 内のローカル CE 宛て受信したパケットを送信した CE を除く 同じ VLAN 内のリモート PE 宛て リモート PE から : 同じ VLAN 内のローカル CE 宛て 14:EVPN リ ト CE PE 15 このホストが重複したパケットを受信 レプリケーション H2 H1( 送信元 ) PE2 H3 PE1 PE3 H4 15:EVPN リ ー ー 13

14 PE1 H1 BUM PE2 PE3 PE2 PE3 VLAN PE2 PE1 BUM PE1 PE2 VLAN H2 H3 PE3 H1 VLAN H3 H4 H3 PE2 PE3 H1 16 緑の VLAN の代表フォワーダ レプリケーション PE2 PE1 PE3 16 EVPN ン レ レプリケーション 代表フォワーダ 赤の VLAN の代表フォワーダ PE2 VLAN PE3 VLAN ESI 17 4 送信元 (IP アドレスでソート ) PE1 ( ) ESI 0:1:1:1:1:1:1:1:1:1 の指定フォワーダテーブル VLAN の指定フォワーダ PE2 ( ) PE 送信元 PE2 からの ESI のイーサネットセグメントルート ( タイプ 4) MP-IBGP NLRI ルートタイプルート識別子 (RD) イーサネットセグメント ID(ESI) 発信元 IP アドレス イーサネットセグメントルート ( タイプ 4)... 0:1:1:1:1:1:1:1:1:1 PE2 の IP アドレス PE ESI 0:1:1:1:1:1:1:1:1:1 拡張コミュニティ ES インポートルートターゲット 他の属性 プ 4 ン ESI 代表フォワーダ 14

15 PE ESI ES ES- MP-BGP PE 3 PE ES ES PE PE ES PE PE ES PE VLAN PE CE 18 送信元が自身のトラフィックのコピーを取得 緑の VLAN の代表フォワーダ PE1 送信元 PE2 レプリケーション 18 EVPN ン レ レプリケーション 代表フォワーダを 送信元 ープ ック トラフィック PE2 BUM PE2 ES/VLAN PE2 PE1 PE1 BUM CE BUM PE PE PE PE BUM - BUM VTEP IP - IP 4 AD - ESI IP 4 ESI BUM - CE 15

16 EVPN MAC L3 L2 EVPN 1 EVPN 2 19 H L1 H L2 L4 L1 L2 L4 H H H アドバタイズシーケンス番号なし (=0) アドバタイズシーケンス番号 1 移動 1 移動 2 アドバタイズシーケンス番号 2 取り消し L1 は シーケンス番号により これが正常ではないアドバタイズメントだと認識 取り消し 19 スト移動によ EVPN MAC 移動 L1 H 2 0 H EVPN H L2 L2 H 2 1 L4 L1 H L4 L4 H 2 2 L2 H 1 L1 L4 2 L2 1 L1 0 L1 L2 1 L1 2 16

17 EVPN EVPN IP L2 QFX5100 Broadcom Trident2 VXLAN 4 VXLAN L3 Clos 20 スパインスイッチ S1 スパインスイッチ S2 スパインスイッチ S3 スパインスイッチ S4 ルーティングインスタンス /24 00:00:5e:00:01: /24 00:00:5e:00:01: /24 00:00:5e:00:01: /24 00:00:5e:00:01: /24 00:00:5e:00:01: /24 00:00:5e:00:01: /24 00:00:5e:00:01: /24 00:00:5e:00:01:01 IRB VLAN のデフォルトゲートウェイ VXLAN トンネル アンダーレイリンク リーフスイッチ ブリッジドメイン = VLAN /24 1:1:1:1:1: /24 2:2:2:2:2: /24 1:1:1:1:1: /24 2:2:2:2:2:2 20 スパイン EVPN の デフォルトゲートウェイ 2 4 VNI / IRB IRB IP MAC SSH 1 HTTPS 1 2 ECMP 2 ECMP / CE IRB IRB 1 ESI 2 MAC PE MAC ESI IRB 2 4 Broadcom Trident2 MPLS MPLS EVPN+MPLS VXLAN 17

18 ESI < 緑の IRB ESI> のイーサネット自動検知 ( タイプ 1) ルート ESI < 緑の IRB ESI> 上のホスト < 緑の IRB エニーキャスト IP> の MAC/IP アドバタイズメント ( タイプ 2) ルート マルチホームド IRB スパイン 1 リーフ 1 マルチホームド IRB スパイン 2 ESI < 緑の IRB ESI> のイーサネット自動検知 ( タイプ 1) ルート ESI < 緑の IRB ESI> 上のホスト < 緑の IRB エニーキャスト IP> の MAC/IP アドバタイズメント ( タイプ 2) ルート リーフ 2 21 EVPN フ ルト ート イのルートアドバタイズメント IRB 1 ESI IRB MAC 2 1 IRB IRB 1 2 EVPN VXLAN EVPN L3 Clos L3 Clos Clos A B コア / ファブリック IP スイッチ ポッド A ポッド B スパイン IP スイッチ スパイン IP スイッチ リーフ IP スイッチ ラックまたは偶数サーバーに接続されたレイヤー 2 サブネット リーフ IP スイッチ ラックまたは偶数サーバーに接続されたレイヤー 2 サブネット 22 5 の L3 Clos フ リッ 18

19 L3 Clos Dijkstra BGP BGP AS ASN 23 ASN 24 AS AS AS AS AS( 階層ごと ポッドごと ) AS AS EBGP BFD マルチパス サーバーのエクスポート ループバックのエクスポート AS AS AS AS /31( リンクごと ) AS AS AS AS L3 Clos ASN AS AS AS AS AS( ルーターごと ) AS AS EBGP BFD マルチパス サーバーのエクスポート ループバックのエクスポート AS AS AS AS /31( リンクごと ) AS AS AS AS L3 Clos ASN IP BGP VXLAN VTEP 19

20 25 コア ge-1/0/ /31 ge-1/0/ /31 ge-1/0/ /31 コア ge-1/0/ /31 ASN65400 xe-0/0/0 xe-0/0/1 xe-0/0/0 xe-0/0/1 スパイン スパイン ASN65401 xe-0/0/ /31 xe-0/0/ /31 xe-0/0/ /31 xe-0/0/ /31 xe-0/0/2 リーフ xe-0/0/4 xe-0/0/33 EVPN/ VXLAN xe-0/0/3 xe-0/0/36 xe-0/0/5 リーフ ASN65402 xe-0/0/34 xe-0/0/32 xe-0/0/37 xe-0/0/38 ハイパーバイザ LAG LAG ハイパーバイザ テナント C テナント D テナント A テナント B テナント A テナント C 25 EVPN/VXLAN ト ー 1 2 MX 3D EVPN IP 1 2 QFX5100 IP 1 2 QFX5100 EVPN PE ASN EVPN - 1 <-> 2-1 <-> 1-1 <-> 2-2 <-> 1-2 <-> 2 BGP EBGP 1 lab@leaf-1> show configuration routing-options router-id ; autonomous-system 65402; forwarding-table { export load-balance; lab@leaf-1> show configuration policy-options policy-statement load-balance term 1 { 20

21 then { load-balance per-packet; BGP 2 1 lo0 2 family inet unicast loops 2 ASN lab@leaf-1> show configuration protocols bgp group underlay type external; advertise-peer-as; family inet { unicast { export lo0; loops 2; peer-as 65401; multipath; neighbor { description spine-1; neighbor { description spine-2; lab@leaf-1> show configuration policy-options policy-statement lo0 from { family inet; protocol direct; route-filter /0 prefix-length-range /32-/32; then accept; advertise-peer-as 1 EBGP ASN 1 2 AS_PATH lab@leaf-1> show route detail table inet.0 inet.0: 13 destinations, 18 routes (13 active, 0 holddown, 0 hidden) /32 (2 entries, 1 announced) *BGP Preference: 170/-101 Next hop type: Router, Next hop index: Address: 0x Next-hop reference count: 84 Source: Next hop: via xe-0/0/2.0 Session Id: 0x0 Next hop: via xe-0/0/4.0, selected 21

22 Session Id: 0x0 State: <Active Ext> Local AS: Peer AS: Age: 2d 4:00:39 Validation State: unverified Task: BGP_ Announcement bits (2): 0-KRT 2-Resolve tree 2 AS path: I (Looped: 65402) Accepted Multipath Localpref: 100 Router ID: BGP Preference: 170/-101 Next hop type: Router, Next hop index: 1723 Address: 0x95f8860 Next-hop reference count: 7 Source: Next hop: via xe-0/0/4.0, selected Session Id: 0x0 State: <NotBest Ext> Inactive reason: Not Best in its group - Active preferred Local AS: Peer AS: Age: 2d 4:00:39 Validation State: unverified Task: BGP_ AS path: I (Looped: 65402) Accepted MultipathContrib Localpref: 100 Router ID: {master: EBGP 2 1 lab@spine-1> show configuration routing-options router-id ; autonomous-system 65401; forwarding-table { export load-balance; ecmp-fast-reroute; lab@spine-1> show configuration policy-options policy-statement load-balance term 1 { then { load-balance per-packet; 22

23 show configuration protocols bgp group underlay-leaf type external; advertise-peer-as; family inet { unicast { loops 2; export lo0; peer-as 65402; multipath; neighbor { description leaf-1; neighbor { description leaf-2; lab@spine-1> show configuration protocols bgp group underlay-core type external; advertise-peer-as; family inet { unicast { loops 2; export lo0; peer-as 65400; multipath; neighbor { description core-1; neighbor { description core-2; lab@spine-1> show configuration policy-options policy-statement lo0 from { family inet; protocol direct; route-filter /0 prefix-length-range /32-/32; then accept; 1 BGP 1 bgp group underlay-leaf advertise-peer-as 1 family inet unicast loops 2 1 ASN EBGP 23

24 1 2 EBGP 2 EBGP 1 ASN ASN 1 1 advertise-peer-as 1 ASN advertise-peer-as EBGP EVPN family inet unicast loops EVPN advertise-peer-as bgp group underlay-core 1 1 lab@core-1> show configuration routing-options router-id ; autonomous-system 65400; forwarding-table { export load-balance; ecmp-fast-reroute; lab@core-1> show configuration policy-options policy-statement load-balance term 1 { then { load-balance per-packet; lab@core-1> show configuration protocols bgp group underlay { type external; advertise-peer-as; family inet { unicast { export lo0; loops 2; peer-as 65401; multipath; neighbor { description spine-1; neighbor { description spine-2; lab@core-1> show configuration policy-options policy-statement lo0 from { 24

25 family inet; protocol direct; route-filter /0 prefix-length-range /32-/32; then accept; family inet unicast loops 2 1 forwarding-table ecmp-fast-reroute MX 5 IP ECMP L3 EVPN/VXLAN EVPN 2 1. Protocols bgp BGP MBGP EVPN EVPN 2. EVPN QFX5100 MX 2 EVPN EVPN VTEP lo0.0 MBGP EVPN RD vrf-import EVPN vrf-export EVPN protocols evpn - VNI - BUM EVPN BUM VLAN - VNI VLAN - BUM QFX switch-options lab@leaf-1> show configuration switch-options vtep-source-interface lo0.0; route-distinguisher :1; vrf-import vrf-imp; vrf-target target:9999:9999; vtep-source-interface lo0.0 route-distinguisher MX MGP MP-MGP QFX5100 vrf-target ESI 1 5 PTX 25

26 vrf-import vrf-imp bgp.evpn.0 default-switch.evpn.0 show configuration policy-options policy-statement vrf-imp term t1 { from community com100; then accept; term t2 { from community com200; then accept; term t3 { from community com300; then accept; term t4 { from community com400; then accept; term t5 { then reject; lab@leaf-1> show configuration policy-options grep members community com100 members target:1:100; community com200 members target:1:200; community com300 members target:1:300; community com400 members target:1:400; protocols evpn lab@leaf-1> show configuration protocols evpn encapsulation vxlan; extended-vni-list [ ]; multicast-mode ingress-replication; vni-routing-options { vni 1100 { vrf-target export target:1:100; vni 1200 { vrf-target export target:1:200; vni 1300 { vrf-target export target:1:300; vni 1400 { vrf-target export target:1:400; 26

27 extended-vni-list EVPN/VXLAN MP-BGP VNI EVPN BUM EVPN VXLAN vni-routing-options VNI RT 2 RT lab@leaf-1> show configuration vlans v100 { vlan-id 100; vxlan { vni 1100; ingress-node-replication; v200 { vlan-id 200; vxlan { vni 1200; ingress-node-replication; v300 { vlan-id 300; vxlan { vni 1300; ingress-node-replication; v400 { vlan-id 400; vxlan { vni 1400; ingress-node-replication; vlans vlan-id VNI EVPN MP-BGP lab@leaf-1> show configuration protocols bgp group EVPN_VXLAN_CORE type external; multihop { ttl 255; no-nexthop-change; local-address ; family evpn { signaling; 27

28 peer-as 65400; neighbor { description core-1; neighbor { description core-2; {master:0 lab@leaf-1> show configuration protocols bgp group EVPN_VXLAN_LEAF type internal; local-address ; family evpn { signaling; neighbor { description leaf-2; family evpn signaling EVPN MP-BGP EBGP EVPN multihop 1 EVPN_VXLAN_CORE no-nexthop-change 1 ASN 1 2 EVPN MP-BGP BGP IBGP ebgp 25 2 Tenant_A CE 1 Tenant_A 2 1 BGP 2 2 EBGP BGP IBGP VTEP 1 1 VTEP 1 1 Tenant_C MAC VTEP 2 EVPN PE PE EVPN 1 2 CE PE no-nexthop-change PE AS IBGP/EBGP EVPN PE EVPN BGP MBGP EBGP Junos OS PE MP-EBGP 1 lab@leaf-1> show configuration interfaces xe-0/0/32 ether-options { 802.3ad ae0; 28

29 {master:0 show configuration interfaces xe-0/0/33 ether-options { 802.3ad ae1; {master:0 lab@leaf-1> show configuration interfaces ae0 esi { 00:01:01:01:01:01:01:01:01:01; all-active; aggregated-ether-options { lacp { active; system-id 00:00:00:01:01:01; unit 0 { family ethernet-switching { interface-mode access; vlan { members v100; {master:0 lab@leaf-1> show configuration interfaces ae1 esi { 00:02:02:02:02:02:02:02:02:02; all-active; aggregated-ether-options { lacp { active; system-id 00:00:00:01:01:01; unit 0 { family ethernet-switching { interface-mode access; vlan { members v200; 29

30 Link Aggregation Control Protocol LACP LAG 2 EVPN esi EVPN ESI all-active PE CE CE 1 1 MX lab@core-1> show configuration routing-instances VRF_Tenant_A { instance-type vrf; interface irb.1100; route-distinguisher :1100; vrf-target target:10:100; VRF_Tenant_B { instance-type vrf; interface irb.1200; route-distinguisher :1200; vrf-target target:10:200; VRF_Tenant_C { instance-type vrf; interface irb.1300; route-distinguisher :1300; vrf-target target:10:300; VRF_Tenant_D { instance-type vrf; interface irb.1400; route-distinguisher :1400; vrf-target target:10:400; VS_VLAN100 { vtep-source-interface lo0.0; instance-type virtual-switch; route-distinguisher :100; vrf-import VS_VLAN100_IMP; vrf-target target:1:100; protocols { evpn { encapsulation vxlan; extended-vni-list 1100; multicast-mode ingress-replication; bridge-domains { bd1100 { vlan-id 100; routing-interface irb.1100; 30

31 vxlan { vni 1100; ingress-node-replication; VS_VLAN200 { vtep-source-interface lo0.0; instance-type virtual-switch; route-distinguisher :200; vrf-import VS_VLAN200_IMP; vrf-target target:1:200; protocols { evpn { encapsulation vxlan; extended-vni-list 1200; multicast-mode ingress-replication; bridge-domains { bd1200 { vlan-id 200; routing-interface irb.1200; vxlan { vni 1200; ingress-node-replication; VS_VLAN300 { vtep-source-interface lo0.0; instance-type virtual-switch; route-distinguisher :300; vrf-import VS_VLAN300_IMP; vrf-target target:1:300; protocols { evpn { encapsulation vxlan; extended-vni-list 1300; multicast-mode ingress-replication; bridge-domains { bd1300 { vlan-id 300; routing-interface irb.1300; vxlan { 31

32 vni 1300; ingress-node-replication; VS_VLAN400 { vtep-source-interface lo0.0; instance-type virtual-switch; route-distinguisher :400; vrf-import VS_VLAN400_IMP; vrf-target target:1:400; protocols { evpn { encapsulation vxlan; extended-vni-list 1400; multicast-mode ingress-replication; bridge-domains { bd1400 { vlan-id 400; routing-interface irb.1400; vxlan { vni 1400; ingress-node-replication; VNI switch-options 1 VS_VLAN400 RD vrf-target vrf-import VNI 1400 EVPN VLAN400 VNI VNI 1400 IRB IP VPN VRF L3 IP VPN VRF MPLS WAN IP VPN 1 1 IRB IRB VRF L3 vrf-import lab@core-1> show configuration policy-options policy-statement VS_VLAN100_IMP { term ESI { from community comm-leaf_esi; 32

33 then accept; term VS_VLAN100 { from community comm-vs_vlan100; then accept; policy-statement VS_VLAN200_IMP { term ESI { from community comm-leaf_esi; then accept; term VS_VLAN200 { from community comm-vs_vlan200; then accept; policy-statement VS_VLAN300_IMP { term ESI { from community comm-leaf_esi; then accept; term VS_VLAN300 { from community comm-vs_vlan300; then accept; policy-statement VS_VLAN400_IMP { term ESI { from community comm-leaf_esi; then accept; term VS_VLAN400 { from community comm-vs_vlan400; then accept; policy-statement lo0 { from { family inet; protocol direct; 33

34 route-filter /0 prefix-length-range /32-/32; then accept; policy-statement load-balance { term 1 { then { load-balance per-packet; community comm-vs_vlan100 members target:1:100; community comm-vs_vlan200 members target:1:200; community comm-vs_vlan300 members target:1:300; community comm-vs_vlan400 members target:1:400; community comm-leaf_esi members target:9999:9999; 2 1 target:9999: ESI 2 VNI RT 2 ESI 25 Tenant_A Tenant_B 2 VS_VLAN100 Tenant_B ESI VS_VLAN200 Tenant_A ESI vrf-target 1 ESI VRF Tenant_A Tenant_B ESI 1 ESI 2 MAC IRB lab@core-1> show configuration interfaces irb unit 1100 { family inet { address /24 { unit 1200 { family inet { virtual-gateway-address ; address /24 { unit 1300 { virtual-gateway-address ; 34

35 family inet { address /24 { virtual-gateway-address ; unit 1400 { family inet { address /24 { virtual-gateway-address ; 2 1 IP 2 IRB virtual-gateway-address 1 2 MAC IP 2 IRB PE 2 LAG EVPN VXLAN EVPN MX QFX10000 QFX5100 Broadcom Trident2 VXLAN Broadcom Trident MP-BGP lab@core-1> show configuration protocols bgp group EVPN_VXLAN type external; local-address ; family evpn { signaling; peer-as 65402; multipath; neighbor { description leaf-1; multihop { ttl 255; neighbor { description leaf-2; multihop { ttl 255; 35

36 EVPN VXLAN EVPN EVPN bgp.evpn.0 = Junos OS RPD EVPN default.switch.evpn.0 QFX5100 = Junos OS RPD EVPN <virtual-switch-name>.evpn.0 MX = Junos OS RPD EVPN L2ALD 2 show ethernet-switching-table show bridge mac-table show route forwarding-table PFE from FPC vty: show l2 manager mac-table CE CE 3 CE 1 Tenant_C 1 MAC MAC 1 2 MAC 1 lab@leaf-1> show ethernet-switching table vlan-id 300 MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static ovsdb MAC) SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - Ethernet switching table : 5 entries, 5 learned Routing instance : default-switch Vlan MAC MAC Logical Active name address flags interface source v300 00:00:5e:00:01:01 DR esi :00:00:ff:78:00:00:05:14:00 v300 00:21:59:c8:24:65 D xe-0/0/34.0 v300 00:21:59:c8:24:69 D vtep v300 40:a6:77:9a:43:f0 D vtep v300 40:a6:77:9a:47:f0 D vtep MAC 00:21:59:c8:24:65 xe-0/0/34.0 Tenant_C CE 1 2 lab@leaf-1> show route advertising-protocol bgp evpn-mac-address 00:21:59:c8:24:65 bgp.evpn.0: 77 destinations, 77 routes (77 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 2: :1::1300::00:21:59:c8:24:65/304 * Self I 2: :1::1400::00:21:59:c8:24:65/304 * Self I default-switch.evpn.0: 67 destinations, 67 routes (67 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 36

37 2: :1::1300::00:21:59:c8:24:65/304 * Self I 2: :1::1400::00:21:59:c8:24:65/304 * Self I default_evpn.evpn.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden) 2 RT MAC 2 xe-0/0/34 Tenant_C Tenant_D VNI RT MAC 1 2 bgp.evpn.0 lab@core-1> show route receive-protocol bgp evpn-mac-address 00:21:59:c8:24:65 extensive table bgp.evpn.0 <output omitted> * 2: :1::1300::00:21:59:c8:24:65/304 (2 entries, 0 announced) Import Accepted Route Distinguisher: :1 Nexthop: AS path: I Communities: target:1:300 encapsulation0:0:0:0:vxlan * 2: :1::1400::00:21:59:c8:24:65/304 (2 entries, 0 announced) Import Accepted Route Distinguisher: :1 Nexthop: AS path: I Communities: target:1:400 encapsulation0:0:0:0:vxlan 00:21:59:c8:24: RT 1 RD :1 1 2 bgp.evpn.0 EVPN Tenant_C target:1: lab@core-1> show route table VS_VLAN300.evpn.0 evpn-mac-address 00:21:59:c8:24:65 grep 00:21:59:c8:24:65 2: :1::1300::00:21:59:c8:24:65/304 lab@core-1> 2 lab@core-1> show route table VS_VLAN300.evpn.0 evpn-mac-address 00:21:59:c8:24:65 extensive <output omitted> 2: :1::1300::00:21:59:c8:24:65/304 (2 entries, 1 announced) *BGP Preference:170/

38 Route Distinguisher: :1 Next hop type: Indirect Address: 0x2cf479c Next-hop reference count: 76 Source: Protocol next hop: Indirect next hop: 0x2 no-forward INH Session ID: 0x0 State: <Secondary Active Ext> Local AS: Peer AS: Age: 4:47 Metric2: 0 Validation State: unverified Task: BGP_ Announcement bits (1): 0-VS_VLAN300-evpn AS path: I Communities: target:1:300 encapsulation0:0:0:0:vxlan Import Accepted Localpref: 100 Router ID: Primary Routing Table bgp.evpn.0 Indirect next hops: 1 Protocol next hop: Indirect next hop: 0x2 no-forward INH Session ID: 0x0 Indirect path forwarding next hops: 2 Next hop type: Router Next hop: via ge-1/0/0.0 Session Id: 0x140 Next hop: via ge-1/0/1.0 Session Id: 0x /32 Originating RIB: inet.0 Node path count: 1 Forwarding nexthops: 2 Nexthop: via ge-1/0/0.0 BGP Preference: 170/-101 Route Distinguisher: :1 Next hop type: Indirect Address: 0x2cf479c Next-hop reference count: 76 Source: Protocol next hop: Indirect next hop: 0x2 no-forward INH Session ID: 0x0 State: <Secondary NotBest Ext> Inactive reason: Not Best in its group - Router ID Local AS: Peer AS: Age: 4:47 Metric2: 0 Validation State: unverified Task: BGP_ AS path: I Communities: target:1:300 encapsulation0:0:0:0:vxlan Import Accepted 38

39 Localpref: 100 Router ID: Primary Routing Table bgp.evpn.0 Indirect next hops: 1 Protocol next hop: Indirect next hop: 0x2 no-forward INH Session ID: 0x0 Indirect path forwarding next hops: 2 Next hop type: Router Next hop: via ge-1/0/0.0 Session Id: 0x140 Next hop: via ge-1/0/1.0 Session Id: 0x /32 Originating RIB: inet.0 Node path count: 1 Forwarding nexthops: 2 Nexthop: via ge-1/0/ no-nexthop-change L2ALD lab@core-1> show bridge mac-table instance VS_VLAN300 MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC) Routing instance : VS_VLAN300 Bridging domain : bd1300, VLAN : 300 MAC MAC Logical Active address flags interface source 00:21:59:c8:24:65 D vtep :21:59:c8:24:69 D vtep :21:59:c8:24:65 vtep lab@core-1> show route forwarding-table family bridge vpn VS_VLAN300 Routing table: VS_VLAN300.evpn-vxlan VPLS: Destination Type RtRef Next hop Type Index NhRef Netif default perm 0 dscd vtep intf 0 comp vtep intf 0 comp Routing table: VS_VLAN300.evpn-vxlan Bridging domain: bd1300.evpn-vxlan VPLS: Destination Type RtRef Next hop Type Index NhRef Netif 00:21:59:c8:24:65/48 user 0 comp

40 00:21:59:c8:24:69/48 user 0 comp x30003/51 user 0 comp Tenant_C MAC 00:21:59:c8:24: NH-Id VNI 1300 VTEP-ID lab@core-1> show l2-learning vxlan-tunnel-end-point remote Logical System Name Id SVTEP-IP IFL L3-Idx <default> lo0.0 0 RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP RVTEP-IP IFL-Idx NH-Id VNID MC-Group-IP :21:59:c8:24:65 PFE # show l2 manager mac-table <output omitted> route table name : VS_VLAN

41 mac counters maximum count 0 2 mac table information mac address BD learn Entry entry hal hardware info Index vlan Flags ifl ifl pfe mask ifl :21:59:c8:24: x0014 vtep vtep D src unknown dest vtep :21:59:c8:24: x0014 vtep vtep D src unknown dest vtep Displayed 2 entries for routing instance VS_VLAN300.7 CE 1 2 Tenant_B CE 1 2 CE 1 2 lab@leaf-1> show ethernet-switching table vlan-id 200 MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC) Ethernet switching table : 4 entries, 4 learned Routing instance : default-switch Vlan MAC MAC Logical Active name address flags interface source v200 00:00:5e:00:01:01 DR esi :00:00:ff:78:00:00:04:b0:00 v200 00:21:59:c8:24:64 DL ae1.0 v200 40:a6:77:9a:43:f0 D vtep v200 40:a6:77:9a:47:f0 D vtep lab@leaf-2> show ethernet-switching table vlan-id 200 MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC) Ethernet switching table : 5 entries, 5 learned Routing instance : default-switch Vlan MAC MAC Logical Active name address flags interface source v200 00:00:5e:00:01:01 DR esi :00:00:ff:78:00:00:04:b0:00 v200 00:21:59:c8:24:41 DL ae1.0 41

42 v200 00:21:59:c8:24:68 DL ae1.0 v200 40:a6:77:9a:43:f0 D vtep v200 40:a6:77:9a:47:f0 D vtep :21:59:c8:24:64 1 Tenant _B NIC MAC 00:21:59:c8:24:68 2 Tenant _B NIC MAC 00:21:59:c8:24:41 MAC 2 ae1 ESI 00:02:02:02:02:02:02:02:02:02 lab@leaf-1> show configuration interfaces ae1 esi 00:02:02:02:02:02:02:02:02:02; all-active; 1 Tenant_B EVPN lab@core-1> show route table VS_VLAN200.evpn.0 VS_VLAN200.evpn.0: 18 destinations, 31 routes (18 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1: :0:: ::FFFF:FFFF/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 1: :0:: ::FFFF:FFFF/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 1: :1:: ::0/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 1: :1:: ::0/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified 42

43 to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 1: :0:: ::FFFF:FFFF/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 1: :0:: ::FFFF:FFFF/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 1: :1:: ::0/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 1: :1:: ::0/304 *[BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 23:27:33, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 <output omitted> 2: :1::1200::00:21:59:c8:24:64/304 *[BGP/170] 4d 10:47:09, localpref 100, from AS path: I, validation-state: unverified > to via ge-1/0/0.0 to via ge-1/0/1.0 [BGP/170] 1d 00:44:04, localpref 100, from

44 AS path: I, validation-state: unverified > to via ge-1/0/0.0 to via ge-1/0/1.0 2: :1::1200::00:21:59:c8:24:41/304 *[BGP/170] 00:14:09, localpref 100, from AS path: I, validation-state: unverified > to via ge-1/0/0.0 to via ge-1/0/1.0 [BGP/170] 00:14:09, localpref 100, from AS path: I, validation-state: unverified > to via ge-1/0/0.0 to via ge-1/0/1.0 2: :1::1200::00:21:59:c8:24:68/304 <output omitted> *[BGP/170] 1d 00:44:04, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/1.0 [BGP/170] 4d 10:47:09, localpref 100, from AS path: I, validation-state: unverified to via ge-1/0/0.0 > to via ge-1/0/ ESI target:9999: target: Tenant_B CE ESI 00:02:02:02:02:02:02:02:02:02 4 a. 1: :0:: ::FFFF:FFFF/304 i. 1 AD 1 EVPN RD ii b. 1: :1:: ::0/304 i. EVI AD 1 EVPN RD QFX5100 ii c. 1: :0:: ::FFFF:FFFF/304 i. 2 AD 1 EVPN RD ii d. 1: :1:: ::0/304 i. EVI AD 1 EVPN RD QFX5100 ii Tenant_B CE 2 MAC 1 MAC 2 1 L2ALD lab@core-1> show bridge mac-table instance VS_VLAN200 44

45 MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC) Routing instance : VS_VLAN200 Bridging domain : bd1200, VLAN : 200 MAC MAC Logical Active address flags interface source 00:21:59:c8:24:41 DR esi :02:02:02:02:02:02:02:02:02 00:21:59:c8:24:64 DR esi :02:02:02:02:02:02:02:02:02 00:21:59:c8:24:68 DR esi :02:02:02:02:02:02:02:02:02 1 lab@core-1> show route forwarding-table vpn VS_VLAN200 Routing table: VS_VLAN200.evpn-vxlan VPLS: Destination Type RtRef Next hop Type Index NhRef Netif default perm 0 dscd vtep intf 0 comp vtep intf 0 comp Routing table: VS_VLAN200.evpn-vxlan Bridging domain: bd1200.evpn-vxlan VPLS: Destination Type RtRef Next hop Type Index NhRef Netif 00:21:59:c8:24:41/48 user 0 indr comp :21:59:c8:24:64/48 user 0 indr comp :21:59:c8:24:68/48 user 0 indr comp x30002/51 user 0 comp VTEP ESI 00:02:02:02:02:02:02:02:02:02 lab@core-1> show l2-learning vxlan-tunnel-end-point esi <output omitted> ESI RTT VLNBH INH ESI-IFL LOC-IFL #RVTEPs 00:01:01:01:01:01:01:01:01:01 VS_VLAN esi RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS vtep vtep <output omitted> 45

46 ESI MAC VTEP esi.703 ECMP unilist unilist lab@core-1> start shell command nhinfo -di 703 <output omitted> NHs in list: 720, 702, <output omitted> vtep vtep EVPN EVPN CE QFX5100 VNI VLAN QFX5100 EVPN+VXLAN VNI VNI QFX5100 VTEP QFX5100 EVPN+VXLAN CE MX QFX5100 VNI VTEP 5 lab@leaf-1> show route receive-protocol bgp inet.0: 13 destinations, 18 routes (13 active, 0 holddown, 0 hidden) :vxlan.inet.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) bgp.evpn.0: 75 destinations, 123 routes (75 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 1: :0::050000ff c00::FFFF:FFFF/304 * I 1: :0::050000ff b000::FFFF:FFFF/304 * I 1: :0::050000ff ::FFFF:FFFF/304 * I 1: :0::050000ff ::FFFF:FFFF/304 * I 2: :300::1300::00:00:5e:00:01:01/304 * I 2: :300::1300::40:a6:77:9a:43:f0/304 * I 2: :100::1100::00:00:5e:00:01:01/304 * I 2: :100::1100::40:a6:77:9a:43:f0/304 * I 2: :400::1400::00:00:5e:00:01:01/304 * I 2: :400::1400::40:a6:77:9a:43:f0/304 * I 2: :200::1200::00:00:5e:00:01:01/304 * I 2: :200::1200::40:a6:77:9a:43:f0/304 46

47 * I 2: :300::1300::00:00:5e:00:01:01:: /304 * I 2: :300::1300::40:a6:77:9a:43:f0:: /304 * I 2: :100::1100::00:00:5e:00:01:01:: /304 * I 2: :100::1100::40:a6:77:9a:43:f0:: /304 * I 2: :400::1400::00:00:5e:00:01:01:: /304 * I 2: :400::1400::40:a6:77:9a:43:f0:: /304 * I 2: :200::1200::00:00:5e:00:01:01:: /304 * I 2: :200::1200::40:a6:77:9a:43:f0:: /304 * I <output omitted> 1 ESI 1 MAC a.b.c a.b.c.3 lab@leaf-1> show route receive-protocol bgp inet.0: 13 destinations, 18 routes (13 active, 0 holddown, 0 hidden) :vxlan.inet.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) bgp.evpn.0: 75 destinations, 123 routes (75 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 1: :0::050000ff c00::FFFF:FFFF/304 * I 1: :0::050000ff b000::FFFF:FFFF/304 * I 1: :0::050000ff ::FFFF:FFFF/304 * I 1: :0::050000ff ::FFFF:FFFF/304 * I 2: :300::1300::00:00:5e:00:01:01/304 * I 2: :300::1300::40:a6:77:9a:47:f0/304 * I 2: :100::1100::00:00:5e:00:01:01/304 * I 2: :100::1100::40:a6:77:9a:47:f0/304 * I 2: :400::1400::00:00:5e:00:01:01/304 * I 47

48 2: :400::1400::40:a6:77:9a:47:f0/304 * I 2: :200::1200::00:00:5e:00:01:01/304 * I 2: :200::1200::40:a6:77:9a:47:f0/304 * I 2: :300::1300::00:00:5e:00:01:01:: /304 * I 2: :300::1300::40:a6:77:9a:47:f0:: /304 * I 2: :100::1100::00:00:5e:00:01:01:: /304 * I 2: :100::1100::40:a6:77:9a:47:f0:: /304 * I 2: :400::1400::00:00:5e:00:01:01:: /304 * I 2: :400::1400::40:a6:77:9a:47:f0:: /304 * I 2: :200::1200::00:00:5e:00:01:01:: /304 * I 2: :200::1200::40:a6:77:9a:47:f0:: /304 <output omitted> 1 the default-switch.evpn.0 VNI 1110 ESI lab@leaf-1> show route table default-switch.evpn.0 evpn-esi-value 05:00:00:ff:78:00:00:04:4c:00 default-switch.evpn.0: 66 destinations, 114 routes (66 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1: :0::050000ff c00::FFFF:FFFF/304 *[BGP/170] 00:23:37, localpref 100, from AS path: I, validation-state: unverified > to via xe-0/0/2.0 to via xe-0/0/4.0 [BGP/170] 00:09:29, localpref 100, from AS path: I, validation-state: unverified > to via xe-0/0/2.0 to via xe-0/0/4.0 1: :0::050000ff c00::FFFF:FFFF/304 *[BGP/170] 00:23:33, localpref 100, from AS path: I, validation-state: unverified > to via xe-0/0/2.0 to via xe-0/0/4.0 [BGP/170] 00:09:29, localpref 100, from AS path: I, validation-state: unverified > to via xe-0/0/2.0 to via xe-0/0/4.0 48

49 ESI RD IBGP 2 1 L2ALD lab@leaf-1> show ethernet-switching table vlan-id 100 MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC) Ethernet switching table : 5 entries, 5 learned Routing instance : default-switch Vlan MAC MAC Logical Active name address flags interface source v100 00:00:5e:00:01:01 DR esi :00:00:ff:78:00:00:04:4c:00 v100 00:21:59:c8:24:63 DL ae0.0 v100 00:21:59:c8:24:69 D vtep v100 40:a6:77:9a:43:f0 D vtep v100 40:a6:77:9a:47:f0 D vtep lab@leaf-1> show route forwarding-table family ethernet-switching <output omitted> Routing table: default-switch.bridge VPLS: Destination Type RtRef Next hop Type Index NhRef Netif default perm 0 dscd vtep intf 0 comp vtep intf 0 comp vtep intf 0 comp <output omitted> Routing table: default-switch.bridge Bridging domain: v100.bridge VPLS: Destination Type RtRef Next hop Type Index NhRef Netif 00:00:5e:00:01:01/48 user 0 comp <output omitted> 1 VNI 1100 vtep vtep lab@leaf-1> show ethernet-switching vxlan-tunnel-end-point esi <output omitted> 05:00:00:ff:78:00:00:04:4c:00 default-switch esi

50 RVTEP-IP RVTEP-IFL VENH MASK-ID FLAGS vtep vtep VTEP QFX5100 EVPN+VXLAN Junos MX unilist 1 set system host-name leaf-1 set chassis aggregated-devices ethernet device-count 2 set interfaces xe-0/0/2 unit 0 family inet address /31 set interfaces xe-0/0/4 unit 0 family inet address /31 set interfaces xe-0/0/32 ether-options 802.3ad ae0 set interfaces xe-0/0/33 ether-options 802.3ad ae1 set interfaces xe-0/0/34 unit 0 family ethernet-switching interface-mode trunk set interfaces xe-0/0/34 unit 0 family ethernet-switching vlan members v300 set interfaces xe-0/0/34 unit 0 family ethernet-switching vlan members v400 set interfaces ae0 esi 00:01:01:01:01:01:01:01:01:01 set interfaces ae0 esi all-active set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 aggregated-ether-options lacp system-id 00:00:00:01:01:01 set interfaces ae0 unit 0 family ethernet-switching interface-mode access set interfaces ae0 unit 0 family ethernet-switching vlan members v100 set interfaces ae1 esi 00:02:02:02:02:02:02:02:02:02 set interfaces ae1 esi all-active set interfaces ae1 aggregated-ether-options lacp active set interfaces ae1 aggregated-ether-options lacp system-id 00:00:00:01:01:01 set interfaces ae1 unit 0 family ethernet-switching interface-mode access set interfaces ae1 unit 0 family ethernet-switching vlan members v200 set interfaces lo0 unit 0 family inet address /32 set routing-options router-id set routing-options autonomous-system set routing-options forwarding-table export load-balance set routing-options forwarding-table ecmp-fast-reroute set protocols bgp group underlay type external set protocols bgp group underlay advertise-peer-as set protocols bgp group underlay family inet unicast loops 2 set protocols bgp group underlay export lo0 set protocols bgp group underlay peer-as set protocols bgp group underlay multipath set protocols bgp group underlay neighbor description spine-1 set protocols bgp group underlay neighbor description spine-2 set protocols bgp group EVPN_VXLAN_CORE type external set protocols bgp group EVPN_VXLAN_CORE multihop ttl 255 set protocols bgp group EVPN_VXLAN_CORE multihop no-nexthop-change set protocols bgp group EVPN_VXLAN_CORE local-address set protocols bgp group EVPN_VXLAN_CORE family evpn signaling set protocols bgp group EVPN_VXLAN_CORE peer-as

51 set protocols bgp group EVPN_VXLAN_CORE local-as set protocols bgp group EVPN_VXLAN_CORE neighbor description core-1 set protocols bgp group EVPN_VXLAN_CORE neighbor description core-2 set protocols bgp group EVPN_VXLAN_LEAF type internal set protocols bgp group EVPN_VXLAN_LEAF local-address set protocols bgp group EVPN_VXLAN_LEAF family evpn signaling set protocols bgp group EVPN_VXLAN_LEAF export LEAF-PREPEND set protocols bgp group EVPN_VXLAN_LEAF neighbor description leaf-2 set protocols evpn encapsulation vxlan set protocols evpn extended-vni-list 1100 set protocols evpn extended-vni-list 1200 set protocols evpn extended-vni-list 1300 set protocols evpn extended-vni-list 1400 set protocols evpn multicast-mode ingress-replication set protocols evpn vni-routing-options vni 1100 vrf-target export target:1:100 set protocols evpn vni-routing-options vni 1200 vrf-target export target:1:200 set protocols evpn vni-routing-options vni 1300 vrf-target export target:1:300 set protocols evpn vni-routing-options vni 1400 vrf-target export target:1:400 set protocols l2-learning traceoptions file l2ald.log set protocols l2-learning traceoptions file size 10m set protocols l2-learning traceoptions level all set protocols l2-learning traceoptions flag all set protocols lldp port-id-subtype interface-name set protocols lldp interface all set policy-options policy-statement LEAF-PREPEND then as-path-prepend set policy-options policy-statement lo0 from family inet set policy-options policy-statement lo0 from protocol direct set policy-options policy-statement lo0 from route-filter /0 prefix-lengthrange /32-/32 set policy-options policy-statement lo0 then accept set policy-options policy-statement load-balance term 1 then load-balance perpacket set policy-options policy-statement vrf-imp term t1 from community com100 set policy-options policy-statement vrf-imp term t1 then accept set policy-options policy-statement vrf-imp term t2 from community com200 set policy-options policy-statement vrf-imp term t2 then accept set policy-options policy-statement vrf-imp term t3 from community com300 set policy-options policy-statement vrf-imp term t3 then accept set policy-options policy-statement vrf-imp term t4 from community com400 set policy-options policy-statement vrf-imp term t4 then accept set policy-options policy-statement vrf-imp term t5 then reject set policy-options community com100 members target:1:100 set policy-options community com200 members target:1:200 set policy-options community com300 members target:1:300 set policy-options community com400 members target:1:400 set switch-options vtep-source-interface lo0.0 set switch-options route-distinguisher :1 set switch-options vrf-import vrf-imp set switch-options vrf-target target:9999:

52 set vlans v100 vlan-id 100 set vlans v100 vxlan vni 1100 set vlans v100 vxlan ingress-node-replication set vlans v200 vlan-id 200 set vlans v200 vxlan vni 1200 set vlans v200 vxlan ingress-node-replication set vlans v300 vlan-id 300 set vlans v300 vxlan vni 1300 set vlans v300 vxlan ingress-node-replication set vlans v400 vlan-id 400 set vlans v400 vxlan vni 1400 set vlans v400 vxlan ingress-node-replication 2 set system host-name leaf-2 set chassis aggregated-devices ethernet device-count 2 set interfaces xe-0/0/3 unit 0 family inet address /31 set interfaces xe-0/0/5 unit 0 family inet address /31 set interfaces xe-0/0/36 ether-options 802.3ad ae0 set interfaces xe-0/0/37 ether-options 802.3ad ae1 set interfaces xe-0/0/38 unit 0 family ethernet-switching interface-mode trunk set interfaces xe-0/0/38 unit 0 family ethernet-switching vlan members v300 set interfaces xe-0/0/38 unit 0 family ethernet-switching vlan members v100 set interfaces ae0 esi 00:01:01:01:01:01:01:01:01:01 set interfaces ae0 esi all-active set interfaces ae0 aggregated-ether-options lacp passive set interfaces ae0 aggregated-ether-options lacp system-id 00:00:00:01:01:01 set interfaces ae0 unit 0 family ethernet-switching interface-mode access set interfaces ae0 unit 0 family ethernet-switching vlan members v100 set interfaces ae1 esi 00:02:02:02:02:02:02:02:02:02 set interfaces ae1 esi all-active set interfaces ae1 aggregated-ether-options lacp passive set interfaces ae1 aggregated-ether-options lacp system-id 00:00:00:01:01:01 set interfaces ae1 unit 0 family ethernet-switching interface-mode access set interfaces ae1 unit 0 family ethernet-switching vlan members v200 set interfaces lo0 unit 0 family inet address /32 set routing-options router-id set routing-options autonomous-system set routing-options forwarding-table export load-balance set routing-options forwarding-table ecmp-fast-reroute set protocols bgp group underlay type external set protocols bgp group underlay advertise-peer-as set protocols bgp group underlay family inet unicast loops 2 set protocols bgp group underlay export lo0 set protocols bgp group underlay peer-as set protocols bgp group underlay multipath set protocols bgp group underlay neighbor description spine-1 set protocols bgp group underlay neighbor description spine-2 set protocols bgp group EVPN_VXLAN_CORE type external 52