VXLAN with Multicast Underlay

One of the control planes with VXLAN is with a Multicast Underlay; this is how it was configured.

The goal is to allow User-Device-1 on 10.1.1.1 to communicate on User-Device-2 on 10.1.1.2 via VXLAN, making both devices appear as if they were in a single broadcast domain.

Site-1 and Site-2 are the enterprise owned routers.

There is a GRE tunnel between Site-1 and Site-2 to act as a VPN for the VXLAN

Core-A, Core-B, Core-C is the internet service providers network, it provides only routing between Site-1 and Site-2 through multiple hops

Testing

User-Device-1#ping 10.1.1.2 source 10.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.2, timeout is 2 seconds:
Packet sent with a source address of 10.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms

User-Device-1 Configuration (Generic L2 Switch)

version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname User-Device-1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
clock timezone GMT 0 0
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode rapid-pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Ethernet0/0
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Vlan1
 ip address 10.1.1.1 255.255.255.0
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 10.1.1.254
!
!
!
!
!
control-plane
!
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
 login
!
!
end

User-Device-2 Configuration(Generic L2 Switch)

version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname User-Device-2
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
clock timezone GMT 0 0
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode rapid-pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Ethernet0/0
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Vlan1
 ip address 10.1.1.2 255.255.255.0
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 10.1.1.254
!
!
!
!
!
control-plane
!
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
 login
!
!
end

Site-1 Configuration (CSR1000V)

version 17.3
service timestamps debug datetime msec
service timestamps log datetime msec
service call-home
platform qfp utilization monitor load 80
platform punt-keepalive disable-kernel-core
platform console serial
!
hostname Site-1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
ip multicast-routing distributed
!
!
!
!
!
!
!
!
!
login on-success log
!
!
!
!
!
!
!
subscriber templating
!
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
crypto pki trustpoint TP-self-signed-1297834211
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-1297834211
revocation-check none
rsakeypair TP-self-signed-1297834211
!
crypto pki trustpoint SLA-TrustPoint
enrollment pkcs12
revocation-check crl
!
!
crypto pki certificate chain TP-self-signed-1297834211
certificate self-signed 01
30820330 30820218 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 31323937 38333432 3131301E 170D3231 31303133 32303039
35315A17 0D333131 30313332 30303935 315A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D31 32393738
33343231 31308201 22300D06 092A8648 86F70D01 01010500 0382010F 00308201
0A028201 0100A133 610C0FE7 4646CB17 EE422451 AD5BAFC5 71D122AD D15682C0
B5847B37 5AE5B325 C509B943 F7518FD1 9AB46BAE B3F05760 0B72D80A 630BD3E4
B41A02CD 642247D9 CF324892 0CE05A6E E600D619 CABA187F C6E4946A 9F808E1B
3BD990A9 5A0E411E 676CC100 1C3B7B94 63CC01AB 909EF611 45DAF74B C29FBAAD
F1C2C488 8121692E 4724B4B2 0907B896 730A4E78 5EAF7FEA 414BA0A3 F16E4ED5
26354B39 B1C1CD5C 2F29B604 0E1F0FAF 5563A625 AC5CEEB5 EEADAA10 9FB82E70
C9A54114 80E0D327 FD112523 4774AD0C 061C5C80 562FAD0F D93ACB53 D3958D54
0173C167 C5BF0B28 75148F5E DC6964A0 9C0EB532 3F67537F A45246D7 4B5C0AD1
2AA8A6B6 1AE10203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF
301F0603 551D2304 18301680 14FA911F 98C7EFE5 49BFBBF5 3A1F997F DD1AE7B7
1F301D06 03551D0E 04160414 FA911F98 C7EFE549 BFBBF53A 1F997FDD 1AE7B71F
300D0609 2A864886 F70D0101 05050003 82010100 7AD58541 EB0F7002 7E1A7FD3
CA945546 D88C623D C1192F04 911FD3B3 40B993B8 412E0BCE 6D4A9841 795CC5B2
DDC4715F 457AC97C 402AE4E7 36CF01F8 CDEEC689 1977EC39 92842175 1642A0F4
BAA3A719 7A2AD763 C34D09C6 00219F00 BD7AD862 D1F63EBD BC13CAD7 5C58D0BF
05FF51B4 4BB7E73A 3EFE14C2 34BF7B91 D8C641C0 9DF70671 BAAE3B26 93C685CF
27DF61F6 23CF420D FCB264C0 FA268BFB C6E3FEF2 CBFDEDAC 17A544F9 D22F8216
CE2AC2E2 E19D48EF 76A82FB2 23FAA71F C5097989 B22D260B F15AD2B6 DC6132D8
FB8A8958 444CD821 02ECD18E F623984A 9A1133DA 4FBFFDD4 A9957D2D F74C2D57
9F943985 F17BD8AF 7A5AC7AB B8F7E0F4 5B998758
quit
crypto pki certificate chain SLA-TrustPoint
certificate ca 01
30820321 30820209 A0030201 02020101 300D0609 2A864886 F70D0101 0B050030
32310E30 0C060355 040A1305 43697363 6F312030 1E060355 04031317 43697363
6F204C69 63656E73 696E6720 526F6F74 20434130 1E170D31 33303533 30313934
3834375A 170D3338 30353330 31393438 34375A30 32310E30 0C060355 040A1305
43697363 6F312030 1E060355 04031317 43697363 6F204C69 63656E73 696E6720
526F6F74 20434130 82012230 0D06092A 864886F7 0D010101 05000382 010F0030
82010A02 82010100 A6BCBD96 131E05F7 145EA72C 2CD686E6 17222EA1 F1EFF64D
CBB4C798 212AA147 C655D8D7 9471380D 8711441E 1AAF071A 9CAE6388 8A38E520
1C394D78 462EF239 C659F715 B98C0A59 5BBB5CBD 0CFEBEA3 700A8BF7 D8F256EE
4AA4E80D DB6FD1C9 60B1FD18 FFC69C96 6FA68957 A2617DE7 104FDC5F EA2956AC
7390A3EB 2B5436AD C847A2C5 DAB553EB 69A9A535 58E9F3E3 C0BD23CF 58BD7188
68E69491 20F320E7 948E71D7 AE3BCC84 F10684C7 4BC8E00F 539BA42B 42C68BB7
C7479096 B4CB2D62 EA2F505D C7B062A4 6811D95B E8250FC4 5D5D5FB8 8F27D191
C55F0D76 61F9A4CD 3D992327 A8BB03BD 4E6D7069 7CBADF8B DF5F4368 95135E44
DFC7C6CF 04DD7FD1 02030100 01A34230 40300E06 03551D0F 0101FF04 04030201
06300F06 03551D13 0101FF04 05300301 01FF301D 0603551D 0E041604 1449DC85
4B3D31E5 1B3E6A17 606AF333 3D3B4C73 E8300D06 092A8648 86F70D01 010B0500
03820101 00507F24 D3932A66 86025D9F E838AE5C 6D4DF6B0 49631C78 240DA905
604EDCDE FF4FED2B 77FC460E CD636FDB DD44681E 3A5673AB 9093D3B1 6C9E3D8B
D98987BF E40CBD9E 1AECA0C2 2189BB5C 8FA85686 CD98B646 5575B146 8DFC66A8
467A3DF4 4D565700 6ADF0F0D CF835015 3C04FF7C 21E878AC 11BA9CD2 55A9232C
7CA7B7E6 C1AF74F6 152E99B7 B1FCF9BB E973DE7F 5BDDEB86 C71E3B49 1765308B
5FB0DA06 B92AFE7F 494E8A9E 07B85737 F3A58BE1 1A48A229 C37C1E69 39F08678
80DDCD16 D6BACECA EEBC7CF9 8428787B 35202CDC 60E4616A B623CDBD 230E3AFB
418616A9 4093E049 4D10AB75 27E86F73 932E35B5 8862FDAE 0275156F 719BB2F0
D697DF7F 28
quit
!
license udi pid CSR1000V sn 9KXQ2Y1ERO2
diagnostic bootup level minimal
memory free low-watermark processor 71507
!
!
spanning-tree extend system-id
!
!
redundancy
!
bridge-domain 1
member vni 4096
member GigabitEthernet1 service-instance 1
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 10.10.10.10 255.255.255.255
ip pim sparse-mode
ip ospf 1 area 0
!
interface Tunnel1
ip address 192.168.1.1 255.255.255.0
ip pim sparse-mode
ip ospf 1 area 0
tunnel source GigabitEthernet2
tunnel destination 1.1.4.2
!
interface GigabitEthernet1
no ip address
negotiation auto
no mop enabled
no mop sysid
service instance 1 ethernet
encapsulation untagged
!
!
interface GigabitEthernet2
ip address 1.1.1.1 255.255.255.0
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
no ip address
shutdown
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet4
no ip address
shutdown
negotiation auto
no mop enabled
no mop sysid
!
interface nve1
no ip address
ip pim sparse-mode
source-interface Loopback0
member vni 4096 mcast-group 230.1.1.1
no mop enabled
no mop sysid
!
router ospf 1
!
ip forward-protocol nd
ip pim rp-address 10.10.10.20
ip http server
ip http authentication local
ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 1.1.1.2
!
!
!
!
!
!
!
control-plane
!
!
!
!
!
!
line con 0
stopbits 1
line vty 0 4
login
transport input ssh
!
call-home
! If contact email address in call-home is configured as sch-smart-licensing@cisco.com
! the email address configured in Cisco Smart License Portal will be used as contact email address to send SCH notifications.
contact-email-addr sch-smart-licensing@cisco.com
profile "CiscoTAC-1"
active
destination transport-method http
!
!
!
!
!
end

Site-2 Configuration (CSR1000V)

version 17.3
service timestamps debug datetime msec
service timestamps log datetime msec
service call-home
platform qfp utilization monitor load 80
platform punt-keepalive disable-kernel-core
platform console serial
!
hostname Site-2
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
ip multicast-routing distributed
!
!
!
!
!
!
!
!
!
login on-success log
!
!
!
!
!
!
!
subscriber templating
!
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
crypto pki trustpoint TP-self-signed-849732361
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-849732361
revocation-check none
rsakeypair TP-self-signed-849732361
!
crypto pki trustpoint SLA-TrustPoint
enrollment pkcs12
revocation-check crl
!
!
crypto pki certificate chain TP-self-signed-849732361
certificate self-signed 01
3082032E 30820216 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
30312E30 2C060355 04031325 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 38343937 33323336 31301E17 0D323131 30313332 30333530
335A170D 33313130 31333230 33353033 5A303031 2E302C06 03550403 1325494F
532D5365 6C662D53 69676E65 642D4365 72746966 69636174 652D3834 39373332
33363130 82012230 0D06092A 864886F7 0D010101 05000382 010F0030 82010A02
82010100 B62E08A0 DE8C4923 E33F1E37 3FEC0FF5 40045B37 8D2F8B22 F3973F3B
05170CA0 34D4605A 024347B9 BD8F72D5 0245A3EB 7BD89D26 05C1C0C8 1E3BE448
DB912ECD E639D29A 68DAB41A D12D96CF CA1B6942 35D5A1BA 2551AAA1 2D149035
9AC1E79C 39149F19 276890BB 6FE1D7C9 B918FA7C 2E5BF6DF B53A8683 885783DA
B5E4FABC F932C1F4 EF34BCC7 B467F6E5 5EC2343A C3099E70 75D272E3 6F5C4E91
49D61599 43B36081 37E3A404 969FB356 4A492FC8 E4331256 4088508D 1131A340
38A36F0C 7C6B508C 9DCC50E2 25FA63A4 BDD57002 2FBE88E2 BC7CD01E 52425207
21C9D7C3 48CBB709 D6B32768 F3368294 CFCC67E2 0ECB8D6C 0E39ABF3 CE903B63
742CC6A3 02030100 01A35330 51300F06 03551D13 0101FF04 05300301 01FF301F
0603551D 23041830 1680143C 96FC975C CF1882ED DA221103 B9C96131 F8061430
1D060355 1D0E0416 04143C96 FC975CCF 1882EDDA 221103B9 C96131F8 0614300D
06092A86 4886F70D 01010505 00038201 01009AA5 3B37C8FA 25F333BE F984AAD7
A4A9F61B 5B05D378 BB8EFAC8 DDE9570A 46C86B00 C46B739D 17D54C9D D44059E5
67964D39 EA0C64DA 759EA038 5D5C0B4C A7910914 E4EC9B85 0136FB13 56D7C106
9FF3B6A6 3B0425DC CE3CF545 B6D3230B 576A4D2A B5052641 ECB331F2 49094794
5DB196A1 B1265715 A33C33D5 9AB11F42 7CE7F875 CE82A874 E938875D 7F4B0DAC
0C613734 216C1390 5E74EF5D 8CB37E29 2FD98125 2C2B8FBB 5C9A3F5E A975C6E5
65F50248 E099F181 A0FF1D3B 439B4263 F6E04174 FE462726 76846479 9DE64645
25502B54 88E23B5F 086285E3 C7027291 41708015 226EB6F7 B356EF7B 5F7FC313
B2DD05DF 8C55795B E75B1264 AAA86EB9 C690
quit
crypto pki certificate chain SLA-TrustPoint
certificate ca 01
30820321 30820209 A0030201 02020101 300D0609 2A864886 F70D0101 0B050030
32310E30 0C060355 040A1305 43697363 6F312030 1E060355 04031317 43697363
6F204C69 63656E73 696E6720 526F6F74 20434130 1E170D31 33303533 30313934
3834375A 170D3338 30353330 31393438 34375A30 32310E30 0C060355 040A1305
43697363 6F312030 1E060355 04031317 43697363 6F204C69 63656E73 696E6720
526F6F74 20434130 82012230 0D06092A 864886F7 0D010101 05000382 010F0030
82010A02 82010100 A6BCBD96 131E05F7 145EA72C 2CD686E6 17222EA1 F1EFF64D
CBB4C798 212AA147 C655D8D7 9471380D 8711441E 1AAF071A 9CAE6388 8A38E520
1C394D78 462EF239 C659F715 B98C0A59 5BBB5CBD 0CFEBEA3 700A8BF7 D8F256EE
4AA4E80D DB6FD1C9 60B1FD18 FFC69C96 6FA68957 A2617DE7 104FDC5F EA2956AC
7390A3EB 2B5436AD C847A2C5 DAB553EB 69A9A535 58E9F3E3 C0BD23CF 58BD7188
68E69491 20F320E7 948E71D7 AE3BCC84 F10684C7 4BC8E00F 539BA42B 42C68BB7
C7479096 B4CB2D62 EA2F505D C7B062A4 6811D95B E8250FC4 5D5D5FB8 8F27D191
C55F0D76 61F9A4CD 3D992327 A8BB03BD 4E6D7069 7CBADF8B DF5F4368 95135E44
DFC7C6CF 04DD7FD1 02030100 01A34230 40300E06 03551D0F 0101FF04 04030201
06300F06 03551D13 0101FF04 05300301 01FF301D 0603551D 0E041604 1449DC85
4B3D31E5 1B3E6A17 606AF333 3D3B4C73 E8300D06 092A8648 86F70D01 010B0500
03820101 00507F24 D3932A66 86025D9F E838AE5C 6D4DF6B0 49631C78 240DA905
604EDCDE FF4FED2B 77FC460E CD636FDB DD44681E 3A5673AB 9093D3B1 6C9E3D8B
D98987BF E40CBD9E 1AECA0C2 2189BB5C 8FA85686 CD98B646 5575B146 8DFC66A8
467A3DF4 4D565700 6ADF0F0D CF835015 3C04FF7C 21E878AC 11BA9CD2 55A9232C
7CA7B7E6 C1AF74F6 152E99B7 B1FCF9BB E973DE7F 5BDDEB86 C71E3B49 1765308B
5FB0DA06 B92AFE7F 494E8A9E 07B85737 F3A58BE1 1A48A229 C37C1E69 39F08678
80DDCD16 D6BACECA EEBC7CF9 8428787B 35202CDC 60E4616A B623CDBD 230E3AFB
418616A9 4093E049 4D10AB75 27E86F73 932E35B5 8862FDAE 0275156F 719BB2F0
D697DF7F 28
quit
!
license udi pid CSR1000V sn 91G3NG7XFED
diagnostic bootup level minimal
memory free low-watermark processor 71507
!
!
spanning-tree extend system-id
!
!
redundancy
!
bridge-domain 1
member vni 4096
member GigabitEthernet1 service-instance 1
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback1
ip address 10.10.10.20 255.255.255.255
ip pim sparse-mode
ip ospf 1 area 0
!
interface Tunnel1
ip address 192.168.1.2 255.255.255.0
ip pim sparse-mode
ip ospf 1 area 0
tunnel source GigabitEthernet2
tunnel destination 1.1.1.1
!
interface GigabitEthernet1
no ip address
negotiation auto
no mop enabled
no mop sysid
service instance 1 ethernet
encapsulation untagged
!
!
interface GigabitEthernet2
ip address 1.1.4.2 255.255.255.0
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet3
no ip address
shutdown
negotiation auto
no mop enabled
no mop sysid
!
interface GigabitEthernet4
no ip address
shutdown
negotiation auto
no mop enabled
no mop sysid
!
interface nve1
no ip address
ip pim sparse-mode
source-interface Loopback1
member vni 4096 mcast-group 230.1.1.1
no mop enabled
no mop sysid
!
router ospf 1
!
ip forward-protocol nd
ip pim rp-address 10.10.10.20
ip http server
ip http authentication local
ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 1.1.4.1
!
!
!
!
!
!
!
control-plane
!
!
!
!
!
!
line con 0
stopbits 1
line vty 0 4
login
transport input ssh
!
call-home
! If contact email address in call-home is configured as sch-smart-licensing@cisco.com
! the email address configured in Cisco Smart License Portal will be used as contact email address to send SCH notifications.
contact-email-addr sch-smart-licensing@cisco.com
profile "CiscoTAC-1"
active
destination transport-method http
!
!
!
!
!
end