Secure Copy Over OTV

In a very recent test, we made a very strange and if I may say so, a very crucial observation over OTV. It may be so that many people are aware of 'ip tcp adjust-mss <value>' and 'ip mtu <value>' commands, but, I did not know that it would be so very important to have these commands to ensure the success of a file transfer over OTV between two virtual machines.

The scenario is laid out as below:

Topology:

     VM1-------SW1-------ED1-------CORE-------ED2-------SW2-------VM2

192.168.11.110                                                                                        192.168.11.111

ED1 and ED2 as known are our edge devices which perform OTV functionality. OTV comes UP as expected and we are able to ping from VM1 to VM2 and vice-versa. It's not important to note, but, for completion let me mention that this is an unicast-core environment. So, back to the topic, what we checked was were we able to ssh to VM2 from VM1, and we were able to. The next thing was to do a secure copy using 'SCP'. To our shock, we happened to notice this was getting stalled and we were forced to kill the session. No matter how many times we did the file would not get copied over OTV.

[root@localhost ~]# scp file1 root@192.168.11.111:/root/
root@192.168.11.111's password:
file1                           0%  192KB 102.0KB/s - stalled -Killed by signal 2.
[root@localhost ~]#

Remove OTV, and boom the file transfer was over within a second. 

After a lot of head scratching, hair pulling, we thought of trying out our secret weapon [secret indeed, since we never knew this would work :-)] and used the above mentioned commands under the 'interface Overlay <num>' configuration on both ED1 and ED2.

Post the configuration, we gave scp a second shot and this time, well, it worked beautifully and our file transfer was complete in a second.

[root@localhost ~]# scp file1 root@192.168.11.111:/root/
root@192.168.11.111's password:
file1                         100%   31MB  30.8MB/s   00:01   
[root@localhost ~]#

The commands used are of very high importance, especially, when we do not have the freedom of changing the MTU across the whole network. 

I have pasted the configurations of the edge devices below, along with the 'ip tcp adjust-mss' and 'ip mtu' commands [in bold].

ED1:

===

otv site bridge-domain 2
!
otv site-identifier 0000.0000.0002
!
interface Overlay11
 no ip address
 no shutdown            ! added by me
 ip mtu 1340
 ip tcp adjust-mss 1300
 otv join-interface GigabitEthernet0/0/0
 otv use-adjacency-server 12.1.11.1 unicast-only
 service instance 11 ethernet
  encapsulation dot1q 11
  bridge-domain 11
 !
!
interface GigabitEthernet0/0/0
 ip address 10.1.11.1 255.255.255.0
 ip ospf 11 area 11

 no shutdown            ! added by me
 negotiation auto
!
interface GigabitEthernet0/0/1
 no ip address
 no shutdown            ! added by me  

 negotiation auto
 service instance 2 ethernet
  encapsulation dot1q 2
  bridge-domain 2
 !
 service instance 11 ethernet
  encapsulation dot1q 11
  bridge-domain 11
 !
!
router ospf 11
 router-id 1.1.1.1
!

ED2:

===

otv site bridge-domain 3
!
otv site-identifier 0000.0000.0003
!
interface Overlay11
 no ip address
 no shutdown          ! added by me
 ip mtu 1340
 ip tcp adjust-mss 1300
 otv join-interface GigabitEthernet0/0/0
 otv use-adjacency-server 11.1.11.1 unicast-only
 otv adjacency-server unicast-only
 service instance 11 ethernet
  encapsulation dot1q 11
  bridge-domain 11
 !
!
interface GigabitEthernet0/0/0
 ip address 12.1.11.1 255.255.255.0
 ip ospf 11 area 11
 no shutdown          ! added by me
 negotiation auto
!
interface GigabitEthernet0/0/1
 no ip address
 no shutdown          ! added by me
 negotiation auto
 service instance 3 ethernet
  encapsulation dot1q 3
  bridge-domain 3
 !
 service instance 11 ethernet
  encapsulation dot1q 11
  bridge-domain 11
 !
!
router ospf 11
 router-id 4.4.4.4
!

The CORE router configuration just needs to ensure you provide L3 connectivity between the edge devices.

The port of the switches connected to the access-interface of the edge devices need to be trunk and the port connected to the VM's should be access with the vlan being 11.

Hope this information has been useful for you.