iii

Author: Elena Crenguta Lindqvist

itnot/index.html

@@ -73,7 +73,13 @@
 					</p>
 				        </tr>
 					<aside class="notes">
-A quick one about ericsson -  40% mobile traffic, erlang, bluetooth - NOT DONE YET , need to write it - advance to next slide - 
+If you don't mind, I'd like to tell you a few things about Ericsson. We're NOT making phones! as most of the people think ....
+I can tell you that no matter where you are in the world, when you are connected to a mobile network and you access the internet, there is a high chance that your traffic goes through our stuff, radio base stations, servers running ericsson software etc...
+40% of all mobile traffic world wide goes through our stuff.
+
+And since we are at an Openstack meetup and talking about open source, it is probably good to mention that Ericsson has given to open source Erlang (Ericsson Language, or named after a Danish mathematician, whichever you prefer)
+In Openstack, RabbitMQ uses Erlang.
+Bluetooth comes from Ericsson too ...
 					</aside>
 				</section>
 <!––  Slide6 ––>	
@@ -94,7 +100,13 @@
 
 Some applications require high throughput and Ericsson's EPC is one notable example.
 EPC is the equivalent of formerly used GPRS, it is there to make mobile data traffic possible. It means you traverse it when surfing the internet from your mobile, or when watching youtube, Netflix, GoT, playing Pokemon?
-The monthly mobile data traffic grew close to 88% between Q4 2017 and Q4 2018, this is mainly due to the increased traffic per smartphone in China. - advance to next slide - 
+According to Ericsson Mobility Report, the monthly mobile data traffic grew close to 88% between Q4 2017 and Q4 2018, this is mainly due to the increased traffic per smartphone in China. 
+According to the same report, mobile traffic is 50% video today and it will increase to 75% video in 2020, driven by, amongst others, AR/VR applications. 
+
+This increase in number of devices using the mobile network and the traffic generated by them, it boils down to EPC will need to handle all this traffic.
+How to cope with this in a performant way, well, maybe smartNICs could have the answer.  
+gtp tunnel to EPC VMs , descapsulate traffic inside the VM
+- advance to next slide - 
 					</aside>
 				</section>
 <!––  Slide7 ––>	
@@ -163,15 +175,70 @@
 What security concerns are raised with introducing a smartNIC with linux running on it. I'm looking at you, Kim!
 
 When should you use a smartNIC?
-(Intel info) If on your host you use more than 4CPUs for OVS, then you should switch to using smartNICs, it makes sense from a business point of view. - advance to next slide - 
+(Intel info) If on your host you use more than 4CPUs for OVS, then you should switch to using smartNICs, it makes sense from a business point of view. 
+(Also, smartNIC is a good idea if you need low latency and don't care so much about migration)
+- advance to next slide - 
                                         </aside>
                                 </section>
 
 <!––  Slide10 ––>
                                 <section> Openstack working with SmartNICs
                                         <aside class="notes">
-Now the question is where are we in Openstack when it comes to integrating the new coming smartNICs?
-Work is done in several Openstack components, like ironic and neutron of course.											
+Now the question is where are we in Openstack when it comes to integrating the wide range of smartNICs appearing on the market?
+Work is done in several Openstack projects, like ironic, nova and neutron of course.	
+
+For instance when it comes to neutron, we need changes in the Neutron OVS driver and Neutron OVS agent in order to bind the Neutron port for the baremetal host with the smartNIC.
+This is needed so that neutron ovs agent can configure the OVS running on the smartNIC.
+
+We can have neutron ovs agent running locally on the smartNIC or remotely and manages the OVS bridges for all baremetal smartNICs.
+
+In ironic
+Steps: 
+(https://specs.openstack.org/openstack/neutron-specs/specs/stein/neutron-ovs-agent-support-baremetal-with-smart-nic.html)
+1. Create Neutron port with smart-nic vnic_type , this is done in Neutron OVS ML2 driver 
+2. local_link_information wiht info like smartNIC hostname , port ID, ssh public key, ovsdb ssl certificate
+
+Picture on the slide with vnic-type 
+root@cic-1:~# openstack  port create
+usage: openstack port create [-h] [-f {json,shell,table,value,yaml}]
+                             [-c COLUMN] [--max-width <integer>] [--fit-width]
+                             [--print-empty] [--noindent] [--prefix PREFIX]
+                             --network <network> [--description <description>]
+                             [--device <device-id>]
+                             [--mac-address <mac-address>]
+                             [--device-owner <device-owner>]
+                             [--vnic-type <vnic-type>] [--host <host-id>]
+                             [--dns-name dns-name]
+                             [--fixed-ip subnet=<subnet>,ip-address=<ip-address>]
+                             [--binding-profile <binding-profile>]
+                             [--enable | --disable] [--project <project>]
+                             [--project-domain <project-domain>]
+                             [--security-group <security-group> | --no-security-group]
+                             [--qos-policy <qos-policy>]
+                             [--enable-port-security | --disable-port-security]
+                             [--allowed-address ip-address=<ip-address>[,mac-address=<mac-address>]]
+                             [--tag <tag> | --no-tag]
+                             <name>
+--vnic-type <vnic-type>
+VNIC type for this port (direct | direct-physical | macvtap | normal | baremetal, default: normal)
+
+There are many interesting questions raised, like how do you know which smartNIC hostname belongs to which server.
+Ovs 
+
+
+Ironic
+Is Openstack ready to use smartNICs?
+In Openstack we need to manage the smartNICs that are running full(y fledged) operating system inside the card.
+
+
+Nova: 
+The smartNICs can do packet processing inside the NIC so in order to support this hw acceleration, nova needs some changes. 
+
+ 
+
+Openstack Cybborg
+
+										
 Some smartNIC vendors use native virtio driver while other use proprietary 
 
 https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/netronome-smartnic-enablement.html
@@ -181,18 +248,90 @@
 Neutron integration with SmartNICs - TO BE ADDED
 Openstack Cyborg for integrating FPGAs with openstack - TO BE ADDED
 
-Is Openstack ready to use smartNICs?
-In Openstack we need to manage the smartNICs that are running full(y fledged) operating system inside the card.
- 
+
 - advance to next slide - 
                                         </aside>
                                 </section>
 
 <!––  Slide11 ––>
                                 <section> Performance
                                         <aside class="notes">
+											
+
+If you care about latency and packet processing performance there are a few options. 
+
+We basically need to overcome the limitations in the Linux kernel which is not ideal for "lots" of packet-processing.
+
+Why is the Linux kernel a problem when we talk about latency and performance ( with performance, I mean higher throughput at a lower CPU cost)? 
+Well, the Linux kernel is monolithic, it's millions of lines of code.
+It contains lots of drivers which makes the Linux kernel work with any hw, not just your specific hw/smartNIC. 
+It allows running many applications at the same time by using a time sharing layer. Resources like CPU, mem exposed by the kernel can be shared between all the processes running. 
+
+The networking stack inside the Linux kernel limits how many packets per second it can process.
+Too many packets per second means CPUs get busy just receiving packets, then either the packets are dropped or we CPU starve the applications.
+
+To get better performance, one can choose to bypass the kernel, fully or partially.
+There are several kernel bypass options like: 
+DPDK, (that would be the poster child of kernel bypass)
+Snabbswitch, 
+PF_RING, 
+Netmap. (I am personally more familiar with DPDK.)
+
+With kernel bypass, we move the NIC to the user-space.
+If the NIC is managed in the user-space, it means we skip things like context switching, networking layer processing, interrupts that happen in the kernel aka IRQ storms and do the packet-processing in the user-space.
+This is relevant at 10Gbps already. EPC today saturates 200Gbps already. 
+
+NUMA awareness together with CPU isolation needs to be considered as well if we need high performance. 
+Remember this is particularly interesting when using smartNICs, with a dual socket system using one smartNIC plugged in on PCIe slot, corresponding to one NUMA socket.
+
+Moving to userspace means losing the abstraction level the kernel provides for e.g. hw resources, it means you need to load own driver.
+Moving to userspace means the kernel space is skipped together with the good stuff too like networking functionality that needs to be reimplemented now. (like the whole TCP stack)
+
+
+How do you move a device from kernel space to user space, in case of DPDK?
+
+echo 0000:18:00.4 > /sys/bus/pci/devices/0000\:18\:00.4/driver/unbind
+echo 0000:18:00.3 > /sys/bus/pci/devices/0000\:18\:00.3/driver/unbind
+modprobe vfio
+modprobe vfio_pci
+
+mkdir -p /dev/hugepages/
+mount -t  hugetlbfs hugetlbfs /dev/hugepages/
+echo 2048 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
+cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
+
+modprobe uio
+cd $RTE_SDK
+insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
+
+./usertools/dpdk-devbind.py -b igb_uio 18:00.2 18:00.3
+./usertools/dpdk-devbind.py -s
+cd $RTE_SDK/$RTE_TARGET/app
+    
+
+XDP
+Another way to achieve high performance would be partially bypassing the Linux kernel, for example using XDP. 
+
+XDP DP (eXpress Data Path) is an eBPF based high performance data path merged in the Linux kernel.
+
+XDP (eXpress Data Path) is shipped with the kernel since version 4.8 and it is enabled by default, with CONFIG_BPF_SYSCALL.
+
+What the heck is eBPF ?
+eBPF stands for "enhanced Berkeley Packet Filter" it's a linux kernel technology 
+ which is an in-kernel virtual machine that was originally used for to run mini-filter programs efficiently, 
+
+To check if XDP it is enabled in the kernel, it's simply grepping for it in the kernel config file 
+
+┌─(ecrehar@elxahkpv4m2:pts/5)────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────(~/Downloads)─┐
+└─(15:09:%)── grep CONFIG_BPF_SYSCALL /boot/config-4.15.0-46-generic                                                                                                                                                 ──(mån,06.03)─┘
+CONFIG_BPF_SYSCALL=y
+
+
+XDP is a hook in the Linux kernel. 
+
+
 Jumping between kernel space and user space cost on performance - TO BE ADDED
-Mention eBPF with XDP kernel hook,  DPDK etc .. TO BE ADDED 
+Mention eBPF with XDP kernel hook,  DPDK, etc .. TO BE ADDED 
 Native VirtIO driver benefits - TO BE ADDED
 - advance to next slide - 
                                         </aside>
@@ -201,6 +340,7 @@
                                 <section> smartNICs with Storage
                                         <aside class="notes">
 smartNICs with Storage - TO BE ADDED
+Using smartNICs for NVMe termination
 NVMe ov TCP , Roce v2
 - advance to next slide - 
                                         </aside>

itnot/notes

@@ -249,5 +249,11 @@ https://technologyevangelist.co/2017/07/17/whats-a-smart-nic/
 https://en.wikipedia.org/wiki/SerDes
 https://cotscomputers.com/blog/pcie-lanes/
 https://blogs.igalia.com/dpino/2019/01/07/a-brief-introduction-to-xdp-and-ebpf/
+http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2018-12-12.log.html#t2018-12-12T17:00:19
+https://blog.cloudflare.com/why-we-use-the-linux-kernels-tcp-stack/
+https://blogs.igalia.com/dpino/2019/01/07/a-brief-introduction-to-xdp-and-ebpf/
+https://blogs.igalia.com/dpino/2019/01/02/build-a-kernel/
+https://www.oreilly.com/ideas/ebpf-and-systems-performance
+