SPDX-License-Identifier: Apache-2.0
Copyright (c) 2019 Intel Corporation
Userspace CNI
Overview
Userspace CNI is a Container Network Interface (CNI) Kubernetes* plugin that was designed to simplify the process of deployment of DPDK-based applications in Kubernetes pods. The plugin uses Kubernetes and Multus CNI CRDs to provide pods with a virtual DPDK-enabled Ethernet port. In this document, you can find details about how to install Smart Edge Open with Userspace CNI support and how to use its main features.
Setup Userspace CNI
Smart Edge Open for Network Edge has been integrated with Userspace CNI to allow users to easily run DPDK- based applications inside Kubernetes pods. To install Smart Edge Open Network Edge with Userspace CNI support, add the value userspace
to variable kubernetes_cnis
in inventory/default/group_vars/all/10-open.yml
and set value of the variable kubeovn_dpdk
in inventory/default/group_vars/all/10-open.yml
to true
:
# inventory/default/group_vars/all/10-open.yml
kubernetes_cnis:
- kubeovn
- userspace
# inventory/default/group_vars/all/10-open.yml
kubeovn_dpdk: true
HugePages configuration
DPDK apps require that a specific number of HugePages are enabled. By default, the Ansible* scripts will enable 1024 of 2M HugePages on a system and then start OVS-DPDK with 1Gb of those HugePages. To change this setting to reflect your specific requirements, set the Ansible variables as defined in the example below. This example enables 4 of 1GB HugePages and appends 1 GB to OVS-DPDK, leaving 3 pages for DPDK applications that will be running in the pods.
# inventory/default/group_vars/controller_group/10-open.yml
hugepage_size: "1G"
hugepage_amount: "4"
default_grub_params: "default_hugepagesz= hugepagesz= hugepages= intel_iommu=on iommu=pt"
# inventory/default/group_vars/edgenode_group/10-open.yml
hugepage_size: "1G"
hugepage_amount: "4"
default_grub_params: "default_hugepagesz= hugepagesz= hugepages= intel_iommu=on iommu=pt"
# inventory/default/group_vars/all/10-open.yml
# Hugepage size to be used with DPDK: 2Mi or 1Gi
kubeovn_dpdk_hugepage_size: "1Gi"
# Overall amount of hugepages available to DPDK
kubeovn_dpdk_hugepages: "1Gi"
Pod deployment
To deploy a pod with a DPDK interface, create a pod with hugepages
mounted to /dev/hugepages
, host directory /var/run/openvswitch/
(with mandatory trailing slash character) mounted into the pod with the volume name shared-dir
(the name shared-dir
is mandatory), and userspace-openness
network annotation. Find an example pod definition with two DPDK ports below:
apiVersion: v1
kind: Pod
metadata:
name: userspace-example
annotations:
k8s.v1.cni.cncf.io/networks: userspace-openness, userspace-openness
spec:
containers:
- name: userspace-example
image: image-name
imagePullPolicy: Never
securityContext:
privileged: true
volumeMounts:
- mountPath: /ovs
name: shared-dir
- mountPath: /dev/hugepages
name: hugepages
resources:
requests:
memory: 1Gi
limits:
hugepages-1Gi: 2Gi
command: ["sleep", "infinity"]
volumes:
- name: shared-dir
hostPath:
path: /var/run/openvswitch/
- name: hugepages
emptyDir:
medium: HugePages
Virtual interface usage
Socket files for virtual interfaces generated by Userspace CNI are created on the host machine in /var/run/openvswitch
directory. This directory has to be mounted into your pod by volume with obligatory name shared-dir
(in our example pod definition /var/run/openvswitch
is mounted to pod as /ovs
). You can then use the sockets available from your mount-point directory in your DPDK-enabled application deployed inside a pod. Find more examples in Userspace CNI’s documentation.
NOTE: When 2MB HugePages are enabled for Userspace CNI there is a limitation to how many memory regions can be used by Virtio - in an event where number of memory regions is exceeded by DPDK application, a flag ‘–single-file-segments’ needs to be added to the applications starting command line.