SPDX-License-Identifier: Apache-2.0
Copyright (c) 2019-2020 Intel Corporation
HugePage support in Smart Edge Open
Overview
Memory is allocated to application processes in terms of pages and by default, 4K pages are supported. For applications dealing with larger datasets, using 4K pages may lead to performance degradation and overhead because of translation lookaside buffer (TLB) misses. To address this, modern CPUs support HugePages which are typically 2M and 1G. This helps avoid TLB miss overhead and therefore improves performance.
Both applications and network functions can improve performance using HugePages. Huge page support, added to Kubernetes* v1.8, enables the discovery, scheduling, and allocation of huge pages as a native, first-class resource. This support addresses low latency and deterministic memory access requirements.
Details of HugePage support on Smart Edge Open
Deployment of Smart Edge Open’ minimal flavor does not enable the hugepages.
To enable hugepages either use flavor that supports hugepages (e.g. flexran) or enable hugepages by editing default_grub_params
variable in group_vars
and/or host_vars
. Suggested value for hugepage enablement is default_hugepagesz= hugepagesz= hugepages=
.
Next, following parameters can be used for tuning hugepages:
hugepage_size
- size, which can be either2M
or1G
hugepage_amount
- amount
Previously default values were:
| Machine type | hugepage_amount
| hugepage_size
|
|————–|——————-|—————–|
| Controller | 1024
| 2M
|
| Node | 1024
| 2M
|
Find below a guide on changing these values. Customizations must be made before Smart Edge Open deployment.
Variables for hugepage customization can be placed in several files:
inventory/default/group_vars/controller_group/10-open.yml
andinventory/default/group_vars/edgenode_group/10-open.yml
will affect Edge Controller and Edge Nodesinventory/default/host_vars/<inventory_host_name>/10-open.yml
will only affect<inventory_host_name>
host present ininventory.yml
- Hugepages can be also specified inside playbook, however due to Ansible’s* variable priority this is not recommended (it will override both
group_vars
andhost_vars
). For example:# network_edge.yml - hosts: edgenode_group vars: hugepage_amount: "5000"
The usage is summarized in the following table:
File | Edge Controller | Edge Node | Comment |
---|---|---|---|
inventory/default/group_vars/controller_group/10-open.yml |
yes | ||
inventory/default/group_vars/edgenode_group/10-open.yml |
yes - every node | ||
inventory/default/host_vars/<inventory_host_name>/10-open.yml |
yes | yes | affects machine specified in inventory.yml with name <inventory_host_name> |
network_edge.yml |
vars under hosts: controller_group |
vars under hosts: edgenode_group - every node |
not recommended |
Note that variables have precedence:
- not recommended:
network_edge.yml
will always take precedence for files from this list (overrides every other var) inventory/default/host_vars/
inventory/default/group_vars/edgenode_group/20-enhanced.yml
andinventory/default/group_vars/controller_group/20-enhanced.yml
inventory/default/group_vars/edgenode_group/10-open.yml
andinventory/default/group_vars/controller_group/10-open.yml
inventory/default/group_vars/all/20-enhanced.yml
inventory/default/group_vars/all/10-open.yml
default/main.yml
in roles’ directory
Examples
Changing size and amount of the hugepages for both controller and nodes
Change the following lines in the inventory/default/group_vars/edgenode_group/10-open.yml
or inventory/default/group_vars/controller_group/10-open.yml
:
- To set 1500 of the hugepages with the page size of 2 MB (which is the default value) for the Edge Controller:
# inventory/default/group_vars/controller_group/10-open.yml hugepage_size: "2M" hugepage_amount: "1500" default_grub_params: "default_hugepagesz= hugepagesz= hugepages="
- To set 10 of the hugepages with the page size of 1GB for the Edge Nodes:
# inventory/default/group_vars/edgenode_group/10-open.yml hugepage_size: "1G" hugepage_amount: "10" default_grub_params: "default_hugepagesz= hugepagesz= hugepages="
Customizing hugepages for specific machine
To specify the size or amount only for a specific machine, hugepage_size
and/or hugepage_amount
can be provided in inventory/default/host_vars/<host_name_from_inventory>/10-open.yml
(i.e., if host is named node01
, then the file is inventory/default/host_vars/node01/10-open.yml
). For example:
# inventory/default/host_vars/node01/10-open.yml
hugepage_size: "2M"
hugepage_amount: "1500"
default_grub_params: "default_hugepagesz= hugepagesz= hugepages="