What is NUMA?
Non-uniform memory access (NUMA) is a processor design, where the memory access time depends on the memory location relative to the processor. Under NUMA, a processor can access its own local memory faster than non-local memory (memory local to another processor or memory shared between processors). [Wikipedia].
Enabling and configuring NUMA properly is highly recommended for MemSQL and will lead to improved cluster performance. It is recommended to bind the memsqld process to the processors and memory in the same NUMA node, but how to bind depends on the environment and NUMA configuration.
When MemSQL is installed with memsql-ops, it will also install numactl which is used to detect and configure NUMA bindings. You can use the command memsql-ops memsql-optimize to see what the recommended configuration is (memsql-optimize). It's recommended to look over and check the suggested configuration first before allowing MemSQL-ops to change it, so select "no" when the prompt asks you to change the config. Once you look over the configuration, you can always go back and allow memsql-ops to deploy them for you by choosing "yes".
You can also make changes to your in the numa configuration file (numactl.cnf), which can found at the following path for each leaf /var/lib/memsql/leaf-*/data/numactl.cnf, assuming your MemSQL install directories are default. The below example highlights the steps needed to update the numactl.cnf file.
Example Steps: Changing NUMA configuration for one Leaf
A. MemSQL Optimizer suggests that this particular leaf should bound to NUMA nodes 0,1.
- Leaf (254C1B48B37E3B35142CE5FC5C26DF3F6816C765)
- Use NUMA nodes 0, 1
B. The current numctl.cnf file indicates that this node is bound to all 8 NUMA nodes
C. Update the file so that the leafs are bound to the recommended NUMA nodes
When the virtual environment host servers have NUMA, numactl and linux cannot detect this. Therefore, the processor and memory binding must be done by the management system of the virtual environment. The recommended setup by that system would be to configure the VMs with MemSQL nodes, such that processors and memory are bound or reserved for that NUMA node. This configuration is recommended and allows for the portability of using a VM, with the performance improvement of optimizing your system for NUMA.
Configuring NUMA properly with MemSQL is highly recommended for performance benefits. The setup and configuration of NUMA is dependent on the environment, and whether you are using virtual machines or physical servers.