<div id="docs-body">
 <div class="section">
  <div class="titlepage">
   <div>
    <div>
     <h4 class="title">
      <a name="mysql-cluster-data-node-memory-management">
      </a>
      25.4.3.13 Data Node Memory Management
     </h4>
    </div>
   </div>
  </div>
  <p>
   All memory allocation for a data node is performed when the node
        is started. This ensures that the data node can run in a stable
        manner without using swap memory, so that
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   can be used for
        latency-sensitive (realtime) applications. The following types
        of memory are allocated on data node startup:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      Data memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Shared global memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Redo log buffers
     </p>
    </li>
    <li class="listitem">
     <p>
      Job buffers
     </p>
    </li>
    <li class="listitem">
     <p>
      Send buffers
     </p>
    </li>
    <li class="listitem">
     <p>
      Page cache for disk data records
     </p>
    </li>
    <li class="listitem">
     <p>
      Schema transaction memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Transaction memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Undo log buffer
     </p>
    </li>
    <li class="listitem">
     <p>
      Query memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Block objects
     </p>
    </li>
    <li class="listitem">
     <p>
      Schema memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Block data structures
     </p>
    </li>
    <li class="listitem">
     <p>
      Long signal memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Shared memory communication buffers
     </p>
    </li>
   </ul>
  </div>
  <p>
   The
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   memory manager, which
        regulates most data node memory, handles the following memory
        resources:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      Data Memory
            (
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-datamemory">
       <code class="literal">
        DataMemory
       </code>
      </a>
      )
     </p>
    </li>
    <li class="listitem">
     <p>
      Redo log buffers
            (
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-redobuffer">
       <code class="literal">
        RedoBuffer
       </code>
      </a>
      )
     </p>
    </li>
    <li class="listitem">
     <p>
      Job buffers
     </p>
    </li>
    <li class="listitem">
     <p>
      Send buffers
            (
      <a class="link" href="mysql-cluster-tcp-definition.html#ndbparam-tcp-sendbuffermemory">
       <code class="literal">
        SendBufferMemory
       </code>
      </a>
      ,
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-totalsendbuffermemory">
       <code class="literal">
        TotalSendBufferMemory
       </code>
      </a>
      ,
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-extrasendbuffermemory">
       <code class="literal">
        ExtraSendBufferMemory
       </code>
      </a>
      )
     </p>
    </li>
    <li class="listitem">
     <p>
      Disk Data record page cache
            (
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-diskpagebuffermemory">
       <code class="literal">
        DiskPageBufferMemory
       </code>
      </a>
      ,
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-diskpagebufferentries">
       <code class="literal">
        DiskPageBufferEntries
       </code>
      </a>
      )
     </p>
    </li>
    <li class="listitem">
     <p>
      Transaction memory
            (
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-transactionmemory">
       <code class="literal">
        TransactionMemory
       </code>
      </a>
      )
     </p>
    </li>
    <li class="listitem">
     <p>
      Query memory
     </p>
    </li>
    <li class="listitem">
     <p>
      Disk access records
     </p>
    </li>
    <li class="listitem">
     <p>
      File buffers
     </p>
    </li>
   </ul>
  </div>
  <p>
   Each of these resources is set up with a reserved memory area
        and a maximum memory area. The reserved memory area can be used
        only by the resource for which it is reserved and cannot be
        shared with other resources; a given resource can never allocate
        more than the maximum memory allowed for the resource. A
        resource that has no maximum memory can expand to use all the
        shared memory in the memory manager.
  </p>
  <p>
   The size of the global shared memory for these resources is
        controlled by the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-sharedglobalmemory">
    <code class="literal">
     SharedGlobalMemory
    </code>
   </a>
   configuration parameter (default: 128 MB).
  </p>
  <p>
   Data memory is always reserved and never acquires any memory
        from shared memory. It is controlled using the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-datamemory">
    <code class="literal">
     DataMemory
    </code>
   </a>
   configuration
        parameter, whose maximum is 16384 GB.
   <code class="literal">
    DataMemory
   </code>
   is where records are stored,
        including hash indexes (approximately 15 bytes per row), ordered
        indexes (10-12 bytes per row per index), and row headers (16-32
        bytes per row).
  </p>
  <p>
   Redo log buffers also use reserved memory only; this is
        controlled by the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-redobuffer">
    <code class="literal">
     RedoBuffer
    </code>
   </a>
   configuration
        parameter, which sets the size of the redo log buffer per LDM
        thread. This means that the actual amount of memory used is the
        value of this parameter multiplied by the number of LDM threads
        in the data node.
  </p>
  <p>
   Job buffers use reserved memory only; the size of this memory is
        calculated by
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   , based on the
        numbers of threads of various types.
  </p>
  <p>
   Send buffers have a reserved part but can also allocate an
        additional 25% of shared global memory. The send buffer reserved
        size is calculated in two steps:
  </p>
  <div class="orderedlist">
   <ol class="orderedlist" type="1">
    <li class="listitem">
     <p>
      Use the value of the
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-totalsendbuffermemory">
       <code class="literal">
        TotalSendBufferMemory
       </code>
      </a>
      configuration parameter (no default value) or the sum of the
            individual send buffers used by all individual connections
            to the data node. A data node is connected to all other data
            nodes, to all API nodes, and to all management nodes. This
            means that, in a cluster with 2 data nodes, 2 management
            nodes, and 10 API nodes each data node has 13 node
            connections. Since the default value for
      <a class="link" href="mysql-cluster-tcp-definition.html#ndbparam-tcp-sendbuffermemory">
       <code class="literal">
        SendBufferMemory
       </code>
      </a>
      for
            a data node connection is 2 MByte, this works out to 26 MB
            total.
     </p>
    </li>
    <li class="listitem">
     <p>
      To obtain the total reserved size for the send buffer, the
            value of the
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-extrasendbuffermemory">
       <code class="literal">
        ExtraSendBufferMemory
       </code>
      </a>
      configuration parameter, if any (default value 0). is added
            to the value obtained in the previous step.
     </p>
    </li>
   </ol>
  </div>
  <p>
   In other words, if
   <code class="literal">
    TotalSendBufferMemory
   </code>
   has
        been set, the send buffer size is
   <code class="literal">
    TotalSendBufferMemory
        + ExtraSendBufferMemory
   </code>
   ; otherwise, the size of the
        send buffer is equal to
   <code class="literal">
    ([
    <em class="replaceable">
     <code>
      number of node
        connections
     </code>
    </em>
    ] * SendBufferMemory) +
        ExtraSendBufferMemory
   </code>
   .
  </p>
  <p>
   The page cache for disk data records uses a reserved resource
        only; the size of this resource is controlled by the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-diskpagebuffermemory">
    <code class="literal">
     DiskPageBufferMemory
    </code>
   </a>
   configuration parameter (default 64 MB). Memory for 32 KB disk
        page entries is also allocated; the number of these is
        determined by the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-diskpagebufferentries">
    <code class="literal">
     DiskPageBufferEntries
    </code>
   </a>
   configuration parameter (default 10).
  </p>
  <p>
   Transaction memory has a reserved part that either is calculated
        by
   <code class="literal">
    NDB
   </code>
   , or is set explicitly using the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-transactionmemory">
    <code class="literal">
     TransactionMemory
    </code>
   </a>
   configuration parameter; transaction memory can also use an
        unlimited amount of shared global memory. Transaction memory is
        used for all operational resources handling transactions, scans,
        locks, scan buffers, and trigger operations. It also holds table
        rows as they are updated, before the next commit writes them to
        data memory.
  </p>
  <p>
   Resources are allocated from a common transaction memory
        resource and can also use resources from global shared memory.
        the size of this resource can be controlled using a single
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-transactionmemory">
    <code class="literal">
     TransactionMemory
    </code>
   </a>
   configuration parameter.
  </p>
  <p>
   Reserved memory for undo log buffers can be set using the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-initiallogfilegroup">
    <code class="literal">
     InitialLogFileGroup
    </code>
   </a>
   configuration parameter. If an undo log buffer is created as
        part of a
   <a class="link" href="create-logfile-group.html" title="15.1.16 CREATE LOGFILE GROUP Statement">
    <code class="literal">
     CREATE LOGFILE GROUP
    </code>
   </a>
   SQL statement, the memory is taken from the transaction memory.
  </p>
  <p>
   A number of resources relating to metadata for Disk Data
        resources also have no reserved part, and use shared global
        memory only. Shared global shared memory is thus shared between
        send buffers, transaction memory,


        and Disk Data metadata.
  </p>
  <p>
   If
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-transactionmemory">
    <code class="literal">
     TransactionMemory
    </code>
   </a>
   is
        not set, it is calculated based on the following parameters:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentoperations">
       <code class="literal">
        MaxNoOfConcurrentOperations
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrenttransactions">
       <code class="literal">
        MaxNoOfConcurrentTransactions
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooffiredtriggers">
       <code class="literal">
        MaxNoOfFiredTriggers
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooflocaloperations">
       <code class="literal">
        MaxNoOfLocalOperations
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentindexoperations">
       <code class="literal">
        MaxNoOfConcurrentIndexOperations
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentscans">
       <code class="literal">
        MaxNoOfConcurrentScans
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooflocalscans">
       <code class="literal">
        MaxNoOfLocalScans
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-batchsizeperlocalscan">
       <code class="literal">
        BatchSizePerLocalScan
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-transactionbuffermemory">
       <code class="literal">
        TransactionBufferMemory
       </code>
      </a>
     </p>
    </li>
   </ul>
  </div>
  <p>
   When
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-transactionmemory">
    <code class="literal">
     TransactionMemory
    </code>
   </a>
   is set explicitly, none of the configuration parameters just
        listed are used to calculate memory size. In addition, the
        parameters
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentindexoperations">
    <code class="literal">
     MaxNoOfConcurrentIndexOperations
    </code>
   </a>
   ,
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooffiredtriggers">
    <code class="literal">
     MaxNoOfFiredTriggers
    </code>
   </a>
   ,
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooflocaloperations">
    <code class="literal">
     MaxNoOfLocalOperations
    </code>
   </a>
   ,
        and
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooflocalscans">
    <code class="literal">
     MaxNoOfLocalScans
    </code>
   </a>
   are incompatible with
   <code class="literal">
    TransactionMemory
   </code>
   and
        cannot be set concurrently with it; if
   <code class="literal">
    TransactionMemory
   </code>
   is set and any of these
        four parameters are also set in the
   <code class="filename">
    config.ini
   </code>
   configuration file, the
        management server cannot start.
  </p>
  <p>
   The
   <code class="literal">
    MaxNoOfConcurrentIndexOperations
   </code>
   ,
   <code class="literal">
    MaxNoOfFiredTriggers
   </code>
   ,
   <code class="literal">
    MaxNoOfLocalOperations
   </code>
   , and
   <code class="literal">
    MaxNoOfLocalScans
   </code>
   parameters are all
        deprecated; you should expect them to be removed from a future
        release of MySQL NDB Cluster.
  </p>
  <p>
   The transaction memory resource contains a large number of
        memory pools. Each memory pool represents an object type and
        contains a set of objects; each pool includes a reserved part
        allocated to the pool at startup; this reserved memory is never
        returned to shared global memory. Reserved records are found
        using a data structure having only a single level for fast
        retrieval, which means that a number of records in each pool
        should be reserved. The number of reserved records in each pool
        has some impact on performance and reserved memory allocation,
        but is generally necessary only in certain very advanced use
        cases to set the reserved sizes explicitly.
  </p>
  <p>
   The size of the reserved part of the pool can be controlled by
        setting the following configuration parameters:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-reservedconcurrentindexoperations">
       <code class="literal">
        ReservedConcurrentIndexOperations
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-reservedfiredtriggers">
       <code class="literal">
        ReservedFiredTriggers
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-reservedconcurrentoperations">
       <code class="literal">
        ReservedConcurrentOperations
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-reservedlocalscans">
       <code class="literal">
        ReservedLocalScans
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-reservedconcurrenttransactions">
       <code class="literal">
        ReservedConcurrentTransactions
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-reservedconcurrentscans">
       <code class="literal">
        ReservedConcurrentScans
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-reservedtransactionbuffermemory">
       <code class="literal">
        ReservedTransactionBufferMemory
       </code>
      </a>
     </p>
    </li>
   </ul>
  </div>
  <p>
   For any of the parameters just listed that is not set explicitly
        in
   <code class="filename">
    config.ini
   </code>
   , the reserved setting is
        calculated as 25% of the corresponding maximum setting. For
        example, if unset,
   <code class="literal">
    ReservedConcurrentIndexOperations
   </code>
   is
        calculated as 25% of
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentindexoperations">
    <code class="literal">
     MaxNoOfConcurrentIndexOperations
    </code>
   </a>
   ,
        and
   <code class="literal">
    ReservedLocalScans
   </code>
   is calculated as 25%
        of
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooflocalscans">
    <code class="literal">
     MaxNoOfLocalScans
    </code>
   </a>
   .
  </p>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    If
    <code class="literal">
     ReservedTransactionBufferMemory
    </code>
    is not
          set, it is calculated as 25% of
    <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-transactionbuffermemory">
     <code class="literal">
      TransactionBufferMemory
     </code>
    </a>
    .
   </p>
  </div>
  <p>
   The number of reserved records is per data node; these records
        are split among the threads handling them (LDM and TC threads)
        on each node. In most cases, it is sufficient to set
   <code class="literal">
    TransactionMemory
   </code>
   alone, and to allow the
        number of records in pools to be governed by its value.
  </p>
  <p>
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentscans">
    <code class="literal">
     MaxNoOfConcurrentScans
    </code>
   </a>
   limits the number of concurrent scans that can be active in each
        TC thread. This is important in guarding against cluster
        overload.
  </p>
  <p>
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentoperations">
    <code class="literal">
     MaxNoOfConcurrentOperations
    </code>
   </a>
   limits the number of operations that can be active at any one
        time in updating transactions. (Simple reads are not affected by
        this parameter.) This number needs to be limited because it is
        necessary to preallocate memory for node failure handling, and a
        resource must be available for handling the maximum number of
        active operations in one TC thread when contending with node
        failures. It is imperative that
   <code class="literal">
    MaxNoOfConcurrentOperations
   </code>
   be set to the
        same number on all nodes (this can be done most easily by
        setting a value for it once, in the
   <code class="literal">
    [ndbd
        default]
   </code>
   section of the
   <code class="filename">
    config.ini
   </code>
   global configuration file).
        While its value can be increased using a rolling restart (see
   <a class="xref" href="mysql-cluster-rolling-restart.html" title="25.6.5 Performing a Rolling Restart of an NDB Cluster">
    Section 25.6.5, “Performing a Rolling Restart of an NDB Cluster”
   </a>
   ), decreasing it
        in this way is not considered safe due to the possibility of a
        node failure occurring during the rolling restart.
  </p>
  <p>
   It is possible to limit the size of a single transaction in NDB
        Cluster through the
   <code class="filename">
    MaxDMLOperationsPerTransaction
   </code>
   parameter. If this is not set, the size of one transaction is
        limited by
   <code class="literal">
    MaxNoOfConcurrentOperations
   </code>
   since
        this parameter limits the total number of concurrent operations
        per TC thread.
  </p>
  <p>
   Schema memory size is controlled by the following set of
        configuration parameters:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofsubscriptions">
       <code class="literal">
        MaxNoOfSubscriptions
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofsubscribers">
       <code class="literal">
        MaxNoOfSubscribers
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofconcurrentsuboperations">
       <code class="literal">
        MaxNoOfConcurrentSubOperations
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofattributes">
       <code class="literal">
        MaxNoOfAttributes
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooftables">
       <code class="literal">
        MaxNoOfTables
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooforderedindexes">
       <code class="literal">
        MaxNoOfOrderedIndexes
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnoofuniquehashindexes">
       <code class="literal">
        MaxNoOfUniqueHashIndexes
       </code>
      </a>
     </p>
    </li>
    <li class="listitem">
     <p>
      <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxnooftriggers">
       <code class="literal">
        MaxNoOfTriggers
       </code>
      </a>
     </p>
    </li>
   </ul>
  </div>
  <p>
   The number of nodes and the number of LDM threads also have a
        major impact on the size of schema memory since the number of
        partitions in each table and each partition (and its fragment
        replicas) have to be represented in schema memory.
  </p>
  <p>
   In addition, a number of other records are allocated during
        startup. These are relatively small. Each block in each thread
        contains block objects that use memory. This memory size is also
        normally quite small compared to the other data node memory
        structures.
  </p>
 </div>
 <br/>
</div>
