<div id="docs-body">
 <div class="section">
  <div class="titlepage">
   <div>
    <div>
     <h4 class="title">
      <a name="mysql-cluster-online-add-node-example">
      </a>
      25.6.7.3 Adding NDB Cluster Data Nodes Online: Detailed Example
     </h4>
    </div>
   </div>
  </div>
  <p>
   In this section we provide a detailed example illustrating how
        to add new NDB Cluster data nodes online, starting with an NDB
        Cluster having 2 data nodes in a single node group and
        concluding with a cluster having 4 data nodes in 2 node groups.
  </p>
  <p>
   <b>
    Starting configuration.
   </b>
   For purposes of illustration, we assume a minimal
          configuration, and that the cluster uses a
   <code class="filename">
    config.ini
   </code>
   file containing only the
          following information:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ini"><div class="docs-select-all right" id="sa13456092"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ini"><span class="token selector">[ndbd default]</span>
<span class="token constant">DataMemory</span> <span class="token attr-value"><span class="token punctuation">=</span> 100M</span>
<span class="token constant">IndexMemory</span> <span class="token attr-value"><span class="token punctuation">=</span> 100M</span>
<span class="token constant">NoOfReplicas</span> <span class="token attr-value"><span class="token punctuation">=</span> 2</span>
<span class="token constant">DataDir</span> <span class="token attr-value"><span class="token punctuation">=</span> /usr/local/mysql/var/mysql-cluster</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 1</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.1</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 2</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.2</span>

<span class="token selector">[mgm]</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.10</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 10</span>

<span class="token selector">[api]</span>
<span class="token constant">Id</span><span class="token attr-value"><span class="token punctuation">=</span>20</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.20</span>

<span class="token selector">[api]</span>
<span class="token constant">Id</span><span class="token attr-value"><span class="token punctuation">=</span>21</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.21</span></code></pre>
  </div>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    We have left a gap in the sequence between data node IDs and
          other nodes. This make it easier later to assign node IDs that
          are not already in use to data nodes which are newly added.
   </p>
  </div>
  <p>
   We also assume that you have already started the cluster using
        the appropriate command line or
   <code class="filename">
    my.cnf
   </code>
   options, and that running
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-show">
    <code class="literal">
     SHOW
    </code>
   </a>
   in the management
        client produces output similar to what is shown here:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa62819682"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token operator">--</span> NDB Cluster <span class="token operator">--</span> Management Client <span class="token operator">--</span>
<span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">SHOW</span>
Connected to Management Server at<span class="token operator">:</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10<span class="token operator">:</span>1186 <span class="token punctuation">(</span>using cleartext<span class="token punctuation">)</span>
Cluster Configuration
<span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">-</span>
<span class="token selector">[ndbd(NDB)]</span>     2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>1    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>1  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">,</span> <span class="token operator">*</span><span class="token punctuation">)</span>
id<span class="token operator">=</span>2    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>2  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">)</span>

<span class="token selector">[ndb_mgmd(MGM)]</span> 1 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>10   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>

<span class="token selector">[mysqld(API)]</span>   2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>20   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>20  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>
id<span class="token operator">=</span>21   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>21  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span></code></pre>
  </div>
  <p>
   Finally, we assume that the cluster contains a single
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDBCLUSTER
    </code>
   </a>
   table created as shown
        here:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-sql"><div class="docs-select-all right" id="sa19068166"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-sql"><span class="token keyword">USE</span> n<span class="token punctuation">;</span>

<span class="token keyword">CREATE</span> <span class="token keyword">TABLE</span> ips <span class="token punctuation">(</span>
    id <span class="token datatype">BIGINT</span> <span class="token operator">NOT</span> <span class="token boolean">NULL</span> <span class="token keyword">AUTO_INCREMENT</span> <span class="token keyword">PRIMARY</span> <span class="token keyword">KEY</span><span class="token punctuation">,</span>
    country_code <span class="token datatype">CHAR</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">)</span> <span class="token operator">NOT</span> <span class="token boolean">NULL</span><span class="token punctuation">,</span>
    <span class="token keyword">type</span> <span class="token datatype">CHAR</span><span class="token punctuation">(</span><span class="token number">4</span><span class="token punctuation">)</span> <span class="token operator">NOT</span> <span class="token boolean">NULL</span><span class="token punctuation">,</span>
    ip_address <span class="token datatype">VARCHAR</span><span class="token punctuation">(</span><span class="token number">15</span><span class="token punctuation">)</span> <span class="token operator">NOT</span> <span class="token boolean">NULL</span><span class="token punctuation">,</span>
    addresses <span class="token datatype">BIGINT</span> <span class="token keyword">UNSIGNED</span> <span class="token keyword">DEFAULT</span> <span class="token boolean">NULL</span><span class="token punctuation">,</span>
    <span class="token datatype">date</span> <span class="token datatype">BIGINT</span> <span class="token keyword">UNSIGNED</span> <span class="token keyword">DEFAULT</span> <span class="token boolean">NULL</span>
<span class="token punctuation">)</span>   <span class="token keyword">ENGINE</span> <span class="token keyword">NDBCLUSTER</span><span class="token punctuation">;</span></code></pre>
  </div>
  <p>
   The memory usage and related information shown later in this
        section was generated after inserting approximately 50000 rows
        into this table.
  </p>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    In this example, we show the single-threaded
    <a class="link" href="mysql-cluster-programs-ndbd.html" title="25.5.1 ndbd — The NDB Cluster Data Node Daemon">
     <span class="command">
      <strong>
       ndbd
      </strong>
     </span>
    </a>
    being used for the data node
          processes. You can also apply this example, if you are using
          the multithreaded
    <a class="link" href="mysql-cluster-programs-ndbmtd.html" title="25.5.3 ndbmtd — The NDB Cluster Data Node Daemon (Multi-Threaded)">
     <span class="command">
      <strong>
       ndbmtd
      </strong>
     </span>
    </a>
    by substituting
    <a class="link" href="mysql-cluster-programs-ndbmtd.html" title="25.5.3 ndbmtd — The NDB Cluster Data Node Daemon (Multi-Threaded)">
     <span class="command">
      <strong>
       ndbmtd
      </strong>
     </span>
    </a>
    for
    <a class="link" href="mysql-cluster-programs-ndbd.html" title="25.5.1 ndbd — The NDB Cluster Data Node Daemon">
     <span class="command">
      <strong>
       ndbd
      </strong>
     </span>
    </a>
    wherever
          it appears in the steps that follow.
   </p>
  </div>
  <p>
   <b>
    Step 1: Update configuration file.
   </b>
   Open the cluster global configuration file in a text editor
          and add
   <code class="literal">
    [ndbd]
   </code>
   sections corresponding to
          the 2 new data nodes. (We give these data nodes IDs 3 and 4,
          and assume that they are to be run on host machines at
          addresses 198.51.100.3 and 198.51.100.4, respectively.) After
          you have added the new sections, the contents of the
   <code class="filename">
    config.ini
   </code>
   file should look like what is
          shown here, where the additions to the file are shown in bold
          type:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ini"><div class="docs-select-all right" id="sa23915192"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ini"><span class="token selector">[ndbd default]</span>
<span class="token constant">DataMemory</span> <span class="token attr-value"><span class="token punctuation">=</span> 100M</span>
<span class="token constant">IndexMemory</span> <span class="token attr-value"><span class="token punctuation">=</span> 100M</span>
<span class="token constant">NoOfReplicas</span> <span class="token attr-value"><span class="token punctuation">=</span> 2</span>
<span class="token constant">DataDir</span> <span class="token attr-value"><span class="token punctuation">=</span> /usr/local/mysql/var/mysql-cluster</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 1</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.1</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 2</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.2</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 3</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.3</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 4</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.4</span>

<span class="token selector">[mgm]</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.10</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 10</span>

<span class="token selector">[api]</span>
<span class="token constant">Id</span><span class="token attr-value"><span class="token punctuation">=</span>20</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.20</span>

<span class="token selector">[api]</span>
<span class="token constant">Id</span><span class="token attr-value"><span class="token punctuation">=</span>21</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.21</span></code></pre>
  </div>
  <p>
   Once you have made the necessary changes, save the file.
  </p>
  <p>
   <b>
    Step 2: Restart the management server.
   </b>
   Restarting the cluster management server requires that you
          issue separate commands to stop the management server and then
          to start it again, as follows:
  </p>
  <div class="orderedlist">
   <ol class="orderedlist" type="1">
    <li class="listitem">
     <p>
      Stop the management server using the management client
      <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-stop">
       <code class="literal">
        STOP
       </code>
      </a>
      command, as shown
            here:
     </p>
     <div class="copytoclipboard-wrapper">
      <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa87708342"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> 10 <span class="token keyword">STOP</span>
<span class="token selector">Node 10</span> has shut down<span class="token punctuation">.</span>
Disconnecting to allow Management Server to shutdown

$<span class="token operator">&gt;</span></code></pre>
     </div>
    </li>
    <li class="listitem">
     <p>
      Because shutting down the management server causes the
            management client to terminate, you must start the
            management server from the system shell. For simplicity, we
            assume that
      <code class="filename">
       config.ini
      </code>
      is in the same
            directory as the management server binary, but in practice,
            you must supply the correct path to the configuration file.
            You must also supply the
      <a class="link" href="mysql-cluster-programs-ndb-mgmd.html#option_ndb_mgmd_reload">
       <code class="option">
        --reload
       </code>
      </a>
      or
      <a class="link" href="mysql-cluster-programs-ndb-mgmd.html#option_ndb_mgmd_initial">
       <code class="option">
        --initial
       </code>
      </a>
      option so that
            the management server reads the new configuration from the
            file rather than its configuration cache. If your
            shell's current directory is also the same as the
            directory where the management server binary is located,
            then you can invoke the management server as shown here:
     </p>
     <div class="copytoclipboard-wrapper">
      <pre class="programlisting language-terminal"><div class="docs-select-all right" id="sa23635339"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-terminal"><span class="token prompt">$&gt; </span><span class="token command">ndb_mgmd</span> <span class="token property">-f</span> config<span class="token punctuation">.</span>ini <span class="token property">--reload</span>
2008-12-08 17<span class="token punctuation">:</span>29<span class="token punctuation">:</span>23 <span class="token punctuation">[</span>MgmSrvr<span class="token punctuation">]</span> INFO     <span class="token property">--</span> NDB Cluster Management Server<span class="token punctuation">.</span> 8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0-ndb-8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0
2008-12-08 17<span class="token punctuation">:</span>29<span class="token punctuation">:</span>23 <span class="token punctuation">[</span>MgmSrvr<span class="token punctuation">]</span> INFO     <span class="token property">--</span> Reading cluster configuration from <span class="token atrule">'config.ini'</span></code></pre>
     </div>
    </li>
   </ol>
  </div>
  <p>
   If you check the output of
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-show">
    <code class="literal">
     SHOW
    </code>
   </a>
   in the management
        client after restarting the
   <a class="link" href="mysql-cluster-programs-ndb-mgm.html" title="25.5.5 ndb_mgm — The NDB Cluster Management Client">
    <span class="command">
     <strong>
      ndb_mgm
     </strong>
    </span>
   </a>
   process,
        you should now see something like this:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa57440547"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token operator">--</span> NDB Cluster <span class="token operator">--</span> Management Client <span class="token operator">--</span>
<span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">SHOW</span>
Connected to Management Server at<span class="token operator">:</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10<span class="token operator">:</span>1186 <span class="token punctuation">(</span>using cleartext<span class="token punctuation">)</span>
Cluster Configuration
<span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">-</span>
<span class="token selector">[ndbd(NDB)]</span>     2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>1    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>1  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">,</span> <span class="token operator">*</span><span class="token punctuation">)</span>
id<span class="token operator">=</span>2    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>2  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">)</span>
id<span class="token operator">=</span>3 <span class="token punctuation">(</span>not connected<span class="token punctuation">,</span> accepting connect from 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>3<span class="token punctuation">)</span>
id<span class="token operator">=</span>4 <span class="token punctuation">(</span>not connected<span class="token punctuation">,</span> accepting connect from 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>4<span class="token punctuation">)</span>

<span class="token selector">[ndb_mgmd(MGM)]</span> 1 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>10   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>

<span class="token selector">[mysqld(API)]</span>   2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>20   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>20  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>
id<span class="token operator">=</span>21   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>21  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span></code></pre>
  </div>
  <p>
   <b>
    Step 3: Perform a rolling restart of the existing data nodes.
   </b>
   This step can be accomplished entirely within the cluster
          management client using the
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-restart">
    <code class="literal">
     RESTART
    </code>
   </a>
   command, as shown
          here:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa40323017"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> 1 <span class="token keyword">RESTART</span>
<span class="token selector">Node 1:</span> Node shutdown initiated
<span class="token selector">Node 1:</span> Node shutdown completed<span class="token punctuation">,</span> restarting<span class="token punctuation">,</span> no start<span class="token punctuation">.</span>
<span class="token selector">Node 1</span> is being restarted

<span class="token prompt">ndb_mgm&gt;</span><span class="token selector"> Node 1:</span> Start initiated <span class="token punctuation">(</span>version 8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>
<span class="token selector">Node 1:</span> Started <span class="token punctuation">(</span>version 8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>

<span class="token prompt">ndb_mgm&gt;</span> 2 <span class="token keyword">RESTART</span>
<span class="token selector">Node 2:</span> Node shutdown initiated
<span class="token selector">Node 2:</span> Node shutdown completed<span class="token punctuation">,</span> restarting<span class="token punctuation">,</span> no start<span class="token punctuation">.</span>
<span class="token selector">Node 2</span> is being restarted

<span class="token prompt">ndb_mgm&gt;</span><span class="token selector"> Node 2:</span> Start initiated <span class="token punctuation">(</span>version 8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>

<span class="token prompt">ndb_mgm&gt;</span><span class="token selector"> Node 2:</span> Started <span class="token punctuation">(</span>version 8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span></code></pre>
  </div>
  <div class="important" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Important
   </div>
   <p>
    After issuing each
    <code class="literal">
     <em class="replaceable">
      <code>
       X
      </code>
     </em>
     RESTART
    </code>
    command, wait until the management client
          reports
    <code class="literal">
     Node
     <em class="replaceable">
      <code>
       X
      </code>
     </em>
     : Started
          (version ...)
    </code>
    <span class="emphasis">
     <em>
      before
     </em>
    </span>
    proceeding
          any further.
   </p>
  </div>
  <p>
   You can verify that all existing data nodes were restarted using
        the updated configuration by checking the
   <a class="link" href="mysql-cluster-ndbinfo-nodes.html" title="25.6.17.48 The ndbinfo nodes Table">
    <code class="literal">
     ndbinfo.nodes
    </code>
   </a>
   table in the
   <a class="link" href="mysql.html" title="6.5.1 mysql — The MySQL Command-Line Client">
    <span class="command">
     <strong>
      mysql
     </strong>
    </span>
   </a>
   client.
  </p>
  <p>
   <b>
    Step 4: Perform a rolling restart of all cluster API nodes.
   </b>
   Shut down and restart each MySQL server acting as an SQL node
          in the cluster using
   <a class="link" href="mysqladmin.html" title="6.5.2 mysqladmin — A MySQL Server Administration Program">
    <span class="command">
     <strong>
      mysqladmin shutdown
     </strong>
    </span>
   </a>
   followed by
   <a class="link" href="mysqld-safe.html" title="6.3.2 mysqld_safe — MySQL Server Startup Script">
    <span class="command">
     <strong>
      mysqld_safe
     </strong>
    </span>
   </a>
   (or another startup
          script). This should be similar to what is shown here, where
   <em class="replaceable">
    <code>
     password
    </code>
   </em>
   is the MySQL
   <code class="literal">
    root
   </code>
   password for a given MySQL server
          instance:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-terminal"><div class="docs-select-all right" id="sa41621347"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-terminal"><span class="token prompt">$&gt; </span><span class="token command">mysqladmin</span> <span class="token property">-uroot</span> <span class="token property">-p<em class="replaceable">password</em></span> shutdown
081208 20<span class="token punctuation">:</span>19<span class="token punctuation">:</span>56 mysqld_safe mysqld from pid file
/usr/local/mysql/var/tonfisk<span class="token punctuation">.</span>pid ended
<span class="token prompt">$&gt; </span><span class="token command">mysqld_safe</span> <span class="token property">--ndbcluster</span> <span class="token constant">--ndb-connectstring</span><span class="token attr-value"><span class="token punctuation">=</span>198.51.100.10</span> &amp;
081208 20<span class="token punctuation">:</span>20<span class="token punctuation">:</span>06 mysqld_safe Logging to <span class="token atrule">'/usr/local/mysql/var/tonfisk.err'</span><span class="token punctuation">.</span>
081208 20<span class="token punctuation">:</span>20<span class="token punctuation">:</span>06 mysqld_safe Starting mysqld daemon with databases
from /usr/local/mysql/var</code></pre>
  </div>
  <p>
   Of course, the exact input and output depend on how and where
        MySQL is installed on the system, as well as which options you
        choose to start it (and whether or not some or all of these
        options are specified in a
   <code class="filename">
    my.cnf
   </code>
   file).
  </p>
  <p>
   <b>
    Step 5: Perform an initial start of the new data nodes.
   </b>
   From a system shell on each of the hosts for the new data
          nodes, start the data nodes as shown here, using the
   <a class="link" href="mysql-cluster-programs-ndbd.html#option_ndbd_initial">
    <code class="option">
     --initial
    </code>
   </a>
   option:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting one-line language-terminal"><div class="docs-select-all right" id="sa49294466"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-terminal"><span class="token prompt">$&gt; </span><span class="token command">ndbd</span> <span class="token property">-c</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10 <span class="token property">--initial</span></code></pre>
  </div>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    Unlike the case with restarting the existing data nodes, you
          can start the new data nodes concurrently; you do not need to
          wait for one to finish starting before starting the other.
   </p>
  </div>
  <p>
   <span class="emphasis">
    <em>
     Wait until both of the new data nodes have started
        before proceeding with the next step
    </em>
   </span>
   . Once the new
        data nodes have started, you can see in the output of the
        management client
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-show">
    <code class="literal">
     SHOW
    </code>
   </a>
   command that they do not yet belong to any node group (as
        indicated with bold type here):
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa1058055"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">SHOW</span>
Connected to Management Server at<span class="token operator">:</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10<span class="token operator">:</span>1186 <span class="token punctuation">(</span>using cleartext<span class="token punctuation">)</span>
Cluster Configuration
<span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">-</span>
<span class="token selector">[ndbd(NDB)]</span>     2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>1    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>1  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">,</span> <span class="token operator">*</span><span class="token punctuation">)</span>
id<span class="token operator">=</span>2    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>2  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">)</span>
id<span class="token operator">=</span>3    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>3  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> no nodegroup<span class="token punctuation">)</span>
id<span class="token operator">=</span>4    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>4  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> no nodegroup<span class="token punctuation">)</span>

<span class="token selector">[ndb_mgmd(MGM)]</span> 1 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>10   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>

<span class="token selector">[mysqld(API)]</span>   2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>20   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>20  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>
id<span class="token operator">=</span>21   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>21  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span></code></pre>
  </div>
  <p>
   <b>
    Step 6: Create a new node group.
   </b>
   You can do this by issuing a
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-create-nodegroup">
    <code class="literal">
     CREATE
          NODEGROUP
    </code>
   </a>
   command in the cluster management client.
          This command takes as its argument a comma-separated list of
          the node IDs of the data nodes to be included in the new node
          group, as shown here:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa11950999"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">CREATE NODEGROUP</span> 3<span class="token punctuation">,</span>4
Nodegroup 1 created</code></pre>
  </div>
  <p>
   By issuing
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-show">
    <code class="literal">
     SHOW
    </code>
   </a>
   again, you
        can verify that data nodes 3 and 4 have joined the new node
        group (again indicated in bold type):
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa95877635"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">SHOW</span>
Connected to Management Server at<span class="token operator">:</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10<span class="token operator">:</span>1186 <span class="token punctuation">(</span>using cleartext<span class="token punctuation">)</span>
Cluster Configuration
<span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">-</span>
<span class="token selector">[ndbd(NDB)]</span>     2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>1    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>1  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">,</span> <span class="token operator">*</span><span class="token punctuation">)</span>
id<span class="token operator">=</span>2    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>2  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 0<span class="token punctuation">)</span>
id<span class="token operator">=</span>3    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>3  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 1<span class="token punctuation">)</span>
id<span class="token operator">=</span>4    @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>4  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">,</span> Nodegroup<span class="token operator">:</span> 1<span class="token punctuation">)</span>

<span class="token selector">[ndb_mgmd(MGM)]</span> 1 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>10   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>

<span class="token selector">[mysqld(API)]</span>   2 node<span class="token punctuation">(</span>s<span class="token punctuation">)</span>
id<span class="token operator">=</span>20   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>20  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span>
id<span class="token operator">=</span>21   @198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>21  <span class="token punctuation">(</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token operator">-</span>ndb<span class="token operator">-</span>8<span class="token punctuation">.</span>4<span class="token punctuation">.</span>0<span class="token punctuation">)</span></code></pre>
  </div>
  <p>
   <b>
    Step 7: Redistribute cluster data.
   </b>
   When a node group is created, existing data and indexes are
          not automatically distributed to the new node group's
          data nodes, as you can see by issuing the appropriate
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-report">
    <code class="literal">
     REPORT
    </code>
   </a>
   command in the
          management client:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa18829082"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">ALL</span> <span class="token keyword">REPORT</span> <span class="token keyword">MEMORY</span>

<span class="token selector">Node 1:</span> Data usage is 5<span class="token operator">%</span><span class="token punctuation">(</span>177 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 1:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>108 8K pages of total 12832<span class="token punctuation">)</span>
<span class="token selector">Node 2:</span> Data usage is 5<span class="token operator">%</span><span class="token punctuation">(</span>177 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 2:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>108 8K pages of total 12832<span class="token punctuation">)</span>
<span class="token selector">Node 3:</span> Data usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>0 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 3:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>0 8K pages of total 12832<span class="token punctuation">)</span>
<span class="token selector">Node 4:</span> Data usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>0 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 4:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>0 8K pages of total 12832<span class="token punctuation">)</span></code></pre>
  </div>
  <p>
   By using
   <a class="link" href="mysql-cluster-programs-ndb-desc.html" title="25.5.9 ndb_desc — Describe NDB Tables">
    <span class="command">
     <strong>
      ndb_desc
     </strong>
    </span>
   </a>
   with the
   <code class="option">
    -p
   </code>
   option, which causes the output to include
        partitioning information, you can see that the table still uses
        only 2 partitions (in the
   <code class="literal">
    Per partition info
   </code>
   section of the output, shown here in bold text):
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-terminal"><div class="docs-select-all right" id="sa52094105"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-terminal"><span class="token prompt">$&gt; </span><span class="token command">ndb_desc</span> <span class="token property">-c</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10 <span class="token property">-d</span> n ips <span class="token property">-p</span>
<span class="token property">--</span> ips <span class="token property">--</span>
Version<span class="token punctuation">:</span> 1
Fragment type<span class="token punctuation">:</span> 9
K Value<span class="token punctuation">:</span> 6
Min load factor<span class="token punctuation">:</span> 78
Max load factor<span class="token punctuation">:</span> 80
Temporary table<span class="token punctuation">:</span> no
Number of attributes<span class="token punctuation">:</span> 6
Number of primary keys<span class="token punctuation">:</span> 1
Length of frm data<span class="token punctuation">:</span> 340
Row Checksum<span class="token punctuation">:</span> 1
Row GCI<span class="token punctuation">:</span> 1
SingleUserMode<span class="token punctuation">:</span> 0
ForceVarPart<span class="token punctuation">:</span> 1
FragmentCount<span class="token punctuation">:</span> 2
TableStatus<span class="token punctuation">:</span> Retrieved
<span class="token property">--</span> Attributes <span class="token property">--</span>
id Bigint PRIMARY KEY DISTRIBUTION KEY AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span> AUTO_INCR
country_code Char<span class="token punctuation">(</span>2<span class="token punctuation">;</span>latin1_swedish_ci<span class="token punctuation">)</span> NOT NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
type Char<span class="token punctuation">(</span>4<span class="token punctuation">;</span>latin1_swedish_ci<span class="token punctuation">)</span> NOT NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
ip_address Varchar<span class="token punctuation">(</span>15<span class="token punctuation">;</span>latin1_swedish_ci<span class="token punctuation">)</span> NOT NULL AT<span class="token attr-value"><span class="token punctuation">=</span>SHORT_VAR</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
addresses Bigunsigned NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
date Bigunsigned NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>

<span class="token property">--</span> Indexes <span class="token property">--</span>
PRIMARY KEY<span class="token punctuation">(</span>id<span class="token punctuation">)</span> - UniqueHashIndex
PRIMARY<span class="token punctuation">(</span>id<span class="token punctuation">)</span> - OrderedIndex

<span class="token property">--</span> Per partition info <span class="token property">--</span>
Partition   Row count   Commit count  Frag fixed memory   Frag varsized memory
0           26086       26086         1572864             557056
1           26329       26329         1605632             557056</code></pre>
  </div>
  <p>
   You can cause the data to be redistributed among all of the data
        nodes by performing, for each
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   table, an
   <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
    <code class="literal">
     ALTER
        TABLE ... ALGORITHM=INPLACE, REORGANIZE PARTITION
    </code>
   </a>
   statement in the
   <a class="link" href="mysql.html" title="6.5.1 mysql — The MySQL Command-Line Client">
    <span class="command">
     <strong>
      mysql
     </strong>
    </span>
   </a>
   client.
  </p>
  <div class="important" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Important
   </div>
   <p>
    <code class="literal">
     ALTER TABLE ... ALGORITHM=INPLACE, REORGANIZE
          PARTITION
    </code>
    does not work on tables that were created
          with the
    <code class="literal">
     MAX_ROWS
    </code>
    option. Instead, use
    <code class="literal">
     ALTER TABLE ... ALGORITHM=INPLACE,
          MAX_ROWS=...
    </code>
    to reorganize such tables.
   </p>
   <p>
    Keep in mind that using
    <code class="literal">
     MAX_ROWS
    </code>
    to set the
          number of partitions per table is deprecated, and you should
          use
    <code class="literal">
     PARTITION_BALANCE
    </code>
    instead; see
    <a class="xref" href="create-table-ndb-comment-options.html" title="15.1.20.12 Setting NDB Comment Options">
     Section 15.1.20.12, “Setting NDB Comment Options”
    </a>
    , for more
          information.
   </p>
  </div>
  <p>
   After issuing the statement
   <code class="literal">
    ALTER TABLE ips
        ALGORITHM=INPLACE, REORGANIZE PARTITION
   </code>
   , you can see
        using
   <a class="link" href="mysql-cluster-programs-ndb-desc.html" title="25.5.9 ndb_desc — Describe NDB Tables">
    <span class="command">
     <strong>
      ndb_desc
     </strong>
    </span>
   </a>
   that the data for this table
        is now stored using 4 partitions, as shown here (with the
        relevant portions of the output in bold type):
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-terminal"><div class="docs-select-all right" id="sa59578409"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-terminal"><span class="token prompt">$&gt; </span><span class="token command">ndb_desc</span> <span class="token property">-c</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10 <span class="token property">-d</span> n ips <span class="token property">-p</span>
<span class="token property">--</span> ips <span class="token property">--</span>
Version<span class="token punctuation">:</span> 16777217
Fragment type<span class="token punctuation">:</span> 9
K Value<span class="token punctuation">:</span> 6
Min load factor<span class="token punctuation">:</span> 78
Max load factor<span class="token punctuation">:</span> 80
Temporary table<span class="token punctuation">:</span> no
Number of attributes<span class="token punctuation">:</span> 6
Number of primary keys<span class="token punctuation">:</span> 1
Length of frm data<span class="token punctuation">:</span> 341
Row Checksum<span class="token punctuation">:</span> 1
Row GCI<span class="token punctuation">:</span> 1
SingleUserMode<span class="token punctuation">:</span> 0
ForceVarPart<span class="token punctuation">:</span> 1
FragmentCount<span class="token punctuation">:</span> 4
TableStatus<span class="token punctuation">:</span> Retrieved
<span class="token property">--</span> Attributes <span class="token property">--</span>
id Bigint PRIMARY KEY DISTRIBUTION KEY AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span> AUTO_INCR
country_code Char<span class="token punctuation">(</span>2<span class="token punctuation">;</span>latin1_swedish_ci<span class="token punctuation">)</span> NOT NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
type Char<span class="token punctuation">(</span>4<span class="token punctuation">;</span>latin1_swedish_ci<span class="token punctuation">)</span> NOT NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
ip_address Varchar<span class="token punctuation">(</span>15<span class="token punctuation">;</span>latin1_swedish_ci<span class="token punctuation">)</span> NOT NULL AT<span class="token attr-value"><span class="token punctuation">=</span>SHORT_VAR</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
addresses Bigunsigned NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>
date Bigunsigned NULL AT<span class="token attr-value"><span class="token punctuation">=</span>FIXED</span> ST<span class="token attr-value"><span class="token punctuation">=</span>MEMORY</span>

<span class="token property">--</span> Indexes <span class="token property">--</span>
PRIMARY KEY<span class="token punctuation">(</span>id<span class="token punctuation">)</span> - UniqueHashIndex
PRIMARY<span class="token punctuation">(</span>id<span class="token punctuation">)</span> - OrderedIndex

<span class="token property">--</span> Per partition info <span class="token property">--</span>
Partition   Row count   Commit count  Frag fixed memory   Frag varsized memory
0           12981       52296         1572864             557056
1           13236       52515         1605632             557056
2           13105       13105         819200              294912
3           13093       13093         819200              294912</code></pre>
  </div>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    Normally,
    <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
     <code class="literal">
      ALTER
          TABLE
      <em class="replaceable">
       <code>
        table_name
       </code>
      </em>
      [ALGORITHM=INPLACE,] REORGANIZE PARTITION
     </code>
    </a>
    is used
          with a list of partition identifiers and a set of partition
          definitions to create a new partitioning scheme for a table
          that has already been explicitly partitioned. Its use here to
          redistribute data onto a new NDB Cluster node group is an
          exception in this regard; when used in this way, no other
          keywords or identifiers follow
    <code class="literal">
     REORGANIZE
          PARTITION
    </code>
    .
   </p>
   <p>
    For more information, see
    <a class="xref" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
     Section 15.1.9, “ALTER TABLE Statement”
    </a>
    .
   </p>
  </div>
  <p>
   In addition, for each table, the
   <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
    <code class="literal">
     ALTER
        TABLE
    </code>
   </a>
   statement should be followed by an
   <a class="link" href="optimize-table.html" title="15.7.3.4 OPTIMIZE TABLE Statement">
    <code class="literal">
     OPTIMIZE TABLE
    </code>
   </a>
   to reclaim wasted
        space. You can obtain a list of all
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDBCLUSTER
    </code>
   </a>
   tables using the
        following query against the Information Schema
   <a class="link" href="information-schema-tables-table.html" title="28.3.38 The INFORMATION_SCHEMA TABLES Table">
    <code class="literal">
     TABLES
    </code>
   </a>
   table:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-sql"><div class="docs-select-all right" id="sa1152625"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-sql"><span class="token keyword">SELECT</span> TABLE_SCHEMA<span class="token punctuation">,</span> <span class="token keyword">TABLE_NAME</span>
    <span class="token keyword">FROM</span> INFORMATION_SCHEMA<span class="token punctuation">.</span><span class="token keyword">TABLES</span>
    <span class="token keyword">WHERE</span> <span class="token keyword">ENGINE</span> <span class="token operator">=</span> <span class="token string">'NDBCLUSTER'</span><span class="token punctuation">;</span></code></pre>
  </div>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    The
    <code class="literal">
     INFORMATION_SCHEMA.TABLES.ENGINE
    </code>
    value
          for an NDB Cluster table is always
    <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
     <code class="literal">
      NDBCLUSTER
     </code>
    </a>
    , regardless of whether
          the
    <code class="literal">
     CREATE TABLE
    </code>
    statement used to create
          the table (or
    <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
     <code class="literal">
      ALTER TABLE
     </code>
    </a>
    statement used to convert an existing table from a different
          storage engine) used
    <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
     <code class="literal">
      NDB
     </code>
    </a>
    or
    <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
     <code class="literal">
      NDBCLUSTER
     </code>
    </a>
    in its
    <code class="literal">
     ENGINE
    </code>
    option.
   </p>
  </div>
  <p>
   You can see after performing these statements in the output of
   <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-report">
    <code class="literal">
     ALL REPORT MEMORY
    </code>
   </a>
   that the
        data and indexes are now redistributed between all cluster data
        nodes, as shown here:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ndbmgm"><div class="docs-select-all right" id="sa24158856"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">ALL</span> <span class="token keyword">REPORT</span> <span class="token keyword">MEMORY</span>

<span class="token selector">Node 1:</span> Data usage is 5<span class="token operator">%</span><span class="token punctuation">(</span>176 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 1:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>76 8K pages of total 12832<span class="token punctuation">)</span>
<span class="token selector">Node 2:</span> Data usage is 5<span class="token operator">%</span><span class="token punctuation">(</span>176 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 2:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>76 8K pages of total 12832<span class="token punctuation">)</span>
<span class="token selector">Node 3:</span> Data usage is 2<span class="token operator">%</span><span class="token punctuation">(</span>80 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 3:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>51 8K pages of total 12832<span class="token punctuation">)</span>
<span class="token selector">Node 4:</span> Data usage is 2<span class="token operator">%</span><span class="token punctuation">(</span>80 32K pages of total 3200<span class="token punctuation">)</span>
<span class="token selector">Node 4:</span> Index usage is 0<span class="token operator">%</span><span class="token punctuation">(</span>50 8K pages of total 12832<span class="token punctuation">)</span></code></pre>
  </div>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    Since only one DDL operation on
    <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
     <code class="literal">
      NDBCLUSTER
     </code>
    </a>
    tables can be executed
          at a time, you must wait for each
    <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
     <code class="literal">
      ALTER TABLE ...
          REORGANIZE PARTITION
     </code>
    </a>
    statement to finish before
          issuing the next one.
   </p>
  </div>
  <p>
   It is not necessary to issue
   <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
    <code class="literal">
     ALTER TABLE ...
        REORGANIZE PARTITION
    </code>
   </a>
   statements for
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDBCLUSTER
    </code>
   </a>
   tables created
   <span class="emphasis">
    <em>
     after
    </em>
   </span>
   the new data nodes have been added;
        data added to such tables is distributed among all data nodes
        automatically. However, in
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDBCLUSTER
    </code>
   </a>
   tables that existed
   <span class="emphasis">
    <em>
     prior to
    </em>
   </span>
   the addition of the new nodes,
        neither existing nor new data is distributed using the new nodes
        until these tables have been reorganized using
   <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
    <code class="literal">
     ALTER TABLE ...
        REORGANIZE PARTITION
    </code>
   </a>
   .
  </p>
  <p>
   <b>
    Alternative procedure, without rolling restart.
   </b>
   It is possible to avoid the need for a rolling restart by
          configuring the extra data nodes, but not starting them, when
          first starting the cluster. We assume, as before, that you
          wish to start with two data nodes—nodes 1 and 2—in
          one node group and later to expand the cluster to four data
          nodes, by adding a second node group consisting of nodes 3 and
          4:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-ini"><div class="docs-select-all right" id="sa82398557"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ini"><span class="token selector">[ndbd default]</span>
<span class="token constant">DataMemory</span> <span class="token attr-value"><span class="token punctuation">=</span> 100M</span>
<span class="token constant">IndexMemory</span> <span class="token attr-value"><span class="token punctuation">=</span> 100M</span>
<span class="token constant">NoOfReplicas</span> <span class="token attr-value"><span class="token punctuation">=</span> 2</span>
<span class="token constant">DataDir</span> <span class="token attr-value"><span class="token punctuation">=</span> /usr/local/mysql/var/mysql-cluster</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 1</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.1</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 2</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.2</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 3</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.3</span>
<span class="token constant">Nodegroup</span> <span class="token attr-value"><span class="token punctuation">=</span> 65536</span>

<span class="token selector">[ndbd]</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 4</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.4</span>
<span class="token constant">Nodegroup</span> <span class="token attr-value"><span class="token punctuation">=</span> 65536</span>

<span class="token selector">[mgm]</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.10</span>
<span class="token constant">Id</span> <span class="token attr-value"><span class="token punctuation">=</span> 10</span>

<span class="token selector">[api]</span>
<span class="token constant">Id</span><span class="token attr-value"><span class="token punctuation">=</span>20</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.20</span>

<span class="token selector">[api]</span>
<span class="token constant">Id</span><span class="token attr-value"><span class="token punctuation">=</span>21</span>
<span class="token constant">HostName</span> <span class="token attr-value"><span class="token punctuation">=</span> 198.51.100.21</span></code></pre>
  </div>
  <p>
   The data nodes to be brought online at a later time (nodes 3 and
        4) can be configured with
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-nodegroup">
    <code class="literal">
     NodeGroup = 65536
    </code>
   </a>
   , in
        which case nodes 1 and 2 can each be started as shown here:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting one-line language-terminal"><div class="docs-select-all right" id="sa8213902"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-terminal"><span class="token prompt">$&gt; </span><span class="token command">ndbd</span> <span class="token property">-c</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10 <span class="token property">--initial</span></code></pre>
  </div>
  <p>
   The data nodes configured with
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-nodegroup">
    <code class="literal">
     NodeGroup = 65536
    </code>
   </a>
   are
        treated by the management server as though you had started nodes
        1 and 2 using
   <a class="link" href="mysql-cluster-programs-ndbd.html#option_ndbd_nowait-nodes">
    <code class="option">
     --nowait-nodes=3,4
    </code>
   </a>
   after waiting for a period of time determined by the setting for
        the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-startnonodegrouptimeout">
    <code class="literal">
     StartNoNodeGroupTimeout
    </code>
   </a>
   data node configuration parameter. By default, this is 15
        seconds (15000 milliseconds).
  </p>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-startnonodegrouptimeout">
     <code class="literal">
      StartNoNodegroupTimeout
     </code>
    </a>
    must be the same for all data nodes in the cluster; for this
          reason, you should always set it in the
    <code class="literal">
     [ndbd
          default]
    </code>
    section of the
    <code class="filename">
     config.ini
    </code>
    file, rather than for
          individual data nodes.
   </p>
  </div>
  <p>
   When you are ready to add the second node group, you need only
        perform the following additional steps:
  </p>
  <div class="orderedlist">
   <ol class="orderedlist" type="1">
    <li class="listitem">
     <p>
      Start data nodes 3 and 4, invoking the data node process
            once for each new node:
     </p>
     <div class="copytoclipboard-wrapper">
      <pre class="programlisting one-line language-terminal"><div class="docs-select-all right" id="sa236962"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-terminal"><span class="token prompt">$&gt; </span><span class="token command">ndbd</span> <span class="token property">-c</span> 198<span class="token punctuation">.</span>51<span class="token punctuation">.</span>100<span class="token punctuation">.</span>10 <span class="token property">--initial</span></code></pre>
     </div>
    </li>
    <li class="listitem">
     <p>
      Issue the appropriate
      <a class="link" href="mysql-cluster-mgm-client-commands.html#ndbclient-create-nodegroup">
       <code class="literal">
        CREATE
            NODEGROUP
       </code>
      </a>
      command in the management client:
     </p>
     <div class="copytoclipboard-wrapper">
      <pre class="programlisting one-line language-ndbmgm"><div class="docs-select-all right" id="sa86732323"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-ndbmgm"><span class="token prompt">ndb_mgm&gt;</span> <span class="token keyword">CREATE NODEGROUP</span> 3<span class="token punctuation">,</span>4</code></pre>
     </div>
    </li>
    <li class="listitem">
     <p>
      In the
      <a class="link" href="mysql.html" title="6.5.1 mysql — The MySQL Command-Line Client">
       <span class="command">
        <strong>
         mysql
        </strong>
       </span>
      </a>
      client, issue
      <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
       <code class="literal">
        ALTER TABLE ...
            REORGANIZE PARTITION
       </code>
      </a>
      and
      <a class="link" href="optimize-table.html" title="15.7.3.4 OPTIMIZE TABLE Statement">
       <code class="literal">
        OPTIMIZE TABLE
       </code>
      </a>
      statements for
            each existing
      <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
       <code class="literal">
        NDBCLUSTER
       </code>
      </a>
      table.
            (As noted elsewhere in this section, existing NDB Cluster
            tables cannot use the new nodes for data distribution until
            this has been done.)
     </p>
    </li>
   </ol>
  </div>
 </div>
 <br/>
</div>
