<div id="docs-body">
 <div class="section">
  <div class="titlepage">
   <div>
    <div>
     <h3 class="title">
      <a name="mysql-cluster-replication-issues">
      </a>
      25.7.3 Known Issues in NDB Cluster Replication
     </h3>
    </div>
   </div>
  </div>
  <a class="indexterm" name="idm46045086732400">
  </a>
  <p>
   This section discusses known problems or issues when using
      replication with NDB Cluster.
  </p>
  <p>
   <b>
    Loss of connection between source and replica.
   </b>
   <a class="indexterm" name="idm46045086729472">
   </a>
   A loss of connection can occur either between the source cluster
        SQL node and the replica cluster SQL node, or between the source
        SQL node and the data nodes of the source cluster. In the latter
        case, this can occur not only as a result of loss of physical
        connection (for example, a broken network cable), but due to the
        overflow of data node event buffers; if the SQL node is too slow
        to respond, it may be dropped by the cluster (this is
        controllable to some degree by adjusting the
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-maxbufferedepochs">
    <code class="literal">
     MaxBufferedEpochs
    </code>
   </a>
   and
   <a class="link" href="mysql-cluster-ndbd-definition.html#ndbparam-ndbd-timebetweenepochs">
    <code class="literal">
     TimeBetweenEpochs
    </code>
   </a>
   configuration parameters). If this occurs,
   <span class="emphasis">
    <em>
     it is
        entirely possible for new data to be inserted into the source
        cluster without being recorded in the source SQL node's
        binary log
    </em>
   </span>
   . For this reason, to guarantee high
        availability, it is extremely important to maintain a backup
        replication channel, to monitor the primary channel, and to fail
        over to the secondary replication channel when necessary to keep
        the replica cluster synchronized with the source. NDB Cluster is
        not designed to perform such monitoring on its own; for this, an
        external application is required.
  </p>
  <p>
   <a class="indexterm" name="idm46045086723264">
   </a>
   <a class="indexterm" name="idm46045086721776">
   </a>
   The source SQL node issues a
   <span class="quote">
    “
    <span class="quote">
     gap
    </span>
    ”
   </span>
   event when
      connecting or reconnecting to the source cluster. (A gap event is
      a type of
   <span class="quote">
    “
    <span class="quote">
     incident event,
    </span>
    ”
   </span>
   which indicates an
      incident that occurs that affects the contents of the database but
      that cannot easily be represented as a set of changes. Examples of
      incidents are server failures, database resynchronization, some
      software updates, and some hardware changes.) When the replica
      encounters a gap in the replication log, it stops with an error
      message. This message is available in the output of
   <a class="link" href="show-replica-status.html" title="15.7.7.35 SHOW REPLICA STATUS Statement">
    <code class="literal">
     SHOW REPLICA STATUS
    </code>
   </a>
   , and indicates
      that the SQL thread has stopped due to an incident registered in
      the replication stream, and that manual intervention is required.
      See
   <a class="xref" href="mysql-cluster-replication-failover.html" title="25.7.8 Implementing Failover with NDB Cluster Replication">
    Section 25.7.8, “Implementing Failover with NDB Cluster Replication”
   </a>
   , for more
      information about what to do in such circumstances.
  </p>
  <div class="important" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Important
   </div>
   <p>
    Because NDB Cluster is not designed on its own to monitor
        replication status or provide failover, if high availability is
        a requirement for the replica server or cluster, then you must
        set up multiple replication lines, monitor the source
    <a class="link" href="mysqld.html" title="6.3.1 mysqld — The MySQL Server">
     <span class="command">
      <strong>
       mysqld
      </strong>
     </span>
    </a>
    on the primary replication line, and
        be prepared fail over to a secondary line if and as necessary.
        This must be done manually, or possibly by means of a
        third-party application. For information about implementing this
        type of setup, see
    <a class="xref" href="mysql-cluster-replication-two-channels.html" title="25.7.7 Using Two Replication Channels for NDB Cluster Replication">
     Section 25.7.7, “Using Two Replication Channels for NDB Cluster Replication”
    </a>
    , and
    <a class="xref" href="mysql-cluster-replication-failover.html" title="25.7.8 Implementing Failover with NDB Cluster Replication">
     Section 25.7.8, “Implementing Failover with NDB Cluster Replication”
    </a>
    .
   </p>
   <p>
    If you are replicating from a standalone MySQL server to an NDB
        Cluster, one channel is usually sufficient.
   </p>
  </div>
  <p>
   <b>
    Circular replication.
   </b>
   <a class="indexterm" name="idm46045086710848">
   </a>
   <a class="indexterm" name="idm46045086709360">
   </a>
   <a class="indexterm" name="idm46045086707872">
   </a>
   NDB Cluster Replication supports circular replication, as shown
        in the next example. The replication setup involves three NDB
        Clusters numbered 1, 2, and 3, in which Cluster 1 acts as the
        replication source for Cluster 2, Cluster 2 acts as the source
        for Cluster 3, and Cluster 3 acts as the source for Cluster 1,
        thus completing the circle. Each NDB Cluster has two SQL nodes,
        with SQL nodes A and B belonging to Cluster 1, SQL nodes C and D
        belonging to Cluster 2, and SQL nodes E and F belonging to
        Cluster 3.
  </p>
  <p>
   Circular replication using these clusters is supported as long as
      the following conditions are met:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      The SQL nodes on all source and replica clusters are the same.
     </p>
    </li>
    <li class="listitem">
     <p>
      All SQL nodes acting as sources and replicas are started with
          the system variable
      <a class="link" href="replication-options-binary-log.html#sysvar_log_replica_updates">
       <code class="literal">
        log_replica_updates
       </code>
      </a>
      enabled.
     </p>
    </li>
   </ul>
  </div>
  <p>
   This type of circular replication setup is shown in the following
      diagram:
  </p>
  <div class="figure">
   <a name="fig-mysql-cluster-circular-all-sources-replicas">
   </a>
   <p class="title">
    <b>
     Figure 25.11 NDB Cluster Circular Replication With All Sources As Replicas
    </b>
   </p>
   <div class="figure-contents">
    <div class="mediaobject">
     <img alt="Some content is described in the surrounding text. The diagram shows three clusters, each with two nodes. Arrows connecting SQL nodes in different clusters illustrate that all sources are also replicas." src="images/cluster-circular-replication-1.png" style="width: 100%; max-width: 557px;"/>
    </div>
   </div>
  </div>
  <br class="figure-break"/>
  <p>
   In this scenario, SQL node A in Cluster 1 replicates to SQL node C
      in Cluster 2; SQL node C replicates to SQL node E in Cluster 3;
      SQL node E replicates to SQL node A. In other words, the
      replication line (indicated by the curved arrows in the diagram)
      directly connects all SQL nodes used as sources and replicas.
  </p>
  <p>
   It should also be possible to set up circular replication in which
      not all source SQL nodes are also replicas, as shown here:
  </p>
  <div class="figure">
   <a name="fig-mysql-cluster-circular-not-all-sources-replicas">
   </a>
   <p class="title">
    <b>
     Figure 25.12 NDB Cluster Circular Replication Where Not All Sources Are Replicas
    </b>
   </p>
   <div class="figure-contents">
    <div class="mediaobject">
     <img alt="Some content is described in the surrounding text. The diagram shows three clusters, each with two nodes. Arrows connecting SQL nodes in different clusters illustrate that not all sources are replicas." src="images/cluster-circular-replication-2.png" style="width: 100%; max-width: 542px;"/>
    </div>
   </div>
  </div>
  <br class="figure-break"/>
  <p>
   In this case, different SQL nodes in each cluster are used as
      sources and replicas. However, you must
   <span class="emphasis">
    <em>
     not
    </em>
   </span>
   start any of the SQL nodes with the
   <a class="link" href="replication-options-binary-log.html#sysvar_log_replica_updates">
    <code class="literal">
     log_replica_updates
    </code>
   </a>
   system
      variable enabled. This type of circular replication scheme for NDB
      Cluster, in which the line of replication (again indicated by the
      curved arrows in the diagram) is discontinuous, should be
      possible, but it should be noted that it has not yet been
      thoroughly tested and must therefore still be considered
      experimental.
  </p>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    The
    <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
     <code class="literal">
      NDB
     </code>
    </a>
    storage engine uses
    <span class="firstterm">
     idempotent execution mode
    </span>
    ,
        which suppresses duplicate-key and other errors that otherwise
        break circular replication of NDB Cluster. This is equivalent to
        setting the global value of the system variable
    <a class="link" href="replication-options-replica.html#sysvar_replica_exec_mode">
     <code class="literal">
      replica_exec_mode
     </code>
    </a>
    to
    <code class="literal">
     IDEMPOTENT
    </code>
    , although this is not necessary in
        NDB Cluster replication, since NDB Cluster sets this variable
        automatically and ignores any attempts to set it explicitly.
   </p>
  </div>
  <p>
   <b>
    NDB Cluster replication and primary keys.
   </b>
   <a class="indexterm" name="idm46045086677184">
   </a>
   In the event of a node failure, errors in replication of
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   tables without primary keys can
        still occur, due to the possibility of duplicate rows being
        inserted in such cases. For this reason, it is highly
        recommended that all
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   tables
        being replicated have explicit primary keys.
  </p>
  <p>
   <b>
    NDB Cluster Replication and Unique Keys.
   </b>
   <a class="indexterm" name="idm46045086671824">
   </a>
   In older versions of NDB Cluster, operations that updated values
        of unique key columns of
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   tables
        could result in duplicate-key errors when replicated. This issue
        is solved for replication between
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   tables by deferring unique key
        checks until after all table row updates have been performed.
  </p>
  <p>
   Deferring constraints in this way is currently supported only by
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   . Thus, updates of unique keys
      when replicating from
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   to a
      different storage engine such as
   <a class="link" href="innodb-storage-engine.html" title="Chapter 17 The InnoDB Storage Engine">
    <code class="literal">
     InnoDB
    </code>
   </a>
   or
   <a class="link" href="myisam-storage-engine.html" title="18.2 The MyISAM Storage Engine">
    <code class="literal">
     MyISAM
    </code>
   </a>
   are still not supported.
  </p>
  <p>
   The problem encountered when replicating without deferred checking
      of unique key updates can be illustrated using
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   table such as
   <code class="literal">
    t
   </code>
   , is created and populated on the source (and
      transmitted to a replica that does not support deferred unique key
      updates) as shown here:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting language-sql"><div class="docs-select-all right" id="sa22455322"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-sql"><span class="token keyword">CREATE</span> <span class="token keyword">TABLE</span> t <span class="token punctuation">(</span>
    p <span class="token datatype">INT</span> <span class="token keyword">PRIMARY</span> <span class="token keyword">KEY</span><span class="token punctuation">,</span>
    c <span class="token datatype">INT</span><span class="token punctuation">,</span>
    <span class="token keyword">UNIQUE</span> <span class="token keyword">KEY</span> u <span class="token punctuation">(</span>c<span class="token punctuation">)</span>
<span class="token punctuation">)</span>   <span class="token keyword">ENGINE</span> <span class="token keyword">NDB</span><span class="token punctuation">;</span>

<span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> t
    <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">,</span><span class="token number">2</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token number">3</span><span class="token punctuation">,</span><span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token number">4</span><span class="token punctuation">,</span><span class="token number">4</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token number">5</span><span class="token punctuation">,</span><span class="token number">5</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre>
  </div>
  <p>
   The following
   <a class="link" href="update.html" title="15.2.17 UPDATE Statement">
    <code class="literal">
     UPDATE
    </code>
   </a>
   statement on
   <code class="literal">
    t
   </code>
   succeeds on the source, since the rows
      affected are processed in the order determined by the
   <code class="literal">
    ORDER BY
   </code>
   option, performed over the entire
      table:
  </p>
  <div class="copytoclipboard-wrapper">
   <pre class="programlisting one-line language-sql"><div class="docs-select-all right" id="sa24553388"><div class="copy-help left">Press ⌘+C to copy</div> <div class="right"><button class="clipboard-btn" title="Copy to Clipboard"><span class="icon-clipboard"></span></button></div></div><code class="language-sql"><span class="token keyword">UPDATE</span> t <span class="token keyword">SET</span> c <span class="token operator">=</span> c <span class="token operator">-</span> <span class="token number">1</span> <span class="token keyword">ORDER</span> <span class="token keyword">BY</span> p<span class="token punctuation">;</span></code></pre>
  </div>
  <p>
   The same statement fails with a duplicate key error or other
      constraint violation on the replica, because the ordering of the
      row updates is performed for one partition at a time, rather than
      for the table as a whole.
  </p>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    Every
    <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
     <code class="literal">
      NDB
     </code>
    </a>
    table is implicitly
        partitioned by key when it is created. See
    <a class="xref" href="partitioning-key.html" title="26.2.5 KEY Partitioning">
     Section 26.2.5, “KEY Partitioning”
    </a>
    , for more information.
   </p>
  </div>
  <p>
   <b>
    GTIDs not supported.
   </b>
   Replication using global transaction IDs is not compatible with
        the
   <code class="literal">
    NDB
   </code>
   storage engine, and is not supported.
        Enabling GTIDs is likely to cause NDB Cluster Replication to
        fail.
  </p>
  <p>
   <b>
    Restarting with --initial.
   </b>
   <a class="indexterm" name="idm46045086647744">
   </a>
   Restarting the cluster with the
   <a class="link" href="mysql-cluster-programs-ndbd.html#option_ndbd_initial">
    <code class="option">
     --initial
    </code>
   </a>
   option causes the
        sequence of GCI and epoch numbers to start over from
   <code class="literal">
    0
   </code>
   . (This is generally true of NDB Cluster and
        not limited to replication scenarios involving Cluster.) The
        MySQL servers involved in replication should in this case be
        restarted. After this, you should use the
   <a class="link" href="reset-binary-logs-and-gtids.html" title="15.4.1.2 RESET BINARY LOGS AND GTIDS Statement">
    <code class="literal">
     RESET BINARY LOGS AND GTIDS
    </code>
   </a>
   and
   <a class="link" href="reset-replica.html" title="15.4.2.3 RESET REPLICA Statement">
    <code class="literal">
     RESET REPLICA
    </code>
   </a>
   statements to clear
        the invalid
   <code class="literal">
    ndb_binlog_index
   </code>
   and
   <code class="literal">
    ndb_apply_status
   </code>
   tables, respectively.
  </p>
  <p>
   <a name="mysql-cluster-replication-ndb-to-non-ndb">
   </a>
   <b>
    Replication from NDB to other storage engines.
   </b>
   It is possible to replicate an
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   table on the source to a table using a different storage engine
        on the replica, taking into account the restrictions listed
        here:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      Multi-source and circular replication are not supported
          (tables on both the source and the replica must use the
      <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
       <code class="literal">
        NDB
       </code>
      </a>
      storage engine for this to
          work).
     </p>
    </li>
    <li class="listitem">
     <p>
      Using a storage engine which does not perform binary logging
          for tables on the replica requires special handling.
     </p>
    </li>
    <li class="listitem">
     <p>
      Use of a nontransactional storage engine for tables on the
          replica also requires special handling.
     </p>
    </li>
    <li class="listitem">
     <p>
      The source
      <a class="link" href="mysqld.html" title="6.3.1 mysqld — The MySQL Server">
       <span class="command">
        <strong>
         mysqld
        </strong>
       </span>
      </a>
      must be started with
      <a class="link" href="mysql-cluster-options-variables.html#option_mysqld_ndb-log-update-as-write">
       <code class="option">
        --ndb-log-update-as-write=0
       </code>
      </a>
      or
      <code class="option">
       --ndb-log-update-as-write=OFF
      </code>
      .
     </p>
    </li>
   </ul>
  </div>
  <p>
   The next few paragraphs provide additional information about each
      of the issues just described.
  </p>
  <p>
   <b>
    Multiple sources not supported when replicating NDB to other storage
        engines.
   </b>
   For replication from
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   to a
        different storage engine, the relationship between the two
        databases must be one-to-one. This means that bidirectional or
        circular replication is not supported between NDB Cluster and
        other storage engines.
  </p>
  <p>
   In addition, it is not possible to configure more than one
      replication channel when replicating between
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   and a different storage engine.
      (An NDB Cluster database
   <span class="emphasis">
    <em>
     can
    </em>
   </span>
   simultaneously
      replicate to multiple NDB Cluster databases.) If the source uses
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   tables, it is still possible to
      have more than one MySQL Server maintain a binary log of all
      changes, but for the replica to change sources (fail over), the
      new source-replica relationship must be explicitly defined on the
      replica.
  </p>
  <p>
   <b>
    Replicating NDB tables to a storage engine that does not perform binary
        logging.
   </b>
   <a class="indexterm" name="idm46045086621024">
   </a>
   If you attempt to replicate from an NDB Cluster to a replica
        that uses a storage engine that does not handle its own binary
        logging, the replication process aborts with the error
   <span class="errortext">
    Binary logging not possible ... Statement cannot be
        written atomically since more than one engine involved and at
        least one engine is self-logging
   </span>
   (Error
   <span class="errorcode">
    1595
   </span>
   ). It is possible to work around this
        issue in one of the following ways:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      <b>
       Turn off binary logging on the replica.
      </b>
      This can be accomplished by setting
      <a class="link" href="replication-options-binary-log.html#sysvar_sql_log_bin">
       <code class="literal">
        sql_log_bin = 0
       </code>
      </a>
      .
     </p>
    </li>
    <li class="listitem">
     <p>
      <b>
       Change the storage engine used for the mysql.ndb_apply_status table.
      </b>
      Causing this table to use an engine that does not handle its
            own binary logging can also eliminate the conflict. This can
            be done by issuing a statement such as
      <a class="link" href="alter-table.html" title="15.1.9 ALTER TABLE Statement">
       <code class="literal">
        ALTER TABLE
            mysql.ndb_apply_status ENGINE=MyISAM
       </code>
      </a>
      on the
            replica. It is safe to do this when using a storage engine
            other than
      <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
       <code class="literal">
        NDB
       </code>
      </a>
      on the replica,
            since you do not need to worry about keeping multiple
            replicas synchronized.
     </p>
    </li>
    <li class="listitem">
     <p>
      <b>
       Filter out changes to the mysql.ndb_apply_status table on the replica.
      </b>
      This can be done by starting the replica with
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-ignore-table">
       <code class="option">
        --replicate-ignore-table=mysql.ndb_apply_status
       </code>
      </a>
      .
            If you need for other tables to be ignored by replication,
            you might wish to use an appropriate
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-wild-ignore-table">
       <code class="option">
        --replicate-wild-ignore-table
       </code>
      </a>
      option instead.
     </p>
    </li>
   </ul>
  </div>
  <div class="important" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Important
   </div>
   <p>
    You should
    <span class="emphasis">
     <em>
      not
     </em>
    </span>
    disable replication or
        binary logging of
    <code class="literal">
     mysql.ndb_apply_status
    </code>
    or
        change the storage engine used for this table when replicating
        from one NDB Cluster to another. See
    <a class="xref" href="mysql-cluster-replication-issues.html#mysql-cluster-replication-issues-filtering" title="Replication and binary log filtering rules with replication between NDB Clusters">
     Replication and binary log filtering rules with replication between NDB
        Clusters
    </a>
    ,
        for details.
   </p>
  </div>
  <p>
   <a name="mysql-cluster-replication-ndb-to-nontransactional">
   </a>
   <b>
    Replication from NDB to a nontransactional storage engine.
   </b>
   When replicating from
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   to a
        nontransactional storage engine such as
   <a class="link" href="myisam-storage-engine.html" title="18.2 The MyISAM Storage Engine">
    <code class="literal">
     MyISAM
    </code>
   </a>
   , you may encounter
        unnecessary duplicate key errors when replicating
   <a class="link" href="insert-on-duplicate.html" title="15.2.7.2 INSERT ... ON DUPLICATE KEY UPDATE Statement">
    <code class="literal">
     INSERT ...
        ON DUPLICATE KEY UPDATE
    </code>
   </a>
   statements. You can suppress
        these by using
   <a class="link" href="mysql-cluster-options-variables.html#option_mysqld_ndb-log-update-as-write">
    <code class="option">
     --ndb-log-update-as-write=0
    </code>
   </a>
   ,
        which forces updates to be logged as writes, rather than as
        updates.
  </p>
  <p>
   <a name="mysql-cluster-replication-issues-tde">
   </a>
   <b>
    NDB Replication and File System Encryption (TDE).
   </b>
   The use of an encrypted filesystem does not have any effect on
        NDB Replication. All of the following scenarios are supported:
  </p>
  <div class="itemizedlist">
   <ul class="itemizedlist" style="list-style-type: disc; ">
    <li class="listitem">
     <p>
      Replication of an NDB Cluster having an encrypted file system
          to an NDB Cluster whose file system is not encrypted.
     </p>
    </li>
    <li class="listitem">
     <p>
      Replication of an NDB Cluster whose file system is not
          encrypted to an NDB Cluster whose file system is encrypted.
     </p>
    </li>
    <li class="listitem">
     <p>
      Replication of an NDB Cluster whose file system is encrypted
          to a standalone MySQL server using
      <a class="link" href="innodb-storage-engine.html" title="Chapter 17 The InnoDB Storage Engine">
       <code class="literal">
        InnoDB
       </code>
      </a>
      tables which are not
          encrypted.
     </p>
    </li>
    <li class="listitem">
     <p>
      Replication of an NDB Cluster with an unencrypted file system
          to a standalone MySQL server using
      <code class="literal">
       InnoDB
      </code>
      tables with file sytem encryption.
     </p>
    </li>
   </ul>
  </div>
  <p>
   <a name="mysql-cluster-replication-issues-filtering">
   </a>
   <b>
    Replication and binary log filtering rules with replication between NDB
        Clusters.
   </b>
   If you are using any of the options
   <code class="option">
    --replicate-do-*
   </code>
   ,
   <code class="option">
    --replicate-ignore-*
   </code>
   ,
   <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-do-db">
    <code class="option">
     --binlog-do-db
    </code>
   </a>
   , or
   <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-ignore-db">
    <code class="option">
     --binlog-ignore-db
    </code>
   </a>
   to filter
        databases or tables being replicated, you must take care not to
        block replication or binary logging of the
   <code class="literal">
    mysql.ndb_apply_status
   </code>
   , which is required for
        replication between NDB Clusters to operate properly. In
        particular, you must keep in mind the following:
  </p>
  <div class="orderedlist">
   <ol class="orderedlist" type="1">
    <li class="listitem">
     <p>
      Using
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-do-db">
       <code class="option">
        --replicate-do-db=
        <em class="replaceable">
         <code>
          db_name
         </code>
        </em>
       </code>
      </a>
      (and no other
      <code class="option">
       --replicate-do-*
      </code>
      or
      <code class="option">
       --replicate-ignore-*
      </code>
      options) means that
      <span class="emphasis">
       <em>
        only
       </em>
      </span>
      tables in database
      <em class="replaceable">
       <code>
        db_name
       </code>
      </em>
      are replicated. In this
          case, you should also use
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-do-db">
       <code class="option">
        --replicate-do-db=mysql
       </code>
      </a>
      ,
      <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-do-db">
       <code class="option">
        --binlog-do-db=mysql
       </code>
      </a>
      , or
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-do-table">
       <code class="option">
        --replicate-do-table=mysql.ndb_apply_status
       </code>
      </a>
      to ensure that
      <code class="literal">
       mysql.ndb_apply_status
      </code>
      is
          populated on replicas.
     </p>
     <p>
      Using
      <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-do-db">
       <code class="option">
        --binlog-do-db=
        <em class="replaceable">
         <code>
          db_name
         </code>
        </em>
       </code>
      </a>
      (and no other
      <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-do-db">
       <code class="option">
        --binlog-do-db
       </code>
      </a>
      options) means that changes
      <span class="emphasis">
       <em>
        only
       </em>
      </span>
      to
          tables in database
      <em class="replaceable">
       <code>
        db_name
       </code>
      </em>
      are
          written to the binary log. In this case, you should also use
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-do-db">
       <code class="option">
        --replicate-do-db=mysql
       </code>
      </a>
      ,
      <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-do-db">
       <code class="option">
        --binlog-do-db=mysql
       </code>
      </a>
      , or
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-do-table">
       <code class="option">
        --replicate-do-table=mysql.ndb_apply_status
       </code>
      </a>
      to ensure that
      <code class="literal">
       mysql.ndb_apply_status
      </code>
      is
          populated on replicas.
     </p>
    </li>
    <li class="listitem">
     <p>
      Using
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-ignore-db">
       <code class="option">
        --replicate-ignore-db=mysql
       </code>
      </a>
      means that no tables in the
      <code class="literal">
       mysql
      </code>
      database
          are replicated. In this case, you should also use
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-do-table">
       <code class="option">
        --replicate-do-table=mysql.ndb_apply_status
       </code>
      </a>
      to ensure that
      <code class="literal">
       mysql.ndb_apply_status
      </code>
      is
          replicated.
     </p>
     <p>
      Using
      <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-ignore-db">
       <code class="option">
        --binlog-ignore-db=mysql
       </code>
      </a>
      means that no changes to tables in the
      <code class="literal">
       mysql
      </code>
      database are written to the binary
          log. In this case, you should also use
      <a class="link" href="replication-options-replica.html#option_mysqld_replicate-do-table">
       <code class="option">
        --replicate-do-table=mysql.ndb_apply_status
       </code>
      </a>
      to ensure that
      <code class="literal">
       mysql.ndb_apply_status
      </code>
      is
          replicated.
     </p>
    </li>
   </ol>
  </div>
  <p>
   You should also remember that each replication rule requires the
      following:
  </p>
  <div class="orderedlist">
   <ol class="orderedlist" type="1">
    <li class="listitem">
     <p>
      Its own
      <code class="option">
       --replicate-do-*
      </code>
      or
      <code class="option">
       --replicate-ignore-*
      </code>
      option, and that
          multiple rules cannot be expressed in a single replication
          filtering option. For information about these rules, see
      <a class="xref" href="replication-options.html" title="19.1.6 Replication and Binary Logging Options and Variables">
       Section 19.1.6, “Replication and Binary Logging Options and Variables”
      </a>
      .
     </p>
    </li>
    <li class="listitem">
     <p>
      Its own
      <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-do-db">
       <code class="option">
        --binlog-do-db
       </code>
      </a>
      or
      <a class="link" href="replication-options-binary-log.html#option_mysqld_binlog-ignore-db">
       <code class="option">
        --binlog-ignore-db
       </code>
      </a>
      option, and
          that multiple rules cannot be expressed in a single binary log
          filtering option. For information about these rules, see
      <a class="xref" href="binary-log.html" title="7.4.4 The Binary Log">
       Section 7.4.4, “The Binary Log”
      </a>
      .
     </p>
    </li>
   </ol>
  </div>
  <p>
   If you are replicating an NDB Cluster to a replica that uses a
      storage engine other than
   <a class="link" href="mysql-cluster.html" title="Chapter 25 MySQL NDB Cluster 8.4">
    <code class="literal">
     NDB
    </code>
   </a>
   , the
      considerations just given previously may not apply, as discussed
      elsewhere in this section.
  </p>
  <p>
   <a name="mysql-cluster-replication-ipv6">
   </a>
   <b>
    NDB Cluster Replication and IPv6.
   </b>
   All types of NDB Cluster nodes support IPv6 in NDB
        8.4; this includes management nodes, data nodes,
        and API or SQL nodes.
  </p>
  <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;">
   <div class="admon-title">
    Note
   </div>
   <p>
    In NDB 8.4, you can disable IPv6 support in the
        Linux kernel if you do not intend to use IPv6 addressing for any
        NDB Cluster nodes.
   </p>
  </div>
  <p>
   <b>
    Attribute promotion and demotion.
   </b>
   NDB Cluster Replication includes support for attribute promotion
        and demotion. The implementation of the latter distinguishes
        between lossy and non-lossy type conversions, and their use on
        the replica can be controlled by setting the global value of the
        system variable
   <a class="link" href="replication-options-replica.html#sysvar_replica_type_conversions">
    <code class="literal">
     replica_type_conversions
    </code>
   </a>
   .
  </p>
  <p>
   For more information about attribute promotion and demotion in NDB
      Cluster, see
   <a class="xref" href="replication-features-differing-tables.html#replication-features-attribute-promotion" title="Row-based replication: attribute promotion and demotion">
    Row-based replication: attribute promotion and demotion
   </a>
   .
  </p>
  <p>
   <code class="literal">
    NDB
   </code>
   , unlike
   <a class="link" href="innodb-storage-engine.html" title="Chapter 17 The InnoDB Storage Engine">
    <code class="literal">
     InnoDB
    </code>
   </a>
   or
   <a class="link" href="myisam-storage-engine.html" title="18.2 The MyISAM Storage Engine">
    <code class="literal">
     MyISAM
    </code>
   </a>
   , does not write changes to
      virtual columns to the binary log; however, this has no
      detrimental effects on NDB Cluster Replication or replication
      between
   <code class="literal">
    NDB
   </code>
   and other storage engines. Changes
      to stored generated columns are logged.
  </p>
 </div>
 <br/>
</div>
