17.5.7.1. MySQL Cluster: Messages in the Cluster Log

Log Message. Node mgm_node_id: Node data_node_id Connected

Description. The data node having node ID node_id has connected to the management server (node mgm_node_id).

Event Name. Connected

Event Type. Connection

Priority. 8

Severity. INFO

Log Message. Node mgm_node_id: Node data_node_id Disconnected

Description. The data node having node ID data_node_id has disconnected from the management server (node mgm_node_id).

Event Name. Disconnected

Event Type. Connection

Priority. 8

Severity. ALERT

Log Message. Node data_node_id: Communication to Node api_node_id closed

Description. The API node or SQL node having node ID api_node_id is no longer communicating with data node data_node_id.

Event Name. CommunicationClosed

Event Type. Connection

Priority. 8

Severity. INFO

Log Message. Node data_node_id: Communication to Node api_node_id opened

Description. The API node or SQL node having node ID api_node_id is now communicating with data node data_node_id.

Event Name. CommunicationOpened

Event Type. Connection

Priority. 8

Severity. INFO

Log Message. Node mgm_node_id: Node api_node_id: API version

Description. The API node having node ID api_node_id has connected to management node mgm_node_id using NDB API version version (generally the same as the MySQL version number).

Event Name. ConnectedApiVersion

Event Type. Connection

Priority. 8

Severity. INFO

Log Message. Node node_id: Global checkpoint gci started

Description. A global checkpoint with the ID gci has been started; node node_id is the master responsible for this global checkpoint.

Event Name. GlobalCheckpointStarted

Event Type. Checkpoint

Priority. 9

Severity. INFO

Log Message. Node node_id: Global checkpoint gci completed

Description. The global checkpoint having the ID gci has been completed; node node_id was the master responsible for this global checkpoint.

Event Name. GlobalCheckpointCompleted

Event Type. Checkpoint

Priority. 10

Severity. INFO

Log Message. Node node_id: Local checkpoint lcp started. Keep GCI = current_gci oldest restorable GCI = old_gci

Description. The local checkpoint having sequence ID lcp has been started on node node_id. The most recent GCI that can be used has the index current_gci, and the oldest GCI from which the cluster can be restored has the index old_gci.

Event Name. LocalCheckpointStarted

Event Type. Checkpoint

Priority. 7

Severity. INFO

Log Message. Node node_id: Local checkpoint lcp completed

Description. The local checkpoint having sequence ID lcp on node node_id has been completed.

Event Name. LocalCheckpointCompleted

Event Type. Checkpoint

Priority. 8

Severity. INFO

Log Message. Node node_id: Local Checkpoint stopped in CALCULATED_KEEP_GCI

Description. The node was unable to determine the most recent usable GCI.

Event Name. LCPStoppedInCalcKeepGci

Event Type. Checkpoint

Priority. 0

Severity. ALERT

Log Message. Node node_id: Table ID = table_id, fragment ID = fragment_id has completed LCP on Node node_id maxGciStarted: started_gci maxGciCompleted: completed_gci

Description. A table fragment has been checkpointed to disk on node node_id. The GCI in progress has the index started_gci, and the most recent GCI to have been completed has the index completed_gci.

Event Name. LCPFragmentCompleted

Event Type. Checkpoint

Priority. 11

Severity. INFO

Log Message. Node node_id: ACC Blocked num_1 and TUP Blocked num_2 times last second

Description. Undo logging is blocked because the log buffer is close to overflowing.

Event Name. UndoLogBlocked

Event Type. Checkpoint

Priority. 7

Severity. INFO

Log Message. Node node_id: Start initiated version

Description. Data node node_id, running NDB version version, is beginning its startup process.

Event Name. NDBStartStarted

Event Type. StartUp

Priority. 1

Severity. INFO

Log Message. Node node_id: Started version

Description. Data node node_id, running NDB version version, has started successfully.

Event Name. NDBStartCompleted

Event Type. StartUp

Priority. 1

Severity. INFO

Log Message. Node node_id: STTORRY received after restart finished

Description. The node has received a signal indicating that a cluster restart has completed.

Event Name. STTORRYRecieved

Event Type. StartUp

Priority. 15

Severity. INFO

Log Message. Node node_id: Start phase phase completed (type)

Description. The node has completed start phase phase of a type start. For a listing of start phases, see Section 17.5.1, "Summary of MySQL Cluster Start Phases". (type is one of initial, system, node, initial node, or <Unknown>.)

Event Name. StartPhaseCompleted

Event Type. StartUp

Priority. 4

Severity. INFO

Log Message. Node node_id: CM_REGCONF president = president_id, own Node = own_id, our dynamic id = dynamic_id

Description. Node president_id has been selected as "president". own_id and dynamic_id should always be the same as the ID (node_id) of the reporting node.

Event Name. CM_REGCONF

Event Type. StartUp

Priority. 3

Severity. INFO

Log Message. Node node_id: CM_REGREF from Node president_id to our Node node_id. Cause = cause

Description. The reporting node (ID node_id) was unable to accept node president_id as president. The cause of the problem is given as one of Busy, Election with wait = false, Not president, Election without selecting new candidate, or No such cause.

Event Name. CM_REGREF

Event Type. StartUp

Priority. 8

Severity. INFO

Log Message. Node node_id: We are Node own_id with dynamic ID dynamic_id, our left neighbor is Node id_1, our right is Node id_2

Description. The node has discovered its neighboring nodes in the cluster (node id_1 and node id_2). node_id, own_id, and dynamic_id should always be the same; if they are not, this indicates a serious misconfiguration of the cluster nodes.

Event Name. FIND_NEIGHBOURS

Event Type. StartUp

Priority. 8

Severity. INFO

Log Message. Node node_id: type shutdown initiated

Description. The node has received a shutdown signal. The type of shutdown is either Cluster or Node.

Event Name. NDBStopStarted

Event Type. StartUp

Priority. 1

Severity. INFO

Log Message. Node node_id: Node shutdown completed [, action] [Initiated by signal signal.]

Description. The node has been shut down. This report may include an action, which if present is one of restarting, no start, or initial. The report may also include a reference to an NDB Protocol signal; for possible signals, refer to Operations and Signals.

Event Name. NDBStopCompleted

Event Type. StartUp

Priority. 1

Severity. INFO

Log Message. Node node_id: Forced node shutdown completed [, action]. [Occured during startphase start_phase.] [ Initiated by signal.] [Caused by error error_code: 'error_message(error_classification). error_status'. [(extra info extra_code)]]

Description. The node has been forcibly shut down. The action (one of restarting, no start, or initial) subsequently being taken, if any, is also reported. If the shutdown occurred while the node was starting, the report includes the start_phase during which the node failed. If this was a result of a signal sent to the node, this information is also provided (see Operations and Signals, for more information). If the error causing the failure is known, this is also included; for more information about NDB error messages and classifications, see MySQL Cluster API Errors.

Event Name. NDBStopForced

Event Type. StartUp

Priority. 1

Severity. ALERT

Log Message. Node node_id: Node shutdown aborted

Description. The node shutdown process was aborted by the user.

Event Name. NDBStopAborted

Event Type. StartUp

Priority. 1

Severity. INFO

Log Message. Node node_id: StartLog: [GCI Keep: keep_pos LastCompleted: last_pos NewestRestorable: restore_pos]

Description. This reports global checkpoints referenced during a node start. The redo log prior to keep_pos is dropped. last_pos is the last global checkpoint in which data node the participated; restore_pos is the global checkpoint which is actually used to restore all data nodes.

Event Name. StartREDOLog

Event Type. StartUp

Priority. 4

Severity. INFO

Log Message. startup_message [Listed separately; see below.]

Description. There are a number of possible startup messages that can be logged under different circumstances.

Event Name. StartReport

Event Type. StartUp

Priority. 4

Severity. INFO

Log Message. Node node_id: Node restart completed copy of dictionary information

Description. Copying of data dictionary information to the restarted node has been completed.

Event Name. NR_CopyDict

Event Type. NodeRestart

Priority. 8

Severity. INFO

Log Message. Node node_id: Node restart completed copy of distribution information

Description. Copying of data distribution information to the restarted node has been completed.

Event Name. NR_CopyDistr

Event Type. NodeRestart

Priority. 8

Severity. INFO

Log Message. Node node_id: Node restart starting to copy the fragments to Node node_id

Description. Copy of fragments to starting data node node_id has begun

Event Name. NR_CopyFragsStarted

Event Type. NodeRestart

Priority. 8

Severity. INFO

Log Message. Node node_id: Table ID = table_id, fragment ID = fragment_id have been copied to Node node_id

Description. Fragment fragment_id from table table_id has been copied to data node node_id

Event Name. NR_CopyFragDone

Event Type. NodeRestart

Priority. 10

Severity. INFO

Log Message. Node node_id: Node restart completed copying the fragments to Node node_id

Description. Copying of all table fragments to restarting data node node_id has been completed

Event Name. NR_CopyFragsCompleted

Event Type. NodeRestart

Priority. 8

Severity. INFO

Log Message. Any of the following:

Description. One of the following (each corresponding to the same-numbered message listed above):

Event Name. NodeFailCompleted

Event Type. NodeRestart

Priority. 8

Severity. ALERT

Log Message. Node mgm_node_id: Node data_node_id has failed. The Node state at failure was state_code

Description. A data node has failed. Its state at the time of failure is described by an arbitration state code state_code: possible state code values can be found in the file include/kernel/signaldata/ArbitSignalData.hpp.

Event Name. NODE_FAILREP

Event Type. NodeRestart

Priority. 8

Severity. ALERT

Log Message. President restarts arbitration thread [state=state_code] or Prepare arbitrator node node_id [ticket=ticket_id] or Receive arbitrator node node_id [ticket=ticket_id] or Started arbitrator node node_id [ticket=ticket_id] or Lost arbitrator node node_id - process failure [state=state_code] or Lost arbitrator node node_id - process exit [state=state_code] or Lost arbitrator node node_id - error_message [state=state_code]

Description. This is a report on the current state and progress of arbitration in the cluster. node_id is the node ID of the management node or SQL node selected as the arbitrator. state_code is an arbitration state code, as found in include/kernel/signaldata/ArbitSignalData.hpp. When an error has occurred, an error_message, also defined in ArbitSignalData.hpp, is provided. ticket_id is a unique identifier handed out by the arbitrator when it is selected to all the nodes that participated in its selection; this is used to ensure that each node requesting arbitration was one of the nodes that took part in the selection process.

Event Name. ArbitState

Event Type. NodeRestart

Priority. 6

Severity. INFO

Log Message. Arbitration check lost - less than 1/2 nodes left or Arbitration check won - all node groups and more than 1/2 nodes left or Arbitration check won - node group majority or Arbitration check lost - missing node group or Network partitioning - arbitration required or Arbitration won - positive reply from node node_id or Arbitration lost - negative reply from node node_id or Network partitioning - no arbitrator available or Network partitioning - no arbitrator configured or Arbitration failure - error_message [state=state_code]

Description. This message reports on the result of arbitration. In the event of arbitration failure, an error_message and an arbitration state_code are provided; definitions for both of these are found in include/kernel/signaldata/ArbitSignalData.hpp.

Event Name. ArbitResult

Event Type. NodeRestart

Priority. 2

Severity. ALERT

Log Message. Node node_id: GCP Take over started

Description. This node is attempting to assume responsibility for the next global checkpoint (that is, it is becoming the master node)

Event Name. GCP_TakeoverStarted

Event Type. NodeRestart

Priority. 7

Severity. INFO

Log Message. Node node_id: GCP Take over completed

Description. This node has become the master, and has assumed responsibility for the next global checkpoint

Event Name. GCP_TakeoverCompleted

Event Type. NodeRestart

Priority. 7

Severity. INFO

Log Message. Node node_id: LCP Take over started

Description. This node is attempting to assume responsibility for the next set of local checkpoints (that is, it is becoming the master node)

Event Name. LCP_TakeoverStarted

Event Type. NodeRestart

Priority. 7

Severity. INFO

Log Message. Node node_id: LCP Take over completed

Description. This node has become the master, and has assumed responsibility for the next set of local checkpoints

Event Name. LCP_TakeoverCompleted

Event Type. NodeRestart

Priority. 7

Severity. INFO

Log Message. Node node_id: Trans. Count = transactions, Commit Count = commits, Read Count = reads, Simple Read Count = simple_reads, Write Count = writes, AttrInfo Count = AttrInfo_objects, Concurrent Operations = concurrent_operations, Abort Count = aborts, Scans = scans, Range scans = range_scans

Description. This report of transaction activity is given approximately once every 10 seconds

Event Name. TransReportCounters

Event Type. Statistic

Priority. 8

Severity. INFO

Log Message. Node node_id: Operations=operations

Description. Number of operations performed by this node, provided approximately once every 10 seconds

Event Name. OperationReportCounters

Event Type. Statistic

Priority. 8

Severity. INFO

Log Message. Node node_id: Table with ID = table_id created

Description. A table having the table ID shown has been created

Event Name. TableCreated

Event Type. Statistic

Priority. 7

Severity. INFO

Log Message. Node node_id: Mean loop Counter in doJob last 8192 times = count

Description.

Event Name. JobStatistic

Event Type. Statistic

Priority. 9

Severity. INFO

Log Message. Mean send size to Node = node_id last 4096 sends = bytes bytes

Description. This node is sending an average of bytes bytes per send to node node_id

Event Name. SendBytesStatistic

Event Type. Statistic

Priority. 9

Severity. INFO

Log Message. Mean receive size to Node = node_id last 4096 sends = bytes bytes

Description. This node is receiving an average of bytes of data each time it receives data from node node_id

Event Name. ReceiveBytesStatistic

Event Type. Statistic

Priority. 9

Severity. INFO

Log Message. Node node_id: Data usage is data_memory_percentage% (data_pages_used 32K pages of total data_pages_total) / Node node_id: Index usage is index_memory_percentage% (index_pages_used 8K pages of total index_pages_total)

Description. This report is generated when a DUMP 1000 command is issued in the cluster management client; for more information, see DUMP 1000, in MySQL Cluster Internals

Event Name. MemoryUsage

Event Type. Statistic

Priority. 5

Severity. INFO

Log Message. Node node1_id: Transporter to node node2_id reported error error_code: error_message

Description. A transporter error occurred while communicating with node node2_id; for a listing of transporter error codes and messages, see NDB Transporter Errors, in MySQL Cluster Internals

Event Name. TransporterError

Event Type. Error

Priority. 2

Severity. ERROR

Log Message. Node node1_id: Transporter to node node2_id reported error error_code: error_message

Description. A warning of a potential transporter problem while communicating with node node2_id; for a listing of transporter error codes and messages, see NDB Transporter Errors, for more information

Event Name. TransporterWarning

Event Type. Error

Priority. 8

Severity. WARNING

Log Message. Node node1_id: Node node2_id missed heartbeat heartbeat_id

Description. This node missed a heartbeat from node node2_id

Event Name. MissedHeartbeat

Event Type. Error

Priority. 8

Severity. WARNING

Log Message. Node node1_id: Node node2_id declared dead due to missed heartbeat

Description. This node has missed at least 3 heartbeats from node node2_id, and so has declared that node "dead"

Event Name. DeadDueToHeartbeat

Event Type. Error

Priority. 8

Severity. ALERT

Log Message. Node node1_id: Node Sent Heartbeat to node = node2_id

Description. This node has sent a heartbeat to node node2_id

Event Name. SentHeartbeat

Event Type. Info

Priority. 12

Severity. INFO

Log Message. Node node_id: Event buffer status: used=bytes_used (percent_used%) alloc=bytes_allocated (percent_available%) max=bytes_available apply_gci=latest_restorable_GCI latest_gci=latest_GCI

Description. This report is seen during heavy event buffer usage, for example, when many updates are being applied in a relatively short period of time; the report shows the number of bytes and the percentage of event buffer memory used, the bytes allocated and percentage still available, and the latest and latest restorable global checkpoints

Event Name. EventBufferStatus

Event Type. Info

Priority. 7

Severity. INFO

Log Message. Node node_id: Entering single user mode, Node node_id: Entered single user mode Node API_node_id has exclusive access, Node node_id: Entering single user mode

Description. These reports are written to the cluster log when entering and exiting single user mode; API_node_id is the node ID of the API or SQL having exclusive access to the cluster (for more information, see Section 17.5.8, "MySQL Cluster Single User Mode"); the message Unknown single user report API_node_id indicates an error has taken place and should never be seen in normal operation

Event Name. SingleUser

Event Type. Info

Priority. 7

Severity. INFO

Log Message. Node node_id: Backup backup_id started from node mgm_node_id

Description. A backup has been started using the management node having mgm_node_id; this message is also displayed in the cluster management client when the START BACKUP command is issued; for more information, see Section 17.5.3.2, "Using The MySQL Cluster Management Client to Create a Backup"

Event Name. BackupStarted

Event Type. Backup

Priority. 7

Severity. INFO

Log Message. Node node_id: Backup backup_id started from node mgm_node_id completed. StartGCP: start_gcp StopGCP: stop_gcp #Records: records #LogRecords: log_records Data: data_bytes bytes Log: log_bytes bytes

Description. The backup having the ID backup_id has been completed; for more information, see Section 17.5.3.2, "Using The MySQL Cluster Management Client to Create a Backup"

Event Name. BackupCompleted

Event Type. Backup

Priority. 7

Severity. INFO

Log Message. Node node_id: Backup request from mgm_node_id failed to start. Error: error_code

Description. The backup failed to start; for error codes, see MGM API Errors

Event Name. BackupFailedToStart

Event Type. Backup

Priority. 7

Severity. ALERT

Log Message. Node node_id: Backup backup_id started from mgm_node_id has been aborted. Error: error_code

Description. The backup was terminated after starting, possibly due to user intervention

Event Name. BackupAborted

Event Type. Backup

Priority. 7

Severity. ALERT