Internal Program Error Failed In Require

4:13 Reporter: Maycon Santos Email Updates: Status: Closed Impact on me: None Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious) Version:7.2.3 OS:Linux (CentOS release 6.3 (Final)) Assigned to: Matthew Montgomery Tags: DBTC, error 2341, internal program error, Internal program error (failed ndbrequire) Triage: Triaged: D3 (Medium) View Add Comment Files Developer Edit Submission View Progress Log Contributions [2 Sep 2013 13:33] Maycon Santos Description: Hi, Yesterday we notice that one of our Data Node were shutdown due to a Internal program error (failed ndbrequire). As I couldn't found any history online based on the logs information, I am doing as sugested and opening this bug report. Logs: ndb_4_out.log 2013-09-01 18:11:25 [ndbd] WARNING -- Ndb kernel thread 27 is stuck in: Scanning Timers elapsed=100 2013-09-01 18:11:28 [ndbd] INFO -- Watchdog: User time: 40830051 System time: 37161110 2013-09-01 18:11:28 [ndbd] WARNING -- Ndb kernel thread 28 is stuck in: Scanning Timers elapsed=100 2013-09-01 18:11:28 [ndbd] INFO -- Watchdog: User time: 40830051 System time: 37161110 2013-09-01 18:11:28 [ndbd] WARNING -- Ndb kernel thread 26 is stuck in: Scanning Timers elapsed=100 2013-09-01 18:11:28 [ndbd] INFO -- Watchdog: User time: 40830052 System time: 37161110 2013-09-01 18:11:28 [ndbd] WARNING -- timerHandlingLab now: 881854613 sent: 881851843 diff: 2770 2013-09-01 18:11:28 [ndbd] INFO -- Watchdog: User time: 40830083 System time: 37161117 2013-09-01 18:11:28 [ndbd] WARNING -- Watchdog: Warning overslept 3339 ms, expected 100 ms. 2013-09-01 18:11:29 [ndbd] INFO -- Internal program error (failed ndbrequire) 2013-09-01 18:11:29 [ndbd] INFO -- DBTC (Line: 7783) 0x00000002 2013-09-01 18:11:29 [ndbd] INFO -- Error handler shutting down system 2013-09-01 18:11:2

Symptom: (mysql-5.6.24 with ndb-7.4.6) [mysql@localhost ndb_data]$ tail -10 ndb_11_out.log 2015-07-02 15:08:42 [ndbd] INFO -- Grant nodes to start phase: 5, nodes: 0000000000001800 2015-07-02 15:08:42 [ndbd] INFO -- NDB start phase 4 completed 2015-07-02 15:08:42 [ndbd] INFO -- As master we will wait for other nodes to reach the state waitpoint52 as well 2015-07-02 15:08:42 [ndbd] INFO -- Node 12 have reached completion of NDB start phase 4 2015-07-02 15:08:42 [ndbd] INFO -- Start NDB start phase 5 (only to DBDIH) 2015-07-02 15:08:47 [ndbd] INFO -- /export/home/pb2/build/sb_0-14878975-1427908966.28/mysql-cluster-com-7.4.6/storage/ndb/src/kernel/blocks/backup/Backup.cpp https://bugs.mysql.com/bug.php?id=70217 2015-07-02 15:08:47 [ndbd] INFO -- BACKUP (Line: 3846) 0x00000002 2015-07-02 15:08:47 [ndbd] INFO -- Error handler shutting down system 2015-07-02 15:08:47 [ndbd] INFO -- Error handler shutdown completed - exiting 2015-07-02 15:08:47 [ndbd] ALERT -- Node 11: Forced node shutdown completed. Occured during startphase 5. Caused by error 2341: 'Internal program error (failed ndbrequire)(Internal error, programming error or missing error http://www.dbaglobe.com/2015/07/caused-by-error-2341-internal-program.html message, please report a bug). Temporary error, restart node'. [mysql@localhost ndb_data]$ more ndb_11_error.log Current byte-offset of file-pointer is: 1067 Time: Thursday 2 July 2015 - 15:08:47 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: Backup.cpp Error object: BACKUP (Line: 3846) 0x00000002 Program: ndbd Pid: 25840 Version: mysql-5.6.24 ndb-7.4.6 Trace: /var/local/mysql/server5-ndb/ndb_data/ndb_11_trace.log.2 [t1..t1] ***EOM*** Root Cause: And I found it was caused by some files inside "datadir" not owned by "mysql" any more, it was changed to "root". After change permission back to "mysql", this problem resolved. [mysql@localhost server2-ndb]$ find . -exec ls -l {} \; |grep root -rw-r--r--. 1 root root 1244 Jul 1 15:11 T52F0.Data -rw-r--r--. 1 root root 1244 Jul 1 15:11 T52F1.Data -rw-r--r--. 1 root root 1244 Jul 1 15:11 ./ndb_data/ndb_11_fs/LCP/1/T52F1.Data -rw-r--r--. 1 root root 1244 Jul 1 15:11 ./ndb_data/ndb_11_fs/LCP/1/T52F0.Data http://bugs.mysql.com/bug.php?id=70217 Posted by Donghua Luo at 11:51 PM Labels: MySQL Newer Post Older Post Home Subscribe to: Post Comments (Atom) Labels AIX (1) Apache Cassandra (7) Apache Spark (5) C++ (1) Cassandra (1) Data Lake (1) DB2 (5) EMC (5) Enterprise Manager (2) GoldenGate (3) HPUX (2) Linux

Italiano Greece Ελληνικά Czech Republic Čeština Slovakia Slovenčina Russia Русский http://knowledgebase.abbyy.com/article/1469 Ukraine Украïнська Ukraine Русский Romania Română Hungary Magyar Asia and Middle East China 中文 South Korea 한국어 Turkey Türkçe Israel עברית Australia Australia English Products and Services Solutions Support Partners Company Store ABBYY.com › ABBYY Technical Support Service › Knowledge Base internal program Knowledge Base Article ID: 1469 | Category: Installation | Type: Hot Fix | Last Modified: 11/22/2013 File processing fails and the "Internal program error: .\Src\SpecialFontFactory.cpp, 138" error message appears in the job log. Description Image processing fails and the "Internal program error: internal program error .\Src\SpecialFontFactory.cpp, 138" error message appears in the job log. Reason This error occurs when *.cache files are missing from the installation folder of ABBYY Recognition Server. We do not currentlyknow what causes the absence of these files. Solution Complete the following steps to resolve this problem. Start ABBYY Recognition Server installation on each processing station. Select Repair. If the problem persists, please contact Technical Support. 196 people think this is helpful. Yes No Was this information helpful to you? Need help? Ask ABBYY Online Support. Search Knowledge Base Advanced Search HAVE A QUESTION? ASK COMMUNITY Related Products ABBYY Recognition Server 4.0 3.5 Related Tags Internal Program Error Builds 3.5.2.4213.5.2.4103.5.1.3433.5.0.137 Contact Us Site Map Legal Privacy Website Feedback © 2016 ABBYY. All rights reserved.