大量数据导入导致mysql自动重启

时间：2015-03-27 12:16:15 阅读：615 评论：0 收藏：0 [点我收藏+]

昨天晚上第十款做数据库迁移，数据库版本：Version: ‘5.1.61‘，数据量27G左右

message报错信息如下：

Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480438] TsysProxy invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480444] Pid: 32626, comm: TsysProxy Not tainted 2.6.32.57-tlinux_xenU-1.1.rc10-default #1
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480447] Call Trace:
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480456]  [<ffffffff810904ab>] oom_kill_process.clone.0+0xab/0x270
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480459]  [<ffffffff81090788>] __out_of_memory+0x118/0x180
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480462]  [<ffffffff81090872>] out_of_memory+0x82/0x170
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480467]  [<ffffffff81098b40>] ? isolate_pages_global+0x0/0x230
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480470]  [<ffffffff81094f09>] __alloc_pages_nodemask+0x629/0x640
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480476]  [<ffffffff810bf376>] alloc_pages_current+0x76/0xd0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480481]  [<ffffffff8108db8b>] __page_cache_alloc+0xb/0x10
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480484]  [<ffffffff810969e1>] __do_page_cache_readahead+0xc1/0x210
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480487]  [<ffffffff81096b4c>] ra_submit+0x1c/0x20
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480490]  [<ffffffff8108e382>] filemap_fault+0x3f2/0x440
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480493]  [<ffffffff810a480f>] __do_fault+0x4f/0x4e0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480500]  [<ffffffff817a7266>] ? schedule_hrtimeout_range+0xb6/0x130
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480503]  [<ffffffff810a6de8>] handle_mm_fault+0x1b8/0x9e0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480507]  [<ffffffff81048e50>] ? default_wake_function+0x0/0x10
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480512]  [<ffffffff810e3954>] ? mntput_no_expire+0x24/0xf0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480517]  [<ffffffff81034460>] do_page_fault+0x110/0x2e0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480520]  [<ffffffff817a87d5>] page_fault+0x25/0x30
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480521] Mem-Info:
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480523] Node 0 DMA per-cpu:
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480525] CPU    0: hi:    0, btch:   1 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480527] CPU    1: hi:    0, btch:   1 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480529] CPU    2: hi:    0, btch:   1 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480530] CPU    3: hi:    0, btch:   1 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480532] Node 0 DMA32 per-cpu:
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480534] CPU    0: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480535] CPU    1: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480537] CPU    2: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480539] CPU    3: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480540] Node 0 Normal per-cpu:
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480542] CPU    0: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480544] CPU    1: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480545] CPU    2: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480547] CPU    3: hi:  186, btch:  31 usd:   0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480551] active_anon:6920538 inactive_anon:552717 isolated_anon:0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480552]  active_file:142 inactive_file:214 isolated_file:32
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480553]  unevictable:0 dirty:0 writeback:0 unstable:0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480554]  free:35240 slab_reclaimable:3770 slab_unreclaimable:3676
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480555]  mapped:2656 shmem:2366 pagetables:16355 bounce:0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480557] Node 0 DMA free:15836kB min:8kB low:8kB high:12kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15260kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480565] lowmem_reserve[]: 0 3768 29570 29570
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480569] Node 0 DMA32 free:105924kB min:2804kB low:3504kB high:4204kB active_anon:2740028kB inactive_anon:547884kB active_file:56kB inactive_file:92kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3858656kB mlocked:0kB dirty:0kB writeback:0kB mapped:1356kB shmem:1068kB slab_reclaimable:1848kB slab_unreclaimable:2584kB kernel_stack:1080kB pagetables:7140kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480578] lowmem_reserve[]: 0 0 25802 25802
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480580] Node 0 Normal free:19200kB min:19200kB low:24000kB high:28800kB active_anon:24942124kB inactive_anon:1662984kB active_file:512kB inactive_file:764kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:26421600kB mlocked:0kB dirty:0kB writeback:0kB mapped:9268kB shmem:8396kB slab_reclaimable:13232kB slab_unreclaimable:12120kB kernel_stack:1424kB pagetables:58280kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:416 all_unreclaimable? no
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480589] lowmem_reserve[]: 0 0 0 0
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480592] Node 0 DMA: 3*4kB 2*8kB 2*16kB 1*32kB 2*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15836kB
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480599] Node 0 DMA32: 581*4kB 485*8kB 423*16kB 361*32kB 291*64kB 202*128kB 78*256kB 22*512kB 2*1024kB 0*2048kB 1*4096kB = 106380kB
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480606] Node 0 Normal: 3841*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 19460kB
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480613] 42760 total pagecache pages
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480615] 39775 pages in swap cache
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480617] Swap cache stats: add 577057, delete 537282, find 6631146/6638910
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480618] Free swap  = 0kB
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.480619] Total swap = 2097144kB
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.567838] 7680000 pages RAM
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.567841] 126998 pages reserved
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.567842] 14402 pages shared
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.567843] 7510002 pages non-shared
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.567848] Out of memory: kill process 28092 (mysqld_safe) score 3710915 or a child
Mar 27 03:44:09 VM_163_210_tlinux kernel: [18465684.567851] Killed process 28250 (mysqld) vsz:34419948kB, anon-rss:29649368kB, file-rss:460kB
Mar 27 04:40:31 VM_163_210_tlinux kernel: [18469067.168641] TCP: Possible SYN flooding on port 3306. Sending cookies.  Check SNMP counters.
Mar 27 06:35:01 VM_163_210_tlinux nslcd[1762]: [5c31df] ldap_result() failed: Can‘t contact LDAP server
Mar 27 06:35:01 VM_163_210_tlinux nslcd[1762]: [5c31df] ldap_abandon() failed to abandon search: Other (e.g., implementation specific) error

mysql error log 如下：

Version: ‘5.1.61‘  socket: ‘/tmp/mysql.sock‘  port: 3306  Source distribution
150327 03:44:11 mysqld_safe Number of processes running now: 0
150327 03:44:11 mysqld_safe mysqld restarted

参考stack-flow上面的分析：http://stackoverflow.com/questions/22522307/mysqld-safe-mysqld-restarted-automatically

技术分享

死循环导致的，究其原因可能是因为innodb_buffer_size配置有问题，系统硬件内存是30G，buffer_size配置的是28G，建议rein下次导数据的时候先把buffer_size调低，导完之后再调高。

大量数据导入导致mysql自动重启

原文：http://www.cnblogs.com/forilen/p/4371193.html

踩

(0)

评论一句话评论（0）

分享档案

更多>

2021年09月23日 (328)
2021年09月24日 (313)
2021年09月17日 (191)
2021年09月15日 (369)
2021年09月16日 (411)
2021年09月13日 (439)
2021年09月11日 (398)
2021年09月12日 (393)
2021年09月10日 (160)
2021年09月08日 (222)