首页 > 数据库技术 > 详细

MongoDB Sharding Cluster 分片集群搭建及使用

时间:2020-07-15 22:42:51      阅读:116      评论:0      收藏:0      [点我收藏+]

1.1 规划

10个实例:38017-380261)configserver:38018-38020
3台构成的复制集(1主两从,不支持arbiter)38018-38020(复制集名字configsvr)
(2)shard节点:
sh1:38021-23    (1主两从,其中一个节点为arbiter,复制集名字sh1)
sh2:38024-26    (1主两从,其中一个节点为arbiter,复制集名字sh2)
(3) mongos:
38017

1.2 Shard节点配置过程

1.2.1 目录创建:

mkdir -p /mongodb/38021/conf  /mongodb/38021/log  /mongodb/38021/data
mkdir -p /mongodb/38022/conf  /mongodb/38022/log  /mongodb/38022/data
mkdir -p /mongodb/38023/conf  /mongodb/38023/log  /mongodb/38023/data
mkdir -p /mongodb/38024/conf  /mongodb/38024/log  /mongodb/38024/data
mkdir -p /mongodb/38025/conf  /mongodb/38025/log  /mongodb/38025/data
mkdir -p /mongodb/38026/conf  /mongodb/38026/log  /mongodb/38026/data

1.2.2 修改配置文件:

第一组复制集搭建:21-23(1主 1从 1Arb)

cat >  /mongodb/38021/conf/mongodb.conf  <<EOF
systemLog:
  destination: file
  path: /mongodb/38021/log/mongodb.log   
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: /mongodb/38021/data
  directoryPerDB: true
  #engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true
net:
  bindIp: 10.0.0.51,127.0.0.1
  port: 38021
replication:
  oplogSizeMB: 2048
  replSetName: sh1
sharding:
  clusterRole: shardsvr
processManagement: 
  fork: true
EOF
\cp  /mongodb/38021/conf/mongodb.conf  /mongodb/38022/conf/
\cp  /mongodb/38021/conf/mongodb.conf  /mongodb/38023/conf/

sed s#38021#38022#g /mongodb/38022/conf/mongodb.conf -i
sed s#38021#38023#g /mongodb/38023/conf/mongodb.conf -i

第二组节点:24-26(1主1从1Arb)

cat > /mongodb/38024/conf/mongodb.conf <<EOF
systemLog:
  destination: file
  path: /mongodb/38024/log/mongodb.log   
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: /mongodb/38024/data
  directoryPerDB: true
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true
net:
  bindIp: 10.0.0.51,127.0.0.1
  port: 38024
replication:
  oplogSizeMB: 2048
  replSetName: sh2
sharding:
  clusterRole: shardsvr
processManagement: 
  fork: true
EOF

\cp  /mongodb/38024/conf/mongodb.conf  /mongodb/38025/conf/
\cp  /mongodb/38024/conf/mongodb.conf  /mongodb/38026/conf/
sed s#38024#38025#g /mongodb/38025/conf/mongodb.conf -i
sed s#38024#38026#g /mongodb/38026/conf/mongodb.conf -i

1.2.3 启动所有节点,并搭建复制集

mongod -f  /mongodb/38021/conf/mongodb.conf 
mongod -f  /mongodb/38022/conf/mongodb.conf 
mongod -f  /mongodb/38023/conf/mongodb.conf 
mongod -f  /mongodb/38024/conf/mongodb.conf 
mongod -f  /mongodb/38025/conf/mongodb.conf 
mongod -f  /mongodb/38026/conf/mongodb.conf  
ps -ef |grep mongod

mongo --port 38021
use  admin
config = {_id: sh1, members: [
                          {_id: 0, host: 10.0.0.51:38021},
                          {_id: 1, host: 10.0.0.51:38022},
                          {_id: 2, host: 10.0.0.51:38023,"arbiterOnly":true}]
           }

rs.initiate(config)
  
 mongo --port 38024 
 use admin
config = {_id: sh2, members: [
                          {_id: 0, host: 10.0.0.51:38024},
                          {_id: 1, host: 10.0.0.51:38025},
                          {_id: 2, host: 10.0.0.51:38026,"arbiterOnly":true}]
           }
  
rs.initiate(config)

1.3 config节点配置

1.3.1 目录创建

mkdir -p /mongodb/38018/conf  /mongodb/38018/log  /mongodb/38018/data
mkdir -p /mongodb/38019/conf  /mongodb/38019/log  /mongodb/38019/data
mkdir -p /mongodb/38020/conf  /mongodb/38020/log  /mongodb/38020/data

1.3.2修改配置文件:

cat > /mongodb/38018/conf/mongodb.conf <<EOF
systemLog:
  destination: file
  path: /mongodb/38018/log/mongodb.conf
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: /mongodb/38018/data
  directoryPerDB: true
  #engine: wiredTiger
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1
      directoryForIndexes: true
    collectionConfig:
      blockCompressor: zlib
    indexConfig:
      prefixCompression: true
net:
  bindIp: 10.0.0.51,127.0.0.1
  port: 38018
replication:
  oplogSizeMB: 2048
  replSetName: configReplSet
sharding:
  clusterRole: configsvr
processManagement: 
  fork: true
EOF

\cp /mongodb/38018/conf/mongodb.conf /mongodb/38019/conf/
\cp /mongodb/38018/conf/mongodb.conf /mongodb/38020/conf/
sed s#38018#38019#g /mongodb/38019/conf/mongodb.conf -i
sed s#38018#38020#g /mongodb/38020/conf/mongodb.conf -i

1.3.3启动节点,并配置复制集

mongod -f /mongodb/38018/conf/mongodb.conf 
mongod -f /mongodb/38019/conf/mongodb.conf 
mongod -f /mongodb/38020/conf/mongodb.conf 

mongo --port 38018
use  admin
 config = {_id: configReplSet, members: [
                          {_id: 0, host: 10.0.0.51:38018},
                          {_id: 1, host: 10.0.0.51:38019},
                          {_id: 2, host: 10.0.0.51:38020}]
           }
rs.initiate(config)  
  
注:configserver 可以是一个节点,官方建议复制集。configserver不能有arbiter。
新版本中,要求必须是复制集。
注:mongodb 3.4之后,虽然要求config server为replica set,但是不支持arbiter

1.4 mongos节点配置:

1.4.1创建目录:

mkdir -p /mongodb/38017/conf  /mongodb/38017/log 

1.4.2配置文件:

cat > /mongodb/38017/conf/mongos.conf <<EOF
systemLog:
  destination: file
  path: /mongodb/38017/log/mongos.log
  logAppend: true
net:
  bindIp: 10.0.0.51,127.0.0.1
  port: 38017
sharding:
  configDB: configReplSet/10.0.0.51:38018,10.0.0.51:38019,10.0.0.51:38020
processManagement: 
  fork: true
EOF

1.4.3启动mongos

 mongos -f /mongodb/38017/conf/mongos.conf 

1.5 分片集群添加节点

 连接到其中一个mongos(10.0.0.51),做以下配置
(1)连接到mongs的admin数据库
# su - mongod
$ mongo 10.0.0.51:38017/admin
(2)添加分片
db.runCommand( { addshard : "sh1/10.0.0.51:38021,10.0.0.51:38022,10.0.0.51:38023",name:"shard1"} )
db.runCommand( { addshard : "sh2/10.0.0.51:38024,10.0.0.51:38025,10.0.0.51:38026",name:"shard2"} )
(3)列出分片
mongos> db.runCommand( { listshards : 1 } )
(4)整体状态查看
mongos> sh.status();

 

1.6 使用分片集群

1.6.1 RANGE分片配置及测试

1、激活数据库分片功能

mongo --port 38017 admin
admin>  ( { enablesharding : "数据库名称" } )
eg:
admin> db.runCommand( { enablesharding : "test" } )

2、指定分片键对集合分片

### 创建索引
use test
> db.vast.ensureIndex( { id: 1 } )
### 开启分片
use admin
> db.runCommand( { shardcollection : "test.vast",key : {id: 1} } )

3、集合分片验证

admin> use test
test> for(i=1;i<1000000;i++){ db.vast.insert({"id":i,"name":"shenzheng","age":70,"date":new Date()}); }
test> db.vast.stats()

4、分片结果测试

shard1:
mongo --port 38021
db.vast.count();

shard2:
mongo --port 38024
db.vast.count();

 

1.6.2 Hash分片例子:

对oldboy库下的vast大表进行hash
创建哈希索引
(1)对于oldboy开启分片功能
mongo --port 38017 admin
use admin
admin> db.runCommand( { enablesharding : "oldboy" } )
(2)对于oldboy库下的vast表建立hash索引
use oldboy
oldboy> db.vast.ensureIndex( { id: "hashed" } )
(3)开启分片 
use admin
admin > sh.shardCollection( "oldboy.vast", { id: "hashed" } )
(4)录入10w行数据测试
use oldboy
for(i=1;i<100000;i++){ db.vast.insert({"id":i,"name":"shenzheng","age":70,"date":new Date()}); }
(5)hash分片结果测试
mongo --port 38021
use oldboy
db.vast.count();
mongo --port 38024
use oldboy
db.vast.count();

 

 

 

 

 

 

 

 

 

 

66

MongoDB Sharding Cluster 分片集群搭建及使用

原文:https://www.cnblogs.com/quzq/p/13308183.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!