mongodb内存使用小结

Z时代
2024-01-10
分类：综合

database

mongodb 内存使用

mongodb通过把文件mapp到内存来访问数据，在os 层可以使用top 命令查看相应的process ,换句访问的文件越多，process 也越多，使用的内存也越多，也可以DB内也可以使用命令来查看虚拟内存的大小，与常驻的内存。eg:

PRIMARY> db.serverStatus().mem { "bits" : 64, #OS位数 "resident" : 79151, #常驻内存/物理内存 "virtual" : 97810, #虚拟内存 "supported" : true, #是否支持额外的内存显示 "mapped" : 0, "mappedWithJournal" : 0 } [vm:24000:mgtest:3.4.6]$top -H top - 09:25:26 up 1652 days, 22:16, 1 user, load average: 2.28, 2.15, 2.09 Tasks: 1832 total, 1 running, 1831 sleeping, 0 stopped, 0 zombie Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 264291560k total, 174980604k used, 89310956k free, 1986892k buffers Swap: 18874364k total, 0k used, 18874364k free, 40814252k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 17821 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.36 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17822 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.15 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17823 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 1025:49 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17829 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 107:48.61 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17830 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 12:51.93 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17831 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 52:03.17 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17832 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 2255:06 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17833 mongodb 20 0 95.5g 77g 13m S 0.7 30.6 2252:49 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17834 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 2253:09 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17835 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 2252:23 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17836 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 93:28.86 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17837 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 22876:13 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17838 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 247:44.60 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17839 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.10 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17840 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:33.99 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17841 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:02.35 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17842 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 392:42.50 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17843 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 52:19.16 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17844 mongodb 20 0 95.5g 77g 13m S 0.4 30.6 454:07.62 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17845 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 209:58.88 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17846 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.02 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17847 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17848 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17849 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:23.83 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17850 mongodb 20 0 95.5g 77g 13m S 0.7 30.6 1325:44 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17851 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 40:42.66 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17852 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 52:46.56 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17853 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17854 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17855 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17856 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17857 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17858 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17859 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17860 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17861 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17862 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17863 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17864 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf 17865 mongodb 20 0 95.5g 77g 13m S 0.0 30.6 0:00.00 /data/mongodb/base/3.4.6/bin/mongod --config=/data/mongodb/etc/mg.cnf [vm:24000:mgtest:3.4.6]

内存建议调整

1.使用cg 设置整个实例的内存使用值
2.storage.wiredTiger.engineConfig.cacheSizeGB 设置为cg 的50%
storage.wiredTiger.engineConfig.cacheSizeGB参数说明：定义 WiredTiger引擎内部cache 的最大值(所有的数据)，但是创建索引时所消耗的内存是独立于WiredTiger 的，从3.4开始，WiredTiger可以设置0.25G~1000GB。

内存淘汰策略

参考：https://mongoing.com/archives/8781

内存使用查看

主要命令：
mem: db.serverStatus.mem
tcmalloc: db.serverStatus().tcmalloc.tcmalloc.formattedString

mgtest:PRIMARY> db.stats()
{
"db" : "meam",
"collections" : 4466,
"views" : 0,
"objects" : 93999866,
"avgObjSize" : 546.7290265179739,
"dataSize" : 51392455231, # 51G
"storageSize" : 4775952384, #4G
"numExtents" : 0,
"indexes" : 12664,
"indexSize" : 6932480000, #7G
"ok" : 1
}
mgtest:PRIMARY>
mgtest:PRIMARY> db.serverStatus().tcmalloc
{
"generic" : {
"current_allocated_bytes" : NumberLong("67615090640"),
"heap_size" : NumberLong("100761542656")
},
"tcmalloc" : {
"pageheap_free_bytes" : 577167360, #
"pageheap_unmapped_bytes" : NumberLong("18533167104"),
"max_total_thread_cache_bytes" : NumberLong(1073741824),
"current_total_thread_cache_bytes" : 15028816,
"total_free_bytes" : NumberLong("14036117552"),
"central_cache_free_bytes" : NumberLong("14020716864"),
"transfer_cache_free_bytes" : 371872,
"thread_cache_free_bytes" : 15028816,
"aggressive_memory_decommit" : 0,
"formattedString" : "------------------------------------------------

MALLOC: 67615090640 (64482.8 MiB) Bytes in use by application

MALLOC: + 577167360 ( 550.4 MiB) Bytes in page heap freelist

MALLOC: + 14020716864 (13371.2 MiB) Bytes in central cache freelist

MALLOC: + 371872 ( 0.4 MiB) Bytes in transfer cache freelist

MALLOC: + 15028816 ( 14.3 MiB) Bytes in thread cache freelists

MALLOC: + 659284160 ( 628.7 MiB) Bytes in malloc metadata

MALLOC: ------------

MALLOC: = 82887659712 (79047.8 MiB) Actual memory used (physical + swap)

MALLOC: + 18533167104 (17674.6 MiB) Bytes released to OS (aka unmapped)

MALLOC: ------------

MALLOC: = 101420826816 (96722.4 MiB) Virtual address space used

MALLOC:

MALLOC: 8139591 Spans in use

MALLOC: 213 Thread heaps in use

MALLOC: 4096 Tcmalloc page size

------------------------------------------------

Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).

Bytes released to the OS take up virtual address space but no physical memory.

"
}
}
mgtest:PRIMARY>

mgtest:PRIMARY> db.serverStatus().mem
{
"bits" : 64, #OS位数
"resident" : 79151, #常驻内存/物理内存
"virtual" : 97810, #虚拟内存
"supported" : true, #是否支持额外的内存显示
"mapped" : 0,
"mappedWithJournal" : 0
}

#查看索引的大小
mgtest:PRIMARY>
db.getCollectionNames().map(name => ({totalIndexSize: db.getCollection(name).stats().totalIndexSize, name: name})).sort((a, b) => a.totalIndexSize - b.totalIndexSize).forEach(printjson)

{ "totalIndexSize" : 10002432, "name" : "T_article_news_flZXB91g18fcqjl75QH6" }
{ "totalIndexSize" : 10059776, "name" : "T_article_news_cSFcNrBaiV7Cud52EaXb" }
{ "totalIndexSize" : 10117120, "name" : "T_article_news_PHymgm317Gz2A2ws0m36" }
{ "totalIndexSize" : 10121216, "name" : "T_article_news_342hVXE31f56wV0H736i" }
{ "totalIndexSize" : 10240000, "name" : "T_article_news_31MEVF98elvUrEZm1hnY" }
{ "totalIndexSize" : 10256384, "name" : "T_article_news_1sjqPaiEZ7DpFHa0H1iv" }
{ "totalIndexSize" : 10285056, "name" : "T_article_news_9ruEXeExFeRXWZvWng3Q" }
{ "totalIndexSize" : 10436608, "name" : "T_article_news_Bu4XmJBN02CEePW7EQiY" }
{ "totalIndexSize" : 10469376, "name" : "T_article_news_Jswrd1tyQbvknwyb7I03" }
{ "totalIndexSize" : 10485760, "name" : "T_article_news_FGMDEyyRg4a4a4Ot2dqW" }
{ "totalIndexSize" : 10575872, "name" : "T_article_news_mxaPsH76hw64HdKwy1i3" }
{ "totalIndexSize" : 10596352, "name" : "T_article_news_3HY77JTj4adyIbHY14Vo" }
{ "totalIndexSize" : 10657792, "name" : "T_article_news_Pkrf2913812E4X32Mh8P" }
{ "totalIndexSize" : 10670080, "name" : "T_article_news_2FhZe8jZObCJ85Nm1P0G" }
{ "totalIndexSize" : 10747904, "name" : "T_article_news_x9XW27Bfz953t7e97O6v" }
{ "totalIndexSize" : 10752000, "name" : "T_article_news_tt7S7C7UX8bb6meRz9oZ" }
{ "totalIndexSize" : 10858496, "name" : "T_article_news_RLbel0jQf1zC3gsd537O" }
{ "totalIndexSize" : 10878976, "name" : "T_article_news_58038F9kMY4BL9U8ACH7" }
{ "totalIndexSize" : 10899456, "name" : "T_article_news_3TPye0a63xb98ac64z75" }
{ "totalIndexSize" : 11051008, "name" : "T_article_news_8170mvJE27MK4wYW03Q2" }
{ "totalIndexSize" : 11177984, "name" : "T_article_news_i1vp48iiMajLq81bj40E" }
{ "totalIndexSize" : 11264000, "name" : "T_article_news_oFKE42IAk7pZ8KqC26BL" }

参考文档

https://segmentfault.com/a/1190000021320867
https://mongoing.com/archives/8781 #MongoDB 如何使用内存？为什么内存满了？
--内存使用说明：
https://www.mongodb.com/blog/post/mongo-db-memory-usage
--内存使用过多问题
https://dba.stackexchange.com/questions/148395/mongodb-using-too-much-memory

https://stackoverflow.com/questions/131303/how-to-measure-actual-memory-usage-of-an-application-or-process
http://gk.link/a/10bqK

以上是 mongodb内存使用小结的全部内容，来源链接： utcz.com/z/535016.html