Redis5.0.3配置文件详解(易读白话翻译)高级配置与碎片整理
############################### ADVANCED CONFIG 高级配置 ################################ 下面的配置需要对Redis的原理，特别5中数据类型的底层数据结构有比较清楚的了解才能看得懂，总的来说就是根据自己的键-值选择5中数据类型在某些条件下使用何种数据结构来存放数据。
# 最常见的高效数据结构就是ziplist、intset,但是他们通常只有元素(条目)较小且元素(条目)较小时才适合
# 要学习这部分内容可以看看redis设计与实现和Redis资深历险两本书，前一本书将原理很多，且深度足够，但是Redis的版本有点太老了，后一步本书可以在原理上对前一本书进行补充，且Redis版本很新，已经到5了。而且它还将了很多实战的知识。
#
# Hashes are encoded using a memory efficient data structure when they have a
# small number of entries, and the biggest entry does not exceed a given
# threshold. These thresholds can be configured using the following directives.
#
# hash数据类型：如果条目数小于512，且条目大小不超过64字节，则使用ziplist作为hash数据类型的底层数据结构
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
# 新版本的Redis针对list数据类型的底层数据结构做了优化采用的是"链表+ziplist"，其思想有点像Java HashMap的"数组+链表/红黑树"
# Lists are also encoded in a special way to save a lot of space.
# The number of entries allowed per internal list node can be specified
# as a fixed maximum size or a maximum number of elements.
# 可以通过"list-max-ziplist-size"设置链表中ziplist的条目数量，其值可以是条目数量，也可以最大字节数
# For a fixed maximum size, use -5 through -1, meaning:
# 下面是5个可能取值，建议使用-1 和 -2，其他选项不推荐使用，除非有特殊需求
# -5: max size: 64 Kb <-- not recommended for normal workloads
# -4: max size: 32 Kb <-- not recommended
# -3: max size: 16 Kb <-- probably not recommended
# -2: max size: 8 Kb <-- good
# -1: max size: 4 Kb <-- good
#
# Positive numbers mean store up to _exactly_ that number of elements
# per list node.
# 上面的负值就是单个链表节点所包含的条目数
#
# The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size),
# but if your use case is unique, adjust the settings as necessary.
# 取值为-1 -2 发挥的性能是最好的
list-max-ziplist-size -2
# Lists may also be compressed.
# Compress depth is the number of quicklist ziplist nodes from *each* side of
# the list to *exclude* from compression. The head and tail of the list
# are always uncompressed for fast push/pop operations. Settings are:
# 0: disable all list compression
# 表示不压缩任何节点
# 1: depth 1 means "don"t start compressing until after 1 node into the list,
# going from either the head or tail"
# So: [head]->node->node->...->node->[tail]
# [head], [tail] will always be uncompressed; inner nodes will compress.
# 表示除链表的头尾以外，其他链表节点都压缩
# 2: [head]->[next]->node->node->...->node->[prev]->[tail]
# 2 here means: don"t compress head or head->next or tail->prev or tail,
# but compress all nodes between them.
# 依次类推，即前两个和后两个以外的都压缩
# 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail]
# 依次类推
# etc.
# 默认压缩深度为0，也就是说不压缩。。。无论如何设置头尾是不会压缩的，比如当list被当做队列使用时，如果压缩了，还需要解压，降低了性能。
list-compress-depth 0
# Sets have a special encoding in just one case: when a set is composed
# of just strings that happen to be integers in radix 10 in the range
# of 64 bit signed integers.
# 当集合(set)存放的值都是64位的无符号10进制整数时，且条目数小于512时会采用intset作为集合的底层数据结构
# The following configuration setting sets the limit in the size of the
# set in order to use this special memory saving encoding.
set-max-intset-entries 512
# 和hash数据类型类似，如果条目数小于128，且条目大小<64会使用ziplist作为有序集合的底层数据结构
# Similarly to hashes and lists, sorted sets are also specially encoded in
# order to save a lot of space. This encoding is only used when the length and
# elements of a sorted set are below the following limits:
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
# HyperLogLog sparse representation bytes limit. The limit includes the
# 16 bytes header. When an HyperLogLog using the sparse representation crosses
# this limit, it is converted into the dense representation.
#
# A value greater than 16000 is totally useless, since at that point the
# dense representation is more memory efficient.
#
# The suggested value is ~ 3000 in order to have the benefits of
# the space efficient encoding without slowing down too much PFADD,
# which is O(N) with the sparse encoding. The value can be raised to
# ~ 10000 when CPU is not a concern, but space is, and the data set is
# composed of many HyperLogLogs with cardinality in the 0 - 15000 range.
# 这个是Redis高级功能，可以用这种数据结构统计网站的UV，能够去重，其准确度接近真实值
# 简单点说：当去重后统计出来的值小于"hll-sparse-max-bytes"指定的值时，Redis会使用稀疏矩阵来存放，一个Key占用的空间比稠密矩阵小，如果统计出来的值大于"hll-sparse-max-bytes"指定的值，那么使用稠密矩阵，此时一个Key占用的空间是12KB
# "hll-sparse-max-bytes"默认为3000，如果设置为16000以上完全是无用的，因为此时稠密矩阵效果更好
hll-sparse-max-bytes 3000
# Streams macro node max size / items. The stream data structure is a radix
# tree of big nodes that encode multiple items inside. Using this configuration
# it is possible to configure how big a single node can be in bytes, and the
# maximum number of items it may contain before switching to a new node when
# appending new stream entries. If any of the following settings are set to
# zero, the limit is ignored, so for instance it is possible to set just a
# max entires limit by setting max-bytes to 0 and max-entries to the desired
# value.
# 设置Stream的单个节点最大字节数和最多能有多少个条目，如果任何一个条件满足就会新增加一个节点用以保存新的数据
# 如果将任何一个配置项设置为0，表示不限制
stream-node-max-bytes 4096
stream-node-max-entries 100
# Redis数据库存放键值对数据结构是一个类型为字典长度为2的数组,假设这个数组名称为"ht"，在rehash的时候就是将其中一个字典(ht[0])中的所有数据搬到另一个字典(ht[1])中,而且rehash是惰性的(因为redis要高效的响应查询或者写，不可能去一次完成rehash操作，不像Java的HashMap)，当方式key时或者CPU比较空闲时会触发，因此也被称之为"渐进式hash"
# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
# order to help rehashing the main Redis hash table (the one mapping top-level
# keys to values). The hash table implementation Redis uses (see dict.c)
# performs a lazy rehashing: the more operation you run into a hash table
# that is rehashing, the more rehashing "steps" are performed, so if the
# server is idle the rehashing is never complete and some more memory is used
# by the hash table.
#
# The default is to use this millisecond 10 times every second in order to
# actively rehash the main dictionaries, freeing memory when possible.
# 默认是使用1秒钟的10毫秒进行rehash，在适当的时候回收内存
#
# If unsure:
# use "activerehashing no" if you have hard latency requirements and it is
# not a good thing in your environment that Redis can reply from time to time
# to queries with 2 milliseconds delay.
# 如果系统有严格的延时要求，在2毫秒内不断的查询出结果，可以将"activerehashing"设置no
# 但是这对你的系统并不是一个好事情，因此不建议这样设置，所以保持不动吧
#
# use "activerehashing yes" if you don"t have such hard requirements but
# want to free memory asap when possible.
# 如果没有非常严格的要求，建议将"activerehashing"设置为yes，这样可以让内存尽可能快的释放
activerehashing yes
# The client output buffer limits can be used to force disconnection of clients
# that are not reading data from the server fast enough for some reason (a
# common reason is that a Pub/Sub client can"t consume messages as fast as the
# publisher can produce them).
# 可以通过设置客户端输出缓冲区大小将待接收数据超过缓冲区大小的客户端断开
# 通常使用pub/sub的时候，客户端没有及时消费而导致超过缓冲区大小
#
# The limit can be set differently for the three different classes of clients:
# 提供三种客户端的设置，分别是普通的、主从复制的、pub/sub的客户端，我们可以分别对这三种客户端的输出缓冲区设置大小
#
# normal -> normal clients including MONITOR clients
# replica -> replica clients
# pubsub -> clients subscribed to at least one pubsub channel or pattern
#
# The syntax of every client-output-buffer-limit directive is the following:
# 下面是三种客户端缓冲区大小设置的语法
#
# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
#
# A client is immediately disconnected once the hard limit is reached, or if
# the soft limit is reached and remains reached for the specified number of
# seconds (continuously).
# 如果客户端输出缓冲区的大小达到了"hard limit"，服务器会立即断开连接
# 如果客户端输出缓冲区的大小达到了"soft limit"，且持续时间达到了"soft seconds"，服务器会立即断开连接
#
# So for instance if the hard limit is 32 megabytes and the soft limit is
# 16 megabytes / 10 seconds, the client will get disconnected immediately
# if the size of the output buffers reach 32 megabytes, but will also get
# disconnected if the client reaches 16 megabytes and continuously overcomes
# the limit for 10 seconds.
# 这上面是一个举例，省略。。。。
#
# By default normal clients are not limited because they don"t receive data
# without asking (in a push way), but just after a request, so only
# asynchronous clients may create a scenario where data is requested faster
# than it can read.
# 默认情况下普通的client不限制，因为它们都是发起请求后等待接收数据，并不像异步的客户端(比如主从复制客户端和PUB/SUB)会造成数据的挤压，挤压的原因就是客户端处理速度跟不上数据产生的速度
#
# Instead there is a default limit for pubsub and replica clients, since
# subscribers and replicas receive data in a push fashion.
#
# Both the hard or the soft limit can be disabled by setting them to zero.
# hard or soft limit 都可以通过设置为0而禁止掉
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
# Client query buffers accumulate new commands. They are limited to a fixed
# amount by default in order to avoid that a protocol desynchronization (for
# instance due to a bug in the client) will lead to unbound memory usage in
# the query buffer. However you can configure it here if you have very special
# needs, such us huge multi/exec requests or alike.
# 客户端查询缓冲区会累加新命令，默认情况下，缓冲区大小是一个固定值以避免协议同步失效(如客户端的bug)导致查询缓冲区出现未绑定的内存(即客户端都已经不存在了，但是它发过来的命令还在缓冲区当中)
# 如果有巨大的multi/exec请求，则可以修改这个值以满足我们的特殊需求
#
# client-query-buffer-limit 1gb
# In the Redis protocol, bulk requests, that are, elements representing single
# strings, are normally limited ot 512 mb. However you can change this limit
# here.
# 如果一个大容量请求(即客户端单次发送过来的字符串)被限制为512MB，我们也可以通过修改"proto-max-bulk-len"值
# 不过我可能一辈子也不会用到
# proto-max-bulk-len 512mb
# Redis calls an internal function to perform many background tasks, like
# closing connections of clients in timeout, purging expired keys that are
# never requested, and so forth.
#
# Not all tasks are performed with the same frequency, but Redis checks for
# tasks to perform according to the specified "hz" value.
#
# By default "hz" is set to 10. Raising the value will use more CPU when
# Redis is idle, but at the same time will make Redis more responsive when
# there are many keys expiring at the same time, and timeouts may be
# handled with more precision.
#
# The range is between 1 and 500, however a value over 100 is usually not
# a good idea. Most users should use the default of 10 and raise this up to
# 100 only in environments where very low latency is required.
# 简单点说：Redis有后台任务，通过设置"hz"可提高或者降低检查这些任务是否应该执行的频率，值越大消耗的CPU越多，反之越少
# 值可以设置在1到500之间，通常不建议将该值设置得比100大，一般都使用10这个默认值，除非我们的系统有非常严格的延时要求，才会将"hz"设置得等于或者超过100
hz 10
# Normally it is useful to have an HZ value which is proportional to the
# number of clients connected. This is useful in order, for instance, to
# avoid too many clients are processed for each background task invocation
# in order to avoid latency spikes.
#
# Since the default HZ value by default is conservatively set to 10, Redis
# offers, and enables by default, the ability to use an adaptive HZ value
# which will temporary raise when there are many connected clients.
#
# When dynamic HZ is enabled, the actual configured HZ will be used as
# as a baseline, but multiples of the configured HZ value will be actually
# used as needed once more clients are connected. In this way an idle
# instance will use very little CPU time while a busy instance will be
# more responsive.
# 英文有时还真的描述很啰唆，还是中文编码更高效。。。吐槽一下英文
# 前面的"ht"配置项是固定值，当连接客户端非常多时，如果"ht"还是10，则可能会导致延迟比较高，因此Redis搞了一个
# "dynamic-hz"配置项，当设置为yes时，可以基于"ht"配置值动态的调整使用的"ht"值，比如连接的客户端很多事，动态将ht调高，可以减少延迟。而当连接客户端比较少，又可以动态降低"ht"，这样消耗的CPU会很少
# 默认值是yes，这个根本不需要我们自己去动，有了它我们也不需要去动"ht"配置
dynamic-hz yes
# When a child rewrites the AOF file, if the following option is enabled
# the file will be fsync-ed every 32 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency spikes.
# 当子进程在重写AOF文件时，如果将"aof-rewrite-incremental-fsync"设置为yes，那么一旦生成32M数据才会调用一次OS的fsync函数，这样可以降低出现访问峰值时系统的延迟。因为可以减少fsync调用次数和IO请求
aof-rewrite-incremental-fsync yes
# When redis saves RDB file, if the following option is enabled
# the file will be fsync-ed every 32 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency spikes.
# 和"aof-rewrite-incremental-fsync"一个意思，只不过是用在生成RDB文件时用。
# 如果持久化采用的混合方式，即AOF文件是由"RDB部分+AOF部分"组成的话，我想"aof-rewrite-incremental-fsync"和"rdb-save-incremental-fsync"都会使用到
rdb-save-incremental-fsync yes
# Redis LFU eviction (see maxmemory setting) can be tuned. However it is a good
# idea to start with the default settings and only change them after investigating
# how to improve the performances and how the keys LFU change over time, which
# is possible to inspect via the OBJECT FREQ command.
#
# There are two tunable parameters in the Redis LFU implementation: the
# counter logarithm factor and the counter decay time. It is important to
# understand what the two parameters mean before changing them.
#
# Redis的LFU实现有两个可调整的参数：计数器对数因子(couter logarithm factor)和计数器衰退时间(counter decay time)
# 一定要充分理解这两个参数之后才能去修改，如果不懂就不要去瞎搞了，如果非要修改，一定要使用"OBJECT FREQ"命令充分调查并知道如何提升性能的情况下才能进行
# The LFU counter is just 8 bits per key, it"s maximum value is 255, so Redis
# uses a probabilistic increment with logarithmic behavior. Given the value
# of the old counter, when a key is accessed, the counter is incremented in
# this way:
# 在介绍maxmemory的时候提到了两个参数作用的原理，这里就不赘述了。
#
# 1. A random number R between 0 and 1 is extracted.
# 2. A probability P is calculated as 1/(old_value*lfu_log_factor+1).
# 3. The counter is incremented only if R < P.
#
# The default lfu-log-factor is 10. This is a table of how the frequency
# counter changes with a different number of accesses with different
# logarithmic factors:
# "lfu-log-factor"的默认值=10,下表是不同对数因子下计数器的改变频率:
#
# +--------+------------+------------+------------+------------+------------+
# | factor | 100 hits | 1000 hits | 100K hits | 1M hits | 10M hits |
# +--------+------------+------------+------------+------------+------------+
# | 0 | 104 | 255 | 255 | 255 | 255 |
# +--------+------------+------------+------------+------------+------------+
# | 1 | 18 | 49 | 255 | 255 | 255 |
# +--------+------------+------------+------------+------------+------------+
# | 10 | 10 | 18 | 142 | 255 | 255 |
# +--------+------------+------------+------------+------------+------------+
# | 100 | 8 | 11 | 49 | 143 | 255 |
# +--------+------------+------------+------------+------------+------------+
#
# NOTE: The above table was obtained by running the following commands:
# 上面的表格可以通过下面的命令得到：
#
# redis-benchmark -n 1000000 incr foo
# redis-cli object freq foo
#
# NOTE 2: The counter initial value is 5 in order to give new objects a chance
# to accumulate hits.
# 默认counter的初始值是5，为了让新的对象有机会累加它的命中率
#
# The counter decay time is the time, in minutes, that must elapse in order
# for the key counter to be divided by two (or decremented if it has a value
# less <= 10).
# 计数器衰减时间是key计数器除以2(如果值小于<=10，则递减)所必须经过的时间，单位为分钟。
#
# The default value for the lfu-decay-time is 1. A Special value of 0 means to
# decay the counter every time it happens to be scanned.
# "lfu-decay-time" 的默认值为 1，0 表示每次都对计数器进行衰减
#
# lfu-log-factor 10
# lfu-decay-time 1
########################### ACTIVE DEFRAGMENTATION #######################
########################### 在线碎片整理 #######################
#
# WARNING THIS FEATURE IS EXPERIMENTAL. However it was stress tested
# even in production and manually tested by multiple engineers for some
# time.
# 这还只是一个实验功能，就像Redis Cluster一样，其实已经有很多人在使用了
# What is active defragmentation?
# -------------------------------
#
# Active (online) defragmentation allows a Redis server to compact the
# spaces left between small allocations and deallocations of data in memory,
# thus allowing to reclaim back memory.
#
# Fragmentation is a natural process that happens with every allocator (but
# less so with Jemalloc, fortunately) and certain workloads. Normally a server
# restart is needed in order to lower the fragmentation, or at least to flush
# away all the data and create it again. However thanks to this feature
# implemented by Oran Agra for Redis 4.0 this process can happen at runtime
# in an "hot" way, while the server is running.
# 活动碎片整理允许Redis服务器压缩内存中由于申请和释放数据块导致的碎片，从而回收内存，就好像window的磁盘整理一样
# 碎片是每次申请内存（幸运的是Jemalloc出现碎片的几率小很多）的时候会自然发生的
# 通常来说，为了降低碎片化程度需要重启服务，或者清除所有的数据然后重新创建。 得益于Oran Agra在Redis 4.0实现的这个特性，进程可以在服务运行时以"热"方式完成
#
# Basically when the fragmentation is over a certain level (see the
# configuration options below) Redis will start to create new copies of the
# values in contiguous memory regions by exploiting certain specific Jemalloc
# features (in order to understand if an allocation is causing fragmentation
# and to allocate it in a better place), and at the same time, will release the
# old copies of the data. This process, repeated incrementally for all the keys
# will cause the fragmentation to drop back to normal values.
# 通常来说当碎片化达到一定程度（查看下面的配置）Redis 会使用Jemalloc创建连续的内存空间，并在此内存空间对现有的值进行拷贝，拷贝完成后会释放掉旧的数据。
# 这个过程会对所有的导致碎片化的key以增量的形式进行，Redis处处使用渐进式的，真实辛苦设计者了
#
# Important things to understand:
# 要重点理解的三点：
#
# 1. This feature is disabled by default, and only works if you compiled Redis
# to use the copy of Jemalloc we ship with the source code of Redis.
# This is the default with Linux builds.
# 默认情况下，该功能是关闭的，并且只有在编译Redis时使用了代码中的Jemalloc才生效(这是 Linux 下的默认行为)
# 2. You never need to enable this feature if you don"t have fragmentation
# issues.
# 如果没有碎片问题，我们永远也不需要启用该功能
#
# 3. Once you experience fragmentation, you can enable this feature when
# needed with the command "CONFIG SET activedefrag yes".
# 可以通过命令"CONFIG SET activefrag yes"来启用并试验
#
# The configuration parameters are able to fine tune the behavior of the
# defragmentation process. If you are not sure about what they mean it is
# a good idea to leave the defaults untouched.
# 相关的配置参数可以很好的调整碎片整理过程，如果你不知道这些选项的作用最好使用默认值。
# Enabled active defragmentation
# 开启在线整理
# activedefrag yes
# Minimum amount of fragmentation waste to start active defrag
# 有多少碎片时开始整理
# active-defrag-ignore-bytes 100mb
# Minimum percentage of fragmentation to start active defrag
# 有多少比例的碎片时开始整理
# active-defrag-threshold-lower 10
# Maximum percentage of fragmentation at which we use maximum effort
# 有多少比例的碎片时开始进行整理
# active-defrag-threshold-upper 100
# Minimal effort for defrag in CPU percentage
# 进行碎片整理时使用多少比例的CPU时间
# active-defrag-cycle-min 5
# Maximal effort for defrag in CPU percentage
# 进行整理时使用多少CPU时间
# active-defrag-cycle-max 75
# Maximum number of set/hash/zset/list fields that will be processed from
# the main dictionary scan
# 进行主字典扫描时处理的 set/hash/zset/list 字段的最大数量(就是说在进行主字典扫描时 set/hash/zset/list 的长度小于这个值才会处理，大于这个值的会放在一个列表中延迟处理)
# 因为如果某一个key过大，一次性处理完会非常耗时的
# active-defrag-max-scan-fields 1000
以上是 Redis5.0.3配置文件详解(易读白话翻译)高级配置与碎片整理 的全部内容， 来源链接： utcz.com/z/513250.html
