https://redis.io/topics/persistence，这是 Google 对 https://redis.io/topics/persistence 的缓存。 这是该网页在 2020年4月5日 03:50:37 GMT 的快照。
This page provides a technical description of Redis persistence, it is a suggested read for all Redis users. For a wider overview of Redis persistence and the durability guarantees it provides you may also want to read Redis persistence demystified.
Redis provides a different range of persistence options:
The most important thing to understand is the different trade-offs between the RDB and AOF persistence. Let‘s start with RDB:
The general indication is that you should use both persistence methods if you want a degree of data safety comparable to what PostgreSQL can provide you.
If you care a lot about your data, but still can live with a few minutes of data loss in case of disasters, you can simply use RDB alone.
There are many users using AOF alone, but we discourage it since to have an RDB snapshot from time to time is a great idea for doing database backups, for faster restarts, and in the event of bugs in the AOF engine.
Note: for all these reasons we‘ll likely end up unifying AOF and RDB into a single persistence model in the future (long term plan).
The following sections will illustrate a few more details about the two persistence models.
By default Redis saves snapshots of the dataset on disk, in a binary file called dump.rdb. You can configure Redis to have it save the dataset every N seconds if there are at least M changes in the dataset, or you can manually call the SAVE or BGSAVE commands.
For example, this configuration will make Redis automatically dump the dataset to disk every 60 seconds if at least 1000 keys changed:
save 60 1000
This strategy is known as snapshotting.
Whenever Redis needs to dump the dataset to disk, this is what happens:
Redis forks. We now have a child and a parent process.
The child starts to write the dataset to a temporary RDB file.
When the child is done writing the new RDB file, it replaces the old one.
This method allows Redis to benefit from copy-on-write semantics.
Snapshotting is not very durable. If your computer running Redis stops, your power line fails, or you accidentally kill -9 your instance, the latest data written on Redis will get lost. While this may not be a big deal for some applications, there are use cases for full durability, and in these cases Redis was not a viable option.
The append-only file is an alternative, fully-durable strategy for Redis. It became available in version 1.1.
You can turn on the AOF in your configuration file:
appendonly yes
From now on, every time Redis receives a command that changes the dataset (e.g. SET) it will append it to the AOF. When you restart Redis it will re-play the AOF to rebuild the state.
As you can guess, the AOF gets bigger and bigger as write operations are performed. For example, if you are incrementing a counter 100 times, you‘ll end up with a single key in your dataset containing the final value, but 100 entries in your AOF. 99 of those entries are not needed to rebuild the current state.
So Redis supports an interesting feature: it is able to rebuild the AOF in the background without interrupting service to clients. Whenever you issue a BGREWRITEAOF Redis will write the shortest sequence of commands needed to rebuild the current dataset in memory. If you‘re using the AOF with Redis 2.2 you‘ll need to run BGREWRITEAOF from time to time. Redis 2.4 is able to trigger log rewriting automatically (see the 2.4 example configuration file for more information).
You can configure how many times Redis will fsync data on disk. There are three options:
appendfsync always: fsync every time a new command is appended to the AOF. Very very slow, very safe.appendfsync everysec: fsync every second. Fast enough (in 2.4 likely to be as fast as snapshotting), and you can lose 1 second of data if there is a disaster.appendfsync no: Never fsync, just put your data in the hands of the Operating System. The faster and less safe method. Normally Linux will flush data every 30 seconds with this configuration, but it‘s up to the kernel exact tuning.The suggested (and default) policy is to fsync every second. It is both very fast and pretty safe. The always policy is very slow in practice, but it supports group commit, so if there are multiple parallel writes Redis will try to perform a single fsync operation.
It is possible that the server crashed while writing the AOF file, or that the volume where the AOF file is stored was full at the time of writing. When this happens the AOF still contains consistent data representing a given point-in-time version of the dataset (that may be old up to one second with the default AOF fsync policy), but the last command in the AOF could be truncated. The latest major versions of Redis will be able to load the AOF anyway, just discarding the last non well formed command in the file. In this case the server will emit a log like the following:
* Reading RDB preamble from AOF file...
* Reading the remaining AOF tail...
# !!! Warning: short read while loading the AOF file !!!
# !!! Truncating the AOF at offset 439 !!!
# AOF loaded anyway because aof-load-truncated is enabled
You can change the default configuration to force Redis to stop in such cases if you want, but the default configuration is to continue regardless the fact the last command in the file is not well-formed, in order to guarantee availability after a restart.
Older versions of Redis may not recover, and may require the following steps:
Fix the original file using the redis-check-aof tool that ships with Redis:
$ redis-check-aof --fix
Optionally use diff -u to check what is the difference between two files.
Restart the server with the fixed file.
If the AOF file is not just truncated, but corrupted with invalid byte sequences in the middle, things are more complex. Redis will complain at startup and will abort:
* Reading the remaining AOF tail...
# Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>
The best thing to do is to run the redis-check-aof utility, initially without the --fix option, then understand the problem, jump at the given offset in the file, and see if it is possible to manually repair the file: the AOF uses the same format of the Redis protocol and is quite simple to fix manually. Otherwise it is possible to let the utility fix the file for us, but in that case all the AOF portion from the invalid part to the end of the file may be discarded, leading to a massive amount of data loss if the corruption happened to be in the initial part of the file.
Log rewriting uses the same copy-on-write trick already in use for snapshotting. This is how it works:
Redis forks, so now we have a child and a parent process.
The child starts writing the new AOF in a temporary file.
The parent accumulates all the new changes in an in-memory buffer (but at the same time it writes the new changes in the old append-only file, so if the rewriting fails, we are safe).
When the child is done rewriting the file, the parent gets a signal, and appends the in-memory buffer at the end of the file generated by the child.
Profit! Now Redis atomically renames the old file into the new one, and starts appending new data into the new file.
There is a different procedure to do this in Redis 2.0 and Redis 2.2, as you can guess it‘s simpler in Redis 2.2 and does not require a restart at all.
Redis >= 2.2
The first CONFIG command enables the Append Only File. In order to do so Redis will block to generate the initial dump, then will open the file for writing, and will start appending all the next write queries.
The second CONFIG command is used to turn off snapshotting persistence. This is optional, if you wish you can take both the persistence methods enabled.
IMPORTANT: remember to edit your redis.conf to turn on the AOF, otherwise when you restart the server the configuration changes will be lost and the server will start again with the old configuration.
Redis 2.0
redis-cli BGREWRITEAOF. This will create the append only file.Redis >= 2.4 makes sure to avoid triggering an AOF rewrite when an RDB snapshotting operation is already in progress, or allowing a BGSAVE while the AOF rewrite is in progress. This prevents two Redis background processes from doing heavy disk I/O at the same time.
When snapshotting is in progress and the user explicitly requests a log rewrite operation using BGREWRITEAOF the server will reply with an OK status code telling the user the operation is scheduled, and the rewrite will start once the snapshotting is completed.
In the case both AOF and RDB persistence are enabled and Redis restarts the AOF file will be used to reconstruct the original dataset since it is guaranteed to be the most complete.
Before starting this section, make sure to read the following sentence: Make Sure to Backup Your Database. Disks break, instances in the cloud disappear, and so forth: no backups means huge risk of data disappearing into /dev/null.
Redis is very data backup friendly since you can copy RDB files while the database is running: the RDB is never modified once produced, and while it gets produced it uses a temporary name and is renamed into its final destination atomically using rename(2) only when the new snapshot is complete.
This means that copying the RDB file is completely safe while the server is running. This is what we suggest:
find command to make sure too old snapshots are deleted: for instance you can take hourly snapshots for the latest 48 hours, and daily snapshots for one or two months. Make sure to name the snapshots with data and time information.If you run a Redis instance with only AOF persistence enabled, you can still copy the AOF in order to create backups. The file may lack the final part but Redis will be still able to load it (see the previous sections about truncated AOF files).
Disaster recovery in the context of Redis is basically the same story as backups, plus the ability to transfer those backups in many different external data centers. This way data is secured even in the case of some catastrophic event affecting the main data center where Redis is running and producing its snapshots.
Since many Redis users are in the startup scene and thus don‘t have plenty of money to spend we‘ll review the most interesting disaster recovery techniques that don‘t have too high costs.
gpg -c (in symmetric encryption mode). Make sure to store your password in many different safe places (for instance give a copy to the most important people of your organization). It is recommended to use multiple storage services for improved data safety.authorized_keys file of your small VPS. You are ready to transfer backups in an automated fashion. Get at least two VPS in two different providers for best results.It is important to understand that this system can easily fail if not implemented in the right way. At least make absolutely sure that after the transfer is completed you are able to verify the file size (that should match the one of the file you copied) and possibly the SHA1 digest if you are using a VPS.
You also need some kind of independent alert system if the transfer of fresh backups is not working for some reason.
此页面提供了有关Redis持久性的技术说明，建议所有Redis用户阅读。 有关Redis持久性及其提供的持久性保证的更广泛概述，您可能还需要阅读神秘化的Redis持久性 。
Redis提供了不同的持久性选项范围：
要理解的最重要的事情是RDB与AOF持久性之间的不同权衡。 让我们从RDB开始：
通常的指示是，如果您想要某种与PostgreSQL可以提供的功能相当的数据安全性，则应同时使用两种持久性方法。
如果您非常关心数据，但是在灾难情况下仍然可以承受几分钟的数据丢失，则只需使用RDB。
有很多用户单独使用AOF，但我们不建议这样做，因为不时拥有RDB快照对于进行数据库备份，加快重启速度以及AOF引擎中存在错误是一个好主意。
注意：由于所有这些原因，我们将来可能会最终将AOF和RDB统一为一个持久性模型（长期计划）。
以下各节将说明有关这两个持久性模型的更多详细信息。
默认情况下，Redis将数据集的快照保存在磁盘上的二进制文件dump.rdb 。 如果数据集中至少有M个更改，可以将Redis配置为使其每N秒保存一次数据集，或者可以手动调用SAVE或BGSAVE命令。
例如，如果更改了至少1000个键，此配置将使Redis每60秒自动将数据集转储到磁盘：
save 60 1000
这种策略称为快照 。
每当Redis需要将数据集转储到磁盘时，就会发生以下情况：
Redis 叉子 。 现在，我们有一个孩子和一个父过程。
子级开始将数据集写入临时RDB文件。
当孩子完成新的RDB文件的写入后，它将替换旧的RDB文件。
此方法使Redis可以受益于写时复制语义。
快照不是很持久。 如果运行Redis的计算机停止运行，电源线出现故障或意外kill -9您的实例，则写入Redis的最新数据将丢失。 尽管这对于某些应用程序可能不是什么大问题，但有些使用案例具有充分的耐用性，在这些情况下，Redis并不是可行的选择。
仅附加文件是Redis的另一种完全持久的策略。 它在1.1版中可用。
您可以在配置文件中打开AOF：
appendonly yes
从现在开始，每次Redis收到更改数据集的命令（例如SET ）时，它将添加到AOF。 重新启动Redis时，它将重新播放AOF以重建状态。
可以猜到，随着执行写操作，AOF越来越大。 例如，如果要增加一个计数器100次，最终将在数据集中包含最终值的键只有一个，而在AOF中却包含100个条目。 不需要其中的99个条目来重建当前状态。
因此，Redis支持一个有趣的功能：它可以在后台重建AOF，而不会中断对客户端的服务。 每当您发出BGREWRITEAOF时， Redis都会编写最短的命令序列来重建内存中的当前数据集。 如果您将AOF与Redis 2.2一起使用，则需要不时运行BGREWRITEAOF 。 Redis 2.4能够自动触发日志重写（有关更多信息，请参见2.4示例配置文件）。
您可以配置Redis将磁盘上的数据同步多少次。 共有三个选项：
appendfsync always ：每次将新命令附加到AOF时，均执行fsync 。 非常非常慢，非常安全。appendfsync everysec ：每秒fsync 。 速度足够快（在2.4中可能与快照速度一样快），如果发生灾难，您可能会丢失1秒的数据。appendfsync no ：永远不要使用fsync ，只需将数据放在操作系统手中即可。 更快，更不安全的方法。 通常，Linux使用此配置每30秒刷新一次数据，但这取决于内核的精确调整。建议（默认）策略是每秒fsync一次。 它既快速又安全。 always策略在实践中非常慢，但是它支持组提交，因此，如果有多个并行写入，Redis将尝试执行单个fsync操作。
写入AOF文件时服务器可能崩溃，或者写入时存储AOF文件的卷已满。 发生这种情况时，AOF仍包含表示数据集给定时间点版本的一致数据（使用默认的AOF fsync策略，该数据可能已过期长达一秒钟），但是AOF中的最后一条命令可能会被截断。 Redis的最新主要版本仍将能够加载AOF，只需丢弃文件中最后一个格式不正确的命令即可。 在这种情况下，服务器将发出如下日志：
* Reading RDB preamble from AOF file... * Reading the remaining AOF tail... # !!! Warning: short read while loading the AOF file !!! # !!! Truncating the AOF at offset 439 !!! # AOF loaded anyway because aof-load-truncated is enabled
如果需要，您可以更改默认配置以强制Redis在这种情况下停止，但是无论文件中的最后一个命令格式是否正确，默认配置都是继续执行，以确保重新启动后的可用性。
旧版本的Redis可能无法恢复，并且可能需要执行以下步骤：
使用Redis redis-check-aof的redis-check-aof工具修复原始文件：
$ redis-check-aof --fix
（可选）使用diff -u检查两个文件之间的区别。
用固定文件重新启动服务器。
如果AOF文件不仅被截断，而且在中间被无效字节序列损坏，则情况会更加复杂。 Redis将在启动时抱怨并中止：
* Reading the remaining AOF tail... # Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>
最好的办法是运行redis-check-aof实用程序，最初不带--fix选项，然后了解问题所在，跳转到文件中给定的偏移量，然后查看是否可以手动修复文件： AOF使用与Redis协议相同的格式，并且手动修复非常简单。 否则，可以让该实用程序为我们修复文件，但是在那种情况下，从无效部分到文件末尾的所有AOF部分都可能会被丢弃，如果损坏恰好是，则会导致大量数据丢失。在文件的初始部分。
日志重写使用与快照相同的写时复制技巧。 它是这样工作的：
Redis forks ，所以现在我们有一个孩子和一个父过程。
孩子开始在临时文件中写入新的AOF。
父级将所有新更改累积在内存缓冲区中（但同时它会将新更改写入旧的仅追加文件中，因此，如果重写失败，我们是安全的）。
当孩子完成文件的重写后，父母会收到一个信号，并在孩子生成的文件末尾附加内存中的缓冲区。
利润！ 现在，Redis原子地将旧文件重命名为新文件，并开始将新数据追加到新文件中。
在Redis 2.0和Redis 2.2中，执行此操作的过程有所不同，因为您可以猜到它在Redis 2.2中更简单，并且根本不需要重新启动。
Redis> = 2.2
第一个CONFIG命令启用“仅追加文件”。 为此， Redis将阻止生成初始转储，然后打开文件进行写入，并开始附加所有下一个写入查询。
第二个CONFIG命令用于关闭快照持久性。 这是可选的，如果您希望可以同时启用两种持久性方法。
重要信息：请记住编辑redis.conf以打开AOF，否则，当您重新启动服务器时，配置更改将丢失，并且服务器将使用旧配置重新启动。
Redis 2.0
redis-cli BGREWRITEAOF 。 这将创建仅追加文件。Redis> = 2.4可以确保避免在RDB快照操作正在进行时触发AOF重写，或者在AOF重写正在进行时允许BGSAVE 。 这样可以防止两个Redis后台进程同时执行大量磁盘I / O。
当进行快照时，用户使用BGREWRITEAOF显式请求日志重写操作时，服务器将以OK状态代码答复，告知用户已安排该操作，并且快照完成后将开始重写。
如果同时启用了AOF和RDB持久性，并且Redis重新启动，则AOF文件将用于重建原始数据集，因为它可以保证是最完整的。
在开始本节之前，请确保阅读以下句子： 确保备份数据库 。 磁盘损坏，云中的实例消失等等：没有备份意味着数据消失在/ dev / null中的巨大风险。
Redis非常便于数据备份，因为您可以在数据库运行时复制RDB文件：RDB一旦生成就永远不会被修改，而RDB在生成时会使用一个临时名称，并且仅使用rename（2）原子地重命名为其最终目标新快照完成后。
这意味着在服务器运行时，复制RDB文件是完全安全的。 这是我们的建议：
find命令以确保删除太旧的快照：例如，您可以在最近48小时内每小时拍摄一次快照，并在一个或两个月内拍摄每日快照。 确保使用数据和时间信息命名快照。如果您在仅启用AOF持久性的情况下运行Redis实例，您仍然可以复制AOF以便创建备份。 该文件可能缺少最后一部分，但是Redis仍然可以加载它（请参阅前面有关AOF文件被截断的部分）。
Redis上下文中的灾难恢复与备份基本相同，并且能够在许多不同的外部数据中心中传输这些备份。 即使在某些灾难性事件影响Redis运行并生成其快照的主数据中心的情况下，也可以通过这种方式保护数据。
由于许多Redis用户都处于启动阶段，因此没有足够的钱可以花，我们将审查成本不高的最有趣的灾难恢复技术。
gpg -c加密数据（在对称加密模式下）。 确保将密码存储在许多不同的安全位置（例如，将副本提供给组织中最重要的人员）。 建议使用多个存储服务以提高数据安全性。authorized_keys文件中。 您已准备好以自动方式传输备份。 在两个不同的提供商中获得至少两个VPS，以获得最佳结果。重要的是要理解，如果不正确地实施该系统，很容易发生故障。 至少要绝对确保在传输完成后，您能够验证文件大小（该大小应与您复制的文件之一匹配），如果使用的是VPS，也可以验证SHA1摘要。
如果由于某些原因无法传输新备份，则还需要某种独立的警报系统。
原文：https://www.cnblogs.com/nightnine/p/12642636.html