analyze: deduplicated size of a set of archives, #5741

docs/changes.rst

@@ -162,6 +162,19 @@ Compatibility notes:
 Change Log 2.x
 ==============
 
+Version 2.0.0b23 (not released yet)
+-----------------------------------
+
+New features:
+
+- analyze: report the deduplicated size of a set of archives, #5741.
+  For the considered (matching) set of archives, ``borg analyze`` now reports the
+  deduplicated size of the set (union of referenced chunks, counted once) and the
+  exclusive size of the set (chunks referenced only by the set, i.e. the space that
+  deleting the whole set would free). Chunk ids and sizes are read from the
+  per-archive references cache maintained by ``borg compact``, so unchanged archives
+  usually do not need to be opened.
+
 Version 2.0.0b22 (2026-07-22)
 -----------------------------
 

docs/usage/analyze.rst.inc

@@ -12,6 +12,10 @@ borg analyze
 
     .. class:: borg-options-table
 
+    +-----------------------------------------------------------------------------+----------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------+
+    | **options**                                                                                                                                                                                                                                              |
+    +-----------------------------------------------------------------------------+----------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------+
+    |                                                                             | ``--by-name``                                | decompose the whole repository by archive name (not combinable with archive filters)                                        |
     +-----------------------------------------------------------------------------+----------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------+
     | .. class:: borg-common-opt-ref                                                                                                                                                                                                                           |
     |                                                                                                                                                                                                                                                          |
@@ -48,6 +52,10 @@ borg analyze
 
 
 
+    options
+        --by-name     decompose the whole repository by archive name (not combinable with archive filters)
+
+
     :ref:`common_options`
         |
 
@@ -65,20 +73,62 @@ borg analyze
 Description
 ~~~~~~~~~~~
 
-Analyze archives to find "hot spots".
+Analyze the archives matching the usual archive selection options (e.g. ``-a series_name``).
+
+**Deduplicated size of a set of archives**
+
+For the considered (matching) set of archives, ``borg analyze`` reports the *source*
+(uncompressed source data) and *stored* (compressed, as stored in the repository) size,
+plus the *compression* factor relating the two (stored / source):
+
+- the *deduplicated size of the set*: the summed size of the union of chunks the
+  considered archives reference (chunks shared *within* the set are counted once);
+- the *exclusive size of the set*: the summed size of chunks referenced *only* by the
+  considered set and by no other archive - i.e. the space that deleting the whole set
+  would free (assuming the other archives stay).
+
+Additionally, the *unreferenced chunks* are reported: chunks in the repository that no
+non-deleted archive references, i.e. what ``borg compact`` could free in the current
+state of the repository. Only their stored size is known - a chunk's source size is
+only recorded in the archives that reference it. Note that chunks belonging to
+soft-deleted archives count as unreferenced here, although ``borg compact`` keeps them
+when the repository is damaged (so ``borg undelete`` stays possible).
+
+If no archive filter is given, all archives are considered - every referenced chunk is
+then trivially exclusive to them, so only the repository-wide deduplicated size is shown.
+
+The stored sizes come from the repository's chunk index; the chunk membership and the
+source sizes come from the per-archive references cache that ``borg compact``
+maintains, so unchanged archives usually do not need to be opened.
+
+**Decomposition by archive name (--by-name)**
+
+With ``--by-name``, the whole repository is decomposed by archive name instead. Archives
+sharing a name form a series, so a name usually groups all backups of one source; for
+old-style archives that do not form a series, each name is just one archive.
+
+Every chunk is counted in exactly one row, so the rows add up to the repository's
+deduplicated size:
+
+- one row per archive name, showing what is *exclusive* to it: no archive of another name
+  references these chunks, so deleting all archives of that name would free them;
+- one row for the chunks shared by two or more names;
+- one row for the unreferenced chunks (see above).
 
-``borg analyze`` relies on the usual archive matching options to select the
-archives that should be considered for analysis (e.g. ``-a series_name``).
-Then it iterates over all matching archives, over all contained files, and
-collects information about chunks stored in all directories it encounters.
+This answers "which name costs how much, and what would I get back by dropping it" for
+all names at once, in a single pass over the archives. As the shared and unreferenced
+rows can only be determined by looking at every archive, ``--by-name`` always covers the
+whole repository and cannot be combined with archive filters.
 
-It considers chunk IDs and their plaintext sizes (we do not have the compressed
-size in the repository easily available) and adds up the sizes of added and removed
-chunks per direct parent directory, and outputs a list of "directory: size".
+**Hot spots**
 
-You can use that list to find directories with a lot of "activity" — maybe
-some of these are temporary or cache directories you forgot to exclude.
+If at least two archives match, ``borg analyze`` additionally iterates over all matching
+archives, over all contained files, and collects information about chunks stored in all
+directories it encounters. It considers chunk IDs and their plaintext sizes and adds up
+the sizes of added and removed chunks per direct parent directory, and outputs a list of
+"directory: size".
 
-To avoid including these unwanted directories in your backups, you can carefully
-exclude them in ``borg create`` (for future backups) or use ``borg recreate``
-to recreate existing archives without them.
+You can use that list to find directories with a lot of "activity" — maybe some of these
+are temporary or cache directories you forgot to exclude. To avoid including these unwanted
+directories in your backups, you can carefully exclude them in ``borg create`` (for future
+backups) or use ``borg recreate`` to recreate existing archives without them.

docs/usage/info.rst.inc

@@ -81,11 +81,9 @@ Description
 
 This command displays detailed information about the specified archive.
 
-Please note that the deduplicated sizes of the individual archives do not add
-up to the deduplicated size of the repository ("all archives"), because the two
-mean different things:
-
-This archive / deduplicated size = amount of data stored ONLY for this archive
-= unique chunks of this archive.
-All archives / deduplicated size = amount of data stored in the repository
-= all chunks in the repository.
+The original size shown here is the total size of the archive's source data
+(uncompressed, counting duplicate content per occurrence).
+
+Deduplicated sizes are not shown here (computing them per archive is expensive).
+For the deduplicated size of a set of archives, use ``borg analyze``; for the
+repository-wide deduplicated size, use ``borg compact --stats``.