File dwarfs.changes of Package dwarfs
-------------------------------------------------------------------
Wed Nov 26 18:45:57 UTC 2025 - Mia Herkt <mia@0x0.st>
- Update to version 0.14.1:
Bugfixes:
* The metadata_builder now recomputes all total sizes
(total_fs_size, total_allocated_fs_size, and
total_hardlink_size) as part of the build() function. This not
only ensures that the totals are correct even if the allocated
size changes between scanning and segmenting (which has been
happening at least on ZFS volumes), but it also allows images
affected by a related bug in Windows builds of DwarFS to be
fixed by rebuilding the metadata.
* Instead of making the FUSE drivers fail hard when seeing the
options that were removed in v0.14.0, they now just log a
warning and ignore them. The options may still be fully removed
in a future release.
gh#mhx/dwarfs#303.
* The pcmaudio categorizer had two minor issues when compressing
a large number of WAV files. One was reporting an unsupported
format: 3/0 or unsupported format: 65,534/3 warning, which
isn't very useful for the end user. These format codes
correspond to IEEE floating point formats, which are indeed
unsupported. However, the format appears to be quite common,
so the warning has been downgraded to an info message that
explicitly mentions the floating point format. The second issue
was an unexpected fmt chunk size of 20 bytes, which caused the
file to be rejected as a PCM audio file (meaning it was added
using a generic compressor instead of FLAC). It turns out that
these non-conforming fmt chunks are also quite common in
practice, so the code has been changed to accept the
non-conforming file, but also logging an info message
mentioning the non-conformance.
gh#mhx/dwarfs#309.
* The help text for the mkdwarfs compress level option (-l) was
misleading in combination with the manual page as neither
mentioned that the table with details was shown only by
-H / --long-help.
gh#mhx/dwarfs#312.
Features:
* Added shell completion for dwarfsck and dwarfsextract.
* Added sample desktop unmount handlers.
- Changes in 0.14.0:
Bugfixes:
* Leading dots in --input-list file paths were incorrectly
treated as literal directory names instead of being expanded.
gh#mhx/dwarfs#292.
* The SPDX license identifier in GPL-licensed source files was
incorrectly specified as GPL-3.0-only instead of
GPL-3.0-or-later.
gh#mhx/dwarfs#275.
* Fixed an off-by-one error when recovering self_index fields in
metadata, which could cause the sentinel directory to have a
non-zero self_entry. While harmless by itself (since that entry
is never actually used), this would cause the metadata
consistency check to fail. The fix covers three aspects:
correcting the off-by-one error; ensuring the self_entry
recovery code does not run for the sentinel directory;
and changing the metadata consistency check to only warn about
a non-zero self_entry rather than fail. Running mkdwarfs with
--rebuild-metadata will also reset a non-zero sentinel
self_entry to zero.
* Fixed the implementation of the read operation in the FUSE
driver to send positive error code values to libfuse. This was
likely never triggered in practice, but in cases where parts of
the filesystem image vanish while being accessed (which
previously caused SIGBUS crashes), libfuse would not understand
the negative error codes.
* When setting CPU thread affinity for worker group threads via
DWARFS_WORKER_GROUP_AFFINITY, the code did not CPU_ZERO the
cpu_set_t structure before setting individual CPUs. This could
pin threads to random CPUs in addition to the requested ones.
* The FITS categorizer would scan entire files for the
end-of-header marker if their size was a multiple of
2880 bytes, causing significant slowdowns on large non-FITS
files. Additional checks now ensure scanning only continues if
the data truly looks like a standards-compliant FITS header.
* GCC caught a potential null-pointer dereference on error when
opening a file in mkdwarfs. This has been fixed.
* Numerous fixes for 32-bit architectures, mostly related to
integer overflows with file sizes larger than 4 GiB.
* Another off-by-one error caused the first regular file inode to
be excluded from the file-size cache. This would be hard to
notice unless that file was highly fragmented. The cache will
be fixed when rebuilding the metadata.
* The FUSE driver’s enable_nlink option is now the default
behavior and cannot be disabled. The previous optimization
skipped building a table of hardlink counts, which produced
inherently incorrect file status information (hardlinked files
share an inode, so reporting a link count of 1 is wrong).
The hardlink table is now stored in the metadata by default;
if there are no hardlinks, it consumes no space. You can still
omit the hardlink table with --no-hardlink-table, at the cost
of building it on-the-fly when the filesystem image is loaded
(typically fast — e.g., ~300 ms for 14 million files).
Features:
* New I/O layer abstraction that supports “classic” mmap-based
file access, granular mmap-based access on 32-bit systems, and
fully mmap-less access if desired. This applies to all DwarFS
tools. By default, tools use the most efficient
method—memory-mapping whole files on 64-bit systems and
mapping file segments on 32-bit systems (to conserve address
space). This can be controlled via the new DWARFS_IOLAYER_OPTS
environment variable described in dwarfs-env(7).
* Full support for sparse files. mkdwarfs now detects and
efficiently processes sparse files, skipping holes where
possible and preserving them in the filesystem image.
dwarfsextract extracts sparse files as such and preserves
sparse representations when extracting to archive formats that
support them (e.g., tar).
Note: Sparse file support is not backwards compatible; images
containing sparse files cannot be processed by DwarFS versions
prior to 0.14.0. By default, mkdwarfs enables sparse file
support if it detects sparse input. Use --no-sparse-files to
disable it and ensure compatibility with older versions.
* Support for subsecond timestamp resolution. The default remains
one second, but finer resolutions (down to nanoseconds) can be
specified with --time-resolution. mkdwarfs will warn if the
requested resolution is finer than the native filesystem
resolution. This is fully backwards compatible: older DwarFS
versions will handle such images but ignore the subsecond
parts.
gh#mhx/dwarfs#294.
* Desktop integration for Linux. A new --auto-mountpoint option
automatically creates or selects a mount-point directory,
making it easier to mount DwarFS images from file managers.
Desktop files and MIME type definitions are now installed to
enable double-click mounting of .dwarfs files.
* Shell completion for mkdwarfs (bash and zsh).
* Improved error handling when DwarFS tools encounter SIGBUS
(usually caused by accessing memory-mapped files on unreliable
or faulty storage like network shares or flaky USB drives).
When SIGBUS is caught, tools now print an error suggesting
switching from mmap- to read-based I/O via DWARFS_IOLAYER_OPTS.
* dwarfsck now checks metadata consistency by default (unless
--no-check is given), improving detection of filesystem image
corruption.
* The FUSE driver exposes new options cache_sparse and
no_cache_sparse to control whether sparse files should be
cached in the kernel page cache. See dwarfs(1) for details.
* The JSON output from dwarfsck now contains a complete raw
metadata dump when the detail level includes
metadata_full_dump.
* dwarfsck no longer artificially limits string sizes when
dumping metadata.
* Accelerated search for the start of a DwarFS image in files
with custom headers; the new code is about four times faster,
scanning at more than 6 GiB/s on a modern CPU.
* The cache size can now be configured for dwarfsck, useful with
the --checksum option.
* Both dwarfsck and dwarfsextract now limit the amount of data
requested from the filesystem image at once to avoid exhausting
memory (and virtual address space on 32-bit systems).
* Improved self-extracting binary stub with better compatibility
for qemu, binfmt_misc, and old kernels. The stub now works on
Linux kernels as old as 2.6.21 (and possibly older), and it now
uses nanoprintf to further reduce binary size.
* The FUSE driver will now show the name of the mounted file
system image in the mount point listing (e.g., in df or mount
output).
Compatibility:
* The accepted minor version for the DwarFS image format has been
incremented. Release v0.16.0 will also increment the written
minor version. This means images produced with v0.16.0 will not
be readable by DwarFS tools prior to v0.14.0.
See the “Features” section in dwarfs-format(7) for details.
* The (no_)cache_image option has been removed from the FUSE
driver.
Build:
* Removed the hard dependency on the date library, which caused
build issues on distributions that no longer bundle it
(e.g., openSUSE).
- Drop remove_hhdate_dependency.patch
- Drop folly-remove-boost_system-dependency.patch
-------------------------------------------------------------------
Fri Oct 3 13:38:35 UTC 2025 - Filippo Bonazzi <filippo.bonazzi@suse.com>
- Remove hhdate dependency
- Add remove_hhdate_dependency.patch: replace date library usage with
C++20 std::chrono
- Add %check section and run tests
- Add test dependencies gtest and gmock
-------------------------------------------------------------------
Thu Oct 2 17:25:56 UTC 2025 - Mia Herkt <mia@0x0.st>
- Remove libboost_system-devel from BuildRequires
- Add folly-remove-boost_system-dependency.patch
Fixes build with Boost >=1.89.0
gh#mhx/dwarfs#288
gh#facebook/folly#2489
-------------------------------------------------------------------
Tue Sep 2 22:55:02 UTC 2025 - Mia Herkt <mia@0x0.st>
- Update to version 0.13.0:
Bugfixes:
* Made section index discovery more robust.
gh#mhx/dwarfs#264
* A recent kernel change (https://lkml.org/lkml/2025/5/5/2868)
caused the tools_test to fail on Linux 6.14 and later.
This has been fixed by accepting both EPERM and ENOSYS as
valid error codes for link() calls.
Features:
* Support for big-endian architectures.
This is still experimental, even though all unit tests pass
with QEMU, and the benchmark suite runs fine on real hardware.
This currently requires forked versions of folly and fsst.
The changes are small and the pull requests will hopefully be
merged upstream soon.
* Experimental support for 32-bit architectures.
While DwarFS should mostly "just work" on 32-bit when using
small images (a few hundred megabytes), the limited address
space is a problem for the extensive use of memory-mapped
files inside DwarFS. There will be changes to limit the use of
mmap in the future (mainly due to other issues), which should
help 32-bit compatibility as a side-effect.
gh#mhx/dwarfs#268.
* The category metadata for categorized blocks is now stored in
the metadata block by default. This allows re-compressing the
blocks with a metadata-dependent algorithm (e.g. FLAC) even if
they were previously compressed using a metadata-independent
algorithm.
This can be disabled using the --no-category-metadata option.
See the mkdwarfs man page for more details.
* The --no-category-names and --no-category-metadata options can
be used to reduce the size of the metadata. However, this will
make it impossible to use metadata-dependent compression
algorithms (e.g. FLAC), or even select category-specific
compression, when recompressing the image.
* Metadata rebuilding is now supported in mkdwarfs using the
--rebuild-metadata option. Previously, the metadata could only
be recompressed, but it was impossible to change it. With the
new option, it is now possible to change metadata packing and
apply operations like --set-owner, --set-group, --set-time,
--time-resolution, --chmod, or --no-create-timestamp.
Note that these are potentially lossy operations that may be
irreversible. By default, the history of metadata rebuilds is
tracked in the metadata itself, but this can be disabled using
--no-metadata-version-history.
* In addition to metadata rebuilding, it is now also possible to
change the block size of an existing image using the
--change-block-size option. This implies --rebuild-metadata
and --recompress=all. This can be useful for tuning the
performance of an existing image without having to re-create
it from scratch.
* mkdwarfs now shows its current memory usage while running.
Note that -L/--memory-limit still only limits the memory used
for the block queue, not the overall memory usage. Fixing this
is on the roadmap, there's no need to file an issue.
* dwarfsextract has new options to control the output format:
--format-options and --format-filters. There is also
--format=auto to automatically "guess" the format and filters
based on the output file name.
* dwarfsck has a new frozen_details detail level that will show
the frozen_analysis content ordered by memory location instead
of memory usage and also shows the address range of each
section.
-------------------------------------------------------------------
Sun Aug 24 12:05:51 UTC 2025 - Jan Engelhardt <jengelh@inai.de>
- Replace wrong BuildRequires pkgconfig(clzma) -> pkgconfig(liblzma);
build only succeeded previously by accident.
-------------------------------------------------------------------
Sat Jun 21 12:19:45 UTC 2025 - Mia Herkt <mia@0x0.st>
- Update to version 0.12.4:
Bugfixes
* Segfault on bad_compression_ratio_error. When recompressing a
filesystem where some blocks cannot be compressed using the
selected algorithm because of a bad_compression_ratio_error,
the resulting block was left empty.
* Add history unless --no-history is given when rewriting a file
system image.
* Allow dumping frozen_layout w/o frozen_analysis in dwarfsck.
* Logging timestamps should show local time.
Features
* More complete breakdown of metadata in dwarfsck.
* Add schema_raw_dump flag to dwarfsck --detail.
Build
* Update folly/fbthrift/fsst.
-------------------------------------------------------------------
Mon Apr 21 19:50:12 UTC 2025 - Mia Herkt <mia@0x0.st>
- Update to version 0.12.3:
Bugfixes
* Automatic image offset detection (for images using a custom
header) did not work correctly if the header contained a
string that would be identified as the start of a v1 section
header (these were only used before dwarfs-0.3.0).
If there was either "DWARFS\x02\x00" or "DWARFS\x02\x01" in
the header, offset detection would fail. The check has been
modified to peek further into the data and ensure this really
is a v1 section header, and also checking if the next section
header position can be derived from the length field.
It is still possible to construct a file system image where
offset detection will ultimately fail, but it is much less
likely with the change.
- Changes in version 0.12.2:
Bugfixes
*The dwarfs-0.12.0 release introduced a performance regression
where FLAC compression took more than twice as long as in the
previous releases. This has been fixed. FLAC decompression was
unaffected.
-------------------------------------------------------------------
Sun Apr 13 19:46:34 UTC 2025 - Mia Herkt <mia@0x0.st>
- Update to version 0.12.1:
Features
* Added --memory-limit=auto to mkdwarfs to use a more reasonable
(hopefully) default for the block queue. The old default of
1 GiB was quite arbitrary and definitely not suitable for
low-end systems. The new auto default will determine the limit
based on the number of workers (which in turn is based on the
number of CPUs), the block size, and the amount of physical
memory of the system.
* Replaced vector_byte_buffer with malloc_byte_buffer, which is
internally based around a simple buffer that doesn't incur the
cost of initializing each element like std::vector. Especially
for large blocks which are known to be overwritten immediately,
this can save a few CPU cycles.
- Changes in version 0.12.0:
* New Licensing Conditions: Instead of being all GPL-3.0 like all
the previous releases, this release changes the license of a
large fraction of the DwarFS code to MIT. All tools and
libraries that only read DwarFS images are now MIT-licensed.
Everything else (e.g. mkdwarfs) is still GPL-3.0 for the time
being.
Bugfixes
* Changes for compatibility with Boost.Process v2.
Features
* Re-licensed all libraries required for reading DwarFS images
under the MIT license. The source of all tools that just read
DwarFS images (i.e. everything except for mkdwarfs) are also
under the MIT license now. Everything else is still GPL-3.0.
gh#mhx/dwarfs#255
* New hotness categorizer in mkdwarfs that allows a list of "hot"
files to be stored in distinct file system blocks.
* New explicit ordering mode in mkdwarfs that allows files to be
ordered accoring to the order in a given list file.
* dwarfs now shows the version of the FUSE library used.
* New dwarfs options preload_all and preload_category to populate
the block cache immediately after mounting.
* New dwarfs option analysis_file that can be used for profiling
and as input to mkdwarfs new hotness categorizer and explicit
ordering mode.
* New dwarfs option block_allocator that allows the user to
switch from a malloc-based block allocator to an mmap-based
one. This can help with returning memory back to the system if
the blocks are evicted from the cache.
-------------------------------------------------------------------
Fri Apr 4 07:31:03 UTC 2025 - Jan Engelhardt <jengelh@inai.de>
- Use SRPM base name for devel subpackage
-------------------------------------------------------------------
Tue Apr 1 03:25:25 UTC 2025 - Mia Herkt <mia@0x0.st>
- Initial package, version 0.11.3
