GitHub - xixu-me/tzst: The next-generation Python library engineered for modern archive management, leveraging cutting-edge Zstandard compression to deliver superior performance, security, and reliability

tzst is a next-generation Python library engineered for modern archive management, leveraging cutting-edge Zstandard compression to deliver superior performance, security, and reliability. Built exclusively for Python 3.12+, this enterprise-grade solution combines atomic operations, streaming efficiency, and a meticulously crafted API to redefine how developers handle .tzst/.tar.zst archives in production environments. 🚀

✨ Features

🗜️ High Compression: Zstandard compression for excellent compression ratios and speed
📁 Tar Compatibility: Creates standard tar archives compressed with Zstandard
💻 Command Line Interface: Intuitive CLI with streaming support and comprehensive options
🐍 Python API: Clean, Pythonic API for programmatic use
🌍 Cross-Platform: Works on Windows, macOS, and Linux
📂 Multiple Extensions: Supports both .tzst and .tar.zst extensions
💾 Memory Efficient: Streaming mode for handling large archives with minimal memory usage
⚡ Atomic Operations: Safe file operations with automatic cleanup on interruption
🔒 Secure by Default: Uses the 'data' filter for maximum security during extraction
🚨 Enhanced Error Handling: Clear error messages with helpful alternatives

📥 Installation

From GitHub Releases

Download standalone executables that don't require Python installation:

Supported Platforms

Platform	Architecture	File
🐧 Linux	x86_64	`tzst-{version}-linux-amd64.zip`
🐧 Linux	ARM64	`tzst-{version}-linux-arm64.zip`
🪟 Windows	x64	`tzst-{version}-windows-amd64.zip`
🪟 Windows	ARM64	`tzst-{version}-windows-arm64.zip`
🍎 macOS	Intel	`tzst-{version}-darwin-amd64.zip`
🍎 macOS	Apple Silicon	`tzst-{version}-darwin-arm64.zip`

🛠️ Installation Steps

📥 Download the appropriate archive for your platform from the latest releases page
📦 Extract the archive to get the tzst executable (or tzst.exe on Windows)
📂 Move the executable to a directory in your PATH:
- 🐧 Linux/macOS: sudo mv tzst /usr/local/bin/
- 🪟 Windows: Add the directory containing tzst.exe to your PATH environment variable
✅ Verify installation: tzst --help

🎯 Benefits of Binary Installation

✅ No Python required - Standalone executable
✅ Faster startup - No Python interpreter overhead
✅ Easy deployment - Single file distribution
✅ Consistent behavior - Bundled dependencies

📦 From PyPI

pip install tzst

🔧 From Source

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install .

🚀 Development Installation

This project uses modern Python packaging standards:

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install -e .[dev]

🚀 Quick Start

💻 Command Line Usage

Note: Download the standalone binary for the best performance and no Python dependency. Alternatively, use uvx tzst for running without installation. See uv documentation for details.

# 📁 Create an archive
tzst a archive.tzst file1.txt file2.txt directory/

# 📤 Extract an archive
tzst x archive.tzst

# 📋 List archive contents
tzst l archive.tzst

# 🧪 Test archive integrity
tzst t archive.tzst

🐍 Python API Usage

from tzst import create_archive, extract_archive, list_archive

# Create an archive
create_archive("archive.tzst", ["file1.txt", "file2.txt", "directory/"])

# Extract an archive
extract_archive("archive.tzst", "output_directory/")

# List archive contents
contents = list_archive("archive.tzst", verbose=True)
for item in contents:
    print(f"{item['name']}: {item['size']} bytes")

💻 Command Line Interface

📁 Archive Operations

➕ Create Archive

# Basic usage
tzst a archive.tzst file1.txt file2.txt

# With compression level (1-22, default: 3)
tzst a archive.tzst files/ -l 15

# Alternative commands
tzst add archive.tzst files/
tzst create archive.tzst files/

📤 Extract Archive

# Extract with full directory structure
tzst x archive.tzst

# Extract to specific directory
tzst x archive.tzst -o output/

# Extract specific files
tzst x archive.tzst file1.txt dir/file2.txt

# Extract without directory structure (flat)
tzst e archive.tzst -o output/

# Use streaming mode for large archives
tzst x archive.tzst --streaming -o output/

📋 List Contents

# Simple listing
tzst l archive.tzst

# Verbose listing with details
tzst l archive.tzst -v

# Use streaming mode for large archives
tzst l archive.tzst --streaming -v

🧪 Test Integrity

# Test archive integrity
tzst t archive.tzst

# Test with streaming mode
tzst t archive.tzst --streaming

📊 Command Reference

Command	Aliases	Description	Streaming Support
`a`	`add`, `create`	Create or add to archive	N/A
`x`	`extract`	Extract with full paths	✓ `--streaming`
`e`	`extract-flat`	Extract without directory structure	✓ `--streaming`
`l`	`list`	List archive contents	✓ `--streaming`
`t`	`test`	Test archive integrity	✓ `--streaming`

⚙️ CLI Options

-v, --verbose: Enable verbose output
-o, --output DIR: Specify output directory (extract commands)
-l, --level LEVEL: Set compression level 1-22 (create command)
--streaming: Enable streaming mode for memory-efficient processing
--filter FILTER: Security filter for extraction (data/tar/fully_trusted)
--no-atomic: Disable atomic file operations (not recommended)

🔒 Security Filters

# Extract with maximum security (default)
tzst x archive.tzst --filter data

# Extract with standard tar compatibility
tzst x archive.tzst --filter tar

# Extract with full trust (dangerous - only for trusted archives)
tzst x archive.tzst --filter fully_trusted

🔐 Security Filter Options:

data (default): Most secure. Blocks dangerous files, absolute paths, and paths outside extraction directory
tar: Standard tar compatibility. Blocks absolute paths and directory traversal
fully_trusted: No security restrictions. Only use with completely trusted archives

🐍 Python API

📦 TzstArchive Class

from tzst import TzstArchive

# Create a new archive
with TzstArchive("archive.tzst", "w", compression_level=5) as archive:
    archive.add("file.txt")
    archive.add("directory/", recursive=True)

# Read an existing archive
with TzstArchive("archive.tzst", "r") as archive:
    # List contents
    contents = archive.list(verbose=True)
    
    # Extract with security filter
    archive.extract("file.txt", "output/", filter="data")
    
    # Test integrity
    is_valid = archive.test()

# For large archives, use streaming mode
with TzstArchive("large_archive.tzst", "r", streaming=True) as archive:
    archive.extract(path="output/")

⚠️ Important Limitations:

❌ Append Mode Not Supported: Create multiple archives or recreate the entire archive instead

🎯 Convenience Functions

📁 create_archive()

from tzst import create_archive

# Create with atomic operations (default)
create_archive(
    archive_path="backup.tzst",
    files=["documents/", "photos/", "config.txt"],
    compression_level=10
)

📤 extract_archive()

from tzst import extract_archive

# Extract with security (default: 'data' filter)
extract_archive("backup.tzst", "restore/")

# Extract specific files
extract_archive("backup.tzst", "restore/", members=["config.txt"])

# Flatten directory structure
extract_archive("backup.tzst", "restore/", flatten=True)

# Use streaming for large archives
extract_archive("large_backup.tzst", "restore/", streaming=True)

📋 list_archive()

from tzst import list_archive

# Simple listing
files = list_archive("backup.tzst")

# Detailed listing
files = list_archive("backup.tzst", verbose=True)

# Streaming for large archives
files = list_archive("large_backup.tzst", streaming=True)

🧪 test_archive()

from tzst import test_archive

# Basic integrity test
if test_archive("backup.tzst"):
    print("Archive is valid")

# Test with streaming
if test_archive("large_backup.tzst", streaming=True):
    print("Large archive is valid")

🔧 Advanced Features

📂 File Extensions

The library automatically handles file extensions with intelligent normalization:

.tzst - Primary extension for tar+zstandard archives
.tar.zst - Alternative standard extension
Auto-detection when opening existing archives
Automatic extension addition when creating archives

# These all create valid archives
create_archive("backup.tzst", files)      # Creates backup.tzst
create_archive("backup.tar.zst", files)  # Creates backup.tar.zst  
create_archive("backup", files)          # Creates backup.tzst
create_archive("backup.txt", files)      # Creates backup.tzst (normalized)

🗜️ Compression Levels

Zstandard compression levels range from 1 (fastest) to 22 (best compression):

Level 1-3: Fast compression, larger files
Level 3 (default): Good balance of speed and compression
Level 10-15: Better compression, slower
Level 20-22: Maximum compression, much slower

🌊 Streaming Mode

Use streaming mode for memory-efficient processing of large archives:

✅ Benefits:

Significantly reduced memory usage
Better performance for archives that don't fit in memory
Automatic cleanup of resources

🎯 When to Use:

Archives larger than 100MB
Limited memory environments
Processing archives with many large files

# Example: Processing a large backup archive
from tzst import extract_archive, list_archive, test_archive

large_archive = "backup_500gb.tzst"

# Memory-efficient operations
is_valid = test_archive(large_archive, streaming=True)
contents = list_archive(large_archive, streaming=True, verbose=True)
extract_archive(large_archive, "restore/", streaming=True)

⚡ Atomic Operations

All file creation operations use atomic file operations by default:

Archives created in temporary files first, then atomically moved
Automatic cleanup if process is interrupted
No risk of corrupted or incomplete archives
Cross-platform compatibility

# Atomic operations enabled by default
create_archive("important.tzst", files)  # Safe from interruption

# Can be disabled if needed (not recommended)
create_archive("test.tzst", files, use_temp_file=False)

🚨 Error Handling

from tzst import TzstArchive
from tzst.exceptions import (
    TzstError,
    TzstArchiveError,
    TzstCompressionError,
    TzstDecompressionError,
    TzstFileNotFoundError
)

try:
    with TzstArchive("archive.tzst", "r") as archive:
        archive.extract()
except TzstDecompressionError:
    print("Failed to decompress archive")
except TzstFileNotFoundError:
    print("Archive file not found")
except KeyboardInterrupt:
    print("Operation interrupted by user")
    # Cleanup handled automatically

🚀 Performance and Comparison

💡 Performance Tips

🗜️ Compression levels: Level 3 is optimal for most use cases
🌊 Streaming: Use for archives larger than 100MB
📦 Batch operations: Add multiple files in single session
📄 File types: Already compressed files won't compress much further

🆚 vs Other Tools

vs tar + gzip:

✅ Better compression ratios
⚡ Faster decompression
🔄 Modern algorithm

vs tar + xz:

🚀 Significantly faster compression
📊 Similar compression ratios
⚖️ Better speed/compression trade-off

vs zip:

🗜️ Better compression
🔐 Preserves Unix permissions and metadata
🌊 Better streaming support

📋 Requirements

🐍 Python 3.12 or higher
📦 zstandard >= 0.19.0

🛠️ Development

🚀 Setting up Development Environment

This project uses modern Python packaging standards:

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install -e .[dev]

🧪 Running Tests

# Run tests with coverage
pytest --cov=tzst --cov-report=html

# Or use the simpler command (coverage settings are in pyproject.toml)
pytest

✨ Code Quality

# Check code quality
ruff check src tests

# Format code
ruff format src tests

🤝 Contributing

We welcome contributions! Please read our Contributing Guide for:

Development setup and project structure
Code style guidelines and best practices
Testing requirements and writing tests
Pull request process and review workflow

🚀 Quick Start for Contributors

git clone https://github.com/xixu-me/tzst.git
cd tzst
pip install -e .[dev]
python -m pytest tests/

🎯 Types of Contributions Welcome

🐛 Bug fixes - Fix issues in existing functionality
✨ Features - Add new capabilities to the library
📚 Documentation - Improve or add documentation
🧪 Tests - Add or improve test coverage
⚡ Performance - Optimize existing code
🔒 Security - Address security vulnerabilities

🙏 Acknowledgments

Meta Zstandard for the excellent compression algorithm
python-zstandard for Python bindings
The Python community for inspiration and feedback

📄 License

Licensed under the BSD 3-Clause license.

Name		Name	Last commit message	Last commit date
Latest commit History 178 Commits
.github		.github
docs		docs
src		src
tests		tests
.codecov.yml		.codecov.yml
.gitignore		.gitignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.ar.md		README.ar.md
README.de.md		README.de.md
README.es.md		README.es.md
README.fr.md		README.fr.md
README.ja.md		README.ja.md
README.ko.md		README.ko.md
README.md		README.md
README.pt.md		README.pt.md
README.ru.md		README.ru.md
README.zh.md		README.zh.md
SECURITY.md		SECURITY.md
pyproject.toml		pyproject.toml

Uh oh!

License

xixu-me/tzst

Folders and files

Latest commit

History

Repository files navigation

✨ Features

📥 Installation

From GitHub Releases

Supported Platforms

🛠️ Installation Steps

🎯 Benefits of Binary Installation

📦 From PyPI

🔧 From Source

🚀 Development Installation

🚀 Quick Start

💻 Command Line Usage

🐍 Python API Usage

💻 Command Line Interface

📁 Archive Operations

➕ Create Archive

📤 Extract Archive

📋 List Contents

🧪 Test Integrity

📊 Command Reference

⚙️ CLI Options

🔒 Security Filters

🐍 Python API

📦 TzstArchive Class

🎯 Convenience Functions

📁 create_archive()

📤 extract_archive()

📋 list_archive()

🧪 test_archive()

🔧 Advanced Features

📂 File Extensions

🗜️ Compression Levels

🌊 Streaming Mode

⚡ Atomic Operations

🚨 Error Handling

🚀 Performance and Comparison

💡 Performance Tips

🆚 vs Other Tools

📋 Requirements

🛠️ Development

🚀 Setting up Development Environment

🧪 Running Tests

✨ Code Quality

🤝 Contributing

🚀 Quick Start for Contributors

🎯 Types of Contributions Welcome

🙏 Acknowledgments

📄 License

About

Topics

Resources

License

Code of conduct

Security policy

Uh oh!

Stars

Watchers

Forks

Releases 10

Sponsor this project

Uh oh!

Uh oh!

Contributors 4

Uh oh!

Languages