Skip to content

PanPath

🌍 Universal sync/async local/cloud path library

A pathlib-compatible interface for Python that works seamlessly across local and cloud storage

GitHub Stars PyPI version Python versions License

✨ Features

  • Unified Interface

    Single API for local and cloud storage (S3, Google Cloud Storage, Azure Blob Storage)

  • Sync & Async

    Choose synchronous or asynchronous operations based on your needs

  • Pathlib Compatible

    Drop-in replacement for pathlib.Path for local files

  • Lazy Loading

    Cloud clients instantiated only when needed for better performance

  • Cross-Storage Operations

    Copy/move files between different storage backends seamlessly

  • Bulk Operations

    Efficient rmtree, copy, copytree for directories

  • Testable

    Local mock infrastructure for testing without cloud resources

  • Optional Dependencies

    Install only what you need - minimal core with optional cloud backends

🚀 Quick Example

from panpath import PanPath

# Local files (pathlib.Path compatible)
local = PanPath("/path/to/file.txt")
content = local.read_text()

# S3 (synchronous)
s3_file = PanPath("s3://bucket/key/file.txt")
content = s3_file.read_text()

# Google Cloud Storage
gs_file = PanPath("gs://bucket/path/file.txt")
content = gs_file.read_text()

# Azure Blob Storage
azure_file = PanPath("az://container/path/file.txt")
content = azure_file.read_text()

from panpath import PanPath

# Same class, async methods with a_ prefix
s3 = PanPath("s3://bucket/file.txt")
content = await s3.a_read_text()

# Works for all storage types
local = PanPath("/path/to/file.txt")
content = await local.a_read_text()

from panpath import PanPath

# Copy from S3 to GCS
s3_file = PanPath("s3://my-bucket/data.csv")
s3_file.copy("gs://other-bucket/data.csv")

# Copy entire directory from cloud to local
cloud_dir = PanPath("s3://data-lake/dataset/")
cloud_dir.copytree("/tmp/dataset/")

# Move between cloud providers
azure_file = PanPath("az://container/file.txt")
azure_file.rename("s3://bucket/file.txt")

📦 Installation

Install the core library:

pip install panpath

With cloud storage support:

pip install panpath[s3]        # Sync
pip install panpath[async-s3]  # Async

pip install panpath[gs]        # Sync
pip install panpath[async-gs]  # Async

pip install panpath[azure]        # Sync
pip install panpath[async-azure]  # Async

pip install panpath[all-sync]   # All sync backends
pip install panpath[all-async]  # All async backends
pip install panpath[all]        # Everything

🎯 Use Cases

Local Development → Cloud Production: Write code using local paths during development, switch to cloud paths in production with minimal changes.

Multi-Cloud Applications: Build applications that work with multiple cloud providers without vendor lock-in.

Data Pipelines: Create ETL pipelines that seamlessly move data between local storage and cloud services.

Async I/O: Leverage async/await for high-performance cloud operations in async frameworks like FastAPI, aiohttp, or asyncio scripts.

Testing: Use local paths in tests, cloud paths in production - same code, different backends.

📚 Documentation Structure

🤝 Contributing

Contributions are welcome! Please see our Contributing Guide for details.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

PanPath is inspired by:

  • pathlib - Python's standard library for filesystem paths
  • cloudpathlib - Path-like interface for cloud storage

Made with ❤️ by the PanPath contributors

GitHub • PyPI • Issues