Skip to content
module

panpath.gs_client

</>

Google Cloud Storage client implementation.

Classes
  • GSClient — Synchronous Google Cloud Storage client implementation.</>
  • GSSyncFileHandle — Sync file handle for GCS with chunked streaming support.</>
class

panpath.gs_client.GSClient(**kwargs)

</>
Bases
panpath.clients.SyncClient panpath.clients.Client

Synchronous Google Cloud Storage client implementation.

Methods
  • copy(source, target, follow_symlinks) — Copy file to target.</>
  • copytree(source, target, follow_symlinks) — Copy directory tree to target recursively.</>
  • delete(path) — Delete GCS blob.</>
  • exists(path) (bool) — Check if GCS blob exists.</>
  • get_metadata(path) (dict) — Get blob metadata.</>
  • glob(path, pattern) (Iterator) — Glob for files matching pattern.</>
  • is_dir(path) (bool) — Check if GCS path is a directory (has blobs with prefix).</>
  • is_file(path) (bool) — Check if GCS path is a file.</>
  • is_symlink(path) (bool) — Check if blob is a symlink (has symlink_target metadata).</>
  • list_dir(path) (list) — List GCS blobs with prefix.</>
  • mkdir(path, parents, exist_ok) — Create a directory marker (empty blob with trailing slash).</>
  • open(path, mode, encoding, **kwargs) (Any) — Open GCS blob for reading/writing with streaming support.</>
  • read_bytes(path) (bytes) — Read GCS blob as bytes.</>
  • read_text(path, encoding) (str) — Read file as text.</>
  • readlink(path) (str) — Read symlink target from metadata.</>
  • rename(source, target) — Rename/move file.</>
  • rmdir(path) — Remove directory marker.</>
  • rmtree(path, ignore_errors, onerror) — Remove directory and all its contents recursively.</>
  • set_metadata(path, metadata) — Set blob metadata.</>
  • stat(path) (Any) — Get GCS blob metadata.</>
  • symlink_to(path, target) — Create symlink by storing target in metadata.</>
  • touch(path, exist_ok) — Create empty file.</>
  • walk(path) (Iterator) — Walk directory tree.</>
  • write_bytes(path, data) — Write bytes to GCS blob.</>
  • write_text(path, data, encoding) — Write text to file.</>
method

read_text(path, encoding='utf-8') → str

</>

Read file as text.

method

write_text(path, data, encoding='utf-8')

</>

Write text to file.

method

exists(path) → bool

</>

Check if GCS blob exists.

method

read_bytes(path) → bytes

</>

Read GCS blob as bytes.

method

write_bytes(path, data)

</>

Write bytes to GCS blob.

method

delete(path)

</>

Delete GCS blob.

method

list_dir(path) → list

</>

List GCS blobs with prefix.

method

is_dir(path) → bool

</>

Check if GCS path is a directory (has blobs with prefix).

method

is_file(path) → bool

</>

Check if GCS path is a file.

method

stat(path) → Any

</>

Get GCS blob metadata.

method

open(path, mode='r', encoding=None, **kwargs)

</>

Open GCS blob for reading/writing with streaming support.

Parameters
  • path (str) — GCS path (gs://bucket/blob)
  • mode (str, optional) — File mode ('r', 'w', 'rb', 'wb', 'a', 'ab')
  • encoding (Optional, optional) — Text encoding (for text modes)
  • **kwargs (Any) — Additional arguments (chunk_size, upload_warning_threshold,upload_interval supported)
Returns (Any)

GSSyncFileHandle with streaming support

method

mkdir(path, parents=False, exist_ok=False)

</>

Create a directory marker (empty blob with trailing slash).

Parameters
  • path (str) — GCS path (gs://bucket/path)
  • parents (bool, optional) — If True, create parent directories as needed (ignored for GCS)
  • exist_ok (bool, optional) — If True, don't raise error if directory already exists
method

get_metadata(path)

</>

Get blob metadata.

Parameters
  • path (str) — GCS path
Returns (dict)

Dictionary of metadata key-value pairs

method

set_metadata(path, metadata)

</>

Set blob metadata.

Parameters
  • path (str) — GCS path
  • metadata (dict) — Dictionary of metadata key-value pairs
generator

glob(path, pattern)

</>

Glob for files matching pattern.

Parameters
  • path (str) — Base GCS path
  • pattern (str) — Glob pattern (e.g., ".txt", "**/.py")
Returns (Iterator)

List of matching paths (as PanPath objects or strings)

generator

walk(path)

</>

Walk directory tree.

Parameters
  • path (str) — Base GCS path
Returns (Iterator)

List of (dirpath, dirnames, filenames) tuples

method

touch(path, exist_ok=True)

</>

Create empty file.

Parameters
  • path (str) — GCS path
  • exist_ok (bool, optional) — If False, raise error if file exists
method

rename(source, target)

</>

Rename/move file.

Parameters
  • source (str) — Source GCS path
  • target (str) — Target GCS path
method

rmdir(path)

</>

Remove directory marker.

Parameters
  • path (str) — GCS path
method

rmtree(path, ignore_errors=False, onerror=None)

</>

Remove directory and all its contents recursively.

Parameters
  • path (str) — GCS path
  • ignore_errors (bool, optional) — If True, errors are ignored
  • onerror (Optional, optional) — Callable that accepts (function, path, excinfo)
method

copy(source, target, follow_symlinks=True)

</>

Copy file to target.

Parameters
  • source (str) — Source GCS path
  • target (str) — Target GCS path
  • follow_symlinks (bool, optional) — If False, symlinks are copied as symlinks (not dereferenced)
method

copytree(source, target, follow_symlinks=True)

</>

Copy directory tree to target recursively.

Parameters
  • source (str) — Source GCS path
  • target (str) — Target GCS path
  • follow_symlinks (bool, optional) — If False, symlinks are copied as symlinks (not dereferenced)
class

panpath.gs_client.GSSyncFileHandle(*args, **kwargs)

</>
Bases
panpath.clients.SyncFileHandle

Sync file handle for GCS with chunked streaming support.

Uses google-cloud-storage's streaming API for efficient reading of large files.

Attributes
  • closed (bool) — Check if file is closed.</>
Methods
  • __del__() — Destructor to ensure stream is closed.</>
  • __enter__() (SyncFileHandle) — Enter context manager.</>
  • __exit__(exc_type, exc_val, exc_tb) — Exit async context manager.</>
  • __iter__() (SyncFileHandle) — Support async iteration over lines.</>
  • __next__() (Union) — Get next line in async iteration.</>
  • close() — Close the file and flush write buffer to cloud storage.</>
  • flush() — Flush write buffer to cloud storage.</>
  • read(size) (Union) — Read and return up to size bytes/characters.</>
  • readline(size) (Union) — Read and return one line from the file.</>
  • readlines() (List) — Read and return all lines from the file.</>
  • reset_stream() — Reset streaming reader/writer.</>
  • seek(offset, whence) (int) — Change stream position (forward seeking only).</>
  • tell() (int) — Return current stream position.</>
  • write(data) (int) — Write data to the file.</>
  • writelines(lines) — Write a list of lines to the file.</>
method

flush()

</>

Flush write buffer to cloud storage.

After open, all flushes append to existing content using provider-native append operations. The difference between 'w' and 'a' modes is that 'w' clears existing content on open, while 'a' preserves it.

method

__enter__() → SyncFileHandle

</>

Enter context manager.

method

__exit__(exc_type, exc_val, exc_tb)

</>

Exit async context manager.

method

read(size=-1)

</>

Read and return up to size bytes/characters.

Parameters
  • size (int, optional) — Number of bytes/chars to read (-1 for all)
Returns (Union)

Data read from file

method

readline(size=-1) → Union

</>

Read and return one line from the file.

method

readlines() → List

</>

Read and return all lines from the file.

method

write(data) → int

</>

Write data to the file.

method

writelines(lines)

</>

Write a list of lines to the file.

method

close()

</>

Close the file and flush write buffer to cloud storage.

method

__iter__() → SyncFileHandle

</>

Support async iteration over lines.

method

__next__() → Union

</>

Get next line in async iteration.

method

tell()

</>

Return current stream position.

Returns (int)

Current position in the file

method

seek(offset, whence=0)

</>

Change stream position (forward seeking only).

Parameters
  • offset (int) — Position offset
  • whence (int, optional) — Reference point (0=start, 1=current, 2=end)
Returns (int)

New absolute position

Raises
  • OSError — If backward seeking is attempted
  • ValueError — If called in write mode or on closed file

Note

  • Only forward seeking is supported due to streaming limitations
  • SEEK_END (whence=2) is not supported as blob size may be unknown
  • Backward seeking requires re-opening the stream
method

reset_stream()

</>

Reset streaming reader/writer.

method

__del__()

</>

Destructor to ensure stream is closed.