Interface

class datalake.interface.IMonitor

The monitoring abstract class defines methods for sending metrics to a Time Series Database

abstract push(measurement)

Sends a measurement to the TSDB backend

Parameters

measurement (Measurement) – the measurement to send

safe_push(measurement)

Same as push() but with all exceptions trapped hence not disrupting the main program

class datalake.interface.ISecret

The secret interface defines methods for fetching a cloud managed secret

abstract property json

Returns the secret as a dict

abstract property plain

Returns the secret in plain UTF-8 text

class datalake.interface.IStorage

The storage interface defines method for manipulating files in a cloud object/blob storage

abstract checksum(key)

Returns the SHA256 hash for the file at the specified path

Parameters

key (str) – the path of a storage object

abstract copy(src, dst, bucket=None)

Copies a storage key to another key in the same storage or another

Parameters

• src (str) – the path of a storage object to copy

• dst (str) – the path of the target storage object

• bucket (str) – if specified then the object is copied in this bucket. If None the object is copied in the same bucket than the source object

abstract delete(key)

Removes an object from the storage

Parameters

key (str) – the path of a storage object

abstract download(src, dst)

Downloads a storage file locally

Parameters

• src (str) – the path of a storage object

• dst (str) – the path of the target local file

abstract exists(key)

Returns True if the specified key exists, False otherwise

Parameters

key (str) – the path of a storage object

abstract get(key)

Returns the content of the specified object

Parameters

key (str) – the path of a storage object

abstract is_folder(key)

Return True if the specified key is a folder-like object, False otherwise

Parameters

key (str) – the path of a storage object

abstract keys_iterator(prefix)

Returns an iterator of keys that match the specified prefix

Parameters

prefix (str) – a partial path of a storage object

abstract move(src, dst, bucket=None)

Moves a storage key to another key in the same storage or another

Parameters

• src (str) – the path of a storage object to move

• dst (str) – the path of the target storage object

• bucket (str) – if specified then the object is moved in this bucket. If None the object is moved in the same bucket than the source object

abstract property name

Returns the name of the storage

abstract put(content, dst, content_type='text/csv', encoding='utf-8', metadata={})

Writes the specified content in a storage object

Parameters

• content (str) – the content to upload

• dst (str) – the path of the target storage object

• content_type (str) – the MIME type for the file

• encoding (str) – the encoding for the file

• metadata (dict) – a map of key/value pairs to add as metadata for the uploaded file

abstract size(key)

Return the size in bytes for the specified object

Parameters

key (str) – the path of a storage object

abstract stream(key, encoding='utf-8')

Returns an iterator on each lines for the specified object

Parameters

• key (str) – the path of a storage object

• encoding (str) – the encoding to use for decoding the byte stream

abstract upload(src, dst, content_type='text/csv', encoding='utf-8', metadata={})

Uploads a local file to the storage

Parameters

• src (str) – the path of a local file

• dst (str) – the path of the target storage object

• content_type (str) – the MIME type for the file

• encoding (str) – the encoding for the file

• metadata (dict) – a map of key/value pairs to add as metadata for the uploaded file

class datalake.interface.IStorageEvent

The storage interface defines methods for handling event notifications from buckets

abstract process(storage, object)

Callback method for handling an object creation

Parameters

• storage (IStorage) – the concrete implementation of the Storage interface

• object (str) – the path of the created object