datawork.instances.data

Instances of Data for common data payloads.

class datawork.instances.data.FileData(desc=None, name=None)

Base class for any disk-native data.

For example, SQLiteData will use this as a base class.

static check_type(value)

Check that value is a filename.

read(filename)

Read by setting the filename.

static serialize(data)

Simply return the filename.

write(filename)

Copy file to new location.

class datawork.instances.data.JSONData(desc=None, name=None)

A Data class for primitive JSON serializable types.

The so-called “primitive types” in JSON are:

• string
• numeric types
• object (in python this is a dict)
• array
• boolean
• null

In this class, hierarchies of the following types are supported:

• bool
• dict
• float
• int
• list
• None
• str

Note that although other types than these may be serializable in Python (by subclassing json.JSONEncoder), the primitive types can be serialized/deserialized unambiguously. For example, we do not support tuples, although the json module supports serializing them by casting them to lists.

static check_type(value)

Check that value is a hierarchy of primitive JSON types.

read(filename)

Read JSON text.

static serialize(data)

Convert to JSON text.

write(filename)

Write as JSON text.

class datawork.instances.data.KerasModelData(desc=None, name=None)

A Data class for Keras models.

static check_type(value)

Check that value is a keras.models.Model.

read(filename)

Read from HDF5.

write(filename)

Write to HDF5.

class datawork.instances.data.PandasData(*args, **kwargs)

Data type for Pandas DataFrames and Series.

__init__(*args, **kwargs)

Construct PandasData.

static check_type(value)

Check that value is a DataFrame or Series.

read(filename)

Read from msgpack.

static serialize(data)

Write to msgpack.

write(filename)

Write msgpack.

class datawork.instances.data.TorchModelData(desc=None, name=None)

A Data class for PyTorch models.

static check_type(value)

Check that value is a torch.nn.Module.

read(filename)

Load state dict and module class.

write(filename)

Write state dict and serialize module class.