dtypes
This module provides custom data types for Spotlight dataset.
Classes
Embedding(data, dtype=None)
Data sample projected onto a new space.
Attributes
data
: 1-dimensional array-like. Sample embedding.
dtype
: Optional data type of embedding. If None
, data type inferred
from data.
Example
>>> import numpy as np
>>> from renumics.spotlight import Dataset, Embedding
>>> value = np.array(np.random.rand(2))
>>> embedding = Embedding(value)
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_embedding_column("embeddings", 5*[embedding])
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(len(dataset["embeddings", 3].data))
2
Ancestors (in MRO)
- renumics.spotlight.dtypes._BaseData
- abc.ABC
Static methods
decode(value)
Restore class from its numpy representation.
Methods
encode(self)
Convert to numpy for storing in dataset.
Args
target
: Optional target format.
Sequence1D(index, value=None, dtype=None)
One-dimensional ndarray with optional index values.
Attributes
index
: 1-dimensional array-like of length num_steps
. Index values (x-axis).
value
: 1-dimensional array-like of length num_steps
. Respective values (y-axis).
dtype
: Optional data type of sequence. If None
, data type inferred
from data.
Example
>>> import numpy as np
>>> from renumics.spotlight import Dataset, Sequence1D
>>> index = np.arange(100)
>>> value = np.array(np.random.rand(100))
>>> sequence = Sequence1D(index, value)
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_sequence_1d_column("sequences", 5*[sequence])
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(len(dataset["sequences", 2].value))
100
Ancestors (in MRO)
- renumics.spotlight.dtypes._BaseData
- abc.ABC
Static methods
decode(value)
Restore class from its numpy representation.
empty()
Create an empty sequence.
Methods
encode(self)
Convert to numpy for storing in dataset.
Args
target
: Optional target format.
Mesh(points, triangles, point_attributes=None, triangle_attributes=None, point_displacements=None)
Triangular 3D mesh with optional per-point and per-triangle attributes and optional per-point displacements over time.
Example
>>> import numpy as np
>>> from renumics.spotlight import Dataset, Mesh
>>> points = np.array([[0,0,0],[1,1,1],[0,1,0],[-1,0,1]])
>>> triangles = np.array([[0,1,2],[2,3,0]])
>>> mesh = Mesh(points, triangles)
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_mesh_column("meshes", 5*[mesh])
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(dataset["meshes", 2].triangles)
[[0 1 2]
[2 3 0]]
Ancestors (in MRO)
- renumics.spotlight.dtypes._BaseFileBasedData
- renumics.spotlight.dtypes._BaseData
- abc.ABC
Static methods
decode(value)
Restore class from its numpy representation.
empty()
Create an empty mesh.
from_file(filepath)
Read mesh from a filepath or an URL.
trimesh
is used inside, so only supported formats are allowed.
from_trimesh(mesh)
Import a trimesh.Trimesh
mesh.
Instance variables
point_attributes
Mapping str -> :code:np.array
with shape (num_points, ...)
. Point-wise
attributes corresponding to points
. All possible shapes of a single
attribute can be found in
renumics.spotlight.mesh_proc.gltf.GLTF_SHAPES
.
point_displacements
List of arrays with shape (num_points, 3)
. Point-wise relative
displacements (offsets) over the time corresponding to points
.
Timestep 0 is omitted since it is explicit stored as absolute values in
points
.
points
:code:np.array
with shape (num_points, 3)
. Mesh points.
triangles
:code:np.array
with shape (num_triangles, 3)
. Mesh triangles stored as their
CCW nodes referring to the points
indices.
Methods
encode(self)
Convert to numpy for storing in dataset.
Args
target
: Optional target format.
interpolate_point_displacements(self, num_timesteps)
subsample time dependent attributes with new time step count
update_attributes(self, point_attributes=None, triangle_attributes=None)
Update point and/or triangle attributes dict-like.
Image(data)
An RGB(A) or grayscale image that will be saved in encoded form.
Attributes
data
: Array-like with shape (num_rows, num_columns)
or
(num_rows, num_columns, num_channels)
with num_channels
equal to
3, or 4; with dtype "uint8".
Example
>>> import numpy as np
>>> from renumics.spotlight import Dataset, Image
>>> data = np.full([100,100,3], 255, dtype=np.uint8) # white uint8 image
>>> image = Image(data)
>>> float_data = np.random.uniform(0, 1, (100, 100)) # random grayscale float image
>>> float_image = Image(float_data)
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_image_column("images", [image, float_image, data, float_data])
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(dataset["images", 0].data[50][50])
... print(dataset["images", 3].data.dtype)
[255 255 255]
uint8
Ancestors (in MRO)
- renumics.spotlight.dtypes._BaseFileBasedData
- renumics.spotlight.dtypes._BaseData
- abc.ABC
Static methods
decode(value)
Restore class from its numpy representation.
empty()
Create a transparent 1 x 1 image.
from_file(filepath)
Read image from a filepath, an URL, or a file-like object.
imageio
is used inside, so only supported formats are allowed.
Methods
encode(self)
Convert to numpy for storing in dataset.
Args
target
: Optional target format.
Audio(sampling_rate, data)
An Audio Signal that will be saved in encoded form.
All formats and codecs supported by AV are supported for read.
Attributes
data
: Array-like with shape (num_samples, num_channels)
with num_channels
<= 5.
If data
has a float dtype, its values should be between -1 and 1.
If data
has an int dtype, its values should be between minimum and
maximum possible values for the particular int dtype.
If data
has an unsigned int dtype, ist values should be between 0
and maximum possible values for the particular unsigned int dtype.
sampling_rate
: Sampling rate (samples per seconds)
Example
>>> import numpy as np
>>> from renumics.spotlight import Dataset, Audio
>>> samplerate = 44100
>>> fs = 100 # 100 Hz audio signal
>>> time = np.linspace(0.0, 1.0, samplerate)
>>> amplitude = np.iinfo(np.int16).max * 0.4
>>> data = np.array(amplitude * np.sin(2.0 * np.pi * fs * time), dtype=np.int16)
>>> audio = Audio(samplerate, np.array([data, data]).T) # int16 stereo signal
>>> float_data = 0.5 * np.cos(2.0 * np.pi * fs * time).astype(np.float32)
>>> float_audio = Audio(samplerate, float_data) # float32 mono signal
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_audio_column("audio", [audio, float_audio])
... dataset.append_audio_column("lossy_audio", [audio, float_audio], lossy=True)
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(dataset["audio", 0].data[100])
... print(f"{dataset['lossy_audio', 1].data[0, 0]:.5g}")
[12967 12967]
0.4596
Ancestors (in MRO)
- renumics.spotlight.dtypes._BaseFileBasedData
- renumics.spotlight.dtypes._BaseData
- abc.ABC
Static methods
decode(value)
Restore class from its numpy representation.
empty()
Create a single zero-value sample stereo audio signal.
from_file(filepath)
Read audio file from a filepath, an URL, or a file-like object.
pyav
is used inside, so only supported formats are allowed.
get_format_codec(target=None)
Get an audio format and an audio codec by an target
.
Methods
encode(self, target=None)
Convert to numpy for storing in dataset.
Args
target
: Optional target format.
Category(...)
A string value that takes only a limited number of possible values (categories).
The corresponding categories can be got and set with get/set_column_attributes['categories'].
Dummy class for window column creation, should not be explicitly used as input data.
Example
>>> import numpy as np
>>> from renumics.spotlight import Dataset
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_categorical_column("my_new_cat",
... categories=["red", "green", "blue"],)
... dataset.append_row(my_new_cat="blue")
... dataset.append_row(my_new_cat="green")
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(dataset["my_new_cat", 1])
green
Example
>>> import numpy as np
>>> import datetime
>>> from renumics.spotlight import Dataset
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_categorical_column("my_new_cat",
... categories=["red", "green", "blue"],)
... current_categories = dataset.get_column_attributes("my_new_cat")["categories"]
... dataset.set_column_attributes("my_new_cat", categories={**current_categories,
... "black":100})
... dataset.append_row(my_new_cat="black")
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(dataset["my_new_cat", 0])
black
Ancestors (in MRO)
- builtins.str
Video(data)
A video object. No encoding or decoding is currently performed on the python side, so all formats will be saved into dataset without compatibility check, but only the formats supported by your browser (apparently .mp4, .ogg, .webm, .mov etc.) can be played in Spotlight.
Ancestors (in MRO)
- renumics.spotlight.dtypes._BaseFileBasedData
- renumics.spotlight.dtypes._BaseData
- abc.ABC
Static methods
decode(value)
Restore class from its numpy representation.
empty()
Create an empty video instance.
from_file(filepath)
Read video from a filepath or an URL.
Methods
encode(self)
Convert to numpy for storing in dataset.
Args
target
: Optional target format.
Window()
A pair of two timestamps in seconds which can be later projected onto
continuous data (only :class:Audio <renumics.spotlight.dtypes.Audio>
is currently supported).
Dummy class for window column creation
(see :func:Dataset.append_column <renumics.spotlight.dataset.Dataset.append_column>
),
should not be explicitly used as input data.
To create a window column, use
:func:Dataset.append_window_column <renumics.spotlight.dataset.Dataset.append_window_column>
method.
Examples
>>> import numpy as np
>>> from renumics.spotlight import Dataset
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_window_column("window", [[1, 2]] * 4)
... dataset.append_row(window=(0, 1))
... dataset.append_row(window=np.array([-1, 0]))
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(dataset["window"])
[[ 1. 2.]
[ 1. 2.]
[ 1. 2.]
[ 1. 2.]
[ 0. 1.]
[-1. 0.]]
>>> import numpy as np
>>> from renumics.spotlight import Dataset
>>> with Dataset("docs/example.h5", "w") as dataset:
... dataset.append_int_column("start", range(5))
... dataset.append_float_column("end", dataset["start"] + 2)
... print(dataset["start"])
... print(dataset["end"])
[0 1 2 3 4]
[2. 3. 4. 5. 6.]
>>> with Dataset("docs/example.h5", "a") as dataset:
... dataset.append_window_column("window", zip(dataset["start"], dataset["end"]))
>>> with Dataset("docs/example.h5", "r") as dataset:
... print(dataset["window"])
[[0. 2.]
[1. 3.]
[2. 4.]
[3. 5.]
[4. 6.]]