lamindb.base.types ¶

Base types.

Classes¶

class lamindb.base.types.CanonicalSuffix¶

Strings that inform a storage format.

The known formats in this class extend the file extensions underlying the international MIME types. The purpose of CanonicalSuffix is to avoid populating the .suffix field of Artifact with random strings like ".abcdef" or ".2022_07_03_txt".

You can extend the set of known formats dynamically to dictate which formats get recognized during the construction of an Artifact. You can also make Pull Request to the set of canonical suffixes on GitHub here.

Upon construction of an Artifact, CanonicalSuffix.from_path() determines whether the path contains a suffix that informs a known storage format or not. If it does, the suffix is extracted and stored in the .suffix field. If not, an empty string is stored in the .suffix field.

Note

Because users can define their own storage formats, the .suffix field of Artifact may contain strings that are not part of the current set of known formats in your compute session.

Examples

Construct from a path:

CanonicalSuffix.from_path("data/sample.csv")  # known storage format
#> CanonicalSuffix('.csv')

CanonicalSuffix.extract_from_path("myfile.abcdedf")  # unknown storage format
#> CanonicalSuffix('')

Often, you’re interested in the canonical suffix and a raw suffix:

CanonicalSuffix.extract_from_path("data/sample.csv")
#> CanonicalSuffix('.csv'), ".csv"

CanonicalSuffix.extract_from_path("myfile.abcdedf")
#> CanonicalSuffix(''), ".abcdedf"

CanonicalSuffix.extract_from_path("data/sample.csv.gz")
#> CanonicalSuffix('.csv.gz'), ".csv.gz"

CanonicalSuffix.extract_from_path("data/sample.anndata.zarr")
#> CanonicalSuffix('.anndata.zarr'), ".anndata.zarr"

CanonicalSuffix.extract_from_path("image.PNG")
#> CanonicalSuffix('.png'), ".png"

CanonicalSuffix.extract_from_path("archive.tar.gz")
#> CanonicalSuffix('.tar.gz'), ".tar.gz"

CanonicalSuffix.extract_from_path("filename.h5ad.tar.xz")
#> CanonicalSuffix('.h5ad.tar.xz'), ".h5ad.tar.xz"

CanonicalSuffix.extract_from_path("file.random.gz")
#> CanonicalSuffix('.gz'), ".gz"

CanonicalSuffix.extract_from_path("sample.OME.ZARR")
#> CanonicalSuffix('.ome.zarr'), ".ome.zarr"

CanonicalSuffix.extract_from_path("unknown.XYZ")
#> CanonicalSuffix(''), ".xyz"

Extend simple suffixes dynamically in a Python session:

CanonicalSuffix.from_path("data/sample.myformat")
#> CanonicalSuffix('')
CanonicalSuffix.simple_formats.add(".myformat")  # extend dynamically
CanonicalSuffix.from_path("data/sample.myformat")
#> CanonicalSuffix('.myformat')

simple_formats: set[str] = {'.3g2', '.3gp', '.3gpp', '.3gpp2', '.7z', '.a', '.aac', '.adts', '.ai', '.aif', '.aifc', '.aiff', '.arrow', '.ass', '.au', '.avi', '.avif', '.avro', '.bam', '.bat', '.bcf', '.bcpio', '.bed', '.bedgraph', '.bigwig', '.bin', '.bmp', '.bw', '.c', '.cdf', '.ckpt', '.cpio', '.cram', '.csh', '.css', '.csv', '.czi', '.data', '.db', '.dcm', '.dll', '.doc', '.dot', '.duckdb', '.dvi', '.embl', '.eml', '.eps', '.etx', '.exe', '.fasta', '.fastq', '.fcs', '.feather', '.gb', '.genbank', '.gff', '.gff3', '.gguf', '.gif', '.gtar', '.gtf', '.h', '.h5', '.h5ad', '.h5mu', '.hdf', '.hdf5', '.heic', '.heif', '.htm', '.html', '.ico', '.ief', '.ipynb', '.joblib', '.jpe', '.jpeg', '.jpg', '.js', '.json', '.jsonl', '.keras', '.ksh', '.latex', '.lif', '.loas', '.loom', '.m1v', '.m3u', '.m3u8', '.man', '.mar', '.markdown', '.md', '.me', '.mht', '.mhtml', '.mid', '.midi', '.mif', '.mjs', '.mlmodel', '.model', '.mov', '.movie', '.mp2', '.mp3', '.mp4', '.mpa', '.mpe', '.mpeg', '.mpg', '.ms', '.mtx', '.n3', '.nc', '.nd2', '.ndjson', '.newick', '.npy', '.npz', '.nq', '.nt', '.nwk', '.nws', '.o', '.obj', '.obo', '.oda', '.onnx', '.opus', '.orc', '.p12', '.p7c', '.parquet', '.pb', '.pbm', '.pbtxt', '.pct', '.pdf', '.pfx', '.pgm', '.pic', '.pickle', '.pict', '.pkl', '.pl', '.png', '.pnm', '.pot', '.ppa', '.ppm', '.pps', '.ppt', '.ps', '.pt', '.pth', '.pwz', '.py', '.pyc', '.pyo', '.qs', '.qt', '.ra', '.ram', '.rar', '.ras', '.rdf', '.rds', '.rgb', '.roff', '.rtf', '.rtx', '.safetensors', '.sam', '.savedmodel', '.sgm', '.sgml', '.sh', '.shar', '.snd', '.so', '.soma', '.sqlite', '.src', '.srt', '.state_dict', '.sv4cpio', '.sv4crc', '.svg', '.svs', '.swf', '.t', '.tar', '.tcl', '.tex', '.texi', '.texinfo', '.tflite', '.tif', '.tiff', '.tiledb', '.tiledbsoma', '.tr', '.trig', '.tsv', '.txt', '.ustar', '.vcf', '.vtt', '.wasm', '.wav', '.webm', '.webmanifest', '.webp', '.wiz', '.wsdl', '.xbm', '.xlb', '.xls', '.xlsx', '.xml', '.xpdl', '.xpm', '.xsl', '.xul', '.xwd', '.yaml', '.yml', '.zarr', '.zip'}¶

Simple formats such as .csv, .h5ad or .parquet.

These correspond to the last component of a filename (path.suffix).

composite_formats: set[str] = {'.anndata.zarr', '.ome.h5', '.ome.hdf5', '.ome.zarr', '.vitessce.json'}¶

Composite formats such as .anndata.zarr or .ome.zarr.

Their meaning is carried by the combination of parts, so they take precedence over the trailing simple suffix (e.g. .anndata.zarr is preferred over .zarr).

encoding_formats: set[str] = {'.bz2', '.gz', '.xz', '.zst'}¶

Stream-encoding formats such as .gz, .bz2, .xz or .zst.

These are appended to another suffix (e.g. .csv.gz, .h5ad.tar.gz).

classmethod from_path(path)¶

Construct a canonical suffix from a path.

Note that this returns the empty string if the path doesn’t contain a suffix that maps on a known formats.

Parameters:: path (Path | UPath) – The path to extract the suffix from.
Return type:: CanonicalSuffix

classmethod extract_from_path(path)¶

Extract a validated canonical suffix and a raw suffix from a path.

This also treats composite (e.g. .csv.gz) suffixes.

Parameters:: path (Path | UPath) – The path to extract the suffix from.
Return type:: tuple[CanonicalSuffix, str]
Returns:: A tuple consisting in the canonical suffix and a raw string suffix.

Simple types¶

lamindb.base.types.ArtifactKind¶: alias of Literal[‘dataset’, ‘model’, ‘plan’, ‘__lamindb_run__’, ‘__lamindb_config__’]

lamindb.base.types.TransformKind¶: alias of Literal[‘pipeline’, ‘notebook’, ‘script’, ‘function’]

lamindb.base.types.BlockKind¶

Block kind, a README.md-type page or comment.

Any block expects Markdown as the formatting language.

lamindb.base.types.BranchStatus¶

Branch status.

status	code	description
`closed`	-2	Change Request was closed without merging.
`merged`	-1	The branch was merged into another branch.
`standalone`	0	A standalone branch without Change Request.
`draft`	1	Change Request exists but is not ready for review.
`review`	2	Change Request is ready for review.

The database stores the branch status as an integer code in field _status_code.

lamindb.base.types.RunStatus¶

Run status.

status	code	description
`scheduled`	-3	The run is scheduled.
`restarted`	-2	The run was restarted.
`started`	-1	The run has started.
`completed`	0	The run completed successfully.
`errored`	1	The run ended with an error.
`aborted`	2	The run was aborted.

The database stores the run status as an integer code in field _status_code.

lamindb.base.types.SimpleDtype¶

Python types for simple scalar dtypes.

See section Data types on the Feature page for more background.

lamindb.base.types.SimpleDtypeStr¶: String-serialized representations for SimpleDtype.

lamindb.base.types.SimpleDvalue¶: Values corresponding to SimpleDtype.

lamindb.base.types.DtypeStr¶: alias of Literal[‘num’, ‘int’, ‘float’, ‘str’, ‘bool’, ‘datetime’, ‘datetime64[ns, UTC]’, ‘date’, ‘dict’, ‘path’, ‘url’, ‘object’]

Basic types¶

lamindb.base.types.AnyPathStr¶: alias of str | Path | UPath

lamindb.base.types.StrField¶: alias of str | DeferredAttribute

lamindb.base.types.ListLike¶: alias of Sequence[str]

lamindb.base.types.FieldAttr¶: alias of DeferredAttribute