Save my data on the store

Each environment has its own storage to save and retrieve any amount of data at any time, from anywhere on the web. This storage is an Amazon S3 bucket that you can manage with the Python SDK of the platform.

You can upload any kind of raw data on the store. Furthermore, the results of the pipelines and metrics generated by the platform can also be saved on the data store.

Info: For the moment, it is not possible to copy data from an environment to another. You have to download it and to re-upload it. In future versions, you will be able to transfer your data.

🆕 The graphical interface of the data store will arrive later on the platform.

Function name	Method	Return type	Description
list_data_store_objects	CraftAiSdk.list_data_store_objects()	list of dict	Get the list of the objects stored in the data store.
upload_data_store_object	CraftAiSdk.upload_data_store_object(filepath_or_buffer, object_path_in_datastore)	None	Upload a file as an object into the data store.
download_data_store_object	CraftAiSdk.download_data_store_object(object_path_in_datastore, filepath_or_buffer)	None	Download an object in the data store and save it into a file.
delete_data_store_object	CraftAiSdk.delete_data_store_object(object_path_in_datastore)	dict	Delete an object on the data store.

Get files

Function definition

Get the list of the objects stored in the data store.

CraftAiSdk.list_data_store_objects()

Parameter

No parameter

Returns

List of objects (dict) in the data store, each object being represented as dict (with keys “path”, “last_modified”, and “size”).

Get file info

Function definition

Get information about a single object in the data store.

CraftAiSdk.get_data_store_object_information(object_path_in_datastore)

Parameter

object_path_in_datastore (str) – Location of the object in the data store.

Returns

Object information, with the following keys:

“path” (str): Location of the object in the data store.
“last_modified” (str): The creation date or last modification date in ISO format.
“size” (int): The size of the object in bytes.

Upload a file

Function definition

Upload a file as an object into the data store.

CraftAiSdk.upload_data_store_object(filepath_or_buffer, object_path_in_datastore)

Parameters

filepath_or_buffer (str, or file-like object) – String, path to the file to be uploaded ; or file-like object implementing a read() method (e.g. via buildin open function). The file object must be opened in binary mode, not text mode.
object_path_in_datastore (str) – Destination of the uploaded file.

Download a file

Function definition

Download an object in the data store and save it into a file.

CraftAiSdk.download_data_store_object(object_path_in_datastore, filepath_or_buffer)

Parameters

object_path_in_datastore (str) – Location of the object to download from the data store.
filepath_or_buffer (str or file-like object) – String, file path to save the file to ; or a file-like object implementing a write() method, (e.g. via builtin open function). The file object must be opened in binary mode, not text mode.

Returns

This function return an None.

Delete a file

Function definition

Delete an object on the data store.

CraftAiSdk.delete_data_store_object(object_path_in_datastore)

Parameters

object_path_in_datastore (str) – Location of the object to delete in the data store.

Returns

Deleted object represented as dict (with key “path”).