4.3 Save my data on the store

Each environment has its own storage to save and retrieve any amount of data at any time, from anywhere on the web. This storage is an Amazon S3 bucket that you can manage with the Python SDK of the platform.

You can upload any kind of raw data on the store. Furthermore, the results of the pipelines and metrics generated by the platform can also be saved on the data store.

Info: For the moment, it is not possible to copy data from an environment to another. You have to download it and to re-upload it. In future versions, you will be able to transfer your data. .. raw:: html

<div style=”background-color:#f3f6f6; padding: 12px;”>

🆕 The graphical interface of the data store will arrive later on the platform.

</div>

4.3.1 Summary

  1. Get files

  2. Get file info

  3. Upload a file

  4. Download a file

  5. Delete a file

Function name

Method

Return type

Description

list_data_store_objects

CraftAiSdk.list_data_store_objects()

list of dict

Get the list of the objects stored in the data store.

upload_data_store_object

CraftAiSdk.upload_data_store_object(filepath_or_buffer, object_path_in_datastore)

None

Upload a file as an object into the data store.

download_data_store_object

CraftAiSdk.download_data_store_object(object_path_in_datastore, filepath_or_buffer)

None

Download an object in the data store and save it into a file.

delete_data_store_object

CraftAiSdk.delete_data_store_object(object_path_in_datastore)

dict

Delete an object on the data store.

4.3.2 Get files

4.3.2.1 Function definition

Get the list of the objects stored in the data store.

CraftAiSdk.list_data_store_objects()

Parameter

No parameter

Returns

List of objects (dict) in the data store, each object being represented as dict (with keys “path”, “last_modified”, and “size”).

4.3.3 Get file info

4.3.3.1 Function definition

Get information about a single object in the data store.

CraftAiSdk.get_data_store_object_information(object_path_in_datastore)

Parameter

  • object_path_in_datastore (str) – Location of the object in the data store.

Returns

Object information, with the following keys:

  • “path” (str): Location of the object in the data store.

  • “last_modified” (str): The creation date or last modification date in ISO format.

  • “size” (int): The size of the object in bytes.

4.3.4 Upload a file

4.3.4.1 Function definition

Upload a file as an object into the data store.

CraftAiSdk.upload_data_store_object(filepath_or_buffer, object_path_in_datastore)

Parameters

  • filepath_or_buffer (str, or file-like object) – String, path to the file to be uploaded ; or file-like object implementing a read() method (e.g. via buildin open function). The file object must be opened in binary mode, not text mode.

  • object_path_in_datastore (str) – Destination of the uploaded file.

4.3.5 Download a file

4.3.5.1 Function definition

Download an object in the data store and save it into a file.

CraftAiSdk.download_data_store_object(object_path_in_datastore, filepath_or_buffer)

Parameters

  • object_path_in_datastore (str) – Location of the object to download from the data store.

  • filepath_or_buffer (str or file-like object) – String, file path to save the file to ; or a file-like object implementing a write() method, (e.g. via builtin open function). The file object must be opened in binary mode, not text mode.

Returns

This function return an None.

4.3.6 Delete a file

4.3.6.1 Function definition

Delete an object on the data store.

CraftAiSdk.delete_data_store_object(object_path_in_datastore)

Parameters

  • object_path_in_datastore (str) – Location of the object to delete in the data store.

Returns

Deleted object represented as dict (with key “path”).