Compose a pipeline
A pipeline is a machine learning workflow, consisting of one or more steps, to deploy code using Docker containers. By specifying the output of one step as the input of another step, the user can create a full pipeline formed with a computed acyclic graph (DAG).
ℹ️ Info: For the moment, the pipelines are only single-step. We are actively working on the integration of multi-steps for the next versions of the platform.
To deploy ML code in production, you need to go through a pipeline. So you have to go through a single-step pipeline in any case to perform an endpoint or another type of deployment.
Summary:
Function name |
Method |
Return type |
Description |
---|---|---|---|
create_pipeline |
create_pipeline(pipeline_name, step_name) |
dict[str, str] |
Create a pipeline containing a single step. |
get_pipeline |
get_pipeline(pipeline_name) |
dict |
Get a single pipeline if it exists. |
delete_pipeline |
delete_pipeline(pipeline_name, force_deployments_deletion=False) |
dict |
Delete a pipeline identified by its name and ID. |
list_pipelines |
list_pipelines() |
list of dict |
Get the list of all pipelines. |
Create a mono-step pipeline
SDK function pipeline
Function definition
pipeline = sdk.create_pipeline(
pipeline_name="my_pipeline",
step_name="my_step",
)
Parameters
pipeline_name
(str) – Name of the pipeline to create.step_name
(str) – Name of the step to include in the pipeline.Note
The step should have the status “Ready” before being used to create the pipeline.
Returns
Created pipeline represented as dict (with key “id”) as dict[str, str] object.
“pipeline_name” (str): Pipeline name.
“created_at” (str): Pipeline date of creation.
“steps” (list): List of step names.
“open_inputs” (list): List of all input of step.
“open_output” (list): List of all output of step.
Information about pipeline store
Pipeline can have multiple inputs and outputs, or no one. In fact, it’s dependent on the inputs and outputs of step inside. All input and output will be the same as the step inside. So, you have nothing to configure on pipeline creation for input and output.
Get information pipeline
Get information about one pipeline
Function definition
Get all information about one pipeline, referred by its name.
CraftAiSdk.get_pipeline(pipeline_name)
Parameters
pipeline_name
(str) – Name of the pipeline to get.
Returns
The pipeline information in a dict or None if the pipeline does not exist.
“id” (str): Pipeline id.
“name” (str): Pipeline name.
“created_at” (str): Pipeline date of creation.
“steps” (list): List of step names.
Get all pipelines
Function definition
Get the list of all pipelines on your environment.
CraftAiSdk.list_pipelines()
Returns
List of pipelines represented as dict with keys :
“pipeline_name” (str): Name of pipeline
“created_at” (str): Create date of the pipeline.
Delete a pipeline
Function definition
Delete a pipeline from the current environment.
CraftAiSdk.delete_pipeline(pipeline_name, force_deployments_deletion=False)
Parameters
pipeline_name
(str) – Name of the pipeline.force_deployments_deletion
(bool, optional) – if True, the associated endpoints will be deleted too. Defaults to False.
Returns
The deleted pipeline and associated deleted endpoints in a dict. The returned dict contains two keys:
“pipeline”(dict): Deleted pipeline represented as dict (with keys “id” and “name”).
“endpoints” (list): List of deleted endpoints represented as dict (with keys “id” and “name”).
Warning
You can’t delete a pipeline that is used in a Deployment
Run a pipeline
Now that your pipeline is created you can run it directly from the SDK. Or you can configure a deployment.