[BUG]: TypeError: 'StepArtifact' object is not subscriptable about zenml

[htahir1]

@Muhammed-Rashid-07 The @pipeline function executes at composition time, and the execution of the pipeline potentially can happen later. This means that when you run a pipeline, zenml first "runs" your pipeline function code and evaluates which steps are connected to which others. After that zenml does not run that code again, so if you have anything dynamic that needs to go between steps, they should be in the step body, not the pipeline body.

[Muhammed-Rashid-07]

I have solved this issue few months back, the problem was I did list subscription in pipeline. In pipeline it is not possible to do it subscription or any operation. If we have to add anything like that do it in steps and import them into the pipeline. I think that's how it works.

@htahir1 just curious, Why is it not possible to do that in pipeline? Can you explain the reason.

[AbdelrhmanNile]

same error
did you find a solution?

[htahir1]

I think there needs to be more context on the original error by @Muhammed-Rashid-07 to resolve this.. need the step code

[AbdelrhmanNile]

@htahir1

from zenml import step
from tensorflow.keras.datasets import mnist  # type: ignore
from tensorflow.keras.utils import to_categorical  # type: ignore

from typing_extensions import Tuple, Annotated, Any

import numpy as np


@step
def load_data() -> (
    Annotated[Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray], "data"]
):
    (x_train, y_train), (x_test, y_test) = mnist.load_data()
    x_train = x_train.reshape((60000, 28, 28, 1))
    x_train = x_train.astype("float32") / 255.0
    x_test = x_test.reshape((10000, 28, 28, 1))
    x_test = x_test.astype("float32") / 255.0
    y_train = to_categorical(y_train)
    y_test = to_categorical(y_test)
    return x_train, y_train, x_test, y_test

now in my pipleine

from zenml import pipeline
from steps.dataloader import load_data
from steps.modelbuilder import build_model
from steps.trainer import train_model
from steps.eval import evaluate_model


@pipeline
def mnist_pipeline() -> None:
    x_train, y_train, x_test, y_test = load_data()

throws the following error

TypeError: cannot unpack non-iterable StepArtifact object

[htahir1]

I see. I think you are annotating the step wrongly here:

@step
def load_data() -> (
    Annotated[Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray], "data"]
):

Your type annotation is saying "My step has one return type and thats a tuple of 4 things"

But rather it should say

"My step has "four" return types and each have their own annotations:

@step
def load_data() -> (
    Tuple[Annotated[np.ndarray, "x_train"], Annotated[np.ndarray, "y_train"], Annotated[np.ndarray, "x_test"], Annotated[np.ndarray, "y_test"]]
):

[strickvl]

@Muhammed-Rashid-07 was this enough information for you to solve your problem / move on? If so, I'll close out this issue.