Getting x and y values about qiskit-experiments

[yaelbh]

Maybe part of BaseAnalysis ?

[eggerdj]

That could also work.

[nkanazawa1989]

Based on our discussion offline, let me add some discussion points.

I was first proposing to have internal data structure in columnar format, so that we can manage arbitrary xvalue (in QPT, xvalue may consist of multiple column of "meas_basis" and "prep_basis", Hamiltonian tomography may have more). More specifically, we can generate a dataframe in the BaseExperiment class. Data added to the instance will be immediately formatted and stored in the dataframe.

The point was if we really need such internal data representation at the cost of extra dependency. Because the list of dictionary shown above is still enough to generate xy values, and we can locally store the exp data instance as a json. However, I see two advantages of having such internal data structure.

1. data handling
   dataframe can provide variety filtering methods. So we don't need to loop over the list manually.

2. saveing/loading data
   json dump is not great method due to the serialization overhead. If we store the experiment result of 3Q QPT, we need to store typ. 1728 entries. I didn't benchmark this yet, but I imagine this will generate huge datafile and dump/load take long time. dataframe can support multiple binary file format such as hdf5.

Perhaps we can start from simple dict structure above, and switch to have proper data structure at some point.

[nkanazawa1989]

As we discussed in #24 , ambiguity of metadata field should be resolved by this extraction function. Sometime we cannot define explicit series, i.e. see Hamiltonian tomography example, it would be better to have API

def extract_xy_values(exp_data: ExperimentData, data_key: str, **filter: Any) -> Tuple[np.array, np.array]:

so that we can write

extract_xy_values(ham_tomo_data, data_key='delay', meas_basis='X', control_state=0)

Another point I want to discuss is how to extend this function to

• no scan (e.g. discriminator)
• multi-axes scan (e.g. chevron)

because current function supports only single data_key.

One possible solution would be:

def extract_xy_values(exp_data: ExperimentData, *data_key: str, **filter: Any) -> Union[np.array, Tuple[np.array, np.array]]:

Then we can extract x-y value more flexibly.

• no data key

extract_xy_values(discr_data, prep_state=0) -> np.ndarray(1, N)

• 1d scan

extract_xy_values(ramsey_data, 'delay', meas_basis='x') -> Tuple[np.array(1, N), np.array(1, N)]

• 2d scan

extract_xy_values(ramsey_data, 'delay', 'detuning') -> Tuple[np.array(2, N), np.array(1, N)]

[chriseclectic]

I updated the curve fit PR to allow a bit more generic way of getting x and y data in its helper functions:

It now has

def curve_fit_data(
    data: List[Dict[str, any]],
    data_processor: Callable,
    x_key: str = "xval"
) -> Tuple[xdata: ndarray(N), ydata: ndarray(N), sigma: ndarray(N)]

and

def multi_curve_fit_data(
    data: List[Dict[str, any]],
    data_processor: Callable,
    x_key: str = "xval",
    series_key: str = "series"
) -> Tuple[xdata: ndarray(N, 2), ydata: ndarray(N), sigma: ndarray(N)]

I think these cover the last two cases naoki mentioned above (minus filtering which would be a nice addition). The no-scan case I don't see as xy data at all. It is just regular data processing since you are converting a list of data into some array of values (Note the array convention is transposed to the one Naoki suggests because of the way i originally wrote the curve_fit and multi_curve_fit functions which these are designed to extract the input for)

Here the data_processor function is a bit different to the data processor PR, but is a callable with specific signature data_processor(data: Dict[str, any]) -> Tuple[y: float, sigma: float]

[nkanazawa1989]

Can this be closed? @chriseclectic

[gadial]

Closing as this issue was addressed