report.plot.classification.py - questions

[rjavadi]

There are a couple of things that I may be missing or need to be clarified in this class:

1. In metrics_trends(): for multi-class plot, is it required to have len(metric_value) == len(num_classes)? If yes, I think we should add assert all similar to metrics_value().
2. In metrics_comparison_scatter(): what happens for colors if what if num of slices > num_classes? Do we have a boundary on numer of slices? In such cases that slices > classes, I get an IndexError: tuple index out of range for colors.
3. In metrics_comparison_bar(): value error in line bar_plot because we pass a single number in the for loop above:

Invalid value of type 'numpy.float64' received for the 'y' property of bar
                  Received value: 0.6385211863242318
              The 'y' property is an array that may be specified as a tuple,
             list, numpy array, or pandas Series

[amrit110]

@mahshidaln could you please help answer the questions above?

[rjavadi]

@mahshidaln
These are the variables that I generate and pass to the ClassificationPlotter's methods:

_slices = [f"slice_#{i}" for i in range(5)]
_metrics = ["recall", "f_beta"]

def binary_slice_metrics():
    """Generate slice metrics dictionary for binary classification."""
    input_dic = defaultdict(lambda: defaultdict(float))
    for i in range(len(_slices)):
        for metric in _metrics:
            input_dic[_slices[i]][metric] = np.random.rand()

    return input_dic

def multiclass_slice_metrics():
    """Generate slice metrics dictionary for multi-class classification."""
    input_dic = defaultdict(lambda: defaultdict(list))
    for i in range(len(_slices)):
        for metric in _metrics:
            input_dic[_slices[i]][metric] = np.random.rand(3)
    return input_dic

And these are the ClassificationPlotters:

_binary_plotter = ClassificationPlotter(
    task_type="binary",
    task_name="b_plot",
    class_num=2,
    class_names=["#1", "#2"],
)
_mclass_plotter = ClassificationPlotter(
    task_type="multiclass",
    task_name="mclass_plot",
    class_num=3,
    class_names=["#1", "#2", "#3"],
)

[mahshidaln]

@rjavadi To answer your questions:

1. Yes that's right the length of metric_data should be the same as the number of classes/labels.
2. No there is no boundary on the number of slices and what needs to be changed is the value we compare the length of colorway with. In the code block starting with if len(self.template.layout.colorway) >= self.class_num, we need to replace class_num with length of slice_metrics.
3. Yes it makes sense the line 871 needs to be updated to metric_values = [metrics[metric_name]]

I can also make the changes If you want me to. Thanks.

[rjavadi]

@mahshidaln @amrit110
I missed something in no. 1. Input type in metrics_trends() should be different with metrics_value(). The type described in pydoc is:

Dictionary of metric trends, where the key is the date/time step \
                and the value is a list of dictionaries with keys metric type, \
                metric value, and slice name

but in method signature it is: Dict[str, Union[float, List[float], npt.NDArray[Any]]].

I don't know why pytest didn't catch it when running tests.