Plotting single channel image with show_sample results in incorrect plot output, with out without masks

[rbavery]

🐛 Bug

Describe the bug
When I plot a sample that has a single channel image and masks, PIL is used in the background I think and it incorrectly converts the single channel image to a color image instead of keeping it in greyscale. The show sample func also adds a false border as seen below.

To Reproduce
I can't provide the underlying data, but this is what my train record looks like for the single channel dataset

import icevision
parser = icevision.parsers.COCOMaskParser(annotations_filepath="../../data/cv2_transfer/instances_slicks_test_v2.json", img_dir="../../data/tiled_image_slicks_test_v2")
train_records = parser.parse(data_splitter=icevision.data.SingleSplitSplitter())
record = icevision.Dataset(train_records[0])[0]
record

BaseRecord

common: 
	- Filepath: ../../data/tiled_image_slicks_test_v2/S1A_IW_GRDH_1SDV_20200729T034859_20200729T034924_033664_03E6D3_93EF_vv-image_tile_0.png
	- Img: 512x512x3 <np.ndarray> Image
	- Record ID: 0
	- Image size ImgSize(width=512, height=512)
detection: 
	- masks: [<icevision.core.mask.RLE object at 0x7fd1ccba2760>]
	- mask_array: <icevision.core.mask.MaskArray object at 0x7fd1ccb98190>
	- Is Crowds: [1]
	- Class Map: <ClassMap: {'background': 0, 'infra_slick': 1, 'natural_seep': 2, 'coincident_vessel': 3, 'recent_vessel': 4, 'old_vessel': 5, 'ambiguous': 6}>
	- Labels: [6]
	- Areas: [17586]
	- BBoxes: [<BBox (xmin:256.0, ymin:275.0, xmax:512.0, ymax:512.0)>]

and to show the plot that looks incorrect

icevision.show_samples([record],display_label = True,
    display_bbox=True,
    display_mask=True)

Expected behavior
Plot the image in greyscale, with colored masks on top

Screenshots

with labels:

Desktop (please complete the following information):

• ubuntu 20.04

Additional context
I'm running icevision .12.0

Plotting the file directly with scikit-image .imshow looks as expected. In general I prefer scikit-image to using PIL because PIL makes too many assumptions about the data for my use cases (that it is rgb, that it is of a certain data type, doesn't support float32 fully python-pillow/Pillow#1888)

[fstroth]

Hey, is your image 8bit or 16bit in grayscale values? We have a set of grayscale record defaults for the purpose of using 16bit grayscale images (https://github.com/airctic/icevision/blob/master/icevision/core/record_defaults.py). It should be enough to just overwrite the template_record function

icevision/icevision/parsers/coco_parser.py

Line 113 in 0e42d3d

def template_record(self) -> BaseRecord:

of the COCOMaskParser.

[FraPochetti]

Closing due to inactivity

[rbavery]

I think this should be re-opened. one of the components that builds the grayscale default record seems to have been commented out and it is non trivial to find all the imports to set this tempalte record function up

https://github.com/airctic/icevision/blob/master/icevision/core/record_components.py#L453

ideally, the COCOMaskParser would handle reading grayscale or n-channel imagery by either using the grayscale record or the other record.

after importing all the record components,, masksrecordcomponent isn't defined

from icevision.parsers import COCOMaskParser
from icevision.core.record import *
from icevision.core.record_components import *
from icevision import GrayScaleInstanceSegmentationRecord

class COCOMaskParserGrayscale(COCOMaskParser):
    def template_record(self) -> BaseRecord:
        return GrayScaleInstanceSegmentationRecord()

testparse = COCOMaskParserGrayscale(annotations_filepath= coco_json_path_train, img_dir=tiled_images_folder_train)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Input In [36], in <cell line: 1>()
----> 1 testparse = COCOMaskParserGrayscale(annotations_filepath= coco_json_path_train, img_dir=tiled_images_folder_train)

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/parsers/coco_parser.py:40, in COCOBaseParser.__init__(self, annotations_filepath, img_dir, idmap)
     37     classes[i] = name
     38 self.class_map = ClassMap(classes)
---> 40 super().__init__(template_record=self.template_record(), idmap=idmap)

Input In [35], in COCOMaskParserGrayscale.template_record(self)
      2 def template_record(self) -> BaseRecord:
----> 3     return GrayScaleInstanceSegmentationRecord()

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/core/record_defaults.py:74, in GrayScaleInstanceSegmentationRecord()
     68 def GrayScaleInstanceSegmentationRecord():
     69     return BaseRecord(
     70         (
     71             GrayScaleRecordComponent(),
     72             InstancesLabelsRecordComponent(),
     73             BBoxesRecordComponent(),
---> 74             MasksRecordComponent(),
     75         )
     76     )

NameError: name 'MasksRecordComponent' is not defined

[rbavery]

I tried to replace the commented out MasksRecordComponent with what looks like a newer component: InstanceMasksRecordComponent. but this doesn't work

class COCOMaskParserGrayscale(COCOMaskParser):
    def template_record(self) -> BaseRecord:
        record = super().template_record()
        record.add_component(GrayScaleRecordComponent())
        record.add_component(InstancesLabelsRecordComponent())
        record.add_component(BBoxesRecordComponent())
        record.add_component(InstanceMasksRecordComponent())
        return record

parsing doesn't fail, but show_results does

show_records(train_records[0:3], ncols=3, class_map=class_map)

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Input In [10], in <cell line: 2>()
      1 get_ipython().run_line_magic('matplotlib', 'inline')
----> 2 show_records(train_records[0:3], ncols=3, class_map=class_map)

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/visualize/show_data.py:151, in show_records(records, class_map, display_label, display_bbox, display_mask, ncols, figsize, show, **draw_sample_kwargs)
    127 def show_records(
    128     records: Sequence[RecordType],
    129     class_map: Optional[ClassMap] = None,
   (...)
    136     **draw_sample_kwargs,
    137 ) -> None:
    138     partials = [
    139         partial(
    140             show_record,
   (...)
    149         for record in records
    150     ]
--> 151     plot_grid(partials, ncols=ncols, figsize=figsize, show=show)

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/utils/imageio.py:141, in plot_grid(fs, ncols, figsize, show, axs_per_iter, **kwargs)
    138     raise ValueError("axs_per_iter has to be greater than 1")
    140 for f, ax in zip(fs, axs):
--> 141     f(ax=ax)
    143 plt.tight_layout()
    144 if show:

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/imports.py:88, in partial.__call__(self, *args, **kwargs)
     87 def __call__(self, *args, **kwargs):
---> 88     return self._partial(*args, **kwargs)

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/visualize/show_data.py:56, in show_record(record, class_map, display_label, display_bbox, display_mask, display_keypoints, ax, show, figsize, **draw_sample_kwargs)
     44 def show_record(
     45     record,
     46     class_map: Optional[ClassMap] = None,
   (...)
     54     **draw_sample_kwargs,
     55 ) -> None:
---> 56     img = draw_record(
     57         record=record,
     58         class_map=class_map,
     59         display_label=display_label,
     60         display_bbox=display_bbox,
     61         display_mask=display_mask,
     62         display_keypoints=display_keypoints,
     63         **draw_sample_kwargs,
     64     )
     65     show_img(img=img, ax=ax, show=show, figsize=figsize)

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/visualize/draw_data.py:383, in draw_record(record, class_map, display_label, display_bbox, display_mask, display_score, display_keypoints, font_path, font_size, label_color, mask_blend, mask_border_thickness, color_map, prettify, prettify_func, return_as_pil_img, exclude_labels, include_only)
    362 def draw_record(
    363     record,
    364     class_map: Optional[ClassMap] = None,
   (...)
    381     include_only: List[str] = None,
    382 ):
--> 383     sample = record.load()
    384     return draw_sample(
    385         sample=sample,
    386         class_map=class_map,
   (...)
    402         include_only=include_only,
    403     )

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/core/record.py:66, in BaseRecord.load(self)
     64 def load(self) -> "BaseRecord":
     65     record = deepcopy(self)
---> 66     record.reduce_on_components("_load")
     67     return record

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/core/components/composite.py:83, in TaskComposite.reduce_on_components(self, fn_name, reduction, **fn_kwargs)
     81 results = {}
     82 for task, composite in self.task_composites.items():
---> 83     result = composite.reduce_on_components(fn_name, reduction, **fn_kwargs)
     84     results[task] = result
     86 return results

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/core/components/composite.py:132, in Composite.reduce_on_components(self, fn_name, reduction, **fn_kwargs)
    130 results = []
    131 for component in self.components:
--> 132     results.append(getattr(component, fn_name)(**fn_kwargs))
    134 if reduction is not None and len(results) > 0:
    135     out = results.pop(0)

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/core/record_components.py:227, in GrayScaleRecordComponent._load(self)
    226 def _load(self):
--> 227     img = open_gray_scale_image(self.filepath)
    228     self.set_img(img)

File ~/work/.ice-env/lib/python3.9/site-packages/icevision/utils/imageio.py:69, in open_gray_scale_image(fn)
     67     img = open_dicom(str(fn))
     68 else:
---> 69     img = PIL.Image.open(str(fn))
     71 img = np.dstack([img, img, img])
     72 img = img.astype(np.float32)

File ~/work/.ice-env/lib/python3.9/site-packages/PIL/Image.py:2975, in open(fp, mode, formats)
   2972     filename = fp
   2974 if filename:
-> 2975     fp = builtins.open(filename, "rb")
   2976     exclusive_fp = True
   2978 try:

FileNotFoundError: [Errno 2] No such file or directory: 'None'

and these are the components present in the first record. it looks like there are duplicates

train_records[1].components

{<icevision.core.record_components.AreasRecordComponent at 0x7f1fec4e5ca0>,
 <icevision.core.record_components.BBoxesRecordComponent at 0x7f1fec4714c0>,
 <icevision.core.record_components.BBoxesRecordComponent at 0x7f1fec471550>,
 <icevision.core.record_components.FilepathRecordComponent at 0x7f1fec4718b0>,
 <icevision.core.record_components.GrayScaleRecordComponent at 0x7f1fec471220>,
 <icevision.core.record_components.InstanceMasksRecordComponent at 0x7f1fec471310>,
 <icevision.core.record_components.InstanceMasksRecordComponent at 0x7f1fec4715e0>,
 <icevision.core.record_components.InstancesLabelsRecordComponent at 0x7f1fec471280>,
 <icevision.core.record_components.InstancesLabelsRecordComponent at 0x7f1fec4713a0>,
 <icevision.core.record_components.IsCrowdsRecordComponent at 0x7f1fec471430>,
 <icevision.core.record_components.RecordIDRecordComponent at 0x7f1fec471940>,
 <icevision.core.record_components.SizeRecordComponent at 0x7f1fec471820>}