Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tensorboard-plugin throws Error when selecting RGB-shader with custom vertex features #6950

Open
3 tasks done
manuelvogel12 opened this issue Sep 6, 2024 · 0 comments
Open
3 tasks done
Labels
bug Not a build issue, this is likely a bug.

Comments

@manuelvogel12
Copy link
Contributor

Checklist

Describe the issue

In the tensorboard plugin, I want to visualize two different colorings ("vertex_colors") and ("vertex_modified_colors") of one object.

Visualizing the color stored in "vertex_colors" works, but visualizing a secondary color option does not.
It works to visualize one channel of the additional vertex attribute with the shader Colormap (Rainbow) and Colormap (Greyscale), but RGB does not work

Steps to reproduce the bug

1. Generate the tensorboard logs with the code below
2. Open tensorboard
3. As Data, select `modified_colors`
4. Use Shader `RGB`
#####################################################################################


import open3d as o3d
import numpy as np

# Monkey-patch torch.utils.tensorboard.SummaryWriter
from open3d.visualization.tensorboard_plugin import summary
# Utility function to convert Open3D geometry to a dictionary format
from open3d.visualization.tensorboard_plugin.util import to_dict_batch
from torch.utils.tensorboard import SummaryWriter

ply_point_cloud = o3d.data.PLYPointCloud()
pcd = o3d.io.read_point_cloud(ply_point_cloud.path)

logdir = "demo_logs/pytorch/small_scale"
writer = SummaryWriter(logdir)

pcd_dict = to_dict_batch([pcd])

# ADD ADDITIONAL VERTEX ATTRIBUTE
pcd_dict["vertex_modified_colors"] = np.copy(pcd_dict["vertex_colors"])

writer.add_3d('obj', pcd_dict, step=1)
writer.add_3d('obj', pcd_dict, step=2)

Error message

Open3D WARNING] Tensor point cloud colors must have DType of Float32 not UInt8. Converting.
[Open3D INFO] OnDataChannelMessage: {"messageId":3,"class_name":"tensorboard/update_rendering","window_uid_list":["window_1"],"tag":"obj","render_state":{"property":"modified_colors","index":"0","shader":"defaultUnlit","colormap":{},"range":[0,255]},"updated":["shader"]}. Error processing message: TypeError: eq(): incompatible function arguments. The following argument types are supported:
1. (self: open3d.cpu.pybind.core.Dtype, arg0: open3d.cpu.pybind.core.Dtype) -> bool

Invoked with: UInt8, <class 'numpy.float32'>

At:
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(407): _normalize
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(670): apply
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/plugin.py(625): _update_rendering

Expected behavior

The expectation is that a colored point cloud should be shown. Instead, it crashes

Open3D, Python and System information

- Operating system: Ubuntu 22.04
- Python version: 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
- Open3D version: 0.18.0 ( also tested on 0.18.0+553ca86)
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip

Additional information

No response

@manuelvogel12 manuelvogel12 added the bug Not a build issue, this is likely a bug. label Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Not a build issue, this is likely a bug.
Projects
None yet
Development

No branches or pull requests

1 participant