draft: add a way to visualize scene box, seed points, and modify if wanted #3363
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
hoanhle
commented
Aug 13, 2024
hoanhle
commented
hoanhle
force-pushed
the
feature/visualize_roi
branch
from
07aca40
to
99a38d1
Compare
hoanhle
force-pushed
the
feature/visualize_roi
branch
from
99a38d1
to
86be60d
Compare
hoanhle
commented
Aug 14, 2024
Comment on lines
+421
to
+433
| def update_scene_box(self): | ||
| min_coords = np.minimum(self.transform_controls[0].position, self.transform_controls[1].position) | ||
| max_coords = np.maximum(self.transform_controls[0].position, self.transform_controls[1].position) | ||
|
|
||
| new_aabb = torch.tensor( | ||
| [min_coords / VISER_NERFSTUDIO_SCALE_RATIO, max_coords / VISER_NERFSTUDIO_SCALE_RATIO], | ||
| device=self.pipeline.model.device, | ||
| requires_grad=False, | ||
| ) | ||
|
|
||
| self.pipeline.model.scene_box = SceneBox(aabb=new_aabb) | ||
| self.pipeline.model.render_aabb = SceneBox(aabb=new_aabb) | ||
| self.pipeline.model.populate_modules() |
There was a problem hiding this comment.
Using device=self.pipeline.model.device for new_aabb seems to cause issues where tensors end up on different devices later in the process. I'm not entirely sure what the best solution is to address this.
@brentyi @tancik, since you both work with this part of the codebase frequently, do you have any suggestions?
hoanhle
commented
Aug 14, 2024
Comment on lines
83
to
91
| self.num_train_data = num_train_data | ||
| self.kwargs = kwargs | ||
| self.collider = None | ||
|
|
||
| self.populate_modules() # populate the modules | ||
| self.device = device | ||
| self.seed_points = seed_points | ||
| self.callbacks = None | ||
| # to keep track of which device the nn.Module is on | ||
| self.device_indicator_param = nn.Parameter(torch.empty(0)) | ||
|
|
||
| @property | ||
| def device(self): | ||
| """Returns the device that the model is on.""" | ||
| return self.device_indicator_param.device | ||
| self.populate_modules() # populate the modules | ||
|
|
There was a problem hiding this comment.
get the device from pipeline.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.