Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added All pixel Sampler #3031

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

IamMohitM
Copy link
Contributor

A sampler that selects a specific image and samples all pixels from that image.

I believe this could be a useful class where the neural field is optimised per image. This is used in nerf2mesh, where the pixels must be sampled from a single image to generate rays, which are then used to rasterise the image from the coarse mesh.

I use this pixel sampler in my implementation of nerf2mesh using nerfstudio.

Happy to make any edits if required.

A sampler that selects a specific image and samples all pixels from that image
@jb-ye
Copy link
Collaborator

jb-ye commented Mar 31, 2024

Just some side question: I am also curious why we didn't have something like AllPixelSampler for implementing next_eval_image.

I spent sometime digging into the code and realized that we currently use RandIndicesEvalDataloader to select a random camera and use get_outputs_for_camera which involves a nested call of camera.generate_rays.

The process to obtain rays is not following the typical workflow we set for next_train() and next_eval(), where a pixel sampler is used.

@IamMohitM
Copy link
Contributor Author

IamMohitM commented Apr 1, 2024

That makes sense. I didn't notice the RandIndicesEvalDataloader earlier.

The AllPixelSampler implementation makes it compatible with the next_train() and next_eval() function in the VanillaDataManager class. Both functions assert a requirement for a pixel sampler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants