Nuofan Xu: Denoiser

CMakeLists.txt

@@ -86,15 +86,33 @@ set(sources
     src/utilities.cpp
     )
 
+set(imgui
+    imgui/imconfig.h
+    imgui/imgui.cpp
+    imgui/imgui.h
+    imgui/imgui_draw.cpp
+    imgui/imgui_internal.h
+    imgui/imgui_widgets.cpp
+    imgui/imgui_demo.cpp
+    imgui/imgui_impl_glfw.cpp
+    imgui/imgui_impl_glfw.h
+    imgui/imgui_impl_opengl2.cpp
+    imgui/imgui_impl_opengl2.h
+    imgui/imgui_impl_opengl3.cpp
+    imgui/imgui_impl_opengl3.h
+    )
+
 list(SORT headers)
 list(SORT sources)
+list(SORT imgui)
 
 source_group(Headers FILES ${headers})
 source_group(Sources FILES ${sources})
+source_group(imgui FILES ${imgui})
 
 #add_subdirectory(stream_compaction)  # TODO: uncomment if using your stream compaction
 
-cuda_add_executable(${CMAKE_PROJECT_NAME} ${sources} ${headers})
+cuda_add_executable(${CMAKE_PROJECT_NAME} ${sources} ${headers} ${imgui})
 target_link_libraries(${CMAKE_PROJECT_NAME}
     ${LIBRARIES}
     #stream_compaction  # TODO: uncomment if using your stream compaction

README.md

@@ -1,13 +1,57 @@
 CUDA Path Tracer
 ================
 
-**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 3**
+**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 4**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+Nuofan Xu, Windows 10, AMD Ryzen 3800X, RTX2080 Super
 
-### (TODO: Your README)
+A CUDA-accelerated path tracer: a global-illumination renderer with a denoiser to provide clearer images. 
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+CUDA Denoiser For CUDA Path Tracer
+==================================
 
+The technical details of the denoising technique used in this project is detailed in ["Edge-Avoiding À-Trous Wavelet Transform for fast Global Illumination Filtering"](https://jo.dreggn.org/home/2010_atrous.pdf). The idea is to blur the image while preserving the edges. We all know that noisy image is usually perceived as worse images compared to images that are not sharp enough. By smoothing over large noisy areas but the edges, this will give the appearance of a sharper, denoised image because shapes will remain intact. We can determine the extent of denoising by adjusting weights based on position, normals, and color of the scene elements. 
+
+Gaussian blur has the effect of reducing the image's high-frequency components; a Gaussian blur is thus a low pass filter. Although a true Gaussian blur is effective, it requires a large amount of computation. By implementing the À-Trous Wavelet transform we are able to spread out the coordinates of the Gaussian kernel. This results in far fewer iterations and computation than a true Gaussian but a larger blur area.
+
+Here we can see the difference between the images where there is no denoising (raw image), a blur with no edge detection (simple blur), and the blur with edge detection (Edge Avoiding A-Trous). All of these images went through 10 iterations of the pathtracer.
+
+| raw pathtraced image | simple blur | blur guided by G-buffers |
+|---|---|---|
+|![](p4_img/raw.PNG)|![](p4_img/simple_blur.PNG)|![](p4_img/gbuffer.PNG)|
+
+The visualization of the per pixel position and normals of the objects which are used as reference of weights are also shown below. This gives us some more insight on how they are able to help us with generating weights for edge detection.
+
+| per-pixel normals | per-pixel position |
+|---|---|
+|![](p4_img/pixel_n.PNG)|![](p4_img/pixel_pos.PNG)|
+
+# Performance Analysis
+
+The relationship between running speed of the pathtracer and the filter size applied is investigated. As we can see from the performance plot, the filter size of the denoiser increases the time it takes for the denoiser to run, but the effect does not scale linearly - as the filter size becomes bigger, the increase speed of running time slows down logarithmically. This makes sense because the denoiser will have to consider a larger area of the image each time, and the À-Trous algorithm depends on a log2 of the filter size.
+
+![](p4_img/performance.PNG)
+
+The relationship between running speed of the pathtracer and the image resolution is also investigated. We can see from the plot that as the resolution increases the runtime also increases, because an image with a higher resolution will have more pixels that the pathtracer and denoiser need to look at with each iteration. This time, the increase looks linear.
+
+![](p4_img/resolution.PNG)
+
+## Qualitative Analysis
+
+The effect of filter size on the actual plot is investigated. From the image below, we can see that the appearance of our image does not scale uniformly with filter size.  After filter size is greater than 150, the difference in the images becomes negligible. In contrast, the difference between a filter of small sizes such as size 25 and size 50 is much more substantial. The experiment stops at filter size of 250 as no difference is observed by further increasing the filter size.
+
+| filter size = 25 | filter size = 50 | filter size = 100 | 
+![](p4_img/filter_size.PNG)
+| filter size = 150 |filter size = 200 | filter size = 250 |
+
+As in the previously implemented setting, the ball is specular and the walls are fully diffusive. We can
+observe that denosing effects are different on different materials. Some surfaces such as diffusive surfaces are much more compatible with the blur. This is as expected as a diffused surface is essentially one color. There is not really any information being lost when the kernel is applied to that part of the image. The surface getting blurred is effective at making it look better as there's a high chance that undesired noise are removed and edges are preserved. This is not the case however with refractive and specular materials. These materials generally create specific reflections of light or the surrounding scene. When these reflections get blurred out by the denoiser they no longer truly look like the material they are supposed to be. This is especially noticeable in the refractive material objects where the contrast between the object and the reflections are not as great and the entire interior of the object gets blurred. It no longer looks like glass because the visuals that make it refractive are not seen.
+
+The relationship between lighting condition in the scene and the quality of pathtracing denosing is also investigated. The below image shows a comparison between the denoised image generated using the exact same filter size, iterations, and weights settings. We can easily observe that the lack of light in the second one makes it appear much noisier. Therefore, lighting impacts the effectiveness of the denoiser. A darker scene usually makes it much more difficult for the denoiser to produce a good image. 
+
+| Good Lighting | Poor Lighting |
+
+![](p4_img/lighting.PNG)
+
+## Feedback
+Any feedback on the project is welcome.

cmake/CUDAComputesList.cmake

@@ -60,6 +60,8 @@ IF(    CUDA_COMPUTE_20
     OR CUDA_COMPUTE_70
     OR CUDA_COMPUTE_72
     OR CUDA_COMPUTE_75
+    OR CUDA_COMPUTE_80
+    OR CUDA_COMPUTE_86
     )
     SET(FALLBACK OFF)
 ELSE()
@@ -70,8 +72,8 @@ LIST(LENGTH COMPUTES_DETECTED_LIST COMPUTES_LEN)
 IF(${COMPUTES_LEN} EQUAL 0 AND ${FALLBACK})
     MESSAGE(STATUS "You can use -DCOMPUTES_DETECTED_LIST=\"AB;XY\" (semicolon separated list of CUDA Compute versions to enable the specified computes")
     MESSAGE(STATUS "Individual compute versions flags are also available under CMake Advance options")
-    LIST(APPEND COMPUTES_DETECTED_LIST "30" "50" "60" "70")
-    MESSAGE(STATUS "No computes detected. Fall back to 30, 50, 60 70")
+    LIST(APPEND COMPUTES_DETECTED_LIST "30" "50" "60" "70" "80")
+    MESSAGE(STATUS "No computes detected. Fall back to 30, 50, 60, 70, 80")
 ENDIF()
 
 LIST(LENGTH COMPUTES_DETECTED_LIST COMPUTES_LEN)
@@ -90,7 +92,7 @@ MACRO(SET_COMPUTE VERSION)
 ENDMACRO(SET_COMPUTE)
 
 # Iterate over compute versions. Create variables and enable computes if needed
-FOREACH(VER 20 30 32 35 37 50 52 53 60 61 62 70 72 75)
+FOREACH(VER 20 30 32 35 37 50 52 53 60 61 62 70 72 75 80 86)
     OPTION(CUDA_COMPUTE_${VER} "CUDA Compute Capability ${VER}" OFF)
     MARK_AS_ADVANCED(CUDA_COMPUTE_${VER})
     IF(${CUDA_COMPUTE_${VER}})

cmake/FindGLFW.cmake

@@ -20,66 +20,66 @@
 include(FindPackageHandleStandardArgs)
 
 if (WIN32)
-	# Find include files
-	find_path(
-		GLFW_INCLUDE_DIR
-		NAMES GLFW/glfw3.h
-		PATHS
-		$ENV{PROGRAMFILES}/include
-		${GLFW_ROOT_DIR}/include
-		DOC "The directory where GLFW/glfw.h resides")
+  # Find include files
+  find_path(
+    GLFW_INCLUDE_DIR
+    NAMES GLFW/glfw3.h
+    PATHS
+    $ENV{PROGRAMFILES}/include
+    ${GLFW_ROOT_DIR}/include
+    DOC "The directory where GLFW/glfw.h resides")
 
-	# Use glfw3.lib for static library
-	if (GLFW_USE_STATIC_LIBS)
-		set(GLFW_LIBRARY_NAME glfw3)
-	else()
-		set(GLFW_LIBRARY_NAME glfw3dll)
-	endif()
+  # Use glfw3.lib for static library
+  if (GLFW_USE_STATIC_LIBS)
+    set(GLFW_LIBRARY_NAME glfw3)
+  else()
+    set(GLFW_LIBRARY_NAME glfw3dll)
+  endif()
 
-	# Find library files
-	find_library(
-		GLFW_LIBRARY
-		NAMES ${GLFW_LIBRARY_NAME}
-		PATHS
-		$ENV{PROGRAMFILES}/lib
-		${GLFW_ROOT_DIR}/lib)
+  # Find library files
+  find_library(
+    GLFW_LIBRARY
+    NAMES ${GLFW_LIBRARY_NAME}
+    PATHS
+    $ENV{PROGRAMFILES}/lib
+    ${GLFW_ROOT_DIR}/lib)
 
-	unset(GLFW_LIBRARY_NAME)
+  unset(GLFW_LIBRARY_NAME)
 else()
-	# Find include files
-	find_path(
-		GLFW_INCLUDE_DIR
-		NAMES GLFW/glfw.h
-		PATHS
-		/usr/include
-		/usr/local/include
-		/sw/include
-		/opt/local/include
-		DOC "The directory where GL/glfw.h resides")
+  # Find include files
+  find_path(
+    GLFW_INCLUDE_DIR
+    NAMES GLFW/glfw.h
+    PATHS
+    /usr/include
+    /usr/local/include
+    /sw/include
+    /opt/local/include
+    DOC "The directory where GL/glfw.h resides")
 
-	# Find library files
-	# Try to use static libraries
-	find_library(
-		GLFW_LIBRARY
-		NAMES glfw3
-		PATHS
-		/usr/lib64
-		/usr/lib
-		/usr/local/lib64
-		/usr/local/lib
-		/sw/lib
-		/opt/local/lib
-		${GLFW_ROOT_DIR}/lib
-		DOC "The GLFW library")
+  # Find library files
+  # Try to use static libraries
+  find_library(
+    GLFW_LIBRARY
+    NAMES glfw3
+    PATHS
+    /usr/lib64
+    /usr/lib
+    /usr/local/lib64
+    /usr/local/lib
+    /sw/lib
+    /opt/local/lib
+    ${GLFW_ROOT_DIR}/lib
+    DOC "The GLFW library")
 endif()
 
 # Handle REQUIRD argument, define *_FOUND variable
 find_package_handle_standard_args(GLFW DEFAULT_MSG GLFW_INCLUDE_DIR GLFW_LIBRARY)
 
 # Define GLFW_LIBRARIES and GLFW_INCLUDE_DIRS
 if (GLFW_FOUND)
-	set(GLFW_LIBRARIES ${OPENGL_LIBRARIES} ${GLFW_LIBRARY})
-	set(GLFW_INCLUDE_DIRS ${GLFW_INCLUDE_DIR})
+  set(GLFW_LIBRARIES ${OPENGL_LIBRARIES} ${GLFW_LIBRARY})
+  set(GLFW_INCLUDE_DIRS ${GLFW_INCLUDE_DIR})
 endif()
 
 # Hide some variables

cmake/FindGLM.cmake

@@ -2,12 +2,12 @@
 # Find GLM
 #
 # Try to find GLM : OpenGL Mathematics.
-# This module defines 
+# This module defines
 # - GLM_INCLUDE_DIRS
 # - GLM_FOUND
 #
 # The following variables can be set as arguments for the module.
-# - GLM_ROOT_DIR : Root library directory of GLM 
+# - GLM_ROOT_DIR : Root library directory of GLM
 #
 # References:
 # - https://github.com/Groovounet/glm/blob/master/util/FindGLM.cmake
@@ -18,34 +18,34 @@
 include(FindPackageHandleStandardArgs)
 
 if (WIN32)
-	# Find include files
-	find_path(
-		GLM_INCLUDE_DIR
-		NAMES glm/glm.hpp
-		PATHS
-		$ENV{PROGRAMFILES}/include
-		${GLM_ROOT_DIR}/include
-		DOC "The directory where glm/glm.hpp resides")
+  # Find include files
+  find_path(
+    GLM_INCLUDE_DIR
+    NAMES glm/glm.hpp
+    PATHS
+    $ENV{PROGRAMFILES}/include
+    ${GLM_ROOT_DIR}/include
+    DOC "The directory where glm/glm.hpp resides")
 else()
-	# Find include files
-	find_path(
-		GLM_INCLUDE_DIR
-		NAMES glm/glm.hpp
-		PATHS
-		/usr/include
-		/usr/local/include
-		/sw/include
-		/opt/local/include
-		${GLM_ROOT_DIR}/include
-		DOC "The directory where glm/glm.hpp resides")
+  # Find include files
+  find_path(
+    GLM_INCLUDE_DIR
+    NAMES glm/glm.hpp
+    PATHS
+    /usr/include
+    /usr/local/include
+    /sw/include
+    /opt/local/include
+    ${GLM_ROOT_DIR}/include
+    DOC "The directory where glm/glm.hpp resides")
 endif()
 
 # Handle REQUIRD argument, define *_FOUND variable
 find_package_handle_standard_args(GLM DEFAULT_MSG GLM_INCLUDE_DIR)
 
 # Define GLM_INCLUDE_DIRS
 if (GLM_FOUND)
-	set(GLM_INCLUDE_DIRS ${GLM_INCLUDE_DIR})
+  set(GLM_INCLUDE_DIRS ${GLM_INCLUDE_DIR})
 endif()
 
 # Hide some variables