Project 5: Utkarsh Dwivedi

.gitignore

@@ -0,0 +1,2 @@
+[Bb]in/
+[Bb]uild/

README.md

@@ -1,12 +1,120 @@
 Vulkan Grass Rendering
 ==================================
 
-**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
+**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 4**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Utkarsh Dwivedi
+  * [LinkedIn](https://www.linkedin.com/in/udwivedi/), [personal website](https://utkarshdwivedi.com/)
+* Tested on: Windows 11 Home, AMD Ryzen 7 5800H @ 3.2GHz 16 GB, Nvidia GeForce RTX 3060 Laptop GPU 6 GB
 
-### (TODO: Your README)
+![](img/grassHighlight.gif)
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+## Introduction
+
+This is a Vulkan based grass renderer heavily based on the paper [Responsive Real-Time Grass Rendering for General 3D Scenes](https://www.cg.tuwien.ac.at/research/publications/2017/JAHRMANN-2017-RRTG/JAHRMANN-2017-RRTG-draft.pdf). It also draws inspiration from certain elements of [Ghost of Tsushima's procedural grass rendering pipeline](https://www.youtube.com/watch?v=Ibe1JBF5i5Y).
+
+## Overview
+
+|Overview of the process|
+|:-:|
+|![](img/processDiagram.png)|
+
+### Representing a grass glade
+
+![](img/grassRepresentation.png)
+
+A grass blade is represented according to the above image from the grass rendering paper. Each blade struct contains:
+- `v0`: position on a **2D plane** where it should be placed,
+- `v1`: control point for a Bezier curve (this idea is also employed in Ghost of Tsushima),
+- `v2`: end point of the blade
+- `up`: vector containing the up direction, in this case `[0,1,0]`, but this can easily be modified to by dynamic based on the ground plane
+- `width`: the width of the blade,
+- `height`: the *length* of the blade (the paper's terminology calls this the height, but I think length makes more sense),
+- `direction`: an angle that represents the orientation of the blade around the up-vector
+
+### Constructing a scene
+
+The paper uses poisson-disk sampling to generate blade positions on the plane. Ghost of Tsushima's pipeline uses a grid of multiple `512x512` artist-authored textures to define which blade of grass should be placed in which areas. 
+
+This project generates the positions using a simple random number generator, but the Ghost of Tsushima method is an interesting direction for the future.
+
+### GPU compute pipeline
+
+This is where the physical model for the grass blades is evaluated and grass blades are culled.
+
+First, orientation and distance culling is applied as described in the paper. Orientation culling culls out blades that are parallel to the view vector, and distance culling culls out increasing number of blades by putting them in buckets of distance from the camera.
+
+|Orientation culling|Distance culling|
+|:-:|:-:|
+|<img src = "img/orientationCulling.gif" width=400>|<img src = "img/distanceCulling.gif" width=400>|
+
+Next, physical forces are evaluated (wind and gravity). The wind force is a simple 2D perlin noise, similar to the way Ghost of Tsushima handles wind.
+
+|Perlin noise based wind|
+|:-:|
+|<img src = "img/wind_perlin.gif" width=400>|
+
+A recovery force is applied based on a stiffness coefficient to return the grass blade to its initial pose. This uses Hooke's Law of elasticity to calculate the blade's position.
+
+`(initial pose - updated pose) * stiffness coefficient`
+
+Applying these forces could lead to invalid configurations for `v2`, the tip of the blade. This is corrected by clamping its position to always be above the ground plane. A final correction is applied to maintain the length of the blade by adjusting v1 and v2.
+
+Finally, frustum culling is applied to cull out blades that do not lie in the frustum and user-defined near and far clip planes. In the below image, the size of the frustum is *very slightly* reduced to show the effect of blades outside the frustum being culled.
+
+|Frustum culling|
+|:-:|
+|<img src = "img/frustumCulling.gif" width=400>|
+
+The blades that remain after culling are sent to the graphics pipeline to be tessellated.
+
+### GPU graphics pipeline
+
+**Vertex Shader**
+
+The vertex shader is a simple pass-through shader that passes information on to the tessellation control shader after converting world-space positions to camera-space positions.
+
+**Tessellation Control Shader**
+
+The tessellation control shader defines the tessellation levels (blade LODs) on a grass blade based on the distance of the blade from the camera.
+
+|Blade LODs based on distance from camera|
+|:-:|
+|<img src = "img/LOD_tesc.gif" width=400>|
+
+**Tessellation Evaluation Shader**
+
+The tessellation evaluation shader does the actual tessellation of the vertices. This is done using the DeCastlejau algorithm to evaluate each vertex's position on along bezier curve and thickness value. Refer the paper for more details.
+
+**Fragment Shader**
+
+The fragment shader applies a simple lambertian shading model to colour the grass blades.
+
+## Performance Analysis
+
+For performance analysis, a scene resolution of 1280x720 was used.
+
+### Culling
+
+For analysing culling, the number of grass blades in the scene was kept at **2<sup>13</sup>**.
+
+|Frame rate at different culling strategies|
+|:-:|
+|![](img/fpsVsCulling.png)|
+
+This is pretty expected. Each culling method is slightly more advanced and improves performance individually. When all culling is applied after physics calculations, the performance gain is improved further. Changing this to applying **orientation** and **distance** culling before computing physics, and only applying the **frustum** culling after computing physics, similar to Ghost of Tsushima, has an further improved performance. This has potential of improvement by implementing occlusion culling, which would really start showing its potential with very high blade counts.
+
+### Varying blade counts
+
+For analysing FPS with increasing number of grass blades, no culling was tested against "pre+post" culling.
+
+|Frame rate at increasing blade counts|
+|:-:|
+|![](img/fpsVsBlades.png)|
+
+The frame rate really starts to take a hit once the blade count increases logarithmically beyond a very small number. This is expected, and this is exactly where strategies like occlusion culling, tiling (ref. Ghost of Tsushima talk), etc. will help.
+
+## References
+
+- [Responsive Real-Time Grass Rendering for General 3D Scenes](https://www.cg.tuwien.ac.at/research/publications/2017/JAHRMANN-2017-RRTG/JAHRMANN-2017-RRTG-draft.pdf)
+- [Sucker Punch Productions, Ghost of Tsushima's procedural grass rendering pipeline talk](https://www.youtube.com/watch?v=Ibe1JBF5i5Y)

src/Blades.cpp

@@ -45,16 +45,17 @@ Blades::Blades(Device* device, VkCommandPool commandPool, float planeDim) : Mode
     indirectDraw.firstInstance = 0;
 
     BufferUtils::CreateBufferFromData(device, commandPool, blades.data(), NUM_BLADES * sizeof(Blade), VK_BUFFER_USAGE_STORAGE_BUFFER_BIT, bladesBuffer, bladesBufferMemory);
-    BufferUtils::CreateBuffer(device, NUM_BLADES * sizeof(Blade), VK_BUFFER_USAGE_STORAGE_BUFFER_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, culledBladesBuffer, culledBladesBufferMemory);
+    // This one is also going to be drawn so we need to also flag it as vertex buffer bit
+    BufferUtils::CreateBuffer(device, NUM_BLADES * sizeof(Blade), VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | VK_BUFFER_USAGE_STORAGE_BUFFER_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, renderableBladesBuffer, renderableBladesBufferMemory);
     BufferUtils::CreateBufferFromData(device, commandPool, &indirectDraw, sizeof(BladeDrawIndirect), VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT, numBladesBuffer, numBladesBufferMemory);
 }
 
 VkBuffer Blades::GetBladesBuffer() const {
     return bladesBuffer;
 }
 
-VkBuffer Blades::GetCulledBladesBuffer() const {
-    return culledBladesBuffer;
+VkBuffer Blades::GetRenderableBladesBuffer() const {
+    return renderableBladesBuffer;
 }
 
 VkBuffer Blades::GetNumBladesBuffer() const {
@@ -64,8 +65,8 @@ VkBuffer Blades::GetNumBladesBuffer() const {
 Blades::~Blades() {
     vkDestroyBuffer(device->GetVkDevice(), bladesBuffer, nullptr);
     vkFreeMemory(device->GetVkDevice(), bladesBufferMemory, nullptr);
-    vkDestroyBuffer(device->GetVkDevice(), culledBladesBuffer, nullptr);
-    vkFreeMemory(device->GetVkDevice(), culledBladesBufferMemory, nullptr);
+    vkDestroyBuffer(device->GetVkDevice(), renderableBladesBuffer, nullptr);
+    vkFreeMemory(device->GetVkDevice(), renderableBladesBufferMemory, nullptr);
     vkDestroyBuffer(device->GetVkDevice(), numBladesBuffer, nullptr);
     vkFreeMemory(device->GetVkDevice(), numBladesBufferMemory, nullptr);
 }

src/Blades.h

@@ -7,10 +7,10 @@
 constexpr static unsigned int NUM_BLADES = 1 << 13;
 constexpr static float MIN_HEIGHT = 1.3f;
 constexpr static float MAX_HEIGHT = 2.5f;
-constexpr static float MIN_WIDTH = 0.1f;
-constexpr static float MAX_WIDTH = 0.14f;
+constexpr static float MIN_WIDTH = 0.05f;
+constexpr static float MAX_WIDTH = 0.07f;
 constexpr static float MIN_BEND = 7.0f;
-constexpr static float MAX_BEND = 13.0f;
+constexpr static float MAX_BEND = 10.0f;
 
 struct Blade {
     // Position and direction
@@ -72,17 +72,17 @@ struct BladeDrawIndirect {
 class Blades : public Model {
 private:
     VkBuffer bladesBuffer;
-    VkBuffer culledBladesBuffer;
+    VkBuffer renderableBladesBuffer;
     VkBuffer numBladesBuffer;
 
     VkDeviceMemory bladesBufferMemory;
-    VkDeviceMemory culledBladesBufferMemory;
+    VkDeviceMemory renderableBladesBufferMemory;
     VkDeviceMemory numBladesBufferMemory;
 
 public:
     Blades(Device* device, VkCommandPool commandPool, float planeDim);
     VkBuffer GetBladesBuffer() const;
-    VkBuffer GetCulledBladesBuffer() const;
+    VkBuffer GetRenderableBladesBuffer() const;
     VkBuffer GetNumBladesBuffer() const;
     ~Blades();
 };

src/Camera.cpp

@@ -15,6 +15,7 @@ Camera::Camera(Device* device, float aspectRatio) : device(device) {
     cameraBufferObject.viewMatrix = glm::lookAt(glm::vec3(0.0f, 1.0f, 10.0f), glm::vec3(0.0f, 1.0f, 0.0f), glm::vec3(0.0f, 1.0f, 0.0f));
     cameraBufferObject.projectionMatrix = glm::perspective(glm::radians(45.0f), aspectRatio, 0.1f, 100.0f);
     cameraBufferObject.projectionMatrix[1][1] *= -1; // y-coordinate is flipped
+    cameraBufferObject.invViewMatrix = glm::inverse(cameraBufferObject.viewMatrix);
 
     BufferUtils::CreateBuffer(device, sizeof(CameraBufferObject), VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT, buffer, bufferMemory);
     vkMapMemory(device->GetVkDevice(), bufferMemory, 0, sizeof(CameraBufferObject), 0, &mappedData);
@@ -37,7 +38,7 @@ void Camera::UpdateOrbit(float deltaX, float deltaY, float deltaZ) {
     glm::mat4 finalTransform = glm::translate(glm::mat4(1.0f), glm::vec3(0.0f)) * rotation * glm::translate(glm::mat4(1.0f), glm::vec3(0.0f, 1.0f, r));
 
     cameraBufferObject.viewMatrix = glm::inverse(finalTransform);
-
+    cameraBufferObject.invViewMatrix = finalTransform;
     memcpy(mappedData, &cameraBufferObject, sizeof(CameraBufferObject));
 }
 

src/Camera.h

@@ -7,6 +7,7 @@
 struct CameraBufferObject {
   glm::mat4 viewMatrix;
   glm::mat4 projectionMatrix;
+  glm::mat4 invViewMatrix;
 };
 
 class Camera {

src/Instance.cpp

@@ -258,6 +258,9 @@ void Instance::PickPhysicalDevice(std::vector<const char*> deviceExtensions, Que
             }
         }
 
+        VkPhysicalDeviceProperties props;
+        vkGetPhysicalDeviceProperties(device, &props);
+
         if (requiredQueues[QueueFlags::Present]) {
             // Get basic surface capabilities
             vkGetPhysicalDeviceSurfaceCapabilitiesKHR(device, surface, &surfaceCapabilities);