Merge pull request #826 from zeux/vc1!

vertexcodec: Stabilize v1 encoding

Makefile

@@ -55,7 +55,7 @@ WASM_DECODER_SOURCES=src/vertexcodec.cpp src/indexcodec.cpp src/vertexfilter.cpp
 WASM_DECODER_EXPORTS=meshopt_decodeVertexBuffer meshopt_decodeIndexBuffer meshopt_decodeIndexSequence meshopt_decodeFilterOct meshopt_decodeFilterQuat meshopt_decodeFilterExp sbrk __wasm_call_ctors
 
 WASM_ENCODER_SOURCES=src/vertexcodec.cpp src/indexcodec.cpp src/vertexfilter.cpp src/vcacheoptimizer.cpp src/vfetchoptimizer.cpp src/spatialorder.cpp tools/wasmstubs.cpp
-WASM_ENCODER_EXPORTS=meshopt_encodeVertexBuffer meshopt_encodeVertexBufferBound meshopt_encodeIndexBuffer meshopt_encodeIndexBufferBound meshopt_encodeIndexSequence meshopt_encodeIndexSequenceBound meshopt_encodeVertexVersion meshopt_encodeIndexVersion meshopt_encodeFilterOct meshopt_encodeFilterQuat meshopt_encodeFilterExp meshopt_optimizeVertexCache meshopt_optimizeVertexCacheStrip meshopt_optimizeVertexFetchRemap meshopt_spatialSortRemap sbrk __wasm_call_ctors
+WASM_ENCODER_EXPORTS=meshopt_encodeVertexBuffer meshopt_encodeVertexBufferBound meshopt_encodeVertexBufferLevel meshopt_encodeIndexBuffer meshopt_encodeIndexBufferBound meshopt_encodeIndexSequence meshopt_encodeIndexSequenceBound meshopt_encodeVertexVersion meshopt_encodeIndexVersion meshopt_encodeFilterOct meshopt_encodeFilterQuat meshopt_encodeFilterExp meshopt_optimizeVertexCache meshopt_optimizeVertexCacheStrip meshopt_optimizeVertexFetchRemap meshopt_spatialSortRemap sbrk __wasm_call_ctors
 
 WASM_SIMPLIFIER_SOURCES=src/simplifier.cpp src/vfetchoptimizer.cpp tools/wasmstubs.cpp
 WASM_SIMPLIFIER_EXPORTS=meshopt_simplify meshopt_simplifyWithAttributes meshopt_simplifyScale meshopt_simplifyPoints meshopt_optimizeVertexFetchRemap sbrk __wasm_call_ctors

README.md

@@ -177,7 +177,11 @@ assert(resvb == 0 && resib == 0);
 
 Note that vertex encoding assumes that vertex buffer was optimized for vertex fetch, and that vertices are quantized; index encoding assumes that the vertex/index buffers were optimized for vertex cache and vertex fetch. Feeding unoptimized data into the encoders will produce poor compression ratios. Both codecs are lossless - the only lossy step is quantization that happens before encoding.
 
-Decoding functions are heavily optimized and can directly target write-combined memory; you can expect both decoders to run at 1-3 GB/s on modern desktop CPUs. Compression ratios depend on the data; vertex data compression ratio is typically around 2-4x (compared to already quantized data), index data compression ratio is around 5-6x (compared to raw 16-bit index data). General purpose lossless compressors can further improve on these results.
+Decoding functions are heavily optimized and can directly target write-combined memory; you can expect both decoders to run at 3-5 GB/s on modern desktop CPUs. Compression ratios depend on the data; vertex data compression ratio is typically around 2-4x (compared to already quantized data), index data compression ratio is around 5-6x (compared to raw 16-bit index data). General purpose lossless compressors can further improve on these results.
+
+For additional improvements in compression ratio and decoding performance, it is recommended to switch to vertex codec v1 (via `meshopt_encodeVertexVersion(1)`). This will result in smaller outputs that decode faster, and provide additional control over compression level - `meshopt_encodeVertexBuffer` will use compression level 2 by default, but using `meshopt_encodeVertexBufferLevel` allows to improve compression in certain cases by using level 3, or to reduce compression ratio and improve encoding speed by using level 1. Note that v1 format requires meshoptimizer v0.23 or later for decoding.
+
+When data is bit packed, using v1 vertex codec (via `meshopt_encodeVertexVersion(1)`) and specifying compression level 3 (`meshopt_encodeVertexBufferLevel`) can improve the compression further by redistributing bits between components.
 
 Index buffer codec only supports triangle list topology; when encoding triangle strips or line lists, use `meshopt_encodeIndexSequence`/`meshopt_decodeIndexSequence` instead. This codec typically encodes indices into ~1 byte per index, but compressing the results further with a general purpose compressor can improve the results to 1-3 bits per index.
 
@@ -237,7 +241,7 @@ For optimal compression results, the values must be quantized to small integers.
 For single-precision floating-point data, it's recommended to use `meshopt_quantizeFloat` to remove entropy from the lower bits of the mantissa. Due to current limitations of the codec, the bit count needs to be 15 (23-8) for good results (7 can be used for more extreme compression).
 For normal or tangent vectors, using octahedral encoding is recommended over three components as it reduces redundancy. Similarly to other quantized values, consider using 10-12 bits per component instead of 16.
 
-> Note: vertex codec v0 is limited to taking advantage of redundancy in high bits of each byte. Because of this, packing multiple 10-bit values into 32 bits will reduce compression ratio, and when storing a 12-bit value in 16 bits, high bits should be zeroed out. This limitation may be lifted in future versions of the codec.
+When data is bit packed, using v1 vertex codec (via `meshopt_encodeVertexVersion(1)`) and specifying compression level 3 (`meshopt_encodeVertexBufferLevel`) can improve the compression further by redistributing bits between components.
 
 To further leverage the inherent structure of some data, the preparation stage can use filters that encode and decode the data in a lossy manner. This is similar to quantization but can be used without having to change the shader code. After decoding, the filter transformation needs to be reversed. This library provides three filters:
 

demo/main.cpp

@@ -1407,7 +1407,7 @@ void processDev(const char* path)
 
 	meshopt_encodeVertexVersion(0);
 	encodeVertex<PackedVertex>(copy, "L");
-	meshopt_encodeVertexVersion(0xe);
+	meshopt_encodeVertexVersion(1);
 	encodeVertex<PackedVertex>(copy, "0", 0);
 	encodeVertex<PackedVertex>(copy, "1", 1);
 	encodeVertex<PackedVertex>(copy, "2", 2);
@@ -1427,7 +1427,7 @@ int main(int argc, char** argv)
 {
 	void runTests();
 
-	meshopt_encodeVertexVersion(0);
+	meshopt_encodeVertexVersion(1);
 	meshopt_encodeIndexVersion(1);
 
 	if (argc == 1)

demo/tests.cpp

@@ -59,7 +59,7 @@ static const unsigned char kVertexDataV0[] = {
 };
 
 static const unsigned char kVertexDataV1[] = {
-    0xae, 0xee, 0xaa, 0xee, 0x00, 0x4b, 0x4b, 0x4b, 0x00, 0x00, 0x4b, 0x00, 0x00, 0x7d, 0x7d, 0x7d,
+    0xa1, 0xee, 0xaa, 0xee, 0x00, 0x4b, 0x4b, 0x4b, 0x00, 0x00, 0x4b, 0x00, 0x00, 0x7d, 0x7d, 0x7d,
     0x00, 0x00, 0x7d, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x62, 0x00, 0x62, // clang-format :-/
 };
@@ -68,7 +68,7 @@ static const unsigned char kVertexDataV1[] = {
 // the encoder that exercised all features of the format; because of this it is much larger
 // and will never be produced by the encoder itself.
 static const unsigned char kVertexDataV1Custom[] = {
-    0xae, 0xd4, 0x94, 0xd4, 0x01, 0x0e, 0x00, 0x58, 0x57, 0x58, 0x02, 0x02, 0x12, 0x00, 0x00, 0x00,
+    0xa1, 0xd4, 0x94, 0xd4, 0x01, 0x0e, 0x00, 0x58, 0x57, 0x58, 0x02, 0x02, 0x12, 0x00, 0x00, 0x00,
     0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x58, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00,
     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
@@ -2060,29 +2060,26 @@ void runTests()
 	encodeIndexSequenceEmpty();
 
 	decodeVertexV0();
-	encodeVertexMemorySafe();
-	decodeVertexMemorySafe();
-	decodeVertexRejectExtraBytes();
-	decodeVertexRejectMalformedHeaders();
-
 	decodeVertexV1();
 	decodeVertexV1Custom();
 
 	for (int version = 0; version <= 1; ++version)
 	{
-		meshopt_encodeVertexVersion(version == 1 ? 0xe : version);
+		meshopt_encodeVertexVersion(version);
 
+		decodeVertexMemorySafe();
+		decodeVertexRejectExtraBytes();
+		decodeVertexRejectMalformedHeaders();
 		decodeVertexBitGroups();
 		decodeVertexBitGroupSentinels();
 		decodeVertexDeltas();
 		decodeVertexBitXor();
 		decodeVertexLarge();
 		decodeVertexSmall();
 		encodeVertexEmpty();
+		encodeVertexMemorySafe();
 	}
 
-	meshopt_encodeVertexVersion(0);
-
 	decodeFilterOct8();
 	decodeFilterOct12();
 	decodeFilterQuat12();

gltf/gltfpack.cpp

@@ -1514,15 +1514,13 @@ int main(int argc, char** argv)
 			settings.compress = true;
 			settings.fallback = true;
 		}
-#ifndef NDEBUG
 		else if (strcmp(arg, "-ce") == 0)
 		{
-			fprintf(stderr, "Warning: experimental compression will produce files that may not decode in the future!\n");
-			meshopt_encodeVertexVersion(0xe);
+			fprintf(stderr, "Warning: experimental compression will produce files that are not compliant with EXT_meshopt_compression\n");
+			meshopt_encodeVertexVersion(1);
 			settings.compress = true;
 			settings.compressmore = true;
 		}
-#endif
 		else if (strcmp(arg, "-v") == 0)
 		{
 			settings.verbose = 1;