Merge branch 'fast-spendability'

.github/workflows/pull-request.yml

@@ -199,7 +199,7 @@ jobs:
         timeout-minutes: 30
         uses: ./.github/actions/setup
       - name: Build and test
-        timeout-minutes: 25
+        timeout-minutes: 30
         run: |
           ./gradlew test
       - name: Collect Artifacts
@@ -292,7 +292,7 @@ jobs:
         timeout-minutes: 30
         uses: ./.github/actions/setup
       - name: Build and test
-        timeout-minutes: 25
+        timeout-minutes: 30
         env:
           ORG_GRADLE_PROJECT_ZCASH_EMULATOR_WTF_API_KEY: ${{ secrets.EMULATOR_WTF_API_KEY }}
         run: |
@@ -336,7 +336,7 @@ jobs:
         run: |
           keytool -genkey -v -keystore $SIGNING_KEY_PATH -keypass android -storepass android -alias androiddebugkey -keyalg RSA -keysize 2048 -validity 100000 -dname "CN=, OU=, O=Test, L=, S=, C=" -noprompt
       - name: Build
-        timeout-minutes: 30
+        timeout-minutes: 35
         env:
           ORG_GRADLE_PROJECT_ZCASH_RELEASE_KEYSTORE_PATH: ${{ format('{0}/release.jks', env.home) }}
           ORG_GRADLE_PROJECT_ZCASH_RELEASE_KEYSTORE_PASSWORD: android
@@ -395,7 +395,7 @@ jobs:
         with:
           name: Demo app release binaries
       - name: Robo test
-        timeout-minutes: 15
+        timeout-minutes: 20
         env:
           # Path depends on `release_build` job, plus path of `Download a single artifact` step
           BINARIES_ZIP_PATH: binaries.zip

.gitignore

@@ -51,6 +51,7 @@ captures/
 .idea/workspace.xml
 .idea/protoeditor.xml
 .idea/appInsightsSettings.xml
+.idea/migrations.xml
 *.iml
 
 # Keystore files

CHANGELOG.md

@@ -1,8 +1,60 @@
-# Change Log
+# Changelog
+All notable changes to this library will be documented in this file.
 
-## 1.21.0-beta01
-Note: This is the last _1.x_ version release. The upcoming version _2.0_ brings the **Spend-before-Sync** feature, 
-which speeds up discovering the wallet's spendable balance.  
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this library adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [2.0.0-rc.1] - 2023-09-12
+
+### Notable Changes
+
+- `CompactBlockProcessor` now processes compact blocks from the lightwalletd
+  server using the **Spend-before-Sync** algorithm, which allows scanning of
+  wallet blocks to be performed in arbitrary order and optimized to make it
+  possible to spend received notes without waiting for synchronization to be
+  complete. This feature shortens the time until a wallet's spendable balance
+  can be used.
+- The block synchronization mechanism is additionally about one-third faster
+  thanks to an optimized `CompactBlockProcessor.SYNC_BATCH_SIZE` (issue **#1206**).
+
+### Removed
+- `CompactBlockProcessor.ProcessorInfo.lastSyncHeight` no longer had a
+  well-defined meaning after implementation of the **SpendBeforeSync**
+  synchronization algorithm and has been removed.
+  `CompactBlockProcessor.ProcessorInfo.overallSyncRange` provides related
+  information.
+- `CompactBlockProcessor.ProcessorInfo.isSyncing`. Use `Synchronizer.status` instead.
+- `CompactBlockProcessor.ProcessorInfo.syncProgress`. Use `Synchronizer.progress` instead.
+- `alsoClearBlockCache` parameter from rewind functions of `Synchronizer` and
+  `CompactBlockProcessor`, as it has no effect on the current behaviour of
+  these functions.
+- Internally, we removed access to the shared block table from the Kotlin
+  layer, which resulted in eliminating these APIs:
+  - `SdkSynchronizer.findBlockHash()`
+  - `SdkSynchronizer.findBlockHashAsHex()`
+
+### Changed
+- `CompactBlockProcessor.quickRewind()` and `CompactBlockProcessor.rewindToNearestHeight()`
+  now might fail due to internal changes in getting scanned height. Thus, these
+  functions now return `Boolean` results.
+- `Synchronizer.new()` and `PersistableWallet` APIs require a new
+  `walletInitMode` parameter of type `WalletInitMode`, which describes wallet
+  initialization mode. See related function and sealed class documentation.
+
+### Fixed
+- `Synchronizer.getMemos()` now correctly returns a flow of strings for sent
+  and received transactions. Issue **#1154**.
+- `CompactBlockProcessor` now triggers transaction polling while block
+  synchronization is in progress as expected. Clients will be notified shortly
+  after every new transaction is discovered via `Synchronizer.transactions`
+  API. Issue **#1170**.
+
+## [1.21.0-beta01]
+
+Note: This is the last _1.x_ version release. The upcoming version _2.0_ brings the **Spend-before-Sync** feature,
+which speeds up discovering the wallet's spendable balance.
 
 ### Changed
 - Updated dependencies:
@@ -18,7 +70,7 @@ which speeds up discovering the wallet's spendable balance.
 
 ## 1.20.0-beta01
 - The SDK internally migrated from `BackendExt` rust backend extension functions to more type-safe `TypesafeBackend`.
-- `Synchronizer.getMemos()` now internally handles expected `RuntimeException` from the rust layer and transforms it 
+- `Synchronizer.getMemos()` now internally handles expected `RuntimeException` from the rust layer and transforms it
   in an empty string.
 
 ## 1.19.0-beta01
@@ -28,16 +80,16 @@ which speeds up discovering the wallet's spendable balance.
 ### Fixed
 - `TransactionOverview` object returned with `SdkSynchronizer.transactions` now contains a correct `TransactionState.
   Pending` in case of the transaction is mined,but not fully confirmed.
-- When the SDK internally works with a recently created transaction there was a moment in which could the transaction 
+- When the SDK internally works with a recently created transaction there was a moment in which could the transaction
   causes the SDK to crash, because of its invalid mined height. Fixed now.
 
 ## 1.18.0-beta01
-- Synchronizer's functions `getUnifiedAddress`, `getSaplingAddress`, `getTransparentAddress`, and `refreshUtxos` now 
-  do not provide `Account.DEFAULT` value for the account argument. As accounts are not fully supported by the SDK 
-  yet, the caller should explicitly set Account.DEFAULT as the account argument to keep the same behavior.   
+- Synchronizer's functions `getUnifiedAddress`, `getSaplingAddress`, `getTransparentAddress`, and `refreshUtxos` now
+  do not provide `Account.DEFAULT` value for the account argument. As accounts are not fully supported by the SDK
+  yet, the caller should explicitly set Account.DEFAULT as the account argument to keep the same behavior.
 - Gradle 8.1.1
 - AGP 8.0.2
- 
+
 ## 1.17.0-beta01
 - Transparent fund balances are now displayed almost immediately
 - Synchronization of shielded balances and transaction history is about 30% faster
@@ -46,7 +98,7 @@ which speeds up discovering the wallet's spendable balance.
 - `Synchronizer.progress` now returns `Flow<PercentDecimal>` instead of `Flow<Int>`. PercentDecimal is a type-safe model. Use `PercentDecimal.toPercentage()` to get a number within 0-100% scale.
 - `Synchronizer.clearedTransactions` has been renamed to `Synchronizer.transactions` and includes sent, received, and pending transactions.  Synchronizer APIs for listing sent, received, and pending transactions have been removed.  Clients can determine whether a transaction is sent, received, or pending by filtering the `TransactionOverview` objects returned by `Synchronizer.transactions`
 - `Synchronizer.send()` and `shieldFunds()` are now `suspend` functions with `Long` return values representing the ID of the newly created transaction.  Errors are reported by thrown exceptions.
- - `DerivationTool` is now an interface, rather than an `object`, which makes it easier to inject alternative implementations into tests.  To adapt to the new API, replace calls to `DerivationTool.methodName()` with `DerivationTool.getInstance().methodName()`.  
+ - `DerivationTool` is now an interface, rather than an `object`, which makes it easier to inject alternative implementations into tests.  To adapt to the new API, replace calls to `DerivationTool.methodName()` with `DerivationTool.getInstance().methodName()`.
  - `DerivationTool` methods are no longer suspending, which should make it easier to call them in various situations.  Obtaining a `DerivationTool` instance via `DerivationTool.getInstance()` frontloads the need for a suspending call.
  - `DerivationTool.deriveUnifiedFullViewingKeys()` no longer has a default argument for `numberOfAccounts`.  Clients should now pass `DerivationTool.DEFAULT_NUMBER_OF_ACCOUNTS` as the value. Note that the SDK does not currently have proper support for multiple accounts.
  - The SDK's internals for connecting with librustzcash have been refactored to a separate Gradle module `backend-lib` (and therefore a separate artifact) which is a transitive dependency of the Zcash Android SDK.  SDK consumers that use Gradle dependency locks may notice this difference, but otherwise it should be mostly an invisible change.
@@ -59,7 +111,7 @@ which speeds up discovering the wallet's spendable balance.
 ## 1.15.0-beta01
 ### Changed
 - A new package `sdk-incubator-lib` is now available as a public API.  This package contains experimental APIs that may be promoted to the SDK in the future.  The APIs in this package are not guaranteed to be stable, and may change at any time.
-- `Synchronizer.refreshUtxos` now takes `Account` type as first parameter instead of transparent address of type 
+- `Synchronizer.refreshUtxos` now takes `Account` type as first parameter instead of transparent address of type
     `String`, and thus it downloads all UTXOs for the given account addresses. The Account object provides a default `0` index Account with `Account.DEFAULT`.
 
 ## 1.14.0-beta01
@@ -68,11 +120,11 @@ which speeds up discovering the wallet's spendable balance.
 
 ## 1.13.0-beta01
 ### Changed
-- The SDK's internal networking has been refactored to a separate Gradle module `lightwallet-client-lib` (and 
+- The SDK's internal networking has been refactored to a separate Gradle module `lightwallet-client-lib` (and
   therefore a separate artifact) which is a transitive dependency of the Zcash Android SDK.
     - The `z.cash.ecc.android.sdk.model.LightWalletEndpoint` class has been moved to `co.electriccoin.lightwallet.client.model.LightWalletEndpoint`
     - The new networking module now provides a `LightWalletClient` for asynchronous calls.
-    - Most unary calls respond with the new `Response` class and its subclasses. Streaming calls will be updated 
+    - Most unary calls respond with the new `Response` class and its subclasses. Streaming calls will be updated
       with the Response class later.
     - SDK clients should avoid using generated GRPC objects, as these are an internal implementation detail and are in process of being removed from the public API.  Any clients using GRPC objects will find these have been repackaged from `cash.z.wallet.sdk.rpc` to `cash.z.wallet.sdk.internal.rpc` to signal they are not a public API.
 
@@ -129,7 +181,7 @@ which speeds up discovering the wallet's spendable balance.
   - `Synchronizer.sendToAddress()` and `Synchronizer.shieldFunds()` return flows that can now be collected multiple times.  Prior versions of the SDK had a bug that could submit transactions multiple times if the flow was collected more than once.
 - Updated dependencies:
   - Kotlin 1.7.21
-  - AndroidX  
+  - AndroidX
   - etc.
 - Updated checkpoints
 
@@ -154,14 +206,14 @@ which speeds up discovering the wallet's spendable balance.
   - `DerivationTool.deriveTransparentSecretKey` (use `DerivationTool.deriveUnifiedSpendingKey` instead).
   - `DerivationTool.deriveShieldedAddress`
   - `DerivationTool.deriveUnifiedViewingKeys` (use `DerivationTool.deriveUnifiedFullViewingKey` instead)
-  - `DerivationTool.validateUnifiedViewingKey` 
+  - `DerivationTool.validateUnifiedViewingKey`
 
 ## Version 1.9.0-beta05
 - The minimum version of Android supported is now API 21
 - Fixed R8/ProGuard consumer rule, which eliminates a runtime crash for minified apps
 
 ## Version 1.9.0-beta04
-- The SDK now stores sapling param files in `no_backup/co.electricoin.zcash` folder instead of the `cache/params` 
+- The SDK now stores sapling param files in `no_backup/co.electricoin.zcash` folder instead of the `cache/params`
   folder. Besides that, `SaplingParamTool` also does validation of downloaded sapling param file hash and size.
 **No action required from client app**.
 
@@ -177,7 +229,7 @@ which speeds up discovering the wallet's spendable balance.
  - Updated checkpoints
 
 ## Version 1.8.0-beta01
-- Enabled automated unit tests run on the CI server 
+- Enabled automated unit tests run on the CI server
 - Added `BlockHeight` typesafe object to represent block heights
 - Significantly reduced memory usage, fixing potential OutOfMemoryError during block download
 - Kotlin 1.7.10

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2017-2021 Electric Coin Company
+Copyright (c) 2017-2023 Electric Coin Company
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -53,3 +53,9 @@ Note that we aim for the main branch of this repository to be stable and releasa
 1. Intel-based machines may have trouble building in Android Studio.  The workaround is to add the following line to `~/.gradle/gradle.properties`: `ZCASH_IS_DEPENDENCY_LOCKING_ENABLED=false`
 1. During builds, a warning will be printed that says "Unable to detect AGP versions for included builds. All projects in the build should use the same AGP version."  This can be safely ignored.  The version under build-conventions is the same as the version used elsewhere in the application.
 1. Android Studio will warn about the Gradle checksum.  This is a [known issue](https://github.com/gradle/gradle/issues/9361) and can be safely ignored.
+
+## Unstable Features
+### Spend-before-Sync compact blocks synchronization algorithm
+`CompactBlockProcessor` now processes compact blocks from the lightwalletd server in non-linear order with the 
+**Spend-before-Sync** algorithm. This feature speeds up discovering the wallet's spendable balance. Please note that 
+this new block synchronization algorithm is still under development.