You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As today in case of crash lucene indexes may get inconsistent with the main storage, this is because there is no cross state tracking between the main storage and the lucene indexes, so is not possible to say which transaction applied to the main storage is applied to the lucene index.
There are multiple way to solve this problem, but all of them involve sharing some state between the main storage log and the lucene indexes, this was quite tricky(almost impossible) in all the implementations 3.2.x and before, but in the last development version some refactor as been done to make this implementation possible, in the specific, in the old implementation for each key the index API was called for apply the change, now there is also an API that pass all the changes present in a transaction for a specific index and the current low level transaction data structure.
This new method allow a better integration from low level OrientDB transaction and potential index changes transactions, so opening to a better durability integration.
for example one way to integrate them would be start a lucene transaction, apply all the changes, and than log the lucene transaction id in the OrientDB main log, this is enough to detect on crash recover if the lucene index is lagging behind or no. To make sure also that the lucene index can be recovered in case of crash is also need to log on the main storage the index key changes and re-execute them on recover if the transaction is missing.
There are still some missing APIs in the core to allow all the cases to be covered but should not be too hard to add them while the work is done on it.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
As today in case of crash lucene indexes may get inconsistent with the main storage, this is because there is no cross state tracking between the main storage and the lucene indexes, so is not possible to say which transaction applied to the main storage is applied to the lucene index.
There are multiple way to solve this problem, but all of them involve sharing some state between the main storage log and the lucene indexes, this was quite tricky(almost impossible) in all the implementations 3.2.x and before, but in the last development version some refactor as been done to make this implementation possible, in the specific, in the old implementation for each key the index API was called for apply the change, now there is also an API that pass all the changes present in a transaction for a specific index and the current low level transaction data structure.
This new method allow a better integration from low level OrientDB transaction and potential index changes transactions, so opening to a better durability integration.
for example one way to integrate them would be start a lucene transaction, apply all the changes, and than log the lucene transaction id in the OrientDB main log, this is enough to detect on crash recover if the lucene index is lagging behind or no. To make sure also that the lucene index can be recovered in case of crash is also need to log on the main storage the index key changes and re-execute them on recover if the transaction is missing.
There are still some missing APIs in the core to allow all the cases to be covered but should not be too hard to add them while the work is done on it.
Beta Was this translation helpful? Give feedback.
All reactions