Program Stalls After Activating Comparison Section

[emanuel-skai]

Hi,

I've been using a nada program to measure the Euclidean distance between an encoded query vector and 10 encoded documents as shown below:

from nada_dsl import *


def nada_main():
    vector_size = 384
    total_docs = 10

    # Create parties
    party1 = Party(name="Party1")

    # Build query vector from inputs
    query_vector = []
    for i in range(vector_size):
        query_vector.append(SecretInteger(Input(name="qv_" + str(i), party=party1)))

    # Build doc vectors from inputs
    docs_vector = []
    for doc_index in range(total_docs):
        doc_vector = []
        for i in range(vector_size):
            doc_vector.append(SecretInteger(Input(name="doc"+str(doc_index)+"_"+str(i), party=party1)))
        docs_vector.append(doc_vector)

    # compute euclidean distance between docs and query vector
    euclidean_distance_array_outputs = []
    euclidean_distance_array = []
    
    for doc_index in range(total_docs):
        # Compute differences
        diff_docindex = []
        for i in range(vector_size):
            diff_docindex_i = query_vector[i] - docs_vector[doc_index][i]
            diff_docindex.append(diff_docindex_i)

        # Compute distance between doc_i and query_vector
        distance_doci = diff_docindex[0] * diff_docindex[0]
        for i in range(1,vector_size):
            distance_doci += diff_docindex[i] * diff_docindex[i]
        

        euclidean_distance_array_outputs.append(Output(distance_doci, "distance"+str(doc_index), party1))
        euclidean_distance_array.append(distance_doci)
    
    # find the closest document, initialize the closest distance and index
    """
    closest_distance = euclidean_distance_array[0]
    closest_index = Integer(0)
    # iterate through the euclidean distance squared array to find the closest document
    for i in range(1, total_docs):
        #cond = euclidean_distance_array[i] < closest_distance
        cond = 
        closest_distance = cond.if_else(euclidean_distance_array[i], closest_distance)
        closest_index = cond.if_else(Integer(i), closest_index)
    euclidean_distance_array_outputs.append(Output(closest_index, "closest_document_index", party1))
    """
    # return the closest document index
    return euclidean_distance_array_outputs
    

The program works fine as long as the commented section is not executed. However, when I uncomment that section to calculate the closest distance index, the program seems to get stuck. This leads me to believe there might be an issue with the comparison operator.

I'm adding the test file I'm using for this program
test-euclidean-distance.zip

[Davetbutler]

Hey @emanuel-skai.

I think I reproduced the issue here.

TL;DR: reduce the number of docs (3-5 docs work fine, 7 docs work but quite slowly).

A few more details:

• This is likely due to the way we handle reuse of variables in the intermediate representation (once the program is compiled).
• Essentially the reuse is currently not very efficient (exaggerated with conditionals), meaning the intermediate representation becomes massive very quickly with every extra doc that is added.

We are actively working on improving this right now. We will be sure to let you know when the refactor is complete. And I hope that this at least unblocks you for now, and let us know if we can do anything to help.

[wwwehr]

@emanuel-skai following up on this - we believe this has been fixed in the network. Please try again with the 0.2.1 SDK (nilup) release.