[$250] [Dupe detection] Use new ResolveDuplicates when approver is resolving duplicates #48416

pecanoro · 2024-09-02T15:10:59Z

Part of the Duplicate Detection project

Main issue: https://github.com/Expensify/Expensify/issues/411008
Project: Dupe detection

Feature Description

Right now, we always call MergeTransactions when resolving duplicates and click Confirm at the last page of resolving duplicates. However, we need to call a new command called ResolveDuplicates when an approver or admin (basically anyone but the submitter) is the person resolving the duplicates. The behaviour (so we can create the proper optimistic actions) should be as follows:

ResolveDuplicates API endpoint needs the following parameters:

            'authToken' => $authToken,
            'transactionIDToKeep' => $transactionIDToKeep,
            'transactionIDList' => $transactionIDList,
            'created' => $created,
            'merchant' => $merchant,
            'amount' => $amount,
            'currency' => $currency,
            'category' => $category,
            'comment' => $comment,
            'billable' => $billable,
            'reimbursable' => $reimbursable,
            'tag' => $tag,
            'optimisticReportActionID' => $optimisticReportActionID,
            'reportActionIDList' => $reportActionIDList,

This command will update the transaction to match the fields we chose during the flow. It should already be doing this.

It won't delete the rest of the transactions in transactionIDList but it will HOLD them by adding the hold NVP to it and will add the corresponding HOLD actions to each of them.
It will add a DISMISSEDVIOLATION action to the transactionIDToKeep
It will dismiss the violation by adding the corresponding transaction NVP for all transactions.

Main issue: https://github.com/Expensify/Expensify/issues/411008

Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~011781060ec2a6c428
Upwork Job ID: 1830626997874011636
Last Price Increase: 2024-09-02
Automatic offers:

nkdengineer | Contributor | 103792699

Issue Owner

Current Issue Owner: @

The text was updated successfully, but these errors were encountered:

pecanoro · 2024-09-02T15:20:20Z

@parasharrajat I am assigning you as the C+ in this issue since you reviewed the main ones.

parasharrajat · 2024-09-02T15:20:31Z

Sure

melvin-bot · 2024-09-02T15:20:44Z

Triggered auto assignment to @sonialiap (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.

melvin-bot · 2024-09-02T15:21:04Z

Job added to Upwork: https://www.upwork.com/jobs/~011781060ec2a6c428

melvin-bot · 2024-09-02T15:21:08Z

Current assignee @parasharrajat is eligible for the External assigner, not assigning anyone new.

nkdengineer · 2024-09-02T15:41:40Z

Proposal

Please re-state the problem that we are trying to solve in this issue.

Add a new command for admin/approver when resolving the duplicate transaction from submitter

What is the root cause of that problem?

We're always calling mergeDuplicates function without checking the current user is admin/approver or not

App/src/pages/TransactionDuplicate/Confirmation.tsx

Lines 45 to 48 in e60b821

    
           const transactionsMergeParams = useMemo(() => TransactionUtils.buildTransactionsMergeParams(reviewDuplicates, transaction), [reviewDuplicates, transaction]); 
        
           const mergeDuplicates = useCallback(() => { 
        
               IOU.mergeDuplicates(transactionsMergeParams); 
        
               Navigation.navigate(ROUTES.REPORT_WITH_ID.getRoute(reportAction?.childReportID ?? '-1'));

What changes do you think we should make in order to solve the problem?

To check the user is admin/approver we can use the same way we do here.

App/src/libs/ReportUtils.ts

Lines 3074 to 3079 in e60b821

    
           const isActionOwner = 
        
               typeof parentReportAction?.actorAccountID === 'number' && 
        
               typeof currentUserPersonalDetails?.accountID === 'number' && 
        
               parentReportAction.actorAccountID === currentUserPersonalDetails?.accountID; 
        
           const isApprover = isMoneyRequestReport(moneyRequestReport) && moneyRequestReport?.managerID !== null && currentUserPersonalDetails?.accountID === moneyRequestReport?.managerID; 
        
           const isAdmin = isPolicyAdmin(moneyRequestReport.policyID ?? '-1', allPolicies);

Create a new function resolveDuplicates that will call ResolveDuplicates comman with the param as described here. It will be almost the same with mergeDuplicates, the only difference with mergeDuplicates is

It won't delete the rest of the transactions in transactionIDList but it will HOLD them by adding the hold NVP to it and will add the corresponding HOLD actions to each of them.

We will not delete the transaction in transactionIDList. Instead, we will add a hold action for these transaction thread reports in optimistic data and reset in failure data and also add hold violation for these transaction

It will add a DISMISSEDVIOLATION action to the transactionIDToKeep

Add a DISMISSEDVIOLATION action with reason in originalMessage is duplicatedTransaction for the transaction thread report of transactionIDToKeep list

It will dismiss the violation by adding the corresponding transaction NVP for all transactions.

Remove violation of all transactions in transactionIDToKeep

What alternative solutions did you explore? (Optional)

parasharrajat · 2024-09-02T18:16:05Z

The proposal sounds good @nkdengineer sounds good to me.

🎀 👀 🎀 C+ reviewed

melvin-bot · 2024-09-02T18:16:16Z

Current assignee @pecanoro is eligible for the choreEngineerContributorManagement assigner, not assigning anyone new.

pecanoro · 2024-09-03T08:59:59Z

Assigning @nkdengineer!

melvin-bot · 2024-09-03T09:00:30Z

📣 @nkdengineer 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job
Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻
Keep in mind: Code of Conduct | Contributing 📖

nkdengineer · 2024-09-03T14:12:31Z

Will raise the PR tomorrow.

nkdengineer · 2024-09-04T05:11:32Z

@pecanoro In the ResolveDuplicates API, what is the structure of reportActionIDList params?

pecanoro · 2024-09-04T11:21:25Z

In the ResolveDuplicates API, what is the structure of reportActionIDList params?

A comma-separated list of report actions

nkdengineer · 2024-09-05T08:17:27Z

A comma-separated list of report actions

@pecanoro Can you give an example? Do we need to add reportID in data?

pecanoro · 2024-09-05T12:06:45Z

@pecanoro Can you give an example?

Imagine we have 3 duplicates with transactionIDs 111, 222 and 333. We want to keep 111 and hold the other two. Then:

'transactionIDToKeep' => '111',
'transactionIDList' => '222,333',

Do we need to add reportID in data?

Nope, no reportID since they are not changing reports

nkdengineer · 2024-09-05T12:10:53Z

@pecanoro I mean I need an example for reportActionIDList param.

nkdengineer · 2024-09-09T04:37:20Z

@pecanoro When I call ResolveDuplicates, BE returns the error like this video. Here is the parameter that I called this API. Am I missing something?

{
    "amount": -200,
    "currency": "VND",
    "created": "2024-09-09",
    "transactionIDList": [
        "4002616464040896678"
    ],
    "billable": false,
    "reimbursable": true,
    "category": "",
    "tag": "",
    "merchant": "ssd",
    "comment": "",
    "transactionIDToKeep": "5332969134080823220",
    "reportActionIDList": [
        "637476526581947356"
    ],
    "optimisticReportActionID": "3766068727586466325"
}

Screen.Recording.2024-09-09.at.11.35.19.mov

pecanoro · 2024-09-09T11:05:25Z

Oh, it's a BE problem, I j just checked the logs. Let me create a PR to fix it

pecanoro · 2024-09-09T12:58:00Z

The PR is merged, we are waiting for a deploy and then you will be able to use the endpoint in staging again. I renamed transactionIDToKeep for just transactionID

dylanexpensify · 2024-09-10T20:50:43Z

@nkdengineer PR is deployed to prod now!

nkdengineer · 2024-09-11T15:21:13Z

@parasharrajat The PR is here.

parasharrajat · 2024-09-11T20:18:21Z

it will HOLD them by adding the hold NVP to it

What do you mean by NVP @pecanoro?

nkdengineer · 2024-09-12T03:20:43Z

@pecanoro While testing I found another BE bug, the dismiss violation action isn't stored from BE side.

pecanoro · 2024-09-12T14:45:33Z

What do you mean by NVP @pecanoro?

Ah sorry, when the transaction is put on HOLD, we add a value to the transaction namevaluepairs in the comment.

pecanoro · 2024-09-12T14:46:06Z

@pecanoro While testing I found another BE bug, the dismiss violation action isn't stored from BE side.

Are you sure? It is added for the transaction we want to keep, not for the others.

nkdengineer · 2024-09-12T14:59:39Z

Yes, I tested and after reset cache this action is removed since BE doesn't return this action. Also the resolve duplicate API doesn't return dismiss violation action data.

pecanoro · 2024-09-12T15:50:56Z

Let me double-check manually, but it's pretty odd since I even wrote automated tests for this and they are passing

pecanoro · 2024-09-12T15:51:21Z

@nkdengineer Can you paste here what the BE is returning for you?

nkdengineer · 2024-09-12T17:25:05Z

{
    "accountID": 18311825,
    "accounts": {
        "18311825": {
            "lastUpdateID": 1775760902,
            "previousUpdateID": 1775614249,
            "updateIDs": [
                1775760897,
                1775760898,
                1775760899,
                1775760900,
                1775760901,
                1775760902
            ]
        }
    },
    "authToken": "ED538D6882FC6B872FD1E38C8B75EC3ED4799DAF2809D40258D5A38DF3F448972D57954AF21C6EF04D9DBA533734CD6FC5B8908331F9904CF87AF73E8B286E9DDC523F8C2153DB499A0D6E433B2BD2E21CB71155B10A7F6D64826D4ACD1CA0443ECF354E9006107CD6DAC056448043E0FBDE3C32367A157108E87CBB53EDBFFD03E891BCEB5DA77E8EBEF24B5AF0D7E9FD9A757BDA391A6568E463678C8EF1BF1E55E96771BEEE6228EF37F1B8FF3BE280B9E5C6B2235E5FAD8E30F652EAA244C06EFE2E8B18BFB3CB8DA94286CC32C74D0B5F03C003CD78D2BB8A9D02A9478C89F64E9674B229779DDC5DEA5EED7B3DF5A513373D224C022CE70012AEAEA03F4BF4CF18ED5EBE8DF44F7A53F64C623731EF5689A91E9FB3E061CDF663B373ED10CECB4758B6D759E25715BB2EF0BE6C729E7A3928CF1F89005D8434D3BD4556A86E95D218CB4353E3B6E6A20E66F7775725A89AEB459453192F36B84EA2742A",
    "email": "[email protected]",
    "fetchableOnyxUpdates": [
        "1775760897",
        "1775760898",
        "1775760899",
        "1775760900",
        "1775760901",
        "1775760902"
    ],
    "onyxUpdates": {
        "1775760897": [
            {
                "key": "transactions_398480206897163307",
                "onyxMethod": "merge",
                "value": {
                    "comment": {
                        "dismissedViolations": {
                            "duplicatedTransaction": {
                                "[email protected]": 1726161878570510
                            }
                        }
                    }
                }
            }
        ],
        "1775760898": [
            {
                "key": "transactions_1367302367436179941",
                "onyxMethod": "merge",
                "value": {
                    "comment": {
                        "comment": "",
                        "dismissedViolations": {
                            "duplicatedTransaction": {
                                "[email protected]": 1726161878570610
                            }
                        }
                    }
                }
            }
        ],
        "1775760899": [
            {
                "key": "report_1113086820473870",
                "onyxMethod": "merge",
                "value": {
                    "participants": {
                        "18301280": {
                            "hidden": true,
                            "notificationPreference": "hidden"
                        },
                        "18308128": {
                            "hidden": true,
                            "notificationPreference": "hidden"
                        },
                        "18311825": {
                            "hidden": true,
                            "notificationPreference": "hidden"
                        }
                    }
                }
            }
        ],
        "1775760900": [
            {
                "key": "report_1113086820473870",
                "onyxMethod": "merge",
                "value": {
                    "chatType": "",
                    "description": "",
                    "lastActorAccountID": 18311825,
                    "lastMessageText": "held this expense with the comment: ",
                    "lastVisibleActionCreated": "2024-09-12 17:24:38.571",
                    "managerID": null,
                    "ownerAccountID": 0,
                    "parentReportActionID": "2951370456346266262",
                    "parentReportID": "8755324512976856",
                    "policyID": "2D62A4DBCE91AD7E",
                    "reportID": "1113086820473870",
                    "reportName": "Chat Report",
                    "state": "OPEN",
                    "stateNum": 0,
                    "statusNum": 0,
                    "type": "chat",
                    "visibility": null
                }
            },
            {
                "key": "reportActions_1113086820473870",
                "onyxMethod": "merge",
                "value": {
                    "2272851721205218606": {
                        "actionName": "HOLD",
                        "actorAccountID": 18311825,
                        "avatar": "https://d2k5nsl2zxldvw.cloudfront.net/images/avatars/default-avatar_18.png",
                        "created": "2024-09-12 17:24:38.571",
                        "lastModified": "2024-09-12 17:24:38.571",
                        "message": [
                            {
                                "html": "",
                                "text": "",
                                "type": "COMMENT",
                                "whisperedTo": []
                            }
                        ],
                        "originalMessage": {
                            "accountID": 18311825,
                            "date": "2024-09-12 17:24:38",
                            "isDuplicate": "true",
                            "lastModified": "2024-09-12 17:24:38.571",
                            "message": ""
                        },
                        "person": [
                            {
                                "style": "strong",
                                "text": "nkd102",
                                "type": "TEXT"
                            }
                        ],
                        "reportActionID": "2272851721205218606",
                        "shouldShow": true
                    }
                }
            }
        ],
        "1775760901": [
            {
                "key": "personalDetailsList",
                "onyxMethod": "merge",
                "value": {
                    "18311825": {
                        "accountID": 18311825,
                        "avatar": "https://d2k5nsl2zxldvw.cloudfront.net/images/avatars/default-avatar_18.png",
                        "displayName": "nkd102",
                        "firstName": "nkd102",
                        "lastName": "",
                        "login": "[email protected]",
                        "phoneNumber": "",
                        "pronouns": "",
                        "status": null,
                        "timezone": {
                            "automatic": true,
                            "selected": "Asia/Ho_Chi_Minh"
                        },
                        "validated": true
                    }
                }
            }
        ],
        "1775760902": [
            {
                "key": "transactions_1367302367436179941",
                "onyxMethod": "merge",
                "value": {
                    "comment": {
                        "hold": "2272851721205218606"
                    }
                }
            }
        ]
    },
    "httpCode": 200,
    "jsonCode": 200,
    "authResponseMessage": "200 OK",
    "requestID": "8c21979b5e5e1fbc-HKG",
    "onyxData": [
        {
            "key": "transactions_398480206897163307",
            "onyxMethod": "merge",
            "value": {
                "comment": {
                    "dismissedViolations": {
                        "duplicatedTransaction": {
                            "[email protected]": 1726161878570510
                        }
                    }
                }
            }
        },
        {
            "key": "transactions_1367302367436179941",
            "onyxMethod": "merge",
            "value": {
                "comment": {
                    "comment": "",
                    "dismissedViolations": {
                        "duplicatedTransaction": {
                            "[email protected]": 1726161878570610
                        }
                    }
                }
            }
        },
        {
            "key": "report_1113086820473870",
            "onyxMethod": "merge",
            "value": {
                "participants": {
                    "18301280": {
                        "hidden": true,
                        "notificationPreference": "hidden"
                    },
                    "18308128": {
                        "hidden": true,
                        "notificationPreference": "hidden"
                    },
                    "18311825": {
                        "hidden": true,
                        "notificationPreference": "hidden"
                    }
                }
            }
        },
        {
            "key": "report_1113086820473870",
            "onyxMethod": "merge",
            "value": {
                "chatType": "",
                "description": "",
                "lastActorAccountID": 18311825,
                "lastMessageText": "held this expense with the comment: ",
                "lastVisibleActionCreated": "2024-09-12 17:24:38.571",
                "managerID": null,
                "ownerAccountID": 0,
                "parentReportActionID": "2951370456346266262",
                "parentReportID": "8755324512976856",
                "policyID": "2D62A4DBCE91AD7E",
                "reportID": "1113086820473870",
                "reportName": "Chat Report",
                "state": "OPEN",
                "stateNum": 0,
                "statusNum": 0,
                "type": "chat",
                "visibility": null
            }
        },
        {
            "key": "reportActions_1113086820473870",
            "onyxMethod": "merge",
            "value": {
                "2272851721205218606": {
                    "actionName": "HOLD",
                    "actorAccountID": 18311825,
                    "avatar": "https://d2k5nsl2zxldvw.cloudfront.net/images/avatars/default-avatar_18.png",
                    "created": "2024-09-12 17:24:38.571",
                    "lastModified": "2024-09-12 17:24:38.571",
                    "message": [
                        {
                            "html": "",
                            "text": "",
                            "type": "COMMENT",
                            "whisperedTo": []
                        }
                    ],
                    "originalMessage": {
                        "accountID": 18311825,
                        "date": "2024-09-12 17:24:38",
                        "isDuplicate": "true",
                        "lastModified": "2024-09-12 17:24:38.571",
                        "message": ""
                    },
                    "person": [
                        {
                            "style": "strong",
                            "text": "nkd102",
                            "type": "TEXT"
                        }
                    ],
                    "reportActionID": "2272851721205218606",
                    "shouldShow": true
                }
            }
        },
        {
            "key": "personalDetailsList",
            "onyxMethod": "merge",
            "value": {
                "18311825": {
                    "accountID": 18311825,
                    "avatar": "https://d2k5nsl2zxldvw.cloudfront.net/images/avatars/default-avatar_18.png",
                    "displayName": "nkd102",
                    "firstName": "nkd102",
                    "lastName": "",
                    "login": "[email protected]",
                    "phoneNumber": "",
                    "pronouns": "",
                    "status": null,
                    "timezone": {
                        "automatic": true,
                        "selected": "Asia/Ho_Chi_Minh"
                    },
                    "validated": true
                }
            }
        },
        {
            "key": "transactions_1367302367436179941",
            "onyxMethod": "merge",
            "value": {
                "comment": {
                    "hold": "2272851721205218606"
                }
            }
        }
    ],
    "previousUpdateID": 1775614249,
    "lastUpdateID": 1775760902
}

nkdengineer · 2024-09-12T17:26:02Z

@pecanoro The response here.

nkdengineer · 2024-09-12T17:26:21Z

The payload here.

pecanoro · 2024-09-12T17:50:49Z

There is something off, also because both hold actions are returned for the same report

pecanoro · 2024-09-12T17:54:32Z

Ah nvm, forget my previous comment, you were resolving duplicates with only two transactions. I am going to check the database

pecanoro · 2024-09-12T18:12:58Z

Ah I found the problem, another parameter name that got cleared because of sanitization. I am going to rename it to something better either way. I will create a PR in the back-end

pecanoro · 2024-09-16T15:59:46Z

@nkdengineer BE fix is on staging, you can test it in the PR now! New parameter is called dismissedViolationReportActionID instead of optimisticReportActionID

parasharrajat · 2024-09-19T16:59:19Z

@pecanoro Does this change affect keep All functionality as well? Do we need to change that too?

pecanoro · 2024-09-19T17:18:55Z

@parasharrajat No, it should not change it

parasharrajat · 2024-09-20T09:32:29Z

Got it. Thanks.

parasharrajat · 2024-09-21T14:23:04Z

Note for future: I noticed that when admin resolves a duplicate, the transaction is no more duplicate for admin but same transaction still shown duplicate for the original user who requested. I confirmed that this is expected #48522 (comment)

nkdengineer · 2024-10-02T08:01:09Z

@parasharrajat The original PR was reverted so I created a new one here to fix another bug #49900

pecanoro changed the title ~~Use new ResolveDuplicates~~ [Dupe detection] Use new ResolveDuplicates when approver is resolving duplicates Sep 2, 2024

pecanoro assigned pecanoro and parasharrajat Sep 2, 2024

pecanoro added the Bug Something is broken. Auto assigns a BugZero manager. label Sep 2, 2024

melvin-bot bot assigned sonialiap Sep 2, 2024

melvin-bot bot added the Daily KSv2 label Sep 2, 2024

pecanoro added External Added to denote the issue can be worked on by a contributor and removed Daily KSv2 labels Sep 2, 2024

melvin-bot bot changed the title ~~[Dupe detection] Use new ResolveDuplicates when approver is resolving duplicates~~ [$250] [Dupe detection] Use new ResolveDuplicates when approver is resolving duplicates Sep 2, 2024

melvin-bot bot added the Help Wanted Apply this label when an issue is open to proposals by contributors label Sep 2, 2024

melvin-bot bot added the Daily KSv2 label Sep 2, 2024

pecanoro assigned nkdengineer Sep 3, 2024

melvin-bot bot removed the Help Wanted Apply this label when an issue is open to proposals by contributors label Sep 3, 2024

nkdengineer mentioned this issue Sep 4, 2024

fix: Use new ResolveDuplicates when approver is resolving duplicates #48522

Merged

50 tasks

melvin-bot bot added Reviewing Has a PR in review Weekly KSv2 and removed Daily KSv2 labels Sep 11, 2024

nkdengineer mentioned this issue Oct 2, 2024

implement resolveDuplicates API and fix regression #50035

Merged

50 tasks

melvin-bot bot added Weekly KSv2 and removed Weekly KSv2 labels Oct 2, 2024

[$250] [Dupe detection] Use new ResolveDuplicates when approver is resolving duplicates #48416

[$250] [Dupe detection] Use new ResolveDuplicates when approver is resolving duplicates #48416

Comments

pecanoro commented Sep 2, 2024 • edited by melvin-bot bot Loading

Part of the Duplicate Detection project

Feature Description

pecanoro commented Sep 2, 2024

parasharrajat commented Sep 2, 2024

melvin-bot bot commented Sep 2, 2024

melvin-bot bot commented Sep 2, 2024

melvin-bot bot commented Sep 2, 2024

nkdengineer commented Sep 2, 2024

Proposal

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

What alternative solutions did you explore? (Optional)

parasharrajat commented Sep 2, 2024

melvin-bot bot commented Sep 2, 2024

pecanoro commented Sep 3, 2024

melvin-bot bot commented Sep 3, 2024

nkdengineer commented Sep 3, 2024

nkdengineer commented Sep 4, 2024

pecanoro commented Sep 4, 2024

nkdengineer commented Sep 5, 2024

pecanoro commented Sep 5, 2024

nkdengineer commented Sep 5, 2024

nkdengineer commented Sep 9, 2024

pecanoro commented Sep 9, 2024

pecanoro commented Sep 9, 2024

dylanexpensify commented Sep 10, 2024

nkdengineer commented Sep 11, 2024

parasharrajat commented Sep 11, 2024

nkdengineer commented Sep 12, 2024

pecanoro commented Sep 12, 2024

pecanoro commented Sep 12, 2024

nkdengineer commented Sep 12, 2024

pecanoro commented Sep 12, 2024

pecanoro commented Sep 12, 2024

nkdengineer commented Sep 12, 2024 • edited Loading

nkdengineer commented Sep 12, 2024

nkdengineer commented Sep 12, 2024

pecanoro commented Sep 12, 2024

pecanoro commented Sep 12, 2024 • edited Loading

pecanoro commented Sep 12, 2024

pecanoro commented Sep 16, 2024

parasharrajat commented Sep 19, 2024 • edited Loading

pecanoro commented Sep 19, 2024

parasharrajat commented Sep 20, 2024

parasharrajat commented Sep 21, 2024

nkdengineer commented Oct 2, 2024

pecanoro commented Sep 2, 2024 •

edited by melvin-bot bot

Loading

nkdengineer commented Sep 12, 2024 •

edited

Loading

pecanoro commented Sep 12, 2024 •

edited

Loading

parasharrajat commented Sep 19, 2024 •

edited

Loading