Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0 #53

Merged
merged 75 commits into from
Dec 3, 2024
Merged

1.0 #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
764ded9
Merge pull request #34 from OpenG2P/develop
venky-ganapathy Oct 1, 2024
87f8445
Update dependency version
PSNAppz Oct 1, 2024
9a47ad7
Merge pull request #35 from PSNAppz/1.0
venky-ganapathy Oct 1, 2024
51ad6f4
Fix disbursement cancellation validations
PSNAppz Oct 3, 2024
dd11e67
Merge pull request #36 from PSNAppz/1.0
venky-ganapathy Oct 3, 2024
5a33591
Fix disbursement validation not checking for empty string in disburse…
PSNAppz Oct 3, 2024
b57c5ab
Add mapper_status and latest_error_code to DisbursementBatchControl m…
PSNAppz Oct 4, 2024
3c6064d
Mapper resolution task to update the status in corresponding disburse…
PSNAppz Oct 4, 2024
cfb0c0c
Merge pull request #37 from PSNAppz/1.0
venky-ganapathy Oct 4, 2024
1171a42
Update version in dockerfile-git
PSNAppz Oct 4, 2024
a5e4d2b
Merge pull request #38 from PSNAppz/1.0
venky-ganapathy Oct 4, 2024
d06fdaf
Bulk update the DisbursementBatchControl data
PSNAppz Oct 4, 2024
c7169e7
Add session.flush()
PSNAppz Oct 4, 2024
57866c8
Update debug message
PSNAppz Oct 4, 2024
5c4f37d
Add debug messages
PSNAppz Oct 4, 2024
7e88e3a
Fix issue in disburse_funds_from_bank
PSNAppz Oct 4, 2024
874e0dc
Disburse only processed in disburse_funds_from_bank
PSNAppz Oct 5, 2024
73ed2ab
Merge pull request #39 from PSNAppz/1.0
venky-ganapathy Oct 7, 2024
9bd00b0
Fix condition in process_and_store_resolution
PSNAppz Oct 8, 2024
c8158f1
Add logger info displaying total attempts remaining
PSNAppz Oct 8, 2024
ec2778a
Pre commit fixes
PSNAppz Oct 8, 2024
c076aa3
Merge pull request #40 from PSNAppz/1.0
venky-ganapathy Oct 14, 2024
1e69ec6
Return None if disbursement id is invalid or not found
PSNAppz Oct 16, 2024
c536d6e
Return None for get_bank_batch_id if disbursement id is invalid
PSNAppz Oct 16, 2024
909408d
Merge pull request #41 from PSNAppz/1.0
venky-ganapathy Oct 16, 2024
e9c10b2
Refactor code for more readability
PSNAppz Oct 16, 2024
e9b9733
Fix error response for envelope status API
PSNAppz Oct 17, 2024
bb6e118
Add unit tests for envelope status API
PSNAppz Oct 17, 2024
3f2990a
Add unit tests for disbursement status API
PSNAppz Oct 17, 2024
ed8faf0
Fix response class in disbursement status error construct method
PSNAppz Oct 17, 2024
f99a2b5
Amend API
PSNAppz Oct 20, 2024
79a29b4
Override disbursement schedule date check through process_future_disb…
PSNAppz Oct 20, 2024
8b3e823
Pre-commit fix
PSNAppz Oct 20, 2024
f4bc273
Pre-commit fix
PSNAppz Oct 21, 2024
a6be372
Merge pull request #42 from PSNAppz/1.0
venky-ganapathy Oct 21, 2024
dc71115
Attempt to acquire the lock before updating envelope_batch_status
PSNAppz Oct 22, 2024
2bd28dc
Merge pull request #43 from PSNAppz/1.0
venky-ganapathy Oct 22, 2024
ba7274e
Acquire lock in the same session
PSNAppz Oct 22, 2024
8e0bda2
pre-commit fixes
PSNAppz Oct 22, 2024
65e25d7
Add debugging logs
PSNAppz Oct 22, 2024
7a94b73
Add sleep for 600 secs
PSNAppz Oct 22, 2024
4ab44e8
Ignore ession.execute(text("SET lock_timeout = '5s'"))
PSNAppz Oct 22, 2024
3d79840
Add .with_for_update(nowait=True)
PSNAppz Oct 22, 2024
6d98be4
Catch general exception
PSNAppz Oct 22, 2024
3788e00
Remove wait time
PSNAppz Oct 22, 2024
528accb
Add a delay before commiting
PSNAppz Oct 22, 2024
789036a
Add .populate_existing()
PSNAppz Oct 22, 2024
c8ff3ac
Add .populate_existing() for mt940_processor
PSNAppz Oct 22, 2024
136c595
remove with_for_update(nowait=True)
PSNAppz Oct 22, 2024
ffd0452
Fix db locking issue for update_envelope_batch_status_reconciled
PSNAppz Oct 23, 2024
80707dd
Refactoring code
PSNAppz Oct 23, 2024
9a63671
Refactoring code
PSNAppz Oct 23, 2024
f854128
Merge pull request #44 from PSNAppz/1.0
venky-ganapathy Oct 23, 2024
6d9696f
Add test for amend_disbursement_envelope
tahzeer Oct 25, 2024
df7345a
Fix exception handeling of error code
tahzeer Nov 11, 2024
117906a
Add tests for brigde celery worker tasks
tahzeer Nov 11, 2024
9843511
Update test_mapper_resolution_task.py
tahzeer Nov 11, 2024
70573a9
Merge pull request #45 from tahzeer/1.0
venky-ganapathy Dec 2, 2024
584fc5e
Fix test cases
PSNAppz Dec 2, 2024
bf5369f
Update pypi versions in dockerfile
PSNAppz Dec 2, 2024
e1bbdda
Merge pull request #46 from PSNAppz/1.0
venky-ganapathy Dec 2, 2024
542f143
Bump fastapi-common and fastapi-auth versions
PSNAppz Dec 3, 2024
468d93f
Fix fastapi-common and fastapi-auth versions
PSNAppz Dec 3, 2024
46ce871
Merge pull request #47 from PSNAppz/1.0
venky-ganapathy Dec 3, 2024
eb54427
Fix g2pconnect-common version
PSNAppz Dec 3, 2024
349801f
Merge pull request #48 from PSNAppz/1.0
venky-ganapathy Dec 3, 2024
b563cee
Bump bridge version to 1.0.1
PSNAppz Dec 3, 2024
1c79d4a
Bump bridge version to 1.0.1
PSNAppz Dec 3, 2024
af83c16
Merge pull request #49 from PSNAppz/1.0
venky-ganapathy Dec 3, 2024
2315853
Fix dependency openg2p-g2p-bridge-celery-workers==1.0.1
PSNAppz Dec 3, 2024
b8345b4
Merge pull request #50 from PSNAppz/1.0
venky-ganapathy Dec 3, 2024
1757b2a
Bump version to 1.0.2
PSNAppz Dec 3, 2024
89cb356
Merge pull request #51 from PSNAppz/1.0
venky-ganapathy Dec 3, 2024
da8fac1
Fix tag workflow docker
PSNAppz Dec 3, 2024
115b804
Merge pull request #52 from PSNAppz/1.0
venky-ganapathy Dec 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/api-tag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ jobs:
python-version: "3.10"
- name: Install app
run: |
python -m pip install git+https://github.com/openg2p/openg2p-fastapi-common@develop\#subdirectory=openg2p-fastapi-common
python -m pip install git+https://github.com/openg2p/openg2p-fastapi-common@develop\#subdirectory=openg2p-fastapi-auth
python -m pip install git+https://github.com/openg2p/openg2p-g2pconnect-common@develop\#subdirectory=openg2p-g2pconnect-common-lib
python -m pip install git+https://github.com/openg2p/openg2p-g2p-bridge@develop\#subdirectory=openg2p-g2p-bridge-models
python -m pip install openg2p-g2p-bridge-api==develop
python -m pip install git+https://github.com/openg2p/openg2p-fastapi-common@v1.1.1\#subdirectory=openg2p-fastapi-common
python -m pip install git+https://github.com/openg2p/openg2p-fastapi-common@v1.1.1\#subdirectory=openg2p-fastapi-auth
python -m pip install git+https://github.com/openg2p/openg2p-g2pconnect-common@1.0.0\#subdirectory=openg2p-g2pconnect-common-lib
python -m pip install git+https://github.com/openg2p/openg2p-g2p-bridge@v1.0.2\#subdirectory=openg2p-g2p-bridge-models
python -m pip install openg2p-g2p-bridge-api==1.0.2
- name: Generate openapi json
run: |
mkdir -p openg2p-g2p-bridge-api/api-docs/generated
Expand Down Expand Up @@ -78,7 +78,7 @@ jobs:

# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
VERSION=TAG_NAME
VERSION=$TAG_NAME
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
echo IMAGE_ID=$IMAGE_ID >> $GITHUB_ENV
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/beat-producers-tag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:

# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
VERSION=TAG_NAME
VERSION=$TAG_NAME
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
echo IMAGE_ID=$IMAGE_ID >> $GITHUB_ENV
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/workers-tag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:

# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
VERSION=TAG_NAME
VERSION=$TAG_NAME
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
echo IMAGE_ID=$IMAGE_ID >> $GITHUB_ENV
Expand Down
8 changes: 4 additions & 4 deletions openg2p-g2p-bridge-api/Dockerfile-git
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ ADD --chown=${container_user}:${container_user_group} main.py /app
RUN python3 -m venv venv \
&& . ./venv/bin/activate
RUN python3 -m pip install \
git+https://github.com/openg2p/openg2p-fastapi-common@develop\#subdirectory=openg2p-fastapi-common \
git+https://github.com/openg2p/openg2p-fastapi-common@develop\#subdirectory=openg2p-fastapi-auth \
git+https://github.com/openg2p/openg2p-g2pconnect-common@develop\#subdirectory=openg2p-g2pconnect-common-lib \
git+https://github.com/openg2p/openg2p-g2p-bridge@develop\#subdirectory=openg2p-g2p-bridge-models \
git+https://github.com/openg2p/openg2p-fastapi-common@v1.1.1\#subdirectory=openg2p-fastapi-common \
git+https://github.com/openg2p/openg2p-fastapi-common@v1.1.1\#subdirectory=openg2p-fastapi-auth \
git+https://github.com/openg2p/openg2p-g2pconnect-common@1.0.0\#subdirectory=openg2p-g2pconnect-common-lib \
git+https://github.com/openg2p/openg2p-g2p-bridge@1.0\#subdirectory=openg2p-g2p-bridge-models \
./src

USER ${container_user}
Expand Down
10 changes: 5 additions & 5 deletions openg2p-g2p-bridge-api/Dockerfile-pypi
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ ADD --chown=${container_user}:${container_user_group} main.py /app
RUN python3 -m venv venv \
&& . ./venv/bin/activate
RUN python3 -m pip install \
openg2p-fastapi-common==develop \
openg2p-fastapi-auth==develop \
openg2p-g2pconnect-common-lib==develop \
openg2p-g2p-bridge-models==develop \
openg2p-g2p-bridge-api==develop
openg2p-fastapi-common==1.1.1 \
openg2p-fastapi-auth==1.1.1 \
openg2p-g2pconnect-common-lib==1.0.0 \
openg2p-g2p-bridge-models==1.0.2 \
openg2p-g2p-bridge-api==1.0.2

USER ${container_user}

Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "1.0.0"
__version__ = "1.0.2"
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ def __init__(self, **kwargs):
responses={200: {"model": DisbursementEnvelopeResponse}},
methods=["POST"],
)
self.router.add_api_route(
"/amend_disbursement_envelope",
self.amend_disbursement_envelope,
responses={200: {"model": DisbursementEnvelopeResponse}},
methods=["POST"],
)

async def create_disbursement_envelope(
self, disbursement_envelope_request: DisbursementEnvelopeRequest
Expand Down Expand Up @@ -80,3 +86,26 @@ async def cancel_disbursement_envelope(
)
_logger.info("Disbursement envelope cancelled successfully")
return disbursement_envelope_response

async def amend_disbursement_envelope(
self, disbursement_envelope_request: DisbursementEnvelopeRequest
) -> DisbursementEnvelopeResponse:
_logger.info("Amending disbursement envelope")
try:
disbursement_envelope_payload: DisbursementEnvelopePayload = (
await self.disbursement_envelope_service.amend_disbursement_envelope(
disbursement_envelope_request
)
)
except DisbursementEnvelopeException as e:
_logger.error("Error amending disbursement envelope")
error_response: DisbursementEnvelopeResponse = await self.disbursement_envelope_service.construct_disbursement_envelope_error_response(
disbursement_envelope_request, e.code
)
return error_response

disbursement_envelope_response: DisbursementEnvelopeResponse = await self.disbursement_envelope_service.construct_disbursement_envelope_success_response(
disbursement_envelope_request, disbursement_envelope_payload
)
_logger.info("Disbursement envelope amended successfully")
return disbursement_envelope_response
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,10 @@ async def validate_disbursement_request(
disbursement_payload.response_error_codes.append(
G2PBridgeErrorCodes.INVALID_DISBURSEMENT_AMOUNT
)
if disbursement_payload.beneficiary_id is None:
if (
disbursement_payload.beneficiary_id is None
or disbursement_payload.beneficiary_id == ""
):
disbursement_payload.response_error_codes.append(
G2PBridgeErrorCodes.INVALID_BENEFICIARY_ID
)
Expand Down Expand Up @@ -561,17 +564,17 @@ async def validate_envelope_for_disbursement_cancellation(
)

no_of_disbursements_after_this_request = (
len(disbursement_payloads)
- disbursement_envelope_batch_status.number_of_disbursements_received
disbursement_envelope_batch_status.number_of_disbursements_received
- len(disbursement_payloads)
)
total_disbursement_amount_after_this_request = (
sum(
disbursement_envelope_batch_status.total_disbursement_amount_received
- sum(
[
disbursement.disbursement_amount
for disbursement in disbursements_in_db
]
)
- disbursement_envelope_batch_status.total_disbursement_amount_received
)

if no_of_disbursements_after_this_request < 0:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -303,3 +303,153 @@ async def construct_disbursement_envelope_batch_status(
)
_logger.info("Disbursement envelope batch status constructed successfully")
return disbursement_envelope_batch_status

async def validate_envelope_amend_request(
self, disbursement_envelope_request: DisbursementEnvelopeRequest
) -> bool:
_logger.info("Validating disbursement envelope amend request")
disbursement_envelope_payload: DisbursementEnvelopePayload = (
disbursement_envelope_request.message
)
if (
disbursement_envelope_payload.disbursement_envelope_id is None
or disbursement_envelope_payload.disbursement_envelope_id == ""
):
_logger.error("Invalid disbursement envelope ID")
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.INVALID_DISBURSEMENT_ENVELOPE_ID
)
if (
disbursement_envelope_payload.number_of_beneficiaries is None
or disbursement_envelope_payload.number_of_beneficiaries < 1
):
_logger.error("Invalid number of beneficiaries")
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.INVALID_NO_OF_BENEFICIARIES
)
if (
disbursement_envelope_payload.number_of_disbursements is None
or disbursement_envelope_payload.number_of_disbursements < 1
):
_logger.error("Invalid number of disbursements")
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.INVALID_NO_OF_DISBURSEMENTS
)
if (
disbursement_envelope_payload.total_disbursement_amount is None
or disbursement_envelope_payload.total_disbursement_amount < 0
):
_logger.error("Invalid total disbursement amount")
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.INVALID_TOTAL_DISBURSEMENT_AMOUNT
)
if (
disbursement_envelope_payload.disbursement_schedule_date is None
or disbursement_envelope_payload.disbursement_schedule_date
< datetime.date(datetime.utcnow())
):
_logger.error("Invalid disbursement schedule date")
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.INVALID_DISBURSEMENT_SCHEDULE_DATE
)
return True

async def update_disbursement_envelope(
self, disbursement_envelope_payload: DisbursementEnvelopePayload, session
) -> DisbursementEnvelopePayload:
_logger.info("Updating disbursement envelope")
disbursement_envelope: DisbursementEnvelope = (
await session.execute(
select(DisbursementEnvelope).where(
DisbursementEnvelope.id == disbursement_envelope_payload.id
)
)
).scalar()

disbursement_envelope.number_of_beneficiaries = (
disbursement_envelope_payload.number_of_beneficiaries
)
disbursement_envelope.number_of_disbursements = (
disbursement_envelope_payload.number_of_disbursements
)
disbursement_envelope.total_disbursement_amount = (
disbursement_envelope_payload.total_disbursement_amount
)
disbursement_envelope.disbursement_schedule_date = (
disbursement_envelope_payload.disbursement_schedule_date
)

await session.commit()
_logger.info("Disbursement envelope updated successfully")
return disbursement_envelope_payload

async def amend_disbursement_envelope(
self, disbursement_envelope_request: DisbursementEnvelopeRequest
) -> DisbursementEnvelopePayload:
_logger.info("Amending disbursement envelope")
session_maker = async_sessionmaker(dbengine.get(), expire_on_commit=False)
async with session_maker() as session:
try:
await self.validate_envelope_amend_request(
disbursement_envelope_request
)
except DisbursementEnvelopeException as e:
raise e

disbursement_envelope_payload: DisbursementEnvelopePayload = (
disbursement_envelope_request.message
)
disbursement_envelope_id: str = (
disbursement_envelope_payload.disbursement_envelope_id
)

disbursement_envelope: DisbursementEnvelope = (
await session.execute(
select(DisbursementEnvelope).where(
DisbursementEnvelope.disbursement_envelope_id
== disbursement_envelope_id
)
)
).scalar()

if disbursement_envelope is None:
_logger.error(
f"Disbursement envelope with ID {disbursement_envelope_id} not found"
)
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.DISBURSEMENT_ENVELOPE_NOT_FOUND
)

if (
disbursement_envelope.cancellation_status
== CancellationStatus.Cancelled.value
):
_logger.error(
f"Disbursement envelope with ID {disbursement_envelope_id} already cancelled"
)
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.DISBURSEMENT_ENVELOPE_ALREADY_CANCELED
)

if disbursement_envelope.disbursement_schedule_date <= datetime.date(
datetime.utcnow()
):
_logger.error(
f"Disbursement envelope with ID {disbursement_envelope_id} date is already passed"
)
raise DisbursementEnvelopeException(
G2PBridgeErrorCodes.DISBURSEMENT_ENVELOPE_DATE_PASSED
)

disbursement_envelope_payload.disbursement_envelope_id = (
disbursement_envelope_id
)
disbursement_envelope_payload.id = disbursement_envelope.id

disbursement_envelope_payload = await self.update_disbursement_envelope(
disbursement_envelope_payload, session
)

await session.commit()
_logger.info("Disbursement envelope amended successfully")
return disbursement_envelope_payload
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ async def construct_disbursement_envelope_status_error_response(
status=StatusEnum.rjct,
status_reason_message=code,
),
message={},
message=None,
)

return response
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
DisbursementRecon,
)
from openg2p_g2p_bridge_models.schemas import (
DisbursementEnvelopeStatusResponse,
DisbursementErrorReconPayload,
DisbursementReconPayload,
DisbursementReconRecords,
Expand Down Expand Up @@ -139,7 +138,7 @@ async def construct_disbursement_status_error_response(
disbursement_status_request: DisbursementStatusRequest,
code: str,
) -> DisbursementStatusResponse:
response = DisbursementEnvelopeStatusResponse(
response = DisbursementStatusResponse(
header=SyncResponseHeader(
message_id=disbursement_status_request.header.message_id,
message_ts=datetime.now().isoformat(),
Expand Down
Loading
Loading