Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong encoding of SBc-AP Write-Replace Warning Request (APER enc) #94

Closed
pespin opened this issue Jul 8, 2022 · 41 comments
Closed

Wrong encoding of SBc-AP Write-Replace Warning Request (APER enc) #94

pespin opened this issue Jul 8, 2022 · 41 comments

Comments

@pespin
Copy link

pespin commented Jul 8, 2022

Hi,

I'm introducing SBc-AP protocol support [1] to osmo-cbc [2] project. For that, I'm using code generated with asn1c commit 2cf625c (mouse07410 branch "vlm_master" including last fix of mine merged yesterday/today).

I'm validating the generated output using several methods:

  • wireshark analysis of pcap files:
    ** Pcap file of another existing external implementation found here.
    ** Pcap file generated by osmo-cbc when running TTCN3 tests [3]
  • libfftranscode decoding [4]
  • https://asn1.io/asn1playground/default.aspx?

I'm adding the SBc-AP support in osmo-cbc.git branch "pespin/sbcap".

First commit adds the skeletons + generated asn1c code (from asn files found in same repo in src/sbcap/asn1/*.asn) with mouse07410/vlm_master plus all the boilerplate to build the content in a library (the Makefile also modified slightly some headers to avoid some circular dependency issues, but that's another topic):
https://cgit.osmocom.org/osmo-cbc/commit/?h=pespin/sbcap&id=469ab793be1d1fe4924a971ad0b6fb84a2ce8d20
asn1c code can be regenerated by calling "make -C src/ regen".

Second commit adds a unit test to encode SBc-AP Write-Replace Warning Request (you can run it with "make check"):
https://cgit.osmocom.org/osmo-cbc/commit/?h=pespin/sbcap&id=8ff4fed49b63d184521aebb3cb3730651c9e49af
It can be seen that the encoded output is:
00 00 00 0e 00 02 00 05 00 02 ab 01 00 0b 00 02 ab cd
When decoding with wireshark, libfftranscode or asn1.io, there seems to be issues parsing the length of the item list (the IE list). When comparing to the message in the existing implementation (available above), it also doesn't match, it starts with 00 00 00 5f 00 00 0b, where of course values 5f and 0b are different since those contain the length of the message and the 11 IEs it contains. However, you can see how in between those there's an extra 00 byte.

In third commit, I apply the proposed changes in my asn1c PR [5]:
https://cgit.osmocom.org/osmo-cbc/commit/?h=pespin/sbcap&id=2c54d1d135018ad57de70a7be47bc701b0a4970f
You can see how the output of the unit test changes to something correct:

-Encoded message: 00 00 00 0e 00 02 00 05 00 02 ab 01 00 0b 00 02 ab cd 
+Encoded message: 00 00 00 0f 00 00 02 00 05 00 02 ab 01 00 0b 00 02 ab cd 

Once this change is applied, wireshark, fftranscode and as1n.io are happy with the message.

[1] 3GPP TS 29.168, https://www.etsi.org/deliver/etsi_ts/129100_129199/129168/16.00.00_60/ts_129168v160000p.pdf
[2] https://osmocom.org/projects/osmo-cbc/wiki
[3] https://gitea.osmocom.org/ttcn3/osmo-ttcn3-hacks/src/branch/pespin/sbcap
[4] https://osmocom.org/projects/cellular-infrastructure/wiki/Titan_TTCN3_Testsuites#Proprietary-APERBER-transcoding-library-for-Iu-tests
[5] #93

@pespin
Copy link
Author

pespin commented Jul 8, 2022

I attach output of "sbcap_test" unit test BEFORE applying the asn1c changes from [5], compiled with -DASN_EMIT_DEBUG=1.

sbcap_test_before_change.txt

@mouse07410
Copy link
Owner

Ok, do people interested in #91 and #93 agree that
A. The current code violates X.691 INTEGER length encoding?
B. The fix proposed in #93 restores conformance?

@pespin
Copy link
Author

pespin commented Jul 8, 2022

@mouse07410 the problem is specifically seen when encoding length of sequence (list of fields), which as it was pointed out, specified by X.691 11.9 (length determinant):

NOTE 1 ... list of fields (with the length count in components of a sequence-of or set-of).

In general in section 11.9, my understanding is that it usually talks about an upper bound "ub" that is less than
64K
, where 64K=65536.

Hence, it usually talks about (ub < 65536 && lb >= 0). So, the maximum ub is 65536-1=65535. However, the code in aper_put_length() uses range, which is range=ub-lb+1. So, the maximum range to check for is 65536.

Hence, (range <= 65536 && range >= 0) is the counterpart of (ub < 65536 && lb >= 0) (note "<=" vs "<").
I'm new to reading ASN1 specs though, so please feel free to provide your own thoughts.

@laf0rge
Copy link

laf0rge commented Jul 8, 2022

Thanks @pespin, this explains a lot. Indeed, it is the case that skeletons/constr_SEQUENCE_OF_aper.c and skeletons/constr_SET_OF_aper.c are using calls like aper_put_length(po, ct->upper_bound - ct->lower_bound + 1, list->count - ct->lower_bound, 0) where it computes a range (ub-lb+1).

In my opinion, to avoid further misunderstnanding, I would suggest to

  • define the aper_put_length() to use n (number of elements in the sequence/set) as argument, not range
  • adjust the callers to pass n and not range. There are only two callers passing a non-negative number as second argument today: the two functions I mentioned above.

This makes IMHO most sense as the chapter 11.9 sections realted to APER indeed never talk about range. Only 11.9.4 for UPER talks about range.

@laf0rge
Copy link

laf0rge commented Jul 8, 2022

Ok, do people interested in #91 and #93 agree that A. The current code violates X.691 INTEGER length encoding? B. The fix proposed in #93 restores conformance?

To clarify/summarize the latest understanding: INTEGER indeed seems fine, but constrained length fields (such as those used for SEQUENCE OF or SET OF where there is a limited number of elements permitted) is wrong. This is caused by the confusing use of different terms than the spec (range instead of n) together with the constants for the term used in the spec (n)

@mouse07410
Copy link
Owner

Thank you! So, does #93 need more work to accommodate suggestions from #94 (comment)?

While it's possible to cherry-pick individual commits, I'd prefer to deal with the entire PR (time factors).

@mouse07410
Copy link
Owner

@pespin and @laf0rge do you confirm that #93 can be merged as-is?

@pespin
Copy link
Author

pespin commented Jul 11, 2022

@mouse07410 IMHO it can already be merged, since it clearly shows some cases are fixed. Later on code can be improved to adapt to terminology used in the specs, but that'd be a separate commit/topic.

Other people having problems when these new patches are merged should provide more detailed bug reports on what is exactly failing on their side.

@mouse07410
Copy link
Owner

@grzegorzniemirowski if you care about this issue, and have (detailed) bug reports (and, maybe, an X.691-compliant recommendation how to fix it), please comment here.

@grzegorzniemirowski
Copy link

grzegorzniemirowski commented Jul 12, 2022

@mouse07410 Detailed bug report has already been given in #62 The issue is that length is encoded in one byte but asn1c expects two bytes. Additionaly I can attach PCAP and test code.

#include <sys/types.h>
#include <stdio.h>
#include "asn_application.h"
#include "asn_internal.h"
#include "F1AP-PDU.h"

uint8_t buf[] = {
  0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x03, 0x00,
  0x4e, 0x00, 0x02, 0x00, 0x00, 0x00, 0x00, 0x40,
  0x01, 0x00, 0x00, 0x30, 0x00, 0x0a, 0x40, 0x01,
  0x00, 0x50, 0x00, 0x04, 0x60, 0x20, 0x00, 0x01
};

int main() {
    F1AP_PDU_t* pdu = NULL;
    int len = sizeof(buf);
    asn_dec_rval_t ret = aper_decode_complete(NULL, &asn_DEF_F1AP_PDU, &pdu, buf, len);
    printf("%ld %d\n", ret.consumed, ret.code);
    return 0;
}

F1AP_Reset.zip

@pespin
Copy link
Author

pespin commented Jul 12, 2022

hi @grzegorzniemirowski , can you provide information on how was that binary stream encoded? which implementation generated that binary stream? asn1c itself? which version?

@grzegorzniemirowski
Copy link

@pespin it was sent by Accelleran gNB-CU

@pespin
Copy link
Author

pespin commented Jul 12, 2022

I tried decoding it with https://asn1.io/asn1playground/default.aspx?

I had to pass "buf" hexstring values above removing the comas.

Console  Output

OSS ASN-1Step Version 10.2.1
Copyright (C) 2022 OSS Nokalva, Inc.  All rights reserved.
This product is licensed for use by "OSS Nokalva, Inc."

C0043I: 0 error messages, 0 warning messages and 0 informatory messages issued.


ASN1STEP: Decoding PDU #1 :

rec1value F1AP-PDU ::= 
  --TYPE INFORMATION: CHOICE
  --OFFSET: 0,0
  --choice index: <.11> (index = 3)
  choice-extension :
  {
    --TYPE INFORMATION: SEQUENCE
    --OFFSET: 0,2
    id 7476,
      --TYPE INFORMATION: INTEGER (0..65535)
      --OFFSET: 0,2; LENGTH: 2,6
      --padding: <010011>
      --contents: .1D.34
    criticality <value not decoded>,
      --TYPE INFORMATION: ENUMERATED {reject,ignore,notify}
      --OFFSET: 3,0; LENGTH: 0,2
      --contents: <.11>
D0071S: Value not among the ENUMERATED: 3; check field 'criticality' of field 'choice-extension' of PDU #1.
value: D0071S: Value not among the ENUMERATED: 3; check field 'criticality' of field 'choice-extension' of PDU #1.
S0014E: Printing of PER details failed with the return code '17'.


(....)
The console diagnostics might be truncated. To avoid this, please Sign In and have a valid license.
Results
Operation failed. See Console Output. 

So at first glance it looks like the initial encoding is wrong?

@grzegorzniemirowski
Copy link

grzegorzniemirowski commented Jul 12, 2022

It looks like your hex string was wrong. Here is PER encoded binary file to upload to the playground: F1AP_Reset_PER.zip. Please note that Wireshark has no problem with decoding as well.

OSS ASN-1Step Version 10.2.1
Copyright (C) 2022 OSS Nokalva, Inc.  All rights reserved.
This product is licensed for use by "OSS Nokalva, Inc."

C0043I: 0 error messages, 0 warning messages and 0 informatory messages issued.


ASN1STEP: Decoding PDU #1 :

rec1value F1AP-PDU ::= 
  --TYPE INFORMATION: CHOICE
  --OFFSET: 0,0
  --choice index: <.00> (index = 0)
  initiatingMessage :
  {
    --TYPE INFORMATION: SEQUENCE
    --OFFSET: 0,2
    procedureCode 0,
      --TYPE INFORMATION: INTEGER (0..255)
      --OFFSET: 0,2; LENGTH: 1,6
      --padding: <000000>
      --contents: .00
    criticality reject,
      --TYPE INFORMATION: ENUMERATED {reject,ignore,notify}
      --OFFSET: 2,0; LENGTH: 0,2
      --contents: <.00>
    --padding: <000000>
    --open type length: .1C (decoded as 28)
    value Reset:
    {
      --TYPE INFORMATION: OpenType - identified as SEQUENCE
      --OFFSET: 4,0
      --extension flag: <.0>
      protocolIEs
      {
        --TYPE INFORMATION: SEQUENCE (SIZE(0..65535)) OF
        --OFFSET: 4,1
        --padding: <0000000>
        --length: .00.03 (decoded as 3)
        {
          --TYPE INFORMATION: SEQUENCE
          --
(....)
The console diagnostics might be truncated. To avoid this, please Sign In and have a valid license.

You can also try different decoders. For example https://www.marben-products.com/decoder-asn1-nr/ Decoding attached PER encoded file produces following output:

<F1AP-PDU>
  <initiatingMessage>
    <procedureCode>0</procedureCode>
    <criticality>
      <reject/>
    </criticality>
    <value>
      <Reset>
        <protocolIEs>
          <SEQUENCE>
            <id>78</id>
            <criticality>
              <reject/>
            </criticality>
            <value>
              <TransactionID>0</TransactionID>
            </value>
          </SEQUENCE>
          <SEQUENCE>
            <id>0</id>
            <criticality>
              <ignore/>
            </criticality>
            <value>
              <Cause>
                <radioNetwork>
                  <unspecified/>
                </radioNetwork>
              </Cause>
            </value>
          </SEQUENCE>
          <SEQUENCE>
            <id>48</id>
            <criticality>
              <reject/>
            </criticality>
            <value>
              <ResetType>
                <partOfF1-Interface>
                  <SEQUENCE>
                    <id>80</id>
                    <criticality>
                      <reject/>
                    </criticality>
                    <value>
                      <UE-associatedLogicalF1-ConnectionItem>
                        <gNB-CU-UE-F1AP-ID>32</gNB-CU-UE-F1AP-ID>
                        <gNB-DU-UE-F1AP-ID>1</gNB-DU-UE-F1AP-ID>
                      </UE-associatedLogicalF1-ConnectionItem>
                    </value>
                  </SEQUENCE>
                </partOfF1-Interface>
              </ResetType>
            </value>
          </SEQUENCE>
        </protocolIEs>
      </Reset>
    </value>
  </initiatingMessage>
</F1AP-PDU>

32 bytes supplied, 32 bytes decoded.
*** DECODING SUCCESSFUL ***

And another one: https://ridenext.co.in/asndecoder.html It needs hex input in following form: 0000001c000003004e0002000000004001000030000a40010050000460200001
And the output is:

{

    "id-Reset": [
        {
            "initiatingMessage": [
                {
                    "procedureCode": 0
                },
                {
                    "criticality": 0
                },
                {
                    "value": [
                        {
                            "protocolIEs": [
                                {
                                    "id-TransactionID": 0
                                },
                                {
                                    "id-Cause": [
                                        {
                                            "radioNetwork": 0
                                        }
                                    ]
                                },
                                {
                                    "id-ResetType": [
                                        {
                                            "partOfF1_Interface": [
                                                [
                                                    {
                                                        "id": 80
                                                    },
                                                    {
                                                        "criticality": 0
                                                    },
                                                    {
                                                        "value": [
                                                            {
                                                                "id-UE-associatedLogicalF1-ConnectionItem": [
                                                                    {
                                                                        "gNB-CU-UE-F1AP-ID": 32
                                                                    },
                                                                    {
                                                                        "gNB-DU-UE-F1AP-ID": 1
                                                                    }
                                                                ]
                                                            }
                                                        ]
                                                    }
                                                ]
                                            ]
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ]

}

Summing up: Wireshark and three ASN1 decoders have no problem with this message.

@mouse07410
Copy link
Owner

@grzegorzniemirowski do #96 and #97 help the problem?

I don't have ASN.1 for F1AP, so am unlikely to reproduce.

@laf0rge
Copy link

laf0rge commented Jul 13, 2022

As there's no OCTET STRING in the example, I don't think #96 will have any impact. #97 might.

@grzegorzniemirowski
Copy link

Indeed. They are not related to SEQUENCE length and they don't help.
F1AP ASN: F1AP-16.7.0.zip

@mouse07410
Copy link
Owner

mouse07410 commented Jul 13, 2022

I would suggest to

  • define the aper_put_length() to use n (number of elements in the sequence/set) as argument, not range
  • adjust the callers to pass n and not range. There are only two callers passing a non-negative number as second argument today: the two functions I mentioned above.

Update

This sounds reasonable, and at least worth trying.
Do I understand correctly that the way length is encoded in APER should be different for, e.g., INTEGER and SEQUENCE OF? Should the fix apply only to skeletons/constr_SEQUENCE_OF_aper.c (and SET_OF), or also involve skeletons/aper_support.c?

I don't have X.691 - what does it say exactly about the length encoding?

What's the semantics of "range" for constraints on SEQUENCE OF?

Anybody cares to submit a PR implementing the above? To avoid messing with what's already working, I'd probably add another function, like aper_put_seq_length() with the parameters as described above.

Comments? PRs?

@grzegorzniemirowski
Copy link

What was wrong with #91 actually? What was broken by this PR?

@mouse07410
Copy link
Owner

mouse07410 commented Jul 14, 2022

@grzegorzniemirowski @laf0rge do you think the line

} else if (aper_put_length(po, ct->upper_bound - ct->lower_bound + 1, list->count - ct->lower_bound, 0) < 0)
should be

} else if (aper_put_length(po, ct->upper_bound - ct->lower_bound + (ct->lower_bound>0)?1:0, list->count - ct->lower_bound, 0) < 0)

and in aper_support.c change<= 65536 to < 65536? That's how I'd interpret X.691 §10.9.

Update

Also, looking at check-APER-support.c, I'm not certain it's correct, matching range against 65536, because it seems to me that X.691 X.691 §10.9 states less than 65536. Comments?

@mouse07410
Copy link
Owner

F1AP ASN: F1AP-16.7.0.zip

Unfortunately, this ASN.1 file does not produce directly-usable (aka, compile-able) code:

.  .  .
clang -O3 -std=gnu18 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk  -DASN_PDU_COLLECTION -I. -o F1AP-PDU.o -c F1AP-PDU.c
In file included from F1AP-PDU.c:8:
In file included from ./F1AP-PDU.h:58:
In file included from ./InitiatingMessage.h:19:
In file included from ./Reset.h:15:
In file included from ./ProtocolIE-Container.h:1089:
In file included from ./ProtocolIE-Field.h:22:
In file included from ./UE-associatedLogicalF1-ConnectionItem.h:48:
In file included from ./ProtocolExtensionContainer.h:3663:
In file included from ./ProtocolExtensionField.h:25:
In file included from ./NPNBroadcastInformation.h:57:
./ProtocolIE-SingleContainer.h:22:9: error: unknown type name 'F1AP_PDU_ExtIEs_t'
typedef F1AP_PDU_ExtIEs_t        ProtocolIE_SingleContainer_10757P0_t;
        ^
./ProtocolIE-SingleContainer.h:23:9: error: unknown type name 'ResetType_ExtIEs_t'
typedef ResetType_ExtIEs_t       ProtocolIE_SingleContainer_10757P1_t;
        ^
./ProtocolIE-SingleContainer.h:24:9: error: unknown type name 'UE_associatedLogicalF1_ConnectionItemRes_t'; did you mean 'UE_associatedLogicalF1_ConnectionItem_t'?
typedef UE_associatedLogicalF1_ConnectionItemRes_t       ProtocolIE_SingleContainer_10757P2_t;
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        UE_associatedLogicalF1_ConnectionItem_t
./UE-associatedLogicalF1-ConnectionItem.h:38:3: note: 'UE_associatedLogicalF1_ConnectionItem_t' declared here
} UE_associatedLogicalF1_ConnectionItem_t;
  ^
In file included from F1AP-PDU.c:8:
In file included from ./F1AP-PDU.h:58:
In file included from ./InitiatingMessage.h:19:
In file included from ./Reset.h:15:
In file included from ./ProtocolIE-Container.h:1089:
In file included from ./ProtocolIE-Field.h:22:
In file included from ./UE-associatedLogicalF1-ConnectionItem.h:48:
In file included from ./ProtocolExtensionContainer.h:3663:
In file included from ./ProtocolExtensionField.h:25:
In file included from ./NPNBroadcastInformation.h:57:
./ProtocolIE-SingleContainer.h:25:9: error: unknown type name 'UE_associatedLogicalF1_ConnectionItemResAck_t'; did you mean 'UE_associatedLogicalF1_ConnectionItem_t'?
typedef UE_associatedLogicalF1_ConnectionItemResAck_t    ProtocolIE_SingleContainer_10757P3_t;
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        UE_associatedLogicalF1_ConnectionItem_t
./UE-associatedLogicalF1-ConnectionItem.h:38:3: note: 'UE_associatedLogicalF1_ConnectionItem_t' declared here
} UE_associatedLogicalF1_ConnectionItem_t;
  ^
In file included from F1AP-PDU.c:8:
In file included from ./F1AP-PDU.h:58:
In file included from ./InitiatingMessage.h:19:
In file included from ./Reset.h:15:
In file included from ./ProtocolIE-Container.h:1089:
In file included from ./ProtocolIE-Field.h:22:
In file included from ./UE-associatedLogicalF1-ConnectionItem.h:48:
In file included from ./ProtocolExtensionContainer.h:3663:
In file included from ./ProtocolExtensionField.h:25:
In file included from ./NPNBroadcastInformation.h:57:
./ProtocolIE-SingleContainer.h:26:9: error: unknown type name 'GNB_DU_Served_Cells_ItemIEs_t'
typedef GNB_DU_Served_Cells_ItemIEs_t    ProtocolIE_SingleContainer_10757P4_t;
        ^
./ProtocolIE-SingleContainer.h:27:9: error: unknown type name 'Cells_to_be_Activated_List_ItemIEs_t'
typedef Cells_to_be_Activated_List_ItemIEs_t     ProtocolIE_SingleContainer_10757P5_t;
        ^
./ProtocolIE-SingleContainer.h:28:9: error: unknown type name 'Served_Cells_To_Add_ItemIEs_t'
typedef Served_Cells_To_Add_ItemIEs_t    ProtocolIE_SingleContainer_10757P6_t;
        ^
./ProtocolIE-SingleContainer.h:29:9: error: unknown type name 'Served_Cells_To_Modify_ItemIEs_t'
typedef Served_Cells_To_Modify_ItemIEs_t         ProtocolIE_SingleContainer_10757P7_t;
        ^
./ProtocolIE-SingleContainer.h:30:9: error: unknown type name 'Served_Cells_To_Delete_ItemIEs_t'
typedef Served_Cells_To_Delete_ItemIEs_t         ProtocolIE_SingleContainer_10757P8_t;
        ^
./ProtocolIE-SingleContainer.h:31:9: error: unknown type name 'Cells_Status_ItemIEs_t'
typedef Cells_Status_ItemIEs_t   ProtocolIE_SingleContainer_10757P9_t;
        ^
./ProtocolIE-SingleContainer.h:32:9: error: unknown type name 'Dedicated_SIDelivery_NeededUE_ItemIEs_t'
typedef Dedicated_SIDelivery_NeededUE_ItemIEs_t  ProtocolIE_SingleContainer_10757P10_t;
        ^
./ProtocolIE-SingleContainer.h:33:9: error: unknown type name 'GNB_DU_TNL_Association_To_Remove_ItemIEs_t'
typedef GNB_DU_TNL_Association_To_Remove_ItemIEs_t       ProtocolIE_SingleContainer_10757P11_t;
        ^
./ProtocolIE-SingleContainer.h:34:9: error: unknown type name 'Cells_to_be_Deactivated_List_ItemIEs_t'
typedef Cells_to_be_Deactivated_List_ItemIEs_t   ProtocolIE_SingleContainer_10757P12_t;
        ^
./ProtocolIE-SingleContainer.h:35:9: error: unknown type name 'GNB_CU_TNL_Association_To_Add_ItemIEs_t'
typedef GNB_CU_TNL_Association_To_Add_ItemIEs_t  ProtocolIE_SingleContainer_10757P13_t;
        ^
./ProtocolIE-SingleContainer.h:36:9: error: unknown type name 'GNB_CU_TNL_Association_To_Remove_ItemIEs_t'
typedef GNB_CU_TNL_Association_To_Remove_ItemIEs_t       ProtocolIE_SingleContainer_10757P14_t;
        ^
./ProtocolIE-SingleContainer.h:37:9: error: unknown type name 'GNB_CU_TNL_Association_To_Update_ItemIEs_t'
typedef GNB_CU_TNL_Association_To_Update_ItemIEs_t       ProtocolIE_SingleContainer_10757P15_t;
        ^
./ProtocolIE-SingleContainer.h:38:9: error: unknown type name 'Cells_to_be_Barred_ItemIEs_t'
typedef Cells_to_be_Barred_ItemIEs_t     ProtocolIE_SingleContainer_10757P16_t;
        ^
./ProtocolIE-SingleContainer.h:39:9: error: unknown type name 'Protected_EUTRA_Resources_ItemIEs_t'
typedef Protected_EUTRA_Resources_ItemIEs_t      ProtocolIE_SingleContainer_10757P17_t;
        ^
./ProtocolIE-SingleContainer.h:40:9: error: unknown type name 'Neighbour_Cell_Information_ItemIEs_t'
typedef Neighbour_Cell_Information_ItemIEs_t     ProtocolIE_SingleContainer_10757P18_t;
        ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [Makefile:23: F1AP-PDU.o] Error 1

Unfortunately, I don't have time to "massage" include files order and whatever else needed to produce a working codec library. So, as I said, I cannot experiment with F1AP myself, and will have to constrain myself to attempts to ensure that asn1c code conforms to X.691.

@grzegorzniemirowski
Copy link

The code is produced by asn1c so it would be asn1c issue, not ASN file issue. It's official ASN for F1AP and I have been using it for months with asn1c. I guess you had some parameters wrong because I have no such problems:

$ asn1c -pdu=F1AP-PDU -fno-include-deps -findirect-choice -fcompound-names -flink-skeletons -gen-APER -no-gen-JER -gen-OER -gen-UPER F1AP-16.7.0.asn
$ clang -O3 -std=gnu18 -DASN_PDU_COLLECTION -I. -o F1AP-PDU.o -c F1AP-PDU.c
$ file F1AP-PDU.o
F1AP-PDU.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

@mouse07410
Copy link
Owner

mouse07410 commented Jul 14, 2022

I guess you had some parameters wrong . . .

Obviously so.

$ clang -O3 -std=gnu18 -DASN_PDU_COLLECTION -I. -o F1AP-PDU.o -c F1AP-PDU.c
$ file F1AP-PDU.o
F1AP-PDU.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

Look, I don't deal with and don't care for Open5G, and am not going to become an expert in it. My interest is strictly limited to improving asn1c. So, whatever you can do to help me reproducing this problem, could potentially improve the chances of getting it fixed. On my own, I don't have time to fool around 5G encoding.

What do I do with F1AP-PDU.o? What I need is converter-example working, so I can point it at .xer file and generate .aper, and vs. versa. Normally, I generate converter-example by

$ make all -f converter-example.mk

Update

Trying to build the converter:

$ make all -f converter-example.mk
.  .  .
clang -O3 -std=gnu18 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk  -DPDU=F1AP_PDU -I. -o ProtocolExtensionField.o -c ProtocolExtensionField.c
ProtocolExtensionField.c:66989:5: error: use of undeclared identifier 'asn_OER_memb_OCTET_STRING_SIZE_3__constr_96'; did you mean 'memb_OCTET_STRING_SIZE_3__constraint_924'?
                        &asn_OER_memb_OCTET_STRING_SIZE_3__constr_96,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         memb_OCTET_STRING_SIZE_3__constraint_924
ProtocolExtensionField.c:17440:1: note: 'memb_OCTET_STRING_SIZE_3__constraint_924' declared here
memb_OCTET_STRING_SIZE_3__constraint_924(const asn_TYPE_descriptor_t *td, const void *sptr,
^
ProtocolExtensionField.c:66992:5: error: use of undeclared identifier 'asn_PER_memb_OCTET_STRING_SIZE_3__constr_96'; did you mean 'memb_OCTET_STRING_SIZE_3__constraint_924'?
                        &asn_PER_memb_OCTET_STRING_SIZE_3__constr_96,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         memb_OCTET_STRING_SIZE_3__constraint_924
.  .  .
2 warnings and 2 errors generated.
make: *** [ProtocolExtensionField.o] Error 1

@mouse07410
Copy link
Owner

mouse07410 commented Jul 14, 2022

@pespin I'd like you to explain why check-APER-support.c validates round-trip against 65536, rather than, e.g., 65535.

Also, I'd like to hear an answer to @grzegorzniemirowski 's question "what breaks when aper_support.c conforms closer to X.691 and uses strict < 65536 instead of the current <= 65536?"

@pespin
Copy link
Author

pespin commented Jul 14, 2022

@mouse07410 I used check_round_trip(65536, ...) since that was the faulty case (small value with range=65536) that I discovered when using the code generated from SBc-AP asn files (3GPP TS 29.168).
Basically the case is an ASN1 sequence type which had a length determinant of range=65536 (0..65535), uint16

I saw the range=65536 in the debug output (-DASN_EMIT_DEBUG=1) when encoding the structure, and it saw it was the part which was encoding the unexpected missing byte. So I basically ported that to a unit test to be able to reproduce it easily, fix it and avoid future regressions.

I guess the best to figure out what's wrong with the decoding from @grzegorzniemirowski would be to use the hexstring he provided, and add a unit test decoding it and enable ASN_DEBUG, so that we can see exactly where it fails and we can reproduce easily.
Then once fixed we also make sure it doesn't break since we have a unit test.

@pespin
Copy link
Author

pespin commented Jul 14, 2022

Actually, the best would be that @grzegorzniemirowski provides the debug output (-DASN_EMIT_DEBUG=1) since he's the once having the support structures from the asn file to decode.
From the debug output we can perhaps find out the exact point at which it fails, and try to write a generic unit test which triggers the issue, so that there's no need to have lots of files in asn1c.git

@pespin
Copy link
Author

pespin commented Jul 14, 2022

Also, looking at check-APER-support.c, I'm not certain it's correct, matching range against 65536, because it seems to me that X.691 X.691 §10.9 states less than 65536. Comments?

I'm looking at https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-X.691-202102-I!!PDF-E&type=items
in there it seems to be section 11.9, not 10.9.
In any case, it was already explained here: #94 (comment)
In summary, the spec talks about "an upper bound that is less than 65536". Which for range=ub-lb+1, lb=0 and maximum possible ub<65536 (0..65535, uint16) it means range<=65536, which is what my patch fixes.

@pespin
Copy link
Author

pespin commented Jul 14, 2022

@grzegorzniemirowski also if possible include a pcap file containing the packet being decoded with ASN debug enabled, so that the hexstring can be identified easily with the protocol fields.

@grzegorzniemirowski
Copy link

@mouse07410 I don't care about Open5G too. I care about 3GPP ASN which should be supported as any other ASN. Or at least it would be very nice if it was. I have spent many hours on this issue so I'm not a person who just ask others to do the work. I'm not even the submitter of #62 but a contributor of PR which in my opinion was fixing the problem.

I don't need F1AP-PDU.o itself, I was just refering to your example of failing compilation and provided correct parameters to help you with code generation.

The converter building error is just another asn1c bug and it was mentioned earlier in #77 (see raoufkh's comment at the end). You need to manually edit generated code using compiler's hint: replace asn_PER_memb_OCTET_STRING_SIZE_3__constr_96 with memb_OCTET_STRING_SIZE_3__constraint_924. Then converter-example would be compiled correctly.

@grzegorzniemirowski
Copy link

grzegorzniemirowski commented Jul 14, 2022

@pespin You can find the PCAP in one of my previous comments. The important part of debug output was in original issue #62. I'm pasting it again in full length. It's visible that in order to get length the nsnnwn is read instead of one byte according to 11.9.a X.691.

  [PER got  2<=256 bits => span 2 +4[2..256]:00 (254) => 0x0] (asn_bit_data.c:132)
CHOICE F1AP-PDU got index 0 in range 2 (constr_CHOICE_aper.c:48)
Discovered CHOICE F1AP-PDU encodes initiatingMessage (constr_CHOICE_aper.c:80)
Decoding InitiatingMessage as SEQUENCE (APER) (constr_SEQUENCE_aper.c:40)
Decoding member "procedureCode" in InitiatingMessage (constr_SEQUENCE_aper.c:130)
Decoding NativeInteger ProcedureCode (APER) (NativeInteger_aper.c:21)
Integer with range 8 bits (INTEGER_aper.c:54)
Aligning 6 bits (aper_support.c:13)
  [PER got  6<=254 bits => span 8 +4[8..256]:00 (248) => 0x0] (asn_bit_data.c:132)
  [PER got  8<=248 bits => span 16 +5[8..248]:00 (240) => 0x0] (asn_bit_data.c:132)
Got value 0 + low 0 (INTEGER_aper.c:114)
NativeInteger ProcedureCode got value 0 (NativeInteger_aper.c:37)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:16)
Decoding member "criticality" in InitiatingMessage (constr_SEQUENCE_aper.c:130)
Decoding Criticality as NativeEnumerated (NativeEnumerated_aper.c:33)
  [PER got  2<=240 bits => span 18 +6[2..240]:00 (238) => 0x0] (asn_bit_data.c:132)
Decoded Criticality = 0 (NativeEnumerated_aper.c:79)
Decoding member "value" in InitiatingMessage (constr_SEQUENCE_aper.c:130)
Getting open type Reset... (aper_opentype.c:25)
Aligning 6 bits (aper_support.c:13)
  [PER got  6<=238 bits => span 24 +6[8..240]:00 (232) => 0x0] (asn_bit_data.c:132)
  [PER got  8<=232 bits => span 32 +7[8..232]:1c (224) => 0x1c] (asn_bit_data.c:132)
  [PER got 24<=224 bits => span 56 +8[24..224]:00 (200) => 0x3] (asn_bit_data.c:132)
  [PER got 24<=200 bits => span 80 +11[24..200]:00 (176) => 0x4e00] (asn_bit_data.c:132)
  [PER got 24<=176 bits => span 104 +14[24..176]:02 (152) => 0x20000] (asn_bit_data.c:132)
  [PER got 24<=152 bits => span 128 +1[24..152]:00 (128) => 0x40] (asn_bit_data.c:132)
  [PER got 24<=128 bits => span 152 +4[24..128]:01 (104) => 0x10000] (asn_bit_data.c:132)
  [PER got 24<=104 bits => span 176 +7[24..104]:30 (80) => 0x30000a] (asn_bit_data.c:132)
  [PER got 24<=80 bits => span 200 +10[24..80]:40 (56) => 0x400100] (asn_bit_data.c:132)
  [PER got 24<=56 bits => span 224 +13[24..56]:50 (32) => 0x500004] (asn_bit_data.c:132)
  [PER got 24<=32 bits => span 248 +0[24..32]:60 (8) => 0x602000] (asn_bit_data.c:132)
  [PER got  8<= 8 bits => span 256 +3[8..8]:01 (0) => 0x1] (asn_bit_data.c:132)
Getting open type Reset encoded in 28 bytes (aper_opentype.c:50)
Decoding Reset as SEQUENCE (APER) (constr_SEQUENCE_aper.c:40)
  [PER got  1<=224 bits => span 1 +0[1..224]:00 (223) => 0x0] (asn_bit_data.c:132)
Decoding member "protocolIEs" in Reset (constr_SEQUENCE_aper.c:130)
getting nsnnwn with range 65536 (aper_support.c:74)
Aligning 7 bits (aper_support.c:13)
  [PER got  7<=223 bits => span 8 +0[8..224]:00 (216) => 0x0] (asn_bit_data.c:132)
  [PER got 16<=216 bits => span 24 +1[16..216]:00 (200) => 0x3] (asn_bit_data.c:132)
Preparing to fetch 3+0 elements from (null) (constr_SET_OF_aper.c:133)
SET OF ResetIEs decoding (constr_SET_OF_aper.c:153)
Decoding ResetIEs as SEQUENCE (APER) (constr_SEQUENCE_aper.c:40)
Decoding member "id" in ResetIEs (constr_SEQUENCE_aper.c:130)
Decoding NativeInteger ProtocolIE-ID (APER) (NativeInteger_aper.c:21)
Integer with range 16 bits (INTEGER_aper.c:54)
  [PER got 16<=200 bits => span 40 +3[16..200]:00 (184) => 0x4e] (asn_bit_data.c:132)
Got value 78 + low 0 (INTEGER_aper.c:114)
NativeInteger ProtocolIE-ID got value 78 (NativeInteger_aper.c:37)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:16)
Decoding member "criticality" in ResetIEs (constr_SEQUENCE_aper.c:130)
Decoding Criticality as NativeEnumerated (NativeEnumerated_aper.c:33)
  [PER got  2<=184 bits => span 42 +5[2..184]:00 (182) => 0x0] (asn_bit_data.c:132)
Decoded Criticality = 0 (NativeEnumerated_aper.c:79)
Decoding member "value" in ResetIEs (constr_SEQUENCE_aper.c:130)
    Getting open type TransactionID... (aper_opentype.c:25)
Aligning 6 bits (aper_support.c:13)
  [PER got  6<=182 bits => span 48 +5[8..184]:00 (176) => 0x0] (asn_bit_data.c:132)
  [PER got  8<=176 bits => span 56 +6[8..176]:02 (168) => 0x2] (asn_bit_data.c:132)
  [PER got 16<=168 bits => span 72 +7[16..168]:00 (152) => 0x0] (asn_bit_data.c:132)
    Getting open type TransactionID encoded in 2 bytes (aper_opentype.c:50)
Decoding NativeInteger TransactionID (APER) (NativeInteger_aper.c:21)
  [PER got  1<=16 bits => span 1 +0[1..16]:00 (15) => 0x0] (asn_bit_data.c:132)
Integer with range 8 bits (INTEGER_aper.c:54)
Aligning 7 bits (aper_support.c:13)
  [PER got  7<=15 bits => span 8 +0[8..16]:00 (8) => 0x0] (asn_bit_data.c:132)
  [PER got  8<= 8 bits => span 16 +1[8..8]:00 (0) => 0x0] (asn_bit_data.c:132)
Got value 0 + low 0 (INTEGER_aper.c:114)
NativeInteger TransactionID got value 0 (NativeInteger_aper.c:37)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:16)
    No padding (aper_opentype.c:77)
ProtocolIE-Container SET OF ResetIEs decoded 0, 0x1398250 (constr_SET_OF_aper.c:156)
SET OF ResetIEs decoding (constr_SET_OF_aper.c:153)
Decoding ResetIEs as SEQUENCE (APER) (constr_SEQUENCE_aper.c:40)
Decoding member "id" in ResetIEs (constr_SEQUENCE_aper.c:130)
Decoding NativeInteger ProtocolIE-ID (APER) (NativeInteger_aper.c:21)
Integer with range 16 bits (INTEGER_aper.c:54)
  [PER got 16<=152 bits => span 88 +9[16..152]:00 (136) => 0x0] (asn_bit_data.c:132)
Got value 0 + low 0 (INTEGER_aper.c:114)
NativeInteger ProtocolIE-ID got value 0 (NativeInteger_aper.c:37)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:16)
Decoding member "criticality" in ResetIEs (constr_SEQUENCE_aper.c:130)
Decoding Criticality as NativeEnumerated (NativeEnumerated_aper.c:33)
  [PER got  2<=136 bits => span 90 +11[2..136]:40 (134) => 0x1] (asn_bit_data.c:132)
Decoded Criticality = 1 (NativeEnumerated_aper.c:79)
Decoding member "value" in ResetIEs (constr_SEQUENCE_aper.c:130)
    Getting open type Cause... (aper_opentype.c:25)
Aligning 6 bits (aper_support.c:13)
  [PER got  6<=134 bits => span 96 +11[8..136]:40 (128) => 0x0] (asn_bit_data.c:132)
  [PER got  8<=128 bits => span 104 +12[8..128]:01 (120) => 0x1] (asn_bit_data.c:132)
  [PER got  8<=120 bits => span 112 +13[8..120]:00 (112) => 0x0] (asn_bit_data.c:132)
    Getting open type Cause encoded in 1 bytes (aper_opentype.c:50)
  [PER got  3<= 8 bits => span 3 +8[3..8]:00 (5) => 0x0] (asn_bit_data.c:132)
CHOICE Cause got index 0 in range 3 (constr_CHOICE_aper.c:48)
Discovered CHOICE Cause encodes radioNetwork (constr_CHOICE_aper.c:80)
Decoding CauseRadioNetwork as NativeEnumerated (NativeEnumerated_aper.c:33)
  [PER got  1<= 5 bits => span 4 +8[4..8]:00 (4) => 0x0] (asn_bit_data.c:132)
  [PER got  4<= 4 bits => span 8 +8[8..8]:00 (0) => 0x0] (asn_bit_data.c:132)
Decoded CauseRadioNetwork = 0 (NativeEnumerated_aper.c:79)
    No padding (aper_opentype.c:77)
ProtocolIE-Container SET OF ResetIEs decoded 0, 0x13982c0 (constr_SET_OF_aper.c:156)
SET OF ResetIEs decoding (constr_SET_OF_aper.c:153)
Decoding ResetIEs as SEQUENCE (APER) (constr_SEQUENCE_aper.c:40)
Decoding member "id" in ResetIEs (constr_SEQUENCE_aper.c:130)
Decoding NativeInteger ProtocolIE-ID (APER) (NativeInteger_aper.c:21)
Integer with range 16 bits (INTEGER_aper.c:54)
  [PER got 16<=112 bits => span 128 +14[16..112]:00 (96) => 0x30] (asn_bit_data.c:132)
Got value 48 + low 0 (INTEGER_aper.c:114)
NativeInteger ProtocolIE-ID got value 48 (NativeInteger_aper.c:37)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:16)
Decoding member "criticality" in ResetIEs (constr_SEQUENCE_aper.c:130)
Decoding Criticality as NativeEnumerated (NativeEnumerated_aper.c:33)
  [PER got  2<=96 bits => span 130 +0[2..96]:00 (94) => 0x0] (asn_bit_data.c:132)
Decoded Criticality = 0 (NativeEnumerated_aper.c:79)
Decoding member "value" in ResetIEs (constr_SEQUENCE_aper.c:130)
    Getting open type ResetType... (aper_opentype.c:25)
Aligning 6 bits (aper_support.c:13)
  [PER got  6<=94 bits => span 136 +0[8..96]:00 (88) => 0x0] (asn_bit_data.c:132)
  [PER got  8<=88 bits => span 144 +1[8..88]:0a (80) => 0xa] (asn_bit_data.c:132)
  [PER got 24<=80 bits => span 168 +2[24..80]:40 (56) => 0x400100] (asn_bit_data.c:132)
  [PER got 24<=56 bits => span 192 +5[24..56]:50 (32) => 0x500004] (asn_bit_data.c:132)
  [PER got 24<=32 bits => span 216 +8[24..32]:60 (8) => 0x602000] (asn_bit_data.c:132)
  [PER got  8<= 8 bits => span 224 +11[8..8]:01 (0) => 0x1] (asn_bit_data.c:132)
    Getting open type ResetType encoded in 10 bytes (aper_opentype.c:50)
  [PER got  2<=80 bits => span 2 +0[2..80]:40 (78) => 0x1] (asn_bit_data.c:132)
CHOICE ResetType got index 1 in range 2 (constr_CHOICE_aper.c:48)
Discovered CHOICE ResetType encodes partOfF1-Interface (constr_CHOICE_aper.c:80)
getting nsnnwn with range 65536 (aper_support.c:74)
Aligning 6 bits (aper_support.c:13)
  [PER got  6<=78 bits => span 8 +0[8..80]:40 (72) => 0x0] (asn_bit_data.c:132)
  [PER got 16<=72 bits => span 24 +1[16..72]:01 (56) => 0x100] (asn_bit_data.c:132)
Got to decode 256 elements (eff -1) (constr_SET_OF_aper.c:146)
SET OF ProtocolIE-SingleContainer decoding (constr_SET_OF_aper.c:153)
Decoding ProtocolIE-SingleContainer as SEQUENCE (APER) (constr_SEQUENCE_aper.c:40)
Decoding member "id" in ProtocolIE-SingleContainer (constr_SEQUENCE_aper.c:130)
Decoding NativeInteger ProtocolIE-ID (APER) (NativeInteger_aper.c:21)
Integer with range 16 bits (INTEGER_aper.c:54)
  [PER got 16<=56 bits => span 40 +3[16..56]:50 (40) => 0x5000] (asn_bit_data.c:132)
Got value 20480 + low 0 (INTEGER_aper.c:114)
NativeInteger ProtocolIE-ID got value 20480 (NativeInteger_aper.c:37)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:16)
Decoding member "criticality" in ProtocolIE-SingleContainer (constr_SEQUENCE_aper.c:130)
Decoding Criticality as NativeEnumerated (NativeEnumerated_aper.c:33)
  [PER got  2<=40 bits => span 42 +5[2..40]:04 (38) => 0x0] (asn_bit_data.c:132)
Decoded Criticality = 0 (NativeEnumerated_aper.c:79)
Decoding member "value" in ProtocolIE-SingleContainer (constr_SEQUENCE_aper.c:130)
Aligning 6 bits (aper_support.c:13)
  [PER got  6<=38 bits => span 48 +5[8..40]:04 (32) => 0x4] (asn_bit_data.c:132)
  [PER got  8<=32 bits => span 56 +6[8..32]:60 (24) => 0x60] (asn_bit_data.c:132)
  [PER got 24<=24 bits => span 80 +7[24..24]:20 (0) => 0x200001] (asn_bit_data.c:132)
UE-associatedLogicalF1-ConnectionListRes SET OF ProtocolIE-SingleContainer decoded 0, 0x1398378 (constr_SET_OF_aper.c:156)
SET OF ProtocolIE-SingleContainer decoding (constr_SET_OF_aper.c:153)
Decoding ProtocolIE-SingleContainer as SEQUENCE (APER) (constr_SEQUENCE_aper.c:40)
Decoding member "id" in ProtocolIE-SingleContainer (constr_SEQUENCE_aper.c:130)
Decoding NativeInteger ProtocolIE-ID (APER) (NativeInteger_aper.c:21)
Integer with range 16 bits (INTEGER_aper.c:54)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:16)
Failed decode id in ProtocolIE-SingleContainer (constr_SEQUENCE_aper.c:145)
UE-associatedLogicalF1-ConnectionListRes SET OF ProtocolIE-SingleContainer decoded 1, 0x13983e0 (constr_SET_OF_aper.c:156)
Failed decoding ProtocolIE-SingleContainer of UE-associatedLogicalF1-ConnectionListRes (SET OF) (constr_SET_OF_aper.c:166)
Freeing ProtocolIE-SingleContainer as SEQUENCE (constr_SEQUENCE.c:76)
Freeing ProtocolIE-ID as INTEGER (1, 0x13983e0, Native) (NativeInteger.c:104)
Freeing Criticality as INTEGER (1, 0x13983e4, Native) (NativeInteger.c:104)
Freeing value as CHOICE (constr_CHOICE.c:164)
Failed to decode partOfF1-Interface in ResetType (CHOICE) 1 (constr_CHOICE_aper.c:91)
Freeing ResetType as CHOICE (constr_CHOICE.c:164)
Freeing ProtocolIE-SingleContainer as SEQUENCE (constr_SEQUENCE.c:76)
Freeing ProtocolIE-ID as INTEGER (1, 0x1398378, Native) (NativeInteger.c:104)
Freeing Criticality as INTEGER (1, 0x139837c, Native) (NativeInteger.c:104)
Freeing value as CHOICE (constr_CHOICE.c:164)
Failed decode value in ResetIEs (constr_SEQUENCE_aper.c:145)
ProtocolIE-Container SET OF ResetIEs decoded 2, 0x1398308 (constr_SET_OF_aper.c:156)
Failed decoding ResetIEs of ProtocolIE-Container (SET OF) (constr_SET_OF_aper.c:166)
Freeing ResetIEs as SEQUENCE (constr_SEQUENCE.c:76)
Freeing ProtocolIE-ID as INTEGER (1, 0x1398308, Native) (NativeInteger.c:104)
Freeing Criticality as INTEGER (1, 0x139830c, Native) (NativeInteger.c:104)
Freeing value as CHOICE (constr_CHOICE.c:164)
Failed decode protocolIEs in Reset (constr_SEQUENCE_aper.c:145)
Freeing Reset as SEQUENCE (constr_SEQUENCE.c:76)
Freeing ResetIEs as SEQUENCE (constr_SEQUENCE.c:76)
Freeing ProtocolIE-ID as INTEGER (1, 0x1398250, Native) (NativeInteger.c:104)
Freeing Criticality as INTEGER (1, 0x1398254, Native) (NativeInteger.c:104)
Freeing value as CHOICE (constr_CHOICE.c:164)
Freeing TransactionID as INTEGER (1, 0x139825c, Native) (NativeInteger.c:104)
Freeing ResetIEs as SEQUENCE (constr_SEQUENCE.c:76)
Freeing ProtocolIE-ID as INTEGER (1, 0x13982c0, Native) (NativeInteger.c:104)
Freeing Criticality as INTEGER (1, 0x13982c4, Native) (NativeInteger.c:104)
Freeing value as CHOICE (constr_CHOICE.c:164)
Freeing Cause as CHOICE (constr_CHOICE.c:164)
Freeing CauseRadioNetwork as INTEGER (1, 0x13982d0, Native) (NativeInteger.c:104)
Failed decode value in InitiatingMessage (constr_SEQUENCE_aper.c:145)
Failed to decode initiatingMessage in F1AP-PDU (CHOICE) 2 (constr_CHOICE_aper.c:91)

@pespin
Copy link
Author

pespin commented Jul 14, 2022

@grzegorzniemirowski that debug output is from 2.5 years ago. I think I'd be great to have an output with current vlm_master to make sure there haven't been side effects with other changes.
In any case, I think my encoding failure was encoding a "SEQUENCE_OF", while your decoding issue seems to be in "SEQUENCE". I don't know the exact differences between those, but that may be a good hint.

@grzegorzniemirowski
Copy link

@pespin you are right and yes, the above output is from current vlm_master, generated when writing the comment.
I suppose length encoding is the same for SEQUENCE_OF and SEQUENCE. I would rather look at the fact that in case of problematic field the lower bound is 1 and upper bound is 65536. So range is 65536 and equal to ub. As upper bound is not less than 64K, nsnnwn shouldn't be used. But the problem is that aper_get_length() checks for range, not upper bound. I guess ct->upper_bound should be passed to this function, not range.

@mouse07410
Copy link
Owner

I care about 3GPP ASN which should be supported as any other ASN

I stopped caring for 3GPP almost 20 years ago. I still do care that asn1c processes ASN.1 files correctly, which is why I'm spending my time here.

I don't need F1AP-PDU.o itself, I was just referring to your example of failing compilation and
provided correct parameters to help you with code generation.

My point is - unless I'm given a way to locally build an executable that can encode and decode F1AP PDUs to/from APER, I cannot help with your specific example. The fact that I can manually compile one .c file does not help.

asn1c generates 1094 .c files from your F1AP ASN file. make all -f converter-example.mk successfully compiles 954 of them. So, showing me that somebody can compile one file doesn't help. Perhaps you can show me how to compile ProtocolExtensionField.c, which fails on my machine?

I think my encoding failure was encoding a "SEQUENCE_OF", while your decoding issue seems to be in "SEQUENCE". I don't know the exact differences between those, but that may be a good hint.

Well, I'm sure you know that, but SEQUENCE is like a C structure, with its length being the total size of all the components, plus possible tagging, etc. SEQUENCE OF is an array, with its length being the length of one element multiplied by the number of elements, plus some "metadata"...

I would rather look at the fact that in case of problematic field the lower bound is 1 and upper bound is 65536. So range is 65536 and equal to ub. As upper bound is not less than 64K, nsnnwn shouldn't be used. But the problem is that aper_get_length() checks for range, not upper bound. I guess ct->upper_bound should be passed to this function, not range.

So, you think it's the decoder's problem? Does it encode this message OK?

@pespin
Copy link
Author

pespin commented Jul 14, 2022

@grzegorzniemirowski are you sure upper bound is 65536 and not 65535 in the field you are mentioning? Which field exactly shows this (1..65536) range? That would indeed explain it perhaps. I think we all agree that it makes sense passing ub and lb to aper_get_length. That may well fix the issue.

@mouse07410
Copy link
Owner

mouse07410 commented Jul 15, 2022

You need to manually edit generated code using compiler's hint: replace asn_PER_memb_OCTET_STRING_SIZE_3__constr_96 with memb_OCTET_STRING_SIZE_3__constraint_924. Then converter-example would be compiled correctly.

You are correct - then the example would be compiled and linked. But I rather doubt such a fix results in a correctly functioning converter.

The resulting converter indeed runs, but fails to convert XER file you posted to APER:

<F1AP-PDU>
  <initiatingMessage>
    <procedureCode>0</procedureCode>
    <criticality>
      <reject/>
    </criticality>
    <value>
      <Reset>
        <protocolIEs>
          <SEQUENCE>
            <id>78</id>
            <criticality>
              <reject/>
            </criticality>
            <value>
              <TransactionID>0</TransactionID>
            </value>
          </SEQUENCE>
          <SEQUENCE>
            <id>0</id>
            <criticality>
              <ignore/>
            </criticality>
            <value>
              <Cause>
                <radioNetwork>
                  <unspecified/>
                </radioNetwork>
              </Cause>
            </value>
          </SEQUENCE>
          <SEQUENCE>
            <id>48</id>
            <criticality>
              <reject/>
            </criticality>
            <value>
              <ResetType>
                <partOfF1-Interface>
                  <SEQUENCE>
                    <id>80</id>
                    <criticality>
                      <reject/>
                    </criticality>
                    <value>
                      <UE-associatedLogicalF1-ConnectionItem>
                        <gNB-CU-UE-F1AP-ID>32</gNB-CU-UE-F1AP-ID>
                        <gNB-DU-UE-F1AP-ID>1</gNB-DU-UE-F1AP-ID>
                      </UE-associatedLogicalF1-ConnectionItem>
                    </value>
                  </SEQUENCE>
                </partOfF1-Interface>
              </ResetType>
            </value>
          </SEQUENCE>
        </protocolIEs>
      </Reset>
    </value>
  </initiatingMessage>
</F1AP-PDU>
$ ./converter-example -dd -s 80000000 -b 1000000 -ixer -oaper f1ap_reset_1.xer > f1ap_reset_1.aper
AD: Processing f1ap_reset_1.xer
AD: Decoding 1632 bytes
AD: decode(0) consumed 181+0b (1632), code 2
AD: Clean up partially decoded F1AP-PDU
AD: ofp 1, no=181, oo=0, dbl=0
f1ap_reset_1.xer: Decode failed past byte 181: Input processing error
$ ./converter-example -dd -s 80000000 -b 1000000 -iaper f1ap_reset_3.aper 
AD: Processing f1ap_reset_3.aper
AD: Decoding 32 bytes
AD: decode(0) consumed 0+0b (32), code 2
AD: Clean up partially decoded F1AP-PDU
AD: ofp 1, no=0, oo=0, dbl=0
f1ap_reset_3.aper: Decode failed past byte 0: Input processing error
$ od -t x1 ~/src/asn-tst/F1AP/f1ap_reset_3.aper
0000000 00 00 00 1c 00 00 03 00 4e 00 02 00 00 00 00 40
0000020 01 00 00 30 00 0a 40 01 00 50 00 04 60 20 00 01
0000040
$ 

Update

I confirm that this patch (your #91 )

diff --git a/skeletons/aper_support.c b/skeletons/aper_support.c
index d6fa6b09..0ce9229b 100644
--- a/skeletons/aper_support.c
+++ b/skeletons/aper_support.c
@@ -22,7 +22,7 @@ aper_get_length(asn_per_data_t *pd, int range, int ebits, int *repeat) {
 
        *repeat = 0;
 
-       if (range <= 65536 && range >= 0)
+       if (range < 65536 && range >= 0)
                return aper_get_nsnnwn(pd, range);
 
        if (aper_get_align(pd) < 0)
@@ -163,7 +163,7 @@ aper_put_length(asn_per_outp_t *po, int range, size_t length, int *need_eom) {
        ASN_DEBUG("APER put length %zu with range %d", length, range);
 
        /* 10.9 X.691 Note 2 */
-       if (range <= 65536 && range >= 0)
+       if (range < 65536 && range >= 0)
                return aper_put_nsnnwn(po, range, length) ? -1 : (ssize_t)length;
 
        if (aper_put_align(po) < 0)

does fix the F1AP case. Changing only aper_get_length() or aper_put_length() was not sufficient - both had to be fixed.

osmocom-gerrit pushed a commit to osmocom/asn1c that referenced this issue Jul 15, 2022
This should help aper_put_length() to take proper decisions on the way
to encode the length, since the range alone is not enough.
A contraint of lb=1 ub=65536 would yield a range=65536, but according to
ITU-T X.691 11.9 it shouldn't be encoded using nsnnwn since that only
applies in case ub<65536.
As a result, it would end up encoding/decoding it using 2 bytes while it
should use only 1.

Related: mouse07410/asn1c#94
osmocom-gerrit pushed a commit to osmocom/asn1c that referenced this issue Jul 15, 2022
This should help aper_put_length() to take proper decisions on the way
to encode the length, since the range alone is not enough.
A contraint of lb=1 ub=65536 would yield a range=65536, but according to
ITU-T X.691 11.9 it shouldn't be encoded using nsnnwn since that only
applies in case ub<65536.
As a result, it would end up encoding/decoding it using 2 bytes while it
should use only 1.

Related: mouse07410/asn1c#94
@pespin
Copy link
Author

pespin commented Jul 15, 2022

@mouse07410 @grzegorzniemirowski @laf0rge please have a look at PR #100
The first bunch of commits contains clean ups and small fixes unrelated to the issue at hand, which I did while reading the code.
The last 2 commits rework aper_put_length and aper_get_length so that "lb" and "ub" are passed to it instead of "range". This allows applying nsnnwn for ub<65536.

I validated the existing unit tests and my SBc-AP generated code still work fine. Please @grzegorzniemirowski give it a try and see if it improves the situation.

@mouse07410
Copy link
Owner

mouse07410 commented Jul 15, 2022

@pespin thank you - my test shows that #100 fixes the F1AP problem. Looks like I can merge it.

Update

Merged, thanks.

mouse07410 pushed a commit that referenced this issue Jul 15, 2022
This should help aper_put_length() to take proper decisions on the way
to encode the length, since the range alone is not enough.
A contraint of lb=1 ub=65536 would yield a range=65536, but according to
ITU-T X.691 11.9 it shouldn't be encoded using nsnnwn since that only
applies in case ub<65536.
As a result, it would end up encoding/decoding it using 2 bytes while it
should use only 1.

Related: #94
mouse07410 pushed a commit that referenced this issue Jul 15, 2022
This should help aper_put_length() to take proper decisions on the way
to encode the length, since the range alone is not enough.
A contraint of lb=1 ub=65536 would yield a range=65536, but according to
ITU-T X.691 11.9 it shouldn't be encoded using nsnnwn since that only
applies in case ub<65536.
As a result, it would end up encoding/decoding it using 2 bytes while it
should use only 1.

Related: #94
acetcom added a commit to open5gs/open5gs that referenced this issue Jul 16, 2022
@acetcom
Copy link

acetcom commented Jul 16, 2022

@mouse07410, @grzegorzniemirowski, @laf0rge and @pespin

Thank you so much for your efforts.

I confirmed that commit 24247e2 works well in Open5GS.

Please let me know if you have any further questions in the Open5GS.

Thanks a lot!
Sukchan

@ruffyontheweb
Copy link

Hi all. I'm working on testing open air interface and would just like to say thank you all for this these sets of timely commits.

@mouse07410
Copy link
Owner

Well, one problem seems solved. Thanks to everybody who helped, especially @pespin , @grzegorzniemirowski , and @laf0rge . I'm closing this issue - please feel free to re-open if needed.

In the meanwhile, the next two hard problems:

  • implementing support for INTEGER constraints that do not fit into uint64 size;
  • implementing support for OCTET STRING xxx CONTAINING ().

Any takers? ;-)

osmocom-gerrit pushed a commit to osmocom/osmo-cbc that referenced this issue Jul 18, 2022
Update skeleton files using newest asn1c with APER support [1],
commit 24247e2813a7510ebabe6a9b6b6b29fffa0eb27b.

This contains some APER decoding and encoding fixes for length
determinants. See [2] for more information.

[1] https://github.com/mouse07410/asn1c/tree/vlm_master/
[2] mouse07410/asn1c#94

Change-Id: I581fc53b124a443e150508811df4cca4593038c4
@laf0rge
Copy link

laf0rge commented Oct 11, 2022 via email

NLag added a commit to securitylab-repository/open5gs_ciot that referenced this issue Oct 19, 2022
* Initialize pgw_s5u_teid (open5gs#1559)

* [SMF] Gy: Send Multiple-Services-Indicator AVP only during Initial CCR (open5gs#1616)

Gy (3GPP TS 32.299 ) refers to AVP in DCCA (RFC4006).

RFC4006 5.1.2:
"[...] by including the Multiple-Services-Indicator AVP in the first
interrogation."

Nokia's infocenter documentation also states it's sent during Initial CCR
only: "(CCR-I only)".

* [SBI] Fix memory leak for nghttp2 session (open5gs#1618)

Delete nghttp2 session to prevent memory leaks.
The issue was detected using valgrind.

* [CORE] fsm: Add asserts to validate ogs_fsm_t is not null (open5gs#1619)

* Change Default MCC/MNC 901/70 -> 999/70 (open5gs#1331)

* [SMF] track and fix scenario where gtp node mempool becomes full (open5gs#1622)

* [SMF] Avoid abort() if gtp_node mempool becomes full

Related: open5gs#1621

* [SMF] metrics: Add new ctr tracking gtp_node allocation failures

This metrics is useful to track whether at some point the mempool went
full, so that config needs to be updated to increase the mempool size.

* [SGWC,SMF] Add specific config opt max.gtp_peer to set gtp_node mempool size (open5gs#1623)

This is needed specially for SMFs handling a pool of SGWs.

* [UPF] Avoid crash if no default subnet configured (open5gs#1624)

In that case, ogs_pfcp_ue_ip_alloc() will fail with  the error message
"CHECK CONFIGURATION: Cannot find subnet [...]" and the assert will make
upf crash.
That's not desirable, let's keep it running and simply reject the
request. The error log is big enoguh to find out.

* [SMF] pfcp-sm: Fix ogs_fsm_dispatch() on NULL sess (open5gs#1628)

It was spotted that if DeleteSessionReq sent by SMF is answered by UPF
with cause="Session context not found", then it contains SEID=0 (this is
correct as per specs). Hence, since SEID=0 session is not looked up, so
sess=NULL.

A follow up commit improves the situation by looking up the SEID in the
originating request message in that case.

* [SMF] Set v4/6 flag in F-TEID IE request type (open5gs#1625)

Signed-off-by: Networkmama <[email protected]>

* Set v4/v6 flags in local F-TEID (open5gs#1625)

* [PFCP] Added DNN/APN in FAR (open5gs#1629, open5gs#1630)

* [PFCP] Added Network Instance to CP-UP FAR (open5gs#1630)

* [SMF] Gn: QoS Profile and PCO IE improvements (open5gs#1631)

* [SMF] pfcp: Retrieve sess when response with SEID=0 is received (open5gs#1620)

3GPP TS 29.244 7.2.2.4.2 documents that the peer will set SEID=0 in the
response when we request something for a session not existing at the peer.
If that's the case, we still want to locate the local session which
originated the request, so let's store the local SEID in the xact when
submitting the message, so that we can retrieve the related SEID and
find the session if we receive SEID=0.

* [SGWC] pfcp: Retrieve sess when SEID=0 (open5gs#1620)

* [GTP] context when TEID=0 (open5gs#1620, open5gs#1606, open5gs#1594)

* [SCTP] Add protection code jumbo frame (open5gs#1632)

* [GTP] Avoid abort if ogs_gtp_node_new() returns NULL (open5gs#1633)

* [GTP] Avoid abort if ogs_gtp_node_new() returns NULL

* [SGWC] Avoid abort if ogs_gtp_node_add_by_addr() returns NULL

* [GTP] avoid abort for ogs_gtp_node_new() (open5gs#1633)

* [UPF] N4: Remove unnecessary assert (open5gs#1634)

* Changes MAX TLV MORE to 16

OGS_MAX_NUM_OF_PDR is 16, but OGS_TLV_MAX_MORE is 8.
To match the size of two macros, increased OGS_TLV_MAX_MORE to 16.

* Revert "[GTP] context when TEID=0 (open5gs#1620, open5gs#1606, open5gs#1594)"

This reverts commit 0d61f7a.

* Revert "[SGWC] pfcp: Retrieve sess when SEID=0 (open5gs#1620)"

This reverts commit 9700563.

* [PFCP] Refine error code path without assertion

Refer to open5gs#1635, open5gs#1620

* [GTP] Refine error code path without assertion

Refer to open5gs#1635, open5gs#1620, open5gs#1606, open5gs#1594

* Fixed a crash when slice/session overflow (open5gs#1637)

* [MISC] Add support for static code analysis

Static code analysis can be executed with following commands:
  meson build
  ninja -C build analyze-cppcheck
  ninja -C build analyze-clang-tidy

These commands are available only if additional tools are installed:
  - cppcheck
  - clang-tidy
  - clang-tools is optional if you want to paralelize the clang-tidy

In case of cppcheck analysis, a file called build/cppchecklog.log is
created with the analysis results.

In case of clang-tidy analysis, some checks are disabled. See file
.clang-tidy, and reenable them if you see fit.
Also it does not scan all the files in the project, since some of them
are imported from other sources. It does not scan any sources under:
  - subprojects/
  - lib/asn1c/
  - lib/ipfw/

* [ALL] Fix differences in function parameter names between definition and declaration

* [CORE] Fix detection of a failed memory allocation

* [CORE] Added memory check (open5gs#1638)

* [PFCP] Fixed a endianness Apply Action (open5gs#1640)

* [SMF] copy UE ip address to uplink PDR rules.

This helps UPF to add ACL based on src ip

Signed-off-by: Networkmama <[email protected]>

* [PFCP] Added UE IP address in the EPC (open5gs#1642)

* d/open5gs-upf.postinst: don't restart service in chroot

Don't attempt to restart systemd-networkd if systemd is not running
(e.g. installing open5gs inside a chroot).

Fix for:
  System has not been booted with systemd as init system (PID 1). Can't operate.
  Failed to connect to bus: Host is down
  dpkg: error processing package open5gs-upf:amd64 (--configure):
   installed open5gs-upf:amd64 package post-installation script subprocess returned error exit status 1

* [SBI] fixed wrong request-nf-type (open5gs#1650)

* [AMF] Do not send Deregistration Event to UDM when UE deregisters

According to TS 23.502, 4.2.2.2.2, AMF sends Registration event to UDM
in the following cases:
- If the AMF has changed since the last Registration procedure, or
- if the UE provides a SUPI which doesn't refer to a valid context in
the AMF,
- or if the UE registers to the same AMF it has already registered
to a non- 3GPP access (i.e. the UE is registered over a non-3GPP access
and initiates this Registration procedure to add a 3GPP access).

In case that UE re-registers to the network with a GUTI, it bypasses
authentication check to the AUSF. In this case, AMF does not send
Registration event to UDM.
Consequently, when UE deregisters again, AMF would send a Deregistration
Event to a UDM, which does not have a context for it.

3GPP standard does not say when AMF sends Deregistration Event to UDM,
only that it is optional.

These (De-)Registration events are for (de-)registering AMF to the UDM
for serving the UE. And not for (de-)registering UE itself for purpose
of tracking when UE is registered on the network.

This partially reverts commit 7be7029

* [NAS] modify library to include both directions of deregistration requests

Definitions in NAS library now include both directions of deregistration
requests/accepts - from UE and from network.

* [SBI] Add support for DeregistrationData in SBI messages

* [AMF] Handle namf-callback DeregNotify message from UDM

UDM may send a Deregistration Notification to AMF, to deregister
specific UE from the network - Network-Initiated Deregistration.
Deregistration procedure includes sending Deregistration Request to UE,
starting a timer T3522, releasing PDU sessions from SMF, releasing PCF
policies from PCF, and waiting for Deregistration Accept from UE.

Not yet implemented is:
- to prevent deregistration of UE in case it has any emergency sessions,
- page UE when UE is in IDLE mode.

* [SBI] incrased session pool of server (open5gs#1652)

* [CORE] Increased memory pool for poll (open5gs#1652)

* [SCP] Support of Indirect Communication

* [asn1c] rework aper from mouse07410/asn1c#94

Merge @pespin the following work
- mouse07410/asn1c#93
- mouse07410/asn1c#100

* [AMF] Fixed 5GMM cause in Reject message (open5gs#1660)

When a UE that requests slices tries to connect and there are no slices configured, the reject message is:

5GMM cause = 0x7 (5GS Services not allowed)

however it should be:

5GMM cause = 0x3e (No network slices available)

All 5GMM cause value in reject message is reviewed in this commit

* Fixed ASSERT when context has already been removed

* fixed the memory leak in test program

* Fixed the crash in UERANSIM 500 test (open5gs#1652)

* Oops! Redundant code is removed (open5gs#1652)

* Add missing pkbuf_free() (open5gs#1652)

* Refactor for the UERANSIM 500 test (open5gs#1652)

* Add more protection code for (open5gs#1652)

* [tests] Fix running unit tests inside docker environment

The issue was introduced with the commit, which builds Open5GS from
local sources instead of downloading them each time from Github.

Fixes: d2cbcf7 ("[build] Use local sources to build applications (open5gs#1583)")

* [SBI] Add function to request NF Instance from NRF by providing it's Instance Id

* [SMF] Send PDU Session Establish Accept to serving AMF

In case there are multiple AMF registered to NRF, SMF would pick only
the first AMF from the list.
In the case of sending PDU Session Establishment Accept from SMF to
AMF, this would mean a high chance of failure since the AMF might
be different than the original requester, and would not know about a
particular UE.

Modify SMF to use ServingNfId field from original request
SmContextCreateData from AMF to determine to which AMF should it send
PDU Session Establishment Accept message.

* Set default Network-Access-Mode to 0

For HSS's which do not include the NAM, the MME should not treat this as a fatal error.  MME should just assume PACKET_AND_CIRCUIT (0), as was decided in a previous PR.

* [MME] Handle Charging Characteristics

Found no support for HSS provided charging characteristics.  Following TS32.251 A.4:
- Use PDN level CC, if one wasn't provided then use subscription level CC
- Don't send CC in S11 if it wasn't included

* Moving handling of assigning sub level cc into the pdn to s11.

* Support Discovery Optional Parameter (open5gs#1671)

To support target-nf-instance-id in the discovery,
Discovery optional parameter is implemeted

* Oops! Warning removed!

* Refine code of discovery option param (open5gs#1671)

* Release v2.4.9

* [MME] Introduce support for S6a Cancel Location Request

- Added diameter dictionary definitions for Cancel Location
- Cancel Location will completely remove UE from MME, allow for a fresh IMSI attach to occur on next attempt.
- T3422 is used for detach request.
- Added new handling for s6a events in mme-sm, as not all s6a messages are at attach now.  Maybe there's something in a state machine I should've been using here instead of a new flag?

- Testing was completed with UE in idle and connected.  With CLR flags indicating re-attach required and without.  Also sending CLR after UE detach.  And then sending again when mme_ue is empty.

* Refact paging module (open5gs#1676)

* [MME] Changed S1AP_Cause in S1AP Release (open5gs#1676)

S1AP_CauseNas_detach -> S1AP_CauseNas_normal_release

* [SBI] Fixed nf_instance memory leak

- Rollback commit ed3444e
- Do not modify reference count when REGISTER/DEREGISTER notified from NRF

* [PFCP] Revert Changes 5e18b2b

* [MME+HSS] AVP Occurring Too Many Times

Do not Set Origin-Hosts with fd_msg_rescode_set before potential use of ogs_diam_message_experimental_rescode_set.  This results in multiple Origin-Host/Realm AVPs.

* [Diameter] Fixed Coding convention (open5gs#1680)

* [NRF] Fixed the nfInstanceUri (open5gs#1683)

* [PFCP] Revert Changes 5e18b2b and d21e9aa

To protect malicious or buggy, we need to check that session context is NULL.

* Changed configuration name from gnb to peer

And restored gtp_peer configuration

* [GTP] gtp_peer override the pool size of GTP node

* [MME] Changed CauseNas_detach in DETACH (open5gs#1676)

* [SBI] Increased the max stream number

* [SBI] CLIENT max concurrent streams to 16384

* Move src/../nf-sm.[ch] to lib/sbi/nf-sm.[ch]

* [PFCP] Fixed security protection

Check the length to prevent buffer overflow attacks.

* Lower Linux version cannot change HTTP2 max stream

CURLMOPT_MAX_CONCURRENT_STREAMS can be supported as of curl 7.67.0

* [SMF]: Update stored PCO IE requested in GTPv2 over S5c in SMF context

As per 3GPP TS 29.274 version 10.5.0, section 7.2.9 and 7.2.10,
Only if PCO IE is included in Delete Session Request then it
must be present in Delete Session Response.

In order to reflect on whether the request contained PCO IE or not
the SMF context containing the GTP request needs to be updated
i.e. update if present else clear the contents

* [SMF]: Update stored PCO IE requested over Gn in SMF context

As per 3GPP TS 29.060 version 15.3.0, section 7.3.3, 7.3.4, 7.3.5 and 7.3.6

Only if PCO IE is included in Update/Delete PDP Context Request then it
must be present in Update/Delete PDP Context Response.

In order to reflect on whether the request contained PCO IE or not
the SMF context containing the GTP request needs to be updated
i.e. update if present else clear the contents

* Update docs @nickvsnetworking and @@s5uishida

* Update docs @s5uishida

* Removed duplicated document link

* Fixed Defects reported by Coverity Scan

* [DOCS] Updated if subscribers changed [open5gs#1694]

* [PFCP] security vulnerability continued in d99491a

* [MME] Cancel Location Handling (open5gs#1698)

* CLR while idle is broken after open5gs@7031856

Cancel Location Request arriving while UE is idle will not proceed to paging due to this check for S1 connection.  Using new flag "isAnswer" to bypass this check to allow paging to occur when we are not doing a AIA/ULA related procedure.

* No Context Setup is required when sending the detach request.  If the paging was due to wanting to send a Detach Request to the UE, then we fast track to sending the detach request.

* emm-sm.c:
In the case of MME initiated detach while UE is idle, there is no initial conext setup.  We go right from the service request after paging into sending the detach request.  TS23.401

mme-path.c:
Using nas_eps.type in the case of MME Initiated Detach while UE is idle does not work.  nas_eps.type would represent the service request.

mme-s11-handler.c:
After S11 action, no action should be taken.  We want to wait for the detach accept from the UE before proceeding with the S1 release (detach).

* InitialContextSetup should occur for detach.

* [MME] Follow-up Cancel Location Handling (open5gs#1698)

* [MME] Fixed GTP transaction crash (open5gs#1696)

* [SGsAP] Changed message if Paging failed (open5gs#1701)

The problem occurred in the following scenario:

1. VLR sent PAGING-REQUEST to the MME
2. MME sent S1-Paging to the UE
3. Paging failed
4. MME responded SERVICE-REQUEST to the VLR
5. VLR sent DOWNLINK-UNITDATA to the MME
6. Even though there is no S1 Context,
   MME try to sent DownlinkNASTransport message to the UE.
7. So, the problem occurred.

I've changed the number 4 PAGING-REJECT instead of SERVICE-REQUEST.

* [MME] Detach removed MME-UE context (open5gs#1698)

* [MME] UE-initiated detach removes S1 only (open5gs#1698)

* [SMF] Fix abort on app exit when no Diameter configuration

In case that SMF was configured to run without Diameter, it would crash
on application exit due to uninitialized variables/pointers.

ERROR  pid:unnamed in [email protected]:324: ERROR: Invalid parameter '(handler && ( ((*handler) != ((void *)0)) && ( ((struct session_handler *)(*handler))->eyec == 0x53554AD1) ))', 22
[smf] FATAL: smf_gx_final: Assertion `ret == 0' failed. (../src/smf/gx-path.c:1353)

* [MME] Dictionary Updates and IDR Support (open5gs#1714)

* Add Diameter Dictionary Elements

* Initial IDR Framework

* Resolve Compile Issues

* Moving Closer

* Compile error

* Somewhat Working stuffing Code

* Add Timestamp Changes

* Cleanup some of this code.  mme_s6a_handle_idr in s6a-handler.c removed for now, since it will only come in handy when IDR flag is set to request current location, which would involve breaking out into paging.  I think there's a few other things we can do just within fd-path first.

* further removal of mme_s6a_handle_idr

* Follow up on open5gs#1714

* Changed sprintf to ogs_snprintf

* Limited to 80 column

* [NRF] Fixed library load error

* [metrics] Fix double-free on application exit (open5gs#1717)

* [SBI] Support service-names in discovery option

* [SBI] Added config for service-names discovery

* diameter: Gx: add AVP 3GPP-Charging-Characteristics

The 3GPP-Charging-Characteristics is an operator specific AVP
(optional). The 3GPP-Charging-Characteristics can be filled by the HSS
and pass through to the Gx interface.

See ETSI 29.212 5.4.0.1 for further details.

* [SMF] send 3GPP-Charging-Characteristics on Gx if received on S5/8c

The 3GPP-Charging-Characteristics is an operator specific AVP
(optional). The 3GPP-Charging-Characteristics can be filled by the HSS
and forwarded by the MME towards the SMF.

* Follow up on open5gs#1715

* Changed <TAB> to <SPACE>*4

* Changed snprintf to ogs_snprintf

* [Conf] Changed MTU size from 1500 to 1400

* [SMF] fixup send 3GPP-Charging-Characteristics on Gx if received on S5/8c

- Gy instead of Gx AVP was used.
- Use correct avp position and avp variables.

Fixes: 657eef9 ("[SMF] send 3GPP-Charging-Characteristics on Gx if received on S5/8c")

* Added Service-based NF discovery

== Known limitation ==
Placing npcf-smpolicycontrol and pcf-policyauthorization
in different NFs is not supported. Both npcf-smpolicycontrol
and pcf-policyauthorization should be placed in the same NF.

* [ALL] Removing trailing whitespace and tab

* [MME] Fixed buffer overflow (open5gs#1728)

* Added simple test program

./tests/registration/registration simple-test
./tests/vonr/vonr simple-test
./tests/attach/attach simple-test
./tests/volte/volte simple-test

* [SMF] Handle upCnxState=ACTIVATING by replying with 200 instead of 204

According to TS 29.502 5.2.2.3.2.2., we should reply with a 200 response
including the upCnxState attribute.

* Follow-up on open5gs#1729

* [SBI] Send NF discovery query with service-names delimited with comma

OpenAPI specification for sending NF discovery query with
"service-names" parameter is defined as folowing:

- name: service-names
  ...
  style: form
  explode: false

According to OpenAPI specification, this means array items
should be delimited with a comma character (example: /users?id=3,4,5).

* [PCF] Check NF service configuration

* npcf-smpolicycontrol - enabled or disabled
* npcf-policyauthorization - enabled or disabled

Only one of npcf-smpolicycontrol and npcf-policyauthorization cannot be enabled. (../src/pcf/sbi-path.c:151)

They can be enabled or disabled together.

* [CORE] Check if timer is double free in SBI module

* removing extra lines

I did not find the purpose of their use

* [AMF] Accept Deregistration Notification from UDM only for registered UE (open5gs#1737)

Add additional check when receiving Deregistration Notification from
UDM. UE should already be in registered state before accepting the
request and deregistering the UE.

Also add additional check that PCF association policy exists before
sending a delete request to PCF.

* When using longer APN name, it is obscured due to short field.

* Also format for pcc_rule.  UE and SMF look okay as medium_data as first section.

* Minor typo fix

* Support service-based NF subscription

* [AMF] Handle APN/DNN names as case-insensitive

In case that APN name sent from UE does not case-match with the one
configured in the database, AMF would reject the registration with the
message:

[gmm] WARNING: [imsi-xxx] DNN Not Supported OR Not Subscribed in the
Slice (../src/amf/gmm-handler.c:1051)

* Follow-up on open5gs#1747

* Release v2.4.10

* Added Release Notes for v2.4.10

* Introduce Cancel Location and Insert Subscriber Data features to HSS. (open5gs#1744)

* Introduce Cancel Location and Insert Subscriber Data features to HSS.
* HSS database will keep track of last known MME and Update Time
* Purged UE flag is established in HSS for future PUR handling
* HSS Thread will connect to database and watch change stream
  mongoDB must be configured with a Replica Set to use this
* HSS will send IDR if subscription data changes
* HSS will send CLR to old MME if MME host or realm changes
* Function created to allow ULA and IDR to generate Subscription-Data AVP
* MME Hostname and Realm shown in WebUI

* Resolve freeDiameter errors

During a ULR, if database does not contain a last known MME, a CLR is being sent to a Null destination.  This will ensure that a destination is available in the database before sending the CLR.

* Removed change streams.  Added PUR handling.

* newline needed at end of file.

* Removed temp variable.

* * Change WebUI to 2x2 display
* Including UE Purged indicator
* Using pointers in ogs_subscription_data_t
* better memory mangement with pointers
* Tweak to Destination used by hss_s6a_send_idr to use last known MME

* Check for null mme_host and mme_realms

Do this before trying to compare the strings.

* Follow-up on open5gs#1744

* [SMF] Wait for both N1&N2 release signals before releasing session

When UE would send a request to release PDU session, AMF would
eventually send "PDU Session Resource Release Command" downlink to both
UE (N1) and gNB (N2). Each UE and gNB would then reply with "PDU Session
Resource Release Response" indicating they released their own resources.

Usually the first one to respond would be gNB. SMF made an assumption
that this would always be the case. And it would wait for signal that UE
resources were freed, before releasing session resources. But
occasionally the situation is that UE responds first, and SMF releases
resources prematurely.

This situation does not normally occur. But under high stress (100's of
UE PDU releases at the same time) this happens occasionally.
According to the standard, this situation is perfectly normal.

3GPP TS 23.502 Rel. 16
4.3.4.2 UE or network requested PDU Session Release for Non-Roaming and
Roaming with Local Breakout
...
Steps 8-10 may happen before steps 6-7.
...

* Added commercial 5G

* Add tested Ericsson gNodeBs and eNodeBs

* [DOC] Fixed alphabet order

* Sponsors logo updated to be dark-mode friendly

* Updated favicon.ico in Document

* [GTP/PFCP] TLV length more acceptable (open5gs#1780)

Acceptable even if the TLV length is smaller than expected

* [AMF] Add amfInfoList to NFProfile

The actual configured GUAMIs and TAIs are used to form NF profile.
Comparing to SMF the "info" section is not introduced into amf.yaml!
Each amf_id (region, set) produces a separate Info in the InfoList.
Guami list consists of all GUAMIs of particular Info.
taiList consists of all TAIs for all PLMNs of particular Info.

Examle:

amf.yaml:
    guami:
      - plmn_id:
          mcc: 999
          mnc: 70
        amf_id:
          region: 2
          set: 2
          pointer: 4
      - plmn_id:
          mcc: 001
          mnc: 01
        amf_id:
          region: 2
          set: 1
      - plmn_id:
          mcc: 001
          mnc: 02
        amf_id:
          region: 2
          set: 2
    tai:
      - plmn_id:
          mcc: 001
          mnc: 01
        tac: [1, 2, 3]
    tai:
      - plmn_id:
          mcc: 002
          mnc: 02
        tac: 4
      - plmn_id:
          mcc: 001
          mnc: 02
        tac: 10
    tai:
      - plmn_id:
          mcc: 004
          mnc: 04
        tac: [6, 7]
      - plmn_id:
          mcc: 005
          mnc: 05
        tac: 8
      - plmn_id:
          mcc: 999
          mnc: 70
        tac: [9, 10]

"amfInfoList":  {
        "1":    {
                "amfSetId":     "002",
                "amfRegionId":  "02",
                "guamiList":    [{
                                "plmnId":       {
                                        "mcc":  "999",
                                        "mnc":  "70"
                                },
                                "amfId":        "020084"
                        }, {
                                "plmnId":       {
                                        "mcc":  "001",
                                        "mnc":  "02"
                                },
                                "amfId":        "020080"
                        }],
                "taiList":      [{
                                "plmnId":       {
                                        "mcc":  "001",
                                        "mnc":  "02"
                                },
                                "tac":  "00000a"
                        }, {
                                "plmnId":       {
                                        "mcc":  "999",
                                        "mnc":  "70"
                                },
                                "tac":  "000009"
                        }, {
                                "plmnId":       {
                                        "mcc":  "999",
                                        "mnc":  "70"
                                },
                                "tac":  "00000a"
                        }]
        },
        "2":    {
                "amfSetId":     "001",
                "amfRegionId":  "02",
                "guamiList":    [{
                                "plmnId":       {
                                        "mcc":  "001",
                                        "mnc":  "01"
                                },
                                "amfId":        "020040"
                        }],
                "taiList":      [{
                                "plmnId":       {
                                        "mcc":  "001",
                                        "mnc":  "01"
                                },
                                "tac":  "000001"
                        }, {
                                "plmnId":       {
                                        "mcc":  "001",
                                        "mnc":  "01"
                                },
                                "tac":  "000002"
                        }, {
                                "plmnId":       {
                                        "mcc":  "001",
                                        "mnc":  "01"
                                },
                                "tac":  "000003"
                        }]
        }
}

* Follow-up on open5gs#1757

* [HSS] Enable Change Streams (open5gs#1758)

* [HSS] Enable Change Streams
* Enable Events and Timers in HSS
* Integrate change streams in dbi
* mongodb should be configured with replica sets enabled to use feature
* Change streams are optional in HSS
* Timer will poll change stream for changes in the database
* As changes are detected, event is created to perform the correct
  action

* Changes made as suggested

* Follow-up on open5gs#1758

* Update document

* [TLV] Added more debug information (open5gs#1767)

* Fixed HTTP2 crashes for random JSON data (open5gs#1769)

* [core] fix timer overflow on 32bit systems (open5gs#16)

must cast ts.tv_sec to 64bits before we multiply it to prevent 32bit math and overflow

* Follow-up on open5gs#1770

* [SGWC] Fixed a crash (open5gs#1765)

Session context could be deleted before a response message is not
received from SMF

* add addr/port to pfcp assoc/de-assoc logs (open5gs#40)

pfcp association log adds addr/port

* [GTP] Changes the print message (open5gs#1772)

* [config,metrics] Move metrics configuration section under respective NF section

Without this change, using metrics with core setup configurations
(configs/vonr.yaml for example) would not be possible. Having one
metrics section for whole config file causes every NF to start metrics
server on same port causing an abort.

* Follow-on up open5gs#1754

* [DOC] iptable setting for security (open5gs#1768)

* [MME] Added protection code if no PDN-Type (open5gs#1756)

* Changes new GA4 in Google Analytics

* Commercial Term by NeoPlane - https://neoplane.io/

* Fix UL and DL URR Usage Report

* Follow-on up open5gs#1793

* [AMF] Fix for switching state when sending Deregistration Request fails

Provide pointer to state machine, instead of pointer to timer structure.
Bug was noticed when switching compiler optimization to -O2.

* [5GC] Fixed session deletion in a BSF (open5gs#1725)

* [UPF] test code for unspecified address (open5gs#1776)

* [Security] Fixed a crash for port scanning (open5gs#1767)

* Release v2.4.11

* Added Release Notes for v2.4.11

* [MME] Support for Insert Subscriber Data (open5gs#1794)

* [MME] Support for Insert Subscriber Data

* Supported AVPs in IDR will overwrite existing subscription information
* Provide error on partial APN updates
* IDR and ULA use same function to process AVPs
* Move subdatamask values into s6a, so both HSS and MME can use them
* Updates are not actioned at this time.  A Re-attach is required for
  most changes to take effect

* Memory issue on IDR exceptions

* Remove of handling MSIDSN change until DSR is used

* Follow-on up open5gs#1794

* [SBI] Client Request timeout

TS29.500
Ch 6.11 Detection and handling of late arriving requests

In Open5GS, this part was hard-corded.

HTTP2 Client sends a request and waits for 10 seconds.
If no response is received from the HTTP2 Server,
HTTP2 Client performs the exception handling.

In this commit, HTTP2 client sends Header with setting Max-Rsp-Time to 10 seconds.
However, HTTP2 server has not yet been implemented to process this value.
The server is still processing using hard-corded values (10 seconds).

* [MME] Cancel Location while Idle (open5gs#1797)

* Cancel Location while Idle Fix

* Forgot about SGSAP on MME Change.

Added "action" to sgsap_send_detach..

* Make handle_clr uniform with other handlers

* Added Robustness for Any Detach Type

* Memory wasn't freed upon CLR for unknown IMSIs

* Moving MME Detach to new PR

* Follow-up on open5gs#1797

* ogs_info swaps CP and UP SEIDs

* Follow-up on open5gs#1797

* fix dropped_dl_traffic_threshold ie.

* [AMF,UDM] Add support to subscribe to SDM changes

AMF subscribes to UDM for each registered UE.

At the moment, UDM does not send callback to AMF when any of the UE's
properties in the database changes.
At the moment, AMF does properly parse the ModificationNotification, but
does not do anything useful.

* [SMF] Update PFCP report error situation (open5gs#1819)

* Revert the previous commit on open5gs#1797

* Updated SBI module

- Introduced NF_INSTANCE_ID/NF_INSTANCE_TYPE
- Skip SCP in configuration validation

* [AMF] Increase size of TMSI pool

Each UE context has 'current' and 'next' TMSI values. AMF first
allocates the 'next' value, before confirming it and releasing the
'previous'. This means that we potentially need pool size of 2x the
amount of maximum configured UE.

Without this change, AMF would crash in case that there are 'x'
configured maximum amount of UE, and there are already 'x' registered
UE.

[gmm] INFO: Registration request (../src/amf/gmm-sm.c:135)
[gmm] INFO: [suci-0-001-01-1234-0-1-1000000000]    SUCI (../src/amf/gmm-handler.c:149)
[gmm] DEBUG:     OLD TSC[UE:0,AMF:0] KSI[UE:7,AMF:0] (../src/amf/gmm-handler.c:179)
[gmm] DEBUG:     NEW TSC[UE:0,AMF:0] KSI[UE:7,AMF:0] (../src/amf/gmm-handler.c:186)
[amf] FATAL: amf_m_tmsi_alloc: Assertion `m_tmsi' failed. (../src/amf/context.c:2160)
[core] FATAL: backtrace() returned 13 addresses (../lib/core/ogs-abort.c:37)

* [AMF] Reject registration requests when pool for UE contexts is empty

AMF does not crash anymore when a new UE registration request arrives,
and there is no available space left in UE context pool. Now it just
rejects the request with an error.

* Follow-up on open5gs#1828

* Follow-up on open5gs#1827

* [DBI] Disable Change Streams with mongo Version

Support for change stream is only available in mongoc >=1.9.0
- Disabled related functions in dbi.
Support for bson to json used in debug statement only in libbson >=1.7.0
- Simple debug message in lower versions

* Follow-up on open5gs#1827

* Update README.md

Signed-off-by: Networkmama <[email protected]>
Co-authored-by: Sukchan Lee <[email protected]>
Co-authored-by: Pau Espin Pedrol <[email protected]>
Co-authored-by: Bostjan Meglic <[email protected]>
Co-authored-by: Networkmama <[email protected]>
Co-authored-by: Bostjan Meglic <[email protected]>
Co-authored-by: Oliver Smith <[email protected]>
Co-authored-by: jmasterfunk84 <[email protected]>
Co-authored-by: herlesupreeth <[email protected]>
Co-authored-by: Alexander Couzens <[email protected]>
Co-authored-by: mitmitmitm <[email protected]>
Co-authored-by: EugeneBogush <[email protected]>
Co-authored-by: neg2led <[email protected]>
Co-authored-by: Gaber Stare <[email protected]>
Co-authored-by: Spencer Sevilla <[email protected]>
Co-authored-by: Dibas Das <[email protected]>
Co-authored-by: safaorhann <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants