Add EXTERNAL support. #140

[velichkov]

• Regenerate asn1p_l.c and asn1p_y.[ch] with (f)lex and bison/yacc on every compile
• Remove EXTERNAL from asn1p_l.l and asn1p_y.y
• Add EXTERNAL ASN.1 definition from ITU-T X.208 in
  skeletons/standard-modules/ASN1C-UsefulInformationObjectClasses.asn1
• Add a test case:
  tests/135-EXTERNAL-OK.asn and asn1c/tests/check-src/check-135.c
• Regenerate all tests/*.asn1.-Pfwide-types files

[mouse07410]

Thank you - but I'm afraid more work is needed:

. . . . .
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-am
  CC       asn1p_y.lo
  CC       asn1parser.lo
  LEX      asn1p_l.c
  CC       asn1p_module.lo
%option yylineno entails a performance penalty ONLY on rules that can match newline characters
REJECT entails a large performance penalty
  CC       asn1p_oid.lo
asn1p_y.y:359:13: error: use of undeclared identifier 'param'
                *(void **)param = (yyvsp[0].a_grammar);
                          ^
1 error generated.
make[3]: *** [asn1p_y.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
+ exit 0

[mouse07410]

@velichkov Also, could you please fix tests/32-sequence-of-OK.asn1.-P? And maybe rebase for the current master?

Thanks!

[velichkov]

Hi @mouse07410

asn1p_y.y:359:13: error: use of undeclared identifier 'param'
*(void **)param = (yyvsp[0].a_grammar);

This looks like version incompatibility problem.

Which version of bison and flex are installed on your platform? I have tested this on two platforms one with flex-2.5.37-3.el7.x86_64 bison-2.7-4.el7.x86_64 and the other with bison-1:2.5.dfsg-2.1 flex-2.5.35-10.1 and also it compiles in the Travis CI container https://travis-ci.org/velichkov/asn1c/builds/213463629

Could you check if configure finds bison and flex - in config.log there will be messages like

configure:4264: checking for bison
configure:4280: found /usr/bin/bison
configure:4291: result: bison -y
configure:4308: checking for flex
configure:4324: found /usr/bin/flex
configure:4335: result: flex
configure:4373: flex conftest.l
configure:4377: $? = 0
configure:4379: checking lex output file root
configure:4393: result: lex.yy
configure:4398: checking lex library

and the output of $ touch libasn1parser/*.y libasn1parser/*.l && make V=1 should be something like

make[2]: Entering directory `/builddir/asn1c/libasn1parser'
/bin/sh ../config/ylwrap asn1p_y.y y.tab.c asn1p_y.c y.tab.h `echo asn1p_y.c | sed -e s/cc$/hh/ -e s/cpp$/hpp/ -e s/cxx$/hxx/ -e s/c++$/h++/ -e s/c$/h/` y.output asn1p_y.output -- bison -y -p asn1p_ -d 
.....
/bin/sh ../config/ylwrap asn1p_l.l lex.yy.c asn1p_l.c -- flex -s -p -Cem -Pasn1p_ -olex.yy.c 
%option yylineno entails a performance penalty ONLY on rules that can match newline characters

@velichkov Also, could you please fix tests/32-sequence-of-OK.asn1.-P? And maybe rebase for the current master?

Yes. I will rebase against your master, regenerate the tests/.asn1. files and push again.

I saw that you have already pushed manually some of these changes in your master in 11c7266 and I don't understand why you did it, is there any specific reason?

[mouse07410]

Yes. I will rebase against your master, regenerate the tests/.asn1. files and push again

Thank you! Will check in a few hours.

I saw that you have already pushed manually some of these changes in your master in 11c7266 and I don't understand why you did it, is there any specific reason?

:-(
I don't recall... (Now that I've learned that answer, can I play a politician? :-)

[mouse07410]

Same problem:

. . . . .
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-am
  CC       asn1parser.lo
  LEX      asn1p_l.c
  CC       asn1p_y.lo
  CC       asn1p_module.lo
%option yylineno entails a performance penalty ONLY on rules that can match newline characters
REJECT entails a large performance penalty
  CC       asn1p_oid.lo
asn1p_y.y:359:13: error: use of undeclared identifier 'param'
                *(void **)param = (yyvsp[0].a_grammar);
                          ^
1 error generated.
make[3]: *** [asn1p_y.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Could you check if configure finds bison and flex - in config.log

$ fgr bison config.log
186:configure:4373: checking for bison
187:configure:4389: found /opt/local/bin/bison
188:configure:4400: result: bison -y
2735:ac_cv_prog_YACC='bison -y'
2986:YACC='bison -y'
$ fgr flex config.log
189:configure:4417: checking for flex
190:configure:4433: found /opt/local/bin/flex
191:configure:4444: result: flex
192:configure:4482: flex conftest.l
221:| /* A lexical scanner generated by flex */
241:| /* flex integer type definitions */
258:| typedef int8_t flex_int8_t;
259:| typedef uint8_t flex_uint8_t;
260:| typedef int16_t flex_int16_t;
261:| typedef uint16_t flex_uint16_t;
262:| typedef int32_t flex_int32_t;
263:| typedef uint32_t flex_uint32_t;
265:| typedef signed char flex_int8_t;
266:| typedef short int flex_int16_t;
267:| typedef int flex_int32_t;
268:| typedef unsigned char flex_uint8_t; 
269:| typedef unsigned short int flex_uint16_t;
270:| typedef unsigned int flex_uint32_t;
600:| 	flex_int32_t yy_verify;
601:| 	flex_int32_t yy_nxt;
603:| static yyconst flex_int16_t yy_acclist[23] =
610:| static yyconst flex_int16_t yy_accept[14] =
653:| static yyconst flex_uint16_t yy_base[13] =
659:| static yyconst flex_int16_t yy_def[13] =
665:| static yyconst flex_uint16_t yy_nxt[19] =
671:| static yyconst flex_int16_t yy_chk[19] =
677:| extern int yy_flex_debug;
678:| int yy_flex_debug = 0;
764:| static void yy_flex_strncpy (char *,yyconst char *,int );
768:| static int yy_flex_strlen (yyconst char * );
814:| 			YY_FATAL_ERROR( "input in flex scanner failed" ); \
824:| 				YY_FATAL_ERROR( "input in flex scanner failed" ); \
1013:| { /* IRIX 6.5 flex 2.5.4 underquotes its yyless argument.  */
1115:| 					 * hoser (like flex itself) wants to
1159:| 			"fatal flex scanner internal error--no action found" );
1181:| 		"fatal flex scanner internal error--end of buffer missed" );
1356:| 			YY_FATAL_ERROR( "flex scanner push-back overflow" );
1890:|         return yy_flex_debug;
1895:|         yy_flex_debug = _bdebug ;
1961:| static void yy_flex_strncpy (char* s1, yyconst char * s2, int n )
1971:| static int yy_flex_strlen (yyconst char * s )
2734:ac_cv_prog_LEX=flex
2948:LEX='flex'
$ bison --version
bison (GNU Bison) 3.0.4
Written by Robert Corbett and Richard Stallman.

Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ flex --version
flex 2.6.0
$ 

and the output of $ touch libasn1parser/*.y libasn1parser/*.l && make V=1 should be something like

$ touch libasn1parser/*.y libasn1parser/*.l
$ make V=1
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in libasn1parser
/bin/sh ../config/ylwrap asn1p_y.y y.tab.c asn1p_y.c y.tab.h `echo asn1p_y.c | sed -e s/cc$/hh/ -e s/cpp$/hpp/ -e s/cxx$/hxx/ -e s/c++$/h++/ -e s/c$/h/` y.output asn1p_y.output -- bison -y -p asn1p_ -d  
/Users/uri/src/asn1c/libasn1parser/asn1p_y.y: warning: 2 shift/reduce conflicts [-Wconflicts-sr]
asn1p_y.h is unchanged
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-am
/bin/sh ../libtool  --tag=CC   --mode=compile clang -DHAVE_CONFIG_H -I. -I..   -I/opt/local/include  -maes -mpclmul -mrdrnd -msse2 -mssse3 -msse4.2 -mtune=native -Os -Ofast -Wall -Wcast-qual -Wchar-subscripts -Wmissing-prototypes -Wmissing-declarations -Wno-error=cast-align -Wno-error=visibility -Wno-error=parentheses-equality -MT asn1p_y.lo -MD -MP -MF .deps/asn1p_y.Tpo -c -o asn1p_y.lo asn1p_y.c
libtool: compile:  clang -DHAVE_CONFIG_H -I. -I.. -I/opt/local/include -maes -mpclmul -mrdrnd -msse2 -mssse3 -msse4.2 -mtune=native -Os -Ofast -Wall -Wcast-qual -Wchar-subscripts -Wmissing-prototypes -Wmissing-declarations -Wno-error=cast-align -Wno-error=visibility -Wno-error=parentheses-equality -MT asn1p_y.lo -MD -MP -MF .deps/asn1p_y.Tpo -c asn1p_y.c  -fno-common -DPIC -o .libs/asn1p_y.o
asn1p_y.y:359:13: error: use of undeclared identifier 'param'
                *(void **)param = (yyvsp[0].a_grammar);
                          ^
1 error generated.
make[3]: *** [asn1p_y.lo] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
$ 

[mouse07410]

@velichkov same thing applies here. Can we you rebase this PR against my 115 branch? It will become the new master soon.

Update.
Look at vlm#82 - it appears bison issues have been seen before. The problem being - I'm running bison v3, while the code was written for v2.

[velichkov]

Hi @mouse07410,

@velichkov same thing applies here. Can we you rebase this PR against my 115 branch? It will become the new master soon.

No problem but instead of rebase I'm going to open a new PR for branch 115

Update.
Look at vlm#82 - it appears bison issues have been seen before. The problem being - I'm running bison v3, while the code was written for v2.

Yes, bison v3 is the problem and vlm#82 fixes it, I just cherry-picked db6c397 and pushed it here.

[mouse07410]

The master branch seems to have some problems here - because of the merged #2 that removes individual encoder/decoder pointers and puts them into a common structure. Here's the log:
test-suite.log.txt

Basically the master will become branch 129, but wouldn't it be nice if it also was complete in functionality? ;-)

[velichkov]

Basically the master will become branch 129, but wouldn't it be nice if it also was complete in functionality?

Sure but the strange thing is that I've already regenerated the tests files, see
https://github.com/velichkov/asn1c/blob/external-gh140/tests/03-enum-OK.asn1.-Pfwide-types#L513-L516
https://github.com/velichkov/asn1c/blob/external-gh140/tests/104-param-1-OK.asn1.-Pfwide-types#L276-L279

Also I just tried to merge manually my branch velichkov/external-gh140 into your latest master. The merge is successful without conflicts, all tests are successful and regenerate doesn't modify the test files.

[mouse07410]

The merge is successful without conflicts

Oh yes, the merge is successful, and so is the compilation. But some of the subsequent tests seem to fail. And I suspect the reason is - #2 (that was merged before) changed some of the internal structures.

[velichkov]

Oh yes, the merge is successful, and so is the compilation. But some of the subsequent tests seem to fail.

I just merged master and pushed the changes. The CI build is successful https://travis-ci.org/mouse07410/asn1c/jobs/215973265#L2309

Note that the Travis actually merge the PR before building it

git clone --depth=50 https://github.com/mouse07410/asn1c.git mouse07410/asn1c
cd mouse07410/asn1c
git fetch origin +refs/pull/3/merge: 
git checkout -qf FETCH_HEAD

The third command actually perform a merge

[mouse07410]

Note that the Travis actually merge the PR before building it

Thanks - to be honest, I didn't realize that (which now, in the hindsight, looks obvious :).

Merged. ;-)