Skip to content
This repository has been archived by the owner on Mar 26, 2023. It is now read-only.

Add build and install of libmagparse.so to the build.rake #352

Closed
AllenOtis opened this issue Oct 22, 2014 · 47 comments
Closed

Add build and install of libmagparse.so to the build.rake #352

AllenOtis opened this issue Oct 22, 2014 · 47 comments
Assignees

Comments

@AllenOtis
Copy link
Contributor

I have created the branch allen/libmagparse .
The commits 22c079c .. 86ea7b1
plus the new server build on seaside.gemtalksystems.com
gss64_maglev_1_9_branch-34618 (v3.1.0.2.1)
are part of moving the ruby parser out of libgcilnk.so
and into a new library , libmagparse.so .

The remaining work before this branch can be merged to master
involves adding C compile and link steps to the build.rake
to build the libmagparse.so and also a rake step to
install the resulting libmagparse.so into the server lib directory.
It would be desirable to have options to compile the library
with and without C optimization depending on whether full debugging
with gdb is needed.
This work is described in src/kernel/parser/README.txt .

Note that a developer working on
the parser would also need to run the yacc.sh step.
A make file that builds byacc and then runs it if the
src/kernel/parser/grammar.y is newer than
src/kernel/parser/rubygrammar.c might be desirable ;
otherwise byacc could be run before every C compile.
The use of byacc is discussed in src/kernel/parser/README.txt .

I have built the new libmagparse.so with a script outside of rake,
and run the tests listed in vmunit.conf successfully under topaz,
building and testing on Ubuntu 10.04 .

For now all of this is supported on Linux only. We haven't
published a v3.1.0.2.1 server for Mac yet.
And some of the byacc scripts or code, and some of the
compile and link steps given in
src/kernel/parser/README.txt would need changes to work on Mac .

Allen

@johnnyt
Copy link
Member

johnnyt commented Oct 23, 2014

Woot! Thank you Allen!

Sent from my iPhone

On Oct 22, 2014, at 5:57 PM, Allen Otis [email protected] wrote:

I have created the branch allen/libmagparse .
The commits 22c079c .. 86ea7b1
plus the new server build on seaside.gemtalksystems.com
gss64_maglev_1_9_branch-34618 (v3.1.0.2.1)
are part of moving the ruby parser out of libgcilnk.so
and into a new library , libmagparse.so .

The remaining work before this branch can be merged to master
involves adding C compile and link steps to the build.rake
to build the libmagparse.so and also a rake step to
install the resulting libmagparse.so into the server lib directory.
It would be desirable to have options to compile the library
with and without C optimization depending on whether full debugging
with gdb is needed.
This work is described in src/kernel/parser/README.txt .

Note that a developer working on
the parser would also need to run the yacc.sh step.
A make file that builds byacc and then runs it if the
src/kernel/parser/grammar.y is newer than
src/kernel/parser/rubygrammar.c might be desirable ;
otherwise byacc could be run before every C compile.
The use of byacc is discussed in src/kernel/parser/README.txt .

I have built the new libmagparse.so with a script outside of rake,
and run the tests listed in vmunit.conf successfully under topaz,
building and testing on Ubuntu 10.04 .

For now all of this is supported on Linux only. We haven't
published a v3.1.0.2.1 server for Mac yet.
And some of the byacc scripts or code, and some of the
compile and link steps given in
src/kernel/parser/README.txt would need changes to work on Mac .

Allen


Reply to this email directly or view it on GitHub.=

@johnnyt
Copy link
Member

johnnyt commented Nov 19, 2014

@AllenOtis -

I'm trying to get this to work (on ubuntu 14.04) and am getting this:

Loading Kernel for maglev.  This may take a few seconds...
rake aborted!
1
 _____________________________________________________________________________
|                             Configuration Files                             |
|                                                                             |
| System File: /opt/maglev/etc/system.conf                                    |
|                                                                             |
| Executable File: /opt/maglev/gem.conf                                       |
| Warning:  File not found (errno=2,ENOENT, The file or directory specified cannot
| be found)                                                                   |
|           using defaults.                                                   |
|_____________________________________________________________________________|
Using GEM_TEMPOBJ_CACHE_SIZE 200000 from -T command line option
 _____________________________________________________________________________
|               Gem Configuration Options for process id 27844                |
|_____________________________________________________________________________|

DUMP_OPTIONS = TRUE;
GEM_GCI_LOG_ENABLED = FALSE;
GEM_ABORT_MAX_CRS = 0;
GEM_FREE_FRAME_CACHE_SIZE = -1;
GEM_FREE_FRAME_LIMIT = -1;
GEM_FREE_PAGEIDS_CACHE = 200;
GEM_HALT_ON_ERROR = 0;
GEM_KEEP_MIN_SOFTREFS = 0;
GEM_MAX_SMALLTALK_STACK_DEPTH = 15000;
GEM_NATIVE_CODE_ENABLED = TRUE;
GEM_PRIVATE_PAGE_CACHE_KB = 960;
GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;
GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
GEM_PGSVR_FREE_FRAME_LIMIT = -1;
GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;
GEM_RPC_KEEPALIVE_INTERVAL = 0;
GEM_RPCGCI_TIMEOUT = 0;
GEM_RPC_USE_SSL = TRUE;
GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
GEM_TEMPOBJ_CACHE_SIZE = 200000;
GEM_TEMPOBJ_MESPACE_SIZE = 0;
GEM_TEMPOBJ_OOPMAP_SIZE = 0;
GEM_TEMPOBJ_SCOPES_SIZE = 20000;
GEM_TEMPOBJ_POMGEN_SIZE = 0;
GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 50;
GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
CFG_GEM_TEMPOBJ_START_ADDR not used on this platform
LOG_WARNINGS = TRUE;
SHR_NUM_FREE_FRAME_SERVERS = -1;
SHR_PAGE_CACHE_LOCKED = FALSE;
SHR_PAGE_CACHE_NUM_PROCS = 4089;
SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
SHR_PAGE_CACHE_PERMISSIONS = 660;
SHR_PAGE_CACHE_SIZE_KB = 1000000;
SHR_TARGET_FREE_FRAME_COUNT = -1;
SHR_WELL_KNOWN_PORT_NUMBER = 0;
(vmGc spaceSizes: eden init 2048K max 37504K , survivor init 448K max 6272K,
 vmGc    old max 149952K, code max 40000K, perm max 20032K, pom 10 * 16704K = 167040K,
 vmGc    remSet 4036K, meSpace max 191368K oopMapSize 1048576 )
 _____________________________________________________________________________
|             GemStone/S64 Object-Oriented Data Management System             |
|                    Copyright (C) VMware, Inc. 1986-2014                     |
|                            All rights reserved.                             |
|                           Covered by U.S Patents:                           |
|            6,256,637 Transactional virtual machine architecture             |
|              6,360,219 Object queues with concurrent updating               |
|                  6,567,905 Generational Garbage Collector.                  |
| 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable Database
+-----------------------------------------------------------------------------+
|    PROGRAM: topaz, Linear GemStone Interface (Linked Session)               |
|    VERSION: 3.1.0.2.1, Beta  Tue Oct 21 16:16:34 2014                       |
|      BUILD: gss64_maglev_1_9_branch-34619                                   |
|  BUILT FOR: x86-64 (Linux)                                                  |
|       MODE: 64 bit                                                          |
| RUNNING ON: 4-CPU db1 x86_64 (Linux 3.13.0-39-generic #66-Ubuntu SMP Tue Oct|
| 28 13:30:27 UTC 2014) 3953MB                                                |
| PROCESS ID: 27844     DATE: 11/19/2014 01:30:43 UTC                         |
|   USER IDS: REAL=ubuntu (1001) EFFECTIVE=ubuntu (1001)                      |
 _____________________________________________________________________________
|   GEMSTONE_GLOBAL_DIR = /opt/maglev                                         |
|_____________________________________________________________________________|
neither .topazini nor $HOME/.topazini were found
TopazError:
/opt/maglev/rakelib/contrib/ottobehrens/topaz.rb:52:in `commands'
/opt/maglev/rakelib/contrib/ottobehrens/stone.rb:271:in `topaz_commands'
/opt/maglev/rakelib/contrib/ottobehrens/stone.rb:187:in `input_file'
/opt/maglev/rakelib/maglev_stone.rb:156:in `reload_prims'
/opt/maglev/Rakefile:185:in `block in task_gemstone'
Tasks: TOP => maglev:reload_prims
(See full trace by running task with --trace)

topaz.log

The end of maglev/log/maglev/topaz.log contains:

topaz> set user DataCurator pass swordfish
Warning: clearing the previous GemStone password.
topaz> login
[Info]: User ID: DataCurator
[Info]: Repository: maglev
[Info]: Session ID: 2
[Info]: GCI Client Host: <Linked>
[Info]: Page server PID: -1
[Info]: Login Time: 11/19/2014 01:33:29.559 UTC
successful login
topaz 1> run
 RubyBridge initialize . "reset generic bridge methods"
 RubyContext commitTransaction  .
%
[268 sz:0 cls: 68097 Boolean] true
topaz 1> run
RubyContext load: #( ) env: 1
%
-----------------------------------------------------
GemStone: Error         Nonfatal
a ArgumentTypeError occurred (error 2094), reason:rtErrBadArgKind,
expected a RubyEvalScope
Error Category: 231169 [GemStone] Number: 2094  Arg Count: 4 Context : 108739841 exception : 21770497
Arg 1: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 2: [27478017 sz:19 cls: 27477761 RubyEvalScope class] RubyEvalScope
Arg 3: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 4: [20 sz:0 cls: 76289 UndefinedObject] nil
topaz > exec iferr 1 : exit

@AllenOtis
Copy link
Contributor Author

You need to be running the topaz executable interactively, or else
maglev script with -d (I think) so you can look at the combined
Smalltalk/ruby stack when the error occurs. Remember our
debugging sessions in Beaverton ? First step is to dig into
the stack to see where the error is coming from, etc.

I suppose unfortunately this is running under rake.
The rake code ideally would have an option to emit to text files
each of the topaz scripts it generates, or else each piece
of topaz script would be a separate text file, not embedded in rake files.

When I do VM development I almost never use rake .

Allen

On Tue, Nov 18, 2014 at 6:13 PM, JohnnyT [email protected] wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this to work (on ubuntu 14.04) and am getting this:

Loading Kernel for maglev. This may take a few seconds...
rake aborted!
1


| Configuration Files |
| |
| System File: /opt/maglev/etc/system.conf |
| |
| Executable File: /opt/maglev/gem.conf |
| Warning: File not found (errno=2,ENOENT, The file or directory specified cannot
| be found) |
| using defaults. |
|_____________________________________________________________________________|
Using GEM_TEMPOBJ_CACHE_SIZE 200000 from -T command line option


| Gem Configuration Options for process id 27844 |
|_____________________________________________________________________________|

DUMP_OPTIONS = TRUE;
GEM_GCI_LOG_ENABLED = FALSE;
GEM_ABORT_MAX_CRS = 0;
GEM_FREE_FRAME_CACHE_SIZE = -1;
GEM_FREE_FRAME_LIMIT = -1;
GEM_FREE_PAGEIDS_CACHE = 200;
GEM_HALT_ON_ERROR = 0;
GEM_KEEP_MIN_SOFTREFS = 0;
GEM_MAX_SMALLTALK_STACK_DEPTH = 15000;
GEM_NATIVE_CODE_ENABLED = TRUE;
GEM_PRIVATE_PAGE_CACHE_KB = 960;
GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;
GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
GEM_PGSVR_FREE_FRAME_LIMIT = -1;
GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;
GEM_RPC_KEEPALIVE_INTERVAL = 0;
GEM_RPCGCI_TIMEOUT = 0;
GEM_RPC_USE_SSL = TRUE;
GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
GEM_TEMPOBJ_CACHE_SIZE = 200000;
GEM_TEMPOBJ_MESPACE_SIZE = 0;
GEM_TEMPOBJ_OOPMAP_SIZE = 0;
GEM_TEMPOBJ_SCOPES_SIZE = 20000;
GEM_TEMPOBJ_POMGEN_SIZE = 0;
GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 50;
GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
CFG_GEM_TEMPOBJ_START_ADDR not used on this platform
LOG_WARNINGS = TRUE;
SHR_NUM_FREE_FRAME_SERVERS = -1;
SHR_PAGE_CACHE_LOCKED = FALSE;
SHR_PAGE_CACHE_NUM_PROCS = 4089;
SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
SHR_PAGE_CACHE_PERMISSIONS = 660;
SHR_PAGE_CACHE_SIZE_KB = 1000000;
SHR_TARGET_FREE_FRAME_COUNT = -1;
SHR_WELL_KNOWN_PORT_NUMBER = 0;
(vmGc spaceSizes: eden init 2048K max 37504K , survivor init 448K max 6272K,
vmGc old max 149952K, code max 40000K, perm max 20032K, pom 10 * 16704K = 167040K,
vmGc remSet 4036K, meSpace max 191368K oopMapSize 1048576 )


| GemStone/S64 Object-Oriented Data Management System |
| Copyright (C) VMware, Inc. 1986-2014 |
| All rights reserved. |
| Covered by U.S Patents: |
| 6,256,637 Transactional virtual machine architecture |
| 6,360,219 Object queues with concurrent updating |
| 6,567,905 Generational Garbage Collector. |
| 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable Database
+-----------------------------------------------------------------------------+
| PROGRAM: topaz, Linear GemStone Interface (Linked Session) |
| VERSION: 3.1.0.2.1, Beta Tue Oct 21 16:16:34 2014 |
| BUILD: gss64_maglev_1_9_branch-34619 |
| BUILT FOR: x86-64 (Linux) |
| MODE: 64 bit |
| RUNNING ON: 4-CPU db1 x86_64 (Linux 3.13.0-39-generic #66-Ubuntu SMP Tue Oct|
| 28 13:30:27 UTC 2014) 3953MB |
| PROCESS ID: 27844 DATE: 11/19/2014 01:30:43 UTC |
| USER IDS: REAL=ubuntu (1001) EFFECTIVE=ubuntu (1001) |


| GEMSTONE_GLOBAL_DIR = /opt/maglev |
|_____________________________________________________________________________|
neither .topazini nor $HOME/.topazini were found
TopazError:
/opt/maglev/rakelib/contrib/ottobehrens/topaz.rb:52:in commands' /opt/maglev/rakelib/contrib/ottobehrens/stone.rb:271:intopaz_commands'
/opt/maglev/rakelib/contrib/ottobehrens/stone.rb:187:in input_file' /opt/maglev/rakelib/maglev_stone.rb:156:inreload_prims'
/opt/maglev/Rakefile:185:in `block in task_gemstone'
Tasks: TOP => maglev:reload_prims
(See full trace by running task with --trace)

topaz.log

The end of maglev/log/maglev/topaz.log contains:

topaz> set user DataCurator pass swordfish
Warning: clearing the previous GemStone password.
topaz> login
[Info]: User ID: DataCurator
[Info]: Repository: maglev
[Info]: Session ID: 2
[Info]: GCI Client Host:
[Info]: Page server PID: -1
[Info]: Login Time: 11/19/2014 01:33:29.559 UTC
successful login
topaz 1> run
RubyBridge initialize . "reset generic bridge methods"
RubyContext commitTransaction .
%
[268 sz:0 cls: 68097 Boolean] true
topaz 1> run
RubyContext load: #( ) env: 1

%

GemStone: Error Nonfatal
a ArgumentTypeError occurred (error 2094), reason:rtErrBadArgKind,
expected a RubyEvalScope
Error Category: 231169 [GemStone] Number: 2094 Arg Count: 4 Context : 108739841 exception : 21770497
Arg 1: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 2: [27478017 sz:19 cls: 27477761 RubyEvalScope class] RubyEvalScope
Arg 3: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 4: [20 sz:0 cls: 76289 UndefinedObject] nil
topaz > exec iferr 1 : exit


Reply to this email directly or view it on GitHub
#352 (comment).

@Monty
Copy link
Member

Monty commented Nov 19, 2014

It's possible the rake script emits the files & only deletes them if there is no error. Worth checking anyway.

-- Monty

Sent from my iPhone

On Nov 18, 2014, at 7:38 PM, Allen Otis [email protected] wrote:

You need to be running the topaz executable interactively, or else
maglev script with -d (I think) so you can look at the combined
Smalltalk/ruby stack when the error occurs. Remember our
debugging sessions in Beaverton ? First step is to dig into
the stack to see where the error is coming from, etc.

I suppose unfortunately this is running under rake.
The rake code ideally would have an option to emit to text files
each of the topaz scripts it generates, or else each piece
of topaz script would be a separate text file, not embedded in rake files.

When I do VM development I almost never use rake .

Allen

On Tue, Nov 18, 2014 at 6:13 PM, JohnnyT [email protected] wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this to work (on ubuntu 14.04) and am getting this:

Loading Kernel for maglev. This may take a few seconds...
rake aborted!
1


| Configuration Files |
| |
| System File: /opt/maglev/etc/system.conf |
| |
| Executable File: /opt/maglev/gem.conf |
| Warning: File not found (errno=2,ENOENT, The file or directory specified cannot
| be found) |
| using defaults. |
|_____________________________________________________________________________|
Using GEM_TEMPOBJ_CACHE_SIZE 200000 from -T command line option


| Gem Configuration Options for process id 27844 |
|_____________________________________________________________________________|

DUMP_OPTIONS = TRUE;
GEM_GCI_LOG_ENABLED = FALSE;
GEM_ABORT_MAX_CRS = 0;
GEM_FREE_FRAME_CACHE_SIZE = -1;
GEM_FREE_FRAME_LIMIT = -1;
GEM_FREE_PAGEIDS_CACHE = 200;
GEM_HALT_ON_ERROR = 0;
GEM_KEEP_MIN_SOFTREFS = 0;
GEM_MAX_SMALLTALK_STACK_DEPTH = 15000;
GEM_NATIVE_CODE_ENABLED = TRUE;
GEM_PRIVATE_PAGE_CACHE_KB = 960;
GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;
GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
GEM_PGSVR_FREE_FRAME_LIMIT = -1;
GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;
GEM_RPC_KEEPALIVE_INTERVAL = 0;
GEM_RPCGCI_TIMEOUT = 0;
GEM_RPC_USE_SSL = TRUE;
GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
GEM_TEMPOBJ_CACHE_SIZE = 200000;
GEM_TEMPOBJ_MESPACE_SIZE = 0;
GEM_TEMPOBJ_OOPMAP_SIZE = 0;
GEM_TEMPOBJ_SCOPES_SIZE = 20000;
GEM_TEMPOBJ_POMGEN_SIZE = 0;
GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 50;
GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
CFG_GEM_TEMPOBJ_START_ADDR not used on this platform
LOG_WARNINGS = TRUE;
SHR_NUM_FREE_FRAME_SERVERS = -1;
SHR_PAGE_CACHE_LOCKED = FALSE;
SHR_PAGE_CACHE_NUM_PROCS = 4089;
SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
SHR_PAGE_CACHE_PERMISSIONS = 660;
SHR_PAGE_CACHE_SIZE_KB = 1000000;
SHR_TARGET_FREE_FRAME_COUNT = -1;
SHR_WELL_KNOWN_PORT_NUMBER = 0;
(vmGc spaceSizes: eden init 2048K max 37504K , survivor init 448K max 6272K,
vmGc old max 149952K, code max 40000K, perm max 20032K, pom 10 * 16704K = 167040K,
vmGc remSet 4036K, meSpace max 191368K oopMapSize 1048576 )


| GemStone/S64 Object-Oriented Data Management System |
| Copyright (C) VMware, Inc. 1986-2014 |
| All rights reserved. |
| Covered by U.S Patents: |
| 6,256,637 Transactional virtual machine architecture |
| 6,360,219 Object queues with concurrent updating |
| 6,567,905 Generational Garbage Collector. |
| 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable Database
+-----------------------------------------------------------------------------+
| PROGRAM: topaz, Linear GemStone Interface (Linked Session) |
| VERSION: 3.1.0.2.1, Beta Tue Oct 21 16:16:34 2014 |
| BUILD: gss64_maglev_1_9_branch-34619 |
| BUILT FOR: x86-64 (Linux) |
| MODE: 64 bit |
| RUNNING ON: 4-CPU db1 x86_64 (Linux 3.13.0-39-generic #66-Ubuntu SMP Tue Oct|
| 28 13:30:27 UTC 2014) 3953MB |
| PROCESS ID: 27844 DATE: 11/19/2014 01:30:43 UTC |
| USER IDS: REAL=ubuntu (1001) EFFECTIVE=ubuntu (1001) |


| GEMSTONE_GLOBAL_DIR = /opt/maglev |
|_____________________________________________________________________________|
neither .topazini nor $HOME/.topazini were found
TopazError:
/opt/maglev/rakelib/contrib/ottobehrens/topaz.rb:52:in commands' /opt/maglev/rakelib/contrib/ottobehrens/stone.rb:271:intopaz_commands'
/opt/maglev/rakelib/contrib/ottobehrens/stone.rb:187:in input_file' /opt/maglev/rakelib/maglev_stone.rb:156:inreload_prims'
/opt/maglev/Rakefile:185:in `block in task_gemstone'
Tasks: TOP => maglev:reload_prims
(See full trace by running task with --trace)

topaz.log

The end of maglev/log/maglev/topaz.log contains:

topaz> set user DataCurator pass swordfish
Warning: clearing the previous GemStone password.
topaz> login
[Info]: User ID: DataCurator
[Info]: Repository: maglev
[Info]: Session ID: 2
[Info]: GCI Client Host:
[Info]: Page server PID: -1
[Info]: Login Time: 11/19/2014 01:33:29.559 UTC
successful login
topaz 1> run
RubyBridge initialize . "reset generic bridge methods"
RubyContext commitTransaction .
%
[268 sz:0 cls: 68097 Boolean] true
topaz 1> run
RubyContext load: #( ) env: 1

%

GemStone: Error Nonfatal
a ArgumentTypeError occurred (error 2094), reason:rtErrBadArgKind,
expected a RubyEvalScope
Error Category: 231169 [GemStone] Number: 2094 Arg Count: 4 Context : 108739841 exception : 21770497
Arg 1: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 2: [27478017 sz:19 cls: 27477761 RubyEvalScope class] RubyEvalScope
Arg 3: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 4: [20 sz:0 cls: 76289 UndefinedObject] nil
topaz > exec iferr 1 : exit


Reply to this email directly or view it on GitHub
#352 (comment).


Reply to this email directly or view it on GitHub.

@AllenOtis
Copy link
Contributor Author

Also, I think the rake scripts should include "where" and "stack"
in some additional topaz iferr commands. Johnny and I will be looking
at this together later today.

Allen

On Tue, Nov 18, 2014 at 8:27 PM, Monty Williams [email protected]
wrote:

It's possible the rake script emits the files & only deletes them if there
is no error. Worth checking anyway.

-- Monty

Sent from my iPhone

On Nov 18, 2014, at 7:38 PM, Allen Otis [email protected]
wrote:

You need to be running the topaz executable interactively, or else
maglev script with -d (I think) so you can look at the combined
Smalltalk/ruby stack when the error occurs. Remember our
debugging sessions in Beaverton ? First step is to dig into
the stack to see where the error is coming from, etc.

I suppose unfortunately this is running under rake.
The rake code ideally would have an option to emit to text files
each of the topaz scripts it generates, or else each piece
of topaz script would be a separate text file, not embedded in rake
files.

When I do VM development I almost never use rake .

Allen

On Tue, Nov 18, 2014 at 6:13 PM, JohnnyT [email protected]
wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this to work (on ubuntu 14.04) and am getting this:

Loading Kernel for maglev. This may take a few seconds...
rake aborted!
1


| Configuration Files |
| |
| System File: /opt/maglev/etc/system.conf |
| |
| Executable File: /opt/maglev/gem.conf |
| Warning: File not found (errno=2,ENOENT, The file or directory
specified cannot
| be found) |
| using defaults. |

|_____________________________________________________________________________|

Using GEM_TEMPOBJ_CACHE_SIZE 200000 from -T command line option


| Gem Configuration Options for process id 27844 |

|_____________________________________________________________________________|

DUMP_OPTIONS = TRUE;
GEM_GCI_LOG_ENABLED = FALSE;
GEM_ABORT_MAX_CRS = 0;
GEM_FREE_FRAME_CACHE_SIZE = -1;
GEM_FREE_FRAME_LIMIT = -1;
GEM_FREE_PAGEIDS_CACHE = 200;
GEM_HALT_ON_ERROR = 0;
GEM_KEEP_MIN_SOFTREFS = 0;
GEM_MAX_SMALLTALK_STACK_DEPTH = 15000;
GEM_NATIVE_CODE_ENABLED = TRUE;
GEM_PRIVATE_PAGE_CACHE_KB = 960;
GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;
GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
GEM_PGSVR_FREE_FRAME_LIMIT = -1;
GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;
GEM_RPC_KEEPALIVE_INTERVAL = 0;
GEM_RPCGCI_TIMEOUT = 0;
GEM_RPC_USE_SSL = TRUE;
GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
GEM_TEMPOBJ_CACHE_SIZE = 200000;
GEM_TEMPOBJ_MESPACE_SIZE = 0;
GEM_TEMPOBJ_OOPMAP_SIZE = 0;
GEM_TEMPOBJ_SCOPES_SIZE = 20000;
GEM_TEMPOBJ_POMGEN_SIZE = 0;
GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 50;
GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
CFG_GEM_TEMPOBJ_START_ADDR not used on this platform
LOG_WARNINGS = TRUE;
SHR_NUM_FREE_FRAME_SERVERS = -1;
SHR_PAGE_CACHE_LOCKED = FALSE;
SHR_PAGE_CACHE_NUM_PROCS = 4089;
SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
SHR_PAGE_CACHE_PERMISSIONS = 660;
SHR_PAGE_CACHE_SIZE_KB = 1000000;
SHR_TARGET_FREE_FRAME_COUNT = -1;
SHR_WELL_KNOWN_PORT_NUMBER = 0;
(vmGc spaceSizes: eden init 2048K max 37504K , survivor init 448K max
6272K,
vmGc old max 149952K, code max 40000K, perm max 20032K, pom 10 *
16704K = 167040K,
vmGc remSet 4036K, meSpace max 191368K oopMapSize 1048576 )


| GemStone/S64 Object-Oriented Data Management System |
| Copyright (C) VMware, Inc. 1986-2014 |
| All rights reserved. |
| Covered by U.S Patents: |
| 6,256,637 Transactional virtual machine architecture |
| 6,360,219 Object queues with concurrent updating |
| 6,567,905 Generational Garbage Collector. |
| 6,681,226 Selective Pessimistic Locking for a Concurrently
Updateable Database

+-----------------------------------------------------------------------------+

| PROGRAM: topaz, Linear GemStone Interface (Linked Session) |
| VERSION: 3.1.0.2.1, Beta Tue Oct 21 16:16:34 2014 |
| BUILD: gss64_maglev_1_9_branch-34619 |
| BUILT FOR: x86-64 (Linux) |
| MODE: 64 bit |
| RUNNING ON: 4-CPU db1 x86_64 (Linux 3.13.0-39-generic #66-Ubuntu SMP
Tue Oct|
| 28 13:30:27 UTC 2014) 3953MB |
| PROCESS ID: 27844 DATE: 11/19/2014 01:30:43 UTC |
| USER IDS: REAL=ubuntu (1001) EFFECTIVE=ubuntu (1001) |


| GEMSTONE_GLOBAL_DIR = /opt/maglev |

|_____________________________________________________________________________|

neither .topazini nor $HOME/.topazini were found
TopazError:
/opt/maglev/rakelib/contrib/ottobehrens/topaz.rb:52:in commands' /opt/maglev/rakelib/contrib/ottobehrens/stone.rb:271:in topaz_commands'
/opt/maglev/rakelib/contrib/ottobehrens/stone.rb:187:in input_file' /opt/maglev/rakelib/maglev_stone.rb:156:inreload_prims'
/opt/maglev/Rakefile:185:in `block in task_gemstone'
Tasks: TOP => maglev:reload_prims
(See full trace by running task with --trace)

topaz.log

The end of maglev/log/maglev/topaz.log contains:

topaz> set user DataCurator pass swordfish
Warning: clearing the previous GemStone password.
topaz> login
[Info]: User ID: DataCurator
[Info]: Repository: maglev
[Info]: Session ID: 2
[Info]: GCI Client Host:
[Info]: Page server PID: -1
[Info]: Login Time: 11/19/2014 01:33:29.559 UTC
successful login
topaz 1> run
RubyBridge initialize . "reset generic bridge methods"
RubyContext commitTransaction .
%
[268 sz:0 cls: 68097 Boolean] true
topaz 1> run
RubyContext load: #( ) env: 1

%

GemStone: Error Nonfatal
a ArgumentTypeError occurred (error 2094), reason:rtErrBadArgKind,
expected a RubyEvalScope
Error Category: 231169 [GemStone] Number: 2094 Arg Count: 4 Context :
108739841 exception : 21770497
Arg 1: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 2: [27478017 sz:19 cls: 27477761 RubyEvalScope class]
RubyEvalScope
Arg 3: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 4: [20 sz:0 cls: 76289 UndefinedObject] nil
topaz > exec iferr 1 : exit


Reply to this email directly or view it on GitHub
#352 (comment).


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#352 (comment).

@AllenOtis
Copy link
Contributor Author

I just committed a change to src/smalltalk/ruby/allprims.topaz
adding iferr commands to print the stack before exiting topaz.

On Tue, Nov 18, 2014 at 6:13 PM, JohnnyT [email protected] wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this to work (on ubuntu 14.04) and am getting this:

Loading Kernel for maglev. This may take a few seconds...
rake aborted!
1


| Configuration Files |
| |
| System File: /opt/maglev/etc/system.conf |
| |
| Executable File: /opt/maglev/gem.conf |
| Warning: File not found (errno=2,ENOENT, The file or directory specified cannot
| be found) |
| using defaults. |
|_____________________________________________________________________________|
Using GEM_TEMPOBJ_CACHE_SIZE 200000 from -T command line option


| Gem Configuration Options for process id 27844 |
|_____________________________________________________________________________|

DUMP_OPTIONS = TRUE;
GEM_GCI_LOG_ENABLED = FALSE;
GEM_ABORT_MAX_CRS = 0;
GEM_FREE_FRAME_CACHE_SIZE = -1;
GEM_FREE_FRAME_LIMIT = -1;
GEM_FREE_PAGEIDS_CACHE = 200;
GEM_HALT_ON_ERROR = 0;
GEM_KEEP_MIN_SOFTREFS = 0;
GEM_MAX_SMALLTALK_STACK_DEPTH = 15000;
GEM_NATIVE_CODE_ENABLED = TRUE;
GEM_PRIVATE_PAGE_CACHE_KB = 960;
GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;
GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
GEM_PGSVR_FREE_FRAME_LIMIT = -1;
GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;
GEM_RPC_KEEPALIVE_INTERVAL = 0;
GEM_RPCGCI_TIMEOUT = 0;
GEM_RPC_USE_SSL = TRUE;
GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
GEM_TEMPOBJ_CACHE_SIZE = 200000;
GEM_TEMPOBJ_MESPACE_SIZE = 0;
GEM_TEMPOBJ_OOPMAP_SIZE = 0;
GEM_TEMPOBJ_SCOPES_SIZE = 20000;
GEM_TEMPOBJ_POMGEN_SIZE = 0;
GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 50;
GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
CFG_GEM_TEMPOBJ_START_ADDR not used on this platform
LOG_WARNINGS = TRUE;
SHR_NUM_FREE_FRAME_SERVERS = -1;
SHR_PAGE_CACHE_LOCKED = FALSE;
SHR_PAGE_CACHE_NUM_PROCS = 4089;
SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
SHR_PAGE_CACHE_PERMISSIONS = 660;
SHR_PAGE_CACHE_SIZE_KB = 1000000;
SHR_TARGET_FREE_FRAME_COUNT = -1;
SHR_WELL_KNOWN_PORT_NUMBER = 0;
(vmGc spaceSizes: eden init 2048K max 37504K , survivor init 448K max 6272K,
vmGc old max 149952K, code max 40000K, perm max 20032K, pom 10 * 16704K = 167040K,
vmGc remSet 4036K, meSpace max 191368K oopMapSize 1048576 )


| GemStone/S64 Object-Oriented Data Management System |
| Copyright (C) VMware, Inc. 1986-2014 |
| All rights reserved. |
| Covered by U.S Patents: |
| 6,256,637 Transactional virtual machine architecture |
| 6,360,219 Object queues with concurrent updating |
| 6,567,905 Generational Garbage Collector. |
| 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable Database
+-----------------------------------------------------------------------------+
| PROGRAM: topaz, Linear GemStone Interface (Linked Session) |
| VERSION: 3.1.0.2.1, Beta Tue Oct 21 16:16:34 2014 |
| BUILD: gss64_maglev_1_9_branch-34619 |
| BUILT FOR: x86-64 (Linux) |
| MODE: 64 bit |
| RUNNING ON: 4-CPU db1 x86_64 (Linux 3.13.0-39-generic #66-Ubuntu SMP Tue Oct|
| 28 13:30:27 UTC 2014) 3953MB |
| PROCESS ID: 27844 DATE: 11/19/2014 01:30:43 UTC |
| USER IDS: REAL=ubuntu (1001) EFFECTIVE=ubuntu (1001) |


| GEMSTONE_GLOBAL_DIR = /opt/maglev |
|_____________________________________________________________________________|
neither .topazini nor $HOME/.topazini were found
TopazError:
/opt/maglev/rakelib/contrib/ottobehrens/topaz.rb:52:in commands' /opt/maglev/rakelib/contrib/ottobehrens/stone.rb:271:intopaz_commands'
/opt/maglev/rakelib/contrib/ottobehrens/stone.rb:187:in input_file' /opt/maglev/rakelib/maglev_stone.rb:156:inreload_prims'
/opt/maglev/Rakefile:185:in `block in task_gemstone'
Tasks: TOP => maglev:reload_prims
(See full trace by running task with --trace)

topaz.log

The end of maglev/log/maglev/topaz.log contains:

topaz> set user DataCurator pass swordfish
Warning: clearing the previous GemStone password.
topaz> login
[Info]: User ID: DataCurator
[Info]: Repository: maglev
[Info]: Session ID: 2
[Info]: GCI Client Host:
[Info]: Page server PID: -1
[Info]: Login Time: 11/19/2014 01:33:29.559 UTC
successful login
topaz 1> run
RubyBridge initialize . "reset generic bridge methods"
RubyContext commitTransaction .
%
[268 sz:0 cls: 68097 Boolean] true
topaz 1> run
RubyContext load: #( ) env: 1

%

GemStone: Error Nonfatal
a ArgumentTypeError occurred (error 2094), reason:rtErrBadArgKind,
expected a RubyEvalScope
Error Category: 231169 [GemStone] Number: 2094 Arg Count: 4 Context : 108739841 exception : 21770497
Arg 1: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 2: [27478017 sz:19 cls: 27477761 RubyEvalScope class] RubyEvalScope
Arg 3: [20 sz:0 cls: 76289 UndefinedObject] nil
Arg 4: [20 sz:0 cls: 76289 UndefinedObject] nil
topaz > exec iferr 1 : exit


Reply to this email directly or view it on GitHub
#352 (comment).

@Monty
Copy link
Member

Monty commented Nov 19, 2014

I’m suspicious of rakelib/contrib/ottobehrens/
I don’t think it was ever updated as the build process evolved. I wouldn’t be surprised if rakelib/contrib/ottobehrens/tests/ won’t run successfully anymore. That may not be worth fixing as long as the critical pieces run OK.

— Monty

@AllenOtis
Copy link
Contributor Author

I just committed a file README_bootstrap.txt which outlines the bootstrap
process , since I keep forgetting it when I am away from maglev for a while.
I believe Johnny's error is from the "allprims.gs" phase which would be
before
any tests were run.

On Wed, Nov 19, 2014 at 10:12 AM, Monty Williams [email protected]
wrote:

I’m suspicious of rakelib/contrib/ottobehrens/
I don’t think it was ever updated as the build process evolved. I wouldn’t
be surprised if rakelib/contrib/ottobehrens/tests/ won’t run successfully
anymore. That may not be worth fixing as long as the critical pieces run
OK.

— Monty


Reply to this email directly or view it on GitHub
#352 (comment).

@Monty
Copy link
Member

Monty commented Nov 19, 2014

Thanks!

On Nov 19, 2014, at 10:15 AM, Allen Otis [email protected] wrote:

I just committed a file README_bootstrap.txt which outlines the bootstrap
process , since I keep forgetting it when I am away from maglev for a while.
I believe Johnny's error is from the "allprims.gs" phase which would be
before
any tests were run.

On Wed, Nov 19, 2014 at 10:12 AM, Monty Williams [email protected]
wrote:

I’m suspicious of rakelib/contrib/ottobehrens/
I don’t think it was ever updated as the build process evolved. I wouldn’t
be surprised if rakelib/contrib/ottobehrens/tests/ won’t run successfully
anymore. That may not be worth fixing as long as the critical pieces run
OK.

— Monty


Reply to this email directly or view it on GitHub
#352 (comment).


Reply to this email directly or view it on GitHub #352 (comment).

@Monty
Copy link
Member

Monty commented Nov 19, 2014

I had almost forgot about Otto’s files until I remembered he contributed stone.rb & topaz.rb

TopazError:
/opt/maglev/rakelib/contrib/ottobehrens/topaz.rb:52:in commands' /opt/maglev/rakelib/contrib/ottobehrens/stone.rb:271:intopaz_commands'
/opt/maglev/rakelib/contrib/ottobehrens/stone.rb:187:in input_file' /opt/maglev/rakelib/maglev_stone.rb:156:inreload_prims'
/opt/maglev/Rakefile:185:in `block in task_gemstone'
Tasks: TOP => maglev:reload_prims

On Nov 19, 2014, at 10:15 AM, Allen Otis [email protected] wrote:

I just committed a file README_bootstrap.txt which outlines the bootstrap
process , since I keep forgetting it when I am away from maglev for a while.
I believe Johnny's error is from the "allprims.gs" phase which would be
before
any tests were run.

On Wed, Nov 19, 2014 at 10:12 AM, Monty Williams [email protected]
wrote:

I’m suspicious of rakelib/contrib/ottobehrens/
I don’t think it was ever updated as the build process evolved. I wouldn’t
be surprised if rakelib/contrib/ottobehrens/tests/ won’t run successfully
anymore. That may not be worth fixing as long as the critical pieces run
OK.

— Monty


Reply to this email directly or view it on GitHub
#352 (comment).


Reply to this email directly or view it on GitHub #352 (comment).

@AllenOtis
Copy link
Contributor Author

I have some documentation additions to rubyom.hf that are in progress,
not yet committed, including the following

 class omObjSType
 {
+ // a C variable of type  omObjSType*  is a pointer to an in-memory object,
+ // where in-memory means the object was created in, or faulted into
+ // the VM's temporary object memory.  For some function calls and returns
+ // the args or result may also be the object id of a special object
+ // such as OOP_NIL or a SmallInteger .
+
+ // When an argument of type omObjSType* is passed to an om:: function 
+ // and the function is documented as "possible gc" or "may cause gc",
+ // then that pointer is not valid after the function returns.
+ // To access the object again you would need to have saved it
+ // in a handle, obtained from OmScopeType::add or 
+ // OmScopeType::newHandle , and then dereference the handle after
+ // the function call which invalidated the pointer.

In code like this (shown with line numbers) from rubygrammar.y .
Here NODE is equivalent to omObjSType and we are concerned about
ensuring that changes to the parser follow the safe style of using object handles.

A handle is a NODE** obtained from an instance of OmScopeType .

1466                 | tUMINUS_NUM tFLOAT tPOW arg
1467                     {
1468                         // $$  = call_op(call_op($2, tPOW, 1, $4, vps), tUMINUS, 0, 0, vps);
1469                       yTrace(vps, "arg: | tUMINUS_NUM tFLOAT tPOW arg"); 
1470                       OmScopeType aScope(vps->omPtr);
1471                       NODE **litH = aScope.add(RubyAbstractNumberNode::s( $2, vps));
1472                       NODE **valH = aScope.add( RubyParser::new_call_1( litH, a_sym_tPOW, & $4     , $3/*srcOffsetSi*/, vps));
1473                       NODE **selH = aScope.add( RpNameToken::s(a_sym_tUMINUS, $3, vps));
1474                       $$ = RubyParser::new_vcall( *valH, *selH , vps);

Note that the object pointers, i.e. NODE *x are saved in handles via
the aScope.add calls. If you were to do this

                       NODE *valO = RubyParser::new_call_1( litH, a_sym_tPOW, & $4     , $3/*srcOffsetSi*/, vps);
                       NODE *selO = RpNameToken::s(a_sym_tUMINUS, $3, vps);
                       $$ = RubyParser::new_vcall( valO, selO , vps);

The C compiler would not complain, and yet the valO object pointer could
become stale if a GC of temporary object memory happened during the RpNameToken::s
call , leading to subsequent SEGV or mysterious errors during the new_vcall call
or later on during execution

Eventually some more sophisticated usage of C++ typing might make these
type of errors harder to accomplish; not sure when or if I will get to adding
that to rubyom.hf .

I would be glad to proofread any parser changes when they are ready
for review.

Allen

@johnnyt
Copy link
Member

johnnyt commented Nov 19, 2014

Hey @AllenOtis -

Currently the file at http://seaside.gemtalksystems.com/maglev/GemStone-34619.Linux-x86_64.tar.gz
when unzipped creates a folder named:
GemStone-345619.Linux-x86_64, but should be
GemStone-34619.Linux-x86_64

( the 345619 should be 34619 - there is an extra 5 in the middle )

Could we get that updated?

@AllenOtis
Copy link
Contributor Author

The directory names within the 34619 download should be fixed now.

@AllenOtis
Copy link
Contributor Author

If a pre-built libmagparse.so is eventually added to the maglev repository,
I would suggest it be built on Ubuntu 12.04 so it can be run on both Ubuntu 12 and 14 .

@jc00ke
Copy link
Contributor

jc00ke commented Feb 17, 2015

@AllenOtis @johnnyt so can we close this issue now? I'm unclear as to the status.

@johnnyt
Copy link
Member

johnnyt commented Feb 17, 2015

@jc00ke - I think we should get the new installer merged in ( #370 ) and then I'll try to get this one ready to merge in too.

@zenspider
Copy link
Contributor

@johnnyt I need the path cleared to get this up and running in order to do my work. Please get #370 done and merged.

@zenspider
Copy link
Contributor

@AllenOtis 34619 is 404'ing right now. If I try to update to the version used by master, I get filein errors because of primitive mismatches. Please advise.

@AllenOtis
Copy link
Contributor Author

Ryan,

If you are trying to merge the libmagparse work into the master, then
you have to change master to use the 34619 server download as part of that
merge. Otherwise please send excerpt of your filein logs showing the topaz
banner
and the filein errors.

After that we can schedule a phone call to discuss later this week.

Allen

On Mon, Mar 16, 2015 at 1:41 PM, Ryan Davis [email protected]
wrote:

@AllenOtis https://github.com/AllenOtis 34619 is 404'ing right now. If
I try to update to the version used by master, I get filein errors because
of primitive mismatches. Please advise.


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

No, I'm just trying to use your branch at this point and can't because the tarball doesn't exist:

% curl -I http://seaside.gemtalksystems.com/maglev/GemStone-34619.Darwin-i386.tar.gz
HTTP/1.1 404 Not Found
Content-Type: text/html
Content-Length: 345
Date: Mon, 16 Mar 2015 21:01:58 GMT
Server: lighttpd/1.4.28

@AllenOtis
Copy link
Contributor Author

You need to do the development on Linux.
We have not yet published any 34619 server for Darwin .

Allen

On Mon, Mar 16, 2015 at 2:02 PM, Ryan Davis [email protected]
wrote:

No, I'm just trying to use your branch at this point and can't because the
tarball doesn't exist:

% curl -I http://seaside.gemtalksystems.com/maglev/GemStone-34619.Darwin-i386.tar.gz
HTTP/1.1 http://seaside.gemtalksystems.com/maglev/GemStone-34619.Darwin-i386.tar.gzHTTP/1.1 404 Not Found
Content-Type: text/html
Content-Length: 345
Date: Mon, 16 Mar 2015 21:01:58 GMT
Server: lighttpd/1.4.28


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

On Mar 16, 2015, at 14:04, Allen Otis [email protected] wrote:

You need to do the development on Linux.
We have not yet published any 34619 server for Darwin .

Then should I try merging master to your branch so I get an up to date version for osx or is that cut of the server the only one with your changes for the parser?

@AllenOtis
Copy link
Contributor Author

Ryan, Johnny,

You have to use the 34619 server build with the libmagparse branch.

I was hoping we could get the libmagparse work integrated on Linux first,
and after it
is completely working we will publish a new Darwin server , and if needed
another
Linux server build to go with it .

Allen

On Mon, Mar 16, 2015 at 2:08 PM, Ryan Davis [email protected]
wrote:

On Mar 16, 2015, at 14:04, Allen Otis [email protected] wrote:

You need to do the development on Linux.
We have not yet published any 34619 server for Darwin .

Then should I try merging master to your branch so I get an up to date
version for osx or is that cut of the server the only one with your changes
for the parser?


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

OK. I'm going to leave that to @johnnyt since he's working on installer changes and I'll focus on the parser changes and work blind for now. Once @johnnyt has this up and running, we'll work towards merging and then towards getting a darwin build available. I have enough tooling that I can just focus on the parser structural changes to get it up to 1.9.

@johnnyt
Copy link
Member

johnnyt commented Mar 16, 2015

@AllenOtis -

I'm trying to get this running on an Ubuntu 12.04 VM and am getting the following error:

vagrant@precise64:~/maglev$ ./bin/maglev start-debug
startstone[Info]: GemStone version '3.1.0.2.1'
startstone[Info]: Starting Stone repository monitor 'maglev'.
startstone[Info]: GEMSTONE is: '/home/vagrant/maglev/gemstone'.
startstone[Info]:
    GEMSTONE_SYS_CONF=/home/vagrant/maglev/etc/system-debug.conf
    GEMSTONE_EXE_CONF=/home/vagrant/maglev/maglev.conf
startstone[Warning]: Using GEMSTONE_LOG environment variable to set log file.
startstone[Info]: Log file is '/home/vagrant/maglev/log/maglev/maglev.log'.
startstone[Info]: GemStone server 'maglev' has been started, process 23473 .
 _____________________________________________________________________________
|             GemStone/S64 Object-Oriented Data Management System             |
|                    Copyright (C) VMware, Inc. 1986-2014                     |
|                            All rights reserved.                             |
+-----------------------------------------------------------------------------+
|    PROGRAM: WAITSTONE, GemStone Remote Process Utility                      |
|    VERSION: 3.1.0.2.1, Beta  Tue Oct 21 16:16:34 2014                       |
|      BUILD: gss64_maglev_1_9_branch-34619                                   |
|  BUILT FOR: x86-64 (Linux)                                                  |
|       MODE: 64 bit                                                          |
| RUNNING ON: 4-CPU precise64 x86_64 (Linux 3.2.0-23-generic #36-Ubuntu SMP Tue
| Apr 10 20:39:51 UTC 2012) 3954MB                                            |
| PROCESS ID: 23496     DATE: 03/16/2015 23:35:55 UTC                         |
|   USER IDS: REAL=vagrant (1000) EFFECTIVE=vagrant (1000)                    |
 _____________________________________________________________________________
|   GEMSTONE_GLOBAL_DIR = /home/vagrant/maglev                                |
|_____________________________________________________________________________|
Network service !#server!maglev is ready.
Loading kernel. It may take a few seconds...
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei
FAILED LOADING PRIMS: 127 See /home/vagrant/maglev/log/maglev/allprims.log
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei

Does that _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei look familiar?

@AllenOtis
Copy link
Contributor Author

Johnny,

First step would be

cd $GEMSTONE/lib
nm -A --demangle *.so |grep DoPrimCheckNumArgs

And if libmagparse is not in $GEMSTONE/lib,
need to apply the nm ... to it also .
That should show what library has an undefined reference .

You might also run the "loading kernel" or whatever in a manner that shows
the topaz banner (omit -q from the topaz args if needed)
to double check the version of topaz ... and to also get a printout of
the loading of the shared libraries into topaz process.

I'm wondering what has changed since we last debugged stuff together.

Allen

On Mon, Mar 16, 2015 at 4:38 PM, JohnnyT [email protected] wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this running on an Ubuntu 12.04 VM and am getting the
following error:

vagrant@precise64:~/maglev$ ./bin/maglev start-debug
startstone[Info]: GemStone version '3.1.0.2.1'
startstone[Info]: Starting Stone repository monitor 'maglev'.
startstone[Info]: GEMSTONE is: '/home/vagrant/maglev/gemstone'.
startstone[Info]:
GEMSTONE_SYS_CONF=/home/vagrant/maglev/etc/system-debug.conf
GEMSTONE_EXE_CONF=/home/vagrant/maglev/maglev.conf
startstone[Warning]: Using GEMSTONE_LOG environment variable to set log file.
startstone[Info]: Log file is '/home/vagrant/maglev/log/maglev/maglev.log'.
startstone[Info]: GemStone server 'maglev' has been started, process 23473 .
_____________________________________________________________________________| GemStone/S64 Object-Oriented Data Management System || Copyright (C) VMware, Inc. 1986-2014 || All rights reserved. |
+-----------------------------------------------------------------------------+| PROGRAM: WAITSTONE, GemStone Remote Process Utility || VERSION: 3.1.0.2.1, Beta Tue Oct 21 16:16:34 2014 || BUILD: gss64_maglev_1_9_branch-34619 || BUILT FOR: x86-64 (Linux) || MODE: 64 bit || RUNNING ON: 4-CPU precise64 x86_64 (Linux 3.2.0-23-generic #36-Ubuntu SMP Tue| Apr 10 20:39:51 UTC 2012) 3954MB || PROCESS ID: 23496 DATE: 03/16/2015 23:35:55 UTC || USER IDS: REAL=vagrant (1000) EFFECTIVE=vagrant (1000) |
| GEMSTONE_GLOBAL_DIR = /home/vagrant/maglev |||
Network service !#server!maglev is ready.
Loading kernel. It may take a few seconds...
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei
FAILED LOADING PRIMS: 127 See /home/vagrant/maglev/log/maglev/allprims.log
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei


Reply to this email directly or view it on GitHub
#352 (comment).

@AllenOtis
Copy link
Contributor Author

Johnny,
Looking at the shared libs that I have here,
it looks like DoPrimCheckNumArgs is only defined and used in the debuggable
libs, i.e. those compiled with FLG_DEBUG defined .

Not sure what type of build we last debugged with on your Linux VM .

However I would suspect your $GEMSTONE is from the released server (
FLG_DEBUG not defined)
and perhaps libmagparse was built with FLG_DEBUG defined ...

Allen

On Mon, Mar 16, 2015 at 4:38 PM, JohnnyT [email protected] wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this running on an Ubuntu 12.04 VM and am getting the
following error:

vagrant@precise64:~/maglev$ ./bin/maglev start-debug
startstone[Info]: GemStone version '3.1.0.2.1'
startstone[Info]: Starting Stone repository monitor 'maglev'.
startstone[Info]: GEMSTONE is: '/home/vagrant/maglev/gemstone'.
startstone[Info]:
GEMSTONE_SYS_CONF=/home/vagrant/maglev/etc/system-debug.conf
GEMSTONE_EXE_CONF=/home/vagrant/maglev/maglev.conf
startstone[Warning]: Using GEMSTONE_LOG environment variable to set log file.
startstone[Info]: Log file is '/home/vagrant/maglev/log/maglev/maglev.log'.
startstone[Info]: GemStone server 'maglev' has been started, process 23473 .
_____________________________________________________________________________| GemStone/S64 Object-Oriented Data Management System || Copyright (C) VMware, Inc. 1986-2014 || All rights reserved. |
+-----------------------------------------------------------------------------+| PROGRAM: WAITSTONE, GemStone Remote Process Utility || VERSION: 3.1.0.2.1, Beta Tue Oct 21 16:16:34 2014 || BUILD: gss64_maglev_1_9_branch-34619 || BUILT FOR: x86-64 (Linux) || MODE: 64 bit || RUNNING ON: 4-CPU precise64 x86_64 (Linux 3.2.0-23-generic #36-Ubuntu SMP Tue| Apr 10 20:39:51 UTC 2012) 3954MB || PROCESS ID: 23496 DATE: 03/16/2015 23:35:55 UTC || USER IDS: REAL=vagrant (1000) EFFECTIVE=vagrant (1000) |
| GEMSTONE_GLOBAL_DIR = /home/vagrant/maglev |||
Network service !#server!maglev is ready.
Loading kernel. It may take a few seconds...
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei
FAILED LOADING PRIMS: 127 See /home/vagrant/maglev/log/maglev/allprims.log
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

The instructions say to use -DFLG_DEBUG=1. I'm going to need that to debug parser changes I'm making so I can trace through a parse. Is there a debug build of the server? Or should we cut that call out for now?

On Mar 16, 2015, at 17:05, Allen Otis [email protected] wrote:

Johnny,
Looking at the shared libs that I have here,
it looks like DoPrimCheckNumArgs is only defined and used in the debuggable
libs, i.e. those compiled with FLG_DEBUG defined .

Not sure what type of build we last debugged with on your Linux VM .

However I would suspect your $GEMSTONE is from the released server (
FLG_DEBUG not defined)
and perhaps libmagparse was built with FLG_DEBUG defined ...

Allen

On Mon, Mar 16, 2015 at 4:38 PM, JohnnyT [email protected] wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this running on an Ubuntu 12.04 VM and am getting the
following error:

vagrant@precise64:~/maglev$ ./bin/maglev start-debug
startstone[Info]: GemStone version '3.1.0.2.1'
startstone[Info]: Starting Stone repository monitor 'maglev'.
startstone[Info]: GEMSTONE is: '/home/vagrant/maglev/gemstone'.
startstone[Info]:
GEMSTONE_SYS_CONF=/home/vagrant/maglev/etc/system-debug.conf
GEMSTONE_EXE_CONF=/home/vagrant/maglev/maglev.conf
startstone[Warning]: Using GEMSTONE_LOG environment variable to set log file.
startstone[Info]: Log file is '/home/vagrant/maglev/log/maglev/maglev.log'.
startstone[Info]: GemStone server 'maglev' has been started, process 23473 .
_____________________________________________________________________________| GemStone/S64 Object-Oriented Data Management System || Copyright (C) VMware, Inc. 1986-2014 || All rights reserved. |
+-----------------------------------------------------------------------------+| PROGRAM: WAITSTONE, GemStone Remote Process Utility || VERSION: 3.1.0.2.1, Beta Tue Oct 21 16:16:34 2014 || BUILD: gss64_maglev_1_9_branch-34619 || BUILT FOR: x86-64 (Linux) || MODE: 64 bit || RUNNING ON: 4-CPU precise64 x86_64 (Linux 3.2.0-23-generic #36-Ubuntu SMP Tue| Apr 10 20:39:51 UTC 2012) 3954MB || PROCESS ID: 23496 DATE: 03/16/2015 23:35:55 UTC || USER IDS: REAL=vagrant (1000) EFFECTIVE=vagrant (1000) |
| GEMSTONE_GLOBAL_DIR = /home/vagrant/maglev |||
Network service !#server!maglev is ready.
Loading kernel. It may take a few seconds...
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei
FAILED LOADING PRIMS: 127 See /home/vagrant/maglev/log/maglev/allprims.log
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error: /home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei


Reply to this email directly or view it on GitHub
#352 (comment).


Reply to this email directly or view it on GitHub.

@AllenOtis
Copy link
Contributor Author

I would make the following edit in rubyom.hf so you can compile
libmagparse
with or without FLG_DEBUG
I don't see any other dependencies from FLG_DEBUG code in libmagparse to
the libgcinlk.

I've got a bunch of documentation comments in my checkout of rubyom.hf now
;
need to test before I checkin, so would you please install the edit below
in your checkout.

 #if defined(FLG_DEBUG)
-void DoPrimCheckNumArgs(om *omPtr, omObjSType **SP, int expectedNumArg);
-#define DOPRIM_ARGS(__omptr, __n) DoPrimCheckNumArgs(__omptr, ARStackPtr,
__n)
+// reenable if doing lots of new primitives in the parser; needs a slow
build
+// void DoPrimCheckNumArgs(om *omPtr, omObjSType **SP, int expectedNumArg);
+// define DOPRIM_ARGS(__omptr, __n) DoPrimCheckNumArgs(__omptr,
ARStackPtr, __n)
+#define DOPRIM_ARGS(__omptr, __n) { }
 #else
 #define DOPRIM_ARGS(__omptr, __n) { }
 #endif

On Mon, Mar 16, 2015 at 5:20 PM, Ryan Davis [email protected]
wrote:

The instructions say to use -DFLG_DEBUG=1. I'm going to need that to
debug parser changes I'm making so I can trace through a parse. Is there a
debug build of the server? Or should we cut that call out for now?

On Mar 16, 2015, at 17:05, Allen Otis [email protected] wrote:

Johnny,
Looking at the shared libs that I have here,
it looks like DoPrimCheckNumArgs is only defined and used in the
debuggable
libs, i.e. those compiled with FLG_DEBUG defined .

Not sure what type of build we last debugged with on your Linux VM .

However I would suspect your $GEMSTONE is from the released server (
FLG_DEBUG not defined)
and perhaps libmagparse was built with FLG_DEBUG defined ...

Allen

On Mon, Mar 16, 2015 at 4:38 PM, JohnnyT [email protected]
wrote:

@AllenOtis https://github.com/AllenOtis -

I'm trying to get this running on an Ubuntu 12.04 VM and am getting the
following error:

vagrant@precise64:~/maglev$ ./bin/maglev start-debug
startstone[Info]: GemStone version '3.1.0.2.1'
startstone[Info]: Starting Stone repository monitor 'maglev'.
startstone[Info]: GEMSTONE is: '/home/vagrant/maglev/gemstone'.
startstone[Info]:
GEMSTONE_SYS_CONF=/home/vagrant/maglev/etc/system-debug.conf
GEMSTONE_EXE_CONF=/home/vagrant/maglev/maglev.conf
startstone[Warning]: Using GEMSTONE_LOG environment variable to set
log file.
startstone[Info]: Log file is
'/home/vagrant/maglev/log/maglev/maglev.log'.
startstone[Info]: GemStone server 'maglev' has been started, process
23473 .

_____________________________________________________________________________|
GemStone/S64 Object-Oriented Data Management System || Copyright (C)
VMware, Inc. 1986-2014 || All rights reserved. |

+-----------------------------------------------------------------------------+|
PROGRAM: WAITSTONE, GemStone Remote Process Utility || VERSION: 3.1.0.2.1,
Beta Tue Oct 21 16:16:34 2014 || BUILD: gss64_maglev_1_9_branch-34619 ||
BUILT FOR: x86-64 (Linux) || MODE: 64 bit || RUNNING ON: 4-CPU precise64
x86_64 (Linux 3.2.0-23-generic #36-Ubuntu SMP Tue| Apr 10 20:39:51 UTC
2012) 3954MB || PROCESS ID: 23496 DATE: 03/16/2015 23:35:55 UTC || USER
IDS: REAL=vagrant (1000) EFFECTIVE=vagrant (1000) |

|
GEMSTONE_GLOBAL_DIR = /home/vagrant/maglev
||
|
Network service !#server!maglev is ready.
Loading kernel. It may take a few seconds...
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error:
/home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined
symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei
FAILED LOADING PRIMS: 127 See
/home/vagrant/maglev/log/maglev/allprims.log
/home/vagrant/maglev/gemstone/bin/topaz: symbol lookup error:
/home/vagrant/maglev/gemstone/lib/libmagparse-3.1.0.2.1-64.so: undefined
symbol: _Z18DoPrimCheckNumArgsP2omPP10omObjSTypei


Reply to this email directly or view it on GitHub
#352 (comment).


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#352 (comment).

@johnnyt
Copy link
Member

johnnyt commented Mar 17, 2015

@AllenOtis - I was compiling the parser with FLG_DEBUG - that was the problem.

I also added src/kernel/parser/parser.sh to help with the process

@johnnyt
Copy link
Member

johnnyt commented Mar 17, 2015

Allrighty - the allen/libmagparse branch has an extra step to build the parser during the bootstrap process.

The InvokeTest.rb is failing on Travis: https://travis-ci.org/MagLev/maglev/jobs/54754268 - which I need to look into, but other than that its looking pretty good.

Eventually - we should build and save a precompiled libmagparse.so in the repo.

@AllenOtis - will we need a different version for Darwin? and what is the process for getting a new darwin server?

@zenspider
Copy link
Contributor

We shouldn't be checking in generated code/binaries into the repo. The parser already has a ton of generated files checked in that need to be removed and marked as ignored. Let's set it up to build from scratch and have proper dependencies in the build system.

We basically need a darwin build of the server before we can merge this to master, don't we?

@AllenOtis
Copy link
Contributor Author

We should have a Darwin build (revision 34619) available by the end of the
week.
Allen

On Tue, Mar 17, 2015 at 2:43 PM, Ryan Davis [email protected]
wrote:

We shouldn't be checking in generated code/binaries into the repo. The
parser already has a ton of generated files checked in that need to be
removed and marked as ignored. Let's set it up to build from scratch and
have proper dependencies in the build system.

We basically need a darwin build of the server before we can merge this to
master, don't we?


Reply to this email directly or view it on GitHub
#352 (comment).

@AllenOtis
Copy link
Contributor Author

@zenspider
Copy link
Contributor

Has anyone gotten a maglev build (on @AllenOtis' parser_fixes branch) against this server to work yet?

Master builds fine for me. But the parser_fixes branch (and my branch off of that) does not.

I get missing headers for unicode stuff:

#include "unicode/ustring.h"
#include "unicode/umachine.h"
#include "unicode/utf.h"

If I bypass that, I get failures for a BUNCH of fields:

Compiling rubygrammar.o
rubygrammar.c: In function 'void heredoc_restore(rb_parse_state*)':
rubygrammar.c:503:64: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
         ps->lineNumber, ps->lineStartOffset, ps->tokenOffset() );
                                                                ^
rubygrammar.c:503:64: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c: In function 'int tokadd_utf8(rb_parse_state*, int, int, int)':
rubygrammar.c:6385:56: error: 'U8_APPEND_UNSAFE' was not declared in this scope
         U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
                                                        ^
rubygrammar.c:6415:56: error: 'U8_APPEND_UNSAFE' was not declared in this scope
         U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
                                                        ^
rubygrammar.c: In function 'int here_document(NODE**, rb_parse_state*)':
rubygrammar.c:6968:62: error: format '%ld' expects argument of type 'long int', but argument 5 has type 'int64 {aka long long int}' [-Werror=format=]
         ps->lineNumber, ps->lineStartOffset, eos, len, indent);
                                                              ^
rubygrammar.c:6968:62: error: format '%ld' expects argument of type 'long int', but argument 5 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c: In function 'int yyparse(rb_parse_state*)':
rubygrammar.c:9078:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
                     YYPREFIX, yystate, yychar, yys);
                                                   ^
rubygrammar.c:9078:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
                     YYPREFIX, yystate, new_state );
                                                  ^
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9145:65: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
                            YYPREFIX, yymarkPtr->state, new_state);
                                                                 ^
rubygrammar.c:9145:65: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9186:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
                     YYPREFIX, yystate, yychar, yys);
                                                   ^
rubygrammar.c:9186:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
                 YYPREFIX, yystate, yyn, yyrule[yyn]);
                                                    ^
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:12373:55: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
                YYPREFIX, yystack->mark->state, yystate);
                                                       ^
rubygrammar.c:12373:55: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
cc1plus: all warnings being treated as errors
compiling rubygrammar.o failed

That's using gcc-4.9. If I use OSX bundled g++, I get:

Compiling rubygrammar.o
rubygrammar.c:503:46: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
        ps->lineNumber, ps->lineStartOffset, ps->tokenOffset() );
                                             ^~~~~~~~~~~~~~~~~
rubygrammar.c:6385:9: error: use of undeclared identifier 'U8_APPEND_UNSAFE'
        U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
        ^
rubygrammar.c:6415:9: error: use of undeclared identifier 'U8_APPEND_UNSAFE'
        U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
        ^
rubygrammar.c:6968:51: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
        ps->lineNumber, ps->lineStartOffset, eos, len, indent);
                                                  ^~~
rubygrammar.c:9078:31: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
                    YYPREFIX, yystate, yychar, yys);
                              ^~~~~~~
rubygrammar.c:9093:31: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
                    YYPREFIX, yystate, new_state );
                              ^~~~~~~
rubygrammar.c:9093:40: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
                    YYPREFIX, yystate, new_state );
                                       ^~~~~~~~~
rubygrammar.c:9145:56: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
                           YYPREFIX, yymarkPtr->state, new_state);
                                                       ^~~~~~~~~
rubygrammar.c:9186:31: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
                    YYPREFIX, yystate, yychar, yys);
                              ^~~~~~~
rubygrammar.c:9197:27: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
                YYPREFIX, yystate, yyn, yyrule[yyn]);
                          ^~~~~~~
rubygrammar.c:9197:36: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
                YYPREFIX, yystate, yyn, yyrule[yyn]);
                                   ^~~
rubygrammar.c:12373:48: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
               YYPREFIX, yystack->mark->state, yystate);
                                               ^~~~~~~
12 errors generated.
compiling rubygrammar.o failed

I'm not sure what tooling or packages I'm missing and I see no doco specifying what versions of what compilers should or should not work.

@zenspider
Copy link
Contributor

I should add, against the apple-supplied compiler, -fcheck-new isn't supported. Seems to work fine with gcc-4.9 tho.

@AllenOtis
Copy link
Contributor Author

Are you trying to compile on Linux or Darwin ?

On Fri, Mar 20, 2015 at 3:30 PM, Ryan Davis [email protected]
wrote:

Has anyone gotten a maglev build (on @AllenOtis
https://github.com/AllenOtis' parser_fixes branch) against this server
to work yet?

Master builds fine for me. But the parser_fixes branch (and my branch off
of that) does not.

I get missing headers for unicode stuff:

#include "unicode/ustring.h"
#include "unicode/umachine.h"
#include "unicode/utf.h"

If I bypass that, I get failures for a BUNCH of fields:

Compiling rubygrammar.o
rubygrammar.c: In function 'void heredoc_restore(rb_parse_state_)':
rubygrammar.c:503:64: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
ps->lineNumber, ps->lineStartOffset, ps->tokenOffset() );
^
rubygrammar.c:503:64: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c: In function 'int tokadd_utf8(rb_parse_state_, int, int, int)':
rubygrammar.c:6385:56: error: 'U8_APPEND_UNSAFE' was not declared in this scope
U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
^
rubygrammar.c:6415:56: error: 'U8_APPEND_UNSAFE' was not declared in this scope
U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
^
rubygrammar.c: In function 'int here_document(NODE*, rb_parse_state)':
rubygrammar.c:6968:62: error: format '%ld' expects argument of type 'long int', but argument 5 has type 'int64 {aka long long int}' [-Werror=format=]
ps->lineNumber, ps->lineStartOffset, eos, len, indent);
^
rubygrammar.c:6968:62: error: format '%ld' expects argument of type 'long int', but argument 5 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c: In function 'int yyparse(rb_parse_state*)':
rubygrammar.c:9078:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
YYPREFIX, yystate, yychar, yys);
^
rubygrammar.c:9078:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
YYPREFIX, yystate, new_state );
^
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9093:50: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9145:65: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
YYPREFIX, yymarkPtr->state, new_state);
^
rubygrammar.c:9145:65: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9186:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
YYPREFIX, yystate, yychar, yys);
^
rubygrammar.c:9186:51: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
YYPREFIX, yystate, yyn, yyrule[yyn]);
^
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:9197:52: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
rubygrammar.c:12373:55: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
YYPREFIX, yystack->mark->state, yystate);
^
rubygrammar.c:12373:55: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'int64 {aka long long int}' [-Werror=format=]
cc1plus: all warnings being treated as errors
compiling rubygrammar.o failed

That's using gcc-4.9. If I use OSX bundled g++, I get:

Compiling rubygrammar.o
rubygrammar.c:503:46: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
ps->lineNumber, ps->lineStartOffset, ps->tokenOffset() );
^~~~~~~~~~~~~~~~~
rubygrammar.c:6385:9: error: use of undeclared identifier 'U8_APPEND_UNSAFE'
U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
^
rubygrammar.c:6415:9: error: use of undeclared identifier 'U8_APPEND_UNSAFE'
U8_APPEND_UNSAFE(dst, offset_counter, codepoint);
^
rubygrammar.c:6968:51: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
ps->lineNumber, ps->lineStartOffset, eos, len, indent);
^~~
rubygrammar.c:9078:31: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yystate, yychar, yys);
^~~~~~~
rubygrammar.c:9093:31: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yystate, new_state );
^~~~~~~
rubygrammar.c:9093:40: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yystate, new_state );
^~~~~~~~~
rubygrammar.c:9145:56: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yymarkPtr->state, new_state);
^~~~~~~~~
rubygrammar.c:9186:31: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yystate, yychar, yys);
^~~~~~~
rubygrammar.c:9197:27: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yystate, yyn, yyrule[yyn]);
^~~~~~~
rubygrammar.c:9197:36: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yystate, yyn, yyrule[yyn]);
^~~
rubygrammar.c:12373:48: error: format specifies type 'long' but the argument has type 'int64' (aka 'long long') [-Werror,-Wformat]
YYPREFIX, yystack->mark->state, yystate);
^~~~~~~
12 errors generated.
compiling rubygrammar.o failed

I'm not sure what tooling or packages I'm missing and I see no doco
specifying what versions of what compilers should or should not work.


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

Darwin.

@AllenOtis
Copy link
Contributor Author

I haven't compile the libmagparse branch on Darwin, and
the Maglev Darwin server just released was built on
Darwin Kernel Version 10.8.0. We haven't releases a server
built on a newer Darwin release yet; we do have one in test, but
are seeing some pretty bad C optimizer problems when
building on Darwin Kernel Version 13.4.0. We will try
building on a system newer than 13.4.0 next month.

You definitely need libicu packages installed .
And make sure you are compiling for 64bit .

Allen

On Fri, Mar 20, 2015 at 4:58 PM, Ryan Davis [email protected]
wrote:

Darwin.


Reply to this email directly or view it on GitHub
#352 (comment).

@AllenOtis
Copy link
Contributor Author

typical compile lines for the server on Darwin 13.4.0 are as follows.

Current directory is where the .o files will go
Additional -I args might be needed.

For debugging

/usr/bin/g++ -fmessage-length=0 -Wno-format -Wtrigraphs -Wcomment
-Wsystem-headers -Wtrigraphs -Wno-aggregate-return -Wswitch -Wshadow
-Wunused-value -Wunused-variable -Wunused-label -Wno-unused-function
-Wchar-subscripts -Wno-conversion -Wmissing-braces -Wmultichar
-Wparentheses -Wsign-compare -Wsign-promo -Wwrite-strings -Wreturn-type
-Wno-c++11-extensions -Wno-deprecated-declarations -Wno-self-assign-field
-Wno-invalid-offsetof -Wno-constant-logical-operand -g -DFLG_DEBUG=1 -m64
-pipe -D_XOPEN_SOURCE -D_REENTRANT -D_GNU_SOURCE -fPIC
-fno-strict-aliasing -D_LARGEFILE64_SOURCE -pipe -D_REENTRANT -fPIC
-I/stever-mbp1/users/otisa/st64_32x/src -x c++ -c
/stever-mbp1/users/otisa/st64_32x/src/strprim.c -o strprim.o

And with optmization

/usr/bin/g++ -DOBJ_pgsvr -fmessage-length=0 -Wno-format -Wtrigraphs
-Wcomment -Wsystem-headers -Wtrigraphs -Wno-aggregate-return -Wswitch
-Wshadow -Wunused-value -Wunused-variable -Wunused-label
-Wno-unused-function -Wchar-subscripts -Wno-conversion -Wmissing-braces
-Wmultichar -Wparentheses -Wsign-compare -Wsign-promo -Wwrite-strings
-Wreturn-type -Wno-c++11-extensions -Wno-deprecated-declarations
-Wno-self-assign-field -Wno-invalid-offsetof -Wno-constant-logical-operand
-O2 -ggdb -DFLG_FAST=1 -m64 -pipe -D_XOPEN_SOURCE -D_REENTRANT
-D_GNU_SOURCE -fPIC -fno-strict-aliasing -D_LARGEFILE64_SOURCE -pipe
-D_REENTRANT -fPIC -I/stever-mbp1/users/otisa/st64_32x/src -x c++ -c
/stever-mbp1/users/otisa/st64_32x/src/strprim.c -o strprim.o
~

On Fri, Mar 20, 2015 at 5:14 PM, Allen Otis [email protected]
wrote:

I haven't compile the libmagparse branch on Darwin, and
the Maglev Darwin server just released was built on
Darwin Kernel Version 10.8.0. We haven't releases a server
built on a newer Darwin release yet; we do have one in test, but
are seeing some pretty bad C optimizer problems when
building on Darwin Kernel Version 13.4.0. We will try
building on a system newer than 13.4.0 next month.

You definitely need libicu packages installed .
And make sure you are compiling for 64bit .

Allen

On Fri, Mar 20, 2015 at 4:58 PM, Ryan Davis [email protected]
wrote:

Darwin.


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

I've got my parser changes into a compilable, but not linkable, state. @johnnyt can you take a whack at it?

@johnnyt
Copy link
Member

johnnyt commented Mar 24, 2015

I'm trying it out - but am getting probably the same stuff you did @zenspider:

+ echo 'Linking libmagparse.so'
Linking libmagparse.so
$CC \
  -shared \
  -m64 -lpthread -ldl -lc -lm -o libmagparse.so \
  -stdlib=libstdc++ \
  -L$GEMSTONE/lib \
  -lgcilnk-$GSVERSION \
  rubyast.o rubygrammar.o
+ /usr/bin/g++ -shared -m64 -lpthread -ldl -lc -lm -o libmagparse.so -stdlib=libstdc++ -L/Users/johnnyt/.rbenv/versions/maglev-2.0.0-dev/gemstone/lib -lgcilnk-3.1.0.2.1-64 rubyast.o rubygrammar.o
Undefined symbols for architecture x86_64:
  "GemErrAnsi(om*, int, char const*, char const*)", referenced from:
      MagParse903(om*, omObjSType**) in rubygrammar.o
      sessionInit(om*, rb_parse_state*) in rubygrammar.o
      resolveAstClass(om*, omObjSType**, AstClassEType) in rubygrammar.o
      initAstSelector(om*, unsigned long long*, AstSelectorEType) in rubygrammar.o
      initAstSymbol(om*, omObjSType**, AstSymbolEType) in rubygrammar.o
  "ComHeapInit(ComStateType*)", referenced from:
      MagParse903(om*, omObjSType**) in rubygrammar.o
  "ComHeapMalloc(ComStateType*, unsigned int)", referenced from:
      VarTable::allocate(rb_parse_state*, int) in rubyast.o
      LocalState::allocate(rb_parse_state*) in rubyast.o
      bstring::balloc(bstring*, long long, rb_parse_state*) in rubyast.o
      bstring::new_(rb_parse_state*) in rubyast.o
      StartPositionList::allocate(rb_parse_state*, int) in rubyast.o
      MagCompileError902(om*, omObjSType**) in rubygrammar.o
      MagParse903(om*, omObjSType**) in rubygrammar.o
      ...
  "GemSupErr_oo_(int, omObjSType*, omObjSType*)", referenced from:
      RubyNode::checkInstanceOf(omObjSType**, AstClassEType, rb_parse_state*) in rubyast.o
  "LrgInt64ToOop(om*, long long)", referenced from:
      RubyLexStrTerm::newHereDoc(rb_parse_state*, char const*, long long, long long, bstring*) in rubygrammar.o
      RubyAbstractNumberNode::s(long long, rb_parse_state*) in rubygrammar.o
  "FloatPrimFetchArg(om*, omObjSType**, double*)", referenced from:
      yyparse(rb_parse_state*) in rubygrammar.o
  "GemDoSessionSymList(om*)", referenced from:
      resolveAstClass(om*, omObjSType**, AstClassEType) in rubygrammar.o
  "GemSupSearchSymList(om*, omObjSType**, omObjSType**)", referenced from:
      resolveAstClass(om*, omObjSType**, AstClassEType) in rubygrammar.o
  "FloatPrimDoubleToOop(om*, double)", referenced from:
      yyparse(rb_parse_state*) in rubygrammar.o
      yylex(rb_parse_state*) in rubygrammar.o
  "HostCallDebuggerMsg_fl(char const*, char const*, int)", referenced from:
      RpNameToken::s(omObjSType**, omObjSType*, rb_parse_state*) in rubyast.o
      RpNameToken::s(AstSymbolEType, omObjSType*, rb_parse_state*) in rubyast.o
      RpNameToken::s(rb_parse_state*, omObjSType*) in rubyast.o
      OOP_OF_SMALL_LONG_(long long) in rubyast.o
      VarTable::allocate(rb_parse_state*, int) in rubyast.o
      OOP_TO_BIT(unsigned long long) in rubyast.o
      RpNameToken::buildQuid(unsigned long long, long long, long long) in rubyast.o
      ...
  "LrgRubyStringToInteger(om*, omObjSType**, long long, int)", referenced from:
      newInteger(rb_parse_state*, int) in rubygrammar.o
  "ObjCanonicalSymFromCStr(om*, unsigned char const*, long long, unsigned long long)", referenced from:
      rb_parser_sym(char const*, rb_parse_state*) in rubygrammar.o
  "ObjExistingCanonicalSym__(om*, omObjSType**)", referenced from:
      initAstSelector(om*, unsigned long long*, AstSelectorEType) in rubygrammar.o
  "LrgNegate(om*, omObjSType**)", referenced from:
      yyparse(rb_parse_state*) in rubygrammar.o
  "ObjNewSym(om*, char const*)", referenced from:
      RpNameToken::s(char const*, omObjSType*, rb_parse_state*) in rubyast.o
      yyparse(rb_parse_state*) in rubygrammar.o
      resolveAstClass(om*, omObjSType**, AstClassEType) in rubygrammar.o
      initAstSymbol(om*, omObjSType**, AstSymbolEType) in rubygrammar.o
  "OmScopeType::initialize_(om*)", referenced from:
      OmScopeType::OmScopeType(om*) in rubyast.o
      OmScopeType::OmScopeType(om*) in rubygrammar.o
  "OmScopeType::deinitialize_()", referenced from:
      OmScopeType::~OmScopeType() in rubyast.o
      OmScopeType::~OmScopeType() in rubygrammar.o
  "OmScopeType::add_(omObjSType*)", referenced from:
      OmScopeType::add(omObjSType*) in rubyast.o
      OmScopeType::add(omObjSType*) in rubygrammar.o
      OmScopeType::newHandle() in rubygrammar.o
  "om::FetchCData(omObjSType*)", referenced from:
      MagParse903(om*, omObjSType**) in rubygrammar.o
  "om::FetchSize_(omObjSType*)", referenced from:
      MagCompileError902(om*, omObjSType**) in rubygrammar.o
      MagParse903(om*, omObjSType**) in rubygrammar.o
      here_document(omObjSType**, rb_parse_state*) in rubygrammar.o
      heredoc_restore(rb_parse_state*) in rubygrammar.o
  "om::corruptObj(omObjSType*, char const*)", referenced from:
      om::StoreSmallInt_(om*, omObjSType**, long long, long long) in rubyast.o
      om::StoreSmallInt_(om*, omObjSType**, long long, long long) in rubygrammar.o
  "om::FetchBytes_(omObjSType*, long long, long long, unsigned char*)", referenced from:
      heredoc_restore(rb_parse_state*) in rubygrammar.o
  "om::NewString__(om*, unsigned char const*, long long)", referenced from:
      RubyLexStrTerm::newHereDoc(rb_parse_state*, char const*, long long, long long, bstring*) in rubygrammar.o
      yylex(rb_parse_state*) in rubygrammar.o
      NEW_STR(char const*, long long, rb_parse_state*) in rubygrammar.o
      newInteger(rb_parse_state*, int) in rubygrammar.o
      om::NewString(om*, long long) in rubygrammar.o
      om::NewString_(om*, char const*) in rubygrammar.o
  "om::strCharSize(omObjSType*)", referenced from:
      MagCompileError902(om*, omObjSType**) in rubygrammar.o
      MagParse903(om*, omObjSType**) in rubygrammar.o
  "om::LocatePomObj(om*, unsigned long long)", referenced from:
      RpNameToken::s(rb_parse_state*, omObjSType*) in rubyast.o
      quidToSymbolObj(omObjSType*, rb_parse_state*) in rubygrammar.o
  "om::isCByteArray(omObjSType*)", referenced from:
      MagParse903(om*, omObjSType**) in rubygrammar.o
  "om::objIdOfObj__(om*, omObjSType*)", referenced from:
      RpNameToken::quidForSym(omObjSType*, om*) in rubyast.o
      rb_parser_sym(char const*, rb_parse_state*) in rubygrammar.o
      initAstSelector(om*, unsigned long long*, AstSelectorEType) in rubygrammar.o
  "om::AppendToArray(om*, omObjSType**, omObjSType**)", referenced from:
      yylex(rb_parse_state*) in rubygrammar.o
  "om::FetchCString_(omObjSType*, char*, long long)", referenced from:
      MagCompileError902(om*, omObjSType**) in rubygrammar.o
      MagParse903(om*, omObjSType**) in rubygrammar.o
      rb_compile_error_q(rb_parse_state*, char const*, omObjSType*) in rubygrammar.o
      here_document(omObjSType**, rb_parse_state*) in rubygrammar.o
  "om::FetchClassId_(omObjSType*)", referenced from:
      omObjSType::classId() in rubygrammar.o
  "om::compilerState()", referenced from:
      MagParse903(om*, omObjSType**) in rubygrammar.o
      VarTable::grow(rb_parse_state*) in rubygrammar.o
  "om::FetchClassObj_(omObjSType*)", referenced from:
      RubyNode::checkInstanceOf(omObjSType**, AstClassEType, rb_parse_state*) in rubyast.o
      RubyNode::is_a(omObjSType*, AstClassEType, rb_parse_state*) in rubyast.o
  "om::rubyParseState()", referenced from:
      MagCompileError902(om*, omObjSType**) in rubygrammar.o
      MagParse903(om*, omObjSType**) in rubygrammar.o
  "om::AppendToString_(om*, omObjSType**, char const*, long long)", referenced from:
      here_document(omObjSType**, rb_parse_state*) in rubygrammar.o
      om::AppendToString(om*, omObjSType**, char const*) in rubygrammar.o
  "om::FetchOopNoFault(omObjSType*, long long)", referenced from:
      om::FetchSmallInt__(omObjSType*, long long) in rubygrammar.o
  "om::IsKindOf_noHist(om*, omObjSType**, omObjSType*)", referenced from:
      RubyAbstractLiteralNode::kind_of(omObjSType*, rb_parse_state*) in rubygrammar.o
  "om::NewGlobalHandle()", referenced from:
      sessionInit(om*, rb_parse_state*) in rubygrammar.o
  "om::IntRecurFromPrim_(om*, omObjSType**, unsigned long long)", referenced from:
      RubyNode::call(omObjSType*, AstSelectorEType, rb_parse_state*) in rubyast.o
  "om::IntRecurFromPrim__(om*, omObjSType**, unsigned long long, long long, omObjSType***)", referenced from:
      RubyNode::call(omObjSType*, AstSelectorEType, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(omObjSType*, AstSelectorEType, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(omObjSType*, AstSelectorEType, omObjSType*, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(omObjSType*, AstSelectorEType, omObjSType*, omObjSType*, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      ...
  "om::set_rubyParseStack(YyStackData*)", referenced from:
      MagParse903(om*, omObjSType**) in rubygrammar.o
      sessionInit(om*, rb_parse_state*) in rubygrammar.o
  "om::set_rubyParseState(rb_parse_state*)", referenced from:
      MagParse903(om*, omObjSType**) in rubygrammar.o
      sessionInit(om*, rb_parse_state*) in rubygrammar.o
  "om::NewObj(om*, omObjSType**)", referenced from:
      RubyNode::basicNew(AstClassEType, rb_parse_state*) in rubyast.o
      RpNameToken::s(omObjSType**, omObjSType*, rb_parse_state*) in rubyast.o
      RpNameToken::s(rb_parse_state*, omObjSType*) in rubyast.o
      RubyLexStrTerm::newStrTerm(short, int, int, rb_parse_state*) in rubygrammar.o
      RubyLexStrTerm::newHereDoc(rb_parse_state*, char const*, long long, long long, bstring*) in rubygrammar.o
  "om::FetchOop(omObjSType*, long long)", referenced from:
      RubyNode::basicNew(AstClassEType, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, omObjSType*, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, omObjSType*, omObjSType*, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      RubyNode::call(AstClassEType, AstSelectorEType, omObjSType*, omObjSType*, omObjSType*, omObjSType*, omObjSType*, rb_parse_state*) in rubyast.o
      ...
  "om::NewArray(om*, long long)", referenced from:
      sessionInit(om*, rb_parse_state*) in rubygrammar.o
      yyparse(rb_parse_state*) in rubygrammar.o
      VarTable::asArray(om*) in rubygrammar.o
      yylex(rb_parse_state*) in rubygrammar.o
  "om::isSymbol(omObjSType*)", referenced from:
      RpNameToken::s(rb_parse_state*, omObjSType*) in rubyast.o
      quidToSymbolObj(omObjSType*, rb_parse_state*) in rubygrammar.o
  "om::StoreOopR(om*, omObjSType**, long long, omObjSType**)", referenced from:
      om::StoreOop(om*, omObjSType**, long long, omObjSType**) in rubyast.o
      om::StoreOop(om*, omObjSType**, long long, omObjSType**) in rubygrammar.o
ld: symbol(s) not found for architecture x86_64

@AllenOtis - any thoughts?

@AllenOtis
Copy link
Contributor Author

I used the following link command the last time I did a link on Linux.
This command is in the README.txt that was part of my last commit .
The args tell the linker that unresolved symbols are warnings only.
And the link should not explicitly link against libgcilnk library. Rather
the VM (topaz -l or gem process) will explicltily load the various libs
from $GEMSTONE/lib in the right order. (libmagparse is loaded last).
Note the magparse.exp file should be in the current directory along
with the .o files from your compiles. magparse.exp specifies what
symbols are exported from libmagparse ; all others are local to the library
being built.

/usr/bin/g++ -shared -Wl,-Bdynamic,-hlibmagparse.so  \
-Wl,--version-script=magparse.exp \
    rubyast.o rubygrammar.o  \
    -m64 -lpthread -lcrypt -ldl -lc -lm -lrt -o libmagparse.so \
    -Wl,--warn-unresolved-symbols

Allen

On Tue, Mar 24, 2015 at 12:20 PM, JohnnyT [email protected] wrote:

I'm trying it out - but am getting probably the same stuff you did
@zenspider https://github.com/zenspider:

  • echo 'Linking libmagparse.so'
    Linking libmagparse.so$CC
    -shared
    -m64 -lpthread -ldl -lc -lm -o libmagparse.so
    -stdlib=libstdc++
    -L$GEMSTONE/lib
    -lgcilnk-$GSVERSION
    rubyast.o rubygrammar.o
  • /usr/bin/g++ -shared -m64 -lpthread -ldl -lc -lm -o libmagparse.so -stdlib=libstdc++ -L/Users/johnnyt/.rbenv/versions/maglev-2.0.0-dev/gemstone/lib -lgcilnk-3.1.0.2.1-64 rubyast.o rubygrammar.o
    Undefined symbols for architecture x86_64:
    "GemErrAnsi(om_, int, char const_, char const_)", referenced from:
    MagParse903(om_, omObjSType**) in rubygrammar.o
    sessionInit(om_, rb_parse_state_) in rubygrammar.o
    resolveAstClass(om*, omObjSType**, AstClassEType) in rubygrammar.o
    initAstSelector(om_, unsigned long long_, AstSelectorEType) in rubygrammar.o
    initAstSymbol(om_, omObjSType__, AstSymbolEType) in rubygrammar.o
    "ComHeapInit(ComStateType_)", referenced from:
    MagParse903(om_, omObjSType__) in rubygrammar.o
    "ComHeapMalloc(ComStateType_, unsigned int)", referenced from:
    VarTable::allocate(rb_parse_state_, int) in rubyast.o
    LocalState::allocate(rb_parse_state_) in rubyast.o
    bstring::balloc(bstring_, long long, rb_parse_state_) in rubyast.o
    bstring::new_(rb_parse_state_) in rubyast.o
    StartPositionList::allocate(rb_parse_state_, int) in rubyast.o
    MagCompileError902(om_, omObjSType__) in rubygrammar.o
    MagParse903(om_, omObjSType**) in rubygrammar.o
    ...
    "GemSupErr_oo_(int, omObjSType_, omObjSType_)", referenced from:
    RubyNode::checkInstanceOf(omObjSType**, AstClassEType, rb_parse_state_) in rubyast.o
    "LrgInt64ToOop(om_, long long)", referenced from:
    RubyLexStrTerm::newHereDoc(rb_parse_state_, char const_, long long, long long, bstring_) in rubygrammar.o
    RubyAbstractNumberNode::s(long long, rb_parse_state_) in rubygrammar.o
    "FloatPrimFetchArg(om_, omObjSType__, double_)", referenced from:
    yyparse(rb_parse_state_) in rubygrammar.o
    "GemDoSessionSymList(om_)", referenced from:
    resolveAstClass(om_, omObjSType__, AstClassEType) in rubygrammar.o
    "GemSupSearchSymList(om_, omObjSType**, omObjSType**)", referenced from:
    resolveAstClass(om_, omObjSType__, AstClassEType) in rubygrammar.o
    "FloatPrimDoubleToOop(om_, double)", referenced from:
    yyparse(rb_parse_state_) in rubygrammar.o
    yylex(rb_parse_state_) in rubygrammar.o
    "HostCallDebuggerMsg_fl(char const_, char const_, int)", referenced from:
    RpNameToken::s(omObjSType**, omObjSType_, rb_parse_state_) in rubyast.o
    RpNameToken::s(AstSymbolEType, omObjSType_, rb_parse_state_) in rubyast.o
    RpNameToken::s(rb_parse_state_, omObjSType_) in rubyast.o
    OOP_OF_SMALL_LONG_(long long) in rubyast.o
    VarTable::allocate(rb_parse_state_, int) in rubyast.o
    OOP_TO_BIT(unsigned long long) in rubyast.o
    RpNameToken::buildQuid(unsigned long long, long long, long long) in rubyast.o
    ...
    "LrgRubyStringToInteger(om_, omObjSType**, long long, int)", referenced from:
    newInteger(rb_parse_state_, int) in rubygrammar.o
    "ObjCanonicalSymFromCStr(om_, unsigned char const_, long long, unsigned long long)", referenced from:
    rb_parser_sym(char const_, rb_parse_state_) in rubygrammar.o
    "ObjExistingCanonicalSym__(om_, omObjSType**)", referenced from:
    initAstSelector(om_, unsigned long long_, AstSelectorEType) in rubygrammar.o
    "LrgNegate(om*, omObjSType**)", referenced from:
    yyparse(rb_parse_state_) in rubygrammar.o
    "ObjNewSym(om_, char const_)", referenced from:
    RpNameToken::s(char const_, omObjSType_, rb_parse_state_) in rubyast.o
    yyparse(rb_parse_state_) in rubygrammar.o
    resolveAstClass(om_, omObjSType**, AstClassEType) in rubygrammar.o
    initAstSymbol(om*, omObjSType**, AstSymbolEType) in rubygrammar.o
    "OmScopeType::initialize_(om_)", referenced from:
    OmScopeType::OmScopeType(om_) in rubyast.o
    OmScopeType::OmScopeType(om_) in rubygrammar.o
    "OmScopeType::deinitialize_()", referenced from:
    OmScopeType::~OmScopeType() in rubyast.o
    OmScopeType::~OmScopeType() in rubygrammar.o
    "OmScopeType::add_(omObjSType_)", referenced from:
    OmScopeType::add(omObjSType_) in rubyast.o
    OmScopeType::add(omObjSType_) in rubygrammar.o
    OmScopeType::newHandle() in rubygrammar.o
    "om::FetchCData(omObjSType_)", referenced from:
    MagParse903(om_, omObjSType**) in rubygrammar.o
    "om::FetchSize_(omObjSType_)", referenced from:
    MagCompileError902(om_, omObjSType**) in rubygrammar.o
    MagParse903(om_, omObjSType__) in rubygrammar.o
    here_document(omObjSType__, rb_parse_state_) in rubygrammar.o
    heredoc_restore(rb_parse_state_) in rubygrammar.o
    "om::corruptObj(omObjSType_, char const_)", referenced from:
    om::StoreSmallInt_(om_, omObjSType**, long long, long long) in rubyast.o
    om::StoreSmallInt_(om*, omObjSType**, long long, long long) in rubygrammar.o
    "om::FetchBytes_(omObjSType_, long long, long long, unsigned char_)", referenced from:
    heredoc_restore(rb_parse_state_) in rubygrammar.o
    "om::NewString__(om_, unsigned char const_, long long)", referenced from:
    RubyLexStrTerm::newHereDoc(rb_parse_state_, char const_, long long, long long, bstring_) in rubygrammar.o
    yylex(rb_parse_state_) in rubygrammar.o
    NEW_STR(char const_, long long, rb_parse_state_) in rubygrammar.o
    newInteger(rb_parse_state_, int) in rubygrammar.o
    om::NewString(om_, long long) in rubygrammar.o
    om::NewString_(om_, char const_) in rubygrammar.o
    "om::strCharSize(omObjSType_)", referenced from:
    MagCompileError902(om_, omObjSType__) in rubygrammar.o
    MagParse903(om_, omObjSType**) in rubygrammar.o
    "om::LocatePomObj(om_, unsigned long long)", referenced from:
    RpNameToken::s(rb_parse_state_, omObjSType_) in rubyast.o
    quidToSymbolObj(omObjSType_, rb_parse_state_) in rubygrammar.o
    "om::isCByteArray(omObjSType_)", referenced from:
    MagParse903(om*, omObjSType**) in rubygrammar.o
    "om::objIdOfObj__(om_, omObjSType_)", referenced from:
    RpNameToken::quidForSym(omObjSType_, om_) in rubyast.o
    rb_parser_sym(char const_, rb_parse_state_) in rubygrammar.o
    initAstSelector(om_, unsigned long long_, AstSelectorEType) in rubygrammar.o
    "om::AppendToArray(om_, omObjSType__, omObjSType__)", referenced from:
    yylex(rb_parse_state_) in rubygrammar.o
    "om::FetchCString_(omObjSType_, char_, long long)", referenced from:
    MagCompileError902(om_, omObjSType__) in rubygrammar.o
    MagParse903(om_, omObjSType**) in rubygrammar.o
    rb_compile_error_q(rb_parse_state_, char const_, omObjSType*) in rubygrammar.o
    here_document(omObjSType**, rb_parse_state_) in rubygrammar.o
    "om::FetchClassId_(omObjSType_)", referenced from:
    omObjSType::classId() in rubygrammar.o
    "om::compilerState()", referenced from:
    MagParse903(om_, omObjSType__) in rubygrammar.o
    VarTable::grow(rb_parse_state_) in rubygrammar.o
    "om::FetchClassObj_(omObjSType_)", referenced from:
    RubyNode::checkInstanceOf(omObjSType__, AstClassEType, rb_parse_state_) in rubyast.o
    RubyNode::is_a(omObjSType_, AstClassEType, rb_parse_state_) in rubyast.o
    "om::rubyParseState()", referenced from:
    MagCompileError902(om_, omObjSType__) in rubygrammar.o
    MagParse903(om_, omObjSType**) in rubygrammar.o
    "om::AppendToString_(om*, omObjSType**, char const_, long long)", referenced from:
    here_document(omObjSType__, rb_parse_state_) in rubygrammar.o
    om::AppendToString(om_, omObjSType__, char const_) in rubygrammar.o
    "om::FetchOopNoFault(omObjSType_, long long)", referenced from:
    om::FetchSmallInt__(omObjSType_, long long) in rubygrammar.o
    "om::IsKindOf_noHist(om_, omObjSType__, omObjSType_)", referenced from:
    RubyAbstractLiteralNode::kind_of(omObjSType_, rb_parse_state_) in rubygrammar.o
    "om::NewGlobalHandle()", referenced from:
    sessionInit(om_, rb_parse_state_) in rubygrammar.o
    "om::IntRecurFromPrim_(om_, omObjSType__, unsigned long long)", referenced from:
    RubyNode::call(omObjSType_, AstSelectorEType, rb_parse_state_) in rubyast.o
    "om::IntRecurFromPrim__(om_, omObjSType**, unsigned long long, long long, omObjSType**)", referenced from:
    RubyNode::call(omObjSType
    , AstSelectorEType, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(omObjSType_, AstSelectorEType, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(omObjSType_, AstSelectorEType, omObjSType_, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(omObjSType_, AstSelectorEType, omObjSType_, omObjSType_, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    ...
    "om::set_rubyParseStack(YyStackData_)", referenced from:
    MagParse903(om_, omObjSType__) in rubygrammar.o
    sessionInit(om_, rb_parse_state_) in rubygrammar.o
    "om::set_rubyParseState(rb_parse_state_)", referenced from:
    MagParse903(om_, omObjSType__) in rubygrammar.o
    sessionInit(om_, rb_parse_state_) in rubygrammar.o
    "om::NewObj(om_, omObjSType**)", referenced from:
    RubyNode::basicNew(AstClassEType, rb_parse_state*) in rubyast.o
    RpNameToken::s(omObjSType**, omObjSType_, rb_parse_state_) in rubyast.o
    RpNameToken::s(rb_parse_state_, omObjSType_) in rubyast.o
    RubyLexStrTerm::newStrTerm(short, int, int, rb_parse_state_) in rubygrammar.o
    RubyLexStrTerm::newHereDoc(rb_parse_state_, char const_, long long, long long, bstring_) in rubygrammar.o
    "om::FetchOop(omObjSType_, long long)", referenced from:
    RubyNode::basicNew(AstClassEType, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, omObjSType_, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, omObjSType_, omObjSType_, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    RubyNode::call(AstClassEType, AstSelectorEType, omObjSType_, omObjSType_, omObjSType_, omObjSType_, omObjSType_, rb_parse_state_) in rubyast.o
    ...
    "om::NewArray(om_, long long)", referenced from:
    sessionInit(om_, rb_parse_state_) in rubygrammar.o
    yyparse(rb_parse_state_) in rubygrammar.o
    VarTable::asArray(om_) in rubygrammar.o
    yylex(rb_parse_state_) in rubygrammar.o
    "om::isSymbol(omObjSType_)", referenced from:
    RpNameToken::s(rb_parse_state_, omObjSType_) in rubyast.o
    quidToSymbolObj(omObjSType_, rb_parse_state_) in rubygrammar.o
    "om::StoreOopR(om_, omObjSType__, long long, omObjSType__)", referenced from:
    om::StoreOop(om_, omObjSType**, long long, omObjSType**) in rubyast.o
    om::StoreOop(om_, omObjSType__, long long, omObjSType_*) in rubygrammar.o
    ld: symbol(s) not found for architecture x86_64

@AllenOtis https://github.com/AllenOtis - any thoughts?


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

@AllenOtis I'm having a hard time getting this up and running and a lot of it has to do with the unknown in this project.

"linux" is vague. "/usr/bin/g++" is vague. I have no idea what my tooling is supposed to be and it doesn't seem to be documented anywhere. Given that the ruby landscape is absolutely dominated by OSX, we need to smooth this out. If there is some specific set of tooling that a developer needs to have installed for this to work, it needs to be well documented.

I'm currently trying to use g++-4.9 and it fails with your build code. Specifically, none of these work:

  -Wl,-Bdynamic,-hlibmagparse.so \
  -Wl,--version-script=magparse.exp \
  -Wl,--warn-unresolved-symbols \

gcc-4.9 doesn't ship with a linker, so it is trying to use the linker from xcode, which is clang's ld.

I've hacked up the following (semi-)equivalents:

@@ -53,8 +54,10 @@ echo "Linking libmagparse.so"
 $CC \
   -shared \
   -m64 -lpthread -ldl -lc -lm -o libmagparse.so \
+  -Wl,-flat_namespace \
+  -Wl,-undefined -Wl,warning \
+  -Wl,-exported_symbols_list -Wl,magparse.exp \
   -L$GEMSTONE/lib \
-  -lgcilnk-$GSVERSION \
   rubyast.o rubygrammar.o

and I've got a link working, finally. But obviously this script won't work for linux users. Is there a cross-platform build setup you guys use, or is it just shell scripts and a bunch of variables?

@AllenOtis
Copy link
Contributor Author

Part of the work to integrate libmagparse branch to master is to develop
the shell scripts and/or make files both of which will require platform
specific
compile and link commands. The gemstone server build has make files
and scripts but those are way to complicated to start from. Thus I just
provided
the linux compile and link command lines in my checkin.

On Thu, Mar 26, 2015 at 4:15 PM, Ryan Davis [email protected]
wrote:

@AllenOtis https://github.com/AllenOtis I'm having a hard time getting
this up and running and a lot of it has to do with the unknown in this
project.

"linux" is vague. "/usr/bin/g++" is vague. I have no idea what my tooling
is supposed to be and it doesn't seem to be documented anywhere. Given that
the ruby landscape is absolutely dominated by OSX, we need to smooth
this out. If there is some specific set of tooling that a developer needs
to have installed for this to work, it needs to be well documented.

I'm currently trying to use g++-4.9 and it fails with your build code.
Specifically, none of these work:

-Wl,-Bdynamic,-hlibmagparse.so
-Wl,--version-script=magparse.exp
-Wl,--warn-unresolved-symbols \

gcc-4.9 doesn't ship with a linker, so it is trying to use the linker from
xcode, which is clang's ld.

I've hacked up the following (semi-)equivalents:

@@ -53,8 +54,10 @@ echo "Linking libmagparse.so"
$CC
-shared
-m64 -lpthread -ldl -lc -lm -o libmagparse.so + -Wl,-flat_namespace + -Wl,-undefined -Wl,warning + -Wl,-exported_symbols_list -Wl,magparse.exp
-L$GEMSTONE/lib - -lgcilnk-$GSVERSION
rubyast.o rubygrammar.o

and I've got a link working, finally. But obviously this script won't work
for linux users. Is there a cross-platform build setup you guys use, or is
it just shell scripts and a bunch of variables?


Reply to this email directly or view it on GitHub
#352 (comment).

@zenspider
Copy link
Contributor

On Mar 26, 2015, at 18:03, Allen Otis [email protected] wrote:

Part of the work to integrate libmagparse branch to master is to develop
the shell scripts and/or make files both of which will require platform
specific
compile and link commands. The gemstone server build has make files
and scripts but those are way to complicated to start from. Thus I just
provided
the linux compile and link command lines in my checkin.

OK. At some point in time, I'll probably convert the scripts to makefiles. Unless creditera wants to pay me to futz with linux, I'll just do the OSX side with appropriate parameterization and let someone else fill in for other platforms.

@johnnyt
Copy link
Member

johnnyt commented Jul 9, 2015

Closed by #405

@johnnyt johnnyt closed this as completed Jul 9, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants