-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME
195 lines (164 loc) · 10.8 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
PreSync update for Cactus
Arrangement Authors : Steve Brandt, Samuel Cupp
--------------------------------------------------------------------------------
This repository contains the thornlist presync.th, which is a list of thorns on
the presync branch which are believed to work and pass tests in the testsuite.
In addition, a docker image is provided which, when built, downloads and
compiles the presync branch of the toolkit and runs the testsuite in both
presync and backward compatible modes. It generates out.txt and pout.txt, which
are the testsuite outputs for the backward compatible and presync tests,
respectively. Below is a brief description of PreSync and some resources for
changing thorns to use PreSync.
--------------------------------------------------------------------------------
Presync now handles synchronization and boundary condition registration and
application from within Carpet. This is done by declaring reads/writes for each
function in the schedule.ccl. This document explains the process of updating an
existing thorn to use the new mechanisms in PreSync. This document consists of
the following sections:
I. Basic Step-by-Step Update Procedure
II. PreSync With Fortran
III. Internal Functions
IV. Registering Boundary Functions
V. Applying Boundary Conditions: Special Cases
VI. Generation of New Macros: rdwr.pl
----------------------------------------
I. Step-By-Step Thorn Update Procedure
----------------------------------------
1) Anything which has Boundary in the interface.ccl (via Requires, Inherits,
Shares, etc.) should be changed to need Boundary2. The changes could be
migrated to Boundary, but they are currently in a different thorn.
2) Previously, a BC was selected for a variable by calling Boundary_SelectGroupForBC
or Boundary_SelectVarForBC. These functions were scheduled whenever boundary
functions needed to be applied, along with ApplyBCs group from the Boundary
thorn. To use PreSync, selection routines are instead scheduled in the GROUP
Boundary2_Selection. The ApplyBCs group is not used in the PreSync and isn't
scheduled. The old boundary scheduling should be encased in the conditional
if(CCTK_ParameterValInt("use_psync","Carpet") == 0) {
}
and the new boundary scheduling should be encased in the conditional
if(CCTK_ParameterValInt("use_psync","Carpet") == 1) {
}
4) All scheduled functions have new macros. Change DECLARE_CCTK_ARGUMENTS to
DECLARE_CCTK_ARGUMENTS_FunctionName. The header file cctk_Arguments_Checked.h
also needs to be included. This will only allow access to those variables with
read/write declarations. Any variables used but not declared will result in a
compiler error. Please note that the macro will use the function name in the
schedule.ccl. While Cactus is case-insensitive, the C preprocessor is not.
Therefore, the macro will take the case of the name in the schedule.ccl in the
event that it differs from the case in the actual code. For this reason, it is
recommended that the two match in case to prevent an easily missable cause
for compilation failure. For Fortran code, please see the next section.
5) Compile errors will appear file-by-file. Each function must have the correct
read/write declarations before the compiler will go to the next file. Since
the given line is the variable's first appearance, it is usually clear if
the variable is read or written. In addition, read-only variables are declared
const, so a variable which is read and written will give another error for
the write. The opposite case is not true - a function which reads and writes
a variable will not give an error if WRITES is present but READS is not.
----------------------------------------
II. PreSync with Fortran
----------------------------------------
Currently, the new macros are not usable with Fortran code. However, they can
still be used to check read/write declarations. As with C code, use the new macro
and header file. The argument list of the function must also be changed from
CCTK_ARGUMENTS to CCTK_ARGUMENTS_FunctionName. The argument list results in an
argument passing error at runtime, but the read/write errors at compile time are
still accessible.
Unfortunately, the compiler (combined with preprocessing) does not provide
the errors in a useful form. Many errors will appear which are caused by the many
undeclared variables. The only errors of interest are implicit variable errors.
The line of code must be found for each error, as it prints out the wrong line
with the error. Otherwise, the process is the same as with C code.
Once the code compiles successfully, revert the macro and argument list to the
original form and compile again. Eventually, full Fortran compatibility will
hopefully be added, but this method is functional in the interim.
----------------------------------------
III. Internal Functions
----------------------------------------
Internal functions can greatly complicate the read/write declarations, as
their usage of variables will not trigger a compiler error in the scheduled
function. This generally manifests in two ways. Either the function uses
variables not declared at all by the parent function, or it uses a variable
in such a way as to change the needed declarations. The second is less clear
without an example. Let Fun1 be a scheduled function, and Fun2 be an internal
function called at the beginning of Fun1. After following the usual procedure,
Fun1 reads and writes var1 in its schedule. However, Fun2 only writes var1.
Then, Fun2 writes var1 before Fun1 reads it. Therefore, the read is not needed.
The best way to handle internal functions is to include the macro of a scheduled
function. If the function is only called by one scheduled funciton, then simply
use that macro. If called by several, there are several options.
First, the macro of one can be included for debugging, and any read/write
declarations resulting from the internal function are added to all scheduled
functions which use it.To be certain that nothing was missed, this may require
recompiling using the macro of each scheduled function. While tedious, this
will work.
Alternatively, the function can be scheduled nowhere. That is, adding something
of the form
schedule FunctionName
{
LANG: C
} ""
to the schedule.ccl will result in a macro being generated for this function.
After determining the read/write declarations for the internal function, they
can be copied into all the scheduled functions which call this internal function.
However, these methods will not catch the case where a read is not needed. For
that problem, the only solution currently is to manually verify the read/write
declarations are correct.
----------------------------------------
IV. Registering Boundary Functions
----------------------------------------
Physical boundary conditions (like those in the Boundary thorn) should be
registered with PreSync using the function Boundary_RegisterPhysicalBC.
This aliased function from Boundary2 is aliased identically to Boundary,
so any routines which already used Boundary to apply their boundary conditions
will not need significant changes. For those which did not use Boundary,
there are several options. The preferred method is to change the scheduled
function to register with PreSync. Then, PreSync can trigger BC application
as needed. Alternatively, the scheduling can simply have the WRITE region be
(boundary). This doesn't benefit from PreSync's features, but it should allow
for compatibility with PreSync.
Symmetry BCs (which are registered with SymBase) should also be registered
with PreSync. However, they are still registered with SymBase. PreSync simply
determines when these functions should be run. The Symmetry boundary conditions
are registered with the function Boundary_RegisterSymmetryBC. Before, these
functions were scheduled in BoundaryConditions GROUP from Boundary. Now, they
are called, and the argument list is different. The old version is copied as
Old_Name, while the original is still Name. The Old_Name is scheduled with
the (use_psync = 0) conditional for backward compatibility.
To see an example of how to use these features, see CactusNumerical/ReflectionSymmetry.
reflection.h includes PreSync.h and a prototype for the registered function.
Boundary2 thorn has a similar header file, but it includes a prototype for
every BC it contains.
register.c includes the registration function. For symmetry functions like
ReflectionSymmetry, the function is also registered with SymBase. In
ReflectionSymmetry_SymbaseRegister, Boundary_RegisterSymmetryBC is added to
the function. Note that it is also contained in a conditional to maintain
backward compatibility.
----------------------------------------
V. Boundary Conditions: Special Cases
----------------------------------------
In some circumstances, boundary conditions or synchronization may need to be
applied manually. This should be avoided and only used if absolutely necessary.
As an example, the variable cXt1 in ML_BSSN is only written, never read. It is
output as a check, but never used in any calculations. Because of this, it will
never trigger synchronization or application of boundary conditions. Thus, the
CarpetIOASCII thorn must be changed to handle it internally. This amounts to
calling ManualSyncGF, which is provided by Carpet. This function must be added
to the interface.ccl. The function checks the region of validity, so it will not
be synchronized if that is not needed.
----------------------------------------
VI. Generation of New Macros: rdwr.pl
----------------------------------------
This section discusses the new Perl script which creates the macros accompanying
the Presync update. The file rdwr.pl in the flesh parses the interface.ccl and
schedule.ccl to create these macros. The top-level subroutine GenerateArguments
is called in the CST and handles all the internal subroutine calls.
For each thorn, the interface.ccl is parsed, and information on variables is
stored in a hash table. After all interface.ccl have been parsed, each thorn's
schedule.ccl is read. The READ/WRITE declarations are parsed for each scheduled
subroutine. In the event that a subroutine is scheduled multiple times with
different declarations, a union of all instances is taken for the macro. Once
the whole file has been parsed, a DECLARE_CCTK_ARGUMENTS_FunctionName macro
is generated for every scheduled subroutine. For Fortran subroutines, a
CCTK_ARGUMENTS_FunctionName macro is also generated. These macros are stored in
a header file called "cctk_Arguments_Checked.h".