Heap Buffer Overflow due to reuse of signatures Buffer

[michaelvanstraten]

In the trace_on function, we invoke initNewTape

ADOL-C/ADOL-C/src/tape_handling.cpp

Line 800 in d5e3c2a

retval = initNewTape(tnum);

which checks if the tag ID has been used previously

ADOL-C/ADOL-C/src/tape_handling.cpp

Line 204 in d5e3c2a

if (tiIter != ADOLC_TAPE_INFOS_BUFFER.end()) {

If the tag ID is found to be already used and no one is currently using it

ADOL-C/ADOL-C/src/tape_handling.cpp

Line 206 in d5e3c2a

if ((*tiIter)->inUse != 0) {

the function calls initTapeInfos_keep

ADOL-C/ADOL-C/src/tape_handling.cpp

Line 224 in d5e3c2a

initTapeInfos_keep(*tiIter);

which in turn reassigns the signatures buffer

ADOL-C/ADOL-C/src/tape_handling.cpp

Line 184 in d5e3c2a

newTapeInfos->signature = signature;

The signatures buffer is initialized during the first call to zos_pl_forward

ADOL-C/ADOL-C/src/uni5_for.c

Line 1102 in d5e3c2a

signature = myalloc1(ADOLC_CURRENT_TAPE_INFOS.stats[NUM_SWITCHES]);

with the number of switching variables determining its size.

The issue arises when the tag ID is reused, leading to the reuse of the signatures buffer without reallocation. If the number of switching variables increases during this process, it can cause a heap buffer overflow due to insufficient space in the reused buffer.

[TimSiebert1]

See #81