QPhotTest.test_qphot_run() failing on 32-bit IRAF

[TGroner]

When trying to verify my recent installation of Lemon it will go nearly to the end but then it fails with a mismatch on the flux value. The output looks like this:

test_get_coords_file (test_qphot.QPhotTest) ... ok
test_qphot_run (test_qphot.QPhotTest) ... FAIL

======================================================================
FAIL: test_qphot_run (test_qphot.QPhotTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ted/lemon/test/test_qphot.py", line 170, in test_qphot_run
    self.assertEqual(phot, expected_phot)
AssertionError: QPhotResult(x=878.62, y=171.496, mag=17.623, sum=6266626.0, flux=3750371.0, stdev=484.9941) != QPhotResult(x=878.62, y=171.496, mag=17.623, sum=6266626, flux=3749740, stdev=484.9941)

----------------------------------------------------------------------
Ran 125 tests in 195.710s

FAILED (failures=1)

Any pointers or ideas on why the test fails in this fashion?

Thanks!

[wschoenell]

It seems to be related to the fact that the test expects 6266626 which is an integer and qphot returns 6266626.0 which is a float. Should be checked but I don't think that impacts the usage at all.

[TGroner]

I did notice that, but the results on flux were much different, 3750371.0 != 3749740. Perhaps the two are related?

[wschoenell]

Sorry... My bad. What version of pyraf do you have?

python -c 'import pyraf; print pyraf.version'

Did you tried the tests on the latest version from the github repository?

On Sun, May 17, 2015 at 2:12 AM TGroner [email protected] wrote:

I did notice that, but the results on flux were much different, 3750371.0
!= 3749740. Perhaps the two are related?

—
Reply to this email directly or view it on GitHub
#57 (comment).

[TGroner]

We have pyraf 2.1.8. I did fetch the latest version of that particular test from github, it produced the exact same output. Thanks for your replies!

[vterron]

This is a strange error indeed. I'm out of office this week, but we will fix this as soon as I'm back next Monday. Thanks for the helping hand, William!

[vterron]

First thing that comes to mind: what IRAF version do you have installed?

[TGroner]

We are using
IRAFNET PC-IRAF Revision 2.13-BETA2 Wed Apr 19 21:21:13 MST 2006

Thanks!

[vterron]

Any chance you could try upgrading to v2.16?

[TGroner]

I am referring that question to our Iraf maintainer, I will let you know.

[TGroner]

Most interesting, it still fails like so with iraf V2.16:

FAIL: test_qphot_run (test_qphot.QPhotTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ted/lemon/test/test_qphot.py", line 170, in test_qphot_run
    self.assertEqual(phot, expected_phot)
AssertionError: QPhotResult(x=736.138, y=64.345, mag=17.777, sum=6035459.0, flux=3254873.0, stdev=572.4526) != QPhotResult(x=736.138, y=64.345, mag=17.777, sum=6035459, flux=3254578, stdev=572.4526)

Very interesting how different the flux values are in my two posts!

[wschoenell]

Can you send the output of this command on your computer?
uname -a

On Thu, May 21, 2015 at 11:13 PM TGroner [email protected] wrote:

Most interesting, it still fails like so with iraf V2.16:
FAIL: test_qphot_run (test_qphot.QPhotTest)

Traceback (most recent call last):
File "/home/ted/lemon/test/test_qphot.py", line 170, in test_qphot_run
self.assertEqual(phot, expected_phot)

AssertionError: QPhotResult(x=736.138, y=64.345, mag=17.777,
sum=6035459.0, flux=3254873.0, stdev=572.4526) != QPhotResult(x=736.138,
y=64.345, mag=17.777, sum=6035459, flux=3254578, stdev=572.4526)

Very interesting how different the flux values are in my two posts!

—
Reply to this email directly or view it on GitHub
#57 (comment).

[TGroner]

Sure, here you go:
bash-4.2# uname -a
Linux tero 3.10.17-smp #2 SMP Wed Oct 23 17:13:14 CDT 2013 i686 AMD Phenom(tm) II X4 965 Processor AuthenticAMD GNU/Linux

[vterron]

Did you run python ./setup.py, as the installation instructions ask to near the end? In other words: does the file /home/ted/lemon/login.cl exist in your machine?

[TGroner]

I did run that ./setup.py, however it returned this error:

ted@tero:~/lemon> python ./setup.py
Setting up IRAF's login.cl in /home/ted/lemon ...
Traceback (most recent call last):
  File "./setup.py", line 119, in <module>
    mkiraf(lemon_path)
  File "./setup.py", line 80, in mkiraf
    raise subprocess.CalledProcessError(p.returncode, args)
subprocess.CalledProcessError: Command '['mkiraf']' returned non-zero exit status 1

At that point I commented out the call to mkiraf and manually did ln -s to my existing uparm and login.cl files. Iraf will start from my lemon directory, and it seems to see my uparm files.

[vterron]

That explains the difference: if the IRAF configuration file is not the exact same, I assume that even the slightest difference in a single parameter could result in qphot producing different values. I do not understand why mkiraf is failing: does the error message say something else?

If you replace the symbolic link with this login.cl file, does the test pass?

[TGroner]

With that login.cl iraf will not even start, with lots of errors. I did get mkiraf working however, and ran setup.py again. This gave a working login.cl, but the original qphot test failed in the same fashion.

[vterron]

We probably also need to remove the symbolic link to your uparm directory. Let's try this:

cd ~/lemon
rm uparm/ -rf
rm pyraf/ -rf
rm login.cl -f
python setup.py

[TGroner]

Did that, same results:

FAIL: test_qphot_run (test_qphot.QPhotTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ted/lemon/test/test_qphot.py", line 170, in test_qphot_run
    self.assertEqual(phot, expected_phot)
AssertionError: QPhotResult(x=736.138, y=64.345, mag=17.777, sum=6035459.0, flux=3254873.0, stdev=572.4526) != QPhotResult(x=736.138, y=64.345, mag=17.777, sum=6035459, flux=3254578, stdev=572.4526)

----------------------------------------------------------------------
Ran 125 tests in 170.586s

[wschoenell]

Could it be due to the fact that the OS is 32bits?

On Sat, May 23, 2015 at 11:20 PM TGroner [email protected] wrote:

Did that, same results:
FAIL: test_qphot_run (test_qphot.QPhotTest)

Traceback (most recent call last):
File "/home/ted/lemon/test/test_qphot.py", line 170, in test_qphot_run
self.assertEqual(phot, expected_phot)

AssertionError: QPhotResult(x=736.138, y=64.345, mag=17.777,
sum=6035459.0, flux=3254873.0, stdev=572.4526) != QPhotResult(x=736.138,
y=64.345, mag=17.777, sum=6035459, flux=3254578, stdev=572.4526)

---

Ran 125 tests in 170.586s

—
Reply to this email directly or view it on GitHub
#57 (comment).

[vterron]

It could be, but in principle I would say that's unlikely to be the case: we have successfully run the unit tests on a couple of 32-bit machines in the past. I still suspect the reason most probably lies with PyRAF seeing a different IRAF version or maybe a .par file with saved parameters for the qphot task.

Ted, could you please tell me what the output of the following commands is?

cd ~/lemon
git status
python -c "print '-' * 50"
pyraf 2>/dev/null | grep -i "Revision"

[TGroner]

# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#       modified:   setup.py
#       modified:   test/test_qphot.py
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       .last-github-commit-cache.json
#       =0.12.0
#       =0.5.1
#       =0.9.7
#       =0.9.9
#       =1.0.1
#       =1.2.1
#       =2.0.1
#       =2.1.1
#       =2.4.1
#       =3.2.6
#       OLD.login.cl
#       lemon.login.cl
#       new.login.cl
#       ouparm
#       test/HIDE.test_qphot.py.HIDE
#       test/test_data/fits/
#       testoid
no changes added to commit (use "git add" and/or "git commit -a")
--------------------------------------------------

NOAO/IRAF PC-IRAF Revision 2.16 EXPORT Thu May 24 15:41:17 MST 2012

I produced the last line manually, yours was invalid for tcsh.

It is not clear why this post blows up fonts either!

[vterron]

There is a more recent version available: IRAF 2.16.1 — would it be possible that you upgrade to it?

It is not clear why this post blows up fonts either!

The weird formatting was caused because Markdown interpreted the # characters as section headings. I have fixed it by indenting the text with four spaces, so that it gets rendered as a block of code. You may want to take a quick look at this guide if you encounter similar problems in the future.

[TGroner]

I installed Iraf 2.16.1 this morning. Current results-

FAIL: test_qphot_run (test_qphot.QPhotTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dilbert/lemon/test/test_qphot.py", line 170, in test_qphot_run
    self.assertEqual(phot, expected_phot)
AssertionError: QPhotResult(x=878.62, y=171.496, mag=17.623, sum=6266626.0, flux=3750371.0, stdev=484.9941) != QPhotResult(x=878.62, y=171.496, mag=17.623, sum=6266626, flux=3749740, stdev=484.9941)

----------------------------------------------------------------------
Ran 125 tests in 195.770s

Also this was from a fresh account, created today, that had zero files in the home directory.

[vterron]

At this point it really looks like a bug in IRAF — somehow, for an arcane reason, the value of the flux is different, but the magnitude, total number of counts and even the standard deviation are the same. It feels like if only that number were being affected by... who knows. What I wonder now is whether this is the value computed by your IRAF installation or, instead, it could be PyRAF's fault.

In order to find this out, could you please run this CL script and paste the output here?

git clone https://gist.github.com/fc5bea80779d08f78be0.git issue-57
cd issue-57/

Then, fire up IRAF and do:

vocl> cl < qphot-issue-57.cl

This will allow us to see the exact output produced by IRAF, without any involvement of PyRAF.

[TGroner]

Did you try that script? It has multiple problems. In order:
1: You need to be in the directory with the data file.
2: The case of the filename matters.
3: You need to specify a fits extension.
4: When I cleared all of those up I get ERROR: No write permission on file (String_File), which I have not figured out yet....

[vterron]

Yes, I tried the script before uploading it!

vterron@feynman:~$ git clone https://gist.github.com/fc5bea80779d08f78be0.git issue-57
Cloning into 'issue-57'...
remote: Counting objects: 12, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 12 (delta 2), reused 10 (delta 2), pack-reused 0
Unpacking objects: 100% (12/12), done.
vterron@feynman:~$ cd issue-57/
vterron@feynman:~/issue-57$ cl
Warning: no login.cl found in login directory
      dataio      language    obsolete    softools    vo          
      dbms        lists       plot        system      
      images      noao        proto       utilities   

vocl> cl < qphot-issue-57.cl
Temporary .mag file: /tmp/qphot_9262d
878.620  171.496  17.623  6266626.  3749740.  484.9941
vocl> logout

Just to make double sure I have tested it on three different Linux machines running IRAF 2.16.1. The code worked in every case and produced the same results. May I see your output when you follow these steps?

[TGroner]

Ok, somehow I must have messed that up. It does work and produces this:

vocl>  cl < qphot-issue-57.cl
Temporary .mag file: /tmp/qphot_4236d
878.620  171.496  17.623  6266626.  3750371.  484.9941

[vterron]

William was right: this has everything to do with you running a 32-bit version of IRAF.

I have configured LEMON on a couple of 32-bit machines (including a virtual one) this weekend and I can reproduce your error. The reason why we had not yet noticed this is that, although we had used the pipeline on 32-bit machines in the past, it was before these unit tests —that examine the exact output of qphot— were written.

This is a very strange and confusing behavior of IRAF. A coworker has mentioned that the difference in the qphot results could be explained by the limitations of floating-point arithmetic, but I find the values to be too different to be explained only by that. Even worse: if that were the sole reason, I would also expect the other values to be different. I cannot understand how flux (total number of counts in the aperture excluding sky) can be different while the sum (total number of counts including sky) remains equal.

I have submitted this to the IRAF developers as a possible bug. In the meantime (because, even if this is indeed a bug and gets fixed, it may easily take years until a new version is released), the solution is to modify the unit tests so that they take into account that some values, mostly the flux, may be different in 32-bit architectures.

Fret not: the magnitude is apparently always the same, so this difference between 32 and 64-bit IRAF versions should only affect (almost negligibly) the signal-to-noise estimation of the differential photometry measurements. That being said — this is but another reason to abandon IRAF and switch to Astropy's photutils, as William suggested in #59. That's what we intend to do as soon as possible, and expect to have completed in a few months at most.

[vterron]

Fixed in 8772356.

[TGroner]

Thanks for sticking with this to the end. Perhaps it is time to peek at the iraf source and see if the problem can be found. I'd have to guess it is some problem in the calculation of the sky value, which is odd indeed!