mwax_calvin_processor

About

The mwax_calvin_processor will run on one of two "calvin" servers at the MRO. The calvin servers have the following specs:

The high level process for the calvins to follow is:

STALE_TIMEOUT = 24 hours

do while running:
    watch watch_path for new fits files (from MWAX servers on site). When new fits arrives execute watch_handler.

    # it's possible some files for an obs just never show up. This next step checks for stale ones and removes them
    check_for_old_incomplete_obsids()

watch_handler(item):
    obs_id = get_obsid_from_filename(item)

    files_needed = get_data_files_info_from_webservice(obs_id)

    files_arrived = get_files_in_watch_dir(obs_id)

    if files_needed == files_arrived:
        create new dir work_dir/obs_id
        move obs_id's files to work_dir/obs_id

check_for_old_incomplete_obsids():
    file_list = get_files_in_watch_dir()

    for each file in file_list:
        ctime = get_file_create_time(file)

        if now() - ctime > STALE_TIMEOUT:
            delete file