CRIRES-MF

Programme ID-driven reduction pipeline for CRIRES+ YJHK (0.95-2.5 um) spectroscopy nodding data. Fetches science and calibration frames from the ESO archive, reduces everything with the cr2res pipeline (util-level calibration cascade), applies telluric correction via vipere, and serves results through a web archive.

Adapted from the CRIRES-LM pipeline.

Requirements

• Python >= 3.10, managed via uv
• cr2res pipeline (esorex + cr2res recipes)
• Emission line catalogs in /home/alavail/cr2res-scripts/catalogs/

Python dependencies are declared per-script (PEP 723) and resolved automatically by uv run.

Pipeline steps

1. Fetch calibrations

Query the ESO archive for deep flats and nightly wavecal frames (UNe lamp + Fabry-Perot etalon) matching the settings and date ranges of your programme.

uv run fetch_calibs.py 114.28HP
uv run fetch_calibs.py --progs programmes.txt    # from file
uv run fetch_calibs.py --query-only 114.28HP      # populate DB, skip download

2. Reduce calibrations

Run the full 13-step util-level calibration cascade per (setting, night). Each calib/{setting}_{night}/ directory is self-contained: raw frames symlinked in, emission line catalogs copied, all intermediate and final products in numbered output subdirectories.

uv run reduce_calib.py --dry-run    # inspect SOFs and commands
uv run reduce_calib.py              # run (4 parallel cascades by default)
uv run reduce_calib.py --jobs 1     # sequential

The cascade steps:

Step	Recipe	Purpose
02	cr2res_util_calib	Collapse flats
03	cr2res_util_trace	Trace spectral orders
04	cr2res_util_slit_curv	Slit curvature from FPET
05	cr2res_util_extract	Extract flat (slit model)
06	cr2res_util_normflat	Normalize flat -> master + BPM
07	cr2res_util_calib	Calibrate UNe lamp
08	cr2res_util_extract	Extract UNe spectrum
09	cr2res_util_genlines	Generate emission line catalog
11	cr2res_util_wave	Wavelength XCORR (degree 0)
12	cr2res_util_wave	Wavelength XCORR (degree 2)
13	cr2res_util_calib	Calibrate FPET
14	cr2res_util_extract	Extract FPET spectrum
15	cr2res_util_wave	Wavelength ETALON (final)

3. Fetch science

uv run fetch_science.py 114.28HP
uv run fetch_science.py --user USERNAME 114.28HP   # proprietary data

4. Generate science SOFs

uv run make_sofs.py

Creates reduced/{object}_{setting}_{tpl}/nodd.sof directories with combined templates and individual AB pairs, referencing calibration products from calib/.

5. Batch science extraction

./reduce_science.sh       # default 6 parallel jobs
./reduce_science.sh 12    # specify job count

6. Telluric correction

Single directory:

uv run tellcorr.py reduced/HD_120411_K2166_2025-01-30_04281_1

Batch (parallel, skips already processed):

ls reduced/*/cr2res_obs_nodding_extractedA.fits | sed 's|/cr2res_obs_nodding_extractedA.fits||' | while read d; do
    [ -e "$d/cr2res_obs_nodding_extractedA_tellcorr.fits" ] || echo "$d"
done > /tmp/tellcorr_dirs.txt

parallel -j10 --bar -a /tmp/tellcorr_dirs.txt 'uv run tellcorr.py {}'

7. Update observation status

Pre-compute PSF FWHM and super-resolution flags from FITS headers into the database (run after science extraction):

uv run update_status.py

8. Web archive

uv run webapp.py --port 8001
uv run webapp.py --port 8001 --root-path /crires-mf   # behind reverse proxy

Directory structure

CRIRES-MF/
  raw/                          # all downloaded FITS frames
  calib/{setting}_{night}/      # self-contained calibration cascades
    02_cr2res_util_calib_out/
    ...
    15_cr2res_util_wave_out/    # final wavelength solution
  reduced/{object}_{setting}_{tpl}/
    nodd.sof
    cr2res_obs_nodding_* products
  calibs.sqlite                 # metadata for flats, wavecals, science

Database

SQLite database calibs.sqlite with three tables: flats, wavecals, science. Created automatically by fetch_calibs.py / fetch_science.py.