The muse_twilight recipe
===============================================================

.. data:: muse_twilight

Synopsis
--------

Combine several twilight skyflats into one cube, compute correction factors for each IFU, and create a smooth 3D illumination correction.

Description
-----------

Processing first handles each raw input image separately: it trims the raw data and records the overscan statistics, subtracts the bias (taking account of the overscan, if --overscan is not "none"), converts the images from adu to count, subtracts the dark, divides by the flat-field and combines all the exposures using input parameters. The input calibrations geometry table, trace table, and wavelength calibration table are used to assign 3D coordinates to each CCD-based pixel, thereby creating a pixel table from the master sky-flat. These pixel tables are then cut in wavelength using the --lambdamin and --lambdamax parameters. The integrated flux in each IFU is computed as the sum of the data in the pixel table, and saved in the header, to be used later as estimate for the relative throughput of each IFU. If an ILLUM exposure was given as input, it is then used to correct the relative illumination between all slices of one IFU. For this, the data of each slice within the pixel table of each IFU is multiplied by the normalized median flux of that slice in the ILLUM exposure. The pixel tables of all IFUs are then merged, using the integrated fluxes as inverse scaling factors, and a cube is reconstructed from the merged dataset, using given parameters. A white-light image is created from the cube. This skyflat cube is then saved to disk, with the white-light image as one extension. To construct a smooth 3D illumination correction, the cube is post-processed in the following way: the white-light image is used to create a mask of the illuminated area. From this area, the optional vignetting mask is removed. The smoothing is then computed for each plane of the cube: the illuminated area is smoothed (by a 5x7 median filter), normalized, fit with a 2D polynomial (of given polynomial orders), and normalized again. A smooth white image is then created by collapsing the smooth cube. If a vignetting mask was given or NFM data is processed, an area close to the edge of the MUSE field is used to compute a 2D correction for the vignetted area: the original unsmoothed white-light image is corrected for large scale gradients by dividing it with the smooth white image. The residuals in the edge area (as defined by the input mask or hardcoded for NFM) are then smoothed using input parameters. This smoothed vignetting correction is the multiplied onto each plane of the smooth cube, normalizing each plane again. This twilight cube is then saved to disk.


Constructor
-----------

.. method:: cpl.Recipe("muse_twilight")
   :noindex:

   Create an object for the recipe muse_twilight.

::

   import cpl
   muse_twilight = cpl.Recipe("muse_twilight")

Parameters
----------

.. py:attribute:: muse_twilight.param.overscan

    If this is "none", stop when detecting discrepant overscan levels (see  ovscsigma), for "offset" it assumes that the mean overscan level  represents the real offset in the bias levels of the exposures  involved, and adjusts the data accordingly; for "vpoly", a polynomial  is fit to the vertical overscan and subtracted from the whole  quadrant. (str; default: 'vpoly') [default="vpoly"].
.. py:attribute:: muse_twilight.param.ovscreject

    This influences how values are rejected when computing overscan  statistics. Either no rejection at all ("none"), rejection using the  DCR algorithm ("dcr"), or rejection using an iterative constant fit  ("fit"). (str; default: 'dcr') [default="dcr"].
.. py:attribute:: muse_twilight.param.ovscsigma

    If the deviation of mean overscan levels between a raw input image and  the reference image is higher than |ovscsigma x stdev|, stop the  processing. If overscan="vpoly", this is used as sigma rejection level  for the iterative polynomial fit (the level comparison is then done  afterwards with |100 x stdev| to guard against incompatible settings).  Has no effect for overscan="offset". (float; default: 30.0) [default=30.0].
.. py:attribute:: muse_twilight.param.ovscignore

    The number of pixels of the overscan adjacent to the data section of  the CCD that are ignored when computing statistics or fits. (int;  default: 3) [default=3].
.. py:attribute:: muse_twilight.param.combine

    Type of combination to use (str; default: 'sigclip') [default="sigclip"].
.. py:attribute:: muse_twilight.param.nlow

    Number of minimum pixels to reject with minmax (int; default: 1) [default=1].
.. py:attribute:: muse_twilight.param.nhigh

    Number of maximum pixels to reject with minmax (int; default: 1) [default=1].
.. py:attribute:: muse_twilight.param.nkeep

    Number of pixels to keep with minmax (int; default: 1) [default=1].
.. py:attribute:: muse_twilight.param.lsigma

    Low sigma for pixel rejection with sigclip (float; default: 3.0) [default=3.0].
.. py:attribute:: muse_twilight.param.hsigma

    High sigma for pixel rejection with sigclip (float; default: 3.0) [default=3.0].
.. py:attribute:: muse_twilight.param.scale

    Scale the individual images to a common exposure time before combining  them. (bool; default: False) [default=False].
.. py:attribute:: muse_twilight.param.resample

    The resampling technique to use for the final output cube. (str;  default: 'drizzle') [default="drizzle"].
.. py:attribute:: muse_twilight.param.crtype

    Type of statistics used for detection of cosmic rays during final  resampling. "iraf" uses the variance information, "mean" uses standard  (mean/stdev) statistics, "median" uses median and the median median of  the absolute median deviation. (str; default: 'median') [default="median"].
.. py:attribute:: muse_twilight.param.crsigma

    Sigma rejection factor to use for cosmic ray rejection during final  resampling. A zero or negative value switches cosmic ray rejection  off. (float; default: 50.0) [default=50.0].
.. py:attribute:: muse_twilight.param.lambdamin

    Minimum wavelength for twilight reconstruction. (float; default:  5000.0) [default=5000.0].
.. py:attribute:: muse_twilight.param.lambdamax

    Maximum wavelength for twilight reconstruction. (float; default:  9000.0) [default=9000.0].
.. py:attribute:: muse_twilight.param.dlambda

    Sampling for twilight reconstruction, this should result in planes of  equal wavelength coverage. (float; default: 250.0) [default=250.0].
.. py:attribute:: muse_twilight.param.xorder

    Polynomial order to use in x direction to fit the full field of view.  (int; default: 2) [default=2].
.. py:attribute:: muse_twilight.param.yorder

    Polynomial order to use in y direction to fit the full field of view.  (int; default: 2) [default=2].
.. py:attribute:: muse_twilight.param.vignmaskedges

    Pixels on edges stronger than this fraction in the normalized image  are excluded from the fit to the vignetted area. Set to non-positive  number to include them in the fit. This has no effect for NFM  skyflats. (float; default: 0.02) [default=0.02].
.. py:attribute:: muse_twilight.param.vignsmooth

    Type of smoothing to use for the vignetted region given by the  VIGNETTING_MASK (for WFM, or the internal mask, for NFM); gaussian  uses (vignxpar + vignypar)/2 as FWHM. (str; default: 'polyfit') [default="polyfit"].
.. py:attribute:: muse_twilight.param.vignxpar

    Parameter used by the vignetting smoothing: x order for polyfit  (default, recommended 4), parameter that influences the FWHM for the  gaussian (recommended: 10), or x dimension of median filter  (recommended 5). If a negative value is found, the default is taken.  (int; default: -1) [default=-1].
.. py:attribute:: muse_twilight.param.vignypar

    Parameter used by the vignetting smoothing: y order for polyfit  (default, recommended 4), parameter that influences the FWHM for the  gaussian (recommended: 10), or y dimension of median filter  (recommended 5). If a negative value is found, the default is taken.  (int; default: -1) [default=-1].
.. py:attribute:: muse_twilight.param.vignnfmmask

    The height of the vignetted region at the top of the MUSE field in  NFM. This is the region modeled separately (the final vignetting model  might be smaller). (int; default: 22) [default=22].


The following code snippet shows the default settings for the available 
parameters.

::

   import cpl
   muse_twilight = cpl.Recipe("muse_twilight")

   muse_twilight.param.overscan = "vpoly"
   muse_twilight.param.ovscreject = "dcr"
   muse_twilight.param.ovscsigma = 30.0
   muse_twilight.param.ovscignore = 3
   muse_twilight.param.combine = "sigclip"
   muse_twilight.param.nlow = 1
   muse_twilight.param.nhigh = 1
   muse_twilight.param.nkeep = 1
   muse_twilight.param.lsigma = 3.0
   muse_twilight.param.hsigma = 3.0
   muse_twilight.param.scale = False
   muse_twilight.param.resample = "drizzle"
   muse_twilight.param.crtype = "median"
   muse_twilight.param.crsigma = 50.0
   muse_twilight.param.lambdamin = 5000.0
   muse_twilight.param.lambdamax = 9000.0
   muse_twilight.param.dlambda = 250.0
   muse_twilight.param.xorder = 2
   muse_twilight.param.yorder = 2
   muse_twilight.param.vignmaskedges = 0.02
   muse_twilight.param.vignsmooth = "polyfit"
   muse_twilight.param.vignxpar = -1
   muse_twilight.param.vignypar = -1
   muse_twilight.param.vignnfmmask = 22


You may also set or overwrite some or all parameters by the recipe 
parameter `param`, as shown in the following example:

::

   import cpl
   muse_twilight = cpl.Recipe("muse_twilight")
   [...]
   res = muse_twilight( ..., param = {"overscan":"vpoly", "ovscreject":"dcr"})


.. seealso:: `cpl.Recipe <https://packages.python.org/python-cpl/recipe.html>`_
   for more information about the recipe object.

Bug reports
-----------

Please report any problems to `Peter Weilbacher <usd-help@eso.org>`_. Alternatively, you may 
send a report to the `ESO User Support Department <usd-help@eso.org>`_.

Copyright
---------

This file is part of the MUSE Instrument Pipeline
Copyright (C) 2005, 2019 European Southern Observatory

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, 
MA  02111-1307  USA

.. codeauthor:: Peter Weilbacher <usd-help@eso.org>
