Image domain gridding ===================== The `image-domain gridder (IDG) `_ is a new, fast gridder that makes *w*-term correction and *a*-term correction computationally very cheap. It performs extremely well on gpus. To use IDG, the `IDG library `_ needs to be available during compilation. See :doc:`installation ` for more information. Considerations -------------- IDG supports the same cleaning and data selections options that WSClean offers in normal mode (without IDG -- so e.g. Cotton-Schwab, multi-frequency multi-scale cleaning, auto-masking, etc.). However there are a few points which require a bit of care to use IDG: * :doc:`Baseline-dependent averaging ` does not work together with IDG and should not be used together. * Imaging multiple polarizations independently (without joining polarizations) is not possible with IDG, see the discussion on polarization settings below. * IDG can only be used on measurement sets that have four polarizations (e.g. XX/XY/YX/YY). Command-line options -------------------- There are several IDG related WSClean parameters: .. code-block:: text -gridder idg This option turns on the IDG gridder. If IDG is not available, WSClean will show an error. .. code-block:: text -idg-mode [cpu/gpu/hybrid] IDG mode to use. The default is CPU, which works on any computer. GPU and Hybrid both make use of the GPU and require the availability of GPU libraries when compiling IDG (see the IDG manual). The "pure" GPU mode is however **not recommended** to be used, as it does not have all the features that the hybrid mode has, hence **hybrid is the recommended method when GPUs are available**. Gridding with the beam ---------------------- IDG allows gridding with a time-variable beam -- currently (as of 2.6), this includes the LOFAR, AARTFAAC and MWA beam. This would increase the accuracy of with which sources are deconvolved (in theory, but it depends of course on the accuracy of the beam model). The option to do this is ``-grid-with-beam``. In summary; IDG with full beam correction: .. code-block:: text wsclean -gridder idg -grid-with-beam ... Note that you do not have to add ``-apply-primary-beam`` to output a pb corrected image: even if you don't specify this parameter, WSClean will grid with the beam and output both the "flat noise" image and the pb corrected image when gridding with the beam. Polarization settings for IDG ----------------------------- IDG always images all Stokes parameters. Therefore, it makes sense to set the polarizations to iquv (with ``-pol iquv``) when using the IDG, as otherwise the other polarizations are imaged and then thrown away. IDG does however require joined or linked polarization cleaning: it is not possible to clean the polarizations individually with IDG in a single run. If this is really desired it can be done by running wsclean 4 times, imaging one polarization at a time, however most science cases will want to use one of the other options described below. There are several distinct cases of interest: * Total intensity science, i.e., not interested in QUV: only image I with ``-pol I``. * Total intensity science, but "nice to have QUV" for e.g. sensivity analysis (V is useful for that): image all polarizations with ``-pol IQUV`` and *link* the deconvolution on polarization I with ``-link-polarization i``. * Interested in rotation measure synthesis, not directly in I and V but nice to have: image all polarizations with ``-pol IQUV`` and link the deconvolution on polarization Q and U with ``-link-polarization qu`` (possibly with squared deconvolution etc., see :doc:`polarized cleaning page `. * Interested in all stokes parameter, cleaning each polarization in a joined way: ``-pol IQUV -join-polarizations``. Some further explanation on this: IDG in combination with ``-pol iquv -join-polarizations`` can be the best choice in some cases, but if one is only interested in Stokes I, it has the downside that QUV are involved in the cleaning of I, and in case these are mostly empty, the net effect is that the noise in the clean-component finding step is increased slightly. Since :doc:`WSClean 2.6 ` it is possible to "link" polarizations. Its use is explained on the :doc:`polarized cleaning page `. To overcome the downside described above, the recommended setting for imaging with IDG is to use polarization linking on I, similar to: .. code-block:: bash wsclean -link-polarization i -gridder idg [..] This will clean Stokes I fully, and clean the components found in I also from the other polarizations. However, it will *not* clean structure from QUV that is not found in I. Performance ----------- On my 4-core home machine, the CPU version is not as fast as the wstacking gridder. However, it can apply *a*-terms and uses less memory, which can be advantageous in some cases. In contrast to the CPU gridder, the GPU gridder *is* considerably faster (up to an order of magnitude), but only for large images (>6k or so). The exact performance benefits depend heavily on image size, bandwidth and number of cores available. Hence if performance is important, I recommend to make a careful comparison for the particular test case you are interested in. IDG has been tested and shown to work on MWA data. It performs well, but does require a lot of memory, caused by the wide field-of-view of the MWA and therefore high w-terms. Advanced *a*-term corrections ----------------------------- WSClean+IDG allows a combination of several direction-dependent corrections to be applied, including TEC screens, diagonal gain correction and position shifts ("dldm screens"). These are discussed on the :doc:`a-term correction page `. Image domain gridding with faceting ----------------------------------- While the option `-gridder idg` uses IDG with screens, i.e. direction-dependent effects are handled by IDG, there's also an option to use IDG with facet corrections: `-gridder facet-idg`, in this case direction-dependent solutions and beam corrections are corrected for each facet. There are some things to take into account when using this mode, mainly that IDG in general grids all (4) instrumental polarizations together, however, combined with faceting `facet-idg` will delegate IDG to grid only a single polarization. Hence, in this mode IDG can grid any polarization provided (not just IQUV). Since parallel gridding is not compatible with IDG, it will likely impact performance when imaging multiple polarizations, though by how much depends on the use-case.