Detailed operation

Figure: Flow chart depicting operation of the dynamic iterative map-maker (DIMM).

Most of the processing steps followed by the DIMM are controlled by a configuration file (see configuration file parameters for the makemap task). An overview of the DIMM operation is shown in Figure , and the numbered boxes are described below:

1. Concatenate:

The makemap task typically takes raw SCUBA-2 data as its input. The best maps are produced by first concatenating as much data from a given observation as possible into single continuous data streams in memory. This step is controlled by the MAXLEN configuration file parameter, as well as the makemap parameter MAXMEM. When the files are concatenated, it is also possible to add extra padding at the beginning and end of the data streams to facilitate filtering (see the PAD and ZEROPAD configuration file parameters). This operation is equivalent to using the sc2concat task.

Whether concatenation is requested or not, it is also worth noting that makemap automatically applies the internally stored flatfield as data files are loaded, so that they have units of pW before estimating the map.

2. Pre-process:

Before the iterative process begins, several pre-processing (data cleaning) steps may be applied. See the configuration file parameters APOD, ORDER, BADFRAC, FLAGSTAT, DCTHRESH, DCBOX, SPIKETHRESH, SPIKEITER, FILT_EDGEHIGH, FILT_EDGELOW, FILT_NOTCHHIGH and FILT_NOTCHLOW. These options provide the same functionality as the sc2clean task. It should be noted that the default configuration file does not perform many of these pre-processing steps; the best results are obtained by flagging spikes, filtering etc., using equivalent operations that can be executed during the iterative step.

3. Fit pre-map estimate models:

Once the iterative process has begun, models are fit to the data in the order described by the MODELORDER configuration file parameter (see Table ). The default order is $COM, GAI, EXT, AST, FLT, NOI$. Components specified before $AST$ are signals that are generally brighter than the astronomical signal. $COM$ and $GAI$ are both used to estimate and remove the bright common-mode signal. $EXT$ is a special model that applies the extinction correction determined from external sensors (see Section ). Rather than subtracting a signal component, $EXT$ applies a time-varying scale factor to each detector, so that the magnitude of the model components calculated before and after $EXT$ are different - total power received by the detectors and power incident on the top of the earth's atmosphere, respectively.

4. Estimate map:

The location of the $AST$ component in MODELORDER indicates when the astronomical image should be estimated. With the default settings $COM$ is fit and removed from the data first, and the extinction correction applied (such that the map has meaningful physical units as mentioned above). Then, once $AST$ is encountered, the signals are re-gridded using nearest-neighbour sampling to produce an estimate of the map. Since many samples typically contribute to the estimate of the signal in a given pixel, the noise is greatly reduced compared to the time-series data. Using the pointing solution, the map is then projected back into the time-domain (effectively the result of `scanning' each detector across the map), storing it as the $AST$ signal component, and then removing it from the detector time streams. This model component is special since both the map and $AST$ are estimated by this step.

5. Fit post-map estimate models:

Once the map has been estimated and the astronomical signal removed from the data, the residual should contain only noise. However, this signal often contains a weak drift ($N_{\rm lf}$ in Eq. ) that is independent from one detector to the next. By specifying the $FLT$ model component after $AST$, it is possible to remove this component using, for example, a low-pass filter. It is advisable to apply this filter after estimating the map to avoid causing ringing around bright astronomical sources. However, in this case at least two iterations are required for this component to have any effect on the estimated map. It is also advisable to specify $NOI$ as the final model component. This model calculates the r.m.s. noise in each detector, and is required if a $\chi^2$ stopping criterion has been requested. Furthermore, the measured r.m.s. values are used to weight each detector in the map estimate. If $NOI$ has not been specified each detector is given the same weight, which can be highly non-optimal if the detectors exhibit a wide range of sensitivities.

6. Check for convergence:

Finally, the solution is checked for convergence - either by reaching the number of pre-defined iterations requested, or because $\chi^2$ has changed by less than the CHITOL value specified in the configuration file. In the latter case, $\chi^2$ is calculated in the following way. In the first iteration, $NOI$ estimates the white-noise contribution to the r.m.s., $\sigma_{\rm w}$, in each detector directly from the flat part of their power spectra (presently defined over the frequency range 2 to 10 Hz). If any astronomical signal, or low-frequency signal components are present in the data, the r.m.s. of the data stream will be much larger than this (the integral over the entire power spectrum). However, as the solution converges, the residual signal should slowly approach a white noise distribution, once the other signal components are estimated and removed. $\chi^2$ is therefore calculated as $(1/N) \sum_i[ r_i^2/\sigma^2_{\rm w}]$, where $r_i$ is the $i$th residual sample for a given detector, and $N$ is the total number of samples for that detector. This number, averaged over all samples and detectors, should tend to 1 as the solution converges, although in practice it will be off by some factor related to the bandwidth used to calculate $\sigma_{\rm w}$.

The final map estimate, the model signal components, and the residual signal may then all be exported to files for examination (see the EXPORTNDF configuration file parameter).