Method 2

Method 2 is to use the DIMM to derive the common mode signal. The common-mode signal will be GAI*COM+FLT. There are a few gotchas to keep in mind though (note that some of these may change as the program gets further optimized):

• The DIMM will do a sc2clean in a preprocessing step which makes it hard to compare the models to the original file. At the least do an external DC removal first (as outlined above). Or do the sc2clean as an external step and switch off all sc2clean parameters in the dimmconfig file submitted to the DIMM.

• The DIMM will add padding on either side of the time series but not remove it from the models before writing them out. There currently is a bug that requires one to use `setorigin' to properly align these cubes with the original one.

• The current FLT model will apodize (i.e. smoothly reduce the ends of the time series to zero; these ends are not used for the mapmaking). The apodization is needed for FLT's current FFT, but a non-apodizing FFT method is under development.

• com.notfirst regulates whether the COM model is run during the first iteration. If set to 1, most of the dominant fridge-related variations will end up in the FLT model. A priori this is not a problem as far as removing the signal is concerned, but the FLT model works on each time series individually i.e. does not really derive a common-mode.

• Related to the previous point, if the fridge variations end up in the FLT model, the GAI model will not really show the gain variation across the bolometers, since the COM model in that case will only contain the residual common-mode variation (often dominated by ringing artefacts from FLT's FFT). i.e. for this exercise we want to make sure that com.notfirst=0.

• The first plane of the GAI model has the multiplicative term but it is distributed around the median (or mean) gain rather than a value of 1 due to the way that the GAI and COM models have been implemented. i.e. divide the gain map by its median in order to get a value distributed around 1. However, note a subtlety: the gain of each bolometer in principle should have been fitted and accounted for by the flat field. Any gain differing from 1 in the GAI model can be interpreted as either a poor flat-field fit, the gain being variable, or some basic bolometers characteristics, e.g. their individual resistances, not (yet) being accounted for sufficiently.

For example, for these observations of CRL618 modify the dimmconfig_bright_compact.lis configuration file as follows. Given that, at the time of writing, the FLT model apodizes, it has been left out.

A reminder: this exercise aims at showing data features and not at showing how well the DIMM can handle these. For the latter one would want to run the DIMM with all its features enabled.

^$STARLINK_DIR/share/smurf/dimmconfig_bright_compact.lis

numiter = 3                        # Just run a few iterations
modelorder = (com,gai,ast)         # Just do common mode part
exportndf = (com,gai,ast,res)      # Write models out
itermap = 1                        # Create map for each iteration
com.gain_box = 600000              # Single gain map for whole spectrum
order = 0                          # Allow for DC level adjustments
dclimcorr = 0                      # No correlated step detection/correction
com.notfirst=0                     # Make sure that COM is run before FLT

The above file exports all relevant models. It produces a moderately smoothed common mode time series and a single gain component for the whole observation. A script that handles combines the output models into a common-mode and common-mode subtracted cube is appended at the end of the document. It actually gives us three useful files to look at: the derived common-mode signal (_commode), the relative gains of the bolometers (_gain), as well as a common-mode subtracted cube (_astres).

The common-mode reduction script is appended at the end of this document.

Figure: Example time series (white) and the derived common-mode signal (red). This time series is the same as the top-left time serie in Figs.  and

Fig.  shows a typical time series with the fitted common-mode signal.

The input cube to makemap had 812 `good' bolometers, the derived gain map 651: makemap has flagged an additional 161 bolometers as bad. A quick inspection of the masked bolometers shows that the majority have steps, increased noise, or multiple spikes. The gain map itself ranges from 0.44 to 1.89 and a histogram shows that of the 651 unflagged bolometers 593 ($\sim$90 per cent) are within a range of [0.75,1.25] and 622 within [0.65,1.35]. To some degree this range indicates that for the S2SRO data the flat field in practice was in general not very accurate or stable probably due to one or more of the aforementioned reasons.

Figure: Histogram of the bolometer gains as derived by the DIMM based on their response to the common-mode signal. Note that these gains are, by default, only used for the subtraction of the common mode and not for the subsequent gridding into a map.

For a further analysis one can also e.g. collapse the common-mode subtracted cube over the time-series to calculate the median and rms:

% collapse ${file}_astres ${file}_astres_median estimator=median \
           axis=3 variance=false wlim=0.0
% collapse ${file}_astres ${file}_astres_rms estimator=rms \
           axis=3 variance=false wlim=0.0

The median signal ranges from $-$33e-04 to 30e-04, with 582 bolometers falling within a range of $-$5e-04 to 5e-04. The median rms is 3e-03 with a maximum of 14e-03 and 578 bolometers below a rms of 6e-03 (twice the median). The three panels in Fig.  summarize this information.

Figure: Left: Gain image: black $<$ 0.75, white $>$ 1.25; Middle: Common-mode subtracted median: black $< -$5e-04, white $>$ 5e-04 Right: Common-mode subtracted rms: white $>$ 6e-03 (twice the median).

% thresh ${file}_gain'(,,~1)' temp \
         thrlo=0.75 thrhi=1.25 newlo=0.0 newhi=2.0
% thresh temp ${file}_gainmsk \
         thrlo=1.5 thrhi=0.5 newlo=1.0 newhi=1.0
% thresh ${file}_astres_median'(,,~1)' temp \
         thrlo=-5e-04 thrhi=5e-04 newlo=-1.0 newhi=1.0
% thresh temp ${file}_astres_medianmsk \
         thrlo=5e-04 thrhi=-5e-04 newlo=0.0 newhi=0.0
% thresh ${file}_astres_rms'(,,~1)' temp \
         thrlo=0 thrhi=6e-03 newlo=-1.0 newhi=1.0
% thresh temp ${file}_astres_rmsmsk \
         thrlo=6e-03 thrhi=0 newlo=0 newhi=0

The three maps have a significant subset of `flagged' bolometers in common. An inspection of the common-mode subtracted data (_astres) shows that many of these bolometers have (multiple) steps that were not removed by sc2clean. Another subset shows variations that don't seem well modeled by the common-mode signal, although one has be careful not to mark the signature from CRL618 as bad. But even for bolometers that pass through all the selection `filters' there are quite a few that still have spikes, steps, or baseline ripples. Although the mapmaker was deliberately crippled for the above presentation, further development of the mapping algorithms will be needed to optimally handle SCUBA-2 data and produce the best possible maps.

Figure: Sample time-series with a simple common-mode subtracted.