How to create custom color images for DP2 DDFs?

I’m attempting to create custom color images of the Data Preview 2 Deep Drilling fields. To do so, I’m trying to follow the process shown in the 103.9 custom color images Notebook tutorial. Namely, I’m trying to reproduce section 4 of that notebook other regions by changing the coordinates used in Cell 23. My intention is to try to make custom color images that resemble the ones shown in Notebook tutorial 301.3:

In Notebook 103.9, when I replace the coordinates for PSZ2 G309.43-72.86 with those of the DDFS defined in Cell 7 of 103.9 (also the DP2 Observations page), various interesting things happen.

For DDF ELAIS S1, I can successfully create a custom color image leaving the rest of the Notebook unchanged.

For DDF XMM LSS, Cell 26 produces

For DDF ECDFS, DDF EDFS a, DDF EDFS b, and DDF COSMOS, Cell 42 produces

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[42], line 1
----> 1 mosaic_image = task.run(coadds).outputRGB

File /opt/lsst/software/stack/conda/envs/lsst-scipipe-12.3.0-exact/share/eups/Linux64/pipe_tasks/g9efd06e670+00bd727f93/python/lsst/pipe/tasks/prettyPictureMaker/_task.py:508, in PrettyPictureTask.run(self, images, image_wcs, image_box)
    505 for item in to_remove:
    506     local_contrast_config["diffusionControl"].pop(item)
--> 508 colorImage = lsstRGB(
    509     colorImage[:, :, 0],
    510     colorImage[:, :, 1],
    511     colorImage[:, :, 2],
    512     local_contrast=self.config.localContrastConfig if self.config.doLocalContrast else None,
    513     scale_lum=self.config.luminanceConfig,
    514     scale_color=self.config.colorConfig,
    515     remap_bounds=self.config.imageRemappingConfig,
    516     bracketing_function=(
    517         self.config.exposureBracketerConfig if self.config.doExposureBrackets else None
    518     ),
    519     gamut_remapping_function=self.config.gamutMapperConfig if self.config.doRemapGamut else None,
    520     cieWhitePoint=tuple(self.config.cieWhitePoint),  # type: ignore
    521     psf=psf if self.config.doPsfDeconvolve else None,
    522 )
    524 # Find the dataset type and thus the maximum values as well
    525 maxVal: int | float

File /opt/lsst/software/stack/conda/envs/lsst-scipipe-12.3.0-exact/share/eups/Linux64/pipe_tasks/g9efd06e670+00bd727f93/python/lsst/pipe/tasks/prettyPictureMaker/_colorMapper.py:199, in lsstRGB(rArray, gArray, bArray, local_contrast, scale_lum, scale_color, remap_bounds, bracketing_function, gamut_remapping_function, psf, cieWhitePoint)
    196     t1 = time.time()
    198 if scale_color is not None:
--> 199     new_a, new_b = scale_color(lum_save, lum, Lab[..., 1], Lab[..., 2])
    200     Lab[..., 1] = new_a
    201     Lab[..., 2] = new_b

File /opt/lsst/software/stack/conda/envs/lsst-scipipe-12.3.0-exact/share/eups/Linux64/pipe_tasks/g9efd06e670+00bd727f93/python/lsst/pipe/tasks/prettyPictureMaker/_functors/_color_scale.py:124, in ColorScaler.__call__(self, old_lum, new_lum, a, b)
    119 else:
    120     correction_factor = np.arcsinh(abs(new_lum - 1) * self.brightFalloff) / np.arcsinh(
    121         self.brightFalloff
    122     )
    123     chroma2_2 = (
--> 124         self.saturation
    125         * correction_factor**2
    126         * sat_original_2
    127         * new_lum**2
    128         / (1 - sat_original_2 * correction_factor**2)
    129     )
    131 # Compute new 'a' values using the square root of adjusted chroma and
    132 # considering hue direction.
    133 chroma2 = np.sqrt(chroma2_2)

ValueError: operands could not be broadcast together with shapes (1842,1842) (1841,1841) 

I’m wondering if perhaps the method shown in tutorial Notebook 103.9 isn’t appropriate for the DDFs somehow, or if I’m doing something else wrong. I’ve kept the various pretty picture configurations at their default values. The value error shown above is familiar to me in other contexts but I don’t see an obvious explanation for what’s going on.

I’ve used other tutorial Notebooks and Firefly to confirm that I can indeed access the DDF images. Any pointers would be very helpful. Thank you in advance!

Hi Melanie – thanks for the detailed post. I’ve again inquired with the Rubin team member who led the creation of DP2 notebook tutorial 103.9, but given the timing, I wouldn’t expect a complete set of answers until this coming week.

Anyway, to address the first error (for DDF XMM LSS), DP2 notebook tutorial 301.3 says:

The XMM LSS field does have visit observations, but is not included in the deep coadd area for DP2.

So that explains why the query for deep_coadd products at the nominal central coordinates of DDF XMM LSS comes up empty, ultimately leading to the EmptyQueryResultError (which I was indeed able to reproduce).

Hi Melanie, the original DP1 version of this tutorial ( 103.6. Color composite Image — DP1 ) had a warning to set a parameter pretty_pic_config.localContrastConfig.doLocalContrast = False which related to an image size error (see section 2.1). The coordinates you mentioned seem to throw up the same value error so the recommended solution for now is set the above parameter to false as well (at the cost of a bit of detail).

I will mark this as the solution but please let us know if it is not working for you. We will probably update the tutorials with this fix, or add some additional checks on the sizes of the input images.