# Datasets for Image Difference Analysis

**URL:** <https://www.rubin.community/t/datasets-for-image-difference-analysis/4415>\
**Category:** Rubin Science Platform\
**Tags:** decam, imagedifference, ingestion\
**Created:** [September 22, 2020, 3:08pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415 "2020-09-22T15:08:38Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\
**Post date:** [September 22, 2020, 3:08pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/1 "2020-09-22T15:08:38Z")

</div>

Hi community,

As I wrote in previous posts, I’m trying to learn the usage of the _ip.diffim package_. I would like to know if it is possible somehow to ingest externally-processed images and if there are test datasets I can use to run the task and see the output.

Thank you in advance!

---

<div class="post-metadata">

**Author:** ![ebellm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ebellm/32/771_2.png) [@ebellm](https://www.rubin.community/u/ebellm)\
**Post date:** [September 22, 2020, 7:37pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/2 "2020-09-22T19:37:50Z")

</div>

The alert production pipelines (which include `ip.diffim`) have some pre-packaged datasets–see [here](https://pipelines.lsst.io/modules/lsst.ap.verify/) for more information.

---

<div class="post-metadata">

**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\
**Post date:** [September 23, 2020, 12:20pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/3 "2020-09-23T12:20:25Z")

</div>

Thank you for the answer.  
Is it possible to launch just the image difference task with externally-processed data or do I need to launch the entire alert production pipelines with raw images?

---

<div class="post-metadata">

**Author:** ![ebellm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ebellm/32/771_2.png) [@ebellm](https://www.rubin.community/u/ebellm)\
**Post date:** [September 24, 2020, 9:27pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/4 "2020-09-24T21:27:47Z")

</div>

I’m not sure what you mean by “externally-processed.” It might be helpful to know what telescope/camera the data come from.

---

<div class="post-metadata">

**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\
**Post date:** [September 25, 2020, 9:39am UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/5 "2020-09-25T09:39:30Z")

</div>

I have some reduced VST images which I would like to test with the image difference analysis procedure. Is it possible or should I ingest all the raw the data?

---

<div class="post-metadata">

**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\
**Post date:** [September 28, 2020, 3:59pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/6 "2020-09-28T15:59:03Z")

</div>

Any idea on how to launch the image difference task with reduced images?

---

<div class="post-metadata">

**Author:** ![ebellm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ebellm/32/771_2.png) [@ebellm](https://www.rubin.community/u/ebellm)\
**Post date:** [September 29, 2020, 8:32pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/7 "2020-09-29T20:32:48Z")

</div>

To process images from any camera you need some translation software (called an “obs package” in the Rubin pipelines) to convert the data into types the pipelines can understand. [Here’s](https://github.com/lsst/obs_subaru/) an example.

Unfortunately there is no such package for VST, and writing a new one is a lot of work and requires specialized knowledge. So I don’t think there’s an easy way to simply run the Rubin pipelines on your data.

---

<div class="post-metadata">

**Author:** ![MelissaGraham](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/melissagraham/32/2549_2.png) [@MelissaGraham](https://www.rubin.community/u/MelissaGraham)\
**Post date:** [October 21, 2020, 11:12pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/8 "2020-10-21T23:12:44Z")

</div>

Hi @Vincenzo, just checking in to see if you decided to try writing an `obs package` for your VST data, or if you decided to go another route?

---

<div class="post-metadata">

**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\
**Post date:** [October 22, 2020, 10:45am UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/9 "2020-10-22T10:45:55Z")

</div>

Hi @MelissaGraham , at the moment I have decided to look at the results of the _alert production_ pipelines on tested datasets which already have an _obs package_ and I am trying to produce light curves for certain sources of interest. Maybe later I could try writing an _obs package_ for VST data, but not in short times.

---

<div class="post-metadata">

**Author:** ![paolillo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/paolillo/32/1245_2.png) [@paolillo](https://www.rubin.community/u/paolillo)\
**Post date:** [October 22, 2020, 1:31pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/10 "2020-10-22T13:31:32Z")

</div>

Hi Melissa, do you know anybody interested in writing such package for VST? I will ask the italian community but I want to avoid duplicated efforts and use all the available expertise.

---

<div class="post-metadata">

**Author:** ![MelissaGraham](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/melissagraham/32/2549_2.png) [@MelissaGraham](https://www.rubin.community/u/MelissaGraham)\
**Post date:** [October 22, 2020, 3:49pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/11 "2020-10-22T15:49:05Z")

</div>

Hi @paolillo, no, unfortunately I don’t know of anyone writing an `obs package` for the VST. Perhaps you could post a new Topic in the [Science](http://www.rubin.community/c/sci/23) and/or [TVS-SC](http://www.rubin.community/c/sci-collabs/tvs-sc/29) categories describing what you’re trying to do, and see if you find anyone to collaborate with?

---

<div class="post-metadata">

**Author:** ![ebellm](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ebellm/32/771_2.png) [@ebellm](https://www.rubin.community/u/ebellm)\
**Post date:** [October 22, 2020, 5:27pm UTC](https://www.rubin.community/t/datasets-for-image-difference-analysis/4415/12 "2020-10-22T17:27:47Z")

</div>

I wasn’t aware of this earlier, but [https://github.com/lsst-uk/obs\_vista](https://github.com/lsst-uk/obs_vista) supports VIRCAM. @Vincenzo @paolillo maybe this will help?
