# HSC images ingestion for image differencing **URL:** \ **Category:** Support\ **Tags:** imagedifference, hsc, ingestion\ **Created:** [June 5, 2020, 2:22pm UTC](https://www.rubin.community/t/hsc-images-ingestion-for-image-differencing/4187 "2020-06-05T14:22:39Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\ **Post date:** [June 5, 2020, 2:22pm UTC](https://www.rubin.community/t/hsc-images-ingestion-for-image-differencing/4187/1 "2020-06-05T14:22:39Z")
Hi community, I was trying to run the image differencing on two downloaded HSC images of the same field, but I have a problem with the ingestion. I put the files into a _test_ directory, with the images into a _img_ folder and an _ups_ directory with the file _test.table_, trying to reply the structure of the tutorial testdata\_ci\_hsc. Then, I have launched these commands: > setup -j -r test > echo $TEST\_DIR > mkdir DATA > echo “lsst.obs.hsc.HscMapper” \> DATA/\_mapper > ingestImages.py DATA $test/img/\*.fits --mode=copy But I got this error message: > root INFO: Loading config overrride file ‘/home/user/lsst\_stack/stack/miniconda3-4.7.10-4d7b902/Linux64/obs\_subaru/19.0.0+2/config/ingest.py’ > HscMapper WARN: Unable to find calib root directory > CameraMapper INFO: Loading exposure registry from /home/user/Scrivania/DATA/registry.sqlite3 > ingest WARN: /img/\*.fits doesn’t match any file > ingest.register INFO: Table “raw” exists. Skipping creation Any idea of what can be the problem? What I would like to do is to use the `lsst.ip.diffim` module, but the images of the tutorial doesn’t seem to be suitable for this purpose. If you could indicate me a set of tested images to launch the task and see the output or any suggested reading about it, I would appreciate it a lot. ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [June 5, 2020, 4:14pm UTC](https://www.rubin.community/t/hsc-images-ingestion-for-image-differencing/4187/2 "2020-06-05T16:14:34Z")
This one looks simple: `/img/*.fits doesn't match any file` is output because `$test` doesn’t have a value. I think you want `$TEST_DIR` there instead. ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [June 5, 2020, 4:16pm UTC](https://www.rubin.community/t/hsc-images-ingestion-for-image-differencing/4187/3 "2020-06-05T16:16:28Z")
> [@Vincenzo](#): > > What I would like to do is to use the `lsst.ip.diffim` module, but the images of the tutorial doesn’t seem to be suitable for this purpose. If you could indicate me a set of tested images to launch the task and see the output or any suggested reading about it, I would appreciate it a lot. Someone more knowledgeable about difference imaging will have to provide this tutorial information. ---
**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\ **Post date:** [June 5, 2020, 5:25pm UTC](https://www.rubin.community/t/hsc-images-ingestion-for-image-differencing/4187/4 "2020-06-05T17:25:51Z")
> [@ktl](#): > > This one looks simple: `/img/*.fits doesn't match any file` is output because `$test` doesn’t have a value. I think you want `$TEST_DIR` there instead. Sorry, it was a distraction error. The error message that I got with the correct procedure is the following: > ingestImages.py DATA $TEST\_DIR/img/\*.fits --mode=copy > root INFO: Loading config overrride file ‘/home/user/lsst\_stack/stack/miniconda3-4.7.10-4d7b902/Linux64/obs\_subaru/19.0.0+2/config/ingest.py’ > HscMapper WARN: Unable to find calib root directory > CameraMapper INFO: Loading Posix exposure registry from /home/user/Scrivania/DATA > ingest.parse WARN: Unable to find value for proposal (derived from PROP-ID) > ingest.parse WARN: Unable to find value for dataType (derived from DATA-TYP) > ingest.parse WARN: Unable to find value for ccd (derived from DET-ID) > ingest.parse WARN: Unable to find value for pa (derived from INST-PA) > ingest.parse WARN: Unable to find value for config (derived from T\_CFGFIL) > ingest.parse WARN: Unable to find value for frameId (derived from FRAMEID) > ingest.parse WARN: Unable to find value for expId (derived from EXP-ID) > ingest.parse WARN: Unable to find value for dateObs (derived from DATE-OBS) > ingest.parse WARN: Unable to find value for taiObs (derived from DATE-OBS) > ingest.parse WARN: translate\_field failed to translate field: ‘OBJECT not found’ > ingest.parse WARN: translate\_visit failed to translate visit: ‘EXP-ID not found’ > ingest.parse WARN: Unable to determine suitable ‘pointing’ value; using 0 > ingest WARN: Failed to ingest file /home/user/Scrivania/images/img/cutout-HSC-R-9813-pdr1\_deep-200514-123452.fits: ‘visit’ > ingest.parse WARN: Unable to find value for proposal (derived from PROP-ID) > ingest.parse WARN: Unable to find value for dataType (derived from DATA-TYP) > ingest.parse WARN: Unable to find value for expTime (derived from EXPTIME) > ingest.parse WARN: Unable to find value for ccd (derived from DET-ID) > ingest.parse WARN: Unable to find value for pa (derived from INST-PA) > ingest.parse WARN: Unable to find value for config (derived from T\_CFGFIL) > ingest.parse WARN: Unable to find value for frameId (derived from FRAMEID) > ingest.parse WARN: Unable to find value for expId (derived from EXP-ID) > ingest.parse WARN: translate\_field failed to translate field: ‘OBJECT not found’ > ingest.parse WARN: translate\_visit failed to translate visit: ‘EXP-ID not found’ > ingest WARN: Failed to ingest file /home/user/Scrivania/images/img/cutout-HSC-R-9813-pdr2\_wide-200514-123417.fits: ‘visit’ Do I need to modify something into the obs.package for a generic ingestion? Thank you again! ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [June 5, 2020, 6:19pm UTC](https://www.rubin.community/t/hsc-images-ingestion-for-image-differencing/4187/5 "2020-06-05T18:19:40Z")
> [@Vincenzo](#): > > EXP-ID not found The `ingestImages.py` script may not be the appropriate tool for what you want to do. It is intended to ingest raw, unprocessed images into a Butler repository. I’m guessing from the filename and lack of appropriate header that this is not such an image. With the possible exception of DECam, I believe the pipelines are not typically set up (at this time) to work with externally-processed images. It should be possible to do so, but I suspect it may involve considerable work: ingesting these into a Butler repository, adding any additional data products expected by the pipeline, and/or adjusting default configurations. ---
**Author:** ![Vincenzo](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/vincenzo/32/1222_2.png) [@Vincenzo](https://www.rubin.community/u/Vincenzo)\ **Post date:** [June 10, 2020, 8:38pm UTC](https://www.rubin.community/t/hsc-images-ingestion-for-image-differencing/4187/6 "2020-06-10T20:38:43Z")
Thank you for the clear answer.