Hi everyone :slight_smile: I’m currently trying to ingest the raw and calibrated images from DECAM following the gen 3 approach of processing images. So, the data that I’m working with was downloaded from the Noir Lab archive, and I organized the raw images by folders of pointings. Meanwhile, the…

I’ve made some progress on this. You can try the following commands, where REPO is the path to your gen3 butler repo. Note that you’ll need a pretty recent pipelines version, as butler register-dataset-type was added at the end of August. Create an astropy-readable table file containing one row pe…

I think K-T is correct. v21 is too old, unfortunately, and you will need to set up a recent weekly or wait for v22 (soon!). You may be able to get past this particular roadblock by setting up a local clone of ap_pipe along with cp_pipe so it finds the correct pipeline files to import, but I can pret…

Okay :slight_smile: thank you both for your input. I was able to install the newest weekly version of the LSST science pipelines :D, and following the draft guide I got a FileNotFoundError (lsst-scipipe-0.6.0) jahumada@leftraru1:/mnt/flock/jahumada/data_hits$ pipetask run -b . -i DECam/raw/all -o D…

[image] Paula: -b . I would advise against working inside your butler repository. You don’t want to delete that gen3.sqlite3 file by mistake and you will have to remind yourself which files you wrote and which files butler wrote. [image] Paula: -p Traceback Looks like your copy and pa…

Okay!, I will work outside the butler repository from now on:). And yes, the -p Traceback was a typo of copying the error with the command line, I will edit it! sorry for that [now the command line is complete]. I did run butler write-curated-calibrations . lsst.obs.decam.DarkEnergyCamera inside th…

As long as you update your input collections (-i DECam/raw/all,DECam/calib), the pipeline should run.

Ok :slight_smile: I used DECam/calib as a second input to the pipetask run and got the following: (lsst-scipipe-0.6.0) [jahumada@leftraru4 jahumada]$ pipetask run -b /mnt/flock/jahumada/data_hits -i DECam/raw/all,DECam/calib -o DECam/raw/crosstalk-sources -p $CP_PIPE_DIR/pipelines/DarkEnergyCamera/…

I think you just need to append --register-dataset-types to the end of your pipetask run command :crossed_fingers: This tells the Butler you intend to create (“register”) a new kind of output dataset it has not seen before.

Ok thanks :slight_smile: adding the --register-dataset-types tag in the end of the pipetask command did avoid the error I was encountering :D. I have been compiling pipetask run -b /mnt/flock/jahumada/data_hits -i DECam/raw/all,DECam/calib -o DECam/raw/crosstalk-sources -p $CP_PIPE_DIR/pipelines/Dar…

We sometimes see errors like this if a disk fills up. Could that be the case here?

Issues with ingestCalib.py using DECAM images

Support

timj (Tim Jenness) July 7, 2021, 9:56pm 13

Use that as a second input collection for your pipetask run. The graph builder needs to know where to find the inputs so you need to tell it what raws to use and what calibrations to use. Your previous command only specified the raw location.

In DECam data reduction, I faced a FileNotFoundError in "Ingest Raw Images" step

Topic		Replies	Views
Gen3 ProcessCcd with DECam Support processccd , decam	18	3919	October 15, 2021
Problem ingesting/applying DECam calibrations generated from raw data Support	1	883	February 2, 2021
processCcd with Obs_decam "No locations for get:" Support	4	1318	December 12, 2018
Error while trying to create master bias frames Support science-pipelines , pipetask	5	1808	June 23, 2022
Problems while trying to create a new LSST obs packag Support butler	26	5321	April 10, 2023

Issues with ingestCalib.py using DECAM images

Related topics