# Problems with tutorial 4 of the LSST Science Pipelines: Global Caibration **URL:** \ **Category:** Support\ **Created:** [February 24, 2023, 2:38pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445 "2023-02-24T14:38:28Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![valeriocarruba](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/valeriocarruba/32/1968_2.png) [@valeriocarruba](https://www.rubin.community/u/valeriocarruba)\ **Post date:** [February 24, 2023, 2:38pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/1 "2023-02-24T14:38:28Z")
Dear Sir or Madam, We are having trouble with the tutorial 4 of the LSST Science Pipelines. While we run tutorial 2 perfectly, when we tried the command: pipetask run -b $RC2\_SUBSET\_DIR/SMALL\_HSC/butler.yaml -p $RC2\_SUBSET\_DIR/pipelines/DRP.yaml#fgcm -i u/$USER/single\_frame -o u/$USER/fgcm –register-dataset-types We got the error message below. We looked for previous comments on this tutorial, and we could not find anything. A post for tutorial 2 that suggested problems of compatibility between the lsst science pipeline and the rc2\_subset data was all that we could find. The solution suggested did not work for us. Can anyone please help us? Thank you, lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback: Traceback (most recent call last): File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/ctrl\_mpexec/g89ce1456cb+8822c702cc/python/lsst/ctrl/mpexec/cli/cmd/commands.py”, line 159, in run pipeline = script.build(\*\*kwargs, show=show) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/ctrl\_mpexec/g89ce1456cb+8822c702cc/python/lsst/ctrl/mpexec/cli/script/build.py”, line 87, in build pipeline = f.makePipeline(args) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/ctrl\_mpexec/g89ce1456cb+8822c702cc/python/lsst/ctrl/mpexec/cmdLineFwk.py”, line 493, in makePipeline pipeline = Pipeline.from\_uri(args.pipeline) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/pipe\_base/gb2e446f0d1+d32b27b84e/python/lsst/pipe/base/pipeline.py”, line 330, in from\_uri pipeline: Pipeline = cls.fromIR(pipelineIR.PipelineIR.from\_uri(uri)) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/pipe\_base/gb2e446f0d1+d32b27b84e/python/lsst/pipe/base/pipelineIR.py”, line 881, in from\_uri with loaded\_uri.open(“r”) as buffer: File “/home/valerio/lsstw/lsstsw/miniconda/envs/lsst-scipipe-5.1.0/lib/python3.10/contextlib.py”, line 135, in **enter** return next(self.gen) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/resources/g2029d1bf85+0e2c4d26cd/python/lsst/resources/file.py”, line 488, in open with open(self.ospath, mode=mode, encoding=encoding) as buffer: FileNotFoundError: [Errno 2] No such file or directory: ‘/home/valerio/lsstw/lsstsw/rc2\_subset/pipelines/DRP.yaml’ ---
**Author:** ![yusra](https://avatars.discourse-cdn.com/v4/letter/y/f4b2a3/32.png) [@yusra](https://www.rubin.community/u/yusra)\ **Post date:** [February 24, 2023, 4:43pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/2 "2023-02-24T16:43:43Z")
I do think this is because you have a mismatch in the version of your tutorial and the version of the pipelines you’re running. Looks like you’re using a very recent version of RC2\_subset and an older version of the tutorial. To find the tutorial corresponding to the most recently weekly. please use: [The LSST Science Pipelines — LSST Science Pipelines](https://pipelines.lsst.io/v/weekly/index.html) ---
**Author:** ![valeriocarruba](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/valeriocarruba/32/1968_2.png) [@valeriocarruba](https://www.rubin.community/u/valeriocarruba)\ **Post date:** [February 28, 2023, 2:40pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/3 "2023-02-28T14:40:57Z")
Thank you for the input. We tried the updated version of the tutorial, but we still got an error message (see below). Any other suggestion would be much appreciated. lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback: Traceback (most recent call last): File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/ctrl\_mpexec/g89ce1456cb+8822c702cc/python/lsst/ctrl/mpexec/cli/cmd/commands.py”, line 159, in run pipeline = script.build(\*\*kwargs, show=show) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/ctrl\_mpexec/g89ce1456cb+8822c702cc/python/lsst/ctrl/mpexec/cli/script/build.py”, line 87, in build pipeline = f.makePipeline(args) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/ctrl\_mpexec/g89ce1456cb+8822c702cc/python/lsst/ctrl/mpexec/cmdLineFwk.py”, line 493, in makePipeline pipeline = Pipeline.from\_uri(args.pipeline) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/pipe\_base/gb2e446f0d1+d32b27b84e/python/lsst/pipe/base/pipeline.py”, line 330, in from\_uri pipeline: Pipeline = cls.fromIR(pipelineIR.PipelineIR.from\_uri(uri)) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/pipe\_base/gb2e446f0d1+d32b27b84e/python/lsst/pipe/base/pipelineIR.py”, line 881, in from\_uri with loaded\_uri.open(“r”) as buffer: File “/home/valerio/lsstw/lsstsw/miniconda/envs/lsst-scipipe-5.1.0/lib/python3.10/contextlib.py”, line 135, in **enter** return next(self.gen) File “/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/resources/g2029d1bf85+0e2c4d26cd/python/lsst/resources/file.py”, line 488, in open with open(self.ospath, mode=mode, encoding=encoding) as buffer: FileNotFoundError: [Errno 2] No such file or directory: ‘/home/valerio/lsstw/lsstsw/stack/lsst-scipipe-5.1.0/Linux64/drp\_pipe/gb906b4b701+012d13edcd/pipelines/HSC/DRP-RC2\_subset.yaml’ ---
**Author:** ![jeffcarlin](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jeffcarlin/32/491_2.png) [@jeffcarlin](https://www.rubin.community/u/jeffcarlin)\ **Post date:** [March 2, 2023, 1:36am UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/4 "2023-03-02T01:36:13Z")
Hi Valerio - We have been finding some issues with the versions of the tutorial not matching up with their respective releases. Here’s what you can try: Make sure you are using v24.0.0 of the pipelines. See what you have set up by typing `eups list lsst_distrib` – this will show you all available versions. If the word “current” doesn’t appear next to `v24_0_0` in the list, then you will want to explicitly set up that version using `setup -t v24_0_0 lsst_distrib`. Once you have set up the v24.0.0 version of the pipelines, you should also make sure you check out the v24 tag of `rc2_subset`. After that, the tutorial at [this link](https://pipelines.lsst.io/v/v24_0_0/index.html) should work for you (note that it has v24\_0\_0 in the URL). Let me/us know if you still have issues. ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [March 2, 2023, 3:46am UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/5 "2023-03-02T03:46:10Z")
Typing `eups list -s lsst_distrib` will show only the version that has been setup, making it easier to check which version that is. In Jeff’s command’s output, the word `setup` is more critical than the word `current` (which is the default version that will be setup if no other is specified). ---
**Author:** ![valeriocarruba](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/valeriocarruba/32/1968_2.png) [@valeriocarruba](https://www.rubin.community/u/valeriocarruba)\ **Post date:** [March 3, 2023, 2:05pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/6 "2023-03-03T14:05:05Z")
Thank you for your suggestions. Indeed, we will need to install v24.0.0 on one of our machine. There were problems in doing that in an Ubuntu OS, so we will try to install CentOS. This may take a while, I will come back to this as soon as possible. Thank you again. ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [March 3, 2023, 2:06pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/7 "2023-03-03T14:06:54Z")
Please let us know what problems you had on Ubuntu. While not an official platform, this should work, and we use Ubuntu ourselves in various places. ---
**Author:** ![valeriocarruba](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/valeriocarruba/32/1968_2.png) [@valeriocarruba](https://www.rubin.community/u/valeriocarruba)\ **Post date:** [March 3, 2023, 2:25pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/8 "2023-03-03T14:25:21Z")
On our machine the installation with the procedure described at: [https://pipelines.lsst.io/install/newinstall.html](https://pipelines.lsst.io/install/newinstall.html) Failed at different stages of the command: eups distrib install -t v24\_0\_0 lsst\_distrib We tried with many different machines. At the end, what worked was the installation through the procedure at: [https://pipelines.lsst.io/install/lsstsw.html](https://pipelines.lsst.io/install/lsstsw.html) This worked fine up to the tutorial 3. For the tutorial 4, we observed the problems described in this posts. We will try again with a machine operating CentOS. Please let me know if you have other questions. ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [March 3, 2023, 2:41pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/9 "2023-03-03T14:41:37Z")
What error messages did you receive during `eups distrib install`? ---
**Author:** ![valeriocarruba](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/valeriocarruba/32/1968_2.png) [@valeriocarruba](https://www.rubin.community/u/valeriocarruba)\ **Post date:** [March 3, 2023, 2:53pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/10 "2023-03-03T14:53:41Z")
This is the latest error message, for package 31 afw: [31/85] afw gb77f0a74f7+8a093cac5b … \*\*\*\*\* error: from /home/valerio/SCRIPTS/LSST/lsst\_track/stack/miniconda3-py38\_4.9.2-5.0.0/EupsBuildDir/Linux64/afw-gb77f0a74f7+8a093cac5b/build.log: 428 | typename Ref\::type\>::type apply\_IMV(int x, int y) { | ^~ include/lsst/afw/image/MaskedImage.h:428:65: error: template argument 1 is invalid 428 | typename Ref\::type\>::type apply\_IMV(int x, int y) { | ^ In file included from src/detection/Psf.cc:7: include/lsst/afw/detection/Psf.h: In member function ‘lsst::geom::Box2I lsst::afw::detection::Psf::computeKernelBBox() const’: include/lsst/afw/detection/Psf.h:333:28: warning: ‘lsst::geom::Box2I lsst::afw::detection::Psf::computeBBox() const’ is deprecated: Default position argument overload is deprecated and will be removed in version 24.0. Please use overload with explicit position. [-Wdeprecated-declarations] 333 | return computeBBox(); | ^ include/lsst/afw/detection/Psf.h:297:23: note: declared here 297 | lsst::geom::Box2I computeBBox() const; | ^ ~~~~~~~~~~ scons: \*\*\* [src/detection/Threshold.os] Error 1 scons: \*\*\* [src/detection/FootprintMerge.os] Error 1 scons: \*\*\* [src/detection/HeavyFootprint.os] Error 1 scons: \*\*\* [src/detection/Footprint.os] Error 1 scons: \*\*\* [src/detection/FootprintSet.os] Error 1 scons: building terminated because of errors. - exit -4 eups distrib: Failed to build afw-gb77f0a74f7+8a093cac5b.eupspkg: Command: source “/home/valerio/miniconda3/envs/lsst-scipipe-5.0.0/eups/bin/setups.sh”; export EUPS\_PATH="/home/valerio/SCRIPTS/LSST/lsst\_track/stack/miniconda3-py38\_4.9.2-5.0.0"; (/home/valerio/SCRIPTS/LSST/lsst\_track/stack/miniconda3-py38\_4.9.2-5.0.0/EupsBuildDir/Linux64/afw-gb77f0a74f7+8a093cac5b/build.sh) \>\> /home/valerio/SCRIPTS/LSST/lsst\_track/stack/miniconda3-py38\_4.9.2-5.0.0/EupsBuildDir/Linux64/afw-gb77f0a74f7+8a093cac5b/build.log 2\>&1 4\>/home/valerio/SCRIPTS/LSST/lsst\_track/stack/miniconda3-py38\_4.9.2-5.0.0/EupsBuildDir/Linux64/afw-gb77f0a74f7+8a093cac5b/build.msg exited with code 252 Sincerely, ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [March 3, 2023, 3:15pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/11 "2023-03-03T15:15:24Z")
You seem to have the wrong version of the rubin-env environment. v24\_0\_0 should be built using lsst-scipipe-4.0.5, not 5.0.0. With that fixed, you can also try a binary install by doing `export EUPS_PKGROOT=https://eups.lsst.codes/stack/redhat/el7/conda-system/miniconda3-py38_4.9.2-4.0.5` before your `eups distrib install` command. You could also try using `lsstinstall` as documented here: [Install with lsstinstall and eups distrib — LSST Science Pipelines](https://pipelines.lsst.io/install/lsstinstall.html) ---
**Author:** ![valeriocarruba](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/valeriocarruba/32/1968_2.png) [@valeriocarruba](https://www.rubin.community/u/valeriocarruba)\ **Post date:** [March 6, 2023, 1:31pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/12 "2023-03-06T13:31:57Z")
Hi, We tried using version 24 for both the science pipelines and the rc\_subset data, as suggested. Below there is the latest error message, what are we doing wrong? We will try again on a CentOS machine, but this will take some time… Thank you, lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback: Traceback (most recent call last): File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/ctrl\_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cli/cmd/commands.py”, line 130, in run qgraph = script.qgraph(pipelineObj=pipeline, \*\*kwargs) File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/ctrl\_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cli/script/qgraph.py”, line 187, in qgraph qgraph = f.makeGraph(pipelineObj, args) File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/ctrl\_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cmdLineFwk.py”, line 565, in makeGraph butler, collections, run = \_ButlerFactory.makeButlerAndCollections(args) File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/ctrl\_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cmdLineFwk.py”, line 352, in makeButlerAndCollections butler, inputs, self = cls.\_makeReadParts(args) File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/ctrl\_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cmdLineFwk.py”, line 288, in \_makeReadParts self = cls(butler.registry, args, writeable=False) File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/ctrl\_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cmdLineFwk.py”, line 218, in **init** self.inputs = tuple(registry.queryCollections(args.input, flattenChains=True)) if args.input else () File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/daf\_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/registries/sql.py”, line 887, in queryCollections for record in query.iter( File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/daf\_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/registry/wildcards.py”, line 673, in iter yield from self.\_search.iter( File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/daf\_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/registry/wildcards.py”, line 507, in iter manager.find(name), File “/home/valerio/lsst\_stack/conda/envs/lsst-scipipe-4.0.5/share/eups/Linux64/daf\_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/registry/collections/\_base.py”, line 448, in find raise MissingCollectionError(f"No collection with name ‘{name}’ found.") lsst.daf.butler.registry.\_exceptions.MissingCollectionError: No collection with name ‘u/valerio/single\_frame’ found. ---
**Author:** ![MelissaGraham](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/melissagraham/32/2549_2.png) [@MelissaGraham](https://www.rubin.community/u/MelissaGraham)\ **Post date:** [December 5, 2024, 6:40pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/13 "2024-12-05T18:40:31Z")
> [@valeriocarruba](#): > > We will try again on a CentOS machine, but this will take some time… Hi @valeriocarruba, the fact that this Topic never received a marked solution was brought to my attention today. It would be great to hear if the suggestions worked to resolve your issue. ---
**Author:** ![valeriocarruba](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/valeriocarruba/32/1968_2.png) [@valeriocarruba](https://www.rubin.community/u/valeriocarruba)\ **Post date:** [December 5, 2024, 9:42pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/14 "2024-12-05T21:42:05Z")
Dear @MelissaGraham, Thank you for your message. Unfortunately, after getting access to the Rubin Science Platform, I no longer worked on this. What we did at the time did not work. Does that answer the question? Best regards, Valerio Carruba ---
**Author:** ![MelissaGraham](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/melissagraham/32/2549_2.png) [@MelissaGraham](https://www.rubin.community/u/MelissaGraham)\ **Post date:** [December 5, 2024, 11:22pm UTC](https://www.rubin.community/t/problems-with-tutorial-4-of-the-lsst-science-pipelines-global-caibration/7445/15 "2024-12-05T23:22:38Z")
Yes thanks!