# Trouble with daf\_persistence 12.1 build at NERSC **URL:** \ **Category:** Support\ **Created:** [November 1, 2016, 9:16pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367 "2016-11-01T21:16:50Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![heather999](https://avatars.discourse-cdn.com/v4/letter/h/f4b2a3/32.png) [@heather999](https://www.rubin.community/u/heather999)\ **Post date:** [November 1, 2016, 9:16pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367/1 "2016-11-01T21:16:50Z")
I was trying to build the sims\_2\_3\_1 tag of lsst\_apps (created by @danielsf for the Twinkles work) at NERSC on Edison, we did this successfully at SLAC last month, but I ran into an error at NERSC: ``` [46/70] daf_persistence 12.1 ... ***** error: from /global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDir/Linux64/daf_persistence-12.1/build.log: File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyyaml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 599, in construct_python_object_apply instance = self.make_python_instance(suffix, node, args, kwds, newobj) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyyaml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 540, in make_python_instance cls = self.find_python_name(suffix, node.start_mark) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyyaml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 513, in find_python_name module. __name__ ), mark) ConstructorError: while constructing a Python object cannot find 'ImgMapper' in the module ' __main__' in "./repositoryCfg.yaml", line 3, column 10 ---------------------------------------------------------------------- Ran 4 tests in 0.151s FAILED (errors=1) The following tests failed: /global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDir/Linux64/daf_persistence-12.1/daf_persistence-12.1/tests/.tests/butlerPickle.py.failed 1 tests failed scons: *** [checkTestStatus] Error 1 scons: building terminated because of errors. + exit -4 eups distrib: Failed to build daf_persistence-12.1.eupspkg: Command: source /global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/eups/bin/setups.sh; export EUPS_PATH=/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1; (/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDir/Linux64/daf_persistence-12.1/build.sh) >> /global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDir/Linux64/daf_persistence-12.1/build.log 2>&1 4>/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDir/Linux64/daf_persistence-12.1/build.msg exited with code 252 ``` Poking into the build.log I see ``` Failed test output: tests/butlerPickle.py /global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/miniconda2/ 3.19.0.lsst4/lib/python2.7/site-packages/astropy/config/configuration.py:687: Conf igurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None warn(ConfigurationMissingWarning(msg)) E... ====================================================================== ERROR: testIO ( __main__.ButlerPickleTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/butlerPickle.py", line 50, in setUp self.butler = dafPersist.Butler(root='.', mapper=MinMapper()) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/bu tler.py", line 285, in __init__ self._addRepo(args, inout='out', defaultMapper=defaultMapper, butlerIOParents= butlerIOParents) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/bu tler.py", line 345, in _addRepo cfg = Storage.getRepositoryCfg(args.cfgRoot) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/st orage.py", line 75, in getRepositoryCfg ret = Storage.storages[parseRes.scheme].getRepositoryCfg(uri) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/po sixStorage.py", line 78, in getRepositoryCfg repositoryCfg = PosixStorage._getRepositoryCfg(uri) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/po sixStorage.py", line 71, in _getRepositoryCfg repositoryCfg = yaml.load(f) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/ __init__.py", line 71, in load return loader.get_single_data() File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 39, in get_single_data return self.construct_document(node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 43, in construct_document data = self.construct_object(node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 88, in construct_object data = constructor(self, node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/re positoryCfg.py", line 61, in v1Constructor d = loader.construct_mapping(node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 610, in construct_python_ob ject_new return self.construct_python_object_apply(suffix, node, newobj=True) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 599, in construct_python_ob ject_apply instance = self.make_python_instance(suffix, node, args, kwds, newobj) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 540, in make_python_instanc e cls = self.find_python_name(suffix, node.start_mark) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 513, in find_python_name module. __name__ ), mark) ConstructorError: while constructing a Python object cannot find 'ImgMapper' in the module ' __main__' in "./repositoryCfg.yaml", line 3, column 10 ---------------------------------------------------------------------- Ran 4 tests in 0.151s FAILED (errors=1) The following tests failed: /global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDir/Linux6 4/daf_persistence-12.1/daf_persistence-12.1/tests/.tests/butlerPickle.py.failed 1 tests failed scons: *** [checkTestStatus] Error 1 scons: building terminated because of errors. + exit -4 ``` I’m trying again with v12\_1 for fun - but I have no reason to believe the result will change. I see something similar about `Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None` here > but I don’t know how that was resolved. Can someone offer any suggestions? Thanks, Heather ---
**Author:** ![danielsf](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/danielsf/32/440_2.png) [@danielsf](https://www.rubin.community/u/danielsf)\ **Post date:** [November 1, 2016, 9:30pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367/2 "2016-11-01T21:30:40Z")
Hi Heather, What does the test fail log say? You should find it in `$DAF_PERSISTENCE_DIR/tests/.tests/butlerPickle.py.failed` ---
**Author:** ![heather999](https://avatars.discourse-cdn.com/v4/letter/h/f4b2a3/32.png) [@heather999](https://www.rubin.community/u/heather999)\ **Post date:** [November 1, 2016, 9:51pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367/3 "2016-11-01T21:51:17Z")
Hi Scott, Does this help? ``` more EupsBuildDir/Linux64/daf_persistence-12.1/daf_persistence-12.1/tests/.tests/butlerPickle.py.failed tests/butlerPickle.py /global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/miniconda2/ 3.19.0.lsst4/lib/python2.7/site-packages/astropy/config/configuration.py:687: Conf igurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None warn(ConfigurationMissingWarning(msg)) E... ====================================================================== ERROR: testIO ( __main__.ButlerPickleTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/butlerPickle.py", line 50, in setUp self.butler = dafPersist.Butler(root='.', mapper=MinMapper()) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/bu tler.py", line 285, in __init__ self._addRepo(args, inout='out', defaultMapper=defaultMapper, butlerIOParents= butlerIOParents) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/bu tler.py", line 345, in _addRepo cfg = Storage.getRepositoryCfg(args.cfgRoot) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/st orage.py", line 75, in getRepositoryCfg ret = Storage.storages[parseRes.scheme].getRepositoryCfg(uri) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/po sixStorage.py", line 78, in getRepositoryCfg repositoryCfg = PosixStorage._getRepositoryCfg(uri) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/po sixStorage.py", line 71, in _getRepositoryCfg repositoryCfg = yaml.load(f) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/ __init__.py", line 71, in load return loader.get_single_data() File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 39, in get_single_data return self.construct_document(node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 43, in construct_document data = self.construct_object(node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 88, in construct_object data = constructor(self, node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/EupsBuildDi r/Linux64/daf_persistence-12.1/daf_persistence-12.1/python/lsst/daf/persistence/re positoryCfg.py", line 61, in v1Constructor d = loader.construct_mapping(node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 610, in construct_python_ob ject_new return self.construct_python_object_apply(suffix, node, newobj=True) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 599, in construct_python_ob ject_apply instance = self.make_python_instance(suffix, node, args, kwds, newobj) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 540, in make_python_instanc e cls = self.find_python_name(suffix, node.start_mark) File "/global/common/edison/contrib/lsst/lsstDM/w.2016.40-sims_2.3.1/Linux64/pyy aml/3.11.lsst1+2/lib/python/yaml/constructor.py", line 513, in find_python_name module. __name__ ), mark) ConstructorError: while constructing a Python object cannot find 'ImgMapper' in the module ' __main__' in "./repositoryCfg.yaml", line 3, column 10 ---------------------------------------------------------------------- Ran 4 tests in 0.151s FAILED (errors=1) ``` ---
**Author:** ![heather999](https://avatars.discourse-cdn.com/v4/letter/h/f4b2a3/32.png) [@heather999](https://www.rubin.community/u/heather999)\ **Post date:** [November 1, 2016, 10:31pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367/4 "2016-11-01T22:31:15Z")
and interestingly, I tried a fresh build using v12\_1 and it got beyond the daf\_persistence 12.1 build… Which is great - but I’m wondering if that sims\_2\_3\_1 tag is either not reproducible or is it just not working in the NERSC case, while it did at SLAC? ---
**Author:** ![danielsf](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/danielsf/32/440_2.png) [@danielsf](https://www.rubin.community/u/danielsf)\ **Post date:** [November 2, 2016, 6:32pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367/5 "2016-11-02T18:32:56Z")
(Sorry; I didn’t see your responses yesterday) Did you try re-starting the sims\_2\_3\_1 build after the 12\_1 build completed? If daf\_persistence is there because of 12\_1, sims\_2\_3\_1 might not need to rebuild it. I know that’s not an answer, but I’m curious to see what happens. ---
**Author:** ![heather999](https://avatars.discourse-cdn.com/v4/letter/h/f4b2a3/32.png) [@heather999](https://www.rubin.community/u/heather999)\ **Post date:** [November 4, 2016, 2:18pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367/6 "2016-11-04T14:18:11Z")
I actually tend to create completely separate installations for each DMstack release or build in a new directory (but in the same directory area) - perhaps that’s archaic! 🙂 I could try the test you suggest though and see what happens, I still have the sims\_2\_3\_1 install set up, it’s just not complete. I’ll see if I can do that and report back. ---
**Author:** ![daues](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/daues/32/762_2.png) [@daues](https://www.rubin.community/u/daues)\ **Post date:** [March 21, 2017, 7:14pm UTC](https://www.rubin.community/t/trouble-with-daf-persistence-12-1-build-at-nersc/1367/7 "2017-03-21T19:14:42Z")
I wanted to mention that I just reproduced this exact same error with daf\_persistence 13.0 on a CentOS 7.3.1611 system. The error occurred when the permissions on my home directory were 700. I changed the permissions to 755 since there was the complaint OSError:Could not find unix home directory The build subsequently succeeded after that change.