# How should I advise a new user to install a new stack version? **URL:** \ **Category:** Data Management\ **Tags:** stack-install\ **Created:** [August 26, 2015, 5:15pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138 "2015-08-26T17:15:03Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![RHL](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rhl/32/25_2.png) [@RHL](https://www.rubin.community/u/RHL)\ **Post date:** [August 26, 2015, 5:15pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/1 "2015-08-26T17:15:04Z")
We need to get a student at Harvard working on calibration up and running with a modern stack. It’ll need to work with obs\_decam, and I don’t think v10\_1 is new enough. There are lots and lots of tags – is there a list? What do we recommend? And are any of them available as binary installs (e.g. from conda, or using docker if necessary). Are there any that build against v10\_1 e.g. without updating anything like boost, in which case a binary base install and a source on top would be fine. ---
**Author:** ![swinbank](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/swinbank/32/24_2.png) [@swinbank](https://www.rubin.community/u/swinbank)\ **Post date:** [August 26, 2015, 6:06pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/2 "2015-08-26T18:06:06Z")
I do not know of a good general-purpose answer to this question. However, there are a few “weekly” tags of the form `w_20XX_YY` (for year XX, week YY). They’re listed (along with a lot of others) at [https://sw.lsstcorp.org/eupspkg/tags/](https://sw.lsstcorp.org/eupspkg/tags/). Building the most-recent weekly tag from source is my currently preferred method for keeping fairly well up to date, although it’s hardly ideal. An additional caveat is that `w_2015_33` doesn’t build for me (likely because of the issues with psfex which Nate has since fixed), so I’m stuck on `_30` for now. ---
**Author:** ![timj](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/timj/32/10_2.png) [@timj](https://www.rubin.community/u/timj)\ **Post date:** [August 26, 2015, 6:09pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/3 "2015-08-26T18:09:35Z")
> [@jdswinbank](#): > > An additional caveat is that w\_2015\_33 doesn’t build for me @frossie builds weeklies on demand so it should be relatively simple for her to generate one next week. 11.0 is also due of course. ---
**Author:** ![RHL](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rhl/32/25_2.png) [@RHL](https://www.rubin.community/u/RHL)\ **Post date:** [August 26, 2015, 6:18pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/4 "2015-08-26T18:18:28Z")
Can we have one every week automatically? Or do I need a cronjob to ask @frossie nicely every Sunday morning so she can submit the job during the sermon? ---
**Author:** ![timj](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/timj/32/10_2.png) [@timj](https://www.rubin.community/u/timj)\ **Post date:** [August 26, 2015, 6:23pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/5 "2015-08-26T18:23:42Z")
The plan is definitely for it to be automatic. From what I hear it’s close to being that way. ---
**Author:** ![RHL](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rhl/32/25_2.png) [@RHL](https://www.rubin.community/u/RHL)\ **Post date:** [August 26, 2015, 6:27pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/6 "2015-08-26T18:27:09Z")
Can we have a manual one every week until it’s automatic? It sounds as if that’s just a sennight or fortnight or so away. ---
**Author:** ![RHL](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rhl/32/25_2.png) [@RHL](https://www.rubin.community/u/RHL)\ **Post date:** [August 26, 2015, 8:47pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/7 "2015-08-26T20:47:14Z")
On hipchat @frossie says > I believe Sims people have been using Mario’s anaconda build for more recent binary but that’s my point. Where do I go to look for details on @mjuric’s anaconda build? ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [August 26, 2015, 9:23pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/8 "2015-08-26T21:23:50Z")
> **[mjuric/conda-lsst](https://github.com/mjuric/conda-lsst)** > > Building Conda packages for the LSST stack. Contribute to mjuric/conda-lsst development by creating an account on GitHub. Looks like the latest you get is w2015.30. ---
**Author:** ![RHL](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rhl/32/25_2.png) [@RHL](https://www.rubin.community/u/RHL)\ **Post date:** [August 26, 2015, 9:30pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/9 "2015-08-26T21:30:32Z")
Where did you find that w2015.30 (presumably the same as w\_2015\_30)? ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [August 26, 2015, 11:39pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/10 "2015-08-26T23:39:43Z")
Started with “The LSST stack builds are kept on [http://eupsforge.net/conda/dev](http://eupsforge.net/conda/dev) channel”, followed that and eventually go to [https://lsst-web.ncsa.illinois.edu/~mjuric/conda/dev/linux-64/](https://lsst-web.ncsa.illinois.edu/~mjuric/conda/dev/linux-64/) ---
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [August 27, 2015, 5:30am UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/11 "2015-08-27T05:30:44Z")
@RHL, @ktl – @ljones (or I) occasionally build and push out a new version of conda-wrapped binary distrib, depending on the needs of the sims team. It was used during the cadence workshop as an install option (the other one was a docker container that AJC prepared). I didn’t want to do (or advertise) one for DM so I don’t cause confusion about what’s officially supported and what isn’t (EUPS is the only supported mechanism right now, I think). It shouldn’t be hard, though – both the code and the instructions are on [https://github.com/mjuric/conda-lsst](https://github.com/mjuric/conda-lsst). I could do a one-off manual build of `lsst_distrib`. For anything more than that (i.e., unattended automated builds), the code would probably need a few more days of cleanups and testing. ---
**Author:** ![RHL](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rhl/32/25_2.png) [@RHL](https://www.rubin.community/u/RHL)\ **Post date:** [August 28, 2015, 5:03am UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/12 "2015-08-28T05:03:34Z")
If you don’t want to do it, maybe square could? With every weekly tag, maybe? ---
**Author:** ![frossie](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/frossie/32/9_2.png) [@frossie](https://www.rubin.community/u/frossie)\ **Post date:** [August 28, 2015, 5:20am UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/13 "2015-08-28T05:20:00Z")
Yes the plan is to indeed do weekly binaries, and once we can drive those from Jenkins we are happy to produce them for people who have a robust recipe that can be automated (eg. I talked to Fabio about building his CernVMFS images last week). ---
**Author:** ![FabioHernandez](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/fabiohernandez/32/82_2.png) [@FabioHernandez](https://www.rubin.community/u/FabioHernandez)\ **Post date:** [September 3, 2015, 1:30pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/14 "2015-09-03T13:30:19Z")
I think that the binary distribution available via CernVM FS should work for this use case. Currently you will find there the latest official stable version, i.e. v10.1, for both Linux and MacOS X. You can use the stack as is or build your own packages on top of this distribution (using EUPS). The relevant details are here: > **[airnandez/lsst-cvmfs](https://github.com/airnandez/lsst-cvmfs)** > > How to configure CernVM FS to use LSST binary distribution \[won't be updated - please see https://sw.lsst.eu\] - airnandez/lsst-cvmfs Look at the [Advanced Usage](https://github.com/airnandez/lsst-cvmfs/blob/master/AdvancedUsage.md) section for details on how to develop on top of a version of the binary stack. This distribution is also available to use via a Docker container. Details: [https://github.com/hepsw/docks/tree/master/cvmfs-lsst](https://github.com/hepsw/docks/tree/master/cvmfs-lsst) ---
**Author:** ![RHL](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/rhl/32/25_2.png) [@RHL](https://www.rubin.community/u/RHL)\ **Post date:** [September 3, 2015, 1:33pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/15 "2015-09-03T13:33:14Z")
The problem is that 10.1 is too old. 11.1 will be out soon, but it’ll soon be superseded in turn; we need a way to get binary distros of at least the weeklies that Frossie produces. ---
**Author:** ![jbosch](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jbosch/32/12_2.png) [@jbosch](https://www.rubin.community/u/jbosch)\ **Post date:** [September 3, 2015, 1:57pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/16 "2015-09-03T13:57:18Z")
I think frequent releases and frequent binaries are important for some kinds of users, but I’ve become convinced that at least right now most core developers of the science pipelines stack are better off just building everything from source and updating that stack in-place frequently, and what we should be providing are better tools to do that. Everyone I know who is not building from source (and instead tries to mix-and-match master packages with released versions) seems to encounter a nonexistent bug or segfault (actually just a version incompatibility) at least once a month. Those don’t just slow down the developer who ran into the problem; they distract everyone else who helps try to reproduce and debug the problem. And while frequent releases will make that less of a problem, I don’t think it will go away anytime soon; there’s simply too much churn. Moreover, installing new releases frequently will either force users to delete old releases frequently or deal with the EUPS sluggishness that comes from having too many versions and tags. And on the other hand, I think the idea that developers need to have a lot of tagged versions on hand to reproduce bugs reported specifically in those versions simply hasn’t materialized (at least not yet). Finally, it makes developers self-sufficient: if they can build everything themselves just as easily as they can ask someone in e.g. SQuaRE to do it, they can’t have their work blocked by SQuaRE availability or CI downtime. A lot of those problems with building against releases are problems that will have to get fixed anyway: lower-level packages will stabilize, EUPS performance will be improved, improved CI automation will reduce human-availability blockage. But we’ve been struggling to solve this problem by having more frequent releases for years now, and we’ve put very little effort into build-the-stack-from-source tools for developers, and essentially abandoned the effort we have done (lsst-build and lsstsw) to “use at your own risk; this was intended for CI”. There might be a faster solution to the new developer problem there. ---
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [September 3, 2015, 10:22pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/17 "2015-09-03T22:22:57Z")
@jbosch I’m not sure the two are exclusive – wouldn’t you still benefit from a tool that shortcircuits any build for which a binary distribution exists? E.g., `lsstsw` could be made smart enough to check if (say) `afw-10.1-38-gf63c96a` has already been built by Jenkins, and fall back to building from source otherwise. Since versions uniquely encode all dependencies, this should be relatively safe to do (I think…). ---
**Author:** ![jbosch](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jbosch/32/12_2.png) [@jbosch](https://www.rubin.community/u/jbosch)\ **Post date:** [September 3, 2015, 11:32pm UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/18 "2015-09-03T23:32:51Z")
I don’t think they’re mutually exclusive at all, and that sounds useful, but it sounds a lot harder to write than a much simpler tool that just built from source. Heck, maybe lsstsw is already there; I’m happy enough using my own tool for this purpose anyway. I’m just not comfortable suggesting either to newcomers, because I know they’re not really being supported or documented for that role. ---
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [September 4, 2015, 12:26am UTC](https://www.rubin.community/t/how-should-i-advise-a-new-user-to-install-a-new-stack-version/138/19 "2015-09-04T00:26:55Z")
Yup, I agree – we officially don’t want to recommend anything unofficial 😄 . I’d find an official build tool helpful as well. There is a proposal [for what lsst-build could evolve into](https://confluence.lsstcorp.org/display/~mjuric/bt%3A+A+build+tool+for+LSST+code). Maybe we could find some resources to move on that now, if there’s interest. Android’s [repo tool](https://source.android.com/source/using-repo.html) may be something to look at as well. I haven’t looked at it much, but it [seems designed to solve our problem](http://www.instructables.com/id/Using-Googles-repo-command-in-your-own-projects/).