# Using Python's venv with newinstall's Anaconda

**URL:** <https://www.rubin.community/t/using-pythons-venv-with-newinstalls-anaconda/2933>\
**Category:** Data Management\
**Created:** [May 25, 2018, 10:02pm UTC](https://www.rubin.community/t/using-pythons-venv-with-newinstalls-anaconda/2933 "2018-05-25T22:02:36Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jsick](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jsick/32/672_2.png) [@jsick](https://www.rubin.community/u/jsick)\
**Post date:** [May 25, 2018, 10:02pm UTC](https://www.rubin.community/t/using-pythons-venv-with-newinstalls-anaconda/2933/1 "2018-05-25T22:02:36Z")

</div>

This post describes how to layer a Python virtual environment on top of the Anaconda Python that comes with standard LSST Science Pipelines installations like [newinstall.sh](https://pipelines.lsst.io/install/newinstall.html) and [lsstsw](https://pipelines.lsst.io/install/lsstsw.html).

## Background

The motivation for this is that you might want to install or develop a regular (as in, PyPI) Python package to use with the LSST Science Pipelines, but you don’t want to install that package and its dependencies in the miniconda site-packages directory. (If you do `pip` install a Python package into the LSST Science Pipelines’ miniconda site-packages, it can be hard to completely uninstall it and its dependencies later).

Documentation engineering is a prime use-case for this technique. Tooling for the LSST Science Pipelines documentation is packaged in [Documenteer](https://github.com/lsst-sqre/documenteer). We can’t install Documenteer with the Pipelines’s EUPS stack, so instead [we install Documenteer](http://www.rubin.community/t/pipelines-package-docs-hack-day-instructions/2742?u=jsick) on top of the Pipelines’s EUPS stack with `pip`.

## The method

I’m going to show you how to use [venv](https://docs.python.org/3.4/library/venv.html) to create the virtual environment because it’s included in the Python 3 standard library.

Before starting, you need to be working in a shell with the LSST environment loaded (`loadLSST.bash`) and set up (`setup lsst_distrib`).

Then create the virtual environment and `pip`-install packages:

```bash
python -m venv --system-site-packages --without-pip .pyvenv
source .pyvenv/bin/activate
curl https://bootstrap.pypa.io/get-pip.py | python
.pyvenv/bin/pip install -r requirements.txt

```

If you look inside the virtual environment, you should see your packages installed there:

```
ls .pyvenv/lib/python3.6/site-packages

```

At the same time, you should be able to import `lsst` from the parent `miniconda` Python:

```
python
>>> import lsst.afw
>>> lsst.afw. __version__ 

```

## Notes

- The `--system-site-packages` argument allows this virtual environment (`.pyvenv`) to see packages installed in the miniconda site-packages. For LSST Science Pipelines, miniconda _is_ the “site.”
- The `--without-pip` is needed because `venv` can’t bootstrap its own `pip`. This is because `ensurepip` is not available in miniconda. The work-around is to create the environment, then add pip later (tip via [https://github.com/ContinuumIO/anaconda-issues/issues/6917#issuecomment-340014721](https://github.com/ContinuumIO/anaconda-issues/issues/6917#issuecomment-340014721)).
- `.pyvenv/bin/pip install -r requirements.txt` can be any `pip` installation command you require. Note that I did find it necessary to directly specify `.pyvenv`'s `pip` because it wasn’t already at the top of the path, despite the environment being activated. If you have a fix for this, let me now.
- `.pyvenv` is the name of the virtual environment, and also the directory containing the virtual environment. You can name it whatever you like. Let me know if there’s a better canonical name.