For those of you using machine learning, one of the critical applications you need to learn to use correctly is Anaconda. This will allow you to run a custom python installation within your user environment which is separate from the system installed python and often newer -- which you’ll find is important for the numerous Machine Learning (ML) and Deep Learning (DL) applications that you want to run (like TensorFlow, for example). Sometimes you just need something newer than the default python version that is installed with the system; other times you’ll need a very specific version for what you’re running.
Plus anaconda includes the conda package management utility which will allow you to easily install many of the ML applications that you want and will automatically install all of their dependencies at the same time (within anaconda) so what would have been 3hrs of installing a dozen different pre-reqs becomes just one line of code.
(sidenote: avoid pip whenever possible -- you will find a ton of online ML documentation telling you how to install their software using the pip system… try using anaconda before you do)
Installation
You can download Anaconda here: https://www.anaconda.com/distribution/#download-section
Select your OS, then if you want the Python3 or Python2.7 version. Download to your home directory and install it there.
./Anaconda3-2019.03-Linux-x86_64.sh
If you cannot execute it, first run:
chmod +x Anaconda3-2019.03-Linux-x86_64.sh
(update with the correct name of the file)
Please note that a> you can install anaconda within your user home directory (which is what we want) and b> you don’t need administrative privileges to install it.
Your only constraint might be space -- so machines where your home directory is your AFS space (which NCSU only provides 10GB of) might have some issues with this as anaconda will want to use all of that 10GB of space. But outside of teaching lab systems or remote access servers, you should find that your research workstations all use local storage now for your home directories.
When installing, go ahead and have anaconda setup its initialization scripts (but see below).
Don’t Start in Anaconda by Default
Last login: Mon Jul 22 18:40:18 2019 from rampart.ece.ncsu.edu
(base) [djgreen@timeflip ~]$
The first thing you’ll notice after you install anaconda (and either logged out and back in OR ran “source ./.bashrc”) is that your prompt changes to the one above.
This indicates that you’re within the anaconda environment… but you don’t want this to be your default. This can break things -- access to your Gnome environment is a common one.
Run the following command:
conda config --set auto_activate_base false
Log back out / in and you’ll find your prompt back to normal. From now on, conda is in your path, but its not running by default.
To enter the conda env, run conda activate and to exit, run conda deactivate
User, not System Packages
[need docs]
Create Conda Environments
Need to install tensorflow?
conda create --name tf_gpu tensorflow-gpu
This is the same as running:
conda create --name tf_gpu
conda activate tf_gpu
conda install tensorflow-gpu
Updating Conda
In most cases what you want to do when you say that you want to update Anaconda is to execute the command:
conda update --all
If you are only interested in updating an individual package then from the command line:
conda update astroid astropy
However, Your root environment is probably not a good place to try and manage an exact set of packages—it is going to be a dynamic working space with new packages installed and packages randomly updated. If you need an exact set of packages, create a conda environment to hold them. Thanks to the conda package cache and the way file linking is used, doing this is typically fast and consumes very little additional disk space. For example:
conda create -n myspecialenv -c bioconda -c conda-forge python=3.5 pandas beautifulsoup seaborn nltk
Recommended Reading
- https://towardsdatascience.com/a-guide-to-conda-environments-bc6180fc533
- https://conda.io/projects/conda/en/latest/_downloads/1f5ecf5a87b1c1a8aaf5a7ab8a7a0ff7/conda-cheatsheet.pdf
- https://towardsdatascience.com/tensorflow-gpu-installation-made-easy-use-conda-instead-of-pip-52e5249374bc
Educational Videos
Uninstall
One of the nice things about installing anaconda in your home directory is that it's incredibly easy to uninstall it… just delete the directory. Open a terminal window, and then remove your entire Anaconda directory, which has a name such as anaconda2 or anaconda3, by entering:
rm -rf ~/anaconda3
Also, remove the conda initialize code from your .bashrc
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/djgreen/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/djgreen/anaconda3/etc/profile.d/conda.sh" ]; then
. "/home/djgreen/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/home/djgreen/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
add anaconda3
There is an installation of anaconda in AFS which you can add to your environment as an alternative to local installation -- “add anaconda3”
The advantage of this installation is that you can minimize the space needed -- for example, you could use this on our HYDRA or GRENDEL remote access systems where your AFS quota is quite small.
Courses are likely to be the largest user of this option.
The following steps, for example, would allow you to call the application into your environment, create a conda environment, activate it, and then add additional anaconda packages (qiskit) to your local storage that’s not provided through the “add” install:
add anaconda3
conda create -n ece792-046
(creates new env in ~/.conda/envs/ece792-046)
source activate ece792-046
#now install what you need
pip install qiskit --user
miniconda
We’ve found that anaconda versions can greatly impact your install, so sometimes the version in AFS isn’t new enough… this is where miniconda might be helpful. It’s the tiniest version of Anaconda and you can install it in your AFS home space.
First you need to download the latest version from https://docs.conda.io/en/latest/miniconda.html#linux-installers
I recommend heading there, copying the link, then using wget to grab it…
wget https://repo.anaconda.com/miniconda/Miniconda3-py39_4.10.3-Linux-x86_64.sh
bash Miniconda3-py39_4.10.3-Linux-x86_64.sh
Follow their instructions… go with the defaults. It’ll want to install to /afs/unity.ncsu.edu/users/d/djgreen/miniconda3 (replace d/djgreen w/ your path), which is fine.
I would recommend that unless you’ll be using it alot, to not add it to your environment by default. In which case, you can activate it with the following command:
eval "$(/afs/unity.ncsu.edu/users/d/djgreen/miniconda3/bin/conda shell.bash hook)"
(yes, its a mouthful and be sure to fix that path for yours)
Alternatively, have it add it, but then immediately run the following command to have it present but disabled by default
conda config --set auto_activate_base false
(ok, the alternative option is probably going to be saner in the long run)