Migrating Anaconda/Miniconda install to a Miniforge install

MGB announced in August 2024 that MGB users could no longer use Anaconda/Miniconda without buying a license.

https://researchlist.partners.org/t/518833/17691226/70031/11/

The license explanation in point 7 of the page above explains that a license covers one designated user who can use it over their desktops, laptops and workstations.  It does not cover other users on the same licenses.  And it does not cover servers — meaning it will not cover using Anaconda on the MLSC job nodes.

These highly restrictive license terms make it so we highly recommend users stop using Anaconda/Miniconda all together and migrate to using Miniforge instead.  Or they can use other python package management methods like venv as we explain on our Martinos python page.

The above MGB page also links to instructions on using Miniforge instead, but I feel there are some points and steps missed so I have created new instructions below.

Step 1 – Get info on current Anaconda/Miniconda install

Get a list of packages and an environment export in YML format from your current Anaconda install by activating it and running

conda list > base_pkg_list.txt
conda env export --name base > base_environment.yml 

If you have multiple environments in your distribution you need to repeat this for each environment after activating them. Make sure to change the file names you write to (e.g. change base in the name above).

Also before you forget, edit all the environment.yml files you make and remove defaults and anything with the full term anaconda in its name from the list of channels and make sure conda-forge is in the list if it is not already.

Step 2 – Save old Anaconda/Miniconda configs

Move your current config files aside so you can refer to them later

mv ~/.condarc ~/SAVE.condarc
mv ~/.anaconda ~/SAVE.anaconda

The files might not exist for you so if they do not just skip it.   Also if ~/.cache/pip exists for you, I highly recommend renaming it as well but also making ~/.cache/pip a symlink to space outside of your home directory as this can get large.

Step 3 – Install Miniforge

First exit the command line shell you used in Step 1 above so the old Anconda/Miniconda paths are no longer in your PATH and other environment variables.  Make sure your login config (~/.bashrc or ~/.cshrc) are not sourcing any Anaconda/Miniconda install or putting such directories in your PATH.  Then start a new command line shell and run printenv to verify there is no vestiges of the old install .

Install Miniforge (do not do this in your home directory)

curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh

After this, you need to type yes to agree the terms and where to install miniforge. The default location is miniforge3 under your home which should NOT be where you install it. If you must install it to the exact same place your Anaconda install was, move that install aside for now. You don’t want to delete it yet.

Also remember we suggest you do NOT let the install add automatic activation of your Miniforge to your login file (e.g. .bashrc) since they causes problems with other programs like Xvnc and OpenOffice. See our main Python help article.

Step 4 – Verify proper channels are configured

After the install is done we need make sure only the proper channels are configured. Check the list created in Step 1 for the channels other than default. Also look at the ~/SAVE.condarc file if you made one in Step 2.

By default only conda-forge should show up

$ conda config --show channels
channels:
  - conda-forge

If you see defaults it needs to be removed with conda config –remove channel defaults as this is the channel that requires the license (as well as any channel with anaconda in its name)

Add any other channels you need as in this example

conda config --add channels bioconda

Look at your ~/SAVE.condarc and enviroment YAML files created above for the extra channels you used in the past.

Step 5 – make a backup of new fresh install

At this point I recommend making a archive backup of your new miniforge install so you can just extract it to start from the beginning easily without having to go through the install and long download times again.  From the directory where you installed it do

tar czf miniforge3.tar.gz miniforge3

assuming you left it the default name of miniforge3.  Then if you need to start over because of package install failures in the next step you do so with

rm -rf miniforge3
tar xzf miniforge3.tar.gz

Step 6 – reinstall missing packages to recreate previous Anaconda install

First active the new miniforge3 install in your shell environment with

eval "$(/path/to/miniforge3/bin/conda shell.bash hook)"

where you need to use YOUR path where you installed it and change bash to tcsh shell if you use tcsh instead.

OPTION 1: If you are fine with just using the latest versions of all packages,  run conda list and compare to base_pkg_list.txt file made above. Install any missing packages with conda install (or just pip if no conda package exists).

If you want specific versions of certain packages you can do that by appending “=version” to the package name. For example:

conda install tensorflow=2.2.0

If you have a lot of these, create a text file with the pkg=version one line at a time and then run

conda install --file=pkglist.txt

Repeat these steps for any sub-environments you want after creating and then activating them.

OPTION 2: If you want to try to recover the exact same python and package versions you had in your Anaconda install then we will use the YML file we exported in Step 1.  However, you need to edit this file to remove default and any other anaconda channels at the top. Then (very tediously) remove build strings from all the conda package lines. For example where you see a package line like

 - bcrypt=3.1.6=py37h7b6447c_0

you need to remove the 2nd equal sign and everything after it so the line is just

 - bcrypt=3.1.6

FYI, these build strings are unique to each channel so you cannot reinstall the same build from another channel which is why you must remove them.

Also look at the package list for any ‘anaconda’ packages like ‘anaconda-navigator’ and remove the lines entirely from the YML file.

Packages in the YML file that are using pip install instead of conda dependencies will often have to compile from scratch and this could result in differences or failures in their installation.  You can avoid most of these issues by making sure to run these commands on the exact same machine you did the Anaconda install on.  You might also have to recreate ENVIRONMENT variables that you set when installing those packages the first time.  This is where I think a lot of groups will run into problems and there is unfortunately not any great advice I can give to solve them.

If you have more than 100 or so packages in your YML this procedure will almost certainly fail.  The conda command will just run for days till it dies suddenly due to running out memory.  Note that running an hour or two could be normal and actually work.  But more than that you probably should abort.

If your install is very old, then it is likely old versions of packages you have specified in the YML file will not exist on conda-forge or be incompatible with some required packages in the base environment.  In this later case you may still be able to recreate it a sub-environment.

Also since builds on conda-forge are different, they may have different requirements. For example in an old Anaconda I had vtk=8.2.0 and libnetcdf=4.6.1 installed, but on conda-forge the vtk=8.2.0 build requires libnetcdf=4.6.2 or higher.

Ultimately if the YML procedure does not work, you may need to resort to Option 1 above and do groups of about 10 packages at a time in a pkglist.txt file using conda install –file=pkglist.txt to try to recreate your old distribution manually step by step.  And you probably will not be able to get every single package at the same version as you had previously.  Concentrate on the versions of the primary package(s) you Python scripts are built around like tensorflow or numpy.

But it is worth giving it a try. Once the YML file is edited properly try creating a test environment with it first by running

conda env create --name test_base --file=base_environment.yml

If that works and you want it to be your base environment you can try running

conda env update --file=base_environment.yml

but this could fail even though the sub-enviroment worked due to old packages being incompatible with things required in the base environment.

After you are done with the base environment, then for each sub-environment you have a *.yml file you can recreate each with

conda env create --name myenvname --file=myenv_environment.yml

Obviously change the myenvname and yml file name as needed.

Step 7 – test and remove old install

Test your new install thoroughly.  When you are confident it works as needed you can delete your old Anaconda/Miniconda installs.  I suggest archiving them using tar or zip before deleting so you have a backup.  Main thing is to prevent accidental use of the old Anaconda/Miniconda and thus violating the license terms.