TCM
UoC crest

Julia in TCM

Julia is a python-like language of the future, but has issues with the present.

It is designed for a future in which per-user installations of Julia and all of the desired packages are sensible (an installation of Julia and the ubiquitous Plots.jl package is well over 1GB and almost 30,000 files). And one in which collaboration is rarely tried, as the chances of two separate installations having the same versions of any given package are slim.

Whereas python works poorly with centralised repositories of packages, Julia is worse, and tends to get very upset if it cannot update a "depot" which contains out-of-date packages. The "solution" of using cron to keep a central repository up-to-date would mean that users suffer unexpected updates which may cause their code to break.

Julia also interacts badly with most package managers. Its website contains the statement "please do not use the version of 'Julia' shipped by Linux or BSD package managers. Many Linux/BSD/Unix package managers ship broken and/or significantly out of date versions of Julia." Ubuntu did package Julia for one release (20.04), found it gave wrong answers sometimes, and dropped it. Julia seems remarkably sensitive to the details of its installation. (The issues with Ubuntu were blamed on interactions with the default Ubuntu LLVM package.)

So in TCM we provide the following Julia versions with no extra packages:

There is also a version of julia-1.10 with Plots.jl. This probably no longer works at all well.

Julia will attempt to place packages you install in ~/.julia in your home directory. This will not work well, being small, slow and quotaed (currently).

The current sensible options are to redirect this to either a local /scratch disk, which will restrict use to a single machine, or to /rscratch. One can achieve this either with symlinks, or by setting the environment variable JULIA_DEPOT_PATH, e.g. spqr1 might set

export JULIA_DEPOT_PATH="/rscratch/spqr1/.julia:"

and might include this line in ~/.bashrc so that it is set on every login. The trailing colon should cause Julia to append its version-specific default path, but package add operations will use the first entry. Maybe spqr1 does not like the confusion of hidden directories (those whose names start with a dot), so perhaps /rscratch/spqr1/julia would be preferable, but more vulnerable to rm -r *.

For reasonable performance from modules one downloads and compiles, it is necessary to add

export JULIA_CPU_TARGET='generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)'

which is how the official Julia distribution is built, and includes generic, Sandy Bridge (AVX), and Haswell (AVX2) optimisations as potentially three separate images, or perhaps just

export JULIA_CPU_TARGET='generic;haswell,-rdrnd,clone_all'

on the basis that no-one still using a pre-Haswell CPU would care about speed.

In TCM, julia-1.7 and julia-1.11 will set the above two environment variables if they are not already set. Currently julia on its own does not, for compatibility with TCM's previous position.