MINIMALLY PAINFUL PYTHON DEVELOPMENT
====================================
[Stef Countryman -- 2019/07/15] [home] [blogs]
This post is meant to help you quickly get a nice, maintainable python environment up and running. You can ignore the explanations and just follow the instructions at the end of each subsection (under TL;DR) if you just want to get going quickly.
MOTIVATION
----------
Developing multiple python projects with arcane dependencies using the
naive approach (install one copy of python, pip --install a
bunch of packages by hand, lose track of which projects need which
packages, face a dependency conflict, suffer, despair) will ruin your
computer and your life.
When it comes to dependency management, packaging, writing extensions, and
even choosing an interpreter, a web search will show dozens of ways to do
the same thing, most of them obsolete, overwrought, unmaintainable, slow,
bug-ridden, inextensible, or otherwise terrible.
This presents a problem to the novice programmer, who has heard that python is easy to learn and has many useful libraries that you can freely use, and therefore starts a project without setting up a decent dev environment, not realizing what horrors await them. Fortunately, practices are improving somewhat, and with a bit of guidance, managing python projects can be as forgiving to novice programmers as the syntax of the code itself.
The aim of this post is to provide such a guide to my friends and colleagues in a central place.
PICK AN INTERPRETER
-------------------
The first thing to do is to install an actual python interpreter. Python is a scripting language, meaning that the programs are just text files which are executed by a program called an interpreter. The language itself changes roughly every year, so you will probably want an interpreter that runs a recent version of the language, as well as the ability to manage multiple versions (in case you find that certain of your programs require older/newer versions than what you typically run).
MacOS computers come with a very old version of python pre-installed.
Linux machines sometimes come with a newer version pre-installed, though
very minimal distributions will come without a python interpreter
altogether.
MacOS, most Linux distributions, and Windows all have system
package managers (either official, like apt on Debian Linux,
or third party, like MacPorts or Homebrew on MacOS/Chocolatey on Windows)
that let you quickly install a version of python;
don't use them.
They are tied in to various tools throughout your system, so that breaking
them might break system-wide tools you depend on, and they tend to be
incomplete and out of date.
Installing python and libraries
[1]
using the system package manager means
possibly having to remove and reinstall everything in your fragile dev
environment at some point in the future, which can easily take over a day
for complex environments, assuming it's possible at all.
This violates what is, in my opinion, the most important rule of a
development environment:
It should be easy to destroy and rebuild your dev environment.
Having this property means you will never live in fear of the day when you inevitably mess up your configuration and need to delete it all and start over. You will not fear adding new team members or switching development computers, and chances are your deployments will become much easier.
The solution is to use Anaconda's python distribution, which includes a
nice, fast, cross-platform python package manager (conda) and
a nice development environment management system, which I'll discuss in the
next section.
TL;DR
~~~~~
PICK AN IDE/EDITOR
------------------
You don't really need an
integrated development environment
(IDE, i.e. a program for editing and running your code)
for python dev, since ipython, Jupyter, ripgrep,
and your favorite text editor can handle 98% of what you'd need an IDE for.
If you're command-line savvy and have a favorite text editor, you can keep
using your favorite tools quite easily.
If you're not yet a command-line guru, however, you should try a nice IDE like PyCharm. It has great code exploration, debugging, and running/testing tools built in, with full support for various arcane interpreter setups (in case you need to deviate in the future from the set up I suggest in this post), and is cross-platform. The community version is free and adequate for most purposes.
TL;DR
~~~~~
------------------------------------------------------------
[1]
Seasoned python devs might correctly observe that you could simply use a
virtualenv.
This is a valid technique, but it does not solve the related problem of
picking and installing an interpreter (let alone managing multiple
versions), nor of the limitations of
pip or the inconvenience of having virtual environments
scattered all over your dev folders.
pipenv uses virtualenv and tracks your
dependencies automatically, but anything that pipenv is
capable of automatically tracking is going to be very easy to track or
reconstruct manually, reducing it's utility (not to mention it was slow and
buggy last I used it).