MINIMALLY PAINFUL PYTHON DEVELOPMENT
====================================

[Stef Countryman -- 2019/07/15] [home] [blogs]

This post is meant to help you quickly get a nice, maintainable python environment up and running. You can ignore the explanations and just follow the instructions at the end of each subsection (under TL;DR) if you just want to get going quickly.

MOTIVATION
----------

Developing multiple python projects with arcane dependencies using the naive approach (install one copy of python, pip --install a bunch of packages by hand, lose track of which projects need which packages, face a dependency conflict, suffer, despair) will ruin your computer and your life. When it comes to dependency management, packaging, writing extensions, and even choosing an interpreter, a web search will show dozens of ways to do the same thing, most of them obsolete, overwrought, unmaintainable, slow, bug-ridden, inextensible, or otherwise terrible.

This presents a problem to the novice programmer, who has heard that python is easy to learn and has many useful libraries that you can freely use, and therefore starts a project without setting up a decent dev environment, not realizing what horrors await them. Fortunately, practices are improving somewhat, and with a bit of guidance, managing python projects can be as forgiving to novice programmers as the syntax of the code itself.

The aim of this post is to provide such a guide to my friends and colleagues in a central place.

PICK AN INTERPRETER
-------------------

The first thing to do is to install an actual python interpreter. Python is a scripting language, meaning that the programs are just text files which are executed by a program called an interpreter. The language itself changes roughly every year, so you will probably want an interpreter that runs a recent version of the language, as well as the ability to manage multiple versions (in case you find that certain of your programs require older/newer versions than what you typically run).

MacOS computers come with a very old version of python pre-installed. Linux machines sometimes come with a newer version pre-installed, though very minimal distributions will come without a python interpreter altogether. MacOS, most Linux distributions, and Windows all have system package managers (either official, like apt on Debian Linux, or third party, like MacPorts or Homebrew on MacOS/Chocolatey on Windows) that let you quickly install a version of python; don't use them. They are tied in to various tools throughout your system, so that breaking them might break system-wide tools you depend on, and they tend to be incomplete and out of date. Installing python and libraries [1] using the system package manager means possibly having to remove and reinstall everything in your fragile dev environment at some point in the future, which can easily take over a day for complex environments, assuming it's possible at all. This violates what is, in my opinion, the most important rule of a development environment:

It should be easy to destroy and rebuild your dev environment.

Having this property means you will never live in fear of the day when you inevitably mess up your configuration and need to delete it all and start over. You will not fear adding new team members or switching development computers, and chances are your deployments will become much easier.

The solution is to use Anaconda's python distribution, which includes a nice, fast, cross-platform python package manager (conda) and a nice development environment management system, which I'll discuss in the next section.

TL;DR
~~~~~

  1. Install Anaconda [MacOS] [Win] [Linux]. If it asks you whether you'd like to install PyCharm as well, say yes if you don't have it installed already (in which case you can skip step 2).

PICK AN IDE/EDITOR
------------------

You don't really need an integrated development environment (IDE, i.e. a program for editing and running your code) for python dev, since ipython, Jupyter, ripgrep, and your favorite text editor can handle 98% of what you'd need an IDE for. If you're command-line savvy and have a favorite text editor, you can keep using your favorite tools quite easily.

If you're not yet a command-line guru, however, you should try a nice IDE like PyCharm. It has great code exploration, debugging, and running/testing tools built in, with full support for various arcane interpreter setups (in case you need to deviate in the future from the set up I suggest in this post), and is cross-platform. The community version is free and adequate for most purposes.

TL;DR
~~~~~

  1. Install PyCharm (if you didn't already do it while installing Anaconda).

------------------------------------------------------------

[1] Seasoned python devs might correctly observe that you could simply use a virtualenv. This is a valid technique, but it does not solve the related problem of picking and installing an interpreter (let alone managing multiple versions), nor of the limitations of pip or the inconvenience of having virtual environments scattered all over your dev folders. pipenv uses virtualenv and tracks your dependencies automatically, but anything that pipenv is capable of automatically tracking is going to be very easy to track or reconstruct manually, reducing it's utility (not to mention it was slow and buggy last I used it).