Python is a pretty useful language: powerful, easy to use and pretty much everything has a python port or interface available. The main drawback is that, as an interpreted language, it is not the fastest solution. But for the kind of “tinkering” that I do, ease of use really outranks speed.
Now since I prefer to use CentOS, using python version 3 takes a little setup and every time I do this, I need to look up something I forgot, so time for a Cheat-Sheet!
1. Install python version 3
Centos 7 still does not have a package for Python version 3. So it’s EPEL to the rescue: (you need to be root for this)
yum -y install epel-release yum -y install python34
This should install python 3.4 alongside the existing version 2 of python:
python --version Python 2.7.5 python3 --version Python 3.4.5
Installing python packages goes much easier once you have pip, the “PyPA recommended tool for installing Python packages” installed. It’s quite easy to do if you have the EPEL repository enabled: (you need to be root for this)
yum -y install python-pip python-wheel
This should go quite quickly. If you don’t know python-wheel yet, its a ZIP-format archive format using the .whl extension and apparently, pip uses it.
To make sure that pip is up to date, use pip to upgrade itself: (you need to be root for this)
pip install -U pip Collecting pip Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB) 100% |....................................| 1.3MB 949kB/s Installing collected packages: pip Found existing installation: pip 8.1.2 Uninstalling pip-8.1.2: Successfully uninstalled pip-8.1.2 Successfully installed pip-9.0.1
3. Install virtualenv
When playing with python on a dev machine, you’ll need to install lots of libraries which are used only by one or two projects. If you install everything in the global system locations, that will very quickly cause bloat and conflicts. The python standard way of dealing with this is to set up a virtual environment for each specific project.
Basically, it means setting up library paths to look at non globally installed libraries and the virtualenv package makes this quite easy to do. An added bonus is that using virtualenv and pip, you are able to set up your project environment as a non-root user, which makes things easier and safer.
First though we still need to install virtualenv and this is done easily using the pip command: (you need to be root for this)
pip install virtualenv Collecting virtualenv Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB) 100% |........................................| 1.8MB 661kB/s Installing collected packages: virtualenv Successfully installed virtualenv-15.1.0
4. Create a virtual environment
Now that all the moving parts are installed, we can go ahead and create a project environment. And as mentioned before, this is the point where we drop root privileges! Everything below is done as a regular user.
virtualenv -p python3 hbaserest Running virtualenv with interpreter /usr/bin/python3 Using base prefix '/usr' New python executable in /home/jhon/hbaserest/bin/python3 Also creating executable in /home/jhon/hbaserest/bin/python Installing setuptools, pip, wheel...done.
The “-p python3” option will make virtualenv create a python version 3 environment. If you leave this option out, you’ll end up with a python version 2 env. I am going to be working on an HBase Rest api project so I named my environment “hbaserest”.
As a result of running that command, the directory structure shown to the side was created.
The top three directories simply hold a local copy of the python libraries. Of course these are not real copies but mostly symbolic links to the global python environment under /usr/lib, /usr/lib64 and /usr/include. At this point, these directories are therefore quite small:
du -hs ./lin ./lib64 ./include 0 ./include 12M ./lib 0 ./lib64
Once you start to install additional libraries for your project, those will be installed directly in these directories and at that point the sizes will increase.
Apart from those directories, there is the json file is just something pip uses for self-checking, as the name implies. The more interesting parts are in the bin directory. There you will the actual virtualenv “activate” script.
5. Using the virtual environment
Every time you want to start working in your virtual environment, you need to “activate” it, meaning you source the activate script to set the execution environment variables. It is recommended to source the script when inside the working directory.
cd hbaserest/ python --version Python 2.7.5 source ./bin/activate python --version Python 3.4.5 deactivate python --version Python 2.7.5
I threw in a couple of “python –version” commands to demonstrate that the environment actually does change. Once you source the activate script, you will also see the command prompt changes to reflect the loaded virtual environment.
Once you’re done with the environment, you can use the “deactivate” function (which was added by the activate script) to undo the virtualenv environment changes.
So now the virtualenv is set up, simply remember to run the “source ./bin/activate” script every time you start working on your project and as long as you use the pip command, everything you add to the python environment should get added to the virtual environment, nicely isolated from the rest of the system.