Python is a pretty useful language: powerful, easy to use and pretty much everything has a python port or interface available. The main drawback is that, as an interpreted language, it is not the fastest solution. But for the kind of “tinkering” that I do, ease of use really outranks speed.
Now since I prefer to use CentOS, using python version 3 takes a little setup and every time I do this, I need to look up something I forgot, so time for a Cheat-Sheet!
1. Install python version 3
Centos 7 still does not have a package for Python version 3. So it’s EPEL to the rescue: (you need to be root for this)
yum -y install epel-release yum -y install python34
This should install python 3.4 alongside the existing version 2 of python:
python --version Python 2.7.5 python3 --version Python 3.4.5
2.Install Pip
Installing python packages goes much easier once you have pip, the “PyPA recommended tool for installing Python packages” installed. It’s quite easy to do if you have the EPEL repository enabled: (you need to be root for this)
yum -y install python-pip python-wheel
This should go quite quickly. If you don’t know python-wheel yet, its a ZIP-format archive format using the .whl extension and apparently, pip uses it.
To make sure that pip is up to date, use pip to upgrade itself: (you need to be root for this)
pip install -U pip Collecting pip Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB) 100% |....................................| 1.3MB 949kB/s Installing collected packages: pip Found existing installation: pip 8.1.2 Uninstalling pip-8.1.2: Successfully uninstalled pip-8.1.2 Successfully installed pip-9.0.1
3. Install virtualenv
When playing with python on a dev machine, you’ll need to install lots of libraries which are used only by one or two projects. If you install everything in the global system locations, that will very quickly cause bloat and conflicts. The python standard way of dealing with this is to set up a virtual environment for each specific project.
Basically, it means setting up library paths to look at non globally installed libraries and the virtualenv package makes this quite easy to do. An added bonus is that using virtualenv and pip, you are able to set up your project environment as a non-root user, which makes things easier and safer.
First though we still need to install virtualenv and this is done easily using the pip command: (you need to be root for this)
pip install virtualenv Collecting virtualenv Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB) 100% |........................................| 1.8MB 661kB/s Installing collected packages: virtualenv Successfully installed virtualenv-15.1.0
4. Create a virtual environment
Now that all the moving parts are installed, we can go ahead and create a project environment. And as mentioned before, this is the point where we drop root privileges! Everything below is done as a regular user.
virtualenv -p python3 hbaserest Running virtualenv with interpreter /usr/bin/python3 Using base prefix '/usr' New python executable in /home/jhon/hbaserest/bin/python3 Also creating executable in /home/jhon/hbaserest/bin/python Installing setuptools, pip, wheel...done.
The “-p python3” option will make virtualenv create a python version 3 environment. If you leave this option out, you’ll end up with a python version 2 env. I am going to be working on an HBase Rest api project so I named my environment “hbaserest”.
As a result of running that command, the directory structure shown to the side was created.
The top three directories simply hold a local copy of the python libraries. Of course these are not real copies but mostly symbolic links to the global python environment under /usr/lib, /usr/lib64 and /usr/include. At this point, these directories are therefore quite small:
du -hs ./lin ./lib64 ./include 0 ./include 12M ./lib 0 ./lib64
Once you start to install additional libraries for your project, those will be installed directly in these directories and at that point the sizes will increase.
Apart from those directories, there is the json file is just something pip uses for self-checking, as the name implies. The more interesting parts are in the bin directory. There you will the actual virtualenv “activate” script.
5. Using the virtual environment
Every time you want to start working in your virtual environment, you need to “activate” it, meaning you source the activate script to set the execution environment variables. It is recommended to source the script when inside the working directory.
cd hbaserest/ python --version Python 2.7.5 source ./bin/activate python --version Python 3.4.5 deactivate python --version Python 2.7.5
I threw in a couple of “python –version” commands to demonstrate that the environment actually does change. Once you source the activate script, you will also see the command prompt changes to reflect the loaded virtual environment.
Once you’re done with the environment, you can use the “deactivate” function (which was added by the activate script) to undo the virtualenv environment changes.
6. Conclusion
So now the virtualenv is set up, simply remember to run the “source ./bin/activate” script every time you start working on your project and as long as you use the pip command, everything you add to the python environment should get added to the virtual environment, nicely isolated from the rest of the system.
Hi Jhon,
I am using CentOS7 and I have a default python version of Python 2.7.5. I want to install virtualenv and Python3.6. However, when I am trying your steps, I am getting the following error:
$ sudo yum -y install epel-release
Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Package epel-release-6-8.noarch already installed and latest version
Nothing to do
$ sudo yum -y install python34
Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Resolving Dependencies
–> Running transaction check
—> Package python34.x86_64 0:3.4.8-1.el6 will be installed
–> Processing Dependency: python34-libs(x86-64) = 3.4.8-1.el6 for package: python34-3.4.8-1.el6.x86_64
–> Processing Dependency: libpython3.4m.so.1.0()(64bit) for package: python34-3.4.8-1.el6.x86_64
–> Running transaction check
—> Package python34-libs.x86_64 0:3.4.8-1.el6 will be installed
–> Processing Dependency: liblzma.so.0()(64bit) for package: python34-libs-3.4.8-1.el6.x86_64
–> Processing Dependency: libgdbm.so.2()(64bit) for package: python34-libs-3.4.8-1.el6.x86_64
–> Processing Dependency: libffi.so.5()(64bit) for package: python34-libs-3.4.8-1.el6.x86_64
–> Finished Dependency Resolution
Error: Package: python34-libs-3.4.8-1.el6.x86_64 (epel)
Requires: libgdbm.so.2()(64bit)
Error: Package: python34-libs-3.4.8-1.el6.x86_64 (epel)
Requires: liblzma.so.0()(64bit)
Error: Package: python34-libs-3.4.8-1.el6.x86_64 (epel)
Requires: libffi.so.5()(64bit)
You could try using –skip-broken to work around the problem
You could try running: rpm -Va –nofiles –nodigest
$ sudo yum -y install python-pip
Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Resolving Dependencies
–> Running transaction check
—> Package python-pip.noarch 0:7.1.0-1.el6 will be installed
–> Processing Dependency: python(abi) = 2.6 for package: python-pip-7.1.0-1.el6.noarch
–> Finished Dependency Resolution
Error: Package: python-pip-7.1.0-1.el6.noarch (epel)
Requires: python(abi) = 2.6
Installed: python-2.7.5-69.el7_5.x86_64 (@rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-16.el7.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-18.el7_1.1.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-34.el7.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-38.el7_2.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-39.el7_2.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-48.el7.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-58.el7.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-68.el7.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python-2.7.5-76.el7.x86_64 (rhui-REGION-rhel-server-releases)
python(abi) = 2.7
python(abi) = 2.7
Available: python34-3.4.8-1.el6.i686 (epel)
python(abi) = 3.4
You could try using –skip-broken to work around the problem
You could try running: rpm -Va –nofiles –nodigest
Can you please help me understand what am I missing here?
Thanks in advance.
This blog is a bit old now, so before debugging your output, I tried the install of Epel and python3.4 on a new CentOS7.4 system and that still work as described.
Looking at your output, your Epel install seems to fail:
—
$ sudo yum -y install epel-release
Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Package epel-release-6-8.noarch already installed and latest version
Nothing to do
—
It tries to install epel-6-8 which is strange if yo are on Centos7. Epel releases have the same version number as the CentOS version you are trying to install on. (On my little test on CentOS7.4.1708, I got Epel release 7.11)
I see that you have a plugin called amazon-id loaded… Are you trying this on an AWS VM?
I have no experience with AWS (I use Azure) but I believe that the Linux versions on AWS are not “plain vanilla” so that may have something to do with the strange version request…
But anyway, looking at the output, when you try to install the correct epel release, it tells you that Epel 6.8 is already installed, which is strange. Since the correct Epel release is never installed, the rest of the steps of course also fail.
You could try to uninstall the epel-6-8 and then reinstall epel, hoping to get the CentOS7 version, but again, if you are on AWS, I have no idea how that will affect your VM.