Intro¶
Welcome to Day 18 of ’31 Days of Python for Ops’.
Today, we’re diving into pip, Python’s package installer!
What is PIP?¶
PIP is a recursive anacronym, for either
PIP Installs Packages
OR
PIP Installs Python
I like both. My favorite recursive anacronym is YAML, but that is a topic for another time.
Introduction to PIP¶
PIP is Python’s package manager, used to install, manage, and uninstall Python software packages.
It connects to the Python Package Index, see www.pypi.org.
This walkthrough is being done in Jupyter Notebook.
Bash/Command Line/Terminal Commands have a preceding ! (exclamation point).
Alternatively, %%bash is used in the line preceding terminal commands.
The command to install packages is:
pip install [package_name]
Install with PIP¶
- Let’s install a package named sklearn with PIP and explore the output and results.
- Then, we will try to import the package, which will result in an error.
- Next, we will then run the command pip install scikit-learn.
- Finally, we will verify that package importing works and the package location.
Verify package not installed¶
In [1]:
%%bash
ls
Untitled.ipynb myenv requirements.txt
In [2]:
!ls myenv/lib/python3.11/site-packages/ | grep sklearn
File is not there! No output.
In [3]:
import sklearn
Uh-Oh… Error because package is not installed.
Let’s install it¶
In [4]:
!pip install scikit-learn
Collecting scikit-learn Using cached scikit_learn-1.4.0-1-cp311-cp311-macosx_12_0_arm64.whl (10.6 MB) Requirement already satisfied: numpy<2.0,>=1.19.5 in ./myenv/lib/python3.11/site-packages (from scikit-learn) (1.26.3) Requirement already satisfied: scipy>=1.6.0 in ./myenv/lib/python3.11/site-packages (from scikit-learn) (1.11.4) Requirement already satisfied: joblib>=1.2.0 in ./myenv/lib/python3.11/site-packages (from scikit-learn) (1.3.2) Requirement already satisfied: threadpoolctl>=2.0.0 in ./myenv/lib/python3.11/site-packages (from scikit-learn) (3.2.0) Installing collected packages: scikit-learn Successfully installed scikit-learn [notice] A new release of pip is available: 23.1 -> 23.3.2 [notice] To update, run: pip install --upgrade pip
It’s installed now let’s check!¶
I had some issues.
Appears you occasionally need to restart the kernel to use after install in Jupyter Notebook.
In [ ]:
# This will restart the kernel in Jupyter Notebook without having to open a terminal or close the session
import os
os._exit(00)
In [1]:
!ls myenv/lib/python3.11/site-packages/ | grep sklearn
sklearn
In [2]:
import sklearn
print("Imported without error")
Imported without error
Important PIP commands¶
Let’s investigate the most used functions of PIP
Unistall with PIP¶
In [3]:
!pip uninstall scikit-learn -y
Found existing installation: scikit-learn 1.4.0 Uninstalling scikit-learn-1.4.0: Successfully uninstalled scikit-learn-1.4.0
List Packages¶
In [4]:
!pip list | head -n 15
Package Version ------------------------- --------------- anyio 4.2.0 appnope 0.1.3 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 async-lru 2.0.4 attrs 23.2.0 Babel 2.14.0 beautifulsoup4 4.12.3 bleach 6.1.0 certifi 2023.11.17 cffi 1.16.0 ERROR: Pipe to stdout was broken [notice] A new release of pip is available: 23.1 -> 23.3.2 [notice] To update, run: pip install --upgrade pip Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> BrokenPipeError: [Errno 32] Broken pipe
PIP Freeze¶
This is similar to pip list, but the format will work with requirement file format.
In [5]:
!pip freeze | head -n 15
anyio==4.2.0 appnope==0.1.3 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 arrow==1.3.0 asttokens==2.4.1 async-lru==2.0.4 attrs==23.2.0 Babel==2.14.0 beautifulsoup4==4.12.3 bleach==6.1.0 certifi==2023.11.17 cffi==1.16.0 charset-normalizer==3.3.2 comm==0.2.1
Note the difference between pip list and pip freeze ¶
Requirements file¶
Use pip freeze to create a requirements file
In [8]:
!pip freeze > requirements.txt
In [9]:
cat requirements.txt | head -n 25
anyio==4.2.0 appnope==0.1.3 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 arrow==1.3.0 asttokens==2.4.1 async-lru==2.0.4 attrs==23.2.0 Babel==2.14.0 beautifulsoup4==4.12.3 bleach==6.1.0 certifi==2023.11.17 cffi==1.16.0 charset-normalizer==3.3.2 comm==0.2.1 debugpy==1.8.0 decorator==5.1.1 defusedxml==0.7.1 executing==2.0.1 fastjsonschema==2.19.1 fqdn==1.5.1 idna==3.6 ipykernel==6.29.0 ipython==8.20.0 ipywidgets==8.1.1
Then you can use or share the requirements.txt file to install from
In [10]:
!pip install -r requirements.txt
Requirement already satisfied: anyio==4.2.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (4.2.0) Requirement already satisfied: appnope==0.1.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (0.1.3) Requirement already satisfied: argon2-cffi==23.1.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (23.1.0) Requirement already satisfied: argon2-cffi-bindings==21.2.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (21.2.0) Requirement already satisfied: arrow==1.3.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (1.3.0) Requirement already satisfied: asttokens==2.4.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 6)) (2.4.1) Requirement already satisfied: async-lru==2.0.4 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (2.0.4) Requirement already satisfied: attrs==23.2.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (23.2.0) Requirement already satisfied: Babel==2.14.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (2.14.0) Requirement already satisfied: beautifulsoup4==4.12.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 10)) (4.12.3) Requirement already satisfied: bleach==6.1.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 11)) (6.1.0) Requirement already satisfied: certifi==2023.11.17 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 12)) (2023.11.17) Requirement already satisfied: cffi==1.16.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 13)) (1.16.0) Requirement already satisfied: charset-normalizer==3.3.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 14)) (3.3.2) Requirement already satisfied: comm==0.2.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 15)) (0.2.1) Requirement already satisfied: debugpy==1.8.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 16)) (1.8.0) Requirement already satisfied: decorator==5.1.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 17)) (5.1.1) Requirement already satisfied: defusedxml==0.7.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 18)) (0.7.1) Requirement already satisfied: executing==2.0.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 19)) (2.0.1) Requirement already satisfied: fastjsonschema==2.19.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 20)) (2.19.1) Requirement already satisfied: fqdn==1.5.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 21)) (1.5.1) Requirement already satisfied: idna==3.6 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 22)) (3.6) Requirement already satisfied: ipykernel==6.29.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 23)) (6.29.0) Requirement already satisfied: ipython==8.20.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 24)) (8.20.0) Requirement already satisfied: ipywidgets==8.1.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 25)) (8.1.1) Requirement already satisfied: isoduration==20.11.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 26)) (20.11.0) Requirement already satisfied: jedi==0.19.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 27)) (0.19.1) Requirement already satisfied: Jinja2==3.1.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 28)) (3.1.3) Requirement already satisfied: joblib==1.3.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 29)) (1.3.2) Requirement already satisfied: json5==0.9.14 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 30)) (0.9.14) Requirement already satisfied: jsonpointer==2.4 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 31)) (2.4) Requirement already satisfied: jsonschema==4.21.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 32)) (4.21.0) Requirement already satisfied: jsonschema-specifications==2023.12.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 33)) (2023.12.1) Requirement already satisfied: jupyter-console==6.6.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 34)) (6.6.3) Requirement already satisfied: jupyter-events==0.9.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 35)) (0.9.0) Requirement already satisfied: jupyter-lsp==2.2.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 36)) (2.2.2) Requirement already satisfied: jupyter_client==8.6.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 37)) (8.6.0) Requirement already satisfied: jupyter_core==5.7.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 38)) (5.7.1) Requirement already satisfied: jupyter_server==2.12.5 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 39)) (2.12.5) Requirement already satisfied: jupyter_server_terminals==0.5.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 40)) (0.5.1) Requirement already satisfied: jupyterlab==4.0.10 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 41)) (4.0.10) Requirement already satisfied: jupyterlab-widgets==3.0.9 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 42)) (3.0.9) Requirement already satisfied: jupyterlab_pygments==0.3.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 43)) (0.3.0) Requirement already satisfied: jupyterlab_server==2.25.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 44)) (2.25.2) Requirement already satisfied: MarkupSafe==2.1.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 45)) (2.1.3) Requirement already satisfied: matplotlib-inline==0.1.6 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 46)) (0.1.6) Requirement already satisfied: mistune==3.0.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 47)) (3.0.2) Requirement already satisfied: nbclient==0.9.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 48)) (0.9.0) Requirement already satisfied: nbconvert==7.14.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 49)) (7.14.2) Requirement already satisfied: nbformat==5.9.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 50)) (5.9.2) Requirement already satisfied: nest-asyncio==1.5.9 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 51)) (1.5.9) Requirement already satisfied: notebook_shim==0.2.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 52)) (0.2.3) Requirement already satisfied: numpy==1.26.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 53)) (1.26.3) Requirement already satisfied: overrides==7.4.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 54)) (7.4.0) Requirement already satisfied: packaging==23.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 55)) (23.2) Requirement already satisfied: pandocfilters==1.5.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 56)) (1.5.0) Requirement already satisfied: parso==0.8.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 57)) (0.8.3) Requirement already satisfied: pexpect==4.9.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 58)) (4.9.0) Requirement already satisfied: platformdirs==4.1.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 59)) (4.1.0) Requirement already satisfied: prometheus-client==0.19.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 60)) (0.19.0) Requirement already satisfied: prompt-toolkit==3.0.43 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 61)) (3.0.43) Requirement already satisfied: psutil==5.9.7 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 62)) (5.9.7) Requirement already satisfied: ptyprocess==0.7.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 63)) (0.7.0) Requirement already satisfied: pure-eval==0.2.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 64)) (0.2.2) Requirement already satisfied: pycparser==2.21 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 65)) (2.21) Requirement already satisfied: Pygments==2.17.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 66)) (2.17.2) Requirement already satisfied: python-dateutil==2.8.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 67)) (2.8.2) Requirement already satisfied: python-json-logger==2.0.7 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 68)) (2.0.7) Requirement already satisfied: PyYAML==6.0.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 69)) (6.0.1) Requirement already satisfied: pyzmq==25.1.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 70)) (25.1.2) Requirement already satisfied: qtconsole==5.5.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 71)) (5.5.1) Requirement already satisfied: QtPy==2.4.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 72)) (2.4.1) Requirement already satisfied: referencing==0.32.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 73)) (0.32.1) Requirement already satisfied: requests==2.31.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 74)) (2.31.0) Requirement already satisfied: rfc3339-validator==0.1.4 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 75)) (0.1.4) Requirement already satisfied: rfc3986-validator==0.1.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 76)) (0.1.1) Requirement already satisfied: rpds-py==0.17.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 77)) (0.17.1) Requirement already satisfied: scipy==1.11.4 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 78)) (1.11.4) Requirement already satisfied: Send2Trash==1.8.2 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 79)) (1.8.2) Requirement already satisfied: six==1.16.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 80)) (1.16.0) Requirement already satisfied: sniffio==1.3.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 81)) (1.3.0) Requirement already satisfied: soupsieve==2.5 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 82)) (2.5) Requirement already satisfied: stack-data==0.6.3 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 83)) (0.6.3) Requirement already satisfied: terminado==0.18.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 84)) (0.18.0) Requirement already satisfied: threadpoolctl==3.2.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 85)) (3.2.0) Requirement already satisfied: tinycss2==1.2.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 86)) (1.2.1) Requirement already satisfied: tornado==6.4 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 87)) (6.4) Requirement already satisfied: traitlets==5.14.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 88)) (5.14.1) Requirement already satisfied: types-python-dateutil==2.8.19.20240106 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 89)) (2.8.19.20240106) Requirement already satisfied: uri-template==1.3.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 90)) (1.3.0) Requirement already satisfied: urllib3==2.1.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 91)) (2.1.0) Requirement already satisfied: wcwidth==0.2.13 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 92)) (0.2.13) Requirement already satisfied: webcolors==1.13 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 93)) (1.13) Requirement already satisfied: webencodings==0.5.1 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 94)) (0.5.1) Requirement already satisfied: websocket-client==1.7.0 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 95)) (1.7.0) Requirement already satisfied: widgetsnbextension==4.0.9 in ./myenv/lib/python3.11/site-packages (from -r requirements.txt (line 96)) (4.0.9) [notice] A new release of pip is available: 23.1 -> 23.3.2 [notice] To update, run: pip install --upgrade pip
In Ops, pip is invaluable for setting up your Python environment.
Whether you’re automating network tasks, processing data, or monitoring systems.
Combine automation with pip and tools like git, virtual enviroments or containers
to keep a consistent environment that is easy to share and reproduce.
Next Up!¶
I am doing a series of 31 short videos of Python topics.
Not everyday will get a blog post.
However, you can:
Check out the video for today where I do most of this, but in a terminal, at:
Check out the full series at:
https://youtube.com/playlist?list=PL_05ep0dPDj9mJdAOlkTZ765w9psDAGe-&si=z38Qh-fK_jmjOJj-
