The Python Libraries That Earned a Permanent Spot in My Projects
By My Code Diary
I used to reinstall the same libraries every time I started a new project. Not because I forgot them, but because I kept convincing myself there was something better out there.
But There wasn’t.
After four years of Python, hundreds of scripts, and more Stack Overflow tabs than I care to admit, I stopped chasing new tools and started paying attention to the ones that showed up in every single project I built. These are those libraries. Not the flashiest ones. Not the ones with the most GitHub stars. The ones that actually solved real problems and kept showing up.
1. httpx requests, but grown up
I used requests for years. Everyone does. It’s fine. But the moment I needed async HTTP calls, and that moment always comes, I had to rewrite half my code.
httpx supports both sync and async out of the box with nearly identical syntax. That means you can start a project with synchronous calls and scale to async without a rewrite.
import httpx
# Sync — simple as always
response = httpx.get("https://api.example.com/data")
print(response.json())
# Async — same feel, zero friction
import asyncio
async def fetch():
async with httpx.AsyncClient() as client:
response = await client.get("https://api.example.com/data")
return response.json()
asyncio.run(fetch())
The real value? It future-proofs your code. You don’t have to decide upfront whether your project needs async. You can decide later; migration costs are minimal.
2. pydantic because dicts will betray you
I spent a year passing dictionaries around like they were perfectly fine. They are not fine.
A dictionary doesn’t tell you what keys should exist. It doesn’t validate types. It doesn’t throw a useful error when something is missing. It just silently fails, usually at 2 am when you’re trying to figure out why your data pipeline broke.
pydantic changed that. You define your data as a class, and it handles validation automatically.
from pydantic import BaseModel, EmailStr
class User(BaseModel):
name: str
age: int
email: EmailStr
user = User(name="Ahmed", age=28, email="ahmed@example.com")
print(user.model_dump())
If the data is the wrong type, missing field, invalid email, you get a clear error immediately. Not three functions later, not in production.
Pro Tip: Pydantic v2 (released 2023) is written in Rust under the hood. It’s not just cleaner code; it’s also significantly faster for large-scale validation tasks.
3. loguru logging without the suffering
Python’s built-in logging module works. It also requires 15 lines of setup before you can log a single message. That’s not a feature.
loguru has one import and zero configuration.
from loguru import logger
logger.info("Script started")
logger.warning("This might be a problem")
logger.error("This definitely is a problem")
It also writes to files, rotates logs automatically, and formats tracebacks beautifully. I added this to a scraper project once and within 20 minutes, I had caught a bug I’d been ignoring for a week simply because I could finally read what was happening.
4. pathlib stop writing os.path.join
This one is embarrassing to admit. I used os.path.join for two years before someone showed me pathlib. Two years of writing this:
import os
full_path = os.path.join(base_dir, "data", "file.csv")
When I could have been writing this:
from pathlib import Path
full_path = Path("data") / "file.csv"
That / operator isn’t a trick. It’s path concatenation. It works across Windows, Mac, and Linux without you thinking about it. And path.read_text(), path.write_text(), path.mkdir(parents=True, exist_ok=True) all built in, all clean.
This is in the standard library. No install needed. There is no reason not to use it.
5. rich your terminal deserves better
Most Python output looks like it was generated in 1998. Plain text, no color, no structure. You print a nested dictionary, and it comes out as a wall of text.
rich solves this.
from rich import print
from rich.table import Table
table = Table(title="Libraries I Use")
table.add_column("Library", style="cyan")
table.add_column("Purpose", style="green")
table.add_row("httpx", "Async HTTP requests")
table.add_row("pydantic", "Data validation")
table.add_row("loguru", "Clean logging")
print(table)
Your terminal output becomes structured, colored, and readable. For any project where you’re reading output often scraping, data processing, automation scripts, this alone saves you meaningful time.
6. tenacity retry logic without the mess
The first time I wrote retry logic, it looked like this:
for attempt in range(3):
try:
result = call_api()
break
except Exception:
time.sleep(2)
Functional. Ugly. Hard to adjust, harder to read.
tenacity wraps this cleanly with decorators.
from tenacity import retry, stop_after_attempt, wait_exponential
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def call_api():
response = httpx.get("https://api.example.com/unstable-endpoint")
response.raise_for_status()
return response.json()
Exponential backoff, max attempts, and custom exceptions are all configurable in one line. This has saved me from flaky APIs more times than I can count.
7. dotenv secrets do not belong in your code
I once pushed an API key to a public GitHub repo. It was revoked within four minutes. I learned fast.
python-dotenv lets you store secrets in a .env file and load them at runtime.
from dotenv import load_dotenv
import os
load_dotenv()
api_key = os.getenv("OPENAI_API_KEY")
Your .env file stays local. Your code stays clean. Your API keys stay secret. This is table stakes for any project that touches external services.
8. tqdm because waiting in silence is its own kind of suffering
Running a loop over 50,000 records and staring at a blank terminal is a specific kind of anxiety. Is it working? Did it freeze? Should I restart it?
tqdm adds a progress bar to any iterable in one line.
from tqdm import tqdm
import time
for item in tqdm(range(10000), desc="Processing"):
time.sleep(0.001) # your actual work here
You get a live progress bar, estimated time remaining, and items per second. It seems small. The first time you run a long job and actually know when it will finish, you’ll wonder how you ever worked without it.
9. schedule automation without a cron headache
Cron jobs are powerful. They’re also cryptic, platform-specific, and require you to leave the Python environment entirely to configure.
schedule lets you write automation logic in plain Python.
import schedule
import time
def run_daily_report():
print("Generating report...")
schedule.every().day.at("09:00").do(run_daily_report)
schedule.every(30).minutes.do(run_daily_report)
while True:
schedule.run_pending()
time.sleep(1)
For personal automation projects, scrapers, report generators, and notification bots, this is fast to set up and easy to read. You can see exactly what runs and when, without memorizing cron syntax.
What these libraries have in common
None of these are exotic. Most are not even particularly new. What they share is that they solve real, recurring problems with minimal overhead. They have clear APIs, good documentation, and they work the way you expect them to.
The best tools are the ones you stop thinking about. These are those tools.
Start a new project this week. Drop all nine of these in from day one. You’ll spend less time fighting your tooling and more time building the thing you actually wanted to build.
That’s the whole point.
Drop your questions or your own permanent-fixture libraries in the comments.
