Data Tools

Apify

freemium

Managed scraping infra + open-source SDK.

— Hosted or self-run. Their SDK alone is worth a look.

cheerio

OSS

jQuery-style server-side HTML parsing.

— We use it for every non-SPA crawl in the KB pipeline.

linkedom

OSS

Spec-compliant DOM for worker/Node environments.

— When you need a real DOM and cheerio's jQuery style isn't enough.

Playwright

OSS

Headless browser for when fetch+parse isn't enough.

— Ships its own browsers; no driver drama. Our default for SPA scraping.

Puppeteer

OSS

Playwright's older sibling. Still fine, still maintained.

— Reach for this when the team already knows it; Playwright when starting fresh.

Scrapy

OSS

Python-native crawling framework with built-in queue + middlewares.

— The right tool at 10K+ pages. Scales without becoming your project.

Dagster

freemium

Type-first data pipelines with asset lineage.

— More opinionated than Prefect. Pick it when asset thinking fits your data.

DuckDB

OSS

Local analytical SQL on files.

— We use it for CSV wrangling in consulting engagements. One binary, no server.

Prefect

freemium

Python workflow orchestration with UI.

— What VORLUX's orchestrator would look like if we used off-the-shelf scheduling.

marimo

OSS

Reactive Python notebooks that aren't a pile of state.

— Open-source, modern. Pick it over Jupyter for anything shared.

Polars

OSS

DataFrames in Rust. Fast.

— Rewrite your slow pandas cells for 10x speedup, often zero code changes.