How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Process Hundreds of GB of Data with DuckDB in the Cloud

calendar_today August 7, 2023 person Patrick Hoefler domain coiled

Aug 7, 2023 4m read Patrick Hoefler DuckDB is great tool for running efficient queries on large datasets. When you want cloud data proximity or need more RAM, Coiled makes it easy to run your Python function in the cloud . In this post we’ll use Coiled Functions API to process the 150 GB Uber-Lyft dataset on a single machine with DuckDB .

open_in_new Read original post