Velamuvi AIAcademy
Data Science & Engineering

Working with Large Datasets

What to do when data does not fit in memory: chunking, efficient formats, Polars, Dask and sampling.

  • Advanced
  • About 6 hours
  • 2 modules
  • 8 lessons
  • Free

This course is not released yet. We publish courses one at a time, each one complete. The full syllabus is below.

What you will learn

  • Staying in MemoryMeasuring Memory Use, Efficient Data Types, Chunked Processing and more
  • Scaling OutPolars, Dask, DuckDB and more

Course syllabus

Not released yet · full syllabus below

  1. 1

    Staying in Memory

    4 lessons
    1. 1Measuring Memory UseSoon
    2. 2Efficient Data TypesSoon
    3. 3Chunked ProcessingSoon
    4. 4Parquet and ArrowSoon
  2. 2

    Scaling Out

    4 lessons
    1. 5PolarsSoon
    2. 6DaskSoon
    3. 7DuckDBSoon
    4. 8Smart SamplingSoon

Recommended next