Skip to content

examples/us_budgetary_impact.py consumes excessive memory (peak ~7.5 GB RSS / 82 GB VMS) #328

@vahid-ahmadi

Description

@vahid-ahmadi

Reported in the discussion on #264 (JOSS paper PR).

When run on the joss-paper branch (4.2.1), examples/us_budgetary_impact.py exhibits very high memory pressure during the baseline + reform simulations on the enhanced CPS 2024 dataset (2026, ~101k people / 41k households):

  • Vahid: OOM-killed at peak ~30 GB RSS on a Mac (rc=137).
  • Anthony (instrumented run): peak 7.48 GiB RSS, 82.35 GiB VMS; baseline_sim.run() 546.6s wall, reform_sim.run() 596.5s wall; VSCode crashed mid-run from memory pressure.

This makes the canonical US population example impractical for normal local development and for JOSS reviewers attempting to reproduce results.

Suggested next steps:

  • Profile baseline_sim.run() and reform_sim.run() to find the dominant allocations (likely in policyengine-us core, but worth confirming).
  • Determine whether intermediate arrays can be released between runs or whether the two simulations can share underlying data.
  • Document peak memory requirement in the example header until the footprint is reduced.

Related: #264

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions