Skip to content

Dataflow construction seems slow-ish #308

@frankmcsherry

Description

@frankmcsherry

I'm not certain this is a real issue, but the event_driven example can be made to create 1000 dataflows each a sequence of 1000 nodes, and doing so takes about 10s on my modern desktop. That seems like a long time to put together what is meant to be a linear amount of information. It does seem to be linear (scaling up either number by 10 changes the time by 10), but it would still be good to get a sense for what is going on. Initial investigation suggests allocations in reachability, which indeed does have a bunch of small per-operator allocations. Perhaps those could be consolidated somehow?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions