Skip to content

Distributed implementation - #58

Closed
matthiasdiener wants to merge 12 commits into
pytato_array_contextfrom
distributed
Closed

Distributed implementation#58
matthiasdiener wants to merge 12 commits into
pytato_array_contextfrom
distributed

Conversation

@matthiasdiener

@matthiasdiener matthiasdiener commented Apr 1, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor

Continued from kaushikcfd#1

See #55.

@matthiasdiener matthiasdiener self-assigned this Apr 1, 2021
@inducer inducer mentioned this pull request Apr 1, 2021
Closed
@matthiasdiener matthiasdiener mentioned this pull request Apr 6, 2021
Closed
matthiasdiener
matthiasdiener commented Apr 8, 2021
View reviewed changes
Comment thread pytato/transform.py
return DictOfNamedArrays({key: self.rec(val.expr)
for key, val in expr.items()})

def map_distributed_send(self, expr: DistributedSend) -> DictOfNamedArrays:

@matthiasdiener matthiasdiener Apr 8, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create deep copy of object

@matthiasdiener

matthiasdiener commented Apr 8, 2021

Copy link
Copy Markdown
Contributor Author
  • Create new Mapper to partition graph

matthiasdiener
matthiasdiener commented Apr 8, 2021
View reviewed changes
Comment thread pytato/transform.py
return self.combine(frozenset([expr]), *(self.rec(ary.expr)
for ary in expr.values()))

def map_distributed_send(self, expr: DistributedSend) -> DictOfNamedArrays:

@matthiasdiener matthiasdiener Apr 8, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(see also CombineMapper in pymbolic) returns set of ?

@kaushikcfd

kaushikcfd commented Apr 8, 2021 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Create new Mapper to partition graph

So, in pytato_array_context branch the result of DictOfNamedArrays is already a special thing called a NamedArray. Probably we could rewire that to make sure that each DictOfNamedArrays emits a single loopy program. I.e. Maybe we could first check that https://github.com/kaushikcfd/pytato/blob/8bd5c3ef169c93fcc06302194329feec5675904a/test/test_codegen.py#L712-L721 results in 2 kernels and then just provide the user with helpers to aggregate bunch of nodes into a DictOfNamedArrays.

@matthiasdiener
matthiasdiener changed the base branch from pytato_array_context to main April 9, 2021 19:27
@matthiasdiener
matthiasdiener changed the base branch from main to pytato_array_context April 9, 2021 19:32
@inducer

inducer commented Apr 9, 2021

Copy link
Copy Markdown
Owner

Likely to be superseded by #65.

@inducer inducer closed this Apr 9, 2021
@inducer
inducer deleted the distributed branch April 9, 2021 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@matthiasdiener matthiasdiener

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@matthiasdiener @kaushikcfd @inducer