[TVMC] TODO items on TVMC backlog

[leandron]

This is a list of items we recorded as TODO, during TVMC code-reviews.

The list is sorted by my perceived priority implementing them:

• In tvmc compile and tvmc tune, modify the way we deal with the output of relay.build, and remove usage of graph_module.get_json(), graph_module.get_lib(), graph_module.get_params(). This will also change the way tvmc run consumes the produced TAR file
• In tvmc run, expand the possibilities for --devices, to cover more alternatives supported by tvm.rpc.client
• tests: make some tests to only check TVM API calls with the right params, suggested in the context of [TVMC] add cl support in tvmc runner #6831
• In tvmc compile, expand the target file to be downloaded from a URL
• In tvmc compile, eliminate the duplicated code in tvmc/compiler.py, inegrating tvmc.common.target_from_cli(target) (this is [TVMC] use common function to obtain target from --target value on 'tvmc compile' #6788)
• We can't, at the moment use TVMC to use composite targets: https://discuss.tvm.apache.org/t/rfc-composite-target/7744
• There is no API so that we can collect all the available targets, to show them to the end-user

The changes will be dealt with, in separate PRs.

There are also other items discussed, that are outside of the immediate TVMC module:

• The SaveToFile interface doesn't offer a robust behavior across different targets: https://discuss.tvm.apache.org/t/rfc-savetofile-file-name-format-expected-behavior/7741

Edit 23/Feb/2021: updated with completed tasks in this backlog.

cc @Mousius @gromero

[tqchen]

@leandron please update the state of the thread accordingly

[leandron]

@leandron please update the state of the thread accordingly

The status is correct as it is displayed. We'll be updating it in the next weeks with new PRs.

[leandron]

@FrozenGene, @comaniac we are currently working on the first item of this TODO list:

In tvmc compile and tvmc tune, modify the way we deal with the output of relay.build, and remove usage of graph_module.get_json(), graph_module.get_lib(), graph_module.get_params(). This will also change the way tvmc run consumes the produced TAR file

Previous discussion here: #6578 (comment)

We are struggling a bit to get rid of the getters usage, as (we think) we need a representation of the graph in order to re-load the graph, input names and shapes at tvmc run time.

Do you have an example in the code base, about how you see that working, that doesn't realy on the getters at all, but still is able to lookup for inputs, shapes, etc?

[FrozenGene]

@leandron I think your point is you want to inspect the graph and query the shape information. Ah, yes, I think previous design doesn't consider this requirement, so after serializing the module, we want users only use graph runtime factory module. cc @tqchen how about meeting this inspect requirement if we package everything into graph runtime factory module?

[leandron]

@leandron I think your point is you want to inspect the graph and query the shape information. Ah, yes, I think previous design doesn't consider this requirement, so after serializing the module, we want users only use graph runtime factory module. cc @tqchen how about meeting this inspect requirement if we package everything into graph runtime factory module?

So we are currently using the JSON for two things:

1. To create the GraphModule or the GraphModuleDebug (when --profile is used)
2. To extract input shapes

For 1, I found a way to create the Graph Runtime directly pointing to the loaded module, but it looks like it is not possible to accomplish the same for the Debug Graph Runtime, that requires the graph_json.

tvm/python/tvm/contrib/debugger/debug_runtime.py

Line 74 in 79b6ef7

class GraphModuleDebug(graph_runtime.GraphModule):

For 2, would that be possible to extract input shapes from get_num_input() and get_input(n)?

[masahi]

Can we close this?