[Tracking Issue] TFLite operator support

[leandron]

In #9187 we implemented quantised version of operators in TFLite frontend. Recently, I just noticed a few more operators (with varying priorities) that can be taken as beginner friendly tasks, for someone who's starting in TVM.

I'm creating this as a tracking issue to mark those operators that need to have quantization support implemented and tested:

• FLOOR_DIV ([TFLite][Frontend] Support quantized floor_div #15724)
• FLOOR_MOD ([TFLite][Frontend] Support quantized floor_mod #15733)
• DIV ([TFLite][Frontend] Support quantized div #15768)
• POW ([TFLite][Frontend] Support quantized Pow #15798)
• REVERSE_SEQUENCE ([TFLite][Frontend] Support quantized Reverse sequence #15915)
• SQUARE ([TFLite][Frontend] Support quantized SQUARE #15914)
• ELU ([TFLite][Frontend] Support quantized ELU #15821)
• LOCAL_RESPONSE_NORMALIZATION (LRN)
• L2_NORMALIZATION
• MIRROR_PAD ([TFLite] Add support for quantized mirror pad #16243)
• ARG_MIN ([TFLite][Frontend] Support quantized ARG_MIN #15922)
• NOT_EQUAL ([TFLite][Frontend] Support quantized NOT_EQUAL #15769)
• LESS ([TFLite][Frontend] Support quantized LESS #15746)
• LESS_EQUAL ([TFLite][Frontend] Support quantized LESS_EQUAL #15790)
• GREATER_EQUAL ([TFLite][Frontend] Support quantized GREATER_EQUAL #15775)

For each operator above, there are actions to be taken:

• Remove the conditional block if self.is_quantized(op): ...
• Implement unit tests
• ask any committer in the project to update this issue, reflecting the PR to support the operators on the list above

#9165 can be used as an example.

In case you're interested, a good starting point is the TFLite frontend:

tvm/python/tvm/relay/frontend/tflite.py

Lines 78 to 185 in bee073b

	self.convert_map = {
	"ABS": self.convert_abs,
	"ADD": self.convert_add,
	"ADD_N": self.convert_add_n,
	"ARG_MAX": self.convert_arg_max,
	"ARG_MIN": self.convert_arg_min,
	"AVERAGE_POOL_2D": self.convert_average_pool2d,
	"BATCH_TO_SPACE_ND": self.convert_batch_to_space_nd,
	"BATCH_MATMUL": self.convert_batch_matmul,
	"CAST": self.convert_cast,
	"CEIL": self.convert_ceil,
	"CONCATENATION": self.convert_concatenation,
	"CONV_2D": self.convert_conv2d,
	"COS": self.convert_cos,
	"DENSIFY": self.convert_densify,
	"DEPTH_TO_SPACE": self.convert_depth_to_space,
	"DEPTHWISE_CONV_2D": self.convert_depthwise_conv2d,
	"DEQUANTIZE": self.convert_dequantize,
	"DETECTION_POSTPROCESS": self.convert_detection_postprocess,
	"DIV": self.convert_div,
	"ELU": self.convert_elu,
	"EQUAL": self.convert_equal,
	"EXP": self.convert_exp,
	"EXPAND_DIMS": self.convert_expand_dims,
	"FAKE_QUANT": self.convert_fake_quant,
	"FILL": self.convert_fill,
	"FLOOR_DIV": self.convert_floor_div,
	"FLOOR_MOD": self.convert_floor_mod,
	"FLOOR": self.convert_floor,
	"FULLY_CONNECTED": self.convert_fully_connected,
	"GATHER": self.convert_gather,
	"GATHER_ND": self.convert_gather_nd,
	"GREATER_EQUAL": self.convert_greater_equal,
	"GREATER": self.convert_greater,
	"HARD_SWISH": self.convert_hard_swish,
	"L2_NORMALIZATION": self.convert_l2_normalization,
	"L2_POOL_2D": self.convert_l2_pool2d,
	"LEAKY_RELU": self.convert_leaky_relu,
	"LESS_EQUAL": self.convert_less_equal,
	"LESS": self.convert_less,
	"LOCAL_RESPONSE_NORMALIZATION": self.convert_lrn,
	"LOG": self.convert_log,
	"LOG_SOFTMAX": self.convert_log_softmax,
	"LOGICAL_AND": self.convert_logical_and,
	"LOGICAL_NOT": self.convert_logical_not,
	"LOGICAL_OR": self.convert_logical_or,
	"LOGISTIC": self.convert_logistic,
	"MATRIX_DIAG": self.convert_matrix_diag,
	"MATRIX_SET_DIAG": self.convert_matrix_set_diag,
	"MAX_POOL_2D": self.convert_max_pool2d,
	"MAXIMUM": self.convert_maximum,
	"MEAN": self.convert_reduce_mean,
	"MINIMUM": self.convert_minimum,
	"MIRROR_PAD": self.convert_mirror_pad,
	"MUL": self.convert_mul,
	"NEG": self.convert_neg,
	"NOT_EQUAL": self.convert_not_equal,
	"ONE_HOT": self.convert_one_hot,
	"PACK": self.convert_pack,
	"PAD": self.convert_pad,
	"PADV2": self.convert_pad,
	"POW": self.convert_pow,
	"PRELU": self.convert_prelu,
	"RANGE": self.convert_range,
	"QUANTIZE": self.convert_quantize,
	"REDUCE_ANY": self.convert_reduce_any,
	"REDUCE_MAX": self.convert_reduce_max,
	"REDUCE_MIN": self.convert_reduce_min,
	"REDUCE_PROD": self.convert_reduce_prod,
	"RELU": self.convert_relu,
	"RELU6": self.convert_relu6,
	"RELU_N1_TO_1": self.convert_relu_n1_to_1,
	"RESHAPE": self.convert_reshape,
	"RESIZE_BILINEAR": self.convert_resize_bilinear,
	"RESIZE_NEAREST_NEIGHBOR": self.convert_resize_nearest_neighbor,
	"ROUND": self.convert_round,
	"RSQRT": self.convert_rsqrt,
	"REVERSE_SEQUENCE": self.convert_reverse_sequence,
	"REVERSE_V2": self.convert_reverse_v2,
	"SELECT": self.convert_select,
	"SHAPE": self.convert_shape,
	"SIN": self.convert_sin,
	"SLICE": self.convert_slice,
	"SOFTMAX": self.convert_softmax,
	"SPACE_TO_BATCH_ND": self.convert_space_to_batch_nd,
	"SPACE_TO_DEPTH": self.convert_space_to_depth,
	"SPARSE_TO_DENSE": self.convert_sparse_to_dense,
	"SPLIT": self.convert_split,
	"SPLIT_V": self.convert_split_v,
	"SQRT": self.convert_sqrt,
	"SQUARE": self.convert_square,
	"SQUARED_DIFFERENCE": self.convert_squared_difference,
	"SQUEEZE": self.convert_squeeze,
	"STRIDED_SLICE": self.convert_strided_slice,
	"SUB": self.convert_sub,
	"SUM": self.convert_reduce_sum,
	"TAN": self.convert_tan,
	"TANH": self.convert_tanh,
	"TILE": self.convert_tile,
	"TOPK_V2": self.convert_topk_v2,
	"TRANSPOSE_CONV": self.convert_transpose_conv,
	"TRANSPOSE": self.convert_transpose,
	"UNPACK": self.convert_unpack,
	"UNIDIRECTIONAL_SEQUENCE_LSTM": self.convert_unidirectional_sequence_lstm,
	"WHERE": self.convert_select,
	"ZEROS_LIKE": self.convert_zeros_like,
	"NON_MAX_SUPPRESSION_V5": self.convert_nms_v5,
	}

[p3achyjr]

hi--if no one has taken this yet, can I give this a shot?

[leandron]

hi--if no one has taken this yet, can I give this a shot?

Absolutely @p3achyjr. Each operator can be submitted individually, so we have self contained, easier to review PRs.

Let us know if you need any help. Thanks!

[tlopex]

@leandron I see that the code has changed a lot from the example#9165. All tests are all combined to one function, so if I just need to remove the conditional block and test it? Besides, because I am totally new to it, I find it a little bit tough to have a test, can you offer me some simple ways?

[p3achyjr]

@tlopex if you're working on this too, would you like to divide up the operators so we don't do redundant work?

[tlopex]

@p3achyjr I'm glad to do that. What operators tasks have you doing now?

[p3achyjr]

I won't be able to work on this for the next couple weeks, so feel free to take anything :)

[leandron]

@leandron I see that the code has changed a lot from the example#9165. All tests are all combined to one function, so if I just need to remove the conditional block and test it? Besides, because I am totally new to it, I find it a little bit tough to have a test, can you offer me some simple ways?

Maybe #14667 can be an updated version of the way to support new operators? Have a look on this one and let me know.

[tlopex]

@p3achyjr So am I. I think I won't have time to do this work until the first few days of September. If you start to work on it, just inform me. Thanks.

[tlopex]

@leandron I have seen #14667 , which is a good example for us to mimic. However, it is not the latest version of code.

[p3achyjr]

I'm back now. I can take a look at FLOOR_DIV.

[tlopex]

@p3achyjr Okay, got it. I am now trying to work on SQUARE.

[p3achyjr]

Finally getting around to this--a draft PR is at #15724

[tlopex]

Got it. Actually I don't know clearly how to write a test for unary element operation... And I will try again in following days.