Please explain subgraphs with 'dangling' intermediary results

[sergei-mironov]

Hi. Could you please check my interpretation of graphs from graph_subst.pb ? I've written some Python code which iterates over all source substitutions and prints their graphs to the text file (see the listing below). The printing format is almost identical to the format of C++ GraphTemp::print from generator.cc.

Most of the rules looks natural, but there are some of them which don't, especially those which include RELU operation. One example is:

#63
0: [CONV2D] (-7,0) (-19,0)
1: [RELU] (0,0)
2: [CONV2D] (-8,0) (-10,0)

We see that the source graph of rule #63 calculates convolution of some inputs, then it applies RELU, but after that it's result is not used. Line number 2 just calculates convolution of other inputs. Why is this? Is this rule correct?

PS I use graph_subst.pb as of commit f4e1486

PPS my printing code is

import sys
import rules_pb2

from typing import Any, Dict, List
from collections import defaultdict
from itertools import chain

def load_orig(path="modules/taso/graph_subst.pb"):
  rules.ParseFromString(open(path, "rb").read())
  return rules

def print_raw_(rules,f):
  for i, rule in enumerate(rules.rule):
    f.write('='*80 + '\n')
    f.write(f"#{i}" + '\n')
    for i in range(len(rule.srcOp)):
      op = rule.srcOp[i]
      inps = [(i.opId, i.tsId) for i in op.input]
      inp_strs = [f"({a},{b})" for a,b in inps]
      f.write(f"{i}: [{OPNAMES[op.type][3:]}] {' '.join(inp_strs)}" + '\n')

def dump_raw(rules, fname):
  with open(fname, 'w') as f:
    print_raw_(rules, f)

dump_raw(load_orig(), 'raw_orig.txt')