add option to remove passes (pytorch#84425)

Summary:
Add a remove_pass method in pass_manager to provide user option to remove any pass.

Reviewed By: wushirong

Differential Revision: D39080077

Pull Request resolved: pytorch#84425
Approved by: https://github.com/yinghai

torch/fx/passes/pass_manager.py

@@ -212,6 +212,16 @@ def add_constraint(self, constraint):
         self.constraints.append(constraint)
         self._validated = False
 
+    def remove_pass(self, _passes: List[Callable]):
+        if _passes is None:
+            return
+        passes_left = []
+        for ps in self.passes:
+            if ps.__name__ not in _passes:
+                passes_left.append(ps)
+        self.passes = passes_left
+        self._validated = False
+
     def validate(self):
         """
         Validates that current pass schedule defined by `self.passes` is valid