Training PET on a personalised task

[xruifan]

Hi, I am trying to train PET on a new task. I have edited custom_task_processor.py and custom_task_pvp.py in /examples for my use. So my question is how could I use them to make the program run them before I call cli.py with --task_name my-task. I have seen people saying copying the classes directly to the respective files but is it a formal way to do that?

Thanks.

[chris-aeviator]

I have in the past also copied it into the tasks.py file and approached (yday) to make it better.

When importing the class from the examples dir, rather than adding it directly to tasks.py, I'm facing a circular import error, to be expected, since custom_task_processor.py itself also imports tasks.py. Will share my findings here but also happy if s.o. could just provide a better instruction

Update - posting some git diffs that seem to get me further

--- a/examples/custom_task_processor.py
-from pet.tasks import DataProcessor, PROCESSORS, TASK_HELPERS

-class MyTaskDataProcessor(DataProcessor):
+class MyTaskDataProcessor():

-PROCESSORS[MyTaskDataProcessor.TASK_NAME] = MyTaskDataProcessor
+#  PROCESSORS[MyTaskDataProcessor.TASK_NAME] = MyTaskDataProcessor

++ b/pet/tasks.py
@@ -25,6 +25,7 @@ from typing import List, Dict, Callable
 import log
 from pet import task_helpers
 from pet.utils import InputExample
+from examples.custom_task_processor import MyTaskDataProcessor

 logger = log.get_logger('root')

@@ -782,6 +783,7 @@ PROCESSORS = {
     "record": RecordProcessor,
     "ax-g": AxGProcessor,
     "ax-b": AxBProcessor,
+    "my-task": MyTaskDataProcessor
 }  # type: Dict[str,Callable[[],DataProcessor]]