Install protoc version 3.19 in order to be compatible with ufw.
mkdir build
cd build
cmake ..
make installRefer to example for how to create test specs.
Execute from SDK environment.
python3 -m tpu_perf.build --time # To build efficiency test models
python3 -m tpu_perf.build # To build precision test models
python3 -m tpu_perf.run # To run efficency test
python3 -m tpu_perf.precision_benchmark # To run precision benchmark
python3 -m tpu_perf.make_table #To make table for model zoo test result| Field | Type | Usage |
|---|---|---|
| root | Preset | Repo top directory |
| home | Preset | Current model directory |
| workdir | Preset | Current model working directory |
| lmdb_out | Preset | Dataset output dir. Only set if lmdb field exists. |
| shape | Preset | Current shape list. Only set if shapes field exists. |
| shape_key | Preset | Current shape string like "1x3x224x224". Only set if shapes field exists. |
| shape_param | Preset | Current shape string like "[1,3,224,224]". Only set if shapes field exists. |
| Field | Type | Usage |
|---|---|---|
| name | Required | Specify network name, should be unique |
| gops | Optional | Specify network FLOPs |
| Field | Type | Usage |
|---|---|---|
| precision | Optional | Boolean type. Indicates whether to build precision model when tpu_perf.build is called. |
| fp_compile_options | Optional | Will use this command to build FP32 bmodel. |
| cali | Optional | Calibration command. tpu_perf.build will do calibrating if this variable exists. |
| time_only_cali | Optional | Calibration command. tpu_perf.build --time will do calibrating if this variable exists. |
| bmnetu_options | Optional | bmnetu compile options. Will do int8 compiling if this variable exists |
| bmnetu_batch_sizes | Optional | Specify int8 output batches |
| harness | Optional | Harness to call when tpu_perf.precision_benchmark is called. |
| lmdb | Optional | This object is provided to dataset and harness plugins to do preprocess. |
Variables in top level config.yaml are also set for all models.