Skip to content

Latest commit

 

History

History
330 lines (285 loc) · 11.3 KB

leaderboard.md

File metadata and controls

330 lines (285 loc) · 11.3 KB
Raw

Leaderboard and Benchmark Results

ETAB Leaderboard

The ETAB leaderboard keeps track of the best performing backbone architectures with respect to benchmark echocardiographic tasks.

Latest update August 28, 2022
Current status ▶ Running 🔴 Cardiac structure identification benchmarks
Progress 5 out of 19 benchmark tasks completed 26%
10 out of 14 baseline models evaluated 71%
Ranking
Backbone
# Parameters
ETAB score
Task-specific performance breakdown
Pre-trained weights
1
MobileNet-V2
(ImageNet-1K weights)
3.5M
0.783
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.825 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.841 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.840 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.709 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.698 | weight: 0.2
2
ResNet-50
(Fully finetuned)
23M
0.769
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.855 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.820 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.822 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.693 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.656 | weight: 0.2
3
MobileNet-V3-Large
(Fully finetuned)
5.5M
0.749
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.838 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.805 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.808 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.656 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.636 | weight: 0.2
4
ResNet-18
(ImageNet-1K weights)
11M
0.702
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.776 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.764 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.753 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.605 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.609 | weight: 0.2
5
ResNet-34
(ImageNet-1K weights)
63M
0.699
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.774 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.734 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.734 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.643 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.611 | weight: 0.2
6
PoolFormer-S24
(ImageNet-1K weights)
21M
0.692
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.719 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.772 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.754 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.597 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.615 | weight: 0.2
7
MiT-B2
(fully tuned)
25M
0.691
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.749 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.748 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.738 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.595 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.626 | weight: 0.2
8
ResNet-50
(ImageNet-1K weights)
23M
0.689
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.787 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.738 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.719 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.604 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.597 | weight: 0.2
9
MiT-B2
(ImageNet-1K weights)
25M
0.653
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.674 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.709 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.708 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.570 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.604 | weight: 0.2
10
ConvNext-Base
(fully tuned)
8M
0.647
Score breakdown (click to expand)  
  • 🔴 a0-A4-E: 0.801 | weight: 0.2
    •  
  • 🔴 a0-A4-C: 0.647 | weight: 0.2
    •  
  • 🔴 a0-A2-C: 0.699 | weight: 0.2
    •  
  • 🔴 a1-A4-C: 0.550 | weight: 0.2
    •  
  • 🔴 a1-A2-C: 0.539 | weight: 0.2
11
DenseNet-121
(ImageNet-1K weights)
8M
---
---
12
ResNeXt-50-32x4d
(ImageNet-1K weights)
25M
---
---
13
Inception_V3
(ImageNet-1K weights)
27M
---
---
14
Inception_V3
(ImageNet-1K weights)
27M
---
---

Current configuration of the ETAB weights for models reported on the leaderboard:

weight_dict          = dict({"a0-A4-E": 0.2, "a0-A4-C": 0.2, "a0-A2-C": 0.2,
                             "a1-A4-C": 0.2, "a1-A2-C": 0.2})

How to contribute?

Instructions on how to submit your model to the ETAB leaderboard will be posted soon!