Move the majority of jobs to the full pipeline

[MarcelKoch]

This PR moves the majority of the gitlab jobs to the full pipeline.

IMO our CI could use a more thourough restructuring, since the job properties are very random, but that should be done later.

[yhmtsai]

I would still prefer the original grouping strategy
The configuration under same compiler version are slightly different, so grouping them together might be easier to configure or distinguish them.

[MarcelKoch]

@yhmtsai can you explain the original grouping strategy? For me all options, except for device+compiler, seem to be choosen very randomly. I.e. wether it's debug or release, shared or static, fast test enabled or not, .... This became especially clear for the #1536 PR.

[yhmtsai]

I mean grouping by device+compiler.
for example, we might have several configuration under cuda 12.2.
we might only adjust a bit for debug/release or some minor configuration settings.
putting them together makes it easier to see minor changes than separating them partially in quick test section and partially in full test.

[pratikvn]

I think it would make sense to group jobs properly to be able to navigate and access them easily. The main groups could be cuda, hip, sycl, mpi. In each of these groups, the number of jobs should be fewer. I dont know if it is possible to be subgroups within groups, but gitlab does allow for creation of groups atleast: https://docs.gitlab.com/ee/ci/jobs/#group-jobs-in-a-pipeline

[MarcelKoch]

@yhmtsai that grouping is still there, just now it's under the full pipeline. I don't think it's reasonable to have our current 'quick' pipeline run with 15+ jobs, while our 'full' pipeline has ~10 jobs. The quick pipeline is always run, even if the PR is still a bit under development. The 15+ jobs are not needed during that phase, so the CI is just clogged with meaningless tests.

[MarcelKoch]

@pratikvn That looks like a nice feature.

Sidenote: I would not count mpi as separated from the other groups, instead mpi should be enabled in almost all of our jobs.

[yhmtsai]

@MarcelKoch I do not mean the quick/full changes in this pr.
more precise example in changes
rocm45 has two jobs at least. one is in quick section and the rest is in full section, but they are located quite far away in the code.
you can still put them together as the original one but only change the condition to quick like now.

original:

...
rocm45/config1: # no matter it is in full or quick
rocm45/config2: # full/quick is irreverent with location 
...

the current change

# Quick
rocm45/config1:
...
...
# Full
rocm45/config2: # it is far from the config1

[MarcelKoch]

Ah, I see. I understand your issue, perhaps the file restructuring would fit better in a later PR, e.g. by naming those test just rocm-quick, etc.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud