have we ever mixed QPhiX && DDalphaAMG in the 1+1 HMC?

[kostrzewa]

@sbacchio @urbach @Finkenrath

I just realised that we might have a bug in tmLQCD + DDalphaAMG + QPhiX if DDalphaAMG with initial guesses is used in the 1+1 sector...

See line 235 below.

tmLQCD/solver/monomial_solve.c

Lines 214 to 256 in 926f87f

	#ifdef DDalphaAMG
	else if (solver_params->type == MG) {
	// if the mg_mms_mass is larger than the smallest shift we use MG
	if (mg_no_shifts > 0 || mg_mms_mass >= solver_params->shifts[0]) {
	int nshifts = solver_params->no_shifts;
	int mg_nshifts = mg_no_shifts > nshifts ? nshifts:mg_no_shifts;
	// if the mg_mms_mass is smaller than the larger shifts, we use CGMMS for those
	// in case mg_no_shifts is used, then mg_mms_mass = 0
	if(mg_mms_mass >= solver_params->shifts[0]) {
	mg_nshifts = solver_params->no_shifts;
	while (mg_mms_mass < solver_params->shifts[mg_nshifts-1]) { mg_nshifts--; }
	}
	// Number of initial guesses provided by gcmms
	// README: tunable value. 1 it's fine for now.
	int no_cgmms_init_guess = 1;
	if(no_cgmms_init_guess > mg_nshifts) {
	no_cgmms_init_guess = mg_nshifts;
	}
	#ifdef TM_USE_QPHIX
	if( solver_params->external_inverter == QPHIX_INVERTER && mg_nshifts < nshifts ) {
	// TODO: no initial guess option with QphiX
	no_cgmms_init_guess = 0;
	spinor ** P_cg = P+(mg_nshifts - no_cgmms_init_guess);
	double * shifts_start = solver_params->shifts;
	solver_params->no_shifts = nshifts - (mg_nshifts - no_cgmms_init_guess);
	solver_params->shifts += (mg_nshifts - no_cgmms_init_guess);
	solver_params->type = CGMMS;
	gamma5(temp[0], Q, VOLUME/2);
	iteration_count = invert_eo_qphix_oneflavour_mshift(P, temp[0],
	solver_params->max_iter, solver_params->squared_solver_prec,
	solver_params->type, solver_params->rel_prec,
	*solver_params,
	solver_params->sloppy_precision,
	solver_params->compression_type);
	for( int shift = 0; shift < solver_params->no_shifts; shift++) {
	mul_gamma5(P[shift], VOLUME/2);
	}
	// Restoring solver_params
	solver_params->no_shifts = nshifts;
	solver_params->shifts = shifts_start;
	solver_params->type = MG;
	} else
	#endif // TM_USE_QPHIX

I guess, however, that we never used this combination in practice, did we?

[sbacchio]

HI @kostrzewa not clear why you say it might be a bug.

The point behind no_cgmms_init_guess is to iterate in the CGMMS procedure also vectors that then will be solved by multigrid.
Meaning that these vectors will not be checked for convergence but still iterated together with the other target solutions. The partial solution then is passed to the multigrid for completion.

The point why this variable with QPhiX is set to zero is because I didn't try know how to ask QPhiX to partially invert some shifts.
On the other hand I did this with CGMMS in tmLQCD.

Independently on the value of that variable, DDalphaAMG should still take care of computing the remaining shifts (mg_nshifts)

[Finkenrath]

Hi @kostrzewa, DDalphaAMG and QPhiX was not used in the same monomial, due to the outlined issue by Simone. Namely that the multishift solver in QPhiX can not exit without iterating all solutions to the target residual. From such optimization would profit mainly the correction term, but the question would be if that would justified the coding effort (would probably require to add additional parameters to QPhiX).