How to combine and own TestTemplateInvocationContextProvider and RepeatedTest and ParameterizedTest

[ahoehma]

This is my extension. I made this because in my business code I have lot of "if logger.isTrace() then trace else debug" ... And I need to have all code covered by tests.

But now I face a problem: I can't combine my new extension with already existing RepeatedTest and ParameterizedTest.

/**
 * This little junit5 extension allows to change the log level of loggers for a {@link TestTemplate test-method}. This allows to run the same test method with different log levels.
 *
 * @author Andreas Höhmann
 */
@Slf4j
public class LogLevelTestTemplateInvocationContextProvider implements TestTemplateInvocationContextProvider {

  @Retention(RetentionPolicy.RUNTIME)
  @Target({ElementType.TYPE, ElementType.METHOD})
  public @interface LogLevel {
    String[] loggers();
    String level();
  }

  @Retention(RetentionPolicy.RUNTIME)
  @Target({ElementType.TYPE, ElementType.METHOD})
  @ExtendWith(LogLevelTestTemplateInvocationContextProvider.class)
  public @interface LogLevels {
    LogLevel[] value();
  }

  @Override
  public boolean supportsTestTemplate(final ExtensionContext context) {
    if (context.getTestMethod().isPresent()) {
      return false;
    }
    final Method testMethod = context.getTestMethod().get();
    if (isAnnotated(testMethod, ParameterizedTest.class)) {
      log.debug("Can't combined with '{}'", ParameterizedTest.class);
      return false;
    }
    if (isAnnotated(testMethod, RepeatedTest.class)) {
      log.debug("Can't combined with '{}'", RepeatedTest.class);
      return false;
    }
    return true;
  }

  @Override
  public Stream<TestTemplateInvocationContext> provideTestTemplateInvocationContexts(final ExtensionContext context) {
    return context.getTestMethod()
        .map(this::extractedLogLevels)
        .stream()
        .flatMap(Arrays::stream)
        .flatMap(logLevels -> Arrays.stream(logLevels.value()))
        .map(logLevel -> new LogLevelInvocationContext(logLevel, context.getRequiredTestMethod().getName()));
  }

  private LogLevels[] extractedLogLevels(final Method method) {
    final Class<?> parentClass = method.getDeclaringClass();
    final LogLevels[] classLevelAnnotations = parentClass.getAnnotationsByType(LogLevels.class);
    final List<LogLevels> cl = Arrays.asList(classLevelAnnotations);
    log.debug("Found {} log level annotations '{}' for method '{}' class '{}'", cl.size(), cl, method.getName(), parentClass.getName());
    final LogLevels[] methodLevelAnnotations = method.getAnnotationsByType(LogLevels.class);
    final List<LogLevels> ml = Arrays.asList(methodLevelAnnotations);
    log.debug("Found {} log level annotations '{}' for method '{}'", ml.size(), ml, method.getName());
    final Set<LogLevels> r = new HashSet<>();
    r.addAll(cl);
    r.addAll(ml);
    log.debug("Found {} log level annotations '{}' for '{}'", r.size(), r, method.getName());
    return r.toArray(new LogLevels[0]);
  }

  private static class LogLevelInvocationContext implements TestTemplateInvocationContext {

    private final LogLevel logLevel;
    private final String methodName;

    LogLevelInvocationContext(final LogLevel logLevel, final String methodName) {
      this.logLevel = logLevel;
      this.methodName = methodName;
    }

    @Override
    public String getDisplayName(final int invocationIndex) {
      return methodName + " - Log level: " + logLevel.level();
    }

    @Override
    public List<Extension> getAdditionalExtensions() {
      return Collections.singletonList(new LogLevelExtension(logLevel));
    }
  }

  private static class LogLevelExtension implements BeforeEachCallback, AfterEachCallback {

    private final LogLevel logLevel;
    private final Map<String, Level> originalLevels = new HashMap<>();

    LogLevelExtension(final LogLevel logLevel) {
      this.logLevel = logLevel;
    }

    @Override
    public void beforeEach(final ExtensionContext context) {
      final Level level = Level.valueOf(logLevel.level().toUpperCase());
      for (final String loggerName : logLevel.loggers()) {
        final Logger logger = (Logger) LoggerFactory.getLogger(loggerName);
        originalLevels.put(loggerName, logger.getLevel());
        logger.setLevel(level);
      }
    }

    @Override
    public void afterEach(final ExtensionContext context) {
      for (final Map.Entry<String, Level> entry : originalLevels.entrySet()) {
        final Logger logger = (Logger) LoggerFactory.getLogger(entry.getKey());
        logger.setLevel(entry.getValue());
      }
    }
  }

}

This is my test:

@LogLevelTestTemplateInvocationContextProvider.LogLevels({
    @LogLevelTestTemplateInvocationContextProvider.LogLevel(loggers = {
        "com.ahoehma.test.foo"},
      level = "TRACE"),
    @LogLevelTestTemplateInvocationContextProvider.LogLevel(loggers = {
        "com.ahoehma.test.foo"},
      level = "DEBUG"),
    @LogLevelTestTemplateInvocationContextProvider.LogLevel(loggers = {
        "com.ahoehma.test.foo"},
      level = "WARN"),
})
class FooTest {

 static Stream<FooData> createFooData() {
    // parameter
  }

 @ParameterizedTest
 @MethodSource("createFooData)
 void checkFoo(final FooData data) throws Exception {
   // test code
  }
}

The problem is now that I can't combine my own LogLevelTestTemplateInvocationContextProvider extension with the ParameterizedTest extension.

So I found only a workaround to skip my extension for tests which are already annotated with ParameterizedTest or RepeatedTest.

But what I really want is my own extension which can combined with these 2 extensions.

I would like to have something like "on top of all existing" TestTemplate-Extensions ... so that I can' create even more test's based on the previously created ones .... like another loop over all calculated TestTemplateInvocationContext's or lets call it a "post-processor" for TestTemplateInvocationContext's.

Hope its understandable :)

Kind regards
Andreas

[kaipe]

LazyParams offers ways to apply ad-hoc parametrization "on top of all existing" ...
Below I have drafted class LogLevelExtension as a BeforeEachCallback implementation, which method beforeEach(...) achieves your desired parametrization regardless of whether your test-method is @ParameterizedTest, @RepeatedTest or just regular @Test.

import org.apache.log4j.Level;  //Using Log4j, because I didn't figure out which flavor ...
import org.apache.log4j.Logger;//... of Slf4j was used in the original extension example.

import org.lazyparams.LazyParams;
import org.lazyparams.showcase.ToPick;

public class LogLevelExtension  implements BeforeEachCallback {

  @Retention(RetentionPolicy.RUNTIME)
  @Target({ElementType.TYPE, ElementType.METHOD})
  public @interface LogLevel {
    String[] loggers();
    String level();
  }

  @Retention(RetentionPolicy.RUNTIME)
  @Target({ElementType.TYPE, ElementType.METHOD})
  @ExtendWith(LogLevelExtension.class)
  public @interface LogLevels {
    LogLevel[] value();
  }

  private LogLevels[] extractedLogLevels(final Method method) {
    final Class<?> parentClass = method.getDeclaringClass();
    final LogLevels[] classLevelAnnotations = parentClass.getAnnotationsByType(LogLevels.class);
    final List<LogLevels> cl = Arrays.asList(classLevelAnnotations);
//    log.debug("Found {} log level annotations '{}' for method '{}' class '{}'", cl.size(), cl, method.getName(), parentClass.getName());
    final LogLevels[] methodLevelAnnotations = method.getAnnotationsByType(LogLevels.class);
    final List<LogLevels> ml = Arrays.asList(methodLevelAnnotations);
//    log.debug("Found {} log level annotations '{}' for method '{}'", ml.size(), ml, method.getName());
    final Set<LogLevels> r = new LinkedHashSet<>();
    r.addAll(cl);
    r.addAll(ml);
//    log.debug("Found {} log level annotations '{}' for '{}'", r.size(), r, method.getName());
    return r.toArray(new LogLevels[0]);
  }

  @Override
  public void beforeEach(final ExtensionContext context) {
    final LogLevels[] levelsOptions = extractedLogLevels(context.getRequiredTestMethod());
    if (0 == levelsOptions.length) {
        return; /* No parameter values*/
    }

    final LogLevel logLevel = Stream.of(levelsOptions)
            .map(LogLevels::value).flatMap(Stream::of)
            .collect(ToPick.from())
            .asParameter(LogLevel::level)//Specify test display-name appendix!
            .pickValue();//Pick value and therewith enforce parametrization!

    final Level level = Level.toLevel(logLevel.level().toUpperCase());
    final Map<String, Level> originalLevels = new HashMap<>();

    for (final String loggerName : logLevel.loggers()) {
      final Logger logger = (Logger) Logger.getLogger(loggerName);
      originalLevels.put(loggerName, logger.getLevel());
      logger.setLevel(level);
    }

    /* Setup retirement-plan that restores original levels: */
    LazyParams.currentScopeConfiguration().setScopedCustomItem(
            new Object(), originalLevels, this::afterRestore);
  }

  private void afterRestore(Map<String,Level> originalLevels) {
    for (final Map.Entry<String, Level> entry : originalLevels.entrySet()) {
      final Logger logger = (Logger) Logger.getLogger(entry.getKey());
      logger.setLevel(entry.getValue());
    }
  }
}

E.g. if above FooData definition is ...

enum FooData {
  foo, fuu, fyy;
}

... then ConsoleLauncher could present this result for your test-class FooTest:

├─ JUnit Jupiter ✔
│  └─ FooTest ✔
│     └─ checkFoo(FooData) ✔
│        ├─ [1] data=foo ✔
│        │  ├─ [1] data=foo TRACE ✔
│        │  ├─ [1] data=foo DEBUG ✔
│        │  └─ [1] data=foo WARN ✔
│        ├─ [2] data=fuu ✔
│        │  ├─ [2] data=fuu TRACE ✔
│        │  ├─ [2] data=fuu DEBUG ✔
│        │  └─ [2] data=fuu WARN ✔
│        └─ [3] data=fyy ✔
│           ├─ [3] data=fyy TRACE ✔
│           ├─ [3] data=fyy DEBUG ✔
│           └─ [3] data=fyy WARN ✔

[marcphilipp]

Essentially, #871 will allow to do the same: have one level of parameterization on the class level and, potentially, another on the method level.