Java Unit Testing

Last modified by Vincent Massol on 2023/05/03 15:57

We are still converting some old unit tests written in JUnit3 or JUnit4 (and using JMock), to JUnit5 tests (using Mockito).

Characteristics

  • A unit test only tests a single class in isolation from other classes. Since in the XWiki project we write code using Components, this means a unit test is testing a Component in isolation from other Components.
  • These are tests performed in isolation using Mock Objects. More specifically we're using JUnit 5.x and Mockito
  • These tests must not interact with the environment (Database, Container, File System, etc) and thus do not need any environmental setup to execute
  • These tests must not output anything to stdout or stderr (or it'll fail the build). 
  • Your Maven module must depend on the xwiki-commons-tool-test-simple (for tests not testing Components) or xwiki-commons-tool-test-component (for tests testing Components) modules.

Writing Unit Tests

Mockito Mocks

You can mock java classes and interfaces using Mockito in several ways:

  • using mock(...). Example:
    @Test
    public void something()
    {
        VelocityEngine engine = mock(VelocityEngine.class);
       ...
    }
  • using the @Mock Mockito annotation. Example:
    @Mock
    private VelocityEngine engine;

    Note that for this example to work, you need to tell Mockito to inject mocks into @Mock annotations by using MockitoAnnotations.initMocks(testInstance); or you can use the @ComponentTest or @OldcoreTest annotations which automatically call MockitoAnnotations.initMocks (see below to know more about these 2 annotations).

  • You can also use the Mockito @InjectMocks annotation which will inject mocks fields automatically. For example; here's how you could combine both @InjectMocks and @InjectMockComponents:
    @InjectMocks
    @InjectMockComponents
    private Component4Impl component4;

    Where Component4Impl is defined as:

    @Component
    @Singleton
    public class Component4Impl implements Component4Role
    {
       private List<String> list;

       @Inject
       private Component1Role component1;
    ...

    In this example, the Component4Impl#list field will be injected with a mock.

@ComponentTest

If you're testing XWiki Components, annotate your test class with @ComponentTest.

This will allow you to have the following features.

Component Manager

A MockitoComponentManager instance will be created under the hood and initialized. It'll be initialized with no component by default. You'll be able to add components to it with one of the following options:

  • Annotate your test class with @AllComponents. This will automatically register all components found in the class path. For example:
    @ComponentTest
    @AllComponents
    public class DefaultConverterManagerTest
    {
    ...
    }
    • You can exclude some component implementations by using the excludes attribute of the @AllComponents annotation (Since 12.1RC1). For example:
      @AllComponents(excludes = {
          DefaultStringDocumentReferenceResolver.class,
          DefaultStringEntityReferenceSerializer.class,
          DefaultOfficeServer.class
      })
  • Annotate your test class with @ComponentList(...) to explicitly list the component implementation to register in the Component Manager. For example:
    @ComponentTest
    @ComponentList({
        ListFilter.class,
        ListItemFilter.class,
        FontFilter.class,
        BodyFilter.class,
        AttributeFilter.class,
        UniqueIdFilter.class,
        DefaultHTMLCleaner.class,
        LinkFilter.class
    })
    public class DefaultHTMLCleanerTest
    {
    ...
    }
  • Programmatically register Components or mock components. For example to register a mock component:
    Environment environment = mockitoComponentManager.registerMockComponent(Environment.class);
  • Using the @MockComponent annotation, see below

@MockComponent

Any test class field annotated with MockComponent will have a mock component created and registered in the ComponentManager.

For example:

@ComponentTest
public class DefaultVelocityContextFactoryTest
{
   @MockComponent
   private VelocityConfiguration configuration;
...
}

You can also specify a component hint using the @Named annotation. For example:

@ComponentTest
public class DefaultVelocityContextFactoryTest
{
   @MockComponent
   @Named("somehint")
   private VelocityConfiguration configuration;
...
}

@InjectMockComponents

Any test class field annotated with InjectMockComponents and referencing a component implementation will have its fields marked with @Inject be injected with mock component instances.

For example:

@ComponentTest
public class DefaultVelocityContextFactoryTest
{
   @InjectMockComponents
   private DefaultVelocityContextFactory factory;

Note that if the @MockComponent annotations are used, they'll be injected with mocks before the component annotated with @InjectMockComponents is instantiated and set up with mocks. 

For example, in the following case, the mocks for VelocityConfiguration and ComponentManager are created first and since DefaultVelocityContextFactory has corresponding fields with @Inject, they'll be injected with those mocks instead of new mocks being generated automatically.

@ComponentTest
public class DefaultVelocityContextFactoryTest
{
   @MockComponent
   private VelocityConfiguration configuration;

   @MockComponent
   private ComponentManager componentManager;

   @InjectMockComponents
   private DefaultVelocityContextFactory factory;
...

If your component implements several roles, you'll need to disambiguate that and specify which role to use. For example:

@InjectMockComponents(role = Component2Role.class)
private Component5Impl component5Role1;

Accessing the Component Manager

You have 2 ways to access it:

  • Either by annotating a field of the test class with @InjectComponentManager. For example:
    @InjectComponentManager
    private MockitoComponentManager componentManager;
  • Either by adding a parameter of type ComponentManagfer or MockitoComponentManager to the various JUnit5 test methods. For example:
    @BeforeEach
    public void before(MockitoComponentManager componentManager)
    {
       ....
    }

    @AfterEach
    public void after(MockitoComponentManager componentManager)
    {
       ....
    }

    @Test
    public void something(MockitoComponentManager componentManager)
    {
       ...
    }

@BeforeComponent / @AfterComponent

If you need to perform some initialization before the mock components are created/set up for the components annotate with @InjectMockComponents, create a method and annotate it with @BeforeComponent.

For example:

@ComponentTest
public class WikiUIExtensionComponentBuilderTest
{
...
   @InjectMockComponents
   private WikiUIExtensionComponentBuilder builder;

   @BeforeComponent
   public void configure() throws Exception
   {
       ...
   }
...
}

Since 12.2 And if you need to test some component that implements Initializable and you need different test fixtures depending on the test, you'll need to use BeforeComponent("<testname>"), such as:

@BeforeComponent("normalizeFromRootServletContextAndServletMapping")
public void beforeNormalizeFromRootServletContextAndServletMapping()
{
 // Code executed before Initializable#initialize() is called and
 // after methods annotated with @BeforeComponent have been called
}

@Test
public void normalizeFromRootServletContextAndServletMapping()
{
  ...
}

If you need access to the Component Manager, you have 2 solutions:

  • Use a field annotated with @InjectComponentManager as described above
  • Or add a parameter of type ComponentManager or MockitoComponentManager to the @BeforeComponent-annotated method. For example:
    @ComponentTest
    public class WikiUIExtensionComponentBuilderTest
    {
    ...
       @InjectMockComponents
       private WikiUIExtensionComponentBuilder builder;

       @BeforeComponent
       public void configure(MockitoComponentManager componentManager) throws Exception
       {
           ...
       }
    ...
    }

You can use @AfterComponent to perform some setup after the mock components have been setup.

Order

It's important to understand the order used to resolve all those annotations:

  1. Create an empty Mockito Component Manager
  2. Inject @InjectComponentManager fields
  3. Inject @MockComponent fields
  4. Call @BeforeComponent methods
  5. Fill the Mockito Component Manager (with components matching the @AllComponent and @ComponentList annotations)
  6. Call @AfterComponent methods
  7. Inject @InjectMockComponents fields
  8. Inject @Inject fields (this is just a shortcut to using @InjectComponentManager and then calling getInstance() on it)
  9. Inject Mockito annotations (@Mock, @InjectMocks, etc)

Providers

If the component being tested is injecting Providers then the test framework will automatically register a mock Component of the provided type, as a singleton component. For example:

// Component under test
...
@Inject
@Named("hint")
private Provider<MyComponent> myComponentProvider;
...

// Test
@ComponentTest
class MyTest
{
...
   @InjectComponentManager
   private MockitoComponentManager componentManager;
...
@Test
void test()
{
    MyComponent myCompnent = this.componentManager.getInstance(MyComponent.class, "hint");
   ...
}

However if the provided component is not a singleton, you'll need to register a Provider yourself to override this default behavior. For example:

// Component under test
...
// Note: DatabaseMailListener is using: @InstantiationStrategy(ComponentInstantiationStrategy.PER_LOOKUP)
@Inject
@Named("database")
private Provider<MailListener> databaseMailListenerProvider;
...
// Test
@BeforeComponent("resendSynchronouslySeveralMessages")
void setupResendSynchronouslySeveralMessages() throws Exception
{
   this.componentManager.registerMockComponent(
       new DefaultParameterizedType(null, Provider.class, MailListener.class), "database");
}

@Test
void resendSynchronouslySeveralMessages() throws Exception
{
   ...
    Provider<MailListener> databaseMailListenerProvider = this.componentManager.getInstance(
       new DefaultParameterizedType(null, Provider.class, MailListener.class), "database");
    DatabaseMailListener databaseMailListener1 = mock(DatabaseMailListener.class);
    DatabaseMailListener databaseMailListener2 = mock(DatabaseMailListener.class);
    when(databaseMailListenerProvider.get()).thenReturn(databaseMailListener1, databaseMailListener2);
   ...
}

@Inject

Any field annotated with @Inject (and optionally with @Named) will get injected with an implementation from the Component Manager if one exists (if not, an exception will be raised).

Example:

@ComponentTest
@AllComponents
class SomeTest
{
   @Inject
   private SomeComponentRole someComponent;
...
}

Capturing and Asserting Logs

If the code under tests output logs, they need to be captured and possibly asserted. For example:

...
/**
 * Capture logs.
 */

@RegisterExtension
private LogCaptureExtension logCapture = new LogCaptureExtension(LogLevel.WARN);
...
assertEquals("Error getting resource [bad resource] because of invalid path format. Reason: [invalid url]",
    logCapture.getMessage(0));
...

Since 12.1RC1 If we have some code that emits logs during a @BeforeAll (for example in some component initializable() method), you should register the LogCaptureExtension statically as in:

@RegisterExtension
private static LogCaptureExtension logCapture = new LogCaptureExtension(LogLevel.INFO);

And perform the verification in a @AfterAll method, as in:

@AfterAll
static void verifyLog() throws Exception
{
 // Assert log happening in the first initialize() call.
 assertEquals(String.format("Using filesystem store directory [%s]",
     new File(new File("."), "store/file").getCanonicalFile()), logCapture.getMessage(0));
}

@OldcoreTest

If you're writing a unit test for some old core code, you should use the @OldcoreTest annotation instead of the @ComponentTest one described above. Tests annotated with @OldcoreTest will behave the same as with @ComponentTest but, in addition, it'll automatically create and configure a helper MockitoOldcore object that you can use to help you write your unit test.

Accessing MockitoOldcore

You have several ways to access it:

  • Either by annotating a field of the test class with @InjectMockitoOldcore. For example:
    @InjectMockitoOldcore
    private MockitoOldcore oldCore;
  • Either by adding a parameter of type MockitoOldcore to the various JUnit5 test methods. For example:
    @BeforeEach
    public void before(MockitoOldcore oldcore)
    {
       ....
    }

    @AfterEach
    public void after(MockitoOldcore oldcore)
    {
       ....
    }

    @Test
    public void something(MockitoOldcore oldcore)
    {
       ...
    }

    Note that this can be combined with the MockitoComponentManager parameter type too, as in:

    @Test
    public void something(MockitoComponentManager componentManager, MockitoOldcore oldcore)
    {
       ...
    }
  • You can also pass a MockitoOldcore parameter to methods annotated with @BeforeComponent and @AfterComponent. For example:
    @BeforeComponent
    public void configure(MockitoComponentManager componentManager, MockitoOldcore oldcore) throws Exception
    {
       ...
    }

Temporary Directory

From time to time, tests require a temporary directory. We must not use the system's temporary directory nor use JUnit5's @TempDir annotation (it also uses the system's temporary directory) for the following reasons:

  • Leftover data in a shared location once the test has finished
  • Creates a state that can make other tests fail
  • Generate errors in Jenkins since Jenkins monitors created files and doesn't allow to remove files outside of the Worskspace

Thus the best practice is to use the XWikiTempDir annotation as in the following examples:

@ComponentTest // or @ExtendWith(XWikiTempDirExtension.class)
public class MyTest
{
 ...
 // New tmp dir once per test class
 @XWikiTempDir
 private static File TEST_DIR;

 // New tmp dir once per test
 @XWikiTempDir
 private File tmpDir;

 // New tmp dir for this specific test
 @Test
 public void testXXX(@XWikiTempDir File tmpDir)
 {
     ...
 }
...

The XWikiTempDir annotation will create a unique temporary directory inside Maven's target directory.

Testing a JUnit5 Extension

If you need to test a JUnit5 extension, user the following example as a basis for your test. It shows how to start a JUInit5 engine from inside a JUnit5 test emoticon_wink

public class LogCaptureExtensionTest
{
   // Test class being tested
   public static class SampleTestCase
   {
       private static final Logger LOGGER = LoggerFactory.getLogger(SampleTestCase.class);

       @RegisterExtension
       private LogCaptureExtension logCapture = new LogCaptureExtension(LogLevel.WARN);

       @Test
       @Order(1)
       void testWhenNotCaptured()
       {
            LOGGER.warn("uncaptured warn");
       }

       @Test
       @Order(2)
       void testWhenCaptured()
       {
            LOGGER.error("captured error");
            assertEquals("captured error", logCapture.getMessage(0));
       }

       @Test
       @Order(3)
       void testWhenLoggerLevelNotCaptured()
       {
            LOGGER.info(("info ignored and not captured"));
       }
   }

   @Test
   void captureLogsWhenNotStatic()
   {
        TestExecutionSummary summary = executeJUnit(SampleTestCase.class);

       // Verify that uncaptured logs are caught and generate an exception
       assertEquals(1, summary.getFailures().size());
        assertEquals("Following messages must be asserted: [uncaptured warn]",
            summary.getFailures().get(0).getException().getMessage());
        assertEquals(2, summary.getTestsSucceededCount());
   }

   private TestExecutionSummary executeJUnit(Class<?> testClass)
   {
        LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
           .selectors(selectClass(testClass))
           .build();
        Launcher launcher = LauncherFactory.create();
        SummaryGeneratingListener summaryListener = new SummaryGeneratingListener();
        launcher.execute(request, summaryListener);

       return summaryListener.getSummary();
   }
...

Examples

Best practices

  • Name the Test class with the name of the class under test suffixed with Test. For example the JUnit test class for XWikiMessageTool should be named XWikiMessageToolTest
  • Have each test method test only 1 use case as much as possible.
  • Name the test methods with the method to test followed by a qualifier describing the test. For example importWithHeterogeneousEncodings(). In the few cases where it's not applicable, name after the use case being tested.
  • Don't use the public keyword since they're not required any more by JUnit5 (not in the Test Class nor in the test methods nor in the lifecycle methods such as BeforeEach).
  • When you're testing for an exception, use the following strategy, shown on an example:
    Throwable exception = assertThrows(IllegalArgumentException.class, () -> {
       // Content throwing exception here
       this.cssIdentifierSerializer.serialize(input);
    });
    assertEquals("Invalid character: the input contains U+0000.", exception.getMessage());
  • We should use Hamcrest assertThat() method when it makes sense (when there's no good JUnit 5.x assertion alternative or without a good reporting when the assertion fails. For example for asserting that a list contains a given entry).

Tips

  • When Mocking, to ignore all debug logging calls and to allow all calls to is*Enabled you could write:
     // Ignore all calls to debug() and enable all logs so that we can assert info(), warn() and error() calls.
    ignoring(any(Logger.class)).method("debug");
    allowing(any(Logger.class)).method("is.*Enabled"); will(returnValue(true));

Using JUnit 4

This is now deprecated but kept for the moment since there are still a lot of JUnit4 tests.

See the guide for writing JUnit4 tests.

Tags:
   

Get Connected