Deal in with user creds under multiple instances of web browser

Hi! This post is going from backend-devs.com test automation team experience. We used that approach in a one of our projects.

It is not a secret, that some applications require authentication (You Don't Say?) to access them. So will we create 100500 accounts and sign in using Selenium WebDriver or HTTP requests using them?

Nope.

In our case application has paid subscription to access a website, and because we tested prod – spamming account creation was not possible.

We had 5 accounts (3, 10 – doesn’t matter, we asked for exactly 5 because we run the regression in 5 threads) and more than 1500 UI tests (and for each – we needed to sign in to start work).

How are credentials for testing usually stored in test framework? Constants in code, or in configuration files, usually in .json or .xlsx formats.

Storing users in constants:

Users

And then you probably do something like: loginPage.login(FIRST_USER); to login with user credentials.

The problem comes from this approach:

  • As I said before we had more than 1500 UI e2e (End-to-end) tests. Of course, we run them in parallel. And when it happens to be the same user in 2 different threads – it may lead to false negative results when testing intersecting functionality.

Example case: first thread is used for testing “Saved search” functionality (adding new “Saved Search” and then accessing it, second – testing “Remove Saved Search” functionality (adding “Saved Search” – verifying it was added and then removing it – second thread may remove wrong one “Saved Search”, since first thread also created saved search for the same user. As a result, both tests are failed.

Solutions:

  1. Slow: run regression in 1 thread. Stable, but very slow -> 16 hours for 1500 tests, instead of 5 hours in a parallel run (in 5 threads, time rate is inaccurate, since manipulating with 5 browser instances in one period of time slows machine a bit).
  2. Not applicable: use a different user account for each test. (Not our case, because every test needed user to start, and we very not able to create so many users for production).
  3. Hard to handle and manage: to keep an eye that users are not intercepting in functionality. (No comments)
  4. Our solution: Credentials Manager. The main idea is in getting a user from the user's pool by and blocking it from other threads by browser session ID (or by any other unique key you want – but it is simple to use Session ID for WebDriver). Session ID will be unique because we creating new browser instance for each test. I recommend to create new browser instance for each test rather than cleaning cookies and start new test in the same browser. (Because each test should be stand alone, independent from any other test or side-effects)

Getting Session ID:

/**
 * You can use any driver because all common drivers like 
 * ChromeDriver or FireFoxDriver
 * Are implementing RemoteWebDriver which has getSessionId() method
 */
String sessionId = ((RemoteWebDriver) driver).getSessionId().toString();

Then you need to lock your credentials, I recommend to do right after creating a new WebDriver instance. To lock credentials – use getCredentials(sessionId) method from Credentials Manager. Where 'T' is a generic type of our credentials.

public T getCredentials(String sessionId) {
    for (int lockedIndex = 0; lockedIndex < taken.length; lockedIndex++) {
        if (indexMap.get(sessionId) != null) {
            /* 
            returns the same credentials 
            (if already taken) for the same sessionId 
            */
            return credentials.get(indexMap.get(sessionId));
        } else if (!taken[lockedIndex]) {
            taken[lockedIndex] = true;
            indexMap.put(sessionId, lockedIndex);
            return credentials.get(lockedIndex);
        }
    }
    throw new NoSuchElementException("No free credentials found");
}

On test finish, you need to call freeCredentials(sessionId) method from Credentials Manager, for example before calling driver.quit(). Don't forget to call freeCredentials(sessionId) or credentials won't be unlocked for other threads after test run!

public void freeCredentials(String sessionId) {
    if (indexMap.get(sessionId) == null) {
        throw new NoSuchElementException(
        format("No locked credentials found for [%s] session", 
        sessionId));
    }
    taken[indexMap.get(sessionId)] = false;
    indexMap.remove(sessionId);
}

More examples how to use CredentialsManager can be available here: Credentials Manager Tests.