I use the before() functions with Cypress to organize my tests. But unfortunately, this runs flaky. By this, I mean that the tests fail the first time and then succeed the subsequent times. This is because the before runs more than once in the tests in describe().
Here is a print-screen of the fault:
Here is the way that I expect that is should work, When I press “Run all Tests again” it works always. I can conclude that this flakyness appears only the first time(when the session weren’t set) I run the test suit:
My test code is this:
describe('portaal', () => {
before(() => {
})
it('test1', () => {
});
it('test2, () => {
});
});
describe('portaal', () => {
before(() => {
})
it('test3', () => {
})
it('test4', () => {
});
});
And I have a e2e config file named e2e.ts which has the following:
before(() => {
const Api = Cypress.env().Api;
cy.ntlm(Api.hostUrl, Api.username, Api.password, Api.domain);
});
Is there a way to make this before work like it should be on the first run?
2
Here is a possible answer to start with, although I prefer to see more detail.
The cy.session() command is used to cache logon details. It should always be used in beforeEach() instead of before().
That is because the first call to cy.session does a real (live) login (when called before test1), but when called before test2 it does not actually do a logon, it just returns the credentials cached after the first call.
Here is the documentation reference session example.
The format for session command is cy.session(id, setup, options), and options can take a validate function, which is recommended when you have flaky tests.
This is the pattern your code should be using. I spread out the code to annotate, but normally you can compact it down.
beforeEach(() => {
cy.session(
'login', // session id
() => { // setup function, called only once
const Api = Cypress.env().Api;
cy.ntlm(Api.hostUrl, Api.username, Api.password, Api.domain);
},
{
validate: () => { // validate function, use to retry login when tests are flaky
cy.contains('a', 'View') // add check on the element that fails when flaky
}
}
})
// Always visit right after cy.session
cy.visit(...)
})
Since the “View” button is mentioned in the error message, it is a good candidate to check in cy.session() validate function, because it is not present on the page when login has failed.
When the validate function fails, the cy.session() knows it cannot trust the cached credentials, and it will re-run the setup function.
This is kind-of the automatic version of “Run all tests again” (expect it just runs the login again). This will ensure the code stays in the beforeEach() and repeats the login until it is successfully logged-in.