There were many cases where we were clicking on a link or (most of the
time) a button and then calling the `visit` method. In the past, it
worked just fine because clicking on buttons usually results in non-AJAX
requests, meaning that the test waited for the request to finish before
continuing.
That's no longer the case, though. In the last few months/years (not
sure since when) we're getting sporadic failures because the test
doesn't wait for the request to finish before making another request
with the `visit` method. This sometimes results in flaky tests.
Some of these tests have recently failed in our CI. Here are a few
examples (note the numbers don't follow an order because these tests
failed in different jobs):
```
1) Admin edit translatable records Current locale translation does not
exist For ActivePoll Shows first available fallback
Failure/Error: expect(page).to have_content "Sondage en Français"
expected to find text "Sondage en Français" in "Language: \n
\nEnglish\nDeutsch\nEspañol\nFrançais\nNederlands\nPortuguês
brasileiro\n中文\n Go back to CONSUL DEMOCRACY\nCONSUL
DEMOCRACY\nADMINISTRATION\nMenu\nNotifications\nMy content\nMy
account\nSign out\nProposals\nDebates\nComments\nPolls\n
Collaborative Legislation\nParticipatory budgets\nVoting booths
\nSignature Sheets\nMessages to users\nSite content\nModerated
content\nProfiles\nStatistics\nSettings\nProposals dashboard\n×
\nPolls description updated successfully.\nList of polls\nPolls
description\nCreate poll\nThere are no polls."
2) Public area translatable records Existing records Update a
translation With valid data Changes the existing translation
Failure/Error: expect(page).to have_field "Debate title",
with: "Title in English"
expected to find field "Debate title" that is not disabled but
there were no matches
2) Admin collaborative legislation Update Edit milestones summary
Failure/Error: expect(page).to have_content "There is still a long
journey ahead of us"
expected to find text "There is still a long journey ahead of us"
in "Language: \n
\nEnglish\nDeutsch\nEspañol\nFrançais\nNederlands\nPortuguês
brasileiro\n中文\n Go back to CONSUL DEMOCRACY\nCONSUL
DEMOCRACY\nADMINISTRATION\nMenu\nNotifications\nMy content\nMy
account\nSign out\nProposals\nDebates\nComments\nPolls\n
Collaborative Legislation\nParticipatory budgets\nVoting booths
\nSignature Sheets\nMessages to users\nSite content\nModerated
content\nProfiles\nStatistics\nSettings\nProposals dashboard\n×
\nProcess updated successfully. Click to visit\nBack\nAn example
legislation process\nInformation\nHomepage\nDebate\nProposals\n
Drafting\nFollowing\n1 language in use\nCurrent language\n
English\nSummary\n Format\n ◢\n Milestone\nManage progress
bars\nDon't have defined milestones\nCreate new milestone".
(However, it was found 1 time including non-visible text.)
3) Admin collaborative legislation SDG related list create Collaborative
Legislation with sdg related list
Failure/Error:
within("tr", text: "Legislation process with SDG related content") do
expect(page).to have_css "td", exact_text: "17"
end
Capybara::ElementNotFound:
Unable to find css "tr"
4) Valuation budget investments Valuate Feasibility can be marked as
pending
Failure/Error: expect(find("#budget_investment_feasibility_undecided"))
.not_to be_checked
Capybara::ElementNotFound:
Unable to find css "#budget_investment_feasibility_undecided"
3) Custom information texts Show custom texts instead of default ones
Failure/Error:
within("#section_help") do
expect(page).to have_content "Custom help with debates"
expect(page).not_to have_content "Help with debates"
end
4) Admin budgets Update Deselect all selected staff
Failure/Error: expect(page).to have_link "Select administrators"
expected to find link "Select administrators" but there were no
matches
3) Admin polls SDG related list edit poll with sdg related list
Failure/Error:
within("tr", text: "Upcoming poll with SDG related content") do
expect(page).to have_css "td", exact_text: "17"
end
Capybara::ElementNotFound:
Unable to find css "tr"
4) Admin polls SDG related list create poll with sdg related list
Failure/Error:
within("tr", text: "Upcoming poll with SDG related content") do
expect(page).to have_css "td", exact_text: "17"
end
Capybara::ElementNotFound:
Unable to find css "tr"
5) Admin custom images Image is replaced on admin newsletters
Failure/Error:
within(".newsletter-body-content") do
expect(page).to have_css("img[src*='logo_email_custom.png']")
end
Capybara::ElementNotFound:
Unable to find css ".newsletter-body-content"
6) Admin custom images Image is replaced on front views
Failure/Error:
within("#map") do
expect(page).to
have_css("img[src*='custom_map.jpg'][alt='Districts list']")
end
Capybara::ElementNotFound:
Unable to find css "#map"
```
This rule was added in rubocop-capybara 2.19.0. We were following it
about 85% of the time.
Now we won't have to check both have_css and have_selector when
searching the code.
We were getting a warning with Ruby 2.7:
```
ruby/gems/2.7.0/gems/capybara-3.37.1/lib/capybara/session.rb:377:
warning: Using the last argument as keyword parameters is deprecated;
maybe ** should be added to the call
```
On Ruby 3.0, the test failed with `Unable to find fieldset
{:text=>"Draft phase"}` and we were also getting another warning:
```
Locator Hash:{:text=>"Draft phase"} for selector :fieldset must be an
instance of String or Symbol. This will raise an error in a future
version of Capybara
```
This way we don't have to write `"spec/fixtures/files"` every time.
Note this method isn't included in factories. We could include it like
so:
```
FactoryBot::SyntaxRunner.class_eval do
include ActiveSupport::Testing::FileFixtures
self.file_fixture_path = RSpec.configuration.file_fixture_path
end
```
However, I'm not sure about the possible side effects, and since we only
use attachments in a few factories, there isn't much gain in applying
the monkey-patch.
When configuring phases in a process, we were validating the start date
or the end date is present, the other date is present too.
However, in other parts of the application we were checking whether a
phase is enabled and then assumed its dates were present if the phase
was enabled. However, we weren't validating this behavior, so it was
possible to enable a phase and leaving its dates blank, causing the
application to crash.
So, as suggested by Alberto, we're changing the validation rule so
phase dates are mandatory when a phase is enabled.
With this rule, the old validation rules are not necessary. I've
considered leaving them in order to avoid database inconsistencies.
However, I realized records having a disabled phase with its start and
end dates have always been valid. This means applications checking for
the presence of these dates instead of checking whether the phase is
enabled have never worked properly.
We don't have to change the logic anywhere else because as mentioned we
were already checking phases are enabled before using their dates.
Users don't care about database content; they care about what they see
on the screen.
Writing tests this way we also avoid potencial database inconsistencies
due to accessing the database after starting the browser.
JavaScript is used by about 98% of web users, so by testing without it
enabled, we're only testing that the application works for a very
reduced number of users.
We proceeded this way in the past because CONSUL started using Rails 4.2
and truncating the database between JavaScript tests with database
cleaner, which made these tests terribly slow.
When we upgraded to Rails 5.1 and introduced system tests, we started
using database transactions in JavaScript tests, making these tests much
faster. So now we can use JavaScript tests everywhere without critically
slowing down our test suite.
System tests are used to test the application from the user's point of
view. To test for specific exceptions, particularly regarding
authorization permissions, controller tests fit better.
Another option would be to test the page displayed shows a certain text,
like "Internal server error". I'm choosing controller tests because
they're faster and we're basically testing the same scenario many times
and we've already got a test checking what happens when users access a
page raising an exception.
Tests are easier to read now. Besides, since we changed the inputs in
the admin section so they don't use jQuery but an HTML date field,
formatting with %d/%m/%Y might not work depending on the browser's
locale.
We were repeating the same code over and over (with a few variants) to
setup tests which require an administrator. We can use a tag and
simplify the code.
In some tables, we had "actions", and some columns were also links
pointing to some places. Having both of them at the same time is
confusing, particularly since traditionally the links in the columns
pointed to the same place as some of the actions (although that's not
the case since commit 48db31cd).
We're still keeping links in tables which don't have an action column.
For instance, the proposals table has a "select" button which would be
harder to use if we had action buttons next to it.
We've had to add a couple of hacks in order to make jQuery UI datepicker
work with Turbolinks, and one of our tests is failing because the
datepicker changes its height when changing from a month with 5 weeks to
a month with 6 weeks.
We could add a workaround so the test still passes (jQuery UI doesn't
provide a configuration option to always displays 6 weeks in the
datepicker), but I think it's easier to just use the HTML5 native date
input field, which also allows us to simplify the code a bit and IMHO it
improves the user experience, particularly when using mobile phones.
Since date fields are not supported in Safari and Internet Explorer,
we're still using the jQuery UI datepicker on those browsers (and on any
other browser not supporting date fields).
Due to these changes, we're moving the tests checking datepicker's
behaviour to the dashboard. I've choosing not to change the public pages
because I'm not 100% sure everybody would like this change (some people
prefer the datepicker because we can configure the way it looks).
The JavaScript involved wasn't working since we removed the disable-date
attribute in commit 73ff6881.
We're also improving the JavaScript in two ways:
First, we trigger the `change` event immediately, so when the page loads
date fields are disabled when phases are disabled.
And second, we don't remove the selected dates when disabling a phase,
so disabling it and enabling it again will keep the selected values.
