People using screen readers had no idea what these links were about (not
that the icons are very usable for people seeing them either... but
that's a different topic). Axe was reporting this error:
```
link-name: Links must have discernible text (serious)
https://dequeuniversity.com/rules/axe/4.10/link-name?application=axeAPI
The following 1 node violate this rule:
Selector: #dashboard_action_2_execute
HTML: <a id="dashboard_action_2_execute" class="unchecked-link"
rel="nofollow" data-method="post"
href="/proposals/16-proposal-6-title/dashboard/actions/2/execute">
<span class="unchecked"></span>
</a>
Fix all of the following:
- Element is in tab order and does not have accessible text
Fix any of the following:
- Element does not have text that is visible to screen readers
- aria-label attribute does not exist or is empty
- aria-labelledby attribute does not exist, references elements that
do not exist or references elements that are empty
- Element has no title attribute
```
All these tests were basically checking the same things. Since system
tests are slow, we're grouping them together so executing them is
slightly faster.
There were many cases where we were clicking on a link or (most of the
time) a button and then calling the `visit` method. In the past, it
worked just fine because clicking on buttons usually results in non-AJAX
requests, meaning that the test waited for the request to finish before
continuing.
That's no longer the case, though. In the last few months/years (not
sure since when) we're getting sporadic failures because the test
doesn't wait for the request to finish before making another request
with the `visit` method. This sometimes results in flaky tests.
Some of these tests have recently failed in our CI. Here are a few
examples (note the numbers don't follow an order because these tests
failed in different jobs):
```
1) Admin edit translatable records Current locale translation does not
exist For ActivePoll Shows first available fallback
Failure/Error: expect(page).to have_content "Sondage en Français"
expected to find text "Sondage en Français" in "Language: \n
\nEnglish\nDeutsch\nEspañol\nFrançais\nNederlands\nPortuguês
brasileiro\n中文\n Go back to CONSUL DEMOCRACY\nCONSUL
DEMOCRACY\nADMINISTRATION\nMenu\nNotifications\nMy content\nMy
account\nSign out\nProposals\nDebates\nComments\nPolls\n
Collaborative Legislation\nParticipatory budgets\nVoting booths
\nSignature Sheets\nMessages to users\nSite content\nModerated
content\nProfiles\nStatistics\nSettings\nProposals dashboard\n×
\nPolls description updated successfully.\nList of polls\nPolls
description\nCreate poll\nThere are no polls."
2) Public area translatable records Existing records Update a
translation With valid data Changes the existing translation
Failure/Error: expect(page).to have_field "Debate title",
with: "Title in English"
expected to find field "Debate title" that is not disabled but
there were no matches
2) Admin collaborative legislation Update Edit milestones summary
Failure/Error: expect(page).to have_content "There is still a long
journey ahead of us"
expected to find text "There is still a long journey ahead of us"
in "Language: \n
\nEnglish\nDeutsch\nEspañol\nFrançais\nNederlands\nPortuguês
brasileiro\n中文\n Go back to CONSUL DEMOCRACY\nCONSUL
DEMOCRACY\nADMINISTRATION\nMenu\nNotifications\nMy content\nMy
account\nSign out\nProposals\nDebates\nComments\nPolls\n
Collaborative Legislation\nParticipatory budgets\nVoting booths
\nSignature Sheets\nMessages to users\nSite content\nModerated
content\nProfiles\nStatistics\nSettings\nProposals dashboard\n×
\nProcess updated successfully. Click to visit\nBack\nAn example
legislation process\nInformation\nHomepage\nDebate\nProposals\n
Drafting\nFollowing\n1 language in use\nCurrent language\n
English\nSummary\n Format\n ◢\n Milestone\nManage progress
bars\nDon't have defined milestones\nCreate new milestone".
(However, it was found 1 time including non-visible text.)
3) Admin collaborative legislation SDG related list create Collaborative
Legislation with sdg related list
Failure/Error:
within("tr", text: "Legislation process with SDG related content") do
expect(page).to have_css "td", exact_text: "17"
end
Capybara::ElementNotFound:
Unable to find css "tr"
4) Valuation budget investments Valuate Feasibility can be marked as
pending
Failure/Error: expect(find("#budget_investment_feasibility_undecided"))
.not_to be_checked
Capybara::ElementNotFound:
Unable to find css "#budget_investment_feasibility_undecided"
3) Custom information texts Show custom texts instead of default ones
Failure/Error:
within("#section_help") do
expect(page).to have_content "Custom help with debates"
expect(page).not_to have_content "Help with debates"
end
4) Admin budgets Update Deselect all selected staff
Failure/Error: expect(page).to have_link "Select administrators"
expected to find link "Select administrators" but there were no
matches
3) Admin polls SDG related list edit poll with sdg related list
Failure/Error:
within("tr", text: "Upcoming poll with SDG related content") do
expect(page).to have_css "td", exact_text: "17"
end
Capybara::ElementNotFound:
Unable to find css "tr"
4) Admin polls SDG related list create poll with sdg related list
Failure/Error:
within("tr", text: "Upcoming poll with SDG related content") do
expect(page).to have_css "td", exact_text: "17"
end
Capybara::ElementNotFound:
Unable to find css "tr"
5) Admin custom images Image is replaced on admin newsletters
Failure/Error:
within(".newsletter-body-content") do
expect(page).to have_css("img[src*='logo_email_custom.png']")
end
Capybara::ElementNotFound:
Unable to find css ".newsletter-body-content"
6) Admin custom images Image is replaced on front views
Failure/Error:
within("#map") do
expect(page).to
have_css("img[src*='custom_map.jpg'][alt='Districts list']")
end
Capybara::ElementNotFound:
Unable to find css "#map"
```
As mentioned in commit 5311daadf, there are several reasons to use
buttons in these situations. And, as mentioned in the previous commits,
using buttons instead of links for actions requiring confirmation will
help us test for accessibility issues.
We had a trait called `:admin_request` for actions that are requests to
administrators, but the default factories were also requests to
administrators.
The tests checking that the "Request" button is not present, which
shouldn't pass with the wrong default factories, were passing by
coincidence. The issue was that we weren't checking whether that the
request had finished before checking that the "Request" button wasn't
present. That meant that we were checking that the "Request" button
wasn't there right at the moment we pressed the link, before the request
was finished.
So we're now checking that the request is finished before checking that
the button isn't there.
On the other hand, the tests checking for the "Request resource" link
being present were checking a behavior that's no longer there since
commit 9d85b3935, when we changed the conditions affecting that link.
This rule was added in rubocop-capybara 2.19.0. We were following it
about 85% of the time.
Now we won't have to check both have_css and have_selector when
searching the code.
Out of the usability issues I've experienced when using Consul
Democracy, the biggest one has arguably been the fact that the link to
edit a proposal opens in a new tab. I guess the reasoning behind it is
that the page to edit a proposal is not part of the proposals dashboard,
but what the hell! Imagine if every link to edit something opened in a
new tab...
So we're reducing the impact of this nonsense by opening most dashboard
links in the same window; for now, we're still opening in a new window
links to download files and links that might point to external websites.
We'll address those ones in the future.
Quoting usability experts Jakob Nielsen and Anna Kaley [1]:
> [Opening PDF files in new tabs] is problematic, because it assumes
> users will always do the exact same things with certain file formats,
> which isn’t always the case.
There are many examples of this situation. For example, some people
(myself included) configure their browser so it downloads PDF files
instead of opening them in the browser. In this situation, a new tab is
opened, a blank page is displayed, the file is downloaded, and then
either the tab is closed or the blank page needs to be manually closed.
The end result is really annoying.
Other situations include people who use a mobile phone browser, where
navigating through tabs is generally much harder than doing so on a
desktop browser.
But IMHO the most important point is: every browser already provides a
way to open "regular" links in a new tab, so people can choose what to
do, but if we decide to open the link in a new tab, we take control away
from them, and people who'd like to open the link in the same tab might
feel frustrated.
In these cases, the links either say "download" or include the word
"PDF", so people know in advance that they're going to download/open a
PDF file, and so we're giving them information and, by removing the
`target` attribute, we're giving them control over their browser so they
can choose what's convenient for them.
[1] https://www.nngroup.com/articles/new-browser-windows-and-tabs
We applied the Capybara/SpecificMatcher in commit f52a86b46. However,
this rule doesn't convert methods finding <a> tags to methods finding
links because <a> tags only count as links when they've got the `href`
attribute. For instance, in the `xss_spec.rb` file we check what happens
when clicking on an anchor tag because we're testing that the `href`
attribute has been removed and so we can't use `click_link`.
So, basically, we can't enable a rule to automatically detect when we're
using `have_css` instead of `have_link`, but we should still do it
because `have_link` adds an extra check which affects accessibility
since it makes sure the tag has the `href` attribute and so it's
recognizable as a link by screen readers.
For the HashAlignment rule, we're using the default `key` style (keys
are aligned and values aren't) instead of the `table` style (both keys
and values are aligned) because, even if we used both in the
application, we used the `key` style a lot more. Furthermore, the
`table` style looks strange in places where there are both very long and
very short keys and sometimes we weren't even consistent with the
`table` style, aligning some keys without aligning other keys.
Ideally we could align hashes to "either key or table", so developers
can decide whether keeping the symmetry of the code is worth it in a
case-per-case basis, but Rubocop doesn't allow this option.
We used "retire" because we translated it literally from the Spanish
verb "retirar" which can mean both "retire" and "withdraw".
Note we're still using "retire" in database fields and method names;
changing that might make it harder to upgrade from a previous version of
CONSUL.
We were updating the database after starting the browser to emulate the
behavior where a user logs in a day before the current request. We can
use `current_sign_in_at` instead and devise will automatically copy that
value to `last_sign_in_at` after users visit a page.
This way we avoid setting up the database after the process runnin the
browser has been started.
Just like we did in commit 0ec8878db, we remove the useless initial
request in the `before` filter since most tests started by visiting a
different URL.
We also reduce the risk of database inconsistency which comes with
setting up the database after the browser has been started.
The test was hanging sometimes on my machine, probably because we
weren't making sure the request submitting the form had finished before
visiting a new page.
In theory the spec should have been fine from a technical point of view:
since submitting the form generates a regular HTTP request (and not an
AJAX one), Capybara/Selenium/Chromedrive should wait until the request
is finished. But that doesn't seem to be the case 100% of the time;
maybe conditions change depending on previous tests.
On the other hand, from a design point of view, the spec wasn't that
fine. The main purpose of system specs is to test the way users interact
with our application, and users don't click a button and immediately
visit a different page. Instead, most users wait until they receive
feedback of their actions, and then they visit a different page.
Of course some users might visit another page without waiting. What
happens then cannot be predicted (it will depend on which request is
handled first), and so there's no point in writing a test for this case
unless there's a specific concurrency issue we'd like to check.
JavaScript is used by about 98% of web users, so by testing without it
enabled, we're only testing that the application works for a very
reduced number of users.
We proceeded this way in the past because CONSUL started using Rails 4.2
and truncating the database between JavaScript tests with database
cleaner, which made these tests terribly slow.
When we upgraded to Rails 5.1 and introduced system tests, we started
using database transactions in JavaScript tests, making these tests much
faster. So now we can use JavaScript tests everywhere without critically
slowing down our test suite.
Content like lowercase letters with `text-transform: uppercase` or
spaces after elements with `display: block` or "You're on page:" are not
seen that way by users with a browser supporting CSS.
So we're testing what most users actually experience.
Using separate tests to check every link on the page made the tests
slower. We were also adding a useless initial request on tests which
started by visiting a different URL.
This useless initial request meant in some tests the browser was started
before using factories to create data. Accessing the database in the
test after the browser starts might cause concurrency issues in
JavaScript tests.
Using `<a>` tags with no `href` means these elements cannot be activated
by keyboard users, so we're replacing them with buttons.
In the future we probably want to add more consistency so all toggle
buttons use the same code. We might also add styles depending on the
`aria-expanded` property.
We've had to add a couple of hacks in order to make jQuery UI datepicker
work with Turbolinks, and one of our tests is failing because the
datepicker changes its height when changing from a month with 5 weeks to
a month with 6 weeks.
We could add a workaround so the test still passes (jQuery UI doesn't
provide a configuration option to always displays 6 weeks in the
datepicker), but I think it's easier to just use the HTML5 native date
input field, which also allows us to simplify the code a bit and IMHO it
improves the user experience, particularly when using mobile phones.
Since date fields are not supported in Safari and Internet Explorer,
we're still using the jQuery UI datepicker on those browsers (and on any
other browser not supporting date fields).
Due to these changes, we're moving the tests checking datepicker's
behaviour to the dashboard. I've choosing not to change the public pages
because I'm not 100% sure everybody would like this change (some people
prefer the datepicker because we can configure the way it looks).
