Since we changed the way we integrate coveralls in commit 8ed8cc8b9,
we're getting 6 additional checks displayed in our pull requests.
We don't need these checks, and they only add noise. The only reason we
use coveralls is to know the test coverage in our master branch.
So we're changing the code so coveralls only runs on the master branch.
There's also a chance that the test suite will be faster because it
doesn't need to keep track of the coverage, although I haven't noticed any
significant differences during my tests.
I haven't found a more elegant way to say that a certain step should
only be run on push on master, so I'm setting the environment variable
we were already using.
Coveralls stopped working back in July when we reached build number 3790
because back when we used Travis we created builds from numbers 3791 to
35700.
After trying to change the numbers in several ways, all of them
resulting in a "No build matching CI build number" message, we're trying
the Github Action provided by Coveralls instead. In order to make it
work, we need to add the `simplecov-lcov` gem and generate the report at
`coverage/lcov.info`. Note that, for some reason, `simplecov-lcov`
doesn't seem to depend on `simplecov` and we need `simplecov` 0.18 or
later, so we're manually adding this dependency as well.
Until now, when institutions made custom changes which made certain
tests obsolete, they didn't have a clear way to deal with the failing
tests. They would either:
1. Stop running the test suite
2. Run the test suite and get test failures 100% of the time
3. Comment the failing tests
4. Modify the failing tests in order to keep them in sync with the code
Solution 1 would be suicide from a maintenance perspective, although it
could work if only a couple of small custom changes were done.
Solution 2 would make it really hard to differenciate between "false
failures" and "real failures" when running the test suite.
Solution 3 would cause many conflicts when updating to a newer version
of Consul.
Solution 4 could make sense sometimes (big tests where only one line
needs to be changed), but it would also cause conflicts when updating
Consul.
So now, we're giving an alternative to solution 3 by making it easier to
exclude a test.
For tests that still need to be changed, when to use this solution
combined with a custom test and when to use solution 4 will have to be
decided on a per-case basis.
The next warning is flooding the tests log:
`Options passed to Node#set but the RackTest driver
doesn't support any - ignoring`
As the rack test driver does not support any option, we
get rid of all `default_set_options` before each system
spec that uses this driver.
There seems to be an issue with capybara or chromedriver which results
in `fill_in` sometimes appending to an input rather than overwriting
[1], causing some tests to fail under certain circumstances.
Clearing fields before filling them in solves the issue.
Note we're now getting warnings on all tests using the rack driver. I
haven't found a way to avoid the `clear: :backspace` option in
non-JavaScript tests, so to avoid the annoying warnings we should reduce
the number of tests using the rack driver even more.
[1] See issue 2419 in https://github.com/teamcapybara/capybara/issues
On JavaScript tests, Rails URL methods don't include the port when
invoked from a test, but they do when invoked from the browser. This was
causing some tests to fail with Selenium.
Somehow I thought it worked automatically, but we had to provide the
token.
The configuration is based on Coveralls instructions to run parallel
builds [1].
Alternatively we could use the Coveralls GitHub Action [2] which
slightly simplifies the workflow configuration and removes the
dependency of the coveralls gem. However, it also adds a dependency on
simplecov-lcov and requires configuring it to renerate LCOV files on
each run, so the benefits of using it are not that big.
[1] https://docs.coveralls.io/parallel-build-webhook
[2] https://github.com/coverallsapp/github-action/
We were doing it manually, but Capybara offers an option which does the
exact same thing.
This way we also apply the NoJavascriptTagHelper ERB rule, which
reported one error in the `disable_animations_in_tests` partial.
Using GitHub Actions has a few advantages over using Travis CI:
* More jobs can be run in parallel
* All CONSUL repositories on GitHub will be configured automatically
Besides, Travis have recently changed their policy twice. First, they
announced their site for free software projects would be shut down but
free software projects could still use their site for private projects.
And then, they limited the usage of their services for free software
projects.
Just like we used to do with Travis, we're enabling builds for pull
requests but not for pushed branches.
We're also building the master branch. Even if we never push to the
master branch directly, we're aware other CONSUL repositories do, so
we're running the tests for this case.
While Rails provides a lot of functionality by default, there's one
missing piece which is present in frameworks like Django or Phoenix: the
so-called "view models", or "components".
It isn't easy to extract methods in a standard Rails view/partial, since
extracting them to a helper will make them available to all views, and
so two helper methods can't have the same name. It's also hard to
organize the code in modules, and due to that it's hard to figure out
where a certain helper method is supposed to be called from.
Furthermore, object-oriented techniques like inheritance can't be
applied, and so in CONSUL customizing views is harder that customizing
models.
Components fix all these issues, and work the way Ruby objects usually
do.
Components are also a pattern whose popularity has increased a lot in
the last few years, with JavaScript frameworks like React using them
heavily. While React's components aren't exactly the same as the
components we're going to use, the concept is really similar.
I've always liked the idea of components. However, there wasn't a stable
gem we could safely use. The most popular gem (cells) hasn't been
maintained for years, and we have to be very careful choosing which gems
CONSUL should depend on.
The view_component gem is maintained by GitHub, which is as a guarantee
of future maintenance as it can be (not counting the Rails core team),
and its usage started growing after RailsConf 2019. While that's
certainly not a huge amount of time, it's not that we're using an
experimental gem either.
There's currently a conflict between view_component and wicked_pdf.
We're adding a monkey-patch with the fix until it's merged in
wicked_pdf.
All the code in the `bin/` and the `config/` folder has been generated
running `rake app:update`, except the `escape_javascript_fix` file,
which we've removed since the code there is already included in Rails
5.2.
With chromedriver >= 80, the tests are freezing sometimes, particularly
when the same editor is loaded again.
We don't know whether it's a CKEditor issue or a chromedriver issue. In
the past we've had some errors related to CKEditor trying to load the
same instance twice and we aren't sure they have been fixed since we
could never reproduce them.
It could be a coincidence, though. If we modify the views so the only
content of the `<body>` tag is a textarea with the `html-area` class,
chromedriver freezes even if we only access the page once. So maybe
we're only detecting the problem on the second visit because the second
request is faster than the first one.
Since chromedriver no longer hangs after this change, we don't have to
force any chromedriver version anymore.
The latest stable version is causing problems on some machines, hanging
forever in tests involving frames. So we're installing an old version
which works with the latest Chrome.
Note this means we're using an unsupported version. Officially, only the
latest chromedriver supports the latest Chrome.
We're using 2.38 instead of a more recent one (like 2.40) because it's
the one we specified in our Dockerfile.
See also:
https://bugs.chromium.org/p/chromedriver/issues/detail?id=3361
The images from OpenStreetMap take a while to load, sometimes even
causing Net::ReadTimeout errors if the internet connection is slow. It's
happened a lot recently on Travis builds.
Using capybara-webmock we guarantee the test suite doesn't fail due to
network issues.
We were having problems under certain conditions with Travis and
Knapsack where tasks were still being loaded twice and so they were
being executed twice.
Calling `load_tasks` in several files made rails load the tasks several
times, and so they were executed several times when called.
Since the milestone migration can't be executed twice in a row (it would
fail with duplicated ID records), loading the tasks several times made
the milestone migrations task specs fail.
We were seeing a log message when running specs locally
```
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.
```
As Coveralls is mainly used in Travis CI, we do not need to load it in
every spec run
