We're still using YAML to serialize the legislation_annotations ranges
column. I'm not sure whether changing the serializer can have
consequences on existing data, and I'm not sure which serializer we
should provide instead. Quoting the Rails configuration guide [1]:
> Unfortunately there isn't really any suitable defaults available in
> Ruby's standard library. JSON could work as a format, but the json
> gems will cast unsupported types to strings which may lead to bugs.
[1] https://guides.rubyonrails.org/v7.1/configuring.html#config-active-record-default-column-serializer
Just like we mentioned in commit 001eee3d6, according to the Rails
configuration guide [1], with this format, Rails serializes cache
entries more efficiently. Most importantly:
> All formats are backward and forward compatible, meaning cache entries
> written in one format can be read when using another format. This
> behavior makes it easy to migrate between formats without invalidating
> the entire cache.
[1] https://guides.rubyonrails.org/v7.1/configuring.html#config-active-support-cache-format-version
Quoting the Rails configuration guide [1]:
> applications running in :zeitwerk mode do not need require_dependency,
> so models, controllers, jobs, etc. do not need to be in $LOAD_PATH.
> Setting this to false saves Ruby from checking these directories when
> resolving require calls with relative paths, and saves Bootsnap work
> and RAM, since it does not need to build an index for them.
[1] https://guides.rubyonrails.org/v7.1/configuring.html#config-add-autoload-paths-to-load-path
We were getting a ton of deprecation warnings:
```
DEPRECATION WARNING: `Rails.application.secrets` is deprecated in favor
of `Rails.application.credentials` and will be removed in Rails 7.2
```
However, we don't plan to replace the secrets with credentials in the
foreseeable future because it would affect existing Consul Democracy
installations, so for now we're simply silencing the warnings.
We're reworking the format validation to correctly interpret feature
collection, feature, and geometry, according to RFC 7946 [1].
Since Leaflet interprets GeoJSON format, we're rendering the GeoJSON as
a layer instead of as a set of points. For that, we're normalizing the
GeoJSON to make sure it contains either a Feature or a
FeatureCollection. We're also adding the Leaflet images to the assets
path so the markers used for point geometries are rendered correctly.
Note we no longer allow a GeoJSON containing a geometry but not a
defined type. Since there might be invalid GeoJSON in existing Consul
Democracy databases, we're normalizing these existing geometry objects
to be part of a feature object.
We're also wrapping the outline points in a FeatureCollection object
because most of the large GIS systems eg ArcGIS, QGIS export geojson as
a complete FeatureCollection.
[1] https://datatracker.ietf.org/doc/html/rfc7946
Co-authored-by: Javi Martín <javim@elretirao.net>
We only want to render the account link and login items in the header.
And we want only render the Multitenancy and Administrators sections in
the admin sidebar.
We include the administrators management so it's possible to give
permissions to other users to manage tenants.
In order to restrict access to other sections by typing the URL or
following a link, we're only enabling the rest of the routes when we
aren't in the multitenancy management mode.
When we upgraded to Rails 7 in commit 8596f1539, we broke the custom
locales since now all nested folders in `config/locales/` are loaded by
default [1]. This meant that the custom folder was now loaded before any
languages whose code alphabetically goes after the word "custom".
As a workaround, we're overwriting the default locales paths so they
don't include the custom folder. We're doing this step before the
`add_locales` initializer; that is, before the default locales paths are
used.
Unfortunately, I haven't found a way to add a test for this behavior,
since we would need to add a file in `config/locales/custom` that
overwrites an internationalization key for an existing language, but
only during a specific test, and the i18n load path is evaluated when
the application starts up.
[1] See pull request 41872 in https://github.com/rails/rails/
We were defining one builder in the `app/lib/` folder and another one
inside a helper module.
So now we're grouping them together. This way we're following the "one
class per file" convention that we follow most of the time. And, by
extracting the `TranslatableFormBuilder` class to its own file, it'll be
easier to add tests for it.
Note that, for consistency, we're renaming the
`TranslationsFieldsBuilder` class so it ends in `FormBuilder`.
We're keeping the old `apply_stylesheet_media_default` option behavior
because removing `media="screen"` from our stylesheets would completely
break our `print` stylesheet, which would now load the default the
styles defined in `application.css`.
We're also keeping the old `:mini_magick` option to process images so
existing installations don't have to install libvips on their server. We
might change it in the future.
This option was deprecated in Rails 7.0 and removed in Rails 7.1 [1]. It
doesn't really affect us because we weren't using `to_s` with a
parameter anywhere in the application.
The Rubocop rule Rails/ToSWithArgument can be used to detect these cases
but, since we've never used them, and adding them now would cause the
application to crash and so it'll be obvious we've done something wrong,
I don't think it's necessary to add the rule.
[1] https://github.com/rails/rails/commit/e420c3380
According to the Rails configuration guide [1], with this format, Rails
serializes cache entries more efficiently. Most importantly:
> All formats are backward and forward compatible, meaning cache entries
> written in one format can be read when using another format. This
> behavior makes it easy to migrate between formats without invalidating
> the entire cache.
[1] https://guides.rubyonrails.org/v7.1/configuring.html#config-active-support-cache-format-version
Note that enabling this options means all encrypted messages and cookies
generated the application become invalid, so we're adding a cookie
rotator in order to keep sessions from expiring when upgrading the
application, as recommended in the "Upgrading Ruby on Rails" guideline
[1].
Since we haven't seen any Consul Democracy applications using encrypted
messages and these messages become invalid with this change, we're also
removing the pre-Rails 5.2 encryption to authenticate messages
(AES-256-CBC) and switching to the default one since Rails 5.2
(AES-256-GCM). Since the configured encryption is used by the cookie
rotator initializer (through the ActiveSupport::MessageEncryptor.key_len
method), at first I thought this might affect the cookie rotator, but it
doesn't: upgrading works as expected, and existing sessions are still
active.
I'm adding a comment to remove the initializer once all cookies have
been migrated. I've added "Rails 7.1" in the comment because we usually
check for these comments when upgrading Rails, but we rarely check for
them when after releasing new versions of Consul Democracy.
[1] https://guides.rubyonrails.org/v7.0/upgrading_ruby_on_rails.html#key-generator-digest-class-changing-to-use-sha256
The config.file_watcher option still exists but it's no longer included
in the default environtment file. Since we don't use it, we're removing
it.
The config.assets.assets.debug option is no longer true by default [1],
so it isn't included anymore.
The config.active_support.deprecation option is now omitted on
production in favor of config.active_support.report_deprecations, which
is false by default. I think it's OK to keep it this way, since we check
deprecations in the development and test environments but never on
production environments.
As mentioned in the Rails upgrade guide, sprockets-rails is no longer a
rails dependency and we need to explicitly include it in our Gemfile.
The behavior of queries trying to find an invalid enum value has changed
[2], so we're updating the tests accordingly.
The `favicon_link_tag` method has removed the deprecated `shortcut`
link type [3], so we're updating the tests accordingly.
The method `raw_filter` in ActiveSupport callbacks has been renamed to
`filter` [4], so we're updating the code accordingly.
[1] https://github.com/rails/rails/commit/adec7e7ba87e3
[2] https://github.com/rails/rails/commit/b68f0954
[3] Pull request 43850 in https://github.com/rails/rails
[4] Pull request 41598 in https://github.com/rails/rails
In Rails 6.1, the classic autoloader is deprecated.
We were getting an error because we were using `autoload` in the
ActiveStorage plugin for CKEditor:
expected file app/lib/ckeditor/backend/active_storage.rb to define
constant Ckeditor::Backend::ActiveStorage
So we're removing the line causing the error.
Finally, we can now restore all the tests that that failed sometimes
with the classic autoloader and that we modified in commits 2af1fc72f
and 8ba37b295.
The purpose of the lib folder is to have code that doesn't necessary
belong in the application but can be shared with other applications.
However, we don't have other applications and, if we did, the way to
share code between them would be using a gem or even a git submodule.
So having both the `app/` and the `lib/` folders is confusing IMHO, and
it causes unnecessary problems with autoloading.
So we're moving the `lib/` folder to `app/lib/`. Originally, some of
these files were in the `app/services/` folder and then they were moved
to the `lib/` folder. We're using `app/lib/` instead of `app/services/`
so the upgrade is less confusing.
There's an exception, though. The `OmniAuth::Strategies::Wordpress`
class needs to be available in the Devise initializer. Since this is an
initializer and trying to autoload a class here will be problematic when
switching to Zeitwerk, we'll keep the `require` clause on top of the
Devise initializer in order to load the file and so it will be loaded
even if it isn't in the autoload paths anymore.
SassC/Libsass has been deprecated for years and has been replaced by
Dart Sass. However, the dartsass-rails gem, maintained by the Rails
team, doesn't support sprockets integration and doesn't allow glob
imports (using `@import something/**/*` or similar). In particular,
dartsass-rails needs to start a separate browser that makes it less
straightforward to change a file and reload the browser.
So we're using sassc-embedded, which provides Dart Sass integration with
sprockets. While there's no guarantee this gem will be maintained a few
years from now, we know for sure that SassC/Libsass won't be maintained
at all, so using sassc-embedded is an improvement over our current
situation.
On my machine, this change reduces compilation times by about 35%.
Note we still depend on the `sassc-rails` gem, for two reasons.
First, we're still importing CSS/Sass content from a couple of gems
(mainly, social-share-button and font-awesome) and we don't know how to
import this content without the `sassc-rails` gem.
And, second, it provides support for glob imports. Without it, we'd have
to manually add every single (S)CSS file we import to the
`application.scss` file instead of being able to write things like
`@import admin/**/*";`.
Note we're removing the `sass` gem from `Gemfile.lock`. We should have
done it as part of e210682ac, but when we developed that branch, it
didn't contain the changes where we removed another gem depending on the
`sass` gem (which we removed in commit 2fa713c64), so Bundler didn't
delete it. However, now that we're changing the Gemfile, Bundler is
finally removing the no-longer-needed `sass` gem and its dependencies.
This will allow us to keep using the timezone used when running
the installer.
We keep using Madrid as default timezone in the installer and in
the application.
As this secret is currently not programmed to be customisable for each Tenant,
we take it out of the security section. The reason is that so far everything inside this security section can be overwritten per tenant. With this change we are trying
to prevent anyone from trying to overwrite it on a per Tenant basis, as it would have
no effect.
Other packages depend on jQuery, so that's why these are the first one
we move from the Gemfile to the package.json file.
This way we can also test whether dependabot correctly opens pull
requests to update Node packages.
I've tried several configuration options for the asset pipeline in order
to be able to include images referenced in jQuery UI CSS files. So far,
adding the `node_modules/jquery-ui/themes/base` folder to the assets
paths is the only way I've found to make it work. Hopefully we can find
a better solution in the future so we don't have to study the internals
of every Node package in order to integrate it with the assets pipeline.
Note that, even if we're excluding the `node_modules/` folder from
version control, we aren't adding it to Capistrano's shared folders
because, when `node_modules` is a symbolic link, NPM removes it when
running `npm install`.
We can remove the `new_framework_defaults_6_1` file by using Rails 6.1
default options and overwriting the one we haven't enabled.
We've experienced problems while running the tests (probably the same
would happen on production) when enabling the `has_many_inversing`
option. For example, after creating a legislation answer for a question
with no answers, calling `question.answers_count` would then return `2`
instead of `1`.
So we aren't enabling this option.
For the HashAlignment rule, we're using the default `key` style (keys
are aligned and values aren't) instead of the `table` style (both keys
and values are aligned) because, even if we used both in the
application, we used the `key` style a lot more. Furthermore, the
`table` style looks strange in places where there are both very long and
very short keys and sometimes we weren't even consistent with the
`table` style, aligning some keys without aligning other keys.
Ideally we could align hashes to "either key or table", so developers
can decide whether keeping the symmetry of the code is worth it in a
case-per-case basis, but Rubocop doesn't allow this option.
We were already applying these rules in most cases.
Note we aren't enabling the `MultilineArrayLineBreaks` rule because
we've got places with many elements whire it isn't clear whether
having one element per line would make the code more readable.
Just like we respond with "not found" for any other record. This
improves the user experience because with the "Not found" error message
people realize the URL is wrong instead of thinking that they broke the
application.
The subdomain elevator we were using, which is included in apartment,
didn't work on hosts already including a subdomain (like
demo.consul.dev, for instance). In those cases, we would manually add
the subdomain to the list of excluded subdomains. Since these subdomains
will be different for different CONSUL installations, it meant each
installation had to customize the code. Furthermore, existing
installations using subdomains would stop working.
So we're using a custom method to find the current tenant, based on the
host defined in `default_url_options`.
In order to avoid any side-effects on single-tenant applications, we're
adding a new configuration option to enable multitenancy
We're enabling two ways to handle this configuration option:
a) Change the application_custom.rb file, which is under version control
b) Change the secrets.yml file, which is not under version control
This way people prefering to handle configuration options through
version control can do so, while people who prefer handling
configuration options through te secrets.yml file can do so as well.
We're also disabling the super-annoying warnings mentioning there are no
tenants which we got every time we run migrations on single-tenant
applications. These messages will only be enabled when the multitenancy
feature is enabled too. For this reason, we're also disabling the
multitenancy feature in the development environment by default.
We were getting a warning because it won't be included by default in
Rails 6.1:
DEPRECATION WARNING: Using I18n fallbacks with an empty `defaults` sets
the defaults to include the `default_locale`. This behavior will change
in Rails 6.1 . If you desire the default locale to be included in the
defaults, please explicitly configure it with
`config.i18n.fallbacks.defaults = [I18n.default_locale]` or
`config.i18n.fallbacks = [I18n.default_locale, {...}]`. If you want to
opt-in to the new behavior, use `config.i18n.fallbacks.defaults = [nil,
{...}] `.
We can remove the `new_framework_defaults_6_0` file by using Rails 6.0
default options and overwriting the ones we haven't enabled.
We're still using the classic autoloader because we still haven't
checked how switching to zeitwerk will affect the way CONSUL
installations customize their code.
And we're using the default queues for Active Storage because we were
already using them and that will be the default option in Rails 6.1.
All the code in the `bin/` and the `config/` folders has been generated
running `rake app:update`. The only exception is the code in
`config/application.rb` where we've excluded the engines that Rails 6.0
has added, since we don't use them.
There are a few changes in Active Storage which aren't compatible with
the code we were using until now.
Since the method to assign an attachment in ActiveStorage has changed
and is incompatible with the hack we used to allow assigning `nil`
attachments, and since ActiveStorage now supports assigning `nil`
attachments, we're removing the mentioned hack. This makes the
HasAttachment module redundant, so we're removing it.
Another change in ActiveStorage is files are no longer saved before
saving the `ActiveStorage::Attachment` record. This means we need to
manually upload the file when using direct uploads. We also have to
change the width and height validations we used for images; however,
doing so results in very complex code, and we currently have to write
that code for both images and site customization images.
So, for now, we're just uploading the file before checking its
dimensions. Not ideal, though. We might use active_storage_validations
in the future to fix this issue (when they support a proc/lambda, as
mentioned in commit 600f5c35e).
We also need to update a couple of tests due to a small change in
response headers. Now the content disposition returns something like:
```
attachment; filename="budget_investments.csv"; filename*=UTF-8''budget_investments.csv
```
So we're updating regular expression we use to check the filename.
Finally, Rails 6.0.1 changed the way the host is set in integration
tests [1] and so both `Capybara.app_host` and `Capybara.default_host`
were ignored when generating URLs in the relationable examples. The only
way I've found to make it work is to explicitely assign the host to the
integration session. Rails 6.1 will change this setup again, so maybe
then we can remove this hack.
[1] https://github.com/rails/rails/pull/36283/commits/fe00711e9
Without this change, we were getting errors when manually creating an
annotation using symbols in the ranges hash (that is, `ranges: [{ start:
"/p[1] }]` instead of `ranges: [{ "start" => "/p[1] }]`.
Using symbols isn't really supported and it might cause strange issues
with code like: `@draft_version.annotations.find_by(range_start: ...)`.
However, this might be an extreme case and using symbols might work
properly most of the time (if not all the time), and we've been using
them in our demo for years.
So I'm not writing a test for this case because I'm not sure we should
support it, but I'm still allowing symbols so we don't get exceptions in
cases where everything might be working fine. Rails already adds Symbol
by default to column permitted classes [1], so it's safe to enable it.
[1] https://github.com/rails/rails/pull/45584/commits/c2b96e3e8
This release introduces an incompatibility in order to fix a security
issue when using YAML for serialization. We use YAML to serialize the
`ranges` column in the `legislation_annotations` table, so we have to
allow the `ActiveSupport::HashWithIndifferentAccess` class in order to
properly read this column.
Ideally we'd use a JSONB column for the ranges (like we do in other
places), but that would require migrating existing data.
Bumps [rails](https://github.com/rails/rails) from 5.2.7.1 to 5.2.8.1.
- [Release notes](https://github.com/rails/rails/releases)
- [Commits](https://github.com/rails/rails/compare/v5.2.7.1...v5.2.8.1)
---
updated-dependencies:
- dependency-name: rails
...
When there was a custom JavaScript file, we weren't loading the original
one, meaning that, in order to customize it, it was necessary to copy
the whole original file and then changing it.
Now we're loading both the original and the custom file, so the custom
file can simply add more functions or overwrite the ones we'd like to
customize, without copying the whole file.
Existing copies of original files will still overwrite the whole file
and won't be affected.
