We forgot to do so in commit b896fc4bb. Back then, we said:
> Note that we aren't providing a proper aria-label for markers on the
> map we use in the form to create a proposal or an investment. Adding
> one isn't trivial given the current code, and keyboard users can't add
> a marker in the first place. We'll have to revisit this issue when we
> add keyboard support for this.
However, in the admin section, the marker is already there, so it should
have a label. In this case, we're using the coordinates as label because
it's the most relevant text for the marker in the context of a form. We
could also use "Default map location" instead, but that information is
already present on the page.
Axe was reporting the same accessibility error we mentioned in commit
b896fc4bb in this situation.
We were using the same texts twice. For the remove marker label text,
however, we were using the text defined in proposals for both proposals
and investments.
Ideally the translation keys for these texts would go in another
namespace, since they no longer refer to just proposals. However,
renaming the translation keys would mean losing the existing
translations in every language we manage through Crowdin. So we aren't
doing so.
The "on_budget_investments" scope in Activity has never been used
anywhere in the codebase. It was introduced in commit d9d38482b3
("extends Activity to include Investment valuations") but no references
were ever added.
Instead of removing it, we make use of the scope by adding the missing
"Budget investments" filter to the admin Activity section. This aligns
it with the rest of the activity filters and gives the scope the purpose
it was originally intended for.
The "sort_by_most_commented" scope in Debate is no longer used anywhere in
the code. Its last use was removed in commit b89f39bfef ("Removes
unused orders from debates controller")
We were calling `parse_remote_to_hash` in the Devise initializer, which
runs when the application starts.
That meant that, if we got an exception when calling that method, the
application wouldn't start. We got exceptions if the single sign-on
(SSO) URL isn't available or we aren't providing the right credentials.
So we're moving the call to `parse_remote_to_hash` to
`OmniauthTenantSetup`, which is only called when actually trying to sign
in with SAML.
Since we're moving the code there, we're also unifying the code so SAML
settings are configured the same way for the main tenant and other
tenants, like we did for OpenID Connect in commit c3b523290.
In order to keep the existing behavior, we're caching the result of
`parse_remote_to_hash` in an instance variable. Not sure about the
advantages and disadvantages of doing so over parsing the remote URL
metadata on every SAML-related request.
Note that the SAML tests in `OmniauthTenantSetup` use the `stub_secrets`
method. But this method is called after the application has started,
meaning it doesn't stub calls to `Rails.application.secrets` in
`config/initializers/`. So, before this commit, the code that parsed the
IDP metadata URL wasn't executed in the tests. Since now we've moved the
code but we don't want to depend on external URLs when running the
tests, we need to stub the call to the external URL. Since we're now
stubbing the call, we're adding expectations in the tests to check that
we correctly use the settings returned in that call.
When we first added OIDC support, we were configuring the redirect URI
in the devise initializer, just like we did for other providers.
Thanks to the changes in the previous commit, that code is no longer in
the devise initializer, which means we can use `url_helpers` to get the
redirect URI.
This means we no longer need to define this URI in the secrets. This is
particularly useful for multitenancy; previously, we had to define the
redirect URI for every tenant because different tenants use different
domains or different subdomains.
We were following the same pattern as we used for other providers like
twitter or facebook, but for OIDC we aren't passing the key and the
secret as separate attributes but only a hash of options. This means we
don't need to duplicate the same logic in the devise initializer and the
`OmniauthTenantSetup` class.
Thanks to these changes, we'll be able to introduce dynamic redirect
URLs for both the default tenant and the other tenants (see next commit).
Note that we could probably apply similar changes for the SAML provider.
We might do so in the future. For other providers, removing the
references to `Rails.application.secrets` broke their configuration when
we tested it back in 2022 as part of the multitenancy feature. We might
check whether that's no longer the case (or whether we made a mistake
during our tests in 2022) in the future.
Note that we are not including Poll::PartialResults for open-ended
questions resutls. The reason is that we do not contemplate the
possibility of there being open questions in booths. Manually
counting and introducing the votes in the system is not feasible.
Ensure GDPR compliance by default (Article 25 GDPR – privacy by design
and by default). Under GDPR, consent must be freely given, specific,
informed and unambiguous [1]. We were subscribing users without
explicity consent, which goes against the "No pre-ticked boxes"
principle.
For compatibility with existing installations, we're using a setting,
disabled by default. Once we release version 2.4.0 we will enable it by
default, which won't affect existing installations but only new ones.
[1] https://gdprinfo.eu/best-gdpr-newsletter-consent-examples-a-complete-guide-to-compliant-email-marketing
Unify the code from app/views/officing/results/index.html.erb with
app/views/admin/poll/results/_result.html.erb. This prepares the ground
to extract a component in the next commit and avoid duplication.
- Remove two redundant go_back_to_new calls in build_results, since
@poll.questions.find already raises RecordNotFound if a question
does not exist.
- Drop the fallback flash translation error_create, which is no longer
used since commit 592fdffe4e and only remained as a default in
go_back_to_new.
- Move check_officer_assignment from Officing::BaseController to
Officing::ResultsController, its only place of use.
- name: :oidc → Identifier for this login provider in the app.
- scope: [:openid, :email, :profile] → Tells the provider we want the user’s ID (openid), their email, and basic profile info (name, picture, etc.).
- response_type: :code → Uses Authorization Code Flow, which is more secure because tokens are not exposed in the URL.
- issuer: Rails.application.secrets.oidc_issuer → The base URL of the OIDC provider (e.g., Auth0). Used to find its config.
- discovery: true → Automatically fetches the provider’s endpoints from its discovery document instead of manually setting them.
- client_auth_method: :basic → Sends client ID and secret using HTTP Basic Auth when exchanging the code for tokens.
Add system tests for OIDC Auth
Edit the oauth docs to support OIDC auth
Note that Rails 7.1 changes `find_or_create_by!` so it calls
`create_or_find_by!` when no record is found, meaning we'll rarely get
`RecordNotUnique` exceptions when using this method during a race
condition.
Adding this index means we need to remove the uniqueness validation.
According to the `create_or_find_by` documentation [1]:
> Columns with unique database constraints should not have uniqueness
> validations defined, otherwise create will fail due to validation
> errors and find_by will never be called.
We're adding a test that checks what happens when using
`create_or_find_by!`.
Note that we're creating voters combining `create_with` with
`find_or_create_by!`. Using `find_or_create_by!(...)` with all
attributes (including non-key ones like `origin`) fails when a voter
already exists with different values, e.g. an existing `origin: "web"`
and an incoming `"booth"`. In this situation the existing record is not
matched and the unique index raises an exception.
`create_with(...).find_or_create_by!(user: ..., poll: ...)` searches by
the unique key only and applies the extra attributes only on creation.
Existing voters are returned unchanged, which is the intended behavior.
For the `take_votes_from` method, we're handling a (highly unlikely, but
theoretically possible) scenario where a user votes at the same time as
taking voters from another user. For that, we're doing something similar
to what `create_or_find_by!` does: we're updating the `user_id` column
inside a new transaction (using a new transactions avoids a
`PG::InFailedSqlTransaction` exception when there are duplicate
records), and deleting the existing voter when we get a
`RecordNotUnique` exception.
On `Poll::WebVote` we're simply raising an exception when there's
already a user who's voted via booth, because the `Poll::WebVote#update`
method should never be called in this case.
We still need to use `with_lock` in `Poll::WebVote`, but not due to
duplicate voters (`find_or_create_by!` method will now handle the unique
record scenario, even in the case of simultaneous transactions), but
because we use a uniqueness validation in `Poll::Answer`; this
validation would cause an error in simultaneous transactions.
[1] https://api.rubyonrails.org/v7.1/classes/ActiveRecord/Relation.html#method-i-create_or_find_by
This could be the case when JavaScript is disabled.
Note that, in `Poll/WebVote` we're calling `given_answers` inside a
transaction. Putting this code before the transaction resulted in a test
failing sometimes, probably because of a bug that might be possible to
reproduce by doing simultaneous requests.
With the old interface, there wasn't a clear way to send a blank ballot.
But now that we've got a form, there's an easy way: clicking on "Vote"
while leaving the form blank.
Our original interface to vote in a poll had a few issues:
* Since there was no button to send the form, it wasn't clear that
selecting an option would automatically store it in the database.
* The interface was almost identical for single-choice questions and
multiple-choice questions, which made it hard to know which type of
question we were answering.
* Adding other type of questions, like open answers, was hard since we
would have to add a different submit button for each answer.
So we're now using radio buttons for single-choice questions and
checkboxes for multiple-choice questions, which are the native controls
designed for these purposes, and a button to send the whole form.
Since we don't have a database table for poll ballots like we have for
budget ballots, we're adding a new `Poll::WebVote` model to manage poll
ballots. We're using WebVote instead of Ballot or Vote because they
could be mistaken with other vote classes.
Note that browsers don't allow removing answers with radio buttons, so
once somebody has voted in a single-choice question, they can't remove
the vote unless they manually edit their HTML. This is the same behavior
we had before commit 7df0e9a96.
As mentioned in c2010f975, we're now adding the `ChangeByZero` rubocop
rule, since we've removed the test that used `and change`.
Without this change the IdpMetaParser would give an error
in the Devise initializer when starting the application.
I found it annoying to have to connect to the VPN so
I decided to add this condition.
Reviewer, feel free to consider this commit unnecessary
and ask to revert it.
We were having an issue because there was a difference of about 11
seconds between the local times of our machines and the time of the IDP
server. Since right now we can't guarantee the time of these machines is
fully synchronized, for now we're adding a margin of error of one
minute.
The default "puma_bind" setting was reintroduced in capistrano3-puma 6.2.0
as "unix://.../tmp/sockets/puma.sock". We no longer need to define it
explicitly in deploy.rb.
NOTE: Although the official capistrano3-puma README still states
that "puma_bind" must be explicitly set after upgrading to v6,
this is no longer true since version 6.2.0. The default value
("unix://.../tmp/sockets/puma.sock") has been restored.
We remove our custom puma.service.erb template located at
lib/capistrano/templates, as the upstream capistrano3-puma gem (v6.2.0)
now supports all the adjustments we previously needed.
This template was originally copied and modified in
PR #5842 (Bump capistrano3-puma from 5.2.0 to 6.0.0),
with the following relevant changes:
1. WatchdogSec=0
We disabled systemd's watchdog feature by setting WatchdogSec=0.
This is now covered upstream since v6.1.0 via:
> <% if fetch(:puma_systemd_watchdog_sec) && fetch(:puma_systemd_watchdog_sec) > 0 %>
> WatchdogSec=<%= fetch(:puma_systemd_watchdog_sec) %>
> <% end %>
To preserve the same behavior, we now explicitly set:
> set :puma_systemd_watchdog_sec, 0
in config/deploy.rb, instead of relying on the template override.
2. SyslogIdentifier=puma
In our custom template, we replaced the default line:
> SyslogIdentifier=<%= fetch(:puma_service_unit_name) %>
with:
> SyslogIdentifier=puma
This was done to avoid introducing a new config variable
in the installer. However, now that we want to remove the overridden template,
we must accept the default behavior from the gem:
> set :puma_service_unit_name, -> { "puma_#{fetch(:application)}_#{fetch(:stage)}" }
As a result, the generated SyslogIdentifier will change from "puma" to something
like "puma_consul_staging".
The only implication is that we'll open a PR in the installer
to replace the hardcoded "puma" with the dynamic default,
so everything stays consistent.
Another option would be to override puma_service_unit_name
with "puma" to keep the SyslogIdentifier exactly as before.
However, this would also affect the names of the systemd services and sockets
(e.g., puma.service instead of puma_consul_staging.service),
which may lead to conflicts or require manual cleanup steps like disabling
and removing the old units. For now, we prefer to avoid that.
3. After=network.target
We had removed the "syslog.target" from the "After=" directive,
as we didn't depend on syslog.
However, keeping "After=syslog.target network.target" does not negatively impact us,
since Puma logs to files directly using StandardOutput=append:....
So we can also adopt the default value from template here.
4. ExecStart
In version 6.2.0, a small change was made to the template logic for the ExecStart line.
This change does not affect us. Since we don't define puma_use_login_shell,
the behavior remains exactly the same as before.
With these changes already supported in the gem, we no longer need
a local copy of the puma service template.
Bumps [rubocop](https://github.com/rubocop/rubocop) from 1.71.2 to 1.75.8.
- [Release notes](https://github.com/rubocop/rubocop/releases)
- [Changelog](https://github.com/rubocop/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop/rubocop/compare/v1.71.2...v1.75.8)
---
updated-dependencies:
- dependency-name: rubocop
dependency-version: 1.75.8
dependency-type: direct:development
update-type: version-update:semver-minor
...
Notes:
This commit also includes several style and lint fixes required after
updating RuboCop:
- Removed redundant parentheses now detected by improved
'Style/RedundantParentheses' (1.72 and 1.75.3).
- Replaced ternary expressions with logical OR when the ternary was
returning 'true', as flagged by 'Style/RedundantCondition' (1.73).
- Adjusted block variables to resolve new 'Lint/ShadowingOuterLocalVariable'
offenses (1.75), helping avoid future conflicts during upgrades with
'rails app:updates'
Signed-off-by: dependabot[bot] <support@github.com>
Ahoy 2.0.0 [1] introduced automatic UUID generation for visit_token and
visitor_token. As a result, the custom ensure_uuid method is no longer
needed and can be safely removed from the initializer.
Since we aren't manually generating UUIDs anymore, we no longer need
the uuidtools dependency.
[1] https://github.com/ankane/ahoy/blob/v2.0.0/README.md#token-generation
IMHO, Spring no longer provides benefits in this project and:
- Spring was already disabled in the test environment since commit e4e0cb5d47
- Rails removed Spring as a default installation option in 2021 [1]
[1] PR #42997 from https://github.com/rails/rails/
capistrano3-puma v6.0.0 updated the defaults for puma_access_log and
puma_error_log to use a single file based on puma_env, like:
> set_if_empty :puma_access_log, -> { File.join(shared_path, 'log', "#{fetch(:puma_env)}.log") }
> set_if_empty :puma_error_log, -> { File.join(shared_path, 'log', "#{fetch(:puma_env)}.log") }
However, our installer expect:
- puma_access.log
- puma_error.log
To keep the existing behavior aligned with the installer, we define
the Puma log paths in config/deploy.rb
In capistrano3-puma v6.0.0, the default for 'puma_service_unit_name' changed to:
> "#{application}_puma_#{stage}"
But the installer uses the older convention:
> "puma_#{application}_#{stage}"
To ensure consistency and avoid unit name conflicts when switching between
versions or deploying older branches, we now define the variable explicitly
in config/deploy.rb:
> set :puma_service_unit_name, -> { "puma_#{fetch(:application)}_#{fetch(:stage)}" }
This commit copies the default puma.service.erb template from the
capistrano3-puma gem into lib/capistrano/templates. This allows us to
customize the generated systemd unit file during deploy.
Note that we are also removing the `:puma_conf` variable from `config/deploy.rb`,
as the new ExecStart line in the systemd template (based on capistrano3-puma 6.0.0)
does not rely on a separate Puma config file. The command now directly invokes:
ExecStart=<%= expanded_bundle_command %> exec puma -e <%= fetch(:puma_env) %>
This replaces the older format used in 5.2.0:
ExecStart=<%= expanded_bundle_command %> exec --keep-file-descriptors puma -C <%= fetch(:puma_conf) %>
which required explicitly setting the Puma config path.
In earlier versions of capistrano3-puma, the puma_bind has a default value to:
unix://.../tmp/sockets/puma.sock via set_if_empty in lib/capistrano/puma.rb.
This default was removed in 6.0.0, requiring to explicitly set :puma_bind in deploy.rb.
This caused the following runtime error during deploy:
> Failed to restart consul_puma_staging.service: Unit
> consul_puma_staging.socket has a bad unit file setting.
capistrano3-puma 6.0.0 removed the `puma:systemd:config` and
`puma:systemd:enable` tasks. This commit updates the deploy script to use
the new `puma:install` and `puma:enable` tasks instead.
We're still using YAML to serialize the legislation_annotations ranges
column. I'm not sure whether changing the serializer can have
consequences on existing data, and I'm not sure which serializer we
should provide instead. Quoting the Rails configuration guide [1]:
> Unfortunately there isn't really any suitable defaults available in
> Ruby's standard library. JSON could work as a format, but the json
> gems will cast unsupported types to strings which may lead to bugs.
[1] https://guides.rubyonrails.org/v7.1/configuring.html#config-active-record-default-column-serializer
I think this doesn't affect us because we use RSpec instead of Rails
test classes. In any case, if it ever affects us, we'll get notified
when a test fails.
As mentioned in the pull request introducing this change [1]:
> FATAL is documented in the Ruby Logger docs as being for "An
> unhandleable error that results in a program crash.", which does not
> really apply to this case since DebugExceptions is handling the error.
So we're using the new default value, which makes more sense.
[1] Pull request 48575 in https://github.com/rails/rails
Since we use a version of Loofah supporting HTML5 since db2d0bb80, the
`Rails::HTML::Sanitizer.best_supported_vendor` method will return the
HTML5 sanitizer. As mentioned in the pull request introducting this
change [1], the libxml2 maintainer wrote:
> it's still a bad idea to use a 20+ years old, unmaintained HTML 4
> parser to sanitize input for the modern web
So we're going with the new default sanitizer.
Note we aren't uncommenting the `action_text.sanitizer_vendor` option
because we don't use Action Text and so it doesn't affect us , and
uncommeting it will raise an error.
Also note we need to change one test because the new sanitizer handles
whitespace slightly differently.
[1] Pull request 48293 in https://github.com/rails/rails
Just like we mentioned in commit 001eee3d6, according to the Rails
configuration guide [1], with this format, Rails serializes cache
entries more efficiently. Most importantly:
> All formats are backward and forward compatible, meaning cache entries
> written in one format can be read when using another format. This
> behavior makes it easy to migrate between formats without invalidating
> the entire cache.
[1] https://guides.rubyonrails.org/v7.1/configuring.html#config-active-support-cache-format-version
It doesn't really affect us because we never use `return`, `break` or
`throw` inside transactions, since it would be confusing exactly because
it wouldn't be 100% clear whether the transaction is committed or not.
So we're using the new default value, which will be the only available
option in Rails 7.2 [1].
[1] Commit eccc6061f4 in https://github.com/rails/rails
This change doesn't affect us, since we don't use `after_commit`
callbacks, and, among our dependencies, AFAIK only the Devise gem uses
them, and it only defines one after_commit callback when creating a
record and another one when updating it, so we're never going to have
more than one callback being executed after a transaction is finished.
