Ensure GDPR compliance by default (Article 25 GDPR – privacy by design
and by default). Under GDPR, consent must be freely given, specific,
informed and unambiguous [1]. We were subscribing users without
explicity consent, which goes against the "No pre-ticked boxes"
principle.
For compatibility with existing installations, we're using a setting,
disabled by default. Once we release version 2.4.0 we will enable it by
default, which won't affect existing installations but only new ones.
[1] https://gdprinfo.eu/best-gdpr-newsletter-consent-examples-a-complete-guide-to-compliant-email-marketing
Note that Rails 7.1 changes `find_or_create_by!` so it calls
`create_or_find_by!` when no record is found, meaning we'll rarely get
`RecordNotUnique` exceptions when using this method during a race
condition.
Adding this index means we need to remove the uniqueness validation.
According to the `create_or_find_by` documentation [1]:
> Columns with unique database constraints should not have uniqueness
> validations defined, otherwise create will fail due to validation
> errors and find_by will never be called.
We're adding a test that checks what happens when using
`create_or_find_by!`.
Note that we're creating voters combining `create_with` with
`find_or_create_by!`. Using `find_or_create_by!(...)` with all
attributes (including non-key ones like `origin`) fails when a voter
already exists with different values, e.g. an existing `origin: "web"`
and an incoming `"booth"`. In this situation the existing record is not
matched and the unique index raises an exception.
`create_with(...).find_or_create_by!(user: ..., poll: ...)` searches by
the unique key only and applies the extra attributes only on creation.
Existing voters are returned unchanged, which is the intended behavior.
For the `take_votes_from` method, we're handling a (highly unlikely, but
theoretically possible) scenario where a user votes at the same time as
taking voters from another user. For that, we're doing something similar
to what `create_or_find_by!` does: we're updating the `user_id` column
inside a new transaction (using a new transactions avoids a
`PG::InFailedSqlTransaction` exception when there are duplicate
records), and deleting the existing voter when we get a
`RecordNotUnique` exception.
On `Poll::WebVote` we're simply raising an exception when there's
already a user who's voted via booth, because the `Poll::WebVote#update`
method should never be called in this case.
We still need to use `with_lock` in `Poll::WebVote`, but not due to
duplicate voters (`find_or_create_by!` method will now handle the unique
record scenario, even in the case of simultaneous transactions), but
because we use a uniqueness validation in `Poll::Answer`; this
validation would cause an error in simultaneous transactions.
[1] https://api.rubyonrails.org/v7.1/classes/ActiveRecord/Relation.html#method-i-create_or_find_by
Bumps [rubocop](https://github.com/rubocop/rubocop) from 1.71.2 to 1.75.8.
- [Release notes](https://github.com/rubocop/rubocop/releases)
- [Changelog](https://github.com/rubocop/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop/rubocop/compare/v1.71.2...v1.75.8)
---
updated-dependencies:
- dependency-name: rubocop
dependency-version: 1.75.8
dependency-type: direct:development
update-type: version-update:semver-minor
...
Notes:
This commit also includes several style and lint fixes required after
updating RuboCop:
- Removed redundant parentheses now detected by improved
'Style/RedundantParentheses' (1.72 and 1.75.3).
- Replaced ternary expressions with logical OR when the ternary was
returning 'true', as flagged by 'Style/RedundantCondition' (1.73).
- Adjusted block variables to resolve new 'Lint/ShadowingOuterLocalVariable'
offenses (1.75), helping avoid future conflicts during upgrades with
'rails app:updates'
Signed-off-by: dependabot[bot] <support@github.com>
I don't think this feature it was ever used. It was introduced in commit
49dec6061 as part of a feature that was removed in commits 1cd47da9d and
c45a0bd8ac.
This method was added in Rails 7.0 and makes the code slihgtly more
readable.
The downside is that it generates two queries instead of one, so it
might generate some confusion when debugging SQL queries. Its impact on
performance is probably negligible.
These cases aren't covered by the `Rails/WhereRange` rubocop rule, but
IMHO using ranges makes them more consistent. Besides, they generate SQL
which is more consistent with what Rails usually generates. For example,
`Poll.where("starts_at <= :time and ends_at >= :time", time:
Time.current)` generates:
```
SELECT \"polls\".\"id\", (...) WHERE \"polls\".\"hidden_at\" IS NULL AND
(starts_at <= '2024-07-(...)' and ends_at >= '2024-07-(...)')
```
And `Poll.where(starts_at: ..Time.current, ends_at: Time.current..)`
generates:
```
SELECT \"polls\".\"id\", (...) WHERE \"polls\".\"hidden_at\" IS NULL AND
\"polls\".\"starts_at\" <= '2024-07-(...)' AND \"polls\".\"ends_at\" >=
'2024-07-(...)'"
```
Note that the `not_archived` scope in proposals slightly changes, since
we were using `>` and now we use the equivalent of `>=`. However, since
the `created_at` field is a time, this will only mean that a proposal
will be archived about one microsecond later.
For consistency, we're also changing the `archived` scope, so a proposal
is never archived and not archived at the same time (not even for a
microsecond).
This rule was added in rubocop-rails 2.25.0. Applying it allows us to
simplify the code a little bit. For example, now there's no need to
specify the `proposals` table in proposal scopes, which was actually
causing a bug in the `Legislation::Proposal` model, which was using the
`proposals` table instead of the `legislation_proposals` table (but,
since we don't use this scope, it didn't affect the application).
Note that, when taking votes from an erased user, since poll answers
don't belong to poll voters, we were not migrating them in the
`take_votes_from` method (and we aren't migrating them now either).
Note that, for everything to work consistently, we need to make sure
that the default locale is one of the available locales.
Also note that we aren't overwriting the `#save ` method set by
globalize. I didn't feel too comfortable changing a monkey-patch which
ideally shouldn't be there in the first place, I haven't found a case
where `Globalize.locale` is `nil` (since it defaults to `I18n.locale`,
which should never be `nil`), so using `I18n.default_locale` probably
doesn't affect us.
When calculating the stats on, say, January 1st 2024, and using a group
age of, say, between 20 and 24 years, we were considering that everyone
born between 2000 and 2004 had between 20 and 24 years. This wasn't
accurate, since most people born in 2004 would have 19 years at that
point, and most people born in 1999 would have 24 years.
Using `>` and `<` for dates is a bit confusing because it usually
requires mentally translating `<` into "before" and `>` into "after",
meaning it takes a few seconds to check which is the starting date and
which is the ending date. When using a range, it's easier to see which
is which.
Note that we were using `>` and `<`, but now we're using the equivalent
of `>=` and `<=`. This makes sense IMHO, since the beginning of the year
and the end of the year should also be included in this case.
We were calculating the age stats based on the age of the users who
participated... at the moment where we were calculating the stats. That
means that, if 20 years ago, 1000 people who were 16 years old
participated, they would be shown as having 36 years in the stats.
Instead, we want to show the stats at the time when the process took
place, so we're implementing a `participation_date` method.
Note that, for polls, we could actually use the `age` column in the
`poll_voters` table. However, doing so would be harder, would only work
for polls but not for budgets, and it wouldn't be statistically very
relevant, since the stats are shown by age groups, and only a small
percentage of people would change their age group (and only to the
nearest one) between the time they participate and the time the process
ends.
We might use the `poll_voters` table in the future, though, since we
have a similar issue with geozones and genders, and using the
information in `poll_voters` would solve it as well (only for polls,
though).
Also note that we're using the `ends_at` dates because some people but
be too young to vote when a process starts but old enough to vote when
the process ends.
Finally, note that we might need to change the way we calculate the
participation date for a budget, since some budgets might not enabled
every phase. Not sure how stats work in that scenario (even before these
changes).
We were using generic names like `args` and `options` which don't really
add anything to `*` or `**` because Ruby required us to.
That's no longer the case in Ruby 3.2, so we can simplify the code a
bit.
This way it won't be possible to browse all user URLs by just going to
/users/1, /users/2, /users/3, ... and collect usernames, which might not
be desirable in some cases.
Note we could use the username as a URL parameter and just find the user
with `@user = User.find_by!(id: id, username: username)`, but since
usernames might contain strange characters, this might lead to
strange/ugly URLs.
Finally, note we're using `username.to_s` in order to cover the case
where the username is `nil` (as is the case with erased users).
In order to the display a warn text on the last attempt
before the account is locked, we need update
config.paranoid to false as the devise documentation
explains.
Adding "config.paranoid: false" implies further changes
to the code, so for now we unncomment the default value
"config.last_attempt_warning = true" and update it to false.
This rule was added in rubocop 1.44.0. It's useful to avoid accidental
`unless !condition` clauses.
Note we aren't replacing `unless zero?` with `if nonzero?` because we
never use `nonzero?`; using it sounds like `if !zero?`.
Replacing `unless any?` with `if none?` is only consistent if we also replace
`unless present?` with `if blank?`, so we're also adding this case. For
consistency, we're also replacing `unless blank?` with `if present?`.
We're also simplifying code dealing with `> 0` conditions in order to
make the code (hopefully) easier to understand.
Also for consistency, we're enabling the `Style/InverseMethods` rule,
which follows a similar idea.
For the HashAlignment rule, we're using the default `key` style (keys
are aligned and values aren't) instead of the `table` style (both keys
and values are aligned) because, even if we used both in the
application, we used the `key` style a lot more. Furthermore, the
`table` style looks strange in places where there are both very long and
very short keys and sometimes we weren't even consistent with the
`table` style, aligning some keys without aligning other keys.
Ideally we could align hashes to "either key or table", so developers
can decide whether keeping the symmetry of the code is worth it in a
case-per-case basis, but Rubocop doesn't allow this option.
Back in commit 36e452437, we wrote:
> The `reload` method added to max_votes validation is needed because
> the author gets here with some changes because of the around_action
> `switch_locale`, which adds some changes to the current user record
> and therefore, the lock method raises an exception when trying to lock
> it requiring us to save or discard those record changes.
This happened when `current_user` didn't have a locale stored in the
database and the `current_locale` method returned the default locale.
And the test "Poll Votation Type Multiple answers" would indeed fail if
we removed the `reload` method. However, we can remove the need to
reload the record by avoiding the mentioned changes on the current user
record.
So we're changing the `User#locale` method so it doesn't modify the user
record.
The Google response contains an `email_verified` field instead of a
`verified_email` field, and so we weren't treating verified Google
accounts as verified.
After a user assigned as a budget admin deletes their account or gets blocked by
a moderator, the application throws an exception while loading the admin
investment index page.
As an erased user is not really deleted and neither its associated roles, the
application was failing when trying to sort and administration without a
username. In this case, the application was throwing an `ArgumentError:
comparison of NilClass with String failed` exception.
As a blocked user is not deleted or its roles, the application failed when trying
to access the user name through the delegation in the Administrator. In this
case, the application was throwing a `NoMethodError: undefined method `name' for
nil:NilClass` exception.
We were returning an (empty) association of users instead of empty
associations of proposals, debates or comments. The code worked because
in the end it returned an empty array, but looked weird nevertheless.
After commit 0214184b2, this method was only used in two places and was
only useful in one of them. IMHO it isn't worth it add a monkey-patch
for such a minor usage.
Just like we did in commit 0214184b2d for investments, we're removing
some possible optimizations (we don't have any benchmarks proving they
affect performance at all) in order to simplify the code.
The investement votes component `delegate` code was accidentally left
but isn't used since commit 0214184b2, so we're removing it now that
we're removing the `voted_for?` helper method.
When we try to register with omniauth and the email or username already exists,
we use the finish_signup and do_finish_signup actions to allow the user to choose
another email or username.
The do_finish_signup action of the registration controller calls the
send_oauth_confirmation_instructions method which is responsible for sending the
confirmation email.
In this method we were only validating the case that the email is duplicated. Now
we add one more condition that allows us to send the instructions for the case in
which we have had to change our username.
Although it wasn't a real security concern because we were only calling
a `find_by` method based on the user input, it's a good practice to
avoid using constants based on user parameters.
Since we don't use the `find_by` method anymore but we still need to
check the associated record exists, we're changing the validations in
the `RelatedContent` model to do exactly that.
In the previous commit I mentioned:
> If I'm right, the `investment_votes` instance variable only exists to
> avoid several database queries to get whether the current user has
> supported each of the investments.
>
> However, that doesn't make much sense when only one investment is
> shown.
Now let's discuss the case when there are several investments, like in
the investments index:
* There are 10 investments per page by default
* Each query takes less than a millisecond
* We still make a query per investment to check whether the current user
voted in a different group
* AFAIK, there have been no performance tests showing these
optimizations make the request to the investments index significantly
faster
* These optimizations make the code way more complex than it is without
them
Considering all these points, I'm removing the optimizations. I'm fine
with adding `includes` calls to preload records and avoid N+1 queries
even if there are no performance tests showing they make the application
faster because the effect on the code complexity is negligible. But
that's not the case here.
Note we're using `defined?` instead of the `||=` operator because the
`||=` operator will not serve its purpose when the result of the
operation returns `false`.
Our manual implementation had a few issues. In particular, it didn't
track changes related to associations, which became more of an issue
when we made investments translatable.
Using audited gives us more functionality while at the same time
simplifies our code. However, it adds one more external dependency to
our project.
The reason for choosing audited over paper trail is audited seems to
make it easier to handle associations.
The current tracking section had a few issues:
* When browsing as an admin, this section becomes useless since no
investments are shown
* Browsing investments in the admin section, you're suddenly redirected
to the tracking section, making navigation confusing
* One test related to the officing dashboard failed due to these changes
and had been commented
* Several views and controller methods were copied from other sections,
leading to duplication and making the code harder to maintain
* Tracking routes were defined for proposals and legislation processes,
but in the tracking section only investments were shown
* Probably many more things, since these issues were detected after only
an hour reviewing and testing the code
So we're removing this untested section before releasing version 1.1. We
might add it back afterwards.
