We noticed there was a performance issue while browsing the SDG
Management section and when one of our tests started failing sometimes
because the request to the relations#index controller took too long.
The issue proved to be `SDG::Target#<=>`. This method calls `.goal` for
each target, meaning we were generating 169 database queries when
sorting all targets.
So we're comparing codes directly to minimize the number of database
queries and improve performance. Requests to the relations index take
now less than third of the time they used to take.
and its relation with relatables
Note about sdg_review factory: Cannot use the constantize method on
the relatable_type as long as the relatable classes will be loaded and
this will throw an exception because the database is not available at
factiry definition time.
So now we'll be able to add them to other sections.
We're also adding a `dependent: :destroy` relation to models having
cards since it doesn't make sense to have cards around when their page
has been destroyed.
Note using `params[:relatable_type].classify` is recognized as a
security risk by some tools. However, it's a false positive, since we've
added constraints to the URL so that paramenter can only have the values
we trust.
Ruby 2.6 introduces `Enumerable#filter` as an alias to
`Enumerable#select`, and so our Filterable.filter method will not work
with Ruby 2.6.
So we're renaming the method to `filter_by`, which is similar to
`find_by`. We could also change the `filter` method so if a block is
given it delegates to `Enumerable#filter`, the same way ActiveRecord
handles the `select` method, but IMHO this is easier to follow.
Using pg_search 2.0.1 with Rails 5.2 results in deprecation warnings:
DEPRECATION WARNING: Dangerous query method (method whose arguments used
as raw SQL) called with non-attribute argument(s):
"pg_search_978c2f8941354cf552831b.rank DESC, \"tags\".\"id\" ASC".
Non-attribute arguments will be disallowed in Rails 6.0. This method
should not be called with user-provided values, such as request
parameters or model attributes. Known-safe values can be passed by
wrapping them in Arel.sql().
We're not upgrading to the latest pg_search because it only supports
ActiveRecord >= 5.2.
Implementation tries to be open for further extensions, such as deciding on
search dictionary based on configuration option or by locale set for
given user.
In this case using `joins` doesn't prevent N+1 queries to get titles for
every record, and since we cannot order translations with just SQL due
to fallbacks, we don't need it.
Automatic SQL injection checks were showing a false positive in this
scope; there was no real vulnerability here because foreign keys, table
names and locales were under our control.
We make the code easier to read and at the same time we remove a SQL
injection false positive regarding the use of `WHERE id = #{id}`.
We still get a warning about SQL injection regarding the `tsv =` part.
It's a false positive, since the value of that parameter does not
depend on user input.
When defining abilities, scopes cover more cases because they can be
used to check permissions for a record and to filter a collection. Ruby
blocks can only be used to check permissions for a record.
Note the `Budget::Phase.kind_or_later` name sounds funny, probably
because we use the word "phase" for both an an attribute in the budgets
table and an object associated with the budget, and so naming methods
for a budget phase is a bit tricky.
The new CSV report was more configurable and could work on proposals,
processes and comments. However, it had several issues.
In the public area, by default it generated a blank file.
In the admin section, the report was hard to configure and it generated
a file with less quality than the old system.
So until we improve this system, we're bringing back the old investment
CSV exporter.
This commit reverts most of commit 9d1ca3bf.
Class variables in Ruby are not the same as instance variables of a
class. They're particularly tricky when it comes to inheritance and
modules.
In the case of the Measurable module, for example, using a class
variable will make all classes including the Measurable module share
the same value. However, that's not what we want; we want the variable
to be different for every class. And that's accomplished using a class
instance variable.
Although in this case it would probably be better to remove the caching
variable. I don't think these methods are called more than once during a
request, and even if they did it's highly unlikely the would become a
bottleneck.
Not doing so has a few gotchas when working with relations, particularly
with records which are not stored in the database.
I'm excluding the related content file because it's got a very peculiar
relationship with itself: the `has_one :opposite_related_content` has no
inverse; the relation itself is its inverse. It's a false positive since
the inverse condition is true:
```
content.opposite_related_content.opposite_related_content.object_id ==
content.object_id
```
We were inconsistent on this one. I consider it particularly useful when
a method starts with a `return` statement.
In other cases, we probably shouldn't have a guard rule in the middle of
a method in any case, but that's a different refactoring.
We were very inconsistent regarding these rules.
Personally I prefer no empty lines around blocks, clases, etc... as
recommended by the Ruby style guide [1], and they're the default values
in rubocop, so those are the settings I'm applying.
The exception is the `private` access modifier, since we were leaving
empty lines around it most of the time. That's the default rubocop rule
as well. Personally I don't have a strong preference about this one.
[1] https://rubystyle.guide/#empty-lines-around-bodies
We were already using `find_by` most of the time.
Since there are false positives related to our `find_by_slug_or_id!` and
`find_by_manger_login` methods, which cannot be replaced with `find_by`,
I'm adding it indicating the "refactor" severity.
Having exceptions is better than having silent bugs.
There are a few methods I've kept the same way they were.
The `RelatedContentScore#score_with_opposite` method is a bit peculiar:
it creates scores for both itself and the opposite related content,
which means the opposite related content will try to create the same
scores as well.
We've already got a test to check `Budget::Ballot#add_investment` when
creating a line fails ("Edge case voting a non-elegible investment").
Finally, the method `User#send_oauth_confirmation_instructions` doesn't
update the record when the email address isn't already present, leading
to the test "Try to register with the email of an already existing user,
when an unconfirmed email was provided by oauth" fo fail if we raise an
exception for an invalid user. That's because updating a user's email
doesn't update the database automatically, but instead a confirmation
email is sent.
There are also a few false positives for classes which don't have bang
methods (like the GraphQL classes) or destroying attachments.
For these reasons, I'm adding the rule with a "Refactor" severity,
meaning it's a rule we can break if necessary.
Usually when we use `try` we actually mean `try!`, which is the same as
the safe navigation operator. However, there are a few cases where we
actually mean to execute a method if the object responds to that method.
In those cases using `try` would actually be OK, but in order to avoid
confusion as to whether we mean to check for `respond_to?` or we mean to
use safe navigation, I'm removing all usages of `try`.
Sanitizing descriptions before saving a record has a few drawbacks:
1. It makes the application rely on data being safe in the database. If
somehow dangerous data enters the database, the application will be
vulnerable to XSS attacks
2. It makes the code complicated
3. It isn't backwards compatible; if we decide to disallow a certain
HTML tag in the future, we'd need to sanitize existing data.
On the other hand, sanitizing the data in the view means we don't need
to triple-check dangerous HTML has already been stripped when we see the
method `auto_link_already_sanitized_html`, since now every time we use
it we sanitize the text in the same line we call this method.
We could also sanitize the data twice, both when saving to the database
and when displaying values in the view. However, doing so wouldn't make
the application safer, since we sanitize text introduced through
textarea fields but we don't sanitize text introduced through input
fields.
Finally, we could also overwrite the `description` method so it
sanitizes the text. But we're already introducing Globalize which
overwrites that method, and overwriting it again is a bit too confusing
in my humble opinion. It can also lead to hard-to-debug behaviour.
Now we take into consideration locales persisted but marked for
destruction to complete some logic and to be able to show best
translations on different situations.
