In this case using `joins` doesn't prevent N+1 queries to get titles for
every record, and since we cannot order translations with just SQL due
to fallbacks, we don't need it.
Automatic SQL injection checks were showing a false positive in this
scope; there was no real vulnerability here because foreign keys, table
names and locales were under our control.
We make the code easier to read and at the same time we remove a SQL
injection false positive regarding the use of `WHERE id = #{id}`.
We still get a warning about SQL injection regarding the `tsv =` part.
It's a false positive, since the value of that parameter does not
depend on user input.
When defining abilities, scopes cover more cases because they can be
used to check permissions for a record and to filter a collection. Ruby
blocks can only be used to check permissions for a record.
Note the `Budget::Phase.kind_or_later` name sounds funny, probably
because we use the word "phase" for both an an attribute in the budgets
table and an object associated with the budget, and so naming methods
for a budget phase is a bit tricky.
The new CSV report was more configurable and could work on proposals,
processes and comments. However, it had several issues.
In the public area, by default it generated a blank file.
In the admin section, the report was hard to configure and it generated
a file with less quality than the old system.
So until we improve this system, we're bringing back the old investment
CSV exporter.
This commit reverts most of commit 9d1ca3bf.
Class variables in Ruby are not the same as instance variables of a
class. They're particularly tricky when it comes to inheritance and
modules.
In the case of the Measurable module, for example, using a class
variable will make all classes including the Measurable module share
the same value. However, that's not what we want; we want the variable
to be different for every class. And that's accomplished using a class
instance variable.
Although in this case it would probably be better to remove the caching
variable. I don't think these methods are called more than once during a
request, and even if they did it's highly unlikely the would become a
bottleneck.
Not doing so has a few gotchas when working with relations, particularly
with records which are not stored in the database.
I'm excluding the related content file because it's got a very peculiar
relationship with itself: the `has_one :opposite_related_content` has no
inverse; the relation itself is its inverse. It's a false positive since
the inverse condition is true:
```
content.opposite_related_content.opposite_related_content.object_id ==
content.object_id
```
We were inconsistent on this one. I consider it particularly useful when
a method starts with a `return` statement.
In other cases, we probably shouldn't have a guard rule in the middle of
a method in any case, but that's a different refactoring.
We were very inconsistent regarding these rules.
Personally I prefer no empty lines around blocks, clases, etc... as
recommended by the Ruby style guide [1], and they're the default values
in rubocop, so those are the settings I'm applying.
The exception is the `private` access modifier, since we were leaving
empty lines around it most of the time. That's the default rubocop rule
as well. Personally I don't have a strong preference about this one.
[1] https://rubystyle.guide/#empty-lines-around-bodies
We were already using `find_by` most of the time.
Since there are false positives related to our `find_by_slug_or_id!` and
`find_by_manger_login` methods, which cannot be replaced with `find_by`,
I'm adding it indicating the "refactor" severity.
Having exceptions is better than having silent bugs.
There are a few methods I've kept the same way they were.
The `RelatedContentScore#score_with_opposite` method is a bit peculiar:
it creates scores for both itself and the opposite related content,
which means the opposite related content will try to create the same
scores as well.
We've already got a test to check `Budget::Ballot#add_investment` when
creating a line fails ("Edge case voting a non-elegible investment").
Finally, the method `User#send_oauth_confirmation_instructions` doesn't
update the record when the email address isn't already present, leading
to the test "Try to register with the email of an already existing user,
when an unconfirmed email was provided by oauth" fo fail if we raise an
exception for an invalid user. That's because updating a user's email
doesn't update the database automatically, but instead a confirmation
email is sent.
There are also a few false positives for classes which don't have bang
methods (like the GraphQL classes) or destroying attachments.
For these reasons, I'm adding the rule with a "Refactor" severity,
meaning it's a rule we can break if necessary.
Usually when we use `try` we actually mean `try!`, which is the same as
the safe navigation operator. However, there are a few cases where we
actually mean to execute a method if the object responds to that method.
In those cases using `try` would actually be OK, but in order to avoid
confusion as to whether we mean to check for `respond_to?` or we mean to
use safe navigation, I'm removing all usages of `try`.
Sanitizing descriptions before saving a record has a few drawbacks:
1. It makes the application rely on data being safe in the database. If
somehow dangerous data enters the database, the application will be
vulnerable to XSS attacks
2. It makes the code complicated
3. It isn't backwards compatible; if we decide to disallow a certain
HTML tag in the future, we'd need to sanitize existing data.
On the other hand, sanitizing the data in the view means we don't need
to triple-check dangerous HTML has already been stripped when we see the
method `auto_link_already_sanitized_html`, since now every time we use
it we sanitize the text in the same line we call this method.
We could also sanitize the data twice, both when saving to the database
and when displaying values in the view. However, doing so wouldn't make
the application safer, since we sanitize text introduced through
textarea fields but we don't sanitize text introduced through input
fields.
Finally, we could also overwrite the `description` method so it
sanitizes the text. But we're already introducing Globalize which
overwrites that method, and overwriting it again is a bit too confusing
in my humble opinion. It can also lead to hard-to-debug behaviour.
Now we take into consideration locales persisted but marked for
destruction to complete some logic and to be able to show best
translations on different situations.
In order to not allow users to remove all persited
translations from any resource. A few exceptions were
added:
* Does not apply to globalizable models without
translatable attributes required
* Make a copy of main model error on current translations to be more realistic
We have changed validate_translation method on Globalize concern.
The objective is skip length validations when locale is distinct to
default_locale.
First we force apply :length validations when locale is equal to
default_locale. After we reject :length from options and apply rest
of validation options only when we have more than 1 options.
Ej: options = { length: "maximum: 10" }
When reject :length option in this example, options is equal to a
empty hash and we cant execute validations.
As we cannot order budget investments by any translatable field through
AR queries we are doing the same using ruby Array sort method and doing
array pagination manually with Kaminari 'paginate_array' helper method.
This method will be used by any translatable model that uses pg_search
feature so it's better to have it within globalizable model concern so
all translatable models can use it.
Paranoia is activated on translation classes by reflection, this is
making Rails to load translation classes before to execute migration
that adds the new column.
With this extra check Rails will not execute this code until translation
table has this column created.
This table will store which reports (stats, results, ...) will be shown
for a certain process (polls, budgets, ...).
Note Rails fails to save a poll and its report when both are new records
if we add a `validate :process, presence: true` rule. Since it caused a
lot of trouble when creating records for tests during factories rule
completely. Instead, I've created the `results_enabled=` and
`stats_enabled=` methods, so tests are easier to set up, while also
automatically creating a report if it doesn't already exist. This also
decouples form structure and database implemenation.
Originally I named this table `enabled_reports` and instead of having
`stats` and `results` columns, it had an `enabled` column and a `kind`
column, which would be set to "stats" or "results". However, although
that table would allow us to add arbitrary reports easily, I found the
way we had to handle the `has_many` relationship was a bit too complex.
These methods are only used while stats are being generated; once stats
are generated, they aren't used anymore. So there's no need to store
them using the Dalli cache.
Furthermore, there are polls (and even budgets) with hundreds of
thousands of participants. Calculating stats for them takes a very long
time because we can't store all those records in the Dalli cache.
However, since these records aren't used once the stats are generated,
we can store them in an instance variable while we generate the stats,
speeding up the process.
We need a way to manually expire the cache for a budget or poll without
expiring the cache of every budget or poll.
Using the `updated_at` column would be dangerous because most of the
times we update a budget or a poll, we don't need to regenerate their
stats.
We've considered adding a `stats_updated_at` column to each of these
tables. However, in that case we would also need to add a similar column
in the future to every process type whose stats we want to generate.
If users participated and were hidden after participating, we should
still count them in the participants stats.
In the tests, we set users' `hidden_at` attribute before they vote.
Although in real life they would vote first and then they would be
hidden, I've written the tests like this for the sake of simplicity.
Even if this class looks very simple now, we're trying a few things
related to these stats. Having a class for it makes future changes
easier and, if there weren't any future changes, at least it makes
current experiments easier.
Note we keep the method `participants_by_geozone` to return a hash
because we're caching the stats and storing GeozoneStats objects would
need a lot more memory and we would get an error.
So if we don't have information regarding gender, age or geozone, stats
regarding those topics will not be shown.
Note we're using `spec/models/statisticable_spec.rb` because having the
same file in `spec/models/concerns` caused the tests to be executed
twice.
Also note the implementation behind the `gender?`, `age?` and `geozone?`
methods is a bit primitive. We might need to make it more robust in the
future.
It will make it far easier to call other methods on the stats object,
and we're already caching the methods.
We had to remove the view fragment caching because the stats object
isn't as easy to cache. The good thing about it is the view will
automatically be updated when we change logic regarding which stats to
show, and the methods taking long to execute are cached in the model.
For now we think showing them would be showing too much data and it
would be a bit confusing.
I've been tempted to just remove the view and keep the methods in the
model in case they're used by other institutions using CONSUL. However,
it's probably better to wait until we're asked to re-implement them, and
in the meantime we don't maintain code nobody uses. The code wasn't that
great to start with (I know it because I wrote it).
