For reasons that might or might not affect production installations, the
test checking simultaneous requests to create poll voters in the
officing voters controller wasn't behaving as expected.
The expected behavior, since commit 9a8bfac5b, is that the second
request reaching the `with_lock` part of the code waits for the first
request to finish and so this second request raises an
`ActiveRecord::RecordInvalid` exception when trying to save a voter with
the same poll and the same user as the first one.
However, 95% of the time that wasn't the case. Instead, when entering
the `@user.with_lock` block, the second request would replace its
`@voter` object with the `@voter` object saved in the same request, so
the second call to `save!` would succeed as it would simply update the
existing record.
This is a behavior that we could accept if it were consistent and
happened 100% of the time, but that isn't the case. 5% of the time, we
do get the `ActiveRecord::RecordInvalid` exception. So 5% of the time we
got a failure in the test:
```
1) Officing::VotersController POST create does not create two records
with two simultaneous requests
Failure/Error: @user.with_lock { @voter.save! }
ActiveRecord::RecordInvalid:
Validation failed: User User has already voted
# ./app/controllers/officing/voters_controller.rb:25:in `block in create'
# ./app/controllers/officing/voters_controller.rb:25:in `create'
# ./app/controllers/application_controller.rb:50:in `switch_locale'
# ./spec/controllers/officing/voters_controller_spec.rb:15:in `block (5 levels) in <top (required)>'
```
So we're changing the `with_lock` block so it includes the
initialization of the object. This way, we get the
`ActiveRecord::RecordInvalid` exception 100% of the time.
Note that in commit 9a8bfac5b we also rescued the
`ActionDispatch::IllegalStateError` exceptions. I'm not why we were
getting those exceptions when running the tests, and I'm not sure
whether we keep getting after these changes, but it doesn't really
matter. The reason is that in Consul Democracy 2.3.0 we're going to add
a unique index to the `poll_voters` table, which (according to the tests
done in the past) will make both the `@user.lock` block and rescuing the
`ActionDispatch::IllegalStateError` unnecessary.
So, in other words, these changes will never make it to production
because this part of the code will be changed again before releasing
version 2.3.0.
When the `multitenancy_management_mode` is enabled.
In order to avoid infinite redirects when regular users try to access
the admin section, we're redirecting to the account page in this case.
Otherwise, the admin section would redirect to the root path, which
would redirect to the admin section, which would redirect to the root
path, and so on.
We only want to render the account link and login items in the header.
And we want only render the Multitenancy and Administrators sections in
the admin sidebar.
We include the administrators management so it's possible to give
permissions to other users to manage tenants.
In order to restrict access to other sections by typing the URL or
following a link, we're only enabling the rest of the routes when we
aren't in the multitenancy management mode.
There are many possible ways to implement this feature:
* Adding a custom middleware
* Using rack-attack with a blocklist
* Using routes constraints
We're choosing to use a controller concern with a redirect because it's
what we do to handle unauthorized cancancan exceptions.
Using a checkbox wasn't very intuitive because checkboxes are
checked/unchecked when clicked on even if there's an error in the
request. Usually, when checkboxes appear on a form, they don't send any
information to the server unless we click a button to send the form.
So we're using a switch instead of a checkbox, like we did to
enable/disable phases in commit 46d8bc4f0.
Note that, since we've got two switches that match the default
`dom_id(record) .toggle-switch` selector, we need to find a way to
differentiate them. We're adding the `form_class` option for that.
Also note that we're now using a separate action and removing the
JavaScript in the `update` action which assumed that AJAX requests to
this action were always related to updating the `visible_to_valuators`
attribute.
This is consistent to what we usually do. Also, we're applying the same
criteria mentioned in commit 72704d776:
> We're also making these actions idempotent, so sending many requests
> to the same action will get the same result, which wasn't the case
> with the `toggle` action. Although it's a low probability case, the
> `toggle` action could result in [selecting an investment] when trying
> to [deselect] it if someone else has [deselected it] it between the
> time the page loaded and the time the admin clicked on the
> "[Selected]" button.
This code isn't used since commit c9f31b8e1.
Since we no longer depend on the content of the `#investments` element
being in a separate partial, we're also moving this element to the
partial itself and adding an HTML class to it, like we usually do.
We're also removing the code that loads all the investments in the
`toggle_selection` action, which wasn't needed since commit 3278b3572,
when we stopped rendering all the investments in this action.
This is consistent to what we usually do. Also, we're applying the same
criteria mentioned in commit 72704d776:
> We're also making these actions idempotent, so sending many requests
> to the same action will get the same result, which wasn't the case
> with the `toggle` action. Although it's a low probability case, the
> `toggle` action could result in [selecting a proposal] when trying to
> [deselect] it if someone else has [deselected it] it between the time
> the page loaded and the time the admin clicked on the "[Selected]"
> button.
This method was added in Rails 7.0 and makes the code slihgtly more
readable.
The downside is that it generates two queries instead of one, so it
might generate some confusion when debugging SQL queries. Its impact on
performance is probably negligible.
This rule was added in rubocop 1.64.0.
For clarity, in order to make it obvious that we're modifying the object
we received, we're excluding the Ahoy initializer, whose code was copied
from the Ahoy documentation.
We're also changing the `Types::BaseObject` class so we don't use a
variable with the same name as the parameter and we don't get a false
positive for this rule.
Until now, we've stored the text of the answer somebody replied to. The
idea was to handle the scenarios where the user voters for an option but
then that option is deleted and restored, or the texts of the options
are accidentally edited and so the option "Yes" is now "Now" and vice
versa.
However, since commit 3a6e99cb8, options can no longer be edited once
the poll starts, so there's no risk of the option changing once somebody
has voted.
This means we can now store the ID of the option that has been voted.
That'll also help us deal with a bug introduced int 673ec075e, since
answers in different locales are not counted as the same answer. Note we
aren't dealing with this bug right now.
We're still keeping (and storing) the answer as well. There are two
reasons for that.
First, we might add an "open answer" type of questions in the future and
use this column for it.
Second, we've still got logic depending on the answer, and we need to be
careful when changing it because there are existing installations where
the answer is present but the option_id is not.
Note that we're using `dependent: nullify`. The reasoning is that, since
we're storing both the option_id and the answer text, we can still use
the answer text when removing the option. In practice, this won't matter
much, though, since we've got a validation rule that makes it impossible
to destroy options once the poll has started.
Also note we're still allowing duplicate records when the option is nil.
We need to do that until we've removed every duplicate record in the
database.
It was confusing to have the action to create an answer in
`QuestionsController#answer` while the action to destroy it was
`AnswersController#destroy`.
The routes for poll questions were accidentally deleted in commit
5bb831e959 when deleting the `:show` action, and restored in commit
9871503c5e. However, the deleted code was:
```
resources :questions, only: [:show], controller: 'polls/questions' (...)
```
While the restored code was:
```
resources :questions, controller: 'polls/questions' (...)
```
Meaning we forgot to add the `only: []` option when restoring the
routes.
We also forgot to remove the `before_action` code when deleting the
`:show` action, so we're removing it now.
Note that, when taking votes from an erased user, since poll answers
don't belong to poll voters, we were not migrating them in the
`take_votes_from` method (and we aren't migrating them now either).
We were using `authorize_resource`, passing it an unnamed parameter.
When that happens, CanCanCan only checks permissions to read that
resource. But, in this case, we want to check the permission to update
that resource before the `update` action.
Most of the time, it doesn't really matter, but, for example, in our
demo we're going to restrict the locales configuration so locales cannot
be updated on the main tenant (but they can be updated on other
tenants).
Having a class named `Poll::Question::Answer` and another class named
`Poll::Answer` was so confusing that no developer working on the project
has ever been capable of remembering which is which for more than a few
seconds.
Furthermore, we're planning to add open answers to polls, and we might
add a reference from the `poll_answers` table to the
`poll_question_answers` table to property differentiate between open
answers and closed answers. Having yet another thing named answer would
be more than what our brains can handle (we know it because we did this
once in a prototype).
So we're renaming `Poll::Question::Answer` to `Poll::Question::Option`.
Hopefully that'll make it easier to remember. The name is also (more or
less) consistent with the `Legislation::QuestionOption` class, which is
similar.
We aren't changing the table or columns names for now in order to avoid
possible issues when upgrading (old code running with the new database
tables/columns after running the migrations but before deployment has
finished, for instance). We might do it in the future.
I've tried not to change the internationalization keys either so
existing translations would still be valid. However, since we have to
change the keys in `activerecord.yml` so methods like
`human_attribute_name` keep working, I'm also changing them in places
where similar keys were used (like `poll_question_answer` or
`poll/question/answer`).
Note that it isn't clear whether we should use `option` or
`question_option` in some cases. In order to keep things simple, we're
using `option` where we were using `answer` and `question_option` where
we were using `question_answer`.
Also note we're adding tests for the admin menu component, since at
first I forgot to change the `answers` reference there and all tests
passed.
This way we can simplify the view, particularly the form. However, we're
still adding some complexity to the form so inputs are inside labels and
so the collection is easier to style with CSS.
We're using different controls depending on the number of available
locales.
When there are only a few locales, the solution is obvious: radio
buttons to select the default language, and checkboxes to select the
available ones are simple and intuitive.
With many languages, showing two consecutive lists of 30 languages could
be confusing, though, particularly on small devices, where scrolling
through both lists could be hard.
So, in this case, we're rendering a <select> to choose the default
language. For selecting the available languages, however, we're sticking
to checkboxes because all the other existing options (like multiple
selects) are hard to use. We think it's OK because the form doesn't have
any additional fields, and there's only one big list of options to
scroll through.
While testing the application, we noticed that if we use the
`admin-fieldset-separator` styles when there's only one fieldset, it's
harder to notice that there's an additional field to select the default
language. So we're only using the `admin-fieldset-separator` styles when
all the fields are grouped in fieldsets.
Regarding the help text for the fieldset, if we leave the help text
outside the <legend> tag, people using screen readers won't hear about
this content. However, if we include it inside the <legend> tag, some
screen readers might read it every time they move to a different
checkbox (or radio button), which can be annoying. Since I don't think
these help messages are really essential, I'm leaving them out of the
<legend> tag. It's also easier to style them if they're outside the
<legend> tag.
Note we're using `display: table` for the labels, for the reasons
mentioned in commit 923c2a7ee.
Also note that, when there's only one available locale, this section is
useless. In this case, we aren't disabling it for now because there's a
chance people see it in the official Consul Democracy demo and then
wonder why it isn't available on their installation. We might disable it
in the future, though.
Note that, for everything to work consistently, we need to make sure
that the default locale is one of the available locales.
Also note that we aren't overwriting the `#save ` method set by
globalize. I didn't feel too comfortable changing a monkey-patch which
ideally shouldn't be there in the first place, I haven't found a case
where `Globalize.locale` is `nil` (since it defaults to `I18n.locale`,
which should never be `nil`), so using `I18n.default_locale` probably
doesn't affect us.
Note that, currently, we take these settings from the database but we
don't provide a way to edit them through the admin interface, so the
locales must be manually introduced through a Rails console.
While we did consider using a comma-separated list, we're using spaces
in order to be consistent with the way we store the allowed content
types settings.
The `enabled_locales` nomenclature, which contrasts with
`available_locales`, is probably subconsciously based on similar
patterns like the one Nginx uses to enable sites.
Note that we aren't using `Setting.enabled_locales` in the globalize
initializer when setting the fallbacks. This means the following test
(which we could add to the shared globalizable examples) would fail:
```
it "Falls back to an enabled locale if the fallback is not enabled" do
Setting["locales.default"] = "en"
Setting["locales.enabled"] = "fr en"
allow(I18n.fallbacks).to receive(:[]).and_return([:fr, :es])
Globalize.set_fallbacks_to_all_available_locales
I18n.with_locale(:fr) do
expect(record.send(attribute)).to eq "In English"
end
end
```
The reason is that the code making this test pass could be:
```
def Globalize.set_fallbacks_to_all_available_locales
Globalize.fallbacks = I18n.available_locales.index_with do |locale|
((I18n.fallbacks[locale] & Setting.enabled_locales) + Setting.enabled_locales).uniq
end
end
```
However, this would make it impossible to run `rake db:migrate` on new
applications because the initializer would try to load the `Setting`
model but the `settings` table wouldn't exist at that point.
Besides, this is a really rare case that IMHO we don't need to support.
For this scenario, an installation would have to enable a locale, create
records with contents in that locale, then disable that locale and have
that locale as a fallback for a language where content for that record
wasn't created. If that happened, it would be solved by creating content
for that record in every enabled language.
In the case of the `edit` action, we're using
`load_and_authorize_resource`, which will always return a
`SiteCustomization::ContentBlock`. In the case of
`edit_heading_content_block`, we're using `Budget::ContentBlock.find`,
which always returns a `Budget::ContentBlock` (or raises an exception).
So, in both cases, the condition to assign `@selected_content_block` can
be removed.
The code works without this action because both the route and the view
are specified, and by default Rails renders the view when there's no
action defined.
However, the code is easier to understand if the action is present in
the controller, which is what we do most of the time.
There was an edge case where a user could configure a locale and then
the application would change the locales to that one would no longer be
available. In that case, we were getting a `I18n::InvalidLocale`
exception when accessing the subscriptions page.
So now, we're defaulting to `I18n.locale`. Note we're using
`I18n.locale`instead of `I18n.default_locale` because `set_user_locale`
is called inside the `switch_locale` block in `ApplicationController`,
which already sets `I18n.locale` based on `I18n.default_locale`.
This way it'll be easier to change it while checking we haven't broken
existing behavior.
While writing the tests, I noticed we were sometimes storing a symbol in
the session while sometimes we were storing a string. So we're adding a
`to_s` call so we always store a string in the session.
Since now generating stats (assuming the results aren't in the cache)
only takes a few seconds even when there are a hundred thousand
participants, as opposed to the several minutes it took to generate them
when we introduced the Cron job, we can simply generate the stats during
the first request to the stats page.
Note that, in order to avoid creating a temporary table when the stats
are cached, we're making sure we only create this table when we need to.
Otherwise, we could spend up to 1 second on every request to the stats
page creating a table that isn't going to be used.
Also note we're using an instance variable to check whether we're
creating a table; I tried to use `table_exists?`, but it didn't work. I
wonder whether `table_exists?` doesn't detect temporary tables.
This page isn't linked from anywhere and most Consul Democracy
installations don't even know it exists, so it's useless for most
people.
If we ever bring it back, we should at least add a link pointing to this
page.
Now that we've moved the logic to generate the events data to a model,
and we've got access to the model in the component rendering the chart,
we can render the data inside the chart instead of doing an extra AJAX
request to get the same data.
Originally, this was problaby done this way so the page wouldn't take
several seconds to load while preparing the data for the chart when
there are thousands of dates being displayed. With an AJAX call, the
page would load as fast as usual, and then the chart would render after
a few seconds. However, we can have an even better performance
improvement in this scenario if we use a Set instead of an Array. The
method `Array#include?`, which we were calling for every date in the
data, is much slower that `Set#merge`. So now both the page and the
chart load as fast as expected.
We could also use something like:
```
def add
(...)
shared_keys.push(*collection.keys)
end
def build
(...)
shared_keys.uniq.each do |k|
(...)
end
def shared_keys
@shared_keys ||= []
end
```
Or other approaches to avoid using `Array#include?`. The performance
would be similar to the one we get when using `Set`. We're using a `Set`
because it makes more obvious that `shared_keys` is supposed to contain
unique elements.
We've had some tests failing in the past due to these AJAX requests
being triggered automatically during the tests and no expectations
checking the requests have finished, so now we're reducing the amount of
flaky tests.
We were tracking some events with Ahoy, but in an inconsistent way. For
example, we were tracking when a debate was created, but (probably
accidentally) we were only tracking proposals when they were created
from the management section. For budget investments and their supports,
we weren't using Ahoy events but checking their database tables instead.
And we were only using ahoy events for the charts; for the other stats,
we were using the real data.
While we could actually fix these issues and start tracking events
correctly, existing production data would remain broken because we
didn't track a certain event when it happened. And, besides, why should
we bother, for instance, to track when a debate is created, when we can
instead access that information in the debates table?
There are probably some features related to tracking an event and their
visits, but we weren't using them, and we were storing more user data
than we needed to.
So we're removing the track events, allowing us to simplify the code and
make it more consistent. We aren't removing the `ahoy_events` table in
case existing Consul Democracy installations use it, but we'll remove it
after releasing version 2.2.0 and adding a warning in the release notes.
This change fixes the proposal created chart, since we were only
tracking proposals created in the management section, and opens the
possibility to add more charts in the future using data we didn't track
with Ahoy.
Also note the "Level 2 user Graph" test wasn't testing the graph, so
we're changing it in order to test it. We're also moving it next to the
other graphs test and, since we were tracking the event when we were
confirming the phone, we're renaming to "Level 3 users".
Finally, note that, since we were tracking events when something was
created, we're including the `with_hidden` scope. This is also
consistent with the other stats shown in the admin section as well as
the public stats.
The only way to use campaigns is to manually insert them in the
database, which IMHO isn't very practical.
We're going to change every piece of code that generates an Ahoy event
and, in this case, the easiest way to change the Campaing model so it
doesn't use Ahoy events is to simply remove it.
Note we're keeping the database tables until we release a new version,
just in case some Consul Democracy installations are using them. We'll
inform in the release notes that we'll remove the campaigns table after
the release, so existing installations using the `campaigns` table can
move the data somewhere else before we remove the table.
We were only using it in one place: the debates controller. All the
other controllers including CommentableActions were overwriting this
action, except the ones in the admin area, where creating proposals,
debates or investments isn't possible.
Note this means that, most of the times, we weren't tracking events
creating a resource.
Also note that since, as mentioned in commit 3752fef6b, there are no
geozones in the debates form, we don't have to load them when creating a
debate fails due to validation rules.
