consul/grecia
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Replace ahoy events with real data

Browse Source 
We were tracking some events with Ahoy, but in an inconsistent way. For
example, we were tracking when a debate was created, but (probably
accidentally) we were only tracking proposals when they were created
from the management section. For budget investments and their supports,
we weren't using Ahoy events but checking their database tables instead.
And we were only using ahoy events for the charts; for the other stats,
we were using the real data.

While we could actually fix these issues and start tracking events
correctly, existing production data would remain broken because we
didn't track a certain event when it happened. And, besides, why should
we bother, for instance, to track when a debate is created, when we can
instead access that information in the debates table?

There are probably some features related to tracking an event and their
visits, but we weren't using them, and we were storing more user data
than we needed to.

So we're removing the track events, allowing us to simplify the code and
make it more consistent. We aren't removing the `ahoy_events` table in
case existing Consul Democracy installations use it, but we'll remove it
after releasing version 2.2.0 and adding a warning in the release notes.

This change fixes the proposal created chart, since we were only
tracking proposals created in the management section, and opens the
possibility to add more charts in the future using data we didn't track
with Ahoy.

Also note the "Level 2 user Graph" test wasn't testing the graph, so
we're changing it in order to test it. We're also moving it next to the
other graphs test and, since we were tracking the event when we were
confirming the phone, we're renaming to "Level 3 users".

Finally, note that, since we were tracking events when something was
created, we're including the `with_hidden` scope. This is also
consistent with the other stats shown in the admin section as well as
the public stats.
This commit is contained in:
Javi Martín
2024-04-23 23:58:02 +02:00
parent 448775a5e9
commit f7e2d724dd
27 changed files with 178 additions and 242 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
app/components/admin/stats/budget_supporting_component.html.erb
View File

@@ -30,7 +30,7 @@
</div>
</div>
<%= render Admin::Stats::ChartComponent.new(name: "user_supported_budgets", event: "", count: user_count) %>
<%= render Admin::Stats::ChartComponent.new(chart) %>
<table class="investment-projects-summary user-count-by-heading">
<thead>

4
app/components/admin/stats/budget_supporting_component.rb
View File

@@ -28,4 +28,8 @@ class Admin::Stats::BudgetSupportingComponent < ApplicationComponent
[heading, headings_stats[heading.id][:total_participants_support_phase]]
end
end
def chart
@chart ||= Ahoy::Chart.new("user_supported_budgets")
end
end

4
app/components/admin/stats/chart_component.html.erb
View File

@@ -1,4 +1,4 @@
<div id="graph" class="small-12 column">
<h2><%= t "admin.stats.graph.#{name || event}" %> (<%= count %>)</h2>
<%= chart_tag id: name, event: event %>
<h2><%= t "admin.stats.graph.#{event}" %> (<%= count %>)</h2>
<%= chart_tag %>
</div>

26
app/components/admin/stats/chart_component.rb
View File

@@ -1,25 +1,21 @@
class Admin::Stats::ChartComponent < ApplicationComponent
attr_reader :name, :event, :count
attr_reader :chart
def initialize(name:, event:, count:)
@name = name
@event = event
@count = count
def initialize(chart)
@chart = chart
end
private
def chart_tag(opt = {})
opt[:data] ||= {}
opt[:data][:graph] = admin_api_stats_path(chart_data(opt))
tag.div(**opt)
def count
chart.count
end
def chart_data(opt = {})
if opt[:id].present?
{ opt[:id] => true }
elsif opt[:event].present?
{ event: opt[:event] }
end
def event
chart.event_name
end
def chart_tag
tag.div("data-graph": admin_api_stats_path(event: event))
end
end

27
app/controllers/admin/api/stats_controller.rb
View File

@@ -1,30 +1,9 @@
class Admin::Api::StatsController < Admin::Api::BaseController
def show
if params[:event].blank? &&
params[:visits].blank? &&
params[:budget_investments].blank? &&
params[:user_supported_budgets].blank?
return render json: {}, status: :bad_request
end
ds = Ahoy::DataSource.new
if params[:event].present?
ds.add params[:event].titleize, Ahoy::Chart.new(params[:event]).group_by_day(:time).count
render json: Ahoy::Chart.new(params[:event]).data_points
else
render json: {}, status: :bad_request
end
if params[:visits].present?
ds.add "Visits", Visit.group_by_day(:started_at).count
end
if params[:budget_investments].present?
ds.add "Budget Investments", Budget::Investment.group_by_day(:created_at).count
end
if params[:user_supported_budgets].present?
ds.add "User supported budgets",
Vote.where(votable_type: "Budget::Investment").group_by_day(:updated_at).count
end
render json: ds.build
end
end

9
app/controllers/admin/stats_controller.rb
View File

@@ -30,14 +30,7 @@ class Admin::StatsController < Admin::BaseController
end
def graph
@name = params[:id]
@event = params[:event]
if params[:event]
@count = Ahoy::Chart.new(params[:event]).count
else
@count = params[:count]
end
@chart = Ahoy::Chart.new(params[:event])
end
def proposal_notifications

4
app/controllers/concerns/commentable_actions.rb
View File

@@ -59,10 +59,6 @@ module CommentableActions
private
def track_event
ahoy.track :"#{resource_name}_created", "#{resource_name}_id": resource.id
end
def tag_cloud
TagCloud.new(resource_model, params[:search])
end

1
app/controllers/debates_controller.rb
View File

@@ -24,7 +24,6 @@ class DebatesController < ApplicationController
@debate.author = current_user
if @debate.save
track_event
redirect_to debate_path(@debate), notice: t("flash.actions.create.debate")
else
render :new

7
app/controllers/legislation/annotations_controller.rb
View File

@@ -51,7 +51,6 @@ class Legislation::AnnotationsController < Legislation::BaseController
@annotation = @draft_version.annotations.new(annotation_params)
@annotation.author = current_user
if @annotation.save
track_event
render json: @annotation.to_json
else
render json: @annotation.errors.full_messages, status: :unprocessable_entity
@@ -100,12 +99,6 @@ class Legislation::AnnotationsController < Legislation::BaseController
[:quote, :text, ranges: [:start, :startOffset, :end, :endOffset]]
end
def track_event
ahoy.track :legislation_annotation_created,
legislation_annotation_id: @annotation.id,
legislation_draft_version_id: @draft_version.id
end
def convert_ranges_parameters
annotation = params[:legislation_annotation]
if annotation && annotation[:ranges] && annotation[:ranges].is_a?(String)

8
app/controllers/legislation/answers_controller.rb
View File

@@ -12,7 +12,6 @@ class Legislation::AnswersController < Legislation::BaseController
if @process.debate_phase.open?
@answer.user = current_user
@answer.save!
track_event
respond_to do |format|
format.js
format.html { redirect_to legislation_process_question_path(@process, @question) }
@@ -35,11 +34,4 @@ class Legislation::AnswersController < Legislation::BaseController
def allowed_params
[:legislation_question_option_id]
end
def track_event
ahoy.track :legislation_answer_created,
legislation_answer_id: @answer.id,
legislation_question_option_id: @answer.legislation_question_option_id,
legislation_question_id: @answer.legislation_question_id
end
end

1
app/controllers/management/proposals_controller.rb
View File

@@ -17,7 +17,6 @@ class Management::ProposalsController < Management::BaseController
published_at: Time.current))
if @resource.save
track_event
redirect_path = url_for(controller: controller_name, action: :show, id: @resource.id)
redirect_to redirect_path, notice: t("flash.actions.create.#{resource_name.underscore}")
else

1
app/controllers/verification/sms_controller.rb
View File

@@ -28,7 +28,6 @@ class Verification::SmsController < ApplicationController
@sms = Verification::Sms.new(sms_params.merge(user: current_user))
if @sms.verified?
current_user.update!(confirmed_phone: current_user.unconfirmed_phone)
ahoy.track(:level_2_user, user_id: current_user.id) rescue nil
if VerifiedUser.phone?(current_user)
current_user.update(verified_at: Time.current)

6
app/helpers/stats_helper.rb
View File

@@ -1,10 +1,4 @@
module StatsHelper
def budget_investments_chart_tag(opt = {})
opt[:data] ||= {}
opt[:data][:graph] = admin_api_stats_path(budget_investments: true)
tag.div(**opt)
end
def number_to_stats_percentage(number, options = {})
number_to_percentage(number, { strip_insignificant_zeros: true, precision: 2 }.merge(options))
end

47
app/models/ahoy/chart.rb
View File

@@ -1,20 +1,59 @@
module Ahoy
class Chart
attr_reader :event_name
delegate :count, :group_by_day, to: :events
delegate :count, to: :records
def initialize(event_name)
@event_name = event_name
end
def self.active_event_names
Ahoy::Event.distinct.order(:name).pluck(:name)
event_names_with_collections.select { |name, collection| collection.any? }.keys
end
def self.event_names_with_collections
{
budget_investment_created: Budget::Investment.with_hidden,
debate_created: Debate.with_hidden,
legislation_annotation_created: Legislation::Annotation.with_hidden,
legislation_answer_created: Legislation::Answer.with_hidden,
level_3_user: User.with_hidden.level_three_verified,
proposal_created: Proposal.with_hidden
}
end
def data_points
ds = Ahoy::DataSource.new
ds.add event_name.to_s.titleize, records_by_day.count
ds.build
end
private
def events
Ahoy::Event.where(name: event_name)
def records
case event_name.to_sym
when :user_supported_budgets
Vote.where(votable_type: "Budget::Investment")
when :visits
Visit.all
else
self.class.event_names_with_collections[event_name.to_sym]
end
end
def records_by_day
raise "Unknown event #{event_name}" unless records.respond_to?(:group_by_day)
records.group_by_day(date_field)
end
def date_field
if event_name.to_sym == :level_3_user
:verified_at
else
:created_at
end
end
end
end

1
app/models/visit.rb
View File

@@ -1,4 +1,5 @@
class Visit < ApplicationRecord
alias_attribute :created_at, :started_at
has_many :ahoy_events, class_name: "Ahoy::Event"
belongs_to :user
end

2
app/views/admin/stats/graph.html.erb
View File

@@ -4,4 +4,4 @@
<%= back_link_to admin_stats_path %>
<%= render Admin::Stats::ChartComponent.new(name: @name, event: @event, count: @count) %>
<%= render Admin::Stats::ChartComponent.new(@chart) %>

11
app/views/admin/stats/show.html.erb
View File

@@ -1,7 +1,3 @@
<% content_for :head do %>
<%= javascript_include_tag "stat_graphs", "data-turbolinks-track" => "reload" %>
<% end %>
<div id="stats" class="stats">
<div class="row">
<div class="small-12 column">
@@ -28,7 +24,7 @@
<div class="small-12 medium-3 column">
<p class="featured">
<%= link_to t("admin.stats.show.summary.visits"),
graph_admin_stats_path(id: "visits", count: @visits) %> <br>
graph_admin_stats_path(event: "visits") %> <br>
<span class="number"><%= number_with_delimiter(@visits) %></span>
</p>
<p>
@@ -115,11 +111,6 @@
</div>
<%= render Admin::Stats::EventLinksComponent.new(@event_names) %>
<% if feature?(:budgets) %>
<h2><%= t "admin.stats.show.budgets_title" %></h2>
<%= budget_investments_chart_tag id: "budget_investments" %>
<% end %>
</div>
</div>
</div>

4
config/locales/en/admin.yml
View File

@@ -1480,7 +1480,6 @@ en:
verified_users_who_didnt_vote_proposals: Verified users who didn't votes proposals
visits: Visits
votes: Total votes
budgets_title: Participatory budgeting
participatory_budgets: Participatory Budgets
direct_messages: Direct messages
proposal_notifications: Proposal notifications
@@ -1488,9 +1487,10 @@ en:
polls: Polls
sdg: SDG
graph:
budget_investment_created: Budget investments created
debate_created: Debates
visit: Visits
level_2_user: Level 2 users
level_3_user: Level 3 users
proposal_created: Citizen proposals
title: Graphs
budgets:

4
config/locales/es/admin.yml
View File

@@ -1480,7 +1480,6 @@ es:
verified_users_who_didnt_vote_proposals: Usuarios verificados que no han votado propuestas
visits: Visitas
votes: Votos
budgets_title: Presupuestos participativos
participatory_budgets: Presupuestos Participativos
direct_messages: Mensajes directos
proposal_notifications: Notificaciones de propuestas
@@ -1488,9 +1487,10 @@ es:
polls: Votaciones
sdg: ODS
graph:
budget_investment_created: Proyectos de gasto creados
debate_created: Debates
visit: Visitas
level_2_user: Usuarios nivel 2
level_3_user: Usuarios nivel 3
proposal_created: Propuestas Ciudadanas
title: Gráficos
budgets:

48
spec/controllers/admin/api/stats_controller_spec.rb
View File

@@ -17,51 +17,19 @@ describe Admin::Api::StatsController, :admin do
time_2 = Time.zone.local(2015, 01, 02)
time_3 = Time.zone.local(2015, 01, 03)
create(:ahoy_event, name: "foo", time: time_1)
create(:ahoy_event, name: "foo", time: time_1)
create(:ahoy_event, name: "foo", time: time_2)
create(:ahoy_event, name: "bar", time: time_1)
create(:ahoy_event, name: "bar", time: time_3)
create(:ahoy_event, name: "bar", time: time_3)
create(:proposal, created_at: time_1)
create(:proposal, created_at: time_1)
create(:proposal, created_at: time_2)
create(:debate, created_at: time_1)
create(:debate, created_at: time_3)
create(:debate, created_at: time_3)
end
it "returns single events formated for working with c3.js" do
get :show, params: { event: "foo" }
get :show, params: { event: "proposal_created" }
expect(response).to be_ok
expect(response.parsed_body).to eq "x" => ["2015-01-01", "2015-01-02"], "Foo" => [2, 1]
end
end
context "visits present" do
it "returns visits formated for working with c3.js" do
time_1 = Time.zone.local(2015, 01, 01)
time_2 = Time.zone.local(2015, 01, 02)
create(:visit, started_at: time_1)
create(:visit, started_at: time_1)
create(:visit, started_at: time_2)
get :show, params: { visits: true }
expect(response).to be_ok
expect(response.parsed_body).to eq "x" => ["2015-01-01", "2015-01-02"], "Visits" => [2, 1]
end
end
context "budget investments present" do
it "returns budget investments formated for working with c3.js" do
time_1 = Time.zone.local(2017, 04, 01)
time_2 = Time.zone.local(2017, 04, 02)
create(:budget_investment, created_at: time_1)
create(:budget_investment, created_at: time_2)
create(:budget_investment, created_at: time_2)
get :show, params: { budget_investments: true }
expect(response).to be_ok
expect(response.parsed_body).to eq "x" => ["2017-04-01", "2017-04-02"], "Budget Investments" => [1, 2]
expect(response.parsed_body).to eq "x" => ["2015-01-01", "2015-01-02"], "Proposal Created" => [2, 1]
end
end
end

28
spec/controllers/debates_controller_spec.rb
View File

@@ -9,34 +9,6 @@ describe DebatesController do
end
end
describe "POST create" do
before do
InvisibleCaptcha.timestamp_enabled = false
end
after do
InvisibleCaptcha.timestamp_enabled = true
end
it "creates an ahoy event" do
debate_attributes = {
terms_of_service: "1",
translations_attributes: {
"0" => {
title: "A sample debate",
description: "this is a sample debate",
locale: "en"
}
}
}
sign_in create(:user)
post :create, params: { debate: debate_attributes }
expect(Ahoy::Event.where(name: :debate_created).count).to eq 1
expect(Ahoy::Event.last.properties["debate_id"]).to eq Debate.last.id
end
end
describe "PUT mark_featured" do
it "ignores query parameters" do
debate = create(:debate)

21
spec/controllers/legislation/annotations_controller_spec.rb
View File

@@ -36,27 +36,6 @@ describe Legislation::AnnotationsController do
end
let(:user) { create(:user, :level_two) }
it "creates an ahoy event" do
sign_in user
post :create, params: {
process_id: legal_process.id,
draft_version_id: draft_version.id,
legislation_annotation: {
"quote" => "ipsum",
"ranges" => [{
"start" => "/p[1]",
"startOffset" => 6,
"end" => "/p[1]",
"endOffset" => 11
}],
"text" => "una anotacion"
}
}
expect(Ahoy::Event.where(name: :legislation_annotation_created).count).to eq 1
expect(Ahoy::Event.last.properties["legislation_annotation_id"]).to eq Legislation::Annotation.last.id
end
it "does not create an annotation if the draft version is a final version" do
sign_in user

14
spec/controllers/legislation/answers_controller_spec.rb
View File

@@ -10,20 +10,6 @@ describe Legislation::AnswersController do
let(:question_option) { create(:legislation_question_option, question: question, value: "Yes") }
let(:user) { create(:user, :level_two) }
it "creates an ahoy event" do
sign_in user
post :create, params: {
process_id: legal_process.id,
question_id: question.id,
legislation_answer: {
legislation_question_option_id: question_option.id
}
}
expect(Ahoy::Event.where(name: :legislation_answer_created).count).to eq 1
expect(Ahoy::Event.last.properties["legislation_answer_id"]).to eq Legislation::Answer.last.id
end
it "creates an answer if the process debate phase is open" do
sign_in user

6
spec/factories/analytics.rb
View File

@@ -1,10 +1,4 @@
FactoryBot.define do
factory :ahoy_event, class: "Ahoy::Event" do
id { SecureRandom.uuid }
time { DateTime.current }
sequence(:name) { |n| "Event #{n} type" }
end
factory :visit do
id { SecureRandom.uuid }
started_at { DateTime.current }

76
spec/models/ahoy/chart_spec.rb Normal file
View File

@@ -0,0 +1,76 @@
require "rails_helper"
describe Ahoy::Chart do
describe "#data_points" do
it "raises an exception for unknown events" do
chart = Ahoy::Chart.new(:mystery)
expect { chart.data_points }.to raise_exception "Unknown event mystery"
end
it "returns data associated with the event" do
time_1 = Time.zone.local(2015, 01, 01)
time_2 = Time.zone.local(2015, 01, 02)
time_3 = Time.zone.local(2015, 01, 03)
create(:proposal, created_at: time_1)
create(:proposal, created_at: time_1)
create(:proposal, created_at: time_2)
create(:debate, created_at: time_1)
create(:debate, created_at: time_3)
chart = Ahoy::Chart.new(:proposal_created)
expect(chart.data_points).to eq x: ["2015-01-01", "2015-01-02"], "Proposal Created" => [2, 1]
end
it "accepts strings as the event name" do
create(:proposal, created_at: Time.zone.local(2015, 01, 01))
create(:debate, created_at: Time.zone.local(2015, 01, 02))
chart = Ahoy::Chart.new("proposal_created")
expect(chart.data_points).to eq x: ["2015-01-01"], "Proposal Created" => [1]
end
it "returns visits data for the visits event" do
time_1 = Time.zone.local(2015, 01, 01)
time_2 = Time.zone.local(2015, 01, 02)
create(:visit, started_at: time_1)
create(:visit, started_at: time_1)
create(:visit, started_at: time_2)
chart = Ahoy::Chart.new(:visits)
expect(chart.data_points).to eq x: ["2015-01-01", "2015-01-02"], "Visits" => [2, 1]
end
it "returns user supports for the user_supported_budgets event" do
time_1 = Time.zone.local(2017, 04, 01)
time_2 = Time.zone.local(2017, 04, 02)
create(:vote, votable: create(:budget_investment), created_at: time_1)
create(:vote, votable: create(:budget_investment), created_at: time_2)
create(:vote, votable: create(:budget_investment), created_at: time_2)
create(:vote, votable: create(:proposal), created_at: time_2)
chart = Ahoy::Chart.new(:user_supported_budgets)
expect(chart.data_points).to eq x: ["2017-04-01", "2017-04-02"], "User Supported Budgets" => [1, 2]
end
it "returns level three verified dates for the level_3_user event" do
time_1 = Time.zone.local(2001, 01, 01)
time_2 = Time.zone.local(2001, 01, 02)
create(:user, :level_two, level_two_verified_at: time_1)
create(:user, :level_three, verified_at: time_2)
create(:user, :level_three, verified_at: time_2, level_two_verified_at: time_1)
chart = Ahoy::Chart.new(:level_3_user)
expect(chart.data_points).to eq x: ["2001-01-02"], "Level 3 User" => [2]
end
end
end

21
spec/models/ahoy/data_source_spec.rb
View File

@@ -2,18 +2,9 @@ require "rails_helper"
describe Ahoy::DataSource do
describe "#build" do
before do
time_1 = Time.zone.local(2015, 01, 01)
time_2 = Time.zone.local(2015, 01, 02)
time_3 = Time.zone.local(2015, 01, 03)
create(:ahoy_event, name: "foo", time: time_1)
create(:ahoy_event, name: "foo", time: time_1)
create(:ahoy_event, name: "foo", time: time_2)
create(:ahoy_event, name: "bar", time: time_1)
create(:ahoy_event, name: "bar", time: time_3)
create(:ahoy_event, name: "bar", time: time_3)
end
let(:january_first) { Time.zone.local(2015, 01, 01) }
let(:january_second) { Time.zone.local(2015, 01, 02) }
let(:january_third) { Time.zone.local(2015, 01, 03) }
it "works without data sources" do
ds = Ahoy::DataSource.new
@@ -22,14 +13,14 @@ describe Ahoy::DataSource do
it "works with single data sources" do
ds = Ahoy::DataSource.new
ds.add "foo", Ahoy::Event.where(name: "foo").group_by_day(:time).count
ds.add "foo", { january_first => 2, january_second => 1 }
expect(ds.build).to eq :x => ["2015-01-01", "2015-01-02"], "foo" => [2, 1]
end
it "combines data sources" do
ds = Ahoy::DataSource.new
ds.add "foo", Ahoy::Event.where(name: "foo").group_by_day(:time).count
ds.add "bar", Ahoy::Event.where(name: "bar").group_by_day(:time).count
ds.add "foo", { january_first => 2, january_second => 1 }
ds.add "bar", { january_first => 1, january_third => 2 }
expect(ds.build).to eq :x => ["2015-01-01", "2015-01-02", "2015-01-03"],
"foo" => [2, 1, 0],
"bar" => [1, 0, 2]

37
spec/system/admin/stats_spec.rb
View File

@@ -72,18 +72,6 @@ describe "Stats", :admin do
expect(page).to have_content "UNVERIFIED USERS\n1"
expect(page).to have_content "TOTAL USERS\n1"
end
scenario "Level 2 user Graph" do
create(:geozone)
visit account_path
click_link "Verify my account"
verify_residence
confirm_phone
visit admin_stats_path
expect(page).to have_content "LEVEL TWO USERS\n1"
end
end
describe "Budget investments" do
@@ -150,15 +138,9 @@ describe "Stats", :admin do
end
end
context "graphs" do
scenario "event graphs", :with_frozen_time do
visit new_debate_path
fill_in_new_debate_title with: "A title for a debate"
fill_in_ckeditor "Initial debate text", with: "This is very important because..."
check "debate_terms_of_service"
click_button "Start a debate"
expect(page).to have_content "Debate created successfully."
describe "graphs", :with_frozen_time do
scenario "event graphs" do
create(:debate)
visit admin_stats_path
@@ -172,6 +154,19 @@ describe "Stats", :admin do
expect(page).to have_content Date.current.strftime("%Y-%m-%d")
end
end
scenario "Level 3 user Graph" do
create(:user, :level_three)
visit admin_stats_path
click_link "level_3_user"
expect(page).to have_content "Level 3 User (1)"
within("#graph") do
expect(page).to have_content Date.current.strftime("%Y-%m-%d")
end
end
end
context "Proposal notifications" do