Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion app/controllers/concerns/dedupable.rb
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ def dedupe_perform
# being deleted as a conflict. Atomic: a failed save rolls the merge back.
ActiveRecord::Base.transaction do
deduper.merge(record_to_keep, record_to_delete)
# After the duplicate's records have moved onto the keeper, reconcile any that
# need it (e.g. re-crediting a moved scholarship to the keeper's registrant).
# Runs inside the transaction so a failure rolls the whole merge back.
config[:after_merge]&.call(record_to_keep)
record_to_keep.save! if record_to_keep.changed?
end

Expand Down Expand Up @@ -147,6 +151,13 @@ def dedupe_perform
# merge_notes: Lambda(keep, delete) returning an array of informational (non-blocking)
# strings to surface on the preview (e.g. "both people have a login") (optional)
# record_extras: Lambda(record) returning extra detail string for index listing (optional)
# back_path: Override the index eyebrow's return path (e.g. when the deduper was
# opened from an event's registrants page, not the model index) (optional)
# back_label: Label for that eyebrow link, without the "← " prefix (optional)
# subtitle: Extra line under the index header, e.g. naming the scope being deduped (optional)
# after_merge: Lambda(keep) run inside the merge transaction AFTER the duplicate's records
# have moved onto the keeper β€” to reconcile anything the move leaves inconsistent
# (e.g. re-credit a moved scholarship to the keeper's registrant). Raise to roll back (optional)
def dedupe_config
raise NotImplementedError, "#{self.class} must implement #dedupe_config"
end
Expand Down Expand Up @@ -186,7 +197,10 @@ def build_dedupe_vars(config)
field_notes: config[:field_notes] || {},
deprecated_columns: Array(config[:deprecated_columns]).map(&:to_s),
preview_images: config[:preview_images],
record_extras: config[:record_extras]
record_extras: config[:record_extras],
back_path: config[:back_path],
back_label: config[:back_label],
subtitle: config[:subtitle]
}
end
end
45 changes: 45 additions & 0 deletions app/controllers/event_registrations_controller.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
class EventRegistrationsController < ApplicationController
include Dedupable

require "csv"

# show redirects to slug URL; kept for backwards compatibility
Expand Down Expand Up @@ -507,6 +509,49 @@ def set_form_variables

private

def dedupe_config
# Opened from an event's registrants page (bulk actions) β†’ scope the suggested
# duplicates to that event and return the eyebrow there; opened from the global
# registrations index β†’ all registrations, default eyebrow.
dedupe_event = Event.find_by(id: params[:event_id])
finder_scope = dedupe_event ? EventRegistration.where(event: dedupe_event) : EventRegistration.all

{
model_class: EventRegistration,
domain: :event_registrations,
candidate_finder: -> { EventRegistrationServices::DuplicateFinder.new(finder_scope).groups },
back_path: dedupe_event ? registrants_event_path(dedupe_event) : nil,
back_label: dedupe_event ? "Registrants" : nil,
subtitle: dedupe_event ? "Showing possible duplicate registrations for #{dedupe_event.title}." : nil,
editable_columns: %w[
status expected_payment_method fee_note shoutout intends_to_pay
someone_else_will_pay invoice_requested scholarship_requested w9_requested
payment_unresolved
],
# The (registrant_id, event_id) index means two registrations only ever
# collide when their registrants are different Person records. Merging keeps
# them separate β€” say so, so the admin also merges the people if they match.
merge_notes: ->(keep, delete) {
next [] if keep.registrant_id == delete.registrant_id

[ "These registrations have different registrants (#{keep.registrant&.full_name} vs #{delete.registrant&.full_name}). Merging combines the registrations only β€” the two people stay separate. Merge them in the people deduper too if they're the same person." ]
},
# A scholarship (and CE registration) moves onto the kept registration with the
# merge, but a scholarship still credits the deleted registrant. Re-credit any
# scholarship now on the keeper to the keeper's registrant so its recipient
# matches its allocation. CE delegates its registrant to the registration, so
# it follows automatically and needs no fixup here.
after_merge: ->(keep) {
keep.scholarships.where.not(recipient_id: keep.registrant_id).find_each do |scholarship|
scholarship.update!(recipient: keep.registrant)
end
},
record_extras: ->(registration) {
[ registration.registrant&.preferred_email.presence, registration.status&.humanize ].compact.join(" Β· ").presence
}
}
end

def redirect_after_failed_create(alert)
case params[:return_to]
when "registrants" then redirect_to registrants_event_path(@event_registration.event), alert: alert
Expand Down
106 changes: 106 additions & 0 deletions app/services/event_registration_services/duplicate_finder.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
require "set"

module EventRegistrationServices
# Surfaces likely-duplicate registrations for the deduper's candidate list. The
# (registrant_id, event_id) unique index already forbids one person registering
# for the same event twice, so a duplicate is one real person registered for the
# same event under two different Person records. Clusters registrations for the
# same event whose registrants share a signal β€” nickname/legal-name-aware name,
# email (or email_2), or FileMaker code β€” then annotates each cluster with why it
# was flagged. Registrants grouped by name but carrying different FileMaker codes
# are surfaced as a caution to verify, never silently treated as a clean merge.
class DuplicateFinder
Group = Struct.new(:key, :label, :records, :reasons, keyword_init: true)

def initialize(scope = EventRegistration.all)
@registrations = scope.includes(:registrant, :event).to_a
end

def groups
union = UnionFind.new(@registrations.map(&:id))
cluster(union) { |registration| name_keys(registration) }
cluster(union) { |registration| email_keys(registration) }
cluster(union) { |registration| Array(filemaker_key(registration)) }

by_id = @registrations.index_by(&:id)
union.components.filter_map do |ids|
next if ids.size < 2
records = ids.map { |id| by_id[id] }.sort_by(&:id)
Group.new(
key: records.map(&:id).join("-"),
label: label_for(records),
records: records,
reasons: reasons_for(records)
)
end.sort_by { |group| group.label.to_s.downcase }
end

private

def cluster(union)
buckets = Hash.new { |hash, key| hash[key] = [] }
@registrations.each do |registration|
yield(registration).each { |key| buckets[key] << registration.id }
end
buckets.each_value { |ids| union.union_all(ids) if ids.size > 1 }
end

def label_for(records)
registrant = records.first.registrant
"#{registrant&.full_name} β€” #{records.first.event&.title}"
end

def reasons_for(records)
reasons = []
reasons << "Same registrant name" if shared?(records) { |registration| name_keys(registration) }
reasons << "Shared registrant email" if shared?(records) { |registration| email_keys(registration) }
reasons.concat(filemaker_reasons(records))
reasons
end

def filemaker_reasons(records)
codes = records.filter_map { |registration| registration.registrant&.filemaker_code.presence }.uniq.sort
return [] if codes.empty?
return [ "⚠ Different FileMaker codes (#{codes.join(", ")}) β€” verify these are the same person" ] if codes.size > 1

[ "Same registrant FileMaker code (#{codes.first})" ]
end

def shared?(records)
keys = records.flat_map { |registration| Array(yield(registration)) }
keys.tally.any? { |_key, count| count > 1 }
end

# Only registrations for the same event can be duplicates, so every signal is
# scoped by event_id β€” the same person on two different events is legitimate.
# First-name variant (nickname or legal first name) paired with the last name,
# so "Bob Smith" and "Robert Smith" on one event cluster together.
def name_keys(registration)
registrant = registration.registrant
last = normalized(registrant&.last_name)
return [] if last.blank?

first_forms = [ registrant&.first_name, registrant&.legal_first_name ].filter_map { |name| name.presence }
first_forms
.flat_map { |name| NicknameMap.variants_for(name) }
.uniq
.map { |variant| "#{registration.event_id}|#{variant}|#{last}" }
end

def email_keys(registration)
registrant = registration.registrant
[ registrant&.email, registrant&.email_2 ]
.filter_map { |email| email.to_s.strip.downcase.presence }
.map { |email| "#{registration.event_id}|#{email}" }
end

def filemaker_key(registration)
code = registration.registrant&.filemaker_code.to_s.strip.downcase
"#{registration.event_id}|fm|#{code}" if code.present?
end

def normalized(value)
NicknameMap.normalize(value)
end
end
end
32 changes: 24 additions & 8 deletions app/services/model_deduper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def merge(record_to_keep, record_to_delete)
# sees exactly what moves (affiliations, event registrations, reports, …).
def reassignment_counts(record)
reassignable_joins.each_with_object({}) do |join, counts|
count = join[:join_class].where(join[:foreign_key] => record.id).count
count = join_references(join, record.id).count
next if count.zero?

counts[join_label(join[:join_class])] = count
Expand All @@ -57,7 +57,7 @@ def reassignment_counts(record)

def reassignment_preview(record)
reassignable_joins.filter_map do |join|
scope = join[:join_class].where(join[:foreign_key] => record.id)
scope = join_references(join, record.id)
count = scope.count
next if count.zero?

Expand Down Expand Up @@ -228,7 +228,13 @@ def has_many_joins
fk = assoc.foreign_key.to_s
next unless join_klass.column_names.include?(fk)

join_for(join_klass, fk)
# A polymorphic `has_many … as:` (allocations, comments) is scoped by its
# `*_type` column too, so a merge only moves this model's own rows and can't
# steal another type's rows that happen to share the deleted record's id.
type_column = assoc.type.to_s if assoc.options[:as]

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸ€– From Claude: Load-bearing behavior change for all five deduper models, not just registrations: polymorphic has_many … as: children (allocations, comments, bookmarks) were previously reassigned by FK id alone. This adds the *_type scope so a merge can only touch this model's own rows β€” worth a close look since it changes existing people/org/workshop merges too.

type_column = nil unless type_column && join_klass.column_names.include?(type_column)

join_for(join_klass, fk, type_column: type_column)
end
end

Expand Down Expand Up @@ -269,14 +275,24 @@ def anonymous_model_for(table)
klass
end

def join_for(join_klass, fk)
def join_for(join_klass, fk, type_column: nil)
{
join_class: join_klass,
foreign_key: fk.to_sym,
type_column: type_column,
natural_key: natural_key_columns(join_klass, fk)
}
end

# Rows of `join` that reference `id` β€” narrowed to this model's polymorphic type
# when the join is a polymorphic `as:` association, so it never touches another
# type's rows sharing the same id.
def join_references(join, id)
scope = join[:join_class].where(join[:foreign_key] => id)
scope = scope.where(join[:type_column] => model_class.polymorphic_name) if join[:type_column]
scope
end

# Columns (other than the FK) of a unique index that includes the FK.
def natural_key_columns(join_klass, fk)
join_klass.connection.indexes(join_klass.table_name)
Expand Down Expand Up @@ -384,14 +400,14 @@ def merge_join(primary, dupe, join)
natural_key = join[:natural_key]

if natural_key.empty?
moved = jc.where(fk => dupe.id).update_all(fk => primary.id)
moved = join_references(join, dupe.id).update_all(fk => primary.id)
logger.info " moved #{moved} #{jc.name} to primary" if moved > 0
return
end

existing = jc.where(fk => primary.id).pluck(*natural_key).map { |values| Array(values) }.to_set
existing = join_references(join, primary.id).pluck(*natural_key).map { |values| Array(values) }.to_set

jc.where(fk => dupe.id).find_each do |item|
join_references(join, dupe.id).find_each do |item|
key = natural_key.map { |col| item.public_send(col) }
if existing.include?(key)
item.destroy!
Expand All @@ -403,7 +419,7 @@ def merge_join(primary, dupe, join)
end
end

remaining = jc.where(fk => dupe.id).count
remaining = join_references(join, dupe.id).count
raise "ABORT: #{remaining} #{jc.name} items still reference #{model_label} #{dupe.id}" if remaining > 0
end
end
6 changes: 5 additions & 1 deletion app/views/dedupes/index.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@

<!-- Eyebrow + header -->
<div>
<%= link_to "← #{@dedupe[:model_label_plural]}", url_for(action: :index),
<%= link_to "← #{@dedupe[:back_label] || @dedupe[:model_label_plural]}",
@dedupe[:back_path] || url_for(action: :index),
class: "text-sm #{eyebrow_link_class} px-2 py-1" %>
<h1 class="text-primary mt-2 font-display text-2xl font-semibold">
Dedupe <%= @dedupe[:model_label_plural].downcase %>
Expand All @@ -13,6 +14,9 @@
Pick which <%= @dedupe[:model_label].downcase %> to delete and which to keep β€” its
records move to the one you keep before it's removed.
</p>
<% if @dedupe[:subtitle].present? %>
<p class="mt-1 text-sm text-gray-500"><%= @dedupe[:subtitle] %></p>
<% end %>
</div>

<!-- Selection form -->
Expand Down
4 changes: 4 additions & 0 deletions app/views/event_registrations/index.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@
event_registrations_path(request.query_parameters.merge(format: :csv)),
class: "text-sm #{eyebrow_link_class} px-2 py-1" %>
<% end %>
<% if allowed_to?(:manage?, EventRegistration) %>
<%= link_to "Dedupe", dedupe_index_event_registrations_path,
class: "text-sm #{eyebrow_link_class} px-2 py-1" %>
<% end %>
<% if allowed_to?(:new?, EventRegistration) %>
<%= link_to "New #{EventRegistration.model_name.human.downcase}",
new_event_registration_path(return_to: "index"),
Expand Down
3 changes: 3 additions & 0 deletions app/views/events/_bulk_actions_menu.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@
<%= link_to "Timesheets", attendance_event_path(@event, return_to: "registrants"), class: item_class %>
<% end %>
<%= link_to "Reconcile affiliations", reconcile_affiliations_event_path(@event), class: item_class %>
<% if allowed_to?(:manage?, EventRegistration) %>
<%= link_to "Dedupe registrations", dedupe_index_event_registrations_path(event_id: @event.id), class: item_class %>
<% end %>
<% if allowed_to?(:import?, EventRegistration) %>
<%= link_to new_event_registration_import_path(event_id: @event.id), class: item_class do %>
Import registrants <i class="fa-solid fa-upload text-gray-400 ml-1"></i>
Expand Down
15 changes: 15 additions & 0 deletions config/features.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2993,3 +2993,18 @@
attendance status, so the history isn't lost.
pro_tips:
- "Everyone else sees only active registrations (registered, attended, incomplete)."

- name: "Merge duplicate registrations"
area: registration
display_status: admin_facing
released_on: 2026-08-31
action_path: "/event_registrations/dedupe_index"
summary: >-
A new "Dedupe" tool on the Event registrations page finds and merges duplicate
registrations β€” the same person signed up for one event under two different
people records. It suggests likely matches, previews exactly what moves, and
combines the two registrations into one.
pro_tips:
- "Duplicates are suggested when two registrations for the same event have registrants with a matching name, email, or FileMaker code."
- "Merging combines the registrations only β€” the two people stay separate, so merge them in the People deduper too if they're the same person."
- "The preview shows every attendance, payment, and organization link that moves to the kept registration before you confirm, and blocks the merge if anything can't be moved safely."
6 changes: 6 additions & 0 deletions config/routes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,12 @@
post :confirm
end
resources :event_registrations do
collection do
get :dedupe_index
get :dedupe_preview
post :dedupe_perform
patch :dedupe_update_keep
end
member do
get :confirm
post :process_confirm
Expand Down
Loading