Leader rejection by clebouleau · Pull Request #876 · PAIR-code/deliberate-lab

clebouleau · 2025-11-28T19:00:33Z

Description:

This PR introduces a new experimental template designed for a Leadership Rejection experiment. The goal of the experiment is to investigate how repeated experiences of rejection from leadership roles affect future leadership ambitions, and whether these dynamics differ by gender.

To implement this, I created a fully new template inspired by the existing Lost-at-Sea template: frontend/src/shared/templates/leader_rejection_template.ts

Key Features Added:

New leader-selection mechanism
A new selection system to get leaders (compared to Condorcet elections):

Applicants receive a weight based on their baseline performance in two initial individual tasks.
A leader is selected via a performance-weighted lottery. If no one applies, all participants enter the lottery under the same rules. A counterfactual is computed for non-candidates and their hypothetical outcome (accepted/rejected) is stored. This logic required a dedicated backend utility system and integration inside ranking-stage processing.

New ranking type (RankingType.LR) and custom ranking view:
To avoid rewriting the entire reveal/payout logic, I reused ranking stages but introduced a new ranking type that:

Does not involve actual ranking input,
Displays instructions only,
Triggers the leader-selection logic on “Next”.

Custom reveal page for leadership status
A new reveal component shows each participant: their status (accepted, rejected, hypothetical).
New frontend file: frontend/src/components/stages/leader_status_reveal_view.ts
New survival task (Desert Scenario)
Added a new set of items, images, and expert solutions for a desert-survival task. Assets added under:
frontend/assets/survival_desert/

Main Files Added / Modified
Backend
functions/src/stages/leadership_rejection.utils.ts — full leader-selection logic & utilities.
utils/src/stages/ranking_stage.ts — new RankingType.LR
utils/src/stages/ranking_stage.validation.ts — validation schema for LR ranking.
utils/src/stages/reveal_stage.ts — support for LR reveal items.
utils/src/stages/reveal_stage.validation.ts — validation schema updates.
utils/src/utils/algebraic.utils.ts — extended helpers for leader-selection logic.

Frontend
frontend/src/shared/templates/leader_rejection_template.ts — new experiment template.
frontend/src/components/experiment_builder/stage_builder_dialog.ts — import LR metadata + card rendering.
frontend/src/components/stages/ranking_editor.ts — new ranking type added to editor UI.
frontend/src/components/stages/ranking_participant_view.ts — new info-only LR ranking view.
frontend/src/components/stages/reveal_summary_view.ts — LR reveal routing.
frontend/src/components/stages/leader_status_reveal_view.ts — new reveal page.
frontend/src/shared/file.utils.ts — export of leader selection outcome + status.
frontend/src/components/stages/payout_summary_view.ts temporarily replaced “Election winner’s answer” with “Leader’s answer” to avoid confusion.

Assets
frontend/assets/survival_desert/ — images + expert solution PDF.

Verifications:
Verified locally in emulator
All stages progress correctly

google-cla · 2025-11-28T19:00:38Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

cjqian · 2025-12-17T13:37:31Z

@vivtsai @jimbojw Could we take an initial review in by EOW?

vivtsai · 2025-12-17T20:57:51Z

Hi @clebouleau!

Quick note that we are in the process of moving from defining templates in code to building and saving them directly in the UI (Implement Templates Gallery Modal & Refine Template Selection UX #927). Can we do this for your template? If there's something that is not configurable via the current UI, let's figure out how we can add those controls in (so that you won't need to add leader_rejection_template.ts).
If I understand "leader rejection" correctly, the leader selection can be calculated in the reveal stage (which would then remove the need for the "empty" ranking stage). Can we do this a new "lottery reveal" component either in the reveal stage or in a new reveal-like stage (specifically for lottery calculations) rather than in ranking?

clebouleau · 2025-12-18T10:23:10Z

Hi @vivtsai!

I completely understand the goal of moving away from code-defined templates toward UI-defined ones. The reason I introduced a template in code (rather than using the UI) when I started this project is specifically that this experiment requires several elements that are not currently configurable via the UI: performance-weighted lotteries to determine the leader at different stages, intermediate reveal stages for leadership acceptance/rejection, and most importantly the propagation of leader identity to downstream group tasks and payout logic. I’m not sure how much work removing the template while preserving all these custom features (and making them available via the UI to build the exact same experiment) would represent. Would there be an acceptable way to keep this template as a temporary, experiment-specific solution (for instance via a temporary, isolated deployment that does not affect the main/public version) with the understanding that it should be removed afterward—assuming this does not conflict with your timeline for implementing the new template system?
The easiest way I found to implement the new leader-selection logic without recoding large parts of the system was to build on the existing ranking-stage infrastructure, following how it is used in the Lost at Sea template. To obtain a “winner” in a way that is compatible with the rest of the pipeline, I relied on ranking stages, which then allows me to reuse all the features already tied to them that are crucial for this experiment: computing payoffs based on the winner’s answers in later group stages, and revealing the winner’s identity using the existing reveal-stage logic. I then added the necessary “intermediate” reveal stages within this structure. I don’t see an easy way to move the leader-selection computation entirely to the reveal stage. From my understanding, doing so would effectively require recreating ranking-like stages anyway, and adjusting existing reveal and payout stages to consume their outputs in the same way they currently do for Lost at Sea. In that sense, the “empty” ranking stages felt like the most straightforward way to leverage existing infrastructure. I’m very open to discussion here and would be curious to hear if you see a simpler or cleaner alternative that I might be missing.

jimbojw · 2025-12-18T18:24:27Z

frontend/index.example.html

Did you mean to edit this file?

Hi @jimbojw , no I think this was an unintended IDE auto-formatting change

jimbojw

Reviewed up to leadership_rejection.utils.ts. Have not yet reviewed beyond that file (alphabetically).

jimbojw · 2025-12-18T18:26:00Z

frontend/src/components/experiment_builder/stage_builder_dialog.ts

    `;
  }

+  private renderLRCard() {


Here and elsewhere, it won't be immediately clear to the reader of the code that "LR" means "Leader Rejection". Recommend spelling it out everywhere. (The renderLRRankingCard() method below, createLRRankingStage(), LRRankingStagePublicData, etc.)

jimbojw · 2025-12-18T18:29:35Z

frontend/src/components/stages/ranking_editor.ts

    </div>`;
    const waitForAllParticipants = this.stage.progress.waitForAllParticipants;

+    console.debug(


Here and elsewhere, are these logs intentionally left in?

jimbojw · 2025-12-18T18:41:59Z

frontend/src/components/stages/leader_status_reveal_view.ts

+    if (showWinner) {
+      const winnerProfile = this.cohortService.participantMap?.[winnerId];
+      if (winnerProfile) {
+        winnerPretty = getParticipantInlineDisplay(winnerProfile); // 👈 magic happens here


What magic happens?

jimbojw · 2025-12-18T19:45:00Z

frontend/src/components/stages/payout_summary_view.ts

              ${rankingWinner !== null
-                ? `Election winner's answer:`
-                : 'Your answer:'}
+                ? `Leader's answer:`


This looks like it would affect experiments other than Leader Rejection.

jimbojw · 2025-12-18T19:50:53Z

frontend/src/shared/file.utils.ts

+      winnerId,
+    );
+    // Column for participant’s leader status
+    columns.push(leaderStatus);


In both branches of this if/else, the code pushes two columns. It's good that they're balanced. I'm a a little concerned though about the risk of adding columns for code that may not be expecting new columns. That is, I don't know what downstream code consumes the output of getRankingStageCSVColumns() but I'd want to make sure that that code is resilient to changing the returned number of columns. (I would hope that downstream code handles this fine, I just don't know for sure.)

jimbojw · 2025-12-18T19:55:34Z

functions/src/stages/leadership_rejection.utils.ts

+  // k > 1
+  const topId = candidateIds[0];
+  const weights: Record<string, number> = {};
+  const P1 = 0.6;


This function uses some magic numbers (0.6, 0.4, rho). Recommendations:

Name the default values as module (file) level constants.

(Optional) Parameterize the function to accept these values, with defaults.

jimbojw · 2025-12-18T19:56:05Z

functions/src/stages/leadership_rejection.utils.ts

+ * Compute geometric tail weights for candidates ranked by performance.
+ * Input: candidates sorted descending by performance.
+ */
+function computeWeights(candidateIds: string[]): Record<string, number> {


Recommend giving this function a more expressive name to indicate the meaning of the weights being computed.

jimbojw · 2025-12-18T19:57:29Z

functions/src/stages/leadership_rejection.utils.ts

+
+/**
+ * Draws a winner given a prob distribution.
+ * We also allow passing in a seed for reproducibility, but here we just Math.random().


I don't see a seed parameter. Copy-paste error?

jimbojw · 2025-12-18T20:00:03Z

functions/src/stages/leadership_rejection.utils.ts

+    }
+  }
+  // fallback in case of float precision
+  const lastId = Object.keys(weights)[Object.keys(weights).length - 1];


Nit: Earlier, you generate a list of Object.entries() in order to loop through them. If you created this outside of the for loop and assigned it to a const, it'd still be in scope here and you could grab the last entry. This would give you the id without recomputing the Object.keys().

clebouleau and others added 30 commits October 28, 2025 13:57

first commit

214ad7b

second commit

b119d2e

changin link to image folder

44464db

changin link to image folder

fc56d45

starting new selection logic

5cca7e4

starting new selection logic 2

6177801

starting new selection logic 3

f972a8d

starting new selection logic 3

3019be8

starting new selection logic 3

de581c6

starting new selection logic 3

1d73017

progress chris and clem in Paris

eeb37bd

progress chris and clem in Paris - debug new changes

c61c654

progress clem

1c0835d

clem 2

db9a3e4

clem 2

3c7b5bf

ajout createLRRankingStagePublicData

d581a84

la page de feedback 1 s'affiche (mais pas avec le bon feedback)

be7637e

debug de la procédure de ranking intial qui marchait plus

293061e

debug de la procédure de ranking intial qui marchait plus

1e9f7bb

debug de la procédure de ranking intial qui marchait plus

bebfd14

m

da1b34b

debugging score calculations

70564f1

updating full LR template

2517f0d

Fixed LR experiment export problem

724a80a

Simplified LR Ranking Stage (only text + button next)

8e72519

adjusting disable button in LR ranking case

28b9a32

display status + group leader ID

d8056a9

adjusting template

7dee7db

fixed compil warnings in algebraic.utils.ts

a48cd4b

Merge remote-tracking branch 'upstream/main' into leader-rejection

3620dbe

CBirch16 and others added 5 commits November 27, 2025 21:21

Reported last leader_rejection_template.ts update (from main branch)

86d9814

adjusting template - changing to consol logs

ea14302

Removed idea files

3eed571

Removed package-lock.json

ad35be9

restored package-lock.json

16fe8ff

CBirch16 added 2 commits December 13, 2025 16:00

Merge branch 'main' into leader-rejection

3e07517

fixed previous merge typo error

5917364

cjqian requested review from jimbojw and vivtsai December 17, 2025 13:35

jimbojw reviewed Dec 18, 2025

View reviewed changes

jimbojw reviewed Dec 19, 2025

View reviewed changes

Conversation

clebouleau commented Nov 28, 2025

Description:

Key Features Added:

Uh oh!

google-cla bot commented Nov 28, 2025

Uh oh!

cjqian commented Dec 17, 2025

Uh oh!

vivtsai commented Dec 17, 2025

Uh oh!

clebouleau commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jimbojw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

clebouleau commented Dec 18, 2025 •

edited

Loading