Summary for stakeholders
Mohammed Beheiry · FEC Contributions Analysis · 2017–2020
Key findings
Under the WinRed-exclusion scenario, itemized contribution records total about USD 15.25 billion. Individuals account for about 83% of that amount. Contributions peaked in October 2020, just before the presidential election. Committee contributions are concentrated: the top 10% of candidates received about 60% of the attributed amount, while the median candidate received about USD 16,000.
- Contributions roughly doubled from the 2018 cycle (about USD 5.09 billion) to the 2020 cycle (about USD 10.16 billion). Individual contributions more than doubled, while committee contributions grew by about 40%.
- The reported-record measure is about USD 1.55 billion higher; the difference is the WinRed-reported records excluded from this scenario. One candidate’s USD 49.6 million committee total falls to about USD 1.2 million without them.
- The source needed cleanup before counting. The 2020 individual table repeats records up to five times (195.5 million rows for 88.8 million unique IDs), and the table labeled 2020 operating expenditures contains 2017–2018 reporting years, so I did not use it.
- Small donations are undercounted. The data contains itemized records, and donors who give USD 200 or less to a committee generally do not have to be itemized.
These figures exclude WinRed conduit reporting to reduce overlap. They are not a verified count of unique donations. ActBlue and WinRed report differently, so comparisons between parties require care. The technical report shows the alternative reported-record measure.
Key numbers
| Result | WinRed-exclusion scenario |
|---|---|
| Individual contributions | 57.5 million records · USD 12.67 billion |
| Committee contributions | 653 thousand records · USD 2.58 billion |
| Busiest month | October 2020 · up 4.4% from September |
| Candidates receiving committee contributions | 2,603 |
| Median / average received per candidate | USD 16,000 / USD 405,000 |
| Top 10 businesses per state | Preliminary version: 4 hours; validated version: estimated 3–4 working days |
Decisions to confirm
Agree on the contribution measure to use for future reporting. For the business ranking, confirm whether to use the payee’s or committee’s state, gross payments or payments net of refunds, whether to combine subsidiaries with parent companies, and how to treat payment processors and other intermediaries.
Next steps and data needed
| Next step | Planning estimate | Data and availability |
|---|---|---|
| Confirm ranking scope | 30 minutes | Stakeholder decisions needed |
| Build a preliminary ranking | 4 hours, including scope | Expenditure tables available; coverage still needs validation |
| Validate businesses and intermediary payments | 3–4 working days total | Available records plus manual review; further evidence may be needed |
| Check completeness of 2018 expenses | Half a day initially | Official bulk files: not included in this analysis; need to be obtained |
| Compare party and incumbent groups | 1 hour initially | Candidate tables available; attribution coverage needs checking |
| Assess unitemized contributions | Estimate after source review | Committee summary reports: not included in this analysis; need to be obtained |
Estimates depend on source coverage and the number of unresolved businesses. They are planning allowances, not measured delivery times, and should not be added together. Summary reports could extend aggregate coverage but would not provide a complete transaction-level list of small donations.
Technical report¶
The stakeholder summary uses the WinRed-exclusion scenario. Q1 and Q2 below show the original reported-record results and explain how that scenario changes them. Both use the 2018 and 2020 cycles, covering January 2017 through December 2020.
Answers by question¶
| Question | Reported-record measure | WinRed-exclusion scenario |
|---|---|---|
| 1a. Individual contributions | 79,847,961 records | 57,464,005 records |
| 1b. Committee contributions | 1,750,504 records | 652,869 records |
| 1c. Individual / committee amounts | USD 14.08B / USD 2.72B | USD 12.67B / USD 2.58B |
| 1d. Peak month / growth from prior month | October 2020 / 12.50% | October 2020 / 4.44% |
| 2a. Candidate IDs receiving committee contributions | 2,614 | 2,603 |
| 2b. Median / average per candidate ID | USD 15,800 / USD 428,635 | USD 15,750 / USD 404,860 |
| 2b. Share received by the top 10% of candidate IDs | 61.91% | 59.61% |
Amounts combine both cycles. Medians are approximate; averages and amounts are rounded. The scenario reduces overlapping reporting but is not a verified count of unique donations.
Q1: sources · Q2: candidates · Q3: analysis plan · QA and model · Conduit comparison · Scope and next steps · Appendix
Q1. Contribution counts and amounts¶
| Source | Cycle | Contribution records | Gross reported amount |
|---|---|---|---|
| Individuals | 2018 | 15,088,198 | USD 4,021,953,507 |
| Individuals | 2020 | 64,759,763 | USD 10,058,325,757 |
| Individuals | Both cycles | 79,847,961 | USD 14,080,279,264 |
| Committees | 2018 | 334,553 | USD 1,067,216,231 |
| Committees | 2020 | 1,415,951 | USD 1,653,263,736 |
| Committees | Both cycles | 1,750,504 | USD 2,720,479,967 |
SQL: 16_model_committee_q1 — answer statement
SELECT
cycle,
COUNT(*) AS n_committee_contributions,
COUNT(DISTINCT sub_id) AS n_distinct_ids,
SUM(amount) AS gross_committee_amount,
MIN(transaction_dt) AS earliest_included_date,
MAX(transaction_dt) AS latest_included_date
FROM `fec-contributions-analysis.fec_analysis.stg_committee_q1`
WHERE is_q1_in_scope
GROUP BY cycle
ORDER BY cycle;
SQL: 17_model_individual_q1 — answer statement
SELECT
cycle,
COUNT(*) AS n_individual_contributions,
COUNT(DISTINCT sub_id) AS n_distinct_ids,
SUM(amount) AS gross_individual_amount,
MIN(transaction_dt) AS earliest_included_date,
MAX(transaction_dt) AS latest_included_date
FROM `fec-contributions-analysis.fec_analysis.stg_individual_q1`
WHERE is_q1_in_scope
GROUP BY cycle
ORDER BY cycle;
Each included record ID is unique within its source and cycle. The counts describe transactions, not distinct donors. Committee activity includes payments to candidate committees and other registered committees.
The table uses the reported-record measure. Under the WinRed-exclusion scenario, individual contributions fall to 57,464,005 records totaling USD 12,667,245,189 and committee contributions fall to 652,869 records totaling USD 2,579,862,501. The stakeholder summary uses these figures. I explain the scenario and its limits in Intermediary reporting and overlap.
Peak month¶
October 2020 recorded USD 2,017,565,781 in combined activity, compared with USD 1,793,420,123 in September. The month-over-month increase is 12.50%.
SQL: 18_model_monthly_contributions — peak-month statement
WITH calendar AS (
SELECT contribution_month
FROM UNNEST(
GENERATE_DATE_ARRAY(
DATE '2017-01-01',
DATE '2020-12-01',
INTERVAL 1 MONTH
)
) AS contribution_month
),
monthly AS (
SELECT
c.contribution_month,
COALESCE(SUM(m.gross_amount), 0) AS total_amount
FROM calendar AS c
LEFT JOIN
`fec-contributions-analysis.fec_analysis.agg_contributions_monthly` AS m
USING (contribution_month)
GROUP BY c.contribution_month
),
growth AS (
SELECT
contribution_month,
total_amount,
LAG(total_amount) OVER (
ORDER BY contribution_month
) AS prior_month_amount
FROM monthly
)
SELECT
contribution_month,
total_amount,
prior_month_amount,
100 * SAFE_DIVIDE(
total_amount - prior_month_amount,
prior_month_amount
) AS month_over_month_growth_pct
FROM growth
QUALIFY DENSE_RANK() OVER (ORDER BY total_amount DESC) = 1;
I aggregated the records by calendar month and used a complete monthly calendar before calculating the prior-month comparison. The monthly table reconciles exactly to the source counts and amounts, with 48 distinct months per source. It also keeps chart queries small.
The peak and growth rate apply to the reported-record definition. Excluding WinRed’s 15E conduit receipts and its outgoing committee contributions leaves October 2020 as the peak, at USD 1,636,196,210 versus USD 1,566,597,035 in September, an increase of 4.44%. This tests the effect of the reporting choice; it is not a fully deduplicated donation estimate.
Source queries: 16_model_committee_q1, 17_model_individual_q1, 18_model_monthly_contributions, 19_qa_monthly_reconciliation and 21_monthly_chart_data.
Definitions and caveats¶
I used gross positive amounts so the contribution measure has a consistent meaning across sources. Refunds, zero amounts, nonblank memo codes, and records with missing or out-of-cycle dates are excluded. Both memo codes X and Y are excluded by the same rule. Amounts are cast to NUMERIC for aggregation.
Individual receipts require entity type IND and transaction types 10, 15, 15E, 30, 30E, 31, 31E, 32, or 32E. This excludes candidate self-contributions, loans and intermediary disbursements. The source contains itemized records; it is not a complete measure of all donations.
Committee contributions use outgoing transaction types 24K and 24Z. For Q1, I used other_committee_tx_2018 and other_committee_tx_2020. For Q2, I used committee_contributions_2018 and committee_contributions_2020 because they contain candidate attribution. I did not add the two committee sources together.
The FEC transaction code definitions informed these filters. The unit of analysis is a qualifying record within a cycle; candidate statistics combine records by candidate ID across both cycles.
import pandas as pd
import matplotlib.pyplot as plt
# In Colab, I uploaded the CSV exported from BigQuery using files.upload().
# I changed this version to read the accompanying CSV directly so it also runs outside Colab.
monthly = pd.read_csv("21_monthly_chart_data.csv")
monthly["contribution_month"] = pd.to_datetime(
monthly["contribution_month"]
)
monthly = monthly.sort_values("contribution_month")
amounts = monthly.set_index("contribution_month")[
["individual_amount", "committee_amount"]
]
# Convert dollars to billions for the chart.
# AI helped me choose the chart colors.
ax = (amounts / 1e9).plot.area(
figsize=(11, 5),
color=["#2563EB", "#F59E0B"]
)
ax.set(
title="Monthly contributions by source, 2017–2020",
xlabel="Month",
ylabel="Gross contributions ($ billions)"
)
from matplotlib.patches import Patch
ax.legend(handles=[
Patch(color="#2563EB", label="Individuals"),
Patch(color="#F59E0B", label="Committees")
])
plt.tight_layout()
plt.show()
Individual receipts make up most of the reported amount at the October 2020 peak. The stacked areas show the contribution of each source to the combined monthly total. Source: 21_monthly_chart_data.csv.
SQL: 21_monthly_chart_data
-- 21_monthly_chart_data
-- I use the saved monthly summary to compare the two sources
-- over time without scanning the original contribution tables again.
WITH monthly AS (
SELECT
contribution_month,
SUM(
IF(source_type = 'individual', gross_amount, 0)
) AS individual_amount,
SUM(
IF(source_type = 'committee', gross_amount, 0)
) AS committee_amount,
SUM(gross_amount) AS total_amount
FROM
`fec-contributions-analysis.fec_analysis.agg_contributions_monthly`
GROUP BY contribution_month
)
SELECT
contribution_month,
individual_amount,
committee_amount,
total_amount,
100 * SAFE_DIVIDE(
total_amount - LAG(total_amount) OVER (
ORDER BY contribution_month
),
LAG(total_amount) OVER (
ORDER BY contribution_month
)
) AS month_over_month_growth_pct
FROM monthly
ORDER BY contribution_month;
Q2. Candidate coverage and distribution¶
Qualifying committee records identify 2,614 candidate IDs and USD 1,120,452,295 in attributed contributions across both cycles. The median total per candidate ID is approximately USD 15,800, compared with a mean of USD 428,635. The top 262 IDs receive 61.91% of the attributed amount.
| Measure | Total per candidate ID | What it means |
|---|---|---|
| Minimum | USD 8 | The lowest total received by a candidate ID in this analysis. |
| 25% approximately | USD 3,500 | About 25% of candidate IDs received this amount or less. |
| Median (50% approximately) | USD 15,800 | About half received this amount or less, and half received this amount or more. |
| Average (mean) | USD 428,635 | Total attributed contributions divided by the number of candidate IDs. Large totals pull the average up. |
| 75% approximately | USD 369,013 | About 75% of candidate IDs received this amount or less. |
| 90% approximately | USD 1,494,038 | About 90% of candidate IDs received this amount or less. |
| 99% approximately | USD 3,946,544 | About 99% of candidate IDs received this amount or less. |
| Maximum | USD 49,608,977 | The highest total received by a candidate ID in this analysis. |
| Standard deviation | USD 1,271,873 | Measures how spread out the totals are around the average, using all candidate IDs in this analysis. A larger value means more variation. |
SQL: 12_q2_candidate_summary
-- 12_q2_candidate_summary
-- I total qualifying contributions for each reported candidate ID
-- across both cycles, then describe the distribution of those totals.
-- Missing candidate IDs remain separate; I cannot tell how many
-- candidates those unidentified records represent.
WITH eligible AS (
SELECT cand_id, amount
FROM
`fec-contributions-analysis.fec_analysis.stg_committee_contributions`
WHERE is_q2_in_scope
),
candidate_totals AS (
SELECT
cand_id,
SUM(amount) AS total_received
FROM eligible
WHERE cand_id IS NOT NULL
GROUP BY cand_id
),
ranked AS (
SELECT
cand_id,
total_received,
ROW_NUMBER() OVER (
ORDER BY total_received DESC, cand_id
) AS candidate_rank,
COUNT(*) OVER () AS n_candidates
FROM candidate_totals
),
summary AS (
SELECT
COUNT(*) AS n_reported_candidate_ids,
SUM(total_received) AS total_attributed_amount,
MIN(total_received) AS minimum_received,
AVG(total_received) AS mean_received,
APPROX_QUANTILES(total_received, 100) AS percentiles,
MAX(total_received) AS maximum_received,
STDDEV_POP(total_received) AS population_stddev,
100 * SAFE_DIVIDE(
SUM(IF(
candidate_rank <= CEIL(n_candidates * 0.10),
total_received, 0
)),
SUM(total_received)
) AS top_10_percent_share_pct
FROM ranked
)
SELECT
n_reported_candidate_ids,
total_attributed_amount,
minimum_received,
percentiles[OFFSET(25)] AS approximate_p25,
percentiles[OFFSET(50)] AS approximate_median,
mean_received,
percentiles[OFFSET(75)] AS approximate_p75,
percentiles[OFFSET(90)] AS approximate_p90,
percentiles[OFFSET(99)] AS approximate_p99,
maximum_received,
population_stddev,
top_10_percent_share_pct,
(
SELECT COUNTIF(cand_id IS NULL)
FROM eligible
) AS n_contributions_without_candidate_id,
(
SELECT SUM(amount)
FROM eligible
WHERE cand_id IS NULL
) AS amount_without_candidate_id
FROM summary;
Definitions and caveats¶
I aggregated by candidate ID before calculating the distribution. The large difference between the mean and median reflects concentration at the upper end.
Populated IDs remain in the analysis even where the same-cycle candidate reference has no match: 138 IDs in 2018 and 125 in 2020. The result is therefore a count of reported candidate IDs, not a verified count of unique people. Another 2,938 qualifying records, totaling USD 3,098,774, have no candidate ID and are excluded from the distribution.
Largest candidate total¶
The USD 49,608,977 maximum belongs to Donald Trump's candidate ID, P80001571. Of that amount, USD 48,407,633 was reported by WinRed, C00694323, in 2020: 257,625 records, all coded 24K, with a largest record of USD 5,600. WinRed accounts for 97.58% of this candidate's total. The full breakdown reconciles to the original maximum.
The WinRed records identify recipient committee C00580100, use entity type CCM, and have no memo text. They establish the reported amount and recipient attribution, but do not identify the original funding source of each payment. I have retained them in the stated committee-record measure. I would not describe the result as USD 49.6 million of independently funded PAC support.
Excluding WinRed from the committee-source measure reduces coverage from 2,614 to 2,603 candidate IDs and the attributed amount to USD 1,053,851,132. The approximate median is USD 15,750 and the mean is USD 404,860. The largest total becomes USD 6,283,681, for candidate ID S4GA11285; Trump’s remaining total is USD 1,201,344. The top 261 IDs still account for 59.61% of the amount. Concentration therefore persists, but the original maximum is heavily affected by conduit reporting. The stakeholder summary uses these figures. Other intermediaries may still be included.
Candidate and committee identities were checked against the FEC candidate record and FEC committee record. Source queries: 11_model_committee_staging, 12_q2_candidate_summary, 22_qa_largest_candidate, 24_qa_winred_candidate_reporting and 27_qa_winred_sensitivity.
import json
from pathlib import Path
import matplotlib.pyplot as plt
# Use the saved Q2 scenario result; no new source scan is needed.
results = json.loads(Path("QA_results/27_qa_winred_sensitivity.json").read_text())
q2 = json.loads(next(row["details"] for row in results
if row["result_type"] == "q2_without_winred"))
top_share = float(q2["top_ten_percent_share_pct"])
fig, ax = plt.subplots(figsize=(9, 2.4))
# AI helped me choose the chart colors.
ax.barh([0], [top_share], color="#2563EB", label="Top 261 candidate IDs")
ax.barh([0], [100 - top_share], left=[top_share], color="#F59E0B",
label="Remaining 2,342 candidate IDs")
ax.text(top_share / 2, 0, f"{top_share:.1f}%", ha="center", va="center", color="white")
ax.text(top_share + (100 - top_share) / 2, 0, f"{100 - top_share:.1f}%",
ha="center", va="center")
ax.set(xlim=(0, 100), yticks=[], xlabel="Share of attributed committee contributions (%)",
title="Top 10% of candidate IDs received about 60% of the amount")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.35), ncol=2, frameon=False)
plt.tight_layout()
plt.show()
WinRed-exclusion scenario, both cycles. The top group contains 261 of 2,603 candidate IDs, rounding 10% up to a whole candidate. Source: query 27. This shows concentration; the table above describes the original distribution.
Q3. Analysis plan: top 10 business payees by state¶
A preliminary ranking is feasible within four hours. A validated ranking is not: I estimate about 26 hours, or three to four working days. The aggregation itself is straightforward. Business identification, name standardization and payments through intermediaries take most of the time, and all three depend on manual review.
| Source | Records | Dates in cycle | Missing dates | Reporting years |
|---|---|---|---|---|
| opex_2018 | 1,800,080 | 0 | 1,800,080 | 2017–2018 |
| opex_2020 | 1,800,080 | 0 | 1,800,080 | 2017–2018 |
| oppexp18 | 1,800,000 | 1,795,535 | 252 | 2017–2018 |
| oppexp20 | 2,315,382 | 2,310,905 | 15 | 2019–2020 |
The query below produced the source assessment. Q3 asks for an analysis plan; I have not produced a business-payee ranking.
SQL: 20_qa_operating_expenditures
-- 20_qa_operating_expenditures
-- Before proposing a business-payee ranking, I need to check whether
-- the expenditure sources cover the intended period and contain
-- usable payee names and states.
-- The two descriptive tables had identical inventory counts,
-- so I am comparing their actual date and reporting-year coverage.
WITH records AS (
SELECT
'opex_2018' AS source_table,
2018 AS cycle,
sub_id, rpt_yr, transaction_dt,
transaction_amt, name, state, entity_tp
FROM `bigquery-public-data.fec.opex_2018`
UNION ALL
SELECT
'opex_2020', 2020,
sub_id, rpt_yr, transaction_dt,
transaction_amt, name, state, entity_tp
FROM `bigquery-public-data.fec.opex_2020`
UNION ALL
SELECT
'oppexp18', 2018,
sub_id, rpt_yr, transaction_dt,
transaction_amt, name, state, entity_tp
FROM `bigquery-public-data.fec.oppexp18`
UNION ALL
SELECT
'oppexp20', 2020,
sub_id, rpt_yr, transaction_dt,
transaction_amt, name, state, entity_tp
FROM `bigquery-public-data.fec.oppexp20`
)
SELECT
source_table,
COUNT(*) AS n_rows,
COUNT(DISTINCT sub_id) AS n_distinct_ids,
COUNTIF(sub_id IS NULL) AS n_missing_ids,
MIN(rpt_yr) AS earliest_report_year,
MAX(rpt_yr) AS latest_report_year,
MIN(transaction_dt) AS earliest_date,
MAX(transaction_dt) AS latest_date,
COUNTIF(transaction_dt IS NULL) AS n_missing_dates,
COUNTIF(
transaction_dt BETWEEN
DATE(cycle - 1, 1, 1) AND DATE(cycle, 12, 31)
) AS n_dates_in_cycle,
COUNTIF(NULLIF(TRIM(name), '') IS NULL) AS n_missing_payee_names,
COUNTIF(NULLIF(TRIM(state), '') IS NULL) AS n_missing_states,
COUNTIF(NULLIF(TRIM(entity_tp), '') IS NULL) AS n_missing_entity_types,
COUNTIF(transaction_amt < 0) AS n_negative_amounts,
SUM(CAST(transaction_amt AS NUMERIC)) AS raw_total_amount
FROM records
GROUP BY source_table
ORDER BY source_table;
Approach and limitations¶
I would start with oppexp18 and oppexp20. Their reporting years match the requested cycles, and most dates fall within those periods. I ruled out opex_2020 for the 2020 analysis: its reporting years are 2017–2018 and every transaction date is missing. Its summary statistics also match opex_2018.
The alternative sources have missing names and states, negative amounts and implausible dates. oppexp18 contains exactly 1,800,000 records. That is a reason to verify coverage, although that count alone is not proof of truncation. opex_2018 has 80 more records and USD 377,626 more than oppexp18, so the two 2018 versions differ even apart from the missing dates.
I would first check record IDs, dates, memo entries and adjustments. Records without usable names or states would be accounted for separately. I would then standardize payee names and review likely matches among the largest recipients, using address details where available. Similar names alone would not justify merging businesses.
Intermediary payments need particular care. A processor or payroll provider may be the reported payee without being the final recipient. Supporting itemization must not be counted again as an additional payment. After applying those rules, I would aggregate accepted amounts by business and state, rank within each state, and review the leading results.
Questions for the stakeholder¶
I would confirm these before building the ranking:
- Should "state" mean the payee’s reported address or the paying committee’s state?
- Should the ranking use gross payments, or payments net of refunds and adjustments?
- Should a business be combined with its subsidiaries or parent company?
- Does a payment processor, payroll provider or conduit count as the business receiving the money?
- Should the two cycles be combined or ranked separately?
Until those are settled, my starting assumption would be total gross payments across both cycles, grouped by the payee’s reported state.
Time estimate¶
| Step | Preliminary (hours) | Validated (hours) | What takes the extra time |
|---|---|---|---|
| Confirm scope with the stakeholder | 0.25 | 0.5 | Agreeing on the questions above and a few edge cases |
| Record checks: IDs, dates, memo entries, negative amounts and coverage | 0.75 | 3 | Comparing oppexp18 with opex_2018 by record ID to confirm whether oppexp18 is complete, and deciding how to treat 38,481 negative amounts and dates such as 9201-10-02 |
| Identify business payees | 0.5 | 4 | Entity type ORG covers any organization, including government agencies and nonprofits, and 167,828 oppexp18 records have no entity type |
| Standardize payee names | 1 | 8 | Basic cleanup is quick, but merging spelling variants of the same business needs address checks and a review of each proposed match |
| Intermediary payments | 0.5 | 6 | Credit card, payroll and processor payments have to be traced to their supporting itemization so the money is not credited to the processor or counted twice |
| Aggregate and rank within each state | 0.25 | 0.5 | The ranking query itself is simple |
| Review the results and document exclusions | 0.75 | 4 | Reviewing the top 10 in every state, not just the largest ones, and walking the stakeholder through unresolved cases |
| Total | 4 | about 26 |
The four-hour version would contain the preliminary ranking, the SQL, the definitions and the value of excluded or unresolved records. I would spend the review time on cases large enough to change a top 10, and I would label the result as a ranking of identified business payees in the available records, not a complete ranking of all businesses.
Source query: 20_qa_operating_expenditures.
Source validation and modeling decisions¶
I compared table coverage before choosing sources. Similar names and matching totals were not sufficient evidence that two versions contained the same records.
| Check | Finding | Decision |
|---|---|---|
| Individual record IDs | 2018 has 21,628,973 unique IDs. 2020 has 195,482,945 rows and 88,823,302 unique IDs. Neither has missing IDs. | Removed repeated copies using stable analysis fields within each cycle. |
| Repeated-ID differences | 104,323 IDs differ in amendment indicator; the other checked fields agree. | Kept the agreed analysis fields without choosing an amendment indicator. |
| Amount and date profile | Positive refunds, memo records, non-individual entities and invalid dates occur in the sources. | Applied type, entity, memo, sign and date filters together. |
| Alternative 2020 candidate sources | 885,084 shared IDs; 2,745 descriptive-only and 4,081 pas220-only IDs. | Kept the versions separate. |
| Candidate attribution | 164 contribution records, worth USD 443,400 before scope filters, have candidate IDs in the descriptive source but not pas220. One USD 1,000 contribution has different populated IDs. | Selected the descriptive source for Q2; attribution differences remain disclosed. |
| Committee-source overlap | 275,881 qualifying IDs overlap in 2018 and 734,239 in 2020; amount, date and committee agree. | Used the broader source alone for Q1. |
| Unmatched committee records | 25 qualifying candidate records worth USD 39,000 are absent from the broader 2020 source. | Disclosed the coverage difference and did not append the unmatched records. |
| Conduit reporting | ActBlue’s side of earmarked donations is coded 24T and falls outside the filters; WinRed’s side is coded 15E and passes them. 1,916,040 one-to-one field matches between WinRed and recipient reports; other matches are ambiguous. | Used a WinRed-exclusion scenario in the stakeholder summary and kept the reported-record measure alongside it. The scenario reduces overlap but does not prove each donation is counted once. |
| Monthly reconciliation | Counts and amounts match for both sources; each has 48 distinct monthly rows. | Used the reconciled monthly aggregate for charts and comparisons. |
Deduplicating record IDs does not identify every repeated underlying donation. Likewise, retaining a populated candidate ID does not validate every attribution. These are separate questions from whether the SQL preserves the selected records.
Staging rows retain eligibility flags. Query 29 saves the deduplicated individual rows in a table clustered by cycle and committee; the staging view now reads that snapshot. The monthly aggregate remains the reporting layer for charts.
The repeated-ID comparison processed 14.73 GB in 13.821 seconds. I used job information to monitor execution cost and runtime; the reconciliation checks provide the evidence for aggregation accuracy.
Source queries: 01_source_inventory through 10_qa_committee_profile; 13_qa_other_committee_profile through 19_qa_monthly_reconciliation.
| Model | Grain | Purpose | Build query |
|---|---|---|---|
| stg_committee_contributions | Source record and cycle | Candidate attribution and Q2 eligibility | 11 |
| stg_committee_q1 | Source record and cycle | Broader committee activity and Q1 eligibility | 16 |
| individual_records_cached / stg_individual_q1 | Selected record fields and cycle | Saved deduplicated rows, including excluded records and eligibility flags | 17, 29 |
| fct_contributions | Source type, cycle and record ID | Q1 amounts, reporting and recipient committees, candidate attribution where available, and reporting-platform flag | 18, 29 |
| dim_committee | Committee ID and cycle | Committee names, types, party and state | 29 |
| dim_candidate | Candidate ID and cycle | Candidate names, party, office and incumbent status | 29 |
| agg_contributions_monthly | Month and source type | Materialized counts and gross amounts | 18 |
The views preserve the selected source fields, not every original column. The monthly table supports repeated reporting without rescanning the individual source. Adding cycle to the record key keeps election-cycle extracts separate.
The dimensions have one row per ID and cycle. Multiple source labels are retained together rather than silently choosing one. Candidate attribution is attached only to matching committee records; individual receipts are not assigned to candidates through an assumed committee relationship. The platform flag identifies the reporting committee, not the original donor. Query 29 checks the richer fact view against the existing monthly counts and amounts before it is used for reporting.
The snapshot is a fixed extract, not an automatic refresh pipeline. Query 29 rebuilds it directly from the original sources. Query 18 can rebuild the monthly table; query 29 should run last to restore the enriched fact view. This keeps the original analysis reproducible while supporting repeated investigation without repeating the large deduplication.
For conduit-reported individual receipts, recipient_committee_id is the reporting conduit’s ID, not necessarily the final recipient committee.
Intermediary reporting and overlap¶
The committee-source comparison found 1,010,120 shared qualifying record IDs. Using one source for Q1 avoids adding those records twice. I also checked a different issue: the same donation appearing under separate IDs in a conduit’s report and a recipient’s report.
ActBlue and WinRed report their side of an earmarked donation with different transaction codes, so my filters treat them differently.
| 2020 individual records | Reported by the conduit | Reported by the recipient | Counted under my filters |
|---|---|---|---|
| ActBlue | 20,733,195 records coded 24T, USD 2,140,427,718 | 18,626,946 records coded 15E, USD 1,801,148,249 | Recipient side only |
| WinRed | 22,383,956 records coded 15E, USD 1,413,034,075 | 4,125,031 records coded 15E, USD 584,456,022 | Both sides |
My transaction filter excludes 24T, so ActBlue donations are counted through the recipient’s report. WinRed codes its own receipts 15E, the same as the recipient, so both sides pass the transaction and memo filters. ActBlue follows the same pattern in 2018, and WinRed has no records in the 2018 data. A blank memo code alone does not resolve conduit reporting.
WinRed’s other_id field identifies an intended recipient on all but 83 of its included records. I compared that ID with the receiving committee’s ID, together with the donor’s name, five-digit ZIP code, date and amount. Names were trimmed and converted to uppercase. The comparison returned 1,916,040 groups with exactly one record on each side, totaling USD 341,092,327 per side. Another 177,252 groups contained multiple records on one or both sides. Many records had no match under these fields.
These matches are strong evidence of repeated reporting, but the matching fields are not a unique donation identifier. I did not automatically delete matching records or treat unmatched records as proof of separate donations. The FEC’s reporting guidance explains that recipients disclose the original contributor and the date the conduit received the contribution. WinRed’s conduit guidance also describes reporting by both parties.
To measure the effect on the answers, I excluded WinRed-reported 15E receipts and WinRed’s outgoing committee contributions while retaining recipient-reported individual receipts. All other filters stayed the same.
| Measure, both cycles | Reported-record measure | WinRed-exclusion scenario |
|---|---|---|
| Individual records | 79,847,961 | 57,464,005 |
| Individual amount | USD 14,080,279,264 | USD 12,667,245,189 |
| Committee records | 1,750,504 | 652,869 |
| Committee amount | USD 2,720,479,967 | USD 2,579,862,501 |
| Peak month | October 2020 | October 2020 |
| Peak-month growth | 12.50% | 4.44% |
| Candidate IDs in Q2 | 2,614 | 2,603 |
| Largest candidate total | USD 49,608,977 | USD 6,283,681 |
The timing of the peak and the concentration of candidate receipts hold under this test. Their size is sensitive to how conduit activity is counted.
The summary uses the WinRed-exclusion scenario. It reduces overlapping reporting but also removes unmatched conduit records. A missing match can reflect differences in fields, reporting coverage or itemization; it is not proof that a recipient never reported the donation. The scenario is therefore not a fully deduplicated measure of original donor funding.
Committee-side follow-up¶
Query 28 checked reporting committees whose names contain ActBlue or WinRed in both committee sources. ActBlue has six included Q1 records totaling USD 237,000: five records in 2018 worth USD 37,000 and one in 2020 worth USD 200,000. It has no included Q2 records. WinRed has 1,097,635 included Q1 records worth USD 140,617,466 and 472,869 included Q2 records worth USD 66,674,728. The Q1 excluded amounts match query 27 exactly. The Q2 amount is USD 73,565 higher than the USD 66,601,163 removed in query 27, which restricts candidate totals to records with a populated candidate ID.
This closes the platform-coding check for the two sources. It does not trace the original funding behind ActBlue’s six records or payments routed through other committees. I retained those records under the stated filters and made no further adjustment to the totals.
Source queries: 23_qa_intermediary_reporting through 28_qa_committee_platform_reporting. Results reviewed September 14, 2026.
SQL: 27_qa_winred_sensitivity
-- 27_qa_winred_sensitivity
-- Measure the effect of excluding WinRed conduit receipts and outgoing payments.
-- This is a reporting-side sensitivity test, not a complete donation deduplication.
WITH excluded AS (
SELECT
'individual' AS source_type,
transaction_dt,
amount
FROM `fec-contributions-analysis.fec_analysis.stg_individual_q1`
WHERE is_q1_in_scope
AND cmte_id = 'C00694323'
AND transaction_tp = '15E'
UNION ALL
SELECT
'committee',
transaction_dt,
amount
FROM `fec-contributions-analysis.fec_analysis.stg_committee_q1`
WHERE is_q1_in_scope
AND cmte_id = 'C00694323'
),
monthly AS (
SELECT
source_type,
DATE_TRUNC(transaction_dt, MONTH) AS contribution_month,
COUNT(*) AS n_records,
SUM(amount) AS amount
FROM excluded
GROUP BY 1, 2
),
candidate_totals AS (
SELECT
cand_id,
SUM(amount) AS amount
FROM `fec-contributions-analysis.fec_analysis.stg_committee_contributions`
WHERE is_q2_in_scope
AND cand_id IS NOT NULL
AND cmte_id != 'C00694323'
GROUP BY 1
),
ranked AS (
SELECT
*,
ROW_NUMBER() OVER (ORDER BY amount DESC, cand_id) AS ranking,
COUNT(*) OVER () AS n_candidates
FROM candidate_totals
),
candidate_summary AS (
SELECT
COUNT(*) AS n_candidates,
SUM(amount) AS amount,
MIN(amount) AS minimum,
AVG(amount) AS mean,
APPROX_QUANTILES(amount, 100)[OFFSET(50)] AS approximate_median,
MAX(amount) AS maximum,
STDDEV_POP(amount) AS population_stddev,
100 * SAFE_DIVIDE(
SUM(IF(ranking <= CEIL(n_candidates * .1), amount, 0)),
SUM(amount)
) AS top_ten_percent_share_pct,
ARRAY_AGG(STRUCT(cand_id, amount) ORDER BY ranking LIMIT 1) AS largest_candidate
FROM ranked
)
SELECT
'excluded_q1_monthly' AS result_type,
TO_JSON_STRING(m) AS details
FROM monthly AS m
UNION ALL
SELECT
'q2_without_winred',
TO_JSON_STRING(c)
FROM candidate_summary AS c;
Scope and next steps¶
Scope¶
I focused the analysis on the three questions, source quality and checks that could change the answers. The report separates the findings from the detailed SQL and QA appendix so a reviewer can read the conclusions first and inspect the supporting work where needed.
Completed refinements¶
I materialized the deduplicated individual records so repeated analysis no longer reruns that step. I also added recipient and reporting-committee fields, a reporting-platform flag, candidate attribution where available, and committee and candidate dimensions. The updated model reconciles to the original totals. Q2 includes a concentration chart showing the share received by the top 10% of candidate IDs.
With more time¶
I would add a full Q2 distribution chart using candidate-level totals, with a log scale to make the smaller amounts visible alongside the largest recipients. The current chart shows concentration, not the shape of the full distribution. I would also automate refreshes and reconciliation for the saved individual table, and extend the conduit fields beyond the named reporting platforms to trace more of the path from the original donor to the recipient. Those additions would support recurring reporting and further analysis beyond this submission.
Technical appendix: complete SQL and captured results
Appendix: SQL and detailed results¶
The complete setup and QA scripts are below. Answer queries 12, 20 and 21, and the sensitivity query 27, appear with their analyses above. All 29 queries are also available as separate files in the SQL folder. Results are captured outputs from the analysis; their provenance is included with each table.
The detailed appendix is available in this HTML and in the SQL and results folders. It is omitted from the printed reading copy.
01_source_inventory¶
Lists source tables, row counts and rounded sizes.
SQL: 01_source_inventory
SELECT
table_id,
row_count,
ROUND(size_bytes / POW(1024, 3), 2) AS size_gib
FROM `bigquery-public-data.fec.__TABLES__`
WHERE REGEXP_CONTAINS(table_id, r'(18|20)$')
ORDER BY table_id;
Full results: 01_source_inventory (27 rows)
Initial source-inventory result screenshots; rounded GiB
| table_id | row_count | size_gib |
|---|---|---|
| candidate_2018 | 7371 | 0 |
| candidate_2020 | 7758 | 0 |
| candidate_committee_2018 | 7180 | 0 |
| candidate_committee_2020 | 7055 | 0 |
| ccl18 | 7180 | 0 |
| ccl20 | 7061 | 0 |
| cm18 | 19022 | 0 |
| cm20 | 18288 | 0 |
| cn18 | 7371 | 0 |
| cn20 | 7771 | 0 |
| committee_2018 | 19022 | 0 |
| committee_2020 | 18286 | 0 |
| committee_contributions_2018 | 349820 | 0.05 |
| committee_contributions_2020 | 887829 | 0.14 |
| indiv18 | 21628973 | 3.82 |
| indiv20 | 195482945 | 33.77 |
| individuals_ingest_2020 | 34484183 | 6.04 |
| opex_2018 | 1800080 | 0.29 |
| opex_2020 | 1800080 | 0.29 |
| oppexp18 | 1800000 | 0.3 |
| oppexp20 | 2315382 | 0.38 |
| oth18 | 1015198 | 0.17 |
| oth20 | 7455026 | 1.31 |
| other_committee_tx_2018 | 1015198 | 0.17 |
| other_committee_tx_2020 | 7401661 | 1.3 |
| pas218 | 349820 | 0.05 |
| pas220 | 889165 | 0.14 |
02_source_schema¶
Lists column names and data types.
SQL: 02_source_schema
-- 02_source_schema: inspect all 2018/2020 table schemas
SELECT
table_name,
COUNT(*) AS n_columns,
STRING_AGG(
CONCAT(column_name, ' (', data_type, ')'),
', ' ORDER BY ordinal_position
) AS columns_and_types
FROM `bigquery-public-data.fec.INFORMATION_SCHEMA.COLUMNS`
WHERE REGEXP_CONTAINS(table_name, r'(18|20)$')
GROUP BY table_name
ORDER BY table_name;
Full results: 02_source_schema (27 rows)
Original JSON export supplied during analysis
| table_name | n_columns | columns_and_types |
|---|---|---|
| candidate_2018 | 15 | cand_id (STRING), cand_name (STRING), cand_pty_affiliation (STRING), cand_election_yr (INT64), cand_office_st (STRING), cand_office (STRING), cand_office_district (STRING), cand_ici (STRING), cand_status (STRING), cand_pcc (STRING), cand_st1 (STRING), cand_st2 (STRING), cand_city (STRING), cand_st (STRING), cand_zip (STRING) |
| candidate_2020 | 15 | cand_id (STRING), cand_name (STRING), cand_pty_affiliation (STRING), cand_election_yr (INT64), cand_office_st (STRING), cand_office (STRING), cand_office_district (STRING), cand_ici (STRING), cand_status (STRING), cand_pcc (STRING), cand_st1 (STRING), cand_st2 (STRING), cand_city (STRING), cand_st (STRING), cand_zip (STRING) |
| candidate_committee_2018 | 7 | cand_id (STRING), cand_election_yr (INT64), fec_election_yr (INT64), cmte_id (STRING), cmte_tp (STRING), cmte_design (STRING), linkage_id (INT64) |
| candidate_committee_2020 | 7 | cand_id (STRING), cand_election_yr (INT64), fec_election_yr (INT64), cmte_id (STRING), cmte_tp (STRING), cmte_design (STRING), linkage_id (INT64) |
| ccl18 | 7 | cand_id (STRING), cand_election_yr (INT64), fec_election_yr (INT64), cmte_id (STRING), cmte_tp (STRING), cmte_design (STRING), linkage_id (INT64) |
| ccl20 | 7 | cand_id (STRING), cand_election_yr (INT64), fec_election_yr (INT64), cmte_id (STRING), cmte_tp (STRING), cmte_design (STRING), linkage_id (INT64) |
| cm18 | 15 | cmte_id (STRING), cmte_nm (STRING), tres_nm (STRING), cmte_st1 (STRING), cmte_st2 (STRING), cmte_city (STRING), cmte_st (STRING), cmte_zip (STRING), cmte_dsgn (STRING), cmte_tp (STRING), cmte_pty_affiliation (STRING), cmte_filing_freq (STRING), org_tp (STRING), connected_org_nm (STRING), cand_id (STRING) |
| cm20 | 15 | cmte_id (STRING), cmte_nm (STRING), tres_nm (STRING), cmte_st1 (STRING), cmte_st2 (STRING), cmte_city (STRING), cmte_st (STRING), cmte_zip (STRING), cmte_dsgn (STRING), cmte_tp (STRING), cmte_pty_affiliation (STRING), cmte_filing_freq (STRING), org_tp (STRING), connected_org_nm (STRING), cand_id (STRING) |
| cn18 | 15 | cand_id (STRING), cand_name (STRING), cand_pty_affiliation (STRING), cand_election_yr (INT64), cand_office_st (STRING), cand_office (STRING), cand_office_district (STRING), cand_ici (STRING), cand_status (STRING), cand_pcc (STRING), cand_st1 (STRING), cand_st2 (STRING), cand_city (STRING), cand_st (STRING), cand_zip (STRING) |
| cn20 | 15 | cand_id (STRING), cand_name (STRING), cand_pty_affiliation (STRING), cand_election_yr (INT64), cand_office_st (STRING), cand_office (STRING), cand_office_district (STRING), cand_ici (STRING), cand_status (STRING), cand_pcc (STRING), cand_st1 (STRING), cand_st2 (STRING), cand_city (STRING), cand_st (STRING), cand_zip (STRING) |
| committee_2018 | 15 | cmte_id (STRING), cmte_nm (STRING), tres_nm (STRING), cmte_st1 (STRING), cmte_st2 (STRING), cmte_city (STRING), cmte_st (STRING), cmte_zip (STRING), cmte_dsgn (STRING), cmte_tp (STRING), cmte_pty_affiliation (STRING), cmte_filing_freq (STRING), org_tp (STRING), connected_org_nm (STRING), cand_id (STRING) |
| committee_2020 | 15 | cmte_id (STRING), cmte_nm (STRING), tres_nm (STRING), cmte_st1 (STRING), cmte_st2 (STRING), cmte_city (STRING), cmte_st (STRING), cmte_zip (STRING), cmte_dsgn (STRING), cmte_tp (STRING), cmte_pty_affiliation (STRING), cmte_filing_freq (STRING), org_tp (STRING), connected_org_nm (STRING), cand_id (STRING) |
| committee_contributions_2018 | 22 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (INT64), other_id (STRING), cand_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| committee_contributions_2020 | 22 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (INT64), other_id (STRING), cand_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| indiv18 | 21 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), other_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| indiv20 | 21 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), other_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| individuals_ingest_2020 | 21 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), other_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| opex_2018 | 25 | cmte_id (STRING), amndt_ind (STRING), rpt_yr (INT64), rpt_tp (STRING), image_num (INT64), line_num (STRING), form_tp_cd (STRING), sched_tp_cd (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), transaction_pgi (STRING), purpose (STRING), category (STRING), category_desc (STRING), memo_cd (STRING), memo_text (STRING), entity_tp (STRING), sub_id (INT64), file_num (INT64), tran_id (STRING), back_ref_tran_id (STRING) |
| opex_2020 | 25 | cmte_id (STRING), amndt_ind (STRING), rpt_yr (INT64), rpt_tp (STRING), image_num (INT64), line_num (STRING), form_tp_cd (STRING), sched_tp_cd (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), transaction_pgi (STRING), purpose (STRING), category (STRING), category_desc (STRING), memo_cd (STRING), memo_text (STRING), entity_tp (STRING), sub_id (INT64), file_num (INT64), tran_id (STRING), back_ref_tran_id (STRING) |
| oppexp18 | 25 | cmte_id (STRING), amndt_ind (STRING), rpt_yr (INT64), rpt_tp (STRING), image_num (INT64), line_num (STRING), form_tp_cd (STRING), sched_tp_cd (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), transaction_pgi (STRING), purpose (STRING), category (STRING), category_desc (STRING), memo_cd (STRING), memo_text (STRING), entity_tp (STRING), sub_id (INT64), file_num (INT64), tran_id (STRING), back_ref_tran_id (STRING) |
| oppexp20 | 25 | cmte_id (STRING), amndt_ind (STRING), rpt_yr (INT64), rpt_tp (STRING), image_num (INT64), line_num (STRING), form_tp_cd (STRING), sched_tp_cd (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), transaction_pgi (STRING), purpose (STRING), category (STRING), category_desc (STRING), memo_cd (STRING), memo_text (STRING), entity_tp (STRING), sub_id (INT64), file_num (INT64), tran_id (STRING), back_ref_tran_id (STRING) |
| oth18 | 21 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), other_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| oth20 | 21 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), other_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| other_committee_tx_2018 | 21 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), other_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| other_committee_tx_2020 | 21 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (FLOAT64), other_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| pas218 | 22 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (INT64), other_id (STRING), cand_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
| pas220 | 22 | cmte_id (STRING), amndt_ind (STRING), rpt_tp (STRING), transaction_pgi (STRING), image_num (INT64), transaction_tp (STRING), entity_tp (STRING), name (STRING), city (STRING), state (STRING), zip_code (STRING), employer (STRING), occupation (STRING), transaction_dt (DATE), transaction_amt (INT64), other_id (STRING), cand_id (STRING), tran_id (STRING), file_num (INT64), memo_cd (STRING), memo_text (STRING), sub_id (INT64) |
03_indiv_id_check¶
Counts missing and repeated individual record IDs.
SQL: 03_indiv_id_check
-- 03_indiv_id_check
-- Before counting contributions, I want to check whether any records
-- have missing or repeated IDs, since that could affect my totals.
-- A repeated ID needs investigation; it does not automatically mean
-- the row should be removed.
WITH id_counts AS (
SELECT
'indiv18' AS source_table,
COUNT(*) AS n_rows,
COUNTIF(sub_id IS NULL) AS n_null_ids,
COUNT(sub_id) AS n_nonnull_ids,
COUNT(DISTINCT sub_id) AS n_distinct_ids
FROM `bigquery-public-data.fec.indiv18`
UNION ALL
SELECT
'indiv20' AS source_table,
COUNT(*) AS n_rows,
COUNTIF(sub_id IS NULL) AS n_null_ids,
COUNT(sub_id) AS n_nonnull_ids,
COUNT(DISTINCT sub_id) AS n_distinct_ids
FROM `bigquery-public-data.fec.indiv20`
)
SELECT
source_table,
n_rows,
n_null_ids,
n_nonnull_ids,
n_distinct_ids,
n_nonnull_ids - n_distinct_ids AS n_repeated_id_rows
FROM id_counts
ORDER BY source_table;
Full results: 03_indiv_id_check (2 rows)
Result supplied during the analysis, September 14, 2026
| source_table | n_rows | n_null_ids | n_nonnull_ids | n_distinct_ids | n_repeated_id_rows |
|---|---|---|---|---|---|
| indiv18 | 21628973 | 0 | 21628973 | 21628973 | 0 |
| indiv20 | 195482945 | 0 | 195482945 | 88823302 | 106659643 |
04_qa_indiv_repeated_ids¶
Compares selected fields within repeated 2020 IDs.
SQL: 04_qa_indiv_repeated_ids
-- 04_qa_indiv_repeated_ids
-- My previous check found repeated IDs in the 2020 table.
-- Before removing any records, I want to see whether rows with
-- the same ID agree on their amount, date, committee, and other
-- details used in this analysis.
-- This checks the fields below, not every column in the table.
WITH records AS (
SELECT
sub_id,
TO_JSON_STRING(
STRUCT(
cmte_id,
transaction_dt,
transaction_amt,
transaction_tp,
entity_tp,
memo_cd,
amndt_ind,
other_id,
tran_id,
file_num
)
) AS contribution_details
FROM `bigquery-public-data.fec.indiv20`
),
id_checks AS (
SELECT
sub_id,
COUNT(*) AS n_copies,
COUNT(DISTINCT contribution_details) AS n_detail_versions
FROM records
GROUP BY sub_id
)
SELECT
COUNT(*) AS n_distinct_ids,
COUNTIF(n_copies = 1) AS n_ids_seen_once,
COUNTIF(n_copies > 1) AS n_repeated_ids,
COUNTIF(
n_copies > 1 AND n_detail_versions = 1
) AS n_repeated_ids_matching_checked_fields,
COUNTIF(
n_detail_versions > 1
) AS n_ids_with_conflicting_details,
MAX(n_copies) AS max_copies_per_id,
SUM(n_copies - 1) AS n_extra_id_occurrences
FROM id_checks;
Full results: 04_qa_indiv_repeated_ids (1 row)
Result supplied during the analysis, September 14, 2026
| n_distinct_ids | n_ids_seen_once | n_repeated_ids | n_repeated_ids_matching_checked_fields | n_ids_with_conflicting_details | max_copies_per_id | n_extra_id_occurrences |
|---|---|---|---|---|---|---|
| 88823302 | 948362 | 87874940 | 87770617 | 104323 | 5 | 106659643 |
05_qa_indiv_conflicting_fields¶
Identifies the fields that differ for repeated IDs.
SQL: 05_qa_indiv_conflicting_fields
-- 05_qa_indiv_conflicting_fields
-- I found 104,323 IDs with conflicting details.
-- I now want to identify which fields differ, especially amounts,
-- dates, and committees, before deciding how to handle these records.
-- I include nulls in the comparison so missing values are not overlooked.
WITH field_checks AS (
SELECT
sub_id,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
transaction_amt AS value
))) AS amount_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
transaction_dt AS value
))) AS date_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
cmte_id AS value
))) AS committee_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
transaction_tp AS value
))) AS transaction_type_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
entity_tp AS value
))) AS entity_type_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
memo_cd AS value
))) AS memo_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
amndt_ind AS value
))) AS amendment_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
other_id AS value
))) AS other_id_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
tran_id AS value
))) AS transaction_id_versions,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
file_num AS value
))) AS file_versions
FROM `bigquery-public-data.fec.indiv20`
GROUP BY sub_id
)
SELECT
COUNTIF(amount_versions > 1) AS ids_with_amount_changes,
COUNTIF(date_versions > 1) AS ids_with_date_changes,
COUNTIF(committee_versions > 1) AS ids_with_committee_changes,
COUNTIF(transaction_type_versions > 1) AS ids_with_type_changes,
COUNTIF(entity_type_versions > 1) AS ids_with_entity_changes,
COUNTIF(memo_versions > 1) AS ids_with_memo_changes,
COUNTIF(amendment_versions > 1) AS ids_with_amendment_changes,
COUNTIF(other_id_versions > 1) AS ids_with_other_id_changes,
COUNTIF(transaction_id_versions > 1) AS ids_with_tran_id_changes,
COUNTIF(file_versions > 1) AS ids_with_file_changes
FROM field_checks;
Full results: 05_qa_indiv_conflicting_fields (1 row)
Result supplied during the analysis, September 14, 2026
| ids_with_amount_changes | ids_with_date_changes | ids_with_committee_changes | ids_with_type_changes | ids_with_entity_changes | ids_with_memo_changes | ids_with_amendment_changes | ids_with_other_id_changes | ids_with_tran_id_changes | ids_with_file_changes |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 | 0 | 104323 | 0 | 0 | 0 |
06_qa_indiv_profile¶
Profiles individual transaction types, memo flags, amounts and dates.
SQL: 06_qa_indiv_profile
-- 06_qa_indiv_profile
-- I want to understand the transaction categories before deciding
-- which records belong in the individual contribution totals.
-- For 2020, I remove repeated copies using the record ID and the
-- analysis fields that my previous checks showed were consistent.
-- I keep negative amounts, memo rows, and unusual dates visible
-- here so I can assess them before applying filters.
WITH records AS (
SELECT
2018 AS cycle,
sub_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
transaction_amt
FROM `bigquery-public-data.fec.indiv18`
UNION ALL
SELECT DISTINCT
2020 AS cycle,
sub_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
transaction_amt
FROM `bigquery-public-data.fec.indiv20`
)
SELECT
cycle,
entity_tp,
transaction_tp,
memo_cd,
COUNT(*) AS n_records,
COUNTIF(transaction_amt > 0) AS n_positive,
COUNTIF(transaction_amt < 0) AS n_negative,
COUNTIF(transaction_amt = 0) AS n_zero,
COUNTIF(transaction_amt IS NULL) AS n_missing_amount,
SUM(CAST(transaction_amt AS NUMERIC)) AS total_amount,
COUNTIF(transaction_dt IS NULL) AS n_missing_date,
COUNTIF(
transaction_dt < DATE(cycle - 1, 1, 1)
OR transaction_dt > DATE(cycle, 12, 31)
) AS n_outside_cycle_dates
FROM records
GROUP BY cycle, entity_tp, transaction_tp, memo_cd
ORDER BY cycle, n_records DESC;
Full results: 06_qa_indiv_profile (248 rows)
Original JSON export supplied during analysis
| cycle | entity_tp | transaction_tp | memo_cd | n_records | n_positive | n_negative | n_zero | n_missing_amount | total_amount | n_missing_date | n_outside_cycle_dates |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2018 | IND | 15 | NULL | 10032679 | 9642445 | 16432 | 373802 | 0 | 2321053382 | 1619 | 189 |
| 2018 | IND | 24T | NULL | 5759441 | 5325259 | 433825 | 357 | 0 | 673295304 | 0 | 38 |
| 2018 | IND | 15E | NULL | 5259480 | 5245530 | 6235 | 7715 | 0 | 703440271 | 0 | 1094 |
| 2018 | IND | 22Y | NULL | 238222 | 224836 | 2001 | 11385 | 0 | 64543054 | 1 | 15 |
| 2018 | IND | 10 | NULL | 200529 | 200470 | 51 | 8 | 0 | 926009663 | 0 | 4 |
| 2018 | IND | 15 | X | 59060 | 40141 | 18469 | 450 | 0 | 24090040 | 48 | 236 |
| 2018 | CAN | 15C | NULL | 12307 | 12258 | 9 | 40 | 0 | 106485087 | 0 | 7 |
| 2018 | IND | 24I | X | 11324 | 11317 | 7 | 0 | 0 | 9655698 | 0 | 0 |
| 2018 | IND | 24I | NULL | 8200 | 8198 | 2 | 0 | 0 | 5519896 | 0 | 0 |
| 2018 | IND | 22Y | X | 7920 | 7909 | 1 | 10 | 0 | 1026556 | 0 | 644 |
| 2018 | ORG | 10 | NULL | 5033 | 5004 | 16 | 13 | 0 | 345332822 | 0 | 0 |
| 2018 | ORG | 15 | NULL | 4843 | 4820 | 21 | 2 | 0 | 17105108 | 0 | 1 |
| 2018 | ORG | 15E | X | 2946 | 2944 | 2 | 0 | 0 | 206708 | 0 | 0 |
| 2018 | IND | 15E | X | 2434 | 2120 | 311 | 3 | 0 | 1630490 | 0 | 1 |
| 2018 | ORG | 11 | NULL | 2319 | 2317 | 2 | 0 | 0 | 8303563 | 0 | 0 |
| 2018 | NULL | 15E | NULL | 1927 | 1927 | 0 | 0 | 0 | 485314 | 0 | 0 |
| 2018 | NULL | 15 | NULL | 1866 | 1859 | 0 | 7 | 0 | 2601068 | 12 | 5 |
| 2018 | ORG | 15 | X | 1823 | 1816 | 6 | 1 | 0 | 371256 | 0 | 0 |
| 2018 | IND | 31 | NULL | 1758 | 1747 | 9 | 2 | 0 | 40945236 | 0 | 0 |
| 2018 | IND | 15C | NULL | 1633 | 1620 | 1 | 12 | 0 | 39438379 | 0 | 19 |
| 2018 | PAC | 15E | X | 1488 | 1484 | 0 | 4 | 0 | 702232 | 0 | 0 |
| 2018 | PAC | 24I | X | 1352 | 1352 | 0 | 0 | 0 | 2098324 | 0 | 0 |
| 2018 | IND | 24T | X | 965 | 965 | 0 | 0 | 0 | 626676 | 0 | 0 |
| 2018 | ORG | 22Y | NULL | 892 | 384 | 507 | 1 | 0 | 292143 | 0 | 0 |
| 2018 | CAN | 15C | X | 778 | 773 | 3 | 2 | 0 | 34635606 | 0 | 4 |
| 2018 | PAC | 10 | NULL | 772 | 768 | 2 | 2 | 0 | 113804301 | 0 | 0 |
| 2018 | COM | 10 | NULL | 674 | 669 | 5 | 0 | 0 | 56015658 | 0 | 0 |
| 2018 | NULL | 11 | NULL | 590 | 590 | 0 | 0 | 0 | 2714750 | 0 | 0 |
| 2018 | IND | 20Y | NULL | 576 | 574 | 2 | 0 | 0 | 1920216 | 0 | 1 |
| 2018 | IND | 32 | NULL | 565 | 565 | 0 | 0 | 0 | 15204467 | 0 | 0 |
| 2018 | CAN | 15 | NULL | 489 | 488 | 1 | 0 | 0 | 284525 | 0 | 0 |
| 2018 | IND | 11 | NULL | 436 | 435 | 1 | 0 | 0 | 955376 | 0 | 0 |
| 2018 | CCM | 24I | X | 405 | 405 | 0 | 0 | 0 | 584275 | 0 | 0 |
| 2018 | NULL | 15E | X | 389 | 389 | 0 | 0 | 0 | 635312 | 0 | 0 |
| 2018 | IND | 10 | X | 315 | 312 | 0 | 3 | 0 | 6569359 | 2 | 1 |
| 2018 | IND | 15C | X | 281 | 273 | 1 | 7 | 0 | 5590290 | 0 | 0 |
| 2018 | NULL | 22Y | NULL | 231 | 219 | 8 | 4 | 0 | 518529 | 1 | 0 |
| 2018 | ORG | 15E | NULL | 209 | 208 | 1 | 0 | 0 | 1276748 | 0 | 0 |
| 2018 | IND | 30 | NULL | 188 | 187 | 1 | 0 | 0 | 7610786 | 0 | 0 |
| 2018 | NULL | 22Y | X | 130 | 130 | 0 | 0 | 0 | 14904 | 0 | 9 |
| 2018 | NULL | 15 | X | 88 | 68 | 18 | 2 | 0 | 10578 | 1 | 0 |
| 2018 | IND | 31 | X | 83 | 83 | 0 | 0 | 0 | 1277329 | 0 | 0 |
| 2018 | ORG | 11 | X | 81 | 60 | 21 | 0 | 0 | 111097 | 0 | 1 |
| 2018 | NULL | 10 | NULL | 73 | 69 | 0 | 4 | 0 | 170094 | 1 | 3 |
| 2018 | COM | 22Y | NULL | 68 | 68 | 0 | 0 | 0 | 56920 | 0 | 0 |
| 2018 | CCM | 24I | NULL | 63 | 63 | 0 | 0 | 0 | 162995 | 0 | 0 |
| 2018 | CCM | 10 | NULL | 56 | 56 | 0 | 0 | 0 | 1287375 | 0 | 0 |
| 2018 | ORG | 21Y | NULL | 52 | 49 | 3 | 0 | 0 | 101407 | 0 | 0 |
| 2018 | CCM | 15E | X | 48 | 48 | 0 | 0 | 0 | 55514 | 0 | 0 |
| 2018 | ORG | 20Y | NULL | 43 | 43 | 0 | 0 | 0 | 790422 | 0 | 0 |
| 2018 | ORG | 10 | X | 41 | 41 | 0 | 0 | 0 | 429539 | 0 | 0 |
| 2018 | CAN | 22Y | NULL | 39 | 39 | 0 | 0 | 0 | 504885 | 0 | 1 |
| 2018 | NULL | 11 | X | 38 | 22 | 16 | 0 | 0 | 14500 | 0 | 0 |
| 2018 | CCM | 15 | NULL | 38 | 37 | 0 | 1 | 0 | 18756 | 0 | 0 |
| 2018 | PAC | 15 | X | 38 | 38 | 0 | 0 | 0 | 28434 | 0 | 0 |
| 2018 | COM | 15E | X | 38 | 38 | 0 | 0 | 0 | 46775 | 0 | 0 |
| 2018 | CAN | 15E | NULL | 37 | 37 | 0 | 0 | 0 | 32394 | 0 | 0 |
| 2018 | PAC | 15 | NULL | 37 | 37 | 0 | 0 | 0 | 76660 | 0 | 0 |
| 2018 | CCM | 15E | NULL | 37 | 37 | 0 | 0 | 0 | 60300 | 0 | 0 |
| 2018 | ORG | 31T | NULL | 33 | 32 | 1 | 0 | 0 | 625100 | 0 | 0 |
| 2018 | COM | 15 | NULL | 33 | 33 | 0 | 0 | 0 | 24237 | 0 | 0 |
| 2018 | NULL | 15C | NULL | 33 | 31 | 0 | 2 | 0 | 31676 | 0 | 0 |
| 2018 | PAC | 15E | NULL | 31 | 30 | 1 | 0 | 0 | 212108 | 0 | 0 |
| 2018 | CCM | 24T | NULL | 25 | 24 | 1 | 0 | 0 | 24128 | 0 | 0 |
| 2018 | PTY | 10 | NULL | 24 | 24 | 0 | 0 | 0 | 359221 | 0 | 0 |
| 2018 | CCM | 22Y | NULL | 21 | 20 | 1 | 0 | 0 | 33750 | 0 | 0 |
| 2018 | PTY | 15 | NULL | 21 | 21 | 0 | 0 | 0 | 16627 | 0 | 0 |
| 2018 | PAC | 11 | NULL | 21 | 19 | 2 | 0 | 0 | 39900 | 0 | 0 |
| 2018 | COM | 15 | X | 18 | 16 | 0 | 2 | 0 | 7902 | 0 | 0 |
| 2018 | ORG | 31 | NULL | 17 | 17 | 0 | 0 | 0 | 476327 | 0 | 0 |
| 2018 | PAC | 24T | NULL | 16 | 16 | 0 | 0 | 0 | 41400 | 0 | 0 |
| 2018 | NULL | 21Y | NULL | 14 | 14 | 0 | 0 | 0 | 25900 | 0 | 0 |
| 2018 | COM | 20Y | NULL | 14 | 14 | 0 | 0 | 0 | 1544670 | 0 | 0 |
| 2018 | IND | 32 | X | 13 | 13 | 0 | 0 | 0 | 444275 | 0 | 2 |
| 2018 | CAN | 15 | X | 13 | 13 | 0 | 0 | 0 | 13903 | 0 | 0 |
| 2018 | IND | 20Y | X | 11 | 11 | 0 | 0 | 0 | 39467 | 0 | 0 |
| 2018 | COM | 11 | NULL | 10 | 10 | 0 | 0 | 0 | 33200 | 0 | 0 |
| 2018 | COM | 10 | X | 10 | 10 | 0 | 0 | 0 | 60515 | 0 | 0 |
| 2018 | PAC | 10 | X | 9 | 8 | 0 | 1 | 0 | 644279 | 0 | 0 |
| 2018 | ORG | 31T | X | 8 | 8 | 0 | 0 | 0 | 139900 | 0 | 0 |
| 2018 | CAN | 10 | NULL | 8 | 8 | 0 | 0 | 0 | 59300 | 0 | 0 |
| 2018 | COM | 15E | NULL | 8 | 8 | 0 | 0 | 0 | 6308 | 0 | 0 |
| 2018 | PTY | 15E | X | 6 | 6 | 0 | 0 | 0 | 28044 | 0 | 0 |
| 2018 | CAN | 22Y | X | 5 | 5 | 0 | 0 | 0 | 1247 | 0 | 0 |
| 2018 | NULL | 10 | X | 5 | 5 | 0 | 0 | 0 | 55699 | 0 | 0 |
| 2018 | PAC | 22Y | NULL | 5 | 5 | 0 | 0 | 0 | 7479 | 0 | 0 |
| 2018 | ORG | 32T | NULL | 4 | 4 | 0 | 0 | 0 | 214300 | 0 | 0 |
| 2018 | PAC | 20Y | NULL | 4 | 4 | 0 | 0 | 0 | 104469 | 0 | 0 |
| 2018 | ORG | 24I | X | 4 | 4 | 0 | 0 | 0 | 3000 | 0 | 0 |
| 2018 | IND | 21Y | NULL | 4 | 4 | 0 | 0 | 0 | 9700 | 0 | 0 |
| 2018 | CCM | 15C | NULL | 4 | 4 | 0 | 0 | 0 | 2229 | 0 | 0 |
| 2018 | PAC | 24T | X | 3 | 3 | 0 | 0 | 0 | 2100 | 0 | 0 |
| 2018 | NULL | 32 | NULL | 3 | 3 | 0 | 0 | 0 | 133206 | 0 | 0 |
| 2018 | COM | 24I | NULL | 3 | 3 | 0 | 0 | 0 | 4750 | 0 | 0 |
| 2018 | PAC | 24I | NULL | 3 | 2 | 1 | 0 | 0 | -3000 | 0 | 0 |
| 2018 | ORG | 24I | NULL | 3 | 0 | 3 | 0 | 0 | -4000 | 0 | 0 |
| 2018 | ORG | 24T | NULL | 3 | 3 | 0 | 0 | 0 | 1065 | 0 | 0 |
| 2018 | COM | 31 | NULL | 2 | 2 | 0 | 0 | 0 | 178900 | 0 | 0 |
| 2018 | CCM | 15 | X | 2 | 2 | 0 | 0 | 0 | 1500 | 0 | 0 |
| 2018 | ORG | 32 | NULL | 2 | 2 | 0 | 0 | 0 | 147800 | 0 | 0 |
| 2018 | IND | 11 | X | 2 | 2 | 0 | 0 | 0 | 7300 | 0 | 0 |
| 2018 | PTY | 15E | NULL | 2 | 2 | 0 | 0 | 0 | 10276 | 0 | 0 |
| 2018 | ORG | 15C | X | 2 | 2 | 0 | 0 | 0 | 3934 | 0 | 0 |
| 2018 | NULL | 31 | NULL | 2 | 2 | 0 | 0 | 0 | 106700 | 0 | 0 |
| 2018 | ORG | 30 | NULL | 2 | 2 | 0 | 0 | 0 | 167800 | 0 | 0 |
| 2018 | NULL | 15C | X | 2 | 2 | 0 | 0 | 0 | 4123 | 0 | 0 |
| 2018 | ORG | 32T | X | 2 | 2 | 0 | 0 | 0 | 115700 | 0 | 0 |
| 2018 | CAN | 15E | X | 2 | 2 | 0 | 0 | 0 | 273 | 0 | 0 |
| 2018 | CCM | 32 | NULL | 2 | 2 | 0 | 0 | 0 | 45500 | 0 | 0 |
| 2018 | ORG | 22Y | X | 2 | 2 | 0 | 0 | 0 | 3700 | 0 | 0 |
| 2018 | COM | 15C | X | 1 | 1 | 0 | 0 | 0 | 25 | 0 | 0 |
| 2018 | ORG | 24T | X | 1 | 1 | 0 | 0 | 0 | 25 | 0 | 0 |
| 2018 | COM | 21Y | NULL | 1 | 1 | 0 | 0 | 0 | 20000 | 0 | 0 |
| 2018 | PTY | 24I | NULL | 1 | 1 | 0 | 0 | 0 | 1000 | 0 | 0 |
| 2018 | COM | 11 | X | 1 | 1 | 0 | 0 | 0 | 1961 | 0 | 0 |
| 2018 | CCM | 11 | NULL | 1 | 1 | 0 | 0 | 0 | 14750 | 0 | 0 |
| 2018 | ORG | 21Y | X | 1 | 1 | 0 | 0 | 0 | 2700 | 0 | 0 |
| 2018 | IND | 31T | X | 1 | 1 | 0 | 0 | 0 | 1100 | 0 | 0 |
| 2018 | NULL | 24T | NULL | 1 | 1 | 0 | 0 | 0 | 200 | 0 | 0 |
| 2018 | ORG | 30T | NULL | 1 | 1 | 0 | 0 | 0 | 14258 | 0 | 0 |
| 2020 | IND | 15E | NULL | 51216509 | 50392258 | 127736 | 696515 | 0 | 4399939509 | 0 | 12818 |
| 2020 | IND | 24T | NULL | 21057871 | 20831325 | 206574 | 19972 | 0 | 2163375547 | 0 | 0 |
| 2020 | IND | 15 | NULL | 13919682 | 13589409 | 66157 | 264116 | 0 | 3439790890 | 7 | 123 |
| 2020 | IND | 22Y | NULL | 1197819 | 1168180 | 2138 | 27501 | 0 | 243138755 | 0 | 1 |
| 2020 | IND | 10 | NULL | 782122 | 781817 | 270 | 35 | 0 | 2027155959 | 0 | 81 |
| 2020 | IND | 15 | X | 540609 | 274293 | 244776 | 21540 | 0 | 34399534 | 0 | 136 |
| 2020 | IND | 15E | X | 17085 | 13992 | 3051 | 42 | 0 | 6281554 | 0 | 5 |
| 2020 | IND | 24I | NULL | 13204 | 2634 | 0 | 10570 | 0 | 1289366 | 0 | 0 |
| 2020 | IND | 24I | X | 12726 | 12718 | 0 | 8 | 0 | 12497977 | 0 | 0 |
| 2020 | CAN | 15C | NULL | 12641 | 12591 | 7 | 43 | 0 | 1457760539 | 0 | 30 |
| 2020 | ORG | 10 | NULL | 7213 | 7204 | 2 | 7 | 0 | 899335905 | 0 | 8 |
| 2020 | IND | 31 | NULL | 6666 | 6617 | 48 | 1 | 0 | 42017870 | 0 | 0 |
| 2020 | ORG | 15 | NULL | 6463 | 6451 | 11 | 1 | 0 | 30115088 | 0 | 0 |
| 2020 | IND | 20Y | NULL | 4703 | 4702 | 0 | 1 | 0 | 16839872 | 0 | 0 |
| 2020 | COM | 15E | X | 3770 | 3770 | 0 | 0 | 0 | 307846949 | 0 | 0 |
| 2020 | ORG | 15 | X | 3459 | 3442 | 16 | 1 | 0 | 74105893 | 0 | 2 |
| 2020 | ORG | 11 | NULL | 2703 | 2697 | 6 | 0 | 0 | 9795393 | 0 | 0 |
| 2020 | IND | 32 | NULL | 2114 | 2110 | 4 | 0 | 0 | 29823568 | 0 | 0 |
| 2020 | PAC | 15E | X | 1791 | 1790 | 0 | 1 | 0 | 60917203 | 0 | 0 |
| 2020 | PAC | 10 | NULL | 1528 | 1526 | 2 | 0 | 0 | 339727554 | 0 | 0 |
| 2020 | CAN | 15 | NULL | 1460 | 1458 | 0 | 2 | 0 | 549199 | 0 | 0 |
| 2020 | IND | 10 | X | 1009 | 990 | 18 | 1 | 0 | 13912424 | 0 | 0 |
| 2020 | CAN | 15C | X | 999 | 994 | 3 | 2 | 0 | 55992353 | 0 | 8 |
| 2020 | PAC | 24I | X | 867 | 867 | 0 | 0 | 0 | 1509138 | 0 | 0 |
| 2020 | COM | 10 | NULL | 845 | 844 | 1 | 0 | 0 | 81520747 | 0 | 0 |
| 2020 | PAC | 15 | NULL | 794 | 692 | 0 | 102 | 0 | 83368 | 0 | 0 |
| 2020 | IND | 24T | X | 671 | 671 | 0 | 0 | 0 | 2164514 | 0 | 0 |
| 2020 | ORG | 22Y | NULL | 629 | 592 | 34 | 3 | 0 | 2229740 | 0 | 0 |
| 2020 | ORG | 15E | X | 553 | 551 | 0 | 2 | 0 | 18884680 | 0 | 0 |
| 2020 | CCM | 15C | NULL | 501 | 501 | 0 | 0 | 0 | 373498 | 0 | 176 |
| 2020 | COM | 24T | X | 479 | 479 | 0 | 0 | 0 | 39426011 | 0 | 0 |
| 2020 | ORG | 15E | NULL | 356 | 350 | 6 | 0 | 0 | 944854 | 0 | 0 |
| 2020 | IND | 11 | NULL | 344 | 344 | 0 | 0 | 0 | 760925 | 0 | 0 |
| 2020 | IND | 30 | NULL | 307 | 306 | 1 | 0 | 0 | 13143381 | 0 | 0 |
| 2020 | IND | 15C | NULL | 256 | 254 | 0 | 2 | 0 | 1015961 | 0 | 6 |
| 2020 | COM | 15 | X | 226 | 226 | 0 | 0 | 0 | 51104 | 0 | 0 |
| 2020 | IND | 22Y | X | 194 | 190 | 1 | 3 | 0 | 269850 | 0 | 0 |
| 2020 | CCM | 24T | NULL | 162 | 162 | 0 | 0 | 0 | 97025 | 0 | 0 |
| 2020 | IND | 32 | X | 140 | 140 | 0 | 0 | 0 | 3886632 | 0 | 19 |
| 2020 | PAC | 15E | NULL | 134 | 134 | 0 | 0 | 0 | 400502 | 0 | 0 |
| 2020 | CCM | 15 | NULL | 132 | 132 | 0 | 0 | 0 | 150199 | 0 | 0 |
| 2020 | IND | 31 | X | 122 | 60 | 62 | 0 | 0 | 1051820 | 0 | 6 |
| 2020 | CCM | 15E | NULL | 99 | 98 | 0 | 1 | 0 | 141002 | 0 | 0 |
| 2020 | CCM | 10 | NULL | 99 | 99 | 0 | 0 | 0 | 792409 | 0 | 0 |
| 2020 | CCM | 24I | X | 79 | 79 | 0 | 0 | 0 | 101385 | 0 | 0 |
| 2020 | ORG | 11 | X | 78 | 60 | 18 | 0 | 0 | 223446 | 0 | 0 |
| 2020 | IND | 42Y | NULL | 70 | 70 | 0 | 0 | 0 | 154666 | 0 | 0 |
| 2020 | ORG | 10 | X | 62 | 62 | 0 | 0 | 0 | 2749138 | 0 | 0 |
| 2020 | ORG | 20Y | NULL | 57 | 55 | 2 | 0 | 0 | 3360046 | 0 | 0 |
| 2020 | CAN | 22Y | NULL | 54 | 51 | 2 | 1 | 0 | 1925926 | 0 | 0 |
| 2020 | ORG | 21Y | X | 49 | 49 | 0 | 0 | 0 | 242294 | 0 | 0 |
| 2020 | PAC | 10 | X | 49 | 48 | 1 | 0 | 0 | 322338 | 0 | 0 |
| 2020 | PAC | 24T | NULL | 46 | 46 | 0 | 0 | 0 | 130228 | 0 | 0 |
| 2020 | NULL | 10 | NULL | 45 | 45 | 0 | 0 | 0 | 106396 | 20 | 0 |
| 2020 | ORG | 21Y | NULL | 44 | 44 | 0 | 0 | 0 | 94805 | 0 | 0 |
| 2020 | ORG | 24T | NULL | 43 | 43 | 0 | 0 | 0 | 158000 | 0 | 0 |
| 2020 | IND | 15C | X | 40 | 40 | 0 | 0 | 0 | 358409 | 0 | 0 |
| 2020 | IND | 15 | Y | 38 | 37 | 1 | 0 | 0 | 15503 | 0 | 0 |
| 2020 | COM | 15 | NULL | 37 | 37 | 0 | 0 | 0 | 26034 | 0 | 0 |
| 2020 | PTY | 10 | NULL | 29 | 28 | 0 | 1 | 0 | 108353 | 0 | 0 |
| 2020 | IND | 20Y | X | 27 | 27 | 0 | 0 | 0 | 54903 | 0 | 0 |
| 2020 | CAN | 15 | X | 25 | 25 | 0 | 0 | 0 | 25481 | 0 | 0 |
| 2020 | CAN | 15E | NULL | 22 | 22 | 0 | 0 | 0 | 4823 | 0 | 0 |
| 2020 | ORG | 24I | X | 21 | 21 | 0 | 0 | 0 | 9660 | 0 | 0 |
| 2020 | NULL | 15 | NULL | 20 | 20 | 0 | 0 | 0 | 48039 | 0 | 0 |
| 2020 | PTY | 15 | NULL | 19 | 19 | 0 | 0 | 0 | 70011 | 0 | 0 |
| 2020 | ORG | 31 | NULL | 19 | 19 | 0 | 0 | 0 | 835153 | 0 | 0 |
| 2020 | IND | 11 | X | 17 | 11 | 6 | 0 | 0 | 11900 | 0 | 0 |
| 2020 | PAC | 32 | NULL | 16 | 16 | 0 | 0 | 0 | 1239825 | 0 | 0 |
| 2020 | COM | 11 | NULL | 15 | 15 | 0 | 0 | 0 | 42300 | 0 | 0 |
| 2020 | ORG | 32T | NULL | 14 | 14 | 0 | 0 | 0 | 415100 | 0 | 0 |
| 2020 | PAC | 15 | X | 13 | 13 | 0 | 0 | 0 | 12459 | 0 | 0 |
| 2020 | ORG | 31T | NULL | 10 | 10 | 0 | 0 | 0 | 559800 | 0 | 0 |
| 2020 | IND | 30 | X | 10 | 9 | 1 | 0 | 0 | 491030 | 0 | 0 |
| 2020 | COM | 15E | NULL | 9 | 9 | 0 | 0 | 0 | 3245 | 0 | 0 |
| 2020 | IND | 15C | Y | 9 | 9 | 0 | 0 | 0 | 129 | 0 | 0 |
| 2020 | CAN | 10 | NULL | 9 | 9 | 0 | 0 | 0 | 8850 | 0 | 0 |
| 2020 | ORG | 32 | NULL | 9 | 9 | 0 | 0 | 0 | 411055 | 0 | 0 |
| 2020 | IND | 21Y | NULL | 8 | 6 | 2 | 0 | 0 | 7600 | 0 | 0 |
| 2020 | CCM | 24I | NULL | 8 | 7 | 1 | 0 | 0 | 23275 | 0 | 0 |
| 2020 | ORG | 30 | NULL | 8 | 8 | 0 | 0 | 0 | 253586 | 0 | 0 |
| 2020 | PAC | 11 | NULL | 7 | 7 | 0 | 0 | 0 | 15800 | 0 | 0 |
| 2020 | PAC | 31 | NULL | 7 | 7 | 0 | 0 | 0 | 287500 | 0 | 0 |
| 2020 | ORG | 30T | NULL | 6 | 6 | 0 | 0 | 0 | 490500 | 0 | 0 |
| 2020 | IND | 41Y | NULL | 6 | 6 | 0 | 0 | 0 | 225 | 0 | 0 |
| 2020 | NULL | 22Y | Y | 6 | 6 | 0 | 0 | 0 | 500 | 0 | 0 |
| 2020 | ORG | 32T | X | 5 | 5 | 0 | 0 | 0 | 66100 | 0 | 0 |
| 2020 | COM | 20Y | NULL | 5 | 5 | 0 | 0 | 0 | 167726 | 0 | 0 |
| 2020 | COM | 32 | NULL | 4 | 4 | 0 | 0 | 0 | 3162862 | 0 | 0 |
| 2020 | PTY | 15E | X | 4 | 4 | 0 | 0 | 0 | 4335 | 0 | 0 |
| 2020 | COM | 22Y | NULL | 4 | 4 | 0 | 0 | 0 | 3743 | 0 | 0 |
| 2020 | ORG | 15C | NULL | 3 | 3 | 0 | 0 | 0 | 770 | 0 | 0 |
| 2020 | PTY | 15E | NULL | 3 | 3 | 0 | 0 | 0 | 2750 | 0 | 0 |
| 2020 | IND | 15E | Y | 3 | 3 | 0 | 0 | 0 | 300 | 0 | 0 |
| 2020 | CCM | 32 | NULL | 3 | 3 | 0 | 0 | 0 | 30132 | 0 | 0 |
| 2020 | NULL | 22Y | NULL | 3 | 3 | 0 | 0 | 0 | 10200 | 0 | 0 |
| 2020 | NULL | 10 | X | 3 | 3 | 0 | 0 | 0 | 894 | 0 | 0 |
| 2020 | ORG | 42Y | NULL | 3 | 3 | 0 | 0 | 0 | 1482 | 0 | 0 |
| 2020 | PAC | 20Y | NULL | 2 | 2 | 0 | 0 | 0 | 176873 | 0 | 0 |
| 2020 | PAC | 21Y | X | 2 | 2 | 0 | 0 | 0 | 15000 | 0 | 0 |
| 2020 | CCM | 15C | X | 2 | 2 | 0 | 0 | 0 | 241 | 0 | 0 |
| 2020 | CCM | 15E | X | 2 | 2 | 0 | 0 | 0 | 75 | 0 | 0 |
| 2020 | PTY | 24I | NULL | 2 | 2 | 0 | 0 | 0 | 10000 | 0 | 0 |
| 2020 | CCM | 22Y | NULL | 2 | 1 | 1 | 0 | 0 | 4750 | 0 | 0 |
| 2020 | NULL | 15C | NULL | 2 | 2 | 0 | 0 | 0 | 7542 | 0 | 0 |
| 2020 | CCM | 31 | NULL | 2 | 2 | 0 | 0 | 0 | 100000 | 0 | 0 |
| 2020 | CAN | 24I | X | 2 | 2 | 0 | 0 | 0 | 2500 | 0 | 0 |
| 2020 | ORG | 31T | X | 2 | 1 | 1 | 0 | 0 | 2300 | 0 | 0 |
| 2020 | CCM | 30 | NULL | 2 | 2 | 0 | 0 | 0 | 125000 | 0 | 0 |
| 2020 | COM | 21Y | X | 2 | 2 | 0 | 0 | 0 | 20000 | 0 | 0 |
| 2020 | PAC | 24I | NULL | 2 | 2 | 0 | 0 | 0 | 52000 | 0 | 0 |
| 2020 | COM | 24T | NULL | 1 | 1 | 0 | 0 | 0 | 1659 | 0 | 0 |
| 2020 | PTY | 15C | NULL | 1 | 1 | 0 | 0 | 0 | 50 | 0 | 0 |
| 2020 | IND | 21Y | X | 1 | 1 | 0 | 0 | 0 | 3414 | 0 | 0 |
| 2020 | PTY | 24T | NULL | 1 | 1 | 0 | 0 | 0 | 2000 | 0 | 0 |
| 2020 | PTY | 15 | X | 1 | 1 | 0 | 0 | 0 | 250 | 0 | 0 |
| 2020 | ORG | 22Y | X | 1 | 1 | 0 | 0 | 0 | 20 | 0 | 0 |
| 2020 | CCM | 24T | X | 1 | 1 | 0 | 0 | 0 | 100 | 0 | 0 |
| 2020 | PAC | 11 | X | 1 | 1 | 0 | 0 | 0 | 1493 | 0 | 0 |
| 2020 | CCM | 21Y | X | 1 | 1 | 0 | 0 | 0 | 3000 | 0 | 0 |
| 2020 | NULL | 15 | X | 1 | 1 | 0 | 0 | 0 | 2000 | 0 | 0 |
| 2020 | NULL | 20Y | NULL | 1 | 1 | 0 | 0 | 0 | 100 | 0 | 0 |
| 2020 | IND | 30T | NULL | 1 | 1 | 0 | 0 | 0 | 35500 | 0 | 0 |
| 2020 | COM | 31T | NULL | 1 | 1 | 0 | 0 | 0 | 7954 | 0 | 0 |
| 2020 | ORG | 15C | X | 1 | 1 | 0 | 0 | 0 | 50 | 0 | 0 |
| 2020 | PTY | 22Y | NULL | 1 | 1 | 0 | 0 | 0 | 3750 | 0 | 0 |
| 2020 | ORG | 24I | NULL | 1 | 1 | 0 | 0 | 0 | 1001 | 0 | 0 |
| 2020 | CAN | 15E | X | 1 | 0 | 1 | 0 | 0 | -2800 | 0 | 0 |
07_qa_committee_source_comparison¶
Compares the four candidate-contribution source versions.
SQL: 07_qa_committee_source_comparison
-- 07_qa_committee_source_comparison
-- Similar committee tables have different row counts.
-- I am comparing their IDs, dates, and filing numbers before
-- choosing a source. More rows alone do not prove a better source.
WITH sources AS (
SELECT
'pas218' AS source_table,
sub_id, transaction_dt, file_num, transaction_amt
FROM `bigquery-public-data.fec.pas218`
UNION ALL
SELECT
'committee_contributions_2018',
sub_id, transaction_dt, file_num, transaction_amt
FROM `bigquery-public-data.fec.committee_contributions_2018`
UNION ALL
SELECT
'pas220',
sub_id, transaction_dt, file_num, transaction_amt
FROM `bigquery-public-data.fec.pas220`
UNION ALL
SELECT
'committee_contributions_2020',
sub_id, transaction_dt, file_num, transaction_amt
FROM `bigquery-public-data.fec.committee_contributions_2020`
)
SELECT
source_table,
COUNT(*) AS n_rows,
COUNT(DISTINCT sub_id) AS n_distinct_ids,
COUNTIF(sub_id IS NULL) AS n_missing_ids,
COUNTIF(transaction_dt IS NULL) AS n_missing_dates,
MIN(transaction_dt) AS earliest_date,
MAX(transaction_dt) AS latest_date,
MAX(file_num) AS highest_file_number,
SUM(CAST(transaction_amt AS NUMERIC)) AS raw_total_amount
FROM sources
GROUP BY source_table
ORDER BY source_table;
Full results: 07_qa_committee_source_comparison (4 rows)
Result supplied during the analysis, September 14, 2026
| source_table | n_rows | n_distinct_ids | n_missing_ids | n_missing_dates | earliest_date | latest_date | highest_file_number | raw_total_amount |
|---|---|---|---|---|---|---|---|---|
| committee_contributions_2018 | 349820 | 349820 | 0 | 2040 | 2010-09-30 | 2117-08-23 | 1664704 | 1881597097 |
| committee_contributions_2020 | 887829 | 887829 | 0 | 5606 | 2015-05-07 | 2022-10-23 | 1851518 | 4042274870 |
| pas218 | 349820 | 349820 | 0 | 2040 | 2010-09-30 | 2117-08-23 | 1633193 | 1881597097 |
| pas220 | 889165 | 889165 | 0 | 5583 | 2015-05-07 | 2022-10-23 | 1634215 | 4040793861 |
08_qa_committee_2020_overlap¶
Compares IDs and fields between the two 2020 candidate sources.
SQL: 08_qa_committee_2020_overlap
-- 08_qa_committee_2020_overlap
-- I want to distinguish extra records from changed records.
-- Both tables passed the unique-ID check, so joining on sub_id
-- will not multiply rows.
WITH compared AS (
SELECT
CASE
WHEN p.sub_id IS NULL THEN 'descriptive_only'
WHEN d.sub_id IS NULL THEN 'pas220_only'
ELSE 'shared_id'
END AS match_status,
p.transaction_amt AS pas_amount,
d.transaction_amt AS descriptive_amount,
p.transaction_amt IS DISTINCT FROM d.transaction_amt
AS amount_differs,
p.transaction_dt IS DISTINCT FROM d.transaction_dt
AS date_differs,
p.cand_id IS DISTINCT FROM d.cand_id
AS candidate_differs,
p.cmte_id IS DISTINCT FROM d.cmte_id
AS committee_differs,
p.transaction_tp IS DISTINCT FROM d.transaction_tp
AS type_differs,
p.memo_cd IS DISTINCT FROM d.memo_cd
AS memo_differs,
p.file_num IS DISTINCT FROM d.file_num
AS file_differs
FROM `bigquery-public-data.fec.pas220` AS p
FULL OUTER JOIN
`bigquery-public-data.fec.committee_contributions_2020` AS d
ON p.sub_id = d.sub_id
)
SELECT
match_status,
COUNT(*) AS n_ids,
SUM(CAST(pas_amount AS NUMERIC)) AS pas_total,
SUM(CAST(descriptive_amount AS NUMERIC)) AS descriptive_total,
COUNTIF(match_status = 'shared_id' AND amount_differs)
AS shared_ids_amount_differs,
COUNTIF(match_status = 'shared_id' AND date_differs)
AS shared_ids_date_differs,
COUNTIF(match_status = 'shared_id' AND candidate_differs)
AS shared_ids_candidate_differs,
COUNTIF(match_status = 'shared_id' AND committee_differs)
AS shared_ids_committee_differs,
COUNTIF(match_status = 'shared_id' AND type_differs)
AS shared_ids_type_differs,
COUNTIF(match_status = 'shared_id' AND memo_differs)
AS shared_ids_memo_differs,
COUNTIF(match_status = 'shared_id' AND file_differs)
AS shared_ids_file_differs
FROM compared
GROUP BY match_status
ORDER BY match_status;
Full results: 08_qa_committee_2020_overlap (3 rows)
Result supplied during the analysis, September 14, 2026
| match_status | n_ids | pas_total | descriptive_total | shared_ids_amount_differs | shared_ids_date_differs | shared_ids_candidate_differs | shared_ids_committee_differs | shared_ids_type_differs | shared_ids_memo_differs | shared_ids_file_differs |
|---|---|---|---|---|---|---|---|---|---|---|
| descriptive_only | 2745 | NULL | 92188285 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| pas220_only | 4081 | 90707276 | NULL | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| shared_id | 885084 | 3950086585 | 3950086585 | 0 | 0 | 329 | 0 | 0 | 0 | 0 |
09_qa_candidate_id_differences¶
Examines candidate attribution differences.
SQL: 09_qa_candidate_id_differences
-- 09_qa_candidate_id_differences
-- The shared records agree on amounts, but 329 candidate IDs differ.
-- I want to see whether one source fills missing IDs or whether
-- the sources assign the records to different candidates.
WITH differences AS (
SELECT
NULLIF(TRIM(p.cand_id), '') AS pas_candidate_id,
NULLIF(TRIM(d.cand_id), '') AS descriptive_candidate_id,
p.transaction_tp,
p.transaction_amt
FROM `bigquery-public-data.fec.pas220` AS p
JOIN `bigquery-public-data.fec.committee_contributions_2020` AS d
ON p.sub_id = d.sub_id
WHERE p.cand_id IS DISTINCT FROM d.cand_id
)
SELECT
CASE
WHEN pas_candidate_id IS NOT DISTINCT FROM descriptive_candidate_id
THEN 'same_after_normalizing_blanks'
WHEN pas_candidate_id IS NULL
THEN 'missing_in_pas220'
WHEN descriptive_candidate_id IS NULL
THEN 'missing_in_descriptive'
ELSE 'different_populated_ids'
END AS difference_type,
pas_candidate_id,
descriptive_candidate_id,
transaction_tp,
COUNT(*) AS n_records,
SUM(CAST(transaction_amt AS NUMERIC)) AS raw_total_amount
FROM differences
GROUP BY
difference_type,
pas_candidate_id,
descriptive_candidate_id,
transaction_tp
ORDER BY n_records DESC;
Full results: 09_qa_candidate_id_differences (147 rows)
Original JSON export supplied during analysis
| difference_type | pas_candidate_id | descriptive_candidate_id | transaction_tp | n_records | raw_total_amount |
|---|---|---|---|---|---|
| different_populated_ids | H6CA19113 | H0MI01211 | 24A | 41 | 3609611 |
| different_populated_ids | H6CA19113 | H0MI01211 | 24E | 33 | 211344 |
| different_populated_ids | H6TX18133 | H0TX35015 | 24A | 22 | 51011 |
| different_populated_ids | H8LA03124 | H8IA03124 | 24A | 8 | 398060 |
| different_populated_ids | H8LA01094 | H8IA01094 | 24A | 8 | 695830 |
| different_populated_ids | H0CA42209 | H2CA37023 | 24A | 8 | 817 |
| different_populated_ids | H0LA02156 | H0IA02156 | 24A | 7 | 1080171 |
| different_populated_ids | H8MI09068 | S4MI00355 | 24A | 7 | 1724 |
| missing_in_pas220 | NULL | H0CA25154 | 24K | 4 | 16500 |
| missing_in_pas220 | NULL | H8NH01210 | 24K | 4 | 8500 |
| different_populated_ids | H8MI09068 | S4MI00355 | 24E | 4 | 7435 |
| different_populated_ids | H0AR02131 | H0AR02206 | 24C | 3 | 95225 |
| missing_in_pas220 | NULL | H8CA48035 | 24K | 3 | 7500 |
| different_populated_ids | S2TN00249 | H4SC03111 | 24C | 3 | 12895 |
| missing_in_pas220 | NULL | H8MI11254 | 24K | 3 | 4000 |
| missing_in_pas220 | NULL | H8CA01109 | 24K | 3 | 3500 |
| missing_in_pas220 | NULL | H6CA44103 | 24K | 2 | -500 |
| missing_in_pas220 | NULL | H6WI03099 | 24K | 2 | 6000 |
| missing_in_pas220 | NULL | H8TX23121 | 24K | 2 | 5500 |
| missing_in_pas220 | NULL | H8CA50098 | 24K | 2 | 7000 |
| missing_in_pas220 | NULL | H8TX22313 | 24K | 2 | 6000 |
| different_populated_ids | H4MI01130 | H0MI01211 | 24E | 2 | 698 |
| different_populated_ids | H8NV03200 | H6NV04020 | 24C | 2 | 1067 |
| missing_in_pas220 | NULL | H0MO02254 | 24K | 2 | 7000 |
| different_populated_ids | H2SD00076 | H2HI02128 | 24E | 2 | 1446 |
| missing_in_pas220 | NULL | H8MN01279 | 24K | 2 | 5000 |
| different_populated_ids | H6CA19113 | H8CA10167 | 24C | 2 | 202 |
| missing_in_pas220 | NULL | H8IA01094 | 24K | 2 | 4000 |
| missing_in_pas220 | NULL | H0IN05219 | 24K | 2 | 7000 |
| missing_in_pas220 | NULL | H0NY01186 | 24K | 2 | 9000 |
| missing_in_pas220 | NULL | H8GA07201 | 24K | 2 | 10000 |
| missing_in_pas220 | NULL | H0NC02125 | 24K | 2 | 9000 |
| missing_in_pas220 | NULL | H8NC13067 | 24K | 2 | 10000 |
| different_populated_ids | H0AK00048 | H6AK00045 | 24C | 2 | 93681 |
| missing_in_pas220 | NULL | H0VA05145 | 24K | 2 | 5500 |
| missing_in_pas220 | NULL | H2OH17109 | 24K | 2 | 7000 |
| missing_in_pas220 | NULL | H2IL17071 | 24K | 2 | 10000 |
| missing_in_pas220 | NULL | H0PA10110 | 24K | 2 | 5500 |
| missing_in_pas220 | NULL | H8NY22151 | 24K | 2 | 4500 |
| missing_in_pas220 | NULL | H0NY02192 | 24K | 2 | 6500 |
| missing_in_pas220 | NULL | H8NY11113 | 24K | 2 | 6000 |
| missing_in_pas220 | NULL | H8NJ07223 | 24K | 2 | 8000 |
| missing_in_pas220 | NULL | H0NJ02171 | 24K | 2 | 9000 |
| missing_in_pas220 | NULL | H8NJ11142 | 24K | 2 | 6000 |
| missing_in_pas220 | NULL | H8FL26039 | 24K | 2 | 2500 |
| different_populated_ids | H6WA03234 | H0WA03187 | 24C | 2 | 94225 |
| missing_in_pas220 | NULL | H8CA01257 | 24K | 2 | 5000 |
| missing_in_pas220 | NULL | H8CA10126 | 24K | 2 | 5400 |
| missing_in_pas220 | NULL | H6AZ01199 | 24K | 2 | 7000 |
| missing_in_pas220 | NULL | S0UT00165 | 24K | 2 | 10000 |
| missing_in_pas220 | NULL | H0TX24175 | 24K | 2 | 2500 |
| missing_in_pas220 | NULL | H8TX32098 | 24K | 2 | 5000 |
| missing_in_pas220 | NULL | S0ND00093 | 24K | 2 | 10000 |
| missing_in_pas220 | NULL | H0NH02181 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8AK00140 | 24K | 1 | 4000 |
| missing_in_pas220 | NULL | S2SD00068 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H0WI03175 | 24K | 1 | 4500 |
| missing_in_pas220 | NULL | H2IL10068 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H8IL04134 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | S0KS00315 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | S0IA00176 | 24K | 1 | 1000 |
| different_populated_ids | S0TX00308 | S0NE00171 | 24C | 1 | 1300 |
| missing_in_pas220 | NULL | H2OH14064 | 24K | 1 | 1500 |
| missing_in_pas220 | NULL | H0UT04076 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H4AZ07043 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | H8AZ09040 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | S0OH00133 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | S0AZ00350 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | S2NM00088 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | S0NM00124 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | H0CA53107 | 24K | 1 | 5000 |
| different_populated_ids | S4TX00417 | S0NE00171 | 24C | 1 | 1300 |
| different_populated_ids | S8ME00130 | H0ME02067 | 24C | 1 | 17521 |
| different_populated_ids | H0IA02180 | H8IA02043 | 24C | 1 | 46900 |
| missing_in_pas220 | NULL | H2CA52089 | 24K | 1 | 5000 |
| different_populated_ids | H0CA42209 | H2CA37023 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H0CA23167 | 24K | 1 | 1500 |
| missing_in_pas220 | NULL | H6TN09068 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | S0TN00169 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | S2WA00189 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H0WA02080 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H8WA03198 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | H6FL11126 | 24K | 1 | 2000 |
| different_populated_ids | H6TX18133 | H0TX35015 | 24E | 1 | 8 |
| missing_in_pas220 | NULL | H0GA05301 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | S0NC00202 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H6IL18088 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | S4PA00121 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | H0NY15160 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H0NY17174 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H4NY11138 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H6MD03292 | 24K | 1 | -2000 |
| missing_in_pas220 | NULL | H0CA10149 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | H8PA18181 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | S8NC00320 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H0PA01325 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8CA05035 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | H8VA10106 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H8LA02054 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H8NJ02166 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H0CA08101 | 24K | 1 | 1500 |
| missing_in_pas220 | NULL | P80000722 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H6PA08277 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | S4NC00162 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H0NC08254 | 24K | 1 | 3000 |
| missing_in_pas220 | NULL | H0NC11175 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | H4NY22051 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H8GA06393 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H6FL13205 | 24K | 1 | -5000 |
| missing_in_pas220 | NULL | H0FL16094 | 24K | 1 | 4000 |
| missing_in_pas220 | NULL | H8OH11141 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H0OH01099 | 24K | 1 | 4000 |
| missing_in_pas220 | NULL | H6VT00160 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H6MI07223 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8MI08102 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H0MI03316 | 24K | 1 | 3000 |
| missing_in_pas220 | NULL | S0NH00219 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H0IA02156 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H6MN02131 | 24K | 1 | 4000 |
| missing_in_pas220 | NULL | H2MA09072 | 24K | 1 | 500 |
| missing_in_pas220 | NULL | H8MT01232 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8IL06139 | 24K | 1 | 4000 |
| missing_in_pas220 | NULL | H8IL14174 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8IL13150 | 24K | 1 | 4000 |
| different_populated_ids | H2TX36126 | H2HI02128 | 24E | 1 | 785 |
| missing_in_pas220 | NULL | H4MO05234 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8KS03155 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H8OK05109 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8TX07140 | 24K | 1 | 2500 |
| different_populated_ids | S2MI00136 | S8MI00372 | 24E | 1 | 24 |
| different_populated_ids | S4MI00371 | S8MI00372 | 24E | 1 | 18 |
| missing_in_pas220 | NULL | H8TX25159 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | H8TX10110 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | H8CO06229 | 24K | 1 | 4000 |
| missing_in_pas220 | NULL | S4CO00395 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H8UT04053 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8AZ08083 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | H0NM03102 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H6NV04020 | 24K | 1 | 1000 |
| missing_in_pas220 | NULL | H8CA34266 | 24K | 1 | 5000 |
| missing_in_pas220 | NULL | H0CA40138 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8CA49074 | 24K | 1 | 4000 |
| missing_in_pas220 | NULL | H8CA49058 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H8CA39174 | 24K | 1 | 2000 |
| missing_in_pas220 | NULL | H8ME02185 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | H0WA10034 | 24K | 1 | 2500 |
| missing_in_pas220 | NULL | S0MN00328 | 24K | 1 | 2500 |
10_qa_committee_profile¶
Profiles candidate-contribution transaction categories.
SQL: 10_qa_committee_profile
-- 10_qa_committee_profile
-- I have selected the descriptive committee-to-candidate tables.
-- Before calculating totals, I want to separate contribution types
-- from other spending and check amounts, dates, and candidate IDs.
-- These are diagnostic totals; no contribution filters are applied.
WITH records AS (
SELECT
2018 AS cycle,
transaction_tp,
memo_cd,
transaction_amt,
transaction_dt,
cand_id
FROM `bigquery-public-data.fec.committee_contributions_2018`
UNION ALL
SELECT
2020 AS cycle,
transaction_tp,
memo_cd,
transaction_amt,
transaction_dt,
cand_id
FROM `bigquery-public-data.fec.committee_contributions_2020`
)
SELECT
cycle,
transaction_tp,
memo_cd,
COUNT(*) AS n_records,
COUNTIF(transaction_amt > 0) AS n_positive,
COUNTIF(transaction_amt < 0) AS n_negative,
COUNTIF(transaction_amt = 0) AS n_zero,
COUNTIF(transaction_amt IS NULL) AS n_missing_amount,
SUM(CAST(transaction_amt AS NUMERIC)) AS raw_total_amount,
COUNTIF(transaction_dt IS NULL) AS n_missing_date,
COUNTIF(
transaction_dt < DATE(cycle - 1, 1, 1)
OR transaction_dt > DATE(cycle, 12, 31)
) AS n_outside_cycle_dates,
COUNTIF(NULLIF(TRIM(cand_id), '') IS NULL)
AS n_missing_candidate
FROM records
GROUP BY cycle, transaction_tp, memo_cd
ORDER BY cycle, n_records DESC;
Full results: 10_qa_committee_profile (28 rows)
Original JSON export supplied during analysis
| cycle | transaction_tp | memo_cd | n_records | n_positive | n_negative | n_zero | n_missing_amount | raw_total_amount | n_missing_date | n_outside_cycle_dates | n_missing_candidate |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2018 | 24K | NULL | 281049 | 271900 | 7722 | 1427 | 0 | 515267295 | 7 | 10 | 1218 |
| 2018 | 24E | NULL | 35169 | 34769 | 69 | 331 | 0 | 330088068 | 337 | 3 | 0 |
| 2018 | 24A | NULL | 20367 | 20230 | 31 | 106 | 0 | 969367979 | 267 | 2 | 0 |
| 2018 | 24Z | NULL | 4021 | 3994 | 16 | 11 | 0 | 1568667 | 0 | 0 | 136 |
| 2018 | 24E | X | 1948 | 1868 | 42 | 38 | 0 | 12765853 | 843 | 21 | 0 |
| 2018 | 24K | X | 1722 | 1511 | 211 | 0 | 0 | 2122627 | 0 | 31 | 4 |
| 2018 | 24F | NULL | 1325 | 1320 | 0 | 5 | 0 | 4286820 | 7 | 1 | 0 |
| 2018 | 24Z | X | 1279 | 1261 | 18 | 0 | 0 | 1288948 | 0 | 25 | 107 |
| 2018 | 24A | X | 1267 | 1213 | 24 | 30 | 0 | 4140306 | 579 | 9 | 0 |
| 2018 | 24C | NULL | 1260 | 1252 | 6 | 2 | 0 | 39092197 | 0 | 0 | 0 |
| 2018 | 24C | X | 379 | 378 | 0 | 1 | 0 | 1520574 | 0 | 1 | 0 |
| 2018 | 24N | NULL | 29 | 29 | 0 | 0 | 0 | 76763 | 0 | 0 | 0 |
| 2018 | 24R | NULL | 4 | 3 | 1 | 0 | 0 | 12000 | 0 | 0 | 0 |
| 2018 | 24P | NULL | 1 | 0 | 1 | 0 | 0 | -1000 | 0 | 0 | 0 |
| 2020 | 24K | NULL | 788654 | 731456 | 45457 | 11741 | 0 | 564981869 | 0 | 11 | 1716 |
| 2020 | 24E | NULL | 53836 | 52259 | 460 | 1117 | 0 | 915457258 | 720 | 14 | 0 |
| 2020 | 24A | NULL | 26973 | 26649 | 136 | 188 | 0 | 2337381277 | 406 | 0 | 0 |
| 2020 | 24E | X | 5791 | 5528 | 91 | 172 | 0 | 90499279 | 2716 | 25 | 0 |
| 2020 | 24K | X | 3766 | 3576 | 185 | 5 | 0 | 2076969 | 1 | 30 | 3 |
| 2020 | 24Z | NULL | 2859 | 2817 | 29 | 13 | 0 | 1135828 | 0 | 0 | 104 |
| 2020 | 24A | X | 2335 | 2274 | 17 | 44 | 0 | 34835289 | 1759 | 2 | 0 |
| 2020 | 24C | NULL | 1246 | 1240 | 5 | 1 | 0 | 87045042 | 0 | 0 | 0 |
| 2020 | 24F | NULL | 1004 | 1002 | 0 | 2 | 0 | 5638350 | 0 | 0 | 0 |
| 2020 | 24C | X | 722 | 722 | 0 | 0 | 0 | 2426895 | 0 | 2 | 0 |
| 2020 | 24Z | X | 620 | 597 | 17 | 6 | 0 | 757777 | 4 | 9 | 44 |
| 2020 | 24N | NULL | 18 | 18 | 0 | 0 | 0 | 38267 | 0 | 0 | 0 |
| 2020 | 24K | Y | 4 | 0 | 4 | 0 | 0 | -4230 | 0 | 0 | 0 |
| 2020 | 24R | X | 1 | 1 | 0 | 0 | 0 | 5000 | 0 | 0 | 0 |
11_model_committee_staging¶
Creates stg_committee_contributions; returns candidate-reference coverage.
SQL: 11_model_committee_staging
-- 11_model_committee_staging
-- I keep the source records and add a flag for my Q2 definition.
-- This lets me reuse the same rules without deleting excluded records.
CREATE OR REPLACE VIEW
`fec-contributions-analysis.fec_analysis.stg_committee_contributions`
AS
WITH source_records AS (
SELECT
2018 AS cycle,
sub_id,
cmte_id,
cand_id,
transaction_dt,
transaction_tp,
memo_cd,
transaction_amt
FROM `bigquery-public-data.fec.committee_contributions_2018`
UNION ALL
SELECT
2020 AS cycle,
sub_id,
cmte_id,
cand_id,
transaction_dt,
transaction_tp,
memo_cd,
transaction_amt
FROM `bigquery-public-data.fec.committee_contributions_2020`
)
SELECT
cycle,
sub_id,
cmte_id,
NULLIF(TRIM(cand_id), '') AS cand_id,
transaction_dt,
transaction_tp,
memo_cd,
CAST(transaction_amt AS NUMERIC) AS amount,
COALESCE(
transaction_tp IN ('24K', '24Z')
AND transaction_amt > 0
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_dt BETWEEN
DATE(cycle - 1, 1, 1) AND DATE(cycle, 12, 31),
FALSE
) AS is_q2_in_scope
FROM source_records;
-- I use one reference row per cycle and candidate ID so the join
-- cannot multiply contribution records.
WITH candidates AS (
SELECT DISTINCT 2018 AS cycle, cand_id
FROM `bigquery-public-data.fec.candidate_2018`
UNION ALL
SELECT DISTINCT 2020 AS cycle, cand_id
FROM `bigquery-public-data.fec.candidate_2020`
)
SELECT
s.cycle,
CASE
WHEN s.cand_id IS NULL THEN 'missing_candidate_id'
WHEN c.cand_id IS NULL THEN 'not_in_candidate_reference'
ELSE 'matched_candidate'
END AS candidate_status,
COUNT(*) AS n_contributions,
COUNT(DISTINCT s.cand_id) AS n_candidate_ids,
SUM(s.amount) AS gross_amount
FROM
`fec-contributions-analysis.fec_analysis.stg_committee_contributions` AS s
LEFT JOIN candidates AS c
ON s.cycle = c.cycle
AND s.cand_id = c.cand_id
WHERE s.is_q2_in_scope
GROUP BY s.cycle, candidate_status
ORDER BY s.cycle, candidate_status;
Full results: 11_model_committee_staging (6 rows)
Result supplied during the analysis, September 14, 2026
| cycle | candidate_status | n_contributions | n_candidate_ids | gross_amount |
|---|---|---|---|---|
| 2018 | matched_candidate | 273257 | 1593 | 529515202 |
| 2018 | missing_candidate_id | 1254 | 0 | 1711645 |
| 2018 | not_in_candidate_reference | 1370 | 138 | 2727735 |
| 2020 | matched_candidate | 731123 | 1565 | 586126034 |
| 2020 | missing_candidate_id | 1684 | 0 | 1387129 |
| 2020 | not_in_candidate_reference | 1457 | 125 | 2083324 |
12_q2_candidate_summary¶
Returns the Q2 candidate count and distribution.
The full SQL appears inline above and in 12_q2_candidate_summary.sql.
Full results: 12_q2_candidate_summary (1 row)
Result supplied during the analysis, September 14, 2026
| n_reported_candidate_ids | total_attributed_amount | minimum_received | approximate_p25 | approximate_median | mean_received | approximate_p75 | approximate_p90 | approximate_p99 | maximum_received | population_stddev | top_10_percent_share_pct | n_contributions_without_candidate_id | amount_without_candidate_id |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2614 | 1120452295 | 8 | 3500 | 15800 | 428635.154934966 | 369013 | 1494038 | 3946544 | 49608977 | 1271873.2626009807 | 61.9099516 | 2938 | 3098774 |
13_qa_other_committee_profile¶
Profiles the broader committee transaction sources.
SQL: 13_qa_other_committee_profile
-- 13_qa_other_committee_profile
-- Q1 asks about contributions originating from committees, not just
-- those directed to candidates. I need to inspect the other committee
-- transactions before deciding which belong in that broader measure.
-- I also check record IDs before trusting the counts.
WITH records AS (
SELECT
2018 AS cycle,
sub_id,
transaction_tp,
memo_cd,
transaction_amt
FROM `bigquery-public-data.fec.other_committee_tx_2018`
UNION ALL
SELECT
2020 AS cycle,
sub_id,
transaction_tp,
memo_cd,
transaction_amt
FROM `bigquery-public-data.fec.other_committee_tx_2020`
)
SELECT
cycle,
transaction_tp,
memo_cd,
COUNT(*) AS n_rows,
COUNT(DISTINCT sub_id) AS n_distinct_ids,
COUNTIF(sub_id IS NULL) AS n_missing_ids,
COUNTIF(transaction_amt > 0) AS n_positive,
COUNTIF(transaction_amt < 0) AS n_negative,
COUNTIF(transaction_amt = 0) AS n_zero,
COUNTIF(transaction_amt IS NULL) AS n_missing_amount,
SUM(CAST(transaction_amt AS NUMERIC)) AS raw_total_amount
FROM records
GROUP BY cycle, transaction_tp, memo_cd
ORDER BY cycle, n_rows DESC;
Full results: 13_qa_other_committee_profile (158 rows)
Original JSON export supplied during analysis
| cycle | transaction_tp | memo_cd | n_rows | n_distinct_ids | n_missing_ids | n_positive | n_negative | n_zero | n_missing_amount | raw_total_amount |
|---|---|---|---|---|---|---|---|---|---|---|
| 2018 | 15J | X | 482913 | 482913 | 0 | 462890 | 5986 | 14037 | 0 | 340640595 |
| 2018 | 24K | NULL | 341001 | 341001 | 0 | 330051 | 9461 | 1489 | 0 | 1039249490 |
| 2018 | 18K | NULL | 57217 | 57217 | 0 | 56748 | 58 | 411 | 0 | 408130787 |
| 2018 | 24E | NULL | 35626 | 35626 | 0 | 35224 | 69 | 333 | 0 | 332016725 |
| 2018 | 24A | NULL | 20390 | 20390 | 0 | 20251 | 31 | 108 | 0 | 971606865 |
| 2018 | 18G | NULL | 17085 | 17085 | 0 | 17013 | 27 | 45 | 0 | 592332590 |
| 2018 | 24G | NULL | 17078 | 17078 | 0 | 16942 | 67 | 69 | 0 | 662342907 |
| 2018 | 18J | X | 12465 | 12465 | 0 | 12257 | 205 | 3 | 0 | 42130339 |
| 2018 | 24Z | NULL | 4550 | 4550 | 0 | 4521 | 18 | 11 | 0 | 3261978 |
| 2018 | 16C | NULL | 4543 | 4543 | 0 | 4517 | 5 | 21 | 0 | 250324566 |
| 2018 | 24K | X | 2107 | 2107 | 0 | 1881 | 226 | 0 | 0 | 3946606 |
| 2018 | 24E | X | 1950 | 1950 | 0 | 1870 | 42 | 38 | 0 | 12775853 |
| 2018 | 20C | NULL | 1837 | 1837 | 0 | 1832 | 1 | 4 | 0 | 56335080 |
| 2018 | 24Z | X | 1719 | 1719 | 0 | 1701 | 18 | 0 | 0 | 7004296 |
| 2018 | 24F | NULL | 1387 | 1387 | 0 | 1381 | 0 | 6 | 0 | 4533306 |
| 2018 | 24G | X | 1374 | 1374 | 0 | 1374 | 0 | 0 | 0 | 4348035 |
| 2018 | 24A | X | 1267 | 1267 | 0 | 1213 | 24 | 30 | 0 | 4140306 |
| 2018 | 24C | NULL | 1260 | 1260 | 0 | 1252 | 6 | 2 | 0 | 39092197 |
| 2018 | 22Z | NULL | 1188 | 1188 | 0 | 1119 | 62 | 7 | 0 | 3988071 |
| 2018 | 18G | X | 1103 | 1103 | 0 | 1088 | 4 | 11 | 0 | 3397040 |
| 2018 | 31J | X | 1024 | 1024 | 0 | 1018 | 6 | 0 | 0 | 32208721 |
| 2018 | 15J | NULL | 930 | 930 | 0 | 929 | 1 | 0 | 0 | 1632627 |
| 2018 | 42 | NULL | 882 | 882 | 0 | 870 | 3 | 9 | 0 | 18466751 |
| 2018 | 31K | NULL | 763 | 763 | 0 | 762 | 1 | 0 | 0 | 23582220 |
| 2018 | 42 | X | 443 | 443 | 0 | 434 | 9 | 0 | 0 | 800439 |
| 2018 | 24C | X | 379 | 379 | 0 | 378 | 0 | 1 | 0 | 1520574 |
| 2018 | 32J | X | 291 | 291 | 0 | 290 | 1 | 0 | 0 | 10432297 |
| 2018 | 20C | X | 244 | 244 | 0 | 244 | 0 | 0 | 0 | 34687375 |
| 2018 | 11J | X | 242 | 242 | 0 | 238 | 4 | 0 | 0 | 1075136 |
| 2018 | 18J | NULL | 215 | 215 | 0 | 215 | 0 | 0 | 0 | 6311297 |
| 2018 | 31G | NULL | 214 | 214 | 0 | 213 | 1 | 0 | 0 | 33577584 |
| 2018 | 15Z | NULL | 181 | 181 | 0 | 181 | 0 | 0 | 0 | 3860163 |
| 2018 | 32K | NULL | 177 | 177 | 0 | 177 | 0 | 0 | 0 | 5715702 |
| 2018 | 18K | X | 165 | 165 | 0 | 147 | 18 | 0 | 0 | 374584 |
| 2018 | 16G | NULL | 131 | 131 | 0 | 129 | 0 | 2 | 0 | 1629431 |
| 2018 | 31F | X | 125 | 125 | 0 | 125 | 0 | 0 | 0 | 1645000 |
| 2018 | 20G | NULL | 108 | 108 | 0 | 108 | 0 | 0 | 0 | 2390961 |
| 2018 | 32G | NULL | 90 | 90 | 0 | 90 | 0 | 0 | 0 | 10173714 |
| 2018 | 41 | NULL | 74 | 74 | 0 | 74 | 0 | 0 | 0 | 8452326 |
| 2018 | 24P | X | 65 | 65 | 0 | 65 | 0 | 0 | 0 | 23269 |
| 2018 | 22Z | X | 53 | 53 | 0 | 49 | 4 | 0 | 0 | 168367 |
| 2018 | 16C | X | 45 | 45 | 0 | 45 | 0 | 0 | 0 | 6078188 |
| 2018 | 30J | X | 45 | 45 | 0 | 44 | 1 | 0 | 0 | 2600150 |
| 2018 | 30K | NULL | 43 | 43 | 0 | 43 | 0 | 0 | 0 | 847088 |
| 2018 | 20F | NULL | 36 | 36 | 0 | 36 | 0 | 0 | 0 | 21857642 |
| 2018 | 24N | NULL | 34 | 34 | 0 | 34 | 0 | 0 | 0 | 110883 |
| 2018 | 32F | X | 29 | 29 | 0 | 29 | 0 | 0 | 0 | 971750 |
| 2018 | 31F | NULL | 15 | 15 | 0 | 15 | 0 | 0 | 0 | 530475 |
| 2018 | 31G | X | 15 | 15 | 0 | 15 | 0 | 0 | 0 | 1028932 |
| 2018 | 16F | NULL | 13 | 13 | 0 | 13 | 0 | 0 | 0 | 32314435 |
| 2018 | 40 | NULL | 9 | 9 | 0 | 9 | 0 | 0 | 0 | 234485 |
| 2018 | 24P | NULL | 7 | 7 | 0 | 5 | 2 | 0 | 0 | 2500 |
| 2018 | 11J | NULL | 6 | 6 | 0 | 6 | 0 | 0 | 0 | 21400 |
| 2018 | 30G | NULL | 6 | 6 | 0 | 6 | 0 | 0 | 0 | 1891030 |
| 2018 | 31J | NULL | 5 | 5 | 0 | 5 | 0 | 0 | 0 | 26612 |
| 2018 | 31K | X | 5 | 5 | 0 | 5 | 0 | 0 | 0 | 116000 |
| 2018 | 24R | NULL | 5 | 5 | 0 | 4 | 1 | 0 | 0 | 27000 |
| 2018 | 30F | X | 4 | 4 | 0 | 4 | 0 | 0 | 0 | 254350 |
| 2018 | 32G | X | 4 | 4 | 0 | 4 | 0 | 0 | 0 | 311134 |
| 2018 | 41Z | NULL | 2 | 2 | 0 | 2 | 0 | 0 | 0 | 117342 |
| 2018 | 20 | NULL | 2 | 2 | 0 | 2 | 0 | 0 | 0 | 101910 |
| 2018 | 16G | X | 2 | 2 | 0 | 2 | 0 | 0 | 0 | 2519 |
| 2018 | 20G | X | 2 | 2 | 0 | 1 | 0 | 1 | 0 | 1026 |
| 2018 | 15Z | X | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1171 |
| 2018 | 23Y | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 2700 |
| 2018 | 32K | X | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 5000 |
| 2018 | 10J | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 290000 |
| 2018 | 30G | X | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 211208 |
| 2018 | 32F | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 9114 |
| 2018 | 42Z | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 5000 |
| 2018 | 16F | X | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 60 |
| 2020 | 15J | X | 5627609 | 5627609 | 0 | 5205661 | 19274 | 402674 | 0 | 1465264073 |
| 2020 | 24K | NULL | 1532175 | 1532175 | 0 | 1412164 | 93402 | 26609 | 0 | 1606838874 |
| 2020 | 24E | NULL | 54445 | 54445 | 0 | 52866 | 460 | 1119 | 0 | 917775979 |
| 2020 | 18K | NULL | 53280 | 53280 | 0 | 53174 | 96 | 10 | 0 | 550607854 |
| 2020 | 24A | NULL | 27007 | 27007 | 0 | 26683 | 136 | 188 | 0 | 2337900002 |
| 2020 | 18G | NULL | 22463 | 22463 | 0 | 22317 | 113 | 33 | 0 | 2175457260 |
| 2020 | 24G | NULL | 21337 | 21337 | 0 | 21233 | 93 | 11 | 0 | 2444802031 |
| 2020 | 18J | X | 12861 | 12861 | 0 | 12788 | 69 | 4 | 0 | 48187962 |
| 2020 | 24E | X | 5829 | 5829 | 0 | 5566 | 91 | 172 | 0 | 90542818 |
| 2020 | 41 | NULL | 4894 | 4894 | 0 | 4830 | 59 | 5 | 0 | 57388182 |
| 2020 | 16C | NULL | 4623 | 4623 | 0 | 4605 | 3 | 15 | 0 | 239981302 |
| 2020 | 24K | X | 4498 | 4498 | 0 | 4283 | 210 | 5 | 0 | 5114015 |
| 2020 | 24Z | NULL | 3853 | 3853 | 0 | 3803 | 31 | 19 | 0 | 8206971 |
| 2020 | 42 | NULL | 2614 | 2614 | 0 | 2594 | 7 | 13 | 0 | 91417965 |
| 2020 | 24A | X | 2337 | 2337 | 0 | 2276 | 17 | 44 | 0 | 34855299 |
| 2020 | 31J | X | 2181 | 2181 | 0 | 2158 | 23 | 0 | 0 | 91580392 |
| 2020 | 20C | NULL | 2140 | 2140 | 0 | 2130 | 4 | 6 | 0 | 71594119 |
| 2020 | 41 | X | 2033 | 2033 | 0 | 1993 | 37 | 3 | 0 | 5097413 |
| 2020 | 24G | X | 1652 | 1652 | 0 | 1652 | 0 | 0 | 0 | 7090403 |
| 2020 | 22Z | NULL | 1515 | 1515 | 0 | 1380 | 133 | 2 | 0 | 4036067 |
| 2020 | 32J | X | 1479 | 1479 | 0 | 1477 | 2 | 0 | 0 | 60570552 |
| 2020 | 15J | NULL | 1424 | 1417 | 0 | 1207 | 207 | 2 | 8 | 1071841 |
| 2020 | 24C | NULL | 1246 | 1246 | 0 | 1240 | 5 | 1 | 0 | 87045042 |
| 2020 | 18G | X | 1150 | 1150 | 0 | 1141 | 8 | 1 | 0 | 2808355 |
| 2020 | 24F | NULL | 1004 | 1004 | 0 | 1002 | 0 | 2 | 0 | 5638350 |
| 2020 | 24Z | X | 988 | 988 | 0 | 964 | 18 | 6 | 0 | 4166978 |
| 2020 | 24C | X | 722 | 722 | 0 | 722 | 0 | 0 | 0 | 2426895 |
| 2020 | 31K | NULL | 585 | 585 | 0 | 585 | 0 | 0 | 0 | 25871164 |
| 2020 | 32K | NULL | 340 | 340 | 0 | 338 | 2 | 0 | 0 | 11297027 |
| 2020 | 15Z | NULL | 335 | 335 | 0 | 334 | 0 | 1 | 0 | 1060648 |
| 2020 | 30J | X | 290 | 290 | 0 | 290 | 0 | 0 | 0 | 15875009 |
| 2020 | 18K | X | 270 | 270 | 0 | 247 | 23 | 0 | 0 | 1251911 |
| 2020 | 11J | X | 261 | 261 | 0 | 259 | 2 | 0 | 0 | 1037473 |
| 2020 | 20C | X | 248 | 248 | 0 | 248 | 0 | 0 | 0 | 46371442 |
| 2020 | 31G | NULL | 224 | 224 | 0 | 221 | 3 | 0 | 0 | 76634640 |
| 2020 | 18J | NULL | 179 | 179 | 0 | 179 | 0 | 0 | 0 | 53648896 |
| 2020 | 16G | NULL | 165 | 165 | 0 | 164 | 0 | 1 | 0 | 323774 |
| 2020 | 32G | NULL | 152 | 152 | 0 | 149 | 3 | 0 | 0 | 59905430 |
| 2020 | 18U | NULL | 146 | 146 | 0 | 146 | 0 | 0 | 0 | 34830039 |
| 2020 | 16C | X | 119 | 119 | 0 | 119 | 0 | 0 | 0 | 37142444 |
| 2020 | 20G | NULL | 119 | 119 | 0 | 118 | 0 | 1 | 0 | 985142 |
| 2020 | 22Z | X | 109 | 109 | 0 | 104 | 5 | 0 | 0 | 517311 |
| 2020 | 42 | X | 107 | 107 | 0 | 102 | 5 | 0 | 0 | 249696 |
| 2020 | 40 | NULL | 96 | 96 | 0 | 96 | 0 | 0 | 0 | 8088673 |
| 2020 | 32F | X | 72 | 72 | 0 | 72 | 0 | 0 | 0 | 1672008 |
| 2020 | 30K | NULL | 69 | 69 | 0 | 69 | 0 | 0 | 0 | 3804191 |
| 2020 | 31F | X | 65 | 65 | 0 | 65 | 0 | 0 | 0 | 1369481 |
| 2020 | 31J | NULL | 60 | 60 | 0 | 60 | 0 | 0 | 0 | 48535 |
| 2020 | 31G | X | 56 | 56 | 0 | 53 | 3 | 0 | 0 | 2140609 |
| 2020 | 32K | X | 45 | 45 | 0 | 32 | 13 | 0 | 0 | 389950 |
| 2020 | 20F | NULL | 23 | 23 | 0 | 23 | 0 | 0 | 0 | 4341271 |
| 2020 | 30F | X | 18 | 18 | 0 | 18 | 0 | 0 | 0 | 786820 |
| 2020 | 24N | NULL | 18 | 18 | 0 | 18 | 0 | 0 | 0 | 38267 |
| 2020 | 31K | X | 17 | 17 | 0 | 3 | 14 | 0 | 0 | -266050 |
| 2020 | 30G | NULL | 17 | 17 | 0 | 17 | 0 | 0 | 0 | 17688340 |
| 2020 | 40 | X | 16 | 16 | 0 | 15 | 1 | 0 | 0 | 325580 |
| 2020 | 16F | NULL | 8 | 8 | 0 | 7 | 0 | 1 | 0 | 2118335 |
| 2020 | 40Z | NULL | 6 | 6 | 0 | 6 | 0 | 0 | 0 | 929647 |
| 2020 | 20 | NULL | 6 | 6 | 0 | 6 | 0 | 0 | 0 | 266809 |
| 2020 | 41Z | NULL | 6 | 6 | 0 | 6 | 0 | 0 | 0 | 135130 |
| 2020 | 24K | Y | 6 | 6 | 0 | 2 | 4 | 0 | 0 | -230 |
| 2020 | 32G | X | 5 | 5 | 0 | 5 | 0 | 0 | 0 | 124100 |
| 2020 | 15Z | X | 4 | 4 | 0 | 4 | 0 | 0 | 0 | 72897 |
| 2020 | 32J | NULL | 4 | 4 | 0 | 4 | 0 | 0 | 0 | 146500 |
| 2020 | 24R | NULL | 4 | 4 | 0 | 3 | 1 | 0 | 0 | 66258 |
| 2020 | 42Z | NULL | 4 | 4 | 0 | 4 | 0 | 0 | 0 | 210692 |
| 2020 | 31F | NULL | 4 | 4 | 0 | 4 | 0 | 0 | 0 | 1515698 |
| 2020 | 32F | NULL | 4 | 4 | 0 | 4 | 0 | 0 | 0 | 350200 |
| 2020 | 11J | NULL | 2 | 2 | 0 | 2 | 0 | 0 | 0 | 3500 |
| 2020 | 11AI-000793757 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
| 2020 | 30J | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1000 |
| 2020 | 10J | X | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 5000 |
| 2020 | 11AI-000793709 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
| 2020 | 15Z | Y | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 21 |
| 2020 | 22H | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 |
| 2020 | 18K | Y | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 367 |
| 2020 | 1403652 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
| 2020 | 11AI-000793708 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
| 2020 | 42Z | X | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 107500 |
| 2020 | 24R | X | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 5000 |
| 2020 | AC0BDB869538D4EF39C1 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
| 2020 | 20R | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 25000 |
| 2020 | 16C | Y | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 2556 |
| 2020 | 11AI-000791467 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
| 2020 | A207714E2F8BD47E48C7 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
| 2020 | 30F | NULL | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1000 |
| 2020 | 11AI-000791387 | NULL | 1 | 1 | 0 | 0 | 0 | 0 | 1 | NULL |
14_qa_committee_overlap¶
Checks overlap between broader and candidate committee sources.
SQL: 14_qa_committee_overlap
-- 14_qa_committee_overlap
-- I want to know whether the broader committee source already
-- contains the candidate contributions.
-- I group the broader source by ID first so repeated copies
-- cannot multiply records during the comparison.
WITH other_records AS (
SELECT
2018 AS cycle,
sub_id, cmte_id, transaction_dt,
transaction_tp, memo_cd, transaction_amt
FROM `bigquery-public-data.fec.other_committee_tx_2018`
UNION ALL
SELECT
2020 AS cycle,
sub_id, cmte_id, transaction_dt,
transaction_tp, memo_cd, transaction_amt
FROM `bigquery-public-data.fec.other_committee_tx_2020`
),
other_by_id AS (
SELECT
cycle,
sub_id,
COUNT(*) AS n_copies,
COUNT(DISTINCT TO_JSON_STRING(STRUCT(
cmte_id, transaction_dt, transaction_tp, transaction_amt
))) AS n_versions,
MIN(CAST(transaction_amt AS NUMERIC)) AS amount,
MIN(transaction_dt) AS transaction_dt,
MIN(cmte_id) AS cmte_id
FROM other_records
WHERE transaction_tp IN ('24K', '24Z')
AND transaction_amt > 0
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_dt BETWEEN
DATE(cycle - 1, 1, 1) AND DATE(cycle, 12, 31)
GROUP BY cycle, sub_id
),
candidate_records AS (
SELECT cycle, sub_id, amount, transaction_dt, cmte_id
FROM
`fec-contributions-analysis.fec_analysis.stg_committee_contributions`
WHERE is_q2_in_scope
)
SELECT
COALESCE(o.cycle, c.cycle) AS cycle,
CASE
WHEN o.cycle IS NULL THEN 'candidate_source_only'
WHEN c.cycle IS NULL THEN 'broader_source_only'
ELSE 'shared_id'
END AS source_coverage,
COUNT(*) AS n_ids,
SUM(COALESCE(o.n_copies - 1, 0)) AS extra_broader_copies,
COUNTIF(o.n_versions > 1) AS broader_ids_with_conflicts,
COUNTIF(
o.cycle IS NOT NULL AND c.cycle IS NOT NULL
AND (
o.amount IS DISTINCT FROM c.amount
OR o.transaction_dt IS DISTINCT FROM c.transaction_dt
OR o.cmte_id IS DISTINCT FROM c.cmte_id
)
) AS shared_ids_with_amount_date_or_committee_conflict
FROM other_by_id AS o
FULL OUTER JOIN candidate_records AS c
ON o.cycle = c.cycle
AND o.sub_id = c.sub_id
GROUP BY cycle, source_coverage
ORDER BY cycle, source_coverage;
Full results: 14_qa_committee_overlap (5 rows)
Result supplied during the analysis, September 14, 2026
| cycle | source_coverage | n_ids | extra_broader_copies | broader_ids_with_conflicts | shared_ids_with_amount_date_or_committee_conflict |
|---|---|---|---|---|---|
| 2018 | broader_source_only | 58672 | 0 | 0 | 0 |
| 2018 | shared_id | 275881 | 0 | 0 | 0 |
| 2020 | broader_source_only | 681712 | 0 | 0 | 0 |
| 2020 | candidate_source_only | 25 | 0 | 0 | 0 |
| 2020 | shared_id | 734239 | 0 | 0 | 0 |
15_qa_committee_unmatched_records¶
Inspects the 25 unmatched candidate records.
SQL: 15_qa_committee_unmatched_records
-- 15_qa_committee_unmatched_records
-- The filtered comparison left 25 candidate records unmatched.
-- I check the original broader table to see whether they are absent
-- or present with details that fail my contribution filters.
WITH qualifying_other_ids AS (
SELECT sub_id
FROM `bigquery-public-data.fec.other_committee_tx_2020`
WHERE transaction_tp IN ('24K', '24Z')
AND transaction_amt > 0
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_dt BETWEEN DATE '2019-01-01' AND DATE '2020-12-31'
),
unmatched_candidates AS (
SELECT
c.sub_id,
c.cmte_id,
c.amount,
c.transaction_dt,
c.transaction_tp,
c.memo_cd
FROM
`fec-contributions-analysis.fec_analysis.stg_committee_contributions` AS c
WHERE c.cycle = 2020
AND c.is_q2_in_scope
AND NOT EXISTS (
SELECT 1
FROM qualifying_other_ids AS o
WHERE o.sub_id = c.sub_id
)
)
SELECT
c.sub_id,
CASE
WHEN o.sub_id IS NULL THEN 'absent_from_broader_source'
ELSE 'present_with_different_details'
END AS finding,
c.amount AS candidate_source_amount,
o.transaction_amt AS broader_source_amount,
c.transaction_dt AS candidate_source_date,
o.transaction_dt AS broader_source_date,
c.transaction_tp AS candidate_source_type,
o.transaction_tp AS broader_source_type,
c.memo_cd AS candidate_source_memo,
o.memo_cd AS broader_source_memo,
c.cmte_id AS candidate_source_committee,
o.cmte_id AS broader_source_committee
FROM unmatched_candidates AS c
LEFT JOIN `bigquery-public-data.fec.other_committee_tx_2020` AS o
ON c.sub_id = o.sub_id
ORDER BY c.sub_id;
Full results: 15_qa_committee_unmatched_records (25 rows)
Original JSON export supplied during analysis
| sub_id | finding | candidate_source_amount | broader_source_amount | candidate_source_date | broader_source_date | candidate_source_type | broader_source_type | candidate_source_memo | broader_source_memo | candidate_source_committee | broader_source_committee |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 4120220241074206716 | absent_from_broader_source | 2000 | NULL | 2020-02-19 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4120220241074206718 | absent_from_broader_source | 2000 | NULL | 2020-02-19 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4120220241074206719 | absent_from_broader_source | 2000 | NULL | 2020-02-19 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036353 | absent_from_broader_source | 1000 | NULL | 2020-12-04 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036354 | absent_from_broader_source | 1000 | NULL | 2020-12-16 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036428 | absent_from_broader_source | 1000 | NULL | 2020-09-30 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036429 | absent_from_broader_source | 2000 | NULL | 2020-07-22 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036430 | absent_from_broader_source | 1000 | NULL | 2020-07-22 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036432 | absent_from_broader_source | 1000 | NULL | 2020-09-09 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036433 | absent_from_broader_source | 2000 | NULL | 2020-05-07 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036434 | absent_from_broader_source | 1000 | NULL | 2020-05-07 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036724 | absent_from_broader_source | 2000 | NULL | 2019-10-17 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036726 | absent_from_broader_source | 2000 | NULL | 2019-07-09 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036728 | absent_from_broader_source | 1000 | NULL | 2019-10-17 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036729 | absent_from_broader_source | 1000 | NULL | 2019-08-26 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036731 | absent_from_broader_source | 2000 | NULL | 2019-09-16 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036732 | absent_from_broader_source | 2000 | NULL | 2019-09-16 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036733 | absent_from_broader_source | 2000 | NULL | 2019-09-16 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036735 | absent_from_broader_source | 1000 | NULL | 2019-10-17 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036737 | absent_from_broader_source | 2000 | NULL | 2019-12-16 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036738 | absent_from_broader_source | 1000 | NULL | 2019-12-16 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036772 | absent_from_broader_source | 2000 | NULL | 2019-02-18 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036774 | absent_from_broader_source | 2000 | NULL | 2019-02-18 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036775 | absent_from_broader_source | 2000 | NULL | 2019-02-18 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
| 4121620241077036776 | absent_from_broader_source | 1000 | NULL | 2019-06-17 | NULL | 24K | NULL | NULL | NULL | C00652883 | NULL |
16_model_committee_q1¶
Creates stg_committee_q1; returns eligible committee totals by cycle.
SQL: 16_model_committee_q1
-- 16_model_committee_q1
-- I use the broader committee source for Q1 and keep the same
-- contribution definition across both cycles.
-- The view preserves excluded records so the rules remain reviewable.
CREATE OR REPLACE VIEW
`fec-contributions-analysis.fec_analysis.stg_committee_q1`
AS
WITH records AS (
SELECT
2018 AS cycle,
sub_id,
cmte_id,
other_id,
transaction_dt,
transaction_tp,
memo_cd,
transaction_amt
FROM `bigquery-public-data.fec.other_committee_tx_2018`
UNION ALL
SELECT
2020 AS cycle,
sub_id,
cmte_id,
other_id,
transaction_dt,
transaction_tp,
memo_cd,
transaction_amt
FROM `bigquery-public-data.fec.other_committee_tx_2020`
)
SELECT
cycle,
sub_id,
cmte_id,
other_id,
transaction_dt,
transaction_tp,
memo_cd,
CAST(transaction_amt AS NUMERIC) AS amount,
COALESCE(
transaction_tp IN ('24K', '24Z')
AND transaction_amt > 0
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_dt BETWEEN
DATE(cycle - 1, 1, 1) AND DATE(cycle, 12, 31),
FALSE
) AS is_q1_in_scope
FROM records;
-- The qualifying records passed the ID check in query 14.
-- I check the counts again here to verify the saved view.
SELECT
cycle,
COUNT(*) AS n_committee_contributions,
COUNT(DISTINCT sub_id) AS n_distinct_ids,
SUM(amount) AS gross_committee_amount,
MIN(transaction_dt) AS earliest_included_date,
MAX(transaction_dt) AS latest_included_date
FROM `fec-contributions-analysis.fec_analysis.stg_committee_q1`
WHERE is_q1_in_scope
GROUP BY cycle
ORDER BY cycle;
Full results: 16_model_committee_q1 (2 rows)
Result supplied during the analysis, September 14, 2026
| cycle | n_committee_contributions | n_distinct_ids | gross_committee_amount | earliest_included_date | latest_included_date |
|---|---|---|---|---|---|
| 2018 | 334553 | 334553 | 1067216231 | 2017-01-01 | 2018-12-31 |
| 2020 | 1415951 | 1415951 | 1653263736 | 2019-01-01 | 2020-12-31 |
17_model_individual_q1¶
Creates stg_individual_q1; returns eligible individual totals by cycle.
SQL: 17_model_individual_q1
-- 17_model_individual_q1
-- I count individual contribution receipts, rather than every
-- transaction involving an individual.
-- I remove repeated 2020 copies using the fields verified earlier.
-- Excluded records remain in the view for review.
CREATE OR REPLACE VIEW
`fec-contributions-analysis.fec_analysis.stg_individual_q1`
AS
WITH records AS (
SELECT
2018 AS cycle,
sub_id,
cmte_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
transaction_amt
FROM `bigquery-public-data.fec.indiv18`
UNION ALL
SELECT DISTINCT
2020 AS cycle,
sub_id,
cmte_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
transaction_amt
FROM `bigquery-public-data.fec.indiv20`
)
SELECT
cycle,
sub_id,
cmte_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
CAST(transaction_amt AS NUMERIC) AS amount,
COALESCE(
entity_tp = 'IND'
AND transaction_tp IN (
'10', '15', '15E',
'30', '30E', '31', '31E', '32', '32E'
)
AND transaction_amt > 0
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_dt BETWEEN
DATE(cycle - 1, 1, 1) AND DATE(cycle, 12, 31),
FALSE
) AS is_q1_in_scope
FROM records;
SELECT
cycle,
COUNT(*) AS n_individual_contributions,
COUNT(DISTINCT sub_id) AS n_distinct_ids,
SUM(amount) AS gross_individual_amount,
MIN(transaction_dt) AS earliest_included_date,
MAX(transaction_dt) AS latest_included_date
FROM `fec-contributions-analysis.fec_analysis.stg_individual_q1`
WHERE is_q1_in_scope
GROUP BY cycle
ORDER BY cycle;
Full results: 17_model_individual_q1 (2 rows)
Result supplied during the analysis, September 14, 2026
| cycle | n_individual_contributions | n_distinct_ids | gross_individual_amount | earliest_included_date | latest_included_date |
|---|---|---|---|---|---|
| 2018 | 15088198 | 15088198 | 4021953507 | 2017-01-01 | 2018-12-31 |
| 2020 | 64759763 | 64759763 | 10058325757 | 2019-01-01 | 2020-12-31 |
18_model_monthly_contributions¶
Creates fct_contributions and agg_contributions_monthly; returns the peak month.
SQL: 18_model_monthly_contributions
-- 18_model_monthly_contributions
-- I combine the two defined contribution measures in a reusable view.
-- Each record is identified by its source, cycle, and record ID.
-- I then save monthly totals to avoid rescanning the large sources
-- whenever I need a chart or a monthly comparison.
CREATE OR REPLACE VIEW
`fec-contributions-analysis.fec_analysis.fct_contributions`
AS
SELECT
'individual' AS source_type,
cycle,
sub_id,
transaction_dt,
amount
FROM `fec-contributions-analysis.fec_analysis.stg_individual_q1`
WHERE is_q1_in_scope
UNION ALL
SELECT
'committee' AS source_type,
cycle,
sub_id,
transaction_dt,
amount
FROM `fec-contributions-analysis.fec_analysis.stg_committee_q1`
WHERE is_q1_in_scope;
CREATE OR REPLACE TABLE
`fec-contributions-analysis.fec_analysis.agg_contributions_monthly`
AS
SELECT
DATE_TRUNC(transaction_dt, MONTH) AS contribution_month,
source_type,
COUNT(*) AS n_contributions,
SUM(amount) AS gross_amount
FROM `fec-contributions-analysis.fec_analysis.fct_contributions`
GROUP BY contribution_month, source_type;
-- I include every calendar month so the previous value really is
-- the previous month, even if a month has no contribution records.
WITH calendar AS (
SELECT contribution_month
FROM UNNEST(
GENERATE_DATE_ARRAY(
DATE '2017-01-01',
DATE '2020-12-01',
INTERVAL 1 MONTH
)
) AS contribution_month
),
monthly AS (
SELECT
c.contribution_month,
COALESCE(SUM(m.gross_amount), 0) AS total_amount
FROM calendar AS c
LEFT JOIN
`fec-contributions-analysis.fec_analysis.agg_contributions_monthly` AS m
USING (contribution_month)
GROUP BY c.contribution_month
),
growth AS (
SELECT
contribution_month,
total_amount,
LAG(total_amount) OVER (
ORDER BY contribution_month
) AS prior_month_amount
FROM monthly
)
SELECT
contribution_month,
total_amount,
prior_month_amount,
100 * SAFE_DIVIDE(
total_amount - prior_month_amount,
prior_month_amount
) AS month_over_month_growth_pct
FROM growth
QUALIFY DENSE_RANK() OVER (ORDER BY total_amount DESC) = 1;
Full results: 18_model_monthly_contributions (1 row)
Result supplied during the analysis, September 14, 2026
| contribution_month | total_amount | prior_month_amount | month_over_month_growth_pct |
|---|---|---|---|
| 2020-10-01 | 2017565781 | 1793420123 | 12.4982237 |
19_qa_monthly_reconciliation¶
Reconciles monthly aggregates to recorded source totals.
SQL: 19_qa_monthly_reconciliation
-- 19_qa_monthly_reconciliation
-- I want to confirm that grouping records by month did not change
-- the contribution counts or amounts calculated in the earlier steps.
-- Expected values come from queries 16 and 17.
WITH expected AS (
SELECT
'individual' AS source_type,
79847961 AS expected_count,
NUMERIC '14080279264' AS expected_amount
UNION ALL
SELECT
'committee',
1750504,
NUMERIC '2720479967'
),
actual AS (
SELECT
source_type,
SUM(n_contributions) AS actual_count,
SUM(gross_amount) AS actual_amount,
COUNT(*) AS n_month_rows,
COUNT(DISTINCT contribution_month) AS n_distinct_months,
MIN(contribution_month) AS first_month,
MAX(contribution_month) AS last_month
FROM
`fec-contributions-analysis.fec_analysis.agg_contributions_monthly`
GROUP BY source_type
)
SELECT
COALESCE(e.source_type, a.source_type) AS source_type,
e.expected_count,
a.actual_count,
e.expected_amount,
a.actual_amount,
a.n_month_rows,
a.n_distinct_months,
a.first_month,
a.last_month,
COALESCE(
a.actual_count = e.expected_count
AND a.actual_amount = e.expected_amount
AND a.n_month_rows = 48
AND a.n_distinct_months = 48
AND a.first_month = DATE '2017-01-01'
AND a.last_month = DATE '2020-12-01',
FALSE
) AS check_passed
FROM expected AS e
FULL OUTER JOIN actual AS a
USING (source_type)
ORDER BY source_type;
Full results: 19_qa_monthly_reconciliation (2 rows)
Result supplied during the analysis, September 14, 2026
| source_type | expected_count | actual_count | expected_amount | actual_amount | n_month_rows | n_distinct_months | first_month | last_month | check_passed |
|---|---|---|---|---|---|---|---|---|---|
| committee | 1750504 | 1750504 | 2720479967 | 2720479967 | 48 | 48 | 2017-01-01 | 2020-12-01 | true |
| individual | 79847961 | 79847961 | 14080279264 | 14080279264 | 48 | 48 | 2017-01-01 | 2020-12-01 | true |
20_qa_operating_expenditures¶
Profiles operating-expenditure coverage for the Q3 plan.
The full SQL appears inline above and in 20_qa_operating_expenditures.sql.
Full results: 20_qa_operating_expenditures (4 rows)
Result supplied during the analysis, September 14, 2026
| source_table | n_rows | n_distinct_ids | n_missing_ids | earliest_report_year | latest_report_year | earliest_date | latest_date | n_missing_dates | n_dates_in_cycle | n_missing_payee_names | n_missing_states | n_missing_entity_types | n_negative_amounts | raw_total_amount |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| opex_2018 | 1800080 | 1800080 | 0 | 2017 | 2018 | NULL | NULL | 1800080 | 0 | 1635 | 6515 | 167889 | 12878 | 4584149613.83 |
| opex_2020 | 1800080 | 1800080 | 0 | 2017 | 2018 | NULL | NULL | 1800080 | 0 | 1635 | 6515 | 167889 | 12878 | 4584149613.83 |
| oppexp18 | 1800000 | 1800000 | 0 | 2017 | 2018 | 1930-02-18 | 9201-10-02 | 252 | 1795535 | 1568 | 6486 | 167828 | 12878 | 4583771987.38 |
| oppexp20 | 2315382 | 2315382 | 0 | 2019 | 2020 | 1919-06-19 | 2029-12-11 | 15 | 2310905 | 149 | 6457 | 4593 | 25603 | 12442584070.620000003 |
21_monthly_chart_data¶
Returns the 48 monthly chart rows and growth rates.
The full SQL appears inline above and in 21_monthly_chart_data.sql.
Full results: 21_monthly_chart_data (48 rows)
Original monthly CSV export
| contribution_month | individual_amount | committee_amount | total_amount | month_over_month_growth_pct |
|---|---|---|---|---|
| 2017-01-01 | 49567473 | 24308484 | 73875957 | |
| 2017-02-01 | 76271836 | 31988292 | 108260128 | 46.5431142 |
| 2017-03-01 | 131847527 | 57101976 | 188949503 | 74.532865 |
| 2017-04-01 | 87810398 | 20541094 | 108351492 | -42.655847 |
| 2017-05-01 | 103127364 | 30583825 | 133711189 | 23.405028 |
| 2017-06-01 | 144710554 | 48804319 | 193514873 | 44.7260132 |
| 2017-07-01 | 70672550 | 19567396 | 90239946 | -53.3679533 |
| 2017-08-01 | 98057010 | 17493667 | 115550677 | 28.0482559 |
| 2017-09-01 | 132645697 | 40738850 | 173384547 | 50.0506544 |
| 2017-10-01 | 109783825 | 31356735 | 141140560 | -18.5968055 |
| 2017-11-01 | 110312681 | 28797568 | 139110249 | -1.4385029 |
| 2017-12-01 | 166671454 | 39961046 | 206632500 | 48.5386602 |
| 2018-01-01 | 98433315 | 32105970 | 130539285 | -36.8253856 |
| 2018-02-01 | 137418195 | 31883001 | 169301196 | 29.693675 |
| 2018-03-01 | 229237757 | 58844261 | 288082018 | 70.1594701 |
| 2018-04-01 | 161175275 | 36718946 | 197894221 | -31.3062917 |
| 2018-05-01 | 214108177 | 47762799 | 261870976 | 32.3287637 |
| 2018-06-01 | 237070789 | 69730091 | 306800880 | 17.1572676 |
| 2018-07-01 | 184811518 | 44256017 | 229067535 | -25.3367412 |
| 2018-08-01 | 290489243 | 53434002 | 343923245 | 50.1405448 |
| 2018-09-01 | 428266710 | 113680353 | 541947063 | 57.5779104 |
| 2018-10-01 | 583983966 | 156276758 | 740260724 | 36.5928104 |
| 2018-11-01 | 113523662 | 20113688 | 133637350 | -81.9472592 |
| 2018-12-01 | 61956531 | 11167093 | 73123624 | -45.2820458 |
| 2019-01-01 | 58011211 | 21653498 | 79664709 | 8.9452418 |
| 2019-02-01 | 96549699 | 29619176 | 126168875 | 58.3748646 |
| 2019-03-01 | 179534181 | 60145980 | 239680161 | 89.9677405 |
| 2019-04-01 | 134298936 | 20261056 | 154559992 | -35.5140653 |
| 2019-05-01 | 137346503 | 30469750 | 167816253 | 8.5767739 |
| 2019-06-01 | 205688694 | 62775620 | 268464314 | 59.9751569 |
| 2019-07-01 | 125594238 | 21929119 | 147523357 | -45.0491744 |
| 2019-08-01 | 144125750 | 16109614 | 160235364 | 8.6169453 |
| 2019-09-01 | 224992402 | 45853963 | 270846365 | 69.0303303 |
| 2019-10-01 | 196054907 | 30881371 | 226936278 | -16.2121751 |
| 2019-11-01 | 194666050 | 27292097 | 221958147 | -2.193625 |
| 2019-12-01 | 296847278 | 45125076 | 341972354 | 54.0706474 |
| 2020-01-01 | 242324355 | 44241458 | 286565813 | -16.2020527 |
| 2020-02-01 | 380755754 | 72418599 | 453174353 | 58.1397126 |
| 2020-03-01 | 327282733 | 77474130 | 404756863 | -10.6840755 |
| 2020-04-01 | 250018392 | 24915800 | 274934192 | -32.0742359 |
| 2020-05-01 | 323522938 | 32797646 | 356320584 | 29.6021355 |
| 2020-06-01 | 595085887 | 76929917 | 672015804 | 88.598648 |
| 2020-07-01 | 648978358 | 74455147 | 723433505 | 7.6512637 |
| 2020-08-01 | 1020656509 | 96715737 | 1117372246 | 54.454036 |
| 2020-09-01 | 1566972414 | 226447709 | 1793420123 | 60.5033711 |
| 2020-10-01 | 1742781090 | 274784691 | 2017565781 | 12.4982237 |
| 2020-11-01 | 850871618 | 85630193 | 936501811 | -53.5825885 |
| 2020-12-01 | 115365860 | 154336389 | 269702249 | -71.2010969 |
22_qa_largest_candidate¶
Breaks down the largest candidate total by committee and cycle.
SQL: 22_qa_largest_candidate
-- 22_qa_largest_candidate
-- The largest candidate total in Q2 is far above anything a single PAC
-- could give, so I break it down by cycle, committee and transaction type
-- to see where the money is reported from.
WITH totals AS (
SELECT
cand_id,
SUM(amount) AS total_amount
FROM `fec-contributions-analysis.fec_analysis.stg_committee_contributions`
WHERE is_q2_in_scope
AND cand_id IS NOT NULL
GROUP BY cand_id
),
top_candidate AS (
SELECT *
FROM totals
QUALIFY DENSE_RANK() OVER (ORDER BY total_amount DESC) = 1
)
SELECT
s.cand_id,
s.cycle,
s.cmte_id,
s.transaction_tp,
COUNT(*) AS n_records,
SUM(s.amount) AS amount,
MAX(s.amount) AS largest_record
FROM `fec-contributions-analysis.fec_analysis.stg_committee_contributions` AS s
JOIN top_candidate AS t
USING (cand_id)
WHERE s.is_q2_in_scope
GROUP BY 1, 2, 3, 4
ORDER BY amount DESC;
Full results: 22_qa_largest_candidate (213 rows)
BigQuery follow-up run; job IDs in query_jobs.json
| cand_id | cycle | cmte_id | transaction_tp | n_records | amount | largest_record |
|---|---|---|---|---|---|---|
| P80001571 | 2020 | C00694323 | 24K | 257625 | 48407633 | 5600 |
| P80001571 | 2020 | C00688069 | 24Z | 7 | 129095 | 45601 |
| P80001571 | 2020 | C00570945 | 24K | 5 | 25000 | 5000 |
| P80001571 | 2020 | C00295527 | 24K | 4 | 20000 | 5000 |
| P80001571 | 2020 | C00326082 | 24K | 3 | 15000 | 5000 |
| P80001571 | 2020 | C00409276 | 24K | 3 | 15000 | 5000 |
| P80001571 | 2020 | C00592089 | 24K | 3 | 15000 | 5000 |
| P80001571 | 2020 | C00693796 | 24K | 3 | 14400 | 10000 |
| P80001571 | 2020 | C00719971 | 24K | 4 | 12800 | 5000 |
| P80001571 | 2020 | C00395590 | 24K | 1 | 12596 | 12596 |
| P80001571 | 2020 | C00038505 | 24K | 7 | 11640 | 2852 |
| P80001571 | 2020 | C00564385 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00543207 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00344648 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00368696 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00244533 | 24K | 1 | 10000 | 10000 |
| P80001571 | 2020 | C00415208 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00545079 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00607374 | 24K | 6 | 10000 | 2500 |
| P80001571 | 2020 | C00428052 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00410621 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00691162 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00497842 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00496257 | 24K | 1 | 10000 | 10000 |
| P80001571 | 2020 | C00440032 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00632893 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00445379 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00569921 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00467431 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00547893 | 24K | 5 | 10000 | 2000 |
| P80001571 | 2020 | C00493072 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00439521 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00500025 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00570770 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00521823 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00328401 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00493924 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00503151 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00536540 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00597062 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00469429 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00390161 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00489427 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00442368 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00582726 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00571208 | 24K | 2 | 10000 | 5000 |
| P80001571 | 2020 | C00316455 | 24K | 4 | 9000 | 2600 |
| P80001571 | 2020 | C00330886 | 24K | 3 | 6000 | 2000 |
| P80001571 | 2020 | C00691147 | 24K | 2 | 5600 | 2800 |
| P80001571 | 2020 | C00565630 | 24K | 2 | 5600 | 2800 |
| P80001571 | 2020 | C00680322 | 24K | 2 | 5600 | 2800 |
| P80001571 | 2018 | C00640664 | 24K | 2 | 5400 | 2700 |
| P80001571 | 2020 | C00611400 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00589994 | 24K | 2 | 5000 | 2500 |
| P80001571 | 2020 | C00652305 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00571216 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00531764 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00461137 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00726729 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00459123 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00467167 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00448373 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00398750 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00526715 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00711119 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00726224 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00298000 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00687020 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00010421 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00279331 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00130658 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00510248 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00559146 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00692087 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00165365 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00053553 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00571521 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00712588 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00448696 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00015552 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00693671 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00525212 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00576249 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00214981 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00376038 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00583153 | 24K | 2 | 5000 | 2800 |
| P80001571 | 2020 | C00094573 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00540146 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00647487 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00691915 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00754283 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00563247 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00041590 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00325076 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00551184 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00010793 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00486050 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00228924 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00481531 | 24K | 1 | 5000 | 5000 |
| P80001571 | 2020 | C00640664 | 24K | 2 | 4600 | 2300 |
| P80001571 | 2020 | C00707992 | 24K | 3 | 4500 | 3000 |
| P80001571 | 2020 | C00187450 | 24K | 2 | 4417 | 3280 |
| P80001571 | 2020 | C00729913 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00607838 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00458463 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00632257 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00476820 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00459297 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00444620 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00473371 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00443218 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00393348 | 24K | 1 | 4000 | 4000 |
| P80001571 | 2020 | C00304758 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00653147 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00498345 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00482737 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00496760 | 24K | 1 | 4000 | 4000 |
| P80001571 | 2020 | C00541862 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00545749 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00313247 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00359034 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00485540 | 24K | 2 | 4000 | 2800 |
| P80001571 | 2020 | C00436386 | 24K | 1 | 4000 | 4000 |
| P80001571 | 2020 | C00394957 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00653220 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00632067 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00416594 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00610451 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00492785 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00296640 | 24K | 1 | 4000 | 4000 |
| P80001571 | 2020 | C00504704 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00500207 | 24K | 1 | 4000 | 4000 |
| P80001571 | 2020 | C00608695 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00257337 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00666396 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00482984 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00376939 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00372532 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00445023 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00658401 | 24K | 1 | 4000 | 4000 |
| P80001571 | 2020 | C00582304 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00462374 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00437913 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00671891 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00311043 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00458679 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00543967 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00468579 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00420935 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00668491 | 24K | 2 | 4000 | 2000 |
| P80001571 | 2020 | C00703173 | 24K | 2 | 4000 | 2800 |
| P80001571 | 2020 | C00728766 | 24K | 2 | 3678 | 2800 |
| P80001571 | 2020 | C00758730 | 24K | 9 | 3400 | 2042 |
| P80001571 | 2020 | C00325464 | 24K | 1 | 2877 | 2877 |
| P80001571 | 2020 | C00100289 | 24K | 1 | 2800 | 2800 |
| P80001571 | 2020 | C00567545 | 24K | 1 | 2800 | 2800 |
| P80001571 | 2020 | C00745265 | 24K | 1 | 2800 | 2800 |
| P80001571 | 2020 | C00376756 | 24K | 2 | 2800 | 1800 |
| P80001571 | 2020 | C00708172 | 24K | 1 | 2800 | 2800 |
| P80001571 | 2020 | C00367680 | 24K | 1 | 2800 | 2800 |
| P80001571 | 2020 | C00641142 | 24K | 1 | 2800 | 2800 |
| P80001571 | 2020 | C00241323 | 24K | 1 | 2700 | 2700 |
| P80001571 | 2020 | C00499798 | 24K | 1 | 2500 | 2500 |
| P80001571 | 2020 | C00663914 | 24K | 2 | 2500 | 2000 |
| P80001571 | 2020 | C00691626 | 24K | 1 | 2500 | 2500 |
| P80001571 | 2020 | C00489906 | 24K | 1 | 2500 | 2500 |
| P80001571 | 2020 | C00681643 | 24K | 1 | 2500 | 2500 |
| P80001571 | 2020 | C00607721 | 24K | 1 | 2500 | 2500 |
| P80001571 | 2020 | C00404392 | 24K | 1 | 2500 | 2500 |
| P80001571 | 2020 | C00332296 | 24K | 3 | 2192 | 1235 |
| P80001571 | 2020 | C00647420 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00548289 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00654723 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00493361 | 24K | 2 | 2000 | 1000 |
| P80001571 | 2020 | C00460063 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00493809 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00550707 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00492231 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00548180 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00368522 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00477745 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00554949 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00544445 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00460659 | 24K | 1 | 2000 | 2000 |
| P80001571 | 2020 | C00130807 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2018 | C00527275 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00724229 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00760603 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00717959 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00577676 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00524314 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00680405 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00760256 | 24K | 2 | 1000 | 500 |
| P80001571 | 2020 | C00669069 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00633164 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00303305 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00487942 | 24K | 1 | 1000 | 1000 |
| P80001571 | 2020 | C00723742 | 24K | 1 | 750 | 750 |
| P80001571 | 2020 | C00041236 | 24K | 1 | 650 | 650 |
| P80001571 | 2020 | C00321141 | 24K | 1 | 634 | 634 |
| P80001571 | 2020 | C00031088 | 24K | 1 | 633 | 633 |
| P80001571 | 2020 | C00706085 | 24K | 1 | 600 | 600 |
| P80001571 | 2020 | C00078196 | 24K | 1 | 571 | 571 |
| P80001571 | 2020 | C00512202 | 24K | 1 | 500 | 500 |
| P80001571 | 2020 | C00544262 | 24K | 1 | 500 | 500 |
| P80001571 | 2020 | C00014092 | 24K | 1 | 398 | 398 |
| P80001571 | 2020 | C00257279 | 24K | 1 | 313 | 313 |
| P80001571 | 2020 | C00697326 | 24K | 1 | 250 | 250 |
| P80001571 | 2020 | C00711689 | 24K | 2 | 130 | 100 |
| P80001571 | 2018 | C00355792 | 24K | 1 | 100 | 100 |
| P80001571 | 2020 | C00461145 | 24K | 1 | 50 | 50 |
| P80001571 | 2020 | C00355792 | 24K | 1 | 35 | 35 |
| P80001571 | 2020 | C00004606 | 24K | 1 | 35 | 35 |
23_qa_intermediary_reporting¶
Separates conduit and recipient reporting groups.
SQL: 23_qa_intermediary_reporting
-- 23_qa_intermediary_reporting
-- Earmarked donations can be reported by the conduit (ActBlue or WinRed)
-- and again by the recipient committee. I split the individual records
-- by who reported them and by transaction type, and flag which rows pass
-- my Q1 transaction filter.
WITH committees AS (
SELECT
2018 AS cycle,
cmte_id,
STRING_AGG(DISTINCT cmte_nm, '; ') AS cmte_nm
FROM `bigquery-public-data.fec.committee_2018`
GROUP BY 1, 2
UNION ALL
SELECT
2020,
cmte_id,
STRING_AGG(DISTINCT cmte_nm, '; ')
FROM `bigquery-public-data.fec.committee_2020`
GROUP BY 1, 2
),
records AS (
SELECT DISTINCT
2018 AS cycle,
sub_id,
cmte_id,
other_id,
transaction_tp,
transaction_dt,
CAST(transaction_amt AS NUMERIC) AS amount
FROM `bigquery-public-data.fec.indiv18`
WHERE entity_tp = 'IND'
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_amt > 0
AND transaction_dt BETWEEN '2017-01-01' AND '2018-12-31'
UNION ALL
SELECT DISTINCT
2020,
sub_id,
cmte_id,
other_id,
transaction_tp,
transaction_dt,
CAST(transaction_amt AS NUMERIC)
FROM `bigquery-public-data.fec.indiv20`
WHERE entity_tp = 'IND'
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_amt > 0
AND transaction_dt BETWEEN '2019-01-01' AND '2020-12-31'
)
SELECT
r.cycle,
CASE
WHEN REGEXP_CONTAINS(UPPER(c.cmte_nm), r'ACTBLUE|WINRED')
THEN CONCAT('reported by ', c.cmte_nm)
WHEN REGEXP_CONTAINS(UPPER(o.cmte_nm), r'ACTBLUE|WINRED')
THEN CONCAT('other_id: ', o.cmte_nm)
ELSE 'other reporting'
END AS flow,
r.transaction_tp,
r.transaction_tp IN (
'10', '15', '15E',
'30', '30E', '31', '31E', '32', '32E'
) AS included_in_q1,
COUNT(*) AS n_records,
SUM(amount) AS amount
FROM records AS r
LEFT JOIN committees AS c
ON r.cycle = c.cycle
AND r.cmte_id = c.cmte_id
LEFT JOIN committees AS o
ON r.cycle = o.cycle
AND r.other_id = o.cmte_id
GROUP BY 1, 2, 3, 4
ORDER BY 1, 2, amount DESC;
Full results: 23_qa_intermediary_reporting (49 rows)
BigQuery follow-up run; job IDs in query_jobs.json
| cycle | flow | transaction_tp | included_in_q1 | n_records | amount |
|---|---|---|---|---|---|
| 2018 | other reporting | 15 | true | 9136570 | 2322716093 |
| 2018 | other reporting | 10 | true | 200467 | 926105496 |
| 2018 | other reporting | 22Y | false | 161803 | 65475860 |
| 2018 | other reporting | 15E | true | 117564 | 47468311 |
| 2018 | other reporting | 31 | true | 1747 | 41003691 |
| 2018 | other reporting | 15C | false | 1601 | 39425560 |
| 2018 | other reporting | 24T | false | 70178 | 23081690 |
| 2018 | other reporting | 32 | true | 565 | 15204467 |
| 2018 | other reporting | 30 | true | 187 | 7610886 |
| 2018 | other reporting | 24I | false | 7516 | 3796105 |
| 2018 | other reporting | 20Y | false | 554 | 1919861 |
| 2018 | other reporting | 11 | false | 435 | 956376 |
| 2018 | other reporting | 21Y | false | 4 | 9700 |
| 2018 | other_id: ACTBLUE | 15E | true | 5126876 | 657259968 |
| 2018 | other_id: ACTBLUE | 24I | false | 682 | 1724291 |
| 2018 | other_id: ACTBLUE | 24T | false | 1412 | 224603 |
| 2018 | reported by ACTBLUE | 24T | false | 5253631 | 687633986 |
| 2018 | reported by ACTBLUE | 15 | true | 504222 | 4584595 |
| 2018 | reported by ACTBLUE | 22Y | false | 63017 | 835660 |
| 2018 | reported by ACTBLUE | 20Y | false | 19 | 2982 |
| 2020 | other reporting | 15 | true | 11528956 | 3448112551 |
| 2020 | other reporting | 10 | true | 781737 | 2027223601 |
| 2020 | other reporting | 15E | true | 5243558 | 683335696 |
| 2020 | other reporting | 22Y | false | 283435 | 128371836 |
| 2020 | other reporting | 24T | false | 97445 | 49799710 |
| 2020 | other reporting | 31 | true | 6617 | 42428188 |
| 2020 | other reporting | 32 | true | 2110 | 29941173 |
| 2020 | other reporting | 20Y | false | 4702 | 16839872 |
| 2020 | other reporting | 30 | true | 306 | 13157881 |
| 2020 | other reporting | 24I | false | 2520 | 1211340 |
| 2020 | other reporting | 15C | false | 248 | 1011367 |
| 2020 | other reporting | 11 | false | 344 | 760925 |
| 2020 | other reporting | 42Y | false | 70 | 154666 |
| 2020 | other reporting | 30T | false | 1 | 35500 |
| 2020 | other reporting | 21Y | false | 6 | 9200 |
| 2020 | other reporting | 41Y | false | 6 | 225 |
| 2020 | other_id: ACTBLUE | 15E | true | 18626946 | 1801148249 |
| 2020 | other_id: ACTBLUE | 24T | false | 467 | 87842 |
| 2020 | other_id: ACTBLUE | 24I | false | 50 | 45014 |
| 2020 | other_id: WINRED | 15E | true | 4125031 | 584456022 |
| 2020 | other_id: WINRED | 24I | false | 64 | 33012 |
| 2020 | other_id: WINRED | 24T | false | 104 | 17002 |
| 2020 | reported by ACTBLUE | 24T | false | 20733195 | 2140427718 |
| 2020 | reported by ACTBLUE | 15 | true | 2060323 | 15355265 |
| 2020 | reported by ACTBLUE | 22Y | false | 108771 | 2154974 |
| 2020 | reported by ACTBLUE | 15E | true | 223 | 133056 |
| 2020 | reported by WINRED | 15E | true | 22383956 | 1413034075 |
| 2020 | reported by WINRED | 22Y | false | 775973 | 115635612 |
| 2020 | reported by WINRED | 24T | false | 114 | 105787 |
24_qa_winred_candidate_reporting¶
Inspects WinRed reporting for the largest candidate.
SQL: 24_qa_winred_candidate_reporting
-- 24_qa_winred_candidate_reporting
-- WinRed reports most of the largest candidate total. I look at the
-- entity type, memo text and other_id on those records to see whether
-- they identify where the money came from.
SELECT
entity_tp,
memo_text,
other_id,
COUNT(*) AS n_records,
SUM(transaction_amt) AS amount
FROM `bigquery-public-data.fec.committee_contributions_2020`
WHERE cand_id = 'P80001571'
AND cmte_id = 'C00694323'
AND transaction_tp IN ('24K', '24Z')
AND transaction_amt > 0
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_dt BETWEEN '2019-01-01' AND '2020-12-31'
GROUP BY 1, 2, 3
ORDER BY amount DESC;
Full results: 24_qa_winred_candidate_reporting (1 row)
BigQuery follow-up run; job IDs in query_jobs.json
| entity_tp | memo_text | other_id | n_records | amount |
|---|---|---|---|---|
| CCM | NULL | C00580100 | 257625 | 48407633 |
25_qa_winred_linkage¶
Profiles the available WinRed linkage fields.
SQL: 25_qa_winred_linkage
-- 25_qa_winred_linkage
-- Inspect linkage fields before deciding how to handle conduit reporting.
WITH records AS (
SELECT DISTINCT
sub_id,
cmte_id,
other_id,
memo_text,
tran_id,
transaction_dt,
CAST(transaction_amt AS NUMERIC) AS amount
FROM `bigquery-public-data.fec.indiv20`
WHERE (cmte_id = 'C00694323' OR other_id = 'C00694323')
AND entity_tp = 'IND'
AND transaction_tp = '15E'
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_amt > 0
AND transaction_dt BETWEEN DATE '2019-01-01' AND DATE '2020-12-31'
)
SELECT
IF(cmte_id = 'C00694323', 'conduit_report', 'recipient_report') AS reporting_side,
COUNT(*) AS n_records,
SUM(amount) AS amount,
COUNTIF(NULLIF(TRIM(other_id), '') IS NOT NULL) AS with_other_id,
COUNTIF(NULLIF(TRIM(memo_text), '') IS NOT NULL) AS with_memo,
ARRAY_AGG(DISTINCT COALESCE(other_id, 'NULL') LIMIT 8) AS example_other_ids,
ARRAY_AGG(DISTINCT COALESCE(memo_text, 'NULL') LIMIT 8) AS example_memos
FROM records
GROUP BY 1;
Full results: 25_qa_winred_linkage (2 rows)
BigQuery follow-up run; job IDs in query_jobs.json
| reporting_side | n_records | amount | with_other_id | with_memo | example_other_ids | example_memos |
|---|---|---|---|---|---|---|
| conduit_report | 22383956 | 1413034075 | 22383873 | 0 | ["C00027466", "C00075820", "C00193342", "C00580100", "C00618371", "C00633362", "C00420935", "C00314575"] | ["NULL"] |
| recipient_report | 4125031 | 584456022 | 4125031 | 2527085 | ["C00694323"] | ["NULL", "EARMARKED FROM WINRED", "EARMARKED FROM WIN RED", "EARMARKED FROM WINRED - CONDUIT", "EARMARKED FROM WIN RED; SEE REDESIGNATION", "EARMARKED THROUGH WINRED RECOUNT RECEIPT AS PER AO 2006-24", "EARMARKED THROUGH WINRED", "EARMARKED FROM WINRED; REFUNDED $30.00 ON 09/24/2020"] |
26_qa_winred_matching_records¶
Compares WinRed and recipient records using five matching fields.
SQL: 26_qa_winred_matching_records
-- 26_qa_winred_matching_records
-- Matching fields indicate possible duplicate reporting, not a unique donation ID.
WITH records AS (
SELECT DISTINCT
sub_id,
cmte_id,
other_id,
UPPER(TRIM(name)) AS donor_name,
SUBSTR(TRIM(zip_code), 1, 5) AS zip5,
transaction_dt,
CAST(transaction_amt AS NUMERIC) AS amount
FROM `bigquery-public-data.fec.indiv20`
WHERE (cmte_id = 'C00694323' OR other_id = 'C00694323')
AND entity_tp = 'IND'
AND transaction_tp = '15E'
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_amt > 0
AND transaction_dt BETWEEN DATE '2019-01-01' AND DATE '2020-12-31'
),
grouped AS (
SELECT
IF(cmte_id = 'C00694323', other_id, cmte_id) AS recipient,
donor_name,
zip5,
transaction_dt,
amount,
COUNTIF(cmte_id = 'C00694323') AS conduit_rows,
COUNTIF(cmte_id != 'C00694323') AS recipient_rows
FROM records
GROUP BY 1, 2, 3, 4, 5
),
classified AS (
SELECT
*,
CASE
WHEN NULLIF(TRIM(recipient), '') IS NULL
OR NULLIF(donor_name, '') IS NULL
OR NULLIF(zip5, '') IS NULL
THEN 'incomplete_match_fields'
WHEN conduit_rows = 1 AND recipient_rows = 1
THEN 'one_to_one_field_match'
WHEN conduit_rows > 0 AND recipient_rows > 0
THEN 'multiple_records_share_fields'
WHEN conduit_rows > 0
THEN 'conduit_only'
ELSE 'recipient_only'
END AS match_status
FROM grouped
)
SELECT
match_status,
COUNT(*) AS n_groups,
SUM(conduit_rows) AS conduit_rows,
SUM(recipient_rows) AS recipient_rows,
SUM(conduit_rows * amount) AS conduit_amount,
SUM(recipient_rows * amount) AS recipient_amount,
SUM(LEAST(conduit_rows, recipient_rows) * amount) AS potential_shared_amount
FROM classified
GROUP BY 1
ORDER BY 1;
Full results: 26_qa_winred_matching_records (5 rows)
BigQuery follow-up run; job IDs in query_jobs.json
| match_status | n_groups | conduit_rows | recipient_rows | conduit_amount | recipient_amount | potential_shared_amount |
|---|---|---|---|---|---|---|
| conduit_only | 18919555 | 20079371 | 0 | 1039829332 | 0 | 0 |
| incomplete_match_fields | 325 | 84 | 259 | 17188 | 30445 | 0 |
| multiple_records_share_fields | 177252 | 388461 | 389931 | 32095228 | 32662236 | 30716064 |
| one_to_one_field_match | 1916040 | 1916040 | 1916040 | 341092327 | 341092327 | 341092327 |
| recipient_only | 1650209 | 0 | 1818801 | 0 | 210671014 | 0 |
27_qa_winred_sensitivity¶
Measures the effect of excluding WinRed conduit reporting.
The full SQL appears inline above and in 27_qa_winred_sensitivity.sql.
Full results: 27_qa_winred_sensitivity (27 rows)
BigQuery follow-up run; job IDs in query_jobs.json
| result_type | details |
|---|---|
| q2_without_winred | {"n_candidates":2603,"amount":1053851132,"minimum":8,"mean":"404860.212063004","approximate_median":15750,"maximum":6283681,"population_stddev":799105.12097680452,"top_ten_percent_share_pct":"59.608895","largest_candidate":[{"cand_id":"S4GA11285","amount":6283681}]} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-12-01","n_records":320995,"amount":31070942} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-10-01","n_records":123309,"amount":17725295} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-11-01","n_records":4340295,"amount":263210282} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-07-01","n_records":1773527,"amount":124377745} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-02-01","n_records":323785,"amount":21960984} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2019-07-01","n_records":2964,"amount":960293} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-05-01","n_records":21082,"amount":1483993} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-11-01","n_records":316397,"amount":34628733} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-03-01","n_records":462201,"amount":27867494} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-04-01","n_records":12872,"amount":1032537} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-07-01","n_records":80751,"amount":16176747} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-10-01","n_records":6336381,"amount":363644276} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-05-01","n_records":980038,"amount":47943731} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2019-11-01","n_records":127158,"amount":8940130} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-09-01","n_records":2992174,"amount":210012778} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-01-01","n_records":148253,"amount":13254619} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2019-10-01","n_records":76778,"amount":6529533} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-06-01","n_records":51552,"amount":6495995} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-06-01","n_records":1549632,"amount":99386766} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2019-08-01","n_records":13762,"amount":2421456} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2019-12-01","n_records":232118,"amount":18681134} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-08-01","n_records":82467,"amount":15192914} |
| excluded_q1_monthly | {"source_type":"committee","contribution_month":"2020-09-01","n_records":88210,"amount":16810310} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-08-01","n_records":2261994,"amount":161413156} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2020-04-01","n_records":715844,"amount":35311083} |
| excluded_q1_monthly | {"source_type":"individual","contribution_month":"2019-09-01","n_records":47052,"amount":7118615} |
28_qa_committee_platform_reporting¶
SQL: 28_qa_committee_platform_reporting
-- 28_qa_committee_platform_reporting
-- Check the named platforms in both committee sources before assuming
-- that the individual-source transaction pattern also applies here.
WITH names AS (
SELECT 2018 AS cycle, cmte_id,
STRING_AGG(DISTINCT cmte_nm, '; ') AS name
FROM `bigquery-public-data.fec.committee_2018`
GROUP BY cycle, cmte_id
UNION ALL
SELECT 2020 AS cycle, cmte_id,
STRING_AGG(DISTINCT cmte_nm, '; ') AS name
FROM `bigquery-public-data.fec.committee_2020`
GROUP BY cycle, cmte_id
), records AS (
SELECT 'Q1 broader source' AS source, cycle, cmte_id,
transaction_tp, amount, is_q1_in_scope AS included
FROM `fec-contributions-analysis.fec_analysis.stg_committee_q1`
UNION ALL
SELECT 'Q2 candidate source', cycle, cmte_id,
transaction_tp, amount, is_q2_in_scope
FROM `fec-contributions-analysis.fec_analysis.stg_committee_contributions`
)
SELECT r.source, r.cycle, n.name, r.cmte_id, r.transaction_tp, r.included,
COUNT(*) AS n_records, SUM(r.amount) AS amount
FROM records AS r
JOIN names AS n ON r.cycle = n.cycle AND r.cmte_id = n.cmte_id
WHERE REGEXP_CONTAINS(UPPER(n.name), r'ACTBLUE|WINRED')
GROUP BY r.source, r.cycle, n.name, r.cmte_id, r.transaction_tp, r.included
ORDER BY r.source, r.cycle, n.name, r.transaction_tp, r.included;Captured results: 28_qa_committee_platform_reporting
[
{
"source": "Q1 broader source",
"cycle": "2018",
"name": "ACTBLUE",
"cmte_id": "C00401224",
"transaction_tp": "24K",
"included": "true",
"n_records": "5",
"amount": "37000"
},
{
"source": "Q1 broader source",
"cycle": "2020",
"name": "ACTBLUE",
"cmte_id": "C00401224",
"transaction_tp": "24K",
"included": "false",
"n_records": "1",
"amount": "-65"
},
{
"source": "Q1 broader source",
"cycle": "2020",
"name": "ACTBLUE",
"cmte_id": "C00401224",
"transaction_tp": "24K",
"included": "true",
"n_records": "1",
"amount": "200000"
},
{
"source": "Q1 broader source",
"cycle": "2020",
"name": "WINRED",
"cmte_id": "C00694323",
"transaction_tp": "24K",
"included": "false",
"n_records": "109752",
"amount": "-13961280"
},
{
"source": "Q1 broader source",
"cycle": "2020",
"name": "WINRED",
"cmte_id": "C00694323",
"transaction_tp": "24K",
"included": "true",
"n_records": "1097635",
"amount": "140617466"
},
{
"source": "Q2 candidate source",
"cycle": "2020",
"name": "ACTBLUE",
"cmte_id": "C00401224",
"transaction_tp": "24K",
"included": "false",
"n_records": "1",
"amount": "-65"
},
{
"source": "Q2 candidate source",
"cycle": "2020",
"name": "WINRED",
"cmte_id": "C00694323",
"transaction_tp": "24K",
"included": "false",
"n_records": "48771",
"amount": "-7880358"
},
{
"source": "Q2 candidate source",
"cycle": "2020",
"name": "WINRED",
"cmte_id": "C00694323",
"transaction_tp": "24K",
"included": "true",
"n_records": "472869",
"amount": "66674728"
}
]29_model_reporting_context¶
SQL: 29_model_reporting_context
-- 29_model_reporting_context
-- Save the verified individual staging rows once for repeated analysis.
-- Cluster only: historical dates must not inherit sandbox partition expiry.
-- Query 17 remains the original build; run this upgrade after query 18.
CREATE OR REPLACE TABLE `fec-contributions-analysis.fec_analysis.individual_records_cached`
CLUSTER BY cycle, cmte_id
AS
WITH records AS (
SELECT
2018 AS cycle,
sub_id,
cmte_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
transaction_amt
FROM `bigquery-public-data.fec.indiv18`
UNION ALL
SELECT DISTINCT
2020 AS cycle,
sub_id,
cmte_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
transaction_amt
FROM `bigquery-public-data.fec.indiv20`
)
SELECT
cycle,
sub_id,
cmte_id,
entity_tp,
transaction_tp,
memo_cd,
transaction_dt,
CAST(transaction_amt AS NUMERIC) AS amount,
COALESCE(
entity_tp = 'IND'
AND transaction_tp IN (
'10', '15', '15E',
'30', '30E', '31', '31E', '32', '32E'
)
AND transaction_amt > 0
AND COALESCE(TRIM(memo_cd), '') = ''
AND transaction_dt BETWEEN
DATE(cycle - 1, 1, 1) AND DATE(cycle, 12, 31),
FALSE
) AS is_q1_in_scope
FROM records;
CREATE OR REPLACE VIEW `fec-contributions-analysis.fec_analysis.stg_individual_q1`
AS SELECT * FROM `fec-contributions-analysis.fec_analysis.individual_records_cached`;
-- One row per committee and cycle. Keep conflicting labels visible.
CREATE OR REPLACE TABLE `fec-contributions-analysis.fec_analysis.dim_committee` AS
WITH records AS (
SELECT 2018 AS cycle, cmte_id, cmte_nm, cmte_tp, cmte_pty_affiliation, cmte_st
FROM `bigquery-public-data.fec.committee_2018`
UNION ALL
SELECT 2020, cmte_id, cmte_nm, cmte_tp, cmte_pty_affiliation, cmte_st
FROM `bigquery-public-data.fec.committee_2020`
)
SELECT cycle, cmte_id,
STRING_AGG(DISTINCT cmte_nm, '; ' ORDER BY cmte_nm) AS committee_name,
STRING_AGG(DISTINCT cmte_tp, '; ' ORDER BY cmte_tp) AS committee_type,
STRING_AGG(DISTINCT cmte_pty_affiliation, '; ' ORDER BY cmte_pty_affiliation) AS party,
STRING_AGG(DISTINCT cmte_st, '; ' ORDER BY cmte_st) AS state,
COUNT(*) AS source_rows,
LOGICAL_OR(REGEXP_CONTAINS(UPPER(COALESCE(cmte_nm, '')), r'ACTBLUE|WINRED')) AS is_named_platform
FROM records
WHERE cmte_id IS NOT NULL
GROUP BY cycle, cmte_id;
CREATE OR REPLACE TABLE `fec-contributions-analysis.fec_analysis.dim_candidate` AS
WITH records AS (
SELECT 2018 AS cycle, cand_id, cand_name, cand_pty_affiliation, cand_office, cand_ici
FROM `bigquery-public-data.fec.candidate_2018`
UNION ALL
SELECT 2020, cand_id, cand_name, cand_pty_affiliation, cand_office, cand_ici
FROM `bigquery-public-data.fec.candidate_2020`
)
SELECT cycle, cand_id,
STRING_AGG(DISTINCT cand_name, '; ' ORDER BY cand_name) AS candidate_name,
STRING_AGG(DISTINCT cand_pty_affiliation, '; ' ORDER BY cand_pty_affiliation) AS party,
STRING_AGG(DISTINCT cand_office, '; ' ORDER BY cand_office) AS office,
STRING_AGG(DISTINCT cand_ici, '; ' ORDER BY cand_ici) AS incumbent_status,
COUNT(*) AS source_rows
FROM records
WHERE cand_id IS NOT NULL
GROUP BY cycle, cand_id;
-- Enrich Q1 without adding the overlapping Q2 contribution records again.
-- A platform flag identifies the reporting committee, not the ultimate donor.
CREATE OR REPLACE VIEW `fec-contributions-analysis.fec_analysis.fct_contributions` AS
WITH attribution AS (
SELECT cycle, sub_id,
IF(COUNT(DISTINCT cand_id) = 1, MIN(cand_id), NULL) AS candidate_id
FROM `fec-contributions-analysis.fec_analysis.stg_committee_contributions`
GROUP BY cycle, sub_id
), records AS (
SELECT 'individual' AS source_type, cycle, sub_id, transaction_dt, amount,
cmte_id AS reporting_committee_id, cmte_id AS recipient_committee_id,
CAST(NULL AS STRING) AS candidate_id, transaction_tp
FROM `fec-contributions-analysis.fec_analysis.stg_individual_q1`
WHERE is_q1_in_scope
UNION ALL
SELECT 'committee', s.cycle, s.sub_id, s.transaction_dt, s.amount,
s.cmte_id, s.other_id, a.candidate_id, s.transaction_tp
FROM `fec-contributions-analysis.fec_analysis.stg_committee_q1` AS s
LEFT JOIN attribution AS a ON s.cycle = a.cycle AND s.sub_id = a.sub_id
WHERE s.is_q1_in_scope
)
SELECT r.*, COALESCE(d.is_named_platform, FALSE) AS is_platform_reported,
d.committee_name AS reporting_committee_name
FROM records AS r
LEFT JOIN `fec-contributions-analysis.fec_analysis.dim_committee` AS d
ON r.cycle = d.cycle AND r.reporting_committee_id = d.cmte_id;
-- Counts and amounts must remain equal to the previously reconciled model.
WITH actual AS (
SELECT source_type, COUNT(*) AS n_records, SUM(amount) AS amount
FROM `fec-contributions-analysis.fec_analysis.fct_contributions`
GROUP BY source_type
), expected AS (
SELECT source_type, SUM(n_contributions) AS n_records, SUM(gross_amount) AS amount
FROM `fec-contributions-analysis.fec_analysis.agg_contributions_monthly`
GROUP BY source_type
)
SELECT e.source_type, a.n_records, a.amount,
COALESCE(a.n_records = e.n_records AND a.amount = e.amount, FALSE) AS check_passed
FROM expected AS e LEFT JOIN actual AS a USING (source_type)
ORDER BY source_type;Captured results: 29_model_reporting_context
[
{
"source_type": "committee",
"n_records": "1750504",
"amount": "2720479967",
"check_passed": "true"
},
{
"source_type": "individual",
"n_records": "79847961",
"amount": "14080279264",
"check_passed": "true"
}
]