Dynamics of Political Involvement Effectiveness: Quantitative Evaluation of Reproductive Rights Advocacy Organizations

Alexandra Shakhnazarov, Gallatin '26

Advised by Professor Natasha Zaretsky

Abstract

America’s reproductive rights organizations adopt varying strategies of advocacy, but the relationship between their political involvement and real-world effectiveness remains largely unexamined. This study asks: Is there a measurable correlation between the political engagement of reproductive rights organizations and their operational effectiveness? To study this, we developed a mixed-methods scoring framework to assess four major national reproductive rights organizations—Planned Parenthood, Center for Reproductive Rights, Reproductive Freedom for All (formerly NARAL), and the Guttmacher Institute—across two axes: Political Involvement Score (PIS) and Effectiveness Score (ERS). PIS incorporated lobbying expenditures, political contributions, endorsements, and a sentiment-based Political Alignment Rhetoric Score (PARS). ERS was calculated based on policy influence, media presence, service provision, financial efficacy, and outreach. Our findings suggest that while political involvement correlates with broader media presence and public influence, it does not uniformly predict organizational effectiveness in service delivery or legal success. Organizations with high political spending did not always score highest in impact. These results highlight the need for multi-dimensional evaluations of advocacy groups and could imply that other factors, such as strategic diversity, rather than just political alignment, contribute to success in the reproductive rights space.

Introduction

Reproductive rights have long been a central and contentious issue in the United States, situated at the intersection of gender, health care, law, and political ideology. These rights encompass a broad range of freedoms—including access to abortion, contraception, sex education, and reproductive health services—and are often shaped by shifting political and judicial landscapes. The historic Roe v. Wade (1973) decision had, for nearly five decades, protected the constitutional right to abortion. However, in 2022, the U.S. Supreme Court’s decision in Dobbs v. Jackson Women’s Health Organization overturned Roe, eliminating federal protections and allowing individual states to enact their abortion laws. This marked a seismic shift in reproductive rights policy, resulting in an increasingly fragmented and polarized landscape.

In response, advocacy organizations have emerged as vital actors in the fight to protect and expand access to reproductive health care. These groups, ranging from legal advocacy centers and health service providers to research institutes and political mobilizers, play an essential role in influencing public policy, conducting public education, providing direct services, and challenging restrictive laws in court. Their ability to mobilize support, frame reproductive rights in public discourse, and shape legislative outcomes has grown even more critical in the wake of Dobbs and subsequent state-level abortion bans or severe restrictions. Their impact is often highly visible during in such political inflection points, but the long-term efficacy of these organizations is difficult to assess. Despite their centrality in the national conversation, there is limited empirical research evaluating how politically involved these organizations are, and whether that involvement meaningfully contributes to their advocacy goals.

Using publicly available data, we construct a quantitative scoring framework to assess both political involvement and effectiveness, offering a new lens for evaluating impact. In doing so, it highlights the challenges of defining and measuring nonprofit impact in a highly politicized environment and opens the door for more data-driven evaluation of social advocacy.

Methodology

To evaluate the political involvement and effectiveness of reproductive rights organizations, we relied on a combination of publicly available quantitative and qualitative data. Financial and lobbying information was sourced primarily from OpenSecrets (Center for Responsive Politics), IRS Form 990 filings, and ProPublica’s Nonprofit Explorer, which provided detailed records on lobbying expenditures, political contributions, and annual revenue. We manually collected press releases, self-published statements, and social media content from each organization’s official accounts across platforms such as Twitter, Facebook, and Instagram. Engagement data was used to evaluate media presence. These sources allowed us to build a multi-dimensional scoring framework that integrated objective financial metrics with sentiment-informed evaluations of political alignment and communication strategy. All data collected was from January 2024 to April 2025, reflecting the most recent publicly available records at the time of research.

We selected four nationally recognized reproductive rights organizations for this study: Planned Parenthood, Center for Reproductive Rights, Reproductive Freedom for All (formerly NARAL), and the Guttmacher Institute. These organizations were chosen based on their visibility, policy relevance, and the diversity of their advocacy approaches. Each of these organizations has a substantial public presence and publishes regular financial and advocacy information, making them suitable for a data-driven, comparative analysis using publicly available sources.

Research Design

This project used a comparative, cross-organizational research design to evaluate four major national reproductive rights organizations and the dynamics between their political involvement and organizational effectiveness using publicly available data. To achieve this, we developed a mixed-method scoring system that combines financial, rhetorical, and engagement data into two composite indices: the Political Involvement Score (PIS) and the Effectiveness Score (ERS). We created a scalable and repeatable scoring framework designed to quantify each organization’s activities along two key dimensions. For political involvement, we incorporated objective metrics such as lobbying expenditures, political contributions, and endorsements, along with a custom-built Political Alignment Rhetoric Score (PARS) based on machine learning-supported sentiment analysis and keyword evaluation of public statements. Sentiment analysis was assisted by OpenAI’s GPT-4 model, which was used to analyze rhetorical tone and classify political alignment intensity.

For effectiveness, we drew on metrics including media presence, service provision, financial efficacy, outreach initiatives, and policy engagement. All scores were generated using either log transformation, fixed-scale normalization, or proportional scoring, depending on the nature of the data. This design allows for meaningful comparisons across organizations with diverse strategies, structures, and budgets.

Scoring Framework

The PIS and ERS were calculated using multiple components that comprised various factors of their political involvement or efficacy. Each component is scored on a standardized 0–10 scale before averaging. PIS is calculated as the average of four component scores rated from 0-10: where L = Lobbying Expenditures, E = Endorsement Activity Score, PS = Political Spending Score, and PR = Politically-Aligning Rhetoric Score.

ES is calculated as the average of five component scores: where PI = Policy Influence Score, CO = Community Outreach Score, MP = Media Presence Score, SP = Service Provision Score, and ESP = Expenses for Service Provision Score.

We used rubrics to score more qualitative components in both the PIS and ES scores, included below in each component section. We used logarithmic scaling to convert raw scores to a 0-10 point scale that adjusted for extremes and outliers that would skew relative scoring.

Political Involvement Score

The Political Involvement Score (PIS) was designed to quantify the extent of each organization’s engagement with formal political systems, policymaking processes, and partisan rhetoric. This composite score includes four key components: lobbying expenditures, direct political contributions, endorsements, and political alignment, and the Political Alignment Rhetoric Score (PARS). Each component was scored on a 0–10 scale using a mix of log scaling, fixed-range normalization, and qualitative coding to ensure comparability across organizations with varying sizes and structures.

Lobbying Expenditures. To assess the significance of lobbying relative to each organization’s overall operations, we calculated the percentage of lobbying expenditures out of total annual spending. These percentages were then log-transformed to reduce skewness caused by extreme disparities in spending levels. After transformation, values were rescaled to a fixed 0–10 range using a standardized log scale. This method allowed for fair comparison without anchoring any organization as a maximum or minimum outlier.

Direct Political Contributions. Political contributions were sourced from Federal Election Commission (FEC) databases and OpenSecrets.org. Because contribution totals also varied widely by organizational size and structure, we applied a logarithmic transformation to the raw spending data and normalized scores on a 0–10 scale. This approach emphasized relative intensity of political giving while minimizing distortions from large, one-time expenditures.

Endorsements and Political Alignment. Endorsements of candidates, legislation, or political parties were collected from each organization’s press releases, websites, and social media channels. We used a rubric-based coding system to evaluate the frequency and explicitness of political endorsements, including support or criticism directed at parties or elected officials. Each organization’s endorsement activity was reviewed manually through a rubric, and then assigned a score on a 0–10 scale based on alignment intensity.

Political Alignment Rhetoric Score (PARS). To measure how partisan or politically charged each organization’s public messaging was, we developed a Political Alignment Rhetoric Score (PARS). This score was based on a manually collected sample of public statements, social media posts, and press releases. Each statement was analyzed using sentiment analysis techniques and a custom-built keyword flagging system that detected references to political parties, ideologies, and emotionally charged partisan language. Statements were then rated on a 0–10 scale based on intensity and partisanship, with higher scores reflecting more polarized or explicit political rhetoric.

Effectiveness Score (ERS)

The Effectiveness Score (ERS) is a composite metric designed to evaluate each organization’s operational reach, public engagement, and real-world policy influence. The ERS includes five core components: policy influence, community outreach, service provision, media presence, and financial efficacy. Each component is independently scored on a 0–10 scale, then averaged to generate the final ERS score.

Policy Influence. This component measures the organization’s success in shaping policy through litigation, legislative support, and public regulatory advocacy. We reviewed public legal victories, bills supported, amicus briefs filed, and policy initiatives led or significantly influenced by each organization. Scores were assigned based on a rubric that considered both the number and substantive impact of policy actions. This qualitative data was translated into a 0–10 scale using relative benchmarks informed by publicly reported case involvement and legislative participation

Community Outreach. To capture an organization’s grassroots reach, we assessed the number and scope of public-facing education programs, events, volunteer initiatives, and partnerships. This included both digital outreach campaigns and in-person events. We assigned scores using a relative rubric based on frequency, geographic scope, and accessibility of outreach initiatives, as reported on the organizations’ websites, newsletters, and community reports.

Service Provision. For organizations involved in direct care or research dissemination, we measured the volume and scale of services provided. This included patient care statistics (e.g., number of individuals served, clinics operated), published research outputs, or technical assistance offered. Scores were assigned using a standardized rubric assessing overall service volume, diversity of services, and alignment with organizational mission.

Media Presence. To assess public visibility, we calculated total social media following across Twitter, Instagram, Facebook, YouTube, and TikTok. These figures were then log-transformed and rescaled to a fixed 0–10 range using a standardized base-10 log scale (log10 range: 10,000 to 10,000,000 followers). This method ensured that extremely high follower counts did not skew results and avoided treating any organization as a maximum or minimum anchor point.

Financial Efficacy. We included a supplemental financial measure to reflect an organization’s ability to raise and manage funds at scale. Revenue figures were collected from IRS Form 990s and ProPublica’s Nonprofit Explorer, and we found what percentage of their finances and spending went to service provision.

Correlation Scoring

To determine the correlation between PIS and ES, we conducted a Pearson correlation test to assess the strength and statistical significance of the relationship and find the correlation coefficient R.

We then determined the statistical significance of the Pearson correlation coefficient, using a computed t-statistic that yielded us our p-value.

Results

A Pearson correlation analysis was conducted to assess the relationship between Political Involvement Score (PIS) and Effectiveness Score (ES) across four major reproductive rights organizations: Planned Parenthood, Center for Reproductive Rights, Reproductive Freedom for All, and the Guttmacher Institute. The calculated Pearson correlation coefficient (R) was 0.943, indicating a high degree of linear association between the two variables. This means that organizations with higher levels of political involvement also tended to score higher in effectiveness, which was measured by their influence on policy, community outreach, media presence, direct service provision, and related expenditures. To assess the statistical significance of this relationship, a t-test for correlation was conducted, having 2 degrees of freedom. The resulting p-value was approximately 0.056, which is marginally above the conventional threshold of statistical significance (p < 0.05). While sample size may confound significance, the strength of R = 0.943) strongly suggests a likely non-random relationship between PIS and ES.

Above: Scatterplot (PIS, ES) points with trendline ES = 4.099 + 0.509*(PIS)

Limitations

This study is limited first and foremost by its small sample size (n = 4), which constrains statistical power and generalizability. With only four organizations analyzed, the degrees of freedom (df = 2) are low, making it difficult to reach traditional thresholds of statistical significance, even with a strong observed correlation. This raises the potential for Type II error and limits the detection of non-linear trends or outliers. Additionally, the analysis is cross-sectional, capturing organizational activity during a narrow 2024–2025 window. As a result, the study offers a snapshot in time without accounting for changes in engagement, effectiveness, or external policy context that may evolve across election cycles or organizational shifts. The challenge of differentiating between engagement (e.g., outreach volume or lobbying effort) and actual impact (e.g., policy wins or improved service outcomes) also complicates interpretation. High activity does not always translate to measurable influence or public benefit.

Furthermore, the scoring methodology relies on rubric-based expert evaluation and machine-learning techniques, which, while structured and transparent, introduces subjectivity. Evaluators may interpret tone, alignment, and strategy differently across organizations. Variability in data availability also presents a risk: some organizations may publicly report more activity or funding detail than others, skewing score accuracy. Finally, causal inference cannot be drawn from this analysis; although political involvement and effectiveness are strongly correlated, the direction of that relationship remains unclear.

Conclusion

These findings support the hypothesis that political involvement is strongly associated with organizational effectiveness in the reproductive rights advocacy sector. Specifically, organizations with greater infrastructure and activity in the political domain—such as lobbying lawmakers, endorsing candidates, investing in political campaigns, or utilizing partisan-aligned rhetoric—also demonstrate greater capacity to deliver services, influence public policy, engage communities, and maintain a strong media footprint. One implication to be drawn from here is that political strategy and advocacy may serve as a force multiplier for effectiveness in issue-based nonprofit sectors. Political capital appears to not only reflect organizational values but also bolster practical reach, visibility, and impact. Furthermore, organizations that score lower in political involvement may face structural disadvantages in achieving their goals, especially in polarized or hostile policy environments.

Future research should aim to test this hypothesis across a larger and more diverse sample of organizations, both within and beyond the reproductive rights field. A broader dataset would allow for greater statistical power, subgroup analysis (e.g., regional, organizational age, issue area), and examination of non-linear or threshold effects. Longitudinal designs could help assess whether political involvement precedes impact or vice versa and provide richer context for interpreting political strategy, community engagement, and policy wins. In addition, incorporating public datasets (e.g., IRS filings, FEC reports, media analytics) could strengthen the validity of scoring frameworks and allow for more granular comparison across time.

References

Center for Reproductive Rights. (n.d.). Our Work. Center for Reproductive Rights. Retrieved April 14, 2025, from https://reproductiverights.org/our-work/#c-heading-Resources-and-Research

Center for Reproductive Rights. (2023). 2023 Annual Report. Center for Reproductive Rights. https://reproductiverights.org/wp-content/uploads/2024/01/2023_Annual_Report_Digital.pdf

Chew, R. (2024, March 30). Opinion | The Persistent Threat to Abortion Rights. The New York Times. https://www.nytimes.com/2024/03/30/opinion/abortion-drug-supreme-court.html

Guttmaacher Institute. (2024). 2024 Impact Report. Guttmacher Institute. Retrieved April, 2025, from https://www.guttmacher.org/sites/default/files/2024-10/Guttmacher%202024_Impact%20Report.pdf

Parenthood, P. (n.d.). Planned Parenthood | Official Site. Retrieved April 14, 2025, from https://www.plannedparenthood.org

Parenthood, P. (2023, June 30). Annual Report 2022-2023. Planned Parenthood. Retrieved April 14, 2025, from https://www.plannedparenthood.org/uploads/filer_public/ce/f6/cef6efdb-919a-4211-bb5c-ce0d61fda7f5/2024-ppfa-annualreport-c3-digital.pdf

Reproductive Freedom for All. (2023). 2023 Annual Report. Reproductive Freedom for All. Retrieved April 14, 2025, from https://reproductivefreedomforall.org/report/2023-annual-report-reproductive-freedom-for-all/

Serchen, J., Erickson, S., & Hilden, D. (2023, February 28). Reproductive Health Policy in the United States: An American College of Physicians Policy Brief. Annals of Internal Medicine, 176(3). https://doi.org/10.7326/M22-3316