-
Can a Digital Job Search Coach Reduce Unemployment? Experimental Evidence from France
Conditionally accepted at the Journal of Labor Economics
Aïcha Ben Dhia, Bruno Crépon, Esther Mbih, Louise Paul-Delvaux, Bertille Picard, and Vincent Pons
Keywords: Labor economics, Online platforms
NBER Working Paper 29914, April 2022 VoxEU Column
AbstractWe evaluate the impact of Bob Emploi, a digital platform designed to provide personalized job search advice and coaching to the unemployed. The platform, developed by a nonprofit organization with access to France’s public employment agency data, aims to replicate traditional counseling services through automated tools. Our experiment included 212,277 individuals, with 56.3% randomly assigned to receive encouragement to use the platform. While our intervention increased Bob Emploi’s usage by 27 percentage points, the effects of the platform remained limited. Users made modest changes to their search methods, showed slightly higher engagement with standard employment services, and felt more supported in their search. However, we find no impact on time spent searching, occupational scope, or job seeker well-being. Most importantly, the platform did not improve any employment outcomes over an 18-month follow-up period, with precise null effects across all subgroups. These results suggest that digital job search assistance platforms may need to combine coaching with specific job recommendations to effectively improve job seekers’ labor market outcomes.
-
Double Machine Learning for Inequality Decompositions: Reference Outcomes, Orthogonal Scores and Limited Overlap
Emmanuel Flachaire, Bertille Picard
Keywords: Kitagawa-Oaxaca-Blinder decomposition, reference outcome, double machine learning, limited overlap
AbstractKitagawa-Oaxaca-Blinder decompositions split a gap in mean outcomes between two groups into a part explained by characteristics and an unexplained part. The unexplained part depends on the counterfactual “reference outcome” that individuals would receive if characteristics were valued equally. Several competing references have been proposed, mostly within the linear regression model. We show that the main proposals, namely using the outcome of either group as the reference and the references of Cotton (1988), Reimers (1983) and Neumark (1988), all belong to a single family of mixtures of the two groups’ potential outcomes. This separates the choice of target from the choice of estimator, and allows nonparametric estimation by double machine learning. We find that the appropriate orthogonal score depends on the reference: Neumark’s propensity-weighted reference requires a pooled regression, while all the others require group-specific regressions. We then show that Neumark’s reference avoids both extrapolation and extreme weights under limited overlap. Finally, we discuss in a general setting the shrinkage effect of the unexplained part associated with the choice of Neumark’s reference.
-
Does Personalized Allocation Make Our Experimental Designs More Fair?
Bertille Picard
Keywords: Fairness, Contextual bandits, Randomized controlled trials, Machine learning
AbstractAlgorithms can optimize treatment allocation within an experimental design. They can progressively identify the most beneficial treatment for the subjects and thus maximize the experiment’s overall impact. However, these designs raise concerns for experimentalists and policymakers because they involve transferring decision-making to an algorithm. Are adaptive experiments inherently fairer and thus a preferred choice over traditional randomized controlled trials? In this paper, I propose a comprehensive examination of fairness by considering multiple criteria that can influence the researchers’ preference for one design over the other: the possibility to increase the benefits of the experiment for the experimental subjects, the transparency of the decision rule, the absence of discrimination regarding the treatment allocation, the protection of individuals’ data. By summarizing and analyzing these distinct criteria through a utility model, I discuss the relative fairness of adaptive experiments and standard randomized controlled trials. Specifically, I show that these different designs align with extreme versions of the fairness utility model, reflecting the pursuit of distinct fairness objectives within experimental settings. I highlight intermediate solutions that can be pursued to reconcile and balance different fairness objectives in experimental designs.
-
An Adaptive Experiment to Boost Online Skill Signaling and Visibility
Morgane Hoffmann, Bertille Picard, Charly Marie, and Guillaume Bied
Keywords: Labor economics, Online platforms, Adaptive experiments
AbstractDigital matching platforms promise to reduce frictions on the labor market by providing low-cost information on available positions and candidates. As such, they may form a welcome addition to the toolbox available to Public Employment Services to bridge labor supply and demand. However, there are certain challenges associated with their adoption. For instance, vulnerable populations may face difficulties in utilizing digital tools effectively. In this study, we evaluate the impact of a communication campaign by email designed to encourage the use of an online matching platform maintained by the French Public Employment Service, Pôle emploi. We designed several email templates that combined information, support or motivational content to encourage jobseekers to engage with their profiles on the platform. In order to discover email effectiveness we implement an adaptive experiment (contextual bandit) where the goal is to use past jobseekers take-up responses and characteristics to determine email allocation in the future, reducing gradually the allocation of less promising templates. Additionally, we built an optimal personalization allocation strategy based on collected data and test its effectiveness. Emails had a positive impact on the usage of the platform, as measured by a wide range of outcomes. However, attempts at learning a personalized emailing strategy did not manage to significantly improve on a random allocation of email templates.
-
Can a Digital Job Search Coach Reduce Unemployment? Experimental Evidence from France
Conditionally accepted at the Journal of Labor Economics
Keywords: Labor economics, Online platforms
-
Double Machine Learning for Inequality Decompositions: Reference Outcomes, Orthogonal Scores and Limited Overlap
Keywords: Kitagawa-Oaxaca-Blinder decomposition, reference outcome, double machine learning, limited overlap
-
Does Personalized Allocation Make Our Experimental Designs More Fair?
Keywords: Fairness, Contextual bandits, Randomized controlled trials, Machine learning
-
An Adaptive Experiment to Boost Online Skill Signaling and Visibility
Keywords: Labor economics, Online platforms, Adaptive experiments