Back to Projects

LLM Bias in Algorithm Evolution

Ongoing research into conservative local optimization by LLMs in ShinkaEvolve and OpenEvolve.

Overview

I am researching why LLMs in ShinkaEvolve and OpenEvolve can favor conservative local optimization while evolving heuristic algorithms. When LLMs repeat small edits, they can keep the search near familiar solutions; I am testing when that happens.

I compare empirical experiments with each framework’s evaluation loop to trace where LLMs become conservative. I also test how changes to evaluation affect their exploration. The experiments are still running.

What I Built

  • Conservative Edits

    I examine when LLMs favor low-risk changes while evolving heuristic algorithms.

  • Local Optimization

    I test why successive small edits can keep a search near a local optimum.

  • Framework Experiments

    I use ShinkaEvolve and OpenEvolve to observe iterative algorithm generation.

  • Working Hypotheses

    I design experiments around where bias may enter the evolution loop and ways to test mitigation.

Problems

  • LLMs often avoid high-impact changes to an algorithm.
  • Repeated edits can remain close to local optimization patterns.
  • Bias in the evaluation loop can limit sustained exploration.

Results

  • Early experiments have surfaced recurring patterns worth testing further.
  • Current evidence supports a working hypothesis about conservative behavior in iterative search.
  • Experiments on mitigation strategies are ongoing.