Pretraining Under Infinite Compute(arxiv.org)

1 points by jedharris 2 hours ago | 1 comments

  • jedharris 2 hours ago
    "Our results show that simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future."