Sizes models and token budgets using Kaplan/Chinchilla scaling laws. Use when reasoning about compute-optimal N and D, tokens-per-parameter ratios, or over-training tradeoffs.
Sizes models and token budgets using Kaplan/Chinchilla scaling laws. Use when reasoning about compute-optimal N and D, tokens-per-parameter ratios, or over-training tradeoffs.