Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

Lizard-LLM Community

  1. Home
  2. Benchmarks
  3. Why the same machine gives different numbers each run

Why the same machine gives different numbers each run

Scheduled Pinned Locked Moved Benchmarks
1 Posts 1 Posters 1 Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • L Offline
    L Offline
    lizardadmin
    wrote on last edited by
    #1

    If you run the same benchmark twice and get different answers, nothing is broken.

    The usual causes, roughly in order:

    1. Heat. Laptops especially will slow themselves down after sustained load. A second run often lands lower than the first purely for this reason.
    2. Other software. A browser with many tabs, or anything using the GPU, competes for exactly the resource that limits generation speed.
    3. Cold vs warm model. The first run after loading includes work that later runs skip.

    This is why comparisons should be run alternating — A, B, A, B — rather than all of A then all of B. Otherwise you're partly measuring how warm the machine got.

    If your numbers swing by more than about 20% with nothing else running, that's worth posting about.

    1 Reply Last reply
    0

    Hello! It looks like you're interested in this conversation, but you don't have an account yet.

    Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

    With your input, this post could be even better 💗

    Register Login
    Reply
    • Reply as topic
    Log in to reply
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes


    • Login

    • Don't have an account? Register

    • Login or register to search.
    Powered by NodeBB Contributors
    • First post
      Last post
    0
    • Categories
    • Recent
    • Tags
    • Popular
    • World
    • Users
    • Groups