PC lab explains how game benchmarking should be made, picks Guru3D as a bad example

Discussion in 'Hardware Reviews and News' started by Asi, Jul 6, 2014.

Tags:
Thread Status:
Not open for further replies.
  1. Asi

    Asi Guest

    Messages:
    1
    Likes Received:
    0
    GPU:
    GeForce GTX 580
    The article is here, the Guru3D example is on page 2 (can't post links yet, you'll have to fix it, or just google "Rzecz o miejscach testowych w grach na przykladzie Far Cry 3"):
    pclab(.)pl/art58059(.)html

    Long story short, they're saying that while benchmarking in games, the test should provide repeatable, consistent results and the parts of the game chosen for testing should be representative of the game and demanding for hardware.

    The bad example is Guru3D's Battlefield 3 test, where Radeon R9 280X and HD 7970 GHz, two versions of the same GPU, basically, got different results (46 and 42 fps, respectively).

    The reason for that is probably the test procedure, shown on the youtube video (sorry again, can't post links, here's the string):
    cedwBQpgooE
    It is too random, the player is inconsistently looking about and shooting. It may be representative of the game, but it's no good for reliable benchmarking.

    The conclusion is that credibility of tests on other websites (including Guru3D) included is questionable. Then, on the following pages, they explain how they do it.

    What do you make of it? Are they nitpicking or is there something to it?
     
    Asi, Jul 6, 2014
    #1
  2. Hilbert Hagedoorn

    Hilbert Hagedoorn Don Vito Corleone Staff Member

    Messages:
    48,509
    Likes Received:
    18,808
    GPU:
    AMD | NVIDIA
    Can't read half of it due to translations. But it sounds like they are thrill-seeking for page-views, by dissing other websites.

    I agree that FRAPS style benchmarking always is less accurate opposed to scripted time-demo's hence we say it is a good indication of performance. We carefully select the scene and tested path, if we can often stay within a 1 FPS offset - and that is time after time doing the same test, then I'd say our numbers are very reliable. Other then that, offsets do happen. The 7970 GHz likely was tested with an older driver or perhaps a game patch messed some stuff up. Dunno, but that article is irrelevant and even insulting as they used copyrighted stuff right from our pages without our consent.
     
    Hilbert Hagedoorn, Jul 6, 2014
    #2
Thread Status:
Not open for further replies.

Share This Page