Guru3D.com Forums

Go Back   Guru3D.com Forums > General Chat > Frontpage news
Frontpage news Perhaps you have some news to report or want to check out the latest Guru3D headlines and comment ? Check it in here.


Reply
 
Thread Tools Display Modes
FCAT benchmarking an article introduction [Guru3D.com]
Old
  (#1)
Hilbert Hagedoorn
Don Vito Corleone
 
Hilbert Hagedoorn's Avatar
 
Videocard: AMD | NVIDIA
Processor: Core i7 5960K
Mainboard: X99
Memory: 16GB
Soundcard: X-Fi - GigaWorks 7.1
PSU: 1200 Watt
Default FCAT benchmarking an article introduction [Guru3D.com] - 04-04-2013, 08:58 | posts: 21,694 | Location: Guru3D testlab

In this article we will introduce you towards FCAT benchmarking. The past couple of months we have seen a some new dynamics in measuring the framerate of your games. Basically the framerate of your ...

FCAT benchmarking an article introduction
   
Reply With Quote
 
Old
  (#2)
hallryu
Don Altobello
 
hallryu's Avatar
 
Videocard: 2x HD7970 - QNIX/U2711
Processor: 3770k delid@4.8GHz - H320
Mainboard: GA-Z77X-UP7 (F5)
Memory: Dominator GT 2134 2x4GB
Soundcard: Asus Xonar D2X + SP2500
PSU: BeQuiet! DP Pro 1200W
Default 04-04-2013, 09:49 | posts: 11,424 | Location: England

Excellent read. Explains the new benchmarking methodology rather nicely.
   
Reply With Quote
Old
  (#3)
Phuncz
Member Guru
 
Phuncz's Avatar
 
Videocard: AMD R9 290X + EK FCWB
Processor: Core i5-4670K + H220
Mainboard: ASUS Maximus VII Impact
Memory: Crucial BallistixVLP 16GB
Soundcard:
PSU: Silverstone ST45SF-G
Default 04-04-2013, 09:50 | posts: 53 | Location: Belgium

Excellent write-up, I'm very pleased you are going to incorporate FCAT testing into your reviews ! This needs to be known, so the manufacturers (AMD more than NVIDIA at this point) can make constant frame-times a priority and eliminate micro-stutter.

One point I'd like to make is that it would benefit all if you would include the same scaling for all your graphs and include zoomed-in versions indented or offset enough that it's clear it isn't the same. Because some aren't really comparable when they use different scaling like these two:


22-44ms scaling (22ms range)


0-30ms scaling (30ms range)

It would have been more helpful if the charts were 0-50ms, which is basically with a broad margin the lowest-highest we'd ever want to see anyway.
   
Reply With Quote
Old
  (#4)
k1net1cs
Ancient Guru
 
Videocard: Radeon HD 5650m (550/800)
Processor: Intel Core i5-520M 2.4GHz
Mainboard: Sony VAIO VPCEA16FG
Memory: 2x4GB CMSO4GX3M1A1333C9
Soundcard: ASUS Xonar U3
PSU: n/a
Default 04-04-2013, 09:59 | posts: 3,716

Hilbert, please don't use the term 'quickie introduction'...just 'quick introduction' would be enough without having the readers being rendered into wild imaginings.




Interested in folding with fellow gurus? Click here to get you started!
   
Reply With Quote
 
Old
  (#5)
Hilbert Hagedoorn
Don Vito Corleone
 
Hilbert Hagedoorn's Avatar
 
Videocard: AMD | NVIDIA
Processor: Core i7 5960K
Mainboard: X99
Memory: 16GB
Soundcard: X-Fi - GigaWorks 7.1
PSU: 1200 Watt
Default 04-04-2013, 10:07 | posts: 21,694 | Location: Guru3D testlab

Quote:
Originally Posted by Phuncz View Post
Excellent write-up, I'm very pleased you are going to incorporate FCAT testing into your reviews ! This needs to be known, so the manufacturers (AMD more than NVIDIA at this point) can make constant frame-times a priority and eliminate micro-stutter.

One point I'd like to make is that it would benefit all if you would include the same scaling for all your graphs and include zoomed-in versions indented or offset enough that it's clear it isn't the same. Because some aren't really comparable when they use different scaling like these two:

Yeah I know and actually mention that a couple of times in the article. Over time you'll see charts with fixed scaling. Percentile charts will range from 0 to 100% and the plots we need to figure out some sort of fixed y-axis for (likely ~60ms).


Follow Guru3D on twitter.
Follow Guru3D on facebook.
   
Reply With Quote
Old
  (#6)
hallryu
Don Altobello
 
hallryu's Avatar
 
Videocard: 2x HD7970 - QNIX/U2711
Processor: 3770k delid@4.8GHz - H320
Mainboard: GA-Z77X-UP7 (F5)
Memory: Dominator GT 2134 2x4GB
Soundcard: Asus Xonar D2X + SP2500
PSU: BeQuiet! DP Pro 1200W
Default 04-04-2013, 10:31 | posts: 11,424 | Location: England

Quote:
Originally Posted by Hilbert Hagedoorn View Post
Yeah I know and actually mention that a couple of times in the article. Over time you'll see charts with fixed scaling. Percentile charts will range from 0 to 100% and the plots we need to figure out some sort of fixed y-axis for (likely ~60ms).
How much time will this add to your reviews? The cost itself is eye-watering!
   
Reply With Quote
Old
  (#7)
xafier
Maha Guru
 
xafier's Avatar
 
Videocard: 7950 @ 1Ghz
Processor: 2500k @ 4.5Ghz
Mainboard:
Memory: 8Gb 1600mhz
Soundcard: Asus Xonar DX
PSU: Antec EcoWatt 620w
Default 04-04-2013, 10:49 | posts: 811 | Location: Sheffield, England

Hilbert,

I've read your description a couple of times now and I'm not really sure why you need to store all the frame data? If all you care about is the differing colours couldn't you just store the 1 pixel from bottom left?

The only reason I can see for not doing that is that it the video capture device can only store the full frame and instead you'd require CPU to pull out just the 1 pixel? But I cannot imagine that would be at all taxing as extracting a value from a known location in a frame is very simple.

Just throwing out my opinion as a software developer, that thats the road I would have gone down to avoid that storage problem. It would also most likely simplify the data analysis later too.
   
Reply With Quote
Old
  (#8)
mohiuddin
Master Guru
 
Videocard: Asus directcu hd6850
Processor: intel core 2 quad q6600
Mainboard: Intel dg33fb
Memory: 6gb ddr2 800bus
Soundcard:
PSU: GIGABYTE 1200w Odin pro
Default 04-04-2013, 10:55 | posts: 550

Thanks.
And explanation of every graph helped me (noob) a lot.
Again thanks.
   
Reply With Quote
Old
  (#9)
Osamar
Newbie
 
Videocard: Gforce 560 Ti 1Gb
Processor: Intel i5 570
Mainboard: P55 microATX
Memory: Kingston DDR3 8Gb (4x2)
Soundcard: Realtek HDA integrated
PSU: Antec Quiet 80 PLUS 650W
Default 04-04-2013, 10:56 | posts: 22 | Location: 40,00º N - 0,00º E

Quote:
Originally Posted by xafier View Post
Hilbert,

I've read your description a couple of times now and I'm not really sure why you need to store all the frame data? If all you care about is the differing colours couldn't you just store the 1 pixel from bottom left?

The only reason I can see for not doing that is that it the video capture device can only store the full frame and instead you'd require CPU to pull out just the 1 pixel? But I cannot imagine that would be at all taxing as extracting a value from a known location in a frame is very simple.

Just throwing out my opinion as a software developer, that thats the road I would have gone down to avoid that storage problem. It would also most likely simplify the data analysis later too.
If I am not wrong, you will need at least one pixel thick bar, becuase you could have mixed frames (tearing ???), so multiple colors in the bar.
   
Reply With Quote
Old
  (#10)
xafier
Maha Guru
 
xafier's Avatar
 
Videocard: 7950 @ 1Ghz
Processor: 2500k @ 4.5Ghz
Mainboard:
Memory: 8Gb 1600mhz
Soundcard: Asus Xonar DX
PSU: Antec EcoWatt 620w
Default 04-04-2013, 11:16 | posts: 811 | Location: Sheffield, England

Quote:
Originally Posted by Osamar View Post
If I am not wrong, you will need at least one pixel thick bar, becuase you could have mixed frames (tearing ???), so multiple colors in the bar.
Good point! But that still reduces the amount of storage by a very large amount.
   
Reply With Quote
 
Old
  (#11)
Ven0m
Maha Guru
 
Ven0m's Avatar
 
Videocard: ASUS GTX 680 D2CU TOP
Processor: i7 920 @ 4GHz
Mainboard: ASUS Rampage II Extreme
Memory: 12GB Patriot Viper
Soundcard: Xonar D2X>Custom amp>K712
PSU: Seasonic M12D 850W
Default 04-04-2013, 11:24 | posts: 1,522 | Location: Warsaw, Poland

Hilbert, thank you for this article. I do hope such analysis will be added to the new card reviews, especially when compering the performance of multi-GPU solutions.

However, the question arises - how reliable and jitter-free is the capture device?
   
Reply With Quote
Old
  (#12)
Pete J
Master Guru
 
Pete J's Avatar
 
Videocard: 3x 780 GTX @1113/7000MHz
Processor: i7 4930k @ 4200MHz
Mainboard: EVGA X79 Dark
Memory: 4x4GB 2400MHz Avexir Blue
Soundcard: X-Fi Titanium Fatal1ty
PSU: Thermaltake 1500W
Default 04-04-2013, 12:53 | posts: 194 | Location: Blighty

Just wanted to say thank you for an excellent article! Nice to see microstutter examined in such detail rather than subjective 'by eye' comments as has seemed to be the case up until now.
   
Reply With Quote
Old
  (#13)
yasamoka
Ancient Guru
 
Videocard: 7970CF @ 1100/1375 H2O
Processor: 5820K @ 4.3GHz 1.25V H2O
Mainboard: MSI X99S SLi Plus
Memory: 16GB Crucial DDR4-2133
Soundcard: Audio-GD NFB-11.32 +DT880
PSU: CM Silent Pro Hybrid 1300
Default 04-04-2013, 12:53 | posts: 3,375 | Location: Lebanon

Quote:
Originally Posted by Ven0m View Post
Hilbert, thank you for this article. I do hope such analysis will be added to the new card reviews, especially when compering the performance of multi-GPU solutions.

However, the question arises - how reliable and jitter-free is the capture device?
That question can probably be answered by feeding the capture card a VSynced signal. If the results turn out flat, there you go, no jitter (oversimplifying it, but oh well).

Thank you Hilbert for a very interesting read!
   
Reply With Quote
Old
  (#14)
BLEH!
Ancient Guru
 
BLEH!'s Avatar
 
Videocard: 7970 CF @ 1050/1500
Processor: 980X @ 4.00 GHz
Mainboard: ASUS P6T7 WS-SC
Memory: 12 GB Corsair @ 1910 MHz
Soundcard: Saffire Pro40/KRK Rokit 5
PSU: Enermax Revo 1500W
Default 04-04-2013, 12:56 | posts: 4,975 | Location: London

Ow, my head. Makes a bit more sense, but owwwww.
   
Reply With Quote
Old
  (#15)
rsnubje
Newbie
 
Videocard: PoV GTX 680
Processor: i7 2600K
Mainboard: Asus Maximus IV Gene-Z
Memory:
Soundcard:
PSU: Coolermaster 1200W
Default 04-04-2013, 13:11 | posts: 5

Quote:
Originally Posted by xafier View Post
Hilbert,

I've read your description a couple of times now and I'm not really sure why you need to store all the frame data? If all you care about is the differing colours couldn't you just store the 1 pixel from bottom left?

The only reason I can see for not doing that is that it the video capture device can only store the full frame and instead you'd require CPU to pull out just the 1 pixel? But I cannot imagine that would be at all taxing as extracting a value from a known location in a frame is very simple.

Just throwing out my opinion as a software developer, that thats the road I would have gone down to avoid that storage problem. It would also most likely simplify the data analysis later too.
Basicly, as it is an analysis tool, you also would like to see the frame that comes with the time. So if the graph says all the frames are for example 30ms and one or more are outstanding with let's say 60ms, you want to be able to look back at that specific frame to see what it looks like, is it or is it not a tearing or stuttering. This way you can tell if it's really giving you a bad gaming experience or not.
   
Reply With Quote
Old
  (#16)
Stukov
Ancient Guru
 
Stukov's Avatar
 
Videocard: 6970 UL / 4870X2 (dead)
Processor: Intel i5 3570k 4.2-4.4Ghz
Mainboard: Asrock Z77 Professional
Memory: 16GB Gskill 1866Mhz
Soundcard: Audigy
PSU: Corsair 850watt
Default 04-04-2013, 13:51 | posts: 4,876 | Location: South Dakota

And Vsync eliminates these issues, correct?
   
Reply With Quote
Old
  (#17)
Norvekh
Maha Guru
 
Norvekh's Avatar
 
Videocard: Sapphire 7970
Processor: Core i7 2600K @ 4.4GHz
Mainboard: ASUS Sabertooth Z77
Memory: 16GB Corsair Vengeance
Soundcard: ASUS Xonar DX
PSU: Corsair 750
Default 04-04-2013, 14:46 | posts: 2,665 | Location: Ohio

More data, when it is applicable to increasing understanding, is always a good thing. This FCAT testing most certainly provides that. That's a ton of work, and expense, but it will provide better and more accurate performance analysis and that is absolutely to be commended. Great job, Hilbert and team. You're showing exactly why this is one top notch site to be kept on the short list that I always check for reviews come new hardware.

On a sidenote, I took the FCAT of an entirely different nature while I lived in Florida. I just thought that was somewhat amusing, even if irrelevent.
   
Reply With Quote
Old
  (#18)
tsunami231
Ancient Guru
 
tsunami231's Avatar
 
Videocard: EVGA 660gtx sig2
Processor: i7 920 CNPS10X Quiet
Mainboard: Evga x58 SLI LE
Memory: 3x2gb Dominator@1600 6Gb
Soundcard: Realtek HD Audio
PSU: Antec Truepower 750
Default 04-04-2013, 15:03 | posts: 3,655 | Location: USA

And so it begins the race for who can render things with the lowest and most stable frame latency, Who will win Nvidia or AMD/ATI. Obviously Nvidia is better at this atm, but who knows what will happen.

Personal it dont mater to if it make all GPU better and smooth as a result its becomes win win situation for all of us
   
Reply With Quote
Old
  (#19)
h0d
Master Guru
 
h0d's Avatar
 
Videocard: MSI R7970 Cfx - EKwb En
Processor: 3930k - EK HF-Supreme
Mainboard: Rog Rampage Extreme IV
Memory: DDR3-2133 16g
Soundcard: HT | Omega - eClaro 7.1
PSU: Silverstone 1200W
Default 04-04-2013, 15:04 | posts: 188 | Location: Québec

Amazing review thank you
   
Reply With Quote
Old
  (#20)
BrainDedd
Master Guru
 
Videocard: ASUS Radeon HD5870
Processor: Intel Core i7-3770k
Mainboard: ASUS Maximus V Formula
Memory: Corsair Dominator 16GB
Soundcard: Onboard
PSU: Corsair AX860i
Default 04-04-2013, 15:52 | posts: 156 | Location: Johannesburg, South Africa

So ... nVidia says they've released the scripts so people can use them, but I can't find a download anywhere?
   
Reply With Quote
Old
  (#21)
djinn_
Newbie
 
djinn_'s Avatar
 
Videocard: AMD 7950
Processor: Core 2 Quad Q9400 @ 3.2
Mainboard: Asus P5Q-E
Memory: DDR2 8GB
Soundcard:
PSU: OCZ 700w
Default 04-04-2013, 16:18 | posts: 1 | Location: Tamworth

Great read comprehensive and easy to understand .
   
Reply With Quote
Old
  (#22)
trajan2448
Newbie
 
Videocard: ATI 4850
Processor: intel core 2 duo
Mainboard:
Memory:
Soundcard:
PSU: thermal
Default 04-04-2013, 17:50 | posts: 15

Quote:
Originally Posted by Hilbert Hagedoorn View Post
In this article we will introduce you towards FCAT benchmarking. The past couple of months we have seen a some new dynamics in measuring the framerate of your games. Basically the framerate of your ...

FCAT benchmarking an article introduction
I think its GREAT you guys are doing this, as it really goes much deeper than FRAPS. I hope you will delve into the whole RUNT frame issue as well.
Great site!.
   
Reply With Quote
Old
  (#23)
Holyneo
Newbie
 
Videocard: Sapphire 7970 3b oc
Processor: Intel 4770k
Mainboard: Asus RoG Max
Memory: 32g
Soundcard:
PSU: Antec 850
Default 04-04-2013, 19:00 | posts: 5

Is there a difference between what Fraps shows us in real time, compare to benchmarks that show fps in real time. In other words do benchmarks like Heaven..etc measure FPS like Fraps?
   
Reply With Quote
Old
  (#24)
JJayzX
Master Guru
 
JJayzX's Avatar
 
Videocard: Evga GTX 660 FTW Sig2 Sli
Processor: i7 4770k
Mainboard: Gigabyte Z87X-UD3H
Memory: RipjawsX 8gb DDR3 2133
Soundcard: Creative Sound Blaster Z
PSU: Corsair GS700
Default 04-04-2013, 19:24 | posts: 365 | Location: RI, USA

Quote:
Originally Posted by Ven0m View Post
However, the question arises - how reliable and jitter-free is the capture device?
It doesn't matter if the capture card is not perfect or compression sucks, as long as the frames are captured. Each frame is tagged so even if you miss a frame it would be known one is missing cause there is a sequence to the tagging.
   
Reply With Quote
Old
  (#25)
Noisiv
Ancient Guru
 
Videocard: R9 290
Processor: i5-4590
Mainboard: GA-Z97X GAMING 3
Memory: 8GB HyperX
Soundcard: Onboard Realtek ALC1150
PSU: Silent Pro M700
Default 04-04-2013, 21:03 | posts: 3,107

Quote:
Originally Posted by xafier View Post
Hilbert,

I've read your description a couple of times now and I'm not really sure why you need to store all the frame data? If all you care about is the differing colours couldn't you just store the 1 pixel from bottom left?

The only reason I can see for not doing that is that it the video capture device can only store the full frame and instead you'd require CPU to pull out just the 1 pixel? But I cannot imagine that would be at all taxing as extracting a value from a known location in a frame is very simple.
They need to store all data so they can process output frames later on.
So to obtain that "1 pixel from bottom left" you _need_ full frames output.

Alternatively script could be running real-time and extracting just frametimes and their length, ie "1 pixel from bottom left"
   
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



Powered by vBulletin®
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skin developed by: vBStyles.com
Copyright (c) 1995-2014, All Rights Reserved. The Guru of 3D, the Hardware Guru, and 3D Guru are trademarks owned by Hilbert Hagedoorn.