Discussion:
[Algorithms] Timing problems
Paul at Home
2009-01-03 23:05:57 UTC
Permalink
I've just fallen foul of QPC not working correctly having just gone from a single processor to a quad core.

I remember reading somewhere that multiple processors could cause problems in the future, but I guess the future (for me) just arrived. I do handle the well-known "jump" problem but I guess that's old news now anyway.

Can anyone point me in the right direction for how to fix this please ?

Regards,
Paul Johnson.
Gregory Junker
2009-01-04 02:07:37 UTC
Permalink
Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a
single core, using SetThreadAffinityMask() (on Win32, of course - not sure
what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a
single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems
in the future, but I guess the future (for me) just arrived. I do handle the
well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.
Paul at Home
2009-01-04 11:43:40 UTC
Permalink
I've seen this around but I'm not using any threads. Not deliberately anyway. Do you put a zero in for "me" ?

Regards,
Paul Johnson.


----- Original Message -----
From: Gregory Junker
To: 'Game Development Algorithms'
Sent: Sunday, January 04, 2009 2:07 AM
Subject: Re: [Algorithms] Timing problems


Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a single core, using SetThreadAffinityMask() (on Win32, of course - not sure what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg




------------------------------------------------------------------------------

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems in the future, but I guess the future (for me) just arrived. I do handle the well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.







------------------------------------------------------------------------------


------------------------------------------------------------------------------





------------------------------------------------------------------------------


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Emil Persson
2009-01-04 14:36:53 UTC
Permalink
The problem doesn't arise from threading your application, but from calling
QPC from different CPUs. In a multi-core machine Windows will schedule each
thread to run on any CPU as it sees fit. First time-slice for your
application may run on CPU0, the next one perhaps on CPU2, and then CPU1
etc. By the time you're reading QPC you could be on any CPU and it could be
different cores every frame. Unless of course you call
SetThreadAffinityMask(). Something like this at the beginning of the
application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately
anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker <mailto:***@dayark.com>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a
single core, using SetThreadAffinityMask() (on Win32, of course - not sure
what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a
single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems
in the future, but I guess the future (for me) just arrived. I do handle the
well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.





_____

----------------------------------------------------------------------------
--



_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Paul at Home
2009-01-04 16:31:54 UTC
Permalink
Thanks Emil, I'll give that a go first. :)

Regards,
Paul Johnson.


----- Original Message -----
From: Emil Persson
To: 'Game Development Algorithms'
Sent: Sunday, January 04, 2009 2:36 PM
Subject: Re: [Algorithms] Timing problems


The problem doesn't arise from threading your application, but from calling QPC from different CPUs. In a multi-core machine Windows will schedule each thread to run on any CPU as it sees fit. First time-slice for your application may run on CPU0, the next one perhaps on CPU2, and then CPU1 etc. By the time you're reading QPC you could be on any CPU and it could be different cores every frame. Unless of course you call SetThreadAffinityMask(). Something like this at the beginning of the application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker

To: 'Game Development Algorithms'

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a single core, using SetThreadAffinityMask() (on Win32, of course - not sure what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg




----------------------------------------------------------------------------

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems in the future, but I guess the future (for me) just arrived. I do handle the well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.






----------------------------------------------------------------------------

------------------------------------------------------------------------------




----------------------------------------------------------------------------

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list



------------------------------------------------------------------------------


------------------------------------------------------------------------------





------------------------------------------------------------------------------


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Gregory Junker
2009-01-04 18:06:40 UTC
Permalink
Correct.


Greg



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 3:44 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately
anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker <mailto:***@dayark.com>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a
single core, using SetThreadAffinityMask() (on Win32, of course - not sure
what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg




_____


From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a
single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems
in the future, but I guess the future (for me) just arrived. I do handle the
well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.






_____


----------------------------------------------------------------------------
--




_____


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Gregory Junker
2009-01-04 18:12:13 UTC
Permalink
Sorry, that's what I get for posting right when I wake up.Emil had the
correct version; since it's a bitmask, you indicate the zero'th core with a
1.


Greg



_____

From: Gregory Junker [mailto:***@dayark.com]
Sent: Sunday, January 04, 2009 10:07 AM
To: 'Game Development Algorithms'
Subject: Re: [Algorithms] Timing problems



Correct.


Greg



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 3:44 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately
anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker <mailto:***@dayark.com>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a
single core, using SetThreadAffinityMask() (on Win32, of course - not sure
what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg




_____


From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a
single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems
in the future, but I guess the future (for me) just arrived. I do handle the
well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.






_____


----------------------------------------------------------------------------
--


_____


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
David Pangerl
2009-01-04 08:43:08 UTC
Permalink
Hi Paul,

I assume you are using RDTSC. If not - stop reading...
Using RDTSC is not a valid way to get time on multiple core CPUs
(http://msdn.microsoft.com/en-us/library/bb173458(VS.85).aspx).
The problem is that time stamp counters are not synchronized properly
between CPUs (from AMD http://lkml.org/lkml/2005/11/4/173).
Instead use /QueryPerformanceCounter/ and /QueryPerformanceFrequency/ as
suggested in attached MS link.

I hope that helps.

Cheers,
DAVID



David Pangerl, CEO

e-mail ***@ActaLogic.com

mobile +386 31 664 297

address ActaLogic <http://www.actalogic.com/>

Teslova 30
<http://maps.google.com/maps?hl=en&geocode=&sll=46.151241,14.995463&sspn=4.486691,5.619507&ie=UTF8&ei=IPezR6ODO5DOoAOh_rj5Dg&cd=10&q=http://bbs.keyhole.com/ubb/placemarks/cl-01-15-06-362758601.kmz&ftid=5599DFBB77E4B2B5>

1000 Ljubljana <http://en.wikipedia.org/wiki/Ljubljana>

Slovenia <http://en.wikipedia.org/wiki/Slovenia>

Europe <http://en.wikipedia.org/wiki/Europe>
Post by Paul at Home
I've just fallen foul of QPC not working correctly having just gone
from a single processor to a quad core.
I remember reading somewhere that multiple processors could cause
problems in the future, but I guess the future (for me) just arrived.
I do handle the well-known "jump" problem but I guess that's old news
now anyway.
Can anyone point me in the right direction for how to fix this please ?
Regards,
Paul Johnson.
------------------------------------------------------------------------
------------------------------------------------------------------------------
------------------------------------------------------------------------
_______________________________________________
GDAlgorithms-list mailing list
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
------------------------------------------------------------------------
No virus found in this incoming message.
Checked by AVG - http://www.avg.com
Version: 8.0.176 / Virus Database: 270.10.2/1873 - Release Date: 1/3/2009 2:14 PM
Paul at Home
2009-01-04 11:45:25 UTC
Permalink
I'm using QPC. Trouble is my timers seem to be a bit grainy and nasty. They still seem to keep accurateish time overall, but the samples I'm making each frame vary a lot more than they used to. I think - it's actually hard to debug this stuff hence me begging for previous fixes :)


Regards,
Paul Johnson.


----- Original Message -----
From: David Pangerl
To: Game Development Algorithms
Sent: Sunday, January 04, 2009 8:43 AM
Subject: Re: [Algorithms] Timing problems


Hi Paul,

I assume you are using RDTSC. If not - stop reading...
Using RDTSC is not a valid way to get time on multiple core CPUs (http://msdn.microsoft.com/en-us/library/bb173458(VS.85).aspx).
The problem is that time stamp counters are not synchronized properly between CPUs (from AMD http://lkml.org/lkml/2005/11/4/173).
Instead use QueryPerformanceCounter and QueryPerformanceFrequency as suggested in attached MS link.

I hope that helps.

Cheers,
DAVID



David Pangerl, CEO

e-mail ***@ActaLogic.com

mobile +386 31 664 297

address ActaLogic

Teslova 30

1000 Ljubljana

Slovenia

Europe



Paul at Home wrote:
I've just fallen foul of QPC not working correctly having just gone from a single processor to a quad core.

I remember reading somewhere that multiple processors could cause problems in the future, but I guess the future (for me) just arrived. I do handle the well-known "jump" problem but I guess that's old news now anyway.

Can anyone point me in the right direction for how to fix this please ?

Regards,
Paul Johnson.


----------------------------------------------------------------------------
------------------------------------------------------------------------------
----------------------------------------------------------------------------
_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list----------------------------------------------------------------------------

No virus found in this incoming message.
Checked by AVG - http://www.avg.com
Version: 8.0.176 / Virus Database: 270.10.2/1873 - Release Date: 1/3/2009 2:14 PM



------------------------------------------------------------------------------


------------------------------------------------------------------------------





------------------------------------------------------------------------------


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Jan Wassenberg
2009-01-04 12:39:56 UTC
Permalink
Hello,
Post by Paul at Home
I'm using QPC. Trouble is my timers seem to be a bit grainy and nasty.
They still seem to keep accurateish time overall, but the samples I'm
making each frame vary a lot more than they used to. I think - it's
actually hard to debug this stuff hence me begging for previous fixes :)
I have developed a workaround that uses HPET hardware (a >= 10 MHz counter
without the annoying flaws of other timers) by means of a mostly user-mode
driver.
The source code, a demo app and an accompanying article explaining the
available timers and their problems are available at
http://www.stud.uni-karlsruhe.de/~urkt/timer_test.zip [1.7 MB]
Comments and suggestions are welcome.

Cheers
Jan
Paul at Home
2009-01-04 12:48:06 UTC
Permalink
Jackpot! Many thanks, Jan.

Regards,
Paul Johnson.


----- Original Message -----
From: "Jan Wassenberg" <***@gmail.com>
To: "Game Development Algorithms" <gdalgorithms-***@lists.sourceforge.net>
Sent: Sunday, January 04, 2009 12:39 PM
Subject: Re: [Algorithms] Timing problems
Post by Jan Wassenberg
Hello,
Post by Paul at Home
I'm using QPC. Trouble is my timers seem to be a bit grainy and nasty.
They still seem to keep accurateish time overall, but the samples I'm
making each frame vary a lot more than they used to. I think - it's
actually hard to debug this stuff hence me begging for previous fixes :)
I have developed a workaround that uses HPET hardware (a >= 10 MHz counter
without the annoying flaws of other timers) by means of a mostly user-mode
driver.
The source code, a demo app and an accompanying article explaining the
available timers and their problems are available at
http://www.stud.uni-karlsruhe.de/~urkt/timer_test.zip [1.7 MB]
Comments and suggestions are welcome.
Cheers
Jan
------------------------------------------------------------------------------
_______________________________________________
GDAlgorithms-list mailing list
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Robert Walker
2013-03-22 02:40:56 UTC
Permalink
Hi Jan, I wondered, is this driver still available by any chance?

I'm trying to access the HPET timer in my software in XP or in later
versions of Windows where it may be enabled in the BIOS but seems it
is disabled by default in the OS.

With the later versions of Windows I could instruct my users to enable
it in Windows using bcdhedit but the problem with that is that in the online
forums a few people report performance issues and possibly even freezes with
it enabled (which may be why the default is for it to be disabled).

In XP there seems to be no way to access it normally.

It looks as if your driver might be a possible solution. But get 404 not found
when I try to download the zip.

This is for a software metronome.

Thanks,

Robert Walker
Jan Wassenberg
2013-03-22 03:18:41 UTC
Permalink
Hi Robert,
this driver is indeed still available. You can always check
http://trac.wildfiregames.com/browser/ps/trunk/source/lib/sysdep/os/win/aken
and I will investigate tonight where the zip file went.

Unfortunately there has been no progress in signing it - that is essential
for it to be usable by non-developers. Is there anyone who can help?

The difficulty is that it provides functions to map physical memory
(HPET is memory-mapped) and write model-specific registers, which
raises security concerns. Perhaps we can change the interface to provide
only 'safe' functionality, such as reading HPET and performance counters,
instead of allowing apps to do that and more.
Anyone willing and able to work together with me on this?
Robert Walker
2013-03-22 11:24:25 UTC
Permalink
Hi Jan,

I've got a Comodo code signing certificate renewed every year for my application
installer signing.

Just tested it on one of the .sys files for 0.A.D. and it signed it successfully
and it says it is digitally signed when I right click on properties.

Not sure how to test to see if the certificate I used is enough to let it be
installed okay.

I'm running Windows 7 64 bit here though, so ideal for testing if it does
install okay if you can tell me what to do to test that.

Presumably once installed then the application doesn't need to be run with admin
privileges to access it?

Yes it is absolutely fine if it is read only for apps like mine that only need
to read the HPET counter.

Yes I'm happy to work together with you on this if it is something I have the
ability to do and if I can find the time to do it. If it is something that can
be done in a few days of work then I can surely find the time for it one way or
another. Longer projects can be harder to find time for.

I have no experience of writing drivers though, also don't use c++, I write in
low level Windows C most of the time. But if it is just a case of removing
functionality and recompiling maybe it is something I can do?

I use Visual C++ 6.0 for my programming which is probably not new enough for
this? I use the Visual Studio family of free compilers as well, but only for
debugging from time to time to take advantage of a few debug capabilities they
have that aren't in Visual C++ 6.0.

Simplest though is just to code sign the existing version if that works, for
now. I don't really see how it is going to be a risk because to access the
driver - wouldn't an application have to be installed on your computer anyway?
Which means that the user has given it admin permissions to install whatever it
likes anyway, and if it does something malicious, surely it would do it directly
and e.g. install a known virus or spyware, rather than try to access a driver to
address the HPET counter? Or am I missing something there?

Though advantage if there is one less thing for the user to have to accept
during install.

Also, how do you use it? I've never installed a driver or interfaced directly
with a driver. Is there example code in the zip?

Thanks,

Robert
Robert Walker
2013-03-23 01:21:44 UTC
Permalink
Hi Jan,

Sorry hope you don't mind me posting again before you reply.

The thing is, I've just had a go at installing it - I found your
install_aken.bat.

Got this message:

"To install the driver, please first enable test mode:
C:\Users\Robert\AppData\Local\0AD~1.ALP\binaries\system\INSTAL~1.BAT enabletest
(This is necessary because Vista/Win7 x64 require signing with
a Microsoft "cross certificate". The Fraunhofer code signing certificate
is not enough, even though its chain of trust is impeccable.
Going the WHQL route, perhaps as an "unclassified" driver, might work.
see http://www.freeotfe.org/docs/Main/impact_of_kernel_driver_signing.htm )
Then reboot (!) and install the driver:
C:\Users\Robert\AppData\Local\0AD~1.ALP\binaries\system\INSTAL~1.BAT install ["p
ath_to_directory_containing_aken*.sys"]
(If no path is given, we will use the directory of this batch file)
To remove the driver and disable test mode, execute the following:
C:\Users\Robert\AppData\Local\0AD~1.ALP\binaries\system\INSTAL~1.BAT remove
C:\Users\Robert\AppData\Local\0AD~1.ALP\binaries\system\INSTAL~1.BAT disabletest

Press any key to continue . . ."

So - it rather looks as if my code signing certificate is no good for this, as I
thought might be a possibility. I expect it is necessary to use a Verisign
certificate.

I was told by someone at Microsoft support that it costs $99 per year for one of
those, so that's the same cost as the certificate I already use - but I've
already bought my certificate for the next year. Still I could consider it if it
solves the problem and if there is no other solution.

ALSO HAD AN IDEA ABOUT AN EASY WAY TO MAKE THE DRIVER READ ONLY.

The idea is - there is no need to remove huge chunks of the driver - I can well
understand it might take thought to do that without introducing new bugs.

You could instead just #ifdef out the routines that write to the mapped
hardware resources. I get the impression from your desciprion that probably,
that would involve very few actual lines of code.

I imagine would be things like #ifdefs to remove the lines

__writemsr(..)

and in
ZwOpenSection
to replace OPEN_ALL_ACCESS
by
SECTION_MAP_READ

that sort of thing.

So - then the driver has all your other code for writing as before, but if some
malware tries to use it to write to memory, it just "goes through the
motions" but doesn't actually do anything.

In the documentation you make clear that the driver is only to be used for read
operations.

Then from your other remarks, presumably if the driver doesn't call any
actual routines that write to hardware resources, then it mightn't need
signing at all at that point?

Is that true?

Just a thought,

Robert
Jan Wassenberg
2013-03-23 07:10:11 UTC
Permalink
We have veered outside the scope of [gd-algorithms] - I will reply directly;
anyone interested in a reduced, ideally WHQL driver capable of accessing
HPET and performance counters, please join the conversation by writing us.

Have you gone through the WHQL experience before? Are there any pitfalls
to avoid? Would love to hear from you.
Konstantin Mironovich
2013-04-12 07:28:00 UTC
Permalink
Hi folks,

Comodo Code Signing CA2 certificate could be used for code signing only.
just run certmgr.msc and look at properties.
Comodo cert is signed by UTN_USERFirst cert. the later one does not support
kernel mode code signing as well.
and only the parental AddTrust Ext CA Root cert has kmcs bit enabled.
So, COMODO does not provide certs for kmcs (sys drivers).
AFAIK the only CAs provide kmcs are Symantec/VeriSign and GlobalSign. and
their partners/franchise of course.


-km

----- Original Message -----
From: "Robert Walker" <***@robertinventor.com>
To: <GDAlgorithms-***@lists.sourceforge.net>
Sent: Friday, March 22, 2013 3:24 PM
Subject: Re: [Algorithms] Timing problems
Post by Robert Walker
Hi Jan,
I've got a Comodo code signing certificate renewed every year for my application
installer signing.
Just tested it on one of the .sys files for 0.A.D. and it signed it successfully
and it says it is digitally signed when I right click on properties.
Not sure how to test to see if the certificate I used is enough to let it be
installed okay.
I'm running Windows 7 64 bit here though, so ideal for testing if it does
install okay if you can tell me what to do to test that.
Presumably once installed then the application doesn't need to be run with admin
privileges to access it?
Yes it is absolutely fine if it is read only for apps like mine that only need
to read the HPET counter.
Yes I'm happy to work together with you on this if it is something I have the
ability to do and if I can find the time to do it. If it is something that can
be done in a few days of work then I can surely find the time for it one way or
another. Longer projects can be harder to find time for.
I have no experience of writing drivers though, also don't use c++, I write in
low level Windows C most of the time. But if it is just a case of removing
functionality and recompiling maybe it is something I can do?
I use Visual C++ 6.0 for my programming which is probably not new enough for
this? I use the Visual Studio family of free compilers as well, but only for
debugging from time to time to take advantage of a few debug capabilities they
have that aren't in Visual C++ 6.0.
Simplest though is just to code sign the existing version if that works, for
now. I don't really see how it is going to be a risk because to access the
driver - wouldn't an application have to be installed on your computer anyway?
Which means that the user has given it admin permissions to install whatever it
likes anyway, and if it does something malicious, surely it would do it directly
and e.g. install a known virus or spyware, rather than try to access a driver to
address the HPET counter? Or am I missing something there?
Though advantage if there is one less thing for the user to have to accept
during install.
Also, how do you use it? I've never installed a driver or interfaced directly
with a driver. Is there example code in the zip?
Thanks,
Robert
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
GDAlgorithms-list mailing list
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Jon Watte
2009-01-05 21:49:10 UTC
Permalink
Post by Jan Wassenberg
I have developed a workaround that uses HPET hardware (a >= 10 MHz counter
without the annoying flaws of other timers) by means of a mostly user-mode
driver.
On a modern version of Windows, with proper motherboard and CPU drivers,
QPC will use that same timer, so you don't need any custom driver.

Sincerely,

jw
Jan Wassenberg
2009-01-06 12:38:29 UTC
Permalink
Post by Jon Watte
On a modern version of Windows, with proper motherboard and CPU drivers,
QPC will use that same timer, so you don't need any custom driver.
Yes, but as mentioned in the accompanying article (excerpt follows), HPET
support hasn't been backported to Windows XP, which means many existing
systems otherwise lack a decent timer.
In the mean time, since Windows does not offer
a ubiquitous software solution, the TSC is not reliable. In an unfortunate
twist, it is precisely this timer that QPC uses on such dual-core
platforms.
One can force it to use the PMT by adding /usepmtimer to boot.ini [QPC
Workaround], but making changes to such a critical system file should not
be asked of users. Finally, despite Microsoft having issued the HPET spec-
ification 5 years ago, support for it has not been patched into Windows XP
and it is only available on Vista.

These circumstances result in the moribund state of affairs that was previ-
ously described: on many systems today, only a mere millisecond-resolution
timer is available. According to [Valve Survey], 93% of the participating
systems are running Windows XP, and 22% have more than one processor
or core. This large proportion cannot be ignored, so we will have to find a
workaround for the above issue.
<<

Best Regards
Jan
Jon Watte
2009-01-05 21:44:26 UTC
Permalink
Post by Paul at Home
I've just fallen foul of QPC not working correctly having just gone
from a single processor to a quad core.
I remember reading somewhere that multiple processors could cause
problems in the future, but I guess the future (for me) just arrived.
I do handle the well-known "jump" problem but I guess that's old news
now anyway.
Can anyone point me in the right direction for how to fix this please ?
There are two fixes:

1) Keep the last value returned, and return max(lastValue, nextValue).
Requires some degree of locking or compare/exchange to work in a
multi-threaded fashion, or using TLS for the "lastValue".

2) Install proper CPU drivers for your system so you don't get
incoherent results. <-- vastly preferred method

Sincerely,

jw
Gregory Junker
2009-01-05 22:13:23 UTC
Permalink
Post by Jon Watte
2) Install proper CPU drivers for your system so you don't get
incoherent results. <-- vastly preferred method
Thus shifting the problem from the developers to post-sales support...
Paul at Home
2009-01-05 22:28:11 UTC
Permalink
or "retard management" as I prefer to think of it. :)

My old man is otherwise extremely intelligent but he still thinks if he
presses the wrong keys at the wrong time that the USA will nuke China or
something. Until that generation has moved on, I'll go with software-side
fixes every time if possible. :)

Regards,
Paul Johnson.


----- Original Message -----
From: "Gregory Junker" <***@dayark.com>
To: "'Game Development Algorithms'"
<gdalgorithms-***@lists.sourceforge.net>
Sent: Monday, January 05, 2009 10:13 PM
Subject: Re: [Algorithms] Timing problems
Post by Gregory Junker
Post by Jon Watte
2) Install proper CPU drivers for your system so you don't get
incoherent results. <-- vastly preferred method
Thus shifting the problem from the developers to post-sales support...
------------------------------------------------------------------------------
_______________________________________________
GDAlgorithms-list mailing list
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Dan White
2009-01-06 19:58:30 UTC
Permalink
We stuck the SetThreadAffinityMask code in our libraries, and our timing
problems went away.

However, about 6 mo later, we ended up with a weird bug:

We have a converter that run as part of our build process. To speed up
the build, we started running multiple instances of the converter on a
multicore machine. However, the build times didn't go down, and closer
examination showed that only one CPU use being used.

Turns out SetThreadAffinityMask was doing exactly what it was supposed
to and making all our processes run on one core. Our solution was to
look at the processor identifier and skip the affinity business CPU's we
know to work correctly.

-D

________________________________

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems


Thanks Emil, I'll give that a go first. :)

Regards,
Paul Johnson.



----- Original Message -----
From: Emil Persson <mailto:***@comhem.se>
To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>
Sent: Sunday, January 04, 2009 2:36 PM
Subject: Re: [Algorithms] Timing problems


The problem doesn't arise from threading your application, but
from calling QPC from different CPUs. In a multi-core machine Windows
will schedule each thread to run on any CPU as it sees fit. First
time-slice for your application may run on CPU0, the next one perhaps on
CPU2, and then CPU1 etc. By the time you're reading QPC you could be on
any CPU and it could be different cores every frame. Unless of course
you call SetThreadAffinityMask(). Something like this at the beginning
of the application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not
deliberately anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker <mailto:***@dayark.com>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time
master to run on only a single core, using SetThreadAffinityMask() (on
Win32, of course - not sure what the equivalent, if any, might be on
Linux).




http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg



________________________________

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly
having just gone from a single processor to a quad core.



I remember reading somewhere that multiple processors
could cause problems in the future, but I guess the future (for me) just
arrived. I do handle the well-known "jump" problem but I guess that's
old news now anyway.



Can anyone point me in the right direction for how to
fix this please ?


Regards,
Paul Johnson.





________________________________


------------------------------------------------------------------------
------



________________________________

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:

http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-lis
t


________________________________





------------------------------------------------------------------------
------





________________________________




_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:

http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-lis
t
Paul at Home
2009-01-06 22:18:28 UTC
Permalink
I did wonder about this actually, especially with the other hot topic being about parallel computing!

My problem that led me to post in the first place had me confused as all the examples people are talking about on here don't fit my case anyway. My entire codebase doesn't use threads in earnest (apart from a background pre-loader that's gone by the time I'm looking) and my chip/mobo and everything else is brand new so shouldn't have any legacy problems.

But using the affinity mask as detailed did indeed make my samples much smoother. I don't see how that could be a bug in my own code so now I'm worried I've stumbled across something new.

And what happens if I decide to do more aggressive threading, possibly utilising the code being discussed elsewhere. How do you guys get your timings to do any profiling with whilst running so many different threads ?

Regards,
Paul Johnson.


----- Original Message -----
From: Dan White
To: Game Development Algorithms
Sent: Tuesday, January 06, 2009 7:58 PM
Subject: Re: [Algorithms] Timing problems


We stuck the SetThreadAffinityMask code in our libraries, and our timing problems went away.

However, about 6 mo later, we ended up with a weird bug:

We have a converter that run as part of our build process. To speed up the build, we started running multiple instances of the converter on a multicore machine. However, the build times didn't go down, and closer examination showed that only one CPU use being used.

Turns out SetThreadAffinityMask was doing exactly what it was supposed to and making all our processes run on one core. Our solution was to look at the processor identifier and skip the affinity business CPU's we know to work correctly.

-D



------------------------------------------------------------------------------
From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems


Thanks Emil, I'll give that a go first. :)

Regards,
Paul Johnson.


----- Original Message -----
From: Emil Persson
To: 'Game Development Algorithms'
Sent: Sunday, January 04, 2009 2:36 PM
Subject: Re: [Algorithms] Timing problems


The problem doesn't arise from threading your application, but from calling QPC from different CPUs. In a multi-core machine Windows will schedule each thread to run on any CPU as it sees fit. First time-slice for your application may run on CPU0, the next one perhaps on CPU2, and then CPU1 etc. By the time you're reading QPC you could be on any CPU and it could be different cores every frame. Unless of course you call SetThreadAffinityMask(). Something like this at the beginning of the application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker

To: 'Game Development Algorithms'

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a single core, using SetThreadAffinityMask() (on Win32, of course - not sure what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg




--------------------------------------------------------------------------

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems in the future, but I guess the future (for me) just arrived. I do handle the well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.






--------------------------------------------------------------------------

------------------------------------------------------------------------------




--------------------------------------------------------------------------

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list



----------------------------------------------------------------------------


------------------------------------------------------------------------------





----------------------------------------------------------------------------


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list




------------------------------------------------------------------------------


------------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB




------------------------------------------------------------------------------


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Gregory Junker
2009-01-06 22:25:39 UTC
Permalink
It doesn't matter whether you use threads or not -- your main process thread
can (and will) be moved by the OS from any core to any other core.
Single-threaded processes that use QPF/QPC will have the same issues as
massively-threaded ones.

Greg


_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Tuesday, January 06, 2009 2:18 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems


I did wonder about this actually, especially with the other hot topic being
about parallel computing!

My problem that led me to post in the first place had me confused as all the
examples people are talking about on here don't fit my case anyway. My
entire codebase doesn't use threads in earnest (apart from a background
pre-loader that's gone by the time I'm looking) and my chip/mobo and
everything else is brand new so shouldn't have any legacy problems.

But using the affinity mask as detailed did indeed make my samples much
smoother. I don't see how that could be a bug in my own code so now I'm
worried I've stumbled across something new.

And what happens if I decide to do more aggressive threading, possibly
utilising the code being discussed elsewhere. How do you guys get your
timings to do any profiling with whilst running so many different threads ?

Regards,
Paul Johnson.



----- Original Message -----
From: Dan White <mailto:***@PipeWorks.Com>
To: Game Development <mailto:gdalgorithms-***@lists.sourceforge.net>
Algorithms
Sent: Tuesday, January 06, 2009 7:58 PM
Subject: Re: [Algorithms] Timing problems

We stuck the SetThreadAffinityMask code in our libraries, and our timing
problems went away.

However, about 6 mo later, we ended up with a weird bug:

We have a converter that run as part of our build process. To speed up the
build, we started running multiple instances of the converter on a multicore
machine. However, the build times didn't go down, and closer examination
showed that only one CPU use being used.

Turns out SetThreadAffinityMask was doing exactly what it was supposed to
and making all our processes run on one core. Our solution was to look at
the processor identifier and skip the affinity business CPU's we know to
work correctly.

-D

_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems


Thanks Emil, I'll give that a go first. :)

Regards,
Paul Johnson.



----- Original Message -----
From: Emil Persson <mailto:***@comhem.se>
To: 'Game Development <mailto:gdalgorithms-***@lists.sourceforge.net>
Algorithms'
Sent: Sunday, January 04, 2009 2:36 PM
Subject: Re: [Algorithms] Timing problems


The problem doesn't arise from threading your application, but from calling
QPC from different CPUs. In a multi-core machine Windows will schedule each
thread to run on any CPU as it sees fit. First time-slice for your
application may run on CPU0, the next one perhaps on CPU2, and then CPU1
etc. By the time you're reading QPC you could be on any CPU and it could be
different cores every frame. Unless of course you call
SetThreadAffinityMask(). Something like this at the beginning of the
application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately
anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory <mailto:***@dayark.com> Junker

To: 'Game Development <mailto:gdalgorithms-***@lists.sourceforge.net>
Algorithms'

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a
single core, using SetThreadAffinityMask() (on Win32, of course - not sure
what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a
single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems
in the future, but I guess the future (for me) just arrived. I do handle the
well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.





_____

----------------------------------------------------------------------------
--



_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list



_____




----------------------------------------------------------------------------
--






_____




_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list





_____




----------------------------------------------------------------------------
--
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB





_____




_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Mat Noguchi
2009-01-06 22:31:07 UTC
Permalink
You can always set the affinity for a thread that controls the jobs submitted to other threads.

MSN
From: Paul at Home [mailto:***@rubicondev.com]
Sent: Tuesday, January 06, 2009 2:18 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems

I did wonder about this actually, especially with the other hot topic being about parallel computing!

My problem that led me to post in the first place had me confused as all the examples people are talking about on here don't fit my case anyway. My entire codebase doesn't use threads in earnest (apart from a background pre-loader that's gone by the time I'm looking) and my chip/mobo and everything else is brand new so shouldn't have any legacy problems.

But using the affinity mask as detailed did indeed make my samples much smoother. I don't see how that could be a bug in my own code so now I'm worried I've stumbled across something new.

And what happens if I decide to do more aggressive threading, possibly utilising the code being discussed elsewhere. How do you guys get your timings to do any profiling with whilst running so many different threads ?

Regards,
Paul Johnson.


----- Original Message -----
From: Dan White<mailto:***@PipeWorks.Com>
To: Game Development Algorithms<mailto:gdalgorithms-***@lists.sourceforge.net>
Sent: Tuesday, January 06, 2009 7:58 PM
Subject: Re: [Algorithms] Timing problems

We stuck the SetThreadAffinityMask code in our libraries, and our timing problems went away.

However, about 6 mo later, we ended up with a weird bug:

We have a converter that run as part of our build process. To speed up the build, we started running multiple instances of the converter on a multicore machine. However, the build times didn't go down, and closer examination showed that only one CPU use being used.

Turns out SetThreadAffinityMask was doing exactly what it was supposed to and making all our processes run on one core. Our solution was to look at the processor identifier and skip the affinity business CPU's we know to work correctly.

-D

________________________________
From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems
Thanks Emil, I'll give that a go first. :)

Regards,
Paul Johnson.


----- Original Message -----
From: Emil Persson<mailto:***@comhem.se>
To: 'Game Development Algorithms'<mailto:gdalgorithms-***@lists.sourceforge.net>
Sent: Sunday, January 04, 2009 2:36 PM
Subject: Re: [Algorithms] Timing problems

The problem doesn't arise from threading your application, but from calling QPC from different CPUs. In a multi-core machine Windows will schedule each thread to run on any CPU as it sees fit. First time-slice for your application may run on CPU0, the next one perhaps on CPU2, and then CPU1 etc. By the time you're reading QPC you could be on any CPU and it could be different cores every frame. Unless of course you call SetThreadAffinityMask(). Something like this at the beginning of the application should do:
SetThreadAffinityMask(GetCurrentThread(), 1);

This will stick the thread to CPU 0.

-Emil


From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems

I've seen this around but I'm not using any threads. Not deliberately anyway. Do you put a zero in for "me" ?

Regards,
Paul Johnson.


----- Original Message -----
From: Gregory Junker<mailto:***@dayark.com>
To: 'Game Development Algorithms'<mailto:gdalgorithms-***@lists.sourceforge.net>
Sent: Sunday, January 04, 2009 2:07 AM
Subject: Re: [Algorithms] Timing problems

Hi Paul

By "jump problem", do you mean this?

http://support.microsoft.com/kb/274323


Either way, simply set the thread that is your time master to run on only a single core, using SetThreadAffinityMask() (on Win32, of course - not sure what the equivalent, if any, might be on Linux).

http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx


Yes, it's a hack, but at present, it's what we got. ;)

Greg

________________________________
From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems

I've just fallen foul of QPC not working correctly having just gone from a single processor to a quad core.

I remember reading somewhere that multiple processors could cause problems in the future, but I guess the future (for me) just arrived. I do handle the well-known "jump" problem but I guess that's old news now anyway.

Can anyone point me in the right direction for how to fix this please ?

Regards,
Paul Johnson.


________________________________
------------------------------------------------------------------------------
________________________________
_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
________________________________
------------------------------------------------------------------------------

________________________________
_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
________________________________
------------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB
________________________________
_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Emil Persson
2009-01-07 18:32:04 UTC
Permalink
Well, that's not necessary, unless the other threads also need to read the
QPC independently (rather than being provided the QPC value from the main
thread). Calling SetThreadAffinityMask() only affects the thread you're
passing to it. So if you have a main thread and a bunch of worker threads
and QPC is only read in the main thread to compute your frame times there's
no need to call SetThreadAffinityMask() for any of the worker threads. It's
probably better to leave it up to Windows to schedule them anyway it likes.
Unless the work is very balanced and you know something about the cache
structure of the CPU so that you could get some benefits from
preserved/shared caches, but generally I would just leave it up to Windows.
If the work load is unbalanced between threads you may artificially throttle
your app performance if you force each thread to only run on a particular
core.



Another thing to note is that you can set the affinity mask to several CPUs.
For instance if you're forcing the main thread to run on CPU0, and the main
thread has a lot of work, it could be beneficial to set the affinity mask on
worker threads to allow all CPUs except CPU0, just to avoid them ever
stealing it from the main thread. Could be a performance gain if the main
thread is the bottleneck.



And just for the record there's also a SetProcessAffinityMask() function,
which sets the affinity mask globally to the process. I would generally
avoid that one.



-Emil





From: Mat Noguchi [mailto:***@bungie.com]
Sent: 06 January 2009 23:31
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



You can always set the affinity for a thread that controls the jobs
submitted to other threads.



MSN

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Tuesday, January 06, 2009 2:18 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I did wonder about this actually, especially with the other hot topic being
about parallel computing!



My problem that led me to post in the first place had me confused as all the
examples people are talking about on here don't fit my case anyway. My
entire codebase doesn't use threads in earnest (apart from a background
pre-loader that's gone by the time I'm looking) and my chip/mobo and
everything else is brand new so shouldn't have any legacy problems.



But using the affinity mask as detailed did indeed make my samples much
smoother. I don't see how that could be a bug in my own code so now I'm
worried I've stumbled across something new.



And what happens if I decide to do more aggressive threading, possibly
utilising the code being discussed elsewhere. How do you guys get your
timings to do any profiling with whilst running so many different threads ?


Regards,
Paul Johnson.





----- Original Message -----

From: Dan White <mailto:***@PipeWorks.Com>

To: Game Development Algorithms
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Tuesday, January 06, 2009 7:58 PM

Subject: Re: [Algorithms] Timing problems



We stuck the SetThreadAffinityMask code in our libraries, and our timing
problems went away.



However, about 6 mo later, we ended up with a weird bug:



We have a converter that run as part of our build process. To speed up the
build, we started running multiple instances of the converter on a multicore
machine. However, the build times didn't go down, and closer examination
showed that only one CPU use being used.



Turns out SetThreadAffinityMask was doing exactly what it was supposed to
and making all our processes run on one core. Our solution was to look at
the processor identifier and skip the affinity business CPU's we know to
work correctly.



-D



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems

Thanks Emil, I'll give that a go first. :)


Regards,
Paul Johnson.





----- Original Message -----

From: Emil Persson <mailto:***@comhem.se>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:36 PM

Subject: Re: [Algorithms] Timing problems



The problem doesn't arise from threading your application, but from calling
QPC from different CPUs. In a multi-core machine Windows will schedule each
thread to run on any CPU as it sees fit. First time-slice for your
application may run on CPU0, the next one perhaps on CPU2, and then CPU1
etc. By the time you're reading QPC you could be on any CPU and it could be
different cores every frame. Unless of course you call
SetThreadAffinityMask(). Something like this at the beginning of the
application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately
anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker <mailto:***@dayark.com>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a
single core, using SetThreadAffinityMask() (on Win32, of course - not sure
what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a
single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems
in the future, but I guess the future (for me) just arrived. I do handle the
well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.





_____

----------------------------------------------------------------------------
--

_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

_____

----------------------------------------------------------------------------
--

_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

_____

----------------------------------------------------------------------------
--
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB

_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Ted Jump
2009-01-06 23:04:01 UTC
Permalink
You could also go so far as always setting affinity, but rotating which CPU
it is done on and letting your timer basis code be thread local.



You could not compare absolute values between threads (without a global
locked timer) but you could at least be assured that if QPC/whatever was
core-specific that you were getting consistent values per-thread.



Or I'm missing something wacked here, very possible too.



-t



Ps: in theory, would keeping threads core-assigned improve cache coherency?



From: Paul at Home [mailto:***@rubicondev.com]
Sent: Tuesday, January 06, 2009 4:18 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I did wonder about this actually, especially with the other hot topic being
about parallel computing!



My problem that led me to post in the first place had me confused as all the
examples people are talking about on here don't fit my case anyway. My
entire codebase doesn't use threads in earnest (apart from a background
pre-loader that's gone by the time I'm looking) and my chip/mobo and
everything else is brand new so shouldn't have any legacy problems.



But using the affinity mask as detailed did indeed make my samples much
smoother. I don't see how that could be a bug in my own code so now I'm
worried I've stumbled across something new.



And what happens if I decide to do more aggressive threading, possibly
utilising the code being discussed elsewhere. How do you guys get your
timings to do any profiling with whilst running so many different threads ?


Regards,
Paul Johnson.





----- Original Message -----

From: Dan White <mailto:***@PipeWorks.Com>

To: Game Development Algorithms
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Tuesday, January 06, 2009 7:58 PM

Subject: Re: [Algorithms] Timing problems



We stuck the SetThreadAffinityMask code in our libraries, and our timing
problems went away.



However, about 6 mo later, we ended up with a weird bug:



We have a converter that run as part of our build process. To speed up the
build, we started running multiple instances of the converter on a multicore
machine. However, the build times didn't go down, and closer examination
showed that only one CPU use being used.



Turns out SetThreadAffinityMask was doing exactly what it was supposed to
and making all our processes run on one core. Our solution was to look at
the processor identifier and skip the affinity business CPU's we know to
work correctly.



-D



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems

Thanks Emil, I'll give that a go first. :)


Regards,
Paul Johnson.





----- Original Message -----

From: Emil Persson <mailto:***@comhem.se>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:36 PM

Subject: Re: [Algorithms] Timing problems



The problem doesn't arise from threading your application, but from calling
QPC from different CPUs. In a multi-core machine Windows will schedule each
thread to run on any CPU as it sees fit. First time-slice for your
application may run on CPU0, the next one perhaps on CPU2, and then CPU1
etc. By the time you're reading QPC you could be on any CPU and it could be
different cores every frame. Unless of course you call
SetThreadAffinityMask(). Something like this at the beginning of the
application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not deliberately
anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker <mailto:***@dayark.com>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your time master to run on only a
single core, using SetThreadAffinityMask() (on Win32, of course - not sure
what the equivalent, if any, might be on Linux).



http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we got. ;)


Greg



_____

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working correctly having just gone from a
single processor to a quad core.



I remember reading somewhere that multiple processors could cause problems
in the future, but I guess the future (for me) just arrived. I do handle the
well-known "jump" problem but I guess that's old news now anyway.



Can anyone point me in the right direction for how to fix this please ?


Regards,
Paul Johnson.





_____

----------------------------------------------------------------------------
--

_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

_____

----------------------------------------------------------------------------
--



_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list

_____

----------------------------------------------------------------------------
--
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB

_____

_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Mat Noguchi
2009-01-06 23:12:45 UTC
Permalink
Only if the threads never context switched while on that core. I forget if context switches flush the CPU caches or not.

MSN
From: Ted Jump [mailto:***@gmail.com]
Sent: Tuesday, January 06, 2009 3:04 PM
To: 'Game Development Algorithms'
Subject: Re: [Algorithms] Timing problems

You could also go so far as always setting affinity, but rotating which CPU it is done on and letting your timer basis code be thread local.

You could not compare absolute values between threads (without a global locked timer) but you could at least be assured that if QPC/whatever was core-specific that you were getting consistent values per-thread.

Or I'm missing something wacked here, very possible too.

-t

Ps: in theory, would keeping threads core-assigned improve cache coherency?

From: Paul at Home [mailto:***@rubicondev.com]
Sent: Tuesday, January 06, 2009 4:18 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems

I did wonder about this actually, especially with the other hot topic being about parallel computing!

My problem that led me to post in the first place had me confused as all the examples people are talking about on here don't fit my case anyway. My entire codebase doesn't use threads in earnest (apart from a background pre-loader that's gone by the time I'm looking) and my chip/mobo and everything else is brand new so shouldn't have any legacy problems.

But using the affinity mask as detailed did indeed make my samples much smoother. I don't see how that could be a bug in my own code so now I'm worried I've stumbled across something new.

And what happens if I decide to do more aggressive threading, possibly utilising the code being discussed elsewhere. How do you guys get your timings to do any profiling with whilst running so many different threads ?

Regards,
Paul Johnson.


----- Original Message -----
From: Dan White<mailto:***@PipeWorks.Com>
To: Game Development Algorithms<mailto:gdalgorithms-***@lists.sourceforge.net>
Sent: Tuesday, January 06, 2009 7:58 PM
Subject: Re: [Algorithms] Timing problems

We stuck the SetThreadAffinityMask code in our libraries, and our timing problems went away.

However, about 6 mo later, we ended up with a weird bug:

We have a converter that run as part of our build process. To speed up the build, we started running multiple instances of the converter on a multicore machine. However, the build times didn't go down, and closer examination showed that only one CPU use being used.

Turns out SetThreadAffinityMask was doing exactly what it was supposed to and making all our processes run on one core. Our solution was to look at the processor identifier and skip the affinity business CPU's we know to work correctly.

-D

________________________________
From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems
Thanks Emil, I'll give that a go first. :)

Regards,
Paul Johnson.


----- Original Message -----
From: Emil Persson<mailto:***@comhem.se>
To: 'Game Development Algorithms'<mailto:gdalgorithms-***@lists.sourceforge.net>
Sent: Sunday, January 04, 2009 2:36 PM
Subject: Re: [Algorithms] Timing problems

The problem doesn't arise from threading your application, but from calling QPC from different CPUs. In a multi-core machine Windows will schedule each thread to run on any CPU as it sees fit. First time-slice for your application may run on CPU0, the next one perhaps on CPU2, and then CPU1 etc. By the time you're reading QPC you could be on any CPU and it could be different cores every frame. Unless of course you call SetThreadAffinityMask(). Something like this at the beginning of the application should do:
SetThreadAffinityMask(GetCurrentThread(), 1);

This will stick the thread to CPU 0.

-Emil


From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems

I've seen this around but I'm not using any threads. Not deliberately anyway. Do you put a zero in for "me" ?

Regards,
Paul Johnson.


----- Original Message -----
From: Gregory Junker<mailto:***@dayark.com>
To: 'Game Development Algorithms'<mailto:gdalgorithms-***@lists.sourceforge.net>
Sent: Sunday, January 04, 2009 2:07 AM
Subject: Re: [Algorithms] Timing problems

Hi Paul

By "jump problem", do you mean this?

http://support.microsoft.com/kb/274323


Either way, simply set the thread that is your time master to run on only a single core, using SetThreadAffinityMask() (on Win32, of course - not sure what the equivalent, if any, might be on Linux).

http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx


Yes, it's a hack, but at present, it's what we got. ;)

Greg

________________________________
From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems

I've just fallen foul of QPC not working correctly having just gone from a single processor to a quad core.

I remember reading somewhere that multiple processors could cause problems in the future, but I guess the future (for me) just arrived. I do handle the well-known "jump" problem but I guess that's old news now anyway.

Can anyone point me in the right direction for how to fix this please ?

Regards,
Paul Johnson.


________________________________
------------------------------------------------------------------------------
________________________________
_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
________________________________
------------------------------------------------------------------------------
________________________________
_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
________________________________
------------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB
________________________________
_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list
Randall Turner
2009-01-07 16:09:08 UTC
Permalink
Context switches don't (trigger cache flushes) on 360/PS3. I'd doubt
they do in PC land, but there's so many flavors of so many
hardware/software combos I'd shudder to have to "prove" that's the case.




Keeping threads core-assigned helps coherency on the 360 and makes no
difference on the PS3 and see "so many flavors" above for the PC.



As far as measuring performance, yet again the PC is challenging. I'd do
stopwatch-tests. (forty bazillion iterations.) The other two are pretty
straightforward, though most of this discussion re: lockfree vs.
critical section is moot for the PS3 as far as performance goes. (But
you'll save on your limited OS objects on the PS3 by using lockfree
implementations.)



Basically, on the 360, you get about 4x performance improvement in
high-contention scenarios using lockfree vs. cs'd containers. Approx.
Sometimes more, sometimes less.



________________________________

From: Mat Noguchi [mailto:***@bungie.com]
Sent: Tuesday, January 06, 2009 5:13 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



Only if the threads never context switched while on that core. I forget
if context switches flush the CPU caches or not.



MSN

From: Ted Jump [mailto:***@gmail.com]
Sent: Tuesday, January 06, 2009 3:04 PM
To: 'Game Development Algorithms'
Subject: Re: [Algorithms] Timing problems



You could also go so far as always setting affinity, but rotating which
CPU it is done on and letting your timer basis code be thread local.



You could not compare absolute values between threads (without a global
locked timer) but you could at least be assured that if QPC/whatever was
core-specific that you were getting consistent values per-thread.



Or I'm missing something wacked here, very possible too.



-t



Ps: in theory, would keeping threads core-assigned improve cache
coherency?



From: Paul at Home [mailto:***@rubicondev.com]
Sent: Tuesday, January 06, 2009 4:18 PM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I did wonder about this actually, especially with the other hot topic
being about parallel computing!



My problem that led me to post in the first place had me confused as all
the examples people are talking about on here don't fit my case anyway.
My entire codebase doesn't use threads in earnest (apart from a
background pre-loader that's gone by the time I'm looking) and my
chip/mobo and everything else is brand new so shouldn't have any legacy
problems.



But using the affinity mask as detailed did indeed make my samples much
smoother. I don't see how that could be a bug in my own code so now I'm
worried I've stumbled across something new.



And what happens if I decide to do more aggressive threading, possibly
utilising the code being discussed elsewhere. How do you guys get your
timings to do any profiling with whilst running so many different
threads ?


Regards,
Paul Johnson.





----- Original Message -----

From: Dan White <mailto:***@PipeWorks.Com>

To: Game Development Algorithms
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Tuesday, January 06, 2009 7:58 PM

Subject: Re: [Algorithms] Timing problems



We stuck the SetThreadAffinityMask code in our libraries, and
our timing problems went away.



However, about 6 mo later, we ended up with a weird bug:



We have a converter that run as part of our build process. To
speed up the build, we started running multiple instances of the
converter on a multicore machine. However, the build times didn't go
down, and closer examination showed that only one CPU use being used.



Turns out SetThreadAffinityMask was doing exactly what it was
supposed to and making all our processes run on one core. Our solution
was to look at the processor identifier and skip the affinity business
CPU's we know to work correctly.



-D




________________________________


From: Paul at Home [mailto:***@rubicondev.com]
Sent: Sunday, January 04, 2009 8:32 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems

Thanks Emil, I'll give that a go first. :)


Regards,
Paul Johnson.





----- Original Message -----

From: Emil Persson <mailto:***@comhem.se>

To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:36 PM

Subject: Re: [Algorithms] Timing problems



The problem doesn't arise from threading your
application, but from calling QPC from different CPUs. In a multi-core
machine Windows will schedule each thread to run on any CPU as it sees
fit. First time-slice for your application may run on CPU0, the next one
perhaps on CPU2, and then CPU1 etc. By the time you're reading QPC you
could be on any CPU and it could be different cores every frame. Unless
of course you call SetThreadAffinityMask(). Something like this at the
beginning of the application should do:

SetThreadAffinityMask(GetCurrentThread(), 1);



This will stick the thread to CPU 0.



-Emil





From: Paul at Home [mailto:***@rubicondev.com]
Sent: 04 January 2009 12:44
To: Game Development Algorithms
Subject: Re: [Algorithms] Timing problems



I've seen this around but I'm not using any threads. Not
deliberately anyway. Do you put a zero in for "me" ?


Regards,
Paul Johnson.





----- Original Message -----

From: Gregory Junker <mailto:***@dayark.com>


To: 'Game Development Algorithms'
<mailto:gdalgorithms-***@lists.sourceforge.net>

Sent: Sunday, January 04, 2009 2:07 AM

Subject: Re: [Algorithms] Timing problems



Hi Paul



By "jump problem", do you mean this?



http://support.microsoft.com/kb/274323





Either way, simply set the thread that is your
time master to run on only a single core, using SetThreadAffinityMask()
(on Win32, of course - not sure what the equivalent, if any, might be on
Linux).




http://msdn.microsoft.com/en-us/library/ms686247(VS.85).aspx





Yes, it's a hack, but at present, it's what we
got. ;)


Greg




________________________________


From: Paul at Home [mailto:***@rubicondev.com]
Sent: Saturday, January 03, 2009 3:06 PM
To: gdalgorithms-***@lists.sourceforge.net
Subject: [Algorithms] Timing problems



I've just fallen foul of QPC not working
correctly having just gone from a single processor to a quad core.



I remember reading somewhere that multiple
processors could cause problems in the future, but I guess the future
(for me) just arrived. I do handle the well-known "jump" problem but I
guess that's old news now anyway.



Can anyone point me in the right direction for
how to fix this please ?


Regards,
Paul Johnson.






________________________________



------------------------------------------------------------------------
------


________________________________


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:

http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-lis
t


________________________________



------------------------------------------------------------------------
------


________________________________


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:

http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-lis
t


________________________________



------------------------------------------------------------------------
------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB


________________________________


_______________________________________________
GDAlgorithms-list mailing list
GDAlgorithms-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:

http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-lis
t
Jon Watte
2009-01-07 23:02:36 UTC
Permalink
Post by Mat Noguchi
Only if the threads never context switched while on that core. I
forget if context switches flush the CPU caches or not.
In general they don't have to, because caches will generally be on
physical (not logical) addresses. However, whatever thread you switch to
is likely (but not guaranteed) to be cache thirsty enough that, when you
get switched back in, the majority of your data is no longer there.


Sincerely,

jw
Loading...