Multicore processing

[Victorio]

Hi,

I did some performance test with multicore processing, here is result:

Two processes need process:

A. serial on one CPU
1.process, then 2.nd proces
time 31 sec

B. serial with random switch between 2 CPU
1.process, then 2.nd proces
time 31 sec

C. paralell on one CPU
1 and 2 process run concurently
time 31 sec

D. paralell with random switch between 2 CPU
1 and 2 process run concurently
time 22 sec

E. paralell , with each module on other CPU, one on CPU1 two on CPU2
1 and 2 process run concurently
time 15sec !!!

As you can see, process need 1/2 time !!!

This help me shorten process time on 12core PC from 12 hours to about 1 hour (I believe... )

So can be some problems with memory, ..... handles,...... etc. necessary many tests.....

[bborzic]

E. paralell , with each module on other CPU, one on CPU1 two on CPU2
1 and 2 process run concurently
time 15sec !!!
As you can see, process need 1/2 time !!!

I was speaking with Roger recently and I happened to mention that it would be nice if Alaska put some effort in improving the core compiler like allowing a single Xbase++ app to utilize more than one processor/CPU. He said he thought there were some changes in Xbase 2.0 and that there were some messages on this newsgroup re. this subject.

The only thing I could find is this thread, but if I read correctly nothing really has changed. This thread talks about starting multiple copies of an application and assigning a different processor to it. Each exe is still running on only one processor/cpu (although a different one from the other copies of the exe).

Re. Xbase 2.0: I also do not think anything has changed. All I can see is that Alaska has added 2 new functions GetLogicalProcessorCount / SetLogicalProcessor. These new functions are essentially wrappers for this

function SmpGetCPU() ; Return DllCall("Xpprt1.dll",DLL_CDECL,"_sysGetCPU")
function SmpSetCPU(n); Return DllCall("Xpprt1.dll",DLL_CDECL,"_sysSetCPU", n)

Is this correct, or have we figured out how to use more than one processor at the same time inside one exe?

[Victorio]

Hi

My application uses one main program, which controlled processes. And after put some process to front , this was broken to several parts , number parts is calculated by number logical processors and run via runshell external exe module. Every exe module was called with parameter - number CPU.
Main app run on CPU1, and on Core 4 running 4 exe modules on CPU1, CPU2, CPU3 and CPU4.
I tested it also on 24 CPU server, and works fine, process shorten to cca 1/24 time.
Comunicating every modules with main app is by shared DBF file.

Broke process is for example, I want search in 2400 files, have 4 CPU, break 2400/4 = 600 files.
Every exe module processing 600 files.

BUT !!!, I saw, if I have not correct set number CPU (not 1,2,4,8,16,32), and set for example to 3,5,... Tasmanager show, then works more than one CPU in one time ?!.

This shows, than Alaska Xbase can work with more than one CPU in one time in one EXE.

I have not time to analyze it, also this do not give me some effect to shorting process time, best results is how I used.

Some problem I have with RAM and Handles, but this is also if I use only one CPU.

Victorio

[rdonnay]

He said he thought there were some changes in Xbase 2.0 and that there were some messages on this newsgroup re. this subject.

Boris -

I did some more research on the subject and realized that all Alaska did was to write new functions in 2.0 called SetLogicalProcessor() and GetLogicalProcessor(). I was wrong when I thought that these functions could set a different processor for each thread.

Roger

[Auge_Ohr]

BUT !!!, I saw, if I have not correct set number CPU (not 1,2,4,8,16,32), and set for example to 3,5,... Taskmanager show, then works more than one CPU in one time ?!.

CPU are set binary in "Register" so 3 mean 1 + 2.

This shows, than Alaska Xbase can work with more than one CPU in one time in one EXE.

have you try it
even if it work with other Apps ( CMD / Cl*pper ) it does NOT work with Xbase++ (> v1.3x )

[Victorio]

sorry, my mistake, I tried this again, and only one CPU in one time running.(with 3 binary 11, only CPU0, with 15 also only CPU0)

I had other problem in application, when :
have CPU with 4 cores.
run 4 times external module each on own CPU, 1.st on CPU0, 2.cnd on CPU1,...
but I swithing bad., with number 1,2,3,4 and not on 1,2,4,8
This caused, that on CPU 0 was more than 1 process running.

My solution with several EXE modules called by Runshell works fine, but better will be to can set CPU for threads in one EXE....

[bborzic]

I did some more research on the subject and realized that all Alaska did was to write new functions in 2.0 called SetLogicalProcessor() and GetLogicalProcessor(). I was wrong when I thought that these functions could set a different processor for each thread.

Thanks, that's what I thought. They have not made any progress on this front - that's too bad because CPU's are not getting any faster and the only way to speed things up is to utilize multiple processors. We need Alaska Software to put more effort into the core compiler instead of creating more window dressing. I have talked to Steffen about this many times in the past, but I'm probably a minority. It's interesting, but the Xbase 3.0 announcement that they sent out 2 weeks ago did not include anything what I would consider 'core' compiler improvements. Nice additions for sure, but not essential because an Xbase++ programmer already has many options available right now, to do what they promise in 3.0. Anyway, that's my 2c on the subject.

[bwolfsohn]

I did some more research on the subject and realized that all Alaska did was to write new functions in 2.0 called SetLogicalProcessor() and GetLogicalProcessor(). I was wrong when I thought that these functions could set a different processor for each thread.

Thanks, that's what I thought. They have not made any progress on this front - that's too bad because CPU's are not getting any faster and the only way to speed things up is to utilize multiple processors. We need Alaska Software to put more effort into the core compiler instead of creating more window dressing. I have talked to Steffen about this many times in the past, but I'm probably a minority. It's interesting, but the Xbase 3.0 announcement that they sent out 2 weeks ago did not include anything what I would consider 'core' compiler improvements. Nice additions for sure, but not essential because an Xbase++ programmer already has many options available right now, to do what they promise in 3.0. Anyway, that's my 2c on the subject.

the 3.0 announcement wasn't worth the paper it wasn't printed on..

[bborzic]

Thanks, that's what I thought. They have not made any progress on this front - that's too bad because CPU's are not getting any faster and the only way to speed things up is to utilize multiple processors. We need Alaska Software to put more effort into the core compiler instead of creating more window dressing. I have talked to Steffen about this many times in the past, but I'm probably a minority. It's interesting, but the Xbase 3.0 announcement that they sent out 2 weeks ago did not include anything what I would consider 'core' compiler improvements. Nice additions for sure, but not essential because an Xbase++ programmer already has many options available right now, to do what they promise in 3.0. Anyway, that's my 2c on the subject.

the 3.0 announcement wasn't worth the paper it wasn't printed on..

Indeed... but the one I got was not even printed on paper. How did you get a printed copy?

[bwolfsohn]

Thanks, that's what I thought. They have not made any progress on this front - that's too bad because CPU's are not getting any faster and the only way to speed things up is to utilize multiple processors. We need Alaska Software to put more effort into the core compiler instead of creating more window dressing. I have talked to Steffen about this many times in the past, but I'm probably a minority. It's interesting, but the Xbase 3.0 announcement that they sent out 2 weeks ago did not include anything what I would consider 'core' compiler improvements. Nice additions for sure, but not essential because an Xbase++ programmer already has many options available right now, to do what they promise in 3.0. Anyway, that's my 2c on the subject.

the 3.0 announcement wasn't worth the paper it wasn't printed on..

Indeed... but the one I got was not even printed on paper. How did you get a printed copy?

Boris,
re-read the message..

"the paper it WASN'T printed on"