GPU threading optimization for frequent work updates

[cdetrio]

No description provided.

[chfast]

This works without issues on Nvidia OpenCL. It can be merged now or can wait for incoming changes.

[chfast]

Can you rebase this?

[pilsy]

Does this still need to be solved?

[cdetrio]

Yes, that's still needed. For example, if the miner's internet connection goes down then maybe there should be a timeout so the GPU stops mining when there's no new work packages coming in. Currently it will just continue to mine uncles indefinitely.

[pilsy]

The change's that i was doing up doesn't quite fit in perfectly with your changes, i am changing the way the kernels are run from the current way. Instead of running 1 kernel then running 2nd kernel before reading the results of 1st, then swap/swap... changing it over to use one main kernel, and one that halts/aborts the job.

So rather than having the kernel finishing and coming back to CPU so often, when a job comes in that miner is created and starts working, it won't ever stop working until you abort it's execution - the idea being that on the CPU you are notified of newWork and simply fire off the abort kernel to kill off the current running one, then queue up this job and fire it off.

How i'm accomplishing this with the kernels:

1. Create two command queues in a context.
2. Create two kernels, one to do the work and another to halt execution. Each kernel has access to a shared global buffer.
3. Load the first kernel into queue1.
4. Load the second kernel into queue2 when we want to halt execution.

(Alternatively we could use an out-of-order queue and load the second kernel into the same command queue to halt execution -- But i think that would be more trouble because we have to then be very careful and use clFinish/clFlush as required/necessary. However it may be a more natural way of writing/doing this... what do you think?

[cdetrio]

@pilsy I don't quite understand what you're describing, but I also have limited knowledge of openCL kernels. I just posted an explanation of the optimization in this PR. Perhaps you could expand a bit on your scheme, in the convo thread?

The only other code I haven't yet pushed is some profiling lines. I'm hoping to finally push the profiling code over the next few days, and double check the improvement.

[chfast]

To simplify: the main kernel is while(true) { if (abort); break; ...}, the second one is just abort = true;.

But I don't get why you need the second kernel to set to abort flag. A buffer shared with CPU is not enough?

[chfast]

Back to the problem, I think the miner thread should just have a flag to stop (maybe it already has). The decision when to stop should be delegated to the network thread. E.g. if the stratum thread is not able to get the work for some time it should stop the miner thread. Does it make sense?

[cdetrio]

The idea here is to reduce the "switching cost" when new work packages arrive.

Currently, whenever a new work package arrives the setWork function is called, which triggers a pause()/kickOff() cycle. pause() sets m_abort to true, which causes the kernel loop to break when it calls the searched hook. This means that the GPU thread (i.e. the kernel loop) is stopped and restarted on every new work package.

This PR is an optimization which keeps the GPU thread running continuously, instead of stopping and restarting. Rather than the GPU thread calling a hook function on every loop iteration to check if if the loop should break, it calls the hook function to check if a new work package has arrived, gets the new work if so, writes the new work to the GPU buffer, and then continues the kernel loop.

[ghost]

The idea sounds very good to me. I think the idea of a fluent GPU work instead of a start/stop behaviour will improve our miner a little bit. 👍

@cdetrio Is you code somehow ready to get merged and is well tested (in terms of "still working and same or better hashrate then the current master code")?

[chfast]

This code requires some internal changes and small improvements. But should work.

[chfast]

Do you know what was this about?

[cdetrio]

It was to support the second of two work packages: current.headerHash and previous.headerHash, where previous is the "stale" block. This was to check if a solution was for the previous block, there was an issue about it here: https://github.com/Genoil/cpp-ethereum/issues/36 (claims to be a 1.2% improvement).

When new work updates are coming very frequently (e.g. adding new tx's to the pending block every 1 second), sometimes a solution to previous is found just as a new one is arriving. But when the new work package arrives, the one in current becomes previous, and the one in previous is purged. So keeping around only the last two work packages is not enough to prevent throwing away solutions.

Rather than keep around three work packages to check solutions against, the solutions themselves now contain the work package information (seedHash, headerHash, nonce, boundary). So now it just checks that the solution is valid before submitting it to the client (eth_submitWork), rather than checking if its a solution to the current (or previous) work package.

[cdetrio]

Rebased on master.

EDIT: fixed rebase

[cdetrio]

Pushed some profiling branches.

Baseline (unoptimized) work switching cost: https://github.com/ethereum-mining/ethminer/commits/switch-profile

Optimized fast switching cost: https://github.com/ethereum-mining/ethminer/compare/fast-switch-profile

Obsolate

[ghost]

Hi @cdetrio / @chfast ,
for me this improvements sounds very good - what is the current state? Is it ready to get merged?

[chfast]

I went through big refactoring first. Some of the changes are in the master branch already (don't stop after finding solution). Some are pending: #200. Some are missing (don't restart mining on new work, just push new work package to the miner thread).

This PR was left for reference, because rebasing it make no sense now.

[chfast]

This has been done in #217.