Optimise work updates on CUDA, was Branch#359 #361

MariusVanDerWijden · 2017-10-28T15:53:08Z

Working on issue #359
The newest commit should work, Please test it!

smurfy · 2017-10-29T08:23:17Z

Just a note, @chfast mentioned before. If possible please make this similiar to the OpenCL variant #217
And also if there are duplicate code move them to the base class. Thanks

chfast · 2017-11-03T20:49:02Z

How is it going?

MariusVanDerWijden · 2017-11-20T16:16:39Z

I am working on it, but i don't have a suitable gpu to test my changes

ZiDanRO · 2017-11-22T11:20:17Z

If you need a tester, i'm availoable. I have gtx 1060 3gb and 6 gb and also gtx 1070 to test it.
Here is my email if i can help you with this part.
[email protected]

LE: I'm using windows 10 64bit 1073 version if this has any relevance for you

kkkrackpot · 2017-11-22T13:29:44Z

I could test it too with 1060 and CUDA 9 on Linux -- if someone can tell me how to patch ethminer from git with these changes.

MariusVanDerWijden · 2017-11-23T11:22:56Z

Travis ci:
"The command "pip install requests" failed and exited with 127 during ."
Maybe the script is wrong

chfast · 2017-11-23T16:21:58Z

Yes, something changed in Travis/macOS, but it is not related. I will try to fix it soon.

chfast · 2017-11-24T08:55:02Z

travis fixed in master branch

chfast · 2017-11-24T16:34:56Z

Can you rebase this? Is this ready?

MariusVanDerWijden · 2017-11-24T18:26:05Z

This is not ready!

ZiDanRO · 2017-11-25T14:57:52Z

Yes this is not ready, it gets stuck at generating DAG

C:\ethminer>ethminer.exe --farm-recheck 5000 -U -S eu1.ethermine.org:4444 -FS eu1.ethermine.org:14444 -O 0x.................. --cuda-parallel-hash 4 --api-port 3333
cu 16:48:04|main Using grid size 8192 , block size 128
i 16:48:04|main Connecting to stratum server eu1.ethermine.org:4444
i 16:48:04|stratum Connected to stratum server eu1.ethermine.org : 4444
i 16:48:04|stratum Starting farm
i 16:48:04|CUDA0 No work. Pause for 3 s.
i 16:48:04|CUDA1 No work. Pause for 3 s.
i 16:48:04|CUDA2 No work. Pause for 3 s.
i 16:48:04|CUDA3 No work. Pause for 3 s.
i 16:48:04|CUDA4 No work. Pause for 3 s.
i 16:48:04|CUDA5 No work. Pause for 3 s.
i 16:48:04|CUDA6 No work. Pause for 3 s.
i 16:48:04|stratum Subscribed to stratum server
i 16:48:04|stratum Authorized worker 0x7.........................
i 16:48:04|stratum Received new job #da64956c
i 16:48:07|CUDA0 set work; seed: #89c875b9, target: #0000000112e0
i 16:48:07|CUDA0 Initialising miner...
i 16:48:07|CUDA2 No work. Pause for 3 s.
i 16:48:07|CUDA1 No work. Pause for 3 s.
i 16:48:07|CUDA3 No work. Pause for 3 s. i 16:48:07|CUDA4 No work. Pause for 3 s.
i 16:48:07|CUDA6 No work. Pause for 3 s.
i 16:48:07|CUDA5 No work. Pause for 3 s.

MariusVanDerWijden · 2017-11-26T17:07:15Z

Thats propably because of a previous bug, I'm going to rebase it later, but thanks for testing 👍

ZiDanRO · 2017-11-26T21:31:51Z

No problem, i will test everything because i can't use 12 final version. It seams it is worst than 12.rc1 on nvidia miners. To much hash drops like other reported already. Now everything seems it is on your shoulders to find out were is the problem and if i can help, i do it gladly.
Im sad i can't help more with the programming, its to much for my understanding and no time to research.

MariusVanDerWijden · 2017-11-26T22:42:37Z

I rebased it now, can you retest it?

ZiDanRO · 2017-11-26T23:03:23Z

Sorry the same, it can't pass the generation of DAG. It generat's it until 97% and it starst again.
And i don't know why the error of no work...

ethminer -U -S eu1.ethermine.org:4444 -O 0xY.............. --cuda-parallel-hash 4
cu 00:57:42|main Using grid size 8192 , block size 128
i 00:57:42|main Connecting to stratum server eu1.ethermine.org:4444
i 00:57:42|stratum Connected to stratum server eu1.ethermine.org : 4444
i 00:57:42|stratum Starting farm
i 00:57:42|CUDA0 No work. Pause for 3 s.
i 00:57:42|CUDA1 No work. Pause for 3 s.
i 00:57:42|CUDA2 No work. Pause for 3 s.
i 00:57:42|CUDA3 No work. Pause for 3 s.
i 00:57:42|CUDA4 No work. Pause for 3 s.
i 00:57:42|CUDA5 No work. Pause for 3 s.
i 00:57:42|CUDA6 No work. Pause for 3 s.
i 00:57:42|stratum Subscribed to stratum server
i 00:57:42|stratum Authorized worker 0x74B9aad.................
i 00:57:42|stratum Received new job #0dcd693a
i 00:57:45|CUDA2 No work. Pause for 3 s.
i 00:57:45|CUDA1 No work. Pause for 3 s.
i 00:57:45|CUDA0 set work; seed: #4be89018, target: #0000000112e0
i 00:57:45|CUDA0 Initialising miner...
i 00:57:45|CUDA4 No work. Pause for 3 s. i 00:57:45|CUDA3 No work. Pause for 3 s.

i 00:57:45|CUDA5 No work. Pause for 3 s. i 00:57:45|CUDA6 No work. Pause for 3 s.

cu 00:57:47|CUDA0 Using device: GeForce GTX 1070 (Compute 6.1)
cu 00:57:47|CUDA0 Generating DAG for GPU # 0
CUDA#0: 0%
CUDA#0: 3%
CUDA#0: 6%
CUDA#0: 8%
i 00:57:48|CUDA1 No work. Pause for 3 s.
i 00:57:48|CUDA2 No work. Pause for 3 s.
i 00:57:48|CUDA4 No work. Pause for 3 s.
i 00:57:48|CUDA3 No work. Pause for 3 s.
i 00:57:48|CUDA5 No work. Pause for 3 s.
i 00:57:48|CUDA6 No work. Pause for 3 s.
CUDA#0: 11%

MariusVanDerWijden · 2017-11-27T08:57:05Z

No work. Pause for 3 s. this is no real problem, the new code polls if there is new work, but i't can't receive new Work if the miner/stratum/dag is not initialized. It's purely semantical, but I'm going to change it in the evening

ZiDanRO · 2017-11-27T11:45:27Z

it looks like it is working, but i don't know how and the rate, no other info than this

LE: it's not submitting anything. On Ethermine.org the miner apears offline!

chfast · 2017-11-29T12:28:56Z

@ZiDanRO from you logs it looks like something is broken in the logic in the miner and it does not see the work updates.

ZiDanRO · 2017-12-03T07:30:06Z

I see that Appveyor can't build so that i could test the new kernel. When is ready i will try it.

MariusVanDerWijden · 2017-12-27T17:48:06Z

Oh man, I really hate it... I'm going to invest my Christmas money into buying a GPU to test that stuff myself. I'm really sorry. Can you run valgrind/gdb or something over it?
Did you test it before the latest commits?

ZiDanRO · 2017-12-27T18:14:14Z

in windows is ok. no errors until now

jean-m-cyr · 2017-12-27T19:08:40Z

Under Ubuntu 16.04 LTS problem is with hwmon. Runs fine without -HWMON option.

Code crashes trying to display per card temp and fan speed data.

Not seeing performance improvement, perhaps even slight performance loss. Will monitor for an hour for stale rate.

smurfy · 2017-12-27T20:00:31Z

I will test the hwmon stuff. soon

jean-m-cyr · 2017-12-28T02:25:59Z

This continues to work well when the hardware monitor is disabled.

I'd get rid of the per card "Set work;" logs. They present identical information for all GPUs, information already provided by the single stratum log.

jean-m-cyr · 2017-12-28T03:39:36Z

Also now works with hwmon enabled (-HWMON) ??? Where'd that get fixed?

NOTE: I deleted the cnote << "set work;"... log in my local build, but that shouldn't have anything to do with it.

chfast · 2017-12-28T09:31:05Z

libethash-cuda/CUDAMiner.cpp

@@ -17,6 +17,8 @@ along with cpp-ethereum.  If not, see <http://www.gnu.org/licenses/>.
 /** @file CUDAMiner.cpp
 * @author Gav Wood <[email protected]>
 * @date 2014
+* @author MariusVanDerWijden


It's better to remove entire comment. Git should be enough to find contributors.

chfast · 2017-12-28T09:31:15Z

libethash-cuda/CUDAMiner.cpp

@@ -113,75 +115,87 @@ void CUDAMiner::report(uint64_t _nonce)
 void CUDAMiner::kickOff()
 {
 	m_hook->reset();
-	startWorking();
+	//startWorking();


You mean the commented out part

chfast · 2017-12-28T09:33:35Z

libethash-cuda/CUDAMiner.cpp

-			delete m_miner;
+		//delete m_miner;
+		//m_miner = new ethash_cuda_miner;
+		if(!m_miner)


You can just embed ethash_cuda_miner in CUDAMiner. I will remove it anyway later on.

I don't know what you mean

Nevermind, I will fix it later.

chfast · 2017-12-28T09:51:40Z

libethash-cuda/CUDAMiner.cpp

-			bytesConstRef lightData = light->data();
+		EthashAux::LightType light;
+		light = EthashAux::light(seed);
+		//bytesConstRef dagData = dag->data();


I don't know what you mean

chfast · 2017-12-28T09:53:24Z

libethash-cuda/CUDAMiner.cpp

+	current.seed = h256{1u};
+	try
+	{
+		while(true)


When this loop ends?

Good point, fixed it

chfast · 2017-12-28T09:53:33Z

libethash-cuda/CUDAMiner.cpp

@@ -194,7 +208,7 @@ void CUDAMiner::workLoop()
 void CUDAMiner::pause()
 {
 	m_hook->abort();
-	stopWorking();
+	//stopWorking();


chfast · 2017-12-28T09:53:59Z

libethash-cuda/ethash_cuda_miner.cpp

@@ -17,6 +17,8 @@
 /** @file ethash_cuda_miner.cpp
 * @author Genoil <[email protected]>
 * @date 2015
+* @author MariusVanDerWijden


Remove entire comment.

chfast · 2017-12-28T09:55:54Z

libethash-cuda/ethash_cuda_miner.cpp

@@ -63,6 +65,11 @@ ethash_cuda_miner::search_hook::~search_hook() {}

 ethash_cuda_miner::ethash_cuda_miner()
 {
+	int devicesCount = getNumDevices();
+	m_light = new hash64_t*[devicesCount];


WTF? Use std::vector<hash64_t*>(deviceCount).

We only have a fixed no. of devices, i use this as a lookup whether a device has a light-struct already allocated. Since the number of devices during an execution doesn't change its save to use an array and index it by device_num

std::vector will do the same (single allocation once) and you will not have to fill it with nullptrs manually and delete in the end.

chfast · 2017-12-28T09:56:33Z

libethash-cuda/ethash_cuda_miner.cpp

+	m_light = new hash64_t*[devicesCount];
+	for(int i = 0; i < devicesCount; i++)
+		m_light[i] = nullptr;
+	m_dag = nullptr;


Init m_dag in declaration.

chfast · 2017-12-28T09:57:09Z

libethash-cuda/ethash_cuda_miner.h

@@ -58,7 +58,14 @@ class ethash_cuda_miner
 	uint64_t m_starting_nonce;
 	uint64_t m_current_index;
 	uint32_t m_sharedBytes;
+
+	///Constants on GPU
+	hash128_t* m_dag;


Init all the fields.

jean-m-cyr · 2017-12-28T13:53:39Z

Left running overnight on 3 cards.

Still has problem in HWMON. All three cards report exactly the same temp and fan speed percentage.

smurfy · 2017-12-28T14:07:29Z

@jean-m-cyr
Just to make sure, you mean nvidia cards, i know this is a cuda fix branch but still, nvidia should work, but i know i had a bug on amd (adl) on windows. I currently only can test amd_sysfs (3 cards) and nvidia on (8 cards) linux and nvidia on windows (one card)

edit: update, i can reproduce the problem on windows, testing on linux soon

smurfy · 2017-12-28T15:26:56Z

libethash-cuda/ethash_cuda_miner.cpp


 		if (device_count == 0)
 			return false;

 		// use selected device
-		m_device_num = std::min<int>((int)_deviceId, device_count - 1);
-
+		unsigned device_num = _deviceId < device_count -1 ? _deviceId : device_count - 1;


need to use class variable m_device_num (or set it to same value), happy to change it to unsigned, but hwmon needs the device_num later to fetch stats for the correct device. See line 404 in this file.

Yep...
Changing m_device_num type to uint32, and all references to device_num in ethash_cuda_miner::init to m_device_num does the trick.

Oh yeah I'm sorry... fixed it now

@subtly is this fine now?

chfast

Only this vector instead of raw dynamic array left. I'm fine with the rest.

chfast · 2017-12-28T17:38:06Z

libethash-cuda/CUDAMiner.cpp

-			delete m_miner;
+		//delete m_miner;
+		//m_miner = new ethash_cuda_miner;
+		if(!m_miner)


Nevermind, I will fix it later.

chfast · 2017-12-28T17:39:14Z

libethash-cuda/ethash_cuda_miner.cpp

+
+ethash_cuda_miner::~ethash_cuda_miner()
+{
+	delete[] m_light;


This will not be needed at at all if std::vector used.

chfast · 2017-12-28T17:51:23Z

libethash-cuda/ethash_cuda_miner.cpp

-	for(int i = 0; i < devicesCount; i++)
+	int deviceCount = getNumDevices();
+	m_light = std::vector<hash64_t*>(deviceCount);
+	for(int i = 0; i < deviceCount; i++)


Not needed. vector's constructor will init all elements with default values, in this case with nullptr.

chfast · 2017-12-28T17:52:50Z

libethash-cuda/ethash_cuda_miner.cpp

-	m_light = new hash64_t*[devicesCount];
-	for(int i = 0; i < devicesCount; i++)
+	int deviceCount = getNumDevices();
+	m_light = std::vector<hash64_t*>(deviceCount);


This is fine, but because m_light is a vector already, you can alternatively write it like m_light.resize(deviceCount). Still it will init new elements with nullptr.

ddobreff · 2017-12-28T18:14:53Z

Confirmed working with -HWMON enabled.

chfast · 2017-12-28T18:20:12Z

Ready to merge?

MariusVanDerWijden · 2017-12-28T18:22:21Z

Yes, Thank you all for bearing with me and for your continuous support

chfast · 2017-12-28T18:25:42Z

Great work! Welcome to the group of the ethminer contributors.

smurfy changed the title ~~Branch #359~~ Optimise work updates on CUDA, was Branch#359 Nov 24, 2017

MariusVanDerWijden added 2 commits November 26, 2017 23:27

Started on ethereum-mining#359

6adfaa8

Rewrote CUDAMiner

a816f1c

MariusVanDerWijden force-pushed the branch_#359 branch 2 times, most recently from 1e282d2 to a816f1c Compare November 26, 2017 22:35

minor fix

1be1486

rewrote the kernel once again

42bd287

MariusVanDerWijden added 4 commits December 3, 2017 09:34

Merge branch 'master' into branch_#359

2853d0c

added support for __shfl_sync since __shfl is deprecated with CUDA 9.0

138173d

added support for shfl_sync

8c7b88e

added Bitmask to all shfl_sync operations, to target all warps

59a1227

MariusVanDerWijden added 4 commits December 28, 2017 01:54

removed unnecessary print

6a251e6

Merge branch 'master' into branch_#359

4d60935

Merge branch 'master' into branch_#359

274500d

fixed merge error

7c0aecf

removed unnecessary cnote

c77a0ca

chfast suggested changes Dec 28, 2017

View reviewed changes

changed requested changes

8aba73b

smurfy suggested changes Dec 28, 2017

View reviewed changes

changed hwmon breaking bug

a8facaf

chfast reviewed Dec 28, 2017

View reviewed changes

use std::vector instead of raw array

5a42eb7

chfast suggested changes Dec 28, 2017

View reviewed changes

fixed init of m_light

6b00137

chfast approved these changes Dec 28, 2017

View reviewed changes

smurfy approved these changes Dec 28, 2017

View reviewed changes

chfast merged commit 627859d into ethereum-mining:master Dec 28, 2017

DLS-bau mentioned this pull request Jan 14, 2018

0.13rc6 - CUDA mining: an illegal instruction was encountered #559

Closed

Optimise work updates on CUDA, was Branch#359 #361

Optimise work updates on CUDA, was Branch#359 #361

Uh oh!

Conversation

MariusVanDerWijden commented Oct 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smurfy commented Oct 29, 2017

Uh oh!

chfast commented Nov 3, 2017

Uh oh!

MariusVanDerWijden commented Nov 20, 2017

Uh oh!

ZiDanRO commented Nov 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kkkrackpot commented Nov 22, 2017

Uh oh!

MariusVanDerWijden commented Nov 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chfast commented Nov 23, 2017

Uh oh!

chfast commented Nov 24, 2017

Uh oh!

chfast commented Nov 24, 2017

Uh oh!

MariusVanDerWijden commented Nov 24, 2017

Uh oh!

ZiDanRO commented Nov 25, 2017

Uh oh!

MariusVanDerWijden commented Nov 26, 2017

Uh oh!

ZiDanRO commented Nov 26, 2017

Uh oh!

MariusVanDerWijden commented Nov 26, 2017

Uh oh!

ZiDanRO commented Nov 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MariusVanDerWijden commented Nov 27, 2017

Uh oh!

ZiDanRO commented Nov 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chfast commented Nov 29, 2017

Uh oh!

ZiDanRO commented Dec 3, 2017

Uh oh!

MariusVanDerWijden commented Dec 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ZiDanRO commented Dec 27, 2017

Uh oh!

jean-m-cyr commented Dec 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smurfy commented Dec 27, 2017

Uh oh!

jean-m-cyr commented Dec 28, 2017

Uh oh!

jean-m-cyr commented Dec 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MariusVanDerWijden commented Oct 28, 2017 •

edited

Loading

ZiDanRO commented Nov 22, 2017 •

edited

Loading

MariusVanDerWijden commented Nov 23, 2017 •

edited

Loading

ZiDanRO commented Nov 26, 2017 •

edited

Loading

ZiDanRO commented Nov 27, 2017 •

edited

Loading

MariusVanDerWijden commented Dec 27, 2017 •

edited

Loading

jean-m-cyr commented Dec 27, 2017 •

edited

Loading

jean-m-cyr commented Dec 28, 2017 •

edited

Loading

MariusVanDerWijden Dec 28, 2017 •

edited

Loading

smurfy commented Dec 28, 2017 •

edited

Loading

smurfy Dec 28, 2017 •

edited

Loading

jean-m-cyr Dec 28, 2017 •

edited

Loading