Jump to content

Welcome to ExtremeHW

Welcome to ExtremeHW, like most online communities you must register to view or post in our community, but don't worry this is a simple free process that requires minimal information for you to signup. Be a part of ExtremeHW by signing in or creating an account.

  • Start new topics and reply to others
  • Subscribe to topics and forums to get email updates
  • Get your own profile page and make new friends
  • Send personal messages to other members.
  • Take advantage of site exclusive features.

ExtremeHW Community Folding Project 2.0


Recommended Posts

Administrators

 

ExtremeHW-Logo-With-slogan-800-black.png.06447b919dad048117f725f83601c905.png                     Corsair-logo.thumb.png.9d2305086a2b7936bf3f225984787e0d.png

 

Hello all, 

 

So we originally had the Community Folding Project Thread which tracked a build of what was initially going to be the CFP for EHW, however upon sitting back and evaluating the build, it would not be able to perform at its best in its mining like rig configuration due to the fact that the PCIe bandwidth would be limited to 1x, which thanks to some very helpful individuals here were able to point out that there would be a fairly hefty performance hit. With this in mind, it was time to re-hash the CFP and do a rebuild best suited to Folding@Home overall. 

 

This thread will serve as a build log/CFP information centre. 

 

Firstly I wish to thank Corsair for sponsoring this project. They have been kind enough to support this project by sending us over the Corsair HX1000 Platinum PSU and a whopping 6x Corsair Premium PCIe 3.0 x16 Extension Cable 300mm riser cables. Corsair have been kind enough to support our folding events, so it is great to see that continued effort here as well!

 

Community Folding Project Specifications 

 

CPU:  Intel i9 10900x

CPU Cooler: EK AIO Basic 360

Motherboard: Asus WS X299 SAGE
RAM:  Corsair Dominator 3200Mhz (64Gb)
Case: XCase Minestation 1

PSU 1: Corsair HX1000 Platinum

PSU 2: Silverstone ST1500-Ti

Riser Cables: 6x Corsair Premium PCIe 3.0 x16 Extension Cable 300mm

Controller GPU: Nvidia GT710

Folding GPU's:  6x Nvidia Tesla M40

 

Due to the cost of GPU's at this point in time, thanks to the silicon shortage, buying gaming GPU's for folding would be prohibitively expensive. As such, thanks to @Avacado , I learned about the Nvidia Tesla M40 cards which are meant for GPU learning but serve Folding@Home very well. The Tesla M40 has the same/approximate folding power of the GTX1070 at about 1 Million PPD+ per GPU.  I was able to get 6x Tesla M40's from the USA for approx £500. If I were to get 6x GTX 1070's, we would be looking at approximately £2100, again this is due to the silicon shortage and mining craze. The maths was easy, it was time to buy the Tesla M40's.

 

Some of you may be wondering why I have such a powerful CPU for what will essentially be a GPU only folding rig. Well the answer is simple, it is all about PCIe lanes. In order to make sure that each GPU was running 4-8x bandwidth, it was essential to get a CPU that could deal out enough PCIe lanes to make that happen. The Intel i9 10900x has a total of 48 lanes which means that it will be able to host the 6x Tesla M40 GPU's without a detrimental performance hit.

 

At the time of buying the XCase Minestation 1 chassis, the manufacturer were out of stock. I managed to find one on Ebay, but of course due to the mining craze and lack of stock, there was some bidding competition to be had. This case was the perfect setup, and without making my own case, there was no other "off the shelf" product like this. So what happened ? I ended up paying through the teeth for it. RRP for the Minestation 1 is £118 (Inc Vat), I paid a healthy £270 for it. 

 

When initially looking at the motherboard and CPU combo to make this project happen, I found a rather nice bundle on Ebay which included the Asus WS X299 SAGE, Corsair Dominator 64GB RAM & an Intel i7 9800x. I managed to nab this at what I think was a good deal at £697. You will notice that I am not using the i7 9800x in this build, the simple reason is that it did not have enough PCIe lanes to make this project a success, hence the upgrade to the i9 10900x. At least I can sell the i7 9800x now. Oh and yes I know 64Gb RAM is overkill for this project, but it came with the board, so what can I say lol.

 

Product Pictures & Bits Required 

 

A build log cannot be complete without pictures of the products being used as well as those little bits and bobs you have to get.

 

IMG20210303141807.thumb.jpg.fcb08e10b1c60654fec84802d85ccc03.jpg

 

The XCase Minestation 1 in its flatpack form.

 

IMG20210325133008.thumb.jpg.ec297a37ce182b9efc109035d2364fd0.jpg

 

The Corsair HX1000 PSU and riser cables that were very kindly provided by Corsair.

 

IMG20210324101134.thumb.jpg.d5ddb1e97ed38dee73d12909cba0f1b0.jpg

 

Due to clearance issues that would likely arise with going air, I decided to go AIO for this project.

 

IMG20210329142141.thumb.jpg.0592b911585211066d6f1bb7658c8a29.jpg 

 

Extra bits and bobs. The ones that are not obvious are new mounting brackets for the Tesla M40's and the power adapters for M40's.

 

Build Progress

 

IMG20210303163912.thumb.jpg.177079a3adc7b9115eeff234f3802dc1.jpg

 

IMG20210303163916.thumb.jpg.804e75f38b965664ad314bae17dfdf6f.jpg

 

IMG20210305131123.thumb.jpg.81c485c6f2f1aae393d7a01e10ff29fb.jpg

 

IMG20210305131130.thumb.jpg.479d05ca969584c3fd6a5ef25c2bb815.jpg

 

IMG20210307190734.thumb.jpg.cbf2823e1bf9009ea35f2fe50bda9e38.jpg

 

A little bit of case modding so my Nvidia GT710 controller card is in the motherboard traditionally.

 

IMG20210307190745.thumb.jpg.bfbb8f231465814fd18a6cb1af9cc980.jpg

 

IMG20210324111734.thumb.jpg.778fc05bd25a74ad45d25c58685da1f8.jpg

 

IMG20210324111726.thumb.jpg.351e7e098c43cbb6fe74c1f31eaf3b3c.jpg

 

The AIO is soon to be put into a push pull config with a dedicated fan controller.

 

IMG20210324111729.thumb.jpg.9c26b1407a420437b898cf67cdd767fc.jpg

 

IMG20210405183356.jpg.abdc2bfcf030173d8b0a0298ea6fb687.thumb.jpg.1c8cd3313edd0f722bba1d59272b9cdb.jpg

 

So I did a little cable management today after getting the Lamptron fan controller installed

 

IMG20210405183408.jpg.c47dabd7a97fb9be131519407d45f5d1.thumb.jpg.93352d361a88e6015d24419c0782d0d0.jpg

 

IMG20210405183417.jpg.d5b7b1e3ba0801dbbbaa019b3cad7107.thumb.jpg.341ec53b52990da53a616a2e41025148.jpg

 

Arrival of cooling fans for M40's as well as some replacement thermal pads

 

IMG20210418111200.jpg.f382b7062436a5199b3d1a8f83237c89.thumb.jpg.4530b638b4d6379cd1b1aaece4c32b80.jpg

 

One awesome PSU sent over to us for review that will get used in the CFP as our GPU PSU 

 

IMG20210430162442.thumb.jpg.b829b9763c73cc0df116c4cdabb04a40.jpg.b242d9fde02b62f2bdd776d988f02a90.thumb.jpg.fc6e323388bf8a43bdd11cb6eeb9477e.jpg

 

Testing each of the received M40's to ensure they work

 

IMG20210417170138.jpg.ba90518c9fd27c8b5734d9d1f080adb1.thumb.jpg.3e47c24acc536df62bfdf8683704a50a.jpg 

 

 

Second PSU is installed some new cable management (not that you would think it) along with the M40 power adapters hooked up to the PCI-E power cables. 

 

IMG20210430180137.jpg.c082c9731ee2a749e0a1630bdf6c7771.thumb.jpg.be7592915f15dd823574a8ad1ab8cb89.jpg 

 

411810619_IMG20210430180206.jpg.ac4975b66da894fb3ef817d25d6f69f2(1).thumb.jpg.b933972ac53a778261a5c137ec30788d.jpg

 

IMG20210430180159.jpg.e641848e383b7560d759515601a90ee2.thumb.jpg.df1695edb041371d9b324efbc374cc8f.jpg

 

IMG20210430180150.jpg.284f66a9341916b8b6ec6b5b2a1c01f8.thumb.jpg.b579e230b6a2369ce124602a2a074a4c.jpg

 

 

Some TLC for the M40's, thermal paste and thermal pads replaced. Removed the shrouds as they are IMHO stupid bits of plastic that only aid to retain heat

 

IMG20210429201734.jpg.329362a32ca14bfc1ad6c968f16906ba.thumb.jpg.c8d05558f1f2e60c47135f5665b9a907.jpg

 

IMG20210429201749.jpg.dae9486ca720c4cae55f9e19d9b393f6.thumb.jpg.72be1637c212d4c2567f95614e27a69e.jpg

 

Initial line-up and testing of all M40's in the rig, though it was quickly apparent they needed extra fans to blow cool air down through the heatsinks 

 

IMG20210502121348.jpg.2150e8bb11fa9ac3177b70e736aa7a3e.thumb.jpg.7653ea5a0b0e8e25b300be4805fc0dae.jpg

 

The addition of the fans to aid with cooling. 

 

IMG20210503123708.jpg.d280c87a1c8dd92854f01062016b5ff8.thumb.jpg.099c103a694d12ea6df86c828dbb9725.jpg

 

I think we can add some MORE fans, so lets start modding the lid panel to add 3x 120mm Gentle Typhoon fans to blow cool air down onto the GPU's directed at the GPU core rear.

 

IMG20210503123713.jpg.33302d4923e750675aea16e1780b52ef.thumb.jpg.7e1020f5c92e37c401db636b26532e4f.jpg

 

So I am pleased to say that we are virtually there now, modding of the lid is now complete with its dust filter and grills. Grills are in place to stop any loose wires from getting snagged up in the fan if anyone was wondering lol. What you see below is pretty much the finished article, now just to set it up. Did I mention this things weighs a fair bit lol.

 

5.thumb.jpg.65d1a3724a1b812cde725ba2903d037d.jpg

 

4.thumb.jpg.7c87b3aee2893452a6e4b20f7b2e6165.jpg

 

1.thumb.jpg.e1a392dc925f44dcb87d9f806ce9ce1a.jpg

 

3.thumb.jpg.26ddbda37cdc0de5ca3c480bcdd16408.jpg

 

2.thumb.jpg.e2ffaa91cda3f7d9cf3adeb516d551a5.jpg

Link to post
Share on other sites
  • Replies 75
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Update time.    So I am pleased to say that we are virtually there now, modding of the lid is now complete with its dust filter and grills. Grills are in place to stop any loose wires from g

Hello all,    So we originally had the Community Folding Project Thread which tracked a build of what was initially going to be the CFP for EHW, however upon sitting back and evaluating the

So I did some more work today now that I received the second PSU from Silverstone. A nice 1500Watt 80+ Titanium.   Installed the last of the fans with their grilles to avoid cables slipping

Posted Images

Administrators

I am now just waiting on the following items to complete the project, at least so far as hardware config: 

 

1x Set of fan extensions 

1x Fan controller so I can best set fan speeds. 

1x Fan filters/covers 

1x New thermal pads and Thermal Grizzly for M40 make over.

6x Nvidia Tesla M40 GPU's (These are being tested by my good man @mllrkllr88 in the states before shipping to me)

 

More to come soon !

Link to post
Share on other sites
EHW News Editor

+Subbed. Thanks for the shout out. Can't wait to see what you come up with.  Wanna share some of that Corsair sponsor love? 😛 Tell them i'm short on ML120's. I only have about 30 of them. 

 

P.S. I freed up 5 of the custom coolers for the M40's as they will be going under water (I think I have 7 laying around). Let me know if you want them and i'll ship them to you. 

Edited by Avacado
Link to post
Share on other sites
Premium Platinum

Just what I needed to make my day even better, an update on the CFP!! :wheee_orange:

Nice writeup E, must be a pleasure to work with all that cool hardware. Did you feel the weight of that motherboard?

 

3 hours ago, ENTERPRISE said:

Firstly I wish to thank Corsair for sponsoring this project.

Thanks a lot for sponsoring this project Corsair! 

 

3 hours ago, ENTERPRISE said:

RRP for the Minestation 1 is £118 (Inc Vat), I paid a healthy £270 for it.

That's a lot of money, but at least you have the perfect case now. Looks like a good quality case with all the features you need for this project to work.

 

37 minutes ago, Avacado said:

Let me know if you want them and I'll ship them to you.

I'm sure they will be needed with 6 M40's in there! 

Link to post
Share on other sites
Administrators

Lamptron fan controller arrived yesterday, so I can finally cable management the large array of fan cables haha. Then all we need is the second PSU and GPU's and we are complete.

 

More pics to come soon.

Link to post
Share on other sites
EHW News Editor

Looking good E I am hoping to get mine running by the end of this weekend as well. Installing 5 GPU waterblocks is a PITA.

 

You want those fans for the Tesla's?

Edited by Avacado
Link to post
Share on other sites
Administrators
2 minutes ago, Avacado said:

Looking good E I am hoping to get mine running by the end of this weekend as well. Installing 5 GPU waterblocks is a PITA.

 

You want those fans for the Tesla's?

 

Yeah water blocks add in another factor that is for sure, I don't envy you. 

 

Sorry, I did see you say about fans, what did they look like again ?

Link to post
Share on other sites
EHW News Editor
2 minutes ago, ENTERPRISE said:

 

Yeah water blocks add in another factor that is for sure, I don't envy you. 

 

Sorry, I did see you say about fans, what did they look like again ?

 

20210122_211936.jpg

Link to post
Share on other sites
EHW Folding@Home
1 hour ago, ENTERPRISE said:

So I did a little cable management today after getting the lamptron fan controller installed.

 

IMG20210405183408.thumb.jpg.e0bce888cc63e22da82483d2a35d1a4a.jpgIMG20210405183356.thumb.jpg.7a2075e58ac33b6fef647d56c76379f7.jpgIMG20210405183417.thumb.jpg.148d711bb334bcfa35401bb1ebe32c93.jpg

 

Looks great. Does that Lampron have any software control? 

 

 

Link to post
Share on other sites
Administrators
3 hours ago, Avacado said:

 

20210122_211936.jpg

Oh nice. Will pm you about those.

2 hours ago, BWG said:

 

Looks great. Does that Lampron have any software control? 

 

 

 

No, hardware only which is how I wanted it. As I may switch to Linux sometime down the line, I wanted something OS independent.

Link to post
Share on other sites
EHW Folding@Home
6 minutes ago, ENTERPRISE said:

Oh nice. Will pm you about those.

 

No, hardware only which is how I wanted it. As I may switch to Linux sometime down the line, I wanted something OS independent.

 

Would be kinda cool to mount the controller internal, then have the knob external. Little hot glue probably to hold it. 

Link to post
Share on other sites
Administrators
1 minute ago, BWG said:

 

Would be kinda cool to mount the controller internal, then have the knob external. Little hot glue probably to hold it. 

It was a thought but lifting the lid to mess with the controller will not be too much bother. Plus I will set the fabs to their optimal speed for cooling and leave it.

Link to post
Share on other sites
Administrators

IMG20210409173258.thumb.jpg.5b51da5618cfe96b715bb976af358135.jpgIMG20210409173253.thumb.jpg.462eb68f2fd102c13ba362380fc634a3.jpg

 

Well we had a test the other day. System seems all well and good. I was testing @Sgt_Swanny new Gigabyte 6900XT to make sure all was well before taking it to him.

 

The CFP was a perfect test bed, plus I got to do a full system test...2 birds one stone and all that.

 

The Tesla M40's are officially on their way to me now. Purchased new thermal pads and Thermal Grizzly for a re-paste, will get onto that once there here. 

 

Also expecting some cooling fans for the M40's soon from @Avacado

Link to post
Share on other sites
EHW News Editor
57 minutes ago, ENTERPRISE said:

IMG20210409173258.thumb.jpg.5b51da5618cfe96b715bb976af358135.jpgIMG20210409173253.thumb.jpg.462eb68f2fd102c13ba362380fc634a3.jpgWell we had a test the other day. System seems all well and good. I was testing @Sgt_Swanny new Gigabyte 6900XT to make sure all was well before taking it to him.

 

The CFP was a perfect test bed, plus I got to do a full system test...2 birds one stone and all that.

 

The Tesla M40's are officially on their way to me now. Purchased new thermal pads and Thermal Grizzly for a re-paste, will get onto that once there here. 

 

Also expecting some cooling fans for the M40's soon from @Avacado

stewie.gif.41937ad519d0a9badc5b8d8df31f5487.gif

Link to post
Share on other sites
Administrators
4 hours ago, Bastiaan_NL said:

Not sure what you guys think, but I love the way that 6900XT looks! 

 

It is not bad, it is a solid card so far as the build and the backplate is nice. Pumps out some heat when folding, I can tell you that much lol.

Link to post
Share on other sites
EHW Content Creator
8 hours ago, ENTERPRISE said:

IMG20210409173258.thumb.jpg.5b51da5618cfe96b715bb976af358135.jpgIMG20210409173253.thumb.jpg.462eb68f2fd102c13ba362380fc634a3.jpgWell we had a test the other day. System seems all well and good. I was testing @Sgt_Swanny new Gigabyte 6900XT to make sure all was well before taking it to him.

 

The CFP was a perfect test bed, plus I got to do a full system test...2 birds one stone and all that.

 

The Tesla M40's are officially on their way to me now. Purchased new thermal pads and Thermal Grizzly for a re-paste, will get onto that once there here. 

 

Also expecting some cooling fans for the M40's soon from @Avacado

Little nervous with an AMD GPU, this will be my frist. AMD have always been the under dogs when it comes to GPUS. However this range has made Nvidia nervous so hopefully the 6900XT will do me well.

Link to post
Share on other sites
Administrators
1 hour ago, Sgt_Swanny said:

Little nervous with an AMD GPU, this will be my frist. AMD have always been the under dogs when it comes to GPUS. However this range has made Nvidia nervous so hopefully the 6900XT will do me well.

I think you will be happy bud. Makes fair more budgetary sense than Nvidia, especially with the scalping. Plus 3080 with 11GB VRAM is shameful, then the 3090 with 24GB is nuts and as we know the 3090 was Nvidia's panic attack at RDNA2.

Link to post
Share on other sites
Administrators
On 4/16/2021 at 4:04 PM, Avacado said:

@ENTERPRISE Can you confirm delivery of the Tesla Fans? I was so worried I effed up the address. 

 

It is why I sent a PM, like to inform people stuff has arrived safely :)

 

So I received my Tesla M40's along with 12 packs worth of new thermal pads.

 

I am pleased to report that initial testing on each M40 GPU has gone well, so after some new thermal paste, new thermal pads and replacement brackets, these will be good to go.

 

IMG20210417170138.thumb.jpg.da441024366072175de1d47d2308aca0.jpg

 

Annoyingly however my second PSU is still delayed. All high wattage PSU's above 1000Watts are gold dust over here right now. Unfortunately without the second PSU it holds the build process back for cable management reasons etc... not to mention having enough power.

 

So this will be on hold until then, though will re-work GPU'S in the meantime.

Link to post
Share on other sites
8 minutes ago, ENTERPRISE said:

 

It is why I sent a PM, like to inform people stuff has arrived safely :)

 

So I received my Tesla M40's along with 12 packs worth of new thermal pads.

 

I am pleased to report that initial testing on each M40 GPU has gone well, so after some new thermal paste, new thermal pads and replacement brackets, these will be good to go.

 

 

 

Annoyingly however my second PSU is still delayed. All high wattage PSU's above 1000Watts are gold dust over here right now. Unfortunately without the second PSU it holds the build process back for cable management reasons etc... not to mention having enough power.

 

So this will be on hold until then, though will re-work GPU'S in the meantime.

 

YHPM

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...

Important Information

This Website may place and access certain Cookies on your computer. ExtremeHW uses Cookies to improve your experience of using the Website and to improve our range of products and services. ExtremeHW has carefully chosen these Cookies and has taken steps to ensure that your privacy is protected and respected at all times. All Cookies used by this Website are used in accordance with current UK and EU Cookie Law. For more information please see our Privacy Policy