Jump to content
We are undertaking some maintenance and improvements over the next few weeks. Apologies in advance for any temporary hiccups during this time. ×

Welcome to ExtremeHW

Welcome to ExtremeHW, like most online communities you must register to view or post in our community, but don't worry this is a simple free process that requires minimal information for you to signup. Be a part of ExtremeHW by signing in or creating an account.

  • Start new topics and reply to others
  • Subscribe to topics and forums to get email updates
  • Get your own profile page and make new friends
  • Send personal messages to other members.
  • Take advantage of site exclusive features.

Best RAID Card For NVMe ?


Recommended Posts

Hey Chaps, 

 

So I wondered if any of you had any recommendations as to a good RAID card for NVMe drives. I did a little searching, most of the add-in cards for NVMe support RAID, but do not setup the RAID independently from the motherboard etc, which ultimately in my mind is better should you have any motherboard issues, at least your array is not tits up ! I did take a look and found this boy: HighPoint M.2 NVMe RAID Controller via PCI-Express 4.0 x16, but while I am sure it is awesome and certainly would fulfill my needs....it is whack expensive lol. 

 

Any ideas ? 

 

Thanks,

E

Link to post
Share on other sites
Premium Platinum

Software RAID, or use AMD or Intel's built in tools. No motherboard is capable of NVMe RAID, they just enable you to use RST/VROC or AMD's RAID tools. Even the Highpoint drive is just a PLX chip and software;

 

https://www.anandtech.com/show/16247/highpoint-updates-nvme-raid-cards-for-pcie-40-up-to-8-m2-ssds

Quote

HighPoint's NVMe RAID is implemented as software RAID bundled with adapter cards featuring Broadcom/PLX PCIe switches. HighPoint provides RAID drivers and management utilities for Windows, macOS and Linux. Competing software NVMe RAID solutions like Intel RST or VROC achieve boot support by bundling a UEFI driver in with the rest of the motherboard's firmware.

 

NVMe drives are already RAID cards. They have LPDDR4 Cache, they have multi-core controllers, and they have stacks of Flash NAND (SSDs) on data buses. They can already push >4GB/s. There is no bus that will give you a performance benefit over PCI-e 4.0 that you can use on modern desktop hardware, and adding another step in the chain will hurt latency. There is simply no controller short of an actual FPGA, ASIC, or true CPU that can keep up.

 

What is your end goal? If you are attempting to protect data, then get a automated networked backup solution. Unless you absolutely need higher sequential read/writes, you will see no benefit from RAID, better to just pony up for next gen Optane or whatever the fastest PCI-e 4.0 NAND SSD is at the time.

Edited by KyadCK
Link to post
Share on other sites
10 hours ago, KyadCK said:

Software RAID, or use AMD or Intel's built in tools. No motherboard is capable of NVMe RAID, they just enable you to use RST/VROC or AMD's RAID tools. Even the Highpoint drive is just a PLX chip and software;

 

https://www.anandtech.com/show/16247/highpoint-updates-nvme-raid-cards-for-pcie-40-up-to-8-m2-ssds

 

NVMe drives are already RAID cards. They have LPDDR4 Cache, they have multi-core controllers, and they have stacks of Flash NAND (SSDs) on data buses. They can already push >4GB/s. There is no bus that will give you a performance benefit over PCI-e 4.0 that you can use on modern desktop hardware, and adding another step in the chain will hurt latency. There is simply no controller short of an actual FPGA, ASIC, or true CPU that can keep up.

 

What is your end goal? If you are attempting to protect data, then get a automated networked backup solution. Unless you absolutely need higher sequential read/writes, you will see no benefit from RAID, better to just pony up for next gen Optane or whatever the fastest PCI-e 4.0 NAND SSD is at the time.

Hey bud,

 

Thanks for the response and info. The purpose of this is not so much day to day gains or for backup reasons. This is more of a study project/review for balls to the wall speed in RAID 0. I understand the use cases are niche.

 

I would need to do some research on the add in cards benefit I mentioned over a standard expansion add in card.

Link to post
Share on other sites
Premium Platinum

It's not going to go as cleanly as you might expect, but you are obviously welcome to try. You're probably going to want to reach out to Level1Techs about it if you plan to go over a few drives.

 

https://forum.level1techs.com/t/fixing-slow-nvme-raid-performance-on-epyc/151909


 

Spoiler

 

 

 

 

 

There is no benefit to add in cards because they do not perform any function. It is that simple, there are no RAID cards for NVMe, only software, and at these speeds the file systems themselves break if the CPU doesn't first.

 

What a time to be alive eh?

Edited by KyadCK
Link to post
Share on other sites
3 hours ago, KyadCK said:

It's not going to go as cleanly as you might expect, but you are obviously welcome to try. You're probably going to want to reach out to Level1Techs about it if you plan to go over a few drives.

 

https://forum.level1techs.com/t/fixing-slow-nvme-raid-performance-on-epyc/151909


 

  Reveal hidden contents

 

 

 

 

 

There is no benefit to add in cards because they do not perform any function. It is that simple, there are no RAID cards for NVMe, only software, and at these speeds the file systems themselves break if the CPU doesn't first.

 

What a time to be alive eh?

Thanks for the great info then. In that case will stick with onboard slots for testing (Saves cash), though an add in card may make life easier in the future as a future WC GPU will block one of my onboard m.2 slots making it a pain to swap out.

 

Not having played with NVMe raid, Is it yet possible to create a bootable array ? So you can have windows boot from it ? 

 

 

Link to post
Share on other sites
Premium Platinum
1 hour ago, KyadCK said:

 

Ahh well that is simple then. Have had RAID arrays before and seems no different now.

Link to post
Share on other sites
On 12/30/2020 at 2:50 AM, ENTERPRISE said:

Hey bud,

 

Thanks for the response and info. The purpose of this is not so much day to day gains or for backup reasons. This is more of a study project/review for balls to the wall speed in RAID 0. I understand the use cases are niche.

 

I would need to do some research on the add in cards benefit I mentioned over a standard expansion add in card.

  

We priced out an alternate solution at the office the other day re. fast backups: U.2 SSDs ..as motherboard vendors rarely offer those onboard, there is a little M.2 > U.2 adapter (older MSI video below). The question is whether that costs U.2 speed, and also whether it is able to work with PCIe 4.0...

 

Intel's new PCIe 4.0 U.2 Optane drives are insane performers, and unfortunately priced accordingly, but there some good used PCIe 3.0 Optane drives to be had...more on those > here

 

spacer.png

 

 

Link to post
Share on other sites
28 minutes ago, J7SC_Orion said:

  

We priced out an alternate solution at the office the other day re. fast backups: U.2 SSDs ..as motherboard vendors rarely offer those onboard, there is a little M.2 > U.2 adapter (older MSI video below). The question is whether that costs U.2 speed, and also whether it is able to work with PCIe 4.0...

 

Intel's new PCIe 4.0 U.2 Optane drives are insane performers, and unfortunately priced accordingly, but there some good used PCIe 3.0 Optane drives to be had...more on those > here

 

spacer.png

 

 

Optane would of course be an option for RAW speed but my project really only calls for NVMe in RAID. However will look into those U.2 Adapters just out of interest.

 

 

Link to post
Share on other sites
Premium Gold

@ENTERPRISE  @Laithan ...a bit off-topic, though this would hold a heck of a lot of nvme raid cards, and apparently also has 2x U.2 onboard...

 

From the 'pressing-your-nose-on-the-Porsche-showroom-window' file:  Gigabyte's upcoming TRX80 Threadripper Pro board...up to 2 TB 8-channel RAM (!), 7x PCIe 4 x 16, dual 10 GbE Lan - and it's black/silver shiny ! 

 

source

 

spacer.png

Edited by J7SC_Orion
Link to post
Share on other sites
Administrators
14 minutes ago, J7SC_Orion said:

@ENTERPRISE  @Laithan ...a bit off-topic, though this would hold a heck of a lot of nvme raid cards, and apparently also has 2x U.2 onboard...

 

From the 'pressing-your-nose-on-the-Porsche-showroom-window' file:  Gigabyte's upcoming TRX80 Threadripper Pro board...up to 2 TB 8-channel RAM (!), 8x PCIe 4 x 16, dual 10 GbE Lan - and it's black/silver shiny ! 

 

source

 

spacer.png

Ha, That is one nice looking board. If I were on that platform, that would certainly be a choice !

Link to post
Share on other sites
  • 4 weeks later...
On 1/1/2021 at 2:12 PM, J7SC_Orion said:

@ENTERPRISE  @Laithan ...a bit off-topic, though this would hold a heck of a lot of nvme raid cards, and apparently also has 2x U.2 onboard...

 

From the 'pressing-your-nose-on-the-Porsche-showroom-window' file:  Gigabyte's upcoming TRX80 Threadripper Pro board...up to 2 TB 8-channel RAM (!), 7x PCIe 4 x 16, dual 10 GbE Lan - and it's black/silver shiny ! 

 

source

 

spacer.png

 

Now that's my kind of motherboard!!

 

With regard to NVMe RAID, I think the current answers are only true... for now...

 

Like they say "If you build it, they will come"... I think that it is only a matter of time but there are a couple of barriers that only time can really address.

 

For one, I think with desktop board the # of PCI-e lanes is extremely limited (for NVMe) creating a bottleneck fairly quickly. PCI-e 4.0 can help with this however it might not be until NVMe is supported on PCI-e 5.0 where this can really shine.

 

The other issue is the "controller".. since NVMe interfaces directly with PCI-e natively, a "controller" isn't really what is needed in a traditional sense. NVMe RAID would essentially have to be an add-in PCI-e device or emulator, in hardware that gives each NVMe dedicate PCI-e bandwidth and presents those drives to the PCI-e interface on the motherboard and "seen" as a single drive. I think this is something we will in the next few years possibly but we are just not there yet IMO... but I could be totally wrong, we shall have to wait and see.

Link to post
Share on other sites
Premium Gold

..one of those new WRX80 TR Pro workstation monsters (which come with 2x U.2) would be nice with 2x Intel's new P5800X Optane (PCIe 4.0) in RAID  ...one probably could even add M.2 to U.2 adapters and then two more P5800X before ever even using one of the PCIe 4.0 x 16 slots .

 

Sounds like a great plan ...until one hits the checkout / cashier 😰

  

IntelP5800X.thumb.jpg.32b6d2bf1313db3f5132015edafd6acb.jpg

 

Edited by J7SC_Orion
Link to post
Share on other sites
Administrators

FYI, I ended up buying the HighPoint M.2 NVMe RAID Controller via PCI-Express 4.0 x16, not so much for playing with RAID (not that it is needed to do RAID) but because it also allows me to swap out NVMe drives more easily than compared to when they are on my motherboard. Will also be reviewing this add in card so keep your eyes peeled for that :)

Link to post
Share on other sites
2 hours ago, ENTERPRISE said:

FYI, I ended up buying the HighPoint M.2 NVMe RAID Controller via PCI-Express 4.0 x16, not so much for playing with RAID (not that it is needed to do RAID) but because it also allows me to swap out NVMe drives more easily than compared to when they are on my motherboard. Will also be reviewing this add in card so keep your eyes peeled for that :)

Are you able to set that up in RAID 0? Did you have to use Windows DM for the RAID? What kind of performance are you getting with ATTO? I guess I should wait for the full review 😛

 

Edited by Laithan
Link to post
Share on other sites
Administrators
4 hours ago, Laithan said:

Are you able to set that up in RAID 0? Did you have to use Windows DM for the RAID? What kind of performance are you getting with ATTO? I guess I should wait for the full review 😛

 

Haha. Yeah these will be in the review. I have not got the card in hand yet. Should be arriving soon. Then I need to get a pair of NVMe drives for RAID 0 Testing.

 

Link to post
Share on other sites

I had heard of these a while back but forgot about them.. The Linus just posted a new video featuring the. The "HoneyBadger". Athough it doesn't use the term RAID, this appears to be a harware based solution that combines multiple NVMe drives together which would essentially be similar in functionality to RAID.

 

This thing is extremely impressive... 24GB/s 

https://www.liqid.com/products/composable-storage/element-lqd4500-pcie-aic-ssd

Edited by Laithan
Link to post
Share on other sites
Administrators
2 hours ago, Laithan said:

I had heard of these a while back but forgot about them.. The Linus just posted a new video featuring the. The "HoneyBadger". Athough it doesn't use the term RAID, this appears to be a harware based solution that combines multiple NVMe drives together which would essentially be similar in functionality to RAID.

 

This thing is extremely impressive... 24GB/s 

https://www.liqid.com/products/composable-storage/element-lqd4500-pcie-aic-ssd

That seems like a fairly similar device to the one I am getting to be honest, will be interesting. Either way they are impressive solutions for maximum throughput, much of which the majority of users would never need, or notice when doing your big standard tasks, aside from in system file transfers however, those will be super hot.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

This Website may place and access certain Cookies on your computer. ExtremeHW uses Cookies to improve your experience of using the Website and to improve our range of products and services. ExtremeHW has carefully chosen these Cookies and has taken steps to ensure that your privacy is protected and respected at all times. All Cookies used by this Website are used in accordance with current UK and EU Cookie Law. For more information please see our Privacy Policy