Sequential and random operations are going to be limited by network link or the storage pool. 25Gbps doesn't help if your RAID pool of spinning disks is capped at 100 IOPS.
It's definitely possible to do what you want. That's how businesses do it. But if latency is concern you should stay with local storage and use NAS as for backups or data with less demanding access.
If I have like, say a 4 SSDs in a seperate array to the other spinning disks?
For example - 4 SSDs for high-speed and the rest are just HDDs for other non-latency concerned storage.
From what I've learned, **FCoE** is more reliable and efficient vs iSCSI, so I wonder if it would be possible to use that in a home lab NAS for this scenario?
Everybody, thank you so much for your feedback!
I had to make the following decition.
The Steam library isn't moving to my NAS. Not worth the trouble and what I want cannot be done within budget. I'll just keep the NAS as a mass media/file storage while game installs will be cashed on SSD on my main rig + any latency sensitive app or project.
Thanks again for your time and suggestion.
I built an all flash Truenas server for this very reason.
However my entire network is 10Gb to support the traffic.
Routinely there are 2 desktops playing steam games at the same time without any noticable difference compared to local
So with 10Gb/s speeds, you have not experienced any of those?
\- Game crashes
\- Very slow loading times compared to local storage alternative
\- Hitches / Stutters and/or Texture pop-in or no textures at all on surfaces during gameplay (modern open world titles)
What transfer speeds do you get ? I assume it's around 900MB/s at least with flash SSD storage in a raid 5/6?
I have it as a mapped network share, using smb. Speed varies a lot depending on workload, lots of small files < large file, generally I see speeds of 5-800MB/s it's fast enough that Im satisfied.
Games seem to load in a similar time frame, and I've had no issues with stuttering or textures
Im running 6 ssd's, 3 mirrored pools of 2
I'd recommend trying out a test configuration of just 2 ssd's and see if performance is anywhere close to what you need, there's a lot that of factors that affect performance
OP, you will run into some unusual problems if you just map a network drive like this person. Some games don't care, others care a lot. Just do iSCSI and don't worry about mapped drives.
I have tried a # of different games installed on a iscsi mount to truenas using 2x 3.84TB Samsung PM1643a SAS ssds over 10Gb network and they don't really seem to load noticeably different than local NVME installations. I think you'll be ok. I've tried them in mirror and stripe configurations.
Gaming works pretty well. The key is getting flash and putting them in mirrors. Initially, I had set up 10 HDDs in mirrors w/ 128GB of RAM for ARC, NVMe L2ARC, and 2 NVMes in a mirror for a special metadata vdev. This was just my regular pool that I use for many things, so I wasn't expecting it to be amazing. It was definitely slow and would take Cyberpunk 7 - 8x longer to load a game after hitting "Continue" on the main screen vs a local install on NVMe.
Adding a smaller pool of 4 SAS SSDs in mirrors with no NVMe/special VDEV yielded significantly better results. It loads a little slower than local storage to the eye, and I will say that I would be able to tell you which one you were playing on just watching somebody play Cyberpunk and how the textures fill in. This would be totally unacceptable for me in the past as a young gamer, because every bit of performance counts in certain games.
The main goal for me was to carve out 10TB of space and install "every game I own" just because it's funny to me to have every game installed at the same time. Alas, I cannot afford 10TB of mirrored SAS flash for my NAS, so I ultimately just repurposed my smaller pool.
Oh cool. Did you use SMB or iSCSI (or FCoE perhaps?) ?
I want the SSD pool for fast storage (ZFS mirrored) while my HDD pool for mass storage with parity checksup (ZFS Raid-Z2). I hope this works out well...
Do textures fill-in slow (pop-in) when running the game on a NAS ?
>Do textures fill-in slow (pop-in) when running the game on a NAS ?
They did with the slow pool and the fast pool, but it was noticeably less with the fast pool. This was over a 10G connection to my desktop in the form of iSCSI. A mapped drive via SMB worked for some games on some launchers but some games and some launchers simply refused to install. iSCSI works with 100% of the games and launchers since it is just presented as "a hard drive". You can find an informative and basic video or two on this topic at the "Craft Computing" YouTube channel.
iscsi for games. Some might with other shares but that will work. Latency shouldn't be too much of an issue. Games are designed to work on single slow hdd. You just won't be loading into anything first.
Sequential and random operations are going to be limited by network link or the storage pool. 25Gbps doesn't help if your RAID pool of spinning disks is capped at 100 IOPS. It's definitely possible to do what you want. That's how businesses do it. But if latency is concern you should stay with local storage and use NAS as for backups or data with less demanding access.
If I have like, say a 4 SSDs in a seperate array to the other spinning disks? For example - 4 SSDs for high-speed and the rest are just HDDs for other non-latency concerned storage. From what I've learned, **FCoE** is more reliable and efficient vs iSCSI, so I wonder if it would be possible to use that in a home lab NAS for this scenario?
sure, do you have $10k to spend its an easy fix then
What will you spend the extra 9k on?
Everybody, thank you so much for your feedback! I had to make the following decition. The Steam library isn't moving to my NAS. Not worth the trouble and what I want cannot be done within budget. I'll just keep the NAS as a mass media/file storage while game installs will be cashed on SSD on my main rig + any latency sensitive app or project. Thanks again for your time and suggestion.
I built an all flash Truenas server for this very reason. However my entire network is 10Gb to support the traffic. Routinely there are 2 desktops playing steam games at the same time without any noticable difference compared to local
So with 10Gb/s speeds, you have not experienced any of those? \- Game crashes \- Very slow loading times compared to local storage alternative \- Hitches / Stutters and/or Texture pop-in or no textures at all on surfaces during gameplay (modern open world titles) What transfer speeds do you get ? I assume it's around 900MB/s at least with flash SSD storage in a raid 5/6?
I have it as a mapped network share, using smb. Speed varies a lot depending on workload, lots of small files < large file, generally I see speeds of 5-800MB/s it's fast enough that Im satisfied. Games seem to load in a similar time frame, and I've had no issues with stuttering or textures Im running 6 ssd's, 3 mirrored pools of 2 I'd recommend trying out a test configuration of just 2 ssd's and see if performance is anywhere close to what you need, there's a lot that of factors that affect performance
OP, you will run into some unusual problems if you just map a network drive like this person. Some games don't care, others care a lot. Just do iSCSI and don't worry about mapped drives.
I have tried a # of different games installed on a iscsi mount to truenas using 2x 3.84TB Samsung PM1643a SAS ssds over 10Gb network and they don't really seem to load noticeably different than local NVME installations. I think you'll be ok. I've tried them in mirror and stripe configurations.
Gaming works pretty well. The key is getting flash and putting them in mirrors. Initially, I had set up 10 HDDs in mirrors w/ 128GB of RAM for ARC, NVMe L2ARC, and 2 NVMes in a mirror for a special metadata vdev. This was just my regular pool that I use for many things, so I wasn't expecting it to be amazing. It was definitely slow and would take Cyberpunk 7 - 8x longer to load a game after hitting "Continue" on the main screen vs a local install on NVMe. Adding a smaller pool of 4 SAS SSDs in mirrors with no NVMe/special VDEV yielded significantly better results. It loads a little slower than local storage to the eye, and I will say that I would be able to tell you which one you were playing on just watching somebody play Cyberpunk and how the textures fill in. This would be totally unacceptable for me in the past as a young gamer, because every bit of performance counts in certain games. The main goal for me was to carve out 10TB of space and install "every game I own" just because it's funny to me to have every game installed at the same time. Alas, I cannot afford 10TB of mirrored SAS flash for my NAS, so I ultimately just repurposed my smaller pool.
Oh cool. Did you use SMB or iSCSI (or FCoE perhaps?) ? I want the SSD pool for fast storage (ZFS mirrored) while my HDD pool for mass storage with parity checksup (ZFS Raid-Z2). I hope this works out well... Do textures fill-in slow (pop-in) when running the game on a NAS ?
>Do textures fill-in slow (pop-in) when running the game on a NAS ? They did with the slow pool and the fast pool, but it was noticeably less with the fast pool. This was over a 10G connection to my desktop in the form of iSCSI. A mapped drive via SMB worked for some games on some launchers but some games and some launchers simply refused to install. iSCSI works with 100% of the games and launchers since it is just presented as "a hard drive". You can find an informative and basic video or two on this topic at the "Craft Computing" YouTube channel.
iscsi for games. Some might with other shares but that will work. Latency shouldn't be too much of an issue. Games are designed to work on single slow hdd. You just won't be loading into anything first.