I have for years now suspected that local files played back from a ZFS pool sound better than when played back from ext4, or similar file systems, and I think I know why.

I have been playing around with ZFS in various forms and on various platforms for a few years now and one thing that has always caught my attention is the fact that I felt the files played from ZFS always sounded better than files played back from other file systems. This didn't make sense to me so I mostly assumed I was hearing things and carried on. That feeling was always there though. Fast forward to present day and I am in a unique position where I have my dedicated machine for HQP playback gone for service and in the interim I have HQP Embedded on my desktop linux machine. This gave me an idea. I put an SSD in the machine, formatted it as EXT4, and copied music files to it. It lives alongside the 3 other SSDs that are in a RAIDZ1 ZFS pool that also hold my music library. Can you see where this is going? Same music files, same machine, same playback chain, file systems on the disks being the only difference. Full disclosure, if we really want to drill down to differences. The ZFS pool is made up of Crucial MX500 SSDs and the single EXT4 SSD is a Samsung 870 Evo. However, I have heard this difference whether ZFS was on all Samsung SSDs, spinning drives, or a mix. As long as they're local to the machine it doesn't matter what type or brand the drive is.

I think I know why the difference in sound quality, and it has to do with the nature of ZFS and how it works. When I press play on an album on JPLAY on my iPhone or iPad what happens, if I selected a local file for playback is this: JPLAY sends that information to HQP Embedded, HQP Embedded requests that file which happens to live on a ZFS pool. ZFS in the blink of an eye reads the file, checks that it's not corrupted by comparing it to parity, and then loads the file into the ARC. The  ARC (Adaptive Replacement Cache) is where all files that are in use from a Zpool live. If you want the TL:DR of it, here is the Explain Like I Am 5 version of it in pdf form: https://papers.freebsd.org/2019/FOSDEM/jude-eli5_zfs_caching.files/ELI5_ZFS_ARC.pdf The ARC is RAM, essentially half of the RAM in a system, unless set up differently, is used by ZFS as the ARC.This is where the ECC memory comes in handy as any changes to the file, whether made by you intentionally or a bit error, are then written back to the file on the drive. ECC memory will fix any single bit errors and help ensure your files are correct, aka: bit perfect.

Anyway, this is the why, in my opinion, of why I think the files played back from a local ZFS pool sound better than files played back from a more normal file system. The entire file is loaded into memory after being checked for errors, and possibly even corrected if any errors were found. Below are 2 screen captures. The first as you can see shows I am really bad at taking screen captures, however if you look in the top the 66.89MB number is the amount of data in the ARC when I start my computer. It is essentially empty except for whatever Minimserver has in there as it scans my music library. In the second image you can see that I a) took a better screen capture, and b) that the ARC current has risen to 144.56MB after I played an album and the first song was loaded into RAM.

This effect does not manifest if your files live on a ZFS system someplace on your network and are accessed by HQP via SMB share.

So, when I compare to the same files, played back via the same way, on the same machine, and the same data, the files from the EXT4 SSD just don't sound as dynamic, attack feels softened and they sound darker, softer, and thicker.

If you have the SSDs and Linux know how, you can easily test this yourself and let me know if you think I am crazy. Having the music files on a ZFS pool essentially means HQPE is playing the files from RAM due to the nature of how ZFS works, and that is why I think files played back from a ZFS pool sound better.

Caveat, before you try this there is one thing you should know. Trying to read music files on a ZFS pool with HQPE directly by trying to add them to the library will use up all your RAM instantly and crash your system. Minimserver however does not have this problem.

Hi AudioDoctor, that was interesting reading for me, although I don't use Linux and Embedded for HQPlayer playaback.

There are many ways how to preload a complete track to RAM. What I am thinking about:

• The difference between traditional disk cache and ARC. Both use dedicated areas of RAM. While the traditional disk cache is disk sector based, ARC is file based. So there is a difference. With traditional disk cache a big file need not to be cached at once (if cache is not empty or hasn't enough space for the whole file). I don't have enough information on disk caching on Linux, if ARC is the only disk cache used with ZFS, or if there is yet one level of OS cache below or above ARC.
   
• The difference between ARC and Freewheel, which also caches the whole track. How about your actual Freewheel setting? Does it influence sound when you change it?
   
• About 10 years ago when that was in discussion with Geoffrey Armstrong, Eurodriver and others, and most of us had music library stored on magnetic disks, I tried many RAM disk implementations and I found that they may have big influence on sound and that they much differ from each other in the way how they affect sound. So the next point is pre-loading complete album to be played to RAM disk versus caching a file by ARC.

IMO all the ways to cache a track to RAM bring some difference in resulting sound. Why one solution sounds better than the other? That's for sure not easy question even for people who implemented that caching code. Those people are usually not interested in high end audio. I would not expect ZFS developer to be a group of audiophiles. If one solution sounds better than the other one, it is usually a coincidence. What is probably not only a coincidence is, that for example HQPlayer with filters disabled, no convolution etc. sounds better with the same content than for example foobar2000. I would not be surprised if Miska is thinking about how to structure his code in order to minimize number of accesses to resources which result to power consumption spikes and to organize processing loops and buffer sizes with intention to generate less noise with negative audible effect.

Yest one question. It looks to me that your observation may be related only to directly connected DAC. Did you experience any advantage of ZFS and ARC when using a NAA?

8 hours ago, bogi said:

est one question. It looks to me that your observation may be related only to directly connected DAC. Did you experience any advantage of ZFS and ARC when using a NAA?

I always use an NAA. Maybe I wasn't clear but I was referring to the fact that the ZFS pool, aka, the music library, has to be local to the machine that also has HQP on it or the benefits of ZFS loading files into RAM have no effect on playback simply because if the library is on another machine, the ZFS ARC would be on that machine.

27 minutes ago, AudioDoctor said:

Maybe I wasn't clear but I was referring to the fact that the ZFS pool, aka, the music library, has to be local to the machine that also has HQP on it or the benefits of ZFS loading files into RAM have no effect on playback simply because if the library is on another machine, the ZFS ARC would be on that machine.

Yes, I understood that.

28 minutes ago, AudioDoctor said:

I always use an NAA.

That's the point which is interesting and rather surprising. Do you use wired Ethernet all the path to NAA or does WiFi appear in that path? Or does fiber LAN appear in the path? Few weeks ago you mentioned your new Intona. Was it connected between your NAA and DAC when you experienced the difference between filesystems?

Of course, we both know that the same data is transferred regardless if you play the same song from ZFS or EXT4 filesystem. So the source of perceived difference can be caused only by different noise patterns generated in that two cases, or maybe by different timing how audio data is processed and sent through LAN.

For me it is easy to get that computer generated noise has influence to perceived audio quality since it is my experience too. Just it is surprising that differences in noise patterns or in timing of data transfer can propagate themself through LAN and NAA device into DAC and result here to audible difference in analog domain.

Miska's long term opinion is, that as soon as HQPLayer computer and NAA are isolated galvanically (WiFi or fiber), computer noise cannot affect DAC. I opposed his standing few times, when we reacted to experience of users in some aspects similar to yours.

Yet one idea. Noise from HQPlayer computer could find its way to DAC device through power cables too since they also create galvanic connection between devices.

I have one NAA, a Sonore Signature Rendu SE that is connected with optical fiber and another, a Pi4 connected via regular ethernet. There is no wifi in the path.

edit: The freewheel setting in HQP has no effect on how ZFS acts when a local file is requested by HQP. Heck, I am not sure if it has any effect on HQP at all when a local file is played. Freewheel or no, ZFS loads the entire file into the ARC.

52 minutes ago, AudioDoctor said:

edit: The freewheel setting in HQP has no effect on how ZFS acts when a local file is requested by HQP. Heck, I am not sure if it has any effect on HQP at all when a local file is played. Freewheel or no, ZFS loads the entire file into the ARC.

You are right, that was my mistake. Local content is not pre-loaded, its done for content streamed from network (streaming services and UPnP). Later I found my 2 years old communication with Miska about source content caching performed by HQPlayer. It starts here: https://audiophilestyle.com/forums/topic/19715-hq-player/?do=findComment&comment=1177435

If you could power your computer from battery (UPS running on battery), that would break the metallic connection of power leads. Such a short test could answer the question if noise is spread to DAC through power cables.

On 2/17/2024 at 4:01 PM, bogi said:

 

You are right, that was my mistake. Local content is not pre-loaded, its done for content streamed from network (streaming services and UPnP). Later I found my 2 years old communication with Miska about source content caching performed by HQPlayer. It starts here: https://audiophilestyle.com/forums/topic/19715-hq-player/?do=findComment&comment=1177435

If you could power your computer from battery (UPS running on battery), that would break the metallic connection of power leads. Such a short test could answer the question if noise is spread to DAC through power cables.

The computer is connected to an APC Dual On Line conversion UPS. So, technically, it is on battery power 100% of the time.

I recall a discussion with Miska about ramdisks and player programs that explicitly say they load data to RAM before playback, where he said it shouldn't make a difference because all programs load their data into RAM before output.

With Audirvana or some other program that allows you to adjust the amount of RAM used for storage of data for playback, you could experiment with varying that setting and seeing whether it makes any difference to your impression of the sound.

1 hour ago, Jud said:

I recall a discussion with Miska about ramdisks and player programs that explicitly say they load data to RAM before playback, where he said it shouldn't make a difference because all programs load their data into RAM before output.

 

With Audirvana or some other program that allows you to adjust the amount of RAM used for storage of data for playback, you could experiment with varying that setting and seeing whether it makes any difference to your impression of the sound.

Shouldn't make a difference, but for some reason does, could be the title of a book about this forum and the many discussions around here. Just to clarify, I don't KNOW that the fact that ZFS loads everything into RAM is the why, it's just my guess based on what I can see happening and what I can hear. I can also see very minimal disk reads going on while playing a file that ZFS has loaded into the RAM, aka, the ARC.

If Audirvana were native to Linux, I would give it a try but sadly, it is not.

4 hours ago, AudioDoctor said:

The computer is connected to an APC Dual On Line conversion UPS. So, technically, it is on battery power 100% of the time.

My point was not only about battery but rather about disconnecting mains cable of your HQPlayer computer (together with monitor and everything that computer is galvanically connected to) from the rest of chain.

Is that UPS used only for HQPlayer computer or for more devices? Is your NAA and DAC on the same UPS? Can you physically disconnect that UPS from 120/230V mains and run your HQPlayer computer purely from its battery?

26 minutes ago, bogi said:

My point was not only about battery but rather about disconnecting mains cable of your HQPlayer computer (together with monitor and everything that computer is galvanically connected to) from the rest of chain.

 

Is that UPS used only for HQPlayer computer or for more devices? Is your NAA and DAC on the same UPS? Can you physically disconnect that UPS from 120/230V mains and run your HQPlayer computer purely from its battery?

Interesting discussion here. 

With regards to Hqplayer computer, potentially mains power noise getting to dac or to the rest of the chain......

That - was actually one of the reasons lately I have been testing my headless RPI5 running HQQ desktop and embedded since theoretically you can't get lower noise than that running HQP with a 27W PC powered by a low noise, low leakage medical grade power supply like a CUI that Miska suggested, still going through the same transformer isolation of ethernet to a NAA connected to an intona before it gets to the DAC. I even disabled the USB port, wifi and bluetooth and only used a SD card to boot Bookworm in order to run HQP...

While I could only managed to do PCM1536 or DSD128 7EC Super with a RPI5, my experience was that it really did not make any difference compared to using my 13600k 3080TI PC other than the fact I save on my energy bill using RPI5 to play music to my Cyan 2 10 hours every night.

I guess next thing I could try is to run the RPI5 with a battery pack and see if that even make any differences but I highly doubt it will.

37 minutes ago, bogi said:

My point was not only about battery but rather about disconnecting mains cable of your HQPlayer computer (together with monitor and everything that computer is galvanically connected to) from the rest of chain.

 

Is that UPS used only for HQPlayer computer or for more devices? Is your NAA and DAC on the same UPS? Can you physically disconnect that UPS from 120/230V mains and run your HQPlayer computer purely from its battery?

You can go ahead and do that all you want, I prefer to listen to music.

13 hours ago, AudioDoctor said:

You can go ahead and do that all you want

I have no such need, since I don't experience your issue.

13 hours ago, AudioDoctor said:

I prefer to listen to music

Why did you open this topic then?

5 hours ago, bogi said:

Why did you open this topic then?

To let others know about it.

ZFS has always sounded best to me because I just don't need to worry about it.

On 2/17/2024 at 2:50 AM, AudioDoctor said:

When I press play on an album on JPLAY on my iPhone or iPad what happens, if I selected a local file for playback is this: JPLAY sends that information to HQP Embedded, HQP Embedded requests that file which happens to live on a ZFS pool.

A bit late to the party but … you forget that, in order to use JPlay, you have Minimserver running and the communication between HQPlayer and your library is actually between HQPlayer and Minimserver, the two communicate through UPNP and network, even if Minimserver is local to the same HQP computer …