'FeralA' decoder -- free-to-use

Here are some Linda Ronstadt 1977 snippets...  Sorry, must be snippets.  Note that the built-in mp3 decoder on dropbox does NOT do these justice.  In fact, these recordings are so intense, they seem to challenge MP3 just a little.

https://www.dropbox.com/sh/o2dx4goe5qgx6dh/AADTYGSw6VaSyHoARquyXgixa?dl=0

3 hours ago, John Dyson said:

Hmmm about the play...   Interesting because the da-avx provides a .wav file through the pipe, and it normally works (for me)...   So, I will definitely research it and try to figure out what I did wrong.  Most likely, I did make a mistake or assume something incorrect on Linux...

I am guessing that the version of sox on Linux that you are using might be happier with a full specification of the file type/data rate.  It is really easy for me to add the full specification to the 'secret' command line that I send sox :-).

 

I will definitely look into it, and check out what I might be doing wrong on Linux.   Thanks for the data point on Linux play though -- and I thought that was the 'easy' one :-).

 

However, thanks for the good news on Windows!!!

 

John

 

Hi @John Dyson,

I setup Fedora in a VMware VM and now the following command works fine:

     da-avx --input=testinfile.wav --info=2 --fa --fgh2 --tone=-12.94 --basic |  play -q -

Funny that it works in Fedora but not Ubuntu.

24 minutes ago, lucretius said:

 

Hi @John Dyson,

 

I setup Fedora in a VMware VM and now the following command works fine:

     da-avx --input=testinfile.wav --info=2 --fa --fgh2 --tone=-12.94 --basic |  play -q -

 

Funny that it works in Fedora but not Ubuntu.

Yea, I use an older version of Fedora...  I wonder if the versions of SOX match?  I use SOX V14.4.2. Maybe older versions don't sense the rate correctly?  Pipes are bit weird at times, as you cannot seek/rewind them.

I'll specify the entire file parameters when internally invoking 'play' or 'sox' on the new version of the decoder.   This should work-around that problem.

When I get the new, final Linda Ronstadt parameters -- I'll publish those also.   There is a lot of possibility of error in the parameters, and there are some ways to 'alias' different sets of parameters that sound similar.  Tricky stuff at times.   Requires lots of patience.

John

1 hour ago, John Dyson said:

I wonder if the versions of SOX match?  I use SOX V14.4.2.

I'm using SoX v14.4.2 in both Fedora and Ubuntu.

You know that the various versioins of Linux do patch their local versions....  I haven't done this, but I'll download the source of the specific Fedora version and then see if they patched SOX.   If they did, they might have fixed the detection problem.   Right now, I only have the source of the generic version of SOX as downloaded directly from the SOX site, so the diffs should explain what is going on.

BTW, the LINDA R stuff has been updated with a bit more controlled results.  That 'lisp' at the beginning of 'Its so easy' really gets on my nerves, and it is obviously an electronic artifact...  I tried to remedy it, but I think that the cure was worse than the disease.   Also, a new decode shifted the gain curve just a little bit to hide the lisp a little.

Everything that I demo in .mp3 snippets or full versions are the results of TESTING, and aren't really the main goal -- the decoder is the goal right now.  So, some day, other people who have better artistic ear can adjust the settings better than I can do!!!

John

53 minutes ago, John Dyson said:

You know that the various versioins of Linux do patch their local versions....  I haven't done this, but I'll download the source of the specific Fedora version and then see if they patched SOX.   If they did, they might have fixed the detection problem.   Right now, I only have the source of the generic version of SOX as downloaded directly from the SOX site, so the diffs should explain what is going on.

 

BTW, the LINDA R stuff has been updated with a bit more controlled results.  That 'lisp' at the beginning of 'Its so easy' really gets on my nerves, and it is obviously an electronic artifact...  I tried to remedy it, but I think that the cure was worse than the disease.   Also, a new decode shifted the gain curve just a little bit to hide the lisp a little.

 

Everything that I demo in .mp3 snippets or full versions are the results of TESTING, and aren't really the main goal -- the decoder is the goal right now.  So, some day, other people who have better artistic ear can adjust the settings better than I can do!!!

 

John

 

Loaded up Linux Mint into a VMware VM with Sox v14.4.2 and tried to do this cmd:

    da-avx --input=testinfile.wav --info=2 --fa --fgh2 --tone=-12.94 --basic |  play -q -

It failed but here's the output:

play WARN alsa: can't encode 0-bit Unknown or not applicable
Audio "DHNRDS DA decoder" V1.2.2A 26Mar2020 -- Author John S. Dyson -- Copyright 2017,2018,2019,2020
License terms are in accompanying documentation

Time Out:
Required CPU type: LINUX: X86-64bit with AVX2: i4xxx or greater
Affinity entirely disabled
Input file: "testinfile.wav"
Output file: "<stdout>"
play WARN wav: wave header missing extended part of fmt chunk
totaldelay: 814
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
S:          1, IN(RMS:-18.67dB,PK*:-13.93dB), INL(PK*:-13.93dB), INR(PK*:-14.43dB), OUT(RMS:-19.11dB,PK:-22.47dB):
    DA LL(-90.01/ -5.89/ -1.03), M(-90.01/ -2.85/ +0.00), H(-90.01/ -9.86/ -5.68), h(-95.01/-14.61/ -9.45)
    DA RL(-90.01/ -6.02/ -0.97), M(-90.01/ -3.05/ +0.00), H(-90.01/ -9.95/ -8.12), h(-95.01/-14.76/-11.00)
play WARN alsa: under-run

etc.

43 minutes ago, lucretius said:

 

Loaded up Linux Mint into a VMware VM with Sox v14.4.2 and tried to do this cmd:

    da-avx --input=testinfile.wav --info=2 --fa --fgh2 --tone=-12.94 --basic |  play -q -

 

It failed but here's the output:

 

play WARN alsa: can't encode 0-bit Unknown or not applicable
Audio "DHNRDS DA decoder" V1.2.2A 26Mar2020 -- Author John S. Dyson -- Copyright 2017,2018,2019,2020
License terms are in accompanying documentation

Time Out:
Required CPU type: LINUX: X86-64bit with AVX2: i4xxx or greater
Affinity entirely disabled
Input file: "testinfile.wav"
Output file: "<stdout>"
play WARN wav: wave header missing extended part of fmt chunk
totaldelay: 814
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
play WARN alsa: under-run
S:          1, IN(RMS:-18.67dB,PK*:-13.93dB), INL(PK*:-13.93dB), INR(PK*:-14.43dB), OUT(RMS:-19.11dB,PK:-22.47dB):
    DA LL(-90.01/ -5.89/ -1.03), M(-90.01/ -2.85/ +0.00), H(-90.01/ -9.86/ -5.68), h(-95.01/-14.61/ -9.45)
    DA RL(-90.01/ -6.02/ -0.97), M(-90.01/ -3.05/ +0.00), H(-90.01/ -9.95/ -8.12), h(-95.01/-14.76/-11.00)
play WARN alsa: under-run

 

etc.

The missing 'fmt' chunk happened because it wasn't in the broadcast spec as a requirement (AFAIR.)   I can add it in but is not a fatal or necessary part of the .wav file.

The under-run is basically about speed issues...   FIrst, the decoder needs at least two full separate cores to run in real time.  The decoder is NOT fast, but should be fast enough to run realtime with two cores scheduled correctly.  Try the --affinity switch to see if it speeds up.   By default, the decoder doesn't work as well real-time when hyperthreading because the threads grab up the entire CPU anyway.  SO, if you have 4 cores and 8 hyperthreads, it is very possible that the decoder is scheduled by the kernel, so that there is a clash so that they are all bunched up onto the same cores/threads instead of spread throuought the 4 cores.  (The decoder has to run on the cores, not the extra hyperthreads that it might also be running on.)

* The decoder is probably the most CPU intensive real-time SIMD DSP program that most people ever run (maybe except games which can sort of adapt to slower CPUS.)   The decoder DEMANDS multi-core CPU power, or it simply cannot run realtime.

So, if you have core0/thread0 core0/thread1 core1/thread0 core1/thread1 running, that is the worst of all worlds.

What you need is core0/thread0 core1/thread0 core2/thread0 core3/thread0, where the threads are scheduled amongst all of the threads, then the world is happier.  (On each core, it doesn't make any difference which thread is running, but that it runs on only one thread on each core.)

So, the --affinity switch tries to modify the scheduling so that it might run faster by pointing the CPU to multiple cores instead of allowing the bunch-up.

Also, try piping to the 'aplay' command, with no switches -- like:

da-avx blah blah blah | aplay

It might work better, I just thought of it, and just verified that it also worked for me (no error message about missing semi-optional fmt..)

John

Before I got back to my hibernation, thought I'd explain a recent improvement to the decoder -- correction of the stereo image.

The DHNRDS and a true DolbyA don't always work the same, especially in this recovery the stereo image because of the games played for the FeralA format.   Before, I was doing all of the correction in placeA or placeB, user selectable.  Those older modes will still be available in case I am wrong, and somebody in the future needs a fix, but I am long gone...

However, the new default mode for pop music uses a combination of placeA and placeB, where the results have bloomed.  Instead of getting that 'low distortion down to the ambiance' type sound, with the combination of the better correction of the stereo image PLUS the lower-midrange EQ, the results have markedly improved.

Because of recent changes, including this one, I am getting better results on the 'problem children' recordings -- Linda Ronstadt and ABBA (along with a few others.)  The new decoder isn't done with an 'artistic intent', but instead undoing technical damage to the recordings.  So, when I make changes, the goals are to uncover the master tape, not to make something sound 'prettier'.   Sometimes, the FeralA form DOES sound better in some ways, or maybe we are more accomodated to the sound.   I try to avoid using my 'taste' when doing the reverse engineering attempts...  My taste can distort the sound, so I am listening ONLY for distortion artifacts or agc artfiacts that need to be undone!!!

John

35 minutes ago, John Dyson said:

The missing 'fmt' chunk happened because it wasn't in the broadcast spec as a requirement (AFAIR.)   I can add it in but is not a fatal or necessary part of the .wav file.

 

The under-run is basically about speed issues...   FIrst, the decoder needs at least two full separate cores to run in real time.  The decoder is NOT fast, but should be fast enough to run realtime with two cores scheduled correctly.  Try the --affinity switch to see if it speeds up.   By default, the decoder doesn't work as well real-time when hyperthreading because the threads grab up the entire CPU anyway.  SO, if you have 4 cores and 8 hyperthreads, it is very possible that the decoder is scheduled by the kernel, so that there is a clash so that they are all bunched up onto the same cores/threads instead of spread throuought the 4 cores.  (The decoder has to run on the cores, not the extra hyperthreads that it might also be running on.)

 

* The decoder is probably the most CPU intensive real-time SIMD DSP program that most people ever run (maybe except games which can sort of adapt to slower CPUS.)   The decoder DEMANDS multi-core CPU power, or it simply cannot run realtime.

 

So, if you have core0/thread0 core0/thread1 core1/thread0 core1/thread1 running, that is the worst of all worlds.

What you need is core0/thread0 core1/thread0 core2/thread0 core3/thread0, where the threads are scheduled amongst all of the threads, then the world is happier.  (On each core, it doesn't make any difference which thread is running, but that it runs on only one thread on each core.)

 

So, the --affinity switch tries to modify the scheduling so that it might run faster by pointing the CPU to multiple cores instead of allowing the bunch-up.

 

Also, try piping to the 'aplay' command, with no switches -- like:

da-avx blah blah blah | aplay

It might work better, I just thought of it, and just verified that it also worked for me (no error message about missing semi-optional fmt..)

 

John

 

I didn't really notice a difference with the --affinity switch.  OTH, I increased the resources available to the Linux Mint VM and that seem to solve the problem.  I tried to do the same for the Ubuntu VM, but I still got the under-run's. (Next, I'll try 'aplay')

Hi @John Dyson,

Getting back to Windows 10, I tried to run the following:

    sox infile.flac --type=wav - | da-avx --fa=2 --tone=-12.90 | sox - outfile.flac

It fails and gives the following output:

    Audio "DHNRDS DA decoder" V1.2.2A 26Mar2020 -- Author John S. Dyson -- Copyright 2017,2018,2019,2020
    License terms are in accompanying documentation

    Time Out:
    sox FAIL formats: can't open input  `-': WAVE chunk fmt not found
    sox FAIL sox: `-' error writing output file: Broken pipe

It's probably just a syntax thing.

Similarly, I tried running the following:

    sox infile.flac --type=wav - | da-avx --outgain=-4.5 --info=2 --fa --basic | sox - outfile.flac

It fails and gives the following output:

    Audio "DHNRDS DA decoder" V1.2.2A 26Mar2020 -- Author John S. Dyson -- Copyright 2017,2018,2019,2020
    License terms are in accompanying documentation

    Time Out:
    Required CPU type: WIN64: x86-64bit with AVX2: (Haswell/Zen/Excavator) or greater
    Input file: "<stdin>"
    Output file: "<stdout>"
    totaldelay: 814

     ...

    Finished

    sox FAIL formats: can't open input  `-': WAVE chunk fmt not found

45 minutes ago, lucretius said:

Hi @John Dyson,

 

Getting back to Windows 10, I tried to run the following:

    sox infile.flac --type=wav - | da-avx --fa=2 --tone=-12.90 | sox - outfile.flac

 

It fails and gives the following output:

 

    Audio "DHNRDS DA decoder" V1.2.2A 26Mar2020 -- Author John S. Dyson -- Copyright 2017,2018,2019,2020
    License terms are in accompanying documentation

    Time Out:
    sox FAIL formats: can't open input  `-': WAVE chunk fmt not found
    sox FAIL sox: `-' error writing output file: Broken pipe

 

It's probably just a syntax thing.

 

Similarly, I tried running the following:

    sox infile.flac --type=wav - | da-avx --outgain=-4.5 --info=2 --fa --basic | sox - outfile.flac

 

 

It fails and gives the following output:

    Audio "DHNRDS DA decoder" V1.2.2A 26Mar2020 -- Author John S. Dyson -- Copyright 2017,2018,2019,2020
    License terms are in accompanying documentation

    Time Out:
    Required CPU type: WIN64: x86-64bit with AVX2: (Haswell/Zen/Excavator) or greater
    Input file: "<stdin>"
    Output file: "<stdout>"
    totaldelay: 814

     ...

    Finished

    sox FAIL formats: can't open input  `-': WAVE chunk fmt not found

 

 

Okay -- I'tl try to see what is going on tomorrow about the shorter command line.  There are a few glitches in the command line handling, and the program is acting ALMOST as if it didn't see the --fa= switch.  It will not run without the --fa switch being used, unless there is a license file.

I WILL resolve this issue tomorrow later in the day -- that is a basic STUPID BUG on my part.   Some of the decoder code is 'beautiful' when in comes to the DSP stuff.   The command line handling is knarly and ugly hackery from hell -- I don't like doing that kind of stuff.   The threading code is quite nice -- it is really useful when tasks need to be threaded because of CPU limitations.  The code also tracks every cycle of delay through the system, so the input and output files should match pretty well (not perfect, but pretty well.)

About the VM -- the decoder really needs 2 full cores, or it CAN NOT run realtime.   So -- I am wondering what is going on there.  I havent' recently run the decoder in a VM.   Earlier on, originally I ran it, with the same infrastructure, but a different program -- a multi-band compressor/expander -- and it worked fine, but NOTHING I have ever written before is as dependent on multiple cores.

Right now, in the highest quality (--xpp)  mode, each CPU intensive thread requires about just a little less than 1/2 Haswell core (about 40%) to run realtime, and there are 8 very intense threads with that requirement.   In the --basic mode, all of the active threads together require about 1.5 Haswell cores to run realtime.   There is a bit of a bypass when running in basic mode so that the 'unused' threads don't get the audio data, but they are all bypassed.

I am wondering if the VM, if you are allocating 2 'CPUS' to it, that it is allocating two hyper threads on the same core?   That would be unproductive and would work just like it only had one core.

If there are actually 2 cores available, there should be zero problem running realtime on an i4xxx or better.  So, I wonder about the VM, if it is allocating the CPU for best performance.

Hyperthreading is the devil for the decoder, and just might (will) run better without hyperthreading.  I have found very little that works much better with hypertheading, but it does feel good.  The newest Ryzen 3 might work better with hyperthreading because their CPUs have more resources, but Ryzen2 might even choke because of limited floating point resources.   I have to use hyperthreading to test because other people usually leave it enabled, and I am trying to avoid performance surprises like what you found.

It is tricky to 'divine' the core/thread architecture on a CPU, and the threads/core usage is dependent on the way that the OS decides to allocate them, I believe.

John

I have literally been beating my head against the wall trying to get a hyper-clean, technically correct ABBA decode for the last three years.  Of course, the decoder wasn't really capable until the last 6 months.

Someone on this forum, it was either 'Audiophile Neuroscience' or 'gmgraves' mentioned IEC vs NAB tape EQ...   I had played with that earlier, in fact the decoder has a mode --iec=yes, which will enable the compensation needed to correct from a recording done with IEC being played back with NAB recorders.

I couldn't find the correct combination of other EQ vs. the IEC, until now.  What happened is that the correction needed with IEC is a little trickier than most, and it always seemed like I needed an even more complex filter set -- but now, things are 'smooooth' sounding, with no sense of distortion down to the ambiance.

I was warned a long time ago -- no-one knows what kind of damage happens to these recordings during the process getting to the consumer.  We have been getting junk quality, especially since CDs.  Might 'sound good', but is no where near artists intent.

Good stuff is coming.  The next release might not come tomorrow, but very soon.  I have only been awake about 4-5Hrs/day, so things are still rough.

John

4 hours ago, John Dyson said:

I am wondering if the VM, if you are allocating 2 'CPUS' to it, that it is allocating two hyper threads on the same core?   That would be unproductive and would work just like it only had one core.

 

If there are actually 2 cores available, there should be zero problem running realtime on an i4xxx or better.  So, I wonder about the VM, if it is allocating the CPU for best performance.

I hope you are well!

My workstation has a single Intel Xeon W-2133 CPU. That has 6 (real) cores, 12 threads.  My host OS is Windows 10 Pro for Workstations (latest build).  I use VMware Workstation 15 Pro to run the VM's.  It allocates (virtual) "processors" based on the # of logical cores (in total) specified.  Since my CPU has 12 threads, it therefore has 12 logical cores max that can be allocated to any specific VM.  I currently have 6 of the 12 logical cores (i.e. threads) allocated for each linux VM (Ubuntu, Linux Mint, and Fedora). As well, I have allocated 16GB out of 32GB memory for each VM.

So far, Linux Mint and Fedora are running fine -- I have only tested the --basic switch with 'da-avx'.  OTH, Ubuntu ran into under-run's when trying to do real-time play piped from 'da-avx to SoX -- I didn't try to create a file yet.

Obviously, I still have much more testing to do, e.g. I need to create files using the --normal, --xtra, and --xpp switches.

8 hours ago, lucretius said:

I currently have 6 of the 12 logical cores (i.e. threads) allocated for each linux VM (Ubuntu, Linux Mint, and Fedora). As well, I have allocated 16GB out of 32GB memory for each VM.

I'll investigate further to determine if I can associate VMs with specific logical cores (CPU affinity).

1 hour ago, lucretius said:

 

I'll investigate further to determine if I can associate VMs with specific logical cores (CPU affinity).

For an idea of what to expect -- for the 'normal' mode deocding, that would be --basic and the --normal modes, the decoder should work better with 2 cores than 1.  However, more cores won't help much.   In the higher decode modes --xtra, --xp, --xpp, the decoder pulls out all stops, and starts running lots of threads -- 8 more.  Those additional threads are 'grinders', and your  6 full cores should easily be able to run significantly faster than realtime even in --xpp mode.  The problem is the darned hyperthreading -- it only helps on memory wait states, which don't happen much on the decoder...  The decoder is careful to do linear accesses and to try to stay within the higher level cache, so the memory wait states don't happen very often.   SO, since the decoder uses up ALL of the computational resources in the SIMD portion of each core -- the scheduling 'mistakes' made because of hyperthreading slow things down.   Thank goodness for affinity.

However, a 6core or more machine should run really nicely in the higher quality modes, but will DEFINITELY be slower than the --basic mode.   How much slower is totally dependent on the scheduling of the hyperthreads being suppressed and how many cores it has to work with.

An item of note -- the higher quality modes won't be a big surprise, but is very subtle.  Normal mode now does about 50% of what --xpp mode can do, iti IS an improvement over --basic mode, which will tend to be edgy, but also almost as fuzzy as a DolbyA.   --xpp mode comes very close to removing as much modulation distortion as possible -- there is a mode that does even more, but not really worth it '--xppp=12 --xf=7', but you'll be waiting an hour for each album.

John

3 hours ago, John Dyson said:

For an idea of what to expect -- for the 'normal' mode deocding, that would be --basic and the --normal modes, the decoder should work better with 2 cores than 1.  However, more cores won't help much.   In the higher decode modes --xtra, --xp, --xpp, the decoder pulls out all stops, and starts running lots of threads -- 8 more.  Those additional threads are 'grinders', and your  6 full cores should easily be able to run significantly faster than realtime even in --xpp mode.  The problem is the darned hyperthreading -- it only helps on memory wait states, which don't happen much on the decoder...  The decoder is careful to do linear accesses and to try to stay within the higher level cache, so the memory wait states don't happen very often.   SO, since the decoder uses up ALL of the computational resources in the SIMD portion of each core -- the scheduling 'mistakes' made because of hyperthreading slow things down.   Thank goodness for affinity.

 

However, a 6core or more machine should run really nicely in the higher quality modes, but will DEFINITELY be slower than the --basic mode.   How much slower is totally dependent on the scheduling of the hyperthreads being suppressed and how many cores it has to work with.

 

An item of note -- the higher quality modes won't be a big surprise, but is very subtle.  Normal mode now does about 50% of what --xpp mode can do, iti IS an improvement over --basic mode, which will tend to be edgy, but also almost as fuzzy as a DolbyA.   --xpp mode comes very close to removing as much modulation distortion as possible -- there is a mode that does even more, but not really worth it '--xppp=12 --xf=7', but you'll be waiting an hour for each album.

 

John

 

Thanks for this.

For 'da-dvx' running on a host OS (Windows 10 in my case), is there any chance of future versions of da-dvx taking advantage of GPU compute power (CUDA or OpenCL)?

1 hour ago, lucretius said:

 

Thanks for this.

 

For 'da-dvx' running on a host OS (Windows 10 in my case), is there any chance of future versions of da-dvx taking advantage of GPU compute power (CUDA or OpenCL)?

You know, I have really thought about it...  In fact, I almost pulled the trigger a year or so ago, except for two things:  1) the code wasn't stable enough to modify, 2) the code has now started using a lot more double precision floating point math.   Some GPUs, except the newest/highest ends don't do DP math as well as one might hope.

If anything is amenable to the 'graphics'/parallel CPUs, the DHNRDS DA and FA modes are.   I'd describe the math to you in full detail, but would be boring and imprecise (because of my suckky language skills....)

------

A simple way to describe the math -- PER SAMPLE:

4 + 3*4 *double precision* and fairly long Hilbert transforms (accurate FIR filters between 1500 -> 5000 taps long!!)  They MUST be accurate, all the way from about 5-10Hz (or less) to MAXFREQ!!!   The minimum frequency is NOT the lowest audio frequency, but is more related to the attack/release speed, which works out to about 200msec in the typical LF case.

Lots (perhaps 40, small 32-256 tap FIR filters)  (accuracy less critical, so SP)

about 3*16 (48) medium sized FIR filters (about 256 to 1024, depending)...  (Some of the longer ones are DP, but shorter ones are SP)

* There is some filtering that is done at 1/2 or 1/4 frequency, as would make sense, but the only time the filters run at different than the processing sample rate is for the measurement filters.  The signal path is very carefully maintained.

This is all PER SAMPLE in the high quality modes, and I can justify each one...   That is one hell of a lot of filters.  The longer filters, which must be longer to gain quality, also require higher precision math (kind of a doubling down the speed decrease.)

Also, the FA mode adds on some IIR filters, just second order, but perhaps 20 or so (actually, they are designed as 2nd order, but end up being mostly 1st order, with a few 2nd order>)

With all of the long filters, DP FP math is really needed.  The shorter filters can get by with SP math.   Actually, I do use DP math for the IIR filters, even though they are short, and could mostly get by with SP math.

* I found VERY audible degradatiion when using SP math on some of the Hilbert transform filters.  These Hilbert transforms are also not likely to be accurate enough if implemented by the normal IIR technique.

* Honestly, I tear the signal apart very aggressively, and is in pieces through the entire decoder when running in the high quality modes especially.  It is *all chopped up*, but is seamlessly put right back together.  Thank God for linear phase filters, but damned the delays :-).

The delays are all very well accounted for, but does cause the propagation through the decoder to be AT LEAST 1/3 seconds, but there is also a lot of buffering.   It is a big, meaty, bunch of processing, no-holds-barred, and as good as possible.  I went nuts on this.

* All of the delays through all of the various tap lengths of all of the filters in the same place in the data path for each frequency range (at the sub-band level, not just the 4 main bands) are all equalized to be at 0 phase error.

-------

So, I have thought about using a GPU, but a machine with 10+cores can really help, but avoiding the hyperthreading is critical.  Some day, I'll figure out a way to determine the computer core/thread architecture.  I can change the scheduling at any time, but I don't know the system config.  On Linux, I just guessed that everyone would be like my machine, so I made --affinity work for my machine, then being hopeful.

I'll be trying to get the release out for tomorrow afternoon (late), but want to be careful.  I'd rather delay by an extra day instead of wasting anyone else's time.

I'll consider ANY bugs brought out before then, and happily delay the release if it allows me to fix a troublesome bug.

John

This post is basic usage hints -- perhaps re-iterated in a different way than before, but I wanna make it as practical to use as possible.

Most decoding is done by a whole set of complex filters (explained with the --fd switch).  Most of the complex filters are pretty much the same for most recordings, so YOU can ignore them except in special cases.

The user is more focused on the main low pass filter as specified in the --fa switch (like --fa=2), and the calibration (--tone=-12.80 or somesuch.)  Forget the tone calibration for now...  The key is to get the correct approximate tonal balance and sound.  That is determined by which of the basic low pass filters that are  used:

--fa=0 (3kHz), --fa=1(4.5kHz), --fa=2(6kHz), --fa=3(9kHz) and --fa=4 (12kHz).

Sometimes a second layer of LPFs are needed -- ignore that for now, only a few recordings need that -- in my tests, ABBAs albums and some of Carpenters albums.

Here is a good choice pattern:

1) Try --fa=2

if it is too dull, increase the number to 3, then try again.

if it is too intense/toomuch HF, decrease the number to 1, try again.

Keep on doing that kind of pattern all the way down to 0 and all the way up to 4.

This is when the second filter comes in...  For example, it sounds good at --fa=1, but somehow too much very high HF (like 10kHz)... What is wrong?   Another LPF is needed.  In that case, start with the 12kHz filter (filter 4), then try 9kHz (filter 3) and even 6khz (filter 2).  How do you specify the additional filter:

--fgh#,

where #=4 for the 12kHz, 3 for the 9kHz, 2 for the 6kHz on down.

Sometimes, some recordings (e.g. ABBA) needs '--fa=1 --fgh3', which means both 4.5kHz and 9kHz (interesting 2:1 pattern, huh?)

In ONE case so far, ABBA, they need IEC conversion also, so I add an --iec=-yes switch not important for now.

-----------------------

So, this --fa=X, is used by the FA user, which starts the FA mode, and specifes the first low pass filter.

2/3 of the recordings can get by with this single setting!!!!!

-----------------------

After you have done 10's or 100's of decoding sessions, you might start noticing some of the finer points, and might want to diddle with the mass of the other filters, but if you need to do that as a beginner, just put that recording aside for now, and let me help you...   By far, most recordings can be done by the --fa=X switch alone, and most of the rest can be done by --fgh# in addition.

Just trying to help...  More to come!!!!

John

Being realistic, the new release wont get done today.  It is NOT a quality problem or anything like that -- it is unforseen things competing for my time.   A release, once I have a solid candidate takes about 1-2Hrs, and requires testing on a VERY slow laptop...   No biggie, it is just that if I must reneg on a 'promise', I try to forewarn as early as possible.  I am hoping for tomorrow, or maybe even still some time early in the morning -- not sure about the final tests along with a few nits that I hadn't fixed yet.

John

Bad news -- I suck at documentation.

Good news -- the next release of the FA decoder will need MUCH LESS detailed user info.  The decoding is now much more simplified, and the 'exceptions to the rule' are now disappearing.   I'll be moving the largest mass of the short users guides to an appendix.  Most not needed anymore!!!

----------

Basically, before, there was the 'FeralA' mode switch:  --fa=xx, and then the calibration to match the levels (like the Dolby tone), whicih is --tone=xx.xxx.   The problem was that there were LOTS of exceptions to the simple rules -- perhaps 1/3 of the recordings -- maybe more for 'tweaking in' most accurately.   There were (still are, but needed infiinitely less often) about 20 other tweaks to the decoding.

---------

Now, there ARE exceptions, but happen much less often.  The main switches -- '--fa=N' and '--tone=xx.xxx' are it for my highest quality decoding on most recordings.

The old exceptions like 'ABBA', and the old Carpenters, and even SUPERTRAMP are covered under the standard rules now, just the two switches.   Also, the --fa mode choices haven't increased -- still N=0,1,2,3,4 and 'classical'.  

Why is this working so easily now?   I could make claims about super secret adaptive programming, but that wouldn't be honest.   The real answer is that I found out what the differences are, and why there were differences in decoding.   Part of it was how the stereo image was mangled and also the other part was an error in my filter chain.   Also, the decoding is infinitely more robust -- I revisit sections of complex code every several months -- usually make improvements when I do so.

When using --fa=0,1,2,3 -- and sometimes tweaking the calibration to zero in the sound precisely, Everything from most of the Carpenters, the Analog productions Nat King Cole, all of my reference ABBA CDs, Olivia Newton John 48 Singles, Brasil'66, Tijuana Brass -- and even -- 'the cars' and some newer stuff...  Wow...  This is actually usable.

Those trying to use the decoder, someitmes getting 'okay' results, but recoginizing the need to tweak at times -- I HOPE that the desire to 'tweak' will be much less.

This is SUCH a relief, I just hope that this improvement wont' be setting expectations too high -- but honestly, I am finally happy with the basic operation of the FA mode now...   Before, I was happy with the quality that could be achived, but now I am happy with the usability.

We certainly need NO MORE sadness, frustration or disappointment right now, do we?

John

5 hours ago, John Dyson said:

New version of FA decoder is available.  I haven't prepared a Linux version yet, but will be ready tomorrow morning.  This new version has MAJOR MAJOR MAJOR quality and usability improvements.  It is almost EASY to use now.   The 'punishment' of  lower quality sound is actually less when the settings are incorrect now.

 

I think that a few things have been fixed -- I hope.   I tried to modify the internal SOX command line to make it work better -- but no guarantees.

 

Some of you know how difficult the first cut in 'Even in the Quietest Moments' has been for me to try to decode -- this version of the decoder can do it EASILY and fairly darned well.   The new version is V1.3.1B, and is in the same place as before -- the pointer is below.  Also, most of the old user docs are fairly accurate, but there is a new one:  using-V1.3X.txt.  I am including a portion of that file at the end of this post -- showing the relative consistency of the commands for each album to be decoded.

 

This is working REALLY well, if it actually works for you.  🙂 The results are actually VERY VERY pretty, and good results on tough material.

The attached example was decoded while using 3 versions of the same recording.  On the 'Give a Little Bit' example, the tonality is very similar to the other decodes (just a little fuller sounding), but the 'excessive' vocal enhancement is very precise -- showing the botched chorus effect very cleanly.   Mp3 does not show the 'botched' chorus effect nearly as cleanly as a raw decode -- there is actually a casually noticeable difference -- not worth splitting hairs on that right now. 

 

Along with the difficult snippet of 'Give a little bit', I am including a much higher quality decode of a selection from Herb Alpert.  I didn't want to only show a difficult decode -- maybe showing an underestimation of the potential quality of the decoder!!!  The 'Spanish Flea' example is actually typical, and not cherry picked.

 

Here is the repository:

https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0

 

Selections from the usage doc:

 

In many cases, on a non-normalized, raw, CD in FeralA form, a simple use directly on wav
file input and output like might be sufficient:

da-avx --input=inputfile.wav --output=outputfile.wav --fa

or

da-avx --input=inputfile.wav --output=outputfile.wav --fa=N
where N might be 0, 1, 2, 3 or 4.  Also, a 'classical' mode might be more approriate.

 

---

The --tone=-xx.xx values might seem to require 'precision', but really do not.  If the number

is within 0.02, there is almost never ANY difference in sound.  Also, the range of settnigs on non-normalized

CDs is now usually within about -13.25 to -13.40...   Even the max or min in this range still results in

typically 'good' results.

 

Carpenters:
1969    Ticket To Ride
        Basic switch: --fa

1970    Close To You
        Basic switch: --fa
        Note: the --mfm switch might be beneficial

1971    Carpenters
        Basic switch: --fa
        Note: the --mfm switch might be beneficial

1972    A Song For You
        Basic switches: --fa=4 --tone=-13.35

1973    Now and Then
        Basic switches: --fa=4 --tone=-13.30

1975    Horizon
        Basic switches: --fa=4 --tone=-13.30

1976    A Kind of Hush
        Basic switches: --fa=4 --tone=-13.30

1977    Passage
        Basic switches: --fa=3 --tone=-13.35 --mfm

1981    Made In America
        Basic switches: --fa=3 --tone=-13.35 --mfm

Linda Ronstadt:
Greatest Hits
        Basic switches: --fa=spc2 --tone=-13.35

1977    Simple Dreams
        Basic switches: --fa=spc2 --tone=-13.35

1975    Prisoner In Disguise
        Basic switches: --fa=spc2 --tone=-13.35

1989    Cry Like A Rainstorm
        Basic switches: --fa=spc2 --tone=-13.35

Nat King Cole
Nat King Cole Story/Analog Productions
        Basic switches: --fa --tone=-13.35

Carly Simon
The Very Best
        Basic switches: --fa

All 8 ABBA Albums
        Basic switches: --fa --tone=-13.35

Styx
The Grand Illusion
        Basic switches: --fa --tone=-13.35

Supertramp
Crime of the Century
        Basic switches: --fa --tone=-13.35
                * for less mellow, use --fa=spc1I
Breakfast In America
        Basic switches: --fa --tone=-13.35
Even In the Quetest Moments
        Basic switches: --fa --tone=-13.35On the 'Give a Little Bit' example, the tonality is very similar to the other decodes (just a little fuller sounding), but the 'excessive' vocal enhancement is very precise -- showing the botched chorus effect very cleanly.

The Cars
Complete Greatest Hits
        Basic switches: --fa --tone=-13.35 --mfm
                * --mfm seems important

Henry Mancini
Greatest Hits
        Basic switches: --fa --tone=-13.35

 

Bread
Bread (1973), Audio Fidelity  AFZ5 197
        Basic switches: --fa=spc1 --tone=-13.35 --mfm

Sergio Mendes, Brasil'66
The Very Best
        Basic switches: --fa --tone=-13.37 --mfm

 

Herb Alpert & Tijuana Brass
        Basic switches: --fa=spc2 --tone=-13.37 --mfm

 

 

 

 

GiveALittleBit.mp3 2.1 MB · 0 downloads 04.Spanish Flea-snippet.mp3 2.1 MB · 1 download

Thanks for this and thanks for simplifying.

@John Dyson

Tried v.1.3.1B (windows).  Everything seems OK so far but I am still having trouble with the SoX pipes --

I run this:

sox infile.flac --type=wav - | da-avx --fa=2 --tone=-12.90 | sox - outfile.flac

and I get this error:

sox FAIL formats: can't open input  `-': WAVE chunk fmt not found

The same command works fine in Linux Mint so maybe this is a Windows issue?

Cheers!

6 hours ago, John Dyson said:

Also, the range of settnigs on non-normalized

CDs is now usually within about -13.25 to -13.40

For previous versions, you said the --tone value will be between -12.80 and -12.95.  Why did this change?