JS8Call-improved
Replies: 29 comments 17 replies
|
I did a test build with 6.11.1 and it builds and runs fine. There's a few deprecation warnings that would have to fixed in our code, but nothing major. @Joe-K0OG would have to try builds on linux and Windows to make sure there's no issues. And then make the decision, do we go with 6.11.3 when it becomes mature, or wait for 6.12? 6.12 will be the next LTS release. Although that doesn't make a lot of difference from our standpoint because we use the open source license and there is no commercial support anyway - we are our own support dept. So yes, definitely worth taking a look at. |
|
O.k., I'm installing Qt6.11.1 now to test. I'll let you know what happens. When I tried this with 6.10 a few months ago, it didn't go well. 73, P.S. Yes, Allan is disordered, but in a good way, ha! ;) |
|
Well, it may be a NO-GO on Linux. I can't even install Qt6.11.1 there. Even installing only the "Desktop" module without any other components, I get this error: So, as far as I'm concerned, we are stuck at Qt6.9.3. I can't go any further, unles this is some bug in Qt6.11.1 that can be fixed. I'm not going to buy a new machine just to keep up with Qt. 73, |
|
In Linux Mint 22.3, if I "Ignore" all the errors, and install the Desktop and Additional Libraries, then try to build with Qt6.11.1, it configures, but miserably fails at build due to my low-spec CPU: 73, |
|
On Windows, Qt6.11.1 installs fine (a higher-spec CPU than my Linux laptop), but attempting to build flags warnings even in the configure stage, then fails miserably at build with unending numbers of errors. No-go... 73, |
This means your processor is too old to support the AVX2 instruction set which Qt's meta object compiler binary was compiled to require. Typically, in the hardware world it seems architectures older than 10 years drop out of mainstream support. I always build my own Qt libraries instead of using the online installer and my current build collection for Apple silicon is 6.9.3, 6.10.3 and 6.11.1. I don't use the online installer because that ships with dual-architecture on Mac, which bloats an arm64 build of JS8Call (or anything else). The last version of Qt that will build on an Intel Mac is 6.8.3. And the oldest version of Qt that will build on an Apple silicon Mac is 6.9.1. Moving MacOS to Qt 6.11 has some definite advantages. We'll see what happens at WWDC coming up in a couple weeks. The Qt Group will be there. Apple is definitely dropping support for Intel with MacOS 27. It is likely Qt will do the same thing and support for Intel in their officially-built distributions will be dropped in Qt 6.12. Whatever Apple says, Qt does because Apple is the 900lb gorilla in the development world. iOS is the same as MacOS since they both use the same Apple silicon processors. Qt 6.11 will build fine on Windows using the clang compiler in Visual Studio 2026 - I just tried it here with arm64 Windows running on the M4 Mac and it builds fine. I suspect it will also build with the MSVC compiler. Linux is an unknown since they kinda tag along and don't have enough desktop marketshare for the Qt Group to worry about it. So they typically rely on linux developers to come with hacks to get the linux version to build, and that's usually a couple years behind the big players in the business. Android will get support before desktop linux does |
|
AVX2 was first supported by 4th gen Haswell processors, circa 2013. What vintage chip are you running, Joe? |
|
On my Linux laptop I'm running a Thinkpad x200, with an Intel Core2 Duo P8600 @ 2.4GHz. This is 2008 vintage. The laptop itself is around 15-years old. There is a LOT of this stuff around, and it still performs well. I would hate to see us lock things like this out of operation. 73, |
|
While we can't support old Intel Core processors forever, for the linux folks that run stuff that's older than the hills our pre-built Qt library can be built so it supports both SSE/SSE2 and AVX2. Just can't do it on an ancient processor. My Intel MacBook Pro with a Core i7 running Debian in a VM can build Qt 6.11 with both instruction sets without any issues. That Core i7 is slower than 85W-140 @ -40° compared to an M1 chip. But it eventually gets the job done if you got enough coffee handy and can put up with a laptop that sounds like a J57 at full afterburner. In theory I can cross-compile on the M4 Pro which can do a Qt library in under 6 minutes for Mac. But I'm not sure if cross-compiling on linux supports all the compiler flags for x86_64, or if arm64 linux even supports cross-compiling. That's one of the potential problems with theory. In theory there is no difference between theory and practice. In practice there is. At any rate, it's still not a show-stopper - we can come up with a pre-built library. |
|
My many-hours-long attempt to build Qt6.11.1 on Linux Mint 22.3 was a miserable failure. It ended up consuming 10s of GB of space until there was none left. So, unless someone provides me with a pre-compiled minimal-cpu-compatible binary, I can't test Qt6.11.1 on Linux. 73, |
|
It requires ~30GB of disk space and can require up to 50GB to build Qt, depending on what modules you build. Building it with a Core2 would be quite painful I would think and might take 2-3 days. However, you should be able to build it on a higher spec machine using the -DCMAKE_CXX_FLAGS="-march=core2" flag and get pre-built binaries that will work. However, building Qt on any sort of Intel processor is not exactly fast. And desktop linux is not really suited to build it either. The GUI itself will eventually crash so it's best to borrow some time from somebody that has a server that you can log in on with SSH, log into your build machine with SSH, or kill the GUI and X11/Wayland and drop to pure commandline before you start. Otherwise your build will fail anyway then the GUI goes down. Qt is a non-trivial monster and they've been moving their baseline up with each version, which is why it will no longer compile on an Intel Mac past 6.8.3. However, on Mac I can cross compile with the M4 Pro for Intel yet, which takes a little longer than a pure Apple silicon build. But with MacOS 27/Xcode 27 I suspect Apple is going to remove that capability because they want Intel Macs to fade into the annals of history, and rightly so. I was never happy with their move to x86_64 - it was a marketing ploy. I've got a PowerMac G5 that has WAAAY more power than the fastest Intel machine they ever made. It might not have the clock speed spec's, but the Power architecture has massive parallel thruput that x86_64 was never able to match and IBM still uses it in their high-end servers. |
|
So, some trivia here; I worked for IBM, primarily at the IBM Burlington facility in Essex Junction, VT. I left IBM during their divestiture of the storage division to Hitachi, as it was clear by that point that IBM was no longer interested in anything hardware related. Yes, the Power architecture, near and dear to my heart, ran rings around Intel; the problem was performance per watt; we couldn't deliver something with adequate performance, at the required power budget, for use in a laptop, in the timeframe that Steve required, and he therefore transitioned to Intel. In a strange case of history repeating itself, we just saw the same thing happen to Intel, for exactly the same reason. |
|
My PowerBook G4 Titanium - still one of the best laptops Apple ever built and still useful today. I still run MacOS X Leopard on it, dual-booted with PowerPC Debian. Yes, it's amazing how fast things change. Intel went from the 900lb gorilla to the verge of bankruptcy struggling to keep up, in a matter of about 7 years. Apple went back to RISC architecture, everybody goes we'll see how this works out. When it did, Qualcomm, MediaTek and Nvidia said we better do something about this before it's too late. Meanwhile the rest of the PC world is still running a 32-bit architecture with 64-bit extensions bolted onto it, which is not power efficient, not code efficient, not suited to today's mobile platforms. Buying a PC these days don't make any sense. It's running an architecture that is 23 years old and is old news in the computing world. Microsoft has made amazing progress in developing the tools and compilers needed for arm64 in the last year. The native arm64 clang compiler in Visual Studio 2026 is pretty impressive running in Win11 on an Apple silicon Mac, even though arm64 Windows is optimized for the Qualcomm Snapdragon. |
|
O.k., I made some progress with Qt6.11.1 -- with great effort! I have an old MacBook Pro which is half-shot, no SSD, dead battery, and flaky touchpad. However, the CPU and display still work, and since it's an Intel processor, I can run Linux and Windows off fast USB3 sticks to use it as a test machine. My Linux Mint 22.3 stick had enough room on it I thought I might be able to build Qt6.11.1 from source on it and get away with it. Sure enough, it worked! I built with modified configure files, adding "-D CMAKE_CXX_FLAGS='-march=core2'" flags everywhere I could find a cmake command (several places!) per @Chris-AC9KH excellent suggestion. It generated a few warnings but completed without error. I then tarballed the resulting Qt6.11.1 installation and copied it to my regular 15-year-old dual-core Linux Mint 22.3 laptop, extracted it, and built JS8Call 3.0.2, pointing to that Qt6.11.1 installation, and it worked! There were some deprecation warning that Qt6.11.1 didn't like (see below), but it built and runs fine. So, I tested notifications audio with this 3.0.2 built with Qt6.11.1, and guess what: No difference -- RATS! It still stacks a bunch of "stuck" notification streams when multiple-simultaneous signals arrive. The good news is that I'm now in business with Qt6.11.1. Here are the warnings I saw (you probably see the same ones in your builds, I'm guessing?). Some are related to audio, so might reveal a weakness we can patch? 73, |
The Qt Group is recommending to NOT build Qt with GCC anymore. They want you to use the LLVM-Clang compiler 
|
|
Encouraging. Yes, the point of the exercise here is the revised and much improved audio interface. I've been running it for about a month now in my SDR application, and it seems to be completely reliable, huge improvement. |
|
Here's the current state of my SDR test, showing the QRhi waterfall / spectrum displays, one per active tuner, and you can see the live transcription of the received audio, which is adequate, but not perfect, as I'm using the standard Apple SpeechTranscriber class here, and it has trouble with the EMS and Fire jargon. For portability, I can run it with a Whisper local model as well, which does a bit better job at the cost of more memory footprint. Uninteresting details, I know, but the fact you can do this sort of thing with a free local model now is great. The new callback interface for the audio is a huge improvement, and the QRhi stuff is very nice -- portable vertex shaders, basically, waterfall performance is amazing, runs entirely on the GPUs, very efficient, very performant. The audio stuff is going to be the real win here, but the waterfall and spectrum could be fantastic. When doing the Qt6 port, I'd looked at Qt's OpenGL stuff, and it reminded me of everything I don't like about OpenGL; it's like a time machine back to 1990, reflecting how video cards worked then. The QRhi stuff is entirely different, definitely worth the effort. 
|
|
It took a little over 8 minutes here to build Qt6.11.1 from source on arm64 Fedora running on a Mac mini M4 in the UTM virtual machine. Note that when you build this you have clone and compile FFmpeg 7.1.3 and use the appropriate flags to compile FFmpeg audio support into your Qt library. When you compile FFmpeg make sure to use the --enable-shared flag or it won't work. Using distro-supplied FFmpeg will NOT work to build Qt. |
|
I got a new build script I whipped up for linux that should build any version of Qt and package it for either x86_64 or arm64. It fetches and builds the proper version of FFmpeg first and deploys it with the Qt package. The script is in the releases on js8lib |
|
@Joe-K0OG if you are looking for something to play with I attempted a port of the audio class to the Qt 6.11 callback-based API. I don't know if it works on anything but Mac yet and there's no linux libraries built for it yet. So to build it you'll have to use Qt 6.11.1 from the Online Installer for now, or build your own Qt6.11.1 library by modifying the Qt6 build script in the source tree. https://github.com/Chris-AC9KH/JS8Call-improved/tree/pr/Qt6.11 |
|
I built your pr/Qt6.11 on Linux Mint 22.3 with the Qt6.11.1 I built last week, and it builds and runs fine, but still throws the same warning of "No QtMultimedia backends found" on Linux as before. As I think about the Qt6.11.1 I built from source for Linux, I did not include ffmpeg in it, so am rebuilding it now on my Ubuntu 24 server using your script (modified to build Qt6.11.1) which will include ffmpeg. When that finishes I'll try again on Linux with the newly-built Qt6.11.1. On Windows, it builds and runs fine. The debug output is this when I run it on Windows: 73, P.S. I forgot to say that on Windows I built it with QtCreator using the latest js8lib. |
|
Oh, that code won't build on Qt 6.9.3 or 6.10 anyway. So I wired up CMakeLists.txt to require 6.11.1 for building |
|
@Joe-K0OG if you try this note that I have to clean up some compiler spam yet. So don't worry about any of the warnings the compiler spits out for now. The main thing on this first pass was to get it to build and see if it works, which it does on MacOS. I'm not too worried about Windows 11. Linux could potentially be the outlier here simply because linux audio is an unmitigated mess and using gcc instead of LLVM-clang on Win/Mac. Since I switched the Windows build from MinGW/gcc to LLVM-clang I'm pretty sure we'll get consistent code quality on the two major platforms. |
|
OK, great. See what errors you get on linux and then we'll go from there. I coded that on the MacOS 27 developer beta with Xcode 27. I think the next step is to get the proper libraries in place for it and set up a functional workflow on my personal repo for easier code validation and testing. |
|
@Chris-AC9KH After properly rebuilding Qt6.11.1 on the Ubuntu 24 server, then installing it on my Linux Mint 22.3 box, JS8Call from your pr/Qt6.11 branch finishes without error, and runs as it should, the command-line output being: So, the build with ffmpeg fixed it! Attached is the build output on Linux Mint 22.3 from the above. 73, |
|
@Chris-AC9KH I seem to have a problem with your source, branch pr/Qt6.11, built on Linux Mint 22.3, with the freshly-minted Qt6.11.1 with ffmpeg. If I run the latest master from js8call-improved, all works well: If I send a HB or other generalized query so that many stations reply, the Time Delta is very close on all the stations (within a few 10s of ms). If I do the same with JS8Call built with your pr/Qt6.11 source, the times are all off by an additional approximately 1000ms, so some very good signals occasionally don't decode. This seems highly repeatable. Maybe something has somewhat broken in the pr/Qt6.11 timer? 73, |
|
@Joe-K0OG can only deal with one issue at a time. On this one: This is coming from QtMultimedia's WASAPI backend, not from the code directly. The m_stream->restart(this) call always creates a brand new QAudioSink, so the old one is being destructed and replaced every single transmission. That turnover, destroying and recreating a QAudioSink every TX, could be racing with WASAPI's internal stop timing under the callback API. Can you test this some more and find out, does this warning appear once at startup only, or every time you transmit? That will single out a one-time initialization race from a per-TX problem. |
|
@Chris-AC9KH I've been testing this afternoon, and it only appears once at startup. So, I did some testing side-by-side, in this screen shot is JS8Call built from current js8call-improved/master, and on the right built from your pr/Qt6.11, simultaneously decoding. This time, the times are close! Before, when I was testing pr/Qt6.11 by itself, I noticed that sometimes a station would be 1500ms off, then the next time would be maybe a few 10s, off, then next transmission back to 1500ms off. Something seems erratic in the timing for Time Delta. I don't know if it's really off, or if it's just a TD presentation error. It did seem like sometimes good strong signals w 73, |
|
A little followup to the previous: If I run pr/Qt6.11 side-by-side with master, the pr/Qt6.11 instance seems on-time. I just did that again, then immediately closed both instances, ran pr/Qt6.11 by itself, sent the same network query, and the times were on the order of THREE seconds off, so only barely got ONE decode out of the list shown above (same query to the same on-air group). I better quit before I really irritate all those stations with repeated SNR? queries! 73, P.S. As I observed before, after a few minutes, the timer seems to revert to correct times for on-air stations. I should mention that the Time Drift function is disabled, and my system clock is GPS synced, rock solid. |
|
@Joe-K0OG So at least it appears we have a one-time initialization race, not a per-Tx race. Likely cause is QtMultimedia probing/opening the default audio output device during its FFmpeg backend initialization before JS8Call has explicitly created its first QAudioSink. This would be a benign issue. Is this on Windows or LInux with the timer being off? That is strange because I didn't touch anything related to frame-time code. |
|
This is Linux. I haven't tested this functionality on Windows because I usually don't have that station on-air. I can try it on Windows too -- I'll get back to you on that. 73, |
|
OK, that's what I thought, which is not surprising on linux. Windows, however, I would expect this issue to not exist. It does not happen on MacOS. Linux, being not designed as a desktop system, with the GUI bolted on as an afterthought, is a totally decoupled client/server system running on sockets. So it has built-in GUI latency compared to Mac/Windows. The suspect on that one would be:
That is now in the Detector and it relies on the main thread's event loop actually firing in a timely fashion. Linux audio is also a server/client system running directly on the kernel, which has built-in latency compared to Mac CoreAudio/Windows WASAPI both of which have high-performance kernel bypass audio threading. We could have a similar issue here as the Linux TCP stack not being able to keep up with Flrig now that we've switched the audio backend to near-zero latency. We'll have to come back to this one, as I suspected Linux is going to have a problem with it. |
|
@Chris-AC9KH I'm sorry to report that your pr/Qt6.11 branch built on Windows with Qt6.11.1 on QtCreator exhibits the same timer strangeness as I observed on Linux, yet when built from js8call-improved/master, there is no timer problem. 73, |
|
@Chris-AC9KH I was able to capture an example of the difference between pr/Qt6.11 (left) and js8call-improved/master (right) on Windows, simultaneously decoding the same response to a group query (they're going to hate me and block me soon!): 73, |
|
Since this is consistent what I'm suspecting is that previously, with the synchronous QIODevice/readyRead path, readyRead fires almost immediately once the OS buffer has data, and onReadyRead ran inline on the same thread with no extra hop. Now with this new method we get OS-level audio buffering compounded with the new setBufferSize/callback-mode buffering, plus a QMetaObject::invokeMethod queued hop to the main thread. So what I tried is defer the buffer position timestamp until we get audio. We can give that a try but this new method may have some issues with JS8Call's internal frame timing. |
|
@Joe-K0OG so there's a new push on that branch that you can try. But I don't have high hopes that it will fix the problem. |
|
@Joe-K0OG I made a couple more changes
I also cleaned up some compiler spam that Qt 6.11 is spitting out. There's still a couple unused parameters in the code spitting out some spam but I'll fix that later. So you can try this and see if it fixes the time delta issue. I have higher hopes for this one. |
|
@Joe-K0OG forget this. The problem still exists. With QIODevice, the backend’s buffering model, plus Qt’s ring buffer, could deliver smaller, more frequent reads even if the total buffer size was large. With the callback API, you get buffers aligned to the backend’s period, which introduces latency we can't have in an app like JS8Call. I have to study the Qt documentation some more. Unless @bazineta has some ideas. |
|
So after studying the Qt docs I did this wrong. I got the audio thread doing one allocation plus one queued event allocation plus one mutex acquisition per callback, three things the Qt 6.11 realtime callback docs specifically warn against. Back to the drawing board as it were. |
|
Good Chris. I'm glad you were able to find the source of the problem, and that we identified it at this early stage. 73, |
|
I need to design a lock-free ring buffer. This might take me awhile. |
|
O.k., take your time. There is no rush on this since we are just prospecting the move to Qt11, and the current master works fine with Qt11. 73, |
|
I use a lock-free ring buffer in my application; it's not a huge lift. Let me know if I can help and if so, I'll try to extract a version of what I've got. You're on a compiler version that makes this easy, fortunately. |
|
@bazineta I think that would be helpful. My first mistake was not studying the docs thoroughly, and as a result I didn't implement the audio thread correctly. If I can get a working implementation Joe is happy with, then we can fine tune it. |
|
@Chris-AC9KH Sent you an MVP extraction of the lock-free ring buffer code in my SDR app, hopefully some useful insight there. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
https://doc.qt.io/qt-6/whatsnew611.html#:~:text=Qt%20Multimedia%20Module,allow%20real%2Dtime%20audio%20processing.
So I'm in the throes of converting
sdrtrunkto C++ / Qt, because apparently converting things written in languages that I hate to languages that I don't hate is some sort of highly-focused mental disorder. I am at peace with it.In 6.11, they've done this:
Introduced in Qt 6.11, the callback-based audio API in Qt Multimedia allows for low-latency, real-time audio processing by enabling direct data access in the audio thread, bypassing the need for a QIODevice. This API is available for QAudioSource (input) and QAudioSink (output).
Key Features
Direct Access: Passes a QSpan directly to a callable, allowing direct reading or writing of audio data.
Low Latency: Suitable for real-time audio applications by reducing buffering overhead.
Thread Safety: The callback runs on a soft-realtime audio thread.
Supported Platforms: Apple CoreAudio, Windows, Linux (PulseAudio/PipeWire), and Android.
Since I've got nothing to lose, and the Qt audio stuff has been brittle and annoying for me, figured I've give that new API a try, and -- night and day. Chalk and cheese. Like water in the desert.
No one should waste any more time with the old APIs. Stupidly better. All the weird hacks go away, and it's extremely performant and low-latency. Implemented the callback with a lock-free circular buffer and it was like falling off a log easy compared to what we had to do for JS8. Definitely should look at moving JS8 to it.
All reactions