HTTP API apps: Part 1

code screenshot

Way back near the turn of the century, Winamp was one of the coolest things around. One of the things that made it cool was plugins, and one of the coolest plugins was one that provided a Web interface. Not only did it let you control your Winamp remotely, but it also had decent interactivity, like a nice indicator of remaining track time, etc. To get the client-side job done, it used plain old HTML, CSS, and JavaScript rather than Flash. And this got me thinking: Why not use HTML, CSS, and JavaScript for the UI for apps in general?

Now I wasn’t the first or only person to think this by any means, but the idea of using Web technologies to build desktop apps didn’t really begin to gain much traction until fairly recently. Frameworks like NW.js and Electron, on which GitHub’s Atom and Microsoft’s Visual Code are built, do just that. But they’re not not quite the same thing as what I had in mind: implementing an app as a server that all sorts of different kinds of UI clients might interact with, including those built with Web front end technologies.

Here’s what I mean. Below is a diagram of what Electron and similar frameworks do:

Conventional hybrid app architecture

In this model, the user interface is rendered inside a dedicated environment provided by the framework using whatever HTML, CSS, and front-end JS frameworks you desire. The UI is tightly bound in a one-to-one relationship with the app engine. The app engine is implemented with a Web back-end language, typically Node.js, and it makes system calls through the engine’s baked-in features or, if the baked-in features don’t do what you need, through generic child_process.exec()-like calls that then invoke custom functions written on the host.

What I’ve got in mind looks like this:

Alternative hybrid app architecture

In this model, the tight UI↔app engine binding is replaced by a flexible REST API binding, and the app engine is simply a REST server embellished with whatever libraries are needed for accessing the required host resources.

The UI can be implemented with any kind of client that can speak the API, including HTML/CSS/JS clients, native mobile clients, terminal clients, etc. The REST server can be implemented in any language on which a REST server can be built. Thus the language the “app” is written in can be one that most easily supports the necessary system interactions or is otherwise best-suited to the app’s requirements. Also, unlike the conventional architecture, the client need not be local, which makes remote-controlled apps almost trivial to implement. As with any network-based communication, adequate measures must be taken to assure secure communication with the REST server.

So a few vacations ago, I set out to see whether this approach was actually workable. Because I have a high threshold of abuse, my test case wasn’t going to be some self-contained desktop app. Instead, I created a mini research project to study this architecture in the context of  real-time control of a physical appliance. (In reality, this was relevant to some other research I was doing. My tolerance of abuse had little to do with it.) The appliance I decided to prototype was an audio preamp with screen-based control.

This is where I ended up:

It’s not much to look at, but there’s a fair amount going on under the hood — which I’ll talk about in my next post.

But for now, that’s all my stories.

Programming Fundamentals with Processing

Kind of a big day here. I’ve decided to put online what I’ve written so far of my book on Processing. I’m pretty sure this will motivate me to do more work on it.

I’m about 80% done with the first half. I’m sort of thinking that once the first half is actually done, I might try a Kickstarter or GoFundMe.

Feel free to kick my butt about this.

Acrobotic’s ESP8266 Tips & Tricks

NodeMCU devkit

There’s a growing series of good videos covering ESP8266 Tips & Tricks on ACROBOTIC’s YouTube channel. The ESP8266 has become quite a darling in the IoT world, and a seriously cool community is growing around it.


NodeMCU devkit picture by Vowstar (Own work) [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons.

Alternative Architectures for Hybrid Applications

Hybrid applications seem to be gaining traction now. In what follows, I’d like to present thoughts on an alternative to the emerging standard hybrid app architecture.

The conventional hybrid architecture

Conventional hybrid app architecture

In this model, which to the best of my knowledge is used by Electron, NW.js, and others, the user interface is rendered as HTML using whatever HTML, CSS, and front-end JS frameworks you desire. The use of Web technologies for the UI is an explicit goal of this architecture.

The UI is tightly bound in a one-to-one relationship with the app engine.1 The app engine is implemented with a Web back-end technology, typically Node.js. The app engine makes system calls through the engine’s baked-in features or through generic child_process.exec()-like calls. This means custom and platform-specific behaviors that the app may require will need to be implemented as external child_process.exec() callable units.

This architecture does a good job of leveraging Web technologies to create secure conventional desktop apps. In addition, frameworks like Electron and NW.js have matured to the point that developing hybrid apps that use many desktop app conventions is relatively easy.

An alternative hybrid architecture

In what follows, I present what I believe is a more flexible approach to developing hybrid apps—one that requires more carefully considered design but that yields greater flexibility.

Alternative hybrid app architecture

In this model, the tightly bound user↔app engine connection is replaced by a REST API. Thus the app engine becomes a REST server, possibly embellished with some needed superpowers for accessing host resources. When the app interface is API driven, any REST client technology can be used for the interface, including HTML/CSS/JS clients, native mobile clients, terminal clients, etc. In addition, the client need not be local, making remote-controlled apps almost trivial to implement. Adequate measures must be taken to assure secure and authorized communication with the REST server.

The other change in the above model is that the REST server is implemented in C++. When this is the case, interacting with the host system can be done directly using a wide range of C++ libraries written for this purpose. The choice of C++ here is somewhat arbitrary; it can be any language that supports ready-to-roll support for the system manipulations that your app requires.

The two changes outlined above are decoupled—meaning that either can be adopted in the absence of the other.

One downside to using C++ (or Java, or Python…) for the server part of this approach is that the server must be able to run on the host platform. This isn’t a significant issue with desktop deployment: only recompiling the REST server for each target platform will be required. But it does currently present a problem for mobile deployment as few mobile platforms provide native support for C++ and its oft used libraries. If you plan to target mobile apps, implementing a REST server in a more universally supported language will likely be required.

Follow updates on my wiki.


1. I’m using “app engine” generically here, not as a reference to Google’s App Engine.

Concerns about Qt’s increasing commercial focus

Road closed sign

It’s not just me. Phoronix is reporting on growing concerns in the Qt community about the project’s increasing commercial focus and apparent deprecation of its open source core.

I have said many times in these pages and elsewhere that I really like Qt: it’s the right tool for a lot of jobs. But increasingly I’ve been feeling that any contribution I make in educating users about Qt is benefiting a commercial enterprise and not a community tool. So much so that I am re-surveying other open source multi-platform libraries to use as an alternative to Qt.

If the community no longer has meaningful Qt ownership, a lot of users will no longer be interested.

Arduino AsyncTimer library

asynctimercode-croped2

So there I was scratching an itch when I realized the scratch would make for a good Arduino library. AsyncTimer lets you create a timer that does something when you start it (or nothing if you prefer), then waits a predetermined time before doing something else. While it’s waiting, it doesn’t lock up your Arudino the way the delay() function does—it just schedules the time-out action to take place some time in the future.

If you’re not the RTFM type, you can just get what you need from the GitHub repository.

On the hidden dangers of highly integrated application frameworks

qupzilla-failure

I have been evaluating WebKit and Blink-based open-source browsers for Linux—mostly because Firefox is often noticeably laggy on an old laptop I like to use. (I still <3 you though, Moz!!) Chromium is an obvious option, but I would like to investigate solutions that don’t rely on Google. This has led me to Midori and QupZilla. Both position themselves as lightweight alternatives to the big kids. Both have their pros and cons.

A con with QupZilla that I want to take on in this post is the result of its use of Qt. I love Qt. It’s the right tool for a lot of jobs. But it harbors a hidden gotcha when it comes to rendering web pages: you are at the mercy of the Qt maintainers’ web rendering engine update policies. The latest version of QupZilla that you can build with the production version of Qt (5.5) uses QtWebEngine, which is based on the Blink-based Chromium. (So much for not relying on Google!)

The current Qt release bases QtWebEngine on Chromium 40—which is almost a year old; the next Qt release (5.6) will base QtWebEngine on Chromium 45—already three months old. The version of QupZilla provided by Debian sid (unstable, rolling release) uses the deprecated QtWebKit, which in Qt (5.5) is based on Webit 538.1.0. This version of WebKit is already a couple years old.

A similar situation exists with Midori. It uses GTK’s WebKitGTK+, which in Debian sid is as of this writing at 2.4.9-2 and also is about about two years old. Probing the user agent indicates it uses WebKit 538.1.5. But as there is in Qt, there is an additional wrinkle in the GTK+ world. There are currently two versions of WebKit for GTK: WebKitGTK+ and WebKit2GTK+ . Depending on which one your GTK+ application uses, you may get an old or a new version of WebKit. The version of WebKit2GTK+ that ships with Debian sid appears to provide WebKit 602.1 at the moment, which is the current point-release WebKit, and new releases of WebKitGTK+ flow regularly.  (Why Midori doesn’t use WebKit2GTK+ is a question for the developers and one that I hope to pursue. FYI, GNOME’s Web, née Epiphany, uses WebKit2GTK+.)

Whatever the specifics may be in this case, the takeaway is this:

When using integrated application frameworks, you need to be aware of versioning limitations with third-party tools that they bundle.

Open source audio remote control

audio-remote

I’ve started a FLOSS remote control receiver project for DIY audio preamplifiers. I think it’s just about good enough to make public.

Remote control is one of the more challenging things for an audio DIY person to implement, so I thought having an open source hardware and software platform for doing this would be useful. It uses our good friend Arduino for brains and works with the Philips RC-5 protocol. I like RC-5 because its the closest thing I know of to a universal, well-documented, brand- and model-agnostic protocol.

The IR command decoding is done using Guy Carpenter’s excellent RC5 library. I also considered using Ken Shirriff’s multi-protocol IR library. Ken’s library works with a large number of protocols, but I thought its larger memory footprint might preclude porting this thing to tiny AVRs.

Details on the project are available in my RC5-Preamp GitHub or GitLab repository.