From Firefox 52 to 60: Dealing with the Deprecation of SDK/XUL Extensions

Introduction

Ignoring all the different variations and forks (simplifying the matter a lot), three different rendering engines for desktop web browsers exist. Representative for the different engines are Google Chrome, Internet Explorer and Firefox.

On Linux, Internet Explorer is not available and thus there are two major choices: Google Chrome and Firefox.

Here, Firefox was first used in version 2.X (at that time still on Windows) when Chrome was not yet available (Konqueror with the KHTML rendering engine was). Since it worked well at the time, Firefox was kept and upgraded to new versions whenever needed.

As Google Chrome (and the open source Chromium) emerged, the were also tested but two major properties of Firefox dismissed them as alterantives:

Firefox Extensions

Very early, Firefox supported powerful extensions already and a lot of useful extensions were found over time which could not adequately be replaced by Chromium-compatible ones.

Browser Behaviour

Different rendering engines have different behaviour. Two major things which were found better in Firefox were the handling of selections and the compatibility with internally-developed local sites (Chromium forbids some of the internally used JavaScript for security reasons, Firefox does not). As a result, for a very long time Firefox was the browser used productively here.

It had long been announced that new Firefox versions would not support “legacy” extensions anymore [1], yet until it was finally required to switch due to the end of security updates [2], no larger attempt to replace existing legacy extensions had been conducted because it had always seemed infeasible to find adequate replacements for all extensions.

Useful Extensions

Why would one need extensions in the first place? Generally, there are multiple categories of issues addressed by the extensions which are described in the following.

Extensions installed in Firefox 52

Web Development

With modern browsers’ integrated web development tools, the need for specialized extensions has certainly decreased. Still, the following extensions were found to useful in addition to the browser-integrated tools:

Web Developer

Previously known as Web Developer Toolbar, this extension provides an additional menu for quickly performing tasks like resizing the browser window or disabling certain features. It can even be used to radically disable all stylesheets or images to check if pages really degrade gracefully. Some of the functionality is also available through browser-supplied tools, still Web Developer allows accessing them more quickly.

User Agent Switcher

As one would expect, this allows arbitrary user agents to be sent which is useful to test how pages behave differently for different browsers without needing the actual browser.

HttpFox

With this extension, one could open a window similar to Wireshark but within the browser and thus more quickly and unaffected by company policies prohibiting the use of Wireshark for security reasons. Of course, being specialized to the browser’s HTTP-requests, this extension’s interface was easier to use than Wireshark’s and thus also often more quickly to navigate.

Opening the Ma_Sys.ma website with HttpFox shows the resulting browser requests.

User Interface Adjustments

For integration with MDVL user interface concepts, two addons were used to enhance the web browser.

Classic Theme Restorer

Mainly used to restore what one was already used to from previous Firefox versions but also disabling unnecessary animations, this extension was not strictly needed but had been successfully used to avoid unexpected user interface changes for some time.

The Classic Theme Restorer’s preferences panel provided fine-tuning for just about any GUI quirk changed between Firefox versions.

VimFx

This enhanced Firefox with vim-like modes and keybindings and therefore unified the interaction with MDVL to some extent: browser, text editor and window manager could all be controlled with the same [H], [J], [K], [L] keys for navigation.

With VimFx being a keyboard-driven extension, there is little GUI to show here. Here, the description of the extension’s functionality with the button and major advantage compared to other similar extensions is highlighted by red borders.

Dealing with bad Websites

Today’s Internet is full of advertising, tracking and malware. Most of it comes in form of JavaScript (other “active elements” like Adobe Flash are slowly dying). Equally annoying is the trend to use gray on white text or illegibly small fonts. A set of extensions was used to integrate the appearance of websites better into MDVL and disable unwanted things with the ability to interactively override the changes if a website would not work.

NoScript

Probably the most famous among all extensions listed on this page, NoScript was one of the earliest extensions added to Firefox and never again removed. It provides the ability to disble JavaScript by default and then activate it on a per-site basis, either temporarily or by updating a whitelist. Additionally, NoScript can block some forms of Cross-Site Request Forgery (XSRF) attacks.

NoScript before it was a WebExtension

Document Font Toggle

After one of the more recent Firefox updates, fonts had ceased to render properly (as if hinting were disabled) and it seemed impossible to recover from that. As a radical, yet beneficial solution, all fonts were changed to Terminus. With some pages failing to work with fonts other than their own (especially considering the trend to use web fonts to display icons on pages), a means of re-enabling the pages’ fonts whenever needed was realized by using the Document Font Toggle extension. With new Firefox versions, font rendering improved again such that text could again be read without Terminus, yet the extension was kept because it still often increased readability and unified the font across most MDVL applications.

Monochro

MDVL uses white-on-black text for all console windows. As file manager, e-mail client and text editor are all console applications on MDVL, this means the web browser is often the only non-console application running and then it is also often the only application with black text on white ground by default. To be able to disable pages’ colors and replace them by a distinguishable and MDVL-compatible colorset, this extension was used to be able to switch-on pages’ colors whenever needed and otherwise display fonts and borders white on dark gray by default. As a side-effect this extension disables background images which can be useful to reduce annoying advertisements but also sometimes makes parts of pages’ user interface illegible.

Ma_Sys.ma Website 4 with Document Font Toggle and Monochro disabled, i. e. regular rendering

Ma_Sys.ma Website 4 with Document Font Toggle and Monochro enabled: Fonts are forced to be Terminus and colors are removed

Managing Internet Information Efficiently

The Internet is constantly changing and information is coming online and offline at seemingly uncontrollable times. As a result, whenever important information is found on the Internet, one should be quick to take a backup of them as not to find them deleted if the same information is needed later. For managing online sources in scientific (and other) writing, it is also beneficial to copy pages not only by their link but also backup their content.

Four extensions were used to provide for most of the cases:

Saved From URL

This plugin adds the original URL a page was downloaded from to all pages saved through the browsers “Save As” dialog also available through [CTRL]-[S]. When managing online sources of information, it is advisable to save all used pages to a directory with [CTRL]-[S]. By default, this will always include the page title and author because those are commonly displayed on the page itself but interestingly, the URL is rarely contained in pages. This extension adds a HTML comment with the download URL to every page downloaded and thus eliminates the need for managing URLs externally when gathering information from online sources.

Fire Captor

This extension allows screenshots of whole pages to be taken (without limitation by screen height or width). Today, the browser-integrated developer tools provide most of this extension’s functionality and external programs like wkhtmltopdf are often better because they produce a PDF output rather than a bitmap. As a result, this addon was generally disabled but could be enabled whenever special circumstances required screenshots of pages to be taken from within the browser.

Print Edit

This allows portions of pages to be removed before printing which includes the ability to export parts of webpages to PDF which is more difficult and sometimes impossible by other means.

ScrapBook

This extension manages a directory of downloaded pages and additionally allows pages to be downloaded recursively. While different external tools (often also with higher performance) exist to achieve this task, e. g. wget and httrack, those generally do not support all the features of modern browsers and thus fail to download more complex pages. ScrapBook always achieved the most complete recursive download of websites and could be operated quickly because it automatically managed the download location of pages. Unlike with [CTRL]-[S], a single downloaded page includes the download URL by default and comes as a single folder instead of a file and separate folder. This way, renaming the directory is less an issue than with [CTRL]-[S]. With improvements in D5Man, ScrapBook’s importance should decrease but as of now, the extension is still required.

The power of ScrapBook is hard to capture in a screenshot but the important feature of recursively saving pages can be seen in the “Save Page As…” dialog

Extensions used before Firefox 60

The following table provides an overview of the extensions used before along with a number of #-characters to indicate their importance (more # means more important).

As one can see, the most important extensions were VimFx, NoScript and Saved From URL with ScrapBook still being very important.

Making Firefox 60 Usable

Upgrading to Firefox 60 resulted in all of the extensions being deactivated except for Web Developer, NoScript. At least the User Agent Switcher was also available for Firefox 60 and could thus be installed again.

With many of the extensions initially lost, functionality was restored by searching for compatible extensions with similar behaviour than the old ones and by setting aside the less frequently used ones.

The following extensions were considered unimportant enough to not need a replacement: Classic Theme Restorer, Fire Captor, Print Edit (all with importance #).

As a result, replacements for the following extensions needed to be sought:

• HttpFox
• VimFx
• Document Font Toggle
• Monochro
• Saved From URL
• ScrapBook

Extensions without sensible Replacements

HttpFox

This extension could not be replaced by any other extension. That is still acceptable because the browser-integrated tools provide the needed functionality.

Saved From URL

Very sadly, there does not seem to exist a replacement for this simple functionality.

Replacement Extensions for VimFx and ScrapBook

Vimium

The description of VimFx already lists some alternatives and Vimium is a really good replacement. The only disadvantage compared to before seems that it does not focus newly created tabs by default.

Web ScrapBook

While it is similar to ScrapBook by its name, Web ScrapBook only provides a fraction of the useful features from ScrapBook: It does not currently seem to support recursive downloads and a sidebar/menu with a list of all downloaded pages is also missing. Still, it stores the original URL and is better than [CTRL]-[S] as explained for ScrapBook before. Time will tell how well it replaces ScrapBook when used regularly.

The Special Cases: Document Font Toggle and Monochro

Although some extensions exist for realizing similar functionality to Document Font Toggle and Monochro, all of these suffer from some drawbacks.

Quite a few extensions for decolorization exist, but the closest alternative to Monochro did not disable the pages’ colors by default rendering it useless to prevent the “radiation” of white from a large browser window when opening a new page.

To make fonts and colors work as expected, it seemed apt to realize this with an userstyle addon which would be able to toggle the application of a stylesheet. In its most simple form, the stylesheet could look as follows:

/* Userstyle concept CSS for Terminus and only two colors */
* { font-family: Terminus; background-color: #3c3c3c; color: #ffffff; }

In practice, some more declarations were needed but even with !important added to all CSS, userstyle extensions’ CSS would still be superseded by pages with !important in their CSS. In the end, a solution without extensions proved to be closest to the previous behavior.

Using the file <profile>/chrome/userContent.css, Firefox users can supply a stylesheet which overrides everything – it can even change Firefox’ GUI! With the following stylesheet, an adequate implementation of “Terminus and few colors only” can be realized as follows:

/* Ma_Sys.ma User Style 1.0.0, Copyright (c) 2018 Ma_Sys.ma */
/* For further info send an e-mail to Ma_Sys.ma@web.de */

/**/

@-moz-document regexp("^(http|ftp|file).*$") {
        * {
            color: white !important;
            background-color: #3c3c3c !important;
            border-color: white !important;
            outline-color: white !important;
            opacity: 1.0 !important;
            font-family: Terminus !important;
            text-shadow: none !important;
            box-shadow: none !important;
        }
        a, button, input[type="submit"] {
            color: #00cccc !important;
            border-color: #00cccc !important;
        }
        body, div, p {
            background-image: none !important;
        }
}

It might look a bit more complex than needed, but there are quite a few annoying visual CSS properties (shadows, opacity, etc.) which need to be considered. Links and buttons should appear as clickable elements (realized by a different color here) and full-page background images should be disabled without disabling the smaller images used for modern webpages’ GUIs.

The @-moz-document regexp disables the stylesheet for other elements than pages’ contents i. e. the GUIs of WebExtensions are not formatted (this is most important for uMatrix to work properly).

This solution is considered OK at the moment, but has the major drawback of being difficult to disable temporarily: Pressing [F12] one can open the developer tools. Then one can activate Browser styles under Inspector >> Computed and go to the Style Editor and find userContent.css. There, one removes the last / to comment-out the whole file which makes pages appear using only their styles.

Deactivating the userContent.css is not as simple as pressing the Monochro button was in the past.

Update (2018/09/12)

A friend of mine told me about Dark Reader and although the result of using this extension over the userstyle is a bit less “deterministic” (sometimes one needs to switch between Static and Dynamic depending on the page and it keeps some colors), it works pretty well and is definitely a sensible replacement for the extensions used before.

The Dark Reader extension works pretty well (both relevant configuration screens depcited on the right)

uMatrix

With many things changing due to the upgrade, an alternative to NoScript was also tested and found a good replacement (although NoScript is available for Firefox 60 onwards): uMatrix. This extension gives better control about what exactly is blocked and uses rules rather than a whitelist to flexibly allow portions of active elements for pages.

It is also slightly more complex to use but the user interface is easy enough to learn while using the extension ☺.

Extensions installed for Firefox 60

Finally, here is an overview on the changes and how well they went.

The probably largest negative impact is in the Managing Internet Information Efficiently category: The advanced downloading abilities of ScrapBook and the simplicity and workflow efficiency of Saved From URL are lost.

Additionally, there is some setback on the MDVL GUI integration: userContent.css is a partial and difficult-to-handle replacement for the handy extensions used before. (Dark Reader might solve this, see Update above)

Extensions installed in Firefox 60

All in all, the number of extensions installed was reduced dramatically which is a clear advantage from a security point of view.

Summary

Why would this whole thing interest you? This post serves at least these three functions:

• It is an opinion that there is some real friction and disadvantages of giving up compatibility with legacy extensions,
• By describing the extensions and their functionality, anybody interested might get some ideas about things one can do with a flexible web browser such as Firefox and it might inspire reader’s to try out some of the extensions or think about the underlying issues – integration into a local working environment and persisting (in?) the shortlived web.
• This is also some form of “internal” documentation which can be used to restore browser behaviour and benchmark new browsers to see if they support the special functionality realized with the extensions here.

To summarize, it might seem that about 10 hours of work just because of a security upgrade is quite a bad thing. And although some things remain rough because no adequate replacements exist for some extensions, Firefox is back to a usable state and with everything being WebExtensions, the door to switching to another compatible browser (e. g. Chromium) in the future is no longer sealed.

Still, with all the interesting and worrying development going on for web technology, it is not clear if the great compatibility between Firefox and Google Chrome is not probably a dangerous thing because it gives up some of the great flexibility previously uniquely found in Firefox extensions. This could strengthen Google Chrome in the long run because there is technically less advantage of Firefox over Chrome and one can doubt that Mozilla can keep up with Google in terms of rendering engine performance. This in turn could result in a monopoly for Google Chrome.

References

[1] Caitlin Neiman: Timeline for disabling legacy add-ons on addons.mozilla.org | Mozilla Add-ons Blog, https://blog.mozilla.org/addons/2018/08/21/timeline-for-disabling-legacy-firefox-add-ons/, checked 2018/09/09.

[2] Moritz Muehlenhoff: Debian – Security Information – DSA-4287-1 firefox-esr, https://www.debian.org/security/2018/dsa-4287, checked 2018/09/09.

Update 2020/01

In the meantime, two changes to the considerations here have come up:

• Extension WebScrapBook ceased to work (i. e. wrote only empty folders?) It could be replaced by using Save Page WE which is flexible enough to also be a suitable replacement for Saved from URL. The browser is more and more back to “fully-featured”.
• It was mentioned at the very beginning of this post that one of the reasons for using Firefox was its ability to process an internally used application (aka. IAL / Information and Links). With the update to Firefox 68, IAL ceased to work locally cf. Mailing List Post. This has finally triggered the re-engineering of D5Man in order to replace IAL by a new D5Man-powered alternative. See d5man2(32) for details (as of this writing, D5Man 2 is an ongoing development effort).

Update 2020/11

Added extension cliget.