Sometimes you have internet access but you have a low bandwidth and the internet is spotty. There are some tools you can use in these situations wich make use of caching, local search and compression techniques. I focus on android and local first.

caching documentation and wiki’s

DevDocs.io

It uses the offline cache of the browser to make searchable documentation of for example languages and projects like Tensorflow, Terraform, Saltstack, Ansible, Ruby, Python, Markdown, Kubernetes, Julia, Homebrew, HAProxy, Mermaid, Vue, Docker available offline.

Wikipedia, wikis and sites

[Kiwix] is a project geared to offline access with so called zim databases, essentially downloaded and indexed websites. Each can be downloaded indivdually. The Kiwix program is around 20M depending on version and OS. The sizes of some example Zim databases are: Wikipedia 0.8 2.11G or 5.8G with images, a selection of 45.000 articles Wikipedia 12.91G or 95.2G with images RationalWiki 116.8M Termux wiki 1.57M Alpine wiki 1.82M Arts and Crafts 44M Energypedia 2.41M Appropedia 438M Chess 70M Puzzling 370M Alternatively one can copy a site with HTTrack, be aware that the bandwidth costs are not necessarily free for the web administratior. Suppose you are downloading a gigabyte and the site server traffic is metered by megabyte.

searching browser cache

  • Recoll desktop search (around 200M depending on version and OS)

With the Recoll WE addon visited html, text, pdf and epub pages are archived to a local installed Recoll desktop search indexer. Set the webdownloadsdir variable to the location of the default downloads directory in your recoll.conf Recoll configuration file (only needed if the location differs from the default $HOME/Downloads). The downloads dialog will appear the first time a page is saved. Click the Save file and *Do this automatically for files like this from now on checkboxes so that things happen automatically in the future (the saved data has a specific MIME type, so this will only affect this addon). Since version 98.0, Firefox automatically opens the download dropdown list when a download completes, so it happens all the time when the Recoll WE extension is active; A workaround a kind user found is to set browser.download.alwaysOpenPanel to false.

Also consisting of Promnesia browser addons and a server with a much wider scope. See the github readme for installation.

  • Diskernet

Full text search archive from your browsing and bookmarks. Only works with Chrome based browsers.

Proxy

  • WWWOffle is a local proxy server specifically for spotty connections. Written by Andrew M. Bishop between 1996 and 2009.

  • Ziproxy is a forwarding (non-caching) compressing HTTP proxy server.

  • It squeezes images by converting them to lower quality JPEGs or JPEG 2000 and compresses (gzip) HTML and other text-like data.

  • It also provides other features such as: HTML/JS/CSS optimization, preemptive hostname resolution, transparent proxying and Ad-Blocker.

  • Both proxies can be chained for a minimum amount of bandwidth. In case some Single Page Applications or logins do not work correctly they need to be configured in the proxy configuration. When combining a proxy with local search the amount of disk space needed is doubled. Wwwoffle cache can be searched as well.

SSH with mosh and tmux