Desktop Apps on Ubuntu Phone

Table Of Content

• Clarification
• Tutorial
• Applications
• Limitations

*also included in the Tutorial part is my digression about Ubuntu Touch platform (my criticism), and how I explain Canonical's failure at selling the concept.

Clarification

The label "Desktop Apps on Ubuntu Phone" is ambiguous and can potentially mean many things. For example, it could refer to enabling ARM based phone running Ubuntu Touch OS to execute x86 or x64 binaries, similarily like it is possible the other way around with virtualization or emulation. Especially interesting aspect is emulation. On Intel based Linux (but also on OSX or Windows) one can run qemu-system-arm based chroot container, and ARM instructions will get translated into Intel's on the fly. On Linux systems you can even better use so called architectural chroot, which is even better than qemu-system-arm based container, because it does not limit how much RAM you can assign to it. I mentionned this technique in my other post: 

Part.1, Telegram Desktop on Raspberry Pi & Ubuntu Touch, The Story Of (2017) [#ArchitecturalChroot]

but in that post I made a reference to a great post that uses architectural chroot to boot into Raspberry Pi's SD card on Ubuntu desktop: Faster compiling on an emulated Raspberry Pi on Linux.

The label can also be interpreted as some sort of wine equivalent for ARM that emulates intel architecture. There even exist a commercial product to claim being just that: ELTECHS: Run Intel x86 applications on ARM-based mini PC. They have a very bold claim about their product: "ExaGear allows you to run x86 applications on ARM device with up to 80% of native performance".

Off course you could even wonder about running a full blown virtual machine manager that runs x86_64 vm on ARM based host, in which case most likely it would be possible by the means of emulating foreign architecture exactly the way qemu does it. 

In fact, ability of ARM to emulate Intel architecture has become an early trend, and in the industry there are some promising pioneers trying to bring it to the market:

• Microsoft: Windows 10 ARM Devices Will Run x86 Code at Near-Native Speed
• ARM Opens Its Chips For x86 Support To All Manufacturers Including Apple & Samsung

So yeah, there are multiple way in which you could choose to interpret the title of this post.

Yet, this post actually is NOT about any of these forementioned approaches.

In fact, what this post talks about, is running apps from Linux based desktop onto Ubuntu Touch phone's screen. 

Remotely.

Over SSH, with X forwarding.

You could ask "but why?", and well, other than answering to you "because I can", please see the Applications chapter of this post.

Tutorial

from zero to having an app running remotely from Ubuntu desktop machine on your phone's screen you need only two steps:

1. setup SSH connection
2. create a custom app launcher

setup SSH connection

Here is the thing: in our scenario we will not be able to prompt user for a password to a remote machine when initiating ssh connection. Unless of course you yourself DIY a GUI for that purpose, then sure it is possible. But here we take a different approach. We setup a public key based authentication for SSH connection from your UT device to your remote Linux machine. And we make sure, that the private key used is not password protected either, otherwise even though using key encryption, there is still a problem of how the user will input the password.

First you need to have so called SSH Identity on your UT device. Here we assume creating one from scratch, on the phone (or tablet?) using Terminal App, just issue this command :

$ ssh-keygen -t rsa -b 4096

I am creating an RSA private/public key pair here, and I use 4096 bit encryption (more than default). That's the way I like it. But you are free to research on your own and tailor this to your own liking. Should you wish to do so, I may suggest you to familiarize yourself with Ubuntu Help: SSH/OpenSSH/Keys.
After issuing the command, you will be asked for the location where to save your SSH identity (I assume you just use the default one, in which case you just hit enter at the prompt), and for a password to protect your SSH Identity with - NOW - do not set the password, leave it empty by just hitting enter on the password and password confirmation prompts. Remember, the reason to do so, is so that you setup a way for your UT device to be able to ssh into your remote machine without password prompt. Again, this is for simplicity, if you insist that password is needed, then it is also possible, but just as mentioned before, you will have to also create some GUI front-end for the user to be able to input this password. Here we just make the password unnecessary entirely.
Now you have to setup your remote machine to accept your UT device to login over ssh using this SSH Identity instead of password. Again, on your UT device, issue this command:

$ ssh-copy-id username@hostname #use actual username and hostname/ip

so, if your remote machine's default user is "kris" and the ip address of the machine is 192.168.0.10, then you'll be entering:

$ ssh-copy-id kris@192.168.0.10

you will be asked for a password (it is the password of the machine you want to connect to, not the password for your local user), and after that the passwordless connection will be configured.

You can verify that it worked by trying to ssh into the remote machine:

$ ssh username@hostname

and if you were not asked for any password then it is all good. If you couldn't connect, or if you were asked for a password, well then, you did not succeed. Try again.

NOTE about doing the other way around: ssh-copy-id  from remote machine into UT device.
You see, just when you use ssh-copy-id you need that one time use password authentication first. But UT devices are configured to entirely reject any ssh connection that tries to login using password authentication. In other word, if you try to ssh-copy-id from remote machine into your UT device, you will fail. To solve this, you should change the direction of doing things. Instead of pushing your SSH Identity to your phone, pull it from your phone. to make this easy I have open-sourced a command line tool and hosted here: https://sourceforge.net/projects/ssh-pull-id/
You can compile the tool on your UT device from the source code (instructions included) or you can download a pre-compiled binary for arm, you choose. Then you just issue an alternative command from your UT device:

$ ssh-pull-id username@hostname

The reson it will work, is because while UT device rejects clients using password authentication, your remote machine likely does not. Otherwise, well, do it yourself ;)

create a custom app launcher

Now that our UT device can ssh into our remote machine without password promt, we will create a custom launcher for a remote app, that will execute ssh command. I will, in this tutorial, begin with Spotify Client For Linux on my Ubuntu 16.04 desktop.
On my UT device I create a directory ~/sshapps/spotify, and its up to you but I have decided that I will organize all of my remote apps in their own respective folders under ~/sshapps/ path. So if you choose differently, just remember to modify my instructions accordingly. In the folder for my remote app, I now will create these things:

• app icon
• app launcher
• app wrapper script

I have downloaded an icon for my Spotify app and saved it as ~/sshapps/spotify/spotify.png.

The launcher I saved as ~/sshapps/spotify/spotify.desktop, and I set it to executable like so:

$ chmod +x spotify.desktop

And then I used mcedit (which is part of mc package) to edit the file, but you can use vi if you are into sadomasochism. Here is the content of the launcher file:

[Desktop Entry]
Name=Spotify (x86_64)
Comment=Spotify over SSH
Exec=gksudo -u phablet /home/phablet/sshapps/spotify/spotify.sh
Icon=/home/phablet/sshapps/spotify/spotify.ico
Terminal=false
Type=Application
Categories=Music;Media;
X-Ubuntu-Touch=true
X-Ubuntu-XMir-Enable=true
X-Ubuntu-Application-ID=SpotifyOverSSH

And now, the only thing left is to create the app wrapper script in ~/sshapps/spotify/spotify.sh file:

#!/bin/sh
#use your own username/ip in the command below:
matchbox-window-manager & ssh -Y kris@192.168.0.10 spotify 

and remember to also set the script to executable:

$ chmod +x spotify.sh

Done! Are we? No, you need to place the launcher in a place where UT will know to look for it:

$ sudo ln -s /home/phablet/sshapps/spotify/spotify.desktop /home/phablet/.local/share/applications/

and mind you, double check if the same path is correct for your device (the /home/phablet part).
Done, at last!!! Or are we...?
matchbox-window-manager is used in the command line in our bash script (spotify.sh). If you have not manually installed it in your UT device, then most likely it is not installed, and you won't be able to start your app. You have 2 options:

1. either install it manually:

   $ sudo mount -o remount,rw / #makes the system image writable
   $ sudo apt-get install matchbox
   $ sudo mount -o remount,ro / #makes the system image read-only again

2. or modify your bash script not to use it:

   #!/bin/sh
   ssh -Y kris@192.168.1.81 spotify

Now you TRULLY done! You can at this very moment start spotify remotely off of your desktop machine and onto your UT phone's screen. Go to your Apps scope. The real Apps scope, not Libertine apps or anything else. If you cannot see the icon for Spotify, then you need to refresh the scope. You do it by scrolling to the top of the apps, and then make one more long and slow scroll up (pull down with your touch). You will see small progress bar briefly at the bottom of the scope, showing that the scope is refreshing. Then your app should appear (see the bottom row):

And when you click on it, if the ssh connection is fine, if your username and hostname are all good, then you will see your desktop's spotify appear on your phone:

You can then use context switch to move between this remote app and other apps currently opened:

And just if you feel that this is too good to be true, here is me recording this from another phone, to show you that these screens are actual thing, not some photoshopped gimmick:

Let me recommend that you in fact install matchbox window manager and use it. The rationale is that without it, Xmir will not provide any window manager to the programs it runs. So your apps will appear without window decorations, without tittle bars, and you won't be able to move/resize them (unless the app itself implements its own title bar or resize bar). Also, the app without a window manager will appear in its default size, not full-screen so you will end up with most of your screen just black and a small app in the upper left corner of that black space. Matchbox is off course not the only window manager out there, but it is very light and does the job. But mostly it is super light, and the resources on your UT device are precious, especially disk space on your system partition.

I want to add, that sometimes it is fine not to use a window manager. For example in my other post I wrote about using this: Geany on Ubuntu Touch device as text editor, source code editor, debugger and compiler for multiple languages
I used a trick there, I uses a small script inside of my wrapper script that just before starting Geany writes to its configuration file to set its window to precisely matching the screen space minus the area where the OSK shows up. Not every app though loads its size/position from a config file.
Another example is Inkscape. On UT without window manager you can resize the window because the app implements its own hot corner (bottom-right) that you can drag to resize the window:

So it is always case by case.

To add more apps, repeat same steps, here I did the same for Telegram Desktop:

the launcher ~/sshapps/telegram/Telegram.desktop:

[Desktop Entry]
Name=Telegram (x86_64)
Comment=Telegram over SSH
Exec=gksudo -u phablet /home/phablet/sshapps/telegram/Telegram.sh
Icon=/home/phablet/sshapps/telegram/telegram.png
Terminal=false
Type=Application
Categories=IM;Communication;
X-Ubuntu-Touch=true
X-Ubuntu-XMir-Enable=true
X-Ubuntu-Application-ID=TelegramOverSSH

the bash script ~/sshapps/telegram/Telegram.sh:

#!/bin/sh
matchbox-window-manager & ssh -Y kris@192.168.0.10 /opt/Telegram/Telegram

And off course I also downloaded icon as ~/sshapps/spotify/telegram/telegram.png.

Why Do I Use Wrapper Script: well, there is a lot that you can do if you use a wrapper, as opposed to just putting all in Exec line of your launcher file:

• call your own custom GUI front-end that supports SSH password prompt
• call your own custom GUI front end that checks if remote app can launch (ie. is the remote host available) and if not, will display a nice banner instead of just closing down without user knowing what happened
• call your own script/program that dynamically decided the ssh connection string: username, hostname, ssh command, based on any logic you may desire
• sky is the limit (remember the Geany example?)

Remember that SSH with X forwarding is an old legacy tech that has been default part of Linux for decades. It will work on any Linux. Using it you can even install GUI apps on a headless Linux server, that does not have any graphical session running, and launch these apps remotely onto your other machine that has graphical environment and can serve as X server. This tutorial was especially made for using Xmir on Ubuntu Touch as X server, and Ubuntu Desktop PC as X client, but ANY other scenario is possible. You can run Raspberry Pi apps on UT phone, or UT phone's app on Ubuntu Desktop. Really, be creative! For example I use X forwarding for writing apps for Ubuntu Touch. I have Lazarus IDE installed on my UT phone, and I start it onto my Ubuntu Desktop machine, so I work on native ARM lazarus and compile apps natively without any cross-compiling setup. I wrote about it in the past: Lazarus development for Ubuntu Phone, and just today I even captured this:

Also, it is possible to start apps remotely onto Windows machine's screen, you just need a 3rd party X server, because Windows doesn't have one by default. There are plenty of 3rd party X servers for various operating systems, and on Windows one that I have personally used is Xming.
Please know, however, that if you wish to run app installed on your UT phone onto a remote machine's screen, you will not be able to do it out of the box. The reason is, that currently the default Ubuntu Touch image does not contain xauth tool that is responsible for generating .Xauthority file on UT device user's profile. You can off course try to create the file yourself, or copy from another machine, but it's the harder way. xauth tool will do that fr you upon the first attempt to use X forwarding over ssh. To install xauth on UT device just use this:

$ sudo mount -o remount,rw /
$ sudo apt-get install xauth
$ sudo mount -o remount,rw /

The fact that this super tiny package is absent from the default UT image and you need to manually install it is a clear evidence that Canonical totally ignored world of possibilities with X forwarding scenario off a UT device. And now that UT is continued by UBports, the tool is still missing.
Also missing are gtk input plugins, so effectively, even though Xmir is provided, there is no way for you to use the OSK (on screen keyboard) with it out of the box. These plugins are also ultra light in size, so they were not ignored out of technical rationale, but rather as a complete ignorance toward a plethora of amazing possibilities of the Ubuntu Touch platform, and incredible workflows and use case scenarios. To get the plugins:

$ sudo mount -o remount,rw /
$ sudo apt-get -q -y install maliit-inputcontext-gtk3 maliit-inputcontext-gtk2
$ sudo mount -o remount,rw / 

Even the fact that Ubuntu phone does not ship with Terminal app by default, and that you need to manually install it, proves it clearly that Canonical was not legit when claiming that Ubuntu phone will initially be targeted at Linux fans and hackers. Without terminal Ubuntu Phone is just a gimmick, and I wholesomely agree with the author of this post: How the Terminal makes Ubuntu Touch worth using. Seems to me, that Ubuntu Touch completely ignored some truly powerful aspects appealing to genuine Linux people and instead focused on end-user aspects in a battle in which it had no chance of competing against Android and iOS. Eventually leading to Canonical abandoning the platform because it could not recognize where the platform's real value was: in offering a full-blown (as opposed to light/stripped down) Ubuntu Linux computer that you can fit in your pocket, that additionally also can be used as a mobile phone. But that is a topic for a separate post. Soon ;)

Applications

The most basic reason why I wanted to run apps from my Ubuntu Desktop remotely over SSH onto my Ubuntu Touch phone's screen was... just because I can. Just for as a proof of concept. Just to explore possibilities. However, some real life applications come to mind.

• Remote Control - say you are at home, where your PC is connected big speakers. You have Spotify on, and it's playing music. But you are not seated in front of the monitor, instead you are chilling on sofa away from your PC. You could instead, since your UT phone is connected to your home LAN over wifi, just start the very same Spotify onto your phone's screen remotely, and control it with your phone from your couch. You would still be controlling the very Spotify that runs off  your PC which is connected by cable to your home theater sound system. Sure, Spotify mobile app also allows for controling playback from another device, but I hope you understand the scenario, because you could be controlling anything else.
  In fact, you could control stuff via multiple levels of abstraction, imagine, onto your UT phone, you remotely start your desktop's installed Teamviewer, and in it you control yet another computer entirely, for example a PC at your office, which has Windows inside!
• App Testing - say you create an app on your desktop. Now you'd like to check how would that very app look and feel like if it was compiled for ARM and ran on your UT phone. You don't actually need to compile it for ARM and deploy to the phone for testing. You can just start it on the phone via SSH and find out.
  Remember that you are not limitted to run apps off of your intel based Linux desktop machine. You could just as well be starting an app off of your Linux powered Raspberry Pi or other single board computer. That one is already ARM based, but you still save time of deploying the app to your phone (copy files, create customized launcher, etc.). You just run it, and test it.
  By doing so, in the development project, you can save the disk footprint on your UT phone, because you do not need to physically deploy anything into the phone.

Limitations

As the old Chinese proverb says: With great powers comes great deal of limitations. With the approach discussed here there are two main bottleneck aspects:

• Bandwidth - obviously we are running GUI apps over SSH connection, where we forward X server messages and data over the Network. As long as you are in a LAN, the modern bandwidths should generally make this experience a breeze, with maybe exception of very hight fps scenarios, like video playback and animations rendering. When you use this approach WAN or VPN, a wholly new level of bandwidth penalty comes to you. Still, could be fine for a real life scenario, depending on your case. 
• Rendering - this bottleneck is on the UT device itself, and you deal with it whether or not you run app remotely. It is relater to Xmir (Mir's compatibility layer for X, based on XWayland), and you deal with it every time that you run legacy apps on your phone via Xmir. For example, currently, Mir does not have a native GTK+ support, so to run apps that use gtk widgetset/libraries (which is most of GUI apps on Linux), you will need to run them in Mir via Xmir. The problem (currently) with Xmir, is that it does not provide hardware accelerated rendering. If the app that you run in it renders a lot FPS, or if it uses a lot of image compositing, then your UX can become less then smooth. Hopefully hardware accelerated rendering end up added to Xmir. Apparently (as I was once told in the UBports Supergroup on Telegram) the lack of fast rendering is only missing from ARM version of XMir, and is present on the x86_64 platform.
• Scaling - this is also related to rendering. XMir reports a fixed dpi of 96. In reality UT devices have high DPI screens. So when you run anything in XMir it will not be aware of the actual dpi of the screen, it will assume it is just a regular 96dpi screen. What it means to you? Well, everything will look super tiny. Tiny buttons, tiny fonts, tiny menus. Unless off course the app itself can scale itself up, there is not much you can do about it. You could scale up fonts, and some things would become bigger, but plenty of controls will remain super tiny. If you connect a mouse to your UT device via bluetooth or USB OTG, then a mouse pointer will appear an you will be able to use these tiny controls easily, but with just touchscreen, unless you have super tiny fingers, it will definitely not be without a challenge. Just remember, practice makes perfect ;) 

So with these limitations in mind, still quite a neat, and subjectively (IMHO) practical approach overall!