Using pyqtdeploy0.5 on Linux to cross compile a PyQt app for Android: Part 1

WORK IN PROGRESS.

This is a log of a program building session.  It duplicates official instructions but discusses more details.  The discussion could well be wrong.

This is about building an app written in Python using PyQt bindings to the Qt cross platform library.  Such an app is portable to the major desktop platforms and the Android and iOS platforms.

Here I use:

  • Ubuntu 14.04
  • Python 3.4.0
  • pyqtdeploy v0.5 (with a few edits needed for v0.6)
  • Qt 5.3

About pyqtdeploy

See Building Static Packages page of the pyqtdeploy documentation.

Pyqtdeploy is a tool for deploying (compiling) apps written in Python and using PyQt.  Pyqtdeploy is actively being developed.  In previous versions, you could deploy to the mobile platforms, but you were on your own for many steps such as compiling Python statically.  In version 0.5, pyqtdeploy itself encapsulates some of those steps (you invoke pyqtdeploy to perform the steps, instead of a lower level tool such as make) :

  • building Python statically
  • building SIP statically
  • building PyQt statically

 pyqtdeploy v0.6

This might be the last version before 1.0, in other words, the developer anticipates few new features .

Note that this blog was written for v0.5.  In v0.6, the app was split into a GUI and a command line.  Everywhere below where you invoke pyqtdeploy from a command line, you should instead invoke pyqtdeploycli.

 

Prerequisites

As a starting point you should have:

Pyqtdeploy depends on:

  • Python3 (to run in)
  • Qt (to display it’s GUI)
  • Qt and its qmake tool (as a source of code and tools.)

 

Preliminaries

Before you start, there are two things to understand:

Configuring the shell environment

Certain environment variables are read by commands in what follows:

>export ANDROID_NDK_ROOT=/home/bootch/android-ndk-r10

That says where you installed the Android NDK.

At least on Ubuntu, as I installed PyQt5 (using ‘python3 configure.py…), PyQt5 (which is imported by pyqtdeploy) installs to /usr/lib/Python3.4/site-packages, and that must be on PYTHONPATH.

>export PYTHONPATH=/usr/lib/python3.4/site-packages

SYSROOT

Certain of the following commands take the parameter SYSROOT.  This names a directory.  While you are cross-compiling, it stands in for the root of an Android device.  Certain commands ‘install’ files there (as if installing to a real Android device.)  Some of the files ‘installed’ there are intermediates, e.g. Python.h header files, and won’t actually be copied to any Android device as part of any install.  Some subsequent commands will build a directory tree under SYSROOT similar to a standard Unix/Linux file system, i.e. having subdirectories /lib and /include.

I named this ‘aRoot’, created it in home, and exported a shell variable for it:

>cd ~
>mkdir aRoot
>export SYSROOT=/home/bootch/aRoot

Installing pyqtdeploy

Pyqtdeploy is a Python app.  Official instructions for installing.

I tried:

>pip3 install pyqtdeploy

That fails with “ImportError: No module named ‘pkg_resources’ “.   I was unable to fix it trying several things much discussed on the web.  Note that pip3 is recently changed for Python3.   My opinion is that this is basically a fault of Python: it is changing too much and is lacking a stable way of downloading modules, especially for multiple versions of Python.  No doubt my problem started when I downloaded a newer version of Python than distributed with Ubuntu.)

Anyway, you can alternatively install pyqtdeploy from source:

>sudo apt-get mercurial     # so you have hg
>hg clone http://www.riverbankcomputing.com/hg/pyqtdeploy
>cd pyqtdeploy          # to the directory you just cloned
>make VERSION           # ? not sure why next two steps are not dependencies in the Makefile
>make pyqtdeploy/version.py
>sudo python3 setup.py install

You can then test by

>which pyqtdeploy
>pyqtdeploy

A   GUI app should open.

Updating pyqtdeploy

If you already have pyqtdeploy, but there is a later version.   The new version might not read your old .pdy, so write down all it’s data.  To update:

>cd pyqtdeploy
>hg pull
>hg update
>make VERSION
>make pyqtdeploy/version.py
>sudo python3 setup.py install

 

Building a host version of SIP

The instructions say:

You must also have a host SIP installation that is the same version as the target version you will be building.

That means that after you downloaded the latest version of SIP (which supports target Android):

>cd ~/Downloads/sip*
>python3 configure.py
>make
>sudo make install

So now /usr/bin/sip (an executable) is the same version as the sip.a (a library) that we will later build for Android.

Building Qt statically

Although this step is described in pyqtdeploy instructions, you MIGHT not need to do this, i.e. it is optional.

When cross-compiling for mobile devices it is recommended that you use one of the binary installers provided by Digia.

This means that we will later be using QtCreator to package our app, and QtCreator will take care of bundling Qt libraries (dynamic?) into the Android package (APK) (or create an APK that uses the Ministro package manager at user’s runtime to download Qt libraries to the Android device.)

(If you have configured QtCreator for building for Android, you will have the Qt libraries cross compiled for Android in ~/Qt/5.3/android_armv7/lib )

(But keep reading, later I found that PyQt would not build, saying there is an issue with licenses.  Possibly I need to build Qt from source.???)

Building Python statically

 

Downloading Python source

Navigate your browser to a Python download page and download the source (say the ‘gzipped source tarball’ for Python version 3.4.0).

Extract it, say to ~/Python-3.4.0 .

Using pyqtdeploy to build Python statically

See below.  You can’t do this twice without starting with a fresh Python source distribution.  Pyqtdeploy patches python and copies the original Python source files to a same named file with .original appended.  So the patching will fail on a second attempt.

>cd /home/bootch/Python-3.4.0
>pyqtdeploycli --package python --target android-32 configure

(target is one of: ‘android-32’, ‘ios-64’, ‘linux-32’, ‘linux-64’, ‘osx-64’, ‘win-32’, ‘win-64’, as I determined by reading targets.py in pyqtdeploy source.  Subject to change.)
When that works, you see about 30 lines of output ending with the line: ‘Installing /home/bootch/Python-3.4.0/python.pro’.  Also, pyqtdeploy creates a python.pro file in the current directory (which is the Python source directory.)

(If you see:

Patching /home/bootch/Python-3.4.0/Include/unicodeobject.h
pyqtdeploy: /home/bootch/Python-3.4.0/Include/unicodeobject.h:2266: line does not match diff context

I think this is a fatal error meaning that pyqtdeploy, while patching the Python source, used a patch that didn’t match the source.  For me, I think this happened on a second attempt, when the source was already patched, and thus would not match the patch.  I don’t see an option to pyqtdeploy to skip patching, so you might need to start over with a fresh Python source.

Explanation: pyqtdeploy maintains a set of patches for each Python version.  This is fraught with maintenance difficulty.  The Python org refuses to support Android, making patching necessary. )

Now we start cross compiling, so define (to qmake) where the Android NDK is:

>export ANDROID_NDK_ROOT=/home/bootch/android-ndk-r10
>/home/bootch/Qt/5.3/android_armv7/bin/qmake SYSROOT=/home/bootch/aRoot

( SYSROOT is where pyqtdeploy installs packages , and I choose to install to ~/aRoot)

(I am not sure if it is necessary to use the qmake from Qt that you downloaded, and the one under ‘android_armv7’, instead of the qmake that may be installed with your distribution, which for me on Ubuntu 14.04 was Qt5.2.1.  I’m not sure whether it is necessary to export ANDROID_NDK_ROOT. )

That works very quickly, without any output to the console, and creates a Makefile in the current directory.

>make
>make install

That takes a few minutes (Python is not a large program.)  You will see a stream of commands to the compiler.  It finally installs the built (static, cross-compiled) Python library into wherever you specified for SYSROOT (which should be separate from your system’s installed Python.)

 

Using pyqtdeploy to build SIP statically

Download SIP, say to your home directory.  Then…

>cd /home/bootch/sip*
>pyqtdeploycli --package sip --target android-32 configure

That tells the pyqtdeploy tool to create a configuration file: sip-android.cfg.  That returns quickly, without any console output.

Now you run configure.py, which creates a .pro file (input to qmake.)

  • It is important to use the same Python that you built statically ??
  • pass the full name of the SYSROOT directory
  • use the name of the configuration file created in the previous step
>python3 configure.py --static --sysroot=/home/bootch/aRoot --no-tools --use-qmake --configuration=sip-android.cfg

That creates a sip.pro file.  Now you run qmake on said file:

  • I’m not sure whether you need to invoke the qmake out of the android distribution
  • this finds the sip.pro file in the present working directory
  • ANDROID_NDK_ROOT must be in the env
 >/home/bootch/Qt/5.3/android_armv7/bin/qmake

That comes back quickly, without console output, and creates a Makefile in current working directory.

 >make

That produces a stream of compiler output to the console, and leaves a file siplib/libsip.a

 >make install

That copies headers and libraries to SYSROOT, e.g.  copies sip header files to /home/bootch/aRoot/include/python3.4/

Using pyqtdeploy to build PyQt5  statically

Generate a configuration file

Download PyQt5, say to your Downloads directory.

>cd /home/bootch/Downloads/PyQt-gpl*
>pyqtdeploycli --package pyqt5 --target android-32 configure

That tells the pyqtdeploy tool to create a configuration file: pyqt5-android.cfg.  That returns quickly, without any console output.

Editing the configuration file

In my case, for subsequent steps, I got errors such as:

sip: QSslConfiguration is undefined

(When compiling the PyQt binding for QWebSockets.)

I don’t think I need that anyway.  So I edit the pyqt5-android.cfg file, removing all Qt modules that I don’t use.  (I know from previous experience with  pyqtdeploy that my app uses a small set of Qt modules.)  I edit it down to QtCore, QtGui, QtWidgets, QtPrintSupport, QtSvg, and QtNetwork.

Continuing to build PyQt statically

Now you run configure.py, which creates a .pro file (input to qmake.)

  • It is important to use the same Python that you built statically ??
  • pass the full name of SYSROOT directory
  • use the name of the configuration file created in the previous step
>python3 configure.py --static --verbose --sysroot=/home/bootch/aRoot --no-tools --no-qsci-api --no-designer-plugin --no-qml-plugin --configuration=pyqt5-android.cfg --qmake=/home/bootch/Qt/5.3/android_armv7/bin/qmake

Here, I pass –verbose because I probably have errors, and I want to know the details.

Here, I pass –qmake .  I’m not sure whether it is important to use that qmake, but qmake seems to be the vehicle by which the proper include files are found.???

Continuing….

>make
>make install

Make compiles for a few minutes, a stream of compiler commands goes to the console.    Make install does not need to be sudo, because it is just installing to SYSROOT, which you own.

If you get:

 mkdir: cannot create directory '/libs': Permission denied

it is because you are using an older version of PyQt subject to QTBUG 39300 as discussed in footnote 5 of the PyQt Building Static Packages.  You need to use the latest snapshots of SIP and PyQt (not the stable GPL or commercial version.)

If you get a message saying the PyQt license is incompatible with the Qt license???

See part 2.

 

Advertisements

5 thoughts on “Using pyqtdeploy0.5 on Linux to cross compile a PyQt app for Android: Part 1

  1. Pingback: Deploying PyQt Python Qt apps cross platform using pyqtdeploy | plashless

  2. why is it that none of the python3 modules get compiled during this process. I am getting many errors using libraries that I need to complie my project that compiles just fine with the Desktop settings. Namely anything that requires a C module such as _ctypes etc

    • As I recall, you may need to edit Python3.4/Modules/Setup and uncommment C extension modules that your app uses. I think I discussed this in another blog about using pyqtdeploy for the desktop, but I omitted that discussion here for some reason, and that was with an earlier version of pyqtdeploy. I assume you mean a link-time error. I can’t recall whether the newer pyqtdeploy understands and patches the Setup file? I started with helloworld and then progressed to my larger app, which I was able to get running on Android (after a fashion.)

      • I am still working on this. did you have to adjust the config.sub for the correct arch? I modified the Setup.dist as you mentioned and I don’t see any difference the libs are still not there when I try to compile my project. is it possible that you might have a copy of your aRoot folder that I could take a look at. thanks

  3. I have mostly gotten everything working except for the _socket module

    Makefile:1258: warning: ignoring old commands for target `.obj/socketmodule.o’
    make: *** No rule to make target `../Python-3.4.1/linux/swab.h’, needed by `.obj/socketmodule.o’. Stop.

    Also i get a tipc.h error.

    So this is the error i get when trying to compile the socketmodule.c I cannot get it to compile.
    if i try pointing to the linux folder in the NDK it gives me a ton of error about Types
    and I get the same when I try pointing it at the linux folder of the host machine.

    If I could see your makefile maybe it would help.. I wouldnt mind if you gave me the _socket.a either lol..
    thanks for the help

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s