Tutorial: Open Source on iOS (Part 3): Creating Pseudo-Frameworks

Posted on September 29, 2012 by idz

On Mac OS X frameworks provide a convenient way to package the dynamic libraries and header files into a single directory structure making it easier to include them in Xcode projects. Apple does not allow dynamic libraries (other than ones they provide) to be used with iOS so it may appear the third-party frameworks are not possible, however it is possible to create a pseudo-framework using a static library giving many of benefits of a real framework. In this tutorial you will combine the static libraries you created in the previous tutorial in a single pseudo-framework that will contain binaries for both the device and simulator.

Creating Ogg.framework

First ensure that you are in the directory where you built libogg.

 
cd $IDZ_BUILD_ROOT/libogg/1.3.0

Now make a directory to hold the framework files.

mkdir Ogg.framework

Copy the header file from one of the install directories to the Headers sub-directory within the framework.

cp -R install-iPhoneSimulator-i386/include/ogg Ogg.framework/Headers

Finally create a fat static library in the framework directory with the same name as the framework

lipo -create -output Ogg.framework/Ogg \
-arch i386 install-iPhoneSimulator-i386/lib/libogg.a \
-arch armv7 install-iPhoneOS-armv7/lib/libogg.a

A script for creating frameworks idz_fw

The steps in the above section can be generalized and made into a script. Using a text editor create the file $IDZ_BUILD_ROOT/bin/idz_fw with the following contents:

#!/bin/bash
#
# Script to create a pseudo-framework
#
idz_usage()
{
  IDZ_SCRIPT_NAME=`basename $0`
  echo "Usage: $IDZ_SCRIPT_NAME <name> <lib_name> <header_dir>"  
  exit 1
}

IDZ_NAME=$1 # Name of the framework (e.g. Ogg)
IDZ_LIBNAME=$2 # Name of .a file (e.g. libogg.a)
IDZ_HEADERS_SRC=$3 # Name of the directory to copy to Headers

IDZ_FW=$IDZ_NAME.framework
IDZ_HEADERS=$IDZ_FW/Headers

if [ -e $IDZ_FW ] ; then
  echo "Backing up existing framework to $IDZ_FW.bak"
  mv $IDZ_FW $IDZ_FW.bak
fi
mkdir -p $IDZ_FW
cp -R $IDZ_HEADERS_SRC $IDZ_HEADERS

for IDZ_ARCH in i386 armv6 armv7 armv7s; do
  case $IDZ_ARCH in
    i386 )
      IDZ_PLATFORM=iPhoneSimulator
      ;;
    armv6 | armv7 | armv7s )
      IDZ_PLATFORM=iPhoneOS
      ;;
    * )
      echo "Unrecognised architecture $IDZ_ARCH"
      idz_usage
      ;;
  esac
  IDZ_INSTALL_DIR=install-$IDZ_PLATFORM-$IDZ_ARCH
  IDZ_LIB=$IDZ_INSTALL_DIR/lib/$IDZ_LIBNAME
  if [ -e $IDZ_LIB ] ; then
    IDZ_LIPO_ARCH="$IDZ_LIPO_ARCH -arch $IDZ_ARCH $IDZ_LIB"
  fi
done

lipo -create -output $IDZ_FW/$IDZ_NAME $IDZ_LIPO_ARCH

Don’t forget to make it executable.

The arguments to this script are the framework name without the .framework extension, the library name and directory containing the include files to be copied to Headers. The script assumes the build naming convention used in the previous tutorial.

Using this script all the steps of the previous section can accomplished with the command:

idz_fw Ogg libogg.a install-iPhoneSimulator/include/ogg

The Ogg.framework is now ready to use, however without a codec it is not all that interesting. In the next tutorial you will use the scripts developed so far to compile libvorbis an open source audio codec.


Posted in Tutorial | Leave a comment

Tutorial: Open Source on iOS (Part 2): Compiling libogg on iOS

Posted on September 29, 2012 by idz

In this tutorial you will learn how to compile open source libraries that use the GNU Build System (GBS) for iOS. I’ll be using the open source libogg library as an example, but the techniques you use should be useful for any source that uses GBS. Make sure you complete the previous tutorial “Open Source on iOS (Part 1): Gathering Your Tools” before attempting this one; it won’t work otherwise!

Preliminaries

The scripts in this tutorial will use the xcode-select command to locate the active Xcode installation. This is part of the Command Line Tools package. This package is not installed by default when installing from the App Store. To see if this is installed on your system, start Xcode and choose Xcode > Preferences…. Click the Downloads icon in the dialog. If Command Line Tools is not installed, click the install button.

To test that xcode-select is correctly installed and is pointing to your preferred Xcode installation type:

xcode-select -print-path

you should see something like:

/Applications/Xcode-4.5.app/Contents/Developer

Your output may be slightly different. I modify the name of my Xcode directory to include the version number to allow multiple versions to co-exist.

If the output of xcode-select does not point to your preferred Xcode installation you can change it using something like:

xcode-select -switch /path/to/your/Xcode

Cross-Compiling with configure

Cross-compiling refers to compiling a binary on one system for use on another. In your case this refers to compiling on a Mac for either the iOS simulator or an iOS device.

With the GNU Build System, a successful cross-compile depends on the value of a few environment variables and some command line options to the configure script. In most cases, setting the following environment variables:

  • CC the path to the C compiler,
  • CXX the path to the C++ compiler,
  • CFLAGS the flags to pass to the C compiler,
  • CXXFLAGS the flags to pass to the C++ compiler,
  • LDFLAGS the flags to pass to the linker

and the following command line arguments:

  • --host the system the binaries will run on,
  • --prefix the installation location,
  • --disable-shared only build static libraries

is sufficient to obtain working static libraries.

The Environment Variable Values

All of the flags (CFLAGS, CXXFLAGS, LDFLAGS) variables must include -isysroot and -arch command line arguments. The -isysroot command line argument specifies the SDK being used for compilation, while the -arch flag specifies the architecture; the arch flag should be one of: i386 (for the simulator), armv6, armv7 or armv7s.

These paths are rather long and tedious to type, so I will describe how they are related in script terms (and from that you will derive a script).

If we assume:

IDZ_XCODE_ROOT=`xcode-select -print-path`

that is, the value of IDZ_XCODE_ROOT is the result of executing xcode-select, then the path to given platform directory is:

IDZ_PLATFORM_PATH=$IDZ_XCODE_ROOT/Platforms/$IDZ_PLATFORM.platform/Developer

where IDZ_PLATFORM is one of: iPhoneSimulator or iPhoneOS.

The above result can be used to determine the platform binary (bin) directory needed for CC and CXX.

IDZ_PLATFORM_BIN_PATH=$IDZ_PLATFORM_PATH/usr/bin

and the IDZ_PLATFORM_PATH the SDK path (needed for -sysroot) can be determined as follows:

IDZ_SDK_PATH=$IDZ_PLATFORM_PATH/SDKs/$IDZ_PLATFORM$IDZ_SDK_VERSION.sdk

where IDZ_SDK_VERSION is, for example, 5.1 or 6.0.

With these variables in hand, the required environment variables for the configure script can be derived as follows:

# Cross-compile environment variables 
IDZ_FLAGS="-isysroot $IDZ_SDK_PATH -arch $IDZ_ARCH"
CC=$IDZ_PLATFORM_BIN_PATH/llvm-gcc
CXX=$IDZ_PLATFORM_BIN_PATH/g++
CFLAGS=$IDZ_FLAGS
CXXFLAGS=$IDZ_FLAGS
LDFLAGS=$IDZ_FLAGS
export CC CXX CFLAGS CXXFLAGS LDFLAGS

provided the values of IDZ_ARCH, IDZ_PLATFORM and IDZ_SDK_VERSION are known.

The Command Line arguments

The --host command line argument is either i386-apple-darwin10 for the simulator or arm-apple-darwin10 for a device. Notice that these values depend only on the value of IDZ_ARCH.

The --prefix command line argument is used to determine where the project will be installed. To ensure that distinct architecture have their own directories I will use install-$IDZ_PLATFORM-$IDZ_ARCH.

Putting It All Together

Putting all the above together and adding a little bit of error checking we obtain the following script. I have not covered all the syntax used in this script. I will come back and add more explanation when I have time.

Using your favorite text editor, copy and paste the following script into $IDZ_BUILD_ROOT/bin/idz_configure:

#!/bin/bash

IDZ_SAVE_DIR=`pwd`
IDZ_SCRIPT_NAME=`basename $0`

# Print usage information and exit
idz_usage()
{
  cd $IDZ_SAVE_DIR
  echo "Usage: $IDZ_SCRIPT_NAME arch sdk_version configure_path"
  echo "  Where arch is one of armv6, armv7, armv7s"
  exit 1
}

# Print error message and exit
idz_error()
{
  cd $IDZ_SAVE_DIR
  echo $1
  exit 1
}

# Test if the last command failed and call idz_error if appropriate
idz_check_error()
{
  if [ $? -ne 0 ] ; then
    idz_error $1
  fi
}

# Print usage if too few arguments
if [ $# -lt 3 ] ; then 
  idz_usage
fi

# Transfer arguments to more meaningful names
IDZ_ARCH=$1
IDZ_SDK_VERSION=$2
IDZ_CONFIGURE=$3

# IDZ_PLATFORM and IDZ_HOST only depend on IDZ_ARCH
case $IDZ_ARCH in 
  i386 )
    IDZ_PLATFORM=iPhoneSimulator
    IDZ_HOST=i386-apple-darwin10 
    ;;
  armv6 | armv7 | armv7s )
    IDZ_PLATFORM=iPhoneOS 
    IDZ_HOST=arm-apple-darwin10
    ;;
  * )
    echo "Unrecognised architecture $IDZ_ARCH" 
    idz_usage
    ;;
esac

# Convert a possible relative path to configure to an absolute path
case $IDZ_CONFIGURE in
  /* )
    ;;
  * )
    IDZ_CONFIGURE=`pwd`/$IDZ_CONFIGURE
    ;;
esac
    
# Derive the environment variables
IDZ_XCODE_ROOT=`xcode-select -print-path`
IDZ_PLATFORM_PATH=$IDZ_XCODE_ROOT/Platforms/$IDZ_PLATFORM.platform/Developer
IDZ_SDK_PATH=$IDZ_PLATFORM_PATH/SDKs/$IDZ_PLATFORM$IDZ_SDK_VERSION.sdk
IDZ_FLAGS="-isysroot $IDZ_SDK_PATH -arch $IDZ_ARCH"
IDZ_PLATFORM_BIN_PATH=$IDZ_PLATFORM_PATH/usr/bin
CC=$IDZ_PLATFORM_BIN_PATH/llvm-gcc
CXX=$IDZ_PLATFORM_BIN_PATH/g++
CFLAGS=$IDZ_FLAGS
CXXFLAGS=$IDZ_FLAGS
LDFLAGS=$IDZ_FLAGS
export CC CXX CFLAGS CXXFLAGS LDFLAGS

# Derive the command line arguments
IDZ_PREFIX=$IDZ_SAVE_DIR/install-$IDZ_PLATFORM-$IDZ_ARCH
IDZ_CONFIGURE_FLAGS="--host=$IDZ_HOST \
		     --prefix=$IDZ_PREFIX \
		     --disable-shared $IDZ_EXTRA_CONFIGURE_FLAGS"

IDZ_BUILD=$IDZ_SAVE_DIR/build-$IDZ_PLATFORM-$IDZ_ARCH
mkdir -p $IDZ_BUILD
cd $IDZ_BUILD
IDZ_SAVE_DIR=$IDZ_BUILD

$IDZ_CONFIGURE $IDZ_CONFIGURE_FLAGS | tee configure_$IDZ_ARCH.log 2>&1
make clean | tee make_clean_$IDZ_ARCH.log 2>&1
idz_check_error "Make clean failed."
make | tee make_$IDZ_ARCH.log 2>&1
idz_check_error "Build failed."
make install | tee make_install_$IDZ_ARCH.log 2>&1
idz_check_error "Install failed."

cd $IDZ_SAVE_DIR

Remember to make it executable using:

chmod u+x $IDZ_BUILD_ROOT/bin/idz_configure

Downloading the Source Code

Many open source projects provide stable releases available via download and “cutting edge” releases via source code control. The stable release of libogg, 1.3.0, available via download was not possible to adapt to cross-compilation so for this tutorial you will use the source control version.

Make sure you are in the idz_build directory:

pushd $IDZ_BUILD_ROOT

Make a directory for libogg

mkdir -p libogg/1.3.0
pushd libogg/1.3.0

Check out the code from source code control 

svn co http://svn.xiph.org/tags/ogg/libogg-1.3.0

The download distribution normally has the configure script pre-made. In this case you will need to create it. 

 
pushd libogg-1.3.0
./autogen.sh

The result of this process is a configure script that has been run for a Mac. This will interfere with your iOS build. To remove this configuration information issue the following command:

 
make distclean

You will also need to make a correction to the configuration script. Open the configure script in a text editor and locate the section near line 11669:

        *-*-darwin*)
                DEBUG="-fno-common -g -Wall -fsigned-char"
                CFLAGS="-fno-common -O4 -Wall -fsigned-char -ffast-math"
                PROFILE="-fno-common -O4 -Wall -pg -g -fsigned-char -ffast-math"
                ;;

Change the -O4 to -O3 to make it look like this:

        *-*-darwin*)
                DEBUG="-fno-common -g -Wall -fsigned-char"
                CFLAGS="-fno-common -O3 -Wall -fsigned-char -ffast-math"
                PROFILE="-fno-common -O3 -Wall -pg -g -fsigned-char -ffast-math"
                ;;

For some reason gcc seems to generate invalid arm object files when optimization level 4 is used. Changing the optimization level to 3 seems to solve this problem.

You should now pop back to parent directory:

popd

Using The idz_configure Script

This script will only work if you are in the correct directory. So as a precaution execute the command:

cd $IDZ_BUILD_ROOT/libogg/1.3.0

To attempt to build the library issue the following command (if you are not using the 6.0 SDK you should change this):

idz_configure armv7 6.0 libogg-1.3.0/configure

It will fail!

But why did it fail? Checking the error messages we see:

libtool: link: /Applications/Xcode-4.5.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/llvm-gcc -D_V_SELFTEST -fno-common -O4 -Wall -fsigned-char -ffast-math -isysroot /Applications/Xcode-4.5.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS6.0.sdk -arch armv7 -isysroot /Applications/Xcode-4.5.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS6.0.sdk -arch armv7 -o test_bitwise test_bitwise-bitwise.o 
ld: warning: ignoring file test_bitwise-bitwise.o, file was built for unsupported file format ( 0x42 0x43 0xc0 0xde 0x21 0x c 0x 0 0x 0 0xf4 0x18 0x 0 0x 0 0x 1 0x10 0x 0 0x 0 ) which is not the architecture being linked (armv7): test_bitwise-bitwise.o
Undefined symbols for architecture armv7:
  "_main", referenced from:
      start in crt1.o
ld: symbol(s) not found for architecture armv7

There is a lot of noise in this message but basically what it means is that, in the course of the build, it tried to run some executable that was not compiled for the build system’s architecture. Of course, since this was a cross-compile it should never have tried to run anything so where was the problem?

It turns out the problem is in the libogg-1.3.0/src/Makefile.am.

If you change the ‘noinst_PROGRAMS’ to ‘check_PROGRAMS’ this will solve the problem. What this change does is that it specifies that some programs were test or check programs and could only be run on the host system.

After making this change you must:

 
pushd libogg-1.3.0
./autogen.sh
make distclean
popd

before running

idz_configure armv7 6.0 libogg-1.3.0/configure

this time the build should complete.

The previous step created the static library for use on a device (running a pre-iOS 6 OS, for iOS 6 and later use armv7s). To create a version for the simulator you need to use:

idz_configure i386 6.0 libogg-1.3.0/configure

In the next tutorial you will learn how to combine multiple static libraries into a pseudo-framework.


Posted in Tutorial | 13 Comments

Tutorial: Open Source on iOS (Part 1): Build Your Tools

Posted on September 28, 2012 by idz

This the first in a series of tutorials explaining how to compile popular open source libraries for iOS. In this tutorial you will download and compile a number of tools used in the GNU Build System used by many open source projects, specifically autoconf, automake and libtool. Since each of these tools is built with the GNU Build System it also provide a useful introduction to the system itself. It should be noted that many projects can be built without having these tools installed, but when things go wrong it is useful to have them around.

Open source builds use the command line extensively. If you are familiar with bash you can probably skip most of my commentary about commands and scripting, if not, I hope I provide enough information for you to understand what is going on. If you ever forget how to use a command you can always type man command_name where command_name is the name of the command you are trying to use. The system will provide you with a manual page for the command.

Creating the Directory Structure

To get started open the terminal application and type the following:

cd ~
mkdir idz_builds
cd idz_builds
mkdir -p autoconf/2.69
mkdir -p automake/1.12
mkdir -p libtool/2.4.2

Let’s look at this a little more closely. The command cd changes directory; the tilde ‘~’ character is a shorthand for your home directory. (If you ever lose track of the directory you are in you can type pwd to print the current working directory). The command mkdir idz_build creates a new directory named idz_build. The following command changes into this newly created directory. This directory will be the root directory containing all your source, builds and scripts for your open source builds.

The remaining commands create directories for each of the tools you are about to build. As time goes on you may need to build updated versions so a sub-directory is created for each version. The -p flag instructs mkdir to make all directories in the path.

Building autoconf

The first tool you will build is autoconf. Type the following command.

pushd autoconf/2.69

The command pushd is similar to cd in that it changes directory but it also pushes the directory you were in onto a directory stack. To pop a directory off the stack and change to it use the popd command.

Now that you are in the appropriate directory you download and decompress the source for autoconf using the following commands.

curl -O http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
tar xvfz autoconf-2.69.tar.gz

The command curl is a command line equivalent of downloading a file via your web browser. The -O argument tells the command to create an output file with the basename of the URL; in this case autoconf-2.69.tar.gz.

The tar command is the command line equivalent of uncompressing an archive file. The argument xvfz means extract, verbose, file, unzip.

mkdir build
pushd build
../autoconf-2.69/configure
make

To test that the build has been successful you can run (WARNING: on my MacBook Air this took quite some time. You can skip this step):

make check

To install the results of the build you type:

sudo make install

This command will prompt you for your password. Before you type it you should understand what is going on here. Your computer has a concept of a superuser (a.k.a. root). The sudo command executes whatever follows it as if it were superuser. The make install command needs to be executes as root because it will install programs in a global directory (in this case /usr/local).

If you want to test that your install has been successful you can type (WARNING: on my MacBook Air this took quite some time. You can skip this step):

make install check

To return to the top-level idz_build directory type:

popd
popd

To make sure your system is using your newly build autoconf type:

autoconf --version

It is quite possible that your system may not find you newly installed version or no version may exist.

To correct this problem type the following commands:

echo 'export PATH=/usr/local/bin:$PATH' >> ~/.profile

The echo command simply prints its arguments, items in single quotes are printed explicitly, but the >> operator causes the output to appended to a file, in this case, a special, hidden file called .profile. The .profile file is used to set variables for your command line terminal (shell). The PATH special environment variable is used to tell the shell where to look for executable commands. The export command makes the changes to PATH globally available. The assignment PATH=/usr/local/bin:$PATH prepends /usr/local/bin to the search path /usr/local/bin first when looking for commands to execute.

Building automake

In the last section, to explain the steps involved in compiling a GNU tool, I suggested typing commands one by one. To repeat this process would be tedious, therefore I suggest gathering these commands into a shell script. A shell script is a sequence of commands gathered into a file.

Let’s review the basic steps:

curl -O $IDZ_URL
tar xvfz $IDZ_TAR_NAME
mkdir build
cd build
../$IDZ_DIR_NAME/configure
make
sudo make install

So essentially all we need to do supply suitable values for the variables, that is, the values prefixed with the $ sign.

It turns out that these values can be easily derived from the program name and version.

IDZ_DIR_NAME=$IDZ_PROGRAM-$IDZ_VERSION
IDZ_TAR_NAME=$IDZ_DIR_NAME.tar.gz
IDZ_URL=http://ftp.gnu.org/gnu/$IDZ_PROGRAM/$IDZ_TAR_NAME

Putting all this together we can create the following minimal script:

#!/bin/bash
IDZ_PROGRAM=$1
IDZ_VERSION=$2

IDZ_DIR_NAME=$IDZ_PROGRAM-$IDZ_VERSION
IDZ_TAR_NAME=$xiIDZ_DIR_NAME.tar.gz
IDZ_URL=http://ftp.gnu.org/gnu/$IDZ_PROGRAM/$IDZ_TAR_NAME

curl -O $IDZ_URL
tar xvfz $IDZ_TAR_NAME
mkdir build
cd build
../$IDZ_DIR_NAME/configure
make
sudo make install

In this script that #!/bin/bash is just some magic to make a script run as a program and the $1 and $2 refer to the first and second arguments given to the program. So, executed in the correct directory, idz_gnu_tool automake 1.12 should download, build and install automake version 1.12.

This is a minimal script because it does not error checking. The following is a more complete script:

#!/bin/bash

# We need a program_name and a version_number
if [ $# -lt 2 ] ; then
  echo "usage: $0 <program_name> <version_number>"
  exit 1
fi

IDZ_PROGRAM=$1
IDZ_VERSION=$2

IDZ_DIR_NAME=$PROGRAM-$VERSION
IDZ_TAR_NAME=$DIR_NAME.tar.gz
IDZ_URL=http://ftp.gnu.org/gnu/$PROGRAM/$TAR_NAME

IDZ_SAVE_DIR=`pwd`

idz_check_error()
{
  if [ $? -ne 0 ] ; then
    echo $1
    cd $IDZ_SAVE_DIR
    exit 1
  fi
}

curl -O $URL
idz_check_error "Failed to download from $URL"

tar xvfz $TAR_NAME
idz_check_error "Failed to extract $TAR_NAME"

mkdir build
cd build

../$DIR_NAME/configure
idz_check_error "Configuration failed."

make
idz_check_error "Build failed."

echo "Enter password to allow installation."
sudo make install
idz_check_error "Configuration failed."

cd $IDZ_SAVE_DIR

This is the first of a number of scripts you will be creating to avoid repetitive tasks. Create a directory to hold scripts and change into it:

cd ~/idz_builds
mkdir bin
pushd bin

You can now either download and decompress the above script:

curl -O https://iosdeveloperzone.com/tutorials/idz_gnu_build.gz
gunzip idz_gnu_build.gz

or if you prefer copy and paste the script using your favorite text editor. If you do not have a favorite text editor the following will create and empty file and open it in the TextEdit app:

touch idz_gnu_build
open -a TextEdit idz_gnu_build

The script file needs to be made executable to allow it to run. Issue the following command:

chmod u+x idz_gnu_build

This command adds (+) execute permission for the user that owns the file.

Finally you need to make sure that shell can find this file. Modify the last lines of your ~/.profile as follows:

export IDZ_BUILD_ROOT=~/idz_builds
export PATH=/usr/local/bin:$IDZ_BUILD_ROOT/bin:$PATH

Line 1 creates a new environment variable IDZ_BUILD_ROOT which you will use in later scripts. Line 2 is a modification of the change you made after compiling autoconf that adds you new script directory to places the shell searches for executables.

If all has gone well you should now be able to type the following commands:

pushd $IDZ_BUILD_ROOT/automake/1.12
idz_gnu_build automake 1.12
popd

This will download, build and install automake.

Compiling libtool

Compiling libtool is now trivial:

pushd $IDZ_BUILD_ROOT/libtool/2.4.2
idz_gnu_build libtool 2.4.2
popd

Verifying Tool Versions

You can check all the tools are successfully installed and that the shell is finding the correct versions by typing:

autoconf --version
automake --version
libtool --version

You will need all of these tools to be correctly installed before continuing to the next tutorial “Compiling libogg for iOS”.


Posted in Tutorial | Tagged , , , , | 5 Comments