Skip to content

Archive

Tag: Build

One word of CAUTION: modify build settings at your own risk. These are MY build settings. Pending our next release, I cannot confirm that these build settings will even work for us.

Project Settings

Architectures

  • Base SDK: latest
  • Valid Architectures: armv7 armv7f armv7k i386
    => I exclude armv6, see ‘Deployment’ section. If you go down that road, make sure to add ‘armv7′ to ‘Required Device Capabilities’ in your info.plist (for the flip side of this, see this stack overflow item ).
    => Once I excluded armv6, my code wouldn’t run into the simulator, with the compiler broadcasting a ‘no architecture to build for’ error. Adding i386 resolves the issue. 

Build Options:

  • Compiler: Apple LLVM 3.0
    => Provisionally.
    With one of our libraries I build with GCC 4.2 because builds will stall with LLVM and (-O2/-O3/-Os). With the same library compiled against Apple LLVM 3.0 , GDB crashes. 

Deployment:

  • iOS deployment target: iOS 4.3
    => targeting iOS 4.3 excludes 2nd generation devices ( ~3.5% of iOS market ) ; more helpfully, this strategy avoids targeting devices that are too slow to run high performance applications.

Code generation:

  • Optimization:
    • Debug: -O0
      => aggressive optimization may confuse debuggers
      => 1.1x to 2x faster build times depending on compilers.
    • Release: -Os or -O3
      => I favor -O3 because our codebase is ‘small’ so I’m not overly worried about executable size. How small is small? Less than 64k lines of code.
  • Relax IEEE compliance: YES
    => The compiler switch for this is -ffast-math, which is what we’re after. Won’t work if you use isnan(x) or other functions.
.
Unit test targets
.
  • Build active architecture only : YES
    => reduces build time

XCode notes

  • Code generation settings are under project > Build Settings > Code generation
  • The ‘main switch’ is under LLVM GCC 4.2 > Optimization level.
  • You can pass additional flags under LLVM GCC 4.2 > Language > Other C flags
LLVM or GCC?
In the best case scenario you can pick between 3 compilers. I don’t have benchmarks; the overall feel from reading around is that GCC may generate faster apps.
Incidentally, my system stalled while building a C++ library with -O2/-O3/-Os turned on (LLVM GCC, LLVM 3.0). Adding -fno-inline-functions to Other C flags ‘fixed’ the issue. But now, that’s quite note what I wanted to do. Switching to GCC 4.2 solved the problem (the target library wraps basic maths with inlines; enforcing inlining yielded 10-15% gain).
.
Read here?
.
Thumbs up?
.
On ARMv6 ( iPhone 3GS and earlier), turning thumbs off, used to yield a dramatic performance increase on floating point calculations. Well not anymore. Put in a better way, there aren’t many ARMv6 devices around anymore ( ~3.5%? )
.
I tested an FP intensive task and observed a marginal improvement ( maybe less than 3%) with an iPad2.
.
-/O0/O1/O2/O3/Os
.
I was curious to see whether -Os (smallest/fastest) would perform significantly worse than -O3. It doesn’t – well not in my case. I observe about 4% speed increase between -O2/O3/Os and -O0 (-O0 : no optimizations).
.
I tested this on a graphic application running OpenGL ES 1.x . I measure the performance increase by looking at % usage per frame for non GPU tasks (In my case, the CPU/GPU balance is fairly even).
.
Measuring performance
.
Running with GDB or Instruments turned on will slow down your app; a crude way to measure release-grade performance may be to stick a frame rate meter right on top of your UI (now that is common practice, right?) and check with an unplugged device.
This won’t tell you where your overheads are going. For this, I still find Instruments an invaluable help.
.
I’m looking at averaging performance over one or several game sessions to get useful figures. Formal test cases can help solving specific performance issues – keeping in mind that optimization issues and level design strongly interact.
.
Stuff that looks interesting
.
.
Conclusion
.
With a background in Java and other virtual languages, I find that playing with compile time settings is, overall, almost a waste of time. Sadly (cf. the thumbs episode) it isn’t always quite so.
.
Wasting a day on belittling tweaks and flags from time to time isn’t altogether bad.

In case you’re not getting it from the title, I may not have learned anything substantial while writing this article. I just created yet another static library, tried linking it against another, and there goes XCode (build 4C104) NYAAAAN.

Since I used to write my own coding tools, I bloody well knew how to compile a java (sigh) program on the command line. I also wrote tools in such a way that they found jars in myland (comforting it was, however restrictive).

XCode workspaces are meant to improve the (sodding) developer experience when dealing with complex projects. Don’t get me wrong, I’m all for wizardry – Clearly, however an ignoramus ex machina is usually at work when I try stringing libraries together.

Useful notes?

Check this article from the Carbon Five blog for useful notes about linking static libraries. I’ll provide a new article on this topic at some point. I just wasted a couple of hours clear because this stuff is having its way. Here’s a lame summary of what I found:

  • Like pixies, build issues appear when you least expect it. Sadly they also disappear mysteriously as soon as you try getting to the root of the problem.
  • You can fix build issues by fiddling:
    • If at first you don’t succeed…
    • Don’t assume a library won’t get found because it appears ‘marked red’
    • Don’t assume a library will get found because it doesn’t appear ‘marked red’
    • Assume duplicate items can easily arise when adding libraries to a project.
    • Remove duplicate items whenever you stumble on them.
    • Play with the ‘Location’ combo (relative to group, relative to project etc…).
  • Don’t be afraid to check build logs. The paths often look infuriatingly convoluted, but while you’re poking around to find what’s really gone wrong, the problem is getting itself fixed (and at least you’re kidding yourself that you have something to do with it).
Need I say this is only the beginning of an idiotic battle? I will get to the bottom of this (if there is one).

The case

Okay, let’s say I made a library called ‘foobar’ (I actually did)

Initially, under ‘products’, libfoobar.a is red. In my file inspector the location is relative to build products (and cannot be changed)

For sake, build (don’t add any source file) and check what happens: nothing. Well that’s good, innit? No source files, no library.

Now, add a source file and rebuild. Check the messages window. We now have 3 items: Precompile…, Compile… and Libtool.

Under Libtool, we can see the intended location of the output:

-o /Users/johndoe/Library/Developer/Xcode/DerivedData/workspace-fdpyddpbphekphdqtzzutyjcagva/Build/Products/Debug-iphonesimulator/libfoobar.a

Feeling paranoid (would Libtool ever lie?) we can verify that the library has been created by browsing to this path (Finder is gracefully setup to overlook whatever path this stuff goes under).

But the item still appears ‘marked red’ under ‘Products’. Suspicious?

Can’t copy libfoobar.a?

Now let’s try linking foobar against a target inside another project (same workspace, right?). Open project settings (yea click on the project in the navigator), go to Build Phases > Link Binary With Libraries, hit the [+] button. A cute browser now appears with libfoobar.a under Workspace (or whatever your workspace is named). Select libfoobar.a, press return and… *build*.

Mind, this operation adds an item to your project, now also visible in the navigator (libfoobar.a, still marked red).

If there is an error, it may look like this:

error: /Users/johndoe/Library/Developer/Xcode/DerivedData/workspace-fdpyddpbphekphdqtzzutyjcagva/Build/Products/Debug-iphonesimulator/../../../../../../../../svn/components/base/build/Release-iphoneos/libfoobar.a: No such file or directory
Command builtin-copy failed with exit code 1

What’s going on here? Let’s go and check the added library item inside our project (select the item and make check sure the inspector is open). I see this:

Location: relative to group

Path: ../../../components/base/build/Release-iphoneos/libfoobar.a

Clearly not right (yet, consistent with the failed copy operation)

Fancy patching the path manually?

Location: relative to build products

Path: ../Debug-iphonesimulator/libfoobar.a

Worked for me, but not right away. Digging around I found yet another duplicate of the same library reference (marked red!). Removed it, cleared (what appears to be) the same item under ‘build phases’ and re-re-re-added it all over again (this time by checking the box I wanted in ‘target memberships’).

Not happy

I felt somewhat unconvinced (notably, I’m not happy with the Debug-iphonesimulator part of the relative-to-build-products-path – how’s that supposed to backfire?).

So I removed the library altogether and started over. Just repeated the initial steps going by the book… sadly it… worked.

Until next time.

We use blender to create game assets; we also use blender as level editor. Blend files are exported to our proprietary format using py scripts.

Exporting files manually is okay… to a point. Soon we find out not only that exporting files manually is a chore, it contributes to creating a beautiful mess.

How to automate the conversion? With XCode, it can be done using build rules. I’m covering the following points:

  • Build integration – add files to your build, automatically translate them using utilities and include the products in your app bundle.
  • Defining useful build rules.
  • Invoking blender from the command line.
  • Modifying blender scripts so that they won’t crash in a windowless environment.
  • Signaling export errors (notably, errors generated by a script running inside blender) using XCode.

While we have a mixed workflow (Blender 2.49 / Blender 2.61), we’re dominantly using 2.49b, so I started with the older version. I’ll be covering 2.61… later.

I used the following articles as a starting point:

This article is not concise; I’ll summarize my findings in a coming article.

The experiment

The first step is to configure build rules to process *.blend files I guess, so in a first approximation I used this:

  • Source files with names matching *.blend
  • Custom Script: /Applications/blender/blender.app/Contents/MacOS/blender ${INPUT_FILE_PATH} (bad)
  • Output Files:
    ${TARGET_BUILD_DIR}/../../${INPUT_FILE_BASE}.scg (bad)
I just thought it would be interesting to try this way although it’s obviously not complete. And indeed, it was.
First surprise, Blender opens in a window. Good for testing (notably because it’s displaying errors already; for a nice smooth build I’ll want to disable this).
Both Blender & XCode are displaying messages in the XCode errors & warnings panel, which is nice.
The initial problem is path related. The file won’t open because spaces in the path are incorrectly interpreted. So the correct form to open the blend is like this:
  • /Applications/blender/blender.app/Contents/MacOS/blender “${INPUT_FILE_PATH}” (good)
Running the script is the next obvious step. For now blender just opens and idles happily until I quit. So I modify my command, like this:
  • /Applications/blender/blender.app/Contents/MacOS/blender -b “${INPUT_FILE_PATH}” -P qwagga.py (bad)

-b to tell blender to ‘render in background’. -P qwagga.py is the wishful step to run my script hoping that scripts are being run from the blender scripts folder. qwagga.py is my export script. There will be a problem with that but let’s try anyway.

Indeed, -b prevents the blender window from opening. Blender is reporting (via XCode) that qwagga.py wasn’t found. Interestingly it would appear that -P could also refer a script included inside the blend file (text node). For now, however, I’ll just provide the full path.

  • /Applications/blender/blender.app/Contents/MacOS/blender -b “${INPUT_FILE_PATH}” -P /Applications/blender/blender.app/Contents/MacOS/.blender/scripts/qwagga.py (good)

Yea this works. As you’d expect since the exporter is trying to pop a file selection panel the script crashes with a fairly explicit message:

File “/Applications/blender/blender.app/Contents/MacOS/.blender/scripts/qwagga.py”, line 213, in <module>   Blender.Window.FileSelector(exportAssets,’Export to File’,PATH)
RuntimeError: the file/image selector is not available in background mode

This error can be patched easily:

    try:
            Blender.Window.FileSelector(exportAssets,’Export to File’,PATH)
    except RuntimeError:
            OUTPUT=Blender.sys.makename(ext=’.scg’)
    exportAssets(OUTPUT)

Telling XCode where the ouptut is located turned out to be a nag. The docs aren’t overly explicit about what environment variables can be used. As it turn out, however, whenever running a custom shell script (like, what we are doing now) the messages window displays a whole batch of setenv commands. That is all we need to know. I adjusted my entry in output files accordingly:

  • ${INPUT_FILE_DIR}/${INPUT_FILE_BASE}.scg

Immediately after I got this work, it seemed there was ‘a bit of weirdness’ in XCode (an error I couldn’t make sense of). For whatever reason this cleared after cleaning the build and re-saving my blend file.

It appears that each file is getting processed/converted whenever building (even if the input file hasn’t changed). I’m a bit worried about how this will turn when I have 200 blend files linked in this way – since blender loads almost instantly I’ll just wait and see.

Single input => multiple output

Now, the above looks quite useful but there are issues:

  • My py scripts don’t output just one file.
  • In fact I have two py scripts used in blender. One outputs a single file, the other outputs a whole lot of files. The names of the files match object names in blender
I would hope that both Blender and XCode can handle this situation gracefully.
For proof of concept, I just hacked a different output file path and created a file hierarchy looking like this :
  • blends
    • car_park.blend
    • build
      • car_park.blz
        • foo.rtf
        • bar.rtf
I added a ‘build’ folder under blends because we don’t want to have output files sprawling around. Then the question is to know whether car_park.blz will be exported along with all its content (for sake, foo.rtf, bar.rtf). The answer is YES.
Not perfect
Using this method I don’t know how to modify the location of the output in the resulting package. Before I would link a ‘blue folder’, so all my media 3D assets would go under [package]/3D/. Controlling the structure of this package is useful in various ways. 
We just demonstrated that we can link a whole folder given a single input file; this means we could input a config file describing which blend files we want to translate and how; then we’d run blender once , translate all the files and producing the correct output.
In the end I think this will turn out to be a better solution, notably because it will avoid translating the same files other and other even if they weren’t modified. Scene files process fast; not the case with files containing large meshes and animations.

For whatever reason, the next steps turned out to be troublesome.
My second py script was symlinked. I didn’t think it would matter (strictly speaking, still don’t think it does) but I found it hard to get the script to refresh (meaning, I would edit the py script and run the build again, find the same error as previously. And again. And again.

An error in the scene export script caused the wrong path to be used.

Passing several scripts on blender command line (-P foo.py -P bar.py) doesn’t work so I ended calling blender twice which is a lesser evil. Maybe.

Complications

  1. I have a mixed workflow using both Blender 2.4x and Blender 2.5x. In theory this means that I would have to go through the motions again, finding out how I can run 2.5x from the command line (likely as not, it won’t work in the same way).
  2. Scene files contain level edits and assets. Asset files do not contain level edits. Both blender 2.4x and 2.5x files are just annotated as *.blend. How do I know which version of blender to run and which scripts to call?

The answer to (1) will be simple for now. Don’t do that. I mainly use 2.4x; the only 2.5x file that really matters so far is the one containing the PC mesh and animations. So I’ll just pass and use the manual workflow for that.

The answer two question (2) will be equally simple. Use a convention. For starters:

  • level file: level.s.blend
  • asset file: asset.a.blend

( and likewise when I get 2.5x scripts working, different conventions may be used )

This is a bit lame. Maybe a better convention might be something like:

  • *2.4x/scenes*.blend
  • *2.5x/assets*.blend

Works – although the build rule does say something like “files with name matching”, the matching rule actually uses the path.

A nag here is that I don’t want to have 2 or 4 ‘build’ folders for my assets. Just one folder (that I can trash as a quick and dirty way to make sure no out of data lingers in the system) would be best. This should be straightforward, matching the output like so:

  • ${INPUT_FILE_DIR}/../../build/${INPUT_FILE_BASE}.blz

Conflating the output from various files (e.g. 2.4x/assets/foo.blend and 2.4x/scenes/foo.blend ) is possible. Collisions are unlikely, however, so I’ll keep it this way.

Polish?

Silencing the output?

When running blender directly from the desktop, getting console output to show up can be tricky. XCode surely doesn’t have this problem. All debug output appears in the messages window. Too much of it, in fact. Worse, something forces the item matching the translation script to expand.

No luck. I was assuming (maybe wrongly) that blender output is displayed because it is sent to stderr. Either way standing messages to stderr or stdout will yield the same result. Until further notice if I want to avoid script messages blotching the output, I need to disable debug output manually.

Displaying errors during the export process

An issue serious enough that putting it under ‘polish’ may be foolish : how to ensure that XCode will alert us when export errors occur? And by the way, what is an export error?

I tried my luck again, this time by looking at the unit test framework output. I thought XCode may be detecting a simple pattern to identify errors, and it is. The pattern is something like this:

  • error: description of error

The space after the colon in error : is required. In python (~2.7), use:

print (“error: something went wrong”)

And XCode will detect the error and display a flamboyant exclamation mark. Neat.

This won’t stop the build from completing but a red exclamation mark is difficult to ignore so it will be enough.

What is an error, then?

A py script failing while running inside blender does NOT cause an error to appear in XCode. This is because blender returns with exit code ‘OK’ regardless. Now that we know how to signal errors to XCode, we can insert validation related errors. But we also want to report an error when our script fails unexpectedly:

    try:
        exportAssets(PATH)
    except:
        print “error: in blender script: “, sys.exc_info()[0], “(see messages window)”
        raise

raise causes the error to propagate, allowing detailed information to appear in the log.

caveat: If the script contains a syntax error, it won’t run. In this case blender will report the error and exit, however we cannot alert XCode since the script didn’t even start running.

(how to cope with) limitations

Limitations with the approach I described derive from the atomicity of the conversion. It has to be done like this:

input file => output file or directory

For level edits, it is fine.

I see a problem when several files want to output to the same deployment directory. For example, it often makes sense to define actors as shared assets (same creatures used in different levels). Although it is a bit messy, it is often convenient to keep several actors in the same blend file. But then what we’ll end up doing is this:

  • myCreatures1 => targetFolderA
  • myCreatures2 => targetFolderA

This is bad in at least two ways:

  • We should cleanup targetFolderA before exporting (to avoid outdated files lingering in the build)
  • When exporting myCreatures2, we’ll end up copying over again data output by myCreatures1.
If we export to separate folders (myCreatures1Out/, myCreatures2Out/) the deployment structure becomes inconvenient. We may end up manually adding every output folder to the path used by a loader, and if we do so retrieving assets will take more time.
For shared assets, we have to choose. Either we start from a file listing all shared asset files and this becomes the input – processed by a special script – or we agree that each file will contain one actor / item. While the first solution is somewhat constraining it does help keeping things neat and requires less work.
So, do what?
Although not ideal, a solution is to modify the way the loader retrieves data blocks. I want to do this conservatively.
Before, a data block would be retrieved by creating a file name against the resource paths (e.g: rabbit => retrieve the mesh called rabbit.dbk)
After, a data block can also be retrieved by matching a file name against a path in a designated shared resource folder (this would have to be the bundle’s root I’m afraid) : rabbit => rabbit.blz/rabbit.dbk

Additionally, the name of the blend file for a shared resource should match the name of the data it contains.

Today, I ‘rescued’ my game project  by migrating all my sources to a sample project file. My original problem clearly came from the source project settings. Since this is my first serious project, I possibly messed with my project settings at some point, and as a result a couple of files refused to get into the main bundle.

A great thing about CocoaTouch development is that XCode provides pre-configured projects which (in the closed world of iSomething development) just work out of the box so we don’t need to learn about the many many gritty details. Unfortunately this can lead to panic when a configuration/build level problem occurs.

For this time, I used a solution that won’t get me an award, but saved my digging further into a nasty problem for which no documentation or online help seems to be available – in short, I duplicated a sample project (containing the code I couldn’t integrate with my old project) and moved all my project files (~130) over. Something you might want to try if you’re having odd build/project configuration issues.

Here’s the steps I followed:

  1. Duplicated a sample project.
  2. Put aside all project source files in a separate group (some stuff can’t be put aside because it can’t be moved, e.g smart groups, targets, frameworks…)
  3. Moved all groups (what looks like folders in XCode, but isn’t) from my old projects to the new project. I did that by dragging and dropping from the source project to the target project, checking the ‘copy as needed’ box at the top. I moved my groups into a new group to keep things tidy.
  4. Step (3) will normally fail with conflicting files. The simplest approach in this case  is probably to replace the conflicting files (in my case, MainWindow.xib and main.m)
  5. Deleted (or at least put aside) unused nib files from the sample project.
  6. Added the frameworks missing from the new project.
  7. Run a build to make sure everything builds OK.
  8. Run the app to make sure I’m running my app versus the sample.

At this point, the template project settings pointed at my application (because the real entry point is the main nib file, and that is a file I replaced).

Note that it may not be a good idea to try replicating the target/creating a new target, especially if you haven’t done that before. In my case, I just use the existing target. For now there’s a few names that don’t match my old project, but these names are cosmetic – no logical dependencies – so it’s OK.

A couple of caveats:

  • I ran my first test build before moving conflicting files over -  this worked because the files that conflict tend to be top level in the dependencies hierarchy. But after overriding the conflicting files, XCode  didn’t detect the change and kept reusing old images from the previous build. I deleted the build folder attached to the project and ran a fresh build.
  • Quite a few warnings appeared. Because of dynamic binding, sometimes XCode can’t detect what’s right or not. These warnings have a way to disappear when running the next builds, but then recur after moving files over to a new project. It’s better to avoid getting any warnings at all if possible. In this particular case, I just went through the list to make sure nothing evil leaked into the project while migrating files.

OK, no glory here, but if like me you got into iSomething dev. by chance, just want to be productive and don’t want to learn all the details of the build process and project config, this might help.