Compiling x64 PySide using VS2010, Python 2.7.4 and Qt.4.8.4 for Windows

I am for the most part a 3D generalist and TD by profession. I enjoy doing tool development, but I try to avoid low level (i.e. C++) programming as much as possible in order to avoid the time consuming process of coding, compiling, restarting the software...ad infinitum.
This is why I use Python as much as possible, and this is also the reason I want to replace the .Net/win32-based menu renderer of our Qmenu for Softimage plugin with one based on Qt, and hence PySide. Coincidentally, there are a couple of precompiled packages of various flavors of PySide downloadable from several official and unofficial sites (like http://qt-project.org/wiki/PySide_Binaries_Windows, http://www.lfd.uci.edu/~gohlke/pythonlibs/ ).
However, due to a bug in the build scripts that has been fixed only recently, to this date none of them contains the shiboken Python module, which is the equivalent of the SIP module of PyQt, (which is for example also needed to get Steven Caron's PyQt for Softimage plugin up and running with PySide).
Even more suprisingly, none of these available packages has been compiled with Visual Studio 2010!
The latter is of importance since most major 3D packages specify VS2010 as the designated compiler for plugin development, including Softimage 2014 and Maya 2014, and mixing compilers is generally not advisable, if not prohibitive in most cases.

Note: The following guidelines are not meant to aide in compiling PySide for Maya, which requires a special version of Qt from Autodesk. The whole procedure has been discussed here by Cyrille Fauvel, in case this is what you are looking for.

Long story short: If you need x64 PySide compiled with the Microsoft VS2010 compiler against your preferred version of Python using the (presumably) latest versions of Qt including the shiboken.pyd Python module, you must build it yourself.

 The catch is that there are a couple of sites telling you how to do it, but none of them tells you the whole story. Most assume a certain existing setup or knowledge of the process, or employ a semi-automatic build- and install procedure that won't let you choose specific versions of the required packages (esp. Qt).
It took me several days to figure out the whole process, and I ended up using most of the information from the following two sites. Thanks and credits go to the individual creators and maintainers of these pages:

Building PySide:
https://pypi.python.org/pypi/PySide#installing-prerequisities

Building Qt:
http://www.holoborodko.com/pavel/2011/02/01/how-to-compile-qt-4-7-with-visual-studio-2010/

The following is a summary of those pages including explanations and experiences from my own build attempts. The build process essentially consists of two parts: compiling Qt and compiling PySide.

Compiling of Qt is, unfortunately, also required since there is no publicly available x64 binary version available anywhere that was built using VS2010, not even from here.
Depending on your own working speed and that of your computer (a fast raid or SSD really speeds things up) this can take half a day to a day, including setting up the required tools and the actual build times.

Prerequisites

It is assumed that you have Visual Studio 2010 already installed and that you know how to set up and change environment variables.
Python (ideally the version you want to compile PySide (in my case this was the installer version 2.7.4 for x64 from http://www.python.org/download/releases/2.7.4/) needs to be installed too.
Make sure that the Python installation path is registered with your PATH environment variable, and in case you have multiple versions of Python installed, that the path to the version you want to use is listed before any other Python folder in the PATH variable.

Note: PySide requires Python 2.6 or later and Qt 4.6 or later.

Building Qt

Install Perl if you are going to compile Qt version 4.8.0 or higher from
http://www.perl.org/get.html
That's right, you need to install yet another scripting language to compile Qt!
Whoever made the build scripts thought it was a smart idea to implement them using Perl :-/

Visual C++ 2010 contains all necessary SDKs for Qt compilation. However if you plan to use Qt with Phonon you need to install the DirectX SDK. The June 2010 version is the last official standalone DX SDK.

Download your desired version of Qt (as of this time of writing the latest 4.8.x version is 4.8.4) from
http://download.qt-project.org/official_releases/qt/4.8/4.8.4/qt-everywhere-opensource-src-4.8.4.zip

Download and extract the archive, then copy contents of the folder qt-everywhere-opensource-src-4.8.4 of this archive to the directory where you intend to install Qt. In my case this was C:\Qt\4.8.4.

That is: The folder C:\Qt\4.8.4 should now contain files and folders like \bin, \doc, configure.exe etc., not the "qt-everywhere-opensource-src-4.8.4" folder itself!

Set up environment variables. In case you chose different folders or software versions you might need to adjust the following lines accordingly. Add the following variables:

QTDIR=C:\Qt\4.8.4

QMAKESPEC=win32-msvc2010

Update the PATH variable to include %QTDIR%\bin

Download the latest version of jom from http://qt-project.org/wiki/jom

It will accelerate the build process by using all available cores at build time, more on this further down.

Extract jom files to C:\Qt\jom

Start Visual Studio 2010 x64 Command Prompt as an administrator. 

On Win7 this can be done by right-clicking Start -> Programs -> Microsoft Visual Studio 2010 -> Visual Studio Tools  -> Visual Studio x64 Win64 Command Prompt (2010) and choosing "Run as Administrator" from the popup menu.
On Win8 you can reach the shortcut by typing "Visual Studio" to get a list of app and shortcuts known to the system, it should list this particular shortcut to. Right-click it and choose to run as administrator from the bottom ribbon.

Run the following commands in it (every line is a different command. Type it, then press Enter):

>> cd c:\Qt\4.8.4

To compile both debug and release versions type or copy/paste:

C:\Qt\4.8.4>configure -debug-and-release -opensource -platform win32-msvc2010

I chose to only compile the release version like so:

C:\Qt\4.8.4>configure -release -opensource -platform win32-msvc2010 

Note: Don't get irritated by the -platform flag reading "win32-msvc2010". It simply means that the MS Visual Studio compiler will be used and does not specify the architecture (x64 or x32, which was in fact already chosen by starting the Visual Studio x64 command prompt above). In other words:  "-platform" is always "win32-msvc2010" for us in the context of this example, even for win64 builds.

This will take a few minutes, depending on your hardware. When it's done it should report back by printing:

"Qt is now configured for building. Just run nmake.

To reconfigure, run nmake confclean and configure."

Build Qt using jom

Jom is an nmake replacement for Qt compilation on multi-core CPUs. Its parameter -j N allows to setup the number of parallel processes for compilation. Larger is better, where N is the number of CPU cores you want to utilize for Qt compilation. The number of physical CPU cores is a good choice for N, e.g. 4 on a typical quad-core CPU.

Usually compilation takes about 10 hours on a single core, with jom and an SSD drive it took only a bit more than 1 hour on my machine (Windows 8, 64bits)

Still in the same command prompt, enter and execute the following:

C:\Qt\4.8.4>..\jom\jom.exe -j 4

You might want to use a different number in the end if you have a system with more or less physical CPU cores. 

Building PySide

Install Cmake

Install OpenSSL (optional)
This is optional, PySide will compile and run fine without it. Since I only intend to use the GUI libraries of Qt I skipped installing it.

Install latest "distribute" distribution into the folder where Python is installed.by downloading the distribute_setup.py script from  http://python-distribute.org/distribute_setup.py
Run it using the python interpreter using the command prompt we have still open from building Qt, or open a new one.
C:\>C:\Python27\python distribute_setup.py
It should print:
Building PySide distribution

Install the PySide setup scripts.
in case you have Git installed, you can clone the PySide setup scripts from git repository into a folder of your choice, e.g.  c:\pyside-setup like so:
C:\>git clone https://github.com/PySide/pyside-setup.git pyside-setup

Alternatively you can download them from https://github.com/PySide/pyside-setup/archive/master.zip
and uncompress the archive to a folder of your choice, e.g. c:\pyside-setup


Switch to the pyside-setup directory:
c:\> cd pyside-setup

Build the PySide windows installer:
C:\pyside-setup>C:\Python27\python.exe setup.py bdist_wininst --msvc-version=10.0 --qmake=c:\Qt\4.8.4\bin\qmake.exe

In case you have OpenSSL installed, the line can be appended with  "--openssl=c:\OpenSSL32bit\bin" , which gives us:
C:\pyside-setup>C:\Python27\python.exe setup.py bdist_wininst --msvc-version=10.0 --qmake=c:\Qt\4.8.4\bin\qmake.exe --openssl=c:\OpenSSL32bit\bin

This can take about an hour on modern hardware.

After the successful build, install the distribution with easy_install and run the post-install script:
C:\pyside-setup>C:\Python27\Scripts\easy_install dist\PySide-1.1.2.win32-py2.7.exe
C:\pyside-setup>C:\Python27\python.exe C:\Python27\Scripts\pyside_postinstall.py -install

After successfull install, the post-install script should print the following information to the console:


Generating file c:\Python27\qt.conf...
The PySide extensions were successfully installed.

This is it.

Cleanup (Optional)

Note: Do the following step after compiling PySide only, otherwise files required by the PySide build process will be deleted too!
After compilation the Qt folder is huge in size (approx. 3.5GB). One can safely shrink its size by deleting temporary files created during the process. Once compiled ,Qt doesn’t depend on these intermediate files – and they can be erased without affecting Qt functionality.

Run this from the command prompt:

>> cd  C:\Qt\jom
>> jom.exe clean

I wrote this article in hope that it is a useful time safer for others. In case it was useful to you, or if you found anything that's wrong, incomplete, misleading, needs additional workarounds, or if you know a more elegant solution, please feel free to leave a comment below.

Industry events in Austria and Germany: Pixel Vienna, Softimage Ubertage

Friends of mine are organizing the Vienna Siggraph Chapter aka "Pixel Vienna" again from Oct 05 - 07.2012.
It's a great event I visit every year, probably the biggest one in Austria besides the Prix Ars Electronika.

The schedule has just been posted, some interesting speakers again this year:
http://www.pixelvienna.com/7/program/schedule

Last years video can be seen here (including Eric Mootz demoing his ICE tools at ~1:40):
https://vimeo.com/49339044

Another event I will attend on Friday 28th of Sept 2012 are the "Softimage Übertage" in Siegen, Germany, organized by Oliver "Owei" Weingarten.
I've been there two years ago and met a lot of interesting people. You will never get closer to fellow Softimage users and developers alike in a single day so easily!
The final schedule has just been announced: http://www.softimage-uebertage.de/pages/line-up.php
Also, if you are considering to buy Vray for Softimage, Chaos group will give all attendees a 25% discount off the regular license price that day.
Some videos from last years events:
2011: Eric Mmootz demoing some of his new tools: https://vimeo.com/34378025
2010: Marcos Fajardo and Borja Morales showing Arnold for Softimage: https://vimeo.com/16155555

If ne1  intends to go to any of these and wants to meet, please feel free to drop me a line: stefan at keyvis dot at.

Open Source...everything?

A few days ago there was a post on Solid Angles Arnold beta mailing list about a Python-based early-stage Blender to Arnold connection. The result was a discussion about the impact of open source software on the 3D industry as a whole, which reminded me on  the moment I first read about the plans for a complete Blender overhaul (Python, Qt, architecture revamp, etc) about two years ago. I thought, that this would change the game in the long run for everybody, if they can deliver.


Meanwhile they have delivered, and it's effects begin to show. More and more people have started using Blender, it's even being used in game production of some mid-sized game companies.
As a life-long user of commercial 3D Software (mostly 3dsmax, Maya, and Softimage), I must say it makes sense. Think about it for a second - it's every larger companies dream to have a scalable architecture with full access to every single line of code! One of the main reasons companies are still struggling with Maya ia because it is very open. You can change almost any aspect of the software and work around issues when something's not working (as it is frequently the case), let alone those who write large parts of their software themselves from the start.
You can do the same with Blender, but even more broadly and low-level because you have access to every line of code. If somethings broken or needs an improvement, you can fix or improve it immediately. Sure, not ye average 3D dude from next door, but companies can. That's how Maya got big too. It started in the companies doing feature production, not with single man shows. After that, everybody went: "Oh, I want to use that Software they did Stuart Little with". For those studios it won't make such a big difference if Blender still has missing features, they will implement them themselves, with and on an architecture they have full access too 24/7/365, without the need to wait for the annual or bi-annual update or bug fix, or an Autodesk technician to implement custom code into their private branch of the software.


I dare to say this:

1. In three years time, Blender will be among the top 3 most widely used 3D apps on the market.
2. In 3 to 5 years, we might not all be using Blender, but we will all be using open source 3D software as our main 3D application.

Yes, sooner or later, Maya, Softimage, Max, Modo, Lightwave, C4D,... whatever they currently put price tags on, will be open sourced.
I don't know if that will also happen to CAD programs so quickly, if ever, but I'm almost 100% certain about DCC.
What's clearly noticable is that Blender is being developed faster than any other software atm. And not just feature wise, the software has seen radical changes of it's architecture on a scale only comparable to what has happened to Softimage over the last four years, only faster.  And those who contribute to Blender do it for mostly no money. How could a company, seeking to sell software for profit, ever hope to compete with this in the long run? I dont think they can, until they change their business model into something along the lines of Red Hat or Suse. That is: making custom solutions for paying customers (those who cannot afford three ore more developers full time), support fees, etc.

The support scheme is something that has been taking shape over the last years anyway, hence all those efforts by Autodesk to get customers to invest in subscription plans.
Sure, they could try to escape the open source competition for some time by providing more and more sophisticated high-level tools, cranking up support quality and stability of their products and update frequency,
but the high tech race is run for the 5% of the upper class customers who need a faster fluid solver, crowd simulation, etc.
For the remaining 95% of all users/customers, it's bread and butter features we keep hearing them asking for, like better polygon modeling tools, etc. But there's not much head room in those departments left,
until a certain level of maturity is reached, and then what? Sooner or later all DCC programs will have the same bread and butter features, and then it only comes down to stability and support/speed of bug fixes, flexibility, extendability, and that's exactly where open source software with a large enough user base like Blender, already shines today.

Personally, tbh, I wouldn't mind. The game will be different for those writing software, and easier for those using it. Open source Softimage any one? I'd not look at Blender for a second longer.

QMenu updated to 0.97

Download from http://code.google.com/p/keyvis-dev/downloads/list

Changes since 0.95

Code Changes:  

• Fixed QMenu rendering on Multi-Monitor setups.
• Fixed: Automatic code generation for ICE and Shading Nodes created and added to a menu interactively was broken and resulted in either no ICE or Shading Nodes added to an ICE/Render Tree at all when called from the menu, or to the wrong one.  If you have created menu items from selected ICE and Render Tree nodes, please delete and re-create them after the update. They won't do any harm, but won't work as expected either, and will prevent similarly named menu items from being created. (Menu Items created from the ICE or Render Tree menus will always first insert an already existing menu item, and only generate a new one when it could not find an existing one with a matching name name).
• Added QMenu awareness of differently named Face Robot View Manager ("frmviewmanager" instead of the usual "vm").
• QMEnu will query to user whether to save config file when closing Softimage even when menu items were only added via ICE and Render Tree views.
• Graceful handling of possible Softimage bug that prevents getting the View Managers "viewportundermouse" Attribute Value in user (i.e. non-factory) Layouts. Now does nothing since v0.96, used to crashed v0.95)

Config file changes:

• Added "Null" Class object awareness to the "Any Object" Context.
• Fixed Parameter editing menu items code so they also work with compound parameters that are not of the "Parameter" class (like scl, ori, pos ).

QMenu updated to 0.95

After quite a long pause I finally had some time to make QMenu fit for Softimage 2012 and add some new features and bug fixes along the way.
One feature I'm a little proud of is the ability to edit Menus on the fly in ICE Tree and Render Tree views.
Selected ICE or shading nodes can be added to pop-up menus without having to open the Configurator (which some find complicated to use). As a result, the user is given a means to quickly organize his most frequently used nodes in easy and quick to access menus. Searching for those nodes in the node repository gets reduced to a minimum.


Video:

QMenu Update 0.95 from keyvis digital imagery on Vimeo.


Download: http://code.google.com/p/keyvis-dev/downloads

Updated documentation and complete change list.: http://www.keyvis.at/?page_id=68

QMenu - a customizable Quad Menu system for Softimage

It's finally done. After almost two years of spare time spent on refinements and bug fixing we have released QMenu for Softimage open beta.

The result is an open and extendable Quad menu system similar to what you might know from 3dsMax, but it offers a lot more control via it's scripting functions. Yet, it is easy to customize existing, or create new, menus via the built-in configuration panel without scripting anything. All your configurations can be saved and loaded in an XML-based configuration file.

Download:
http://code.google.com/p/keyvis-dev/downloads/list

Here are some demo videos showing installation...


...and current feature set and configuration





For more information and documentation please visit http://www.keyvis.at/?page_id=68

Sproing -> keyvis

They say that becoming is more rewarding than being.
In this spirit, I decided it was time for a change and left my former position
as a Lead Artist at Sproing Interactive to found my own company together with my long-time friend a colleague Eugen Sares.

We both have 15 years experience in the field of CGI, with Eugen being more focused on artistic and design aspects, while I'm more technically oriented, which is a good match.

This decision didn't come easy as I have spent almost 8 years at different computer games companies (Similis and Rockstar Vienna, besides Sproing) doing game art from animation to modeling and texturing to writing tools for 3dsMax, Maya and Softimage in Python and Max Script, let alone the fact that one of the Sproing founders, Harald Riegler, is a good and old friend of mine. But sometimes you just have to make a small jump to move forward.

For the time being we'll concentrate on the things we do best - game art, architectural visualisations and TV commercials, as well as writing tools for us and whoever needs them.

If you read this and have work to do that is in some way related to above topics, don't hesitate to contact us.
For a list of references and example work please look at our portfolio page:

http://www.keyvis.at/?page_id=385

Note that the site is still under construction, especially  a lot of our game and animation related work is still not uploaded or ready for viewing - please request some demo material directly if you are interested.

Stefan