Informix box must be preconfigured and joined AD domain like in this my example for Solaris and MS AD. Installing the latest patches is strongly recommended as some related bugs were fixed recently in Solaris and Informix.

1. On any Domain Controller:
   • create a service account in AD, one per server/alias
   • run setspn -A <sso_alias>/<informix_server>.domain.com@DOMAIN.COM <informix_server>
   • run ktpass -princ <sso_alias>/<informix_server>.domain.com@DOMAIN.COM -mapuser <serv_acc>@DOMAIN.COM -crypto RC4-HMAC-NT -ptype KRB5_NT_PRINCIPAL -mapop set -pass <serv_acc_password> -out my.keytab
   • upload generated keytab file to Informix server
2. On the Informix box:
   • run ktutil and insert generated key to existing keys file:
     ktutil:  rkt /upload/my.keytab
ktutil:  wkt /etc/krb5/krb5.keytab
ktutil:  quit
   • run klist -e -k /etc/krb5/krb5.keytab to check keys file
   • create <informix_home>ids/etc/concsm.cfg file with one row like this:
     GSSCSM("/app/informix/ids/lib/csm/libixgss.so", "", "c=1,i=1")
   • add sso alias to Informix onconfig file
   • add sso alias to sqlhosts file:
     ssoalias         ontlitcp        hostname      1526   s=7,csm=(GSSCSM)
3. On all Windows workstations:
   • latest version of IBM Informix-Connect must be installed
   • create concsm.cfg file in the C:\Program Files\IBM\Informix\Connect\etc folder with one row like this:
     GSSCSM("client=C:\Program Files\IBM\Informix\Connect\lib\client\csm\igsss11a.dll", "", "c=1,i=1")
   • run setnet32 and describe server like on my screenshot, don’t forget specify options: s=7,csm=(GSSCSM)
   • test using ilogin or define ODBC source; leave username and password fields empty

To check AD accounts from Unix or debug Kerberos and SSO use the following tools:

• klist, ldapsearch, ldaplist, getent
• krb-diag

./swinstall

Script will collect installation data:
Installing TIBCO iProcess Engine version 11.0.2

Location, Identification and OS Accounts Menu

* ) Installation Directory : /export/home/tibco/tibco/iprocess
2 ) iProcess Engine Nodename : s-bpm01
3 ) iProcess Engine Licensee Name : TIBCO iPE 11.0.2 Install
4 ) iProcess Engine Background User Name : pro
5 ) iProcess Engine Administration User Name : swadmin
6 ) iProcess Engine User Group Name : staffwar

ORACLE Database Connection and Account Details

1 ) Oracle DB TNS Identifier : orcl
2 ) Oracle DB Administrator Name : system
3 ) Oracle DB Administrator Password : ********
4 ) iProcess Engine DB Schema Owner Name : swpro
5 ) iProcess Engine DB Schema Owner Password : staffpro1
6 ) iProcess Engine DB User Name : swuser
7 ) iProcess Engine DB User Password : swuser1
8 ) Data Tablespace Name : STAFFWAR
9 ) Temporary Tablespace Name : TEMP
10) Schema Sizing Configuration : Small

Display configuration summary and start installation:
==============================================
Configuration Summary
==============================================

General
===============================================
Install type: install (MASTER)
Version: 11.0.2
Target location: /export/home/tibco/tibco/iprocess
Licensee: TIBCO iPE 11.0.2 Install

iProcess Objects Server Version: 11.0.2
iProcess Objects Director Version: 11.0.2

Node Details
===============================================
Node name: s-bpm01
Client-Server RPC port: 391875

Environment Settings
===============================================
iProcess Engine User group: staffwar
iProcess Engine bkg. account: pro
iProcess Engine admin. account: swadmin

Optional Settings
===============================================
Autostart Server: Y
Passwords required for login: Y
Enable Prediction (Global): N
Enable Case Data Normalization: Y
Enable Activity Publishing: N
Configure iProcess E-Mail Plug-in: Y
Enable iProcess Objects Server: Y
Enable iProcess Objects Director: N
Install TIBCO Hawk 4.8.1: N
Enable Webdav write access: N

DataBase Settings
===============================================
Database Type: ORACLE
TNS Identifier: orcl
DBA Name: system
DB Schema Owner: swpro
DB User: swuser
Data Tablespace: STAFFWAR
Temp Tablespace: TEMP

The final step:
Your TIBCO iProcess Engine installation has now been configured as follows:

--------------------------------------------------------------------------------
Machine ID Machine Name Master Check Error Files Machine Comment
--------------------------------------------------------------------------------
1 S-BPM01 Y Y s-bpm01

Checking and setting file permissions ...

TIBCO iProcess Engine Installation Complete

Display engine password:
TIBCO iProcess Engine Password is:
********************************************
* 3BFD-7292-DBAF-A3E7-823D-4720-351E *
********************************************
Licensee Name is:
TIBCO iPE 11.0.2 Install
(The existing TIBCO iProcess Engine Password and Licensee Name may also be
displayed later by running 'swconfig').

Reminder:
All users of TIBCO iProcess Engine (Staffware) should have the
environment variable $SWDIR set to
/export/home/tibco/tibco/iprocess
before invoking or starting TIBCO iProcess Engine.

Installer will run the final check and complete:
TIBCO iProcess Engine Nodename ( s-bpm01 ) checked OK.
TIBCO iProcess Engine RPC Number ( 391875 ) checked OK.
TIBCO iProcess Engine service ports checked OK
TIBCO iProcess Engine process entries OK

Then I have to create this .profile for pro user:
SWDIR=/export/home/tibco/tibco/iprocess
export SWDIR
ORACLE_HOME=/export/home/oracle/product/10.2.0/db_1
ORACLE_SID=orcl
export ORACLE_HOME ORACLE_SID
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib:$SWDIR/libs
export LD_LIBRARY_PATH

To start iProcess engine:
su - pro
cd bin
./swstart -p
./swstart

Admin tool:
su - pro
cd util
./swadm

1. Synchronize the system clock with AD server
   domain ntp server(s) must be in /etc/inet/ntp.conf
   then restart ntp daemon svcadm restart /network/ntp
2. Solaris server must have a record in the DNS
3. Domain name and name servers (DNS servers) must be in /etc/resolv.conf
4. In the /etc/nsswitch.conf file dns and files must be specified for hosts and ipnodes
   ...
hosts: dns files
ipnodes: dns files
...
5. In the /etc/nodename and /etc/hostname.<nic> files host name must be specified only, not a fully qualified domain name
6. Run adjoin script. You can find it here. It will:
   • auto-detects the Active Directory domain controllers
   • creates a machine account (also called a Computer object) for the Solaris host in Active Directory and generates a random password for this account
   • configures the Solaris host as a Kerberos client of the Active Directory domain controller by using the /etc/krb5/krb5.conf file
   • configures the /etc/krb5/krb5.keytab file on the Solaris host by using the keys for the machine account (also called host credentials)

   Execute adjoin script with following options:
   ./adjoin -d <domain_name> -p <administrator_principal> -f -x
   where -f to delete any pre-existing computer account for this host and -x to debug output.

   If your domain if geographically distributed with a lot of domain controllers (DC), script can detect inappropriate controllers. Just before entering admin password, check prepared adjoin-krb5.conf.XXXXXX file in the /tmp folder and remove unnecessary controllers from it.

   Adjoin script can stop work with pkcs11_kernel.so syntax error on some SUN servers:
   + ./adjoin[859]: /usr/lib/security/$ISA/pkcs11_kernel.so:: syntax error
   Then all you need is just to temporary rename this file and execute adjoin again
   mv /usr/lib/security/$ISA/pkcs11_kernel.so /usr/lib/security/$ISA/pkcs11_kernel.so.orig
   when adjoin finished successfully, rename it back

7. Run ldapsearch and klist to check Kerberos
   ldapsearch -R -T -h dc1.xxxxxx.com -o authzid= -o mech=gssapi -b CN=Computers,DC=xxxxxx,DC=com -s sub cn=<computer_name>
klist
klist -e -k /etc/krb5/krb5.keytab
8. Enable dns client and cache daemons
   svcadm enable /network/dns/client
svcadm enable /system/name-service-cache
9. In the /etc/nsswitch.ldap file dns and files must be specified for hosts and ipnodes
   ...
hosts: dns files
ipnodes: dns files
...
10. Set up a server as a client of an LDAP. Execute ldapclient
    ldapclient -v manual \
-a credentialLevel=self \
-a authenticationMethod=sasl/gssapi \
-a defaultSearchBase=dc=xxxxxx,dc=com \
-a defaultSearchScope=sub \
-a domainName=xxxxxx.com \
-a defaultServerList="dc1.xxxxxx.com dc2.xxxxxx.com dc3.xxxxxx.com" \
-a attributeMap=passwd:gecos=cn \
-a attributeMap=passwd:homedirectory=unixHomeDirectory \
-a objectClassMap=group:posixGroup=group \
-a objectClassMap=passwd:posixAccount=user \
-a objectClassMap=shadow:shadowAccount=user \
-a serviceSearchDescriptor="passwd:ou=Accounts,ou=European office,dc=xxxxxx,dc=com?sub;ou=Accounts,ou=American Office,dc=xxxxxx,dc=com?sub" \
-a serviceSearchDescriptor=group:ou=Groups,dc=xxxxxx,dc=com?sub
    ldapclient should finish without errors. To check use ldapclient list
11. Edit the /etc/nsswitch.conf file: files and ldap must be specified for passwd and group only
    ...
passwd: files ldap
group: files ldap
hosts: dns files
ipnodes: dns files
networks: files
protocols: files
...
    remove ldap from everywhere else
12. Restart LDAP client
    svcadm restart /network/ldap/client
13. Add pam_krb5.so.1 in the /etc/pam.conf file
    ...
login auth sufficient pam_krb5.so.1
krlogin auth required pam_krb5.so.1
krsh auth required pam_krb5.so.1
ktelnet auth required pam_krb5.so.1
other auth sufficient pam_krb5.so.1
other account required pam_krb5.so.1
other password sufficient pam_krb5.so.1
...

To ensure that users could login on the host under their AD accounts, accounts in AD must have following additional attributes:
uid the same as sAMAccountName
uidNumber unique number
gidNumber number
unixHomeDirectory for example /tmp
loginShell for example /usr/bin/bash or /bin/false

To check it use getent or ldapsearch
getent passwd <uid>
ldapsearch -R -T -h dc1.xxxxxx.com -b "ou=Accounts,ou=American Office,dc=xxxxxx,dc=com" -o mech=gssapi -o authzid='' "uid=<uid>"

If you would like read more: link to SUN’s article “Using Kerberos to Authenticate a Solaris 10 OS LDAP Client With Microsoft Active Directory”.