Title: buildbot_setup_lin_win_mac.txt Subject: Setup buildbot master on linux and slaves on linux, windows, and osx. Author: John Stile ------------------------------------- Sections: 1. Master on linux (gentoo) - buildbot-0.7.5 2. Slave on linux (gentoo) - buildbot-0.7.5 - gcc-4.1.2 - commercial qt 3. Slave on windows (xp) - cygwin (buildbot,python,twisted,sshd) - VisualStuido2005 - commercial qt 4. Slave on osx - xcode - commercial qt - macports - osx 10.4 - vine ------------------------------------- Section 1. Master on linux (gentoo) #==================================== # Host Software #==================================== Standard Gentoo system # # Boot from knoppix # boot knoppix # # PCI BUS shows # 1. SATA: Intel 2801GB/GR (ICH7 Family) # Driver: CONFIG_SATA_AHCI=y # 2. Ethernet: by Intel 82573E/82573L # Driver: CONFIG_E1000=m # 3. VGA: Xabre Graphics Inc Volari Z7 (http://www.mjmwired.net/kernel/Documentation/fb/sisfb.txt) # Package: x11-drivers/xf86-video-xgi # lspci 00:00.0 Host bridge: Intel Corporation E7230 Memory Controller Hub (rev c0) 00:01.0 PCI bridge: Intel Corporation E7230 PCI Express Root Port (rev c0) 00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01) 00:1c.4 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 5 (rev 01) 00:1c.5 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 6 (rev 01) 00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #1 (rev 01) 00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #2 (rev 01) 00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #3 (rev 01) 00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI #4 (rev 01) 00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 01) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1) 00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01) 00:1f.2 IDE interface: Intel Corporation 82801GB/GR/GH (ICH7 Family) Serial ATA Storage Controller IDE (rev 01) 00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01) 0d:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet Controller (Copper) (rev 03) 0e:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller 0f:00.0 VGA compatible controller: XGI - Xabre Graphics Inc Volari Z7 # # partition disk # boot=200Mb,root=145Gb,swap=remaining # swap is huge because this is a build machine too. # fdisk /dev/sda n -> p -> 1 -> +200M n -> p -> 26 -> +145G n -> p -> 18993 -> t -> 3 -> 82 w # # format drive, and mount # mkfs.ext3 /dev/sda1 mkfs.reiserfs /dev/sda2 mount /dev/sda2 /mnt/sda2/ mkdir /mnt/sda2/boot mount /dev/sda1 /mnt/sda2/boot cd /mnt/sda1/ # # Download un-pack a stage3 into mounted drive # http://funtoo.org/linux/i686/funtoo-i686-2008.01.14/stage3-i686-2008.01.14.tar.bz2 tar -jxvpf stage3-i686-2008.01.14.tar.bz2 # # Prepare to chroot: some mounting # mount -o bind /sys sys mount -o bind /proc proc mount -o bind /dev dev # # Copy host's resolv.conf into chroot directory # cp /etc/resolv.conf /mnt/sda2/etc/ # # Copy kernel config over # cp /boot/config-* /mn/sda2/ # # Chroot # chroot /mnt/sda2/ /bin/bash --login # # Make some portage configs # cat >> /etc/portage/package.use <<'EOF' dev-util/buildbot doc irc mail test web dev-python/epydoc doc pdf sys-apps/pciutils -zlib EOF cat >> /etc/portage/package.keywords <<'EOF' gnome-extra/hal-device-manager sys-apps/hwinfo ~x86 sys-fs/sysfsutils ~x86 EOF # # Create make.conf # cat >> /etc/make.conf <<'EOF' CHOST="i686-pc-linux-gnu" CFLAGS="-march=prescott -O2 -pipe -fomit-frame-pointer" CXXFLAGS="${CFLAGS}" EOF # # Rebuild system and world # emerge --sync emerge system emerge system emerge world emerge world # # Create /etc/fstab # cat >> /etc/fstab <<'EOF' /dev/sda3 none swap sw 0 0 /dev/sda2 / reiserfs defaults,notail,noatime 0 1 /dev/sda1 /boot ext3 defaults,noatime 1 2 none /proc proc defaults 0 0 shm /dev/shm tmpfs nodev,nosuid,noexec 0 0 /dev/cdrom /mnt/cdrom iso9660 noauto,ro 0 0 EOF # # # Build a kernel # emerge genkernel gentoo-sources cd /usr/src/linux cp /config-* .config make oldconfig genkernel --menuconfig --no-clean all echo 'e1000' >> /etc/modules.autoload.d/kernel-2.6 ##################BEGIN KERNEL CONFIG################# # Note: Kernels >= 2.6.20, sata disks appear as hda with some SATA controllers. # The Intel 82801GB/GR/GH (ICH7 Family) using kernel moduel ata_piix is one of them. # To fix, Select the following kernel options # # REFERENCE: http://forums.gentoo.org/viewtopic-t-641789-highlight-piix.html # <*> ATA/ATAPI/MFM/RLL support ---> --- ATA/ATAPI/MFM/RLL support <*> Enhanced IDE/MFM/RLL disk/cdrom/tape/floppy support --- Please see Documentation/ide.txt for help/info on IDE drives [ ] Support for SATA (deprecated; conflicts with libata SATA driver) [ ] Use old disk-only driver on primary interface < > Include IDE/ATA-2 DISK support [*] Use multi-mode by default <*> Include IDE/ATAPI CDROM support < > Include IDE/ATAPI TAPE support (EXPERIMENTAL) < > Include IDE/ATAPI FLOPPY support < > SCSI emulation support [ ] IDE Taskfile Access [*] legacy /proc/ide/ support --- IDE chipset support/bugfixes < > generic/default IDE chipset support [ ] CMD640 chipset bugfix/support [ ] PCI IDE chipset support SCSI device support ---> < > RAID Transport Class --- SCSI device support < > SCSI target support [*] legacy /proc/scsi/ support --- SCSI support type (disk, tape, CD-ROM) <*> SCSI disk support < > SCSI tape support < > SCSI OnStream SC-x0 tape support < > SCSI CDROM support <*> SCSI generic support < > SCSI media changer support --- Some SCSI devices (e.g. CD jukebox) support multiple LUNs [ ] Probe all LUNs on each SCSI device [ ] Verbose SCSI error reporting (kernel size +=12K) [ ] SCSI logging facility [ ] Asynchronous SCSI scanning SCSI Transports ---> SCSI low-level drivers ---> <*> Serial ATA (prod) and Parallel ATA (experimental) drivers ---> --- Serial ATA (prod) and Parallel ATA (experimental) drivers <*> AHCI SATA support < > ServerWorks Frodo / Apple K2 SATA support <*> Intel ESB, ICH, PIIX3, PIIX4 PATA/SATA support < > Marvell SATA support (HIGHLY EXPERIMENTAL) < > NVIDIA SATA support < > Pacific Digital ADMA support < > Pacific Digital SATA QStor support # # When broken, dmesg looks like this # dmesg |grep ata libata version 2.21 loaded. ata_piix 0000:00:1f.2: version 2.12 ata_piix 0000:00:1f.2: MAP [ P0 P2 IDE IDE ] ata_piix 0000:00:1f.2: failed to request/iomap BARs for port 0 (errno=-16) ata_piix 0000:00:1f.2: failed to request/iomap BARs for port 1 (errno=-16) ata_piix 0000:00:1f.2: no available native port # # When Fixed, dmesg looks like this # dmesg |grep ata libata version 2.21 loaded. ata_piix 0000:00:1f.2: version 2.12 ata_piix 0000:00:1f.2: MAP [ P0 P2 IDE IDE ] scsi0 : ata_piix scsi1 : ata_piix ata1: SATA max UDMA/133 cmd 0x000101f0 ctl 0x000103f6 bmdma 0x000130b0 irq 14 ata2: PATA max UDMA/100 cmd 0x00010170 ctl 0x00010376 bmdma 0x000130b8 irq 15 ata1.00: ATA-7: ST3160815AS, 3.AAD, max UDMA/133 ata1.00: 312581808 sectors, multi 16: LBA48 NCQ (depth 0/32) ata1.00: configured for UDMA/133 ata2.00: ATAPI: CD-224E-N, 1.AA, max UDMA/33 ata2.00: configured for UDMA/33 ata1: soft resetting port ata1.00: configured for UDMA/133 ata1: EH complete # # Check kernel options # egrep -i 'piix|CONFIG_IDE=|CONFIG_ATA=' /usr/src/linux/.config CONFIG_ATA_PIIX=y # CONFIG_PATA_MPIIX is not set # CONFIG_PATA_OLDPIIX is not set CONFIG_I2C_PIIX4=m # # Also set these options # CONFIG_IDE=n CONFIG_ATA=y ##################END KERNEL CONFIG################# # # Install grub # emerge grub emerge vim cat >> /boot/grub/menu.lst <<'EOF' bon# timeout 30 default 0 fallback 1 # x86-2.6.23-gentoo-r6 title Gentoo-genkernel-x86-2.6.23-gentoo-r6 kernel (hd0,0)/kernel-genkernel-x86-2.6.23-gentoo-r6 ro real_root=/dev/sda2 initrd (hd0,0)/initramfs-genkernel-x86-2.6.23-gentoo-r6 EOF # # Exit chroot and install grub in MBR # exit grub-install --root-directory=/mnt/sda2/boot --rescan /dev/sda REBOOT emerge sys-process/cronbase emerge sys-process/dcron #==================================== # Install COMPONENT_1_REPOSITORY specific software #==================================== * QT for mac os x V4.2.2 #----------------------------------------------------- # QT for mac os x: #----------------------------------------------------- URLs: http://dist.trolltech.com//qt/4.2 Username: Password: Download qt-license-4.2.2 qt-mac-commercial-src-4.2.2.tar.gz qt-x11-commercial-src-4.2.2.tar.gz cd ~/ wget http://:@dist.trolltech.com//qt/4.2/qt-license-4.2.2 ln -s qt-license-4.2.2 $HOME/.qt-license sudo su cd /opt/ wget http://:@dist.trolltech.com//qt/4.2/qt-x11-commercial-src-4.2.2.tar.gz tar -zxvpf qt-x11-commercial-src-4.2.2.tar.gz cd /opt/qt-x11-commercial-src-4.2.2/ ln -s /home/jstile/qt-license-4.2.2 /root/.qt-license ./configure -prefix /opt/qt make mkdir /opt/qt make install exit #==================================== # Build COMPONENT_2_REPOSITORY/COMPONENT_3_REPOSITORY #==================================== As jstile: export QTDIR=/opt/qt export PATH=$QTDIR/bin:$PATH #---------------------------------------------- # Date used to create current build directory # Date=`date +%Y%m%d_%H%M%S` #---------------------------------------------- # Make build directory structure # mkdir -p ~/COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_1_REPOSITORY/trunk mkdir -p ~/COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_3_REPOSITORY/trunk mkdir -p ~/COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_2_REPOSITORY/trunk #---------------------------------------------- # Checkout sources from repository # Dirs="audiomodule ggnet global littleguys muscle protocols" for Dir in $Dirs; do svn co --username builder --password MyBuildPassword \ https://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/${Dir} \ ~/COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_1_REPOSITORY/trunk/${Dir} done svn co --username builder --password MyBuildPassword \ https://svn.stilen.com/COMPONENT_3_REPOSITORY/trunk \ ~/COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_3_REPOSITORY/trunk svn co --username builder --password MyBuildPassword \ https://svn.stilen.com/COMPONENT_2_REPOSITORY/trunk \ ~/COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_2_REPOSITORY/trunk #---------------------------------------------- # Get into position to build, # and let qmake create COMPONENT_3_REPOSITORY.xcodeproj # export QTDIR=/opt/qt export PATH=$QTDIR/bin:$PATH pushd ~/COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_2_REPOSITORY/trunk/Source qmake make #---------------------------------------------- # Run it export LD_LIBRARY_PATH=/opt/qt/lib ~/COMPONENT_2_REPOSITORY_20070522_091456/svn_COMPONENT_2_REPOSITORY/trunk/Source/ProgramUnix/COMPONENT_3_REPOSITORY ======================================================= BUILDBOT ------------------------------------------------------- #----------------------------------- # Install buildbot (just to try) #----------------------------------- # REFERENCE: Howto: http://agiletesting.blogspot.com/2006/02/continuous-integration-with-buildbot.html # REFERENCE: Howto: http://mousebender.wordpress.com/2006/05/26/installing-buildbot/ # REFERENCE: Home: http://buildbot.sourceforge.net/ # Install buildbot and its dependencies for a 'BUILD MASTER' # # Note: I had to run the install of buildbot from an X session, # due to some test it does before installing. # echo 'dev-util/buildbot doc irc mail test web' >> /etc/portage/package.use echo 'dev-python/epydoc doc pdf' >> /etc/portage/package.use emerge --update --newuse --deep -ta buildbot Calculating dependencies... done! [ebuild N ] dev-util/buildbot-0.7.5 USE="doc irc mail test web" [nomerge ] dev-util/buildbot-0.7.5 USE="doc irc mail test web" [ebuild N ] dev-python/twisted-words-0.4.0-r1 [ebuild N ] dev-python/twisted-mail-0.3.0 [ebuild N ] dev-python/twisted-names-0.3.0 [ebuild R ] dev-python/epydoc-2.1-r2 USE="doc* pdf*" [ebuild N ] app-text/tetex-3.0_p1-r3 USE="X -Xaw3d -doc -lesstif -motif -neXt -tk" [ebuild N ] net-libs/libwww-5.4.0-r7 USE="mysql ssl" # # Create build slave # useradd \ -c "Build Slave" \ -d /home/build \ -m \ -g users \ -G cron \ -s /bin/bash \ -p MyBuildPassword \ build # # Initialize the build slave directory # su - build mkdir COMPONENT_2_REPOSITORY_linux mkdir COMPONENT_2_REPOSITORY_linux/bot_scripts, buildbot create-slave /home/build/COMPONENT_2_REPOSITORY_linux localhost:9989 lnx-johns.stilen.com MyBuildPassword # updating existing installation # chdir /home/build/COMPONENT_2_REPOSITORY_linux # creating Makefile.sample # mkdir /home/build/COMPONENT_2_REPOSITORY_linux/info # Creating info/admin, you need to edit it appropriately # Creating info/host, you need to edit it appropriately # Please edit the files in /home/build/COMPONENT_2_REPOSITORY_linux/info appropriately. # buildslave configured in /home/build/COMPONENT_2_REPOSITORY_linux # # Fix permissions # chown -R buildmaster:users COMPONENT_2_REPOSITORY_win32 # # # echo 'Buildxp2: 32bit Windows buildbot Host' > /home/build/COMPONENT_2_REPOSITORY_linux/info/host echo 'Buildbot Admin ' > /home/build/COMPONENT_2_REPOSITORY_linux/info/admin ############################################### ## BEGIN: /home/build/COMPONENT_2_REPOSITORY_linux/buildbot.tac ############################################### from twisted.application import service from buildbot.slave.bot import BuildSlave basedir = r'/Users/build/COMPONENT_2_REPOSITORY_mac' host = 'lnx-johns.stilen.com' port = 9989 slavename = 'lnx-johns.stilen.com' passwd = 'MyBuildPassword' keepalive = 600 usepty = 1 umask = None application = service.Application('buildslave') s = BuildSlave(host, port, slavename, passwd, basedir, keepalive, usepty, umask=umask) s.setServiceParent(application) ############################################### ## END: /home/build/COMPONENT_2_REPOSITORY_linux/buildbot.tac ############################################### ###################################################################### # buildslave: # # START: # su build -c buildbot start /home/build/COMPONENT_2_REPOSITORY_linux # # STOP: # su build -c buildbot stop /home/build/COMPONENT_2_REPOSITORY_linux # # RE-LOAD: # su build -c buildbot sighup /home/build/COMPONENT_2_REPOSITORY_linux # ###################################################################### # # Create user named buildmaster # useradd \ -c "Build Master" \ -d /home/buildmaster \ -m \ -g users \ -G cron \ -s /bin/bash \ -p M@st3rBl@5ter \ buildmaster # # Password Psneumonic: masterblaster # # # Initialize the buildmaster directory # su - buildmaster mkdir /home/buildmaster/COMPONENT_2_REPOSITORY buildbot create-master /home/buildmaster/COMPONENT_2_REPOSITORY pushd /home/buildmaster/COMPONENT_2_REPOSITORY cp master.cfg.sample master.cfg popd # # MAKE THE FOLLOWING CHANGES IN THE master.cfg # vi COMPONENT_2_REPOSITORY/master.cfg # # Defining the project name and URL # Change: c['projectName'] = "COMPONENT_2_REPOSITORY" c['projectURL'] = "http://lnx-johns.stilen.com/COMPONENT_2_REPOSITORY" # # Defining the URL for the builbot status page # Change: c['buildbotURL'] = "http://lnx-johns.stilen.com:9000/" # # Defining the buildslaves # Change: c['bots'] = [("buildmac2.stilen.com", "MyBuildPassword")] # # Modify Scheduler method to include Periotic # Comment: # #from buildbot.scheduler import Scheduler # Add: from buildbot.scheduler import Scheduler, Periodic # # Configuring the schedulers # Commented out all the lines in that section and I added these lines: ##################################################### # We build and test every 24 hours # Defined a schedule of type: Periodic # Name of schedule: Every_24_hours # Will run a builder named: Eruopa_trunk # Periodicity of: 24*60*60 seconds # periodic = Periodic("every_24_hours", ["mac_COMPONENT_2_REPOSITORY_trunk"], 24*60*60) c['schedulers'] = [periodic] ##################################################### # # Configuring the build steps # # Search for c['builders'] # Comment out all lines # FROM: # cvsroot = ":pserver:anonymous@cvs.sourceforge.net:/cvsroot/buildbot" # THROUGH: # c['builders'] = [b1] # # Add our config: source = s(step.SVN, mode='update', baseURL='http://svn.stilen.com/COMPONENT_2_REPOSITORY', defaultBranch='trunk') from buildbot.steps import source, shell from buildbot.process import factory f = factory.BuildFactory() f.addStep(shell.ShellCommand, command=["mkdir, "-p" "svn_COMPONENT_1_REPOSITORY/trunk"]) f.addStep(shell.ShellCommand, command=["mkdir, "-p" "svn_COMPONENT_3_REPOSITORY/trunk"]) f.addStep(shell.ShellCommand, command=["mkdir, "-p" "svn_COMPONENT_2_REPOSITORY/trunk"]) f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/audiomodule") f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/ggnet") f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/global") f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/littleguys") f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/muscle") f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/protocols") f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_3_REPOSITORY/trunk/") f.addStep(source.SVN, svnurl="http://svn.stilen.com/COMPONENT_2_REPOSITORY/trunk/") f.addStep(shell.ShellCommand, command=["cd", "/svn_COMPONENT_2_REPOSITORY/trunk/Source"]) f.addStep(shell.ShellCommand, command=["qmake"]) f.addStep(shell.ShellCommand, command=["xcodebuild", "USE_HEADERMAP='NO'", "ARCHS='ppc i386'", "GCC_ENABLE_FIX_AND_CONTINUE='NO'"]) f.addStep(shell.ShellCommand, command=["MOD_PROJECT_XML='svn_COMPONENT_2_REPOSITORY/trunk/Source/COMPONENT_2_REPOSITORY_installer_script.xml'"]) f.addStep(shell.ShellCommand, command=["REVISION=`svn info COMPONENT_2_REPOSITORY_${Date}/svn_COMPONENT_2_REPOSITORY/trunk/ |grep Revision: |awk '{print $2}'`"]) f.addStep(shell.ShellCommand, command=["cd", "svn_COMPONENT_2_REPOSITORY/trunk/Source" ]) f.addStep(shell.ShellCommand, command=["/Users/build/installbuilder-4.2.0/bin/Builder.app/Contents/MacOS/installbuilder", "build", "/Users/build/bin/COMPONENT_2_REPOSITORY_installer_script.xml", "osx" ]) f.addStep(shell.ShellCommand, command=[zip -c "Installer_for_MyProgram-osx.zip" -b /Users/build/output/MyProgram-$REVISION-osx-installer.app ]) c['builders'] = [ {'name':'mac_COMPONENT_2_REPOSITORY_trunk', 'slavename':'buildmac2.stilen.com', 'builddir':'COMPONENT_2_REPOSITORY_mac', 'factory':f }, ] ###################################################################### # buildmaster: # # START: # su buildmaster -c 'buildbot start /home/buildmaster/COMPONENT_2_REPOSITORY' # # STOP: # su buildmaster -c 'buildbot stop /home/buildmaster/COMPONENT_2_REPOSITORY' # # RE-LOAD: # su buildmaster -c 'buildbot sighup /home/buildmaster/COMPONENT_2_REPOSITORY' # ###################################################################### ###################################################################### # buildslave MAC: # # START: # su - build -c 'buildbot start /Users/build/COMPONENT_2_REPOSITORY_mac' # # STOP: # su - build -c 'buildbot stop /Users/build/COMPONENT_2_REPOSITORY_mac' # # RE-LOAD: # su - build -c 'buildbot sighup /Users/build/COMPONENT_2_REPOSITORY_mac' # ###################################################################### ###################################################################### # buildslave LINUX: # # START: # su build -c 'buildbot start /home/build/COMPONENT_2_REPOSITORY_linux' # # STOP: # su build -c 'buildbot stop /home/build/COMPONENT_2_REPOSITORY_linux' # # RE-LOAD: # su build -c 'buildbot sighup /home/build/COMPONENT_2_REPOSITORY_linux' # ###################################################################### ------------------------------------- Section 2. Slave on linux (gentoo) ------------------------------------- Section 3. Slave on windows (xp) ################################################### # Administrator Password: MyRootPassword # Build Password: MyBuildPassword ################################################### # # NOTE: shutdown OR reboot from Remote Desktop # In Remote Desktop # Disconnect and Log Off are the only commands in Start menu. # The Shutdown command is not available on the Start menu. # # To shut down a remote computer from Remote Desktop: # press CTRL+ALT+END, and then click Shutdown. # Or, use the shutdown command at a command prompt: # shutdown -r -f 0 # ################################################### ##-------------------------- # Install XP # Windows updates # Enable "Remote Desktop" for user build # Tweek Windows configs. # Install Norton Antivirus # Install Veritas Backup Agent # Install cygwin # Activate ssh shell # Activate chron # Activate syslog # Install Subversion # Install Visual Stuido 2005 # Install QT # Rebuild QT with static libraries # Build in DOS from "QT 4.3.3 command prompt" # Build in cygiwn from bash script. # Build in cygwin over ssh connection. # Install buildbot ##-------------------------- ##-------------------------- # Steps to get build working # 1. Build from a "dos" shell 2. Build from a cygwin shell 2. Run buildbot (without actually building your software) 3. get buildbot to work for building your software 4. get buildbot running as a service ##-------------------------- # Windows updates (note fail & fix) Failed: KB892130 Download "MGADiag" from http://go.microsoft.com/fwlink/?linkid=52012 Activate activeX controls allowing MS to install what they want on your system This installed the patch Failed: KB928255 ##DIDNT WORK## System -> Advanced-> Performance-> Optimum Performance ##DIDNT WORK## start->run->%windir%\windowsupdate.log I had to install as a Domain Administrator Local Administrator didn't have rights to install patch. ##-------------------------- # Remote Desktop: Right click My comptuer select properties select Remote Under Remote Desktop Check box "Allow users to connect remotely to this computer" Click "Select Remote Users" On window that opens titled "Remote Desktop Users" Click "Add" Button. Enter remote users: ad.stilen.com\sjohns ad.stilen.com\johns build administrator ##-------------------------- # Windows Configuration Changes * Turned OFF all automatic updates * Turned ON firewall; Exceptions: BackupExec Remote Agent RDP SSH * Turned off all MS networking, deselect "Client for Microsoft networks" deselect "File and Printer Sharing for Microsoft Netowrks" deselect "QoS Packet Scheduler" select "Internet Protocol (TCP/IP)" * Created local account build #OLD NOTES# Created cltbld account * Turned on RDP; allowed cltbld and Administrator to connect * Turned performance settings: "Adjust for best performance" * Turned off screensavers * Set networking to DHCP (so, will have to assign an IP) * Set system wide variables: CYGWINBASE: C:\cygwin VCVARS: REFERENCE: ReferencePlatforms/Win32 http://wiki.mozilla.org/ReferencePlatforms/Win32 ##-------------------------- # Install Norton AntiVirus Browse to \\fa.ad.stilen.com\IT\Warez\Symantec Antivirus 10\CD1 Double click setup.exe Choose full install Choose unmanaged ##-------------------------- # Install Firefox (version 2.0.0.11) Brows to mozilla.org download firefox install ##-------------------------- # Install Veritas backup agent Remote Deskotp to buildxp Disable firewall Remote Desktop to Veritas backup servre (ben.ad.stilen.com) Launch Symantech Backup Exec (C:\Program Files\Symantec\Backup Exec\BkupExec.exe) Tools-> Install remote agent-> Click next Right click "Windows Remote Agent"->Add Remote Comptuer Active Directory->ad.stilen.com->buildxp2 Enter user: build Domain: buildxp2.ad.stilen.com Pass: MyBuildPassword Click Next Firewall rules See: http://seer.support.veritas.com/docs/270139.htm On Symantec Backup Exec Remote Install screen, Click "Remote Agent for Windows systems, Select "Install this fearture..." On "Publish Remote Agent Information" windows Check "Enable the Remote Agnet to publish information to the media server..." Names or IP Adderesses: BEN Click Next. Click Install. Click Next Click Finish. ##-------------------------- # Install cygwin (version 2.573.2.2) # Base Install Right click My comptuer select properties select Advanced select Environment variables System Variables Select New Variable Name: CYGWIN Variable Value: binmode tty ntsec Edit Path Append ; c:\cygwin\bin Brows to http://www.cygwin.com/ download setup.exe Run setup.exe Instll type: Install from the internet Root Directory: c:\cygwin Select local package dir: c:\cygwin_installfiles Conneciton type: Direct Connection Choose Download site: http://cygwin.elite-systems.org Admin: cron: Vixie's cron cygrunsrv: NT/W2k service initiatio shutdown: syslog-ng Archive unzip zip Audio Base Database Devel autoconf autoconf2.1 autoconf2.5 automake cvs gcc gdb subversion Doc xpdf Editor mc nedit vim Games Gnome Graphics Interpreters Libs Mail Math bc Mingw Net openssh openssl ping rsync Perl Publishing Python Numeric: Neumeric Python Module asciidoc: Text based document generatorion python: An interactive object-oriented scripting language python-pyrex: Compile native-code modules for Python from python-like syntax Security Shells ash bash bash completion rxvt System procps Utils for monitoring your system psmisc Utils for managing processes on your system util-linux Random collection of Linux Utils. Text Utils Web X11 + wget Allow icon on desktop #NOTE: I skipped this step because it messed-up permissions # # Run cygwin shortcut ( c:\cygwin\Cygwin.bat ) # In the shell run: # cp /etc/passwd /etc/passwd.orig # cp /etc/group /etc/group.orig # mkpasswd -l -d > /etc/passwd # exit # # Run cygwin shortcut ( c:\cygwin\Cygwin.bat ) # In the shell run: # mkgroup -l -d /etc/group # exit # Setup SSH Run cygwin shortcut ( c:\cygwin\Cygwin.bat ) In the shell run: ssh-host-config Overwrite existing /etc/ssh_config file? yes Overwrite existing /etc/sshd_config? yes Should privilege separation be used? no Do you want to install sshd as a service? yes Which value should the environment variable CYGWIN have when sshd starts? ntsec net start sshd (Service is called CYGWIN sshd) cygrunsrv -S sshd # Set firewall to allow port 22 Start->Settings->Control Pannel->Security Center ->Windows Firewall OR Start->Settings->Control Pannel->Windows Firewall Click "Exceptions" tab Click "Add Port" Name "SSH" Port Number "22" ssh-keygen -t dsa chmod 755 /home/build chmod 700 /home/build/.ssh touch /home/build/.ssh/authorized_keys2 chmod 400 /home/build/.ssh/authorized_keys2 # Remove access for everyone but the user cacls.exe .ssh/known-hosts cacles.exe .ssh/known-hosts /E /R "Everyone" cacles.exe .ssh/known-hosts /E cd .ssh scp jstile@192.168.60.151:~/.ssh/id_rsa.pub authorized_keys2 chmod 600 authorized_keys2 cacles authorized_keys2 cacles.exe authorized_keys2 /E /R "Everyone" cacles.exe authorized_keys2 /E /R "ad.stilen.com\Domain Users" ssh-host-config # Generating /etc/ssh_host_key # Generating /etc/ssh_host_rsa_key # Generating /etc/ssh_host_dsa_key # Generating /etc/ssh_config file # Privilege separation is set to yes by default since OpenSSH 3.3. # However, this requires a non-privileged account called 'sshd'. # For more info on privilege separation read /usr/share/doc/openssh/README.privsep. # Should # Copy over my ssh key rsync -avz -e ssh .ssh/id_dsa.pub build@buildxp2:~/.ssh/authorized_keys2 #----------------------- # Install cron and Run cygwin shortcut ( c:\cygwin\Cygwin.bat ) In the shell run: chmod +r /etc/passwd chmod +r /etc/group cron-config Do you want to install the cron daemon as a service? yes Enter the value of CYGWIN for the daemon: [ntsec smbntsec] ntsec Do ou want to the cron daemon to run as yourself? no Do you want to continue anyway? (yes/no) yes Do you want to start the cron daemon as a service now? (yes/no) Log Location: /var/log/cron.log crontab -e * * * * * "/home/build/test.sh" > /home/build/test.txt #----------------------- # Install syslog Run cygwin shortcut ( c:\cygwin\Cygwin.bat ) In the shell run: syslog-ng-config Creating default /etc/syslog-ng.conf file Do you want to install syslog-ng as service? yes cygrunsrv -S syslog-ng ##-------------------------- # Install Subversion ##-------------------------- http://subversion.tigris.org/ select download http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=91 Select svn-1.4.5-setup.exe double click svn-1.4.5-setup.exe and follow installer, select defaults. NOTE: Installer add System Environment variables: APR_ICONV_PATH="C:\Program Files\Subversion\iconv" reboot -r -f 0 ##-------------------------- # Install Visual Stuido 2005 ##-------------------------- Pop in the disk Brows to disk, double click setup.exe On "Visual Studio 2005 Setup" Click "Install Visual Studio 2005" Accept defauts Enter Product key: Start->Programs->Microsoft Visual Studio 2005->Microsoft Visual Studio 2005 Select to install invironment variables. ##-------------------------- # Install QT ##-------------------------- # Set system environment variable: Add to PATH: C:\Program Files\Microsoft Visual Studio 8\VC\bin Add to PATH: c:\Qt\4.3.3\bin # Download # Download: # qt-license-4.3.3 # qt-win-commercial-4.3.3-vs2005.exe http://dist.trolltech.com//qt/4.3/ Username: Password: # Run the windows installer, which will build the # shared library version of qt. Double click qt-win-commercial-4.3.3-vs2005.exe Click Run on the Securuity Warning Window, if it appears. Click Next on Welcome page. QT License: Licensee name: QT License key: Click Next License Agreement: Select "I accept the terms in the license Agreement" Click Next Choose Components Click Next Choose Install Location Destination Folder: c:\Qt\4.3.3 Click Next Choose Start Menu Folder Click Install Installation Complete Click Next Completing the QT Commercial 4.3.3 Setup Wizard Selected: Run Examples and Demos Selected: Show Documentation Click Finish # ##-------------------------------- # Rebuild QT with static libraries ##-------------------------------- # Start->Program Files->QT by Troll Tech v4.3.3 (VS2005)->QT 4.3.3 command prompt configure.exe -release -static nmake confclean nmake NOTE: Shortcut Properties for "QT 4.3.3 command prompt" TARGET: %COMSPEC% /k "C:\Qt\4.3.3\bin\qtvars.bat vsvars" Startin: C:\Qt\4.3.3 echo %COMSPEC% # contains: c:\WINDOWS\system32\cmd.exe ####################################### # Build MyProgram ##------------------------------------- # Build in DOS from "QT 4.3.3 command prompt" ##------------------------------------- cd c:\Documents and Settings\build mkdir test cd test mkdir svn_COMPONENT_3_REPOSITORY mkdir svn_COMPONENT_2_REPOSITORY mkdir svn_COMPONENT_1_REPOSITORY cd svn_COMPONENT_1_REPOSITORY mkdir trunk cd trunk mkdir audiomodule mkdir ggnet mkdir global mkdir littleguys mkdir muscle mkdir protocols cd ..\.. cd svn_COMPONENT_3_REPOSITORY svn co -q --username 'builder' --password 'MyBuildPassword' https://svn.stilen.com/COMPONENT_3_REPOSITORY/trunk cd .. cd svn_COMPONENT_2_REPOSITORY svn co --username 'builder' --password 'MyBuildPassword' https://svn.stilen.com/COMPONENT_2_REPOSITORY/trunk cd .. cd svn_COMPONENT_1_REPOSITORY/trunk svn co -q --username builder --password 'MyBuildPassword' https://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk/ cd ..\.. REM let qmake create COMPONENT_3_REPOSITORY.xcodeproj cd svn_COMPONENT_2_REPOSITORY\trunk\Source qmake REM Build devenv COMPONENT_2_REPOSITORY.sln /build RELEASE ##-------------------------- # Build from cygwin ##-------------------------- ########################## ### BEGIN BUILD SCRITP ### ########################## cat >> ~/bin/build_COMPONENT_2_REPOSITORY.bash <<'ENDBUILD' #!/bin/bash # # This script will build MyProgram # from repositories COMPONENT_2_REPOSITORY, COMPONENT_3_REPOSITORY, and COMPONENT_1_REPOSITORY. # Using MS Visual Studio 2005 # # # Make directory structure # mkdir current cd current mkdir svn_COMPONENT_3_REPOSITORY mkdir svn_COMPONENT_2_REPOSITORY mkdir svn_COMPONENT_1_REPOSITORY cd svn_COMPONENT_1_REPOSITORY mkdir trunk cd trunk mkdir audiomodule mkdir ggnet mkdir global mkdir littleguys mkdir muscle mkdir protocols cd ../.. # # Checkout # cd svn_COMPONENT_3_REPOSITORY svn co -q --username 'builder' --password 'MyBuildPassword' https://svn.stilen.com/COMPONENT_3_REPOSITORY/trunk cd .. cd svn_COMPONENT_2_REPOSITORY svn co -q --username 'builder' --password 'MyBuildPassword' https://svn.stilen.com/COMPONENT_2_REPOSITORY/trunk cd .. cd svn_COMPONENT_1_REPOSITORY/trunk svn co -q --username builder --password 'MyBuildPassword' https://svn.stilen.com/COMPONENT_1_REPOSITORY/trunk cd ../.. # # Launch DOS shell and build # cd svn_COMPONENT_2_REPOSITORY/trunk/Source echo %COMSPEC% /k "C:\Qt\4.3.3\bin\qtvars.bat vsvars" "devenv COMPONENT_2_REPOSITORY.sln /build RELEASE" /cygdrive/c/WINDOWS/system32/cmd.exe /C "C:\cygwin\home\build\bin\qtvars_build.bat vsvars" # # Back to cygwin bash shell # for packaging # rsync -avz ~/current/svn_COMPONENT_2_REPOSITORY/trunk/Source/Program/MyProgram.exe ~/ ENDBUILD ######################## ### END BUILD SCRITP ### ######################## ########################### ### BEGIN QTVARS SCRITP ### ########################### cat >> bin/qtvars_build.bat <<'ENDQTVARS' @echo off rem rem This file is generated by John rem echo Setting up a Qt environment... echo -- QTDIR set to C:\Qt\4.3.3 echo -- Added C:\Qt\4.3.3\bin to PATH echo -- QMAKESPEC set to win32-msvc2005 set QTDIR=C:\Qt\4.3.3 set PATH=C:\Qt\4.3.3\bin;%PATH% set QMAKESPEC=win32-msvc2005 if not "%1"=="vsvars" goto END call "C:\Program Files\Microsoft Visual Studio 8\\Common7\Tools\vsvars32.bat" :END if not "%1"=="vsstart" goto ENDSTARTVS call "C:\Program Files\Microsoft Visual Studio 8\\Common7\Tools\vsvars32.bat" devenv /useenv :ENDSTARTVS rem rem Start Build rem devenv COMPONENT_2_REPOSITORY.sln /build RELEASE ENDQTVARS ######################## ### END BUILD SCRITP ### ######################## chmod +X ~/bin/build_COMPONENT_2_REPOSITORY.bash ~/bin/build_COMPONENT_2_REPOSITORY.bash ##-------------------------- # Install buildbot In cygwin: cd /home/build: cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/tools/buildbot In cmd.exe DOS prompt : cd c:\cygwin\home\build\mozilla\tools\buildbot python setup.py build python setup.py install Edit ppath in C:\cygwin\home\build\mozilla\tools\buildbot\contrib\windows\buildbot.bat ##-------------------------- # Install Buildbot ##-------------------------- REFERENCE: ReferencePlatforms/BuildBot/Win32 http://wiki.mozilla.org/ReferencePlatforms/BuildBot/Win32 Howto buildbot on windows http://buildbot.net/trac/wiki/RunningBuildbotOnWindows Running Buildbot as a WinXP Service http://www.stolennotebook.com/anthony/2006/07/03/running-buildbot-as-a-windows-xp-service/ ssh buildxp2 -l build # Check PATHEXT for '.PY' env |grep PATHEXT # PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.PY # Install ZOPE wget http://www.zope.org/Products/ZopeInterface/3.3.0/zope.interface-3.3.0.tar.gz tar -xvpf zope.interface-3.3.0.tar.gz pushd zope.interface-3.3.0 python setup.py build python setup.py install popd # Install TWISTED wget http://tmrc.mit.edu/mirror/twisted/Twisted/2.5/Twisted-2.5.0.tar.bz2 tar -jxvpf Twisted-2.5.0.tar.bz2 pushd Twisted-2.5.0/ python setup.py build python setup.py install popd # TEST python /lib/python2.5/site-packages/twisted/scripts/trial.py -o -R twisted # Install Instal Buildbot wget http://downloads.sourceforge.net/buildbot/buildbot-0.7.6.tar.gz tar -zxvpf buildbot-0.7.6.tar.gz pushd buildbot-0.7.6 python setup.py install cd ~/ buildbot create-slave /home/build/COMPONENT_2_REPOSITORY_win32 localhost:9989 lnx-johns.stilen.com MyBuildPassword # mkdir /home/build/COMPONENT_2_REPOSITORY_win32 # chdir /home/build/COMPONENT_2_REPOSITORY_win32 # creating Makefile.sample # mkdir /home/build/COMPONENT_2_REPOSITORY_win32/info # Creating info/admin, you need to edit it appropriately # Creating info/host, you need to edit it appropriately # Please edit the files in /home/build/COMPONENT_2_REPOSITORY_win32/info appropriately. # buildslave configured in /home/build/COMPONENT_2_REPOSITORY_win32 # # Setup email and build host info. # echo 'Buildxp2: 32bit Windows buildbot Host' > /home/build/COMPONENT_2_REPOSITORY_win32/info/host echo 'Buildbot Admin ' > /home/build/COMPONENT_2_REPOSITORY_win32/info/admin cat >> ~/bin/buildslave_win32.bash <<'BUILDSLAVE_STARTUP' #!/bin/sh case "$1" in start) su - build -c 'buildbot start /home/build/COMPONENT_2_REPOSITORY_win32' ;; stop ) su - build -c 'buildbot stop /home/build/COMPONENT_2_REPOSITORY_win32' ;; restart) $0 stop $0 start ;; status) su - build -c 'buildbot statuslog /home/build/COMPONENT_2_REPOSITORY_win32' ;; *) echo "Usage: buildslave.bash [start|stop|restart|status] " exit ;; esac BUILDSLAVE_STARTUP chmod +x ~/bin/buildslave_win32.bash ~/bin/buildslave_win32.bash start Traceback (most recent call last): File "/bin/buildbot", line 4, in runner.run() File "/usr/lib/python2.5/site-packages/buildbot/scripts/runner.py", line 889, in run start(so) File "/usr/lib/python2.5/site-packages/buildbot/scripts/startup.py", line 66, in start os.chdir(config['basedir']) OSError: [Errno 2] No such file or directory: '/home/build/COMPONENT_2_REPOSITORY_linux' ----------------------------------------- https://connect.microsoft.com/VisualStudio/Downloads/DownloadDetails.aspx?DownloadID=3988&wa=wsignin1.0 Description FIX: You may receive a "Fatal error C1902" error message and a solution build may fail when you try to use the AT command or a scheduled task to automate a build of a C or C++ project in Visual Studio 2005 ------------------------------------------- TR suggests that I got through this site and set their env variables system wide http://resources.kairo.at/mozilla/build/make-win-tinderbox-vc8express.html http://support.microsoft.com/kb/310519 ------------------------------------------- Output from buildbot shows it can't find cmd.exe (launched from devenv.com) %PATH% before and after running devenv.com: %PATH% BEFORE C:\Qt\4.3.3\bin;C:\cygwin\bin;c:\Program Files\Microsoft Visual Studio 8\Common7\IDE %PATH% AFTER: C:\Qt\4.3.3\bin;C:\cygwin\bin;c:\Program Files\Microsoft Visual Studio 8\Common7\IDE ================================================ I can build from cygwin shell from rdesktop, AND ssh-without-pubkey-authenticaiton. I get that error: "fatal error C1902: Program database manager mismatch; please check your installation" if I build from cron, buildbot, or ssh-with-publickey-authenticaiton. Ref: http://www.bokebb.com/dev/english/1960/posts/196053880.shtml Many people have this problem for different reasons. 1. Version mismatch or missing: mspdbsrv.exe, mspdbcore.dll, and mspdb80.dll http://msdn2.microsoft.com/en-us/library/8y7hea02.aspx "This error usually indicates that mspdbsrv.exe or mspdbcore.dll are missing or have different versions from mspdb80.dll. Ensure matching versions of mspdbsrv.exe, mspdbcore.dll, and mspdb80.dll are installed on your system." 2. MSVC8 corrupt or a mspdb80.dll is in VC/Bin http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=81208&SiteID=1 3. mspdb80.dll is in system32 http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=10577&SiteID=1 4. I don't understand http://forums.microsoft.com/msdn/showpost.aspx?postid=120192&siteid=1 5. http://support.microsoft.com/kb/920770/EN-US/ THE REAL PROBLEM: This issue is caused because cygwin does not implement a full login process. If I: move .ssh out of the way, log-in via ssh, run the bash build script IT WORKS If I:login via rdesktop, launch cygwin shell, run the bash build script IT WORKS If I:Launch bash buld from cron Error: "fatal error C1902: Program database manager mismatch; please check your installation" If I:Launch bash buld from buildbot Error: "fatal error C1902: Program database manager mismatch; please check your installation" If I: og-in via ssh (using public key crypto), run the bash build script Error: "fatal error C1902: Program database manager mismatch; please check your installation" This issue is caused because cygwin does not implement a full login process. Cygwin does not make the necessary call to LsaLogonUser. As a result, the wrong SID is in the token as the primary user. According to filemon, the debug server calls into secur32, which suggests that it is doing interpretive access control. It finds the service's SID instead of the users SID in some slot in the token that it gets via RPC, and then everything goes downhill from there. In ssh shell that fails, USERNAME=SYSTEM, In ssh shell that works, USERNAME=build If I: og-in via ssh (using public key crypto), export USERNAME=build run the bash build script Error: "fatal error C1902: Program database manager mismatch; please check your installation" http://www.jetbrains.net/jira/browse/TW-1191 ======================================================= ------------------------------------- Section 4. Slave on osx (10.4) #------------------------------------ # Title: Setup Notes for buildmac2.stilen.com # For The COMPONENT_2_REPOSITORY Mac Autobuild System # By: John Stile # Start: Mon Apr 23 18:50:59 PDT 2007 # Update: Thu May 24 15:20:24 PDT 2007 # ---------------------------------- # REFERENCE: software requirements # http://wiki.lcscanada.com/wiki/COMPONENT_1_REPOSITORY/AutobuildServers # ---------------------------------- # Specifications: # Mac OSX 10.4 (tiger) # XCode Tools Version V2.4.1 # QT for mac os x V4.3.3 # Subversion V1.4.3 #==================================== # SOFTWARE STOREAGE: fs.stilen.com:/software/COMPONENT_1_REPOSITORY_autobuild_system_software/COMPONENT_2_REPOSITORY_buildmac_build_software #==================================== #==================================== # Installing the base os #==================================== Hold down c when booting with install disk 1 in cdrom drive. -> Select English -> Select Utilities->Disk Utility ->Select the 149.1Gb Hitachi drive ->Select Partition ->Format-> Mac OS Extended (Case-sensetive) -> Select Partition ->Close the dialog box. -> Select English License -> Agree to the license -> Select Destination Drive ->Select the 149.1Gb Hitachi drive -> Select Installtion -> Select Customize -> Deselect Printer drivers -> Deselect Additional Fonts -> Deselect Language Translations -> Select X11 -> Deselect All Bundled Software ->it spends an hour verifying the DVD then installs -> You can click 'Skip' ->Country->United States ->Don't transfer my data ->Select Keyboard->US ->Apple ID -> skip it ->Registration-> Cant skip ->A few more questions: ->Where is computer use->small business ->What best describes what you do->music ->Deselect "Stay in touch!..." ->Create Your Account ->Name: build ->Short name: build ->Password: MyBuildPassword ->Get the Full Mac Expericne ->I don't want to purchase .Mac right now. ->Try .Mac For Free ->I don't want to try .Mac now ->Your mac is ready ->Go Desktop starts ->Only Allow Update of Mac OSX Update Combined -> Restart computer #========================== # Remove unwanted software #========================== Go to Macintosh HD->Applications Toss stuff in the trash can: -> Remove programs iTunes, iCal, iChat, iWork, Mail, Address Book, ImageCapture, Quicktime Stickies, Sherlock, Preview, Chess -> Remove folders Documents, Movies, Music, #========================== # Disable energy Saver #========================== Go to apple menu -> System Preferences -> Energy Saver ->"Put computer to sleep when it is inactive for:" -> never ->"Put the display to sleep when the cmputer is inacive for:" -> never #=================== # Activate services #=================== Go to apple menu -> System Preferences -> Sharing Edit Host name, set to buildmac2 Enable Remote Login => ssh From console, create ssh keys: build$ ssh-keygen -t dsa # NOTE: settings stored in: /Users/build/.ssh/ From my desktop, setup ssh-key based authentication jstile@genx ~ $ rsync -avz -e ssh ~/.ssh/id_dsa.pub build@192.168.60.198:/Users/build/.ssh/authorized_keys2 jstile@genx ~ $ ssh build@192.168.60.198 # no passwd needed, success!!! vi /etc/sshd_config # Enabled xforwareding by uncommenting this line and change no to yes. X11Forwarding yes Restart Service "Remote Login" #=================================== # Add extra Software: MacPorts-1.4.0 #=================================== NOTE: Update Procedure: port selfupdate port -d sync portindex port --all clean all port upgrade installed #--------------------------------- # Install Vine (remote vnc access) #--------------------------------- Apple "Apple Remote Desktop" (vnc) doesn't work easily with Linux vncviewr client. Vine does the right thing Download Vine (http://sourceforge.net/projects/osxvnc/) ->Install, ->Configure Vine Server-> Startup->"Start server when the applicaiton launches" "Restart server if it terminates unexpectedly" Start System Server Sharing->"Allowy only one VNC connection at a time" "keep existing client if a new client tries to connect" Connection->"Display Number"-> "Auto" "Password"->MyBuildPassword From my desktop, connect over vnc vncviewer 192.168.60.198:5900 # connected. success!!! #------------------ # Install MacPorts #------------------ MacPorts allow you to install handy unix tools Go to http://www.macports.org/ ->Click Download ->Downloads Directory ->MacPorts-1.4.0/ ->MacPorts-1.4.0-10.4.dmg ->"MacPorts-1.4.0-10.4.dmg" contains an applicaiton..." ->Continue The gui installer starts "Introduction"->Contineu "Read Me"->Contineu "License"->Contineu->Agree "Select Destination"->Macintosh HD "Installation Type"->Continue "Install"->Enter password for build "Finish Up"->Close Done #----------------------------------- # Install Gui diff & GUI editor #----------------------------------- . ~/.bash_profile prot -d sync port install tkdif port install nedit port install vim port install wget port install coreutils #----------------------------------- # Install GUI for settingup init scripts # (used to run buildbot at startup ) #----------------------------------- port install Lingon #----------------------------------- # Start X windows from terminal #----------------------------------- open /Applications/Utilities/X11.app #----------------------------------- # Install buildbot (just to try) #----------------------------------- # REFERENCE: Howto: http://agiletesting.blogspot.com/2006/02/continuous-integration-with-buildbot.html # REFERENCE: Howto: http://mousebender.wordpress.com/2006/05/26/installing-buildbot/ # REFERENCE: Home: http://buildbot.sourceforge.net/ # Install buildbot and its dependencies: . ~/.bash_profile port install buildbot ---> Fetching python24 ---> Attempting to fetch Python-2.4.3.tar.bz2 from http://www.python.org//ftp/python/2.4.3/ ---> Verifying checksum(s) for python24 ---> Extracting python24 ---> Applying patches to python24 ---> Configuring python24 ---> Building python24 with target all ---> Staging python24 into destroot ---> Installing python24 2.4.3_1+darwin_8 ---> Activating python24 2.4.3_1+darwin_8 ---> Cleaning python24 ---> Fetching py-zopeinterface ---> Attempting to fetch zope.interface-3.3.0.tar.gz from http://www.zope.org/Products/ZopeInterface/3.3.0/ ---> Verifying checksum(s) for py-zopeinterface ---> Extracting py-zopeinterface ---> Configuring py-zopeinterface ---> Building py-zopeinterface with target build ---> Staging py-zopeinterface into destroot ---> Installing py-zopeinterface 3.3.0_0 ---> Activating py-zopeinterface 3.3.0_0 ---> Cleaning py-zopeinterface ---> Fetching py-twisted ---> Attempting to fetch Twisted-2.5.0.tar.bz2 from http://tmrc.mit.edu/mirror/twisted/Twisted/2.5/ ---> Verifying checksum(s) for py-twisted ---> Extracting py-twisted ---> Configuring py-twisted ---> Building py-twisted with target build ---> Staging py-twisted into destroot ---> Installing py-twisted 2.5.0_0 ---> Activating py-twisted 2.5.0_0 ---> Cleaning py-twisted ---> Fetching buildbot ---> Attempting to fetch buildbot-0.7.4.tar.gz from http://downloads.sourceforge.net/buildbot ---> Verifying checksum(s) for buildbot ---> Extracting buildbot ---> Configuring buildbot ---> Building buildbot with target build ---> Staging buildbot into destroot ---> Installing buildbot 0.7.4_0 ---> Activating buildbot 0.7.4_0 ---> Cleaning buildbot # Creat the buildmaster on linux ( See buildlinux2_setup_notes.txt ) # # The machine which triggers the build-and-test process # by sending commands to other machines known as the buildslaves. # The Slaves sent the results back to the master, # which displays them in a nice HTML format. # # # Prepare the clinet. # # The user acocunt: builder # HOME /Users/build/ su - build mkdir COMPONENT_2_REPOSITORY_mac mkdir COMPONENT_2_REPOSITORY_mac/bot_scripts, buildbot create-slave /Users/build/COMPONENT_2_REPOSITORY_mac localhost:9989 buildmac2.stilen.com MyBuildPassword # # updating existing installation # chdir /Users/build/COMPONENT_2_REPOSITORY_mac # creating Makefile.sample # mkdir /Users/build/COMPONENT_2_REPOSITORY_mac/info # Creating info/admin, you need to edit it appropriately # Creating info/host, you need to edit it appropriately # Please edit the files in /Users/build/COMPONENT_2_REPOSITORY_mac/info appropriately. # buildslave configured in /Users/build/COMPONENT_2_REPOSITORY_mac # ls -alF /Users/build/COMPONENT_2_REPOSITORY_mac/buildbot.tac ############################################### ## BEGIN: /Users/build/COMPONENT_2_REPOSITORY_mac/buildbot.tac ############################################### from twisted.application import service from buildbot.slave.bot import BuildSlave basedir = r'/Users/build/COMPONENT_2_REPOSITORY_mac' host = 'lnx-johns.stilen.com' port = 9989 slavename = 'buildmac2.stilen.com' passwd = 'MyBuildPassword' keepalive = 600 usepty = 1 umask = None application = service.Application('buildslave') s = BuildSlave(host, port, slavename, passwd, basedir, keepalive, usepty, umask=umask) s.setServiceParent(application) ############################################### ## END: /Users/build/COMPONENT_2_REPOSITORY_mac/buildbot.tac ############################################### #----------------------------------- # Autostart buildbot (just to try) #----------------------------------- ############################################### ## BEGIN: /Library/LaunchDaemons/net.buildbot.slave.compass2.plist ############################################### EnvironmentVariables PYTHONPATH /opt/local/lib/python2.4/site-packages QTDIR /Users/build/qt-mac-commercial-4.2.2 PATH ${QTDIR}/bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin:${PATH} DYLD_LIBRARY_PATH $QTDIR/lib:$DYLD_LIBRARY_PATH GroupName daemon KeepAlive SuccessfulExit Label net.buildbot.slave.compass2 OnDemand ProgramArguments /opt/local/bin/twistd --nodaemon --python=COMPONENT_2_REPOSITORY_mac/buildbot.tac --logfile=COMPONENT_2_REPOSITORY_mac/buildbot.log --prefix=net.buildbot.slave.compass2 --pidfile=/var/run/net.buildbot.slave.compass2.pid RunAtLoad UserName build WorkingDirectory /Users/build ############################################### ## END: /Library/LaunchDaemons/net.buildbot.slave.compass2.plist ############################################### # # Test: # 1) Loading *.plist, # 2) Start via launchd, # 3) Follow buildbot.log # launchctl load /Library/LaunchDaemons/net.buildbot.slave.compass2.plist launchctl list |grep net launchctl start net.buildbot.slave.compass2 tail -f /var/log/system.log # # # If needed stop and unload the buildbot .plist file # launchctl stop net.buildbot.slave.compass2 launchctl unload /Library/LaunchDaemons/net.buildbot.slave.compass2.plist # # Reboot to verify it will start # ############################################## # NOTES: # REFERENCE: http://buildbot.net/trac/wiki/UsingLaunchd # # NOTE: # There is a bug in hdiutil (used to make disk image) that causes # it to fail if the process that started the buildbot daemon ends. # e.g. if daemon is started from a shell and then you exit, hdiutil will fail. # # The solution is to start buldbot slave via launchd (OSX init system) # That is what this section does. # # The problem lies in hdiutil's use of RPC, and when the parent process disappers, things go south. # However, when buildbot is started via a plist, hdiutil runs fine. # # # This is the base command our plist file will exectue # # /opt/local/bin/twistd --nodaemon --python=COMPONENT_2_REPOSITORY_mac/buildbot.tac --logfile=COMPONENT_2_REPOSITORY_mac/buildbot.log # # # I found that I had to reboot after creating this .plist file # Because duing my testing, I had started too many slave processes # And the plist file kept restarting buldbot slave over and over. # # # # Create startup script using launchd method # #==================================== # Install COMPONENT_1_REPOSITORY specific software #==================================== REFER: http://wiki.lcscanada.com/wiki/COMPONENT_1_REPOSITORY/AutobuildServers Summery of updates: * XCode Tools Version V2.4.1 * QT for mac os x V4.3.3 * Subversion V1.4.3 * Bitrock installer #----------------------------------------------------- # Subversion command line tools from http://www.codingmonkeys.de/mbo/ #----------------------------------------------------- http://www.codingmonkeys.de/mbo/ ->"Download my Subversion 1.4.3 package" ->Wecome to Subversion Installer ->Continue ->Important Information ->Continue ->License ->Continue ->Agree ->Select a Destination ->Continue ->Easy install on "Macintosh HD" ->Install ->Enter build passwd ->Software successfully installed ->Close Done. Close all the windows. #----------------------------------------------------- # XCode Tools - C compiler & creates universl binaries #----------------------------------------------------- http://developer.apple.com ->Xcode Development Tools user: john@stilen.com pass: 1234ABCD # Current downloads is Xcode-2.4.1 xcode_2.4.1_8m19106936315.dmg CD Image (MacBinary) 923M October20,2006 ->Double click xcode_2.4.1_8m19106936315.dmg ->Double click XcodeTools.mpkg ->Welcome -> Continue ->License ->Continue->Agree ->Select Destination->MacintoshHD->continue ->Easy Install->Install ->Enter password for build Done. Close all the windows. # Update to Xcode 2.5 Developer Tools https://connect.apple.com/cgi-bin/WebObjects/MemberSite.woa/wa/getSoftware?bundleID=19907 user: john@bla.com pass: foobar ->License ->Continue->Agree ->Update Professional Profile ->Double click xcode25_8m2558_developerdvd.dmg ->Double click XcodeTools.mpkg ->Welcome -> Continue ->License ->Continue->Agree ->Select Destination->MacintoshHD->continue ->Easy Install->Install ->Enter password for build Done. Close all the windows. #----------------------------------------------------- # QT for mac os x: #----------------------------------------------------- # # Download: # qt-license-4.3.3 # qt-mac-commercial-src-4.3.3.tar.gz # cd ~build wget http://:@dist.trolltech.com//qt/4.3/qt-license-4.3.3 wget http://:@dist.trolltech.com//qt/4.3/qt-mac-commercial-src-4.3.3.tar.gz unlink .qt-license ln -s qt-license-4.3.3 .qt-license # # Unpack and enter directory # tar -xzvpf qt-mac-commercial-src-4.3.3.tar.gz cd qt-mac-commercial-src-4.3.3/ # # Build it # make confclean time ./configure \ -prefix ~/qt-mac-commercial-4.3.3 \ -debug-and-release \ -static \ -largefile \ -no-accessibility \ -no-sql-db2 \ -no-sql-ibase \ -no-sql-mysql \ -no-sql-oci \ -no-sql-odbc \ -no-sql-psql \ -no-sql-sqlite \ -no-sql-sqlite2 \ -no-sql-tds \ -qt3support \ -platform macx-g++ \ -no-nis \ -no-cups \ -universal Do you accept the terms of the Qt COMMERCIAL LICENSE AGREEMENT? yes # real 8m5.128s # user 6m39.826s # sys 1m41.367s time make # real 327m5.595s # user 142m29.390s # sys 37m18.325s time make install # # # #============================================= # Setup user build's environment. #============================================= # REFERENCE: http://wiki.lcscanada.com/wiki/Autobuild # # Become build # su - build # # Put subversion in the path # echo 'PATH=$PATH:/usr/local/bin:/opt/local/bin' >> .bash_profile . .bash_profile # # Remove Default Apple Directories # cd ~/ rm -rf Sites Pictures Documents Music Movies Public #---------------------------------------------- # Build COMPONENT_2_REPOSITORY: #---------------------------------------------- # # Create Script that can build COMPONENT_3_REPOSITORY.app # mkdir bin ############################ #### BEGIN SCRIPT ########## ############################ /Users/build/bin/build_COMPONENT_2_REPOSITORY.bash Script too big to fit here. Placed scritp in Repo ############################ #### END SCRIPT ############ ############################ chmod +x bin/build_COMPONENT_2_REPOSITORY.bash ./bin/build_COMPONENT_2_REPOSITORY.bash #---------------------------------------------- # NOTES on Xcode Things I 've tried to get this in there: SUCCESS: xcodebuild USE_HEADERMAP=NO SUCCESS: xcodebuild USE_HEADERMAP=NO ARCHS="ppc i386" FAILED: xcodebuild USE_SEPARATE_HEADERMAPS=YES FAILED: export USE_SEPARATE_HEADERMAPS=YES FAILED: xcodebuild OTHER_CFLAGS=-DUSE_HEADERMAP=NO REFERENCE: http://book.itzero.com/read/others/0602/Addison.Wesley.Step.into.Xcode.Mac.OS.X.Development.Jan.2006_html/0321334221/appBlev1sec1.html USE_HEADERMAP - If set to NO, Xcode follows the more expected practice of searching only project directories for quoted include files, at the expense of some performance. (In Xcode, gcc treats angle-bracket #includes the same as it treats quoted ones: It searches the project header directories for the named header and then the system directories.) In Xcode 2.1 and later, USE_SEPARATE_HEADERMAPS is the preferred solution. xcodebuild USE_HEADERMAP=NO ARCHS="ppc" #---------------------------------------------- # We decided not to create an installer package # simply distrubute a disk image # User drag-n-drop where they want it installed # uninstall is drag-n-drop into the trash #---------------------------------------------- # Notes on the pacakge manager. Package Maker: REFERENCES: Tutorial http://steven.bitsetters.com/articles/2007/05/09/xcode-packagemaker-tutorial/ Installing: http://developer.apple.com/tools/installerpolicy.html Xcode tools: http://developer.apple.com/tools/ Create directory: mkdir -p ~/mypakcage/Applicaitons Copy our App into ~/mypakcage/Applicaitons Open Macintosh HD Developer->Applicaitons->Utilities->PackageMaker.app Select: Single Package Projec Installer should contain: Add ~/mypakcage/Applicaitons/MyProgram.app Add documentation.pdf Add release-notes.txt Location in buid product svn_COMPONENT_2_REPOSITORY/trunk/Source/ProgramMacx/COMPONENT_3_REPOSITORY.app svn_COMPONENT_2_REPOSITORY/trunk/Source/Program/release-notes.txt svn_COMPONENT_2_REPOSITORY/trunk/Source/Program/documentation.pdf svn_COMPONENT_2_REPOSITORY/trunk/Source/Program/Resources -------------------------------------