Apple Window Programming Guide Manuel Apple sur Fnac.com - Pour voir la liste complète des manuels APPLE, cliquez ici

 

 

TELECHARGER LE PDF sur :

https://developer.apple.com/library/mac/documentation//Cocoa/Conceptual/WinPanel/WinPanel.pdf

Commander un produit Apple sur Fnac.com

 

 

Voir également d'autres Guides et documentation APPLE :

Apple-TV_2nd_gen_Setup_Guide.pdf-manuel

Apple-Archives-and-Serializations-Programming-manuel

Apple-SafariWebContent.pdf-Guide-manuel

Apple-iTunes_ExtrasandiTunes_LPTestGuide1.1.pdf-manuel

Apple-Text-System-User-Interface-Layer-Programming-manuel

Apple-CocoaTextArchitecture.pdf-manuel

Apple-Key-Value-Observing-Programming-Guide-manuel

Apple-Location-Awareness-Programming-Guide-manuel

Apple-SharkUserGuide.pdf-manuel

Apple-drawingprintingios.pdf-manuel

Apple-QuickTime7_User_Guide.pdf-manuel

Apple-Event-Handling-Guide-for-iOS-manuel

Apple-ipod_nano_3rd_gen_features_guide.pdf-manuel

Apple-iTunes_VideoandAudio_Asset_Guide5.0.pdf-manuel

Apple-ARD3_AdminGuide.pdf-manuel

Apple-SafariWebContent.pdf-Guide-manue

Apple-iphone_3gs_finger_tips.pdf-manuel

Apple-InstrumentsUserGuide.pdf-manuel

Apple-Logic-Pro-9-TDM-Guide-manuel

Apple-macbook_air_users_guide.pdf-manuel

Apple-macbook_air-13-inch_mid-2012-qs_ta.pdf-manuel

Apple-AppStoreMarketingGuidelines-JP.pdf-Japon-manuel

Apple-macbook_pro_retina_qs_ta.pdf-manuel

Apple-ipad_user_guide_tu.pdf-manuel

Apple-ipad_user_guide_th.pdf-manuel

Apple-iphone_user_guide_gr.pdf-manuel

Apple-Nike_Plus_iPod_Sensor_UG_2A.pdf-manuel

Apple-ipad_manual_del_usuario.pdf-manuel

Apple-ipad_uzivatelska_prirucka.pdf-manuel

Apple-ipad_wifi_informations_importantes.pdf-manuel

Apple-Xsan_2_Admin_Guide_v2.3.pdf-manuel

Apple-macbook_pro-13-inch-late-2012-quick_start.pdf-manuel

Apple-CocoaDrawingGuide.pdf-manuel

Apple-Cryptographic-Services-Guide-manuel

Apple-Resource-Programming-Guide-manuel

AppleSafariVisualEffectsProgGuide.pdf-manuel

/Apple-WorkingWithUSB.pdf-manuel

Apple-macbook_pro-retina-mid-2012-important_product_info_f.pdf-manuel

Apple-iOS_Security_May12.pdf-manue

Apple-Mac-Pro-2008-Performance-and-Productivity-for-Creative-Pros

Apple-iPod_shuffle_4thgen_Manuale_utente.pdf-Italie-Manuel

Apple-KernelProgramming.pdf-manuel

Apple-Core-Data-Model-Versioning-and-Data-Migration-Programming-Guide-manuel

Apple-RED_Workflows_with_Final_Cut_Pro_X.pdf-manuel

Apple-Transitioning-to-ARC-Release-Notes-manuel

Apple-iTunes-Connect-Sales-and-Trends-Guide-manuel

Apple-App-Sandbox-Design-Guide-manuel

Apple-String-Programming-Guide-manuel

Apple-Secure-Coding-Guide-manuel

Apple_AirPort_Networks_Early2009.pdf-manuel

Apple-TimeCapsule_SetupGuide_TA.pdf-manuel

Apple-time_capsule_4th_gen_setup.pdf-manuel

Apple-TimeCapsule_SetupGuide.pdf-manuel

Apple-TimeCapsule_SetupGuide_CH.pdf-Chinois-manuel

Apple-CodeSigningGuide.pdf-manuel

Apple-ViewControllerPGforiOS.pdf-manuel

Apple-KeyValueObserving.pdf-manuel

Apple-mac_mini-late-2012-quick_start.pdf-manuel

Apple-OS-X-Mountain-Lion-Core-Technologies-Overview-June-2012-manuel

Apple-OS-X-Server-Product-Overview-June-2012-manuel

Apple-Apple_Server_Diagnostics_UG_109.pdf-manuel

Apple-PackageMaker_UserGuide.pdf-manuel

Apple-InstrumentsUserGuide.pdf-manuel

Apple-Logic-Pro-9-TDM-Guide-manuel

Apple-macbook_air_users_guide.pdf-manuel

Apple-macbook_air-13-inch_mid-2012-qs_ta.pdf-manuel

Apple-AppStoreMarketingGuidelines-JP.pdf-Japon-manuel

Apple-macbook_pro_retina_qs_ta.pdf-manuel

Apple-ipad_user_guide_tu.pdf-manuel

Apple-ipad_user_guide_th.pdf-manuel

Apple-iphone_user_guide_gr.pdf-manuel

Apple-Nike_Plus_iPod_Sensor_UG_2A.pdf-manuel

Apple-ipad_manual_del_usuario.pdf-manuel

Apple-ipad_uzivatelska_prirucka.pdf-manuel

Apple-ipad_wifi_informations_importantes.pdf-manuel

Apple-Xsan_2_Admin_Guide_v2.3.pdf-manuel

Apple-macbook_pro-13-inch-late-2012-quick_start.pdf-manuel

Apple-CocoaDrawingGuide.pdf-manuel

Apple-Cryptographic-Services-Guide-manuel

Apple-Resource-Programming-Guide-manuel

AppleSafariVisualEffectsProgGuide.pdf-manuel

/Apple-WorkingWithUSB.pdf-manuel

Apple-macbook_pro-retina-mid-2012-important_product_info_f.pdf-manuel

Apple-iOS_Security_May12.pdf-manue

Apple-Mac-Pro-2008-Performance-and-Productivity-for-Creative-Pros

Apple-iPod_shuffle_4thgen_Manuale_utente.pdf-Italie-Manuel

Apple-KernelProgramming.pdf-manuel

Apple-Core-Data-Model-Versioning-and-Data-Migration-Programming-Guide-manuel

Apple-RED_Workflows_with_Final_Cut_Pro_X.pdf-manuel

Apple-Transitioning-to-ARC-Release-Notes-manuel

Apple-iTunes-Connect-Sales-and-Trends-Guide-manuel

Apple-App-Sandbox-Design-Guide-manuel

Apple-String-Programming-Guide-manuel

Apple-Secure-Coding-Guide-manuel

Apple_AirPort_Networks_Early2009.pdf-manuel

Apple-TimeCapsule_SetupGuide_TA.pdf-manuel

Apple-time_capsule_4th_gen_setup.pdf-manuel

Apple-TimeCapsule_SetupGuide.pdf-manuel

Apple-TimeCapsule_SetupGuide_CH.pdf-Chinois-manuel

Apple-CodeSigningGuide.pdf-manuel

Apple-ViewControllerPGforiOS.pdf-manuel

Apple-KeyValueObserving.pdf-manuel

Apple-mac_mini-late-2012-quick_start.pdf-manuel

Apple-OS-X-Mountain-Lion-Core-Technologies-Overview-June-2012-manuel

Apple-OS-X-Server-Product-Overview-June-2012-manuel

Apple-Apple_Server_Diagnostics_UG_109.pdf-manuel

Apple-PackageMaker_UserGuide.pdf-manuel

Apple-Instrumentos_y_efectos_de_Logic_Studio.pdf-Manuel

Apple-ipod_nano_kayttoopas.pdf-Finlande-Manuel

Apple_ProRes_White_Paper_October_2012.pdf-Manuel

Apple-wp_osx_configuration_profiles.pdf-Manuel

Apple-UsingiTunesProducerFreeBooks.pdf-Manuel

Apple-ipad_manual_do_usuario.pdf-Portugais-Manuel

Apple-Instruments_et_effets_Logic_Studio.pdf-Manuel

Apple-ipod_touch_gebruikershandleiding.pdf-Neerlandais-Manuel

AppleiPod_shuffle_4thgen_Manual_del_usuario.pdf-Espagnol-Manuel

Apple-Premiers-contacts-avec-votre-PowerBook-G4-Manuel

Apple_Composite_AV_Cable.pdf-Manuel

Apple-iPod_shuffle_3rdGen_UG_DK.pdf-Danemark-Manuel

Apple-iPod_classic_160GB_Benutzerhandbuch.pdf-Allemand-Manuel

Apple-VoiceOver_GettingStarted-Manuel

Apple-iPod_touch_2.2_Benutzerhandbuch.pdf-Allemand-Manuel

Apple-Apple_TV_Opstillingsvejledning.pdf-Allemand-Manuel

Apple-iPod_shuffle_4thgen_Manuale_utente.pdf-Italie-Manuel

Apple-iphone_prirucka_uzivatela.pdf-Manuel

Apple-Aan-de-slag-Neerlandais-Manuel

Apple-airmac_express-80211n-2nd-gen_setup_guide.pdf-Thailande-Manuel

Apple-ipod_nano_benutzerhandbuch.pdf-Allemand-Manuel

Apple-aperture3.4_101.pdf-Manuel

Apple-Pages09_Anvandarhandbok.pdf-Manuel

Apple-nike_plus_ipod_sensor_ug_la.pdf-Mexique-Manuel

Apple-ResEdit-Reference-For-ResEdit02.1-Manuel

Apple-ipad_guide_de_l_utilisateur.pdf-Manuel

Apple-Compressor-4-Benutzerhandbuch-Allemand-Manuel

Apple-AirPort_Networks_Early2009_DK.pdf-Danemark-Manuel

Apple-MacBook_Pro_Mid2007_2.4_2.2GHz_F.pdf-Manuel

Apple-MacBook_13inch_Mid2010_UG_F.pdf-Manuel

Apple-Xserve-RAID-Presentation-technologique-Janvier-2004-Manuel

Apple-MacBook_Pro_15inch_Mid2010_F.pdf-Manuel

Apple-AirPort_Express-opstillingsvejledning.pdf-Danemark-Manuel

Apple-DEiPod_photo_Benutzerhandbuch_DE0190269.pdf-Allemand-Manuel

Apple-Final-Cut-Pro-X-Logic-Effects-Reference-Manuel

Apple-iPod_touch_2.1_Brugerhandbog.pdf-Danemark-Manuel

Apple-Remote-Desktop-Administratorhandbuch-Version-3.1-Allemand-Manuel

Apple-Qmaster-4-User-Manual-Manuel

Apple-Server_Administration_v10.5.pdf-Manuel

Apple-ipod_classic_features_guide.pdf-Manuel

Apple-Lecteur-Optique-Manuel

Apple-Carte-AirPort-Manuel

Apple-iPhone_Finger_Tips_Guide.pdf-Anglais-Manuel

Apple-Couvercle-Manuel

Apple-battery.cube.pdf-Manuel

Apple-Boitier-de-l-ordinateur-Manuel

Apple-Pile-Interne-Manuel

Apple-atacable.pdf-Manuel

Apple-videocard.pdf-Manuel

Apple-Guide_de_configuration_de_l_Airport_Express_5.1.pdf-Manuel

Apple-iMac_Mid2010_UG_F.pdf-Manuel

Apple-MacBook_13inch_Mid2009_F.pdf-Manuel

Apple-MacBook_Mid2007_UserGuide.F.pdf-Manuel

Apple-Designing_AirPort_Networks_10.5-Windows_F.pdf-Manuel

Apple-Administration_de_QuickTime_Streaming_et_Broadcasting_10.5.pdf-Manuel

Apple-Opstillingsvejledning_til_TimeCapsule.pdf-Danemark-Manuel

Apple-iPod_nano_5th_gen_Benutzerhandbuch.pdf-Manuel

Apple-iOS_Business.pdf-Manuel

Apple-AirPort_Extreme_Installationshandbuch.pdf-Manuel

Apple-Final_Cut_Express_4_Installation_de_votre_logiciel.pdf-Manuel

Apple-MacBook_Pro_15inch_2.53GHz_Mid2009.pdf-Manuel

Apple-Network_Services.pdf-Manuel

Apple-Aperture_Performing_Adjustments_f.pdf-Manuel

Apple-Supplement_au_guide_Premiers_contacts.pdf-Manuel

Apple-Administration_des_images_systeme_et_de_la_mise_a_jour_de_logiciels_10.5.pdf-Manuel

Apple-Mac_OSX_Server_v10.6_Premiers_contacts.pdf-Francais-Manuel

Apple-Designing_AirPort_Networks_10.5-Windows_F.pdf-Manuel

Apple-Mise_a_niveau_et_migration_v10.5.pdf-Manue

Apple-MacBookPro_Late_2007_2.4_2.2GHz_F.pdf-Manuel

Apple-Mac_mini_Late2009_SL_Server_F.pdf-Manuel

Apple-Mac_OS_X_Server_10.5_Premiers_contacts.pdf-Manuel

Apple-iPod_touch_2.0_Guide_de_l_utilisateur_CA.pdf-Manuel

Apple-MacBook_Pro_17inch_Mid2010_F.pdf-Manuel

Apple-Comment_demarrer_Leopard.pdf-Manuel

Apple-iPod_2ndGen_USB_Power_Adapter-FR.pdf-Manuel

Apple-Feuille_de_operations_10.4.pdf-Manuel

Apple-Time_Capsule_Installationshandbuch.pdf-Allemand-Manuel

Apple-F034-2262AXerve-grappe.pdf-Manuel

Apple-Mac_Pro_Early2009_4707_UG_F

Apple-imacg5_17inch_Power_Supply

Apple-Logic_Studio_Installieren_Ihrer_Software_Retail

Apple-IntroductionXserve1.0.1

Apple-Aperture_Getting_Started_d.pdf-Allemand

Apple-getting_started_with_passbook

Apple-iPod_mini_2nd_Gen_UserGuide.pdf-Anglais

Apple-Deploiement-d-iPhone-et-d-iPad-Reseaux-prives-virtuels

Apple-F034-2262AXerve-grappe

Apple-Mac_OS_X_Server_Glossaire_10.5

Apple-FRLogic_Pro_7_Guide_TDM

Apple-iphone_bluetooth_headset_userguide

Apple-Administration_des_services_reseau_10.5

Apple-imacg5_17inch_harddrive

Apple-iPod_nano_4th_gen_Manuale_utente

Apple-iBook-G4-Getting-Started

Apple-XsanGettingStarted

Apple-Mac_mini_UG-Early2006

Apple-Guide_des_fonctionnalites_de_l_iPod_classic

Apple-Guide_de_configuration_d_Xsan_2

Apple-MacBook_Late2006_UsersGuide

Apple-sur-Fnac.com

Apple-Mac_mini_Mid2010_User_Guide_F.pdf-Francais

Apple-PowerBookG3UserManual.PDF.Anglais

Apple-Installation_de_votre_logiciel_Logic_Studio_Retail

Apple-Pages-Guide-de-l-utilisateur

Apple-MacBook_Pro_13inch_Mid2009.pdf.Anglais

Apple-MacBook_Pro_15inch_Mid2009

Apple-Installation_de_votre_logiciel_Logic_Studio_Upgrade

Apple-FRLogic_Pro_7_Guide_TDM

Apple-airportextreme_802.11n_userguide

Apple-iPod_shuffle_3rdGen_UG

Apple-iPod_classic_160GB_User_Guide

Apple-iPod_nano_5th_gen_UserGuide

Apple-ipod_touch_features_guide

Apple-Wireless_Mighty_Mouse_UG

Apple-Advanced-Memory-Management-Programming-Guide

Apple-iOS-App-Programming-Guide

Apple-Concurrency-Programming-Guide

Apple-MainStage-2-User-Manual-Anglais

Apple-iMacG3_2002MultilingualUserGuide

Apple-iBookG3_DualUSBUserGuideMultilingual.PDF.Anglais

Apple-imacG5_20inch_AirPort

Apple-Guide_de_l_utilisateur_de_Mac_Pro_Early_2008

Apple-Installation_de_votre_logiciel_Logic_Express_8

Apple-iMac_Guide_de_l_utilisateur_Mid2007

Apple-imacg5_20inch_OpticalDrive

Apple-FCP6_Formats_de_diffusion_et_formats_HD

Apple-prise_en_charge_des_surfaces_de_controle_logic_pro_8

Apple-Aperture_Quick_Reference_f

Apple-Shake_4_User_Manual

Apple-aluminumAppleKeyboard_wireless2007_UserGuide

Apple-ipod_shuffle_features_guide

Apple-Color-User-Manual

Apple-XsanGettingStarted

Apple-Migration_10.4_2e_Ed

Apple-MacBook_Air_SuperDrive

Apple-MacBook_Late2007-f

ApplePowerMacG5_(Early_2005)_UserGuide

Apple-iSightUserGuide

Apple-MacBook_Pro_Early_2008_Guide_de_l_utilisateur

Apple-Nouvelles-fonctionnalites-aperture-1.5

Apple-premiers_contacts_2e_ed_10.4.pdf-Mac-OS-X-Server

Apple-premiers_contacts_2e_ed_10.4

Apple-eMac_2005UserGuide

Apple-imacg5_20inch_Inverter

Apple-Keynote2_UserGuide.pdf-Japon

Apple-Welcome_to_Tiger.pdf-Japon

Apple-XsanAdminGuide_j.pdf-Japon

Apple-PowerBookG4_UG_15GE.PDF-Japon

Apple-Xsan_Migration.pdf-Japon

Apple-Xserve_Intel_DIY_TopCover_JA.pdf-Japon

Apple-iPod_nano_6thgen_User_Guide_J.pdf-Japon

Apple-Aperture_Photography_Fundamentals.pdf-Japon

Apple-nikeipod_users_guide.pdf-Japon

Apple-QuickTime71_UsersGuide.pdf-Japon

Apple-iMacG5_iSight_UG.pdf-Japon

Apple-Aperture_Performing_Adjustments_j.pdf-Japon

Apple-iMacG5_17inch_HardDrive.pdf-Japon

Apple-iPod_shuffle_Features_Guide_J.pdf-Japon

Apple-MacBook_Air_User_Guide.pdf-Japon

Apple-MacBook_UsersGuide.pdf-Japon

Apple-iPad_iOS4_Brukerhandbok.pdf-Norge-Norvege

Apple-Apple_AirPort_Networks_Early2009_H.pd-Norge-Norvege

Apple-iPod_classic_120GB_no.pdf-Norge-Norvege

Apple-StoreKitGuide.pdf-Japon

Apple-Xserve_Intel_DIY_ExpansionCardRiser_JA.pdf-Japon

Apple-iMacG5_Battery.pdf-Japon

Apple-Logic_Pro_8_Getting_Started.pdf-Japon

Apple-PowerBook-handbok-Norge-Norveg

Apple-iWork09_formler_og_funksjoner.pdf-Norge-Norvege

Apple-MacBook_Pro_15inch_Mid2010_H.pdf-Norge-Norvege

Apple-MacPro_HardDrive_DIY.pdf-Japon

Apple-iPod_Fifth_Gen_Funksjonsoversikt.pdf-Norge-Norvege

Apple-MacBook_13inch_white_Early2009_H.pdf-Norge-Norvege

Apple-GarageBand_09_Komme_i_gang.pdf-Norge-Norvege

Apple-MacBook_Pro_15inch_Mid2009_H.pdf-Norge-Norvege

Apple-imac_mid2011_ug_h.pdf-Norge-Norvege

Apple-iDVD_08_Komme_i_gang.pdf-Norge-Norvege

Apple-MacBook_Air_11inch_Late2010_UG_H.pdf-Norge-Norvege

Apple-iMac_Mid2010_UG_H.pdf-Norge-Norvege

Apple-MacBook_13inch_Mid2009_H.pdf-Norge-Norvege

/Apple-iPhone_3G_Viktig_produktinformasjon_H-Norge-Norvege

Apple-MacBook_13inch_Mid2010_UG_H.pdf-Norge-Norvege

Apple-macbook_air_13inch_mid2011_ug_no.pdf-Norge-Norvege

Apple-Mac_mini_Early2009_UG_H.pdf-Norge-Norvege

Apple-ipad2_brukerhandbok.pdf-Norge-Norvege

Apple-iPhoto_08_Komme_i_gang.pdf-Norge-Norvege

Apple-MacBook_Air_Brukerhandbok_Late2008.pdf-Norge-Norvege

Apple-Pages09_Brukerhandbok.pdf-Norge-Norvege

Apple-MacBook_13inch_Late2009_UG_H.pdf-Norge-Norvege

Apple-iPhone_3GS_Viktig_produktinformasjon.pdf-Norge-Norvege

Apple-MacBook_13inch_Aluminum_Late2008_H.pdf-Norge-Norvege

Apple-Wireless_Keyboard_Aluminum_2007_H-Norge-Norvege

Apple-NiPod_photo_Brukerhandbok_N0190269.pdf-Norge-Norvege

Apple-MacBook_Pro_13inch_Mid2010_H.pdf-Norge-Norvege

Apple-MacBook_Pro_17inch_Mid2010_H.pdf-Norge-Norvege

Apple-Velkommen_til_Snow_Leopard.pdf-Norge-Norvege.htm

Apple-TimeCapsule_Klargjoringsoversikt.pdf-Norge-Norvege

Apple-iPhone_3GS_Hurtigstart.pdf-Norge-Norvege

Apple-Snow_Leopard_Installeringsinstruksjoner.pdf-Norge-Norvege

Apple-iMacG5_iSight_UG.pdf-Norge-Norvege

Apple-iPod_Handbok_S0342141.pdf-Norge-Norvege

Apple-ipad_brukerhandbok.pdf-Norge-Norvege

Apple-GE_Money_Bank_Handlekonto.pdf-Norge-Norvege

Apple-MacBook_Air_11inch_Late2010_UG_H.pdf-Norge-Norvege

Apple-iPod_nano_6thgen_Brukerhandbok.pdf-Norge-Norvege

Apple-iPod_touch_iOS4_Brukerhandbok.pdf-Norge-Norvege

Apple-MacBook_Air_13inch_Late2010_UG_H.pdf-Norge-Norvege

Apple-MacBook_Pro_15inch_Early2011_H.pdf-Norge-Norvege

Apple-Numbers09_Brukerhandbok.pdf-Norge-Norvege

Apple-Welcome_to_Leopard.pdf-Japon

Apple-PowerMacG5_UserGuide.pdf-Norge-Norvege

Apple-iPod_touch_2.1_Brukerhandbok.pdf-Norge-Norvege

Apple-Boot_Camp_Installering-klargjoring.pdf-Norge-Norvege

Apple-MacOSX10.3_Welcome.pdf-Norge-Norvege

Apple-iPod_shuffle_3rdGen_UG_H.pdf-Norge-Norvege

Apple-iPhone_4_Viktig_produktinformasjon.pdf-Norge-Norvege

Apple_TV_Klargjoringsoversikt.pdf-Norge-Norvege

Apple-iMovie_08_Komme_i_gang.pdf-Norge-Norvege

Apple-iPod_classic_160GB_Brukerhandbok.pdf-Norge-Norvege

Apple-Boot_Camp_Installering_10.6.pdf-Norge-Norvege

Apple-Network-Services-Location-Manager-Veiledning-for-nettverksadministratorer-Norge-Norvege

Apple-iOS_Business_Mar12_FR.pdf

Apple-PCIDualAttachedFDDICard.pdf

Apple-Aperture_Installing_Your_Software_f.pdf

Apple-User_Management_Admin_v10.4.pdf

Apple-Compressor-4-ユーザーズマニュアル Japon

Apple-Network_Services_v10.4.pdf

Apple-iPod_2ndGen_USB_Power_Adapter-DE

Apple-Mail_Service_v10.4.pdf

Apple-AirPort_Express_Opstillingsvejledning_5.1.pdf

Apple-MagSafe_Airline_Adapter.pdf

Apple-L-Apple-Multiple-Scan-20-Display

Apple-Administration_du_service_de_messagerie_10.5.pdf

Apple-System_Image_Admin.pdf

Apple-iMac_Intel-based_Late2006.pdf-Japon

Apple-iPhone_3GS_Finger_Tips_J.pdf-Japon

Apple-Power-Mac-G4-Mirrored-Drive-Doors-Japon

Apple-AirMac-カード取り付け手順-Japon

Apple-iPhone開発ガイド-Japon

Apple-atadrive_pmg4mdd.j.pdf-Japon

Apple-iPod_touch_2.2_User_Guide_J.pdf-Japon

Apple-Mac_OS_X_Server_v10.2.pdf

Apple-AppleCare_Protection_Plan_for_Apple_TV.pdf

Apple_Component_AV_Cable.pdf

Apple-DVD_Studio_Pro_4_Installation_de_votre_logiciel

Apple-Windows_Services

Apple-Motion_3_New_Features_F

Apple-g4mdd-fw800-lowerfan

Apple-MacOSX10.3_Welcome

Apple-Print_Service

Apple-Xserve_Setup_Guide_F

Apple-PowerBookG4_17inch1.67GHzUG

Apple-iMac_Intel-based_Late2006

Apple-Installation_de_votre_logiciel

Apple-guide_des_fonctions_de_l_iPod_nano

Apple-Administration_de_serveur_v10.5

Apple-Mac-OS-X-Server-Premiers-contacts-Pour-la-version-10.3-ou-ulterieure

Apple-boot_camp_install-setup

Apple-iBookG3_14inchUserGuideMultilingual

Apple-mac_pro_server_mid2010_ug_f

Apple-Motion_Supplemental_Documentation

Apple-imac_mid2011_ug_f

Apple-iphone_guide_de_l_utilisateur

Apple-macbook_air_11inch_mid2011_ug_fr

Apple-NouvellesfonctionnalitesdeLogicExpress7.2

Apple-QT_Streaming_Server

Apple-Web_Technologies_Admin

Apple-Mac_Pro_Early2009_4707_UG

Apple-guide_de_l_utilisateur_de_Numbers08

Apple-Decouverte_d_Aperture_2

Apple-Guide_de_configuration_et_d'administration

Apple-mac_integration_basics_fr_106.

Apple-iPod_shuffle_4thgen_Guide_de_l_utilisateur

Apple-ARA_Japan

Apple-081811_APP_iPhone_Japanese_v5.4.pdf-Japan

Apple-Recycle_Contract120919.pdf-Japan

Apple-World_Travel_Adapter_Kit_UG

Apple-iPod_nano_6thgen_User_Guide

Apple-RemoteSupportJP

Apple-Mac_mini_Early2009_UG_F.pdf-Manuel-de-l-utilisateur

Apple-Compressor_3_Batch_Monitor_User_Manual_F.pdf-Manuel-de-l-utilisateur

Apple-Premiers__contacts_avec_iDVD_08

Apple-Mac_mini_Intel_User_Guide.pdf

Apple-Prise_en_charge_des_surfaces_de_controle_Logic_Express_8

Apple-mac_integration_basics_fr_107.pdf

Apple-Final-Cut-Pro-7-Niveau-1-Guide-de-preparation-a-l-examen

Apple-Logic9-examen-prep-fr.pdf-Logic-Pro-9-Niveau-1-Guide-de-preparation-a-l-examen

Apple-aperture_photography_fundamentals.pdf-Manuel-de-l-utilisateu

Apple-emac-memory.pdf-Manuel-de-l-utilisateur

Apple-Apple-Installation-et-configuration-de-votre-Power-Mac-G4

Apple-Guide_de_l_administrateur_d_Xsan_2.pdf

Apple-premiers_contacts_avec_imovie6.pdf

Apple-Tiger_Guide_Installation_et_de_configuration.pdf

Apple-Final-Cut-Pro-7-Level-One-Exam-Preparation-Guide-and-Practice-Exam

Apple-Open_Directory.pdf

Apple-Nike_+_iPod_User_guide

Apple-ard_admin_guide_2.2_fr.pdf

Apple-systemoverviewj.pdf-Japon

Apple-Xserve_TO_J070411.pdf-Japon

Apple-Mac_Pro_User_Guide.pdf

Apple-iMacG5_iSight_UG.pdf

Apple-premiers_contacts_avec_iwork_08.pdf

Apple-services_de_collaboration_2e_ed_10.4.pdf

Apple-iPhone_Bluetooth_Headset_Benutzerhandbuch.pdf

Apple-Guide_de_l_utilisateur_de_Keynote08.pdf

APPLE/Apple-Logic-Pro-9-Effectsrfr.pdf

Apple-Logic-Pro-9-Effectsrfr.pdf

Apple-iPod_shuffle_3rdGen_UG_F.pdf

Apple-iPod_classic_160Go_Guide_de_l_utilisateur.pdf

Apple-iBookG4GettingStarted.pdf

Apple-Administration_de_technologies_web_10.5.pdf

Apple-Compressor-4-User-Manual-fr

Apple-MainStage-User-Manual-fr.pdf

Apple-Logic_Pro_8.0_lbn_j.pdf

Apple-PowerBookG4_15inch1.67-1.5GHzUserGuide.pdf

Apple-MacBook_Pro_15inch_Mid2010_CH.pdf

Apple-LED_Cinema_Display_27-inch_UG.pdf

Apple-MacBook_Pro_15inch_Mid2009_RS.pdf

Apple-macbook_pro_13inch_early2011_f.pdf

Apple-iMac_Mid2010_UG_BR.pdf

Apple-iMac_Late2009_UG_J.pdf

Apple-iphone_user_guide-For-iOS-6-Software

Apple-iDVD5_Getting_Started.pdf

Apple-guide_des_fonctionnalites_de_l_ipod_touch.pdf

Apple_iPod_touch_User_Guide

Apple_macbook_pro_13inch_early2011_f

Apple_Guide_de_l_utilisateur_d_Utilitaire_RAID

Apple_Time_Capsule_Early2009_Setup_F

Apple_iphone_4s_finger_tips_guide_rs

Apple_iphone_upute_za_uporabu

Apple_ipad_user_guide_ta

Apple_iPod_touch_User_Guide

apple_earpods_user_guide

apple_iphone_gebruikershandleiding

apple_iphone_5_info

apple_iphone_brukerhandbok

apple_apple_tv_3rd_gen_setup_tw

apple_macbook_pro-retina-mid-2012-important_product_info_ch

apple_Macintosh-User-s-Guide-for-Macintosh-PowerBook-145

Apple_ipod_touch_user_guide_ta

Apple_TV_2nd_gen_Setup_Guide_h

Apple_ipod_touch_manual_del_usuario

Apple_iphone_4s_finger_tips_guide_tu

Apple_macbook_pro_retina_qs_th

Apple-Manuel_de_l'utilisateur_de_Final_Cut_Server

Apple-iMac_G5_de_lutilisateur

Apple-Cinema_Tools_4.0_User_Manual_F

Apple-Personal-LaserWriter300-User-s-Guide

Apple-QuickTake-100-User-s-Guide-for-Macintosh

Apple-User-s-Guide-Macintosh-LC-630-DOS-Compatible

Apple-iPhone_iOS3.1_User_Guide

Apple-iphone_4s_important_product_information_guide

Apple-iPod_shuffle_Features_Guide_F

Liste-documentation-apple

Apple-Premiers_contacts_avec_iMovie_08

Apple-macbook_pro-retina-mid-2012-important_product_info_br

Apple-macbook_pro-13-inch-mid-2012-important_product_info

Apple-macbook_air-11-inch_mid-2012-qs_br

Apple-Manuel_de_l_utilisateur_de_MainStage

Apple-Compressor_3_User_Manual_F

Apple-Color_1.0_User_Manual_F

Apple-guide_de_configuration_airport_express_4.2

Apple-TimeCapsule_SetupGuide

Apple-Instruments_et_effets_Logic_Express_8

Apple-Manuel_de_l_utilisateur_de_WaveBurner

Apple-Macmini_Guide_de_l'utilisateur

Apple-PowerMacG5_UserGuide

Disque dur, ATA parallèle Instructions de remplacement

Apple-final_cut_pro_x_logic_effects_ref_f

Apple-Leopard_Installationshandbok

Manuale Utente PowerBookG4

Apple-thunderbolt_display_getting_started_1e

Apple-Compressor-4-Benutzerhandbuch

Apple-macbook_air_11inch_mid2011_ug

Apple-macbook_air-mid-2012-important_product_info_j

Apple-iPod-nano-Guide-des-fonctionnalites

Apple-iPod-nano-Guide-des-fonctionnalites

Apple-iPod-nano-Guide-de-l-utilisateur-4eme-generation

Apple-iPod-nano-Guide-de-l-utilisateur-4eme-generation

Apple-Manuel_de_l_utilisateur_d_Utilitaire_de_reponse_d_impulsion

Apple-Aperture_2_Raccourcis_clavier

AppleTV_Setup-Guide

Apple-livetype_2_user_manual_f

Apple-imacG5_17inch_harddrive

Apple-macbook_air_guide_de_l_utilisateur

Apple-MacBook_Early_2008_Guide_de_l_utilisateur

Apple-Keynote-2-Guide-de-l-utilisateur

Apple-PowerBook-User-s-Guide-for-PowerBook-computers

Apple-Macintosh-Performa-User-s-Guide-5200CD-and-5300CD

Apple-Macintosh-Performa-User-s-Guide

Apple-Workgroup-Server-Guide

Apple-iPod-nano-Guide-des-fonctionnalites

Apple-iPad-User-Guide-For-iOS-5-1-Software

Apple-Boot-Camp-Guide-d-installation-et-de-configuration

Apple-iPod-nano-Guide-de-l-utilisateur-4eme-generation

Power Mac G5 Guide de l’utilisateur APPLE

Guide de l'utilisateur PAGE '08 APPLE

Guide de l'utilisateur KEYNOTE '09 APPLE

Guide de l'Utilisateur KEYNOTE '3 APPLE

Guide de l'Utilisateur UTILITAIRE RAID

Guide de l'Utilisateur Logic Studio

Power Mac G5 Guide de l’utilisateur APPLE

Guide de l'utilisateur PAGE '08 APPLE

Guide de l'utilisateur KEYNOTE '09 APPLE

Guide de l'Utilisateur KEYNOTE '3 APPLE

Guide de l'Utilisateur UTILITAIRE RAID

Guide de l'Utilisateur Logic Studio

Guide de l’utilisateur ipad Pour le logiciel iOS 5.1

PowerBook G4 Premiers Contacts APPLE

Guide de l'Utilisateur iphone pour le logiciel ios 5.1 APPLE

Guide de l’utilisateur ipad Pour le logiciel iOS 4,3

Guide de l’utilisateur iPod nano 5ème génération

Guide de l'utilisateur iPod Touch 2.2 APPLE

Guide de l’utilisateur QuickTime 7  Mac OS X 10.3.9 et ultérieur Windows XP et Windows 2000

Guide de l'utilisateur MacBook 13 pouces Mi 2010

Guide de l’utilisateur iPhone (Pour les logiciels iOS 4.2 et 4.3)

Guide-de-l-utilisateur-iPod-touch-pour-le-logiciel-ios-4-3-APPLE

Guide-de-l-utilisateur-iPad-2-pour-le-logiciel-ios-4-3-APPLE

Guide de déploiement en entreprise iPhone OS

Guide-de-l-administrateur-Apple-Remote-Desktop-3-1

Guide-de-l-utilisateur-Apple-Xserve-Diagnostics-Version-3X103

Guide-de-configuration-AirPort-Extreme-802.11n-5e-Generation

Guide-de-configuration-AirPort-Extreme-802-11n-5e-Generation

Guide-de-l-utilisateur-Capteur-Nike-iPod

Guide-de-l-utilisateur-iMac-21-5-pouces-et-27-pouces-mi-2011-APPLE

Guide-de-l-utilisateur-Apple-Qadministrator-4

Guide-d-installation-Apple-TV-3-eme-generation

User-Guide-iPad-For-ios-5-1-Software

Window Programming GuideContents Introduction 6 Organization of This Document 6 See Also 8 How Windows Work 9 How a Window is Displayed 11 How Modal Windows Work 12 How Panels Work 14 How Window Controllers Work 15 Window Closing Behavior 16 Opening and Closing Windows 17 Window Layering and Types of Windows 18 Window Layering 18 Key and Main Windows 19 The Key Window 20 The Main Window 20 Changing a Window’s Status 21 Window Layers and Levels 22 Window Levels 22 Setting Ordering and Level Programmatically 22 Setting Window Collection Behavior 24 Spaces Collection Behavior 24 Exposé Collection Behavior 24 Window Cycling Behavior 25 Sizing and Placing Windows 26 Setting a Window’s Size and Location 26 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 2Window Cascading 27 Window Zooming 27 Constraining a Window’s Size and Location 27 Saving a Window’s Position into the User’s Defaults 29 Minimizing Windows 30 Using the Window Menu 31 Setting a Window’s Appearance 32 Setting a Window’s Style 32 Setting a Window’s Color and Transparency 33 Setting a Window’s Color Space 33 Setting a Window’s Content Border Thickness 33 Setting a Window’s Title and Represented File 34 Setting Attributes for the Window’s Image 35 Specifying How To Store the Window’s Image 35 Specifying Where To Store the Window’s Image 36 Specifying When the Window’s Image Is Created 36 Specifying Whether the Window’s Image Persists When Offscreen 37 Specifying the Depth Limit for the Window’s Image 37 Specifying Whether the Depth Limit Changes to the Screen’s Capacity 37 Specifying Whether Window Content Can Be Read or Written by Another Process 37 Handling Events in Windows 38 Using Keyboard Interface Control in Windows 39 Using the Window’s Field Editor 40 Using Window Notifications and Delegate Methods 41 Dragging Images to and from Windows 42 Updating the Cursor Image in a Window 43 Caching Window Images 44 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 3Document Revision History 45 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 4Figures and Listings Window Layering and Types of Windows 18 Figure 1 Main, key, and inactive windows 19 Saving a Window’s Position into the User’s Defaults 29 Listing 1 Saving a window’s frame automatically 29 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 5An application displays windows on the screen that must be managed and coordinated. A window object corresponds to at most one on-screen window. The two principal functions of windows are to provide an area in which views can be placed and to accept and distribute events the user sends through actions with the mouse and keyboard. The term window sometimes refers to the Application Kit object and sometimes to the window server’s window device; which meaning is intended is made clear in context. Panels are a special kind of window, typically serving an auxiliary function in an application, such as utility windows. This document is intended for Cocoa developers who need to work with windows and panels in their applications. Organization of This Document This programming topic describes how to use windows and panels. These articles give you basic information on the different types of windows and how they work: ● “How Windows Work” (page 9) describes the classes that define objects that manage and coordinate the windows an application displays. ● “How a Window is Displayed” (page 11) describes how window drawing is accomplished. ● “How Modal Windows Work” (page 12) describes the behavior of modal windows. ● “How Panels Work” (page 14) describes the various uses of panels. ● “How Window Controllers Work” (page 15) describesthe relationship between a window and its controller. ● “Window Layering and Types of Windows” (page 18) describes window layering and the concepts of key and main windows, and how a window can avoid becoming key or main. ● “Window Layers and Levels” (page 22) describes window levels, and how to place a window in a specific level, such as the level for document windows, palettes, or tear-off menus. ● “Setting Window Collection Behavior” (page 24) describes how to set a window’s behavior with Spaces, Exposé, and window cycles. These articles describe how to use windows: ● “Opening and Closing Windows” (page 17) describes how to open and close, or just show and hide, a window. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 6 Introduction● “Sizing and Placing Windows” (page 26) describes how to control a window’s size and position, including how to set its minimum and maximum size, how to constrain it to the screen, how to cascade it so its title bar remains visible, how to zoom it as though the user pressed the zoom button, and how to center it on the screen. ● “Saving a Window’s Position into the User’s Defaults” (page 29) describes how to store a window’s position in the user defaults system, so that it appears in the same location the next time the user starts the application. ● “Minimizing Windows” (page 30) describes how to replace a window with a smaller counterpart in the Dock. ● “Using the Window Menu” (page 31) describes how to place a window’s name in the Windows menu that appears in most Cocoa applications. These articles describe how to change what a window looks like: ● “Setting a Window’s Appearance” (page 32) describes how to choose whether to display a window’s peripheral elements, including its title bar, close box, zoom box, or size box. It also describes how to set a window’s background color and transparency, ● “Setting a Window’s Title and Represented File” (page 34) describes how to set a window’s title with either a string or the filename of the window’s represented file. ● “Setting Attributes for the Window’s Image” (page 35) describes how to set attributes for the window’s device, which stores the window’s image, including how the image is stored, when the image is created, and the image’s color depth. These articles describe how to handle a window’s events: ● “Handling Events in Windows” (page 38) gives basic information on how a window handles events. ● “Using Keyboard Interface Control in Windows” (page 39) describes how to navigate between a window’s fields using the Tab key and how to use the Return and Escape keys to select default buttons. ● “Using the Window’s Field Editor” (page 40) describes how to use the window’s text object, which is shared for light editing tasks. These articles describe some advanced features of windows: ● “Using Window Notifications and Delegate Methods” (page 41) describes the notifications and delegate methods used when a window gains or loses key or main window status, minimizes, moves or resizes, becomes exposed, or closes. ● “Dragging Images to and from Windows” (page 42) describes what happens when the user wants to drag an object into or out of a window. Introduction Organization of This Document 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 7● “Updating the Cursor Image in a Window” (page 43) directs you to information on how to change the cursor image when the cursor is over a specified area in a view. ● “Caching Window Images” (page 44) describes how to temporarily cache a portion of a window’s image so that it can be restored later. Thisis useful when highly dynamic drawing must be done over an otherwise static image of the window. See Also For additional information on specific types of windows and panels, you can also see the following programming topics: ● Sheet Programming Topics describes a dialog attached to a specific window, ensuring that a user never loses track of which window the dialog belongs to. ● Drawer Programming Topics describes a type of view that slides out from one side of a window. ● Toolbar Programming Topics for Cocoa describes a standard way to display a toolbar for a titled window below its title bar and provide users with a way to customize toolbars and save those customizations. ● Dialogs and Special Panels describes alert panels and other specialized types of panels, such as Font, Save, and Print panels. ● Document-Based App Programming Guide for Mac describes how to use the architecture supplied by AppKit to create applications that can create, open, load, and save multiple document files. ● Cocoa Event Handling Guide discusses the variety of ways your application objects can handle the events they receive. Introduction See Also 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 8The NSWindow class defines objects that manage and coordinate the windows an application displays on the screen. A single NSWindow object corresponds to at most one onscreen window. The two principal functions of an NSWindow object are to provide an area in which NSView objects can be placed and to accept and distribute, to the appropriate views, events the user instigates through actions with the mouse and keyboard. Note that the term window sometimes refers to the Application Kit object and sometimes to the window server’s display device; which meaning is intended is made clear in context. AppKit also defines an abstract subclass of NSWindow—NSPanel—that adds behavior more appropriate for auxiliary windows. An NSWindow object is defined by a frame rectangle that encloses the entire window, including its title bar, border, and other peripheral elements (such as the resize control), and by a content rectangle that encloses just its content area. Both rectangles are specified in the screen coordinate system and are restricted to integer values. The frame rectangle establishesthe window’s base coordinate system. This coordinate system is always aligned with and measured in the same increments as the screen coordinate system (in other words, the base coordinate system can’t be rotated or scaled). The origin of the base coordinate system is the bottom-left corner of the window’s frame rectangle. Typically, you create windows using Interface Builder, which allows you to position them, set many of their attributes, and lay out their views. The programmatic work you do with windows more often involves bringing them on and off the screen; changing dynamic attributes such as the window’s title; running modal windows to restrict user input; and assigning a delegate that can monitor certain of the window’s actions,such as closing, zooming, and resizing. You can also create a window programmatically with one of itsinitializers by specifying, among other attributes, the size and location of its content rectangle. The frame rectangle is derived from the dimensions of the content rectangle. When it’s created, a window automatically createstwo views: an opaque frame view that fillsthe frame rectangle and draws the border, title bar, other peripheral elements, and background, and a transparent content view that fills the content rectangle. The frame view and its peripheral elements are private objects that your application can’t access directly. The content view is the “highest” accessible view in the window; you can replace the default content view with a view of your own creation using the setContentView: method. The window determines the placement of the content view; you can’t position it using the NSView methods that begin with setFrame; you must use the NSWindow class’s placement methods, as described in “Opening and Closing Windows” (page 17). 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 9 How Windows WorkYou add other views to the window as subviews of the content view or as subviews of any of the content view’s subviews, and so on, via the addSubview: method of NSView. This tree of views is called the window’s view hierarchy. When a window is told to display itself, it does so by sending display... messages to the top-level view in its view hierarchy. Because displaying is carried out in a determined order, the content view (which is drawn first) may be wholly or partially obscured by itssubviews, and these subviews may be obscured by their subviews (and so on). How Windows Work 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 10Displaying an NSWindow object begins with the drawing performed by its view objects, which accumulates in the window’s display buffer or appears immediately on the screen. Windows, like NSView objects, can be displayed unconditionally or merely marked as needing display, using the display and setViewsNeedDisplay: methods, respectively. A displayIfNeeded message causes the window’s views to display only if they’ve been marked as needing display. Normally, any time a view is marked as needing display, the window makes note of this fact and automatically displays itself shortly thereafter. This automatic display is typically performed on each pass through the event loop, but can be turned off using the setAutodisplay: method. If you turn off autodisplay for a window, you’re then responsible for displaying it whenever necessary. A window’s views can be drawn concurrently. You can use the methods allowsConcurrentViewDrawing and setAllowsConcurrentViewDrawing: to determine and set, respectively, whether or not a window draws its views concurrently. By default, a window’s views are drawn concurrently. On each passthrough the event loop, the application object invokesits updateWindows method, which sends an update message to each window. Subclasses of NSWindow can override this method to examine the state of the application and change their own state or appearance accordingly—enabling or disabling menus, buttons, and other controls based on the object that’s selected, for example. In addition to displaying itself on the screen, a window can print itself in its entirety, just as a view can. The print: method runs the application’s Print panel and causes the window’s frame view to print itself. dataWithEPSInsideRect: behaves similarly. For additional information see Printing Programming Guide for OS X . 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 11 How a Window is DisplayedYou can make a whole window or panel run in application-modal fashion, using the application’s normal event loop machinery but restricting input to the modal window or panel. Modal operation is useful for windows and panels that require the user’s attention before an action can proceed. Examples include error messages and warnings, as well as operations that require input, such as open dialogs, or dialogs that apply to multiple windows. There are two mechanisms for operating an application-modal window or panel. The first, and simpler, is to invoke the runModalForWindow: method of NSApplication, which monopolizes events for the specified window until one of stopModal, abortModal, or stopModalWithCode: is invoked, typically by a button’s action method. The stopModal method ends the modal status of the window or panel from within the event loop. It doesn’t work if invoked from a method invoked by a timer or by a distributed object because those mechanisms operate outside of the event loop. To terminate the modal loop in these situations, you can use abortModal. The stopModal method is typically invoked when the user clicks the OK button (or equivalent), abortModal when the user clicks the Cancel button (or presses the Escape key). These two methods are equivalent to stopModalWithCode: with the appropriate argument. The second mechanism for operating a modal window or panel, called a modal session, allowsthe application to perform a long operation while it still sends events to the window or panel. Modal sessions are particularly useful for panels that allow the user to cancel or modify an operation. To begin a modal session, invoke beginModalSessionForWindow: on the application, which sets the window up for the session and returns an identifier used for other session-controlling methods. At this point, the application can run in a loop that performsthe operation, invoking runModalSession: on the application object on each passso that pending events can be dispatched to the modal window. This method returns a code indicating whether the operation should continue, stop, or abort, which is typically established by the methods described above for runModalForWindow:. After the loop concludes, you can remove the window from the screen and invoke endModalSession: on the application to restore the normal event loop. Note: You can write a modal event loop for a view object so that the object has access to all events pertaining to a particular task, such as tracking the mouse in the view. For an example, see “Responding to User Events and Actions” in “Creating a Custom View”. The normal behavior of a modal window or session is to exclude all other windows and panels from receiving events. For windows and panels that serve as general auxiliary controls, such as menus and the Font panel, this behavior is overly restrictive. The user must be able to use menu key equivalents (such as those for Cut 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 12 How Modal Windows Workand for Paste) and change the font of text in the modal window, and this requires that non-modal panels be able to receive events. To support this behavior, an NSWindow subclass overridesthe worksWhenModal method to return YES. This allows the window to receive mouse and keyboard events even when a modal window is present. If a subclass needs to work when a modal window is present, it should generally be a subclass of NSPanel, not of NSWindow. Modal windows and modal sessions provide different levels of control to the application and the user. Modal windows restrict all action to the window itself and any methods invoked from the window. Modal sessions allow the application to continue an operation while accepting input only through the modal session window. Beyond this, you can use distributed objects to perform background operations in a separate thread, while allowing the user to perform other actions with any part of the application. The background thread can communicate with the main thread, allowing the application to display the status of the operation in a non-modal panel, perhaps including controls to stop or affect the operation as it occurs. Note that because AppKit isn’t thread-safe, the background thread should communicate with a designated object in the main thread that in turn interacts with the AppKit. Before OS X version 10.6, if a modal window was open, application termination would be prevented if the user attempted to terminate that window’s application. Beginning in OS X version 10.6, you can call setPreventsApplicationTerminationWhenModal: with a value of NO, and the window will not prevent application termination when modal. The current value of this property may be accessed by calling preventsApplicationTerminationWhenModal. The default value is NO. How Modal Windows Work 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 13A panel is a special kind of window, typically serving an auxiliary function in an application. The NSPanel subclass of NSWindow adds a few special behaviors to windows in support of the role panels play: ● By default panels are not released when they’re closed, because they’re usually lightweight and often reused. ● Onscreen panels, except for alert dialogs, are removed from the screen when the application isn’t active and are restored when the application again becomes active. This reduces screen clutter. Specifically, the NSWindow implementation of the hidesOnDeactivate method returns NO, but the NSPanel implementation of the same method returns YES. ● Panels can become the key window, but they cannot become the main window. ● If a panel is the key window and has a close button, it closes itself when the user presses the Escape key. In addition to these automatic behaviors, the NSPanel class allows you to configure certain other behaviors common to some kinds of panels: ● You can prevent a panel from becoming the key window unless the user clicks in a view that responds to typing. This prevents the key window from shifting to the panel unnecessarily. The setBecomesKeyOnlyIfNeeded: method controls this behavior. ● Palettes and similar panels can be made to float above standard windows and other panels. This prevents them from being covered and keepsthem readily available to the user. The setFloatingPanel: method controls this behavior. ● A panel can be made to receive mouse and keyboard events even when another window or panel is being run modally or in a modal session. This permits actions in the panel to affect the modal window or panel. The setWorksWhenModal: method controls this behavior. See “How Modal Windows Work” (page 12) for more information on modal windows and panels. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 14 How Panels WorkA controller object (in this case, an instance of the NSWindowController class) manages a window; this object is usually stored in a nib file. This management entails the following: ● Loading and displaying the window ● Closing the window when appropriate ● Customizing the window’s title ● Storing the window’s frame (size and location) in the defaults database ● Cascading the window in relation to other document windows of the application A window controller can manage a window by itself or as a participant in AppKit’s document-based architecture, which also includes the NSDocument and NSDocumentController classes. In this architecture, a window controller is created and managed by a document (an instance of an NSDocument subclass) and, in turn, keeps a reference to the document. For a discussion of this architecture, see Document-Based App Programming Guide for Mac . The relationship between a window controller and a nib file is important. Although a window controller can manage a programmatically created window, it usually manages a window in a nib file. The nib file can contain other top-level objects, including other windows, but the window controller’s responsibility is this primary window. The window controller is usually the owner of the nib file, even when it is part of a document-based application. For simple documents—that is, documents with only one nib file containing a window—you need do little directly with NSWindowController objects. AppKit creates one for you. However, if the default window controller is not sufficient, you can create a custom subclass of NSWindowController. For documents with multiple windows or panels, your document must create separate instances of NSWindowController (or of custom subclasses of NSWindowController), one for each window or panel. An example is a CAD application that has different windows for side, top, and front views of drawn objects. What you do in your NSDocument subclass determines whether the default NSWindowController object or separately created and configured NSWindowController objects are used. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 15 How Window Controllers WorkWindow Closing Behavior When a window is closed and it is part of a document-based application, the document removes the window’s window controller from itslist of window controllers. Thisresultsin the system deallocating the window controller and the window, and possibly the NSDocument object itself. When a window controller is not part of a document-based application, closing the window does not by default result in the deallocation of the window or window controller. This is the desired behavior for a window controller that manages something like an inspector; you shouldn’t have to load the nib file again and re-create the objectsthe next time the user requests the inspector. If you want the closing of a window to make both window and window controller go away when it isn’t part of a document, yoursubclass of NSWindowController can observe the NSWindowWillCloseNotification notification or, as the window delegate, implement the windowWillClose: method. How Window Controllers Work Window Closing Behavior 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 16This article describes how to open and close a window. Opening a window—that is, making a window visible—is normally accomplished by placing the window into the application's window list by invoking one of the methods makeKeyAndOrderFront:, orderFront:, etc., in NSWindow, and so on. Also, with certain bits set in Interface Builder, the window is shown when the nib file is loaded in some cases. Closing a window involves explicit use of either the close method, which simply removes the window from the screen, or performClose:, which highlights the close button as though the user clicked it. Closing a window involves at least removing it from the screen but may include disposing of it altogether. The setReleasedWhenClosed: method specifies whether a window releases itself when it receives a close message. A window’s delegate is also notified when it’s about to close, as described in “Using Window Notifications and Delegate Methods” (page 41). These methods hide a window without closing it. The method orderOut: removes a window from the screen. You can also set a window to be removed from the screen automatically when its application isn’t active using setHidesOnDeactivate:. The isVisible method returns whether a window is on or off the screen. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 17 Opening and Closing WindowsEach window is placed on the screen by a particular application, and each application typically owns a variety of windows. Windows have numerous characteristics. They can be located onscreen or offscreen. Onscreen windows are placed on the screen in levels managed by the window server. Windows onscreen are ordered from front to back. Like sheets of paper loosely stacked together, windows in front can overlap, or even completely cover, those behind them. Each window has a unique position in the order. When two windows are placed side-by-side, one is still technically in front of the other. If any window could be in front of any other window, then small but important windows—like menus and tool palettes—might get lost behind larger ones. Windows that require user action, like attention panels and pop-up lists, might disappear behind another window and go unnoticed. To prevent this, all the windows onscreen are organized into levels. When two windows belong to the same level, either one can be in front. When two windows belong to different levels, however, the one in the higher level will always be above the other. Onscreen windows can also carry a status: main or key . Offscreen windows are hidden or minimized on Dock, and do not carry either status. Onscreen windows that are neither main nor key are inactive. Window Layering Each application and document window exists in its own layer, so documents from different applications can be interleaved. Clicking a window to bring it to the front doesn’t disturb the layering order of any other window. A window’s depth in the layers is determined by when the window was last accessed. When a user clicks an inactive document or chooses it from the Window menu, only that document, and any open utility windows, should be brought to the front. Users can bring all windows of an application forward by clicking its icon in the Dock or by choosing Bring All to Front in the application’s Window menu. These actions should bring forward all of the application’s open windows, maintaining their onscreen location, size, and layering order within the application. For more information, see “UI Element Guidelines: Menus” in OS X Human Interface Guidelines. Utility windows are alwaysin the same layer: the top layer. They are visible only when their application is active. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 18 Window Layering and Types of WindowsKey and Main Windows Windows have different looks based on how the user is interacting with them. The foremost document or application window that is the focus of the user’s attention is referred to as the main window. Each application also has only one main window at a given time. This main window often has key status, as well. The main window is the principal focus of user actions for an application. Often, user actions in a modal key window (typically a panel such as the Font window or an Info window) have a direct effect on the main window. Main and key windows are both active windows. Active windows are visually distinct from inactive windows in that their controls have color, while the controls in inactive windows do not have color. Inactive windows are windows the user has open but that are not in the foreground. Main and key windows are always in the foreground and their controls always have color. If the main and key window are different windows, they are distinguished from one another by the look of their title bars. Note the visual distinctions between main, key, and inactive windows in Figure 1. Figure 1 Main, key, and inactive windows Inactive window Main window Key window A good example of the difference between key and main windows can be seen in most well-behaved Mac apps. Selecting “Save As...” in a text document, for example, displays a panel with a field to type the document’s name and a pull-down menu of locations to save it. The panel represents the key window. It will accept your Window Layering and Types of Windows Key and Main Windows 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 19keyboard input (the file name), but will directly affect the main window under it (by saving it to the location you specified). Once you save the document, the save panel disappears, the main window becomes key again, and will accept keyboard input once more. The Key Window The key window responds to user input, whether from the keyboard, mouse, or alternative input devices, for an application and is the primary recipient of messages from menus and panels. Usually, a window is made key when the user clicks it. Each application can have only one key window at a given time. Users expect to see their actions on the keyboard and mouse take effect not only in a particular application, but also in a particular window of that application. Each user action is associated with a window by the window server and AppKit. Before acting, the user needs to know which window will be affected; there should be no surprises. Since the mouse controls the pointer, it's quite easy for the user to determine which window a mouse action is associated with. It's whatever window the pointer is over. But the keyboard doesn’t have a pointer, so there’s no natural way to determine where typed characters will appear. To mark the key window for users, AppKit highlights its title bar. You can think of the highlighting as a kind of pointer for the keyboard. It shifts from window to window as the key window changes. Key-window status also moves from application to application as the active application changes. Only one window on the screen is marked at a time, and it is in the active application. There’s just one key window on the Desktop. Even a system that has two screens, but only one keyboard, has at most one key window. Note: A window doesn’t have to become the key window to receive, and act on, keyboard shortcuts. It does, however, have to be a window in the active application. Since the key window belongs to the active application, its highlighted title bar has the secondary effect of helping to show which application is currently active. The key window isthe most prominently marked window in the active application, making it “key” in a second sense: it’s the main focus of the user’s attention on the screen. The Main Window The main window is the standard window where the user is currently working. The main window is not always the key window. There are times when a window other than the main window takes the focus of the input device, while the main window still remains the focus of the user’s attention and of user actions carried out in panels and menus. For example, when a person is using an inspector, a Find dialog, or the Fonts or Colors windows, the document is the main window and the other window is the key window. The Find panel requires Window Layering and Types of Windows Key and Main Windows 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 20the user to supply information by typing it. Since the panel is the destination of the user’s keystrokes, it’s marked as the key window. But the panel is just an instrument through which users can do work in another window—the main window. In a document-based application, the main window isthe window for the current document. Whenever a standard window becomesthe key window, it also becomesthe main window. When key-window status shifts from a standard window to a panel, main-window status remains with the standard window. So that users can pick out the main window when it’s not the key window, the Application Kit highlights its title bar and colorsthe window buttons. If the main window is also the key window, it has only the highlighting of the key window. A menu command might affect either the key window or the main window, depending on the command. For example, the Paste command can be used to enter text in a Find panel. But the Save command saves the document displayed in the main window, and the Bold command turns the current selection in the main window bold. For this reason, user actions in a panel or menu are associated with both the key window and the main window: ● An action is first associated with the key window. ● If the key window is a panel and it can’t handle the action, the action is next associated with the main window. Note that this order of precedence is reflected in the way windows are highlighted: The key window is always marked, but the main window is marked only when it’s not the key window. The main window is always in the same application as the key window, the active application. Changing a Window’s Status Windows that are already onscreen automatically change their status as the key or main window based on the user’s actions with the mouse and on how clicked views handle those mouse events. You can also set the key and main windows programmatically by sending the relevant windows a makeKeyWindow or makeMainWindow message. Setting the key and main windows programmatically is particularly useful when creating a new window. Because making a window key is often combined with ordering the window to the front of the screen, the NSWindow class defines a convenience method, makeKeyAndOrderFront:, that performs both operations. Not all windows are suitable as key or main windows. For example, a window that merely displays information and contains no objectsthat need to respond to events or action messages can completely forgo ever becoming the key window. Similarly, a window that acts as a floating palette of itemsthat are only dragged out by mouse actions never needs to be the key window. Such a window can be defined as a subclass of NSWindow that overridesthe methods canBecomeKeyWindow and canBecomeMainWindow to return NO instead of the default of YES. Defining a window this way prevents it from ever becoming the key or main window. Although the NSWindow class defines these methods, only subclasses of NSPanel typically refuse to accept key or main window status. Window Layering and Types of Windows Key and Main Windows 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 21Windows can be placed on the screen in three dimensions. Besides horizontal and vertical placement, windows are layered back-to-front within distinct levels. Each application and document window exists in its own layer, so documents from different applications can be interleaved. Clicking a window to bring it to the front doesn’t disturb the layering order of any other window. A window’s depth in the layers is determined by when the window was last accessed. When a user clicks an inactive document or chooses it from the Window menu, only that document and any open utility windows should be brought to the front. Window Levels Windows are ordered within several distinct levels. Window levels group windows ofsimilar type and purpose so that the more “important” ones(such as alert panels) appear in front of those lesser importance. A window’s level serves as a high-order bit to determine its position with regard to other windows. Windows can be reordered with respect to each other within a given level; a given window, however, cannot be layered above other windows in a higher level. There are a number of predefined window levels, specified by constants defined by the NSWindow class. The levels you typically use are: NSNormalWindowLevel, which specifies the default level; NSFloatingWindowLevel, which specifiesthe level for floating palettes; and NSScreenSaverWindowLevel, which specifies the level for a screen saver window. You might also use NSStatusWindowLevel for a status window, or NSModalPanelWindowLevel for a modal panel. If you need to implement your own popup menus you use NSPopUpMenuWindowLevel. The remaining two levels, NSTornOffMenuWindowLevel and NSMainMenuWindowLevel, are reserved for system use. Setting Ordering and Level Programmatically You can use the orderWindow:relativeTo: method to order a window within its level in front of or in back of another window. You more typically use convenience methods to specify ordering, such as makeKeyAndOrderFront: (which also affectsstatus), orderFront:, and orderBack:, as well as orderOut:, which removes a window from the screen. You use the isVisible method to determine whether a window is on or off the screen. You can also set a window to be removed from the screen automatically when its application isn’t active using setHidesOnDeactivate:. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 22 Window Layers and LevelsTypically you should have no need to programmatically set the level of a window, since Cocoa automatically determines the appropriate level for a window based on its characteristics. A utility panel, for example, is automatically assigned to NSFloatingWindowLevel. You can nevertheless set a window’s level using the setLevel: method; for example, you can set the level of a standard window to NSFloatingWindowLevel if you want a utility window that looks like a standard window (for example to act as an inspector). This has two disadvantages, however: firstly, it may violate the human interface guidelines; secondly, if you assign a window to a floating level, you must ensure that you also set it to hide on deactivation of your application or reset its level when your application is hidden. Cocoa automatically takes care of the latter aspect for you if you use default window configurations. There is currently no level specified to allow you to place a window above a screen saver window. If you need to do this (for example, to show an alert while a screen saver is running), you can set the window’s level to be greater than that of the screen saver, as shown in the following example. [aWindow setLevel:NSScreenSaverWindowLevel + 1]; Other than this specific case, you are discouraged from setting windows in custom levels since this may lead to unexpected behavior. Window Layers and Levels Setting Ordering and Level Programmatically 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 23The are a number of different options that can be set regarding the window collection behavior of a window. They include a window’s behavior when using Spaces, Exposé, and the “Cycle Through Windows” command. These options can be set using the setCollectionBehavior: method of NSWindow, by passing in at most one constant from each group, combined using bitwise or operators. The current options may be accessed via the collectionBehavior method. Spaces Collection Behavior There are three options that can be set for a window’s Spaces collection behavior. The default is NSWindowCollectionBehaviorDefault, which allows the window to be associated with one space at a time. The second option is NSWindowCollectionBehaviorCanJoinAllSpaces. This option causes the window to appear on all spaces, like the menu bar. The third option is NSWindowCollectionBehaviorMoveToActiveSpace. This causesthe window to switch to the active space when it is made active. Only one of these options may be used at a time. If a window is currently associated with the active space, isOnActiveSpace returns YES. Otherwise, it returns NO. Additionally, you can get an array of the window numbers of windows on one or all spaces using the method windowNumbersWithOptions: and specified your desired options. The possible options are specified by NSWindowNumberListOptions. Exposé Collection Behavior There are also three options that can be set for a window’s Exposé collection behavior. If a window has a window level of NSNormalWindowLevel, the default behavior is NSWindowCollectionBehaviorManaged, which causes the window to participate in both Spaces and Exposé. NSWindowCollectionBehaviorTransient causes the window to float in Spaces and be hidden in Exposé. This is the default behavior if the window level is not NSNormalWindowLevel. The final option is NSWindowCollectionBehaviorStationary, which causes the window to be unaffected by Exposé; i.e. it stays visible and does not move, like the desktop window. Only one of these options may be used at a time. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 24 Setting Window Collection BehaviorWindow Cycling Behavior There are two options: NSWindowCollectionBehaviorParticipatesInCycle and NSWindowCollectionBehaviorIgnoresCycle. These options cause the window to participate in the window cycle for the “Cycle Through Windows” menu option or not participate in it, respectively. Setting Window Collection Behavior Window Cycling Behavior 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 25This article describes how to control a window’s size and position, including how to set a window’s minimum and maximum size, how to constrain a window to the screen, how to cascade windowsso their title barsremain visible, how to zoom a window as though the user pressed the zoom button, and how to center a window on the screen. Setting a Window’s Size and Location The center method places a window in the most prominent location on the screen, one suitable for important messages and alert dialogs. You can resize or reposition a window using setFrame:display: or setFrame:display:animate:—the former is equivalent to the latter with the animate flag NO. You might use these methodsin particular to expand or contract a window to show or hide a subview (such as a control that may be exposed by clicking a disclosure triangle). If the animate argument in setFrame:display:animate: is YES, the method performs a smooth resize of the window, where the total time for the resize can be obtained by calling animationResizeTime:. The user can resize windows by clicking and dragging on the bottom right corner of the window. While the user is resizing the window, inLiveResize will return YES. Otherwise, it returns NO. The user can generally reposition windows by dragging only the title bar. If you want usersto be able to drag your window by clicking elsewhere, you should override mouseDownCanMoveWindow so that it returns YES in any views that you want to be draggable window regions. The methods isMovable and setMovable: determine whether the user can move the window by clicking in its title bar or background. To keep the window’s top-left hand corner fixed when resizing, you must typically also reposition the origin, as illustrated in the following example. - (IBAction)showAdditionalControls:sender { NSRect frame = [myWindow frame]; if (frame.size.width <= MIN_WIDTH_WITH_ADDITIONS) frame.size.width = MIN_WIDTH_WITH_ADDITIONS; frame.size.height += ADDITIONS_HEIGHT; frame.origin.y -= ADDITIONS_HEIGHT; 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 26 Sizing and Placing Windows[myWindow setFrame:frame display:YES animate:YES]; // implementation continues... Note that the window’s delegate does not receive windowWillResize:toSize: messages when the window is resized in this way. It is your responsibility to ensure that the window’s new size is acceptable. The window’s delegate doesreceive windowDidResize: messages. You can implement windowDidResize: to add or remove subviews at suitable junctures. There are no additional flags to denote that the window is performing an animated resize operation (as distinct from a user-initiated resize). It is therefore up to you to capture relevant state information so that you can update the window contents appropriately in windowDidResize:. Window Cascading If you use the Cocoa document architecture, you can use the setShouldCascadeWindows: method of NSWindowController to set whether the window, when it is displayed, should cascade in relation to other document windows(that is, have a slightly offset location so that the title bars of previously displayed windows are still visible). The default is true, so typically you have no additional work to perform. If you are not using the document architecture, you can use the cascadeTopLeftFromPoint: method of NSWindow to cascade windows yourself. The method returns a point shifted from the top-left corner of the window that can be passed to a subsequent invocation of cascadeTopLeftFromPoint: to position the next window so the title bars of both windows are fully visible. Window Zooming You use the zoom: method to toggle the size and location of a window between its standard state, as determined by the application, and its user state: a new size and location the user may have set by moving or resizing the window. Constraining a Window’s Size and Location You can use setContentMinSize: and setContentMaxSize: to limit the user’s ability to resize the window—note that you can still set it to any size programmatically. Similarly, you can use setContentAspectRatio: to keep a window’s width and height at the same proportions as the user resizes it, and setContentResizeIncrements: to make the window resize in discrete amountslarger than a single pixel. (Aspect ratio and resize increments are mutually exclusive attributes.) In general, you should use the Sizing and Placing Windows Constraining a Window’s Size and Location 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 27setContent... methodsinstead of those that affect the window’sframe (setAspectRatio:, setMaxSize:, and so on). These are preferred because they avoid confusion for windows with toolbars, and also are typically a better model since you control the content of the window but not the frame. You can use the constrainFrameRect:toScreen: method to adjust a proposed frame rectangle so that it lies on the screen in such a way that the user can move and resize a window. However, you should make sure your window fits onscreen before display. Note that any NSWindow with a title bar automatically constrains itself to the screen. The cascadeTopLeftFromPoint: method shifts the top left point by an amount that allows one window to be placed relative to another so that both their title bars are visible. Additionally, when a window is about to be resized, the window’s delegate will be sent a windowWillResize:toSize: message. You can implement that method in your delegate to easily control your window’s size. Sizing and Placing Windows Constraining a Window’s Size and Location 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 28A window can store its placement in the user defaults system, so that it appears in the same location the next time the user starts the application. The saveFrameUsingName: method stores the frame rectangle, and setFrameUsingName: setsit from the value in user defaults. You can also use the setFrameAutosaveName: method to have a window save the frame rectangle any time it changes. However, for the correct frame to be saved, you must ensure that the window controller for the window in question doesn’t cascade the windows under its charge. You accomplish this task by sending setShouldCascadeWindows:NO to the controller, as shown in Listing 1. Listing 1 Saving a window’s frame automatically NSWindow *window = // the window in question [[window windowController] setShouldCascadeWindows:NO]; // Tell the controller to not cascade its windows. [window setFrameAutosaveName:[window representedFilename]]; // Specify the autosave name for the window. To expunge a frame rectangle from the defaults system, use the class method removeFrameUsingName:. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 29 Saving a Window’s Position into the User’s DefaultsWhen a user minimizes a window, it’s removed from the screen and replaced with a smaller counterpart in the Dock. The miniaturize: and deminiaturize: methods reduce and reconstitute a window, and performMiniaturize: simulatesthe user clicking the window’s minimize button. You can also set the image and title displayed in a freestanding mini-window by sending setMiniwindowImage: and setMiniwindowTitle: messages to the NSWindow object. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 30 Minimizing WindowsMost Cocoa applications include the Window menu, which displays the titles of various of the application’s windows. When you change a window’s title, this change is automatically reflected in the Window menu. This menu automatically lists windowsthat have a title bar and are resizable and that can become the main window (as described in “Window Layering and Types of Windows” (page 18)). Typically you can rely on the automatic updating provided by Cocoa. In rare circumstances, however, you might want to modify the default behavior. You can exclude a window that would otherwise be listed in the Window menu by sending it a setExcludedFromWindowsMenu:YES message. Since they cannot become main, NSPanel objects are excluded from the Windows menu. Instances of subclasses of NSPanel can be included in the menu by returning NO from its isExcludedFromWindowsMenu method and YES from its canBecomeMainWindow method. If you change a window’s configuration such that it should be added to or removed from the Window menu, you can update the Window menu by sending the shared application instance addWindowsItem:title:filename: or removeWindowsItem:. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 31 Using the Window MenuYou usually configure most aspects of a window’s appearance in Interface Builder. Sometimes, however, you may need to create a window programmatically, or alter its appearance after it has been created. Setting a Window’s Style The peripheral elements that a window displays define its style. Though you can’t access and manipulate them directly, you can determine at initialization whether a window has them by providing a style mask to the initializer. There are four possible style elements,specifiable by combining their mask values using the C bitwise OR operator: Element Mask Value A title bar NSTitledWindowMask A close button NSClosableWindowMask A minimize button NSMiniaturizableWindowMask A resize bar, border, or box NSResizableWindowMask You can also specify NSBorderlessWindowMask, in which case none of these style elements is used. Typically, you set a window’s appearance once, when it is first created. Sometimes, however, you want to enable or disable a button in the title bar to reflect changed context. To do this, you first retrieve the button from the window using the standardWindowButton: of NSWindow method and then set its enabled state, as in the following example. NSButton *closeButton = [window standardWindowButton:NSWindowCloseButton]; [closeButton setEnabled:NO]; The constants required to access standard title bar widgets are defined in the API reference for NSWindow. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 32 Setting a Window’s AppearanceSetting a Window’s Color and Transparency You can set a window’s background color and transparency using the methods setBackgroundColor: and setAlphaValue:, respectively. You can set a window’s background color to a non-opaque color. This does not affect the window’s title bar; it only makes the background itself transparent if the window is not opaque, as illustrated in the following example. [myWindow setOpaque:NO]; // YES by default NSColor *semiTransparentBlue = [NSColor colorWithDeviceRed:0.0 green:0.0 blue:1.0 alpha:0.5]; [myWindow setBackgroundColor:semiTransparentBlue]; Views placed on a non-opaque window with a transparent background color retain their own opacity. If you want to make the entire window (including the title bar and views placed on the window) transparent, you should use setAlphaValue:. Setting a Window’s Color Space You can set a window’s colorspace using setColorSpace: and can retrieve the window’s current colorspace using colorSpace. NSColorSpace objects for use with setColorSpace: may be obtained using the class methods documented in NSColorSpace Class Reference . Setting a Window’s Content Border Thickness Beginning in OS X version 10.5, windows automatically have a textured gradient applied to their backgrounds. The area on which the gradient is drawn is determined automatically. At times, however, this may not work correctly. If your window does not look correct with automatic gradient calculation, disable it by calling setAutorecalculatesContentBorderThickness:forEdge: with a value of NO and the edge to disable automatic calculation for. The value of this property may be accessed using the method autorecalculatesContentBorderThicknessForEdge:. You can also set and access the content border thickness manually using setContentBorderThickness:forEdge: and contentBorderThicknessForEdge:, respectively. Setting a Window’s Appearance Setting a Window’s Color and Transparency 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 33A titled window can display an arbitrary title or one derived from a filename. The setTitle: method puts an arbitrary string on the title bar. The setTitleWithRepresentedFilename: method formats a filename in the title bar in a readable format and associates the window with that file. You can set the associated file without changing the title using setRepresentedFilename:. You can use the association between the window and the file in any way you see fit. One convenience offered by the NSWindow class is marking the file as having been changed, so that the user is prompted to save it on closing the window. The method for marking the document as having been changed is setDocumentEdited:. When the window closes, its delegate can check if the files has been changed using isDocumentEdited to see whether the document needs to be saved. Additionally, starting in OS X version 10.5, you can set a window’s represented document by URL using the setRepresentedURL: method. You can get the URL of the document currently represented by a window using the representedURL method. The window will automatically use the known icon for the file type of the specified file, if one exists. To customize the document icon, you can use the following code segment: [[NSWindow standardWindowButton:NSWindowDocumentIconButton] setImage:customImage]. By default, a Command-click or Control-click on the rectangle containing a window’s document icon button and title will show a path popup. To customize this behavior, you can implement window:shouldPopUpDocumentPathMenu: in your window’s delegate. You can return NO from this method to stop the window from showing the path popup. You can also customize the document icon’s default drag behavior by implementing the window:shouldDragDocumentWithEvent:from:withPasteboard: in the window’s delegate. You can return NO to prohibit dragging the document icon. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 34 Setting a Window’s Title and Represented FileNearly every window has a corresponding display window device in the window server. The window device holdsthe window’s drawn image, and hastwo attributes determined by the window server and many attributes that the window controls. The window server assigns the window device a unique identifier (within an application). This is the window number, and it can be accessed using the windowNumber method. Each window also has a graphics state that most of its views share for drawing (views can create their own as well). The gState method returns its identifier. The attributes under direct window control are the following: ● Backing store type, described in “Specifying How To Store the Window’s Image” (page 35) ● Backing location, described in “Specifying Where To Store the Window’s Image” (page 36) ● Window device creation, described in “Specifying When the Window’s Image Is Created” (page 36) ● One shot, described in “Specifying Whether the Window’s Image Persists When Offscreen” (page 37) ● Depth limit, described in “Specifying the Depth Limit for the Window’s Image” (page 37) ● Dynamic depth limit, described in “Specifying Whether the Depth Limit Changes to the Screen’s Capacity” (page 37) ● Content sharing, described in “Specifying Whether Window Content Can Be Read or Written by Another Process” (page 37). Specifying How To Store the Window’s Image A window device’s backing store type determines how the window’simage isstored. It’sset when the window is initialized and can be one of three types. A buffered window device renders all drawing into a display buffer and then flushes it to the screen. Always drawing to the buffer produces very smooth display, but can require significant amounts of memory. Buffered windows are best for displaying material that must be redrawn often, such as text. You must also use buffered windows if you want your windows to support transparency. A retained window device also uses a buffer, but draws directly to the screen where possible and to the buffer for any portions that are obscured. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 35 Setting Attributes for the Window’s ImageA nonretained window device has no buffer at all, and must redraw portions as they’re exposed. Further, this redrawing is suspended when the window’s display mechanism is preempted. For example, if the user drags a window across a nonretained window, the nonretained window is “erased” and isn’t redrawn until the user releases the mouse. Both retained and nonretained windows are also subject to a flashing effect as individual drawing operations are performed, but their results do get to the screen more quickly than those of buffered windows. You can change the backing store type between buffered and retained after initialization using the setBackingType: method. Specifying Where To Store the Window’s Image The window server chooses whether to place the backing store for a buffered window in main memory or video memory. It will choose the location that providesthe best overall performance. You can query the window server to determine where your window’s backing store is located using the preferredBackingLocation method. You may choose to set a preferred location for a Window’s backing store using the setPreferredBackingLocation: method. While the window server is not required to respect this preferred backing location, it will attempt to do so. You should not change the preferred backing location without testing how it affects the performance of your application. Specifying When the Window’s Image Is Created The defer argument to the initializer specifies whether the window creates its window device immediately or only when it’s moved on screen. Deferring creation of the window device can offer some performance gain for windows that aren’t displayed immediately because it reduces the amount of work that needs to be performed up front. Deferring creation of the window device is particularly useful when creation of the window itself can’t be deferred or when an window is needed for purposes other than displaying content. Submenus with key equivalents, for example, must exist for the key equivalents to work, but may never actually be displayed. Setting Attributes for the Window’s Image Specifying Where To Store the Window’s Image 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 36Specifying Whether the Window’s Image Persists When Offscreen Memory can also be saved by destroying the window device when the window is removed from the screen. The setOneShot: method controls this behavior. One-shot window devices exist only when their windows are onscreen. Specifying the Depth Limit for the Window’s Image Like the display hardware, a window device’s buffer has a depth, or a limit to the memory allotted each pixel. Buffered and retained windows start out with the same depth as the main display or 16 bits, whichever is deeper. These settings stay in effect unless changed using the setDepthLimit: method, which takes as an argument a window depth limit created using the NSBestDepth function. SpecifyingWhether the Depth Limit Changesto the Screen’s Capacity Keeping a window’s depth at its richest preserves the displayed image, but may incur unnecessary memory overhead when the window buffer depth is deeper than the screen depth. You can use the setDynamicDepthLimit: method to tell a window to match the depth of the screen it’s on. When it’s moved to a new screen, a window with a dynamic depth limit adjusts its buffer to the new depth before redrawing. Making a window’s depth limit dynamic overrides the limit set using setDepthLimit:, and removing the dynamic limit reverts the window to the default limit. Specifying Whether Window Content Can Be Read or Written by Another Process The contents of your window can be made available to other processes. By default, the contents of your window can be read but not written to by other processes. This allows system services to work with your window’s contents and also allows other applications to capture a snapshot of your windows contents. You can override the default behavior using the setSharingType: method. Changing the sharing type to NSWindowSharingNone prevents other systems from capturing your window’s image data. If you do this, however, your window will not be able to participate in a number of system services; therefore, this setting should be used with caution. If you set your window’s sharing type to NSWindowSharingReadWrite, other processes can both read and modify the window’s content. Setting Attributes for the Window’s Image Specifying Whether the Window’s Image Persists When Offscreen 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 37As described in NSResponder Class Reference , most events coming into an application make their way to a window in a sendEvent: message. A key event is directed at the key window, while a mouse event is directed at whatever window lies under the pointer. If an event affects the window directly—resizing or moving it, for example—it performs the appropriate operation itself and sends messages to its delegate informing it of its intentions, thus allowing your application to intercede. The window sends other events up its responder chain from the appropriate starting point: the first responder for a key event, the view under the pointer for a mouse event. These events are then typically handled by some view object in the window. See Cocoa Event Handling Guide for more information on how to intercept and handle events. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 38 Handling Events in WindowsA window’s first responder is often a view object selected by the user clicking it. For text fields and other view objects (mainly subclasses of NSControl), the user can select the first responder with the keyboard using the Tab and Shift keys. The NSView class defines the methods for setting up and examining the loop of objects that the user can select in this manner. A view that’s the first responder is called the key view, and the views that can become the key view in a window are linked together in the window’s key view loop. You normally set up the key view loop using Interface Builder, establishing connections between the nextKeyView outlets of views in the window and setting the window’s initialFirstResponder outlet to the view that you want selected when the window is first placed onscreen. If you do not set this outlet, the window sets a key loop (not necessarily the same as the one you would have specified!) and picks a default initial first responder for you. In addition to the key view loop, a window can have a default button cell, which uses the Return (or Enter) key as its key equivalent. The setDefaultButtonCell: method establishes this button cell; you can also set it in Interface Builder by setting a button cell’s key equivalent to '\r'. The default button cell draws itself as a focal element for keyboard interface control unless another button cell is focused on. In this case, it temporarily draws itself as normal and disables its key equivalent. Another default key established by the NSWindow class is the Escape key, which immediately aborts a modal loop (described in “How Modal Windows Work” (page 12)). See NSResponder Class Reference for more information on keyboard interface control. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 39 Using Keyboard Interface Control in WindowsEach window has a text object that is shared for light editing tasks. This object, the window’s field editor, is inserted in the view hierarchy when an object needsto editsome text and removed when the object isfinished. The field editor is used by NSTextField objects and other controls, for example, to edit the text that they display. The fieldEditor:forObject: method returns a window’s field editor, after asking the delegate for a substitute using windowWillReturnFieldEditor:toObject:. You can override the fieldEditor:forObject: method of NSWindow in subclasses or provide a delegate to substitute a class of text object different from the NSTextView default, thereby customizing text editing in your application. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 40 Using the Window’s Field EditorThe NSWindow class offers observers a rich set of notifications, which it broadcasts on such occurrences as gaining or losing key or main window status, minimizing, moving or resizing, becoming exposed, and closing. Each notification is matched to a delegate method, so a window’s delegate is automatically registered for all notifications that it has methods for. The NSWindow class also offers its delegate a few other methods, such as windowShouldClose:, which requests approval to close, windowWillResize:toSize:, which allows the delegate to constrain the window’ssize, windowWillUseStandardFrame:defaultFrame:, which allows the delegate to set the window frame for zooming, and windowWillReturnFieldEditor:toObject:, which gives the delegate a chance to modify the field editor or substitute a different editor. See the individual notification and delegate method descriptions for more information. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 41 Using Window Notifications and Delegate MethodsThe NSWindow class defines some methods for image dragging, in case the user wants to drag an object into or out of a window. Although most dragging operations are initiated by and occur between view objects, the NSWindow class also defines an image-dragging method, dragImage:at:offset:event:pasteboard:source:slideBack:. A window can also serve as the destination for dragging operations, registering the types it accepts with registerForDraggedTypes: and unregisterDraggedTypes. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 42 Dragging Images to and from WindowsYou can change the cursor image when the cursor is within a specified area of a view in a window. To do this, use the NSTrackingArea class, along with the cursorUpdate: method of the NSResponder class. For specifics, read “Using Tracking-Area Objects” in Cocoa Event Handling Guide . For details on the NSTrackingArea class itself, refer to NSTrackingArea Class Reference . 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 43 Updating the Cursor Image in a WindowTo support transitory drawing by views, the NSWindow class defines methods that temporarily cache a portion of its raster image so that it can be restored later. This feature is useful for situations where highly dynamic drawing must be done over the otherwise static image of the window. For example, in a drawing program where the user drags lines and other shapes directly onto a canvas, it’s more efficient to restore the window’s cached image and draw anew over that than to have all the views send display instructions to the window server. For more information,see the method descriptionsfor cacheImageInRect:, restoreCachedImage, and discardCachedImage. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 44 Caching Window ImagesThis table describes the changes to Window Programming Guide . Date Notes Revised the article “Updating the Cursor Image in a Window” (page 43), previously titled “Setting Pointer Rectangles for Windows.” 2009-11-27 2009-05-15 Updated for OS X v10.6. Added information on the use of backing locations to improve performance. 2009-02-04 2008-10-15 Provided links to delegate methods. Clarified the behavior of the setFrameAutosaveName: method in conjunction with a window's window controller. 2006-10-03 Added window-controller requirement for the NSWindow setFrameAutosaveName: method to “Saving a Window’s Position into the User’s Defaults” (page 29). Made correction to "Using the Windows Menu" article. Changed title from "Windows and Panels." 2005-09-08 Updated “Setting a Window’s Appearance” (page 32) to cover enabling and disabling buttons in the title bar, and to discuss setting a window’s background color and transparency. 2004-08-31 “Setting a Window’s Level” renamed “Window Layers and Levels” (page 22) and augmented. “Changing the Key and Main Windows” renamed to “Window Layering and Types of Windows” (page 18) and augmented. 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 45 Document Revision HistoryDate Notes Augmented “Sizing and Placing Windows” (page 26) to discuss animated resizing, window cascading, and constraining window size and position. Minor changes to “Using the Window Menu” (page 31). Clarified the concepts of key and main windowsin “Window Layering and Types of Windows” (page 18)“. 2003-06-05 2002-11-12 Revision history was added to existing topic. Document Revision History 2009-11-27 | © 2002, 2009 Apple Inc. All Rights Reserved. 46Apple Inc. © 2002, 2009 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, Cocoa, Exposé, Mac, Numbers, OS X, and Spaces are trademarks of Apple Inc., registered in the U.S. and other countries. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. AV Foundation Programming GuideContents About the AV Foundation Framework 4 At a Glance 5 Representing and Using Media with AV Foundation 5 Concurrent Programming with AV Foundation 7 Prerequisites 8 Using Assets 9 Creating an Asset Object 9 Options for Initializing an Asset 9 Accessing the User’s Assets 10 Preparing an Asset for Use 11 Getting Still Images From a Video 12 Generating a Single Image 13 Generating a Sequence of Images 14 Trimming and Transcoding a Movie 15 Reading and Writing Assets 17 Playback 18 Playing Assets 18 Handling Different Types of Asset 20 Playing an Item 21 Changing the Playback Rate 21 Seeking—Repositioning the Playhead 22 Playing Multiple Items 23 Monitoring Playback 23 Responding to a Change in Status 24 Tracking Readiness for Visual Display 25 Tracking Time 25 Reaching the End of an Item 26 Putting it all Together: Playing a Video File Using AVPlayerLayer 26 The Player View 27 A Simple View Controller 27 Creating the Asset 28 Responding to the Player Item’s Status Change 30 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 2Playing the Item 31 Media Capture 32 Use a Capture Session to Coordinate Data Flow 33 Configuring a Session 34 Monitoring Capture Session State 35 An AVCaptureDevice Object Represents an Input Device 35 Device Characteristics 36 Device Capture Settings 36 Configuring a Device 40 Switching Between Devices 41 Use Capture Inputs to Add a Capture Device to a Session 41 Use Capture Outputs to Get Output from a Session 42 Saving to a Movie File 43 Processing Frames of Video 46 Capturing Still Images 47 Showing the User What’s Being Recorded 49 Video Preview 49 Showing Audio Levels 50 Putting it all Together: Capturing Video Frames as UIImage Objects 50 Create and Configure a Capture Session 51 Create and Configure the Device and Device Input 51 Create and Configure the Data Output 52 Implement the Sample Buffer Delegate Method 52 Starting and Stopping Recording 53 Time and Media Representations 54 Representation of Assets 54 Representations of Time 55 CMTime Represents a Length of Time 55 CMTimeRange Represents a Time Range 57 Representations of Media 58 Converting a CMSampleBuffer to a UIImage 59 Document Revision History 62 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 3 ContentsAV Foundation is one of several frameworks that you can use to play and create time-based audiovisual media. It provides an Objective-C interface you use to work on a detailed level with time-based audiovisual data. For example, you can use it to examine, create, edit, or reencode media files. You can also get input streams from devices and manipulate video during realtime capture and playback. Core Audio UIKit Media Player AV Foundation iOS 3 Audio classes Core Media Core Animation You should typically use the highest-level abstraction available that allows you to perform the tasks you want. For example, in iOS: ● If you simply want to play movies, you can use the Media Player Framework (MPMoviePlayerController or MPMoviePlayerViewController), or for web-based media you could use a UIWebView object. ● To record video when you need only minimal control over format, use the UIKit framework (UIImagePickerController). Note, however, thatsome of the primitive data structuresthat you use in AV Foundation—including time-related data structures and opaque objectsto carry and describemedia data—are declared in the Core Media framework. AV Foundation is available in iOS 4 and later, and OS X 10.7 and later. This document describes AV Foundation as introduced in iOS 4.0. To learn about changes and additions to the framework in subsequent versions, you should also read the appropriate release notes: ● AV Foundation Release Notes describe changes made for iOS 5. ● AV Foundation Release Notes (iOS 4.3) describe changes made for iOS 4.3 and included in OS X 10.7. 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 4 About the AV Foundation FrameworkRelevant Chapters: “Time and Media Representations” (page 54) At a Glance There are two facets to the AV Foundation framework—API related just to audio, which was available prior to iOS 4; and API introduced in iOS 4 and later. The older audio-related classes provide easy ways to deal with audio. They are described in Multimedia Programming Guide , not in this document. ● To play sound files, you can use AVAudioPlayer. ● To record audio, you can use AVAudioRecorder. You can also configure the audio behavior of your application using AVAudioSession; this is described in Audio Session Programming Guide . Representing and Using Media with AV Foundation The primary class that the AV Foundation framework uses to represent media is AVAsset. The design of the framework is largely guided by this representation. Understanding its structure will help you to understand how the framework works. An AVAsset instance is an aggregated representation of a collection of one or more pieces of media data (audio and video tracks). It provides information about the collection as a whole, such as its title, duration, natural presentation size, and so on. AVAsset is not tied to particular data format. AVAsset is the superclass of other classes used to create asset instances from media at a URL (see “Using Assets” (page 9)) and to create new compositions (see “Editing” (page 7)). Each of the individual pieces of media data in the asset is of a uniform type and called a track. In a typical simple case, one track represents the audio component, and another represents the video component; in a complex composition, however, there may be multiple overlapping tracks of audio and video. Assets may also have metadata. A vital concept in AV Foundation is that initializing an asset or a track does not necessarily mean that it is ready for use. It may require some time to calculate even the duration of an item (an MP3 file, for example, may not contain summary information). Rather than blocking the current thread while a value is being calculated, you ask for values and get an answer back asynchronously through a callback that you define using a block. About the AV Foundation Framework At a Glance 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 5Relevant Chapters: “Using Assets” (page 9) “Time and Media Representations” (page 54) Playback AVFoundation allows you to manage the playback of asset in sophisticated ways. To support this, it separates the presentation state of an asset from the asset itself. This allows you to, for example, play two different segments of the same asset at the same time rendered at different resolutions. The presentation state for an asset is managed by a player item object; the presentation state for each tracks within an asset is managed by a player item track objects. Using the player item and player item tracks you can, for example, set the size at which the visual portion of the item is presented by the player, set the audio mix parameters and video composition settings to be applied during playback, or disable components of the asset during playback. You play player items using a player object, and direct the output of a player to Core Animation layer. On iOS 4.1 and later, you can use a player queue to schedule playback of a collection of player items in sequence. Relevant Chapters: “Playback” (page 18) Reading, Writing, and Reencoding Assets AV Foundation allows you to create new representations of an asset in several ways. You can simply reencode an existing asset, or—on iOS 4.1 and later—you can perform operations on the contents of an asset and save the result as a new asset. You use an export session to reencode an existing asset into a format defined by one of a small number of commonly-used presets. If you need more control over the transformation, on iOS 4.1 and later you can use an asset reader and asset writer object in tandem to convert an asset from one representation to another. Using these objects you can, for example, choose which of the tracks you want to be represented in the output file, specify your own output format, or modify the asset during the conversion process. To produce a visual representation of the waveform, you use an asset reader to read the audio track of an asset. About the AV Foundation Framework At a Glance 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 6Relevant Chapters: “Using Assets” (page 9) Thumbnails To create thumbnail images of video presentations, you initialize an instance of AVAssetImageGenerator using the asset from which you want to generate thumbnails. AVAssetImageGenerator uses the default enabled video track(s) to generate images. Relevant Chapters: “Using Assets” (page 9) Editing AV Foundation uses compositions to create new assets from existing pieces of media (typically, one or more video and audio tracks). You use a mutable composition to add and remove tracks, and adjust their temporal orderings. You can also set the relative volumes and ramping of audio tracks; and set the opacity, and opacity ramps, of video tracks. A composition is an assemblage of pieces of media held in memory. When you export a composition using an export session, it's collapsed to a file. On iOS 4.1 and later, you can also create an asset from media such as sample buffers or still images using an asset writer. Media Capture and Access to Camera Recording input from cameras and microphonesis managed by a capture session. A capture session coordinates the flow of data from input devices to outputs such as a movie file. You can configure multiple inputs and outputs for a single session, even when the session is running. You send messages to the session to start and stop data flow. In addition, you can use an instance of preview layer to show the user what a camera is recording. Relevant Chapters: “Media Capture” (page 32) Concurrent Programming with AV Foundation Callouts from AV Foundation—invocations of blocks, key-value observers, or notification handlers—are not guaranteed to be made on any particular thread or queue. Instead, AV Foundation invokes these handlers on threads or queues on which it performs its internal tasks. You are responsible for testing whether the thread or queue on which a handler isinvoked is appropriate for the tasks you want to perform. If it’s not (for example, if you want to update the user interface and the callout is not on the main thread), you must redirect the execution of your tasks to a safe thread or queue that you recognize, or that you create for the purpose. About the AV Foundation Framework At a Glance 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 7If you’re writing amultithreaded application, you can use the NSThreadmethod isMainThread or [[NSThread currentThread] isEqual:<#A stored thread reference#>] to test whether the invocation thread is a thread you expect to perform your work on. You can redirect messages to appropriate threads using methods such as performSelectorOnMainThread:withObject:waitUntilDone: and performSelector:onThread:withObject:waitUntilDone:modes:. You could also use dispatch_async(3) OS X Developer Tools Manual Page to “bounce”to your blocks on an appropriate queue, either the main queue for UI tasks or a queue you have up for concurrent operations. For more about concurrent operations, see Concurrency Programming Guide ; for more about blocks, see Blocks Programming Topics. Prerequisites AV Foundation is an advanced Cocoa framework. To use it effectively, you must have: ● A solid understanding of fundamental Cocoa development tools and techniques ● A basic grasp of blocks ● A basic understanding of key-value coding and key-value observing ● For playback, a basic understanding of Core Animation (see Core Animation Programming Guide ) About the AV Foundation Framework Prerequisites 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 8Asset can come from a file or from media in the user’s iPod Library or Photo library. Simply creating an asset object, though, does not necessarily mean that all the information that you might want to retrieve for that item is immediately available. Once you have a movie asset, you can extract still images from it, transcode it to another format, or trim the contents. Creating an Asset Object To create an asset to represent any resource that you can identify using a URL, you use AVURLAsset. The simplest case is creating an asset from a file: NSURL *url = <#A URL that identifies an audiovisual asset such as a movie file#>; AVURLAsset *anAsset = [[AVURLAsset alloc] initWithURL:url options:nil]; Options for Initializing an Asset AVURLAsset’s initialization methods take as their second argument an options dictionary. The only key used in the dictionary is AVURLAssetPreferPreciseDurationAndTimingKey. The corresponding value is a boolean (contained in an NSValue object) that indicates whether the asset should be prepared to indicate a precise duration and provide precise random access by time. Getting the exact duration of an asset may require significant processing overhead. Using an approximate duration is typically a cheaper operation and sufficient for playback. Thus: ● If you only intend to play the asset, either pass nil instead of a dictionary, or pass a dictionary that contains the AVURLAssetPreferPreciseDurationAndTimingKey key and a corresponding value of NO (contained in an NSValue object). ● If you want to add the asset to a composition (AVMutableComposition), you typically need precise randomaccess. Pass a dictionary that containsthe AVURLAssetPreferPreciseDurationAndTimingKey key and a corresponding value of YES (contained in an NSValue object—recall that NSNumber inherits from NSValue): 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 9 Using AssetsNSURL *url = <#A URL that identifies an audiovisual asset such as a movie file#>; NSDictionary *options = @{ AVURLAssetPreferPreciseDurationAndTimingKey : @YES }; AVURLAsset *anAssetToUseInAComposition = [[AVURLAsset alloc] initWithURL:url options:options]; Accessing the User’s Assets To access the assets managed the iPod Library or by the Photos application, you need to get a URL of the asset you want. ● To access the iPod Library, you create an MPMediaQuery instance to find the item you want, then get its URL using MPMediaItemPropertyAssetURL. For more about the Media Library, see Multimedia Programming Guide . ● To access the assets managed by the Photos application, you use ALAssetsLibrary. The following example shows how you can get an asset to represent the first video in the Saved Photos Album. ALAssetsLibrary *library = [[ALAssetsLibrary alloc] init]; // Enumerate just the photos and videos group by using ALAssetsGroupSavedPhotos. [library enumerateGroupsWithTypes:ALAssetsGroupSavedPhotos usingBlock:^(ALAssetsGroup *group, BOOL *stop) { // Within the group enumeration block, filter to enumerate just videos. [group setAssetsFilter:[ALAssetsFilter allVideos]]; // For this example, we're only interested in the first item. [group enumerateAssetsAtIndexes:[NSIndexSet indexSetWithIndex:0] options:0 usingBlock:^(ALAsset *alAsset, NSUInteger index, BOOL *innerStop) { // The end of the enumeration is signaled by asset == nil. if (alAsset) { Using Assets Creating an Asset Object 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 10ALAssetRepresentation *representation = [alAsset defaultRepresentation]; NSURL *url = [representation url]; AVAsset *avAsset = [AVURLAsset URLAssetWithURL:url options:nil]; // Do something interesting with the AV asset. } }]; } failureBlock: ^(NSError *error) { // Typically you should handle an error more gracefully than this. NSLog(@"No groups"); }]; Preparing an Asset for Use Initializing an asset (or track) does not necessarily mean that all the information that you might want to retrieve for that item is immediately available. It may require some time to calculate even the duration of an item (an MP3 file, for example, may not contain summary information). Rather than blocking the current thread while a value is being calculated, you should use the AVAsynchronousKeyValueLoading protocol to ask for values and get an answer back later through a completion handler you define using a block. (AVAsset and AVAssetTrack conform to the AVAsynchronousKeyValueLoading protocol.) You test whether a value is loaded for a property using statusOfValueForKey:error:. When an asset is first loaded, the value of most or all of its properties is AVKeyValueStatusUnknown. To load a value for one or more properties, you invoke loadValuesAsynchronouslyForKeys:completionHandler:. In the completion handler, you take whatever action is appropriate depending on the property’s status. You should always be prepared for loading to not complete successfully, either because it failed for some reason such as a network-based URL being inaccessible, or because the load was canceled. . NSURL *url = <#A URL that identifies an audiovisual asset such as a movie file#>; AVURLAsset *anAsset = [[AVURLAsset alloc] initWithURL:url options:nil]; NSArray *keys = @[@"duration"]; [asset loadValuesAsynchronouslyForKeys:keys completionHandler:^() { Using Assets Preparing an Asset for Use 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 11NSError *error = nil; AVKeyValueStatus tracksStatus = [asset statusOfValueForKey:@"duration" error:&error]; switch (tracksStatus) { case AVKeyValueStatusLoaded: [self updateUserInterfaceForDuration]; break; case AVKeyValueStatusFailed: [self reportError:error forAsset:asset]; break; case AVKeyValueStatusCancelled: // Do whatever is appropriate for cancelation. break; } }]; If you want to prepare an asset for playback, you should load its tracks property. For more about playing assets, see “Playback” (page 18). Getting Still Images From a Video To get still images such as thumbnails from an asset for playback, you use an AVAssetImageGenerator object. You initialize an image generator with your asset. Initialization may succeed, though, even if the asset possesses no visual tracks at the time of initialization, so if necessary you should test whether the asset has any tracks with the visual characteristic using tracksWithMediaCharacteristic:. AVAsset anAsset = <#Get an asset#>; if ([anAsset tracksWithMediaCharacteristic:AVMediaTypeVideo]) { AVAssetImageGenerator *imageGenerator = [AVAssetImageGenerator assetImageGeneratorWithAsset:anAsset]; // Implementation continues... Using Assets Getting Still Images From a Video 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 12You can configure several aspects of the image generator, for example, you can specify the maximum dimensions for the images it generates and the aperture mode using maximumSize and apertureMode respectively.You can then generate a single image at a given time, or a series of images. You must ensure that you keep a strong reference to the image generator until it has generated all the images. Generating a Single Image You use copyCGImageAtTime:actualTime:error: to generate a single image at a specific time. AV Foundation may not be able to produce an image at exactly the time you request, so you can pass as the second argument a pointer to a CMTime that upon return contains the time at which the image was actually generated. AVAsset *myAsset = <#An asset#>]; AVAssetImageGenerator *imageGenerator = [[AVAssetImageGenerator alloc] initWithAsset:myAsset]; Float64 durationSeconds = CMTimeGetSeconds([myAsset duration]); CMTime midpoint = CMTimeMakeWithSeconds(durationSeconds/2.0, 600); NSError *error; CMTime actualTime; CGImageRef halfWayImage = [imageGenerator copyCGImageAtTime:midpoint actualTime:&actualTime error:&error]; if (halfWayImage != NULL) { NSString *actualTimeString = (NSString *)CMTimeCopyDescription(NULL, actualTime); NSString *requestedTimeString = (NSString *)CMTimeCopyDescription(NULL, midpoint); NSLog(@"Got halfWayImage: Asked for %@, got %@", requestedTimeString, actualTimeString); // Do something interesting with the image. CGImageRelease(halfWayImage); } Using Assets Getting Still Images From a Video 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 13Generating a Sequence of Images To generate a series of images, you send the image generator a generateCGImagesAsynchronouslyForTimes:completionHandler:message. The first argument is an array of NSValue objects, each containing a CMTime, specifying the asset times for which you want images to be generated. The second argument is a block that serves as a callback invoked for each image that is generated. The block arguments provide a result constant that tells you whether the image was created successfully or if the operation was canceled, and, as appropriate: ● The image. ● The time for which you requested the image and the actual time for which the image was generated. ● An error object that describes the reason generation failed. In your implementation of the block, you should check the result constant to determine whether the image was created. In addition, you must ensure that you keep a strong reference to the image generator until it has finished creating the images. AVAsset *myAsset = <#An asset#>]; // Assume: @property (strong) AVAssetImageGenerator *imageGenerator; self.imageGenerator = [AVAssetImageGenerator assetImageGeneratorWithAsset:myAsset]; Float64 durationSeconds = CMTimeGetSeconds([myAsset duration]); CMTime firstThird = CMTimeMakeWithSeconds(durationSeconds/3.0, 600); CMTime secondThird = CMTimeMakeWithSeconds(durationSeconds*2.0/3.0, 600); CMTime end = CMTimeMakeWithSeconds(durationSeconds, 600); NSArray *times = @[NSValue valueWithCMTime:kCMTimeZero], [NSValue valueWithCMTime:firstThird], [NSValue valueWithCMTime:secondThird], [NSValue valueWithCMTime:end]]; [imageGenerator generateCGImagesAsynchronouslyForTimes:times completionHandler:^(CMTime requestedTime, CGImageRef image, CMTime actualTime, AVAssetImageGeneratorResult result, NSError *error) { NSString *requestedTimeString = (NSString *) CFBridgingRelease(CMTimeCopyDescription(NULL, requestedTime)); Using Assets Getting Still Images From a Video 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 14NSString *actualTimeString = (NSString *) CFBridgingRelease(CMTimeCopyDescription(NULL, actualTime)); NSLog(@"Requested: %@; actual %@", requestedTimeString, actualTimeString); if (result == AVAssetImageGeneratorSucceeded) { // Do something interesting with the image. } if (result == AVAssetImageGeneratorFailed) { NSLog(@"Failed with error: %@", [error localizedDescription]); } if (result == AVAssetImageGeneratorCancelled) { NSLog(@"Canceled"); } }]; You can cancel the generation of the image sequence by sending the image generator a cancelAllCGImageGeneration message. Trimming and Transcoding a Movie You can transcode a movie from one format to another, and trim a movie, using an AVAssetExportSession object. An export session is a controller object that manages asynchronous export of an asset. You initialize the session using the asset you want to export and the name of a export preset that indicates the export options you want to apply (see allExportPresets). You then configure the export session to specify the output URL and file type, and optionally other settings such as the metadata and whether the output should be optimized for network use. Asset Export preset AVAssetExportSession URL You can check whether you can export a given asset using a given preset using exportPresetsCompatibleWithAsset: as illustrated in this example: Using Assets Trimming and Transcoding a Movie 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 15AVAsset *anAsset = <#Get an asset#>; NSArray *compatiblePresets = [AVAssetExportSession exportPresetsCompatibleWithAsset:anAsset]; if ([compatiblePresets containsObject:AVAssetExportPresetLowQuality]) { AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:anAsset presetName:AVAssetExportPresetLowQuality]; // Implementation continues. } You complete configuration of the session by providing the output URL (The URL must be a file URL.) AVAssetExportSession can infer the output file type from the URL’s path extension; typically, however, you set it directly using outputFileType. You can also specify additional properties such as the time range, a limit for the output file length, whether the exported file should be optimized for network use, and a video composition. The following example illustrates how to use the timeRange property to trim the movie: exportSession.outputURL = <#A file URL#>; exportSession.outputFileType = AVFileTypeQuickTimeMovie; CMTime start = CMTimeMakeWithSeconds(1.0, 600); CMTime duration = CMTimeMakeWithSeconds(3.0, 600); CMTimeRange range = CMTimeRangeMake(start, duration); exportSession.timeRange = range; To create the new file you invoke exportAsynchronouslyWithCompletionHandler:. The completion handler block is called when the export operation finishes; in your implementation of the handler, you should check the session’s status to determine whether the export was successful, failed, or was canceled: [exportSession exportAsynchronouslyWithCompletionHandler:^{ switch ([exportSession status]) { case AVAssetExportSessionStatusFailed: NSLog(@"Export failed: %@", [[exportSession error] localizedDescription]); break; case AVAssetExportSessionStatusCancelled: NSLog(@"Export canceled"); Using Assets Trimming and Transcoding a Movie 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 16break; default: break; } }]; You can cancel the export by sending the session a cancelExport message. The export will fail if you try to overwrite an existing file, or write a file outside of the application’s sandbox. It may also fail if: ● There is an incoming phone call ● Your application is in the background and another application starts playback In these situations, you should typically inform the user that the export failed, then allow the user to restart the export. Reading and Writing Assets You use an AVAssetReader when you want to perform an operation on the contents of an asset. For example, you might read the audio track of an asset to produce a visual representation of the waveform. To produce an asset from media such as sample buffers or still images, you use an AVAssetWriter object. You can use an asset reader and asset writer object in tandem to convert an asset from one representation to another. Using these objects you have more control over the conversion than you do with AVExportSession. For example of you want to choose which of the tracks you want to be represented in the output file, specify your own output format, or modify the asset during the conversion process. Using Assets Reading and Writing Assets 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 17To control the playback of assets, you use an AVPlayer object. During playback, you can use an AVPlayerItem object to manage the presentation state of an asset as a whole, and an AVPlayerItemTrack to manage the presentation state of an individual track. To display video, you use an AVPlayerLayer object. Playing Assets A player is a controller object that you use to manage playback of an asset, for example starting and stopping playback, and seeking to a particular time. You use an instance of AVPlayer to play a single asset. On iOS 4.1 and later, you can use an AVQueuePlayer object to play a number of items in sequence (AVQueuePlayer is a subclass of AVPlayer). A player provides you with information about the state of the playback so, if you need to, you can synchronize your user interface with the player’s state. You typically direct the output of a player to specialized Core Animation Layer (an instance of AVPlayerLayer or AVSynchronizedLayer). To learn more about layers, see Core Animation Programming Guide . 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 18 PlaybackMultiple player layers: You can create arbitrarily many AVPlayerLayer objects from a single AVPlayer instance, but only the most-recently-created such layer will display any video content on-screen. Although ultimately you want to play an asset, you don’t provide assets directly to an AVPlayer object. Instead, you provide an instance of AVPlayerItem. A player item manages the presentation state of an asset with which it is associated. A player item contains player item tracks—instances of AVPlayerItemTrack—that correspond to the tracks in the asset. AVAsset AVAssetTrack AVAssetTrack AVPlayerItem AVPlayerItemTrack AVPlayer AVPlayerLayer AVPlayerItemTrack This abstraction means that you can play a given asset using different players simultaneously, but rendered in different ways by each player. Using the item tracks, you can, for example, disable a particular track during playback (you might not want to play the sound component). AVAsset AVPlayer 1 AVPlayer 2 • Video • Audio R • Audio L AVPlayerItem 1 AVPlayerItem 2 AVPlayerItemTracks time = 4:15 time = 2:10 Video Audio R Audio L Off Off Playback Playing Assets 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 19You can initialize a player item with an existing asset, or you can initialize a player item directly from a URL so that you can play a resource at a particular location (AVPlayerItem will then create and configure an asset for the resource). As with AVAsset, though, simply initializing a player item doesn’t necessarily mean it’s ready for immediate playback. You can observe (using key-value observing) an item’s status property to determine if and when it’s ready to play. Handling Different Types of Asset The way you configure an asset for playback may depend on the sort of asset you want to play. Broadly speaking, there are two main types: file-based assets, to which you have random access (such as from a local file, the camera roll, or the Media Library), and stream-based (HTTP Live Stream format). To load and play a file-based asset. There are several steps to playing a file-based asset: ● Create an asset using AVURLAsset and load its tracks using loadValuesAsynchronouslyForKeys:completionHandler:. ● When the asset has loaded its tracks, create an instance of AVPlayerItem using the asset. ● Associate the item with an instance of AVPlayer. ● Wait until the item’s status indicatesthat it’sready to play (typically you use key-value observing to receive a notification when the status changes). This approach is illustrated in “Putting it all Together: Playing a Video File Using AVPlayerLayer” (page 26). To create and prepare an HTTP live stream for playback. Initialize an instance of AVPlayerItem using the URL. (You cannot directly create an AVAsset instance to represent the media in an HTTP Live Stream.) NSURL *url = [NSURL URLWithString:@"<#Live stream URL#>]; // You may find a test stream at . self.playerItem = [AVPlayerItem playerItemWithURL:url]; [playerItem addObserver:self forKeyPath:@"status" options:0 context:&ItemStatusContext]; self.player = [AVPlayer playerWithPlayerItem:playerItem]; When you associate the player item with a player, it starts to become ready to play. When it is ready to play, the player item createsthe AVAsset and AVAssetTrack instances, which you can use to inspect the contents of the live stream. Playback Handling Different Types of Asset 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 20If you simply want to play a live stream, you can take a shortcut and create a player directly using the URL: self.player = [AVPlayer playerWithURL:<#Live stream URL#>]; [player addObserver:self forKeyPath:@"status" options:0 context:&PlayerStatusContext]; As with assets and items, initializing the player does not mean it’s ready for playback. You should observe the player’s status property, which changes to AVPlayerStatusReadyToPlay when it is ready to play. You can also observe the currentItem property to access the player item created for the stream. If you don’t know what kind of URL you have. Follow these steps: 1. Try to initialize an AVURLAsset using the URL, then load its tracks key. If the tracks load successfully, then you create a player item for the asset. 2. If 1 fails, create an AVPlayerItem directly from the URL. Observe the player’s status property to determine whether it becomes playable. If either route succeeds, you end up with a player item that you can then associate with a player. Playing an Item To start playback, you send a play message to the player. - (IBAction)play:sender { [player play]; } In addition to simply playing, you can manage various aspects of the playback,such asthe rate and the location of the playhead. You can also monitor the play state of the player; this is useful if you want to, for example, synchronize the user interface to the presentation state of the asset—see “Monitoring Playback” (page 23). Changing the Playback Rate You change the rate of playback by setting the player’s rate property. aPlayer.rate = 0.5; aPlayer.rate = 2.0; Playback Playing an Item 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 21A value of 1.0 means “play at the natural rate of the current item”. Setting the rate to 0.0 is the same as pausing playback—you can also use pause. Seeking—Repositioning the Playhead To move the playhead to a particular time, you generally use seekToTime:. CMTime fiveSecondsIn = CMTimeMake(5, 1); [player seekToTime:fiveSecondsIn]; The seekToTime: method, however, is tuned for performance rather than precision. If you need to move the playhead precisely, instead you use seekToTime:toleranceBefore:toleranceAfter:. CMTime fiveSecondsIn = CMTimeMake(5, 1); [player seekToTime:fiveSecondsIn toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero]; Using a tolerance of zero may require the framework to decode a large amount of data. You should only use zero if you are, for example, writing a sophisticated media editing application that requires precise control. After playback, the player’s head is set to the end of the item, and further invocations of play have no effect. To position the play head back at the beginning of the item, you can register to receive an AVPlayerItemDidPlayToEndTimeNotification from the item. In the notification’s callback method, you invoke seekToTime: with the argument kCMTimeZero. // Register with the notification center after creating the player item. [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(playerItemDidReachEnd:) name:AVPlayerItemDidPlayToEndTimeNotification object:<#The player item#>]; - (void)playerItemDidReachEnd:(NSNotification *)notification { [player seekToTime:kCMTimeZero]; } Playback Playing an Item 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 22Playing Multiple Items On iOS 4.1 and later, you can use an AVQueuePlayer object to play a number of items in sequence. AVQueuePlayer is a subclass of AVPlayer. You initialize a queue player with an array of player items: NSArray *items = <#An array of player items#>; AVQueuePlayer *queuePlayer = [[AVQueuePlayer alloc] initWithItems:items]; You can then play the queue using play, just as you would an AVPlayer object. The queue player plays each item in turn. If you want to skip to the next item, you send the queue player an advanceToNextItem message. You can modify the queue using insertItem:afterItem:, removeItem:, and removeAllItems. When adding a new item, you should typically check whether it can be inserted into the queue, using canInsertItem:afterItem:. You pass nil as the second argument to test whether the new item can be appended to the queue: AVPlayerItem *anItem = <#Get a player item#>; if ([queuePlayer canInsertItem:anItem afterItem:nil]) { [queuePlayer insertItem:anItem afterItem:nil]; } Monitoring Playback You can monitor a number of aspects of the presentation state of a player and the player item being played. This is particularly useful for state changes that are not under your direct control, for example: ● If the user uses multitasking to switch to a different application, a player’s rate property will drop to 0.0. ● If you are playing remotemedia, a playeritem’s loadedTimeRanges and seekableTimeRanges properties will change as more data becomes available. These properties tell you what portions of the player item’s timeline are available. ● A player’s currentItem property changes as a player item is created for an HTTP live stream. ● A player item’s tracks property may change while playing an HTTP live stream. This may happen if the stream offers different encodings for the content; the tracks change if the player switches to a different encoding. ● A player or player item’s status may change if playback fails for some reason. You can use key-value observing to monitor changes to values of these properties. Playback Playing Multiple Items 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 23Important: You should register for KVO change notifications and unregister from KVO change notifications on the main thread. This avoids the possibility of receiving a partial notification if a change is being made on another thread. AV Foundation invokes observeValueForKeyPath:ofObject:change:context: on the main thread, even if the change operation is made on another thread. Responding to a Change in Status When a player or player item’s status changes, it emits a key-value observing change notification. If an object is unable to play for some reason (for example, if the media services are reset), the status changes to AVPlayerStatusFailed or AVPlayerItemStatusFailed as appropriate. In thissituation, the value of the object’s error property is changed to an error object that describes why the object is no longer be able to play. AV Foundation does not specify what thread that the notification is sent on. If you want to update the user interface, you must make sure that any relevant code is invoked on the main thread. This example uses dispatch_async(3) OS X Developer Tools Manual Page to execute code on the main thread. - (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context { if (context == <#Player status context#>) { AVPlayer *thePlayer = (AVPlayer *)object; if ([thePlayer status] == AVPlayerStatusFailed) { NSError *error = [<#The AVPlayer object#> error]; // Respond to error: for example, display an alert sheet. return; } // Deal with other status change if appropriate. } // Deal with other change notifications if appropriate. [super observeValueForKeyPath:keyPath ofObject:object change:change context:context]; return; } Playback Monitoring Playback 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 24Tracking Readiness for Visual Display You can observe an AVPlayerLayer object’s readyForDisplay property to be notified when the layer has user-visible content. In particular, you might insert the player layer into the layer tree only when there is something for the user to look at, and perform a transition from Tracking Time To track changes in the position of the playhead in an AVPlayer object, you can use addPeriodicTimeObserverForInterval:queue:usingBlock: or addBoundaryTimeObserverForTimes:queue:usingBlock:. You might do this to, for example, update your user interface with information about time elapsed or time remaining, or perform some other user interface synchronization. ● With addPeriodicTimeObserverForInterval:queue:usingBlock:,the block you provide isinvoked at the interval you specify, and if time jumps, and when playback starts or stops. ● With addBoundaryTimeObserverForTimes:queue:usingBlock:, you pass an array of CMTimes contained in NSValue objects. The block you provide is invoked whenever any of those times is traversed. Both of the methods return an opaque object that serves as an observer. You must keep a strong reference to the returned object as long as you want the time observation block to be invoked by the player. You must also balance each invocation of these methods with a corresponding call to removeTimeObserver:. With both of these methods, AV Foundation does not guarantee to invoke your block for every interval or boundary passed. AV Foundation does not invoke a block if execution of a previously-invoked block has not completed. You must make sure, therefore, that the work you perform in the block does not overly tax the system. // Assume a property: @property (strong) id playerObserver; Float64 durationSeconds = CMTimeGetSeconds([<#An asset#> duration]); CMTime firstThird = CMTimeMakeWithSeconds(durationSeconds/3.0, 1); CMTime secondThird = CMTimeMakeWithSeconds(durationSeconds*2.0/3.0, 1); NSArray *times = @[[NSValue valueWithCMTime:firstThird], [NSValue valueWithCMTime:secondThird]]; self.playerObserver = [<#A player#> addBoundaryTimeObserverForTimes:times queue:NULL usingBlock:^{ NSString *timeDescription = (NSString *) Playback Monitoring Playback 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 25CFBridgingRelease(CMTimeCopyDescription(NULL, [self.player currentTime])); NSLog(@"Passed a boundary at %@", timeDescription); }]; Reaching the End of an Item You can register to receive an AVPlayerItemDidPlayToEndTimeNotification notification when a player item has completed playback: [[NSNotificationCenter defaultCenter] addObserver:<#The observer, typically self#> selector:@selector(<#The selector name#>) name:AVPlayerItemDidPlayToEndTimeNotification object:<#A player item#>]; Putting it all Together: Playing a Video File Using AVPlayerLayer This brief code example to illustrates how you can use an AVPlayer object to play a video file. It shows how to: ● Configure a view to use an AVPlayerLayer layer ● Create an AVPlayer object ● Create an AVPlayerItem object for a file-based asset, and use key-value observing to observe its status ● Respond to the item becoming ready to play by enabling a button ● Play the item, then restore the player’s head to the beginning. Note: To focus on the most relevant code, this example omits several aspects of a complete application,such as memory management, and unregistering as an observer (for key-value observing or for the notification center). To use AV Foundation, you are expected to have enough experience with Cocoa to be able to infer the missing pieces. For a conceptual introduction to playback, skip to “Playing Assets” (page 18). Playback Putting it all Together: Playing a Video File Using AVPlayerLayer 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 26The Player View To play the visual component of an asset, you need a view containing an AVPlayerLayer layer to which the output of an AVPlayer object can be directed. You can create a simple subclass of UIView to accommodate this: #import #import @interface PlayerView : UIView @property (nonatomic) AVPlayer *player; @end @implementation PlayerView + (Class)layerClass { return [AVPlayerLayer class]; } - (AVPlayer*)player { return [(AVPlayerLayer *)[self layer] player]; } - (void)setPlayer:(AVPlayer *)player { [(AVPlayerLayer *)[self layer] setPlayer:player]; } @end A Simple View Controller Assume you have a simple view controller, declared as follows: @class PlayerView; @interface PlayerViewController : UIViewController @property (nonatomic) AVPlayer *player; @property (nonatomic) AVPlayerItem *playerItem; @property (nonatomic, weak) IBOutlet PlayerView *playerView; @property (nonatomic, weak) IBOutlet UIButton *playButton; - (IBAction)loadAssetFromFile:sender; Playback Putting it all Together: Playing a Video File Using AVPlayerLayer 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 27- (IBAction)play:sender; - (void)syncUI; @end The syncUI method synchronizes the button’s state with the player’s state: - (void)syncUI { if ((self.player.currentItem != nil) && ([self.player.currentItem status] == AVPlayerItemStatusReadyToPlay)) { self.playButton.enabled = YES; } else { self.playButton.enabled = NO; } } You can invoke syncUI in the view controller’s viewDidLoad method to ensure a consistent user interface when the view is first displayed. - (void)viewDidLoad { [super viewDidLoad]; [self syncUI]; } The other properties and methods are described in the remaining sections. Creating the Asset You create an asset from a URL using AVURLAsset. Creating the asset, however, does not necessarily mean that it’s ready for use. To be used, an asset must have loaded its tracks. To avoid blocking the current thread, you load the asset’s tracks asynchronously using loadValuesAsynchronouslyForKeys:completionHandler:. (The following example assumes your project contains a suitable video resource.) - (IBAction)loadAssetFromFile:sender { Playback Putting it all Together: Playing a Video File Using AVPlayerLayer 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 28NSURL *fileURL = [[NSBundle mainBundle] URLForResource:<#@"VideoFileName"#> withExtension:<#@"extension"#>]; AVURLAsset *asset = [AVURLAsset URLAssetWithURL:fileURL options:nil]; NSString *tracksKey = @"tracks"; [asset loadValuesAsynchronouslyForKeys:@[tracksKey] completionHandler: ^{ // The completion block goes here. }]; } In the completion block, you create an instance of AVPlayerItem for the asset, and set it as the player for the player view. As with creating the asset, simply creating the player item does not mean it’s ready to use. To determine when it’s ready to play, you can observe the item’s status. You trigger its preparation to play when you associate it with the player. // Define this constant for the key-value observation context. static const NSString *ItemStatusContext; // Completion handler block. dispatch_async(dispatch_get_main_queue(), ^{ NSError *error; AVKeyValueStatus status = [asset statusOfValueForKey:tracksKey error:&error]; if (status == AVKeyValueStatusLoaded) { self.playerItem = [AVPlayerItem playerItemWithAsset:asset]; [self.playerItem addObserver:self forKeyPath:@"status" options:0 context:&ItemStatusContext]; [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(playerItemDidReachEnd:) name:AVPlayerItemDidPlayToEndTimeNotification Playback Putting it all Together: Playing a Video File Using AVPlayerLayer 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 29object:self.playerItem]; self.player = [AVPlayer playerWithPlayerItem:self.playerItem]; [self.playerView setPlayer:self.player]; } else { // You should deal with the error appropriately. NSLog(@"The asset's tracks were not loaded:\n%@", [error localizedDescription]); } }); Responding to the Player Item’s Status Change When the player item’s status changes, the view controller receives a key-value observing change notification. AV Foundation does not specify what thread that the notification is sent on. If you want to update the user interface, you must make sure that any relevant code is invoked on the main thread. This example uses dispatch_async(3) OS X Developer Tools Manual Page to queue a message on the main thread to synchronize the user interface. - (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context { if (context == &ItemStatusContext) { dispatch_async(dispatch_get_main_queue(), ^{ [self syncUI]; }); return; } [super observeValueForKeyPath:keyPath ofObject:object change:change context:context]; return; } Playback Putting it all Together: Playing a Video File Using AVPlayerLayer 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 30Playing the Item Playing the item is trivial: you send a play message to the player. - (IBAction)play:sender { [player play]; } This only playsthe item once, though. After playback, the player’s head isset to the end of the item, and further invocations of play will have no effect. To position the play head back at the beginning of the item, you can register to receive an AVPlayerItemDidPlayToEndTimeNotification from the item. In the notification’s callback method, invoke seekToTime: with the argument kCMTimeZero. // Register with the notification center after creating the player item. [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(playerItemDidReachEnd:) name:AVPlayerItemDidPlayToEndTimeNotification object:[self.player currentItem]]; - (void)playerItemDidReachEnd:(NSNotification *)notification { [self.player seekToTime:kCMTimeZero]; } Playback Putting it all Together: Playing a Video File Using AVPlayerLayer 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 31To manage the capture from a device such as a camera or microphone, you assemble objects to represent inputs and outputs, and use an instance of AVCaptureSession to coordinate the data flow between them. Minimally you need: ● An instance of AVCaptureDevice to represent the input device, such as a camera or microphone ● An instance of a concrete subclass of AVCaptureInput to configure the ports from the input device ● An instance of a concrete subclass of AVCaptureOutput to manage the output to a movie file or still image ● An instance of AVCaptureSession to coordinate the data flow from the input to the output To show the user what a camera is recording, you can use an instance of AVCaptureVideoPreviewLayer (a subclass of CALayer). You can configure multiple inputs and outputs, coordinated by a single session: AVCapture Device Input AVCapture Device Input AVCaptureMovieFileOutput AVCaptureStillImageOutput AVCaptureVideoPreviewLayer AVCapture Session Capture Session For many applications, thisis as much detail as you need. Forsome operations, however, (if you want to monitor the power levels in an audio channel, for example) you need to consider how the various ports of an input device are represented, how those ports are connected to the output. 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 32 Media CaptureA connection between a capture input and a capture output in a capture session is represented by an AVCaptureConnection object. Capture inputs(instances of AVCaptureInput) have one or more input ports (instances of AVCaptureInputPort). Capture outputs (instances of AVCaptureOutput) can accept data from one or more sources (for example, an AVCaptureMovieFileOutput object accepts both video and audio data). When you add an input or an output to a session, the session “greedily” forms connections between all the compatible capture inputs’ ports and capture outputs. A connection between a capture input and a capture output is represented by an AVCaptureConnection object. Capture Device Input Capture connection Capture connection Capture input port (Video) Capture input port (Audio) Capture Device Input Capture input port (Audio) Connections AVCaptureMovieFileOutput AVCaptureStillImageOutput Connections Capture Session Capture connection You can use a capture connection to enable or disable the flow of data from a given input or to a given output. You can also use a connection to monitor the average and peak power levels in an audio channel. Use a Capture Session to Coordinate Data Flow AVCaptureSession object is the central coordinating object you use to manage data capture. You use an instance to coordinate the flow of data from AV input devices to outputs. You add the capture devices and outputs you want to the session, then start data flow by sending the session a startRunning message, and stop recording by sending a stopRunning message. AVCaptureSession *session = [[AVCaptureSession alloc] init]; // Add inputs and outputs. [session startRunning]; Media Capture Use a Capture Session to Coordinate Data Flow 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 33Configuring a Session You use a preset on the session to specify the image quality and resolution you want. A preset is a constant that identifies one of a number of possible configurations; in some cases the actual configuration is device-specific: Symbol Resolution Comments Highest recording quality. This varies per device. AVCaptureSessionPresetHigh High Suitable for WiFi sharing. The actual values may change. AVCaptureSessionPresetMedium Medium Suitable for 3G sharing. The actual values may change. AVCaptureSessionPresetLow Low AVCaptureSessionPreset640x480 640x480 VGA. AVCaptureSessionPreset1280x720 1280x720 720p HD. Full photo resolution. This is not supported for video output. AVCaptureSessionPresetPhoto Photo For examples of the actual valuesthese presetsrepresent for various devices,see “Saving to a Movie File” (page 43) and “Capturing Still Images” (page 47). If you want to set a size-specific configuration, you should check whether it is supported before setting it: if ([session canSetSessionPreset:AVCaptureSessionPreset1280x720]) { session.sessionPreset = AVCaptureSessionPreset1280x720; } else { // Handle the failure. } In many situations, you create a session and the various inputs and outputs all at once. Sometimes, however, you may want to reconfigure a running session, perhaps as different input devices become available, or in response to user request. This can present a challenge, since, if you change them one at a time, a new setting may be incompatible with an existing setting. To deal with this, you use beginConfiguration and commitConfiguration to batch multiple configuration operations into an atomic update. After calling Media Capture Use a Capture Session to Coordinate Data Flow 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 34beginConfiguration, you can for example add or remove outputs, alter the sessionPreset, or configure individual capture input or output properties. No changes are actually made until you invoke commitConfiguration, at which time they are applied together. [session beginConfiguration]; // Remove an existing capture device. // Add a new capture device. // Reset the preset. [session commitConfiguration]; Monitoring Capture Session State A capture session posts notifications that you can observe to be notified, for example, when it starts or stops running, or when it is interrupted. You can also register to receive an AVCaptureSessionRuntimeErrorNotification if a runtime error occurs. You can also interrogate the session’s running property to find out if it is running, and its interrupted property to find out if it is interrupted. An AVCaptureDevice Object Represents an Input Device An AVCaptureDevice object abstracts a physical capture device that provides input data (such as audio or video) to an AVCaptureSession object. There is one object for each input device, so for example on an iPhone 3GS there is one video input for the camera and one audio input for the microphone; on an iPhone 4 there are two video inputs—one for front-facing the camera, one for the back-facing camera—and one audio input for the microphone. You can find out what capture devices are currently available using the AVCaptureDevice class methods devices and devicesWithMediaType:, and if necessary find out what featuresthe devices offer (see “Device Capture Settings” (page 36)). The list of available devices may change, though. Current devices may become unavailable (if they’re used by another application), and new devices may become available, (if they’re relinquished by another application). You should register to receive AVCaptureDeviceWasConnectedNotification and AVCaptureDeviceWasDisconnectedNotificationnotifications to be alerted when the list of available devices changes. You add a device to a capture session using a capture input (see “Use Capture Inputs to Add a Capture Device to a Session” (page 41)). Media Capture An AVCaptureDevice Object Represents an Input Device 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 35Device Characteristics You can ask a device about several different characteristics. You can test whether it provides a particular media type or supports a given capture session preset using hasMediaType: and supportsAVCaptureSessionPreset: respectively. To provide information to the user, you can find out the position of the capture device (whether it is on the front or the back of the unit they’re using), and its localized name. This may be useful if you want to present a list of capture devices to allow the user to choose one. The following code example iterates over all the available devices and logs their name, and for video devices their position on the unit. NSArray *devices = [AVCaptureDevice devices]; for (AVCaptureDevice *device in devices) { NSLog(@"Device name: %@", [device localizedName]); if ([device hasMediaType:AVMediaTypeVideo]) { if ([device position] == AVCaptureDevicePositionBack) { NSLog(@"Device position : back"); } else { NSLog(@"Device position : front"); } } } In addition, you can find out the device’s model ID and its unique ID. Device Capture Settings Different devices have different capabilities; for example, some may support different focus or flash modes; some may support focus on a point of interest. Feature iPhone 3G iPhone 3GS iPhone 4 (Back) iPhone 4 (Front) Focus mode NO YES YES NO Media Capture An AVCaptureDevice Object Represents an Input Device 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 36Feature iPhone 3G iPhone 3GS iPhone 4 (Back) iPhone 4 (Front) Focus point of interest NO YES YES NO Exposure mode YES YES YES YES Exposure point of interest NO YES YES YES White balance mode YES YES YES YES Flash mode NO NO YES NO Torch mode NO NO YES NO The following code fragment shows how you can find video input devices that have a torch mode and support a given capture session preset: NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo]; NSMutableArray *torchDevices = [[NSMutableArray alloc] init]; for (AVCaptureDevice *device in devices) { [if ([device hasTorch] && [device supportsAVCaptureSessionPreset:AVCaptureSessionPreset640x480]) { [torchDevices addObject:device]; } } If you find multiple devices that meet your criteria, you might let the user choose which one they want to use. To display a description of a device to the user, you can use its localizedName property. You use the various different features in similar ways. There are constants to specify a particular mode, and you can ask a device whether it supports a particular mode. In several cases you can observe a property to be notified when a feature is changing. In all cases, you should lock the device before changing the mode of a particular feature, as described in “Configuring a Device” (page 40). Note: Focus point of interest and exposure point of interest are mutually exclusive, as are focus mode and exposure mode. Focus modes There are three focus modes: Media Capture An AVCaptureDevice Object Represents an Input Device 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 37● AVCaptureFocusModeLocked: the focal length is fixed. This is useful when you want to allow the user to compose a scene then lock the focus. ● AVCaptureFocusModeAutoFocus: the camera does a single scan focus then reverts to locked. This is suitable for a situation where you want to select a particular item on which to focus and then maintain focus on that item even if it is not the center of the scene. ● AVCaptureFocusModeContinuousAutoFocus: the camera continuously auto-focuses as needed. You use the isFocusModeSupported: method to determine whether a device supports a given focus mode, then set the mode using the focusMode property. In addition, a device may support a focus point of interest. You test for support using focusPointOfInterestSupported. If it’ssupported, you set the focal point using focusPointOfInterest. You pass a CGPoint where {0,0} representsthe top left of the picture area, and {1,1} representsthe bottom right in landscape mode with the home button on the right—this applies even if the device is in portrait mode. You can use the adjustingFocus property to determine whether a device is currently focusing. You can observe the property using key-value observing to be notified when a device starts and stops focusing. If you change the focus mode settings, you can return them to the default configuration as follows: if ([currentDevice isFocusModeSupported:AVCaptureFocusModeContinuousAutoFocus]) { CGPoint autofocusPoint = CGPointMake(0.5f, 0.5f); [currentDevice setFocusPointOfInterest:autofocusPoint]; [currentDevice setFocusMode:AVCaptureFocusModeContinuousAutoFocus]; } Exposure modes There are two exposure modes: ● AVCaptureExposureModeLocked: the exposure mode is fixed. ● AVCaptureExposureModeAutoExpose: the camera continuously changesthe exposure level as needed. You use the isExposureModeSupported: method to determine whether a device supports a given exposure mode, then set the mode using the exposureMode property. Media Capture An AVCaptureDevice Object Represents an Input Device 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 38In addition, a device may support an exposure point of interest. You test for support using exposurePointOfInterestSupported. If it’s supported, you set the exposure point using exposurePointOfInterest. You pass a CGPoint where {0,0} represents the top left of the picture area, and {1,1} represents the bottom right in landscape mode with the home button on the right—this applies even if the device is in portrait mode. You can use the adjustingExposure property to determine whether a device is currently changing its exposure setting. You can observe the property using key-value observing to be notified when a device starts and stops changing its exposure setting. If you change the exposure settings, you can return them to the default configuration as follows: if ([currentDevice isExposureModeSupported:AVCaptureExposureModeContinuousAutoExposure]) { CGPoint exposurePoint = CGPointMake(0.5f, 0.5f); [currentDevice setExposurePointOfInterest:exposurePoint]; [currentDevice setExposureMode:AVCaptureExposureModeContinuousAutoExposure]; } Flash modes There are three flash modes: ● AVCaptureFlashModeOff: the flash will never fire. ● AVCaptureFlashModeOn: the flash will always fire. ● AVCaptureFlashModeAuto: the flash will fire if needed. You use hasFlash to determine whether a device has a flash. You use the isFlashModeSupported: method to determine whether a device supports a given flash mode, then set the mode using the flashMode property. Torch mode Torch mode is where a camera uses the flash continuously at a low power to illuminate a video capture. There are three torch modes: ● AVCaptureTorchModeOff: the torch is always off. ● AVCaptureTorchModeOn: the torch is always on. ● AVCaptureTorchModeAuto: the torch is switched on and off as needed. Media Capture An AVCaptureDevice Object Represents an Input Device 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 39You use hasTorch to determine whether a device has a flash. You use the isTorchModeSupported: method to determine whether a device supports a given flash mode, then set the mode using the torchMode property. For devices with a torch, the torch only turns on if the device is associated with a running capture session. White balance There are two white balance modes: ● AVCaptureWhiteBalanceModeLocked: the white balance mode is fixed. ● AVCaptureWhiteBalanceModeContinuousAutoWhiteBalance: the camera continuously changes the white balance as needed. You use the isWhiteBalanceModeSupported: method to determine whether a device supports a given white balance mode, then set the mode using the whiteBalanceMode property. You can use the adjustingWhiteBalance property to determine whether a device is currently changing its white balance setting. You can observe the property using key-value observing to be notified when a device starts and stops changing its white balance setting. Configuring a Device To set capture properties on a device, you must first acquire a lock on the device using lockForConfiguration:. This avoids making changes that may be incompatible with settings in other applications. The following code fragment illustrates how to approach changing the focus mode on a device by first determining whether the mode is supported, then attempting to lock the device for reconfiguration. The focus mode is changed only if the lock is obtained, and the lock is released immediately afterward. if ([device isFocusModeSupported:AVCaptureFocusModeLocked]) { NSError *error = nil; if ([device lockForConfiguration:&error]) { device.focusMode = AVCaptureFocusModeLocked; [device unlockForConfiguration]; } else { // Respond to the failure as appropriate. You should only hold the device lock if you need settable device properties to remain unchanged. Holding the device lock unnecessarily may degrade capture quality in other applications sharing the device. Media Capture An AVCaptureDevice Object Represents an Input Device 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 40Switching Between Devices Sometimes you may want to allow the user to switch between input devices—for example, on an iPhone 4 they could switch from using the front to the back camera. To avoid pauses or stuttering, you can reconfigure a session while it is running, however you should use beginConfiguration and commitConfiguration to bracket your configuration changes: AVCaptureSession *session = <#A capture session#>; [session beginConfiguration]; [session removeInput:frontFacingCameraDeviceInput]; [session addInput:backFacingCameraDeviceInput]; [session commitConfiguration]; When the outermost commitConfiguration is invoked, all the changes are made together. This ensures a smooth transition. Use Capture Inputs to Add a Capture Device to a Session To add a capture device to a capture session, you use an instance of AVCaptureDeviceInput (a concrete subclass of the abstract AVCaptureInput class). The capture device input manages the device’s ports. NSError *error; AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:device error:&error]; if (!input) { // Handle the error appropriately. } You add inputs to a session using addInput:. If appropriate, you can check whether a capture input is compatible with an existing session using canAddInput:. AVCaptureSession *captureSession = <#Get a capture session#>; AVCaptureDeviceInput *captureDeviceInput = <#Get a capture device input#>; if ([captureSession canAddInput:captureDeviceInput]) { [captureSession addInput:captureDeviceInput]; Media Capture Use Capture Inputs to Add a Capture Device to a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 41} else { // Handle the failure. } See “Configuring a Session” (page 34) for more details on how you might reconfigure a running session. An AVCaptureInput vends one or more streams of media data. For example, input devices can provide both audio and video data. Each media stream provided by an input is represented by an AVCaptureInputPort object. A capture session uses an AVCaptureConnection object to define the mapping between a set of AVCaptureInputPort objects and a single AVCaptureOutput. Use Capture Outputs to Get Output from a Session To get output from a capture session, you add one or more outputs. An output is an instance of a concrete subclass of AVCaptureOutput; you use: ● AVCaptureMovieFileOutput to output to a movie file ● AVCaptureVideoDataOutput if you want to process frames from the video being captured ● AVCaptureAudioDataOutput if you want to process the audio data being captured ● AVCaptureStillImageOutput if you want to capture still images with accompanying metadata You add outputs to a capture session using addOutput:. You check whether a capture output is compatible with an existing session using canAddOutput:. You can add and remove outputs as you want while the session is running. AVCaptureSession *captureSession = <#Get a capture session#>; AVCaptureMovieFileOutput *movieInput = <#Create and configure a movie output#>; if ([captureSession canAddOutput:movieInput]) { [captureSession addOutput:movieInput]; } else { // Handle the failure. } Media Capture Use Capture Outputs to Get Output from a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 42Saving to a Movie File You save movie data to a file using an AVCaptureMovieFileOutput object. (AVCaptureMovieFileOutput is a concrete subclass of AVCaptureFileOutput, which defines much of the basic behavior.) You can configure various aspects of the movie file output, such as the maximum duration of the recording, or the maximum file size. You can also prohibit recording if there is less than a given amount of disk space left. AVCaptureMovieFileOutput *aMovieFileOutput = [[AVCaptureMovieFileOutput alloc] init]; CMTime maxDuration = <#Create a CMTime to represent the maximum duration#>; aMovieFileOutput.maxRecordedDuration = maxDuration; aMovieFileOutput.minFreeDiskSpaceLimit = <#An appropriate minimum given the quality of the movie format and the duration#>; The resolution and bit rate for the output depend on the capture session’s sessionPreset. The video encoding is typically H.264 and audio encoding AAC. The actual values vary by device, as illustrated in the following table. Preset iPhone 3G iPhone 3GS iPhone 4 (Back) iPhone 4 (Front) 640x480 3.5 mbps 1280x720 10.5 mbps 640x480 3.5 mbps No video Apple Lossless High 480x360 700 kbps 480x360 700 kbps 480x360 700 kbps No video Apple Lossless Medium 192x144 128 kbps 192x144 128 kbps 192x144 128 kbps No video Apple Lossless Low 640x480 3.5 mbps 640x480 3.5 mbps 640x480 3.5 mbps No video Apple Lossless 640x480 No video 64 kbps AAC No video 64 kbps AAC No video 64 kbps AAC No video Apple Lossless 1280x720 Notsupported for video output Notsupported for video output Not supported for video output Not supported for video output Photo Media Capture Use Capture Outputs to Get Output from a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 43Starting a Recording You start recording a QuickTime movie using startRecordingToOutputFileURL:recordingDelegate:. You need to supply a file-based URL and a delegate. The URL must not identify an existing file, as the movie file output does not overwrite existing resources. You must also have permission to write to the specified location. The delegate must conform to the AVCaptureFileOutputRecordingDelegate protocol, and must implement the captureOutput:didFinishRecordingToOutputFileAtURL:fromConnections:error: method. AVCaptureMovieFileOutput *aMovieFileOutput = <#Get a movie file output#>; NSURL *fileURL = <#A file URL that identifies the output location#>; [aMovieFileOutput startRecordingToOutputFileURL:fileURL recordingDelegate:<#The delegate#>]; In the implementation of captureOutput:didFinishRecordingToOutputFileAtURL:fromConnections:error:, the delegate might write the resulting movie to the camera roll. Itshould also check for any errorsthat might have occurred. Ensuring the File Was Written Successfully To determine whether the file was saved successfully, in the implementation of captureOutput:didFinishRecordingToOutputFileAtURL:fromConnections:error: you check not only the error, but also the value of the AVErrorRecordingSuccessfullyFinishedKey in the error’s user info dictionary: - (void)captureOutput:(AVCaptureFileOutput *)captureOutput didFinishRecordingToOutputFileAtURL:(NSURL *)outputFileURL fromConnections:(NSArray *)connections error:(NSError *)error { BOOL recordedSuccessfully = YES; if ([error code] != noErr) { // A problem occurred: Find out if the recording was successful. id value = [[error userInfo] objectForKey:AVErrorRecordingSuccessfullyFinishedKey]; if (value) { recordedSuccessfully = [value boolValue]; } Media Capture Use Capture Outputs to Get Output from a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 44} // Continue as appropriate... You should check the value of the AVErrorRecordingSuccessfullyFinishedKey in the error’s user info dictionary because the file might have been saved successfully, even though you got an error. The error might indicate that one of your recording constraints wasreached, for example AVErrorMaximumDurationReached or AVErrorMaximumFileSizeReached. Other reasons the recording might stop are: ● The disk is full—AVErrorDiskFull. ● The recording device was disconnected (for example, the microphone was removed from an iPod touch)—AVErrorDeviceWasDisconnected. ● The session wasinterrupted (for example, a phone call wasreceived)—AVErrorSessionWasInterrupted. Adding Metadata to a File You can set metadata for the movie file at any time, even while recording. This is useful for situations where the information is not available when the recording starts, as may be the case with location information. Metadata for a file output is represented by an array of AVMetadataItem objects; you use an instance of its mutable subclass, AVMutableMetadataItem, to create metadata of your own. AVCaptureMovieFileOutput *aMovieFileOutput = <#Get a movie file output#>; NSArray *existingMetadataArray = aMovieFileOutput.metadata; NSMutableArray *newMetadataArray = nil; if (existingMetadataArray) { newMetadataArray = [existingMetadataArray mutableCopy]; } else { newMetadataArray = [[NSMutableArray alloc] init]; } AVMutableMetadataItem *item = [[AVMutableMetadataItem alloc] init]; item.keySpace = AVMetadataKeySpaceCommon; item.key = AVMetadataCommonKeyLocation; CLLocation *location - <#The location to set#>; item.value = [NSString stringWithFormat:@"%+08.4lf%+09.4lf/" location.coordinate.latitude, location.coordinate.longitude]; Media Capture Use Capture Outputs to Get Output from a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 45[newMetadataArray addObject:item]; aMovieFileOutput.metadata = newMetadataArray; Processing Frames of Video An AVCaptureVideoDataOutput object uses delegation to vend video frames. You set the delegate using setSampleBufferDelegate:queue:. In addition to the delegate, you specify a serial queue on which they delegate methods are invoked. You must use a serial queue to ensure that frames are delivered to the delegate in the proper order. You should not pass the queue returned by dispatch_get_current_queue since there is no guarantee as to which thread the current queue is running on. You can use the queue to modify the priority given to delivering and processing the video frames. The frames are presented in the delegate method, captureOutput:didOutputSampleBuffer:fromConnection:, as instances of the CMSampleBuffer opaque type (see “Representations of Media” (page 58)). By default, the buffers are emitted in the camera’s most efficient format. You can use the videoSettings property to specify a custom output format. The video settings property is a dictionary; currently, the only supported key is kCVPixelBufferPixelFormatTypeKey. The recommended pixel format choices for iPhone 4 are kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange or kCVPixelFormatType_32BGRA; for iPhone 3G the recommended pixel format choices are kCVPixelFormatType_422YpCbCr8 or kCVPixelFormatType_32BGRA. Both Core Graphics and OpenGL work well with the BGRA format: AVCaptureSession *captureSession = <#Get a capture session#>; NSDictionary *newSettings = @{ (NSString *)kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA) }; captureSession.videoSettings = newSettings; Performance Considerations for Processing Video You should set the session output to the lowest practical resolution for your application. Setting the output to a higher resolution than necessary wastes processing cycles and needlessly consumes power. You must ensure that your implementation of captureOutput:didOutputSampleBuffer:fromConnection: is able to process a sample buffer within the amount of time allotted to a frame. If it takes too long, and you hold onto the video frames, AV Foundation will stop delivering frames, not only to your delegate but also other outputs such as a preview layer. Media Capture Use Capture Outputs to Get Output from a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 46You can use the capture video data output’s minFrameDuration property to ensure you have enough time to process a frame—at the cost of having a lower frame rate than would otherwise be the case. You might also ensure that the alwaysDiscardsLateVideoFrames property is set to YES (the default). This ensures that any late video frames are dropped rather than handed to you for processing. Alternatively, if you are recording and it doesn’t matter if the output fames are a little late, you would prefer to get all of them, you can set the property value to NO. This does not mean that frames will not be dropped (that is, frames may still be dropped), but they may not be dropped as early, or as efficiently. Capturing Still Images You use an AVCaptureStillImageOutput output if you want to capture still images with accompanying metadata. The resolution of the image depends on the preset for the session, as illustrated in this table: Preset iPhone 3G iPhone 3GS iPhone 4 (Back) iPhone 4 (Front) High 400x304 640x480 1280x720 640x480 Medium 400x304 480x360 480x360 480x360 Low 400x304 192x144 192x144 192x144 640x480 N/A 640x480 640x480 640x480 1280x720 N/A N/A 1280x720 N/A Photo 1600x1200 2048x1536 2592x1936 640x480 Pixel and Encoding Formats Different devices support different image formats: iPhone 3G iPhone 3GS iPhone 4 yuvs, 2vuy, BGRA, jpeg 420f, 420v, BGRA, jpeg 420f, 420v, BGRA, jpeg You can find out what pixel and codec types are supported using availableImageDataCVPixelFormatTypes and availableImageDataCodecTypes respectively. You set the outputSettings dictionary to specify the image format you want, for example: AVCaptureStillImageOutput *stillImageOutput = [[AVCaptureStillImageOutput alloc] init]; NSDictionary *outputSettings = @{ AVVideoCodecKey : AVVideoCodecJPEG}; [stillImageOutput setOutputSettings:outputSettings]; Media Capture Use Capture Outputs to Get Output from a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 47If you want to capture a JPEG image, you should typically not specify your own compression format. Instead, you should let the still image output do the compression for you,since its compression is hardware-accelerated. If you need a data representation of the image, you can use jpegStillImageNSDataRepresentation: to get an NSData object without re-compressing the data, even if you modify the image’s metadata. Capturing an Image When you want to capture an image, you send the output a captureStillImageAsynchronouslyFromConnection:completionHandler: message. The first argument is the connection you want to use for the capture. You need to look for the connection whose input port is collecting video: AVCaptureConnection *videoConnection = nil; for (AVCaptureConnection *connection in stillImageOutput.connections) { for (AVCaptureInputPort *port in [connection inputPorts]) { if ([[port mediaType] isEqual:AVMediaTypeVideo] ) { videoConnection = connection; break; } } if (videoConnection) { break; } } The second argument to captureStillImageAsynchronouslyFromConnection:completionHandler: is a block that takes two arguments: a CMSampleBuffer containing the image data, and an error. The sample buffer itself may contain metadata,such as an Exif dictionary, as an attachment. You can modify the attachments should you want, but note the optimization for JPEG images discussed in “Pixel and Encoding Formats” (page 47). [stillImageOutput captureStillImageAsynchronouslyFromConnection:videoConnection completionHandler: ^(CMSampleBufferRef imageSampleBuffer, NSError *error) { CFDictionaryRef exifAttachments = CMGetAttachment(imageSampleBuffer, kCGImagePropertyExifDictionary, NULL); if (exifAttachments) { // Do something with the attachments. } Media Capture Use Capture Outputs to Get Output from a Session 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 48// Continue as appropriate. }]; Showing the User What’s Being Recorded You can provide the user with a preview of what’s being recorded by the camera using a preview layer, or by the microphone by monitoring the audio channel. Video Preview You can provide the user with a preview of what’s being recorded using an AVCaptureVideoPreviewLayer object. AVCaptureVideoPreviewLayer is a subclass ofCALayer (see Core Animation Programming Guide . You don’t need any outputs to show the preview. Unlike a capture output, a video preview layer maintains a strong reference to the session with which it is associated. This is to ensure that the session is not deallocated while the layer is attempting to display video. This is reflected in the way you initialize a preview layer: AVCaptureSession *captureSession = <#Get a capture session#>; CALayer *viewLayer = <#Get a layer from the view in which you want to present the preview#>; AVCaptureVideoPreviewLayer *captureVideoPreviewLayer = [[AVCaptureVideoPreviewLayer alloc] initWithSession:captureSession]; [viewLayer addSublayer:captureVideoPreviewLayer]; In general, the preview layer behaves like any other CALayer object in the render tree (see Core Animation Programming Guide ). You can scale the image and perform transformations, rotations and so on just as you would any layer. One difference is that you may need to set the layer’s orientation property to specify how itshould rotate images coming from the camera. In addition, on iPhone 4 the preview layersupports mirroring (this is the default when previewing the front-facing camera). Video Gravity Modes The preview layer supports three gravity modes that you set using videoGravity: ● AVLayerVideoGravityResizeAspect: This preserves the aspect ratio, leaving black bars where the video does not fill the available screen area. Media Capture Showing the User What’s Being Recorded 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 49● AVLayerVideoGravityResizeAspectFill: This preservesthe aspect ratio, but fillsthe available screen area, cropping the video when necessary. ● AVLayerVideoGravityResize: This simply stretches the video to fill the available screen area, even if doing so distorts the image. Using “Tap to Focus” With a Preview You need to take care when implementing tap-to-focus in conjunction with a preview layer. You must account for the preview orientation and gravity of the layer, and the possibility that the preview may be mirrored. Showing Audio Levels To monitor the average and peak power levels in an audio channel in a capture connection, you use an AVCaptureAudioChannel object. Audio levels are not key-value observable, so you must poll for updated levels as often as you want to update your user interface (for example, 10 times a second). AVCaptureAudioDataOutput *audioDataOutput = <#Get the audio data output#>; NSArray *connections = audioDataOutput.connections; if ([connections count] > 0) { // There should be only one connection to an AVCaptureAudioDataOutput. AVCaptureConnection *connection = [connections objectAtIndex:0]; NSArray *audioChannels = connection.audioChannels; for (AVCaptureAudioChannel *channel in audioChannels) { float avg = channel.averagePowerLevel; float peak = channel.peakHoldLevel; // Update the level meter user interface. } } Putting it all Together: Capturing Video Frames as UIImage Objects This brief code example to illustrates how you can capture video and convert the frames you get to UIImage objects. It shows you how to: ● Create an AVCaptureSession object to coordinate the flow of data from an AV input device to an output Media Capture Putting it all Together: Capturing Video Frames as UIImage Objects 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 50● Find the AVCaptureDevice object for the input type you want ● Create an AVCaptureDeviceInput object for the device ● Create an AVCaptureVideoDataOutput object to produce video frames ● Implement a delegate for the AVCaptureVideoDataOutput object to process video frames ● Implement a function to convert the CMSampleBuffer received by the delegate into a UIImage object Note: To focus on the most relevant code, this example omits several aspects of a complete application, including memory management. To use AV Foundation, you are expected to have enough experience with Cocoa to be able to infer the missing pieces. Create and Configure a Capture Session You use an AVCaptureSession object to coordinate the flow of data from an AV input device to an output. Create a session, and configure it to produce medium resolution video frames. AVCaptureSession *session = [[AVCaptureSession alloc] init]; session.sessionPreset = AVCaptureSessionPresetMedium; Create and Configure the Device and Device Input Capture devices are represented by AVCaptureDevice objects; the class provides methods to retrieve an object for the input type you want. A device has one or more ports, configured using an AVCaptureInput object. Typically, you use the capture input in its default configuration. Find a video capture device, then create a device input with the device and add it to the session. AVCaptureDevice *device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo]; NSError *error = nil; AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:device error:&error]; if (!input) { // Handle the error appropriately. } [session addInput:input]; Media Capture Putting it all Together: Capturing Video Frames as UIImage Objects 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 51Create and Configure the Data Output You use an AVCaptureVideoDataOutput object to process uncompressed frames from the video being captured. You typically configure several aspects of an output. For video, for example, you can specify the pixel format using the videoSettings property, and cap the frame rate by setting the minFrameDuration property. Create and configure an output for video data and add it to the session; cap the frame rate to 15 fps by setting the minFrameDuration property to 1/15 second: AVCaptureVideoDataOutput *output = [[AVCaptureVideoDataOutput alloc] init]; [session addOutput:output]; output.videoSettings = @{ (NSString *)kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA) }; output.minFrameDuration = CMTimeMake(1, 15); The data output object uses delegation to vend the video frames. The delegate must adopt the AVCaptureVideoDataOutputSampleBufferDelegate protocol. When you set the data output’s delegate, you must also provide a queue on which callbacks should be invoked. dispatch_queue_t queue = dispatch_queue_create("MyQueue", NULL); [output setSampleBufferDelegate:self queue:queue]; dispatch_release(queue); You use the queue to modify the priority given to delivering and processing the video frames. Implement the Sample Buffer Delegate Method In the delegate class, implement the method (captureOutput:didOutputSampleBuffer:fromConnection:) that is called when a sample buffer is written. The video data output object delivers frames as CMSampleBuffers, so you need to convert from the CMSampleBuffer to a UIImage object. The function for this operation isshown in “Converting a CMSampleBuffer to a UIImage” (page 59). - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection { Media Capture Putting it all Together: Capturing Video Frames as UIImage Objects 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 52UIImage *image = imageFromSampleBuffer(sampleBuffer); // Add your code here that uses the image. } Remember that the delegate method is invoked on the queue you specified in setSampleBufferDelegate:queue:; if you want to update the user interface, you must invoke any relevant code on the main thread. Starting and Stopping Recording After configuring the capture session, you send it a startRunning message to start the recording. [session startRunning]; To stop recording, you send the session a stopRunning message. Media Capture Putting it all Together: Capturing Video Frames as UIImage Objects 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 53Time-based audio-visual data such as a movie file or a video stream is represented in the AV Foundation framework by AVAsset. Its structure dictates much of the framework works. Several low-level data structures that AV Foundation uses to represent time and media such as sample buffers come from the Core Media framework. Representation of Assets AVAsset is the core class in the AV Foundation framework. It provides a format-independent abstraction of time-based audiovisual data, such as a movie file or a video stream. In many cases, you work with one of its subclasses: you use the composition subclasses when you create new assets (see “Editing” (page 7)), and you use AVURLAsset to create a new asset instance from media at a given URL (including assetsfrom the MPMedia framework or the Asset Library framework—see “Using Assets” (page 9)). AVURLAsset AVMutableComposition AVComposition AVAsset NSObject An asset contains a collection of tracks that are intended to be presented or processed together, each of a uniform media type, including (but not limited to) audio, video, text, closed captions, and subtitles. The asset object providesinformation about whole resource,such asits duration or title, as well as hintsfor presentation, such as its natural size. Assets may also have metadata, represented by instances of AVMetadataItem. 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 54 Time and Media RepresentationsA track is represented by an instance of AVAssetTrack. In a typical simple case, one track represents the audio component and another represents the video component; in a complex composition, there may be multiple overlapping tracks of audio and video. AVAsset AVMetadataItem AVMetadataItem AVAssetTrack AVAssetTrack AVAssetTrack AVAssetTrack A track has a number of properties, such as its type (video or audio), visual and/or audible characteristics (as appropriate), metadata, and timeline (expressed in terms of its parent asset). A track also has an array of format descriptions. The array contains CMFormatDescriptions (see CMFormatDescriptionRef), each of which describes the format of media samples referenced by the track. A track that contains uniform media (for example, all encoded using to the same settings) will provide an array with a count of 1. A track may itself be divided into segments, represented by instances of AVAssetTrackSegment. A segment is a time mapping from the source to the asset track timeline. Representations of Time Time in AV Foundation is represented by primitive structures from the Core Media framework. CMTime Represents a Length of Time CMTime is a C structure that represents time as a rational number, with a numerator (an int64_t value), and a denominator (an int32_t timescale).Conceptually, the timescale specifies the fraction of a second each unit in the numerator occupies. Thusif the timescale is 4, each unit represents a quarter of a second; if the timescale is 10, each unit represents a tenth of a second, and so on. You frequently use a timescale of 600, since this is a common multiple of several commonly-used frame-rates: 24 frames per second (fps) for film, 30 fps for NTSC (used for TV in North America and Japan), and 25 fps for PAL (used for TV in Europe). Using a timescale of 600, you can exactly represent any number of frames in these systems. In addition to a simple time value, a CMTime can represent non-numeric values: +infinity, -infinity, and indefinite. It can also indicate whether the time been rounded at some point, and it maintains an epoch number. Time and Media Representations Representations of Time 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 55Using CMTime You create a time using CMTimeMake, or one of the related functions such as CMTimeMakeWithSeconds (which allows you to create a time using a float value and specify a preferred time scale). There are several functions for time-based arithmetic and to compare times, as illustrated in the following example. CMTime time1 = CMTimeMake(200, 2); // 200 half-seconds CMTime time2 = CMTimeMake(400, 4); // 400 quarter-seconds // time1 and time2 both represent 100 seconds, but using different timescales. if (CMTimeCompare(time1, time2) == 0) { NSLog(@"time1 and time2 are the same"); } Float64 float64Seconds = 200.0 / 3; CMTime time3 = CMTimeMakeWithSeconds(float64Seconds , 3); // 66.66... third-seconds time3 = CMTimeMultiply(time3, 3); // time3 now represents 200 seconds; next subtract time1 (100 seconds). time3 = CMTimeSubtract(time3, time1); CMTimeShow(time3); if (CMTIME_COMPARE_INLINE(time2, ==, time3)) { NSLog(@"time2 and time3 are the same"); } For a list of all the available functions, see CMTime Reference . Special Values of CMTime Core Media provides constants for special values: kCMTimeZero, kCMTimeInvalid, kCMTimePositiveInfinity, and kCMTimeNegativeInfinity. There are many ways, though in which a CMTime can, for example, represent a time that is invalid. If you need to test whether a CMTime is valid, or a non-numeric value, you should use an appropriate macro, such as CMTIME_IS_INVALID, CMTIME_IS_POSITIVE_INFINITY, or CMTIME_IS_INDEFINITE. CMTime myTime = <#Get a CMTime#>; if (CMTIME_IS_INVALID(myTime)) { Time and Media Representations Representations of Time 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 56// Perhaps treat this as an error; display a suitable alert to the user. } You should not compare the value of an arbitrary CMTime with kCMTimeInvalid. Representing a CMTime as an Object If you need to use CMTimes in annotations or Core Foundation containers, you can convert a CMTime to and from a CFDictionary (see CFDictionaryRef) using CMTimeCopyAsDictionary and CMTimeMakeFromDictionary respectively. You can also get a string representation of a CMTime using CMTimeCopyDescription. Epochs The epoch number of a CMTime is usually set to 0, but you can use it to distinguish unrelated timelines. For example, the epoch could be incremented each cycle through a presentation loop, to differentiate between time N in loop 0 from time N in loop 1. CMTimeRange Represents a Time Range CMTimeRange is a C structure that has a start time and duration, both expressed as CMTimes. A time range does not include the time that is the start time plus the duration. You create a time range using CMTimeRangeMake or CMTimeRangeFromTimeToTime. There are constraints on the value of the CMTimes’ epochs: ● CMTimeRanges cannot span different epochs. ● The epoch in a CMTime that represents a timestamp may be non-zero, but you can only perform range operations (such as CMTimeRangeGetUnion) on ranges whose start fields have the same epoch. ● The epoch in a CMTime that represents a duration should always be 0, and the value must be non-negative. Working with Time Ranges Core Media provides functions you can use to determine whether a time range contains a given time or other time range, or whether two time ranges are equal, and to calculate unions and intersections of time ranges, such as CMTimeRangeContainsTime, CMTimeRangeEqual, CMTimeRangeContainsTimeRange, and CMTimeRangeGetUnion. Given that a time range does not include the time that is the start time plus the duration, the following expression always evaluates to false: Time and Media Representations Representations of Time 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 57CMTimeRangeContainsTime(range, CMTimeRangeGetEnd(range)) For a list of all the available functions, see CMTimeRange Reference . Special Values of CMTimeRange Core Media provides constants for a zero-length range and an invalid range, kCMTimeRangeZero and kCMTimeRangeInvalid respectively. There are many ways, though in which a CMTimeRange can be invalid, or zero—or indefinite (if one of the CMTimes is indefinite. If you need to test whether a CMTimeRange is valid, zero, or indefinite, you should use an appropriate macro: CMTIMERANGE_IS_VALID, CMTIMERANGE_IS_INVALID, CMTIMERANGE_IS_EMPTY, or CMTIMERANGE_IS_EMPTY. CMTimeRange myTimeRange = <#Get a CMTimeRange#>; if (CMTIMERANGE_IS_EMPTY(myTimeRange)) { // The time range is zero. } You should not compare the value of an arbitrary CMTimeRange with kCMTimeRangeInvalid. Representing a CMTimeRange as an Object If you need to use CMTimeRangesin annotations or Core Foundation containers, you can convert a CMTimeRange to and from a CFDictionary (see CFDictionaryRef) using CMTimeRangeCopyAsDictionary and CMTimeRangeMakeFromDictionary respectively. You can also get a string representation of a CMTime using CMTimeRangeCopyDescription. Representations of Media Video data and its associated metadata is represented in AV Foundation by opaque objects from the Core Media framework. Core Media represents video data using CMSampleBuffer (see CMSampleBufferRef). CMSampleBuffer is a Core Foundation-style opaque type; an instance contains the sample buffer for a frame of video data as a Core Video pixel buffer (see CVPixelBufferRef). You access the pixel buffer from a sample buffer using CMSampleBufferGetImageBuffer: CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(<#A CMSampleBuffer#>); Time and Media Representations Representations of Media 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 58From the pixel buffer, you can accessthe actual video data. For an example,see “Converting a CMSampleBuffer to a UIImage” (page 59). In addition to the video data, you can retrieve a number of other aspects of the video frame: ● Timing information You get accurate timestamps for both the original presentation time and the decode timeusingCMSampleBufferGetPresentationTimeStampandCMSampleBufferGetDecodeTimeStamp respectively. ● Format information The format information is encapsulated in a CMFormatDescription object (see CMFormatDescriptionRef). From the format description, you can get for example the pixel type and video dimensions using CMVideoFormatDescriptionGetCodecType and CMVideoFormatDescriptionGetDimensions respectively. ● Metadata Metadata are stored in a dictionary as an attachment. You use CMGetAttachment to retrieve the dictionary: CMSampleBufferRef sampleBuffer = <#Get a sample buffer#>; CFDictionaryRef metadataDictionary = CMGetAttachment(sampleBuffer, CFSTR("MetadataDictionary", NULL); if (metadataDictionary) { // Do something with the metadata. } Converting a CMSampleBuffer to a UIImage The following function shows how you can convert a CMSampleBuffer to a UIImage object. You should consider your requirements carefully before using it. Performing the conversion is a comparatively expensive operation. It is appropriate to, for example, create a still image from a frame of video data taken every second or so. You should not use this as a means to manipulate every frame of video coming from a capture device in real time. UIImage *imageFromSampleBuffer(CMSampleBufferRef sampleBuffer) { CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); // Lock the base address of the pixel buffer. CVPixelBufferLockBaseAddress(imageBuffer,0); // Get the number of bytes per row for the pixel buffer. Time and Media Representations Converting a CMSampleBuffer to a UIImage 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 59size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); // Get the pixel buffer width and height. size_t width = CVPixelBufferGetWidth(imageBuffer); size_t height = CVPixelBufferGetHeight(imageBuffer); // Create a device-dependent RGB color space. static CGColorSpaceRef colorSpace = NULL; if (colorSpace == NULL) { colorSpace = CGColorSpaceCreateDeviceRGB(); if (colorSpace == NULL) { // Handle the error appropriately. return nil; } } // Get the base address of the pixel buffer. void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer); // Get the data size for contiguous planes of the pixel buffer. size_t bufferSize = CVPixelBufferGetDataSize(imageBuffer); // Create a Quartz direct-access data provider that uses data we supply. CGDataProviderRef dataProvider = CGDataProviderCreateWithData(NULL, baseAddress, bufferSize, NULL); // Create a bitmap image from data supplied by the data provider. CGImageRef cgImage = CGImageCreate(width, height, 8, 32, bytesPerRow, colorSpace, kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder32Little, dataProvider, NULL, true, kCGRenderingIntentDefault); CGDataProviderRelease(dataProvider); // Create and return an image object to represent the Quartz image. UIImage *image = [UIImage imageWithCGImage:cgImage]; CGImageRelease(cgImage); Time and Media Representations Converting a CMSampleBuffer to a UIImage 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 60CVPixelBufferUnlockBaseAddress(imageBuffer, 0); return image; } Time and Media Representations Converting a CMSampleBuffer to a UIImage 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 61This table describes the changes to AV Foundation Programming Guide . Date Notes 2011-10-12 Updated for iOS5 to include references to release notes. 2011-04-28 First release for OS X v10.7. 2010-09-08 TBD First version of a document that describes a low-level framework you use to play, inspect, create, edit, capture, and transcode media assets. 2010-08-16 2011-10-12 | © 2011 Apple Inc. All Rights Reserved. 62 Document Revision HistoryApple Inc. © 2011 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, Cocoa, iPhone, iPod, iPod touch, Mac, Objective-C, OS X, Quartz, and QuickTime are trademarks of Apple Inc., registered in the U.S. and other countries. OpenGL is a registered trademark of Silicon Graphics, Inc. Times is a registered trademark of Heidelberger Druckmaschinen AG, available from Linotype Library GmbH. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. Preferences and Settings Programming GuideContents About Preferences and Settings 5 At a Glance 5 You Decide What Preferences You Want to Expose 5 Apps Provide Their Own Preferences Interface 5 Apps Access Preferences Using the User Defaults Object 6 iCloud Stores Shared Preference and Configuration Data 6 Defaults Are Grouped into Domains in OS X 6 A Settings Bundle Manages Preferences for iOS Apps 6 See Also 7 About the User Defaults System 8 What Makes a Good Preference? 8 Providing a Preference Interface 8 The Organization of Preferences 9 The Argument Domain 10 The Application Domain 10 The Global Domain 11 The Languages Domains 11 The Registration Domain 11 Viewing Preferences Using the Defaults Tool 12 Accessing Preference Values 13 Registering Your App’s Default Preferences 13 Getting and Setting Preference Values 14 Synchronizing and Detecting Preference Changes 15 Managing Preferences Using Cocoa Bindings 16 Managing Preferences Using Core Foundation 16 Setting a Preference Value Using Core Foundation 16 Getting a Preference Value Using Core Foundation 17 Storing Preferences in iCloud 19 Strategies for Using the iCloud Key-Value Store 19 Configuring Your App to Use the Key-Value Store 20 Accessing Values in the Key-Value Store 21 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 2Defining the Scope of Key-Value Store Changes 22 Implementing an iOS Settings Bundle 24 The Settings App Interface 24 The Settings Bundle 26 The Settings Page File Format 27 Hierarchical Preferences 27 Localized Resources 28 Creating and Modifying the Settings Bundle 29 Adding the Settings Bundle 29 Preparing the Settings Page for Editing 29 Configuring a Settings Page: A Tutorial 31 Creating Additional Settings Page Files 33 Debugging Preferences for Simulated Apps 34 Document Revision History 35 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 3 ContentsFigures, Tables, and Listings About the User Defaults System 8 Table 1-1 Options for displaying preferences to the user 8 Table 1-2 Search order for domains 10 Accessing Preference Values 13 Listing 2-1 Registering default preference values 14 Listing 2-2 Writing a simple default 17 Listing 2-3 Reading a simple default 17 Storing Preferences in iCloud 19 Listing 3-1 Updating local preference values using iCloud 21 Implementing an iOS Settings Bundle 24 Figure 4-1 Organizing preferences using child panes 28 Figure 4-2 Formatted contents of the Root.plist file 30 Figure 4-3 A root Settings page 31 Table 4-1 Preference control types 25 Table 4-2 Contents of the Settings.bundle directory 26 Table 4-3 Root-level keys of a preferences Settings page file 27 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 4Preferences are pieces of information that you store persistently and use to configure your app. Apps often expose preferences to users so that they can customize the appearance and behavior of the app. Most preferences are stored locally using the Cocoa preferences system—known as the user defaults system. Apps can also store preferences in a user’s iCloud account using the key-value store. The user defaultssystem and key-value store are both designed forstoring simple data types—strings, numbers, dates, Boolean values, URLs, data objects, and so forth—in a property list. The use of a property list also means you can organize your preference data using array and dictionary types. It is also possible to store other objects in a property list by encoding them into an NSData object first. At a Glance Apps integrate preferences in several ways, including programmatically at various points throughout your code and as part of the user interface. Preferences are supported in both iOS and Mac apps. You Decide What Preferences You Want to Expose Preferences are different for each app, and it is up to you to decide what parts of your app you want to make configurable. Configuration involves checking the value of a stored preference from your code and taking action based on that value. Thus, the preference value itself should always be simple and have a specific meaning that is then implemented by your app. Relevant section: “What Makes a Good Preference?” (page 8) Apps Provide Their Own Preferences Interface Because each app’s preferences are different, the app itself is responsible for deciding how best to present those preferences to the user, if at all. Both iOS and OS X provide some standard places for you to incorporate a preferences interface, but you are still responsible for designing that interface and displaying it at the appropriate time. 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 5 About Preferences and SettingsRelevant section: “Providing a Preference Interface” (page 8) Apps Access Preferences Using the User Defaults Object Apps accesslocally stored preferences using a user defaults object, which is either an NSUserDefaults object (iOS and OS X) or an NSUserDefaultsController object (OS X only). In addition to retrieving preference values, apps can use this object to register default values for preferences and manage other aspects of the preferences system. Relevant chapter: “Accessing Preference Values” (page 13) iCloud Stores Shared Preference and Configuration Data Apps that support iCloud can put some of their preference data in the user’s iCloud account and make it available to instances of the app running on the user’s other devices. You use this capability to supplement (not replace) your app’s existing preferences data and provide a more coherent experience across the user’s devices. For example, a magazine app might store information about the page number and issue last read by the user so that the app running on a different device can show that same page. Relevant chapter: “Storing Preferences in iCloud” (page 19) Defaults Are Grouped into Domains in OS X OS X preferences are grouped by domainsso thatsystem preferences can be differentiated from app preferences. Splitting preferences in this manner lets the user specify some preferences globally and then override one or more of those preferences inside an app. Relevant section: “The Organization of Preferences” (page 9) A Settings Bundle Manages Preferences for iOS Apps An iOS, apps can display preferences from the Settings app, which is a good place to put preferences that the user does not need to configure frequently. To display preferences in the Settings app, an app’s bundle must include a special resource called a Settings bundle that defines the preferences to display, the proper way to display them, and the information needed to record the user’s selections. About Preferences and Settings At a Glance 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 6Note: Apps are not required to use a Settings bundle to manage all preferences. For preferences that the user islikely to change frequently, the app can display its own custom interface for managing those preferences. Relevant chapter: “Implementing an iOS Settings Bundle” (page 24) See Also For information about property lists, see Property List Programming Guide . For more advanced information about using Core Foundation to manage preferences, see Preferences Programming Topics for Core Foundation . About Preferences and Settings See Also 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 7The user defaults system manages the storage of preferences for each user. Most preferences are stored persistently and therefore do not change between subsequent launch cycles of your app. Apps use preferences to track user-initiated and program-initiated configuration changes. What Makes a Good Preference? When defining your app’s preferences, it is better to use simple values and data types whenever possible. The preferences system is built around property-list data types such as strings, numbers, and dates. Although you can use an NSData object to store arbitrary objects in preferences, doing so is not recommended in most cases. Storing objects persistently means that your app has to decode that object at some point. In the case of preferences, a stored object means decoding the object every time you access the preference. It also means that a newer version of your app has to ensure that it is able to decode objects created and written to disk using an earlier version of your app, which is potentially error prone. A better approach for preferences is to store simple strings and values and use them to create the objects your app needs. Storing simple values meansthat your app can always accessthe value. The only thing that changes from release to release is the interpretation of the simple value and the objects your app creates in response. Providing a Preference Interface For user-facing preferences, Table 1-1 lists the options for displaying those preferences to the user. As you can see from this table, most options involve the creation of a custom user interface for managing and presenting preferences. If you are creating an iOS app, you can use a Settings bundle to present preferences, but you should do so only for settings the user changes infrequently. Table 1-1 Options for displaying preferences to the user Preference iOS OS X Frequently changed preferences Custom UI Custom UI Infrequently changed preferences Settings bundle Custom UI 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 8 About the User Defaults SystemNote: An example of preferencesthat might change frequently include thingslike the volume levels or control options of a game. An example of preferences that might change infrequently are the email address and server settings in the Mail app. For iOS apps, it is ultimately up to you to decide whether it is appropriate to expose preferences from the Settings app or from inside your app. Preferences in Mac apps should be accessible from a Preferences menu item in the app menu. Cocoa apps created using the Xcode templates provide such a menu item for you automatically. It is your responsibility to present an appropriate user interface when the user choosesthis menu item. You can provide that user interface by defining an action method in your app delegate that displays a custom preferences window and connecting that action method to the menu item in Interface Builder. There is no standard way to display custom preferences from inside an iOS app. You can integrate preferences in many ways, including using a separate tab in a tab-bar interface or using a custom button from one of your app’s screens. Preferences should generally be presented using a distinct view controller so that changes in preferences can be recorded when that view controller is dismissed by the user. The Organization of Preferences Preferences are grouped into domains, each of which has a name and a specific usage. For example, there’s a domain for app-specific preferences and another for systemwide preferences that apply to all apps. All preferences are stored and accessed on a per-user basis. There is no support for sharing preferences between users. Each preference has three components: ● The domain in which it is stored ● Its name (specified as an NSString object) ● Its value, which can be any property-list object (NSData, NSString, NSNumber, NSDate, NSArray, or NSDictionary) The lifetime of a preference depends on which domain you store it in. Some domains store preferences persistently by writing them to the user’s defaults database. Such preferences continue to exist from one app launch to the next. Other domains store preferences in a more volatile way, preserving preference values only for the life of the corresponding user defaults object. About the User Defaults System The Organization of Preferences 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 9A search for the value of a given preference proceeds through the domains in an NSUserDefaults object’s search list. Only domains in the search list are searched and they are searched in the order shown in Table 1-2, starting with the NSArgumentDomain domain. A search ends when a preference with the specified name is found. If multiple domains contain the same preference, the value is taken from the domain nearest the beginning of the search list. Table 1-2 Search order for domains Domain State NSArgumentDomain volatile Application (Identified by the app’s identifier) persistent NSGlobalDomain persistent Languages (Identified by the language names) volatile NSRegistrationDomain volatile The Argument Domain The argument domain comprises values set from command- line arguments (if you started the app from the command line) and is identified by the NSArgumentDomain constant. Values set from the command line are automatically placed into this domain by the system. To add a value to this domain, specify the preference name on the command line (preceded with a hyphen) and follow it with the corresponding value. For example, the following command launches Xcode and sets the value of its IndexOnOpen preference to NO: localhost> Xcode.app/Contents/MacOS/Xcode -IndexOnOpen NO Preferencesset from the command line temporarily override the established valuesstored in the user’s defaults database. In the preceding example,setting the IndexOnOpen preference to NO prevents Xcode from indexing projects automatically, even if the preference is set to YES in the user defaults database. The Application Domain The application domain contains app-specific preferences that are stored in the user defaults database of the current user. When you use the shared NSUserDefaults object (or a NSUserDefaultsController object in OS X) to write preferences, those preferences are automatically placed in this domain. About the User Defaults System The Organization of Preferences 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 10Because this domain is app-specific, the contents of the domain are tied to your app’s bundle identifier. The contents of this domain are stored in a file that is managed by the system. Currently, this file is located in the $HOME/Library/Preferences/ directory, where $HOME is either the app’s home directory or the user’s home directory (depending on the platform and whether your app is in a sandbox). The name of the user defaults database file is .plist, where is your app’s bundle identifier. You should not modify this file directly but can inspect it during debugging to make sure preference values are being written by your app. The Global Domain The global domain contains preferencesthat are applicable to all apps and isidentified by the NSGlobalDomain constant. This domain is typically used by system frameworks to store system-wide values and should not be used by your app to store app-specific values. If you want to change the value of a preference in the global domain, write that same preference to the application domain with the new value. Examples of how the system frameworks use this domain: ● Instances of the NSRuleView class store the user’s preferred measurement units in the AppleMeasurementUnits key. Using this storage location causes ruler views in all apps to use the same units. ● The system uses the AppleLanguages key to store the user’s preferred languages as an array of strings. For example, a user could specify English as the preferred language, followed by Spanish, French, German, Italian, and Swedish. The Languages Domains For each language in the AppleLanguages preference, the system recordslanguage-specific preference values in a domain whose name is based on the language name. Each language-specific domain contains preferences for the corresponding locale. Many classes in the Foundation framework (such as the NSDate, NSDateFormatter, NSTimeZone, NSString, and NSScanner classes) use this locale information to modify their behavior. For example, when you request a string representation of an NSCalendarDate object, the NSCalendarDate object uses the locale information to find the names of months and the days of the week for the user’s preferred language. The Registration Domain The registration domain defines the set of default values to use if a given preference is not set explicitly in one of the other domains. At launch time, an app can call the registerDefaults: method of NSUserDefaults to specify a default set of values for important preferences. When an app launches for the first time, most preferences have no values,so retrieving them would yield undefined results. Registering a set of default values ensures that your app always has a known good set of values to operate on. About the User Defaults System The Organization of Preferences 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 11The contents of the registration domain can be set only by using the registerDefaults: method. Viewing Preferences Using the Defaults Tool In OS X, the defaults command-line tool provides a way for you to examine the contents of the user defaults database. During app development, you might use this tool to validate the preferences your app is writing to disk. To do that, you would use a command of the following form from the Terminal app: defaults read To read the contents of the global domain, you would use the following command: defaults read NSGlobalDomain For more information about using the defaults tool to read and write preference values, see defaults man page. About the User Defaults System Viewing Preferences Using the Defaults Tool 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 12You use the NSUserDefaults class to gain access to your app’s preferences. Each app is provided with a single instance of this class, accessible from the standardUserDefaults class method. You use the shared user defaults object to: ● Specify any default values for your app’s preferences at launch time. ● Get and set individual preference values stored in the app domain. ● Remove preference values. ● Examine the contents of the volatile preference domains. Mac appsthat use Cocoa bindings can use an NSUserDefaultsController object to set and get preferences automatically. You typically add such an object to the same nib file you use for displaying user-facing preferences. You bind your user interface controls to items in the user defaults controller, which handles the process of getting and setting values in the user defaults database. Preference values must be one of the standard property list object types: NSData, NSString, NSNumber, NSDate, NSArray, or NSDictionary. The NSUserDefaults class also provides built-in manipulations for storing NSURL objects as preference values. For more information about property lists and their contents, see Property List Programming Guide . Registering Your App’s Default Preferences At launch time, an app should register default values for any preferences that it expects to be present and valid. When you request the value of a preference that has never been set, the methods of the NSUserDefaults class return default values that are appropriate for the data type. For numerical scalar values, this typically means returning 0, but for strings and other objects it means returning nil. If these standard default values are not appropriate for your app, you can register your own default values using the registerDefaults: method. This method places your custom default values in the NSRegistrationDomain domain, which causes them to be returned when a preference is not explicitly set. When calling the registerDefaults: method, you must provide a dictionary of all the default values you need to register. Listing 2-1 shows an example where an iOS app registers its default values early in the launch cycle. You can register default values at any time, of course, butshould alwaysregister them before attempting to retrieve any preference values. 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 13 Accessing Preference ValuesListing 2-1 Registering default preference values - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions { // Register the preference defaults early. NSDictionary *appDefaults = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES] forKey:@"CacheDataAgressively"]; [[NSUserDefaults standardUserDefaults] registerDefaults:appDefaults]; // Other initialization... } When registering default values for scalar types, use an NSNumber object to specify the value for the number. If you want to register a preference whose value is a URL, use the archivedDataWithRootObject: method of NSKeyedArchiver to encode the URL in an NSData object first. Although you can use a similar technique for other types of objects, you should avoid doing so when a simpler option is available. Getting and Setting Preference Values You get and set preference values using the methods of the NSUserDefaults class. This class has methods for getting and setting preferences with scalar values of type Boolean, integer, float, and double. It also has methodsfor getting and setting preferences whose value is an object of type NSData, NSDate, NSString, NSNumber, NSArray, NSDictionary, and NSURL. There are two situations where you might get preference values and one where you might set them: ● Get preference values: ● When you need to use the value to configure your app’s behavior. ● When you need to display the value in your preferences interface. ● Set preference values when the user changes them in your preferences interface. The following code shows how you might get a preference value in your code. In this example, the code retrieves the value of the CacheDataAggressively key, which is custom key that the app might use to determine its caching strategy. Code like this can be used anywhere to handle custom configuration of your app. If you wanted to display this particular preference value to the user, you would use similar code to configure the controls of your preferences interface. Accessing Preference Values Getting and Setting Preference Values 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 14if ([[NSUserDefaults standardUserDefaults] boolForKey:@"CacheDataAggressively"]) { // Delete the backup file. } To set a preference value programmatically, you call the corresponding setter methods of NSUserDefaults. When setting object values, you must use the setObject:forKey: method. When calling this method, you must make sure that the object is one of the standard property list types. The following example sets some preferences based on the state of the app’s preferences interface. NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults]; if ([cacheAgressivelyButton state] == NSOnState) { // The user wants to cache files aggressively. [defaults setBool:YES forKey:@"CacheDataAggressively"]; [defaults setObject:[NSDate dateWithTimeIntervalSinceNow:(3600 * 24 * 7)] forKey:@"CacheExpirationDate"]; // Set a 1-week expiration } else { // The user wants to use lazy caching. [defaults setBool:NO forKey:@"CacheDataAggressively"]; [defaults removeObjectForKey:@"CacheExpirationDate"]; } You do not have to display a preferences interface to manage all values. Your app can use preferences to cache interesting information. For example, NSWindow objectsstore their current location in the user defaultssystem. This data allows them to return to the same location the next time the user starts the app. Synchronizing and Detecting Preference Changes Because the NSUserDefaults class caches values, it issometimes necessary to synchronize the cached values with the current contents of the user defaults database. Your app is not always the only entity modifying the user defaults database. In iOS, the Settings app can modify the values of preferences for apps that have a Settings bundle. In OS X, the system and other apps might modify preferences values in response to user actions. For example, if the user changes preferred languages, the system writes the new values to the user defaults database. In OS X v10.5 and later, the shared NSUserDefaults object synchronizes its caches automatically at periodic intervals. However, apps can call the synchronize method manually to force an update of the cached values. Accessing Preference Values Synchronizing and Detecting Preference Changes 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 15To detect when changes to a preference value occur, apps can also register for the notification NSUserDefaultsDidChangeNotification. The shared NSUserDefaults object sends this notification to your app whenever it detects a change to a preference located in one of the persistent domains. You can use this notification to respond to changes that might impact your user interface. For example, you could use it to detect changes to the user’s preferred language and update your app content appropriately. Managing Preferences Using Cocoa Bindings Mac apps can use Cocoa bindings to set preference values directly from their user interfaces. Modifying preferences using bindings involves adding an NSUserDefaultsController object to the appropriate nib files and binding the values of your controls to the preference values in the user defaults database. When your app showsthe interface, the user defaults controller automatically loads valuesfrom the user defaults database and uses them to set the value of controls. Similarly, when the user changes the value in a control, the user defaults controller updates the value in the user defaults database. For more information on how to use the NSUserDefaultsController class to bind preference values to your user interface, see “User Defaults and Bindings” in Cocoa Bindings Programming Topics. Managing Preferences Using Core Foundation The Core Foundation framework provides its own set of interfaces for accessing preferences stored in the user defaults database. Like the NSUserDefaults class, you can use Core Foundation functions to get and set preference values and synchronize the user defaults database. Unlike NSUserDefaults, you can use the Core Foundation functions to write preferences for different apps and on different computers. Note that modifying some preferences domains(those not belonging to the current app and user) requiresroot privileges(or admin privileges prior to OS X v10.6); for information on how to gain suitable privileges, see Authorization Services Programming Guide . Writing outside the app domain is not possible for apps installed in a sandbox. For information about the Core Foundation functions for getting and setting preferences, see Preferences Utilities Reference . Setting a Preference Value Using Core Foundation Preferences are stored as key-value pairs. The key must be a CFString object, but the value can be any Core Foundation property list value (see Property List Programming Topics for Core Foundation ), including the container types. For example, you might have a key called defaultWindowWidth that defines the width in Accessing Preference Values Managing Preferences Using Cocoa Bindings 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 16pixels of any new windows that your app creates. Its value would most likely be of type CFNumber. You might also decide to combine window width and height into a single preference called defaultWindowSize and make its value be a CFArray object containing two CFNumber objects. The code in Listing 2-2 demonstrates how to create a simple preference for the app MyTextEditor. The example sets the default text color for the app to blue. Listing 2-2 Writing a simple default CFStringRef textColorKey = CFSTR("defaultTextColor"); CFStringRef colorBLUE = CFSTR("BLUE"); // Set up the preference. CFPreferencesSetAppValue(textColorKey, colorBLUE, kCFPreferencesCurrentApplication); // Write out the preference data. CFPreferencesAppSynchronize(kCFPreferencesCurrentApplication); Notice that CFPreferencesSetAppValue by itself is not sufficient to create the new preference. A call to CFPreferencesAppSynchronize isrequired to actually save the value. If you are writing multiple preferences, it is more efficient to sync only once after the last value has been set than to sync after each individual value is set. For example, if you implement a preference pane you might synchronize only when the user presses an OK button. In other cases you might not want to sync at all until the app quits—although note that if the app crashes, all unsaved preferences settings will be lost. Getting a Preference Value Using Core Foundation The simplest way to locate and retrieve a preference value is to use the CFPreferencesCopyAppValue function. This call searches through the various preference domains in order until it finds the key you have specified. If a preference has been set in a less specific domain—Any Application, for example —its value is retrieved with this call if a more specific version cannot be found. Listing 2-3 shows how to retrieve the text color preference saved in Listing 2-2 (page 17). Listing 2-3 Reading a simple default CFStringRef textColorKey = CFSTR("defaultTextColor"); CFStringRef textColor; Accessing Preference Values Managing Preferences Using Core Foundation 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 17// Read the preference. textColor = (CFStringRef)CFPreferencesCopyAppValue(textColorKey, kCFPreferencesCurrentApplication); // When finished with value, you must release it // CFRelease(textColor); All values returned from preferences are immutable, even if you have just set the value using a mutable object. Accessing Preference Values Managing Preferences Using Core Foundation 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 18An app can use the iCloud key-value store to share small amounts of data with other instances of itself on the user’s other computers and iOS devices. The key-value store is intended for simple data types like those you might use for preferences. For example, a magazine app might store the current issue and page number being read by the user so that other instances of the app can open to the same page when launched. You should not use this store for large amounts of data or for complex data types. To use the iCloud key-value store, do the following: 1. In Xcode, configure the com.apple.developer.ubiquity-kvstore-identifier entitlement for your app. 2. In your code, create the shared NSUbiquitousKeyValueStore object and register for change notifications. 3. Use the methods of NSUbiquitousKeyValueStore to get and set values. Key-value data in iCloud is limited to simple property-list types (strings, numbers, dates, and so on). Strategies for Using the iCloud Key-Value Store The key-value store is not intended for storing large amounts of data. It is intended for storing configuration data, preferences, and small amounts of app-related data. To help you decide whether the key-value store is appropriate for your needs, consider the following: ● Each app is limited to 1 MB of total space in the key-value store. (There is also a separate per-key limit of 1 MB and a maximum of 1024 keys are allowed.) Thus, you cannot use the key-value store to share large amounts of data. ● The key-value store supports only property-list types. Property-list types include simple types such as NSNumber, NSString, and NSDate objects. You can also store raw blocks of data in NSData objects and arrange all of the types using NSArray and NSDictionary objects. ● The key-value store is intended for storing data that changes infrequently. If the apps on a device make frequent changes to the key-value store, the system may defer the synchronization of some changes in order to minimize the number of round trips to the server. The more frequently apps make changes, the more likely it is that later changes will be deferred and not show up on other devices right away. 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 19 Storing Preferences in iCloud● The key-value store is not a replacement for preferences or other local techniques for saving the same data. The purpose of the key-value store is to share data between apps, but if iCloud is not enabled or is not available on a given device, you still might want to keep a local copy of the data. If you are using the key-value store to share preferences, one approach is to store the actual values in the user defaults database and synchronize them using the key-value store. (If you do not want to use the preferences system, you could also save the changes in a custom property-list file or some other local storage.) When you change the value of a key locally, write that change to both the user defaults database and to the iCloud key-value store at the same time. To receive changesfrom externalsources, add an observer for the notification NSUbiquitousKeyValueStoreDidChangeExternallyNotification and use your handler method to detect which keys changed externally and update the corresponding data in the user defaults database. By doing this, your user defaults database always contains the correct configuration values. The iCloud key-value store simply becomes a mechanism for ensuring that the user defaults database has the most recent changes. Configuring Your App to Use the Key-Value Store In order to use of the key-value store, an app must be explicitly configured with the com.apple.developer.ubiquity-kvstore-identifier entitlement. You use Xcode to enable this entitlement and specify its value for your app: 1. In your Xcode project, select the target for your app. 2. In the Summary tab, enable the Entitlements option. 3. Specify a value for the iCloud Key-Value Store field. When you enable entitlements, Xcode automatically fills in a default value for the iCloud Key-Value Store field that is based on the bundle identifier of your app. For most apps, the default value is what you want. However, if your app shares its key-value storage with another app, you must specify the bundle identifier for the other app instead. For example, if you have a lite version of your app, you might want it to use the same key-value store as the paid version. Enabling the entitlement is all you have to do to use the shared NSUbiquitousKeyValueStore object. As long as the entitlement is configured and contains a valid value, the key-value store object writes its data to the appropriate location in the user’s iCloud account. If there is a problem attaching to the specified iCloud container, any attemptsto read or write key values will fail. To ensure the key-value store is configured properly and accessible, you should execute code similar to the following early in your app’s launch cycle: NSUbiquitousKeyValueStore* store = [NSUbiquitousKeyValueStore defaultStore]; [[NSNotificationCenter defaultCenter] addObserver:self Storing Preferences in iCloud Configuring Your App to Use the Key-Value Store 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 20selector:@selector(updateKVStoreItems:) name:NSUbiquitousKeyValueStoreDidChangeExternallyNotification object:store]; [store synchronize]; Creating the key-value store object early in your app’s launch cycle is recommended because it ensures that your app receives updates from iCloud in a timely manner. The best way to determine if changes have been made to keys and values is to register for the notification NSUbiquitousKeyValueStoreDidChangeExternallyNotification. And at launch time, you should call the synchronize method manually to detect if any changes were made externally. You do not need to call that method at other times during you app’s execution. For more information about how to configure entitlements for an iOS app, see “Configuring Apps” in Tools Workflow Guide for iOS . Accessing Values in the Key-Value Store You get and set key-value store values using the methods of the NSUbiquitousKeyValueStore class. This class has methods for getting and setting preferences with scalar values of type Boolean, long long, and double. It also has methods for getting and setting keys whose values are NSData, NSDate, NSString, NSNumber, NSArray, or NSDictionary objects. If you are using the key-value store as a way to update locally stored preferences, you could use code similar to that in Listing 3-1 to coordinate updates to the user defaults database. This example assumes that you use the same key names and corresponding values in both iCloud and the user defaults database. It also assumes that you previously registered the updateKVStoreItems: method as the method to call in response to the notification NSUbiquitousKeyValueStoreDidChangeExternallyNotification. Listing 3-1 Updating local preference values using iCloud - (void)updateKVStoreItems:(NSNotification*)notification { // Get the list of keys that changed. NSDictionary* userInfo = [notification userInfo]; NSNumber* reasonForChange = [userInfo objectForKey:NSUbiquitousKeyValueStoreChangeReasonKey]; NSInteger reason = -1; // If a reason could not be determined, do not update anything. Storing Preferences in iCloud Accessing Values in the Key-Value Store 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 21if (!reasonForChange) return; // Update only for changes from the server. reason = [reasonForChange integerValue]; if ((reason == NSUbiquitousKeyValueStoreServerChange) || (reason == NSUbiquitousKeyValueStoreInitialSyncChange)) { // If something is changing externally, get the changes // and update the corresponding keys locally. NSArray* changedKeys = [userInfo objectForKey:NSUbiquitousKeyValueStoreChangedKeysKey]; NSUbiquitousKeyValueStore* store = [NSUbiquitousKeyValueStore defaultStore]; NSUserDefaults* userDefaults = [NSUserDefaults standardUserDefaults]; // This loop assumes you are using the same key names in both // the user defaults database and the iCloud key-value store for (NSString* key in changedKeys) { id value = [store objectForKey:key]; [userDefaults setObject:value forKey:key]; } } } Defining the Scope of Key-Value Store Changes Every call to one of the NSUbiquitousKeyValueStore methods is treated as a single atomic transaction. When transferring the data for that transaction to iCloud, the whole transaction either fails or succeeds. If it succeeds, all of the keys are written to the store and if it fails no keys are written. There is no partial writing of keys to the store. When a failure occurs, the system also generates a NSUbiquitousKeyValueStoreDidChangeExternallyNotification notification that containsthe reason for the failure. If you are using the key-value store, you should use that notification to detect possible problems. Storing Preferences in iCloud Defining the Scope of Key-Value Store Changes 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 22If you have a group of keys whose values must all be updated at the same time in order to be valid, save them together in a single transaction. To write multiple keys and values in a single transaction, create an NSDictionary object with all of the keys and values. Then write the dictionary object to the key-value store using the setDictionary:forKey: method. Writing an entire dictionary of changes ensures that all of the keys are written or none of them are. Storing Preferences in iCloud Defining the Scope of Key-Value Store Changes 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 23In iOS, the Foundation framework provides the low-level mechanism for storing the preference data. Apps then have two options for presenting preferences: ● Display preferences inside the app. ● Use a Settings bundle to manage preferences from the Settings app. Which option you choose depends on how you expect users to interact with preferences. The Settings bundle is generally the preferred mechanism for displaying preferences. However, games and other apps that contain configuration options or other frequently accessed preferences might want to present them inside the app instead. Regardless of how you present them, you use the NSUserDefaults class to access preference values from your code. This chapter focuses on the creation of a Settings bundle for your app. A Settings bundle contains files that describe the structure and presentation style of your preferences. The Settings app uses this information to create an entry for your app and to display your custom preference pages. For guidelines on how to manage and present settings and configuration options, see iOS Human Interface Guidelines. The Settings App Interface The Settings app implements a hierarchical set of pages for navigating app preferences. The main page of the Settings app liststhe system and third-party apps whose preferences can be customized. Selecting a third-party app takes the user to the preferences for that app. Every app with a Settings bundle has at least one page of preferences, referred to as the main page . If your app has only a few preferences, the main page may be the only one you need. If the number of preferences gets too large to fit on the main page, however, you can create child pages that link off the main page or other child pages. There is no specific limit to the number of child pages you can create, but you should strive to keep your preferences as simple and easy to navigate as possible. The contents of each page consists of one or more controls that you configure. Table 4-1 lists the types of controls supported by the Settings app and describes how you might use each type. The table also lists the raw key name stored in the configuration files of your Settings bundle. 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 24 Implementing an iOS Settings BundleTable 4-1 Preference control types Controltype Description The text field type displays a title (optional) and an editable text field. You can use this type for preferences that require the user to specify a custom string value. The key for this type is PSTextFieldSpecifier. Text field The title type displays a read-only string value. You can use thistype to display read-only preference values. (If the preference contains cryptic or nonintuitive values, this type lets you map the possible values to custom strings.) The key for this type is PSTitleValueSpecifier. Title The toggle switch type displays an ON/OFF toggle button. You can use this type to configure a preference that can have only one of two values. Although you typically use this type to represent preferences containing Boolean values, you can also use it with preferences containing non-Boolean values. The key for this type is PSToggleSwitchSpecifier. Toggle switch The slider type displays a slider control. You can use this type for a preference that represents a range of values. The value for this type is a real number whose minimum and maximum value you specify. The key for this type is PSSliderSpecifier. Slider The multivalue type lets the user select one value from a list of values. You can use this type for a preference that supports a set of mutually exclusive values. The values can be of any type. The key for this type is PSMultiValueSpecifier. Multivalue The group type is for organizing groups of preferences on a single page. The group type does not represent a configurable preference. It simply contains a title string that is displayed immediately before one or more configurable preferences. The key for this type is PSGroupSpecifier. Group The child pane type lets the user navigate to a new page of preferences. You use this type to implement hierarchical preferences. For more information on how you configure and use this preference type, see “Hierarchical Preferences” (page 27). The key for this type is PSChildPaneSpecifier. Child pane For detailed information about the format of each preference type, see Settings Application Schema Reference . To learn how to create and edit Settings page files, see “Creating and Modifying the Settings Bundle” (page 29). Implementing an iOS Settings Bundle The Settings App Interface 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 25The Settings Bundle A Settings bundle hasthe name Settings.bundle and residesin the top-level directory of your app’s bundle. This bundle contains one or more Settings page files that describe the individual pages of preferences. It may also include other support files needed to display your preferences, such as images or localized strings. Table 4-2 lists the contents of a typical Settings bundle. Table 4-2 Contents of the Settings.bundle directory Item name Description The Settings page file containing the preferences for the root page. The name of thisfile must be Root.plist. The contents of thisfile are described in more detail in “The Settings Page File Format” (page 27). Root.plist If you build a set of hierarchical preferences using child panes, the contents for each child pane are stored in a separate Settings page file. You are responsible for naming these files and associating them with the correct child pane. Additional .plist files These directories store localized string resources for your Settings page files. Each directory contains a single strings file, whose title is specified in your Settings page file. The strings files provide the localized strings to display for your preferences. One or more .lproj directories If you use the slider control, you can store the images for your slider in the top-level directory of the bundle. Additional images In addition to the Settings bundle, the app bundle can contain a custom icon for your app settings. The Settings app displays the icon you provide next to the entry for your app preferences. For information about app icons and how you specify them, see iOS App Programming Guide . When the Settings app launches, it checks each custom app for the presence of a Settings bundle. For each custom bundle it finds, it loadsthat bundle and displaysthe corresponding app’s name and icon in the Settings main page. When the user taps the row belonging to your app, Settings loads the Root.plist Settings page file for your Settings bundle and uses that file to build your app’s main page of preferences. In addition to loading your bundle’s Root.plist Settings page file, the Settings app also loads any language-specific resources for that file, as needed. Each Settings page file can have an associated .strings file containing localized values for any user-visible strings. As it prepares your preferences for display, the Settings app looksforstring resourcesin the user’s preferred language and substitutesthem in your preferences page prior to display. Implementing an iOS Settings Bundle The Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 26The Settings Page File Format Each Settings page file is stored in the iPhone Settings property-list file format, which is a structured file format. The simplest way to edit Settings page files is to use the built-in editor facilities of Xcode; see “Preparing the Settings Page for Editing” (page 29). You can also edit property-list files using the Property List Editor app that comes with the Xcode tools. Note: Xcode converts any XML-based property files in your project to binary format when building your app. This conversion saves space and is done for you automatically. The root element of each Settings page file contains the keys listed in Table 4-3. Only one key is actually required, but it is recommended that you include both of them. Table 4-3 Root-level keys of a preferences Settings page file Key Type Value The value for this key is an array of dictionaries, with each dictionary containing the information for a single control. For a list of control types, see Table 4-1 (page 25). For a description of the keys associated with each control, see Settings Application Schema Reference . PreferenceSpecifiers Array (required) The name of the strings file associated with this file. A copy of this file (with appropriate localized strings) should be located in each of your bundle’s language-specific project directories. If you do not include this key, the strings in this file are not localized. For information on how these strings are used, see “Localized Resources” (page 28). StringsTable String Hierarchical Preferences If you plan to organize your preferences hierarchically, each page you define must have its own separate .plist file. Each .plist file contains the set of preferences displayed only on that page. Your app’s main preferences page is always stored in a file called Root.plist. Additional pages can be given any name you like. To specify a link between a parent page and a child page, you include a child pane control in the parent page. A child pane control creates a row that, when tapped, displays a new page of settings. The File key of the child pane control identifies the name of the .plist file with the contents of the child page. The Title key Implementing an iOS Settings Bundle The Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 27identifies the title of the child page; this title is also used as the text of the control used to display the child page. The Settings app automatically provides navigation controls on the child page to allow the user to navigate back to the parent page. Figure 4-1 shows how this hierarchical set of pages works. The left side of the figure shows the .plist files, and the right side shows the relationships between the corresponding pages. Figure 4-1 Organizing preferences using child panes Sounds New Voicemail Group 1 Group 2 New Email Sent Mail Ringtones Sounds page Settings Group 1 Usage Sounds Group 2 Group 3 Brightness Wallpaper General Root page Sounds.plist Root.plist General.plist General page General Date & Time Group 1 Network Keyboard For more information about child pane controls and their associated keys, see Settings Application Schema Reference . Localized Resources Because preferences contain user-visible strings, you should provide localized versions of those strings with your Settings bundle. Each page of preferences can have an associated .strings file for each localization supported by your bundle. When the Settings app encounters a key that supports localization, it checks the appropriately localized .strings file for a matching key. If it finds one, it displays the value associated with that key. When looking for localized resources such as .strings files, the Settings app follows the same rules that other iOS apps follow. It first tries to find a localized version of the resource that matches the user’s preferred language setting. If no such resource exists, an appropriate fallback language is selected. Implementing an iOS Settings Bundle The Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 28For information about the format ofstringsfiles, language-specific project directories, and how language-specific resources are retrieved from bundles, see Internationalization Programming Topics. Creating and Modifying the Settings Bundle Xcode provides a template for adding a Settings bundle to your current project. The default Settings bundle contains a Root.plist file and a default language directory for storing any localized resources. You can expand this bundle as needed to include additional property list files and resources needed by your Settings bundle. Adding the Settings Bundle To add a Settings bundle to your Xcode project: 1. Choose File > New > New File. 2. Under iOS, choose Resource, and then select the Settings Bundle template. 3. Name the file Settings.bundle. In addition to adding a new Settings bundle to your project, Xcode automatically addsthat bundle to the Copy Bundle Resources build phase of your app target. Thus, all you have to do is modify the property list files of your Settings bundle and add any needed resources. The new Settings bundle has the following structure: Settings.bundle/ Root.plist en.lproj/ Root.strings Preparing the Settings Page for Editing Before editing any of the property-list files in your Settings bundle, you should configure the Xcode editor to format the contents of those files as iPhone settings. Xcode does this automatically for the Root.plist file, but you may need to format additional property-list files manually. To format a file as iPhone Settings, do the following: 1. Select the file. 2. Control-click the editor window and choose Property List Type > iPhone Settings plist if it is not already chosen. Implementing an iOS Settings Bundle Creating and Modifying the Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 29Formatting a property list makes it easier to understand and edit the file’s contents. Xcode substitutes human-readable strings (as shown in Figure 4-2) that are appropriate for the selected format. Figure 4-2 Formatted contents of the Root.plist file Implementing an iOS Settings Bundle Creating and Modifying the Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 30Configuring a Settings Page: A Tutorial This section shows you how to configure a Settings page to display the controls you want. The goal of the tutorial is to create a page like the one in Figure 4-3. If you have not yet created a Settings bundle for your project, you should do so as described in “Adding the Settings Bundle” (page 29) before proceeding with these steps. Figure 4-3 A root Settings page 1. Disclose the Preference Items key to display the default items that come with the template. 2. Change the title of Item 0 to Sound. ● Disclose Item 0 of Preference Items. ● Change the value of the Title key from Group to Sound. ● Leave the Type key set to Group. ● Click the disclosure triangle of the item to hide its contents. 3. Create the first toggle switch for the renamed Sound group. ● Select Item 2 (the toggle switch item) of Preference Items and choose Edit > Cut. ● Select Item 0 and choose Edit > Paste. (This moves the toggle switch item in front of the text field item.) ● Disclose the toggle switch item to reveal its configuration keys. ● Change the value of the Title key to Play Sounds. ● Change the value of the Identifier key to play_sounds_preference. Implementing an iOS Settings Bundle Creating and Modifying the Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 31● Click the disclosure triangle of the item to hide its contents. 4. Create a second toggle switch for the Sound group. ● Select Item 1 (the Play Sounds toggle switch). ● Choose Edit > Copy. ● Choose Edit >Paste to place a copy of the toggle switch right after the first one. ● Disclose the new toggle switch item to reveal its configuration keys. ● Change the value of its Title key to 3D Sound. ● Change the value of its Identifier key to 3D_sound_preference. ● Click the disclosure triangle of the item to hide its contents. At this point, you have finished the first group of settings and are ready to create the User Info group. 5. Change Item 3 into a Group control and name it User Info. ● Click Item 3 in the Preferences Items. This displays a pop-up menu with a list of item types. ● From the pop-up menu, choose Group to change the type of the control. ● Disclose the contents of Item 3. ● Set the value of the Title key to User Info. ● Click the disclosure triangle of the item to hide its contents. 6. Create the Name field. ● Select Item 4 in the Preferences Items. ● Using the pop-up menu, change its type to Text Field. ● Set the value of the Title key to Name. ● Set the value of the Identifier key to user_name. ● Click the disclosure triangle of the item to hide its contents. 7. Create the Experience Level settings. ● Select Item 4. ● Control-click the editor window and select Add Row to add a new item. ● Set the type of the new item to Multi Value. ● Disclose the item’s contents and set its title to Experience Level, its identifier to experience_preference, and its default value to 0. ● With the Default Value key selected, Control-click and select Add Row to add a Titles array. ● Select the Titles array and press Return to add a new subitem. ● Add two more subitems to create a total of three items. Implementing an iOS Settings Bundle Creating and Modifying the Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 32● Set the values of the subitems to Beginner, Expert, and Master. ● Hide the key’s subitems. ● Add a new item for the Values array. ● Add three subitems to the Values array and set their values to 0, 1, and 2. ● Hide the contents of Item 5. 8. Add the final group to your settings page. ● Create a new item and set its type to Group and its title to Gravity. ● Create another new item and set itstype to Slider, itsidentifier to gravity_preference, its default value to 1, and its maximum value to 2. Creating Additional Settings Page Files The Settings Bundle template includes the Root.plist file, which defines your app’s top Settings page. To define additional Settings pages, you must add additional property list files to your Settings bundle. To add a property list file to your Settings bundle in Xcode, do the following: 1. Choose File > New > New File. 2. Under iOS, select Resource, and then select the Property List template. 3. Select the new file to display its contents in the editor. 4. Control-click the editor pane and choose Property List Type > iPhone Settings plist to format the contents. 5. Control-click the editor pane again and choose Add Row to add a new key. 6. Add and configure any additional keys you need. After adding a new Settings page to your Settings bundle, you can edit the page’s contents as described in “Configuring a Settings Page: A Tutorial” (page 31). To display the settings for your page, you must reference it from a child pane control as described in “Hierarchical Preferences” (page 27). Implementing an iOS Settings Bundle Creating and Modifying the Settings Bundle 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 33Note: In Xcode 4, adding a property-list file to your project does not automatically associate it with your Settings bundle. You must use the Finder to move any additional property-list files into your Settings bundle. Debugging Preferences for Simulated Apps When running your app, iOS Simulatorstores any preferences valuesfor your app in ~/Library/Application Support/iOS Simulator/User/Applications//Library/Preferences, where is a programmatically generated directory name that iOS uses to identify your app. Each time you build your app, Xcode preserves your app preferences and other relevant library files. If you want to remove the current preferences for testing purposes, you can delete the app from Simulator or choose Reset Contents and Settings from the iOS Simulator menu. Implementing an iOS Settings Bundle Debugging Preferences for Simulated Apps 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 34This table describes the changes to Preferences and Settings Programming Guide . Date Notes 2012-03-01 Updated the document to reflect new limits for key and value sizes. Updated the document to include information about Settings bundles and iOS in general. Also incorporated iCloud information. 2011-10-12 Removed the articles on storing NSColor objects and using Cocoa bindings and now link to their locations instead. Changed document name from User Defaults Programming Topics. 2007-10-31 Updated information about periodic autosave behavior. 2007-01-08 Corrected typos and capitalization mistakes. Added overview of procedure forstoring non-property-list objectsin user defaults, and linked to related article. 2006-11-07 2006-09-05 Made small additions to the content. Changed title from "User Defaults." Expanded explanation of user defaults in introduction. Noted requirement that a default’s value must be a property list value at the beginning of the “Using NSUserDefaults” article. Included an article that describes the use of NSUserDefaultsController. Corrected minor typographical errors. 2005-08-11 2004-02-03 Added article “Storing NSColor in User Defaults”. Linked to the Core Foundation Preferences Programming Topic, which was also incorrectly named. 2003-05-09 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 35 Document Revision HistoryDate Notes Added link in limitations area to CFPreferences. Corrected class name in Defaults Domains Concept. 2003-01-13 Revision history was added to existing topic. It will be used to record changes to the content of the topic. 2002-11-12 Document Revision History 2012-03-01 | © 2012 Apple Inc. All Rights Reserved. 36Apple Inc. © 2012 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, Cocoa, Finder, iPhone, Mac, OS X, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries. .Mac and iCloud are service marks of Apple Inc., registered in the U.S. and other countries. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. OpenGL Programming Guide for MacContents About OpenGL for OS X 11 At a Glance 11 OpenGL Is a C-based, Platform-Neutral API 12 Different Rendering Destinations Require Different Setup Commands 12 OpenGL on Macs Exists in a Heterogenous Environment 12 OpenGL Helps Applications Harness the Power of Graphics Processors 13 Concurrency in OpenGL Applications Requires Additional Effort 13 Performance Tuning Allows Your Application to Provide an Exceptional User Experience 14 How to Use This Document 14 Prerequisites 15 See Also 15 OpenGL on the Mac Platform 17 OpenGL Concepts 17 OpenGL Implements a Client-Server Model 18 OpenGL Commands Can Be Executed Asynchronously 18 OpenGL Commands Are Executed In Order 19 OpenGL Copies Client Data at Call-Time 19 OpenGL Relies on Platform-Specific Libraries For Critical Functionality 19 OpenGL in OS X 20 Accessing OpenGL Within Your Application 21 OpenGL APIs Specific to OS X 22 Apple-Implemented OpenGL Libraries 23 Terminology 24 Renderer 24 Renderer and Buffer Attributes 24 Pixel Format Objects 24 OpenGL Profiles 25 Rendering Contexts 25 Drawable Objects 25 Virtual Screens 26 Offline Renderer 31 Running an OpenGL Program in OS X 31 Making Great OpenGL Applications on the Macintosh 33 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 2Drawing to a Window or View 35 General Approach 35 Drawing to a Cocoa View 36 Drawing to an NSOpenGLView Class: A Tutorial 37 Drawing OpenGL Content to a Custom View 40 Optimizing OpenGL for High Resolution 44 Enable High-Resolution Backing for an OpenGL View 44 Set Up the Viewport to Support High Resolution 45 Adjust Model and Texture Assets 46 Check for Calls Defined in Pixel Dimensions 46 Tune OpenGL Performance for High Resolution 47 Use a Layer-Backed View to Overlay Text on OpenGL Content 48 Use an Application Window for Fullscreen Operation 49 Convert the Coordinate Space When Hit Testing 49 Drawing to the Full Screen 50 Creating a Full-Screen Application 50 52 Drawing Offscreen 53 Rendering to a Framebuffer Object 53 Using a Framebuffer Object as a Texture 54 Using a Framebuffer Object as an Image 58 Rendering to a Pixel Buffer 60 Setting Up a Pixel Buffer for Offscreen Drawing 61 Using a Pixel Buffer as a Texture Source 61 Rendering to a Pixel Buffer on a Remote System 63 Choosing Renderer and Buffer Attributes 64 OpenGL Profiles (OS X v10.7) 64 Buffer Size Attribute Selection Tips 65 Ensuring That Back Buffer Contents Remain the Same 66 Ensuring a Valid Pixel Format Object 66 Ensuring a Specific Type of Renderer 67 Ensuring a Single Renderer for a Display 68 Allowing Offline Renderers 69 OpenCL 70 Deprecated Attributes 70 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 3 ContentsWorking with Rendering Contexts 72 Update the Rendering Context When the Renderer or Geometry Changes 72 Tracking Renderer Changes 73 Updating a Rendering Context for a Custom Cocoa View 73 Context Parameters Alter the Context’s Behavior 76 Swap Interval Allows an Application to Synchronize Updates to the Screen Refresh 76 Surface Opacity Specifies How the OpenGL Surface Blends with Surfaces Behind It 77 Surface Drawing Order Specifies the Position of the OpenGL Surface Relative to the Window 77 Determining Whether Vertex and Fragment Processing Happens on the GPU 78 Controlling the Back Buffer Size 78 Sharing Rendering Context Resources 79 Determining the OpenGL Capabilities Supported by the Renderer 83 Detecting Functionality 83 Guidelines for Code That Checks for Functionality 87 OpenGL Renderer Implementation-Dependent Values 88 OpenGL Application Design Strategies 89 Visualizing OpenGL 89 Designing a High-Performance OpenGL Application 91 Update OpenGL Content Only When Your Data Changes 94 Synchronize with the Screen Refresh Rate 96 Avoid Synchronizing and Flushing Operations 96 Using glFlush Effectively 97 Avoid Querying OpenGL State 98 Use Fences for Finer-Grained Synchronization 98 Allow OpenGL to Manage Your Resources 99 Use Double Buffering to Avoid Resource Conflicts 100 Be Mindful of OpenGL State Variables 101 Replace State Changes with OpenGL Objects 102 Use Optimal Data Types and Formats 102 Use OpenGL Macros 103 Best Practices for Working with Vertex Data 104 Understand How Vertex Data Flows Through OpenGL 105 Techniques for Handling Vertex Data 107 Vertex Buffers 107 Using Vertex Buffers 108 Buffer Usage Hints 110 Flush Buffer Range Extension 113 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 4 ContentsVertex Array Range Extension 113 Vertex Array Object 116 Best Practices for Working with Texture Data 118 Using Extensions to Improve Texture Performance 119 Pixel Buffer Objects 121 Apple Client Storage 124 Apple Texture Range and Rectangle Texture 125 Combining Client Storage with Texture Ranges 127 Optimal Data Formats and Types 128 Working with Non–Power-of-Two Textures 129 Creating Textures from Image Data 131 Creating a Texture from a Cocoa View 131 Creating a Texture from a Quartz Image Source 133 Getting Decompressed Raw Pixel Data from a Source Image 135 Downloading Texture Data 136 Double Buffering Texture Data 137 Customizing the OpenGL Pipeline with Shaders 139 Shader Basics 141 Advanced Shading Extensions 142 Transform Feedback 142 GPU Shader 4 143 Geometry Shaders 143 Uniform Buffers 143 Techniques for Scene Antialiasing 144 Guidelines 145 General Approach 145 Hinting for a Specific Antialiasing Technique 147 Concurrency and OpenGL 148 Identifying Whether an OpenGL Application Can Benefit from Concurrency 149 OpenGL Restricts Each Context to a Single Thread 149 Strategies for Implementing Concurrency in OpenGL Applications 150 Multithreaded OpenGL 150 Perform OpenGL Computations in a Worker Task 151 Use Multiple OpenGL Contexts 153 Guidelines for Threading OpenGL Applications 154 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 5 ContentsTuning Your OpenGL Application 155 Gathering and Analyzing Baseline Performance Data 156 Using OpenGL Driver Monitor to Measure Stalls 161 Identifying Bottlenecks with Shark 161 Legacy OpenGL Functionality by Version 163 Version 1.1 163 Version 1.2 164 Version 1.3 165 Version 1.4 165 Version 1.5 166 Version 2.0 166 Version 2.1 167 Updating an Application to Support the OpenGL 3.2 Core Specification 168 Removed Functionality 168 Extension Changes on OS X 169 Setting Up Function Pointers to OpenGL Routines 171 Obtaining a Function Pointer to an Arbitrary OpenGL Entry Point 171 Initializing Entry Points 172 Document Revision History 175 Glossary 179 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 6 ContentsFigures, Tables, and Listings OpenGL on the Mac Platform 17 Figure 1-1 OpenGL provides the reflections in iChat 17 Figure 1-2 OpenGL client-server model 18 Figure 1-3 Graphics platform model 18 Figure 1-4 MacOS X OpenGL driver model 20 Figure 1-5 Layers of OpenGL for OS X 21 Figure 1-6 The programing interfaces used for OpenGL content 22 Figure 1-7 Data flow through OpenGL 26 Figure 1-8 A virtual screen displays what the user sees 27 Figure 1-9 Two virtual screens 28 Figure 1-10 A virtual screen can represent more than one physical screen 29 Figure 1-11 Two virtual screens and two graphics cards 30 Figure 1-12 The flow of data through OpenGL 31 Drawing to a Window or View 35 Figure 2-1 OpenGL content in a Cocoa view 35 Figure 2-2 The output from the Golden Triangle program 39 Listing 2-1 The interface for MyOpenGLView 37 Listing 2-2 Include OpenGL/gl.h 38 Listing 2-3 The drawRect: method for MyOpenGLView 38 Listing 2-4 Code that draws a triangle using OpenGL commands 38 Listing 2-5 The interface for a custom OpenGL view 40 Listing 2-6 The initWithFrame:pixelFormat: method 41 Listing 2-7 The lockFocus method 42 Listing 2-8 The drawRect method for a custom view 42 Listing 2-9 Detaching the context from a drawable object 43 Optimizing OpenGL for High Resolution 44 Figure 3-1 Enabling high-resolution backing for an OpenGL view 45 Figure 3-2 A text overlay scales automatically for standard resolution (left) and high resolution (right) 48 Listing 3-1 Setting up the viewport for drawing 45 Drawing to the Full Screen 50 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 7Figure 4-1 Drawing OpenGL content to the full screen 50 Drawing Offscreen 53 Listing 5-1 Setting up a framebuffer for texturing 57 Listing 5-2 Setting up a renderbuffer for drawing images 59 Choosing Renderer and Buffer Attributes 64 Table 6-1 Renderer types and pixel format attributes 67 Listing 6-1 Using the CGL API to create a pixel format object 66 Listing 6-2 Setting an NSOpenGLContext object to use a specific display 68 Listing 6-3 Setting a CGL context to use a specific display 69 Working with Rendering Contexts 72 Figure 7-1 A fixed size back buffer and variable size front buffer 79 Figure 7-2 Shared contexts attached to the same drawable object 80 Figure 7-3 Shared contexts and more than one drawable object 80 Listing 7-1 Handling context updates for a custom view 74 Listing 7-2 Using CGL to set up synchronization 76 Listing 7-3 Using CGL to set surface opacity 77 Listing 7-4 Using CGL to set surface drawing order 77 Listing 7-5 Using CGL to check whether the GPU is processing vertices and fragments 78 Listing 7-6 Using CGL to set up back buffer size control 79 Listing 7-7 Setting up an NSOpenGLContext object for sharing 81 Listing 7-8 Setting up a CGL context for sharing 82 Determining the OpenGL Capabilities Supported by the Renderer 83 Table 8-1 Common OpenGL renderer limitations 88 Table 8-2 OpenGL shader limitations 88 Listing 8-1 Checking for OpenGL functionality 84 Listing 8-2 Setting up a valid rendering context to get renderer functionality information 86 OpenGL Application Design Strategies 89 Figure 9-1 OpenGL graphics pipeline 90 Figure 9-2 OpenGL client-server architecture 91 Figure 9-3 Application model for managing resources 92 Figure 9-4 Single-buffered vertex array data 100 Figure 9-5 Double-buffered vertex array data 101 Listing 9-1 Setting up a Core Video display link 94 Listing 9-2 Setting up synchronization 96 Listing 9-3 Disabling state variables 102 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 8 Figures, Tables, and ListingsListing 9-4 Using CGL macros 103 Best Practices for Working with Vertex Data 104 Figure 10-1 Vertex data sets can be quite large 104 Figure 10-2 Vertex data path 105 Figure 10-3 Immediate mode requires a copy of the current vertex data 105 Listing 10-1 Submitting vertex data using glDrawElements. 106 Listing 10-2 Using the vertex buffer object extension with dynamic data 109 Listing 10-3 Using the vertex buffer object extension with static data 110 Listing 10-4 Geometry with different usage patterns 111 Listing 10-5 Using the vertex array range extension with dynamic data 115 Listing 10-6 Using the vertex array range extension with static data 116 Best Practices for Working with Texture Data 118 Figure 11-1 Textures add realism to a scene 118 Figure 11-2 Texture data path 119 Figure 11-3 Data copies in an OpenGL program 120 Figure 11-4 The client storage extension eliminates a data copy 124 Figure 11-5 The texture range extension eliminates a data copy 126 Figure 11-6 Combining extensions to eliminate data copies 127 Figure 11-7 Normalized and non-normalized coordinates 129 Figure 11-8 An image segmented into power-of-two tiles 130 Figure 11-9 Using an image as a texture for a cube 131 Figure 11-10 Single-buffered data 137 Figure 11-11 Double-buffered data 138 Listing 11-1 Using texture extensions for a rectangular texture 127 Listing 11-2 Using texture extensions for a power-of-two texture 128 Listing 11-3 Building an OpenGL texture from an NSView object 132 Listing 11-4 Using a Quartz image as a texture source 134 Listing 11-5 Getting pixel data from a source image 135 Listing 11-6 Code that downloads texture data 136 Customizing the OpenGL Pipeline with Shaders 139 Figure 12-1 OpenGL fixed-function pipeline 139 Figure 12-2 OpenGL shader pipeline 140 Listing 12-1 Loading a Shader 141 Techniques for Scene Antialiasing 144 Table 13-1 Antialiasing hints 147 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 9 Figures, Tables, and ListingsConcurrency and OpenGL 148 Figure 14-1 CPU processing and OpenGL on separate threads 152 Figure 14-2 Two contexts on separate threads 153 Listing 14-1 Enabling the multithreaded OpenGL engine 151 Tuning Your OpenGL Application 155 Figure 15-1 Output produced by the top application 157 Figure 15-2 The OpenGL Profiler window 158 Figure 15-3 A statistics window 159 Figure 15-4 A Trace window 160 Figure 15-5 The graph view in OpenGL Driver Monitor 161 Legacy OpenGL Functionality by Version 163 Table A-1 Functionality added in OpenGL 1.1 163 Table A-2 Functionality added in OpenGL 1.2 164 Table A-3 Functionality added in OpenGL 1.3 165 Table A-4 Functionality added in OpenGL 1.4 165 Table A-5 Functionality added in OpenGL 1.5 166 Table A-6 Functionality added in OpenGL 2.0 166 Table A-7 Functionality added in OpenGL 2.1 167 Updating an Application to Support the OpenGL 3.2 Core Specification 168 Table B-1 Extensions described in this guide 169 Setting Up Function Pointers to OpenGL Routines 171 Listing C-1 Using NSLookupAndBindSymbol to obtain a symbol for a symbol name 172 Listing C-2 Using NSGLGetProcAddress to obtain an OpenGL entry point 173 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 10 Figures, Tables, and ListingsOpenGL is an open, cross-platform graphics standard with broad industry support. OpenGL greatly eases the task of writing real-time 2D or 3D graphics applications by providing a mature, well-documented graphics processing pipeline that supports the abstraction of current and future hardware accelerators. OpenGL client OpenGL server Graphics hardware Application OpenGL framework OpenGL driver Runs on GPU Runs on CPU At a Glance OpenGL is an excellent choice for graphics development on the Macintosh platform because it offers the following advantages: ● Reliable Implementation. The OpenGL client-server model abstracts hardware details and guarantees consistent presentation on any compliant hardware and software configuration. Every implementation of OpenGL adheres to the OpenGL specification and must pass a set of conformance tests. ● Performance. Applications can harness the considerable power of the graphics hardware to improve rendering speeds and quality. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 11 About OpenGL for OS X● Industry acceptance. The specification for OpenGL is controlled by the Khronos Group, an industry consortium whose members include many of the major companies in the computer graphics industry, including Apple. In addition to OpenGL for OS X, there are OpenGL implementations for Windows, Linux, Irix, Solaris, and many game consoles. OpenGL Is a C-based, Platform-Neutral API Because OpenGL is a C-based API, it is extremely portable and widely supported. As a C API, it integrates seamlessly with Objective-C based Cocoa applications. OpenGL provides functions your application uses to generate 2D or 3D images. Your application presents the rendered images to the screen or copies them back to its own memory. The OpenGL specification does not provide a windowing layer of its own. It relies on functions defined by OS X to integrate OpenGL drawing with the windowing system. Your application creates an OS X OpenGL rendering context and attaches a rendering target to it (known as a drawable object). The rendering context manages OpenGL state changes and objects created by calls to the OpenGL API. The drawable object is the final destination for OpenGL drawing commands and is typically associated with a Cocoa window or view. Relevant Chapters: “OpenGL on the Mac Platform” (page 17) Different Rendering Destinations Require Different Setup Commands Depending on whether your application intends to draw OpenGL content to a window, to draw to the entire screen, or to perform offscreen image processing, it takes different steps to create the rendering context and associate it with a drawable object. Relevant Chapters: “Drawing to a Window or View” (page 35), “Drawing to the Full Screen” (page 50) and “Drawing Offscreen” (page 53) OpenGL on Macs Exists in a Heterogenous Environment Macs support different types of graphics processors, each with different rendering capabilities, supporting versions of OpenGL from 1.x through OpenGL 3.2. When creating a rendering context, your application can accept a broad range of renderers or it can restrict itself to devices with specific capabilities. Once you have a context, you can configure how that context executes OpenGL commands. About OpenGL for OS X At a Glance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 12OpenGL on the Mac is not only a heterogenous environment, but it is also a dynamic environment. Users can add or remove displays, or take a laptop running on battery power and plug it into a wall. When the graphics environment on the Mac changes, the renderer associated with the context may change. Your application must handle these changes and adjust how it uses OpenGL. Relevant Chapters: “Choosing Renderer and Buffer Attributes” (page 64), “Working with Rendering Contexts” (page 72), and “Determining the OpenGL Capabilities Supported by the Renderer” (page 83) OpenGL Helps Applications Harness the Power of Graphics Processors Graphics processors are massively parallelized devices optimized for graphics operations. To access that computing power adds additional overhead because data must move from your application to the GPU over slower internal buses. Accessing the same data simultaneously from both your application and OpenGL is usually restricted. To get great performance in your application, you must carefully design your application to feed data and commands to OpenGL so that the graphics hardware runs in parallel with your application. A poorly tuned application may stall either on the CPU or the GPU waiting for the other to finish processing. When you are ready to optimize your application’s performance, Apple provides both general-purpose and OpenGL-specific profiling tools that make it easy to learn where your application spends its time. Relevant Chapters: “Optimizing OpenGL for High Resolution” (page 44), “OpenGL on the Mac Platform” (page 17),“OpenGL Application Design Strategies” (page 89), “Best Practices for Working with Vertex Data” (page 104), “Best Practicesfor Working with Texture Data” (page 118), “Customizing the OpenGL Pipeline with Shaders” (page 139), and “Tuning Your OpenGL Application” (page 155) Concurrency in OpenGL Applications Requires Additional Effort Many Macs ship with multiple processors or multiple cores, and future hardware is expected to add more of each. Designing applications to take advantage of multiprocessing is critical. OpenGL places additional restrictions on multithreaded applications. If you intend to add concurrency to an OpenGL application, you must ensure that the application does not access the same context from two different threads at the same time. About OpenGL for OS X At a Glance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 13Relevant Chapters: “Concurrency and OpenGL” (page 148) Performance Tuning Allows Your Application to Provide an Exceptional User Experience Once you’ve improved the performance of your OpenGL application and taken advantage of concurrency, put some of the freed processing power to work for you. Higher resolution textures, detailed models, and more complex lighting and shading algorithms can improve image quality. Full-scene antialiasing on modern graphics hardware can eliminate many of the “jaggies” common on lower resolution images. Relevant Chapters: “Customizing the OpenGL Pipeline with Shaders” (page 139),“Techniques for Scene Antialiasing” (page 144) How to Use This Document If you have never programmed in OpenGL on the Mac, you should read this book in its entirety, starting with “OpenGL on the Mac Platform” (page 17). Critical Mac terminology is defined in that chapter as well as in the “Glossary” (page 179). If you already have an OpenGL application running on the Mac, but have not yet updated it for OS X v10.7, read “Choosing Renderer and Buffer Attributes” (page 64) to learn how to choose an OpenGL profile for your application. To find out how to update an existing OpenGL app for high resolution, see “Optimizing OpenGL for High Resolution” (page 44). Once you have OpenGL content in your application, read “OpenGL Application Design Strategies” (page 89) to learn fundamental patterns for implementing high-performance OpenGL applications, and the chapters that follow to learn how to apply those patterns to specific OpenGL problems. About OpenGL for OS X How to Use This Document 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 14Important: Although this guide describes how to create rendering contexts that support OpenGL 3.2, most code examples and discussion in the rest of the book describe the earlier legacy versions of OpenGL. See “Updating an Application to Support the OpenGL 3.2 Core Specification” (page 168) for more information on migrating your application to OpenGL 3.2. Prerequisites This guide assumes that you have some experience with OpenGL programming, but want to learn how to apply that knowledge to create software for the Mac. Although this guide provides advice on optimizing OpenGL code, it does not provide entry-level information on how to use the OpenGL API. If you are unfamiliar with OpenGL, you should read “OpenGL on the Mac Platform” (page 17) to get an overview of OpenGL on the Mac platform, and then read the following OpenGL programming guide and reference documents: ● OpenGL Programming Guide, by Dave Shreiner and the Khronos OpenGL Working Group; otherwise known as "The Red book.” ● OpenGL Shading Language , by Randi J. Rost, is an excellent guide for those who want to write programs that compute surface properties (also known as shaders). ● OpenGL Reference Pages. Before reading this document, you should be familiar with Cocoa windows and views asintroduced in Window Programming Guide and View Programming Guide . See Also Keep these reference documents handy as you develop your OpenGL program for OS X: ● NSOpenGLView Class Reference , NSOpenGLContext Class Reference , NSOpenGLPixelBuffer Class Reference , and NSOpenGLPixelFormat Class Reference provide a complete description of the classes and methods needed to integrate OpenGL content into a Cocoa application. ● CGL Reference describes low-level functions that can be used to create full-screen OpenGL applications. ● OpenGL Extensions Guide provides information about OpenGL extensions supported in OS X. The OpenGL Foundation website, http://www.opengl.org, provides information on OpenGL commands, the Khronos OpenGL Working Group, logo requirements, OpenGL news, and many other topics. It's a site that you'll want to visit regularly. Among the many resources it provides, the following are important reference documents for OpenGL developers: About OpenGL for OS X Prerequisites 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 15● OpenGL Specification provides detailed information on how an OpenGL implementation is expected to handle each OpenGL command. ● OpenGL Reference describes the main OpenGL library. ● OpenGL GLU Reference describes the OpenGL Utility Library, which contains convenience functions implemented on top of the OpenGL API. ● OpenGL GLUT Reference describes the OpenGL Utility Toolkit, a cross-platform windowing API. ● OpenGL API Code and Tutorial Listings provides code examples for fundamental tasks, such as modeling and texture mapping, as well as for advanced techniques, such as high dynamic range rendering (HDRR). About OpenGL for OS X See Also 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 16You can tell that Apple has an implementation of OpenGL on its platform by looking at the user interface for many of the applications that are installed with OS X. The reflections built into iChat (Figure 1-1) provide one of the more notable examples. The responsiveness of the windows, the instant results of applying an effect in iPhoto, and many other operations in OS X are due to the use of OpenGL. OpenGL is available to all Macintosh applications. OpenGL for OS X is implemented as a set of frameworks that contain the OpenGL runtime engine and its drawing software. These frameworks use platform-neutral virtual resourcesto free your programming as much as possible from the underlying graphics hardware. OS X provides a set of application programming interfaces (APIs) that Cocoa applications can use to support OpenGL drawing. Figure 1-1 OpenGL provides the reflections in iChat This chapter provides an overview of OpenGL and the interfaces your application uses on the Mac platform to tap into it. OpenGL Concepts To understand how OpenGL fits into OS X and your application, you should first understand how OpenGL is designed. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 17 OpenGL on the Mac PlatformOpenGL Implements a Client-Server Model OpenGL uses a client-server model, as shown in Figure 1-2. When your application calls an OpenGL function, it talks to an OpenGL client. The client delivers drawing commands to an OpenGL server. The nature of the client, the server, and the communication path between them is specific to each implementation of OpenGL. For example, the server and clients could be on different computers, or they could be different processes on the same computer. Figure 1-2 OpenGL client-server model Application OpenGL client OpenGL server A client-server model allows the graphics workload to be divided between the client and the server. For example, all Macintosh computersship with dedicated graphics hardware that is optimized to perform graphics calculations in parallel. Figure 1-3 shows a common arrangement of CPUs and GPUs. With this hardware configuration, the OpenGL client executes on the CPU and the server executes on the GPU. Figure 1-3 Graphics platform model CPU RAM Core Core GPU RAM Core Core Core Core Core Core System OpenGL Commands Can Be Executed Asynchronously A benefit of the OpenGL client-server model is that the client can return control to the application before the command has finished executing. An OpenGL client may also buffer or delay execution of OpenGL commands. If OpenGL required all commands to complete before returning control to the application, then either the CPU or the GPU would be idle waiting for the other to provide it data, resulting in reduced performance. OpenGL on the Mac Platform OpenGL Concepts 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 18Some OpenGL commandsimplicitly or explicitly require the client to wait untilsome or all previously submitted commands have completed. OpenGL applicationsshould be designed to reduce the frequency of client-server synchronizations. See “OpenGL Application Design Strategies” (page 89) for more information on how to design your OpenGL application. OpenGL Commands Are Executed In Order OpenGL guarantees that commands are executed in the order they are received by OpenGL. OpenGL Copies Client Data at Call-Time When an application calls an OpenGL function, the OpenGL client copies any data provided in the parameters before returning control to the application. For example, if a parameter points at an array of vertex data stored in application memory, OpenGL must copy that data before returning. Therefore, an application is free to change memory it owns regardless of calls it makes to OpenGL. The data that the client copies is often reformatted before it is transmitted to the server. Copying, modifying, and transmitting parameters to the server adds overhead to calling OpenGL. Applications should be designed to minimize copy overhead. OpenGL Relies on Platform-Specific Libraries For Critical Functionality OpenGL provides a rich set of cross-platform drawing commands, but does not define functions to interact with an operating system’s graphics subsystem. Instead, OpenGL expects each implementation to define an interface to create rendering contexts and associate them with the graphics subsystem. A rendering context holds all of the data stored in the OpenGL state machine. Allowing multiple contexts allows the state in one machine to be changed by an application without affecting other contexts. Associating OpenGL with the graphic subsystem usually means allowing OpenGL content to be rendered to a specific window. When content is associated with a window, the implementation creates whatever resources are required to allow OpenGL to render and display images. OpenGL on the Mac Platform OpenGL Concepts 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 19OpenGL in OS X OpenGL in OS X implementsthe OpenGL client-server model using a common OpenGL framework and plug-in drivers. The framework and driver combine to implement the client portion of OpenGL, as shown in Figure 1-4. Dedicated graphics hardware provides the server. Although this is the common scenario, Apple also provides a software renderer implemented entirely on the CPU. Figure 1-4 MacOS X OpenGL driver model OpenGL client OpenGL server Graphics hardware Application OpenGL framework OpenGL driver Runs on GPU Runs on CPU OS X supports a display space that can include multiple dissimilar displays, each driven by different graphics cards with different capabilities. In addition, multiple OpenGL renderers can drive each graphics card. To accommodate this versatility, OpenGL for OS X is segmented into well-defined layers: a window system layer, OpenGL on the Mac Platform OpenGL in OS X 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 20a framework layer, and a driver layer, as shown in Figure 1-5. This segmentation allows for plug-in interfaces to both the window system layer and the framework layer. Plug-in interfaces offer flexibility in software and hardware configuration without violating the OpenGL standard. Figure 1-5 Layers of OpenGL for OS X Software GLD plug-in ATI GLD plug-in NVIDIA GLD plug-in Intel GLD plug-in Application Hardware Window system layer Common OpenGL framework Driver layer NSOpenGL CGL OpenGL The window system layer is an OS X–specific layer that your application uses to create OpenGL rendering contexts and associate them with the OS X windowing system. The NSOpenGL classes and Core OpenGL (CGL) API also provide some additional controlsfor how OpenGL operates on that context. See “OpenGL APIs Specific to OS X” (page 22) for more information. Finally, this layer also includes the OpenGL libraries—GL, GLU, and GLUT. (See “Apple-Implemented OpenGL Libraries” (page 23) for details.) The common OpenGL framework layer is the software interface to the graphics hardware. This layer contains Apple's implementation of the OpenGL specification. The driver layer contains the optional GLD plug-in interface and one or more GLD plug-in drivers, which may have different software and hardware support capabilities. The GLD plug-in interface supports third-party plug-in drivers, allowing third-party hardware vendors to provide drivers optimized to take best advantage of their graphics hardware. Accessing OpenGL Within Your Application The programming interfacesthat your application callsfall into two categories—those specific to the Macintosh platform and those defined by the OpenGL Working Group. The Apple-specific programming interfaces are what Cocoa applications use to communicate with the OS X windowing system. These APIs don't create OpenGL content, they manage content, direct it to a drawing destination, and control various aspects of the rendering OpenGL on the Mac Platform Accessing OpenGL Within Your Application 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 21operation. Your application calls the OpenGL APIs to create content. OpenGL routines accept vertex, pixel, and texture data and assemble the data to create an image. The final image resides in a framebuffer, which is presented to the user through the windowing-system specific API. Figure 1-6 The programing interfaces used for OpenGL content OpenGL engine and drivers GLUT CGL OpenGL NSOpenGL classes GLUT application Cocoa application OpenGL APIs Specific to OS X OS X offers two easy-to-use APIs that are specific to the Macintosh platform: the NSOpenGL classes and the CGL API. Throughout this document, these APIs are referred to as the Apple-specific OpenGL APIs. Cocoa provides many classes specifically for OpenGL: ● The NSOpenGLContext class implements a standard OpenGL rendering context. ● The NSOpenGLPixelFormat class is used by an application to specify the parameters used to create the OpenGL context. ● The NSOpenGLView class is a subclass of NSView that uses NSOpenGLContext and NSOpenGLPixelFormat to display OpenGL content in a view. Applicationsthatsubclass NSOpenGLView do not need to directly subclass NSOpenGLPixelFormat or NSOpenGLContext. Applications that need customization or flexibility, can subclass NSView and create NSOpenGLPixelFormat and NSOpenGLContext objects manually. ● The NSOpenGLLayer class allows your application to integrate OpenGL drawing with Core Animation. ● The NSOpenGLPixelBuffer class provides hardware-accelerated offscreen drawing. OpenGL on the Mac Platform Accessing OpenGL Within Your Application 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 22The Core OpenGL API (CGL) residesin the OpenGL framework and is used to implement the NSOpenGL classes. CGL offersthe most direct accessto system functionality and providesthe highest level of graphics performance and control for drawing to the full screen. CGL Reference provides a complete description of this API. Apple-Implemented OpenGL Libraries OS X also provides the full suite of graphics libraries that are part of every implementation of OpenGL: GL, GLU, GLUT, and GLX. Two of these—GL and GLU—provide low-level drawing support. The other two—GLUT and GLX—support drawing to the screen. Your application typically interfaces directly with the core OpenGL library (GL), the OpenGL Utility library (GLU), and the OpenGL Utility Toolkit (GLUT). The GL library provides a low-level modular API that allows you to define graphical objects. Itsupportsthe core functions defined by the OpenGL specification. It providessupport for two fundamental types of graphics primitives: objects defined by sets of vertices, such as line segments and simple polygons, and objects that are pixel-based images, such as filled rectangles and bitmaps. The GL API does not handle complex custom graphical objects; your application must decompose them into simpler geometries. The GLU library combines functions from the GL library to support more advanced graphics features. It runs on all conforming implementations of OpenGL. GLU is capable of creating and handling complex polygons (including quartic equations), processing nonuniform rational b-spline curves (NURBs), scaling images, and decomposing a surface to a series of polygons (tessellation). The GLUT library provides a cross-platform API for performing operations associated with the user windowing environment—displaying and redrawing content, handling events, and so on. It isimplemented on most UNIX, Linux, and Windows platforms. Code that you write with GLUT can be reused across multiple platforms. However, such code is constrained by a generic set of user interface elements and event-handling options. This document does not show how to use GLUT. The GLUTBasics sample project shows you how to get started with GLUT. GLX is an OpenGL extension that supports using OpenGL within a window provided by the X Window system. X11 for OS X is available as an optional installation. (It's not shown in Figure 1-6 (page 22).) See OpenGL Programming for the X Window System, published by Addison Wesley for more information. This document does not show how to use these libraries. For detailed information, either go to the OpenGL Foundation website http://www.opengl.org or see the most recent version of "The Red book"—OpenGL Programming Guide, published by Addison Wesley. OpenGL on the Mac Platform Accessing OpenGL Within Your Application 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 23Terminology There are a number of termsthat you’ll want to understand so that you can write code effectively using OpenGL: renderer, renderer attributes, buffer attributes, pixel format objects, rendering contexts, drawable objects, and virtual screens. As an OpenGL programmer, some of these may seem familiar to you. However, understanding the Apple-specific nuances of these terms will help you get the most out of OpenGL on the Macintosh platform. Renderer A renderer isthe combination of the hardware and software that OpenGL usesto execute OpenGL commands. The characteristics of the final image depend on the capabilities of the graphics hardware associated with the renderer and the device used to display the image. OS X supports graphics accelerator cards with varying capabilities, as well as a software renderer. It is possible for multiple renderers, each with different capabilities or features, to drive a single set of graphics hardware. To learn how to determine the exact features of a renderer, see “Determining the OpenGL Capabilities Supported by the Renderer” (page 83). Renderer and Buffer Attributes Your application uses renderer and buffer attributes to communicate renderer and buffer requirements to OpenGL. The Apple implementation of OpenGL dynamically selectsthe best renderer for the current rendering task and doesso transparently to your application. If your application has very specific rendering requirements and wants to control renderer selection, it can do so by supplying the appropriate renderer attributes. Buffer attributes describe such things as color and depth buffer sizes, and whether the data is stereoscopic or monoscopic. Renderer and buffer attributes are represented by constants defined in the Apple-specific OpenGL APIs. OpenGL uses the attributes you supply to perform the setup work needed prior to drawing content. “Drawing to a Window or View” (page 35) provides a simple example that shows how to use renderer and buffer attributes. “Choosing Renderer and Buffer Attributes” (page 64) explains how to choose renderer and buffer attributes to achieve specific rendering goals. Pixel Format Objects A pixel format describes the format for pixel data storage in memory. The description includes the number and order of components as well as their names (typically red, blue, green and alpha). It also includes other information, such as whether a pixel contains stencil and depth values. A pixel format object is an opaque data structure that holds a pixel format along with a list of renderers and display devices that satisfy the requirements specified by an application. OpenGL on the Mac Platform Terminology 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 24Each of the Apple-specific OpenGL APIs defines a pixel format data type and accessor routines that you can use to obtain the information referenced by this object. See “Virtual Screens” (page 26) for more information on renderer and display devices. OpenGL Profiles OpenGL profiles are new in OS X 10.7. An OpenGL profile is a renderer attribute used to request a specific version of the OpenGL specification. When your application provides an OpenGL profile as part of its renderer attributes, it only receives renderers that provide the complete feature set promised by that profile. The render can implement a different version of the OpenGL so long asthe version itsuppliesto your application provides the same functionality that your application requested. Rendering Contexts A rendering context, or simply context, contains OpenGL state information and objects for your application. State variables include such things as drawing color, the viewing and projection transformations, lighting characteristics, and material properties. State variables are set per context. When your application creates OpenGL objects (for example, textures), these are also associated with the rendering context. Although your application can maintain more than one context, only one context can be the current context in a thread. The current context is the rendering context that receives OpenGL commands issued by your application. Drawable Objects A drawable object refers to an object allocated by the windowing system that can serve as an OpenGL framebuffer. A drawable object is the destination for OpenGL drawing operations. The behavior of drawable objects is not part of the OpenGL specification, but is defined by the OS X windowing system. A drawable object can be any of the following: a Cocoa view, offscreen memory, a full-screen graphics device, or a pixel buffer. OpenGL on the Mac Platform Terminology 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 25Note: A pixel buffer (pbuffer) is an OpenGL buffer designed for hardware-accelerated offscreen drawing and as a source for texturing. An application can render an image into a pixel buffer and then use the pixel buffer as a texture for other OpenGL commands. Although pixel buffers are supported on Apple’s implementation of OpenGL, Apple recommends you use framebuffer objects instead. See “Drawing Offscreen” (page 53) for more information on offscreen rendering. Before OpenGL can draw to a drawable object, the object must be attached to a rendering context. The characteristics of the drawable object narrow the selection of hardware and software specified by the rendering context. Apple’s OpenGL automatically allocates buffers, creates surfaces, and specifies which renderer is the current renderer. The logical flow of data from an application through OpenGL to a drawable object is shown in Figure 1-7. The application issues OpenGL commands that are sent to the current rendering context. The current context, which contains state information, constrains how the commands are interpreted by the appropriate renderer. The renderer converts the OpenGL primitives to an image in the framebuffer. (See also “Running an OpenGL Program in OS X ” (page 31).) Figure 1-7 Data flow through OpenGL Rendered Image Application Possible renderers OpenGL buffers Current Drawable objects CONTEXT Virtual Screens The characteristics and quality of the OpenGL content that the user sees depend on both the renderer and the physical display used to view the content. The combination of renderer and physical display is called a virtual screen. This important concept has implications for any OpenGL application running on OS X. A simple system, with one graphics card and one physical display, typically has two virtual screens. One virtual screen consists of a hardware-based renderer and the physical display and the other virtual screen consists of a software-based renderer and the physical display. OS X provides a software-based renderer as a fallback. It's possible for your application to decline the use of thisfallback. You'llsee how in “Choosing Renderer and Buffer Attributes” (page 64). OpenGL on the Mac Platform Terminology 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 26The green rectangle around the OpenGL image in Figure 1-8 surrounds a virtual screen for a system with one graphics card and one display. Note that a virtual screen is not the physical display, which is why the green rectangle is drawn around the application window thatshowsthe OpenGL content. In this case, it isthe renderer provided by the graphics card combined with the characteristics of the display. Figure 1-8 A virtual screen displays what the user sees Graphics card Virtual screen OpenGL on the Mac Platform Terminology 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 27Because a virtual screen is not simply the physical display, a system with one display can use more than one virtualscreen at a time, asshown in Figure 1-9. The green rectangles are drawn to point out each virtualscreen. Imagine that the virtual screen on the right side uses a software-only renderer and that the one on the left uses a hardware-dependent renderer. Although this is a contrived example, it illustrates the point. Figure 1-9 Two virtual screens Graphics card Virtual screen 2 (Software renderer) Virtual screen 1 (Hardware renderer) OpenGL on the Mac Platform Terminology 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 28It's also possible to have a virtualscreen that can represent more than one physical display. The green rectangle in Figure 1-10 is drawn around a virtual screen that spans two physical displays. In this case, the same graphics hardware drives a pair of identical displays. A mirrored display also has a single virtual screen associated with multiple physical displays. Figure 1-10 A virtual screen can represent more than one physical screen Dual-headed graphics card Identical displays Virtual screen The concept of a virtualscreen is particularly important when the user drags an image from one physicalscreen to another. When this happens, the virtual screen may change, and with it, a number of attributes of the imaging process, such as the current renderer, may change. With the dual-headed graphics card shown in Figure 1-10 (page 29), dragging between displays preserves the same virtual screen. However, Figure 1-11 shows the case for which two displays represent two unique virtual screens. Not only are the two graphics cards different, but it's possible that the renderer, buffer attributes, and pixel characteristics are different. A change in any of these three items can result in a change in the virtual screen. OpenGL on the Mac Platform Terminology 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 29When the user drags an image from one display to another, and the virtualscreen isthe same for both displays, the image quality should appear similar. However, for the case shown in Figure 1-11, the image quality can be quite different. Figure 1-11 Two virtual screens and two graphics cards Graphics card 1 Graphics card 2 Virtual screen 1 Virtual screen 2 OpenGL for OS X transparently manages rendering across multiple monitors. A user can drag a window from one monitor to another, even though their display capabilities may be different or they may be driven by dissimilar graphics cards with dissimilar resolutions and color depths. OpenGL dynamically switches renderers when the virtual screen that contains the majority of the pixels in an OpenGL window changes. When a window issplit between multiple virtualscreens, the framebuffer israsterized entirely by the renderer driving the screen that contains the largest segment of the window. The regions of the window on the other virtual screens are drawn by copying the rasterized image. When the entire OpenGL drawable object is displayed on one virtual screen, there is no performance impact from multiple monitor support. Applications need to track virtual screen changes and, if appropriate, update the current application state to reflect changes in renderer capabilities. See “Working with Rendering Contexts” (page 72). OpenGL on the Mac Platform Terminology 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 30Offline Renderer An offline renderer is one that is not currently associated with a display. For example, a graphics processor might be powered down to conserve power, or there might not be a display hooked up to the graphics card. Offline renderers are not normally visible to your application, but your application can enable them by adding the appropriate renderer attribute. Taking advantage of offline renderers is useful because it gives the user a seamless experience when they plug in or remove displays. For more information about configuring a context to see offline renderers, see “Choosing Renderer and Buffer Attributes” (page 64). To enable your application to switch to a renderer when a display is attached,see “Update the Rendering Context When the Renderer or Geometry Changes” (page 72). Running an OpenGL Program in OS X Figure 1-12 shows the flow of data in an OpenGL program, regardless of the platform that the program runs on. Figure 1-12 The flow of data through OpenGL Rasterization Fragment shading and per-fragment operations Per-pixel operations Texture assembly Framebuffer Vertex shading and per-vertex operations Pixel data Vertex data Per-vertex operations include such things as applying transformation matrices to add perspective or to clip, and applying lighting effects. Per-pixel operations include such things as color conversion and applying blur and distortion effects. Pixels destined for textures are sent to texture assembly, where OpenGL stores textures until it needs to apply them onto an object. OpenGL rasterizesthe processed vertex and pixel data, meaning that the data are converged to create fragments. A fragment encapsulates all the values for a pixel, including color, depth, and sometimes texture values. These values are used during antialiasing and any other calculations needed to fill shapes and to connect vertices. OpenGL on the Mac Platform Running an OpenGL Program in OS X 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 31Per-fragment operations include applying environment effects, depth and stencil testing, and performing other operations such as blending and dithering. Some operations—such as hidden-surface removal—end the processing of a fragment. OpenGL draws fully processed fragments into the appropriate location in the framebuffer. The dashed arrows in Figure 1-12 indicate reading pixel data back from the framebuffer. They represent operations performed byOpenGL functionssuch as glReadPixels, glCopyPixels, and glCopyTexImage2D. So far you've seen how OpenGL operates on any platform. But how do Cocoa applications provide data to the OpenGL for processing? A Mac application must perform these tasks: ● Set up a list of buffer and renderer attributes that define the sort of drawing you want to perform. (See “Renderer and Buffer Attributes” (page 24).) ● Request the system to create a pixel format object that contains a pixel format that meets the constraints of the buffer and render attributes and a list of all suitable combinations of displays and renderers. (See “Pixel Format Objects” (page 24) and “Virtual Screens” (page 26).) ● Create a rendering context to hold state information that controls such things as drawing color, view and projection matrices, characteristics of light, and conventions used to pack pixels. When you set up this context, you must provide a pixel format object because the rendering context needs to know the set of virtual screens that can be used for drawing. (See “Rendering Contexts” (page 25).) ● Bind a drawable object to the rendering context. The drawable object is what capturesthe OpenGL drawing sent to that rendering context. (See “Drawable Objects” (page 25).) ● Make the rendering context the current context. OpenGL automatically targets the current context. Although your application might have several rendering contexts set up, only the current one is the active one for drawing purposes. ● Issue OpenGL drawing commands. ● Flush the contents of the rendering context. This causes previously submitted commands to be rendered to the drawable object and displays them to the user. The tasks described in the first five bullet items are platform-specific. “Drawing to a Window or View” (page 35) provides simple examples of how to perform them. As you read other parts of this document, you'll see there are a number of other tasks that, although not mandatory for drawing, are really quite necessary for any application that wantsto use OpenGL to perform complex 3D drawing efficiently on a wide variety of Macintosh systems. OpenGL on the Mac Platform Running an OpenGL Program in OS X 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 32Making Great OpenGL Applications on the Macintosh OpenGL lets you create applications with outstanding graphics performance as well as a great user experience—but neither of these things come for free. Your application performs best when it works with OpenGL rather than against it. With that in mind, here are guidelines you should follow to create high-performance, future-looking OpenGL applications: ● Ensure your application runs successfully with offline renderers and multiple graphics cards. Apple ships many sophisticated hardware configurations. Your application should handle renderer changes seamlessly. You should test your application on a Mac with multiple graphics processors and include tests for attaching and removing displays. For more information on how to implement hot plugging correctly, see “Working with Rendering Contexts” (page 72) ● Avoid finishing and flushing operations. Pay particular attention to OpenGL functions that force previously submitted commands to complete. Synchronizing the graphics hardware to the CPU may result in dramatically lower performance. Performance is covered in detail in “OpenGL Application Design Strategies” (page 89). ● Use multithreading to improve the performance of your OpenGL application. Many Macs support multiple simultaneous threads of execution. Your application should take advantage of concurrency. Well-behaved applications can take advantage of concurrency in just a few line of code. See “Concurrency and OpenGL” (page 148). ● Use buffer objects to manage your data. Vertex buffer objects (VBOs) allow OpenGL to manage your application’s vertex data. Using vertex buffer objects gives OpenGL more opportunities to cache vertex data in a format that is friendly to the graphics hardware, improving application performance. For more information see “Best Practices for Working with Vertex Data” (page 104). Similarly, pixel buffer objects (PBOs) should be used to manage your image data. See “Best Practices for Working with Texture Data” (page 118) ● Use framebuffer objects (FBOs) when you need to render to offscreen memory. Framebuffer objects allow your application to create offscreen rendering targets without many of the limitations of platform-dependent interfaces. See “Rendering to a Framebuffer Object” (page 53). ● Generate objects before binding them. Earlier version of OpenGL allowed your applications to create its own object names before binding them. However, you should avoid this. Always use the OpenGL API to generate object names. ● Migrate your OpenGL Applications to OpenGL 3.2 OpenGL on the Mac Platform Making Great OpenGL Applications on the Macintosh 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 33The OpenGL 3.2 Core profile provides a clean break from earlier versions of OpenGL in favor of a simpler shader-based pipeline. For better compatibility with future hardware and OS X releases, migrate your applications away from legacy versions of OpenGL. Many of the recommendationslisted above are required when your application uses OpenGL 3.2. ● Harness the power of Apple’s development tools. Apple provides many toolsthat help create OpenGL applications and analyze and tune their performance. Learning how to use these tools helps you create fast, reliable applications. “Tuning Your OpenGL Application” (page 155) describes many of these tools. OpenGL on the Mac Platform Making Great OpenGL Applications on the Macintosh 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 34The OpenGL programming interface provides hundreds of drawing commands that drive graphics hardware. It doesn't provide any commands that interface with the windowing system of an operating system. Without a windowing system, the 3D graphics of an OpenGL program are trapped inside the GPU. Figure 2-1 shows a cube drawn to a Cocoa view. Figure 2-1 OpenGL content in a Cocoa view This chapter shows how to display OpenGL drawing onscreen using the APIs provided by OS X. (This chapter does not show how to use GLUT.) The first section describes the overall approach to drawing onscreen and provides an overview of the functions and methods used by each API. General Approach To draw your content to a view or a layer, your application uses the NSOpenGL classes from within the Cocoa application framework. While the CGL API is used by your applications only to create full-screen content, every NSOpenGLContext object contains a CGL context object. This object can be retrieved from the NSOpenGLContext when your application needs to reference it directly. To show the similarities between the two, this chapter discusses both the NSOpenGL classes and the CGL API. To draw OpenGL content to a window or view using the NSOpenGL classes, you need to perform these tasks: 1. Set up the renderer and buffer attributes that support the OpenGL drawing you want to perform. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 35 Drawing to a Window or ViewEach of the OpenGL APIs in OS X has its own set of constants that represent renderer and buffer attributes. For example, the all-renderers attribute is represented by the NSOpenGLPFAAllRenderers constant in Cocoa and the kCGLPFAAllRenderers constant in the CGL API. 2. Request, from the operating system, a pixel format object that encapsulates pixel storage information and the renderer and buffer attributes required by your application. The returned pixel format object contains all possible combinations of renderers and displays available on the system that your program runs on and that meets the requirements specified by the attributes. The combinations are referred to as virtual screens. (See “Virtual Screens” (page 26).) There may be situationsfor which you want to ensure that your program uses a specific renderer. “Choosing Renderer and Buffer Attributes” (page 64) discusses how to set up an attributes array that guarantees the system passes back a pixel format object that uses only that renderer. If an error occurs, your application may receive a NULL pixel format object. Your application must handle this condition. 3. Create a rendering context and bind the pixel format object to it. The rendering context keeps track of state information that controls such things as drawing color, view and projection matrices, characteristics of light, and conventions used to pack pixels. Your application needs a pixel format object to create a rendering context. 4. Release the pixel format object. Once the pixel format object is bound to a rendering context, itsresources are no longer needed. 5. Bind a drawable object to the rendering context. For a windowed context, this is typically a Cocoa view. 6. Make the rendering context the current context. The system sends OpenGL drawing to whichever rendering context is designated as the current one. It's possible for you to set up more than one rendering context, so you need to make sure that the one you want to draw to is the current one. 7. Perform your drawing. The specific functions or methods that you use to perform each of the steps are discussed in the sections that follow. Drawing to a Cocoa View There are two ways to draw OpenGL content to a Cocoa view. If your application has modest drawing requirements, then you can use the NSOpenGLView class. See “Drawing to an NSOpenGLView Class: A Tutorial” (page 37). Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 36If your application is more complex and needs to support drawing to multiple rendering contexts, you may want to consider subclassing the NSView class. For example, if your application supports drawing to multiple views at the same time, you need to set up a custom NSView class. See “Drawing OpenGL Content to a Custom View” (page 40). Drawing to an NSOpenGLView Class: A Tutorial The NSOpenGLView class is a lightweight subclass of the NSView class that provides convenience methods for setting up OpenGL drawing. An NSOpenGLView object maintains an NSOpenGLPixelFormat object and an NSOpenGLContext object into which OpenGL calls can be rendered. It provides methods for accessing and managing the pixel format object and the rendering context, and handles notification of visible region changes. An NSOpenGLView object does notsupportsubviews. You can, however, divide the view into multiple rendering areas using the OpenGL function glViewport. This section provides step-by-step instructions for creating a simple Cocoa application that draws OpenGL content to a view. The tutorial assumes that you know how to use Xcode and Interface Builder. If you have never created an application using the Xcode development environment, see Getting Started with Tools. 1. Create a Cocoa application project named Golden Triangle. 2. Add the OpenGL framework to your project. 3. Add a new file to your project using the Objective-C class template. Name the file MyOpenGLView.m and create a header file for it. 4. Open the MyOpenGLView.h file and modify the file so that it looks like the code shown in Listing 2-1 to declare the interface. Listing 2-1 The interface for MyOpenGLView #import @interface MyOpenGLView : NSOpenGLView { } - (void) drawRect: (NSRect) bounds; @end 5. Save and close the MyOpenGLView.h file. 6. Open the MyOpenGLView.m file and include the gl.h file, as shown in Listing 2-2. Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 37Listing 2-2 Include OpenGL/gl.h #import "MyOpenGLView.h" #include @implementation MyOpenGLView @end 7. Implement the drawRect: method asshown in Listing 2-3, adding the code after the @implementation statement. The method sets the clear color to black and clears the color buffer in preparation for drawing. Then, drawRect: calls your drawing routine, which you’ll add next. The OpenGL command glFlush draws the content provided by your routine to the view. Listing 2-3 The drawRect: method for MyOpenGLView -(void) drawRect: (NSRect) bounds { glClearColor(0, 0, 0, 0); glClear(GL_COLOR_BUFFER_BIT); drawAnObject(); glFlush(); } 8. Add the code to perform your drawing. In your own application, you'd perform whatever drawing is appropriate. But for the purpose of learning how to draw OpenGL content to a view, add the code shown in Listing 2-4. This code draws a 2D, gold-colored triangle, whose dimensions are not quite the dimensions of a true golden triangle, but good enough to show how to perform OpenGL drawing. Make sure that you insert this routine before the drawRect: method in the MyOpenGLView.m file. Listing 2-4 Code that draws a triangle using OpenGL commands static void drawAnObject () { glColor3f(1.0f, 0.85f, 0.35f); glBegin(GL_TRIANGLES); { Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 38glVertex3f( 0.0, 0.6, 0.0); glVertex3f( -0.2, -0.3, 0.0); glVertex3f( 0.2, -0.3 ,0.0); } glEnd(); } 9. Open the MainMenu.xib in Interface Builder. 10. Change the window’s title to Golden Triangle. 11. Drag an NSOpenGLView object from the Library to the window. Resize the view to fit the window. 12. Change the class of this object to MyOpenGLView. 13. Open the Attributes pane of the inspector for the view, and take a look at the renderer and buffer attributes that are available to set. These settings save you from setting attributes programmatically. Only those attributes listed in the Interface Builder inspector are set when the view is instantiated. If you need additional attributes, you need to set them programmatically. 14. Build and run your application. You should see content similar to the triangle shown in Figure 2-2. Figure 2-2 The output from the Golden Triangle program This example is extremely simple. In a more complex application, you'd want to do the following: ● Replace the immediate-mode drawing commands with commands that persist your vertex data inside OpenGL. See “OpenGL Application Design Strategies” (page 89). Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 39● In the interface for the view, declare a variable that indicates whether the view is ready to accept drawing. A view is ready for drawing only if it is bound to a rendering context and that context is set to be the current one. ● Cocoa does not call initialization routines for objects created in Interface Builder. If you need to perform any initialization tasks, do so in the awakeFromNib method for the view. Note that because you set attributes in the inspector, there is no need to set them up programmatically unless you need additional ones. There is also no need to create a pixel format object programmatically; it is created and loaded when Cocoa loads the nib file. ● Your drawRect: method should test whether the view is ready to draw into. You need to provide code that handles the case when the view is not ready to draw into. ● OpenGL is at its best when doing real-time and interactive graphics. Your application needs to provide a timer or support user interaction. For more information about creating animation in your OpenGL application, see “Synchronize with the Screen Refresh Rate” (page 96). Drawing OpenGL Content to a Custom View This section provides an overview of the key tasks you need to perform to customize the NSView class for OpenGL drawing. Before you create a custom view for OpenGL drawing, you should read “Creating a Custom View” in View Programming Guide . When you subclass the NSView class to create a custom view for OpenGL drawing, you override any Quartz drawing or other content that is in that view. To set up a custom view for OpenGL drawing, subclass NSView and create two private variables—one which is an NSOpenGLContext object and the other an NSOpenGLPixelFormat object, as shown in Listing 2-5. Listing 2-5 The interface for a custom OpenGL view @class NSOpenGLContext, NSOpenGLPixelFormat; @interface CustomOpenGLView : NSView { @private NSOpenGLContext* _openGLContext; NSOpenGLPixelFormat* _pixelFormat; } + (NSOpenGLPixelFormat*)defaultPixelFormat; - (id)initWithFrame:(NSRect)frameRect pixelFormat:(NSOpenGLPixelFormat*)format; - (void)setOpenGLContext:(NSOpenGLContext*)context; Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 40- (NSOpenGLContext*)openGLContext; - (void)clearGLContext; - (void)prepareOpenGL; - (void)update; - (void)setPixelFormat:(NSOpenGLPixelFormat*)pixelFormat; - (NSOpenGLPixelFormat*)pixelFormat; @end In addition to the usual methods for the private variables (openGLContext, setOpenGLContext:, pixelFormat, and setPixelFormat:) you need to implement the following methods: ● + (NSOpenGLPixelFormat*) defaultPixelFormat Use this method to allocate and initialize the NSOpenGLPixelFormat object. ● - (void) clearGLContext Use this method to clear and release the NSOpenGLContext object. ● - (void) prepareOpenGL Use this method to initialize the OpenGL state after creating the NSOpenGLContext object. You need to override the update and initWithFrame: methods of the NSView class. ● update calls the update method of the NSOpenGLContext class. ● initWithFrame:pixelFormat retains the pixel format and sets up the notification NSViewGlobalFrameDidChangeNotification. See Listing 2-6. Listing 2-6 The initWithFrame:pixelFormat: method - (id)initWithFrame:(NSRect)frameRect pixelFormat:(NSOpenGLPixelFormat*)format { self = [super initWithFrame:frameRect]; if (self != nil) { _pixelFormat = [format retain]; [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(_surfaceNeedsUpdate:) name:NSViewGlobalFrameDidChangeNotification object:self]; } Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 41return self; } - (void) _surfaceNeedsUpdate:(NSNotification*)notification { [self update]; } If the custom view is not guaranteed to be in a window, you must also override the lockFocus method of the NSView class. See Listing 2-7. This method makes sure that the view is locked prior to drawing and that the context is the current one. Listing 2-7 The lockFocus method - (void)lockFocus { NSOpenGLContext* context = [self openGLContext]; [super lockFocus]; if ([context view] != self) { [context setView:self]; } [context makeCurrentContext]; } The reshape method is not supported by the NSView class. You need to update bounds in the drawRect: method, which should take the form shown in Listing 2-8. Listing 2-8 The drawRect method for a custom view -(void) drawRect { [context makeCurrentContext]; //Perform drawing here [context flushBuffer]; } Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 42There may be other methods that you want to add. For example, you might consider detaching the context from the drawable object when the custom view is moved from the window, as shown in Listing 2-9. Listing 2-9 Detaching the context from a drawable object -(void) viewDidMoveToWindow { [super viewDidMoveToWindow]; if ([self window] == nil) [context clearDrawable]; } Drawing to a Window or View Drawing to a Cocoa View 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 43OpenGL is a pixel-based API so the NSOpenGLView class does not provide high-resolution surfaces by default. Because adding more pixelsto renderbuffers has performance implications, you must explicitly opt in to support high-resolution screens. It’s easy to enable high-resolution backing for an OpenGL view. When you do, you’ll want to perform a few additional tasks to ensure the best possible high-resolution experience for your users. Enable High-Resolution Backing for an OpenGL View You can opt in to high resolution by calling the method setWantsBestResolutionOpenGLSurface: when you initialize the view, and supplying YES as an argument: [self setWantsBestResolutionOpenGLSurface:YES]; If you don’t opt in, the system magnifies the rendered results. The wantsBestResolutionOpenGLSurface property is relevant only for views to which an NSOpenGLContext object is bound. Its value does not affect the behavior of other views. For compatibility, wantsBestResolutionOpenGLSurface defaultsto NO, providing a 1-pixel-per-point framebuffer regardless of the backing scale factor for the display the view occupies. Setting this property to YES for a given view causes AppKit to allocate a higher-resolution framebuffer when appropriate for the backing scale factor and target display. To function correctly with wantsBestResolutionOpenGLSurface set to YES, a view must perform correct conversions between view units (points) and pixel units as needed. For example, the common practice of passing the width and height of [self bounds] to glViewport() will yield incorrect results at high resolution, because the parameters passed to the glViewport() function must be in pixels. As a result, you’ll get only partial instead of complete coverage of the render surface. Instead, use the backing store bounds: [self convertRectToBacking:[self bounds]]; 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 44 Optimizing OpenGL for High ResolutionYou can also opt in to high resolution by enabling the Supports Hi-Res Backing setting for the OpenGL view in Xcode, as shown in Figure 3-1. Figure 3-1 Enabling high-resolution backing for an OpenGL view Set Up the Viewport to Support High Resolution The viewport dimensions are in pixelsrelative to the OpenGL surface. Passthe width and height to glViewPort and use 0,0 for the x and y offsets. Listing 3-1 shows how to get the view dimensions in pixels and take the backing store size into account. Listing 3-1 Setting up the viewport for drawing - (void)drawRect:(NSRect)rect // NSOpenGLView subclass { // Get view dimensions in pixels NSRect backingBounds = [self convertRectToBacking:[self bounds]]; Optimizing OpenGL for High Resolution Set Up the Viewport to Support High Resolution 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 45GLsizei backingPixelWidth = (GLsizei)(backingBounds.size.width), backingPixelHeight = (GLsizei)(backingBounds.size.height); // Set viewport glViewport(0, 0, backingPixelWidth, backingPixelHeight); // draw… } You don’t need to perform rendering in pixels, but you do need to be aware of the coordinate system you want to render in. For example, if you want to render in points, this code will work: glOrtho(NSWidth(bounds), NSHeight(bounds),...) Adjust Model and Texture Assets If you opt in to high-resolution drawing, you also need to adjust the model and texture assets of your app. For example, when running on a high-resolution display, you might want to choose larger models and more detailed textures to take advantage of the increased number of pixels. Conversely, on a standard-resolution display, you can continue to use smaller models and textures. If you create and cache textures when you initialize your app, you might want to consider a strategy that accommodates changing the texture based on the resolution of the display. Check for Calls Defined in Pixel Dimensions These functions use pixel dimensions: ● glViewport (GLint x, GLint y, GLsizei width, GLsizei height) ● glScissor (GLint x, GLint y, GLsizei width, GLsizei height) ● glReadPixels (GLint x, GLint y, GLsizei width, GLsizei height, ...) ● glLineWidth (GLfloat width) ● glRenderbufferStorage (..., GLsizei width, GLsizei height) ● glTexImage2D (..., GLsizei width, GLsizei height, ...) Optimizing OpenGL for High Resolution Adjust Model and Texture Assets 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 46Tune OpenGL Performance for High Resolution Performance is an important factor when determining whether to support high-resolution content. The quadrupling of pixels that occurs when you opt in to high resolution requires more work by the fragment processor. If your app performs many per-fragment calculations, the increase in pixels might reduce its frame rate. If your app runs significantly slower at high resolution, consider the following options: ● Optimize fragment shader performance. (See “Tuning Your OpenGL Application” (page 155).) ● Choose a simpler algorithm to implement in your fragment shader. This reduces the quality of each individual pixel to allow for rendering the overall image at a higher resolution. ● Use a fractional scale factor between 1.0 and 2.0. A scale factor of 1.5 provides better quality than a scale factor of 1.0, but it needs to fill fewer pixels than an image scaled to 2.0. ● Multisampling antialiasing can be costly with marginal benefit at high resolution. If you are using it, you might want to reconsider. The best solution depends on the needs of your OpenGL app; you should test more than one of these options and choose the approach that provides the best balance between performance and image quality. Optimizing OpenGL for High Resolution Tune OpenGL Performance for High Resolution 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 47Use a Layer-Backed View to Overlay Text on OpenGL Content When you draw standard controls and Cocoa text to a layer-backed view, the system handles scaling the contents of that layer for you. You need to perform only a few steps to set and use the layer. Compare the controls and text in standard and high resolutions, as shown in Figure 3-2. The text looks the same on both without any additional work on your part. Figure 3-2 A text overlay scales automatically for standard resolution (left) and high resolution (right) To set up a layer-backed view for OpenGL content 1. Set the wantsLayer property of your NSOpenGLView subclass to YES. Enabling the wantsLayer property of an NSOpenGLView object activates layer-backed rendering of the OpenGL view. Drawing a layer-backed OpenGL view proceeds mostly normally through the view’s drawRect: method. The layer-backed rendering mode usesits own NSOpenGLContext object, which is distinct from the NSOpenGLContext that the view uses for drawing in non-layer-backed mode. AppKit automatically creates this context and assigns it to the view by invoking the setOpenGLContext: method. The view’s openGLContext accessor will return the layer-backed OpenGL context (rather than the non-layer-backed context) while the view is operating in layer-backed mode. Optimizing OpenGL for High Resolution Use a Layer-Backed View to Overlay Text on OpenGL Content 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 482. Create the layer content either as a XIB file or programmatically. The controls shown in Figure 3-2 were created in a XIB file by subclassing NSBox and using static text with a variety of standard controls. Using this approach allows the NSBox subclass to ignore mouse events while still allowing the user to interact with the OpenGL content. 3. Add the layer to the OpenGL view by calling the addSublayer: method. Use an Application Window for Fullscreen Operation For the best user experience, if you want your app to run full screen, create a window that covers the entire screen. This approach offers two advantages: ● The system provides optimized context performance. ● Users will be able to see critical system dialogs above your content. You should avoid changing the display mode of the system. Convert the Coordinate Space When Hit Testing Always convert window event coordinates when performing hit testing in OpenGL. The locationInWindow method of the NSEvent class returns the receiver’s location in the base coordinate system of the window. You then need to call the convertPoint:fromView: method to get the local coordinates for the OpenGL view. NSPoint aPoint = [theEvent locationInWindow]; NSPoint localPoint = [myOpenGLView convertPoint:aPoint fromView:nil]; Optimizing OpenGL for High Resolution Use an Application Window for Fullscreen Operation 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 49In OS X, you have the option to draw to the entire screen. This is a common scenario for games and other immersive applications, and OS X applies additional optimizations to improve the performance of full-screen contexts. Figure 4-1 Drawing OpenGL content to the full screen OS X v10.6 and later automatically optimize the performance ofscreen-sized windows, allowing your application to take complete advantage of the window server environment on OS X. For example, critical operating system dialogs may be displayed over your content when necessary. For information about high-resolution and full-screen drawing, see “Use an Application Window for Fullscreen Operation” (page 49). Creating a Full-Screen Application Creating a full-screen context is very simple. Your application should follow these steps: 1. Create a screen-sized window on the display you want to take over: NSRect mainDisplayRect = [[NSScreen mainScreen] frame]; 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 50 Drawing to the Full ScreenNSWindow *fullScreenWindow = [[NSWindow alloc] initWithContentRect: mainDisplayRect styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:YES]; 2. Set the window level to be above the menu bar.: [fullScreenWindow setLevel:NSMainMenuWindowLevel+1]; 3. Perform any other window configuration you desire: [fullScreenWindow setOpaque:YES]; [fullScreenWindow setHidesOnDeactivate:YES]; 4. Create a view with a double-buffered OpenGL context and attach it to the window: NSOpenGLPixelFormatAttribute attrs[] = { NSOpenGLPFADoubleBuffer, 0 }; NSOpenGLPixelFormat* pixelFormat = [[NSOpenGLPixelFormat alloc] initWithAttributes:attrs]; NSRect viewRect = NSMakeRect(0.0, 0.0, mainDisplayRect.size.width, mainDisplayRect.size.height); MyOpenGLView *fullScreenView = [[MyOpenGLView alloc] initWithFrame:viewRect pixelFormat: pixelFormat]; [fullScreenWindow setContentView: fullScreenView]; 5. Show the window: [fullScreenWindow makeKeyAndOrderFront:self]; That’s all you need to do. Your content is in a window that is above most other content, but because it is in a window, OS X can still show critical UI elements above your content when necessary (such as error dialogs). When there is no content above your full-screen window, OS X automatically attemptsto optimize this context’s performance. For example, when your application calls flushBuffer on the NSOpenGLContext object, the Drawing to the Full Screen Creating a Full-Screen Application 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 51system may swap the buffers rather than copying the contents of the back buffer to the front buffer. These performance optimizations are not applied when your application adds the NSOpenGLPFABackingStore attribute to the context. Because the system may choose to swap the buffers rather than copy them, your application must completely redraw the scene after every call to flushBuffer. For more information on NSOpenGLPFABackingStore, see “Ensuring That Back Buffer Contents Remain the Same” (page 66). Avoid changing the display resolution from that chosen by the user. If your application needs to render data at a lower resolution for performance reasons, you can explicitly create a back buffer at the desired resolution and allow OpenGL to scale those results to the display. See “Controlling the Back Buffer Size” (page 78). Drawing to the Full Screen 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 52OpenGL applications may want to use OpenGL to render images without actually displaying them to the user. For example, an image processing application might render the image, then copy that image back to the application and save it to disk. Another useful strategy is to create intermediate images that are used later to render additional content. For example, your application might want to render an image and use it as a texture in a future rendering pass. For best performance, offscreen targets should be managed by OpenGL. Having OpenGL manage offscreen targets allows you to avoid copying pixel data back to your application, except when this is absolutely necessary. OS X offers two useful options for creating offscreen rendering targets: ● Framebuffer objects. The OpenGL framebuffer extension allows your application to create fully supported offscreen OpenGL framebuffers. Framebuffer objects are fully supported as a cross-platform extension, so they are the preferred way to create offscreen rendering targets. See “Rendering to a Framebuffer Object” (page 53). ● Pixel buffer drawable objects. Pixel buffer drawable objects are an Apple-specific technology for creating an offscreen target. Each of the Apple-specific OpenGL APIs provides routines to create an offscreen hardware accelerated pixel buffer. Pixel buffers are recommended for use only when framebuffer objects are not available. See “Rendering to a Pixel Buffer” (page 60). Rendering to a Framebuffer Object The OpenGL framebuffer extension (GL_EXT_framebuffer_object) allows applications to create offscreen rendering targets from within OpenGL. OpenGL manages the memory for these framebuffers. Note: Extensions are available on a per-renderer basis. Before you use framebuffer objects you must check each renderer to make sure that itsupportsthe extension. See “Detecting Functionality” (page 83) for more information. A framebuffer object(FBO) issimilar to a drawable object, except a drawable object is a window-system-specific object, whereas a framebuffer object is a window-agnostic object that's defined in the OpenGL standard. After drawing to a framebuffer object, it is straightforward to read the pixel data to the application, or to use it as source data for other OpenGL commands. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 53 Drawing OffscreenFramebuffer objects offer a number of benefits: ● They are window-system independent, which makes porting code easier. ● They are easy to set up and save memory. There is no need to set up attributes and obtain a pixel format object. ● They are associated with a single OpenGL context, whereas each pixel buffer must be bound to a context. ● You can switch between them faster since there is no context switch as with pixel buffers. Because all commands are rendered by a single context, no additional serialization is required. ● They can share depth buffers; pixel buffers cannot. ● You can use them for 2D pixel images and texture images. Completeness is a key concept to understanding framebuffer objects. Completeness is a state that indicates whether a framebuffer object meets all the requirements for drawing. You test for this state after performing all the necessary setup work. If a framebuffer object is not complete, it cannot be used as the destination for rendering operations and as a source for read operations. Completeness is dependent on many factors that are not possible to condense into one or two statements, but these factors are thoroughly defined in the OpenGL specification for the framebuffer object extension. The specification describes the requirements for internal formats of images attached to the framebuffer, how to determine if a format is color-, depth-, and stencil-renderable, as well as other requirements. Prior to using framebuffer objects, read the OpenGL specification, which not only defines the framebuffer object API, but provides detailed definitions of all the terms necessary to understand their use and shows several code examples. The remainder of thissection provides an overview of how to use a framebuffer as either a texture or an image. The functions used to set up textures and images are slightly different. The API for images usesthe renderbuffer terminology defined in the OpenGL specification. A renderbuffer image is simply a 2D pixel image. The API for textures uses texture terminology, as you might expect. For example, one of the calls for setting up a framebuffer object for a texture is glFramebufferTexture2DEXT, whereasthe call forsetting up a framebuffer object for an image is glFramebufferRenderbufferEXT. You'll see how to set up a simple framebuffer object for each type of drawing, starting first with textures. Using a Framebuffer Object as a Texture These are the basic steps needed to set up a framebuffer object for drawing a texture offscreen: 1. Make sure the framebuffer extension (GL_EXT_framebuffer_object) is supported on the system that your code runs on. See “Determining the OpenGL Capabilities Supported by the Renderer” (page 83). Drawing Offscreen Rendering to a Framebuffer Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 542. Check the renderer limits. For example, you might want to call the OpenGL function glGetIntegerv to check the maximum texture size (GL_MAX_TEXTURE_SIZE) or find out the maximum number of color buffers you can attach to the framebuffer object(GL_MAX_COLOR_ATTACHMENTS_EXT). 3. Generate a framebuffer object name by calling the following function: void glGenFramebuffersEXT (GLsizei n, GLuint *ids); n is the number of framebuffer object names that you want to create. On return, *ids points to the generated names. 4. Bind the framebuffer object name to a framebuffer target by calling the following function: void glBindFramebufferEXT(GLenum target, GLuint framebuffer); target should be the constant GL_FRAMEBUFFER_EXT. framebuffer is set to an unused framebuffer object name. On return, the framebuffer object is initialized to the state values described in the OpenGL specification for the framebuffer object extension. Each attachment point of the framebuffer is initialized to the attachment point state values described in the specification. The number of attachment points is equal to GL_MAX_COLOR_ATTACHMENTS_EXT plus 2 (for depth and stencil attachment points). Whenever a framebuffer object is bound, drawing commands are directed to it instead of being directed to the drawable associated with the rendering context. 5. Generate a texture name. void glGenTextures(GLsizei n, GLuint *textures); n is the number of texture object names that you want to create. On return, *textures points to the generated names. 6. Bind the texture name to a texture target. void glBindTexture(GLenum target, GLuint texture); target is the type of texture to bind. texture is the texture name you just created. 7. Set up the texture environment and parameters. Drawing Offscreen Rendering to a Framebuffer Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 558. Define the texture by calling the appropriate OpenGL function to specify the target, level of detail, internal format, dimensions, border, pixel data format, and texture data storage. 9. Attach the texture to the framebuffer by calling the following function: void glFramebufferTexture2DEXT (GLenum target, GLenum attachment, GLenum textarget, GLuint texture, GLint level); target must be GL_FRAMEBUFFER_EXT. attachment must be one of the attachment points of the framebuffer: GL_STENCIL_ATTACHMENT_EXT, GL_DEPTH_ATTACHMENT_EXT, or GL_COLOR_ATTACHMENTn_EXT, where n is a number from 0 to GL_MAX_COLOR_ATTACHMENTS_EXT-1. textarget is the texture target. texture is an existing texture object. level is the mipmap level of the texture image to attach to the framebuffer. 10. Check to make sure that the framebuffer is complete by calling the following function: GLenum glCheckFramebufferStatusEXT(GLenum target); target must be the constant GL_FRAMEBUFFER_EXT. This function returns a status constant. You must test to make sure that the constant is GL_FRAMEBUFFER_COMPLETE_EXT. If it isn't, see the OpenGL specification for the framebuffer object extension for a description of the other constants in the status enumeration. 11. Render content to the texture. You must make sure to bind a different texture to the framebuffer object or disable texturing before you render content. If you render to a framebuffer object texture attachment with that same texture currently bound and enabled, the result is undefined. 12. To draw the contents of the texture to a window, make the window the target of all rendering commands by calling the function glBindFramebufferEXT and passing the constant GL_FRAMEBUFFER_EXT and 0. The window is always specified as 0. 13. Use the texture attachment as a normal texture by binding it, enabling texturing, and drawing. 14. Delete the texture. 15. Delete the framebuffer object by calling the following function: void glDeleteFramebuffersEXT (GLsizei n, const GLuint *framebuffers); Drawing Offscreen Rendering to a Framebuffer Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 56n is the number of framebuffer objects to delete. *framebuffers points to an array that contains the framebuffer object names. Listing 5-1 shows code that performs these tasks. This example creates and draws to a single framebuffer object. Listing 5-1 Setting up a framebuffer for texturing GLuint framebuffer, texture; GLenum status; glGenFramebuffersEXT(1, &framebuffer); // Set up the FBO with one texture attachment glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, framebuffer); glGenTextures(1, &texture); glBindTexture(GL_TEXTURE_2D, texture); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, TEXWIDE, TEXHIGH, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL); glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_2D, texture, 0); status = glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT); if (status != GL_FRAMEBUFFER_COMPLETE_EXT) // Handle error here // Your code to draw content to the FBO // ... // Make the window the target glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0); //Your code to use the contents of the FBO // ... //Tear down the FBO and texture attachment glDeleteTextures(1, &texture); glDeleteFramebuffersEXT(1, &framebuffer); Drawing Offscreen Rendering to a Framebuffer Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 57Using a Framebuffer Object as an Image There is a lot of similarity between setting up a framebuffer object for drawing images and setting one up to draw textures. These are the basic steps needed to set up a framebuffer object for drawing a 2D pixel image (a renderbuffer image) offscreen: 1. Make sure the framebuffer extension (EXT_framebuffer_object) is supported on the renderer that your code runs on. 2. Check the renderer limits. For example, you might want to call the OpenGL function glGetIntegerv to find out the maximum number of color buffers (GL_MAX_COLOR_ATTACHMENTS_EXT). 3. Generate a framebuffer object name by calling the function glGenFramebuffersEXT. 4. Bind the framebuffer object name to a framebuffer target by calling the function glBindFramebufferEXT. 5. Generate a renderbuffer object name by calling the following function: void glGenRenderbuffersEXT (GLsizei n, GLuint *renderbuffers ); n is the number of renderbuffer object names to create. *renderbuffers points to storage for the generated names. 6. Bind the renderbuffer object name to a renderbuffer target by calling the following function: void glBindRenderbufferEXT (GLenum target, GLuint renderbuffer); target must be the constant GL_RENDERBUFFER_EXT. renderbuffer is the renderbuffer object name generated previously. 7. Create data storage and establish the pixel format and dimensions of the renderbuffer image by calling the following function: void glRenderbufferStorageEXT (GLenum target, GLenum internalformat, GLsizei width, GLsizei height); target must be the constant GL_RENDERBUFFER_EXT. internalformat is the pixel format of the image. The value must be RGB, RGBA, DEPTH_COMPONENT, STENCIL_INDEX, or one of the other formats listed in the OpenGL specification. width is the width of the image, in pixels. height is the height of the image, in pixels. Drawing Offscreen Rendering to a Framebuffer Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 588. Attach the renderbufferto a framebuffertarget by calling the function glFramebufferRenderbufferEXT. void glFramebufferRenderbufferEXT(GLenum target, GLenum attachment, GLenum renderbuffertarget, GLuint renderbuffer); target must be the constant GL_FRAMEBUFFER_EXT. attachment should be one of the attachment points of the framebuffer: GL_STENCIL_ATTACHMENT_EXT, GL_DEPTH_ATTACHMENT_EXT, or GL_COLOR_ATTACHMENTn_EXT, where n is a number from 0 to GL_MAX_COLOR_ATTACHMENTS_EXT–1. renderbuffertarget must be the constant GL_RENDERBUFFER_EXT. renderbuffer should be set to the name of the renderbuffer object that you want to attach to the framebuffer. 9. Check to make sure that the framebuffer is complete by calling the following function: enum glCheckFramebufferStatusEXT(GLenum target); target must be the constant GL_FRAMEBUFFER_EXT. This function returns a status constant. You must test to make sure that the constant is GL_FRAMEBUFFER_COMPLETE_EXT. If it isn't, see the OpenGL specification for the framebuffer object extension for a description of the other constants in the status enumeration. 10. Render content to the renderbuffer. 11. To access the contents of the renderbuffer object, bind the framebuffer object and then use OpenGL functions such as glReadPixels or glCopyTexImage2D. 12. Delete the framebuffer object with its renderbuffer attachment. Listing 5-2 shows code that sets up and draws to a single renderbuffer object. Your application can set up more than one renderbuffer object if it requires them. Listing 5-2 Setting up a renderbuffer for drawing images GLuint framebuffer, renderbuffer; GLenum status; // Set the width and height appropriately for your image GLuint imageWidth = 1024, imageHeight = 1024; Drawing Offscreen Rendering to a Framebuffer Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 59//Set up a FBO with one renderbuffer attachment glGenFramebuffersEXT(1, &framebuffer); glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, framebuffer); glGenRenderbuffersEXT(1, &renderbuffer); glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, renderbuffer); glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_RGBA8, imageWidth, imageHeight); glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_RENDERBUFFER_EXT, renderbuffer); status = glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT); if (status != GL_FRAMEBUFFER_COMPLETE_EXT) // Handle errors //Your code to draw content to the renderbuffer // ... //Your code to use the contents // ... // Make the window the target glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0); // Delete the renderbuffer attachment glDeleteRenderbuffersEXT(1, &renderbuffer); Rendering to a Pixel Buffer The OpenGL extension string GL_APPLE_pixel_buffer provides hardware-accelerated offscreen rendering to a pixel buffer. A pixel buffer is typically used as a texture source. It can also be used for remote rendering. Important: Pixel buffers are deprecated starting with OS X v10.7 and are not supported by the OpenGL 3.2 Core profile; use framebuffer objects instead. You must create a rendering context for each pixel buffer. For example, if you want to use a pixel buffer as a texture source, you create one rendering context attached to the pixel buffer and a second context attached to a window or view. The first step in using a pixel buffer is to create it. The Apple-specific OpenGL APIs each provide a routine for this purpose: Drawing Offscreen Rendering to a Pixel Buffer 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 60● The NSOpenGLPixelBuffer method initWithTextureTarget:textureInternalFormat:textureMaxMipMapLevel:pixelsWide:pixelsHigh: ● The CGL function CGLCreatePBuffer Each of these routinesrequiresthat you provide a texture target, an internal format, a maximum mipmap level, and the width and height of the texture. The texture target must be one of these OpenGL texture constants: GL_TEXTURE_2D for a 2D texture, GL_TEXTURE_RECTANGLE_ARB for a rectangular (not power-of-two) texture, or GL_TEXTURE_CUBE_MAP for a cube map texture. The internal format specifies how to interpret the data for texturing operations. You can supply any of these options: GL_RGB (each pixel is a three-component group), GL_RGBA (each pixel is a four-component group), or GL_DEPTH_COMPONENT (each pixel is a single depth component). The maximum mipmap level should be 0 for a pixel buffer that does not have a mipmap. The value that you supply should not exceed the actual maximum number of mipmap levels that can be represented with the given width and height. Note that none of the routines that create a pixel buffer allocate the storage needed. The storage is allocated by the system at the time that you attach the pixel buffer to a rendering context. Setting Up a Pixel Buffer for Offscreen Drawing After you create a pixel buffer, the general procedure for using a pixel buffer for drawing is similar to the way you set up windows and views for drawing: 1. Specify renderer and buffer attributes. 2. Obtain a pixel format object. 3. Create a rendering context and make it current. 4. Attach a pixel buffer to the context using the appropriate Apple OpenGL attachment function: ● The setPixelBuffer:cubeMapFace:mipMapLevel:currentVirtualScreen: method of the NSOpenGLContext class instructs the receiver to render into a pixel buffer. ● The CGL function CGLSetPBuffer attaches a CGL rendering context to a pixel buffer. 5. Draw, as you normally would, using OpenGL. Using a Pixel Buffer as a Texture Source Pixel bufferslet you perform direct texturing without incurring the cost of extra copies. After drawing to a pixel buffer, you can create a texture by following these steps: Drawing Offscreen Rendering to a Pixel Buffer 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 611. Generate a texture name by calling the OpenGL function glGenTextures. 2. Bind the named texture to a target by calling the OpenGL function glBindTexture. 3. Set the texture parameters by calling OpenGL function glTexEnvParameter. 4. Set up the pixel buffer asthe source for the texture by calling one of the following Apple OpenGL functions: ● The setTextureImageToPixelBuffer:colorBuffer: method of the NSOpenGLContext class attaches the image data in the pixel buffer to the texture object currently bound by the receiver. ● The CGL function CGLTexImagePBuffer binds the contents of a CGL pixel buffer as the data source for a texture object. The context that you attach to the pixel buffer is the target rendering context: the context that uses the pixel buffer as the source of the texture data. Each of these routines requires a source parameter, which is an OpenGL constant that specifies the source buffer to texture from. The source parameter must be a valid OpenGL buffer, such as GL_FRONT, GL_BACK, or GL_AUX0, and should be compatible with the buffer attributes used to create the OpenGL context associated with the pixel buffer. This means that the pixel buffer must possess the buffer in question for texturing to succeed. For example, if the buffer attribute used with the pixel buffer is only single buffered, then texturing from the GL_BACK buffer will fail. If you modify content of any pixel buffer that contains mipmap levels, you must call the appropriate Apple OpenGL function again (setTextureImageToPixelBuffer:colorBuffer: or CGLTexImagePBuffer) before drawing with the pixel buffer to ensure that the content issynchronized with OpenGL. To synchronize the content of pixel buffers without mipmaps, simply rebind to the texture object using glBind. 5. Draw primitives using the appropriate texture coordinates. (See "The Red book"—OpenGL Programming Guide—for details.) 6. Call glFlush to cause all drawing commands to be executed. 7. When you no longer need the texture object, call the OpenGL function glDeleteTextures. 8. Set the current context to NULL using one of the Apple OpenGL routines: ● The makeCurrentContext method of the NSOpenGLContext class ● The CGL function CGLSetCurrentContext 9. Destroy the pixel buffer by calling CGLDestroyPBuffer. 10. Destroy the context by calling CGLDestroyContext. 11. Destroy the pixel format by calling CGLDestroyPixelFormat. You might find these guidelines useful when using pixel buffers for texturing: Drawing Offscreen Rendering to a Pixel Buffer 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 62● You cannot make OpenGL texturing calls that modify pixel buffer content (such as glTexSubImage2D or glCopyTexImage2D) with the pixel buffer as the destination. You can use texturing commands to read data from a pixel buffer, such as glCopyTexImage2D, with the pixel buffer texture as the source. You can also use OpenGL functions such as glReadPixels to read the contents of a pixel buffer directly from the pixel buffer context. ● Texturing can fail to produce the intended results without reporting an error. You must make sure that you enable the proper texture target, set a compatible filter mode, and adhere to other requirements described in the OpenGL specification. ● You are not required to set up contextsharing when you texture from a pixel buffer. You can have different pixel format objects and rendering contexts for both the pixel buffer and the target drawable object, without sharing resources, and still texture using a pixel buffer in the target context. Rendering to a Pixel Buffer on a Remote System Follow these steps to render to a pixel buffer on a remote system. The remote system does not need to have a display attached to it. 1. When you set the renderer and buffer attributes, include the remote pixel buffer attribute kCGLPFARemotePBuffer. 2. Log in to the remote machine using the ssh command to ensure security. 3. Run the application on the target system. 4. Retrieve the content. Drawing Offscreen Rendering to a Pixel Buffer 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 63Renderer and buffer attributes determine the renderers that the system chooses for your application. Each of the Apple-specific OpenGL APIs provides constants that specify a variety of renderer and buffer attributes. You supply a list of attribute constants to one of the Apple OpenGL functions for choosing a pixel format object. The pixel format object maintains a list of renderers that meet the requirements defined by those attributes. In a real-world application, selecting attributes is an art because you don't know the exact combination of hardware and software that your application will run on. An attribute list that is too restrictive may miss out on future capabilities or it may fail to return renderers on some systems. For example, if you specify a buffer of a specific depth, your application won't be able to take advantage of a larger buffer when more memory is available in the future. In this case, you might specify a required minimum and direct OpenGL to use the maximum available. Although you might specify attributes that make your OpenGL content look and run its best, you also need to consider whether your application should run on a less-capable system with less speed or detail. If tradeoffs are acceptable, you need to set the attributes accordingly. OpenGL Profiles (OS X v10.7) When your application is running on OS X v10.7, it should always include the kCGLPFAOpenGLProfile attribute, followed by a constant for the profile whose functionality your application requires. A profile affects different parts of OpenGL in OS X: ● A profile requires that a specific version of the OpenGL API must provided by the renderer. The renderer may implement a different version of the OpenGL specification only if that version implements the same functions and constants required by the profile; typically, this means a renderer that supports a later version of the OpenGL specification that did not remove or alter behavior specified in the version of the OpenGL specification your application requested. ● The profile alters the list of OpenGL extensions returned by the renderer. For example, extensions whose functionality is provided by the version of the OpenGL specification you requested are not also returned in the list of extensions. ● On OS X, the profile affects what other renderer and buffer attributes may be included in the attributes list. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 64 Choosing Renderer and Buffer AttributesFollow these guidelines to choose an OpenGL profile: ● If you are developing a new OS X v10.7 application, implement your OpenGL functionality using the OpenGL 3.2 Core profile; include the kCGLOGLPVersion_3_2_Core constant. The OpenGL 3.2 core profile is defined by Khronos and explicitly removes removes deprecated features described in earlier versions of the OpenGL specification; further the core profile prohibits these functions from being added back into OpenGL using extensions. OpenGL 3.2 core represents a complete break from the fixed function pipeline of OpenGL 1.x in favor of a clean, lean shader-based pipeline. When you use the OpenGL 3.2 Core profile on OS X, legacy extensions are removed wherever their functionality is already provided by OpenGL 3.2. Further, pixel and buffer format attributesthat are marked as deprecated may not be used in conjunction with the OpenGL 3.2 core profile. ● If you are updating an existing OS X application, include the kCGLOGLPVersion_Legacy constant. The legacy profile provides the same functionality found in earlier versions of OS X, with no changes. It continues to support older extensions as well as deprecated pixel and buffer format attributes. No new functionality will be added to the legacy profile in future versions of OS X. ● If you want to use OpenGL 3.2 in your application, but also want to support earlier versions of OS X or Macsthat lack hardware support for OpenGL 3.2, you must implement multiple OpenGL rendering options in your application. On OS X v10.7, your application should first test to see if OpenGL 3.2 is supported. If OpenGL 3.2 is supported, create a context and provide it to your OpenGL 3.2 rendering path. Otherwise, search for a pixel format using the legacy profile instead. For more information on migrating an application to OpenGL 3.2, see “Updating an Application to Support the OpenGL 3.2 Core Specification” (page 168). Buffer Size Attribute Selection Tips Follow these guidelines to choose buffer attributes that specify buffer size: ● To choose color, depth, and accumulation buffers that are greater than or equal to a size you specify, use the minimum policy attribute (NSOpenGLPFAMinimumPolicy or kCGLPFAMinimumPolicy). ● To choose color, depth, and accumulation buffers that are closest to a size you specify, use the closest policy attribute (NSOpenGLPFAClosestPolicy or kCGLPFAClosestPolicy). ● To choose the largest color, depth, and accumulation buffers available, use the maximum policy attribute (NSOpenGLPFAMaximumPolicy or kCGLPFAMaximumPolicy). Aslong as you pass a value that is greater than 0, this attribute specifies the use of color, depth, and accumulation buffers that are the largest size possible. Choosing Renderer and Buffer Attributes Buffer Size Attribute Selection Tips 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 65Ensuring That Back Buffer Contents Remain the Same When your application uses a double-buffered context, it displays the rendered image by calling a function to flush the image to the screen— theNSOpenGLContext class’s flushBuffer method or the CGL function CGLFlushDrawable. When the image is displayed, the contents of the back buffer are not preserved. The next time your application wants to update the back buffer, it must completely redraw the scene. Your application can add a backing store attribute (NSOpenGLPFABackingStore or kCGLPFABackingStore) to preserve the contents of the buffer after the back buffer is flushed. Adding this attribute disables some optimizations that the system can perform, which may impact the performance of your application. Ensuring a Valid Pixel Format Object The pixel format routines (the initWithAttributes: method of the NSOpenGLPixelFormat class and the CGLChoosePixelFormat function) return a pixel format object to your application that you use to create a rendering context. The buffer and renderer attributes that you supply to the pixel format routine determine the characteristics of the OpenGL drawing sent to the rendering context. If the system can't find at least one pixel format that satisfies the constraints specified by the attribute array, it returns NULL for the pixel format object. In this case, your application should have an alternative that ensures it can obtain a valid object. One alternative is to set up your attribute array with the least restrictive attribute first and the most restrictive attribute last. Then, it is fairly easy to adjust the attribute list and make another request for a pixel format object. The code in Listing 6-1 illustrates this technique using the CGL API. Notice that the initial attributes list is set up with the supersample attribute last in the list. If the function CGLChoosePixelFormat returns NULL, it clears the supersample attribute to NULL and tries again. Listing 6-1 Using the CGL API to create a pixel format object int last_attribute = 6; CGLPixelFormatAttribute attribs[] = { kCGLPFAAccelerated, kCGLPFAColorSize, 24 kCGLPFADepthSize, 16, kCGLPFADoubleBuffer, kCGLPFASupersample, 0 }; Choosing Renderer and Buffer Attributes Ensuring That Back Buffer Contents Remain the Same 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 66CGLPixelFormatObj pixelFormatObj; GLint numPixelFormats; long value; CGLChoosePixelFormat (attribs, &pixelFormatObj, &numPixelFormats); if( pixelFormatObj == NULL ) { attribs[last_attribute] = NULL; CGLChoosePixelFormat (attribs, &pixelFormatObj, &numPixelFormats); } if( pixelFormatObj == NULL ) { // Your code to notify the user and take action. } Ensuring a Specific Type of Renderer There are times when you want to ensure that you obtain a pixel format that supports a specific renderer type, such as a hardware-accelerated renderer. Table 6-1 lists attributes that support specific types of renderers. The table reflects the following tips for setting up pixel formats: ● To select only hardware-accelerated renderers, use both the accelerated and no-recovery attributes. ● To use only the floating-point software renderer, use the appropriate generic floating-point constant. ● To render to system memory, use the offscreen pixel attribute. Note that this rendering option does not use hardware acceleration. ● To render offscreen with hardware acceleration, specify a pixel buffer attribute. (See “Rendering to a Pixel Buffer” (page 60).) Table 6-1 Renderer types and pixel format attributes Renderer type CGL Cocoa NSOpenGLPFAAccelerated NSOpenGLPFANoRecovery kCGLPFAAccelerated kCGLPFANoRecovery Hardware-accelerated onscreen Choosing Renderer and Buffer Attributes Ensuring a Specific Type of Renderer 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 67Renderer type CGL Cocoa NSOpenGLPFARendererID kCGLRendererGenericFloatID kCGLPFARendererID kCGLRendererGenericFloatID Software (floating-point) System memory (not kCGLPFAOffScreen NSOpenGLPFAOffScreen accelerated) Hardware-accelerated kCGLPFAPBuffer NSOpenGLPFAPixelBuffer offscreen Ensuring a Single Renderer for a Display In some cases you may want to use a specific hardware renderer and nothing else. Since the OpenGL framework normally provides a software renderer as a fallback in addition to whatever hardware renderer it chooses, you need to prevent OpenGL from choosing the software renderer as an option. To do this, specify the no-recovery attribute for a windowed drawable object. Limiting a context to use a specific display, and thus a single renderer, has its risks. If your application runs on a system that uses more than one display, dragging a windowed drawable object from one display to the other is likely to yield a less than satisfactory result. Either rendering fails, or OpenGL uses the specified renderer and then copiesthe result to the second display. The same unsatisfactory result happens when attaching a full-screen context to another display. If you choose to use the hardware renderer associated with a specific display, you need to add code that detects and handles display changes. The code examples that follow show how to use each of the Apple-specific OpenGL APIs to set up a context that uses a single renderer. Listing 6-2 shows how to set up an NSOpenGLPixelFormat object that supports a single renderer. The attribute NSOpenGLPFANoRecovery specifies to OpenGL not to provide the fallback option of the software renderer. Listing 6-2 Setting an NSOpenGLContext object to use a specific display #import + (NSOpenGLPixelFormat*)defaultPixelFormat { NSOpenGLPixelFormatAttribute attributes [] = { NSOpenGLPFAScreenMask, 0, NSOpenGLPFANoRecovery, Choosing Renderer and Buffer Attributes Ensuring a Single Renderer for a Display 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 68NSOpenGLPFADoubleBuffer, (NSOpenGLPixelFormatAttribute)nil }; CGDirectDisplayID display = CGMainDisplayID (); // Adds the display mask attribute for selected display attributes[1] = (NSOpenGLPixelFormatAttribute) CGDisplayIDToOpenGLDisplayMask (display); return [[(NSOpenGLPixelFormat *)[NSOpenGLPixelFormat alloc] initWithAttributes:attributes] autorelease]; } Listing 6-3 shows how to use CGL to set up a context that uses a single renderer. The attribute kCGLPFANoRecovery ensures that OpenGL does not provide the fallback option of the software renderer. Listing 6-3 Setting a CGL context to use a specific display #include CGLPixelFormatAttribute attribs[] = { kCGLPFADisplayMask, 0, kCGLPFANoRecovery, kCGLPFADoubleBuffer, 0 }; CGLPixelFormatObj pixelFormat = NULL; GLint numPixelFormats = 0; CGLContextObj cglContext = NULL; CGDirectDisplayID display = CGMainDisplayID (); // Adds the display mask attribute for selected display attribs[1] = CGDisplayIDToOpenGLDisplayMask (display); CGLChoosePixelFormat (attribs, &pixelFormat, &numPixelFormats); Allowing Offline Renderers Adding the attribute NSOpenGLPFAAllowOfflineRenderers allows OpenGL to include offline renderers in the list of virtual screens returned in the pixel format object. Apple recommends you include this attribute, because it allows your application to work better in environments where renderers come and go,such as when a new display is plugged into a Mac. Choosing Renderer and Buffer Attributes Allowing Offline Renderers 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 69If your application includes NSOpenGLPFAAllowOfflineRenderers in the list of attributes, your application must also watch for display changes and update its rendering context. See “Update the Rendering Context When the Renderer or Geometry Changes” (page 72). OpenCL If your applications uses OpenCL to perform other computations, you may want to find an OpenGL renderer that also supports OpenCL. To do this, add the attribute NSOpenGLPFAAcceleratedCompute to the pixel format attribute list. Adding this attribute restricts the list of renderers to those that also support OpenCL. More information on OpenCL can be found in the OpenCL Programming Guide for Mac . Deprecated Attributes There are several renderer and buffer attributes that are no longer recommended either because they are too narrowly focused or no longer useful. Your application should move away from using any of these attributes: ● The robust attribute (NSOpenGLPFARobust or kCGLPFARobust) specifies only those renderers that do not have any failure modes associated with a lack of video card resources. ● The multiple-screen attribute (NSOpenGLPFAMultiScreen or kCGLPFAMultiScreen) specifies only those renderers that can drive more than one screen at a time. ● The multiprocessing-safe attribute (kCGLPFAMPSafe) specifies only those renderers that are thread safe. This attribute is deprecated in OS X because all renderers can accept commands for threads running on a second processor. However, this does not mean that all renderers are thread safe or reentrant. See “Concurrency and OpenGL” (page 148). ● The compliant attribute (NSOpenGLPFACompliant or kCGLPFACompliant) specifies only OpenGL-compliant renderers. All OS X renderers are OpenGL-compliant, so this attribute is no longer useful. ● The fullscreen attribute (kCGLPFAFullScreen) requested special fullscreen contexts. The window screen attribute (kCGLPFAWindow) required the context to support windowed contexts. OS X no longer requires a special full screen context to be created, as it automatically provides the same performance benefits with a properly formatted window. ● The offscreen buffer attribute (kCGLPFAOffScreen) selects renderers capable of rendering to offscreen memory. Instead, use a frame buffer object as the rendering target and read the final results back to application memory. Choosing Renderer and Buffer Attributes OpenCL 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 70● The pixel buffer attributes(kCGLPFAPBuffer and kCGLPFARemotePBuffer are no longer recommended; use frame buffer objects instead. ● The auxiliary buffers attribute (kCGLPFAAuxBuffers) specifies the number of required auxiliary buffers your application requires. Auxiliary buffers are not supported by the OpenGL 3.2 Core profile. Because auxiliary buffers are not supported, the kCGLPFAAuxDepthStencil attribute that modifies it is also deprecated. ● The accumulation buffersize attribute (kCGLPFAAccumSize)specifiesthe desired size for the accumulation buffer. Accumulation buffers are not supported by the OpenGL 3.2 Core Profile. Important: Your application may not use any of the deprecated attributes in conjunction with a profile other than the legacy profile; if you do, pixel format creation fails. Choosing Renderer and Buffer Attributes Deprecated Attributes 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 71A rendering context is a container forstate information. When you designate a rendering context asthe current rendering context,subsequent OpenGL commands modify that context’sstate, objects attached to that context, or the drawable object associated with that context. The actual drawing surfaces are never owned by the rendering context but are created, as needed, when the rendering context is actually attached to a drawable object. You can attach multiple rendering contexts to the same drawing surfaces. Each context maintains its own drawing state. “Drawing to a Window or View” (page 35), “Drawing to the Full Screen” (page 50), and “Drawing Offscreen” (page 53) show how to create a rendering context and attach it to a drawable object. This chapter describes advanced ways to interact with rendering contexts. Update the Rendering Context When the Renderer or Geometry Changes A renderer change can occur when the user drags a window from one display to another or when a display is attached or removed. Geometry changes occur when the display mode changes or when a window is resized or moved. If your application uses an NSOpenGLView object to maintain the context, it is automatically updated. An application that creates a custom view to hold the rendering context must track the appropriate system events and update the context when the geometry or display changes. Updating a rendering context notifies it of geometry changes; it doesn't flush content. Calling an update function updates the attached drawable objects and ensures that the renderer is properly updated for any virtual screen changes. If you don't update the rendering context, you may see rendering artifacts. The routine that you call for updating determines how events related to renderer and geometry changes are handled. For applications that use or subclass NSOpenGLView, Cocoa calls the update method automatically. Applications that create an NSOpenGLContext object manually must call the update method of NSOpenGLContext directly. For a full-screen Cocoa application, calling the setFullScreen method of NSOpenGLContext ensures that depth, size, or display changes take affect. Your application must update the rendering context after the system event but before drawing to the context. If the drawable object is resized, you may want to issue a glViewport command to ensure that the content scales properly. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 72 Working with Rendering ContextsNote: Some system-level events(such as display mode changes) that require a context update could reallocate the buffers of the context; thus you need to redraw the entire scene after all context updates. It's important that you don't update rendering contexts more than necessary. Your application should respond to system-level events and notifications rather than updating every frame. For example, you'll want to respond to window move and resize operations and to display configuration changes such as a color depth change. Tracking Renderer Changes It's fairly straightforward to track geometry changes, but how are renderer changes tracked? This is where the concept of a virtual screen becomes important (see “Virtual Screens” (page 26)). A change in the virtual screen indicates a renderer change, a change in renderer capability, or both. When your application detects a window resize event, window move event, or display change, it should check for a virtual screen change and respond to the change to ensure that the current application state reflects any changes in renderer capabilities. Each of the Apple-specific OpenGL APIs has a function that returns the current virtual screen number: ● The currentVirtualScreen method of the NSOpenGLContext class ● The CGLGetVirtualScreen function The virtual screen number represents an index in the list of virtual screens that were set up specifically for the pixel format object used for the rendering context. The number is unique to the list but is meaningless otherwise. When the renderer changes, the limits and extensions available to OpenGL may also change. Your application should retest the capabilities of the renderer and use these to choose its rendering algorithms appropriately. See “Determining the OpenGL Capabilities Supported by the Renderer” (page 83). Updating a Rendering Context for a Custom Cocoa View If you subclass NSView instead of using the NSOpenGLView class, your application must update the rendering context. That's due to a slight difference between the events normally handled by the NSView class and those handled by the NSOpenGLView class. Cocoa does not call a reshape method for the NSView class when the size changes because that class does not export a reshape method to override. Instead, you need to perform reshape operations directly in your drawRect: method, looking for changes in view bounds prior to drawing content. Using this approach provides results that are equivalent to using the reshape method of the NSOpenGLView class. Working with Rendering Contexts Update the Rendering Context When the Renderer or Geometry Changes 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 73Listing 7-1 is a partial implementation of a custom view thatshows how to handle context updates. The update method is called after move, resize, and display change events and when the surface needs updating. The class adds an observer to the notification NSViewGlobalFrameDidChangeNotification, which is posted whenever an NSView object that has attached surfaces (that is, NSOpenGLContext objects) resizes, moves, or changes coordinate offsets. It's slightly more complicated to handle changes in the display configuration. For that, you need to register for the notification NSApplicationDidChangeScreenParametersNotification through the NSApplication class. This notification is posted whenever the configuration of any of the displays attached to the computer is changed (either programmatically or when the user changes the settings in the interface). Listing 7-1 Handling context updates for a custom view #import #import #import @class NSOpenGLContext, NSOpenGLPixelFormat; @interface CustomOpenGLView : NSView { @private NSOpenGLContext* _openGLContext; NSOpenGLPixelFormat* _pixelFormat; } - (id)initWithFrame:(NSRect)frameRect pixelFormat:(NSOpenGLPixelFormat*)format; - (void)update; @end @implementation CustomOpenGLView - (id)initWithFrame:(NSRect)frameRect pixelFormat:(NSOpenGLPixelFormat*)format { Working with Rendering Contexts Update the Rendering Context When the Renderer or Geometry Changes 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 74self = [super initWithFrame:frameRect]; if (self != nil) { _pixelFormat = [format retain]; [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(_surfaceNeedsUpdate:) name:NSViewGlobalFrameDidChangeNotification object:self]; } return self; } - (void)dealloc [[NSNotificationCenter defaultCenter] removeObserver:self name:NSViewGlobalFrameDidChangeNotification object:self]; [self clearGLContext]; } - (void)update { if ([_openGLContext view] == self) { [_openGLContext update]; } } - (void) _surfaceNeedsUpdate:(NSNotification*)notification { [self update]; } @end Working with Rendering Contexts Update the Rendering Context When the Renderer or Geometry Changes 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 75Context Parameters Alter the Context’s Behavior A rendering context has a variety of parameters that you can set to suit the needs of your OpenGL drawing. Some of the most useful, and often overlooked, context parameters are discussed in thissection:swap interval, surface opacity, surface drawing order, and back-buffer size control. Each of the Apple-specific OpenGL APIs provides a routine forsetting and getting rendering context parameters: ● The setValues:forParameter: method of the NSOpenGLContext class takes as arguments a list of values and a list of parameters. ● The CGLSetParameter function takes as parameters a rendering context, a constant that specifies an option, and a value for that option. Some parameters need to be enabled for their values to take effect. The reference documentation for a parameter indicates whether a parameter needs to be enabled. See NSOpenGLContext Class Reference , and CGL Reference . Swap Interval Allows an Application to Synchronize Updatesto the Screen Refresh If the swap interval is set to 0 (the default), buffers are swapped as soon as possible, without regard to the vertical refresh rate of the monitor. If the swap interval is set to any other value, the buffers are swapped only during the vertical retrace of the monitor. For more information, see “Synchronize with the Screen Refresh Rate” (page 96). You can use the following constants to specify that you are setting the swap interval value: ● For Cocoa, use NSOpenGLCPSwapInterval. ● If you are using the CGL API, use kCGLCPSwapInterval as shown in Listing 7-2. Listing 7-2 Using CGL to set up synchronization GLint sync = 1; // ctx must be a valid context CGLSetParameter (ctx, kCGLCPSwapInterval, &sync); Working with Rendering Contexts Context Parameters Alter the Context’s Behavior 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 76Surface Opacity Specifies How the OpenGL Surface Blends with Surfaces Behind It OpenGL surfaces are typically rendered as opaque. Thus the background color for pixels with alpha values of 0.0 is the surface background color. If you set the value of the surface opacity parameter to 0, then the contents of the surface are blended with the contents of surfaces behind the OpenGL surface. This operation is equivalent to OpenGL blending with a source contribution proportional to the source alpha and a background contribution proportional to 1 minus the source alpha. A value of 1 means the surface is opaque (the default); 0 means completely transparent. You can use the following constants to specify that you are setting the surface opacity value: ● For Cocoa, use NSOpenGLCPSurfaceOpacity. ● If you are using the CGL API, use kCGLCPSurfaceOpacity as shown in Listing 7-3. Listing 7-3 Using CGL to set surface opacity GLint opaque = 0; // ctx must be a valid context CGLSetParameter (ctx, kCGLCPSurfaceOpacity, &opaque); Surface Drawing Order Specifies the Position of the OpenGL Surface Relative to the Window A value of 1 means that the position is above the window; a value of –1 specifies a position that is below the window. When you have overlapping views, setting the order to -1 causes OpenGL to draw underneath, 1 causes OpenGL to draw on top. This parameter is useful for drawing user interface controls on top of an OpenGL view. You can use the following constants to specify that you are setting the surface drawing order value: ● For Cocoa, use NSOpenGLCPSurfaceOrder. ● If you are using the CGL API, use kCGLCPSurfaceOrder as shown in Listing 7-4. Listing 7-4 Using CGL to set surface drawing order GLint order = –1; // below window // ctx must be a valid context CGLSetParameter (ctx, kCGLCPSurfaceOrder, &order); Working with Rendering Contexts Context Parameters Alter the Context’s Behavior 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 77Determining Whether Vertex and Fragment Processing Happens on the GPU CGL provides two parameters for checking whether the system is using the GPU for processing: kCGLCPGPUVertexProcessing and kCGLCPGPUFragmentProcessing. To check vertex processing, pass the vertex constant to the CGLGetParameter function. To check fragment processing, pass the fragment constant to CGLGetParameter. Listing 7-5 demonstrates how to use these parameters. Important: Although you can perform these queries at any time, keep in mind that such queries force an internal state validation, which can impact performance. For best performance, do not use these queries inside your drawing loop. Instead, perform the queries once at initialization or context setup time to determine whether OpenGL is using the CPU or the GPU for processing, and then act appropriately in your drawing loop. Listing 7-5 Using CGL to check whether the GPU is processing vertices and fragments BOOL gpuProcessing; GLint fragmentGPUProcessing, vertexGPUProcessing; CGLGetParameter (CGLGetCurrentContext(), kCGLCPGPUFragmentProcessing, &fragmentGPUProcessing); CGLGetParameter(CGLGetCurrentContext(), kCGLCPGPUVertexProcessing, &vertexGPUProcessing); gpuProcessing = (fragmentGPUProcessing && vertexGPUProcessing) ? YES : NO; Controlling the Back Buffer Size Normally, the back buffer is the same size as the window or view that it's drawn into, and it changes size when the window or view changes size. For a window whose size is 720×pixels, the OpenGL back buffer is sized to match. If the window grows to 1024×768 pixels, for example, then the back buffer is resized as well. If you do not want this behavior, use the back buffer size control parameter. Using this parameter fixes the size of the back buffer and lets the system scale the image automatically when it moves the data to a variable size buffer (see Figure 7-1). The size of the back buffer remains fixed at the size that you set up regardless of whether the image is resized to display larger onscreen. You can use the following constants to specify that you are setting the surface backing size: ● If you are using the CGL API, use kCGLCPSurfaceBackingSize, as shown in Listing 7-6. Working with Rendering Contexts Context Parameters Alter the Context’s Behavior 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 78Listing 7-6 Using CGL to set up back buffer size control GLint dim[2] = {720, 480}; // ctx must be a valid context CGLSetParameter(ctx, kCGLCPSurfaceBackingSize, dim); CGLEnable (ctx, kCGLCESurfaceBackingSize); Figure 7-1 A fixed size back buffer and variable size front buffer Sharing Rendering Context Resources A rendering context does not own the drawing objects attached to it, which leaves open the option forsharing. Rendering contexts can share resources and can be attached to the same drawable object (see Figure 7-2 (page 80)) or to different drawable objects (see Figure 7-3 (page 80)). You set up context sharing—either with more than one drawable object or with another context—at the time you create a rendering context. Contexts can share object resources and their associated object state by indicating a shared context at context creation time. Shared contexts share all texture objects, display lists, vertex programs, fragment programs, and buffer objects created before and after sharing is initiated. The state of the objects is also shared but not other Working with Rendering Contexts Sharing Rendering Context Resources 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 79contextstate,such as current color, texture coordinate settings, matrix and lighting settings, rasterization state, and texture environment settings. You need to duplicate context state changes as required, but you need to set up individual objects only once. Figure 7-2 Shared contexts attached to the same drawable object Context Context Drawable object Shared object state When you create an OpenGL context, you can designate another context whose object resources you want to share. Allsharing is peer to peer. Shared resources are reference-counted and thus are maintained until explicitly released or when the last context-sharing resource is released. Not every context can be shared with every other context. Both contexts must share the same OpenGL profile. You must also ensure that both contexts share the same set of renderers. You meet these requirements by ensuring each context uses the same virtual screen list, using either of the following techniques: ● Use the same pixel format object to create all the rendering contexts that you want to share. ● Create pixel format objects using attributes that narrow down the choice to a single display. This practice ensures that the virtual screen is identical for each pixel format object. Figure 7-3 Shared contexts and more than one drawable object Context Context Drawable object Drawable object Shared object state Setting up shared rendering contextsis very straightforward. Each Apple-specific OpenGL API providesfunctions with an option to specify a context to share in its context creation routine: ● Use the share argument for the initWithFormat:shareContext: method of the NSOpenGLContext class. See Listing 7-7 (page 81). ● Use the share parameter for the function CGLCreateContext. See Listing 7-8 (page 82). Working with Rendering Contexts Sharing Rendering Context Resources 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 80Listing 7-7 ensures the same virtual screen list by using the same pixel format object for each of the shared contexts. Listing 7-7 Setting up an NSOpenGLContext object for sharing #import + (NSOpenGLPixelFormat*)defaultPixelFormat { NSOpenGLPixelFormatAttribute attributes [] = { NSOpenGLPFADoubleBuffer, (NSOpenGLPixelFormatAttribute)nil }; return [(NSOpenGLPixelFormat *)[NSOpenGLPixelFormat alloc] initWithAttributes:attribs]; } - (NSOpenGLContext*)openGLContextWithShareContext:(NSOpenGLContext*)context { if (_openGLContext == NULL) { _openGLContext = [[NSOpenGLContext alloc] initWithFormat:[[self class] defaultPixelFormat] shareContext:context]; [_openGLContext makeCurrentContext]; [self prepareOpenGL]; } return _openGLContext; } - (void)prepareOpenGL { // Your code here to initialize the OpenGL state } Listing 7-8 ensures the same virtual screen list by using the same pixel format object for each of the shared contexts. Working with Rendering Contexts Sharing Rendering Context Resources 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 81Listing 7-8 Setting up a CGL context for sharing #include CGLPixelFormatAttribute attrib[] = {kCGLPFADoubleBuffer, 0}; CGLPixelFormatObj pixelFormat = NULL; Glint numPixelFormats = 0; CGLContextObj cglContext1 = NULL; CGLContextObj cglContext2 = NULL; CGLChoosePixelFormat (attribs, &pixelFormat, &numPixelFormats); CGLCreateContext(pixelFormat, NULL, &cglContext1); CGLCreateContext(pixelFormat, cglContext1, &cglContext2); Working with Rendering Contexts Sharing Rendering Context Resources 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 82One of the benefits of using OpenGL isthat it is extensible. An extension istypically introduced by one or more vendors and then later is accepted by the OpenGL Working Group. Some extensions are promoted from a vendor-specific extension to one shared by more than one vendor, sometimes even being incorporated into the core OpenGL API. Extensions allow OpenGL to embrace innovation, but require you to verify that the OpenGL functionality you want to use is available. Because extensions can be introduced at the vendor level, more than one extension can provide the same basic functionality. There might also be an ARB-approved extension that has functionality similar to that of a vendor-specific extension. Your application should prefer core functionality or ARB-approved extensions over those specific to a particular vendor, when both are offered by the same renderer. This makes it easier to transparently support new renderers from other vendors. As particular functionality becomes widely adopted, it can be moved into the core OpenGL API by the ARB. As a result, functionality that you want to use could be included as an extension, as part of the core API, or both. For example, the ability to combine texture environments is supported through the GL_ARB_texture_env_combine and the GL_EXT_texture_env_combine extensions. It's also part of the core OpenGL version 1.3 API. Although each has similar functionality, they use a different syntax. You may need to check in several places (core OpenGL API and extension strings) to determine whether a specific renderer supports functionality that you want to use. Detecting Functionality OpenGL hastwo types of commands—those that are part of the core API and those that are part of an extension to OpenGL. Your application first needs to check for the version of the core OpenGL API and then check for the available extensions. Keep in mind that OpenGL functionality is available on a per-renderer basis. For example, a software renderer might notsupport fog effects even though fog effects are available in an OpenGL extension implemented by a hardware vendor on the same system. For this reason, it's important that you check for functionality on a per-renderer basis. Regardless of what functionality you are checking for, the approach is the same. You need to call the OpenGL function glGetString twice. The first time pass the GL_VERSION constant. The function returns a string that specifies the version of OpenGL. The second time, pass the GL_EXTENSIONS constant. The function returns a pointer to an extension name string. The extension name string is a space-delimited list of the OpenGL 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 83 Determining the OpenGL Capabilities Supported by the Rendererextensions that are supported by the current renderer. This string can be rather long, so do not allocate a fixed-length string for the return value of the glGetString function. Use a pointer and evaluate the string in place. Pass the extension name string to the function gluCheckExtension along with the name of the extension you want to check for. The gluCheckExtension function returns a Boolean value that indicates whether or not the extension is available for the current renderer. If an extension becomes part of the core OpenGL API, OpenGL continues to export the name strings of the promoted extensions. It also continuesto support the previous versions of any extension that has been exported in earlier versions of OS X. Because extensions are not typically removed, the methodology you use today to check for a feature works in future versions of OS X. Checking for functionality, although fairly straightforward, involves writing a large chunk of code. The best way to check for OpenGL functionality is to implement a capability-checking function that you call when your program starts up, and then any time the renderer changes. Listing 8-1 shows a code excerpt that checks for a few extensions. A detailed explanation for each line of code appears following the listing. Listing 8-1 Checking for OpenGL functionality GLint maxRectTextureSize; GLint myMaxTextureUnits; GLint myMaxTextureSize; const GLubyte * strVersion; const GLubyte * strExt; float myGLVersion; GLboolean isVAO, isTexLOD, isColorTable, isFence, isShade, isTextureRectangle; strVersion = glGetString (GL_VERSION); // 1 sscanf((char *)strVersion, "%f", &myGLVersion); strExt = glGetString (GL_EXTENSIONS); // 2 glGetIntegerv(GL_MAX_TEXTURE_UNITS, &myMaxTextureUnits); // 3 glGetIntegerv(GL_MAX_TEXTURE_SIZE, &myMaxTextureSize); // 4 isVAO = gluCheckExtension ((const GLubyte*)"GL_APPLE_vertex_array_object",strExt); // 5 isFence = gluCheckExtension ((const GLubyte*)"GL_APPLE_fence", strExt); // 6 isShade = gluCheckExtension ((const GLubyte*)"GL_ARB_shading_language_100", strExt); // 7 Determining the OpenGL Capabilities Supported by the Renderer Detecting Functionality 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 84isColorTable = gluCheckExtension ((const GLubyte*)"GL_SGI_color_table", strExt) || gluCheckExtension ((const GLubyte*)"GL_ARB_imaging", strExt); // 8 isTexLOD = gluCheckExtension ((const GLubyte*)"GL_SGIS_texture_lod", strExt) || (myGLVersion >= 1.2); // 9 isTextureRectangle = gluCheckExtension ((const GLubyte*) "GL_EXT_texture_rectangle", strExt); if (isTextureRectangle) glGetIntegerv (GL_MAX_RECTANGLE_TEXTURE_SIZE_EXT, &maxRectTextureSize); else maxRectTextureSize = 0; // 10 Here is what the code does: 1. Gets a string that specifies the version of OpenGL. 2. Gets the extension name string. 3. Calls the OpenGL function glGetIntegerv to get the value of the attribute passed to it which, in this case, is the maximum number of texture units. 4. Gets the maximum texture size. 5. Checks whether vertex array objects are supported. 6. Checks for the Apple fence extension. 7. Checks for support for version 1.0 of the OpenGL shading language. 8. Checks for RGBA-format color lookup table support. In this case, the code needs to check for the vendor-specific string and for the ARB string. If either is present, the functionality is supported. 9. Checks for an extension related to the texture level of detail parameter (LOD). In this case, the code needs to check for the vendor-specific string and for the OpenGL version. If the vendor string is present or the OpenGL version is greater than or equal to 1.2, the functionality is supported. 10. Getsthe OpenGL limit for rectangle textures. Forsome extensions,such asthe rectangle texture extension, it may not be enough to check whether the functionality is supported. You may also need to check the limits. You can use glGetIntegerv and related functions (glGetBooleanv, glGetDoublev, glGetFloatv) to obtain a variety of parameter values. You can extend this example to make a comprehensive functionality-checking routine for your application. For more details, see the GLCheck.c file in the Cocoa OpenGL sample application. Determining the OpenGL Capabilities Supported by the Renderer Detecting Functionality 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 85The code in Listing 8-2 shows one way to query the current renderer. It uses the CGL API, which can be called from Cocoa applications. In reality, you need to iterate over all displays and all renderers for each display to get a true picture of the OpenGL functionality available on a particular system. You also need to update your functionality snapshot each time the list of displays or display configuration changes. Listing 8-2 Setting up a valid rendering context to get renderer functionality information #include #include CGDirectDisplayID display = CGMainDisplayID (); // 1 CGOpenGLDisplayMask myDisplayMask = CGDisplayIDToOpenGLDisplayMask (display); // 2 { // Check capabilities of display represented by display mask CGLPixelFormatAttribute attribs[] = {kCGLPFADisplayMask, myDisplayMask, 0}; // 3 CGLPixelFormatObj pixelFormat = NULL; GLint numPixelFormats = 0; CGLContextObj myCGLContext = 0; CGLContextObj curr_ctx = CGLGetCurrentContext (); // 4 CGLChoosePixelFormat (attribs, &pixelFormat, &numPixelFormats); // 5 if (pixelFormat) { CGLCreateContext (pixelFormat, NULL, &myCGLContext); // 6 CGLDestroyPixelFormat (pixelFormat); // 7 CGLSetCurrentContext (myCGLContext); // 8 if (myCGLContext) { // Check for capabilities and functionality here } } CGLDestroyContext (myCGLContext); // 9 CGLSetCurrentContext (curr_ctx); // 10 } Here's what the code does: 1. Gets the display ID of the main display. Determining the OpenGL Capabilities Supported by the Renderer Detecting Functionality 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 862. Maps a display ID to an OpenGL mask. 3. Fills a pixel format attributes array with the display mask attribute and the mask value. 4. Saves the current context so that it can be restored later. 5. Gets the pixel format object for the display. The numPixelFormats parameter specifies how many pixel formats are listed in the pixel format object. 6. Creates a context based on the first pixel format in the list supplied by the pixel format object. Only one renderer will be associated with this context. In your application, you would need to iterate through all pixel formats for this display. 7. Destroys the pixel format object when it is no longer needed. 8. Sets the current context to the newly created, single-renderer context. Now you are ready to check for the functionality supported by the current renderer. See Listing 8-1 (page 84) for an example of functionality-checking code. 9. Destroys the context because it is no longer needed. 10. Restores the previously saved context as the current context, thus ensuring no intrusion upon the user. Guidelines for Code That Checks for Functionality The guidelines in this section ensure that your functionality-checking code is thorough yet efficient. ● Don't rely on what's in a header file. A function declaration in a header file does not ensure that a feature is supported by the current renderer. Neither does linking against a stub library that exports a function. ● Make sure that a renderer is attached to a valid rendering context before you check the functionality of that renderer. ● Check the API version or the extension name string for the current renderer before you issue OpenGL commands. ● Check only once per renderer. After you've determined that the current renderer supports an OpenGL command, you don't need to check for that functionality again for that renderer. ● Make sure that you are aware of whether a feature is being used as part of the Core OpenGL API or as an extension. When a feature is implemented both as part of the core OpenGL API and as an extension, it uses different constants and function names. Determining the OpenGL Capabilities Supported by the Renderer Guidelines for Code That Checks for Functionality 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 87OpenGL Renderer Implementation-Dependent Values The OpenGL specification definesimplementation-dependent valuesthat define the limits of what an OpenGL implementation is capable of. For example, the maximum size of a texture and the number of texture units are both common implementation-dependent values that an application is expected to check. Each of these values provides a minimum value that all conforming OpenGL implementations are expected to support. If your application’s usage exceeds these minimums, it must check the limit first, and fail gracefully if the implementation cannot provide the limit desired. Your application may need to load smaller textures, disable a rendering feature, or choose a different implementation. Although the specification provides a comprehensive list of these limitations, a few stand out in most OpenGL applications. Table 8-1 lists values that applications should test if they require more than the minimum values in the specification. Table 8-1 Common OpenGL renderer limitations Maximum size of the texture GL_MAX_TEXTURE_SIZE Number of depth buffer planes GL_DEPTH_BITS Number of stencil buffer planes GL_STENCIL_BITS The limit on the size and complexity of your shaders is a key area you need to test. All graphics hardware supportslimited memory to pass attributesinto the vertex and fragmentshaders. Your application must either keep its usage below the minimums as defined in the specification, or it must check the shader limitations documented in Table 8-2 and choose shaders that are within those limits. Table 8-2 OpenGL shader limitations Maximum number of vertex attributes GL_MAX_VERTEX_ATTRIBS Maximum number of uniform vertex vectors GL_MAX_VERTEX_UNIFORM_COMPONENTS Maximum number of uniform fragment vectors GL_MAX_FRAGMENT_UNIFORM_COMPONENTS Maximum number of varying vectors GL_MAX_VARYING_FLOATS Maximum number of texture units usable in a GL_MAX_VERTEX_TEXTURE_IMAGE_UNITS vertex shader Maximum number of texture units usable in a GL_MAX_TEXTURE_IMAGE_UNITS fragment shader Determining the OpenGL Capabilities Supported by the Renderer OpenGL Renderer Implementation-Dependent Values 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 88OpenGL performs many complex operations—transformations, lighting, clipping, texturing, environmental effects, and so on—on large data sets. The size of your data and the complexity of the calculations performed on it can impact performance, making your stellar 3D graphics shine less brightly than you'd like. Whether your application is a game using OpenGL to provide immersive real-time images to the user or an image processing application more concerned with image quality, use the information in this chapter to help you design your application. Visualizing OpenGL The most common way to visualize OpenGL is as a graphics pipeline, as shown in Figure 9-1 (page 90). Your application sends vertex and image data, configuration and state changes, and rendering commandsto OpenGL. Vertices are processed, assembled into primitives, and rasterized into fragments. Each fragment is calculated and merged into the framebuffer. The pipeline model is useful for identifying exactly what work your application must perform to generate the results you want. OpenGL allows you to customize each stage of the graphics pipeline, either through customized shader programs or by configuring a fixed-function pipeline through OpenGL function calls. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 89 OpenGL Application Design StrategiesIn most implementations, each pipeline stage can act in parallel with the others. This is a key point. If any one pipeline stage performs too much work, then the other stages sit idle waiting for it to complete. Your design should balance the work performed in each pipeline stage to the capabilities of the renderer. When you tune your application’s performance, the firststep is usually to determine which stage the application is bottlenecked in, and why. Figure 9-1 OpenGL graphics pipeline Geometry Fragment Framebuffer operations Texturing Fog Alpha, stencil, and depth tests Framebuffer blending Primitive assembly Clipping Vertex Application Primitives and image data Transform and lighting Another way to visualize OpenGL is as a client-server architecture, as shown in Figure 9-2 (page 91). OpenGL state changes, texture and vertex data, and rendering commands must all travel from the application to the OpenGL client. The client transforms these items so that the graphics hardware can understand them, and then forwards them to the GPU. Not only do these transformations add overhead, but the bandwidth between the CPU and the graphics hardware is often lower than other parts of the system. OpenGL Application Design Strategies Visualizing OpenGL 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 90To achieve great performance, an application must reduce the frequency of callsthey make to OpenGL, minimize the transformation overhead, and carefully manage the flow of data between the application and the graphics hardware. For example, OpenGL provides mechanismsthat allow some kinds of data to be cached in dedicated graphics memory. Caching reusable data in graphics memory reduces the overhead of transmitting data to the graphics hardware. Figure 9-2 OpenGL client-server architecture OpenGL client OpenGL server Graphics hardware Application OpenGL framework OpenGL driver Runs on GPU Runs on CPU Designing a High-Performance OpenGL Application To summarize, a well-designed OpenGL application needs to: ● Exploit parallelism in the OpenGL pipeline. ● Manage data flow between the application and the graphics hardware. OpenGL Application Design Strategies Designing a High-Performance OpenGL Application 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 91Figure 9-3 shows a suggested process flow for an application that uses OpenGL to perform animation to the display. Figure 9-3 Application model for managing resources Update dynamic resources Execute rendering commands Read back results Present to display Free up resources Render loop Slower process Faster process Create static resources When the application launches, it creates and initializes any static resources it intends to use in the renderer, encapsulating those resources into OpenGL objects where possible. The goal is to create any object that can remain unchanged for the runtime of the application. Thistradesincreased initialization time for better rendering performance. Ideally, complex commands or batches ofstate changesshould be replaced with OpenGL objects that can be switched in with a single function call. For example, configuring the fixed-function pipeline can take dozens of function calls. Replace it with a graphics shader that is compiled at initialization time, and you can switch to a different program with a single function call. In particular, OpenGL objects that are expensive to create or modify should be created as static objects. The rendering loop processes all of the items you intend to render to the OpenGL context, then swaps the buffersto display the resultsto the user. In an animated scene,some data needsto be updated for every frame. In the inner rendering loop shown in Figure 9-3, the application alternates between updating rendering resources(possibly creating or modifying OpenGL objectsin the process) and submitting rendering commands that use those resources. The goal of this inner loop is to balance the workload so that the CPU and GPU are working in parallel, without blocking each other by using the same resources simultaneously. OpenGL Application Design Strategies Designing a High-Performance OpenGL Application 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 92A goal for the inner loop is to avoid copying data back from the graphics processor to the CPU. Operations that require the CPU to read results back from the graphics hardware are sometimes necessary, but in general reading back results should be used sparingly. If those results are also used to render the current frame, as shown in the middle rendering loop, this can be very slow. Copying data from the GPU to the CPU often requires that some or all previously submitted drawing commands have completed. After the application submits all drawing commands needed in the frame, it presents the results to the screen. Alternatively, a non-interactive application might read the final image back to the CPU, but this is also slower than presenting results to the screen. This step should be performed only for results that must be read back to the application. For example, you might copy the image in the back buffer to save it to disk. Finally, when your application is ready to shut down, it deletes static and dynamic resources to make more hardware resources available to other applications. If your application is moved to the background, releasing resources to other applications is also good practice. To summarize the important characteristics of this design: ● Create static resources, whenever practical. ● The inner rendering loop alternates between modifying dynamic resources and submitting rendering commands. Enough work should be included in this loop so that when the application needs to read or write to any OpenGL object, the graphics processor has finished processing any commands that used it. ● Avoid reading intermediate rendering results into the application. The rest of this chapter provides useful OpenGL programming techniques to implement the features of this rendering loop. Later chapters demonstrate how to apply these general techniquesto specific areas of OpenGL programming. ● “Update OpenGL Content Only When Your Data Changes” (page 94) ● “Avoid Synchronizing and Flushing Operations” (page 96) ● “Allow OpenGL to Manage Your Resources” (page 99) ● “Use Optimal Data Types and Formats” (page 102) ● “Use Double Buffering to Avoid Resource Conflicts” (page 100) ● “Be Mindful of OpenGL State Variables” (page 101) ● “Use OpenGL Macros” (page 103) ● “Replace State Changes with OpenGL Objects” (page 102) OpenGL Application Design Strategies Designing a High-Performance OpenGL Application 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 93Update OpenGL Content Only When Your Data Changes OpenGL applications should avoid recomputing a scene when the data has not changed. This is critical on portable devices, where power conservation is critical to maximizing battery life. You can ensure that your application draws only when necessary by following a few simple guidelines: ● If your application isrendering animation, use a Core Video display link to drive the animation loop. Listing 9-1 (page 94) provides code that allows your application to be notified when a new frame needs to be displayed. This code also synchronizes image updates to the refresh rate of the display. See “Synchronize with the Screen Refresh Rate” (page 96) for more information. ● If your application does not animate its OpenGL content, you should allow the system to regulate drawing. For example, in Cocoa call the setNeedsDisplay: method when your data changes. ● If your application does not use a Core Video display link, you should still advance an animation only when necessary. To determine when to draw the next frame of an animation, calculate the difference between the current time and the start of the last frame. Use the difference to determine how much to advance the animation. You can use the Core Foundation function CFAbsoluteTimeGetCurrent to obtain the current time. Listing 9-1 Setting up a Core Video display link @interface MyView : NSOpenGLView { CVDisplayLinkRef displayLink; //display link for managing rendering thread } @end - (void)prepareOpenGL { // Synchronize buffer swaps with vertical refresh rate GLint swapInt = 1; [[self openGLContext] setValues:&swapInt forParameter:NSOpenGLCPSwapInterval]; // Create a display link capable of being used with all active displays CVDisplayLinkCreateWithActiveCGDisplays(&displayLink); // Set the renderer output callback function CVDisplayLinkSetOutputCallback(displayLink, &MyDisplayLinkCallback, self); OpenGL Application Design Strategies Update OpenGL Content Only When Your Data Changes 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 94// Set the display link for the current renderer CGLContextObj cglContext = [[self openGLContext] CGLContextObj]; CGLPixelFormatObj cglPixelFormat = [[self pixelFormat] CGLPixelFormatObj]; CVDisplayLinkSetCurrentCGDisplayFromOpenGLContext(displayLink, cglContext, cglPixelFormat); // Activate the display link CVDisplayLinkStart(displayLink); } // This is the renderer output callback function static CVReturn MyDisplayLinkCallback(CVDisplayLinkRef displayLink, const CVTimeStamp* now, const CVTimeStamp* outputTime, CVOptionFlags flagsIn, CVOptionFlags* flagsOut, void* displayLinkContext) { CVReturn result = [(MyView*)displayLinkContext getFrameForTime:outputTime]; return result; } - (CVReturn)getFrameForTime:(const CVTimeStamp*)outputTime { // Add your drawing codes here return kCVReturnSuccess; } - (void)dealloc { // Release the display link CVDisplayLinkRelease(displayLink); [super dealloc]; } OpenGL Application Design Strategies Update OpenGL Content Only When Your Data Changes 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 95Synchronize with the Screen Refresh Rate Tearing is a visual anomaly caused when part of the current frame overwrites previous frame data in the framebuffer before the current frame is fully rendered on the screen. To avoid tearing, applications use a double-buffered context and synchronize buffer swaps with the screen refresh rate (sometimes called VBL , vertical blank , or vsynch ) to eliminate frame tearing. Note: During development, it's best to disable synchronization so that you can more accurately benchmark your application. Enable synchronization when you are ready to deploy your application. The refresh rate of the display limits how often the screen can be refreshed. The screen can be refreshed at rates that are divisible by integer values. For example, a CRT display that has a refresh rate of 60 Hz can support screen refresh rates of 60 Hz, 30 Hz, 20 Hz, and 15 Hz. LCD displays do not have a vertical retrace in the CRT sense and are typically considered to have a fixed refresh rate of 60 Hz. After you tell the context to swap the buffers, OpenGL must defer any rendering commands that follow that swap until after the buffers have successfully been exchanged. Applications that attempt to draw to the screen during this waiting period waste time that could be spent performing other drawing operations or saving battery life and minimizing fan operation. Listing 9-2 shows how an NSOpenGLView object can synchronize with the screen refresh rate; you can use a similar approach if your application uses CGL contexts. It assumes that you set up the context for double buffering. The swap interval can be set only to 0 or 1. If the swap interval is set to 1, the buffers are swapped only during the vertical retrace. Listing 9-2 Setting up synchronization GLint swapInterval = 1; [[self openGLContext] setValues:&swapInt forParameter:NSOpenGLCPSwapInterval]; Avoid Synchronizing and Flushing Operations OpenGL is not required to execute most commandsimmediately. Often, they are queued to a command buffer and read and executed by the hardware at a later time. Usually, OpenGL waits until the application has queued up a significant number of commands before sending the buffer to the hardware—allowing the graphics hardware to execute commands in batches is often more efficient. However, some OpenGL functions must flush the buffer immediately. Other functions not only flush the buffer, but also block until previously submitted commands have completed before returning control to the application. Your application should restrict the OpenGL Application Design Strategies Avoid Synchronizing and Flushing Operations 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 96use of flushing and synchronizing commands only to those cases where that behavior is necessary. Excessive use of flushing or synchronizing commands add additional stalls waiting for the hardware to finish rendering. On a single-buffered context, flushing may also cause visual anomalies, such as flickering or tearing. These situations require OpenGL to submit the command buffer to the hardware for execution. ● The function glFlush waits until commands are submitted but does not wait for the commands to finish executing. ● The function glFinish waits for all previously submitted commands to complete executing. ● Functions that retrieve OpenGL state (for example, glGetError), also wait for submitted commands to complete. ● Buffer swapping routines (the flushBuffer method of the NSOpenGLContext class or the CGLFlushDrawable function) implicitly call glFlush. Note that when using the NSOpenGLContext class or the CGL API, the term flush actually refers to a buffer-swapping operation. For single-buffered contexts, glFlush and glFinish are equivalent to a swap operation, since all rendering is taking place directly in the front buffer. ● The command buffer is full. Using glFlush Effectively Most of the time you don't need to call glFlush to move image data to the screen. There are only a few cases that require you to call the glFlush function: ● If your application submits rendering commands that use a particular OpenGL object, and it intends to modify that object in the near future. If you attempt to modify an OpenGL object that has pending drawing commands, your application may be forced to wait until those commands have been completed. In this situation, calling glFlush ensures that the hardware begins processing commands immediately. After flushing the command buffer, your application should perform work that does not need that resource. It can perform other work (even modifying other OpenGL objects). ● Your application needs to change the drawable object associated with the rendering context. Before you can switch to another drawable object, you must call glFlush to ensure that all commands written in the command queue for the previous drawable object have been submitted. ● When two contexts share an OpenGL object. After submitting any OpenGL commands, call glFlush before switching to the other context. ● To keep drawing synchronized across multiple threads and prevent command buffer corruption, each thread should submit its rendering commands and then call glFlush. OpenGL Application Design Strategies Avoid Synchronizing and Flushing Operations 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 97Avoid Querying OpenGL State Calls to glGet*(), including glGetError(), may require OpenGL to execute previous commands before retrieving any state variables. This synchronization forces the graphics hardware to run lockstep with the CPU, reducing opportunities for parallelism. Your application should keep shadow copies of any OpenGL state that you need to query, and maintain these shadow copies as you change the state. When errors occur, OpenGL sets an error flag that you can retrieve with the function glGetError. During development, it's crucial that your code contains error checking routines, not only for the standard OpenGL calls, but for the Apple-specific functions provided by the CGL API. If you are developing a performance-critical application, retrieve error information only in the debugging phase. Calling glGetError excessively in a release build degrades performance. Use Fences for Finer-Grained Synchronization Avoid using glFinish in your application, because it waits until all previously submitted commands are completed before returning control to your application. Instead, you should use the fence extension (APPLE_fence). This extension was created to provide the level of granularity that is not provided by glFinish. A fence is a token used to mark the current point in the command stream. When used correctly, it allows you to ensure that a specific series of commands has been completed. A fence helps coordinate activity between the CPU and the GPU when they are using the same resources. Follow these steps to set up and use a fence: 1. At initialization time, create the fence object by calling the function glGenFencesAPPLE. GLint myFence; glGenFencesAPPLE(1,&myFence); 2. Call the OpenGL functions that must complete prior to the fence. 3. Set up the fence by calling the function glSetFenceAPPLE. Thisfunction inserts a token into the command stream and sets the fence state to false. void glSetFenceAPPLE(GLuint fence); fence specifies the token to insert. For example: glSetFenceAPPLE(myFence); OpenGL Application Design Strategies Avoid Synchronizing and Flushing Operations 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 984. Call glFlush to force the commands to be sent to the hardware. This step is optional, but recommended to ensure that the hardware begins processing OpenGL commands. 5. Perform other work in your application. 6. Wait for all OpenGL commands issued prior to the fence to complete by calling the function glFinishFenceAPPLE. glFinishFenceAPPLE(myFence); As an alternative to calling glFinishFenceAPPLE, you can call glTestFenceAPPLE to determine whether the fence has been reached. The advantage of testing the fence is that your application does not block waiting for the fence to complete. This is useful if your application can continue processing other work while waiting for the fence to trigger. glTestFenceAPPLE(myFence); 7. When your application no longer needsthe fence, delete it by calling the function glDeleteFencesAPPLE. glDeleteFencesAPPLE(1,&myFence); There is an art to determining where to insert a fence in the command stream. If you insert a fence for too few drawing commands, you risk having your application stall while it waits for drawing to complete. You'll want to set a fence so your application operates as asynchronously as possible without stalling. The fence extension also lets you synchronize buffer updates for objects such as vertex arrays and textures. For that you call the function glFinishObjectAPPLE, supplying an object name along with the token. For detailed information on this extension, see the OpenGL specification for the Apple fence extension. Allow OpenGL to Manage Your Resources OpenGL allows many data types to be stored persistently inside OpenGL. Creating OpenGL objects to store vertex, texture, or other forms of data allows OpenGL to reduce the overhead of transforming the data and sending them to the graphics processor. If data is used more frequently than it is modified, OpenGL can substantially improve the performance of your application. OpenGL Application Design Strategies Allow OpenGL to Manage Your Resources 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 99OpenGL allows your application to hint how it intends to use the data. These hints allow OpenGL to make an informed choice of how to process your data. For example, static data might be placed in high-speed graphics memory directly connected to the graphics processor. Data that changes frequently might be kept in main memory and accessed by the graphics hardware through DMA. Use Double Buffering to Avoid Resource Conflicts Resource conflicts occur when your application and OpenGL want to access a resource at the same time. When one participant attempts to modify an OpenGL object being used by the other, one of two problems results: ● The participant that wantsto modify the object blocks until it is no longer in use. Then the other participant is not allowed to read from or write to the object until the modifications are complete. This is safe, but these can be hidden bottlenecks in your application. ● Some extensions allow OpenGL to access application memory that can be simultaneously accessed by the application. In this situation, synchronizing between the two participants is left to the application to manage. Your application calls glFlush to force OpenGL to execute commands and uses a fence or glFinish to ensure that no commands that access that memory are pending. Whether your application relies on OpenGL to synchronize access to a resource, or it manually synchronizes access, resource contention forces one of the participants to wait, rather than allowing them both to execute in parallel. Figure 9-4 demonstrates this problem. There is only a single buffer for vertex data, which both the application and OpenGL want to use and therefore the application must wait until the GPU finishes processing commands before it modifies the data. Figure 9-4 Single-buffered vertex array data CPU GPU Vertex array 1 Vertex array 1 Vertex array 1 Vertex array 1 glFlush glFlush glFinishObject(..., 1) glFinishObject(..., 1) Time Frame 1 Frame 2 OpenGL Application Design Strategies Use Double Buffering to Avoid Resource Conflicts 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 100To solve this problem, your application could fill this idle time with other processing, even other OpenGL processing that does not need the objects in question. If you need to process more OpenGL commands, the solution is to create two of the same resource type and let each participant access a resource. Figure 9-5 illustrates the double-buffered approach. While the GPU operates on one set of vertex array data, the CPU is modifying the other. After the initialstartup, neither processing unit isidle. This example uses a fence to ensure that access to each buffer is synchronized. Figure 9-5 Double-buffered vertex array data CPU Vertex array 1 Vertex array 1 GPU Vertex array 1 Vertex array 1 Vertex array 2 Vertex array 2 Vertex array 2 Vertex array 2 glFlush glFlush glFlush glFlush glFinishObject(..., 1) glFinishObject(..., 1) glFinishObject(..., 2) glFinishObject(..., 2) Time Frame 1 Frame 2 Frame 3 Frame 4 Double buffering issufficient for most applications, but it requiresthat both participantsfinish processing their commands before a swap can occur. For a traditional producer-consumer problem, more than two buffers may prevent a participant from blocking. With triple buffering, the producer and consumer each have a buffer, with a third idle buffer. If the producer finishes before the consumer finishes processing commands, it takes the idle buffer and continues to process commands. In this situation, the producer idles only if the consumer falls badly behind. Be Mindful of OpenGL State Variables The hardware has one current state, which is compiled and cached. Switching state is expensive, so it's best to design your application to minimize state switches. Don't set a state that's already set. Once a feature is enabled, it does not need to be enabled again. Calling an enable function more than once does nothing except waste time because OpenGL does not check the state of a feature when you call glEnable or glDisable. For instance, if you call glEnable(GL_LIGHTING) more than once, OpenGL does not check to see if the lighting state is already enabled. It simply updates the state value even if that value is identical to the current value. OpenGL Application Design Strategies Be Mindful of OpenGL State Variables 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 101You can avoid setting a state more than necessary by using dedicated setup or shutdown routines rather than putting such callsin a drawing loop. Setup and shutdown routines are also useful for turning on and off features that achieve a specific visual effect—for example, when drawing a wire-frame outline around a textured polygon. If you are drawing 2D images, disable all irrelevant state variables, similar to what's shown in Listing 9-3. Listing 9-3 Disabling state variables glDisable(GL_DITHER); glDisable(GL_ALPHA_TEST); glDisable(GL_BLEND); glDisable(GL_STENCIL_TEST); glDisable(GL_FOG); glDisable(GL_TEXTURE_2D); glDisable(GL_DEPTH_TEST); glPixelZoom(1.0,1.0); // Disable other state variables as appropriate. Replace State Changes with OpenGL Objects The “Be Mindful of OpenGL State Variables” (page 101) section suggests that reducing the number of state changes can improve performance. Some OpenGL extensions also allow you to create objects that collect multiple OpenGL state changes into an object that can be bound with a single function call. Where such techniques are available, they are recommended. For example, configuring the fixed-function pipeline requires many function calls to change the state of the various operators. Not only does this incur overhead for each function called, but the code is more complex and difficult to manage. Instead, use a shader. A shader, once compiled, can have the same effect but requires only a single call to glUseProgram. Other examples of objects that take the place of multiple state changes include the “Vertex Array Range Extension” (page 113) and “Uniform Buffers” (page 143). Use Optimal Data Types and Formats If you don't use data types and formats that are native to the graphics hardware, OpenGL must convert those data types into a format that the graphics hardware understands. OpenGL Application Design Strategies Replace State Changes with OpenGL Objects 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 102For vertex data, use GLfloat, GLshort, or GLubyte data types. Most graphics hardware handle these types natively. For texture data, you’ll get the best performance if you use the following format and data type combination: GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV These format and data type combinations also provide acceptable performance: GL_BGRA, GL_UNSIGNED_SHORT_1_5_5_5_REV GL_YCBCR_422_APPLE, GL_UNSIGNED_SHORT_8_8_REV_APPLE The combination GL_RGBA and GL_UNSIGNED_BYTE needs to be swizzled by many cards when the data is loaded, so it's not recommended. Use OpenGL Macros OpenGL performs a global context and renderer lookup for each command it executesto ensure that all OpenGL commands are issued to the correct rendering context and renderer. There is significant overhead associated with these lookups; applicationsthat have extremely high call frequenciesmay find that the overheadmeasurably affects performance. OS X allows your application to use macros to provide a local context variable and cache the current renderer in that variable. You get more benefit from using macros when your code makes millions of function calls per second. Before implementing this technique, consider carefully whether you can redesign your application to perform less function calls. Frequently changing OpenGL state, pushing or popping matrices, or even submitting one vertex at a time are all examples of techniques that should be replaced with more efficient operations. You can use the CGL macro header (CGL/CGLMacro.h) if your application uses CGL from a Cocoa application. You must define the local variable cgl_ctx to be equal to the current context. Listing 9-4 shows what's needed to set up macro use for the CGL API. First, you need to include the correct macro header. Then, you must set the current context. Listing 9-4 Using CGL macros #include // include the header CGL_MACRO_DECLARE_VARIABLES // set the current context glBegin (GL_QUADS); // This code now uses the macro // draw here glEnd (); OpenGL Application Design Strategies Use OpenGL Macros 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 103Complex shapes and detailed 3D models require large amounts of vertex data to describe them in OpenGL. Moving vertex data from your application to the graphics hardware incurs a performance cost that can be quite large depending on the size of the data set. Figure 10-1 Vertex data sets can be quite large Applications that use large vertex data sets can adopt one or more of the strategies described in “OpenGL Application Design Strategies” (page 89) to optimize how vertex data is delivered to OpenGL.This chapter expands on those best practices with specific techniques for working with vertex data. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 104 Best Practices for Working with Vertex DataUnderstand How Vertex Data Flows Through OpenGL Understanding how vertex data flows through OpenGL is important to choosing strategies for handling the data. Vertex data enters into the vertex stage, where it is processed by either the built-in fixed function vertex stage or a custom vertex. Figure 10-2 Vertex data path Rasterization Fragment shading and per-fragment operations Per-pixel operations Texture assembly Framebuffer Vertex shading and per-vertex operations Pixel data Vertex data Figure 10-3 takes a closer look at the vertex data path when using immediate mode. Without any optimizations, your vertex data may be copied at various points in the data path. If your application uses immediate mode to each vertex separately, calls to OpenGL first modify the current vertex, which is copied into the command buffer whenever your application makes a glVertex* call. Thisis not only expensive in terms of copy operations, but also in function overhead to specify each vertex. Figure 10-3 Immediate mode requires a copy of the current vertex data GPU VRAM Copy Copy Original Command buffer Current vertex Application Best Practices for Working with Vertex Data Understand How Vertex Data Flows Through OpenGL 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 105The OpenGL commands glDrawRangeElements, glDrawElements, and glDrawArrays render multiple geometric primitives from array data, using very few subroutine calls. Listing 10-1 shows a typical implementation. Your application creates a vertex structure that holds all the elements for each vertex. For each element , you enable a client array and provide a pointer and offset to OpenGL so that it knows how to find those elements. Listing 10-1 Submitting vertex data using glDrawElements. typedef struct _vertexStruct { GLfloat position[2]; GLubyte color[4]; } vertexStruct; void DrawGeometry() { const vertexStruct vertices[] = {...}; const GLubyte indices[] = {...}; glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(2, GL_FLOAT, sizeof(vertexStruct), &vertices[0].position); glEnableClientState(GL_COLOR_ARRAY); glColorPointer(4, GL_UNSIGNED_BYTE, sizeof(vertexStruct), &vertices[0].color); glDrawElements(GL_TRIANGLE_STRIP, sizeof(indices)/sizeof(GLubyte), GL_UNSIGNED_BYTE, indices); } Each time you call glDrawElements, OpenGL must copy all of the vertex data into the command buffer, which is later copied to the hardware. The copy overhead is still expensive. Best Practices for Working with Vertex Data Understand How Vertex Data Flows Through OpenGL 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 106Techniques for Handling Vertex Data Avoiding unnecessary copies of your vertex data is critical to application performance. Thissection summarizes common techniques for managing your vertex data using either built-in functionality or OpenGL extensions. Before using these techniques, you must ensure that the necessary functions are available to your application. See “Detecting Functionality” (page 83). ● Avoid the use of glBegin and glEnd to specify your vertex data. The function and copying overhead makes this path useful only for very small data sets. Also, applications written with glBegin and glEnd are not portable to OpenGL ES on iOS. ● Minimize data type conversions by supplying OpenGL data types for vertex data. Use GLfloat, GLshort, or GLubyte data types because most graphics processors handle these types natively. If you use some other type, then OpenGL may need to perform a costly data conversion. ● The preferred way to manage your vertex data is with vertex buffer objects. Vertex buffer objects are buffers owned by OpenGL that hold your vertex information. These buffers allow OpenGL to place your vertex data into memory that is accessible to the graphics hardware. See “Vertex Buffers” (page 107) for more information. ● If vertex buffer objects are not available, your application can search for the GL_APPLE_vertex_array_range and APPLE_fence extensions. Vertex array ranges allow you to prevent OpenGL from copying your vertex data into the command buffer. Instead, your application must avoid modifying or deleting the vertex data until OpenGL finishes executing drawing commands. This solution requires more effort from the application, and is not compatible with other platforms, including iOS. See “Vertex Array Range Extension” (page 113) for more information. ● Complex vertex operations require many array pointers to be enabled and set before you call glDrawElements. The GL_APPLE_vertex_array_object extension allows your application to consolidate a group of array pointers into a single object. Your application switches multiple pointers by binding a single vertex array object, reducing the overhead of changing state. See “Vertex Array Object” (page 116). ● Use double buffering to reduce resource contention between your application and OpenGL. See “Use Double Buffering to Avoid Resource Conflicts” (page 100). ● If you need to compute new vertex information between frames, consider using vertex shaders and buffer objects to perform and store the calculations. Vertex Buffers Vertex buffers are available as a core feature starting in OpenGL 1.5, and on earlier versions of OpenGL through the vertex buffer object extension (GL_ARB_vertex_buffer_object). Vertex buffers are used to improve the throughput of static or dynamic vertex data in your application. Best Practices for Working with Vertex Data Techniques for Handling Vertex Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 107A buffer object is a chunk of memory owned by OpenGL. Your application reads from or writes to the buffer using OpenGL callssuch as glBufferData, glBufferSubData, and glGetBufferSubData. Your application can also gain a pointer to this memory, an operation referred to as mapping a buffer. OpenGL prevents your application and itself from simultaneously using the data stored in the buffer. When your application maps a buffer or attempts to modify it, OpenGL may block until previous drawing commands have completed. Using Vertex Buffers You can set up and use vertex buffers by following these steps: 1. Call the function glGenBuffers to create a new name for a buffer object. void glGenBuffers(sizei n, uint *buffers ); n is the number of buffers you wish to create identifiers for. buffers specifies a pointer to memory to store the buffer names. 2. Call the function glBindBuffer to bind an unused name to a buffer object. After this call, the newly created buffer object is initialized with a memory buffer of size zero and a default state. (For the default setting, see the OpenGL specification for ARB_vertex_buffer_object.) void glBindBuffer(GLenum target, GLuint buffer); target must be set to GL_ARRAY_BUFFER. buffer specifies the unique name for the buffer object. 3. Fill the buffer object by calling the function glBufferData. Essentially, this call uploads your data to the GPU. void glBufferData(GLenum target, sizeiptr size, const GLvoid *data, GLenum usage); target must be set to GL_ARRAY_BUFFER. size specifies the size of the data store. *data points to the source data. If this is not NULL, the source data is copied to the data stored of the buffer object. If NULL, the contents of the data store are undefined. Best Practices for Working with Vertex Data Vertex Buffers 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 108usage is a constant that provides a hint as to how your application plans to use the data stored in the buffer object. These examples use GL_STREAM_DRAW, which indicates that the application plans to both modify and draw using the buffer, and GL_STATIC_DRAW, which indicates that the application will define the data once but use it to draw many times. For more details on buffer hints,see “Buffer Usage Hints” (page 110) 4. Enable the vertex array by calling glEnableClientState and supplying the GL_VERTEX_ARRAY constant. 5. Point to the contents of the vertex buffer object by calling a function such as glVertexPointer. Instead of providing a pointer, you provide an offset into the vertex buffer object. 6. To update the data in the buffer object, your application calls glMapBuffer. Mapping the buffer prevents the GPU from operating on the data, and gives your application a pointer to memory it can use to update the buffer. void *glMapBuffer(GLenum target, GLenum access); target must be set to GL_ARRAY_BUFFER. access indicatesthe operations you plan to performon the data. You can supply READ_ONLY, WRITE_ONLY, or READ_WRITE. 7. Write pixel data to the pointer received from the call to glMapBuffer. 8. When your application hasfinished modifying the buffer contents, call the function glUnmapBuffer. You must supply GL_ARRAY_BUFFER as the parameter to this function. Once the buffer is unmapped, the pointer is no longer valid, and the buffer’s contents are uploaded again to the GPU. Listing 10-2 shows code that usesthe vertex buffer object extension for dynamic data. This example overwrites all of the vertex data during every draw operation. Listing 10-2 Using the vertex buffer object extension with dynamic data // To set up the vertex buffer object extension #define BUFFER_OFFSET(i) ((char*)NULL + (i)) glBindBuffer(GL_ARRAY_BUFFER, myBufferName); glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(3, GL_FLOAT, stride, BUFFER_OFFSET(0)); // When you want to draw using the vertex data draw_loop { Best Practices for Working with Vertex Data Vertex Buffers 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 109glBufferData(GL_ARRAY_BUFFER, bufferSize, NULL, GL_STREAM_DRAW); my_vertex_pointer = glMapBuffer(GL_ARRAY_BUFFER, GL_WRITE_ONLY); GenerateMyDynamicVertexData(my_vertex_pointer); glUnmapBuffer(GL_ARRAY_BUFFER); PerformDrawing(); } Listing 10-3 shows code that uses the vertex buffer object extension with static data. Listing 10-3 Using the vertex buffer object extension with static data // To set up the vertex buffer object extension #define BUFFER_OFFSET(i) ((char*)NULL + (i)) glBindBuffer(GL_ARRAY_BUFFER, myBufferName); glBufferData(GL_ARRAY_BUFFER, bufferSize, NULL, GL_STATIC_DRAW); GLvoid* my_vertex_pointer = glMapBuffer(GL_ARRAY_BUFFER, GL_WRITE_ONLY); GenerateMyStaticVertexData(my_vertex_pointer); glUnmapBuffer(GL_ARRAY_BUFFER); glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(3, GL_FLOAT, stride, BUFFER_OFFSET(0)); // When you want to draw using the vertex data draw_loop { PerformDrawing(); } Buffer Usage Hints A key advantage of buffer objectsisthat the application can provide information on how it usesthe data stored in each buffer. For example, Listing 10-2 and Listing 10-3 differentiated between cases where the data were expected to never change (GL_STATIC_DRAW) and cases where the buffer data might change (GL_DYNAMIC_DRAW). The usage parameter allows an OpenGL renderer to alter its strategy for allocating the vertex buffer to improve performance. For example, static buffers may be allocated directly in GPU memory, while dynamic buffers may be stored in main memory and retrieved by the GPU via DMA. If OpenGL ES compatibility is useful to you, you should limit your usage hints to one of three usage cases: Best Practices for Working with Vertex Data Vertex Buffers 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 110● GL_STATIC_DRAW should be used for vertex data that isspecified once and never changed. Your application should create these vertex buffers during initialization and use them repeatedly until your application shuts down. ● GL_DYNAMIC_DRAW should be used when the buffer is expected to change after it is created. Your application should still allocate these buffers during initialization and periodically update them by mapping the buffer. ● GL_STREAM_DRAW is used when your application needs to create transient geometry that is rendered and then discarded. This is most useful when your application must dynamically change vertex data every frame in a way that cannot be performed in a vertex shader. To use a stream vertex buffer, your application initially fills the buffer using glBufferData, then alternates between drawing using the buffer and modifying the buffer. Other usage constants are detailed in the vertex buffer specification. If different elements in your vertex format have different usage characteristics, you may want to split the elements into one structure for each usage pattern and allocate a vertex buffer for each. Listing 10-4 shows how to implement this. In this example, position data is expected to be the same in each frame, while color data may be animated in every frame. Listing 10-4 Geometry with different usage patterns typedef struct _vertexStatic { GLfloat position[2]; } vertexStatic; typedef struct _vertexDynamic { GLubyte color[4]; } vertexDynamic; // Separate buffers for static and dynamic data. GLuint staticBuffer; GLuint dynamicBuffer; GLuint indexBuffer; const vertexStatic staticVertexData[] = {...}; vertexDynamic dynamicVertexData[] = {...}; Best Practices for Working with Vertex Data Vertex Buffers 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 111const GLubyte indices[] = {...}; void CreateBuffers() { glGenBuffers(1, &staticBuffer); glGenBuffers(1, &dynamicBuffer); glGenBuffers(1, &indexBuffer); // Static position data glBindBuffer(GL_ARRAY_BUFFER, staticBuffer); glBufferData(GL_ARRAY_BUFFER, sizeof(staticVertexData), staticVertexData, GL_STATIC_DRAW); glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer); glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(indices), indices, GL_STATIC_DRAW); // Dynamic color data // While not shown here, the expectation is that the data in this buffer changes between frames. glBindBuffer(GL_ARRAY_BUFFER, dynamicBuffer); glBufferData(GL_ARRAY_BUFFER, sizeof(dynamicVertexData), dynamicVertexData, GL_DYNAMIC_DRAW); } void DrawUsingVertexBuffers() { glBindBuffer(GL_ARRAY_BUFFER, staticBuffer); glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(2, GL_FLOAT, sizeof(vertexStatic), (void*)offsetof(vertexStatic,position)); glBindBuffer(GL_ARRAY_BUFFER, dynamicBuffer); glEnableClientState(GL_COLOR_ARRAY); glColorPointer(4, GL_UNSIGNED_BYTE, sizeof(vertexDynamic), (void*)offsetof(vertexDynamic,color)); glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer); Best Practices for Working with Vertex Data Vertex Buffers 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 112glDrawElements(GL_TRIANGLE_STRIP, sizeof(indices)/sizeof(GLubyte), GL_UNSIGNED_BYTE, (void*)0); } Flush Buffer Range Extension When your application unmaps a vertex buffer, the OpenGL implementation may copy the full contents of the buffer to the graphics hardware. If your application changes only a subset of a large buffer, this is inefficient. The APPLE_flush_buffer_range extension allows your application to tell OpenGL exactly which portions of the buffer were modified, allowing it to send only the changed data to the graphics hardware. To use the flush buffer range extension, follow these steps: 1. Turn on the flush buffer extension by calling glBufferParameteriAPPLE. glBufferParameteriAPPLE(GL_ARRAY_BUFFER,GL_BUFFER_FLUSHING_UNMAP_APPLE, GL_FALSE); This disables the normal flushing behavior of OpenGL. 2. Before you unmap a buffer, you must call glFlushMappedBufferRangeAPPLE for each range of the buffer that was modified by the application. void glFlushMappedBufferRangeAPPLE(enum target, intptr offset, sizeiptr size); target is the type of buffer being modified; for vertex data it’s ARRAY_BUFFER. offset is the offset into the buffer for the modified data. size is the length of the modified data in bytes. 3. Call glUnmapBuffer. OpenGL unmaps the buffer, but it is required to update only the portions of the buffer your application explicitly marked as changed. For more information see the APPLE_flush_buffer_range specification. Vertex Array Range Extension The vertex array range extension (APPLE_vertex_array_range) lets you define a region of memory for your vertex data. The OpenGL driver can optimize memory usage by creating a single memory mapping for your vertex data. You can also provide a hint as to how the data should be stored: cached or shared. The cached Best Practices for Working with Vertex Data Vertex Array Range Extension 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 113option specifiesto cache vertex data in video memory. The shared option indicatesthat data should be mapped into a region of memory that allows the GPU to access the vertex data directly using DMA transfer. This option is best for dynamic data. If you use shared memory, you'll need to double buffer your data. You can set up and use the vertex array range extension by following these steps: 1. Enable the extension by calling glEnableClientState and supplying the GL_VERTEX_ARRAY_RANGE_APPLE constant. 2. Allocate storage for the vertex data. You are responsible for maintaining storage for the data. 3. Define an array of vertex data by calling a function such as glVertexPointer. You need to supply a pointer to your data. 4. Optionally set up a hint about handling the storage of the array data by calling the function glVertexArrayParameteriAPPLE. GLvoid glVertexArrayParameteriAPPLE(GLenum pname, GLint param); pname must be VERTEX_ARRAY_STORAGE_HINT_APPLE. param is a hint that specifies how your application expects to use the data. OpenGL uses this hint to optimize performance. You can supply either STORAGE_SHARED_APPLE or STORAGE_CACHED_APPLE. The default value is STORAGE_SHARED_APPLE, which indicates that the vertex data is dynamic and that OpenGL should use optimization and flushing techniques suitable for this kind of data. If you expect the supplied data to be static, use STORAGE_CACHED_APPLE so that OpenGL can optimize appropriately. 5. Call the OpenGL function glVertexArrayRangeAPPLE to establish the data set. void glVertexArrayRangeAPPLE(GLsizei length, GLvoid *pointer); length specifies the length of the vertex array range. The length is typically the number of unsigned bytes. *pointer points to the base of the vertex array range. 6. Draw with the vertex data using standard OpenGL vertex array commands. 7. If you need to modify the vertex data,set a fence object after you’ve submitted all the drawing commands. See “Use Fences for Finer-Grained Synchronization” (page 98) 8. Perform other work so that the GPU has time to process the drawing commands that use the vertex array. 9. Call glFinishFenceAPPLE to gain access to the vertex array. 10. Modify the data in the vertex array. 11. Call glFlushVertexArrayRangeAPPLE. Best Practices for Working with Vertex Data Vertex Array Range Extension 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 114void glFlushVertexArrayRangeAPPLE(GLsizei length, GLvoid *pointer); length specifies the length of the vertex array range, in bytes. *pointer points to the base of the vertex array range. For dynamic data, each time you change the data, you need to maintain synchronicity by calling glFlushVertexArrayRangeAPPLE. You supply as parameters an array size and a pointer to an array, which can be a subset of the data, as long as it includes all of the data that changed. Contrary to the name of the function, glFlushVertexArrayRangeAPPLE doesn't actually flush data like the OpenGL function glFlush does. It simply makes OpenGL aware that the data has changed. Listing 10-5 shows code thatsets up and usesthe vertex array range extension with dynamic data. It overwrites all of the vertex data during each iteration through the drawing loop. The call to the glFinishFenceAPPLE command guaranteesthat the CPU and the GPU don't accessthe data at the same time. Although this example calls the glFinishFenceAPPLE function almost immediately after setting the fence, in reality you need to separate these calls to allow parallel operation of the GPU and CPU. To see how that's done, read “Use Double Buffering to Avoid Resource Conflicts” (page 100). Listing 10-5 Using the vertex array range extension with dynamic data // To set up the vertex array range extension glVertexArrayParameteriAPPLE(GL_VERTEX_ARRAY_STORAGE_HINT_APPLE, GL_STORAGE_SHARED_APPLE); glVertexArrayRangeAPPLE(buffer_size, my_vertex_pointer); glEnableClientState(GL_VERTEX_ARRAY_RANGE_APPLE); glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(3, GL_FLOAT, 0, my_vertex_pointer); glSetFenceAPPLE(my_fence); // When you want to draw using the vertex data draw_loop { glFinishFenceAPPLE(my_fence); GenerateMyDynamicVertexData(my_vertex_pointer); glFlushVertexArrayRangeAPPLE(buffer_size, my_vertex_pointer); PerformDrawing(); glSetFenceAPPLE(my_fence); } Best Practices for Working with Vertex Data Vertex Array Range Extension 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 115Listing 10-6 shows code that usesthe vertex array range extension with static data. Unlike the setup for dynamic data, the setup forstatic data includes using the hint for cached data. Because the data isstatic, it's unnecessary to set a fence. Listing 10-6 Using the vertex array range extension with static data // To set up the vertex array range extension GenerateMyStaticVertexData(my_vertex_pointer); glVertexArrayParameteriAPPLE(GL_VERTEX_ARRAY_STORAGE_HINT_APPLE, GL_STORAGE_CACHED_APPLE); glVertexArrayRangeAPPLE(array_size, my_vertex_pointer); glEnableClientState(GL_VERTEX_ARRAY_RANGE_APPLE); glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(3, GL_FLOAT, stride, my_vertex_pointer); // When you want to draw using the vertex data draw_loop { PerformDrawing(); } For detailed information on this extension, see the OpenGL specification for the vertex array range extension. Vertex Array Object Look at the DrawUsingVertexBuffers function in Listing 10-4 (page 111). It configures buffer pointers for position, color, and indexing before calling glDrawElements. A more complex vertex structure may require additional buffer pointers to be enabled and changed before you can finally draw your geometry. If your application swaps frequently between multiple configurations of elements, changing these parameters adds significant overhead to your application. The APPLE_vertex_array_object extension allows you to combine a collection of buffer pointers into a single OpenGL object, allowing you to change all the buffer pointers by binding a different vertex array object. To use this extension, follow these steps during your application’s initialization routines: 1. Generate a vertex array object for a configuration of pointers you wish to use together. Best Practices for Working with Vertex Data Vertex Array Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 116void glGenVertexArraysAPPLE(sizei n, const uint *arrays); n is the number of arrays you wish to create identifiers for. arrays specifies a pointer to memory to store the array names. glGenVertexArraysAPPLE(1,&myArrayObject); 2. Bind the vertex array object you want to configure. void glBindVertexArrayAPPLE(uint array); array is the identifier for an array that you received from glGenVertexArraysAPPLE. glBindVertexArrayAPPLE(myArrayObject); 3. Call the pointer routines (glColorPointer and so forth.) that you would normally call inside your rendering loop. When a vertex array object is bound, these calls change the currently bound vertex array object instead of the default OpenGL state. glBindBuffer(GL_ARRAY_BUFFER, staticBuffer); glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(2, GL_FLOAT, sizeof(vertexStatic), (void*)offsetof(vertexStatic,position)); ... 4. Repeat the previous steps for each configuration of vertex pointers. 5. Inside your rendering loop, replace the calls to configure the array pointers with a call to bind the vertex array object. glBindVertexArrayAPPLE(myArrayObject); glDrawArrays(...); 6. If you need to get back to the default OpenGL behavior, call glBindVertexArrayAPPLE and pass in 0. glBindVertexArrayAPPLE(0); Best Practices for Working with Vertex Data Vertex Array Object 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 117Textures add realism to OpenGL objects. They help objects defined by vertex data take on the material properties of real-world objects, such as wood, brick, metal, and fur. Texture data can originate from many sources, including images. Many of the same techniques your application uses on vertex data can also be used to improve texture performance. Figure 11-1 Textures add realism to a scene 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 118 Best Practices for Working with Texture DataTextures start as pixel data that flows through an OpenGL program, as shown in Figure 11-2. Figure 11-2 Texture data path Rasterization Fragment shading and per-fragment operations Per-pixel operations Texture assembly Framebuffer Vertex shading and per-vertex operations Pixel data Vertex data The precise route that texture data takesfrom your application to itsfinal destination can impact the performance of your application. The purpose of this chapter is to provide techniques you can use to ensure optimal processing of texture data in your application. This chapter ● shows how to use OpenGL extensions to optimize performance ● lists optimal data formats and types ● provides information on working with textures whose dimensions are not a power of two ● describes creating textures from image data ● shows how to download textures ● discusses using double buffers for texture data Using Extensions to Improve Texture Performance Without any optimizations, texture data flows through an OpenGL program as shown in Figure 11-3. Data from your application first goes to the OpenGL framework, which may make a copy of the data before handing it to the driver. If your data is not in a native format for the hardware (see “Optimal Data Formats and Types” (page Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 119128)), the driver may also make a copy of the data to convert it to a hardware-specific format for uploading to video memory. Video memory, in turn, can keep a copy of the data. Theoretically, there could be four copies of your texture data throughout the system. Figure 11-3 Data copies in an OpenGL program GPU VRAM OpenGL driver OpenGL framework Application Data flows at different rates through the system, as shown by the size of the arrows in Figure 11-3. The fastest data transfer happens between VRAM and the GPU. The slowest transfer occurs between the OpenGL driver and VRAM. Data moves between the application and the OpenGL framework, and between the framework and the driver at the same "medium" rate. Eliminating any of the data transfers, but the slowest one in particular, will improve application performance. There are several extensions you can use to eliminate one or more data copies and control how texture data travels from your application to the GPU: ● GL_ARB_pixel_buffer_object allows your application to use OpenGL buffer objectsto manage texture and image data. As with vertex buffer objects, they allow your application to hint how a buffer is used and to decide when data is copied to OpenGL. ● GL_APPLE_client_storage allows you to prevent OpenGL from copying your texture data into the client. Instead, OpenGL keepsthe memory pointer you provided when creating the texture. Your application must keep the texture data at that location until the referencing OpenGL texture is deleted. ● GL_APPLE_texture_range, along with a storage hint, either GL_STORAGE_CACHED_APPLE or GL_STORAGE_SHARED_APPLE, allows you to specify a single block of texture memory and manage it as you see fit. ● GL_ARB_texture_rectangle provides support for non-power of-two textures. Here are some recommendations: Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 120● If your application requires optimal texture upload performance, use GL_APPLE_client_storage and GL_APPLE_texture_range together to manage your textures. ● If your application requires optimal texture download performance, use pixel buffer objects. ● If your application requires cross-platform techniques, use pixel buffer objects for both texture uploads and texture downloads. ● Use GL_ARB_texture_rectangle when your source images are not aligned to a power-of-2 size. The sections that follow describe the extensions and show how to use them. Pixel Buffer Objects Pixel buffer objects are a core feature of OpenGL 2.1 and also available through the GL_ARB_pixel_buffer_object extension. The procedure for setting up a pixel buffer object is almost identical to that of vertex buffer objects. Using Pixel Buffer Objects to Efficiently Load Textures 1. Call the function glGenBuffers to create a new name for a buffer object. void glGenBuffers(sizei n, uint *buffers ); n is the number of buffers you wish to create identifiers for. buffers specifies a pointer to memory to store the buffer names. 2. Call the function glBindBuffer to bind an unused name to a buffer object. After this call, the newly created buffer object is initialized with a memory buffer of size zero and a default state. (For the default setting, see the OpenGL specification for ARB_vertex_buffer_object.) void glBindBuffer(GLenum target, GLuint buffer); target should be be set to GL_PIXEL_UNPACK_BUFFER to use the buffer as the source of pixel data. buffer specifies the unique name for the buffer object. 3. Create and initialize the data store of the buffer object by calling the function glBufferData. Essentially, this call uploads your data to the GPU. void glBufferData(GLenum target, sizeiptr size, const GLvoid *data, GLenum usage); Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 121target must be set to GL_PIXEL_UNPACK_BUFFER. size specifies the size of the data store. *data pointsto the source data. If thisis not NULL, the source data is copied to the data store of the buffer object. If NULL, the contents of the data store are undefined. usage is a constant that provides a hint as to how your application plans to use the data store. For more details on buffer hints, see “Buffer Usage Hints” (page 110) 4. Whenever you call glDrawPixels, glTexSubImage or similar functions that read pixel data from the application, those functions use the data in the bound pixel buffer object instead. 5. To update the data in the buffer object, your application calls glMapBuffer. Mapping the buffer prevents the GPU from operating on the data, and gives your application a pointer to memory it can use to update the buffer. void *glMapBuffer(GLenum target, GLenum access); target must be set to PIXEL_UNPACK_BUFFER. access indicatesthe operations you plan to performon the data. You can supply READ_ONLY, WRITE_ONLY, or READ_WRITE. 6. Modify the texture data using the pointer provided by map buffer. 7. When you have finished modifying the texture, call the function glUnmapBuffer. You should supplyPIXEL_UNPACK_BUFFER. Once the buffer is unmapped, your application can no longer access the buffer’s data through the pointer, and the buffer’s contents are uploaded again to the GPU. Using Pixel Buffer Objects for Asynchronous Pixel Transfers glReadPixels normally blocks until previous commands have completed, which includes the slow process of copying the pixel data to the application. However, if you call glReadPixels while a pixel buffer object is bound, the function returns immediately. It does not block until you actually map the pixel buffer object to read its content. 1. Call the function glGenBuffers to create a new name for a buffer object. void glGenBuffers(sizei n, uint *buffers ); n is the number of buffers you wish to create identifiers for. buffers specifies a pointer to memory to store the buffer names. Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 1222. Call the function glBindBuffer to bind an unused name to a buffer object. After this call, the newly created buffer object is initialized with a memory buffer of size zero and a default state. (For the default setting, see the OpenGL specification for ARB_vertex_buffer_object.) void glBindBuffer(GLenum target, GLuint buffer); target should be be set to GL_PIXEL_PACK_BUFFER to use the buffer as the destination for pixel data. buffer specifies the unique name for the buffer object. 3. Create and initialize the data store of the buffer object by calling the function glBufferData. void glBufferData(GLenum target, sizeiptr size, const GLvoid *data, GLenum usage); target must be set to GL_ARRAY_BUFFER. size specifies the size of the data store. *data pointsto the source data. If thisis not NULL, the source data is copied to the data store of the buffer object. If NULL, the contents of the data store are undefined. usage is a constant that provides a hint as to how your application plans to use the data store. For more details on buffer hints, see “Buffer Usage Hints” (page 110) 4. Call glReadPixels or a similar function. The function inserts a command to read the pixel data into the bound pixel buffer object and then returns. 5. To take advantage of asynchronous pixel reads, your application should perform other work. 6. To retrieve the data in the pixel buffer object, your application calls glMapBuffer. This blocks OpenGL until the previously queued glReadPixels command completes, maps the data, and provides a pointer to your application. void *glMapBuffer(GLenum target, GLenum access); target must be set to GL_PIXEL_PACK_BUFFER. access indicatesthe operations you plan to performon the data. You can supply READ_ONLY, WRITE_ONLY, or READ_WRITE. 7. Write vertex data to the pointer provided by map buffer. 8. When you no longer need the vertex data, call the function glUnmapBuffer. You should supply GL_PIXEL_PACK_BUFFER. Once the buffer is unmapped, the data is no longer accessible to your application. Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 123Using Pixel Buffer Objects to Keep Data on the GPU There is no difference between a vertex buffer object and a pixel buffer object except for the target to which they are bound. An application can take the results in one buffer and use them as another buffer type. For example, you could use the pixel resultsfrom a fragmentshader and reinterpret them as vertex data in a future pass, without ever leaving the GPU: 1. Set up your first pass and submit your drawing commands. 2. Bind a pixel buffer object and call glReadPixels to fetch the intermediate results into a buffer. 3. Bind the same buffer as a vertex buffer. 4. Set up the second pass of your algorithm and submit your drawing commands. Keeping your intermediate data inside the GPU when performing multiple passes can result in great performance increases. Apple Client Storage The Apple client storage extension (APPLE_client_storage) lets you provide OpenGL with a pointer to memory that your application allocates and maintains. OpenGL retains a pointer to your data but does not copy the data. Because OpenGL references your data, your application must retain its copy of the data until all referencing textures are deleted. By using this extension you can eliminate the OpenGL framework copy as shown in Figure 11-4. Note that a texture width must be a multiple of 32 bytes for OpenGL to bypass the copy operation from the application to the OpenGL framework. Figure 11-4 The client storage extension eliminates a data copy GPU VRAM OpenGL driver OpenGL framework Application The Apple clientstorage extension defines a pixelstorage parameter, GL_UNPACK_CLIENT_STORAGE_APPLE, that you pass to the OpenGL function glPixelStorei to specify that your application retains storage for textures. The following code sets up client storage: Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 124glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE); For detailed information, see the OpenGL specification for the Apple client storage extension. Apple Texture Range and Rectangle Texture The Apple texture range extension (APPLE_texture_range) lets you define a region of memory used for texture data. Typically you specify an address range that encompasses the storage for a set of textures. This allows the OpenGL driver to optimize memory usage by creating a single memory mapping for all of the textures. You can also provide a hint as to how the data should be stored: cached or shared. The cached hint specifies to cache texture data in video memory. This hint is recommended when you have textures that you plan to use multiple times or that use linear filtering. The shared hint indicates that data should be mapped into a region of memory that enables the GPU to access the texture data directly (via DMA) without the need to copy it. This hint is best when you are using large images only once, perform nearest-neighbor filtering, or need to scale down the size of an image. The texture range extension defines the following routine for making a single memory mapping for all of the textures used by your application: void glTextureRangeAPPLE(GLenum target, GLsizei length, GLvoid *pointer); target is a valid texture target, such as GL_TEXTURE_2D. length specifies the number of bytes in the address space referred to by the pointer parameter. *pointer points to the address space that your application provides for texture storage. You provide the hint parameter and a parameter value to to the OpenGL function glTexParameteri. The possible values for the storage hint parameter (GL_TEXTURE_STORAGE_HINT_APPLE) are GL_STORAGE_CACHED_APPLE or GL_STORAGE_SHARED_APPLE. Some hardware requires texture dimensions to be a power-of-two before the hardware can upload the data using DMA. The rectangle texture extension (ARB_texture_rectangle) was introduced to allow texture targets for textures of any dimensions—that is, rectangle textures (GL_TEXTURE_RECTANGLE_ARB). You need to use the rectangle texture extension together with the Apple texture range extension to ensure OpenGL uses DMA to access your texture data. These extensions allow you to bypass the OpenGL driver, as shown in Figure 11-5. Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 125Note that OpenGL does not use DMA for a power-of-two texture target (GL_TEXTURE_2D). So, unlike the rectangular texture, the power-of-two texture will incur one additional copy and performance won't be quite as fast. The performance typically isn't an issue because games, which are the applications most likely to use power-of-two textures, load textures at the start of a game or level and don't upload textures in real time as often as applications that use rectangular textures, which usually play video or display images. The next section has code examples that use the texture range and rectangle textures together with the Apple client storage extension. Figure 11-5 The texture range extension eliminates a data copy GPU VRAM OpenGL driver OpenGL framework Application For detailed information on these extensions,see the OpenGL specification for the Apple texture range extension and the OpenGL specification for the ARB texture rectangle extension. Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 126Combining Client Storage with Texture Ranges You can use the Apple client storage extension along with the Apple texture range extension to streamline the texture data path in your application. When used together, OpenGL moves texture data directly into video memory, as shown in Figure 11-6. The GPU directly accesses your data (via DMA). The set up is slightly different for rectangular and power-of-two textures. The code examples in this section upload textures to the GPU. You can also use these extensions to download textures, see “Downloading Texture Data” (page 136). Figure 11-6 Combining extensions to eliminate data copies GPU VRAM OpenGL driver OpenGL framework Application Listing 11-1 shows how to use the extensions for a rectangular texture. After enabling the texture rectangle extension you need to bind the rectangular texture to a target. Next, set up the storage hint. Call glPixelStorei to set up the Apple client storage extension. Finally, call the function glTexImage2D with a with a rectangular texture target and a pointer to your texture data. Note: The texture rectangle extension limits what can be done with rectangular textures. To understand the limitationsin detail, read the OpenGL extension for texture rectangles. See “Working with Non–Power-of-Two Textures” (page 129) for an overview of the limitations and an alternative to using this extension. Listing 11-1 Using texture extensions for a rectangular texture glEnable (GL_TEXTURE_RECTANGLE_ARB); glBindTexture(GL_TEXTURE_RECTANGLE_ARB, id); glTexParameteri(GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_STORAGE_HINT_APPLE, GL_STORAGE_CACHED_APPLE); glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE); Best Practices for Working with Texture Data Using Extensions to Improve Texture Performance 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 127glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_RGBA, sizex, sizey, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, myImagePtr); Setting up a power-of-two texture to use these extensions is similar to what's needed to set up a rectangular texture, as you can see by looking at Listing 11-2. The difference is that the GL_TEXTURE_2D texture target replaces the GL_TEXTURE_RECTANGLE_ARB texture target. Listing 11-2 Using texture extensions for a power-of-two texture glBindTexture(GL_TEXTURE_2D, myTextureName); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_STORAGE_HINT_APPLE, GL_STORAGE_CACHED_APPLE); glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, sizex, sizey, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, myImagePtr); Optimal Data Formats and Types The best format and data type combinations to use for texture data are: GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV GL_BGRA, GL_UNSIGNED_SHORT_1_5_5_5_REV) GL_YCBCR_422_APPLE, GL_UNSIGNED_SHORT_8_8_REV_APPLE The combination GL_RGBA and GL_UNSIGNED_BYTE needs to be swizzled by many cards when the data is loaded, so it's not recommended. Best Practices for Working with Texture Data Optimal Data Formats and Types 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 128Working with Non–Power-of-Two Textures OpenGL is often used to process video and images, which typically have dimensionsthat are not a power-of-two. Until OpenGL 2.0, the texture rectangle extension (ARB_texture_rectangle) provided the only option for a rectangular texture target. This extension, however, imposesthe following restrictions on rectangular textures: ● You can't use mipmap filtering with them. ● You can use only these wrap modes: GL_CLAMP, GL_CLAMP_TO_EDGE, and GL_CLAMP_TO_BORDER. ● The texture cannot have a border. ● The texture uses non-normalized texture coordinates. (See Figure 11-7.) OpenGL 2.0 adds another option for a rectangular texture target through the ARB_texture_non_power_of_two extension, which supports these textures without the limitations of the ARB_texture_rectangle extension. Before using it, you must check to make sure the functionality is available. You'll also want to consult the OpenGL specification for the non—power-of-two extension. Figure 11-7 Normalized and non-normalized coordinates Normalized Non-normalized 0 1 1 0 Width Height If your code runs on a system that does not support either the ARB_texture_rectangle or ARB_texture_non_power_of_two extensions you have these options for working with with rectangular images: ● Use the OpenGL function gluScaleImage to scale the image so that it fitsin a rectangle whose dimensions are a power of two. The image undoes the scaling effect when you draw the image from the properly sized rectangle back into a polygon that has the correct aspect ratio for the image. Note: This option can result in the loss of some data. But if your application runs on hardware that doesn'tsupport the ARB_texture_rectangle extension, you may need to use this option. Best Practices for Working with Texture Data Working with Non–Power-of-Two Textures 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 129● Segment the image into power-of-two rectangles, as shown in Figure 11-8 by using one image buffer and different texture pointers. Notice how the sides and corners of the image shown in Figure 11-8 are segmented into increasingly smaller rectangles to ensure that every rectangle has dimensions that are a power of two. Special care may be needed at the borders between each segment to avoid filtering artifacts if the texture is scaled or rotated. Figure 11-8 An image segmented into power-of-two tiles Best Practices for Working with Texture Data Working with Non–Power-of-Two Textures 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 130Creating Textures from Image Data OpenGL on the Macintosh provides several options for creating high-quality textures from image data. OS X supports floating-point pixel values, multiple image file formats, and a variety of color spaces. You can import a floating-point image into a floating-point texture. Figure 11-9 shows an image used to texture a cube. Figure 11-9 Using an image as a texture for a cube For Cocoa, you need to provide a bitmap representation. You can create an NSBitmapImageRep object from the contents of an NSView object. You can use the Image I/O framework (see CGImageSource Reference ). This framework has support for many different file formats, floating-point data, and a variety of color spaces. Furthermore, it is easy to use. You can import image data as a texture simply by supplying a CFURL object that specifies the location of the texture. There is no need for you to convert the image to an intermediate integer RGB format. Creating a Texture from a Cocoa View You can use the NSView class or a subclass of it for texturing in OpenGL. The process is to first store the image data from an NSView object in an NSBitmapImageRep object so that the image data is in a format that can be readily used as texture data by OpenGL. Then, after setting up the texture target, you supply the bitmap data to the OpenGL function glTexImage2D. Note that you must have a valid, current OpenGL context set up. Best Practices for Working with Texture Data Creating Textures from Image Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 131Note: You can't create an OpenGL texture from image data that's provided by a view created from the following classes: NSProgressIndicator, NSMovieView, and NSOpenGLView. Thisis because these views do not use the window backing store, which is what the method initWithFocusedViewRect: reads from. Listing 11-3 shows a routine that uses this process to create a texture from the contents of an NSView object. A detailed explanation for each numbered line of code appears following the listing. Listing 11-3 Building an OpenGL texture from an NSView object -(void)myTextureFromView:(NSView*)theView textureName:(GLuint*)texName { NSBitmapImageRep * bitmap = [theView bitmapImageRepForCachingDisplayInRect: [theView visibleRect]]; // 1 int samplesPerPixel = 0; [theView cacheDisplayInRect:[theView visibleRect] toBitmapImageRep:bitmap]; // 2 samplesPerPixel = [bitmap samplesPerPixel]; // 3 glPixelStorei(GL_UNPACK_ROW_LENGTH, [bitmap bytesPerRow]/samplesPerPixel); // 4 glPixelStorei (GL_UNPACK_ALIGNMENT, 1); // 5 if (*texName == 0) // 6 glGenTextures (1, texName); glBindTexture (GL_TEXTURE_RECTANGLE_ARB, *texName); // 7 glTexParameteri(GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_MIN_FILTER, GL_LINEAR); // 8 if(![bitmap isPlanar] && (samplesPerPixel == 3 || samplesPerPixel == 4)) { // 9 glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, samplesPerPixel == 4 ? GL_RGBA8 : GL_RGB8, [bitmap pixelsWide], [bitmap pixelsHigh], 0, Best Practices for Working with Texture Data Creating Textures from Image Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 132samplesPerPixel == 4 ? GL_RGBA : GL_RGB, GL_UNSIGNED_BYTE, [bitmap bitmapData]); } else { // Your code to report unsupported bitmap data } } Here's what the code does: 1. Allocates an NSBitmapImageRep object. 2. Initializes the NSBitmapImageRep object with bitmap data from the current view. 3. Gets the number of samples per pixel. 4. Sets the appropriate unpacking row length for the bitmap. 5. Sets the byte-aligned unpacking that's needed for bitmaps that are 3 bytes per pixel. 6. If a texture object is not passed in, generates a new texture object. 7. Binds the texture name to the texture target. 8. Sets filtering so that it does not use a mipmap, which would be redundant for the texture rectangle extension. 9. Checks to see if the bitmap is nonplanar and is either a 24-bit RGB bitmap or a 32-bit RGBA bitmap. If so, retrievesthe pixel data using the bitmapData method, passing it along with other appropriate parameters to the OpenGL function for specifying a 2D texture image. Creating a Texture from a Quartz Image Source Quartz images (CGImageRef data type) are defined in the Core Graphics framework (ApplicationServices/CoreGraphics.framework/CGImage.h) while the image source data type for reading image data and creating Quartz images from an image source is declared in the Image I/O framework (ApplicationServices/ImageIO.framework/CGImageSource.h). Quartz provides routines that read a wide variety of image data. To use a Quartz image as a texture source, follow these steps: 1. Create a Quartz image source by supplying a CFURL object to the function CGImageSourceCreateWithURL. 2. Create a Quartz image by extracting an image from the image source, using the function CGImageSourceCreateImageAtIndex. Best Practices for Working with Texture Data Creating Textures from Image Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 1333. Extract the image dimensions using the function CGImageGetWidth and CGImageGetHeight. You'll need these to calculate the storage required for the texture. 4. Allocate storage for the texture. 5. Create a color space for the image data. 6. Create a Quartz bitmap graphics context for drawing. Make sure to set up the context for pre-multiplied alpha. 7. Draw the image to the bitmap context. 8. Release the bitmap context. 9. Set the pixel storage mode by calling the function glPixelStorei. 10. Create and bind the texture. 11. Set up the appropriate texture parameters. 12. Call glTexImage2D, supplying the image data. 13. Free the image data. Listing 11-4 shows a code fragment that performsthese steps. Note that you must have a valid, current OpenGL context. Listing 11-4 Using a Quartz image as a texture source CGImageSourceRef myImageSourceRef = CGImageSourceCreateWithURL(url, NULL); CGImageRef myImageRef = CGImageSourceCreateImageAtIndex (myImageSourceRef, 0, NULL); GLint myTextureName; size_t width = CGImageGetWidth(myImageRef); size_t height = CGImageGetHeight(myImageRef); CGRect rect = {{0, 0}, {width, height}}; void * myData = calloc(width * 4, height); CGColorSpaceRef space = CGColorSpaceCreateDeviceRGB(); CGContextRef myBitmapContext = CGBitmapContextCreate (myData, width, height, 8, width*4, space, kCGBitmapByteOrder32Host | kCGImageAlphaPremultipliedFirst); CGContextSetBlendMode(myBitmapContext, kCGBlendModeCopy); CGContextDrawImage(myBitmapContext, rect, myImageRef); Best Practices for Working with Texture Data Creating Textures from Image Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 134CGContextRelease(myBitmapContext); glPixelStorei(GL_UNPACK_ROW_LENGTH, width); glPixelStorei(GL_UNPACK_ALIGNMENT, 1); glGenTextures(1, &myTextureName); glBindTexture(GL_TEXTURE_RECTANGLE_ARB, myTextureName); glTexParameteri(GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_RGBA8, width, height, 0, GL_BGRA_EXT, GL_UNSIGNED_INT_8_8_8_8_REV, myData); free(myData); For more information on using Quartz, see Quartz 2D Programming Guide , CGImage Reference , and CGImageSource Reference . Getting Decompressed Raw Pixel Data from a Source Image You can use the Image I/O framework together with a Quartz data provider to obtain decompressed raw pixel data from a source image, as shown in Listing 11-5. You can then use the pixel data for your OpenGL texture. The data has the same format as the source image, so you need to make sure that you use a source image that has the layout you need. Alpha is not premultiplied for the pixel data obtained in Listing 11-5, but alpha is premultiplied for the pixel data you get when using the code described in “Creating a Texture from a Cocoa View” (page 131) and “Creating a Texture from a Quartz Image Source” (page 133). Listing 11-5 Getting pixel data from a source image CGImageSourceRef myImageSourceRef = CGImageSourceCreateWithURL(url, NULL); CGImageRef myImageRef = CGImageSourceCreateImageAtIndex (myImageSourceRef, 0, NULL); CFDataRef data = CGDataProviderCopyData(CGImageGetDataProvider(myImageRef)); void *pixelData = CFDataGetBytePtr(data); Best Practices for Working with Texture Data Creating Textures from Image Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 135Downloading Texture Data A texture download operation uses the same data path as an upload operation except that the data path is reversed. Downloading transfers texture data, using direct memory access (DMA), from VRAM into a texture that can then be accessed directly by your application. You can use the Apple client range, texture range, and texture rectangle extensions for downloading, just as you would for uploading. To download texture data using the Apple client storage, texture range, and texture rectangle extensions: ● Bind a texture name to a texture target. ● Set up the extensions ● Call the function glCopyTexSubImage2D to copy a texture subimage from the specified window coordinates. This call initiates an asynchronous DMA transfer to system memory the next time you call a flush routine. The CPU doesn't wait for this call to complete. ● Call the function glGetTexImage to transfer the texture into system memory. Note that the parameters must match the ones that you used to set up the texture when you called the function glTexImage2D. This call is the synchronization point; it waits until the transfer is finished. Listing 11-6 shows a code fragment that downloads a rectangular texture that uses cached memory. Your application processes data between the glCopyTexSubImage2D and glGetTexImage calls. How much processing? Enough so that your application does not need to wait for the GPU. Listing 11-6 Code that downloads texture data glBindTexture(GL_TEXTURE_RECTANGLE_ARB, myTextureName); glTexParameteri(GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_STORAGE_HINT_APPLE, GL_STORAGE_SHARED_APPLE); glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE); glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_RGBA, sizex, sizey, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, myImagePtr); glCopyTexSubImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, 0, 0, 0, 0, image_width, image_height); glFlush(); // Do other work processing here, using a double or triple buffer glGetTexImage(GL_TEXTURE_RECTANGLE_ARB, 0, GL_BGRA, Best Practices for Working with Texture Data Downloading Texture Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 136GL_UNSIGNED_INT_8_8_8_8_REV, pixels); Double Buffering Texture Data When you use any technique that allowsthe GPU to access your texture data directly,such asthe texture range extension, it's possible for the GPU and CPU to access the data at the same time. To avoid such a collision, you must synchronize the GPU and the CPU. The simplest way is shown in Figure 11-10. Your application works on the data, flushes it to the GPU and waits until the GPU is finished before working on the data again. One technique for ensuring that the GPU is finished executing commands before your application sends more data is to insert a token into the command stream and use that to determine when the CPU can touch the data again, as described in “Use Fences for Finer-Grained Synchronization” (page 98). Figure 11-10 uses the fence extension command glFinishObject to synchronize buffer updates for a stream of single-buffered texture data. Notice that when the CPU is processing texture data, the GPU is idle. Similarly, when the GPU is processing texture data, the CPU is idle. It's much more efficient for the GPU and CPU to work asynchronously than to work synchronously. Double buffering data is a technique that allows you to process data asynchronously, as shown in Figure 11-11 (page 138). Figure 11-10 Single-buffered data CPU GPU glFinishObject(..., 1) glFinishObject(..., 1) TIME Frame 1 Frame 2 glFlush glFlush Best Practices for Working with Texture Data Double Buffering Texture Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 137To double buffer data, you must supply two sets of data to work on. Note in Figure 11-11 that while the GPU is rendering one frame of data, the CPU processes the next. After the initial startup, neither processing unit is idle. Using the glFinishObject function provided by the fence extension ensures that buffer updating is synchronized. Figure 11-11 Double-buffered data CPU GPU glFinishObject(..., 1) glFinishObject(..., 1) glFinishObject(..., 2) glFinishObject(..., 2) Time Frame 1 Frame 2 Frame 3 Frame 4 glFlush glFlush glFlush glFlush Best Practices for Working with Texture Data Double Buffering Texture Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 138OpenGL 1.x used fixed functions to deliver a useful graphics pipeline to application developers. To configure the various stages of the pipeline shown in Figure 12-1, applications called OpenGL functions to tweak the calculationsthat were performed for each vertex and fragment. Complex algorithmsrequired multiple rendering passes and dozens of function calls to configure the calculations that the programmer desired. Extensions offered new configuration options, but did not change the complex nature of OpenGL programming. Figure 12-1 OpenGL fixed-function pipeline Geometry Fragment Framebuffer operations Texturing Fog Alpha, stencil, and depth tests Framebuffer blending Primitive assembly Clipping Vertex Application Primitives and image data Transform and lighting 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 139 Customizing the OpenGL Pipeline with ShadersStarting with OpenGL 2.0, some stages of the OpenGL pipeline can be replaced with shaders. A shader is a program written in a special shading language. This program is compiled by OpenGL and uploaded directly into the graphics hardware. Figure 12-2 shows where your applications can hook into the pipeline with shaders. Figure 12-2 OpenGL shader pipeline Geometry Fragment Framebuffer operations Fragment shaders Alpha, stencil, and depth tests Framebuffer blending Geometry shaders Clipping Vertex Application Primitives and image data Vertex shaders Shaders offer a considerable number of advantages to your application: ● Shaders give you precise control over the operations that are performed to render your images. ● Shaders allow for algorithmsto be written in a terse, expressive format. Rather than writing complex blocks of configuration callsto implement a mathematical operation, you write code that expressesthe algorithm directly. ● Older graphics processors implemented the fixed-function pipeline in hardware or microcode, but now graphics processors are general-purpose computing devices. The fixed function pipeline is itself implemented as a shader. ● Shaders allow for longer and more complex algorithms to be implemented using a single rendering pass. Because you have extensive control over the pipeline, it is also easier to implement multipass algorithms without requiring the data to be read back from the GPU. ● Your application can switch between different shaders with a single function call. In contrast, configuring the fixed-function pipeline incurs significant function-call overhead. If your application uses the fixed-function pipeline, a critical task is to replace those tasks with shaders. Customizing the OpenGL Pipeline with Shaders 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 140If you are new to shaders, OpenGL Shading Language , by Randi J. Rost, is an excellent guide for those looking to learn more about writing shaders and integrating them into your application. The rest of this chapter provides some boilerplate code, briefly describe the extensions that implement shaders, and discusses tools that Apple provides to assist you in writing shaders. Shader Basics OpenGL 2.0 offers vertex and fragmentshaders, to take over the processing of those two stages of the graphics pipeline. These same capabilities are also offered by the ARB_shader_objects, ARB_vertex_shader and ARB_fragment_shaderextensions. Vertex shading is available on all hardware running OS X v10.5 or later. Fragment shading is available on all hardware running OS X v10.6 and the majority of hardware running OS X v10.5. Creating a shader program is an expensive operation compared to other OpenGL state changes. Listing 12-1 presents a typical strategy to load, compile, and verify a shader program. Listing 12-1 Loading a Shader /** Initialization-time for shader **/ GLuint shader, prog; GLchar *shaderText = "... shader text ..."; // Create ID for shader shader = glCreateShader(GL_VERTEX_SHADER); // Define shader text glShaderSource(shaderText); // Compile shader glCompileShader(shader); // Associate shader with program glAttachShader(prog, shader); // Link program glLinkProgram(prog); // Validate program glValidateProgram(prog); // Check the status of the compile/link glGetProgramiv(prog, GL_INFO_LOG_LENGTH, &logLen); if(logLen > 0) { // Show any errors as appropriate glGetProgramInfoLog(prog, logLen, &logLen, log); Customizing the OpenGL Pipeline with Shaders Shader Basics 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 141fprintf(stderr, "Prog Info Log: %s\n", log); } // Retrieve all uniform locations that are determined during link phase for(i = 0; i < uniformCt; i++) { uniformLoc[i] = glGetUniformLocation(prog, uniformName); } // Retrieve all attrib locations that are determined during link phase for(i = 0; i < attribCt; i++) { attribLoc[i] = glGetAttribLocation(prog, attribName); } /** Render stage for shaders **/ glUseProgram(prog); This code loads the text source for a vertex shader, compiles it, and adds it to the program. A more complex example might also attach fragment and geometry shaders. The program islinked and validated for correctness. Finally, the program retrieves information about the inputs to the shader and stores then in its own arrays. When the application is ready to use the shader, it calls glUseProgram to make it the current shader. For best performance, your application should create shaders when your application is initialized, and not inside the rendering loop. Inside your rendering loop, you can quickly switch in the appropriate shaders by calling glUseProgram. For best performance, use the vertex array object extension to also switch in the vertex pointers. See “Vertex Array Object” (page 116) for more information. Advanced Shading Extensions In addition to the standard shader,some Macs offer additionalshading extensionsto reveal advanced hardware capabilities. Not all of these extensions are available on all hardware,so you need to assess whether the features of each extension are worth implementing in your application. Transform Feedback The EXT_transform_feedback extension is available on all hardware running OS X v10.5 or later. With the feedback extension, you can capture the results of the vertex shader into a buffer object, which can be used as an input to future commands. This is similar to the pixel buffer object technique described in “Using Pixel Buffer Objects to Keep Data on the GPU” (page 124), but more directly captures the results you desire. Customizing the OpenGL Pipeline with Shaders Advanced Shading Extensions 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 142GPU Shader 4 The EXT_gpu_shader4 extension extends the OpenGL shading language to offer new operations, including: ● Full integer support. ● Built-in shader variable to reference the current vertex. ● Built-in shader variable to reference the current primitive. This makes it easier to use a shader to use the same static vertex data to render multiple primitives, using a shader and uniform variables to customize each instance of that primitive. ● Unfiltered texture fetches using integer coordinates. ● Querying the size of a texture within a shader. ● Offset texture lookups. ● Explicit gradient and LOD texture lookups. ● Depth Cubemaps. Geometry Shaders The EXT_geometry_shader4 extension allows your create geometry shaders. A geometry shader accepts transformed vertices and can add or remove vertices before passing them down to the rasterizer. This allows the application to add or remove geometry based on the calculated values in the vertex. For example, given a triangle and its neighboring vertices, your application could emit additional vertices to better create a more accurate appearance of a curved surface. Uniform Buffers The EXT_bindable_uniform extension allows your application to allocate buffer objects and use them as the source for uniform data in your shaders. Instead of relying on a single block of uniform memory supplied by OpenGL, your application allocates buffer objects using the same API that it uses to implement vertex buffer objects (“Vertex Buffers” (page 107)). Instead of making a function call for each uniform variable you want to change, you can swap all of the uniform data by binding to a different uniform buffer. Customizing the OpenGL Pipeline with Shaders Advanced Shading Extensions 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 143Aliasing is the bane of the digital domain. In the early days of the personal computer, jagged edges and blocky graphics were accepted by the user simply because not much could be done to correct them. Now with faster hardware and higher-resolution displays, there are several antialiasing techniques that can smooth edges to achieve a more realistic scene. OpenGL supports antialiasing that operates at the level of lines and polygons as well as at the level of the full scene. This chapter discusses techniques for full scene antialiasing (FSAA). If your application needs point or line antialiasing instead of full scene antialiasing, use the built in OpenGL point and line antialiasing functions. These are described in Section 3.4.2 in the OpenGL Specification. The three antialiasing techniques in use today are multisampling, supersampling, and alpha channel blending: ● Multisampling defines a technique for sampling pixel content at multiple locations for each pixel. This is a good technique to use if you want to smooth polygon edges. ● Supersampling renders at a much higher resolution than what's needed for the display. Prior to drawing the content to the display, OpenGL scales and filters the content to the appropriate resolution. This is a good technique to use when you want to smooth texture interiors in addition to polygon edges. ● Alpha channel blending uses the alpha value of a fragment to control how to blend the fragment with the pixel values that are already in the framebuffer. It's a good technique to use when you want to ensure that foreground and background images are composited smoothly. The ARB_multisample extension defines a specification for full scene antialiasing. It describes multisampling and alpha channel sampling. The specification does not specifically mention supersampling but its wording doesn't preclude supersampling. The antialiasing methods that are available depend on the hardware and the actual implementation depends on the vendor. Some graphics cards support antialiasing using a mixture of multisampling and supersampling. The methodology used to select the samples can vary as well. Your best approach is to query the renderer to find out exactly what is supported. OpenGL lets you provide a hint to the renderer asto which antialiasing technique you prefer. Hints are available asrenderer attributesthat you supply when you create a pixel format object. A smallersubset of rendererssupport the EXT_framebuffer_blit and EXT_framebuffer_multisample extensions. These extensions allow your application to create multisampled offscreen frame buffer objects, render detailed scenesto them, with precise control over when the multisampled renderbuffer isresolved to a single displayable color per pixel. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 144 Techniques for Scene AntialiasingGuidelines Keep the following in mind when you set up full scene antialiasing: ● Although a system may have enough VRAM to accommodate a multisample buffer, a large buffer can affect the ability of OpenGL to maintain a properly working texture set. Keep in mind that the buffers associated with the rendering context—depth and stencil—increase in size by a factor equal to number of samples per pixel. ● The OpenGL driver allocates the memory needed for the multisample buffer; your application should not allocate this memory. ● Any antialiasing algorithm that operates on the full scene requires additional computing resources. There is a tradeoff between performance and quality. For that reason, you may want to provide a user interface that allows the user to enable and disable FSAA, or to choose the level of quality for antialiasing. ● The commands glEnable(GL_MULTISAMPLE) and glDisable(GL_MULTISAMPLE) are ignored on some hardware because some graphics cards have the feature enabled all the time. That doesn't mean that you should not call these commands because you'll certainly need them on hardware that doesn't ignore them. ● A hint as to the variant of sampling you want is a suggestion, not a command. Not all hardware supports all types of antialiasing. Other hardware mixes multisampling with supersampling techniques. The driver dictates the type of antialiasing that's actually used in your application. ● The best way to find out which sample modes are supported is to call the CGL function CGLDescribeRenderer with the renderer property kCGLRPSampleModes or kCGLRPSampleAlpha. You can also determine how many samples the renderer supports by calling CGLDescribeRenderer with the renderer property kCGLRPMaxSamples. General Approach The general approach to setting up full scene antialiasing is as follows: 1. Check to see what's supported. Not all renderers support the ARB multisample extension, so you need to check for this functionality (see “Detecting Functionality” (page 83)). To find out what type of antialiasing a specific renderersupports, call the function CGLDescribeRenderer. Supply the renderer property kCGLRPSampleModes to find out whether the renderer supports multisampling and supersampling. Supply kCGLRPSampleAlpha to see whether the renderer supports alpha sampling. Techniques for Scene Antialiasing Guidelines 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 145You can choose to exclude unsupported hardware from the pixel format search by specifying only the hardware thatsupports multisample antialiasing. Keep in mind that if you exclude unsupported hardware, the unsupported displays will not render anything. If you include unsupported hardware, OpenGL uses normal aliased rendering to the unsupported displays and multisampled rendering to supported displays. 2. Include these buffer attributes in the attributes array: ● The appropriate sample buffer attribute constant (NSOpenGLPFASampleBuffers or kCGLPFASampleBuffers) along with the number of multisample buffers. At thistime the specification allows only one multisample buffer. ● The appropriate samples constant (NSOpenGLPFASamples or kCGLPFASamples) along with the number ofsamples per pixel. You can supply 2, 4, 6, or more depending on what the renderersupports and the amount of VRAM available. The value that you supply affects the quality, memory use, and speed of the multisampling operation. For fastest performance, and to use the least amount of video memory, specify 2 samples. When you need more quality, specify 4 or more. ● The no recovery attribute ( NSOpenGLPFANoRecovery or kCGLPFANoRecovery). Although enabling this attribute is not mandatory, it's recommended to prevent OpenGL from using software fallback as a renderer. Multisampled antialiasing performance is slow in the software renderer. 3. Optionally provide a hint for the type of antialiasing you want—multisampling, supersampling, or alpha sampling. See “Hinting for a Specific Antialiasing Technique” (page 147). 4. Enable multisampling with the following command: glEnable(GL_MULTISAMPLE); Regardless of the enabled state, OpenGL always uses the multisample buffer if you supply the appropriate buffer attributes when you set up the pixel format object. If you haven'tsupplied the appropriate attributes, enabling multisampling has no effect. When multisampling is disabled, all coverage values are set to 1, which gives the appearance of rendering without multisampling. Some graphics hardware leaves multisampling enabled all the time. However, don't rely on hardware to have multisampling enabled; use glEnable to programmatically turn on this feature. 5. Optionally provide hints for the rendering algorithm. You perform this optional step only if you want OpenGL to compute coverage values by a method other than uniformly weighting samples and averaging them. Some hardware supports a multisample filter hint through an OpenGL extension—GL_NV_multisample_filter_hint. This hint allows an OpenGL implementation to use an alternative method of resolving the color of multisampled pixels. Techniques for Scene Antialiasing General Approach 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 146You can specify that OpenGL usesfaster or nicer rendering by calling the OpenGL function glHint, passing the constant GL_MULTISAMPLE_FILTER_HINT_NV asthe target parameter and GL_FASTEST or GL_NICEST as the mode parameter. Hints allow the hardware to optimize the output if it can. There is no performance penalty or returned error for issuing a hint that's not supported. For more information, see the OpenGL extension registry for NV_multisample_filter_hint. Hinting for a Specific Antialiasing Technique When you set up your renderer and buffer attributes for full scene antialiasing, you can specify a hint to prefer one antialiasing technique over the others. If the underlying renderer does not have sufficient resources to support what you request, OpenGL ignores the hint. If you do not supply the appropriate buffer attributes when you create a pixel format object, then the hint does nothing. Table 13-1 lists the hinting constants available for the NSOpenGLPixelFormat class and CGL. Table 13-1 Antialiasing hints Multisampling Supersampling Alpha blending NSOpenGLPFAMultisample NSOpenGLPFASupersample NSOpenGLPFASampleAlpha kCGLPFAMultisample kCGLPFASupersample kCGLPFASampleAlpha Techniques for Scene Antialiasing Hinting for a Specific Antialiasing Technique 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 147Concurrency is the notion of multiple things happening at the same time. In the context of computers, concurrency usually refers to executing tasks on more than one processor at the same time. By performing work in parallel, tasks complete sooner, and applications become more responsive to the user. The good news isthat well-designed OpenGL applications already exhibit a specific form of concurrency—concurrency between application processing on the CPU and OpenGL processing on the GPU. Many of the techniques introduced in “OpenGL Application Design Strategies” (page 89) are aimed specifically at creating OpenGL applications that exhibit great CPU-GPU parallelism. However, modern computers not only contain a powerful GPU, but also contain multiple CPUs. Sometimesthose CPUs have multiple cores, each capable of performing calculations independently of the others. It is critical that applications be designed to take advantage of concurrency where possible. Designing a concurrent application means decomposing the work your application performs into subtasks and identifying which tasks can safely operate in parallel and which tasks must be executed sequentially—that is, which tasks are dependent on either resources used by other tasks or results returned from those tasks. Each process in OS X is made up of one or more threads. A thread is a stream of execution that runs code for the process. Multicore systems offer true concurrency by allowing multiple threads to execute simultaneously. Apple offers both traditional threads and a feature called Grand CentralDispatch (GCD). Grand Central Dispatch allows you to decompose your application into smaller tasks without requiring the application to manage threads. GCD allocates threads based on the number of cores available on the system and automatically schedules tasks to those threads. At a higher level, Cocoa offers NSOperation and NSOperationQueue to provide an Objective-C abstraction for creating and scheduling units of work. On OS X v10.6, operation queues use GCD to dispatch work; on OS X v10.5, operation queues create threads to execute your application’s tasks. This chapter does not attempt describe these technologiesin detail. Before you consider how to add concurrency to your OpenGL application, you should first readConcurrency Programming Guide . If you plan on managing threads manually, you should also read Threading Programming Guide . Regardless of which technique you use, there are additional restrictions when calling OpenGL on multithreaded systems. This chapter helps you understand when multithreading improves your OpenGL application’s performance, the restrictions OpenGL places on multithreaded applications, and common design strategies you might use to implement concurrency in an OpenGL application. Some of these design techniques can get you an improvement in just a few lines of code. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 148 Concurrency and OpenGLIdentifying Whether an OpenGL Application Can Benefit from Concurrency Creating a multithreaded application requires significant effort in the design, implementation, and testing of your application. Threads also add complexity and overhead to an application. For example, your application may need to copy data so that it can be handed to a worker thread, or multiple threads may need to synchronize access to the same resources. Before you attempt to implement concurrency in an OpenGL application, you should optimize your OpenGL code in a single-threaded environment using the techniques described in “OpenGL Application Design Strategies” (page 89). Focus on achieving great CPU-GPU parallelism first and then assess whether concurrent programming can provide an additional performance benefit. A good candidate has either or both of the following characteristics: ● The application performs many tasks on the CPU that are independent of OpenGL rendering. Games, for example, simulate the game world, calculate artificial intelligence from computer-controlled opponents, and play sound. You can exploit parallelism in thisscenario because many of these tasks are not dependent on your OpenGL drawing code. ● Profiling your application has shown that your OpenGL rendering code spends a lot of time in the CPU. In this scenario, the GPU is idle because your application is incapable of feeding it commands fast enough. If your CPU-bound code has already been optimized, you may be able to improve its performance further by splitting the work into tasks that execute concurrently. If your application is blocked waiting for the GPU, and has no work it can perform in parallel with its OpenGL drawing commands, then it is not a good candidate for concurrency. If the CPU and GPU are both idle, then your OpenGL needs are probably simple enough that no further tuning is useful. For more information on how to determine where your application spends its time, see “Tuning Your OpenGL Application” (page 155). OpenGL Restricts Each Context to a Single Thread Each thread in an OS X process has a single current OpenGL rendering context. Every time your application calls an OpenGL function, OpenGL implicitly looks up the context associated with the current thread and modifies the state or objects associated with that context. OpenGL is not reentrant. If you modify the same context from multiple threads simultaneously, the results are unpredictable. Your application might crash or it might render improperly. If for some reason you decide to set more than one thread to target the same context, then you must synchronize threads by placing a mutex around all OpenGL calls to the context, such as gl* and CGL*. OpenGL commands that block—such as fence commands—do not synchronize threads. Concurrency and OpenGL Identifying Whether an OpenGL Application Can Benefit from Concurrency 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 149GCD and NSOperationQueue objects can both execute your tasks on a thread of their choosing. They may create a thread specifically for that task, or they may reuse an existing thread. But in either case, you cannot guarantee which thread executes the task. For an OpenGL application, that means: ● Each task must set the context before executing any OpenGL commands. ● Your application must ensure that two tasks that access the same context are not allowed to execute concurrently. Strategies for Implementing Concurrency in OpenGL Applications A concurrent OpenGL application wants to focus on CPU parallelism so that OpenGL can provide more work to the GPU. Here are a few recommended strategies for implementing concurrency in an OpenGL application: ● Decompose your application into OpenGL and non-OpenGL tasks that can execute concurrently. Your OpenGL rendering code executes as a single task, so it still executes in a single thread. This strategy works best when your application has other tasks that require significant CPU processing. ● If performance profiling reveals that your application spends a lot of CPU time inside OpenGL, you can move some of that processing to another thread by enabling the multithreading in the OpenGL engine. The advantage of this method is its simplicity; enabling the multithreaded OpenGL engine takes just a few lines of code. See “Multithreaded OpenGL” (page 150). ● If your application spends a lot of CPU time preparing data to send to openGL, you can divide the work between tasks that prepare rendering data and tasks that submit rendering commands to OpenGL. See “Perform OpenGL Computations in a Worker Task” (page 151) ● If your application has multiple scenes it can render simultaneously or work it can perform in multiple contexts, it can create multiple tasks, with an OpenGL context per task. If the contexts can share the same resources, you can use contextsharing when the contexts are created to share surfaces or OpenGL objects: display lists, textures, vertex and fragment programs, vertex array objects, and so on. See “Use Multiple OpenGL Contexts” (page 153) Multithreaded OpenGL Whenever your application calls OpenGL, the renderer processes the parameters to put them in a format that the hardware understands. The time required to process these commands varies depending on whether the inputs are already in a hardware-friendly format, but there is always some overhead in preparing commands for the hardware. Concurrency and OpenGL Strategies for Implementing Concurrency in OpenGL Applications 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 150If your application spends a lot of time performing calculations inside OpenGL, and you’ve already taken steps to pick ideal data formats, your application might gain an additional benefit by enabling multithreading inside the OpenGL engine. The multithreaded OpenGL engine automatically creates a worker thread and transfers some of its calculationsto that thread. On a multicore system, this allowsinternal OpenGL calculations performed on the CPU to act in parallel with your application, improving performance. Synchronizing functions continue to block the calling thread. Listing 14-1 shows the code required to enable the multithreaded OpenGL engine. Listing 14-1 Enabling the multithreaded OpenGL engine CGLError err = 0; CGLContextObj ctx = CGLGetCurrentContext(); // Enable the multithreading err = CGLEnable( ctx, kCGLCEMPEngine); if (err != kCGLNoError ) { // Multithreaded execution may not be available // Insert your code to take appropriate action } Note: Enabling or disabling multithreaded execution causes OpenGL to flush previous commands as well as incurring the overhead of setting up the additional thread. You should enable or disable multithreaded execution in an initialization function rather than in the rendering loop. Enabling multithreading comes at a cost—OpenGL must copy parameters to transmit them to the worker thread. Because of this overhead, you should always test your application with and without multithreading enabled to determine whether it provides a substantial performance improvement. Perform OpenGL Computations in a Worker Task Some applications perform lots of calculations on their data before passing that data down to the OpenGL renderer. For example, the application might create new geometry or animate existing geometry. Where possible,such calculationsshould be performed inside OpenGL. For example, vertex shaders and the transform Concurrency and OpenGL Perform OpenGL Computations in a Worker Task 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 151feedback extension might allow you to perform these calculations entirely within OpenGL. Thistakes advantage of the greater parallelism available inside the GPU, and reduces the overhead of copying results between your application and OpenGL. The approach described in Figure 9-3 (page 92) alternates between updating OpenGL objects and executing rendering commands that use those objects. OpenGL renders on the GPU in parallel with your application’s updates running on the CPU. If the calculations performed on the CPU take more processing time than those on the GPU, then the GPU spends more time idle. In this situation, you may be able to take advantage of parallelism on systems with multiple CPUs. Split your OpenGL rendering code into separate calculation and processing tasks, and run them in parallel. Figure 14-1 shows a clear division of labor. One task produces data that is consumed by the second and submitted to OpenGL. Figure 14-1 CPU processing and OpenGL on separate threads CPU Processing Shared data Framebuffer OpenGL context Texture data Vertex data OpenGL state OpenGL surface Thread 1 Thread 2 For best performance, your application should avoid copying data between the tasks. For example, rather than calculating the data in one task and copying it into a vertex buffer object in the other, map the vertex buffer object in the setup code and hand the pointer directly to the worker task. If your application can further decompose the modifications task into subtasks, you may see better benefits. For example, assume two or more vertex buffers, each of which needsto be updated before submitting drawing commands. Each can be recalculated independently of the others. In this scenario, the modifications to each buffer becomes an operation, using an NSOperationQueue object to manage the work: 1. Set the current context. 2. Map the first buffer. 3. Create an NSOperation object whose task is to fill that buffer. 4. Queue that operation on the operation queue. Concurrency and OpenGL Perform OpenGL Computations in a Worker Task 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 1525. Perform steps 2 through 4 for the other buffers. 6. Call waitUntilAllOperationsAreFinished on the operation queue. 7. Unmap the buffers. 8. Execute rendering commands. On a multicore system, multiple threads of execution may allow the buffers to be filled simultaneously. Steps 7 and 8 could even be performed by a separate operation queued onto the same operation queue, provided that operation set the proper dependencies. Use Multiple OpenGL Contexts If your application has multiple scenes that can be rendered in parallel, you can use a context for each scene you need to render. Create one context for each scene and assign each context to an operation or task. Because each task has its own context, all can submit rendering commands in parallel. The Apple-specific OpenGL APIs also provide the option for sharing data between contexts, as shown in Figure 14-2. Shared resources are automatically set up as mutual exclusion (mutex) objects. Notice that thread 2 draws to a pixel buffer that is linked to the shared state as a texture. Thread 1 can then draw using that texture. Figure 14-2 Two contexts on separate threads Pbuffer surface Framebuffer OpenGL context 1 OpenGL state 1 OpenGL context 2 OpenGL state 2 OpenGL surface Thread 1 Thread 2 OpenGL shared state OpenGL shared state OpenGL shared state This is the most complex model for designing an application. Changes to objects in one context must be flushed so that other contextssee the changes. Similarly, when your application finishes operating on an object, it must flush those commands before exiting, to ensure that all rendering commands have been submitted to the hardware. Concurrency and OpenGL Use Multiple OpenGL Contexts 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 153Guidelines for Threading OpenGL Applications Follow these guidelines to ensure successful threading in an application that uses OpenGL: ● Use only one thread per context. OpenGL commands for a specific context are not thread safe. You should never have more than one thread accessing a single context simultaneously. ● Contexts that are on different threads can share object resources. For example, it is acceptable for one context in one thread to modify a texture, and a second context in a second thread to modify the same texture. The shared object handling provided by the Apple APIs automatically protects against thread errors. And, your application is following the "one thread per context" guideline. ● When you use an NSOpenGLView object with OpenGL calls that are issued from a thread other than the main one, you must set up mutex locking. Mutex locking is necessary because unless you override the default behavior, the main thread may need to communicate with the view for such things as resizing. Applications that use Objective-C with multithreading can lock contexts using the functions CGLLockContext and CGLUnlockContext. If you want to perform rendering in a thread other than the main one, you can lock the context that you want to access and safely execute OpenGL commands. The locking calls must be placed around all of your OpenGL calls in all threads. CGLLockContext blocks the thread it is on until all other threads have unlocked the same context using the function CGLUnlockContext. You can use CGLLockContext recursively. Context-specific CGL calls by themselves do not require locking, but you can guarantee serial processing for a group of calls by surrounding them with CGLLockContext and CGLUnlockContext. Keep in mind that calls from the OpenGL API (the API provided by the Khronos OpenGL Working Group) require locking. ● Keep track of the current context. When switching threadsit is easy to switch contextsinadvertently, which causes unforeseen effects on the execution of graphic commands. You must set a current context when switching to a newly created thread. Concurrency and OpenGL Guidelines for Threading OpenGL Applications 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 154After you design and implement your application, it is important that you spend some time analyzing its performance. The key to performance tuning your OpenGL application is to successively refine the design and implementation of your application. You do this by alternating between measuring your application, identifying where the bottleneck is, and removing the bottleneck. If you are unfamiliar with general performance issues on the Macintosh platform, you will want to read Getting Started with Performance and Performance Overview. Performance Overview contains general performance tips that are useful to all applications. It also describes most of the performance tools provided with OS X. Next, take a close look at Instruments. Instruments consolidates many measurement tools into a single comprehensive performance-tuning application. There are two tools other than OpenGL Profiler that are specific for OpenGL development—OpenGL Driver Monitor and OpenGL Shader Builder. OpenGL Driver Monitor collectsreal-time data from the hardware. OpenGL Shader Builder provides immediate feedback on vertex and fragment programs that you write. For more information on these tools, see: ● OpenGL Tools for Serious Graphics Development ● Optimizing with Shark: Big Payoff, Small Effort ● Instruments User Guide ● Shark User Guide ● Real world profiling with the OpenGL Profiler ● OpenGL Driver Monitor User Guide ● OpenGL Shader Builder User Guide The following books contain many techniques for getting the most performance from the GPU: ● GPU Gems: Programming Techniques, Tips and Tricks for Real Time Graphics, Randima Fernando. In particular, Graphics Pipeline Performance is a critical article for understanding how to find the bottlenecks in your OpenGL application. ● GPU Gems 2: Programming Techniques for High-Performance Graphics and General-Purpose Computation , Matt Pharr and Randima Fernando. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 155 Tuning Your OpenGL ApplicationThis chapter focuses on two main topics: ● “Gathering and Analyzing Baseline Performance Data” (page 156) shows how to use top and OpenGL Profiler to obtain and interpret baseline performance data. ● “Identifying Bottlenecks with Shark” (page 161) discussesthe patterns of usage that the Shark performance tool can make apparent and that indicate places in your code that you may want to improve. Gathering and Analyzing Baseline Performance Data Analyzing performance is a systematic process that starts with gathering baseline data. OS X provides several applications that you can use to assess baseline performance for an OpenGL application: ● top is a command-line utility that you run in the Terminal window. You can use top to assess how much CPU time your application consumes. ● OpenGL Profiler is an application that determines how much time an application spends in OpenGL. It also provides function traces that you can use to look for redundant calls. ● OpenGL Driver Monitor lets you gather real-time data on the operation of the GPU and lets you look at information (OpenGL extensions supported, buffer modes, sample modes, and so forth) for the available renderers. For more information, see OpenGL Tools for Serious Graphics Development. This section shows how to use top along with OpenGL Profiler to analyze where to spend your optimization efforts—in your OpenGL code, your other application code, or in both. You'll see how to gather baseline data and how to determine the relationship of OpenGL performance to overall application performance. 1. Launch your OpenGL application. 2. Open a Terminal window and place it side-by-side with your application window. 3. In the Terminal window, type top and press Return. You'll see output similar to that shown in Figure 15-1. Tuning Your OpenGL Application Gathering and Analyzing Baseline Performance Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 156The top program indicates the amount of CPU time that an application uses. The CPU time serves as a good baseline value for gauging how much tuning your code needs. Figure 15-1 shows the percentage of CPU time for the OpenGL application GLCarbon1C (highlighted). Note this application utilizes 31.5% of CPU resources. Figure 15-1 Output produced by the top application Tuning Your OpenGL Application Gathering and Analyzing Baseline Performance Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 1574. Open the OpenGL Profiler application, located in /Developer/Applications/Graphics Tools/. In the window that appears, select the options to collect a trace and include backtraces, as shown in Figure 15-2. Figure 15-2 The OpenGL Profiler window 5. Select the option “Attach to application”, then select your application from the Application list. You may see small pauses or stutters in the application, particularly when OpenGL Profiler is collecting a function trace. This is normal and does not significantly affect the performance statistics. The glitches are due to the large amount of data that OpenGL Profiler is writing out. 6. Click Suspend to stop data collection. 7. Open the Statistics and Trace windows by choosing them from the Views menu. Figure 15-3 provides an example of what the Statistics window looks like. Figure 15-4 (page 160) shows a Trace window. The estimated percentage of time spent in OpenGL is shown at the bottom of Figure 15-3. Note that for this example, it is 28.91%. The higher this number, the more time the application is spending in OpenGL and the more opportunity there may be to improve application performance by optimizing OpenGL code. Tuning Your OpenGL Application Gathering and Analyzing Baseline Performance Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 158You can use the amount of time spent in OpenGL along with the CPU time to calculate a ratio of the application time versus OpenGL time. Thisratio indicates where to spend most of your optimization efforts. Figure 15-3 A statistics window 8. In the Trace window, look for duplicate function calls and redundant or unnecessary state changes. Look for back-to-back function calls with the same or similar data. These are areas that can typically be optimized. Functions that are called more than necessary include glTexParameter, glPixelStore, glEnable, and glDisable. For most applications, these functions can be called once from a setup or state modification routine and called only when necessary. It's generally good practice to keep state changes out of rendering loops(which can be seen in the function trace as the same sequence of state changes and drawing over and over again) as much as possible and use separate routines to adjust state as necessary. Tuning Your OpenGL Application Gathering and Analyzing Baseline Performance Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 159Look at the time value to the left of each function call to determine the cost of the call. Figure 15-4 A Trace window Use these to determine the cost of a call 9. Determine what the performance gain would be if it were possible to reduce the time to execute all OpenGL calls to zero. For example, take the performance data from the GLCarbon1C application used in thissection to determine the performance attributable to the OpenGL calls. Total Application Time (from top) = 31.5% Total Time in OpenGL (from OpenGL Profiler) = 28.91% At first glance, you might think that optimizing the OpenGL code could improve application performance by almost 29%, thusreducing the total application time by 29%. Thisisn't the case. Calculate the theoretical performance increase by multiplying the total CPU time by the percentage of time spent in OpenGL. The theoretical performance improvement for this example is: 31.5 X .2891 = 9.11% If OpenGL took no time at all to execute, the application would see a 9.11% increase in performance. So, if the application runs at 60 frames per second (FPS), it would perform as follows: New FPS = previous FPS * (1 +(% performance increase)) = 60 fps *(1.0911) = 65.47 fps The application gains almost 5.5 frames per second by reducing OpenGL from 28.91% to 0%. This shows that the relationship of OpenGL performance to application performance is not linear. Simply reducing the amount of time spent in OpenGL may or may not offer any noticeable benefit in application performance. Tuning Your OpenGL Application Gathering and Analyzing Baseline Performance Data 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 160Using OpenGL Driver Monitor to Measure Stalls You can use OpenGL Driver Monitor to measure how long the CPU waits for the GPU, as shown in Figure 15-5. OpenGL Driver Monitor is useful for analyzing other parameters as well. You can choose which parameters to monitor simply by clicking a parameter name from the drawer shown in the figure. Figure 15-5 The graph view in OpenGL Driver Monitor Identifying Bottlenecks with Shark Shark is an extremely useful tool for identifying places in your code that are slow and could benefit from optimization. Once you learn the basics, you can use it on your OpenGL applications to identify bottlenecks. There are three issues to watch out for in Shark when using it to analyze OpenGL performance: ● Costly data conversions. If you notice the glgProcessPixels call (in the libGLImage.dylib library) showing up in the analysis, it's an indication that the driver is not handling a texture upload optimally. The call is used when your application makes a glTexImage or glTexSubImage call using data that is in a nonnative format for the driver, which meansthe data must be converted before the driver can upload it. You can improve performance by changing your data so that it is in a native format for the driver. See “Use Optimal Data Types and Formats” (page 102). Tuning Your OpenGL Application Identifying Bottlenecks with Shark 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 161Note: If your data needs only to be swizzled, glgProcessPixels performs the swizzling reasonably fast although not as fast as if the data didn't need swizzling. But non-native data formats are converted one byte at a time and incurs a performance cost that is best to avoid. ● Time in the mach_kernel library. If you see time spent waiting for a timestamp or waiting for the driver, it indicates that your application is waiting for the GPU to finish processing. You see this during a texture upload, for example. ● Misleading symbols. You may see a symbol, such as glgGetString, that appears to be taking time but shouldn't be taking time in your application. Thatsometimes happens because the underlying optimizations performed by the system don't have any symbols attached to them on the driver side. Without a symbol to display, Shark shows the last symbol. You need to look for the call that your application made prior to that symbol and focus your attention there. You don't need to concern yourself with the calls that were made "underneath" your call. Tuning Your OpenGL Application Identifying Bottlenecks with Shark 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 162OpenGL functionality changes with each version of the OpenGL API. This appendix describes the functionality that was added with each version. See the official OpenGL specification for detailed information. The functionality for each version is guaranteed to be available through the OpenGL API even if a particular renderer does not support all of the extensions in a version. For example, a renderer that claims to support OpenGL 1.3 might not export the GL_ARB_texture_env_combine or GL_EXT_texture_env_combine extensions. It's important that you query both the renderer version and extension string to make sure that the renderer supports any functionality that you want to use. Note: It's possible for vendor and ARB extensions to provide similar functionality. As particular functionality becomes widely adopted, it can be moved into the core OpenGL API. As a result, functionality that you want to use could be included as an extension, as part of the core API, or both. You should read the extensions and the core OpenGL specifications carefully to see the differences. Furthermore, as an extension is promoted, the API associated with that functionality can change. For more information,see “Determining the OpenGL Capabilities Supported by the Renderer” (page 83). In the following tables, the extensions describe the feature that the core functionality is based on. The core functionality might not be the same as the extension. For example, compare the core texture crossbar functionality with the extension that it's based on. Version 1.1 Table A-1 Functionality added in OpenGL 1.1 Functionality Extension Copy texture and subtexture GL_EXT_copy_texture and GL_EXT_subtexture Logical operation GL_EXT_blend_logic_op Polygon offset GL_EXT_polygon_offset Texture image formats GL_EXT_texture 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 163 Legacy OpenGL Functionality by VersionFunctionality Extension Texture objects GL_EXT_texture_object Texture proxies GL_EXT_texture Texture replace environment GL_EXT_texture Vertex array GL_EXT_vertex_array There were a number of other minor changes outlined in Appendix C section 9 of the OpenGL specification. See http://www.opengl.org. Version 1.2 Table A-2 Functionality added in OpenGL 1.2 Functionality Extension BGRA pixel formats GL_EXT_bgra GL_SGI_color_table , GL_EXT_color_subtable, GL_EXT_convolution,GL_HP_convolution_border_modes, GL_SGI_color_matrix, GL_EXT_histogram, GL_EXT_blend_minmax, and GL_EXT_blend_subtract Imaging subset (optional) Normal rescaling GL_EXT_rescale_normal Packed pixel formats GL_EXT_packed_pixels Separate specular color GL_EXT_separate_specular_color Texture coordinate edge clamping GL_SGIS_texture_edge_clamp Texture level of detail control GL_SGIS_texture_lod Three-dimensional texturing GL_EXT_texture3D Vertex array draw element range GL_EXT_draw_range_elements Legacy OpenGL Functionality by Version Version 1.2 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 164Note: The imaging subset might not be present on all implementations; you must verify by checking for the ARB_imaging extension. OpenGL 1.2.1 introduced ARB extensions with no specific core API changes. Version 1.3 Table A-3 Functionality added in OpenGL 1.3 Functionality Extension Compressed textures GL_ARB_texture_compression Cube map textures GL_ARB_texture_cube_map Multisample GL_ARB_multisample Multitexture GL_ARB_multitexture Texture add environment mode GL_ARB_texture_env_add Texture border clamp GL_ARB_texture_border_clamp Texture combine environment mode GL_ARB_texture_env_combine Texture dot3 environment mode GL_ARB_texture_env_dot3 Transpose matrix GL_ARB_transpose_matrix Version 1.4 Table A-4 Functionality added in OpenGL 1.4 Functionality Extension Automatic mipmap generation GL_SGIS_generate_mipmap Blend function separate GL_ARB_blend_func_separate Blend squaring GL_NV_blend_square Depth textures GL_ARB_depth_texture Legacy OpenGL Functionality by Version Version 1.3 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 165Functionality Extension Fog coordinate GL_EXT_fog_coord Multiple draw arrays GL_EXT_multi_draw_arrays Point parameters GL_ARB_point_parameters Secondary color GL_EXT_secondary_color Separate blend functions GL_EXT_blend_func_separate, GL_EXT_blend_color Shadows GL_ARB_shadow Stencil wrap GL_EXT_stencil_wrap Texture crossbar environment mode GL_ARB_texture_env_crossbar Texture level of detail bias GL_EXT_texture_lod_bias Texture mirrored repeat GL_ARB_texture_mirrored_repeat Window raster position GL_ARB_window_pos Version 1.5 Table A-5 Functionality added in OpenGL 1.5 Functionality Extension Buffer objects GL_ARB_vertex_buffer_object Occlusion queries GL_ARB_occlusion_query Shadow functions GL_EXT_shadow_funcs Version 2.0 Table A-6 Functionality added in OpenGL 2.0 Functionality Extension Multiple render targets GL_ARB_draw_buffers Legacy OpenGL Functionality by Version Version 1.5 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 166Functionality Extension Non–power-of-two textures GL_ARB_texture_non_power_of_two Point sprites GL_ARB_point_sprite Separate blend equation GL_EXT_blend_equation_separate GL_ATI_separate_stencil GL_EXT_stencil_two_side Separate stencil Shading language GL_ARB_shading_language_100 Shader objects GL_ARB_shader_objects GL_ARB_fragment_shader GL_ARB_vertex_shader Shader programs Version 2.1 Table A-7 Functionality added in OpenGL 2.1 Functionality Extension Pixel buffer objects GL_ARB_pixel_buffer_object sRGB textures GL_EXT_texture_sRGB Legacy OpenGL Functionality by Version Version 2.1 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 167The OpenGL 3.0 specification deprecated many areas of functionality defined in earlier versions of the OpenGL specification. The OpenGL 3.2 Core profile explicitly removesthese deprecated features and adjusts other parts of the specification to provide a streamlined, clean programming interface to OpenGL. Use this chapter to assist you in migrating your application away from this deprecated functionality. Removed Functionality The features that were removed from OpenGL are described in in Appendix E of the OpenGL 3.2 Core specification, and you should use that as the definitive guide for the changes you need to make in your application. Here is a summary of most significant areas that changed: ● If your application uses the fixed-function pipeline, it must be rewritten to use shaders instead. ● If your application uses shaders, you must rewrite your shaders to use OpenGL Shading Language 1.5; many built-in shader variables provided in earlier versions of the OpenGL Shading Language were explicitly removed from the OpenGL Shading Language 1.5 specification. Similarly, your application may no longer provide vertex data using the fixed-function routines; all vertex attributes are now specified as generic vertex attributes. ● Your application must explicitly generate object names using the OpenGL API. ● Vertex data must be provided to OpenGL using buffer objects. ● The built-in matrix stack functionality from earlier versions of OpenGL has been removed; you must recreate this functionality using shader inputs. ● Support for auxiliary and accumulation buffers has been removed; use framebuffer objects instead. ● Your application no longer fetches the list of extensions as a single string. Instead, you first fetch the number of extensions and then separately fetch each extension string. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 168 Updating an Application to Support the OpenGL 3.2 Core SpecificationExtension Changes on OS X OpenGL 3.2 providesfunctionality that earlier versions ofOpenGL provided through extensions.Other extensions that were previously supported on OS X are no longer supported when your application uses the OpenGL 3.2 Core profile. Table B-1 lists extensions described elsewhere in this guide; use this table to determine whether the extension is supported, and if not, what equivalent functionality is supported. Table B-1 Extensions described in this guide Extension Status Obsolete. Use the ARB_Sync functionality provided by OpenGL 3.2 (Core). APPLE_fence ARB_vertex_buffer_object Functionality provided by OpenGL 3.2 (Core). Obsolete. Use the ARB_vertex_array_object functionality provided by OpenGL 3.2 (Core). APPLE_vertex_array_object Obsolete. Use the ARB_map_buffer_range functionality provided by OpenGL 3.2 (Core). APPLE_vertex_array_range Obsolete. Use the ARB_map_buffer_range functionality provided by OpenGL 3.2 (Core). APPLE_flush_buffer_range APPLE_client_storage Supported. APPLE_texture_range Supported. ARB_texture_rectangle Functionality provided by OpenGL 3.2 (Core). ARB_shader_objects Functionality provided by OpenGL 3.2 (Core). ARB_vertex_shader Functionality provided by OpenGL 3.2 (Core). ARB_fragment_shader Functionality provided by OpenGL 3.2 (Core). EXT_transform_feedback Functionality provided by OpenGL 3.2 (Core). EXT_gpu_shader4 Obsolete. Functionality included in GLSL 1.5 EXT_geometry_shader4 Functionality provided by OpenGL 3.2 (Core). Obsolete. Use the ARB_uniform_buffer_object functionality provided by OpenGL 3.2 (Core). EXT_bindable_uniform ARB_pixel_buffer_object Functionality provided by OpenGL 3.2 (Core). Updating an Application to Support the OpenGL 3.2 Core Specification Extension Changes on OS X 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 169Extension Status Obsolete. Use the ARB_framebuffer_object functionality provided by OpenGL 3.2 (Core). EXT_framebuffer_object APPLE_pixel_buffer Obsolete. Use framebuffer objects instead. Obsolete. Use multisampled renderbuffers to precisely control multisampling. NV_multisample_filter_hint Updating an Application to Support the OpenGL 3.2 Core Specification Extension Changes on OS X 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 170Function pointers to OpenGL routines allow you to deploy your application across multiple versions of OS X regardless of whether the entry point is supported at link time or runtime. This practice also provides support for code that needs to run cross-platform—in both OS X and Windows. Note: If you are deploying your application only in OS X v10.4 or later, you do not need to read this chapter. Instead, consider the alternative, which is to set the gcc attribute that allows weak linking of symbols. Keep in mind, however, that weak linking may impact your application's performance. For more information, see “Frameworks and Weak Linking”. This appendix discusses the tasks needed to set up and use function pointers as entry points to OpenGL routines: ● “Obtaining a Function Pointer to an Arbitrary OpenGL Entry Point” (page 171)shows how to write a generic routine that you can reuse for any OpenGL application on the Macintosh platform. ● “Initializing Entry Points” (page 172) describes how to declare function pointer type definitions and initialize them with the appropriate OpenGL command entry points for your application. Obtaining a Function Pointer to an Arbitrary OpenGL Entry Point Getting a pointer to an OpenGL entry point function is fairly straightforward from Cocoa. You can use the Dynamic Loader function NSLookupAndBindSymbol to get the address of an OpenGL entry point. Keep in mind that getting a valid function pointer means that the entry point is exported by the OpenGL framework; it does not guarantee that a particular routine is supported and valid to call from within your application. You still need to check for OpenGL functionality on a per-renderer basis as described in “Detecting Functionality” (page 83). Listing C-1 shows how to use NSLookupAndBindSymbol from within the function MyNSGLGetProcAddress. When provided a symbol name, this application-defined function returns the appropriate function pointer from the global symbol table. A detailed explanation for each numbered line of code appears following the listing. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 171 Setting Up Function Pointers to OpenGL RoutinesListing C-1 Using NSLookupAndBindSymbol to obtain a symbol for a symbol name #import #import #import void * MyNSGLGetProcAddress (const char *name) { NSSymbol symbol; char *symbolName; symbolName = malloc (strlen (name) + 2); // 1 strcpy(symbolName + 1, name); // 2 symbolName[0] = '_'; // 3 symbol = NULL; if (NSIsSymbolNameDefined (symbolName)) // 4 symbol = NSLookupAndBindSymbol (symbolName); free (symbolName); // 5 return symbol ? NSAddressOfSymbol (symbol) : NULL; // 6 } Here's what the code does: 1. Allocates storage for the symbol name plus an underscore character ('_'). The underscore character is part of the UNIX C symbol-mangling convention, so make sure that you provide storage for it. 2. Copiesthe symbol name into the string variable,starting at the second character, to leave room for prefixing the underscore character. 3. Copies the underscore character into the first character of the symbol name string. 4. Checks to make sure that the symbol name is defined, and if it is, looks up the symbol. 5. Frees the symbol name string because it is no longer needed. 6. Returns the appropriate pointer if successful, or NULL if not successful. Before using this pointer, you should make sure that is it valid. Initializing Entry Points Listing C-2 shows how to use the MyNSGLGetProcAddress function from Listing C-1 (page 172) to obtain a few OpenGL entry points. A detailed explanation for each numbered line of code appears following the listing. Setting Up Function Pointers to OpenGL Routines Initializing Entry Points 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 172Listing C-2 Using NSGLGetProcAddress to obtain an OpenGL entry point #import "MyNSGLGetProcAddress.h" // 1 static void InitEntryPoints (void); static void DeallocEntryPoints (void); // Function pointer type definitions typedef void (*glBlendColorProcPtr)(GLclampf red,GLclampf green, GLclampf blue,GLclampf alpha); typedef void (*glBlendEquationProcPtr)(GLenum mode); typedef void (*glDrawRangeElementsProcPtr)(GLenum mode, GLuint start, GLuint end,GLsizei count,GLenum type,const GLvoid *indices); glBlendColorProcPtr pfglBlendColor = NULL; // 2 glBlendEquationProcPtr pfglBlendEquation = NULL; glDrawRangeElementsProcPtr pfglDrawRangeElements = NULL; static void InitEntryPoints (void) // 3 { pfglBlendColor = (glBlendColorProcPtr) MyNSGLGetProcAddress ("glBlendColor"); pfglBlendEquation = (glBlendEquationProcPtr)MyNSGLGetProcAddress ("glBlendEquation"); pfglDrawRangeElements = (glDrawRangeElementsProcPtr)MyNSGLGetProcAddress ("glDrawRangeElements"); } // ------------------------- static void DeallocEntryPoints (void) // 4 { pfglBlendColor = NULL; pfglBlendEquation = NULL; pfglDrawRangeElements = NULL;; } Here's what the code does: 1. Imports the header file that contains the MyNSGLProcAddress function from Listing C-1 (page 172). Setting Up Function Pointers to OpenGL Routines Initializing Entry Points 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 1732. Declares function pointers for the functions of interest. Note that each function pointer uses the prefix pf to distinguish it from the function it points to. Although using this prefix is not a requirement, it's best to avoid using the exact function names. 3. Initializes the entry points. This function repeatedly calls the MyNSGLProcAddress function to obtain function pointers for each of the functions of interest—glBlendColor, glBlendEquation, and glDrawRangeElements. 4. Sets each of the function pointers to NULL when they are no longer needed. Setting Up Function Pointers to OpenGL Routines Initializing Entry Points 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 174This table describes the changes to OpenGL Programming Guide for Mac . Date Notes 2012-07-23 Updated with information on supporting high-resolution displays. 2011-06-06 Added new context options. 2010-11-15 Fixed a few small errors in the texture chapter. Updated the recommendations on when to use each texture uploading and downloading technique. Updated the code for creating a texture from a view’s contents to use newer, better supported techniques. 2010-06-14 Corrected texture creation code snippets. 2010-03-24 Minor updates and clarifications. Substantial revisions to describe behaviors for OpenGL on OS X v10.5 and OS X v10.6. Removed information on obsolete and deprecated behaviors. 2010-02-24 Corrected errors in code listings. Pixel format attribute lists should be terminated with 0, not NULL. One call to glTexImage2D had an incorrect number of parameters. 2009-08-28 Updated the Cocoa OpenGL tutorial and made numerous other minor changes. 2008-06-09 Fixed compilation errors in Listing 8-1 (page 84). Added “Getting Decompressed Raw Pixel Data from a Source Image” (page 135). Updated links to OpenGL extensions. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 175 Document Revision HistoryDate Notes Made several minor edits. 2007-12-04 Corrected minor typographical and technical errors. Added “Ensuring That Back Buffer Contents Remain the Same” (page 66). Revised “Deprecated Attributes” (page 70). 2007-08-07 Fixed several technical issues. 2007-05-29 Fixed a broken link. 2007-05-17 Fixed a few technical inaccuracies in the code listings. Changed attribs to attributes in Listing 6-2 (page 68). Fixed drawRect method implementation in “Drawing to a Window or View” (page 35). 2006-12-20 Fixed minor errors. Added information concerning the Apple client storage extension. Fixed a typographical error. 2006-11-07 Added information about performance issues and processor queries. See “Determining Whether Vertex and Fragment Processing Happens on the GPU” (page 78). 2006-10-03 Added a section on checking for GPU processing. Added “Determining Whether Vertex and Fragment Processing Happens on the GPU” (page 78). Fixed a number of minor typos in the code and in the text. 2006-09-05 Fixed minor technical problems. 2006-07-24 Made minor technical and typograhical changes throughout. Added information to “Surface Drawing Order Specifies the Position of the OpenGL Surface Relative to the Window” (page 77). Document Revision History 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 176Date Notes Changed glCopyTexSubImage to glCopyTexSubImage2D in “Downloading Texture Data” (page 136). Made minor improvements to Listing 11-6 (page 136). Removed information about 1-D textures. 2006-06-28 Made several minor technical corrections. Redirected links to the OpenGL specification for the framebuffer object extension so that they point to the SGI Open Source website, which hosts the most up-to-date version of this specification. Removed the logic operation blending entry from Table A-6 (page 166) because this functionality is not available in OpenGL 2.0. 2006-05-23 First version. This document replaces Macintosh OpenGL Programming Guide and AGL Programming Guide . This document incorporates information from the following Technical Notes: TN2007 “The CGDirectDisplay API” TN2014 “Insights on OpenGL” TN2080 “Understanding and Detecting OpenGL Functionality” TN2093 “OpenGL Performance Optimization: The Basics” This document incorporates information from the following Technical Q&As: Technical Q&A OGL01 “aglChoosePixelFormat, The Inside Scoop” Technical Q&A OGL02 “Correct Setup of an AGLDrawable” Technical Q&A QA1158 “glFlush() vs. glFinish()” Technical Q&A QA1167 “Using Interface Builder's NSOpenGLView or Custom View objects for an OpenGL application” Technical Q&A QA1188 “GetProcAdress and OpenGL Entry Points” Document Revision History 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 177Date Notes Technical Q&A QA1209 “Updating OpenGL Contexts” Technical Q&A QA1248 “Context Sharing Tips” Technical Q&A QA1268 “Sharpening Full Scene Anti-Aliasing Details” Technical Q&A QA1269 “OS X OpenGL Interfaces” Technical Q&A QA1325 “Creating an OpenGL texture from an NSView” Document Revision History 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 178This glossary containstermsthat are used specifically for the Apple implementation of OpenGL and a few terms that are common in graphics programming. For definitions of additional OpenGL terms, see OpenGL Programming Guide, by the Khronos OpenGL Working Group aliased Said of graphics whose edges appear jagged; can be remedied by performing antialiasing operations. antialiasing In graphics, a technique used to smooth and soften the jagged (or aliased) edges that are sometimes apparent when graphical objects such as text, line art, and images are drawn. ARB The Khronos OpenGL Working Group, which is the group that oversees the OpenGL specification and extensions to it. attach To establish a connection between two existing objects. Compare bind. bind To create a new object and then establish a connection between that object and a rendering context. Compare attach. bitmap A rectangular array of bits. bitplane A rectangular array of pixels. buffer A block of memory dedicated to storing a specific kind of data, such as depth values, green color values, stencil index values, and color index values. CGL (Core OpenGL) framework The Apple framework for using OpenGL graphics in OS X applications that need low-level access to OpenGL. clipping An operation that identifies the area of drawing. Anything not in the clipping region is not drawn. clip coordinates The coordinate system used for view-volume clipping. Clip coordinates are applied after applying the projection matrix and prior to perspective division. color lookup table A table of values used to map color indexes into actual color values. completeness A state that indicates whether a framebuffer object meets all the requirements for drawing. context A set of OpenGL state variables that affect how drawing is performed for a drawable object attached to that context. Also called a rendering context. culling Eliminating parts of a scene that can't be seen by the observer. current context The rendering context to which OpenGL routes commands issued by your application. current matrix A matrix used by OpenGL to transform coordinates in one system to those of another system, such as the modelview matrix, the perspective matrix, and the texture matrix. GL shading language allows user-defined matrices. 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 179 Glossarydepth In OpenGL, refers to the z coordinate and specifies how far a pixel lies from the observer. depth buffer A block of memory used to store a depth value for each pixel. The depth buffer is used to determine whether or not a pixel can be seen by the observer. Those that are hidden are typically removed. display list A list of OpenGL commands that have an associated name and that are uploaded to the GPU, preprocessed, and then executed at a later time. Display lists are often used for computing-intensive commands. double buffering The practice of using a front and back color buffer to achieve smooth animation. The back buffer is not displayed, but swapped with the front buffer. drawable object In OS X, an object allocated outside of OpenGL that can serve as an OpenGL framebuffer. A drawable object can be any of the following: a window, a view, a pixel buffer, offscreen memory, or a full-screen graphics device. See also framebuffer object extension A feature of OpenGL that's not part of the OpenGL core API and therefore not guaranteed to be supported by every implementation of OpenGL. The naming conventions used for extensions indicate how widely accepted the extension is. The name of an extension supported only by a specific company includes an abbreviation of the company name. If more then one company adoptsthe extension,the extension name is changed to include EXT instead of a company abbreviation. If the Khronos OpenGL Working Group approves an extension, the extension name changes to include ARB instead of EXT or a company abbreviation. eye coordinates The coordinate system with the observer at the origin. Eye coordinates are produced by the modelview matrix and passed to the projection matrix. fence A token used by the GL_APPLE_fence extension to determine whether a given command has completed or not. filtering A process that modifies an image by combining pixels or texels. fog An effect achieved by fading colors to a background color based on the distance from the observer. Fog provides depth cues to the observer. fragment The color and depth values for a single pixel; can also include texture coordinate values. A fragment is the result of rasterizing primitives. framebuffer The collection of buffers associated with a window or a rendering context. framebuffer attachable image The rendering destination for a framebuffer object. framebuffer object An OpenGL extension that allows rendering to a destination other than the usual OpenGL buffers or destinations provided by the windowing system. A framebuffer object (FBO) contains state information for the OpenGL framebuffer and its set of images. A framebuffer object is similar to a drawable object, except that a drawable object is a window-system specific object whereas a framebuffer object is a window-agnostic object. The context that's bound to a framebuffer object can be bound to a window-system-provided drawable object for the purpose of displaying the content associated with the framebuffer object. frustum The region of space that is seen by the observer and that is warped by perspective division. Glossary 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 180FSAA (full scene antialiasing) A technique that takes multiple samples at a pixel and combinesthem with coverage values to arrive at a final fragment. gamma correction A function that changes color intensity valuesto correct for the nonlinear response of the eye or of a display. GLU Graphics library utilities. GL Graphics library. GLUT Graphics Library Utilities Toolkit, which is independent of the window system. In OS X, GLUT is implemented on top of Cocoa. GLX An OpenGL extension that supports using OpenGL within a window provided by the X Window system. image A rectangular array of pixels. immediatemode The practice ofOpenGL executing commands at the time an application issues them. To prevent commands from being issued immediately, an application can use a display list. interleaved data Arrays of dissimilar data that are grouped together, such as vertex data and texture coordinates. Interleaving can speed data retrieval. mipmaps A set of texture maps, provided at various resolutions, whose purpose is to minimize artifacts that can occur when a texture is applied to a geometric primitive whose onscreen resolution doesn't match the source texture map. Mipmapping derivesfromthe latin phrasemultumin parvo , which means "many things in a small place." modelview matrix A 4 X 4 matrix used by OpenGL to transforms points, lines, polygons, and positions from object coordinates to eye coordinates. mutex A mutual exclusion object in a multithreaded application. NURBS (nonuniform rational basis spline) A methodology use to specify parametric curves and surfaces. packing Converting pixel color components from a buffer into the format needed by an application. pbuffer See pixel buffer. pixel A picture element; the smallest element that the graphics hardware can display on the screen. A pixel is made up of all the bits at the location x , y , in all the bitplanes in the framebuffer. pixel buffer A type of drawable object that allows the use of offscreen buffers as sources for OpenGL texturing. Pixel buffers allow hardware-accelerated rendering to a texture. pixel depth The number of bits per pixel in a pixel image. pixel format A format used to store pixel data in memory. The format describesthe pixel components (that is, red, blue, green, alpha), the number and order of components, and other relevant information,such as whether a pixel containsstencil and depth values. primitives The simplest elements in OpenGL—points, lines, polygons, bitmaps, and images. projection matrix A matrix that OpenGL uses to transform points, lines, polygons, and positionsfrom eye coordinates to clip coordinates. rasterization The process of converting vertex and pixel data to fragments, each of which corresponds to a pixel in the framebuffer. renderbuffer A rendering destination for a 2D pixel image, used for generalized offscreen rendering, as defined in the OpenGL specification for the GL_EXT_framebuffer_object extension. Glossary 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 181renderer A combination of hardware and software that OpenGL uses to create an image from a view and a model. The hardware portion of a renderer is associated with a particular display device and supports specific capabilities, such as the ability to support a certain color depth or buffering mode. A renderer that uses only software is called a software renderer and is typically used as a fallback. rendering context A container forstate information. rendering pipeline The order of operations used by OpenGL to transform pixel and vertex data to an image in the framebuffer. render-to-texture An operation that draws content directly to a texture target. RGBA Red, green, blue, and alpha color components. shader A programthat computessurface properties. shading language A high-level language, accessible in C, used to produce advanced imaging effects. stencil buffer Memory used specifically for stencil testing. A stencil test is typically used to identify masking regions, to identify solid geometry that needs to be capped, and to overlap translucent polygons. surface The internal representation of a single buffer that OpenGL actually drawsto and readsfrom. For windowed drawable objects, thissurface is what the OS X window server uses to composite OpenGL content on the desktop. tearing A visual anomaly caused when part of the current frame overwrites previous frame data in the framebuffer before the current frame is fully rendered on the screen. tessellation An operation that reduces a surface to a mesh of polygons, or a curve to a sequence of lines. texel A texture element used to specify the color to apply to a fragment. texture Image data used to modify the color of rasterized fragments; can be one-, two-, or threedimensional or be a cube map. texture mapping The process of applying a texture to a primitive. texture matrix A 4 x 4 matrix that OpenGL uses to transform texture coordinates to the coordinates that are used for interpolation and texture lookup. texture object An opaque data structure used to store all data related to a texture. A texture object can include such things as an image, a mipmap, and texture parameters (width, height, internal format, resolution, wrapping modes, and so forth). vertex A three-dimensional point. A set of vertices specify the geometry of a shape. Vertices can have a number of additional attributes such as color and texture coordinates. See vertex array. vertex array A data structure that stores a block of data thatspecifiessuch things as vertex coordinates, texture coordinates, surface normals, RGBA colors, color indices, and edge flags. virtual screen A combination of hardware, renderer, and pixel format that OpenGL selects as suitable for an imaging task. When the current virtual screen changes, the current renderer typically changes. Glossary 2012-07-23 | © 2004, 2012 Apple Inc. All Rights Reserved. 182Apple Inc. © 2004, 2012 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, Carbon, Cocoa, iChat, Instruments, iPhoto, Logic, Mac, Macintosh, Objective-C, OS X, Pages, Quartz, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries. OpenCL is a trademark of Apple Inc. DEC is a trademark of Digital Equipment Corporation. OpenGL is a registered trademark of Silicon Graphics, Inc. UNIX is a registered trademark of The Open Group. X Window System is a trademark of the Massachusetts Institute of Technology. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. View Programming Guide for iOSContents About Windows and Views 7 At a Glance 7 Views Manage Your Application’s Visual Content 7 Windows Coordinate the Display of Your Views 8 Animations Provide the User with Visible Feedback for Interface Changes 8 The Role of Interface Builder 8 See Also 9 View and Window Architecture 10 View Architecture Fundamentals 10 View Hierarchies and Subview Management 11 The View Drawing Cycle 12 Content Modes 13 Stretchable Views 15 Built-In Animation Support 16 View Geometry and Coordinate Systems 17 The Relationship of the Frame, Bounds, and Center Properties 18 Coordinate System Transformations 20 Points Versus Pixels 21 The Runtime Interaction Model for Views 23 Tips for Using Views Effectively 25 Views Do Not Always Have a Corresponding View Controller 25 Minimize Custom Drawing 26 Take Advantage of Content Modes 26 Declare Views as Opaque Whenever Possible 26 Adjust Your View’s Drawing Behavior When Scrolling 26 Do Not Customize Controls by Embedding Subviews 27 Windows 28 Tasks That Involve Windows 28 Creating and Configuring a Window 29 Creating Windows in Interface Builder 29 Creating a Window Programmatically 30 Adding Content to Your Window 30 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 2Changing the Window Level 31 Monitoring Window Changes 31 Displaying Content on an External Display 32 Handling Screen Connection and Disconnection Notifications 33 Configuring a Window for an External Display 35 Configuring the Screen Mode of an External Display 37 Views 38 Creating and Configuring View Objects 38 Creating View Objects Using Interface Builder 39 Creating View Objects Programmatically 39 Setting the Properties of a View 40 Tagging Views for Future Identification 42 Creating and Managing a View Hierarchy 42 Adding and Removing Subviews 43 Hiding Views 46 Locating Views in a View Hierarchy 47 Translating, Scaling, and Rotating Views 47 Converting Coordinates in the View Hierarchy 50 Adjusting the Size and Position of Views at Runtime 51 Being Prepared for Layout Changes 51 Handling Layout Changes Automatically Using Autoresizing Rules 52 Tweaking the Layout of Your Views Manually 54 Modifying Views at Runtime 54 Interacting with Core Animation Layers 56 Changing the Layer Class Associated with a View 56 Embedding Layer Objects in a View 57 Defining a Custom View 58 Checklist for Implementing a Custom View 58 Initializing Your Custom View 59 Implementing Your Drawing Code 60 Responding to Events 62 Cleaning Up After Your View 63 Animations 64 What Can Be Animated? 64 Animating Property Changes in a View 66 Starting Animations Using the Block-Based Methods 66 Starting Animations Using the Begin/Commit Methods 68 Nesting Animation Blocks 72 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 3 ContentsImplementing Animations That Reverse Themselves 73 Creating Animated Transitions Between Views 73 Changing the Subviews of a View 74 Replacing a View with a Different View 76 Linking Multiple Animations Together 77 Animating View and Layer Changes Together 77 Document Revision History 80 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 4 ContentsFigures, Tables, and Listings View and Window Architecture 10 Figure 1-1 Architecture of the views in a sample application 11 Figure 1-2 Content mode comparisons 14 Figure 1-3 Stretching the background of a button 15 Figure 1-4 Coordinate system orientation in UIKit 17 Figure 1-5 Relationship between a view's frame and bounds 19 Figure 1-6 Rotating a view and its content 21 Figure 1-7 UIKit interactions with your view objects 23 Table 1-1 Screen dimensions for iOS-based devices 22 Windows 28 Listing 2-1 Registering for screen connect and disconnect notifications 33 Listing 2-2 Handling connect and disconnect notifications 34 Listing 2-3 Configuring a window for an external display 35 Views 38 Figure 3-1 Layered views in the Clock application 43 Figure 3-2 Rotating a view 45 degrees 49 Figure 3-3 Converting values in a rotated view 51 Figure 3-4 View autoresizing mask constants 53 Table 3-1 Usage of some key view properties 40 Table 3-2 Autoresizing mask constants 52 Listing 3-1 Adding a view to a window 44 Listing 3-2 Adding views to an existing view hierarchy 45 Listing 3-3 Adding a custom layer to a view 57 Listing 3-4 Initializing a view subclass 59 Listing 3-5 A drawing method 61 Listing 3-6 Implementing the dealloc method 63 Animations 64 Table 4-1 Animatable UIView properties 64 Table 4-2 Methods for configuring animation blocks 69 Listing 4-1 Performing a simple block-based animation 66 Listing 4-2 Creating an animation block with custom options 67 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 5Listing 4-3 Performing a simple begin/commit animation 69 Listing 4-4 Configuring animation parameters using the begin/commit methods 70 Listing 4-5 Nesting animations that have different configurations 72 Listing 4-6 Swapping an empty text view for an existing one 74 Listing 4-7 Changing subviews using the begin/commit methods 75 Listing 4-8 Toggling between two views in a view controller 76 Listing 4-9 Mixing view and layer animations 78 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 6 Figures, Tables, and ListingsIn iOS, you use windows and views to present your application’s content on the screen. Windows do not have any visible content themselves but provide a basic container for your application’s views. Views define a portion of a window that you want to fill with some content. For example, you might have views that display images, text, shapes, or some combination thereof. You can also use views to organize and manage other views. At a Glance Every application has at least one window and one view for presenting its content. UIKit and other system frameworks provide predefined viewsthat you can use to present your content. These viewsrange from simple buttons and text labels to more complex views such as table views, picker views, and scroll views. In places where the predefined views do not provide what you need, you can also define custom views and manage the drawing and event handling yourself. Views Manage Your Application’s Visual Content A view is an instance of the UIView class (or one of its subclasses) and manages a rectangular area in your application window. Views are responsible for drawing content, handling multitouch events, and managing the layout of any subviews. Drawing involves using graphics technologies such as Core Graphics, OpenGL ES, or UIKit to draw shapes, images, and text inside a view’s rectangular area. A view responds to touch events in its rectangular area either by using gesture recognizers or by handling touch events directly. In the view hierarchy, parent views are responsible for positioning and sizing their child views and can do so dynamically. This ability to modify child views dynamically lets your views adjust to changing conditions, such as interface rotations and animations. You can think of views as building blocks that you use to construct your user interface. Rather than use one view to present all of your content, you often use several views to build a view hierarchy. Each view in the hierarchy presents a particular portion of your user interface and is generally optimized for a specific type of content. For example, UIKit has views specifically for presenting images, text and other types of content. 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 7 About Windows and ViewsRelevant Chapters: “View and Window Architecture” (page 10) “Views” (page 38) Windows Coordinate the Display of Your Views A window is an instance of the UIWindow class and handles the overall presentation of your application’s user interface. Windows work with views (and their owning view controllers) to manage interactions with, and changes to, the visible view hierarchy. For the most part, your application’s window never changes. After the window is created, it stays the same and only the views displayed by it change. Every application has at least one window that displays the application’s user interface on a device’s main screen. If an external display is connected to the device, applications can create a second window to present content on that screen as well. Relevant Chapters: “Windows” (page 28) Animations Provide the User with Visible Feedback for Interface Changes Animations provide users with visible feedback about changes to your view hierarchy. The system defines standard animationsfor presenting modal views and transitioning between different groups of views. However, many attributes of a view can also be animated directly. For example, through animation you can change the transparency of a view, its position on the screen, its size, its background color, or other attributes. And if you work directly with the view’s underlying Core Animation layer object, you can perform many other animations as well. Relevant Chapters: “Animations” (page 64) The Role of Interface Builder Interface Builder is an application that you use to graphically construct and configure your application’s windows and views. Using Interface Builder, you assemble your views and place them in a nib file, which is a resource file that stores a freeze-dried version of your views and other objects. When you load a nib file at runtime, the objects inside it are reconstituted into actual objects that your code can then manipulate programmatically. Interface Builder greatly simplifiesthe work you have to do in creating your application’s user interface. Because support for Interface Builder and nib files is incorporated throughout iOS, little effort is required to incorporate nib files into your application’s design. About Windows and Views The Role of Interface Builder 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 8For more information about how to use Interface Builder, see Interface Builder User Guide . For information about how view controllers manage the nib files containing their views, see “Custom View Controllers” in View Controller Programming Guide for iOS . See Also Because views are very sophisticated and flexible objects, it would be impossible to cover all of their behaviors in one document. However, other documents are available to help you learn about other aspects of managing views and your user interface as a whole. ● View controllers are an important part of managing your application’s views. A view controller presides over all of the viewsin a single view hierarchy and facilitatesthe presentation of those views on the screen. For more information about view controllers and the role they play, see View Controller Programming Guide for iOS . ● Views are the key recipients of gesture and touch events in your application. For more information about using gesture recognizers and handling touch events directly, see Event Handling Guide for iOS . ● Custom views must use the available drawing technologies to render their content. For information about using these technologies to draw within your views, see Drawing and Printing Guide for iOS . ● In places where the standard view animations are notsufficient, you can use Core Animation. For information about implementing animations using Core Animation, see Core Animation Programming Guide . About Windows and Views See Also 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 9Views and windows present your application’s user interface and handle the interactions with that interface. UIKit and other system frameworks provide a number of views that you can use as-is with little or no modification. You can also define custom views for places where you need to present content differently than the standard views allow. Whether you use the system views or create your own custom views, you need to understand the infrastructure provided by the UIView and UIWindow classes. These classes provide sophisticated facilities for managing the layout and presentation of views. Understanding how those facilities work is important for making sure your views behave appropriately when changes occur in your application. View Architecture Fundamentals Most of the things you might want to do visually are done with view objects—instances of the UIView class. A view object defines a rectangular region on the screen and handles the drawing and touch events in that region. A view can also act as a parent for other views and coordinate the placement and sizing of those views. The UIView class does most of the work in managing these relationships between views, but you can also customize the default behavior as needed. Views work in conjunction with Core Animation layers to handle the rendering and animating of a view’s content. Every view in UIKit is backed by a layer object (usually an instance of the CALayer class), which manages the backing store for the view and handles view-related animations. Most operations you perform should be through the UIView interface. However, in situations where you need more control over the rendering or animation behavior of your view, you can perform operations through its layer instead. To understand the relationship between views and layers, it helps to look at an example. Figure 1-1 shows the view architecture from the ViewTransitions sample application along with the relationship to the underlying Core Animation layers. The views in the application include a window (which is also a view), a generic UIView object that acts as a container view, an image view, a toolbar for displaying controls, and a bar button item (which is not a view itself but which manages a view internally). (The actual ViewTransitions sample application includes an additional image view that is used to implement transitions. For simplicity, and because that view is usually hidden, it is not included in Figure 1-1.) Every view has a corresponding layer object that can be 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 10 View and Window Architectureaccessed from that view’s layer property. (Because a bar button item is not a view, you cannot access its layer directly.) Behind those layer objects are Core Animation rendering objects and ultimately the hardware buffers used to manage the actual bits on the screen. Figure 1-1 Architecture of the views in a sample application UIKit views Core Animation layers UIImageView UIView UIWindow UIToolbar UIBarButtonItem (internal view) The use of Core Animation layer objects has important implications for performance. The actual drawing code of a view object is called as little as possible, and when the code is called, the results are cached by Core Animation and reused as much as possible later. Reusing already-rendered content eliminates the expensive drawing cycle usually needed to update views. Reuse of this content is especially important during animations, where the existing content can be manipulated. Such reuse is much less expensive than creating new content. View Hierarchies and Subview Management In addition to providing its own content, a view can act as a container for other views. When one view contains another, a parent-child relationship is created between the two views. The child view in the relationship is known asthe subview and the parent view is known asthe superview. The creation of thistype of relationship has implications for both the visual appearance of your application and the application’s behavior. Visually, the content of a subview obscures all or part of the content of its parent view. If the subview is totally opaque, then the area occupied by the subview completely obscures the corresponding area of the parent. If the subview is partially transparent, the content from the two viewsis blended together prior to being displayed View and Window Architecture View Architecture Fundamentals 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 11on the screen. Each superview stores its subviews in an ordered array and the order in that array also affects the visibility of each subview. If two sibling subviews overlap each other, the one that was added last (or was moved to the end of the subview array) appears on top of the other. The superview-subview relationship also impacts several view behaviors. Changing the size of a parent view has a ripple effect that can cause the size and position of any subviews to change too. When you change the size of a parent view, you can control the resizing behavior of each subview by configuring the view appropriately. Other changes that affect subviews include hiding a superview, changing a superview’s alpha (transparency), or applying a mathematical transform to a superview’s coordinate system. The arrangement of views in a view hierarchy also determines how your application responds to events. When a touch occurs inside a specific view, the system sends an event object with the touch information directly to that view for handling. However, if the view does not handle a particular touch event, it can pass the event object along to its superview. If the superview does not handle the event, it passes the event object to its superview, and so on up the responder chain. Specific views can also pass the event object to an intervening responder object, such as a view controller. If no object handles the event, it eventually reaches the application object, which generally discards it. For more information about how to create view hierarchies,see “Creating and Managing a View Hierarchy” (page 42). The View Drawing Cycle The UIView class uses an on-demand drawing model for presenting content. When a view first appears on the screen, the system asks it to draw its content. The system captures a snapshot of this content and uses that snapshot as the view’s visual representation. If you never change the view’s content, the view’s drawing code may never be called again. The snapshot image is reused for most operations involving the view. If you do change the content, you notify the system that the view has changed. The view then repeats the process of drawing the view and capturing a snapshot of the new results. When the contents of your view change, you do not redraw those changes directly. Instead, you invalidate the view using either the setNeedsDisplay or setNeedsDisplayInRect: method. These methods tell the system that the contents of the view changed and need to be redrawn at the next opportunity. The system waits until the end of the current run loop before initiating any drawing operations. This delay gives you a chance to invalidate multiple views, add or remove views from your hierarchy, hide views, resize views, and reposition views all at once. All of the changes you make are then reflected at the same time. View and Window Architecture View Architecture Fundamentals 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 12Note: Changing a view’s geometry does not automatically cause the system to redraw the view’s content. The view’s contentMode property determines how changes to the view’s geometry are interpreted. Most content modes stretch or reposition the existing snapshot within the view’s boundaries and do not create a new one. For more information about how content modes affect the drawing cycle of your view, see “Content Modes” (page 13). When the time comes to render your view’s content, the actual drawing process varies depending on the view and its configuration. System views typically implement private drawing methods to render their content. Those same system views often expose interfaces that you can use to configure the view’s actual appearance. For custom UIView subclasses, you typically override the drawRect: method of your view and use that method to draw your view’s content. There are also other ways to provide a view’s content, such as setting the contents of the underlying layer directly, but overriding the drawRect: method is the most common technique. For more information about how to draw content for custom views, see “Implementing Your Drawing Code” (page 60). Content Modes Each view has a content mode that controls how the view recycles its content in response to changes in the view’s geometry and whether it recycles its content at all. When a view is first displayed, it renders its content as usual and the results are captured in an underlying bitmap. After that, changes to the view’s geometry do not always cause the bitmap to be recreated. Instead, the value in the contentMode property determines whether the bitmap should be scaled to fit the new bounds or simply pinned to one corner or edge of the view. The content mode of a view is applied whenever you do the following: ● Change the width or height of the view’s frame or bounds rectangles. ● Assign a transform that includes a scaling factor to the view’s transform property. View and Window Architecture View Architecture Fundamentals 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 13By default, the contentMode property for most views is set to UIViewContentModeScaleToFill, which causes the view’s contents to be scaled to fit the new frame size. Figure 1-2 shows the results that occur for some content modes that are available. As you can see from the figure, not all content modes result in the view’s bounds being filled entirely, and those that do might distort the view’s content. Figure 1-2 Content mode comparisons UIViewContentModeScaleToFill Distorting Nondistorting UIViewContentModeScaleAspectFit UIViewContentModeScaleAspectFill Nondistorting Nondistorting UIViewContentModeLeft View and Window Architecture View Architecture Fundamentals 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 14Content modes are good for recycling the contents of your view, but you can also set the content mode to the UIViewContentModeRedraw value when you specifically want your custom views to redraw themselves during scaling and resizing operations. Setting your view’s content mode to this value forces the system to call your view’s drawRect: method in response to geometry changes. In general, you should avoid using this value whenever possible, and you should certainly not use it with the standard system views. For more information about the available content modes, see UIView Class Reference . Stretchable Views You can designate a portion of a view asstretchable so that when the size of the view changes only the content in the stretchable portion is affected. You typically use stretchable areas for buttons or other views where part of the view defines a repeatable pattern. The stretchable area you specify can allow for stretching along one or both axes of the view. Of course, when stretching a view along two axes, the edges of the view must also define a repeatable pattern to avoid any distortion. Figure 1-3 shows how this distortion manifests itself in a view. The color from each of the view’s original pixels is replicated to fill the corresponding area in the larger view. Figure 1-3 Stretching the background of a button (0,0) (1,1) View and Window Architecture View Architecture Fundamentals 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 15You specify the stretchable area of a view using the contentStretch property. This property accepts a rectangle whose values are normalized to the range 0.0 to 1.0. When stretching the view, the system multiplies these normalized values by the view’s current bounds and scale factor to determine which pixel or pixels need to be stretched. The use of normalized values alleviates the need for you to update the contentStretch property every time the bounds of your view change. The view’s content mode also plays a role in determining how the view’s stretchable area is used. Stretchable areas are only used when the content mode would cause the view’s content to be scaled. This means that stretchable views are supported only with the UIViewContentModeScaleToFill, UIViewContentModeScaleAspectFit, and UIViewContentModeScaleAspectFill content modes. If you specify a content mode that pins the content to an edge or corner (and thus does not actually scale the content), the view ignores the stretchable area. Note: The use of the contentStretch property isrecommended over the creation of a stretchable UIImage object when specifying the background for a view. Stretchable views are handled entirely in the Core Animation layer, which typically offers better performance. Built-In Animation Support One of the benefits of having a layer object behind every view is that you can animate many view-related changes easily. Animations are a useful way to communicate information to the user and should always be considered during the design of your application. Many properties of the UIView class are animatable—that is, semiautomatic support exists for animating from one value to another. To perform an animation for one of these animatable properties, all you have to do is: 1. Tell UIKit that you want to perform an animation. 2. Change the value of the property. Among the properties you can animate on a UIView object are the following: frame—Use this to animate position and size changes for the view. bounds—Use this to animate changes to the size of the view. center—Use this to animate the position of the view. transform—Use this to rotate or scale the view. alpha—Use this to change the transparency of the view. backgroundColor—Use this to change the background color of the view. contentStretch—Use this to change how the view’s contents stretch. View and Window Architecture View Architecture Fundamentals 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 16One place where animations are very important is when transitioning from one set of viewsto another. Typically, you use a view controller to manage the animations associated with major changes between parts of your user interface. For example, for interfaces that involve navigating from higher-level to lower-level information, you typically use a navigation controller to manage the transitions between the views displaying each successive level of data. However, you can also create transitions between two sets of views using animations instead of a view controller. You might do so in places where the standard view-controller animations do not yield the results you want. In addition to the animations you create using UIKit classes, you can also create animations using Core Animation layers. Dropping down to the layer level gives you much more control over the timing and properties of your animations. For details about how to perform view-based animations, see “Animations” (page 64). For more information about creating animations using Core Animation, see Core Animation Programming Guide and Core Animation Cookbook . View Geometry and Coordinate Systems The default coordinate system in UIKit has its origin in the top-left corner and has axes that extend down and to the right from the origin point. Coordinate values are represented using floating-point numbers, which allow for precise layout and positioning of content regardless of the underlying screen resolution. Figure 1-4 shows this coordinate system relative to the screen. In addition to the screen coordinate system, windows and views define their own local coordinate systems that allow you to specify coordinates relative to the view or window origin instead of relative to the screen. Figure 1-4 Coordinate system orientation in UIKit y x (0,0) View and Window Architecture View Geometry and Coordinate Systems 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 17Because every view and window defines its own local coordinate system, you need to be aware of which coordinate system is in effect at any given time. Every time you draw into a view or change its geometry, you do so relative to some coordinate system. In the case of drawing, you specify coordinates relative to the view’s own coordinate system. In the case of geometry changes, you specify coordinates relative to the superview’s coordinate system. The UIWindow and UIView classes both include methods to help you convert from one coordinate system to another. Important: Some iOS technologies define default coordinate systems whose origin point and orientation differ from those used by UIKit. For example, Core Graphics and OpenGL ES use a coordinate system whose origin lies in the lower-left corner of the view or window and whose y-axis points upward relative to the screen. Your code must take such differences into account when drawing or creating content and adjust coordinate values (or the default orientation of the coordinate system) as needed. The Relationship of the Frame, Bounds, and Center Properties A view object tracks its size and location using its frame, bounds, and center properties: ● The frame property contains the frame rectangle, which specifies the size and location of the view in its superview’s coordinate system. ● The bounds property contains the bounds rectangle, which specifies the size of the view (and its content origin) in the view’s own local coordinate system. ● The center property contains the known center point of the view in the superview’s coordinate system. You use the center and frame properties primarily for manipulating the geometry of the current view. For example, you use these properties when building your view hierarchy or changing the position or size of a view at runtime. If you are changing only the position of the view (and not its size), the center property is the preferred way to do so. The value in the center property is always valid, even if scaling or rotation factors have been added to the view’s transform. The same is not true for the value in the frame property, which is considered invalid if the view’s transform is not equal to the identity transform. You use the bounds property primarily during drawing. The bounds rectangle is expressed in the view’s own local coordinate system. The default origin of this rectangle is (0, 0) and its size matches the size of the frame rectangle. Anything you draw inside this rectangle is part of the view’s visible content. If you change the origin of the boundsrectangle, anything you draw inside the new rectangle becomes part of the view’s visible content. View and Window Architecture View Geometry and Coordinate Systems 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 18Figure 1-5 shows the relationship between the frame and bounds rectangles for an image view. In the figure, the upper-left corner of the image view is located at the point (40, 40) in its superview’s coordinate system and the size of the rectangle is 240 by 380 points. For the bounds rectangle, the origin point is (0, 0) and the size of the rectangle is similarly 240 by 380 points. Figure 1-5 Relationship between a view's frame and bounds Frame rectangle Center (160,230) Bounds rectangle 240 240 380 380 (40,40) (0,0) Although you can change the frame, bounds, and center properties independent of the others, changes to one property affect the others in the following ways: ● When you set the frame property, the size value in the bounds property changes to match the new size of the frame rectangle. The value in the center property similarly changes to match the new center point of the frame rectangle. ● When you set the center property, the origin value in the frame changes accordingly. ● When you set the size of the bounds property, the size value in the frame property changes to match the new size of the bounds rectangle. By default, a view’s frame is not clipped to its superview’s frame. Thus, any subviews that lie outside of their superview’sframe are rendered in their entirety. You can change this behavior, though, by setting the superview’s clipsToBounds property to YES. Regardless of whether or not subviews are clipped visually, touch events always respect the bounds rectangle of the target view’s superview. In other words, touch events occurring in a part of a view that lies outside of its superview’s bounds rectangle are not delivered to that view. View and Window Architecture View Geometry and Coordinate Systems 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 19Coordinate System Transformations Coordinate system transformations offer a way to alter your view (or its contents) quickly and easily. An affine transform is a mathematical matrix that specifies how points in one coordinate system map to points in a different coordinate system. You can apply affine transforms to your entire view to change the size, location, or orientation of the view relative to its superview. You can also use affine transforms in your drawing code to perform the same types of manipulations to individual pieces of rendered content. How you apply the affine transform therefore depends on context: ● To modify your entire view, modify the affine transform in the transform property of your view. ● To modify specific pieces of content in your view’s drawRect: method, modify the affine transform associated with the active graphics context. You typically modify the transform property of a view when you want to implement animations. For example, you could use this property to create an animation of your view rotating around its center point. You would not use this property to make permanent changes to your view, such as modifying its position or size a view within its superview’s coordinate space. For that type of change, you should modify the frame rectangle of your view instead. Note: When modifying the transform property of your view, all transformations are performed relative to the center point of the view. In your view’s drawRect: method, you use affine transforms to position and orient the items you plan to draw. Rather than fix the position of an object at some location in your view, it is simpler to create each object relative to a fixed point, typically (0, 0), and use a transform to position the object immediately prior to drawing. That way, if the position of the object changes in your view, all you have to do is modify the transform, which is much faster and less expensive than recreating the object at its new location. You can retrieve the affine transform associated with a graphics context using the CGContextGetCTM function and you can use the related Core Graphics functions to set or modify this transform during drawing. The current transformation matrix (CTM) is the affine transform in use at any given time. When manipulating the geometry of your entire view, the CTM is the affine transform stored in your view’s transform property. Inside your drawRect: method, the CTM is the affine transform associated with the active graphics context. The coordinate system of each subview builds upon the coordinate systems of its ancestors. So when you modify a view’s transform property, that change affects the view and all of its subviews. However, these changes affect only the final rendering of the views on the screen. Because each view draws its content and lays out its subviews relative to its own bounds, it can ignore its superview’s transform during drawing and layout. View and Window Architecture View Geometry and Coordinate Systems 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 20Figure 1-6 demonstrates how two different rotation factors combine visually when rendered. Inside the view’s drawRect: method, applying a 45 degree rotation factor to a shape causes that shape to appear rotated by 45 degrees. Applying a separate 45 degree rotation factor to the view then causes the shape to appear to be rotated by 90 degrees. The shape is still rotated by only 45 degrees relative to the view that drew it, but the view rotation makes it appear to be rotated by more. Figure 1-6 Rotating a view and its content No rotations Shape rotated 45˚ during drawing Shape and view each rotated 45˚ Important: If a view’s transform property is not the identity transform, the value of that view’s frame property is undefined and must be ignored. When applying transforms to a view, you must use the view’s bounds and center properties to get the size and position of the view. The frame rectangles of any subviews are still valid because they are relative to the view’s bounds. For information about modifying your view’s transform property at runtime, see “Translating, Scaling, and Rotating Views” (page 47). For information about how to use transforms to position content during drawing, see Drawing and Printing Guide for iOS . Points Versus Pixels In iOS, all coordinate values and distances are specified using floating-point valuesin unitsreferred to as points. The measurable size of a point variesfrom device to device and islargely irrelevant. The main thing to understand about points is that they provide a fixed frame of reference for drawing. View and Window Architecture View Geometry and Coordinate Systems 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 21Table 1-1 lists the screen dimensions (measured in points) for different types of iOS-based devices in a portrait orientation. The width dimension is listed first, followed by the height dimension of the screen. As long as you design your interface to these screen sizes, your views will display correctly on the corresponding type of device. Table 1-1 Screen dimensions for iOS-based devices Device Screen dimensions (in points) iPhone and iPod touch 320 x 480 iPad 768 x 1024 The point-based measuring system used for each type of device defines what is known as the user coordinate space. This is the standard coordinate space you use for nearly all of your code. For example, you use points and the user coordinate space when manipulating the geometry of a view or calling Core Graphics functions to draw the contents of your view. Although coordinates in the user coordinate space sometimes map directly to the pixels on the device’s screen, you should never assume that this is the case. Instead, you should always remember the following: One point does not necessarily correspond to one pixel on the screen. At the device level, all coordinates you specify in your view must be converted to pixels atsome point. However, the mapping of pointsin the user coordinate space to pixelsin the device coordinate space is normally handled by the system. Both UIKit and Core Graphics use a primarily vector-based drawing model where all coordinate values are specified using points. Thus, if you draw a curve using Core Graphics, you specify the curve using the same values, regardless of the resolution of the underlying screen. When you need to work with images or other pixel-based technologies such as OpenGL ES, iOS provides help in managing those pixels. For static image files stored as resources in your application bundle, iOS defines conventionsforspecifying your images at different pixel densities and for loading the image that best matches the current screen resolution. Views also provide information about the current scale factor so that you can adjust any pixel-based drawing code manually to accommodate higher-resolution screens. The techniques for dealing with pixel-based content at different screen resolutions is described in “Supporting High-Resolution Screens” in Drawing and Printing Guide for iOS . View and Window Architecture View Geometry and Coordinate Systems 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 22The Runtime Interaction Model for Views Any time a user interacts with your user interface, or any time your own code programmatically changes something, a complex sequence of events takes place inside of UIKit to handle that interaction. At specific points during that sequence, UIKit calls out to your view classes and gives them a chance to respond on behalf of your application. Understanding these callout points is important to understanding where your views fit into the system. Figure 1-7 shows the basic sequence of events that starts with the user touching the screen and ends with the graphics system updating the screen content in response. The same sequence of events would also occur for any programmatically initiated actions. Figure 1-7 UIKit interactions with your view objects Your Application iPhone OS Touches • Buffers • Images • Attributes • Geometry • Animations touches layoutSubviews drawRect Compositor Draw images, text, etc. Touch Framework Graphics hardware UIKit setNeedsDisplay frame, alpha, etc. setNeedsLayout setNeedsDisplay frame, alpha, etc. The following steps break the event sequence in Figure 1-7 (page 23) down even further and explain what happens at each stage and how you might want your application to react in response. 1. The user touches the screen. 2. The hardware reports the touch event to the UIKit framework. 3. The UIKit framework packages the touch into a UIEvent object and dispatches it to the appropriate view. (For a detailed explanation of how UIKit delivers events to your views, see Event Handling Guide for iOS .) 4. The event-handling code of your view responds to the event. For example, your code might: ● Change the properties (frame, bounds, alpha, and so on) of the view or its subviews. ● Call the setNeedsLayout method to mark the view (or its subviews) as needing a layout update. ● Call the setNeedsDisplay or setNeedsDisplayInRect: method to mark the view (or itssubviews) as needing to be redrawn. ● Notify a controller about changes to some piece of data. View and Window Architecture The Runtime Interaction Model for Views 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 23Of course, it is up to you to decide which of these things the view should do and which methods it should call. 5. If the geometry of a view changed for any reason, UIKit updates its subviews according to the following rules: a. If you have configured autoresizing rules for your views, UIKit adjusts each view according to those rules. For more information about how autoresizing rules work, see “Handling Layout Changes Automatically Using Autoresizing Rules” (page 52). b. If the view implements the layoutSubviews method, UIKit calls it. You can override this method in your custom views and use it to adjust the position and size of any subviews. For example, a view that provides a large scrollable area would need to use severalsubviews as “tiles” rather than create one large view, which is not likely to fit in memory anyway. In its implementation of this method, the view would hide any subviewsthat are now offscreen or reposition them and use them to draw newly exposed content. As part of this process, the view’s layout code can also invalidate any views that need to be redrawn. 6. If any part of any view was marked as needing to be redrawn, UIKit asks the view to redraw itself. For custom viewsthat explicitly define a drawRect: method, UIKit callsthat method. Your implementation of this method should redraw the specified area of the view as quickly as possible and nothing else. Do not make additional layout changes at this point and do not make other changes to your application’s data model. The purpose of this method is to update the visual content of your view. Standard system viewstypically do not implement a drawRect: method but instead manage their drawing at this time. 7. Any updated views are composited with the rest of the application’s visible content and sent to the graphics hardware for display. 8. The graphics hardware transfers the rendered content to the screen. Note: The preceding update model applies primarily to applicationsthat use standard system views and drawing techniques. Applications that use OpenGL ES for drawing typically configure a single full-screen view and draw directly to the associated OpenGL graphics context. In such a case, the view would still handle touch events but, because it is full-screen, it would not need to lay out subviews or implement a drawRect: method. For more information about using OpenGL ES, see OpenGL ES Programming Guide for iOS . In the preceding set of steps, the primary integration points for your own custom views are: ● The event-handling methods: ● touchesBegan:withEvent: View and Window Architecture The Runtime Interaction Model for Views 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 24● touchesMoved:withEvent: ● touchesEnded:withEvent: ● touchesCancelled:withEvent: ● The layoutSubviews method ● The drawRect: method These are the most commonly overridden methods for views but you may not need to override all of them. If you use gesture recognizersto handle events, you do not need to override any of the event-handling methods. Similarly, if your view does not contain subviews or its size does not change, there is no reason to override the layoutSubviews method. Finally, the drawRect: method is needed only when the contents of your view can change at runtime and you are using native technologiessuch as UIKit or Core Graphicsto do your drawing. It is also important to remember that these are the primary integration points but not the only ones. Several methods of the UIView class are designed to be override points for subclasses. You should look at the method descriptions in UIView Class Reference to see which methods might be appropriate for you to override in your custom implementations. Tips for Using Views Effectively Custom views are useful for situations where you need to draw something the standard system views do not provide, but it is your responsibility to ensure that the performance of your views is good enough. UIKit does everything it can to optimize view-related behaviors and help you achieve good performance in your custom views. However, you can help UIKit in this aspect by considering the following tips. Important: Before optimizing your drawing code, you should always gather data about your view’s current performance. Measuring the current performance lets you confirm whether there actually is a problem and, if there is, gives you a baseline measurement against which you can compare future optimizations. Views Do Not Always Have a Corresponding View Controller There is rarely a one-to-one relationship between individual views and view controllers in your application. The job of a view controller is to manage a view hierarchy, which often consists of more than one view used to implement some self-contained feature. For iPhone applications, each view hierarchy typically fills the entire screen, although for iPad applications a view hierarchy may fill only part of the screen. View and Window Architecture Tips for Using Views Effectively 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 25As you design your application’s user interface, it is important to consider the role that view controllers will play. View controllers provide a lot of important behaviors, such as coordinating the presentation of views on the screen, coordinating the removal of those views from the screen, releasing memory in response to low-memory warnings, and rotating views in response to interface orientation changes. Circumventing these behaviors could cause your application to behave incorrectly or in unexpected ways. For more information view controllers and their role in applications, see View Controller Programming Guide for iOS . Minimize Custom Drawing Although custom drawing is necessary at times, it is also something you should avoid whenever possible. The only time you should truly do any custom drawing is when the existing system view classes do not provide the appearance or capabilities that you need. Any time your content can be assembled with a combination of existing views, your best bet is to combine those view objects into a custom view hierarchy. Take Advantage of Content Modes Content modes minimize the amount of time spent redrawing your views. By default, views use the UIViewContentModeScaleToFill content mode, which scales the view’s existing contents to fit the view’s frame rectangle. You can change this mode as needed to adjust your content differently, but you should avoid using the UIViewContentModeRedraw content mode if you can. Regardless of which content mode is in effect, you can always force your view to redraw its contents by calling setNeedsDisplay or setNeedsDisplayInRect:. Declare Views as Opaque Whenever Possible UIKit uses the opaque property of each view to determine whether the view can optimize compositing operations. Setting the value of this property to YES for a custom view tells UIKit that it does not need to render any content behind your view. Less rendering can lead to increased performance for your drawing code and is generally encouraged. Of course, if you set the opaque property to YES, your view must fills its bounds rectangle completely with fully opaque content. Adjust Your View’s Drawing Behavior When Scrolling Scrolling can incur numerous view updates in a short amount of time. If your view’s drawing code is not tuned appropriately, scrolling performance for your view could be sluggish. Rather than trying to ensure that your view’s content is pristine at all times, consider changing your view’s behavior when a scrolling operation begins. View and Window Architecture Tips for Using Views Effectively 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 26For example, you can reduce the quality of your rendered content temporarily or change the content mode while a scroll isin progress. When scrolling stops, you can then return your view to its previousstate and update the contents as needed. Do Not Customize Controls by Embedding Subviews Although it is technically possible to add subviews to the standard system controls—objects that inherit from UIControl—you should never customize them in this way. Controlsthatsupport customizations do so through explicit and well-documented interfaces in the control class itself. For example, the UIButton class contains methods for setting the title and background images for the button. Using the defined customization points meansthat your code will always work correctly. Circumventing these methods, by embedding a custom image view or label inside the button, might cause your application to behave incorrectly now or at some point in the future if the button’s implementation changes. View and Window Architecture Tips for Using Views Effectively 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 27Every iOS application needs at least one window—an instance of the UIWindow class—and some may include more than one window. A window object has several responsibilities: ● It contains your application’s visible content. ● It plays a key role in the delivery of touch events to your views and other application objects. ● It works with your application’s view controllers to facilitate orientation changes. In iOS, windows do not have title bars, close boxes, or any other visual adornments. A window is always just a blank container for one or more views. Also, applications do not change their content by showing new windows. When you want to change the displayed content, you change the frontmost views of your window instead. Most iOS applications create and use only one window during their lifetime. This window spans the entire main screen of the device and is loaded from the application’s main nib file (or created programmatically) early in the life of the application. However, if an application supports the use of an external display for video out, it can create an additional window to display content on that external display. All other windows are typically created by the system, and are usually created in response to specific events, such as an incoming phone call. Tasks That Involve Windows For many applications, the only time the application interacts with its window is when it creates the window at startup. However, you can use your application’s window object to perform a few application-related tasks: ● Use the window object to convert points and rectangles to or from the window’s local coordinate system. For example, if you are provided with a value in window coordinates, you might want to convert it to the coordinate system of a specific view before trying to use it. For information on how to convert coordinates, see “Converting Coordinates in the View Hierarchy” (page 50). ● Use window notifications to track window-related changes. Windows generate notifications when they are shown or hidden or when they accept or resign the key status. You can use these notifications to perform actions in other parts of your application. For more information, see “Monitoring Window Changes” (page 31). 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 28 WindowsCreating and Configuring a Window You can create and configure your application’s main window programmatically or using Interface Builder. In either case, you create the window at launch time and should retain it and store a reference to it in your application delegate object. If your application creates additional windows, have the application create them lazily when they are needed. For example, if your application supports displaying content on an external display, it should wait until a display is connected before creating the corresponding window. You should always create your application’s main window at launch time regardless of whether your application is being launched into the foreground or background. Creating and configuring a window is not an expensive operation by itself. However, if your application is launched straight into the background, you should avoid making the window visible until your application enters the foreground. Creating Windows in Interface Builder Creating your application’s main window using Interface Builder issimple because the Xcode project templates do it for you. Every new Xcode application project includes a main nib file (usually with the name MainWindow.xib or some variant thereof) that includes the application’s main window. In addition, these templates also define an outlet for that window in the application delegate object. You use this outlet to access the window object in your code. Important: When creating your window in Interface Builder, it is recommended that you enable the Full Screen at Launch option in the attributes inspector. If this option is not enabled and your window is smaller than the screen of the target device, touch events will not be received by some of your views. Thisis because windows (like all views) do not receive touch events outside of their bounds rectangle. Because views are not clipped to the window’s bounds by default, the views still appear visible but events do not reach them. Enabling the Full Screen at Launch option ensures that the window is sized appropriately for the current screen. If you are retrofitting a project to use Interface Builder, creating a window using Interface Builder is a simple matter of dragging a window object to your nib file. Of course, you should also do the following: ● To access the window at runtime, you should connect the window to an outlet, typically one defined in your application delegate or the File’s Owner of the nib file. ● If your retrofit plans include making your new nib file the main nib file of your application, you must also set the NSMainNibFile key in your application’s Info.plist file to the name of your nib file. Changing the value of this key ensures that the nib file is loaded and available for use by the time the application:didFinishLaunchingWithOptions: method of your application delegate is called. For more information about creating and configuring nib files,see Interface Builder User Guide . For information about how to load nib files into your application at runtime, see “Nib Files” in Resource Programming Guide . Windows Creating and Configuring a Window 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 29Creating a Window Programmatically If you prefer to create your application’s main window programmatically, you should include code similar to the following in the application:didFinishLaunchingWithOptions: method of your application delegate: self.window = [[[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]] autorelease]; In the preceding example, self.window is assumed to be a declared property of your application delegate that is configured to retain the window object. If you were creating a window for an external display instead, you would assign it to a different variable and you would need to specify the bounds of the non main UIScreen object representing that display. When creating windows, you should always set the size of the window to the full bounds of the screen. You should not reduce the size of the window to accommodate the status bar or any other items. The status bar always floats on top of the window anyway, so the only thing you should shrink to accommodate the status bar is the view you put into your window. And if you are using view controllers, the view controller should handle the sizing of your views automatically. Adding Content to Your Window Each window typically has a single root view object (managed by a corresponding view controller) that contains all of the other views representing your content. Using a single root view simplifies the process of changing your interface; to display new content, all you have to do is replace the root view. To install a view in your window, use the addSubview: method. For example, to install a view that is managed by a view controller, you would use code similar to the following: [window addSubview:viewController.view]; In place of the preceding code, you can alternatively configure the rootViewController property of the window in your nib file. This property offers a convenient way to configure the root view of the window using a nib file instead of programmatically. If this property is set when the window is loaded from its nib file, UIKit automatically installsthe view from the associated view controller asthe root view of the window. This property is used only to install the root view and is not used by the window to communicate with the view controller. You can use any view you want for a window’s root view. Depending on your interface design, the root view can be a generic UIView object that acts as a container for one or more subviews, the root view can be a standard system view, or the root view can be a custom view that you define. Some standard system views that are commonly used as root views include scroll views, table views, and image views. Windows Creating and Configuring a Window 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 30When configuring the root view of the window, you are responsible for setting its initial size and position within the window. For applications that do not include a status bar, or that display a translucent status bar, set the view size to match the size of the window. For applications that show an opaque status bar, position your view below the status bar and reduce its size accordingly. Subtracting the status bar height from the height of your view prevents the top portion of your view from being obscured. Note: If the root view of your window is provided by a container view controller (such as a tab bar controller, navigation controller, or split-view controller), you do not need to set the initial size of the view yourself. The container view controller automatically sizes its view appropriately based on whether the status bar is visible. Changing the Window Level Each UIWindow object has a configurable windowLevel property that determines how that window is positioned relative to other windows. For the most part, you should not need to change the level of your application’s windows. New windows are automatically assigned to the normal window level at creation time. The normal window level indicates that the window presents application-related content. Higher window levels are reserved for information that needs to float above the application content, such as the system status bar or alert messages. And although you can assign windows to these levels yourself, the system usually does this for you when you use specific interfaces. For example, when you show or hide the status bar or display an alert view, the system automatically creates the needed windows to display those items. Monitoring Window Changes If you want to track the appearance or disappearance of windows inside your application, you can do so using these window-related notifications: ● UIWindowDidBecomeVisibleNotification ● UIWindowDidBecomeHiddenNotification ● UIWindowDidBecomeKeyNotification ● UIWindowDidResignKeyNotification These notifications are delivered in response to programmatic changes in your application’s windows. Thus, when your application shows or hides a window, the UIWindowDidBecomeVisibleNotification and UIWindowDidBecomeHiddenNotification notifications are delivered accordingly. These notifications are not delivered when your application moves into the background execution state. Even though your window is not displayed on the screen while your application is in the background, it is still considered visible within the context of your application. Windows Monitoring Window Changes 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 31The UIWindowDidBecomeKeyNotification and UIWindowDidResignKeyNotification notifications help your application keep track of which window is the key window—that is, which window is currently receiving keyboard events and other non touch-related events. Whereas touch events are delivered to the window in which the touch occurred, events that do not have an associated coordinate value are delivered to the key window of your application. Only one window at a time may be key. Displaying Content on an External Display To display content on an external display, you must create an additional window for your application and associate it with the screen object representing the external display. New windows are normally associated with the main screen by default. Changing the window’s associated screen object causes the contents of that window to be rerouted to the corresponding display. Once the window is associated with the correct screen, you can add views to it and show it just like you do for your application’s main screen. The UIScreen class maintains a list of screen objects representing the available hardware displays. Normally, there is only one screen object representing the main display for any iOS-based device, but devicesthatsupport connecting to an external display can have an additional screen object available. Devices that support an external display include iPhone and iPod touch devices that have Retina displays and the iPad. Older devices, such as iPhone 3GS, do not support external displays. Note: Because external displays are essentially a video-out connection, you should not expect touch events for views and controls in a window that is associated with an external display. In addition, it is your application’s responsibility to update the contents of the window as needed. Thus, to mirror the contents of your main window, your application would need to create a duplicate set of views for the external display’s window and update them in tandem with the views in your main window. The process for displaying content on an external display is described in the following sections. However, the following steps summarize the basic process: 1. At application startup, register for the screen connection and disconnection notifications. 2. When it is time to display content on the external display, create and configure a window. ● Use the screens property of UIScreen to obtain the screen object for the external display. ● Create a UIWindow object and size it appropriately for the screen (or for your content). ● Assign the UIScreen object for the external display to the screen property of the window. ● Adjust the resolution of the screen object as needed to support your content. ● Add any appropriate views to the window. Windows Displaying Content on an External Display 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 323. Show the window and update it normally. Handling Screen Connection and Disconnection Notifications Screen connection and disconnection notifications are crucial for handling changes to external displays gracefully. When the user connects or disconnects a display, the system sends appropriate notifications to your application. You should use these notifications to update your application state and create or release the window associated with the external display. The important thing to remember about the connection and disconnection notificationsisthat they can come at any time, even when your application is suspended in the background. Therefore, it is best to observe the notifications from an object that is going to exist for the duration of your application’s runtime, such as your application delegate. If your application is suspended, the notifications are queued until your application exits the suspended state and starts running in either the foreground or background. Listing 2-1 shows the code used to register for connection and disconnection notifications. This method is called by the application delegate at initialization time but you could register for these notificationsfrom other places in your application, too. The implementation of the handler methods is shown in Listing 2-2 (page 34). Listing 2-1 Registering for screen connect and disconnect notifications - (void)setupScreenConnectionNotificationHandlers { NSNotificationCenter* center = [NSNotificationCenter defaultCenter]; [center addObserver:self selector:@selector(handleScreenConnectNotification:) name:UIScreenDidConnectNotification object:nil]; [center addObserver:self selector:@selector(handleScreenDisconnectNotification:) name:UIScreenDidDisconnectNotification object:nil]; } If your application is active when an external display is attached to the device, itshould create a second window for that display and fill it with some content. The content does not need to be the final content you want to present. For example, if your application is not ready to use the extra screen, it can use the second window to display some placeholder content. If you do not create a window for the screen, or if you create a window but do not show it, a black field is displayed on the external display. Windows Displaying Content on an External Display 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 33Listing 2-2 shows how to create a secondary window and fill it with some content. In this example, the application creates the window in the handler methods it uses to receive screen connection notifications. (For information about registering for connection and disconnection notifications, see Listing 2-1 (page 33).) The handler method for the connection notification creates a secondary window, associates it with the newly connected screen and calls a method of the application’s main view controller to add some content to the window and show it. The handler method for the disconnection notification releases the window and notifies the main view controller so that it can adjust its presentation accordingly. Listing 2-2 Handling connect and disconnect notifications - (void)handleScreenConnectNotification:(NSNotification*)aNotification { UIScreen* newScreen = [aNotification object]; CGRect screenBounds = newScreen.bounds; if (!_secondWindow) { _secondWindow = [[UIWindow alloc] initWithFrame:screenBounds]; _secondWindow.screen = newScreen; // Set the initial UI for the window. [viewController displaySelectionInSecondaryWindow:_secondWindow]; } } - (void)handleScreenDisconnectNotification:(NSNotification*)aNotification { if (_secondWindow) { // Hide and then delete the window. _secondWindow.hidden = YES; [_secondWindow release]; _secondWindow = nil; // Update the main screen based on what is showing here. [viewController displaySelectionOnMainScreen]; Windows Displaying Content on an External Display 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 34} } Configuring a Window for an External Display To display a window on an external screen, you must associate it with the correct screen object. This process involves locating the proper UIScreen object and assigning it to the window’s screen property. You can get the list of screen objects from the screens class method of UIScreen. The array returned by this method always contains at least one object representing the main screen. If a second object is present, that object represents a connected external display. Listing 2-3 shows a method that is called at application startup to see if an external display is already attached. If it is, the method creates a window, associatesit with the external display, and addssome placeholder content before showing the window. In this case, the placeholder content is a white background and a label indicating that there is no content to display. To show the window, this method changes the value of its hidden property rather than calling makeKeyAndVisible. It does this because the window contains only static content and is not used to handle events. Listing 2-3 Configuring a window for an external display - (void)checkForExistingScreenAndInitializeIfPresent { if ([[UIScreen screens] count] > 1) { // Associate the window with the second screen. // The main screen is always at index 0. UIScreen* secondScreen = [[UIScreen screens] objectAtIndex:1]; CGRect screenBounds = secondScreen.bounds; _secondWindow = [[UIWindow alloc] initWithFrame:screenBounds]; _secondWindow.screen = secondScreen; // Add a white background to the window UIView* whiteField = [[UIView alloc] initWithFrame:screenBounds]; whiteField.backgroundColor = [UIColor whiteColor]; [_secondWindow addSubview:whiteField]; Windows Displaying Content on an External Display 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 35[whiteField release]; // Center a label in the view. NSString* noContentString = [NSString stringWithFormat:@""]; CGSize stringSize = [noContentString sizeWithFont:[UIFont systemFontOfSize:18]]; CGRect labelSize = CGRectMake((screenBounds.size.width - stringSize.width) / 2.0, (screenBounds.size.height - stringSize.height) / 2.0, stringSize.width, stringSize.height); UILabel* noContentLabel = [[UILabel alloc] initWithFrame:labelSize]; noContentLabel.text = noContentString; noContentLabel.font = [UIFont systemFontOfSize:18]; [whiteField addSubview:noContentLabel]; // Go ahead and show the window. _secondWindow.hidden = NO; } } Important: You should always associate a screen with a window before showing the window. While it is possible to change screens for a window that is currently visible, doing so is an expensive operation and should be avoided. Assoon asthe window for an externalscreen is displayed, your application can begin updating it like any other window. You can add and remove subviews as needed, change the contents of subviews, animate changes to the views, and invalidate their contents as needed. Windows Displaying Content on an External Display 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 36Configuring the Screen Mode of an External Display Depending on your content, you might want to change the screen mode before associating your window with it. Many screens support multiple resolutions, some of which use different pixel aspect ratios. Screen objects use the most common screen mode by default, but you can change that mode to one that is more suitable for your content. For example, if you are implementing a game using OpenGL ES and your textures are designed for a 640 x 480 pixel screen, you might change the screen mode for screens with higher default resolutions. If you plan to use a screen mode other than the default one, you should apply that mode to the UIScreen object before associating the screen with a window. The UIScreenMode class definesthe attributes of a single screen mode. You can get a list of the modes supported by a screen from its availableModes property and iterate through the list for one that matches your needs. For more information about screen modes, see UIScreenMode Class Reference . Windows Displaying Content on an External Display 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 37Because view objects are the main way your application interacts with the user, they have many responsibilities. Here are just a few: ● Layout and subview management ● A view defines its own default resizing behaviors in relation to its parent view. ● A view can manage a list of subviews. ● A view can override the size and position of its subviews as needed. ● A view can convert points in its coordinate system to the coordinate systems of other views or the window. ● Drawing and animation ● A view draws content in its rectangular area. ● Some view properties can be animated to new values. ● Event handling ● A view can receive touch events. ● A view participates in the responder chain. This chapter focuses on the steps for creating, managing, and drawing views and for handling the layout and management of view hierarchies. For information about how to handle touch events (and other events) in your views, see Event Handling Guide for iOS . Creating and Configuring View Objects You create views as self-contained objects either programmatically or using Interface Builder, and then you assemble them into view hierarchies for use. 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 38 ViewsCreating View Objects Using Interface Builder The simplest way to create viewsisto assemble them graphically using Interface Builder. From Interface Builder, you can add views to your interface, arrange those views into hierarchies, configure each view’s settings, and connect view-related behaviors to your code. Because Interface Builder uses live view objects—that is, actual instances of the view classes—what you see at design time is what you get at runtime. You then save those live objects in a nib file, which is a resource file that preserves the state and configuration of your objects. You usually create nib filesin order to store an entire view hierarchy for one of your application’s view controllers. The top level of the nib file usually contains a single view object that represents your view controller’s view. (The view controller itself is typically represented by the File’s Owner object.) The top-level view should be sized appropriately for the target device and contain all of the other views that are to be presented. It is rare to use a nib file to store only a portion of your view controller’s view hierarchy. When using nib files with a view controller, all you have to do is initialize the view controller with the nib file information. The view controller handles the loading and unloading of your views at the appropriate times. However, if your nib file is not associated with a view controller, you can load the nib file contents manually using an NSBundle or UINib object, which use the data in the nib file to reconstitute your view objects. For more information about how to use Interface Builder to create and configure your views, see Interface Builder User Guide . For information about how view controllers load and manage their associated nib files, see “Custom View Controllers” in View Controller Programming Guide for iOS . For more information about how to load views programmatically from a nib file, see “Nib Files” in Resource Programming Guide . Creating View Objects Programmatically If you prefer to create views programmatically, you can do so using the standard allocation/initialization pattern. The default initialization method for views is the initWithFrame: method, which sets the initial size and position of the view relative to its (soon-to-be-established) parent view. For example, to create a new generic UIView object, you could use code similar to the following: CGRect viewRect = CGRectMake(0, 0, 100, 100); UIView* myView = [[UIView alloc] initWithFrame:viewRect]; Views Creating and Configuring View Objects 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 39Note: Although all views support the initWithFrame: method, some may have a preferred initialization method that you should use instead. For information about any custom initialization methods, see the reference documentation for the class. After you create a view, you must add it to a window (or to another view in a window) before it can become visible. For information on how to add viewsto your view hierarchy,see “Adding and Removing Subviews” (page 43). Setting the Properties of a View The UIView class hasseveral declared properties for controlling the appearance and behavior of the view. These properties are for manipulating the size and position of the view, the view’stransparency, its background color, and its rendering behavior. All of these properties have appropriate default values that you can change later as needed. You can also configure many of these propertiesfrom Interface Builder using the Inspector window. Table 3-1 lists some of the more commonly used properties (and some methods) and describes their usage. Related properties are listed together so that you can see the options you have for affecting certain aspects of the view. Table 3-1 Usage of some key view properties Properties Usage These properties affect the opacity of the view. The alpha and hidden properties change the view’s opacity directly. The opaque property tells the system how it should composite your view. Set this property to YES if your view’s content is fully opaque and therefore does not reveal any of the underlying view’s content. Setting this property to YES improves performance by eliminating unnecessary compositing operations. alpha, hidden, opaque Views Creating and Configuring View Objects 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 40Properties Usage These properties affect the size and position of the view. The center and frame properties represent the position of the view relative to its parent view. The frame also includes the size of the view. The bounds property defines the view’s visible content area in its own coordinate system. The transform property is used to animate or move the entire view in complex ways. For example, you would use a transform to rotate or scale the view. If the current transform is not the identity transform, the frame property is undefined and should be ignored. For information about the relationship between the bounds, frame, and center properties, see “The Relationship of the Frame, Bounds, and Center Properties” (page 18). For information about how transforms affect a view, see “Coordinate System Transformations” (page 20). bounds, frame, center, transform These properties affect the automatic resizing behavior of the view and its subviews. The autoresizingMask property controls how a view responds to changes in its parent view’s bounds. The autoresizesSubviews property controls whether the current view’s subviews are resized at all. autoresizingMask, autoresizesSubviews These properties affect the rendering behavior of content inside the view. The contentMode and contentStretch properties determine how the content is treated when the view’s width or height changes. The contentScaleFactor property is used only when you need to customize the drawing behavior of your view for high-resolution screens. For more information on how the content mode affects your view, see “Content Modes” (page 13). For information about how the content stretch rectangle affects your view, see “Stretchable Views” (page 15). For information about how to handle scale factors, see “Supporting High-Resolution Screens” in Drawing and Printing Guide for iOS . contentMode, contentStretch, contentScaleFactor These properties affect how your view processes touch events. The gestureRecognizers property contains gesture recognizers attached to the view. The other properties control what touch events the view supports. For information about how to respond to eventsin your views,see Event Handling Guide for iOS . gestureRecognizers, userInteractionEnabled, multipleTouchEnabled, exclusiveTouch Views Creating and Configuring View Objects 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 41Properties Usage These properties and methods help you manage the actual content of your view. For simple views, you can set a background color and add one ormore subviews. The subviews property itself contains a read-only list of subviews, but there are several methods for adding and rearranging subviews. For views with custom drawing behavior, you must override the drawRect: method. For more advanced content, you can work directly with the view’s Core Animation layer. To specify an entirely different type of layer for the view (such as a layer that supports OpenGL ES drawing calls), you must override the layerClass method. backgroundColor, subviews, drawRect: method, layer, (layerClass method) Forinformation aboutthe basic properties common to all views,seeUIViewClass Reference . Formore information about specific properties of a view, see the reference documentation for that view. Tagging Views for Future Identification The UIView class contains a tag property that you can use to tag individual view objects with an integer value. You can use tags to uniquely identify views inside your view hierarchy and to perform searches for those views at runtime. (Tag-based searches are faster than iterating the view hierarchy yourself.) The default value for the tag property is 0. To search for a tagged view, use the viewWithTag: method of UIView. This method performs a depth-first search of the receiver and its subviews. It does not search superviews or other parts of the view hierarchy. Thus, calling this method from the root view of a hierarchy searches all views in the hierarchy but calling it from a specific subview searches only a subset of views. Creating and Managing a View Hierarchy Managing view hierarchies is a crucial part of developing your application’s user interface. The organization of your views influences both the visual appearance of your application and how your application responds to changes and events. For example, the parent-child relationships in the view hierarchy determine which objects might handle a specific touch event. Similarly, parent-child relationships define how each view responds to interface orientation changes. Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 42Figure 3-1 shows an example of how the layering of views creates the desired visual effect for an application. In the case of the Clock application, the view hierarchy is composed of a mixture of views derived from different sources. The tab bar and navigation views are special view hierarchies provided by the tab bar and navigation controller objects to manage portions of the overall user interface. Everything between those bars belongs to the custom view hierarchy that the Clock application provides. Figure 3-1 Layered views in the Clock application Window Tab bar view Navigation view Custom view hierarchy There are several ways to build view hierarchies in iOS applications, including graphically in Interface Builder and programmatically in your code. The following sections show you how to assemble your view hierarchies and, having done that, how to find views in the hierarchy and convert between different view coordinate systems. Adding and Removing Subviews Interface Builder is the most convenient way to build view hierarchies because you assemble your views graphically, see the relationships between the views, and see exactly how those views will appear at runtime. When using Interface Builder, you save your resulting view hierarchy in a nib file, which you load at runtime as the corresponding views are needed. If you prefer to create your views programmatically instead, you create and initialize them and then use the following methods to arrange them into hierarchies: Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 43● To add a subview to a parent, call the addSubview: method of the parent view. This method adds the subview to the end of the parent’s list of subviews. ● To insert a subview in the middle of the parent’s list of subviews, call any of the insertSubview:... methods of the parent view. Inserting a subview in the middle of the list visually places that view behind any views that come later in the list. ● To reorder existing subviewsinside their parent, callthe bringSubviewToFront:, sendSubviewToBack:, or exchangeSubviewAtIndex:withSubviewAtIndex: methods of the parent view. Using these methods is faster than removing the subviews and reinserting them. ● To remove a subview from its parent, call the removeFromSuperview method of the subview (not the parent view). When adding a subview to its parent, the subview’s current frame rectangle denotes its initial position inside the parent view. A subview whose frame lies outside of its superview’s visible bounds is not clipped by default. If you want yoursubview to be clipped to the superview’s bounds, you must explicitly set the clipsToBounds property of the superview to YES. The most common example of adding a subview to another view occurs in the application:didFinishLaunchingWithOptions: method of almost every application. Listing 3-1 shows a version of this method that installs the view from the application’s main view controller into the application window. Both the window and the view controller are stored in the application’s main nib file, which is loaded before the method is called. However, the view hierarchy managed by the view controller is not actually loaded until the view property is accessed. Listing 3-1 Adding a view to a window - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions { // Override point for customization after application launch. // Add the view controller's view to the window and display. [window addSubview:viewController.view]; [window makeKeyAndVisible]; return YES; } Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 44Another common place where you might add subviewsto a view hierarchy isin the loadView or viewDidLoad methods of a view controller. If you are building your views programmatically, you put your view creation code in the loadView method of your view controller. Whether you create your views programmatically or load them from a nib file, you could include additional view configuration code in the viewDidLoad method. Listing 3-2 showsthe viewDidLoad method of the TransitionsViewController classfrom the UICatalog sample application. The TransitionsViewController class manages the animations associated with transitioning between two views. The application’s initial view hierarchy (consisting of a root view and toolbar) is loaded from a nib file. The code in the viewDidLoad method subsequently creates the container view and image views used to manage the transitions. The purpose of the container view is to simplify the code needed to implement the transition animations between the two image views. The container view has no real content of its own. Listing 3-2 Adding views to an existing view hierarchy - (void)viewDidLoad { [super viewDidLoad]; self.title = NSLocalizedString(@"TransitionsTitle", @""); // create the container view which we will use for transition animation (centered horizontally) CGRect frame = CGRectMake(round((self.view.bounds.size.width - kImageWidth) / 2.0), kTopPlacement, kImageWidth, kImageHeight); self.containerView = [[[UIView alloc] initWithFrame:frame] autorelease]; [self.view addSubview:self.containerView]; // The container view can represent the images for accessibility. [self.containerView setIsAccessibilityElement:YES]; [self.containerView setAccessibilityLabel:NSLocalizedString(@"ImagesTitle", @"")]; // create the initial image view frame = CGRectMake(0.0, 0.0, kImageWidth, kImageHeight); self.mainView = [[[UIImageView alloc] initWithFrame:frame] autorelease]; self.mainView.image = [UIImage imageNamed:@"scene1.jpg"]; Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 45[self.containerView addSubview:self.mainView]; // create the alternate image view (to transition between) CGRect imageFrame = CGRectMake(0.0, 0.0, kImageWidth, kImageHeight); self.flipToView = [[[UIImageView alloc] initWithFrame:imageFrame] autorelease]; self.flipToView.image = [UIImage imageNamed:@"scene2.jpg"]; } Important: Superviews automatically retain their subviews, so after embedding a subview it is safe to release that subview. In fact, doing so is recommended because it prevents your application from retaining the view one time too many and causing a memory leak later. Just remember that if you remove a subview from its superview and intend to reuse it, you must retain the subview again. The removeFromSuperview method autoreleases a subview before removing it from its superview. If you do not retain the view before the next event loop cycle, the view will be released. Formore information about Cocoamemorymanagement conventions,see AdvancedMemoryManagement Programming Guide . When you add a subview to another view, UIKit notifies both the parent and child views of the change. If you implement custom views, you can intercept these notifications by overriding one or more of the willMoveToSuperview:, willMoveToWindow:, willRemoveSubview:, didAddSubview:, didMoveToSuperview, or didMoveToWindow methods. You can use these notifications to update any state information related to your view hierarchy or to perform additional tasks. After creating a view hierarchy, you can navigate it programmatically using the superview and subviews properties of your views. The window property of each view containsthe window in which that view is currently displayed (if any). Because the root view in a view hierarchy has no parent, its superview property is set to nil. For views that are currently onscreen, the window object is the root view of the view hierarchy. Hiding Views To hide a view visually, you can either set its hidden property to YES or change its alpha property to 0.0. A hidden view does not receive touch events from the system. However, hidden views do participate in autoresizing and other layout operations associated with the view hierarchy. Thus, hiding a view is often a convenient alternative to removing views from your view hierarchy, especially if you plan to show the views again at some point soon. Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 46Important: If you hide a view that is currently the first responder, the view does not automatically resign its first responder status. Events targeted at the first responder are still delivered to the hidden view. To prevent this from happening, you should force your view to resign the first responder status when you hide it. For more information about the responder chain, see Event Handling Guide for iOS . If you want to animate a view’s transition from visible to hidden (or the reverse), you must do so using the view’s alpha property. The hidden property is not an animatable property, so any changes you make to it take effect immediately. Locating Views in a View Hierarchy There are two ways to locate views in a view hierarchy: ● Store pointers to any relevant views in an appropriate location, such as in the view controller that owns the views. ● Assign a unique integer to each view’s tag property and use the viewWithTag: method to locate it. Storing references to relevant views is the most common approach to locating views and makes accessing those views very convenient. If you used Interface Builder to create your views, you can connect objects in your nib file (including the File’s Owner object that represents the managing controller object) to one another using outlets. For views you create programmatically, you can store referencesto those viewsin private member variables. Whether you use outlets or private member variables, you are responsible for retaining the views as needed and then releasing them as well. The best way to ensure objects are retained and released properly is to use declared properties. Tags are a useful way to reduce hard-coded dependencies and support more dynamic and flexible solutions. Rather than storing a pointer to a view, you could locate it using its tag. Tags are also a more persistent way of referring to views. For example, if you wanted to save the list of views that are currently visible in your application, you would write out the tags of each visible view to a file. This is simpler than archiving the actual view objects, especially in situations where you are tracking only which views are currently visible. When your application is subsequently loaded, you would then re-create your views and use the saved list of tags to set the visibility of each view, and thereby return your view hierarchy to its previous state. Translating, Scaling, and Rotating Views Every view has an associated affine transform that you can use to translate, scale, or rotate the view’s content. View transforms alter the final rendered appearance of the view and are often used to implement scrolling, animations, or other visual effects. Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 47The transform property of UIView contains a CGAffineTransform structure with the transformations to apply. By default, this property is set to the identity transform, which does not modify the appearance of the view. You can assign a new transform to this property at any time. For example, to rotate a view by 45 degrees, you could use the following code: // M_PI/4.0 is one quarter of a half circle, or 45 degrees. CGAffineTransform xform = CGAffineTransformMakeRotation(M_PI/4.0); self.view.transform = xform; Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 48Applying the transform in the preceding code to a view would rotate that view clockwise about its center point. Figure 3-2 shows how this transformation would look if it were applied to an image view embedded in an application. Figure 3-2 Rotating a view 45 degrees Unrotated Rotated 45˚ When applying multiple transformations to a view, the order in which you add those transformations to the CGAffineTransform structure is significant. Rotating the view and then translating it is not the same as translating the view and then rotating it. Even if the amounts of rotation and translation are the same in each case, the sequence of the transformations affects the final results. In addition, any transformations you add are applied to the view relative to its center point. Thus, applying a rotation factor rotates the view around its center point. Scaling a view changes the width and height of the view but does not change its center point. For more information about creating and using affine transforms, see “Transforms” in Quartz 2D Programming Guide . Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 49Converting Coordinates in the View Hierarchy At various times, particularly when handling events, an application may need to convert coordinate values from one frame of reference to another. For example, touch events report the location of each touch in the window’s coordinate system but view objects often need that information in the view’slocal coordinate system. The UIView class defines the following methods for converting coordinates to and from the view’s local coordinate system: convertPoint:fromView: convertRect:fromView: convertPoint:toView: convertRect:toView: The convert...:fromView: methods convert coordinates from some other view’s coordinate system to the local coordinate system (bounds rectangle) of the current view. Conversely, the convert...:toView: methods convert coordinates from the current view’s local coordinate system (bounds rectangle) to the coordinate system of the specified view. If you specify nil as the reference view for any of the methods, the conversions are made to and from the coordinate system of the window that contains the view. In addition to the UIView conversion methods, the UIWindow class also defines several conversion methods. These methods are similar to the UIView versions except that instead of converting to and from a view’s local coordinate system, these methods convert to and from the window’s coordinate system. convertPoint:fromWindow: convertRect:fromWindow: convertPoint:toWindow: convertRect:toWindow: When converting coordinates in rotated views, UIKit converts rectangles under the assumption that you want the returned rectangle to reflect the screen area covered by the source rectangle. Figure 3-3 shows an example of how rotations can cause the size of the rectangle to change during a conversion. In the figure, an outer Views Creating and Managing a View Hierarchy 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 50parent view contains a rotated subview. Converting a rectangle in the subview’s coordinate system to the parent’s coordinate system yields a rectangle that is physically larger. This larger rectangle is actually the smallest rectangle in the bounds of outerView that completely encloses the rotated rectangle. Figure 3-3 Converting values in a rotated view Rectangle in rotatedView coordinate system Rectangle converted to outerView coordinate system outerView superview subviews frame rotatedView superview subviews frame Adjusting the Size and Position of Views at Runtime Whenever the size of a view changes, the size and position of its subviews must change accordingly. The UIView class supports both the automatic and manual layout of views in a view hierarchy. With automatic layout, you set the rules that each view should follow when its parent view resizes, and then forget about resizing operations altogether. With manual layout, you manually adjust the size and position of views as needed. Being Prepared for Layout Changes Layout changes can occur whenever any of the following events happens in a view: ● The size of a view’s bounds rectangle changes. ● An interface orientation change occurs, which usually triggers a change in the root view’s boundsrectangle. ● The set of Core Animation sublayers associated with the view’s layer changes and requires layout. ● Your application forces layout to occur by calling the setNeedsLayout or layoutIfNeeded method of a view. ● Your application forces layout by calling the setNeedsLayout method of the view’s underlying layer object. Views Adjusting the Size and Position of Views at Runtime 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 51Handling Layout Changes Automatically Using Autoresizing Rules When you change the size of a view, the position and size of any embedded subviews usually needs to change to account for the new size of their parent. The autoresizesSubviews property of the superview determines whether the subviewsresize at all. If this property isset to YES, the view usesthe autoresizingMask property of each subview to determine how to size and position that subview. Size changes to any subviews trigger similar layout adjustments for their embedded subviews. For each view in your view hierarchy,setting that view’s autoresizingMask property to an appropriate value is an important part of handling automatic layout changes. Table 3-2 lists the autoresizing options you can apply to a given view and describes their effects during layout operations. You can combine constants using an OR operator or just add them together before assigning them to the autoresizingMask property. If you are using Interface Builder to assemble your views, you use the Autosizing inspector to set these properties. Table 3-2 Autoresizing mask constants Autoresizing mask Description UIViewAutoresizingNone The view does not autoresize. (This is the default value.) The view’s height changes when the superview’s height changes. If this constant is not included, the view’s height does not change. UIViewAutoresizingFlexibleHeight The view’s width changes when the superview's width changes. If this constant is not included, the view’s width does not change. UIViewAutoresizingFlexibleWidth The distance between the view’s left edge and the superview’s left edge grows or shrinks as needed. If this constant is not included, the view’s left edge remains a fixed distance from the left edge of the superview. UIViewAutoresizingFlexibleLeftMargin The distance between the view’s right edge and the superview’s right edge grows or shrinks as needed. If this constant is not included, the view’s right edge remains a fixed distance from the right edge of the superview. UIViewAutoresizingFlexibleRightMargin The distance between the view’s bottom edge and the superview’s bottom edge grows or shrinks as needed. If this constant is not included, the view’s bottom edge remains a fixed distance from the bottom edge of the superview. UIViewAutoresizingFlexibleBottomMargin The distance between the view’s top edge and the superview’s top edge grows or shrinks as needed. If this constant is not included, the view’s top edge remains a fixed distance from the top edge of the superview. UIViewAutoresizingFlexibleTopMargin Views Adjusting the Size and Position of Views at Runtime 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 52Figure 3-4 shows a graphical representation of how the options in the autoresizing mask apply to a view. The presence of a given constant indicates that the specified aspect of the view is flexible and may change when the superview’s bounds change. The absence of a constant indicates that the view’s layout is fixed in that aspect. When you configure a view that has more than one flexible attribute along a single axis, UIKit distributes any size changes evenly among the corresponding spaces. Figure 3-4 View autoresizing mask constants UIViewAutoresizingFlexibleWidth UIViewAutoresizingFlexibleRightMargin UIViewAutoresizingFlexibleBottomMargin UIViewAutoresizingFlexibleHeight Superview View UIViewAutoresizingFlexibleTopMargin UIViewAutoresizingFlexibleLeftMargin The easiest way to configure autoresizing rulesis using the Autosizing controlsin the Size inspector of Interface Builder. The flexible width and height constants from the preceding figure have the same behavior as the width and size indicatorsin the Autosizing controls diagram. However, the behavior and use of margin indicators is effectively reversed. In Interface Builder, the presence of a margin indicator means that the margin has a fixed size and the absence of the indicator means the margin has a flexible size. Fortunately, Interface Builder provides an animation to show you how changes to the autoresizing behaviors affect your view. Important: If a view’s transform property does not contain the identity transform, the frame of that view is undefined and so are the results of its autoresizing behaviors. After the automatic autoresizing rules for all affected views have been applied, UIKit goes back and gives each view a chance to make any necessary manual adjustments to its superview. For more information about how to manage the layout of views manually, see “Tweaking the Layout of Your Views Manually” (page 54). Views Adjusting the Size and Position of Views at Runtime 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 53Tweaking the Layout of Your Views Manually Whenever the size of a view changes, UIKit appliesthe autoresizing behaviors of that view’ssubviews and then calls the layoutSubviews method of the view to let it make manual changes. You can implement the layoutSubviews method in custom views when the autoresizing behaviors by themselves do not yield the results you want. Your implementation of this method can do any of the following: ● Adjust the size and position of any immediate subviews. ● Add or remove subviews or Core Animation layers. ● Force a subview to be redrawn by calling its setNeedsDisplay or setNeedsDisplayInRect: method. One place where applications often lay out subviews manually is when implementing a large scrollable area. Because it is impractical to have a single large view for its scrollable content, applications often implement a root view that contains a number of smaller tile views. Each tile represents a portion of the scrollable content. When a scroll event happens, the root view calls its setNeedsLayout method to initiate a layout change. Its layoutSubviews method then repositions the tile views based on the amount of scrolling that occurred. As tiles scroll out of the view’s visible area, the layoutSubviews method moves the tiles to the incoming edge, replacing their contents in the process. When writing your layout code, be sure to test your code in the following ways: ● Change the orientation of your views to make sure the layout looks correct in all supported interface orientations. ● Make sure your code responds appropriately to changes in the height of the status bar. When a phone call is active, the status bar height increasesin size, and when the user endsthe call, the status bar decreases in size. For information about how autoresizing behaviors affect the size and position of your views, see “Handling Layout Changes Automatically Using Autoresizing Rules” (page 52). For an example of how to implement tiling, see the ScrollViewSuite sample. Modifying Views at Runtime As applications receive input from the user, they adjust their user interface in response to that input. An application might modify its views by rearranging them, changing their size or position, hiding or showing them, or loading an entirely new set of views. In iOS applications, there are several places and ways in which you perform these kinds of actions: ● In a view controller: Views Modifying Views at Runtime 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 54● A view controller has to create its views before showing them. It can load the views from a nib file or create them programmatically. When those views are no longer needed, it disposes of them. ● When a device changes orientations, a view controller might adjust the size and position of views to match. As part of its adjustment to the new orientation, it might hide some views and show others. ● When a view controller manages editable content, it might adjust its view hierarchy when moving to and from edit mode. For example, it might add extra buttons and other controls to facilitate editing various aspects of its content. This might also require the resizing of any existing viewsto accommodate the extra controls. ● In animation blocks: ● When you want to transition between different sets of views in your user interface, you hide some views and show others from inside an animation block. ● When implementing special effects, you might use an animation block to modify various properties of the view. For example, to animate changes to the size of a view, you would change the size of its frame rectangle. ● Other ways: ● When touch events or gestures occur, your interface might respond by loading a new set of views or changing the current set of views. For information about handling events, see Event Handling Guide for iOS . ● When the user interacts with a scroll view, a large scrollable area might hide and show tile subviews. For more information about supporting scrollable content, see Scroll View Programming Guide for iOS . ● When the keyboard appears, you might reposition or resize views so that they do not lie underneath the keyboard. For information about how to interact with the keyboard, see Text, Web, and Editing Programming Guide for iOS . View controllers are a common place to initiate changes to your views. Because a view controller manages the view hierarchy associated with the content being displayed, it is ultimately responsible for everything that happens to those views. When loading its views or handling orientation changes, the view controller can add new views, hide or replace existing ones, and make any number of changes to make the views ready for the display. And if you implement support for editing your view’s content, the setEditing:animated: method in UIViewController gives you a place to transition your views to and from their editable versions. Animation blocks are another common place to initiate view-related changes. The animation support built into the UIView class makes it easy to animate changes to view properties. You can also use the transitionWithView:duration:options:animations:completion: or transitionFromView:toView:duration:options:completion: methods to swap out entire sets of views for new ones. Views Modifying Views at Runtime 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 55For more information about animating views and initiating view transitions, see “Animations” (page 64). For more information on how you use view controllers to manage view-related behaviors, see View Controller Programming Guide for iOS . Interacting with Core Animation Layers Each view object has a dedicated Core Animation layer that manages the presentation and animation of the view’s content on the screen. Although you can do a lot with your view objects, you can also work directly with the corresponding layer objects as needed. The layer object for the view is stored in the view’s layer property. Changing the Layer Class Associated with a View The type of layer associated with a view cannot be changed after the view is created. Therefore, each view uses the layerClass class method to specify the class of its layer object. The default implementation of this method returns the CALayer class and the only way to change this value is to subclass, override the method, and return a different value. You might want to change this value in the following circumstances: ● Your application uses OpenGL ES for drawing, in which case, the layer must be an instance of the CAEAGLLayer class. ● Your view uses tiling to display a large scrollable area, in which case you might want to use the CATiledLayer class to back your view instead. Implementation of the layerClass method should simply create the desired Class object and return it. For example, a view that supports OpenGL ES drawing would have the following implementation for this method: + (Class)layerClass { return [CAEAGLLayer class]; } Each view calls its layerClass method early in its initialization process and uses the returned class to create its layer object. In addition, the view always assigns itself as the delegate of its layer object. At this point, the view ownsitslayer and the relationship between the view and layer must not change. You must also not assign the same view as the delegate of any other layer object. Changing the ownership or delegate relationships of the view will cause drawing problems and potential crashes in your application. Views Interacting with Core Animation Layers 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 56For more information about the different types of layer objects provided by Core Animation,see Core Animation Reference Collection . Embedding Layer Objects in a View If you prefer to work primarily with layer objects instead of views, you can incorporate custom layer objects into your view hierarchy as needed. A custom layer object is any instance of CALayer that is not owned by a view. You typically create custom layers programmatically and incorporate them using Core Animation routines. Custom layers do not receive events or participate in the responder chain but do draw themselves and respond to size changes in their parent view or layer according to the Core Animation rules. Listing 3-3 shows an example of the viewDidLoad method from a view controller that creates a custom layer object and addsit to itsroot view. The layer is used to display a static image that is animated. Instead of adding the layer to the view itself, you add it to the view’s underlying layer. Listing 3-3 Adding a custom layer to a view - (void)viewDidLoad { [super viewDidLoad]; // Create the layer. CALayer* myLayer = [[CALayer alloc] init]; // Set the contents of the layer to a fixed image. And set // the size of the layer to match the image size. UIImage layerContents = [[UIImage imageNamed:@"myImage"] retain]; CGSize imageSize = layerContents.size; myLayer.bounds = CGRectMake(0, 0, imageSize.width, imageSize.height); myLayer = layerContents.CGImage; // Add the layer to the view. CALayer* viewLayer = self.view.layer; [viewLayer addSublayer:myLayer]; // Center the layer in the view. CGRect viewBounds = backingView.bounds; Views Interacting with Core Animation Layers 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 57myLayer.position = CGPointMake(CGRectGetMidX(viewBounds), CGRectGetMidY(viewBounds)); // Release the layer, since it is retained by the view's layer [myLayer release]; } You can add any number of sublayers and arrange them into sublayer hierarchies, if you want. However, at some point, those layers must be attached to the layer object of a view. For information on how to work with layers directly, see Core Animation Programming Guide . Defining a Custom View If the standard system views do not do exactly what you need, you can define a custom view. Custom views give you total control over the appearance of your application’s content and how interactions with that content are handled. Checklist for Implementing a Custom View The job of a custom view is to present content and manage interactions with that content. The successful implementation of a custom view involves more than just drawing and handling events, though. The following checklist includes the more important methods you can override (and behaviors you can provide) when implementing a custom view: ● Define the appropriate initialization methods for your view: ● For views you plan to create programmatically, override the initWithFrame: method or define a custom initialization method. ● For views you plan to load from nib files, override the initWithCoder: method. Use this method to initialize your view and put it into a known state. ● Implement a dealloc method to handle the cleanup of any custom data. ● To handle any custom drawing, override the drawRect: method and do your drawing there. ● Set the autoresizingMask property of the view to define its autoresizing behavior. ● If your view class manages one or more integral subviews, do the following: ● Create those subviews during your view’s initialization sequence. ● Set the autoresizingMask property of each subview at creation time. Views Defining a Custom View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 58● If your subviews require custom layout, override the layoutSubviews method and implement your layout code there. ● To handle touch-based events, do the following: ● Attach any suitable gesture recognizersto the view by using the addGestureRecognizer: method. ● For situations where you want to process the touches yourself, override the touchesBegan:withEvent:, touchesMoved:withEvent:, touchesEnded:withEvent:, and touchesCancelled:withEvent: methods. (Remember that you should always override the touchesCancelled:withEvent: method, regardless of which other touch-related methods you override.) ● If you want the printed version of your view to look different from the onscreen version, implement the drawRect:forViewPrintFormatter: method. For detailed information about how to support printing in your views, see Drawing and Printing Guide for iOS . In addition to overriding methods, remember that there is a lot you can do with the view’s existing properties and methods. For example, the contentMode and contentStretch properties let you change the final rendered appearance of your view and might be preferable to redrawing the content yourself. In addition to the UIView class itself, there are many aspects of a view’s underlying CALayer object that you can configure directly or indirectly. You can even change the class of the layer object itself (which you must do if you plan to use OpenGL ES to draw your view’s content). For more information about the methods and properties of the view class, see UIView Class Reference . Initializing Your Custom View Every new view object you define should include a custom initWithFrame: initializer method. This method is responsible for initializing the class at creation time and putting your view object into a known state. You use this method when creating instances of your view programmatically in your code. Listing 3-4 shows a skeletal implementation of a standard initWithFrame: method. This method calls the inherited implementation of the method first and then initializes the instance variables and state information of the class before returning the initialized object. Calling the inherited implementation istraditionally performed first so that if there is a problem, you can abort your own initialization code and return nil. Listing 3-4 Initializing a view subclass - (id)initWithFrame:(CGRect)aRect { self = [super initWithFrame:aRect]; if (self) { // setup the initial properties of the view Views Defining a Custom View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 59... } return self; } If you plan to load instances of your custom view class from a nib file, you should be aware that in iOS, the nib-loading code does not use the initWithFrame: method to instantiate new view objects. Instead, it uses the initWithCoder: method that is part of the NSCoding protocol. Even if your view adopts the NSCoding protocol, Interface Builder does not know about your view’s custom properties and therefore does not encode those properties into the nib file. As a result, your own initWithCoder: method should perform whatever initialization code it can to put the view into a known state. You can also implement the awakeFromNib method in your view class and use that method to perform additional initialization. Implementing Your Drawing Code For viewsthat need to do custom drawing, you need to override the drawRect: method and do your drawing there. Custom drawing is recommended only as a last resort. In general, if you can use other views to present your content, that is preferred. The implementation of your drawRect: method should do exactly one thing: draw your content. This method is not the place to be updating your application’s data structures or performing any tasks not related to drawing. It should configure the drawing environment, draw your content, and exit as quickly as possible. And if your drawRect: method might be called frequently, you should do everything you can to optimize your drawing code and draw as little as possible each time the method is called. Before calling your view’s drawRect: method, UIKit configures the basic drawing environment for your view. Specifically, it creates a graphics context and adjusts the coordinate system and clipping region to match the coordinate system and visible bounds of your view. Thus, by the time your drawRect: method is called, you can begin drawing your content using native drawing technologies such as UIKit and Core Graphics. You can get a pointer to the current graphics context using the UIGraphicsGetCurrentContext function. Views Defining a Custom View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 60Important: The current graphics context is valid only for the duration of one call to your view’s drawRect: method. UIKit might create a different graphics context for each subsequent call to this method, so you should not try to cache the object and use it later. Listing 3-5 shows a simple implementation of a drawRect: method that draws a 10-pixel-wide red border around the view. Because UIKit drawing operations use Core Graphics for their underlying implementations, you can mix drawing calls, as shown here, to get the results you expect. Listing 3-5 A drawing method - (void)drawRect:(CGRect)rect { CGContextRef context = UIGraphicsGetCurrentContext(); CGRect myFrame = self.bounds; // Set the line width to 10 and inset the rectangle by // 5 pixels on all sides to compensate for the wider line. CGContextSetLineWidth(context, 10); CGRectInset(myFrame, 5, 5); [[UIColor redColor] set]; UIRectFrame(myFrame); } If you know that your view’s drawing code always covers the entire surface of the view with opaque content, you can improve system performance by setting the opaque property of your view to YES. When you mark a view as opaque, UIKit avoids drawing content that is located immediately behind your view. This not only reduces the amount of time spent drawing but also minimizes the work that must be done to composite your view with other content. However, you should set this property to YES only if you know your view’s content is completely opaque. If your view cannot guarantee that its contents are always opaque, you should set the property to NO. Another way to improve drawing performance, especially during scrolling, is to set the clearsContextBeforeDrawing property of your view to NO. When this property is set to YES, UIKIt automatically fills the area to be updated by your drawRect: method with transparent black before calling your method. Setting this property to NO eliminates the overhead for that fill operation but puts the burden on your application to fill the update rectangle passed to your drawRect: method with content. Views Defining a Custom View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 61Responding to Events View objects are responder objects—instances of the UIResponder class—and are therefore capable of receiving touch events. When a touch event occurs, the window dispatches the corresponding event object to the view in which the touch occurred. If your view is not interested in an event, it can ignore it or passit up the responder chain to be handled by a different object. In addition to handling touch events directly, views can also use gesture recognizers to detect taps, swipes, pinches, and other types of common touch-related gestures. Gesture recognizers do the hard work of tracking touch events and making sure that they follow the right criteria to qualify them as the target gesture. Instead of your application having to track touch events, you can create the gesture recognizer, assign an appropriate target object and action method to it, and install it on your view using the addGestureRecognizer: method. The gesture recognizer then calls your action method when the corresponding gesture occurs. If you prefer to handle touch events directly, you can implement the following methods for your view, which are described in more detail in Event Handling Guide for iOS : touchesBegan:withEvent: touchesMoved:withEvent: touchesEnded:withEvent: touchesCancelled:withEvent: The default behavior for views is to respond to only one touch at a time. If the user puts a second finger down, the system ignoresthe touch event and does not report it to your view. If you plan to track multifinger gestures from your view’s event-handler methods, you need to enable multitouch events by setting the multipleTouchEnabled property of your view to YES. Some views, such as labels and images, disable event handling altogether initially. You can control whether a view is able to receive touch events by changing the value of the view’s userInteractionEnabled property. You might temporarily set this property to NO to prevent the user from manipulating the contents of your view while a long operation is pending. To prevent events from reaching any of your views, you can also use the beginIgnoringInteractionEvents and endIgnoringInteractionEvents methods of the UIApplication object. These methods affect the delivery of events for the entire application, not just for a single view. Views Defining a Custom View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 62Note: The animation methods of UIView typically disable touch events while animations are in progress. You can override this behavior by configuring the animation appropriately. For more information about performing animations, see “Animations” (page 64). As it handles touch events, UIKit uses the hitTest:withEvent: and pointInside:withEvent: methods of UIView to determine whether a touch event occurred inside a given view’s bounds. Although you rarely need to override these methods, you could do so to implement custom touch behaviors for your view. For example, you could override these methods to prevent subviews from handling touch events. Cleaning Up After Your View If your view class allocates any memory, stores references to any custom objects, or holds resources that must be released when the view is released, you must implement a dealloc method. The system calls the dealloc method when your view’s retain count reaches zero and it is time to deallocate the view. Your implementation of this method should release any objects or resources held by the view and then call the inherited implementation, as shown in Listing 3-6. You should not use this method to perform any other types of tasks. Listing 3-6 Implementing the dealloc method - (void)dealloc { // Release a retained UIColor object [color release]; // Call the inherited implementation [super dealloc]; } Views Defining a Custom View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 63Animations provide fluid visual transitions between different states of your user interface. In iOS, animations are used extensively to reposition views, change theirsize, remove them from view hierarchies, and hide them. You might use animations to convey feedback to the user or to implement interesting visual effects. In iOS, creating sophisticated animations does not require you to write any drawing code. All of the animation techniques described in this chapter use the built-in support provided by Core Animation. All you have to do is trigger the animation and let Core Animation handle the rendering of individual frames. This makes creating sophisticated animations very easy with only a few lines of code. What Can Be Animated? Both UIKit and Core Animation provide support for animations, but the level of support provided by each technology varies. In UIKit, animations are performed using UIView objects. Views support a basic set of animations that cover many common tasks. For example, you can animate changes to properties of views or use transition animations to replace one set of views with another. Table 4-1 liststhe animatable properties—the propertiesthat have built-in animation support—of the UIView class. Being animatable does not mean animations happen automatically. Changing the value of these properties normally just updates the property (and the view) immediately without an animation. To animate such a change, you must change the property’s value from inside an animation block, which is described in “Animating Property Changes in a View” (page 66). Table 4-1 Animatable UIView properties Property Changes you can make Modify this property to change the view’s size and position relative to its superview’s coordinate system. (If the transform property does not contain the identity transform, modify the bounds or center properties instead.) frame bounds Modify this property to change the view’s size. Modify this property to change the view’s position relative to its superview’s coordinate system. center 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 64 AnimationsProperty Changes you can make Modify this property to scale, rotate, or translate the view relative to its center point. Transformations using this property are always performed in 2D space. (To perform 3D transformations, you must animate the view’slayer object using Core Animation.) transform alpha Modify this property to gradually change the transparency of the view. backgroundColor Modify this property to change the view’s background color. Modify this property to change the way the view’s contents are stretched to fill the available space. contentStretch Animated view transitions are a way for you to make changes to your view hierarchy beyond those offered by view controllers. Although you should use view controllers to manage succinct view hierarchies, there may be times when you want to replace all or part of a view hierarchy. In those situations, you can use view-based transitions to animate the addition and removal of your views. In places where you want to perform more sophisticated animations, or animations not supported by the UIView class, you can use Core Animation and the view’s underlying layer to create the animation. Because view and layer objects are intricately linked together, changes to a view’s layer affect the view itself. Using Core Animation, you can animate the following types of changes for your view’s layer: ● The size and position of the layer ● The center point used when performing transformations ● Transformations to the layer or its sublayers in 3D space ● The addition or removal of a layer from the layer hierarchy ● The layer’s Z-order relative to other sibling layers ● The layer’s shadow ● The layer’s border (including whether the layer’s corners are rounded) ● The portion of the layer that stretches during resizing operations ● The layer’s opacity ● The clipping behavior for sublayers that lie outside the layer’s bounds ● The current contents of the layer ● The rasterization behavior of the layer Animations What Can Be Animated? 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 65Note: If your view hosts custom layer objects—that is, layer objects without an associated view—you must use Core Animation to animate any changes to them. Although this chapter addresses a few Core Animation behaviors, it does so in relation to initiating them from your view code. For more complete information about how to use Core Animation to animate layers, see Core Animation Programming Guide and Core Animation Cookbook . Animating Property Changes in a View In order to animate changesto a property of the UIView class, you must wrap those changesinside an animation block. The term animation block is used in the generic sense to refer to any code that designates animatable changes. In iOS 4 and later, you create an animation block using block objects. In earlier versions of iOS, you mark the beginning and end of an animation block using special class methods of the UIView class. Both techniques support the same configuration options and offer the same amount of control over the animation execution. However, the block-based methods are preferred whenever possible. The following sections focus on the code you need in order to animate changes to view properties. For information about how to create animated transitions between sets of views,see “Creating Animated Transitions Between Views” (page 73). Starting Animations Using the Block-Based Methods In iOS 4 and later, you use the block-based class methods to initiate animations. There are several block-based methods that offer different levels of configuration for the animation block. These methods are: ● animateWithDuration:animations: ● animateWithDuration:animations:completion: ● animateWithDuration:delay:options:animations:completion: Because these are class methods, the animation blocks you create with them are not tied to a single view. Thus, you can use these methods to create a single animation that involves changes to multiple views. For example, Listing 4-1 showsthe code needed to fade in one view while fading out another over a one second time period. When this code executes, the specified animations are started immediately on another thread so as to avoid blocking the current thread or your application’s main thread. Listing 4-1 Performing a simple block-based animation [UIView animateWithDuration:1.0 animations:^{ Animations Animating Property Changes in a View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 66firstView.alpha = 0.0; secondView.alpha = 1.0; }]; The animations in the preceding example run only once using using an ease-in, ease-out animation curve. If you want to change the default animation parameters, you must use the animateWithDuration:delay:options:animations:completion:method to performyour animations. This method lets you customize the following animation parameters: ● The delay to use before starting the animation ● The type of timing curve to use during the animation ● The number of times the animation should repeat ● Whether the animation should reverse itself automatically when it reaches the end ● Whether touch events are delivered to views while the animations are in progress ● Whether the animation should interrupt any in-progress animations or wait until those are complete before starting Another thing that both the animateWithDuration:animations:completion: and animateWithDuration:delay:options:animations:completion: methods support is the ability to specify a completion handler block. You might use a completion handler to signal your application that a specific animation has finished. Completion handlers are also the way to link separate animations together. Listing 4-2 shows an example of an animation block that uses a completion handler to initiate a new animation after the first one finishes. The first call to animateWithDuration:delay:options:animations:completion: sets up a fade-out animation and configures it with some custom options. When that animation is complete, its completion handler runs and sets up the second half of the animation, which fades the view back in after a delay. Using a completion handler is the primary way that you link multiple animations. Listing 4-2 Creating an animation block with custom options - (IBAction)showHideView:(id)sender { // Fade out the view right away [UIView animateWithDuration:1.0 delay: 0.0 options: UIViewAnimationOptionCurveEaseIn Animations Animating Property Changes in a View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 67animations:^{ thirdView.alpha = 0.0; } completion:^(BOOL finished){ // Wait one second and then fade in the view [UIView animateWithDuration:1.0 delay: 1.0 options:UIViewAnimationOptionCurveEaseOut animations:^{ thirdView.alpha = 1.0; } completion:nil]; }]; } Important: Changing the value of a property while an animation involving that property is already in progress does not stop the current animation. Instead, the current animation continues and animates to the new value you just assigned to the property. Starting Animations Using the Begin/Commit Methods If your application runs in iOS 3.2 and earlier, you must use the beginAnimations:context: and commitAnimations class methods of UIView to define your animation blocks. These methods mark the beginning and end of your animation block. Any animatable properties you change between these methods are animated to their new values after you call the commitAnimations method. Execution of the animations occurs on a secondary thread so as to avoid blocking the current thread or your application’s main thread. Note: If you are writing an application for iOS 4 or later, you should use the block-based methods for animating your content instead. For information on how to use those methods, see “Starting Animations Using the Block-Based Methods” (page 66). Listing 4-3 shows the code needed to implement the same behavior as Listing 4-1 (page 66) but using the begin/commit methods. Asin Listing 4-1, this code fades one view out while fading another in over one second of time. However, in this example, you must set the duration of the animation using a separate method call. Animations Animating Property Changes in a View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 68Listing 4-3 Performing a simple begin/commit animation [UIView beginAnimations:@"ToggleViews" context:nil]; [UIView setAnimationDuration:1.0]; // Make the animatable changes. firstView.alpha = 0.0; secondView.alpha = 1.0; // Commit the changes and perform the animation. [UIView commitAnimations]; By default, all animatable property changes within an animation block are animated. If you want to animate some changes but not others, use the setAnimationsEnabled: method to disable animations temporarily, make any changesthat you do not want animated, and then call setAnimationsEnabled: again to reenable animations. You can determine if animations are current enabled by calling the areAnimationsEnabled class method. Note: Changing the value of a property while an animation involving that property is in progress does not stop the current animation. Instead, the animation continues and animates to the new value you just assigned to the property. Configuring the Parameters for Begin/Commit Animations To configure the animation parameters for a begin/commit animation block, you use any of several UIView class methods. Table 4-2 lists these methods and describes how you use them to configure your animations. Most of these methods should be called only from inside a begin/commit animation block but some may also be used with block-based animations. If you do not call one of these methods from your animation block, a default value for the corresponding attribute is used. For more information about the default value associated with each method, see the method description in UIView Class Reference . Table 4-2 Methods for configuring animation blocks Method Usage Use either of these methodsto specify when the executionsshould begin executing. If the specified start date is in the past (or the delay is 0), the animations begin as soon as possible. setAnimationStartDate: setAnimationDelay: Use this method to set the period of time over which to execute the animations. setAnimationDuration: Animations Animating Property Changes in a View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 69Method Usage Use this method to set the timing curve of the animations. This controls whether animations execute linearly or change speed at certain times. setAnimationCurve: Use these methods to set the number of times the animation repeats and whether the animation runs in reverse at the end of each complete cycle. For more information about using these methods, see “Implementing Animations That Reverse Themselves” (page 73). setAnimationRepeatCount: setAnimationRepeatAutoreverses: Use these methods to execute code immediately before or after the animations. For more information about using a delegate,see “Configuring an Animation Delegate” (page 71). setAnimationDelegate: setAnimationWillStartSelector: setAnimationDidStopSelector: Use this method to stop all previous animations immediately and start the new animations from the stopping point. If you pass NO to this method, instead of YES, the new animations do not begin executing until the previous animations stop. setAnimationBeginsFromCurrentState: Listing 4-4 shows the code needed to implement the same behavior as the code in Listing 4-2 (page 67) but using the begin/commit methods. As before, this code fades out a view, waits one second, and then fades it back in. In order to implement the second part of the animation, the code sets up an animation delegate and implements a did-stop handler method. That handler method then sets up the second half of the animations and runs them. Listing 4-4 Configuring animation parameters using the begin/commit methods // This method begins the first animation. - (IBAction)showHideView:(id)sender { [UIView beginAnimations:@"ShowHideView" context:nil]; [UIView setAnimationCurve:UIViewAnimationCurveEaseIn]; [UIView setAnimationDuration:1.0]; [UIView setAnimationDelegate:self]; [UIView setAnimationDidStopSelector:@selector(showHideDidStop:finished:context:)]; Animations Animating Property Changes in a View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 70// Make the animatable changes. thirdView.alpha = 0.0; // Commit the changes and perform the animation. [UIView commitAnimations]; } // Called at the end of the preceding animation. - (void)showHideDidStop:(NSString *)animationID finished:(NSNumber *)finished context:(void *)context { [UIView beginAnimations:@"ShowHideView2" context:nil]; [UIView setAnimationCurve:UIViewAnimationCurveEaseOut]; [UIView setAnimationDuration:1.0]; [UIView setAnimationDelay:1.0]; thirdView.alpha = 1.0; [UIView commitAnimations]; } Configuring an Animation Delegate If you want to execute code immediately before or after an animation, you must associate a delegate object and a start or stop selector with your begin/commit animation block. You set your delegate object using the setAnimationDelegate: class method of UIView and you set your start and stop selectors using the setAnimationWillStartSelector: and setAnimationDidStopSelector: class methods. During the animation, the animation system calls your delegate methods at the appropriate times to give you a chance to perform your code. The signatures of your animation delegate methods need to be similar to the following: - (void)animationWillStart:(NSString *)animationID context:(void *)context; - (void)animationDidStop:(NSString *)animationID finished:(NSNumber *)finished context:(void *)context; The animationID and context parameters for both methods are the same parameters that you passed to the beginAnimations:context: method at the beginning of the animation block: Animations Animating Property Changes in a View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 71● animationID—An application-supplied string used to identify the animation. ● context—An application-supplied object that you can use to pass additional information to the delegate. The setAnimationDidStopSelector: selector method has an additional parameter—a Boolean value that is YES if the animation ran to completion. If the value of this parameter is NO, the animation was either canceled or stopped prematurely by another animation. Note: Although animation delegates can be used in the block-based methods, there is generally no need to use them there. Instead, place any code you want to run before the animations at the beginning of your block and place any code you want to run after the animationsfinish in a completion handler. Nesting Animation Blocks You can assign different timing and configuration options to parts of an animation block by nesting additional animation blocks. As the name implies, a nested animation block is a new animation block created inside an existing animation block. Nested animations are started at the same time as any parent animations but run (for the most part) with their own configuration options. By default, nested animations do inherit the parent’s duration and animation curve but even those options can be overridden as needed. Listing 4-5 shows an example of how a nested animation is used to change the timing, duration, and behavior of some animations in the overall group. In this case, two views are being faded to total transparency, but the transparency of the anotherView object is changed back and forth several times before it is finally hidden. The UIViewAnimationOptionOverrideInheritedCurve and UIViewAnimationOptionOverrideInheritedDuration keys used in the nested animation block allow the curve and duration values from the first animation to be modified for the second animation. If these keys were not present, the duration and curve of the outer animation block would be used instead. Listing 4-5 Nesting animations that have different configurations [UIView animateWithDuration:1.0 delay: 1.0 options:UIViewAnimationOptionCurveEaseOut animations:^{ aView.alpha = 0.0; // Create a nested animation that has a different // duration, timing curve, and configuration. Animations Animating Property Changes in a View 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 72[UIView animateWithDuration:0.2 delay:0.0 options: UIViewAnimationOptionOverrideInheritedCurve | UIViewAnimationOptionCurveLinear | UIViewAnimationOptionOverrideInheritedDuration | UIViewAnimationOptionRepeat | UIViewAnimationOptionAutoreverse animations:^{ [UIView setAnimationRepeatCount:2.5]; anotherView.alpha = 0.0; } completion:nil]; } completion:nil]; If you are using the begin/commit methods to create your animations, nesting works in much the same way as with the block-based methods. Each successive call to beginAnimations:context: within an already open animation block creates a new nested animation block that you can configure as needed. Any configuration changes you make apply to the most recently opened animation block. All animation blocks must be closed with a call to commitAnimations before the animations are submitted and executed. Implementing Animations That Reverse Themselves When creating reversible animations in conjunction with a repeat count, consider specifying a non integer value for the repeat count. For an autoreversing animation, each complete cycle of the animation involves animating from the original value to the new value and back again. If you want your animation to end on the new value, adding 0.5 to the repeat count causes the animation to complete the extra half cycle needed to end at the new value. If you do not include this half step, your animation will animate to the original value and then snap quickly to the new value, which may not be the visual effect you want. Creating Animated Transitions Between Views View transitions help you hide sudden changes associated with adding, removing, hiding, or showing views in your view hierarchy. You use view transitions to implement the following types of changes: Animations Creating Animated Transitions Between Views 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 73● Change the visible subviews of an existing view. You typically choose this option when you want to make relatively small changes to an existing view. ● Replace one view in your view hierarchy with a different view. You typically choose this option when you want to replace a view hierarchy that spans all or most of the screen. Important: View transitions should not be confused with transitions initiated by view controllers, such as the presentation of modal view controllers or the pushing of new view controllers onto a navigation stack. View transitions affect the view hierarchy only, whereas view-controller transitions change the active view controller as well. Thus, for view transitions, the view controller that was active when you initiated the transition remains active when the transition finishes. For more information about how you can use view controllers to present new content, see View Controller Programming Guide for iOS . Changing the Subviews of a View Changing the subviews of a view allows you to make moderate changes to the view. For example, you might add or remove subviews to toggle the superview between two different states. By the time the animations finish, the same view is displayed but its contents are now different. In iOS 4 and later, you use the transitionWithView:duration:options:animations:completion: method to initiate a transition animation for a view. In the animations block passed to this method, the only changesthat are normally animated are those associated with showing, hiding, adding, or removing subviews. Limiting animations to this set allows the view to create a snapshot image of the before and after versions of the view and animate between the two images, which is more efficient. However, if you need to animate other changes, you can include the UIViewAnimationOptionAllowAnimatedContent option when calling the method. Including that option prevents the view from creating snapshots and animates all changes directly. Listing 4-6 is an example of how to use a transition animation to make it seem as if a new text entry page has been added. In this example, the main view contains two embedded text views. The text views are configured identically, but one is always visible while the other is always hidden. When the user taps the button to create a new page, this method toggles the visibility of the two views, resulting in a new empty page with an empty text view ready to accept text. After the transition is complete, the view saves the text from the old page using a private method and resets the now hidden text view so that it can be reused later. The view then arranges its pointers so that it can be ready to do the same thing if the user requests yet another new page. Listing 4-6 Swapping an empty text view for an existing one - (IBAction)displayNewPage:(id)sender { Animations Creating Animated Transitions Between Views 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 74[UIView transitionWithView:self.view duration:1.0 options:UIViewAnimationOptionTransitionCurlUp animations:^{ currentTextView.hidden = YES; swapTextView.hidden = NO; } completion:^(BOOL finished){ // Save the old text and then swap the views. [self saveNotes:temp]; UIView* temp = currentTextView; currentTextView = swapTextView; swapTextView = temp; }]; } If you need to perform view transitions in iOS 3.2 and earlier, you can use the setAnimationTransition:forView:cache: method to specify the parameters for the transition. The view you pass to that method is the same one you would pass in as the first parameter to the transitionWithView:duration:options:animations:completion: method. Listing 4-7 shows the basic structure of the animation block you need to create. Note that to implement the completion block shown in Listing 4-6 (page 74), you would need to configure an animation delegate with a did-stop handler as described in “Configuring an Animation Delegate” (page 71). Listing 4-7 Changing subviews using the begin/commit methods [UIView beginAnimations:@"ToggleSiblings" context:nil]; [UIView setAnimationTransition:UIViewAnimationTransitionCurlUp forView:self.view cache:YES]; [UIView setAnimationDuration:1.0]; // Make your changes [UIView commitAnimations]; Animations Creating Animated Transitions Between Views 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 75Replacing a View with a Different View Replacing views is something you do when you want your interface to be dramatically different. Because this technique swaps only views (and not view controllers), you are responsible for designing your application’s controller objects appropriately. This technique is simply a way of presenting new views quickly using some standard transitions. In iOS 4 and later, you use the transitionFromView:toView:duration:options:completion: method to transition between two views. This method actually removes the first view from your hierarchy and inserts the other, so you should make sure you have a reference to the first view if you want to keep it. If you want to hide views instead of remove them from your view hierarchy, pass the UIViewAnimationOptionShowHideTransitionViews key as one of the options. Listing 4-8 shows the code needed to swap between two main views managed by a single view controller. In this example, the view controller’s root view always displays one of two child views (primaryView or secondaryView). Each view presents the same content but does so in a different way. The view controller uses the displayingPrimary member variable (a Boolean value) to keep track of which view is displayed at any given time. The flip direction changes depending on which view is being displayed. Listing 4-8 Toggling between two views in a view controller - (IBAction)toggleMainViews:(id)sender { [UIView transitionFromView:(displayingPrimary ? primaryView : secondaryView) toView:(displayingPrimary ? secondaryView : primaryView) duration:1.0 options:(displayingPrimary ? UIViewAnimationOptionTransitionFlipFromRight : UIViewAnimationOptionTransitionFlipFromLeft) completion:^(BOOL finished) { if (finished) { displayingPrimary = !displayingPrimary; } }]; } Animations Creating Animated Transitions Between Views 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 76Note: In addition to swapping out views, your view controller code needs to manage the loading and unloading of both the primary and secondary views. For information on how views are loaded and unloaded by a view controller, see View Controller Programming Guide for iOS . Linking Multiple Animations Together The UIView animation interfaces provide support for linking separate animation blocks so that they perform sequentially instead of at the same time. The process for linking animation blocks depends on whether you are using the block-based animation methods or the begin/commit methods: ● For block-based animations, use the completion handler supported by the animateWithDuration:animations:completion: and animateWithDuration:delay:options:animations:completion: methods to execute any follow-on animations. ● For begin/commit animations, associate a delegate object and a did-stop selector with the animation. For information about how to associate a delegate with your animations, see “Configuring an Animation Delegate” (page 71). An alternative to linking animations together is to use nested animations with different delay factors so as to start the animations at different times. For more information on how to nest animations,see “Nesting Animation Blocks” (page 72). Animating View and Layer Changes Together Applications can freely mix view-based and layer-based animation code as needed but the process for configuring your animation parameters depends on who owns the layer. Changing a view-owned layer is the same as changing the view itself, and any animations you apply to the layer’s properties respect the animation parameters of the current view-based animation block. The same is not true for layers that you create yourself. Custom layer objects ignore view-based animation block parameters and use the default Core Animation parameters instead. If you want to customize the animation parametersfor layers you create, you must use Core Animation directly. Typically, animating layers using Core Animation involves creating a CABasicAnimation object orsome other concrete subclass of CAAnimation. You then add that animation to the corresponding layer. You can apply the animation from either inside or outside a view-based animation block. Animations Linking Multiple Animations Together 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 77Listing 4-9 shows an animation that modifies a view and a custom layer at the same time. The view in this example contains a custom CALayer object at the center of its bounds. The animation rotatesthe view counter clockwise while rotating the layer clockwise. Because the rotations are in opposite directions, the layer maintains its original orientation relative to the screen and does not appear to rotate significantly. However, the view beneath that layerspins 360 degrees and returnsto its original orientation. This example is presented primarily to demonstrate how you can mix view and layer animations. Thistype of mixing should not be used in situations where precise timing is needed. Listing 4-9 Mixing view and layer animations [UIView animateWithDuration:1.0 delay:0.0 options: UIViewAnimationOptionCurveLinear animations:^{ // Animate the first half of the view rotation. CGAffineTransform xform = CGAffineTransformMakeRotation(DEGREES_TO_RADIANS(-180)); backingView.transform = xform; // Rotate the embedded CALayer in the opposite direction. CABasicAnimation* layerAnimation = [CABasicAnimation animationWithKeyPath:@"transform"]; layerAnimation.duration = 2.0; layerAnimation.beginTime = 0; //CACurrentMediaTime() + 1; layerAnimation.valueFunction = [CAValueFunction functionWithName:kCAValueFunctionRotateZ]; layerAnimation.timingFunction = [CAMediaTimingFunction functionWithName:kCAMediaTimingFunctionLinear]; layerAnimation.fromValue = [NSNumber numberWithFloat:0.0]; layerAnimation.toValue = [NSNumber numberWithFloat:DEGREES_TO_RADIANS(360.0)]; layerAnimation.byValue = [NSNumber numberWithFloat:DEGREES_TO_RADIANS(180.0)]; [manLayer addAnimation:layerAnimation forKey:@"layerAnimation"]; } completion:^(BOOL finished){ // Now do the second half of the view rotation. [UIView animateWithDuration:1.0 Animations Animating View and Layer Changes Together 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 78delay: 0.0 options: UIViewAnimationOptionCurveLinear animations:^{ CGAffineTransform xform = CGAffineTransformMakeRotation(DEGREES_TO_RADIANS(-359)); backingView.transform = xform; } completion:^(BOOL finished){ backingView.transform = CGAffineTransformIdentity; }]; }]; Note: In Listing 4-9 (page 78), you could also create and apply the CABasicAnimation object outside of the view-based animation block to achieve the same results. All of the animations ultimately rely on Core Animation for their execution. Thus, if they are submitted at approximately the same time, they run together. If precise timing between your view and layer based animations is required, it is recommended that you create all of the animations using Core Animation. You may find that some animations are easier to perform using Core Animation anyway. For example, the view-based rotation in Listing 4-9 (page 78) requires a multistep sequence for rotations of more than 180 degrees, whereas the Core Animation portion uses a rotation value function that rotates from start to finish through a middle value. For more information about how to create and configure animations using Core Animation,see Core Animation Programming Guide and Core Animation Cookbook . Animations Animating View and Layer Changes Together 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 79This table describes the changes to View Programming Guide for iOS . Date Notes 2011-03-08 Reorganized and expanded the content of the document. Added information on how to create view-based animations. Incorporated information on how to display content on an external display. Added information about how to work with high-resolution screens. New document describing the creation and management of views, windows, and other visual interface elements. 2010-05-17 2011-03-08 | © 2011 Apple Inc. All Rights Reserved. 80 Document Revision HistoryApple Inc. © 2011 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, Cocoa, iPad, iPhone, iPod, iPod touch, Quartz, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries. Retina is a trademark of Apple Inc. OpenGL is a registered trademark of Silicon Graphics, Inc. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. Stream Programming GuideContents Introduction to Stream Programming Guide for Cocoa 4 Organization of This Document 4 See Also 4 Cocoa Streams 6 Reading From Input Streams 8 Preparing the Stream Object 8 Handling Stream Events 9 Disposing of the Stream Object 11 Writing To Output Streams 12 Preparing the Stream Object 12 Handling Stream Events 13 Disposing of the Stream Object 15 Polling Versus Run-Loop Scheduling 17 Handling Stream Errors 20 Setting Up Socket Streams 22 Basic Procedure 22 Securing and Configuring the Connection 23 Initiating an HTTP Request 24 For More Information 25 Document Revision History 26 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 2Figures and Listings Cocoa Streams 6 Figure 1 Sources and destinations of stream objects 6 Reading From Input Streams 8 Listing 1 Creating and initializing an NSInputStream object 8 Listing 2 Handling a bytes-available event 10 Listing 3 Closing and releasing the NSInputStream object 11 Writing To Output Streams 12 Listing 1 Creating and initializing an NSOutputStream object for memory 13 Listing 2 Handling a space-available event 14 Listing 3 Closing and releasing the NSInputStream object 15 Polling Versus Run-Loop Scheduling 17 Listing 1 Writing to an output stream using polling 17 Handling Stream Errors 20 Listing 1 Handling stream errors 20 Setting Up Socket Streams 22 Listing 1 Setting up a network socket stream 22 Listing 2 Making an HTTP GET request 24 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 3A stream is a fundamental abstraction in programming: a sequence of bits transmitted serially from one point to another point. Cocoa provides three classes to represent streams and facilitate their use in your programs: NSStream, NSInputStream, and NSOutputStream. With the instances of these classes you can read data from, and write data to, files and application memory. You can also use these objects in socket-based connections to exchange data with remote hosts. You can also subclass the stream classes to obtain specialized stream behavior. Organization of This Document This document includes the following articles: ● “Cocoa Streams” (page 6) gives an overview of the Cocoa stream classes, describing architecture, capabilities, and general usage. ● “Reading From Input Streams” (page 8) explains how to create and prepare a (non-socket) input-stream object. It also describes how to handle stream events generated by all types of NSInputStream objects. ● “Writing To Output Streams” (page 12) explains how to create and prepare a (non-socket) output-stream object. It also describes how to handle stream events generated by all types of NSOutputStream objects. ● “Polling Versus Run-Loop Scheduling” (page 17) discusses the relative merits of the two techniques used to avoid blocking when reading and writing to streams. It also illustrates how to poll for stream data using the API of the stream classes. ● “Handling Stream Errors” (page 20) describes how to handle errors that occur in stream processing. ● “Setting Up Socket Streams” (page 22) explains how to set up stream objects used to communicate with remote hosts via sockets. See Also You may find the following external resources helpful if you are implementing socket-based network streams: ● OpenSSL — http://www.openssl.org/ ● Apache SSL — http://www.apache-ssl.org/ 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 4 Introduction to Stream Programming Guide for Cocoa● SOCKS — http://tools.ietf.org/html/rfc1928 Introduction to Stream Programming Guide for Cocoa See Also 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 5Streams provide an easy way for a program to exchange data with a variety of media in a device-independent way. A stream is a contiguous sequence of bits transmitted serially over a communications path. It is unidirectional and hence, from the perspective of a program, a stream can be an input (or read) stream or an output (or write) stream. Except for ones that are file-based, streams are non-seekable; once stream data has been provided or consumed, it cannot be retrieved again from the stream. Cocoa includes three stream-related classes: NSStream, NSInputStream, and NSOutputStream. NSStream is an abstract classthat definesthe fundamental interface and propertiesfor allstream objects. NSInputStream and NSOutputStream are subclasses of NSStream and implement default input-stream and output-stream behavior. You can create NSOutputStream instances for stream data located in memory or written to a file or C buffer; you can create NSInputStream instances for stream data read from an NSData object or a file. You can also have NSInputStream and NSOutputStream objects at the end points of a socket-based network connection and you can use stream objects without loading all of the stream data into memory at once. Figure 1 illustrates the types of input-stream and output-stream objects in terms of their sources or destinations. Figure 1 Sources and destinations of stream objects NSOutputStream Buffer Memory (NSData) Network socket Data (NSData) Network socket Client program NSInputStream File File Because they deal with such a basic computing abstraction (streams), NSStream and itssubclasses are intended for lower-level programming tasks. If there is a higher-level Cocoa API that is more suited for a particular task (for example, NSURL or NSFileHandle) use it instead. 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 6 Cocoa StreamsStream objects have properties associated with them. Most properties have to do with network security and configuration, namely secure-socket (SSL) levels and SOCKS proxy information. Two important additional properties are NSStreamDataWrittenToMemoryStreamKey, which permits retrieval of data written to memory for an output stream, and NSStreamFileCurrentOffsetKey, which allows you to manipulate the current read or write position in file-based streams. A stream object also has a delegate associated with it. If a delegate is not explicitly set, the stream object itself becomesthe delegate (a useful convention for custom subclasses). A stream object invokesthe sole delegation method stream:handleEvent: for each stream-related event it handles. Of particular importance are the events that indicate when bytes are available to read from an input stream and when an output stream signals that it’s ready to accept bytes. For these two events, the delegate sends the stream the appropriate message—read:maxLength: or write:maxlength:, depending on type of stream—to get the bytes from the stream or to put bytes on the stream. NSStream is built on the CFStream layer of Core Foundation. This close relationship means that the concrete subclasses of NSStream, NSOutputStream and NSInputStream, are toll-free bridged with their Core Foundation counterparts CFWriteStream and CFReadStream. Although there are strong similarities between the Cocoa and Core Foundation stream APIs, their implementations are not exactly coincident. The Cocoa stream classes use the delegation model for asynchronous behavior (assuming run-loop scheduling) while Core Foundation uses client callbacks. The Core Foundation stream types sets the client (termed a context in Core Foundation) differently than the NSStream setsthe delegate; callsto set the delegate should not be mixed with calls to set the context. Otherwise you can freely intermix calls from the two APIs in your code. Despite their strong similarities, NSStream does give you a major advantage over CFStream. Because of its Objective-C underpinnings, it is extensible. You can subclass NSStream, NSInputStream, or NSOutputStream to customize stream attributes and behavior. For example, you could create an input stream that maintains statistics on the bytes it reads; or you could make a NSStream subclass whose instances can seek through their stream, putting back bytes that have been read. NSStream has its own set of required overrides, as do NSInputStream and NSOutputStream. See the reference documentation for NSStream, NSInputStream, and NSOutputStream for details on subclassing these classes. Cocoa Streams 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 7In Cocoa, reading from an NSInputStream instance consists of several steps: 1. Create and initialize an instance of NSInputStream from a source of data. 2. Schedule the stream object on a run loop and open the stream. 3. Handle the events that the stream object reports to its delegate. 4. When there is no more data to read, dispose of the stream object. The following discussion goes into each of these steps in more detail. Note: The examples in this document show the strategy of scheduling stream objects on run loops and setting a delegate to handle stream events. You may use polling instead of run-loop scheduling if you prefer that approach. However, run-loop scheduling with delegation isthe preferred approach for various reasons (described in “Polling Versus Run-Loop Scheduling” (page 17)), and that is why it is highlighted in this document. Preparing the Stream Object To begin using an NSInputStream object you must have (after first locating, if necessary) a source of data for the stream. The source of data can be a file, an NSData object, or a network socket. Note: The procedure for initializing input-stream objects from network sockets is different from the procedure for the other two data sources, and is not covered in this article. To learn about initializing an NSInputStream instance for a network connection, see “Setting Up Socket Streams” (page 22). The initializers and factory methods for NSInputStream allow you to create and initialize the instance from an NSData or file. Listing 1 shows an NSInputStream instance created from a file. Listing 1 Creating and initializing an NSInputStream object - (void)setUpStreamForFile:(NSString *)path { // iStream is NSInputStream instance variable 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 8 Reading From Input StreamsiStream = [[NSInputStream alloc] initWithFileAtPath:path]; [iStream setDelegate:self]; [iStream scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [iStream open]; } As this example shows, after you create the object you should set the delegate (more often than not to self). The delegate receives stream:handleEvent: messages from the NSInputStream object when that object is scheduled on the run loop and hasstream-related eventsto report,such as when there are bytes on the stream to be read. Before you open the stream to begin the streaming of data,send a scheduleInRunLoop:forMode: message to the stream object to schedule it to receive stream events on a run loop. By doing this, you are helping the delegate to avoid blocking when there is no data on the stream to read. If streaming is taking place on another thread, be sure to schedule the stream object on that thread’s run loop. You should never attempt to access a scheduled stream from a thread different than the one owning the stream’s run loop. Finally, send the NSInputStream instance an open message to start the streaming of data from the input source. Handling Stream Events After a stream object is sent open, you can find out about its status, whether it has bytes available to read, and the nature of any error with the following messages: streamStatus hasBytesAvailable streamError The returned status is an NSStreamStatus constant indicating that the stream is opening, reading, at the end of the stream, and so on. The returned error is an NSError object encapsulating information about any error that took place. (See the reference documentation for NSStream for descriptions of NSStreamStatus and other stream types.) More importantly, once the streamobject has been opened, it keepssending stream:handleEvent:messages to its delegate until it encounters the end of the stream. These messages include a parameter with an NSStreamEvent constant that indicates the type of event. For NSInputStream objects, the most common types of events are NSStreamEventOpenCompleted, NSStreamEventHasBytesAvailable, and Reading From Input Streams Handling Stream Events 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 9NSStreamEventEndEncountered. The delegate is typically most interested in NSStreamEventHasBytesAvailable events. Listing 2 illustrates a good approach for handling this type of event. Listing 2 Handling a bytes-available event - (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode { switch(eventCode) { case NSStreamEventHasBytesAvailable: { if(!_data) { _data = [[NSMutableData data] retain]; } uint8_t buf[1024]; unsigned int len = 0; len = [(NSInputStream *)stream read:buf maxLength:1024]; if(len) { [_data appendBytes:(const void *)buf length:len]; // bytesRead is an instance variable of type NSNumber. [bytesRead setIntValue:[bytesRead intValue]+len]; } else { NSLog(@"no buffer!"); } break; } // continued In this implementation of stream:handleEvent: the delegate uses a switch statement to identify the passed-in NSStreamEvent constant. If the constant is NSStreamEventHasBytesAvailable, the delegate first lazily creates(if necessary) an NSMutableData object (_data) to hold the retrieved bytes. Then it declares a buffer of a certain size (1024 bytes, in this case) and invokes the stream object’s read:maxLength: method, which fills up the buffer with the specified number of bytes. If the read operation successfully fetched bytes from the stream, the delegate appends these bytes to the NSMutableData object. Reading From Input Streams Handling Stream Events 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 10There is no firm guideline on how many bytes to read at one time. Although it may be possible to read all the data in the stream in one event, this depends on the length of the stream (that is, the number of bytes in it) as well as the behavior of the kernel, including device and socket characteristics. The best approach is to use some reasonable buffer size, such as 512 bytes, one kilobyte (as in the example above), or a page size (four kilobytes). When the NSInputStream object experiences errors processing the stream, it stops streaming and notifies its delegate with a NSStreamEventErrorOccurred. The delegate should handle the error in its stream:handleEvent: method as described in “Handling Stream Errors” (page 20). Disposing of the Stream Object When an NSInputStream object reaches the end of a stream, it sends the delegate a NSStreamEventEndEncountered event in a stream:handleEvent: message. The delegate should dispose of the object by doing the mirror-opposite of what it did to prepare the object. In other words, it should first close the stream object, remove it from the run loop, and finally release it. Listing 3 gives an example of how you might do this. Listing 3 Closing and releasing the NSInputStream object - (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode { switch(eventCode) { case NSStreamEventEndEncountered: { [stream close]; [stream removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [stream release]; stream = nil; // stream is ivar, so reinit it break; } // continued ... } } Reading From Input Streams Disposing of the Stream Object 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 11Using an NSOutputStream instance to write to an output stream requires several steps: 1. Create and initialize an instance of NSOutputStream with a repository for the written data. Also set a delegate. 2. Schedule the stream object on a run loop and open the stream. 3. Handle the events that the stream object reports to its delegate. 4. If the stream object has written data to memory, obtain the data by requesting the NSStreamDataWrittenToMemoryStreamKey property. 5. When there is no more data to write, dispose of the stream object. The following discussion goes into each of these steps in more detail. Note: The examples in this document show the strategy of scheduling stream objects on run loops and setting a delegate to handle stream events. You may use polling instead of run-loop scheduling if you prefer that approach. However, run-loop scheduling with delegation isthe preferred approach for various reasons (described in “Polling Versus Run-Loop Scheduling” (page 17)), and that is why it is highlighted in this document. Preparing the Stream Object To begin using an NSOutputStream object you must specify a destination for the data written to the stream. The destination for an output-stream object can be a file, a C buffer, application memory, or a network socket. 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 12 Writing To Output StreamsNote: The procedure for initializing output-stream objects from network sockets is different from the procedure for the other data destinations, and is not covered in this article. To learn about initializing an NSOutputStream instance for a network connection, see “Setting Up Socket Streams” (page 22). The initializers and factory methods for NSOutputStream allow you to create and initialize the instance with a file, a buffer, or memory. Listing 1 shows the creation of an NSOutputStream instance that will write data to application memory. Listing 1 Creating and initializing an NSOutputStream object for memory - (void)createOutputStream { NSLog(@"Creating and opening NSOutputStream..."); // oStream is an instance variable oStream = [[NSOutputStream alloc] initToMemory]; [oStream setDelegate:self]; [oStream scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [oStream open]; } As the code in Listing 1 shows, after you create the object you should set the delegate (more often than not to self). The delegate receives stream:handleEvent: messages from the NSOutputStream object when that object has stream-related events to report, such as when the stream has space for bytes. Before you open the stream to begin the streaming of data,send a scheduleInRunLoop:forMode: message to the stream object to schedule it to receive stream events on a run loop. By doing this, you are helping the delegate to avoid blocking when the stream is unable to accept more bytes. If streaming is taking place on another thread, be sure to schedule the stream object on that thread’s run loop. You should never attempt to access a scheduled stream from a thread different than the one owning the stream’s run loop. Finally, send the NSOutputStream instance an open message to start the streaming of data to the output container. Handling Stream Events After a stream object is sent open, you can find out about its status, whether it has space for writing data, and the nature of any error with the following messages: streamStatus Writing To Output Streams Handling Stream Events 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 13hasSpaceAvailable streamError The returned status is an NSStreamStatus constant indicating that the stream is opening, writing, at the end of the stream, and so on. The returned error is an NSError object encapsulating information about any error that took place. (See the reference documentation for NSStream for descriptions of NSStreamStatus and other stream types.) More importantly, once the streamobject has been opened, it keepssending stream:handleEvent:messages to its delegate (as long as the delegate continues to put bytes on the stream) until it encounters the end of the stream. These messages include a parameter with an NSStreamEvent constant that indicates the type of event. For NSOutputStream objects, the most common types of events are NSStreamEventOpenCompleted, NSStreamEventHasSpaceAvailable, and NSStreamEventEndEncountered. The delegate is typically most interested in NSStreamEventHasSpaceAvailable events. Listing 2 illustrates one approach you could take to handle this type of event. Listing 2 Handling a space-available event - (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode { switch(eventCode) { case NSStreamEventHasSpaceAvailable: { uint8_t *readBytes = (uint8_t *)[_data mutableBytes]; readBytes += byteIndex; // instance variable to move pointer int data_len = [_data length]; unsigned int len = ((data_len - byteIndex >= 1024) ? 1024 : (data_len-byteIndex)); uint8_t buf[len]; (void)memcpy(buf, readBytes, len); len = [stream write:(const uint8_t *)buf maxLength:len]; byteIndex += len; break; } // continued ... } } Writing To Output Streams Handling Stream Events 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 14In this implementation of stream:handleEvent: the delegate uses a switch statement to identify the passed-in NSStreamEvent constant. If the constant is NSStreamEventHasSpacesAvailable, the delegate gets the bytes held by a NSMutableData object (_data) and advances the pointer for the current write operation. It next determines the byte capacity of the impending write operation (1024 or the remaining bytes to write), declares a buffer of that size, and copies that amount of data to the buffer. Next the delegate invokes the output-stream object’s write:maxLength: method to put the buffer’s contents onto the output stream. Finally it advances the index used to advance the readBytes pointer for the next operation. If the delegate receives an NSStreamEventHasSpaceAvailable event and does not write anything to the stream, it does not receive further space-available events from the run loop until the NSOutputStream object receives more bytes. When this happens, the run loop is restarted for space-available events. If this scenario is likely in your implementation, you can have the delegate set a flag when it doesn’t write to the stream upon receiving an NSStreamEventHasSpaceAvailable event. Later, when your program has more bytesto write, it can check this flag and, if set, write to the output-stream instance directly. There is no firm guideline on how many bytes to write at one time. Although it may be possible to write all the data to the stream in one event, this depends on external factors, such as the behavior of the kernel and device and socket characteristics. The best approach is to use some reasonable buffer size, such as 512 bytes, one kilobyte (as in the example above), or a page size (four kilobytes). When the NSOutputStream object experiences errors writing to the stream, it stops streaming and notifies its delegate with a NSStreamEventErrorOccurred. The delegate should handle the error in its stream:handleEvent: method as described in “Handling Stream Errors” (page 20). Disposing of the Stream Object When an NSOutputStream object concludes writing data to an output stream, it sends the delegate a NSStreamEventEndEncountered event in a stream:handleEvent: message. At this point the delegate should dispose of the stream object by doing the mirror-opposite of what it did to prepare the object. In other words, it should first close the stream object, remove it from the run loop, and finally release it. Furthermore, if the destination for the NSOutputStream object is application memory (that is, you created the instance using initToMemory or the factory method outputStreamToMemory), you might now want to retrieve the data held in memory. Listing 3 illustrates how you might do all of these things. Listing 3 Closing and releasing the NSInputStream object - (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode { switch(eventCode) { case NSStreamEventEndEncountered: Writing To Output Streams Disposing of the Stream Object 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 15{ NSData *newData = [oStream propertyForKey: NSStreamDataWrittenToMemoryStreamKey]; if (!newData) { NSLog(@"No data written to memory!"); } else { [self processData:newData]; } [stream close]; [stream removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [stream release]; oStream = nil; // oStream is instance variable break; } // continued ... } } You get the stream data written to memory by sending the NSOutputStream object a propertyForKey: message, specifying a key of NSStreamDataWrittenToMemoryStreamKey The stream object returns the data in an NSData object. Writing To Output Streams Disposing of the Stream Object 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 16A potential problem with stream processing is blocking. A thread that is writing to or reading from a stream might have to wait indefinitely until there is (respectively) space on the stream to put bytes or bytes on the stream that can be read. In effect, the thread is at the mercy of the stream, and that can spell trouble for an application. Blocking can especially be a problem with socketstreams because they are dependent on responses from a remote host. With Cocoa streams you have two ways to handle stream events: ● Run-loop scheduling. You schedule a stream object on a run loop so that the delegate receives messages reporting stream-related events only when blocking is unlikely to take place. For read and write operations, the pertinent NSStreamEvent constants are NSStreamHasBytesAvailable and NSStreamHasSpaceAvailable. ● Polling. In a closed loop broken only at the end of the stream or upon error, you keep asking the stream object if it has (for read streams) bytes available to read or (for write streams) space available for writing. The pertinent methods are hasBytesAvailable (NSInputStream) and hasSpaceAvailable (NSOutputStream). Run-loop scheduling is almost always preferable over polling, and that is why the code examples in “Reading From Input Streams” (page 8) and “Writing To Output Streams” (page 12) exclusively show the use of run loops. With polling, your program is locked in a tight loop, waiting for stream events that might or might not be imminent. With run-loop scheduling, your program can go off and do other things, knowing that it will be notified when there is a stream event to handle. Moreover, run loops save you from having to manage state and are more efficient than polling. Polling is also CPU-intensive; there are other things you can be doing with your processing time. That said, there can be situations where polling is a viable option. For example, if you are porting legacy code, you might choose to use polling because it is better suited the threading model in the legacy code. Listing 1 illustrates a method that writes data to an output stream using polling. Listing 1 Writing to an output stream using polling - (void)createNewFile { oStream = [[NSOutputStream alloc] initToMemory]; [oStream open]; 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 17 Polling Versus Run-Loop Schedulinguint8_t *readBytes = (uint8_t *)[data mutableBytes]; uint8_t buf[1024]; int len = 1024; while (1) { if (len == 0) break; if ( [oStream hasSpaceAvailable] ) { (void)strncpy(buf, readBytes, len); readBytes += len; if ([oStream write:(const uint8_t *)buf maxLength:len] == -1) { [self handleError:[oStream streamError]]; break; } [bytesWritten setIntValue:[bytesWritten intValue]+len]; len = (([data length] - [bytesWritten intValue] >= 1024) ? 1024 : [data length] - [bytesWritten intValue]); } } NSData *newData = [oStream propertyForKey: NSStreamDataWrittenToMemoryStreamKey]; if (!newData) { NSLog(@"No data written to memory!"); } else { [self processData:newData]; } [oStream close]; [oStream release]; oStream = nil; } It should be pointed out that neither the polling nor run-loop scheduling approaches are airtight defenses against blocking. If the NSInputStream hasBytesAvailable method or the NSOutputStream hasSpaceAvailable method returns NO, it means in both cases that the stream definitely has no available bytes or space. However, if either of these methods returns YES, it can mean that there is available bytes or Polling Versus Run-Loop Scheduling 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 18space or that the only way to find out isto attempt a read or a write operation (which could lead to a momentary block). The NSStreamEventHasBytesAvailable and NSStreamEventHasSpaceAvailable stream events have identical semantics. Polling Versus Run-Loop Scheduling 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 19Occasionally, and especially with sockets, streams can experience errors that prevent further processing of stream data. Generally, errors indicate the absence of something at one end of a stream, such as the crash of a remote host or the deletion of a file being streamed. There is a little that a client of a stream can do when most errors occur except report the error to the user. Although a stream object that has reported an error can be queried for state before it is closed, it cannot be reused for read or write operations. The NSStream and NSOutputStream classes inform you if an error occurred in several ways: ● If the stream object is scheduled on a run loop, the object reports a NSStreamEventErrorOccurred event to its delegate in a stream:handleEvent: message. ● At any time, the client can send a streamStatus message to a stream object and see if it returns NSStreamStatusError. ● If you attempt to write to an NSOutputStream object by sending it write:maxLength: and it returns -1, a write error has occurred. Once you have determined that a stream object experienced an error, you can query the object with a streamError message to get more information about the error (in the form of an NSError object). Next, inform the user about the error. Listing 1 shows how the delegate of a run loop-scheduled stream object might handle an error. Listing 1 Handling stream errors - (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode { NSLog(@"stream:handleEvent: is invoked..."); switch(eventCode) { case NSStreamEventErrorOccurred: { NSError *theError = [stream streamError]; NSAlert *theAlert = [[NSAlert alloc] init]; [theAlert setMessageText:@"Error reading stream!"]; [theAlert setInformativeText:[NSString stringWithFormat:@"Error %i: %@", 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 20 Handling Stream Errors[theError code], [theError localizedDescription]]]; [theAlert addButtonWithTitle:@"OK"]; [theAlert beginSheetModalForWindow:[NSApp mainWindow] modalDelegate:self didEndSelector:@selector(alertDidEnd:returnCode:contextInfo:) contextInfo:nil]; [stream close]; [stream release]; break; } // continued .... } } For some errors, you can attempt to do more than inform the user. For example, if you try to set an SSL security level on a socket connection but the remote host is not secure, the stream object will report an error. You can then release the old stream object and create a new one for a non-secure socket connection. Handling Stream Errors 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 21You can use the CFStream API to establish a socket connection and, with the stream object (or objects) created as a result,send data to and receive data from a remote host. You can also configure the connection forsecurity. Basic Procedure The NSStream class does notsupport connecting to a remote host on iOS. CFStream does support this behavior, however, and once you have created your streams with the CFStream API, you can take advantage of the toll-free bridge between CFStream and NSStream to cast your CFStreams to NSStreams. Just call the CFStreamCreatePairWithSocketToHost function, providing a host name and a port number, to receive both a CFReadStreamRef and a CFWriteStreamRef for the given host. You can then cast these objects to an NSInputStream and an NSOutputStream and proceed. Listing 1 illustrates the use of CFStreamCreatePairWithSocketToHost. This example shows the creation of both a CFReadStreamRef object and a CFWriteStreamRef object. If you want to receive only one of these objects, just specify NULL as the parameter value for the unwanted object. Listing 1 Setting up a network socket stream - (IBAction)searchForSite:(id)sender { NSString *urlStr = [sender stringValue]; if (![urlStr isEqualToString:@""]) { NSURL *website = [NSURL URLWithString:urlStr]; if (!website) { NSLog(@"%@ is not a valid URL"); return; } CFReadStreamRef readStream; CFWriteStreamRef writeStream; CFStreamCreatePairWithSocketToHost(NULL, (CFStringRef)[website host], 80, &readStream, &writeStream); 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 22 Setting Up Socket StreamsNSInputStream *inputStream = (__bridge_transfer NSInputStream *)readStream; NSOutputStream *outputStream = (__bridge_transfer NSOutputStream *)writeStream; [inputStream setDelegate:self]; [outputStream setDelegate:self]; [inputStream scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [outputStream scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [inputStream open]; [outputStream open]; /* Store a reference to the input and output streams so that they don't go away.... */ ... } } If you pass in invalid parameters, one or both of the requested CFReadStreamRef and CFWriteStreamRef objects are NULL. Once you have cast the CFStreams to NSStreams, set the delegate, schedule the stream on a run loop, and open the stream as usual. The delegate should begin to receive stream-event messages (stream:handleEvent:). See “Reading From Input Streams” (page 8) and “Writing To Output Streams” (page 12) for more information. Securing and Configuring the Connection Before you open a stream object, you might want to set security and other features for the connection to the remote host (which might be, for example, an HTTPS server). NSStream defines properties that affect the security of TCP/IP socket connections in two ways: ● Secure Socket Layer (SSL). A security protocol using digital certificates to provide data encryption, server authentication, message integrity, and (optionally) client authentication for TCP/IP connections. ● SOCKS proxy server. Setting Up Socket Streams Securing and Configuring the Connection 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 23A server that sits between a client application and a real server over a TCP/IP connection. It intercepts requests to the real server and, if it cannot fulfill them from a cache of recently requested files, forwards them to the real server. SOCKS proxy servers help improve performance over a network and can also be used to filter requests. For SSL security, NSStream defines various security-level properties (for example, NSStreamSocketSecurityLevelSSLv2). You set these properties by sending setProperty:forKey: to the stream object using the key NSStreamSocketSecurityLevelKey, as in this sample message: [inputStream setProperty:NSStreamSocketSecurityLevelTLSv1 forKey:NSStreamSocketSecurityLevelKey]; You must set the property before you open the stream. Once it opens, it goes through a handshake protocol to find out what level of SSL security the other side of the connection is using. If the security level is not compatible with the specified property, the stream object generates an error event. However, if you request a negotiated security level (NSStreamSocketSecurityLevelNegotiatedSSL), the security level becomes the highest that both sides of the connection can implement. Still, if you try to set an SSL security level when the remote host is not secure, an error is generated. To configure a SOCKS proxy server for a connection, you need to construct a dictionary with keys of the form NSStreamSOCKSProxyNameKey (for example, NSStreamSOCKSProxyHostKey). The value of each key is the SOCKS proxy setting that Name refers to. Then using setProperty:forKey:, set the dictionary as the value of the NSStreamSOCKSProxyConfigurationKey. Initiating an HTTP Request If you are opening a connection to an HTTP server (that is, a website), then you may have to initiate a transaction with that server by sending it an HTTP request. A good time to make this request is when the delegate of the NSOutputStream objectreceives a NSStreamEventHasSpaceAvailable event via a stream:handleEvent: message. “Making an HTTP GET request” shows the delegate creating an HTTP GET request and writing it to the output stream, after which it immediately closes the stream object. Listing 2 Making an HTTP GET request - (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode { NSLog(@"stream:handleEvent: is invoked..."); switch(eventCode) { Setting Up Socket Streams Initiating an HTTP Request 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 24case NSStreamEventHasSpaceAvailable: { if (stream == oStream) { NSString * str = [NSString stringWithFormat: @"GET / HTTP/1.0\r\n\r\n"]; const uint8_t * rawstring = (const uint8_t *)[str UTF8String]; [oStream write:rawstring maxLength:strlen(rawstring)]; [oStream close]; } break; } // continued ... } } For More Information To learn more about using streams for networking, read Networking Overview. Setting Up Socket Streams For More Information 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 25This table describes the changes to Stream Programming Guide . Date Notes 2012-09-19 Clarified behavior of CFStreamCreatePairWithSocketToHost. 2009-12-16 Updated code listings in the Setting Up Socket Streams chapter. 2009-08-28 Added links to related concepts. 2009-05-06 Added a missing comment to a code sample. 2008-10-15 Fixed broken links. 2006-10-03 Fixed a broken link. Changed event in code listing on writing to a network stream to NSStreamEventHasSpaceAvailable. 2006-04-04 2005-07-07 Fixed bugs and changed title from "Streams." 2004-07-21 Fixed bug in code example (Radar 3597799). 2004-02-20 First version of Streams. 2012-09-19 | © 2004, 2012 Apple Inc. All Rights Reserved. 26 Document Revision HistoryApple Inc. © 2004, 2012 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, Cocoa, Mac, Objective-C, and Spaces are trademarks of Apple Inc., registered in the U.S. and other countries. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. URL Loading System Programming GuideContents Introduction 5 Organization of This Document 5 See Also 6 URL Loading System Overview 7 URL Loading 7 Cache Management 9 Authentication and Credentials 9 Cookie Storage 10 Protocol Support 11 Using NSURLConnection 12 Creating a Connection 12 Controlling Response Caching 15 Estimating Upload Progress 16 Downloading Data Synchronously 17 Using NSURLDownload 18 Downloading to a Predetermined Destination 18 Downloading a File Using the Suggested Filename 20 Displaying Download Progress 22 Resuming Downloads 24 Decoding Encoded Files 24 Handling Redirects and Other Request Changes 26 Authentication Challenges 28 Deciding How to Respond to an Authentication Challenge 28 Responding to an Authentication Challenge 29 Providing Credentials 29 Continuing Without Credentials 30 Canceling the Connection 30 Understanding Cache Access 32 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 2Using the Cache for a Request 32 Cache Use Semantics for the http Protocol 33 Document Revision History 34 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 3Figures and Listings URL Loading System Overview 7 Figure 1 The URL loading system class hierarchy 7 Using NSURLConnection 12 Listing 1 Creating a connection using NSURLConnection 12 Listing 2 Example connection:didReceiveResponse: implementation 13 Listing 3 Example connection:didReceiveData: implementation 14 Listing 4 Example connectionDidFailWithError: implementation 14 Listing 5 Example connectionDidFinishLoading: implementation 15 Listing 6 Example connection:withCacheResponse: implementation 16 Using NSURLDownload 18 Listing 1 Using NSURLDownload with a predetermined destination file location 18 Listing 2 Using NSURLDownload with a filename derived from the download 20 Listing 3 Logging the finalized filename using download:didCreateDestination: 22 Listing 4 Displaying the download progress 22 Listing 5 Example implementation of download:shouldDecodeSourceDataOfMIMEType: method. 24 Handling Redirects and Other Request Changes 26 Listing 1 Example of an implementation of connection:willSendRequest:redirectResponse: 26 Authentication Challenges 28 Listing 1 An example of using the connection:didReceiveAuthenticationChallenge: delegate method 30 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 4This guide describes the Foundation framework classes available for interacting with URLs and communicating with servers using standard Internet protocols. Together these classes are referred to asthe URL loading system. The NSURL class provides the ability to manipulate URLs and the resources they refer to. The Foundation framework also provides a rich collection of classes that include support for URL loading, cookie storage, response caching, credentialstorage and authentication, and writing custom protocol extensions. The URL loading system provides support for accessing resources using the following protocols: ● File Transfer Protocol (ftp://) ● Hypertext Transfer Protocol (http://) ● Secure 128-bit Hypertext Transfer Protocol (https://) ● Local file URLs (file:///) It also transparently supports both proxy servers and SOCKS gateways using the user’s system preferences. Organization of This Document This guide includes the following articles: ● “URL Loading System Overview” (page 7) describes the classes of the URL loading system and their interaction. ● “Using NSURLConnection” (page 12) describes using NSURLConnection for asynchronous connections. ● “Using NSURLDownload” (page 18) describes using NSURLDownload to download files asynchronously to disk. ● “Handling Redirects and Other Request Changes” (page 26) describesthe options you have for responding to a change to your URL request. ● “Authentication Challenges” (page 28) describes the process for authenticating your connection against a secure server. ● “Understanding Cache Access” (page 32) describes how a connection uses the cache during a request. 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 5 IntroductionSee Also The following sample code is available through Apple Developer Connection: ● SpecialPictureProtocol implements a custom NSURLProtocol that creates jpeg images in memory as data is downloaded. ● AutoUpdater demonstrates how to check for, and download, an application update using NSURLConnection and NSURLDownload. Introduction See Also 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 6The URL loading system is a set of classes and protocols that provide the underlying capability for an application to access the data specified by a URL. These classes fall into five categories: URL loading, cache management, authentication and credentials, cookie storage, and protocol support. Figure 1 The URL loading system class hierarchy NSObject URL Loading NSURLConnection NSURLRequest NSMutableURLRequest NSURLResponse NSHTTPURLResponse NSURLDownload Cache Management NSCacheURLRequest NSURLCache Cookie Storage NSHTTPCookie NSHTTPCookieStorage Protocol Support NSURLProtocolClient NSURLProtocol Authentication and Credentials NSURLCredential NSURLCredentialStorage NSURLProtectionSpace NSURLAuthenticationChallenge NSURLAuthenticationChallengeSender URL Loading The most commonly used classes in the URL loading system allow an application to create a request for the content of a URL and download it from the source. 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 7 URL Loading System OverviewA request for the contents of a URL is represented by an NSURLRequest object. The NSURLRequest class encapsulates a URL and any protocol-specific properties, in a protocol-independent manner. It also provides an interface to set the timeout for a connection and specifiesthe policy regarding the use of any locally cached data. The NSMutableURLRequest classis a mutable subclass of NSURLRequest that allows a client application to alter an existing request. Note: When a client application initiates a connection or download using an instance of NSMutableURLRequest, a deep copy is made of the request. Changes made to the initiating request have no effect once a download has been initialized. Protocols,such as HTTP, thatsupport protocol-specific properties must create categories on the NSURLRequest and NSMutableURLRequest classesto provide accessorsfor those properties. As an example, the HTTP protocol adds methods to NSURLRequest to return the HTTP request body, headers, and transfer method. It also adds methodsto NSMutableURLRequest to set the corresponding values. Methodsforsetting and getting property values in those accessors are exposed in the NSURLProtocol class. The response from a server to a request can be viewed as two parts: metadata describing the contents and the URL content data. The metadata that is common to most protocolsis encapsulated by the NSURLResponse class and consists of the MIME type, expected content length, text encoding (where applicable), and the URL that provided the response. Protocols can create subclasses of NSURLResponse to store protocol-specific metadata. NSHTTPURLResponse, for example, stores the headers and the status code returned by the web server. Note: It’s important to remember that only the metadata for the response is stored in an NSURLResponse object. An NSCachedURLResponse instance is used to encapsulate an NSURLResponse, the URL content data, and any application-provided information. See “Cache Management” (page 9) for details. The NSURLConnection and NSURLDownload classes provide the interface to make a connection specified by an NSURLRequest object and download the contents. An NSURLConnection object provides data to the delegate as it is received from the originating source, whereas an NSURLDownload object writes the request data directly to disk. Both classes provide extensive delegate support for responding to redirects, authentication challenges, and error conditions. The NSURLConnection class provides a delegate method that allows an application to control the caching of a response on a per-request basis. Downloads initiated by an NSURLDownload instance are not cached. URL Loading System Overview URL Loading 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 8Cache Management The URL loading system provides a composite on-disk and in-memory cache allowing an application to reduce its dependence on a network connection and provide faster turnaround for previously cached responses. The cache is stored on a per-application basis. The cache is queried by NSURLConnection according to the cache policy specified by the initiating NSURLRequest. The NSURLCache class provides methods to configure the cache size and its location on disk. It also provides methods to manage the collection of NSCachedURLResponse objects that contain the cached responses. An NSCachedURLResponse object encapsulates the NSURLResponse and the URL content data. NSCachedURLResponse also provides a user info dictionary that can be used by an application to cache any custom data. Not all protocol implementations support response caching. Currently only http and https requests are cached, and https requests are never cached to disk. An NSURLConnection can control whether a response is cached and whether the response should be cached only in memory by implementing the connection:willCacheResponse: delegate method. Authentication and Credentials Some servers restrict access to certain content, requiring a user to authenticate with a valid user name and password in order to gain access. In the case of a web server, restricted content is grouped together into a realm that requires a single set of credentials. The URL loading system provides classesthat model credentials and protected areas as well as providing secure credential persistence. Credentials can be specified to persist for a single request, for the duration of an application’s launch, or permanently in the user’s keychain. Note: Credentials stored in persistent storage are kept in the user's keychain and shared among all applications. The NSURLCredential class encapsulates a credential consisting of the user name, password, and the type of persistence to use. The NSURLProtectionSpace classrepresents an area that requires a specific credential. A protection space can be limited to a single URL, encompass a realm on a web server, or refer to a proxy. URL Loading System Overview Cache Management 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 9A shared instance of the NSURLCredentialStorage class manages credential storage and provides the mapping of an NSURLCredential object to the corresponding NSURLProtectionSpace object for which it provides authentication. The NSURLAuthenticationChallenge class encapsulates the information required by an NSURLProtocol implementation to authenticate a request: a proposed credential, the protection space involved, the error or response that the protocol used to determine that authentication isrequired, and the number of authentication attemptsthat have been made. An NSURLAuthenticationChallenge instance also specifiesthe object that initiated the authentication. The initiating object, referred to as the sender, must conform to the NSURLAuthenticationChallengeSender protocol. NSURLAuthenticationChallenge instances are used by NSURLProtocol subclasses to inform the URL loading system that authentication is required. They are also provided to the delegate methods of NSURLConnection and NSURLDownload that facilitate customized authentication handling. Cookie Storage Due to the stateless nature of the HTTP protocol, cookies are often used to provide persistent storage of data across URL requests. The URL loading system provides interfaces to create and manage cookies as well as sending and receiving cookies from web servers. The NSHTTPCookie class encapsulates a cookie, providing accessors for many of the common cookie attributes. It also provides methods to convert HTTP cookie headers to NSHTTPCookie instances and convert an NSHTTPCookie instance to headers suitable for use with an NSURLRequest. The URL loading system automatically sends any stored cookies appropriate for an NSURLRequest. unless the request specifies not to send cookies. Likewise, cookies returned in an NSURLResponse are accepted in accordance with the current cookie acceptance policy. The NSHTTPCookieStorage class provides the interface for managing the collection of NSHTTPCookie objects shared by all applications. iOS Note: Cookies are not shared by applications in iOS. NSHTTPCookieStorage allows an application to specify a cookie acceptance policy. The cookie acceptance policy controls whether cookies should always be accepted, never be accepted, or accepted only from the same domain as the main document URL. URL Loading System Overview Cookie Storage 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 10Note: Changing the cookie acceptance policy in an application affectsthe cookie acceptance policy for all other running applications. When another application changesthe cookie storage or the cookie acceptance policy, NSHTTPCookieStorage notifies an application by posting the NSHTTPCookieStorageCookiesChangedNotification and NSHTTPCookieStorageAcceptPolicyChangedNotification notifications. Protocol Support The URL loading system design allows a client application to extend the protocols that are supported for transferring data. The URL loading system natively supports http, https, file, and ftp protocols. Custom protocols are implemented by subclassing NSURLProtocol and then registering the new class with the URL loading system using the NSURLProtocol class method registerClass:. When an NSURLConnection or NSURLDownload object initiates a connection for an NSURLRequest, the URL loading system consults each of the registered classes in the reverse order of their registration. The first class that returns YES for a canInitWithRequest: message is used to handle the request. The URL loading system is responsible for creating and releasing NSURLProtocol instances when connections start and complete. An application should never create an instance of NSURLProtocol directly. When an NSURLProtocol subclass is initialized by the URL loading system, it is provided a client object that conforms to the NSURLProtocolClient protocol. The NSURLProtocol subclass sends messages from the NSURLProtocolClient protocol to the client object to inform the URL loading system of its actions as it creates a response, receives data, redirectsto a new URL, requires authentication, and completesthe load. If the custom protocolsupports authentication, then it must conform to the NSURLAuthenticationChallengeSender protocol. URL Loading System Overview Protocol Support 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 11NSURLConnection provides the most flexible method of downloading the contents of a URL. It provides a simple interface for creating and canceling a connection, and supports a collection of delegate methods that provide feedback and control of many aspects of the connection. These classes fall into five categories: URL loading, cache management, authentication and credentials, cookie storage, and protocol support. Creating a Connection In order to download the contents of a URL, an application needsto provide a delegate object that, at a minimum, implements the following delegate methods: connection:didReceiveResponse:, connection:didReceiveData:, connection:didFailWithError: and connectionDidFinishLoading:. The example in Listing 1 initiates a connection for a URL. It begins by creating an NSURLRequest instance for the URL, specifying the cache access policy and the timeout interval for the connection. It then creates an NSURLConnection instance,specifying the request and a delegate. If NSURLConnection can’t create a connection for the request, initWithRequest:delegate: returns nil. If the connection is successful, an instance of NSMutableData is created to store the data that is provided to the delegate incrementally. Listing 1 Creating a connection using NSURLConnection // Create the request. NSURLRequest *theRequest=[NSURLRequest requestWithURL:[NSURL URLWithString:@"http://www.apple.com/"] cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:60.0]; // create the connection with the request // and start loading the data NSURLConnection *theConnection=[[NSURLConnection alloc] initWithRequest:theRequest delegate:self]; if (theConnection) { // Create the NSMutableData to hold the received data. // receivedData is an instance variable declared elsewhere. 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 12 Using NSURLConnectionreceivedData = [[NSMutableData data] retain]; } else { // Inform the user that the connection failed. } The download starts immediately upon receiving the initWithRequest:delegate: message. It can be canceled any time before the delegate receives a connectionDidFinishLoading: or connection:didFailWithError: message by sending the connection a cancel message. When the server has provided sufficient data to create an NSURLResponse object, the delegate receives a connection:didReceiveResponse: message. The delegate method can examine the provided NSURLResponse and determine the expected content length of the data, MIME type, suggested filename and other metadata provided by the server. You should be prepared for your delegate to receive the connection:didReceiveResponse: message multiple times for a single connection. This message can be sent due to server redirects, or in rare cases multi-part MIME documents. Each time the delegate receives the connection:didReceiveResponse: message, it should reset any progress indication and discard all previously received data. The example implementation in Listing 2 simply resets the length of the received data to 0 each time it is called. Listing 2 Example connection:didReceiveResponse: implementation - (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response { // This method is called when the server has determined that it // has enough information to create the NSURLResponse. // It can be called multiple times, for example in the case of a // redirect, so each time we reset the data. // receivedData is an instance variable declared elsewhere. [receivedData setLength:0]; } The delegate is periodically sent connection:didReceiveData: messages as the data is received. The delegate implementation is responsible for storing the newly received data. In the example implementation in Listing 3, the new data is appended to the NSMutableData object created in Listing 1. Using NSURLConnection Creating a Connection 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 13Listing 3 Example connection:didReceiveData: implementation - (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data { // Append the new data to receivedData. // receivedData is an instance variable declared elsewhere. [receivedData appendData:data]; } You can also use the connection:didReceiveData: method to provide an indication of the connection’s progress to the user. If an error is encountered during the download, the delegate receives a connection:didFailWithError: message. The NSError object passed as the parameter specifies the details of the error. It also provides the URL of the request that failed in the user info dictionary using the key NSURLErrorFailingURLStringErrorKey. After the delegate receives a message connection:didFailWithError:, it receives no further delegate messages for the specified connection. The example in Listing 4 releases the connection, as well as any received data, and logs the error. Listing 4 Example connectionDidFailWithError: implementation - (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error { // release the connection, and the data object [connection release]; // receivedData is declared as a method instance elsewhere [receivedData release]; // inform the user NSLog(@"Connection failed! Error - %@ %@", [error localizedDescription], [[error userInfo] objectForKey:NSURLErrorFailingURLStringErrorKey]); } Using NSURLConnection Creating a Connection 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 14Finally, if the connection succeeds in downloading the request, the delegate receives the connectionDidFinishLoading: message. The delegate will receive no further messagesfor the connection and the NSURLConnection object can be released. The example implementation in Listing 5 logsthe length of the received data and releases both the connection object and the received data. Listing 5 Example connectionDidFinishLoading: implementation - (void)connectionDidFinishLoading:(NSURLConnection *)connection { // do something with the data // receivedData is declared as a method instance elsewhere NSLog(@"Succeeded! Received %d bytes of data",[receivedData length]); // release the connection, and the data object [connection release]; [receivedData release]; } This represents the simplest implementation of a client using NSURLConnection. Additional delegate methods provide the ability to customize the handling of server redirects, authorization requests and caching of the response. Controlling Response Caching By default the data for a connection is cached according to the support provided by the NSURLProtocolsubclass that handles the request. An NSURLConnection delegate can further refine that behavior by implementing connection:willCacheResponse:. This delegate method can examine the provided NSCachedURLResponse object and change how the response is cached, for example restricting its storage to memory only or preventing it from being cached altogether. It is also possible to insert objects in an NSCachedURLResponse’s user info dictionary, causing them to be stored in the cache as part of the response. Using NSURLConnection Controlling Response Caching 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 15Note: The delegate receives connection:willCacheResponse: messages only for protocols that support caching. The example in Listing 6 prevents the caching of https responses. It also adds the current date to the user info dictionary for responses that are cached. Listing 6 Example connection:withCacheResponse: implementation -(NSCachedURLResponse *)connection:(NSURLConnection *)connection willCacheResponse:(NSCachedURLResponse *)cachedResponse { NSCachedURLResponse *newCachedResponse = cachedResponse; if ([[[[cachedResponse response] URL] scheme] isEqual:@"https"]) { newCachedResponse = nil; } else { NSDictionary *newUserInfo; newUserInfo = [NSDictionary dictionaryWithObject:[NSCalendarDate date] forKey:@"Cached Date"]; newCachedResponse = [[[NSCachedURLResponse alloc] initWithResponse:[cachedResponse response] data:[cachedResponse data] userInfo:newUserInfo storagePolicy:[cachedResponse storagePolicy]] autorelease]; } return newCachedResponse; } Estimating Upload Progress You can estimate the progress of an HTTP POST upload with the connection:didSendBodyData:totalBytesWritten:totalBytesExpectedToWrite: delegatemethod. Note that this is not an exact measurement of upload progress, because the connection may fail or the connection may encounter an authentication challenge. Using NSURLConnection Estimating Upload Progress 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 16Downloading Data Synchronously NSURLConnection provides support for downloading the contents of an NSURLRequest in a synchronous manner using the class method sendSynchronousRequest:returningResponse:error:. Using this method is not recommended, because it has severe limitations: ● The client application blocks until the data has been completely received, an error is encountered, or the request times out. ● Minimal support is provided for requests that require authentication. ● There is no means of modifying the default behavior of response caching or accepting server redirects. If the download succeeds, the contents of the request are returned as an NSData object and an NSURLResponse for the request is returned by reference. If NSURLConnection is unable to download the URL, the method returns nil and any available NSError instance by reference in the appropriate parameter. If the request requires authentication in order to make the connection, valid credentials must already be available in the NSURLCredentialStorage, or must be provided as part of the requested URL. If the credentials are not available or fail to authenticate, the URL loading system responds by sending the NSURLProtocol subclass handling the connection a continueWithoutCredentialForAuthenticationChallenge: message. When a synchronous connection attempt encounters a server redirect, the redirect is always honored. Likewise the response data is stored in the cache according to the default support provided by the protocol implementation. Using NSURLConnection Downloading Data Synchronously 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 17NSURLDownload provides an application the ability to download the contents of a URL directly to disk. It provides an interface similar to NSURLConnection, adding an additional method forspecifying the destination of the file. NSURLDownload can also decode commonly used encoding schemes such as MacBinary, BinHex and gzip. Unlike NSURLConnection, data downloaded using NSURLDownload is notstored in the cache system. If your application is not restricted to using Foundation classes, the WebKit framework includes WebDownload, a subclass of NSURLDownload that provides a user interface for authentication. iOS Note: The NSURLDownload class is not available in iOS, because downloading directly to the file system is discouraged. Use the NSURLConnection class instead. See “Using NSURLConnection” (page 12) for more information. Downloading to a Predetermined Destination One usage pattern for NSURLDownload is downloading a file to a predetermined filename on disk. If the application knows the destination of the download, it can set it explicitly using setDestination:allowOverwrite:. Multiple setDestination:allowOverwrite: messages to an NSURLDownload instance are ignored. The download starts immediately upon receiving the initWithRequest:delegate: message. It can be canceled any time before the delegate receives a downloadDidFinish: or download:didFailWithError: message by sending the download a cancel message. The example in Listing 1 sets the destination, and thus requires the delegate only implement the download:didFailWithError: and downloadDidFinish: methods. Listing 1 Using NSURLDownload with a predetermined destination file location - (void)startDownloadingURL:sender { // Create the request. NSURLRequest *theRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:@"http://www.apple.com"] 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 18 Using NSURLDownloadcachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:60.0]; // Create the connection with the request and start loading the data. NSURLDownload *theDownload = [[NSURLDownload alloc] initWithRequest:theRequest delegate:self]; if (theDownload) { // Set the destination file. [theDownload setDestination:@"/tmp" allowOverwrite:YES]; } else { // inform the user that the download failed. } } - (void)download:(NSURLDownload *)download didFailWithError:(NSError *)error { // Release the connection. [download release]; // Inform the user. NSLog(@"Download failed! Error - %@ %@", [error localizedDescription], [[error userInfo] objectForKey:NSURLErrorFailingURLStringErrorKey]); } - (void)downloadDidFinish:(NSURLDownload *)download { // Release the connection. [download release]; // Do something with the data. NSLog(@"%@",@"downloadDidFinish"); } Using NSURLDownload Downloading to a Predetermined Destination 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 19Additional methods can be implemented by the delegate to customize the handling of authentication, server redirects and file decoding. Downloading a File Using the Suggested Filename Sometimesthe application must derive the destination filename from the downloaded data itself. Thisrequires you to implement the delegate method download:decideDestinationWithSuggestedFilename: and call setDestination:allowOverwrite: with the suggested filename. The example in Listing 2 saves the downloaded file to the desktop using the suggested filename. Listing 2 Using NSURLDownload with a filename derived from the download - (void)startDownloadingURL:sender { // Create the request. NSURLRequest *theRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:@"http://www.apple.com/index.html"] cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:60.0]; // Create the download with the request and start loading the data. NSURLDownload *theDownload = [[NSURLDownload alloc] initWithRequest:theRequest delegate:self]; if (!theDownload) { // Inform the user that the download failed. } } - (void)download:(NSURLDownload *)download decideDestinationWithSuggestedFilename:(NSString *)filename { NSString *destinationFilename; NSString *homeDirectory = NSHomeDirectory(); destinationFilename = [[homeDirectory stringByAppendingPathComponent:@"Desktop"] stringByAppendingPathComponent:filename]; Using NSURLDownload Downloading a File Using the Suggested Filename 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 20[download setDestination:destinationFilename allowOverwrite:NO]; } - (void)download:(NSURLDownload *)download didFailWithError:(NSError *)error { // Release the download. [download release]; // Inform the user. NSLog(@"Download failed! Error - %@ %@", [error localizedDescription], [[error userInfo] objectForKey:NSURLErrorFailingURLStringErrorKey]); } - (void)downloadDidFinish:(NSURLDownload *)download { // Release the download. [download release]; // Do something with the data. NSLog(@"%@",@"downloadDidFinish"); } The downloaded file is stored on the user's desktop with the name index.html, which was derived from the downloaded content. Passing NO to setDestination:allowOverwrite: prevents an existing file from being overwritten by the download. Instead a unique filename is created by inserting a sequential number after the filename, for example, index-1.html. The delegate is informed when a file is created on disk if it implements the download:didCreateDestination: method. This method also gives the application the opportunity to determine the finalized filename with which the download is saved. The example in Listing 3 logs the finalized filename. Using NSURLDownload Downloading a File Using the Suggested Filename 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 21Listing 3 Logging the finalized filename using download:didCreateDestination: -(void)download:(NSURLDownload *)download didCreateDestination:(NSString *)path { // path now contains the destination path // of the download, taking into account any // unique naming caused by -setDestination:allowOverwrite: NSLog(@"Final file destination: %@",path); } This message is sent to the delegate after it has been given an opportunity to respond to the download:shouldDecodeSourceDataOfMIMEType: and download:decideDestinationWithSuggestedFilename: messages. Displaying Download Progress The progress of the download can be determined by implementing the delegate methods download:didReceiveResponse: and download:didReceiveDataOfLength:. The download:didReceiveResponse: method provides the delegate an opportunity to determine the expected content length from the NSURLResponse. The delegate should reset the progress each time this message is received. The example implementation in Listing 4 demonstrates using these methods to provide progress feedback to the user. Listing 4 Displaying the download progress - (void)setDownloadResponse:(NSURLResponse *)aDownloadResponse { [aDownloadResponse retain]; // downloadResponse is an instance variable defined elsewhere. [downloadResponse release]; downloadResponse = aDownloadResponse; } Using NSURLDownload Displaying Download Progress 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 22- (void)download:(NSURLDownload *)download didReceiveResponse:(NSURLResponse *)response { // Reset the progress, this might be called multiple times. // bytesReceived is an instance variable defined elsewhere. bytesReceived = 0; // Retain the response to use later. [self setDownloadResponse:response]; } - (void)download:(NSURLDownload *)download didReceiveDataOfLength:(unsigned)length { long long expectedLength = [[self downloadResponse] expectedContentLength]; bytesReceived = bytesReceived + length; if (expectedLength != NSURLResponseUnknownLength) { // If the expected content length is // available, display percent complete. float percentComplete = (bytesReceived/(float)expectedLength)*100.0; NSLog(@"Percent complete - %f",percentComplete); } else { // If the expected content length is // unknown, just log the progress. NSLog(@"Bytes received - %d",bytesReceived); } } The delegate receives a download:didReceiveResponse: message before it begins receiving download:didReceiveDataOfLength: messages. Using NSURLDownload Displaying Download Progress 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 23Resuming Downloads In some cases, you can resume a download that was canceled or that failed while in progress. To do so, first make sure your original download doesn’t delete its data upon failure by passing NO to the download’s setDeletesFileUponFailure: method. If the original download fails, you can obtain its data with the resumeData method. You can then initialize a new download with the initWithResumeData:delegate:path: method. When the download resumes, the download’s delegate receives the download:willResumeWithResponse:fromByte: message. You can resume a download only if both the protocol of the connection and the MIME type of the file being downloaded support resuming. You can determine whether your file’s MIME type is supported with the canResumeDownloadDecodedWithEncodingMIMEType: method. Decoding Encoded Files NSURLDownload provides support for decoding selected file formats: MacBinary, BinHex and gzip. If NSURLDownload determines that a file is encoded in a supported format, it attempts to send the delegate a download:shouldDecodeSourceDataOfMIMEType: message. If the delegate implements this method, it should examine the passed MIME type and return YES if the file should be decoded. The example in Listing 5 compares the MIME type of the file and allows decoding of MacBinary and BinHex encoded content. Listing 5 Example implementation of download:shouldDecodeSourceDataOfMIMEType: method. - (BOOL)download:(NSURLDownload *)download shouldDecodeSourceDataOfMIMEType:(NSString *)encodingType; { BOOL shouldDecode = NO; if ([encodingType isEqual:@"application/macbinary"]) { shouldDecode = YES; } else if ([encodingType isEqual:@"application/binhex"]) { shouldDecode = YES; } else if ([encodingType isEqual:@"application/gzip"]) { shouldDecode = NO; } return shouldDecode; Using NSURLDownload Resuming Downloads 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 24} Using NSURLDownload Decoding Encoded Files 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 25A server may redirect a request for one URL to another URL. The delegates for NSURLConnection and NSURLDownload can be notified when this occurs for their connection. To handle a redirect for an instance of NSURLConnection, implement the connection:willSendRequest:redirectResponse: delegate method (for NSURLDownload, implement download:willSendRequest:redirectResponse:). If the delegate implements this method, it can examine the new request and the response that caused the redirect, and respond in one of four ways: ● The delegate can allow the redirect by simply returning the provided request. ● The delegate can create a new request, pointing to a different URL, and return that request. ● The delegate can reject the redirect and receive any existing data from the connection by returning nil. ● The delegate can cancel both the redirect and the connection by sending the cancelmessage to the NSURLConnection or NSURLDownload. The delegate also receives the connection:willSendRequest:redirectResponse: message if the NSURLProtocol subclass that handles the request has changed the NSURLRequest in order to standardize its format, for example, changing a request for http://www.apple.com to http://www.apple.com/. This occurs because the standardized, or canonical, version of the request is used for cache management. In this special case, the response passed to the delegate is nil and the delegate should simply return the provided request. The example implementation in Listing 1 allows canonical changes and denies all server redirects. Listing 1 Example of an implementation of connection:willSendRequest:redirectResponse: -(NSURLRequest *)connection:(NSURLConnection *)connection willSendRequest:(NSURLRequest *)request redirectResponse:(NSURLResponse *)redirectResponse { NSURLRequest *newRequest = request; if (redirectResponse) { newRequest = nil; } 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 26 Handling Redirects and Other Request Changesreturn newRequest; } If the delegate doesn't implement connection:willSendRequest:redirectResponse:, all canonical changes and server redirects are allowed. Handling Redirects and Other Request Changes 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 27An NSURLRequest object often encounters an authentication challenge, or a request for credentials from the server it is connecting to. The delegates for NSURLConnection and NSURLDownload can be notified when their request encounters an authentication challenge, so that they can act accordingly. Deciding How to Respond to an Authentication Challenge If an NSURLRequest object requires authentication, the delegate of the NSURLConnection (or NSURLDownload) object associated with the request first receives a connection:canAuthenticateAgainstProtectionSpace: (or download:canAuthenticateAgainstProtectionSpace:) message. This allows the delegate to analyze properties of the server, including its protocol and authentication method, before attempting to authenticate against it. If your delegate is not prepared to authenticate against the server’s protection space, you can return NO, and the system attempts to authenticate with information from the user’s keychain. Note: If your delegate does not implement the connection:canAuthenticateAgainstProtectionSpace: method and the protection space uses client certificate authentication or server trust authentication, the system behaves as if you returned NO. The system behaves as if you returned YES for all other authentication methods. If your delegate returns YES from connection:canAuthenticateAgainstProtectionSpace: or doesn’t implement it, and there are no valid credentials available, either as part of the requested URL or in the shared NSURLCredentialStorage,the delegate receives a connection:didReceiveAuthenticationChallenge: message. In order for the connection to continue, the delegate has three options: ● Provide authentication credentials ● Attempt to continue without credentials ● Cancel the authentication challenge 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 28 Authentication ChallengesTo help determine the correct course of action, the NSURLAuthenticationChallenge instance passed to the method contains information about what triggered the authentication challenge, how many attempts were made for the challenge, any previously attempted credentials, the NSURLProtectionSpace that requires the credentials, and the sender of the challenge. If the authentication challenge has tried to authenticate previously and failed, you can obtain the attempted credentials by calling proposedCredential on the authentication challenge. The delegate can then use these credentials to populate a dialog that it presents to the user. Calling previousFailureCount on the authentication challenge returns the total number of previous authentication attempts, including those from different authentication protocols. The delegate can provide this information to the end user, to determine whether the credentials it supplied previously are failing, or to limit the maximum number of authentication attempts. Responding to an Authentication Challenge The following are the three ways you can respond to the connection:didReceiveAuthenticationChallenge: delegate method. Providing Credentials To attempt to authenticate, the application should create an NSURLCredential object with authentication information of the form expected by the server. You can determine the server’s authentication method by calling authenticationMethod on the protection space of the provided authentication challenge. Some authentication methods supported by NSURLCredential are: HTTP Basic Authentication (NSURLAuthenticationMethodHTTPBasic) The basic authentication method requires a user name and password. Prompt the user for the necessary information and create an NSURLCredential object with credentialWithUser:password:persistence:. HTTP Digest Authentication (NSURLAuthenticationMethodHTTPDigest) Like basic authentication, digest authentication just requires a user name and password (the digest is generated automatically). Prompt the user for the necessary information and create an NSURLCredential object with credentialWithUser:password:persistence:. Client Certificate Authentication (NSURLAuthenticationMethodClientCertificate) Client certificate authentication requires the system identity and all certificates needed to authenticate with the server. Create an NSURLCredential object with credentialWithIdentity:certificates:persistence:. Authentication Challenges Responding to an Authentication Challenge 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 29Server Trust Authentication (NSURLAuthenticationMethodServerTrust) Server trust authentication requires a trust provided by the protection space of the authentication challenge. Create an NSURLCredential object with credentialForTrust:. After you’ve created the NSURLCredential object, pass it to the authentication challenge’s sender with useCredential:forAuthenticationChallenge:. Continuing Without Credentials If the delegate chooses not to provide a credential for the authentication challenge, it can attempt to continue without one by calling continueWithoutCredentialsForAuthenticationChallenge: on [challenge sender]. Depending on the protocol implementation, continuing without credentials may either cause the connection to fail, resulting in a connectionDidFailWithError: message, or return alternate URL contents that don’t require authentication. Canceling the Connection The delegate may also choose to cancel the authentication challenge by calling cancelAuthenticationChallenge: on [challenge sender]. The delegate receives a connection:didCancelAuthenticationChallenge: message, providing the opportunity to give the user feedback. Para The implementation shown in Listing 1 attempts to authenticate the challenge by creating an NSURLCredential instance with a user name and password supplied by the application’s preferences. If the authentication has failed previously, it cancels the authentication challenge and informs the user. Listing 1 An example of using the connection:didReceiveAuthenticationChallenge: delegate method -(void)connection:(NSURLConnection *)connection didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge { if ([challenge previousFailureCount] == 0) { NSURLCredential *newCredential; newCredential = [NSURLCredential credentialWithUser:[self preferencesName] password:[self preferencesPassword] persistence:NSURLCredentialPersistenceNone]; [[challenge sender] useCredential:newCredential Authentication Challenges Responding to an Authentication Challenge 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 30forAuthenticationChallenge:challenge]; } else { [[challenge sender] cancelAuthenticationChallenge:challenge]; // inform the user that the user name and password // in the preferences are incorrect [self showPreferencesCredentialsAreIncorrectPanel:self]; } } If the delegate doesn’t implement connection:didReceiveAuthenticationChallenge: and the request requires authentication, valid credentials must already be available in the URL credential storage or must be provided as part of the requested URL. If the credentials are not available or if they fail to authenticate, a continueWithoutCredentialForAuthenticationChallenge: message is sent by the underlying implementation. Authentication Challenges Responding to an Authentication Challenge 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 31The URL loading system provides a composite on-disk and in-memory cache of responses to requests. This cache allows an application to reduce its dependency on a network connection and increase its performance. Using the Cache for a Request An NSURLRequest instance specifies how the local cache is used by setting the cache policy to one of the NSURLRequestCachePolicy values: NSURLRequestUseProtocolCachePolicy, NSURLRequestReloadIgnoringCacheData, NSURLRequestReturnCacheDataElseLoad, or NSURLRequestReturnCacheDataDontLoad. The default cache policy for an NSURLRequest instance is NSURLRequestUseProtocolCachePolicy. The NSURLRequestUseProtocolCachePolicy behavior is protocol specific and is defined as being the best conforming policy for the protocol. Setting the cache policy to NSURLRequestReloadIgnoringCacheData causes the URL loading system to load the data from the originating source, ignoring the cache completely. The NSURLRequestReturnCacheDataElseLoad cache policy will cause the URL loading system to use cached data ignoring its age or expiration date, if it exists, and load the data from the originating source only if there is no cached version. The NSURLRequestReturnCacheDataDontLoad policy allows an application to specify that only data in the cache should be returned. Attempting to create an NSURLConnection or NSURLDownload instance with this cache policy returns nil immediately if the response is not in the local cache. This is similar in function to an “offline” mode and never brings up a network connection. 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 32 Understanding Cache AccessNote: Currently, only responsesto http and https requests are cached. The ftp and file protocols attempt to access the originating source as allowed by the cache policy. Custom NSURLProtocol classes can provide caching if they choose. Cache Use Semantics for the http Protocol The most complicated cache use situation is when a request uses the http protocol and has set the cache policy to NSURLRequestUseProtocolCachePolicy. If an NSCachedURLResponse does not exist for the request, then the data isfetched from the originating source. If there is a cached response for the request, the URL loading system checks the response to determine if it specifies that the contents must be revalidated. If the contents must be revalidated a connection is made to the originating source to see if it has changed. If it has not changed, then the response is returned from the local cache. If it has changed, the data is fetched from the originating source. If the cached response doesn’t specify that the contents must be revalidated, the maximum age or expiration specified in the response is examined. If the cached response is recent enough, then the response is returned from the local cache. If the response is determined to be stale, the originating source is checked for newer data. If newer data is available, the data is fetched from the originating source, otherwise it is returned from the cache. RFC 2616, Section 13 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13) specifies the semantics involved in detail. Understanding Cache Access Cache Use Semantics for the http Protocol 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 33This table describes the changes to URL Loading System Programming Guide . Date Notes 2010-09-01 Fixed typos and removed deprecated symbols from code examples. Restructured content and added discussions of new authentication functionality. 2010-03-24 2009-08-14 Added links to Cocoa Core Competencies. 2008-05-20 Updated to include content about NSURLDownload availability in iOS. 2008-05-06 Made minor editorial changes. 2007-07-10 Corrected minor typos. 2006-05-23 Added links to sample code. 2006-03-08 Updated sample code. 2005-09-08 Corrected connectionDidFinishLoading: method signature. 2005-04-08 Added accessor method to sample code. Corrected minor typos. 2004-08-31 Corrected minor typos. Corrected table of contents ordering. Corrected willSendRequest:redirectResponse: method signature throughout topic. 2003-07-03 Added additional article outlining differences in behavior between NSURLDownload and NSURLConnection. 2003-06-11 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 34 Document Revision HistoryDate Notes First release of conceptual and task material covering the usage of new classes in Mac OS X v10.2 with Safari 1.0 for downloading content from the Internet. 2003-06-06 Document Revision History 2010-09-01 | © 2003, 2010 Apple Inc. All Rights Reserved. 35Apple Inc. © 2003, 2010 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, Cocoa, Mac, Mac OS, OS X, and Safari are trademarks of Apple Inc., registered in the U.S. and other countries. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. External Accessory Programming TopicsContents About External Accessories 4 At a Glance 4 Including the External Accessory Framework in Your Project 4 Declaring the Protocols Your App Supports 5 Communicating with an Accessory 5 See Also 5 Connecting to an Accessory 6 Monitoring Accessory-Related Events 9 Document Revision History 10 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 2Listings Connecting to an Accessory 6 Listing 1 Creating a communications session for an accessory 6 Listing 2 Processing stream events 8 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 3The External Accessory framework (ExternalAccessory.framework) provides a conduit for communicating with accessories attached to any iOS-based device. App developers can use this conduit to integrate accessory-level features into their apps. Communicating with an external accessory requires you to work closely with the accessory manufacturer to understand the services provided by that accessory. Manufacturers must build explicit support into their accessory hardware for communicating with iOS. As part of this support, an accessory must support at least one command protocol, which is a custom scheme for sending data back and forth between the accessory and an attached app. Apple does not maintain a registry of protocols; it is up to the manufacturer to decide which protocols to support and whether to use custom protocols or standard protocols supported by other manufacturers. As part of your communication with the accessory manufacturer, you must find out what protocols a given accessory supports. To prevent namespace conflicts, protocol names are specified as reverse-DNS strings of the form com.apple.myProtocol. This allows each manufacturer to define as many protocols as needed to support their line of accessories. Note: If you are interested in becoming a developer of accessories for iPad, iPhone, or iPod touch, you can find information about how to do so on http://developer.apple.com. At a Glance Communicating with accessories requires information about the accessory itself, which you must obtain from the hardware manufacturer. From there, you use the classes of the External Accessory framework to create the bridge between the hardware and your app. Including the External Accessory Framework in Your Project To use the features of the External Accessory framework, you must add ExternalAccessory.framework to your Xcode project and link against it in any relevant targets. To access the classes and headers of the framework, include an #import statement at the top of any relevant source files. 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 4 About External AccessoriesDeclaring the Protocols Your App Supports Appsthat are able to communicate with an external accessory must declare the protocolsthey support in their Info.plist file. Declaring support for specific protocols lets the system know that your app can be launched when that accessory is connected. If no app supports the connected accessory, the system may choose to launch the App Store and point out apps that do. To declare the protocols your app supports, you must include the UISupportedExternalAccessoryProtocols key in your app’s Info.plist file. This key contains an array ofstringsthat identify the communications protocolsthat your app supports. Your app can include any number of protocols in this list and the protocols can be in any order. The system does not use this list to determine which protocol your app should choose; it uses it only to determine if your app is capable of communicating with the accessory. It is up to your code to choose an appropriate communications protocol when it begins talking to the accessory. For more information about the keys you put into your app’s Info.plist file, see Information Property List Key Reference . Communicating with an Accessory An app communicates with an accessory by creating an EASession object for managing the accessory interactions. Session objects work with the underlying system to transfer data packetsto and from the accessory. Data transfer in your app occurs through NSInputStream and NSOutputStream objects, which are vended by the session object once the connection is made. To receive data from the accessory, monitor the input stream using a custom delegate object. To send data to the accessory, write data packets to the output stream. The format of the incoming and outgoing data packetsis determined by the protocol you use to communicate with the accessory. Relevant Article: “Connecting to an Accessory” (page 6), “Monitoring Accessory-Related Events” (page 9) See Also For information about the classes of the External Accessory framework, see External Accessory Framework Reference About External Accessories See Also 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 5Accessories are not visible through the External Accessory framework until they have been connected by the system and made ready for use. When an accessory does become visible, your app can get the appropriate accessory object and open a session using one or more of the protocols supported by the accessory. The shared EAAccessoryManager object provides the main entry point for apps looking to communicate with accessories. This class contains an array of already connected accessory objects that you can enumerate to see if there is one your app supports. Most of the information in an EAAccessory object (such asthe name, manufacturer, and model information) is intended for display purposes only. To determine whether your app can connect to an accessory, you must look at the accessory’s protocols and see if there is one your app supports. Note: It is possible for more than one accessory object to support the same protocol. If that happens, your code is responsible for choosing which accessory object to use. For a given accessory object, only one session at a time is allowed for a specific protocol. The protocolStrings property of each EAAccessory object contains a dictionary whose keys are the supported protocols. If you attempt to create a session using a protocol that is already in use, the External Accessory framework closes the existing session before opening the new one. Listing 1 shows a method that checks the list of connected accessories and grabs the first one that the app supports. It creates a session for the designated protocol and configures the input and output streams of the session. By the time this method returnsthe session object, it is connected to the accessory and ready to begin sending and receiving data. Listing 1 Creating a communications session for an accessory - (EASession *)openSessionForProtocol:(NSString *)protocolString { NSArray *accessories = [[EAAccessoryManager sharedAccessoryManager] connectedAccessories]; EAAccessory *accessory = nil; EASession *session = nil; 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 6 Connecting to an Accessoryfor (EAAccessory *obj in accessories) { if ([[obj protocolStrings] containsObject:protocolString]) { accessory = obj; break; } } if (accessory) { session = [[EASession alloc] initWithAccessory:accessory forProtocol:protocolString]; if (session) { [[session inputStream] setDelegate:self]; [[session inputStream] scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [[session inputStream] open]; [[session outputStream] setDelegate:self]; [[session outputStream] scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; [[session outputStream] open]; [session autorelease]; } } return session; } After the input and output streams are configured, the final step is to process the stream-related data. Listing 2 shows the fundamental structure of a delegate’s stream processing code. This method responds to events from both input and output streams of the accessory. As the accessory sends data to your app an event arrives indicating there are bytes available to be read. Similarly, when the accessory is ready to receive data from your app, events arrive indicating that fact. (Of course, your app does not always have to wait for an event to arrive Connecting to an Accessory 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 7before it can write bytes to the stream. It can also call the stream’s hasBytesAvailable method to see if the accessory is still able to receive data.) For more information on streams and handling stream-related events, see Stream Programming Guide . Listing 2 Processing stream events // Handle communications from the streams. - (void)stream:(NSStream*)theStream handleEvent:(NSStreamEvent)streamEvent { switch (streamEvent) { case NSStreamHasBytesAvailable: // Process the incoming stream data. break; case NSStreamEventHasSpaceAvailable: // Send the next queued command. break; default: break; } } Connecting to an Accessory 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 8The External Accessory framework is capable of sending notifications whenever a hardware accessory is connected or disconnected. Although it is capable, it does not do so automatically. Your app must specifically request that notifications be generated by calling the registerForLocalNotifications method of the EAAccessoryManager class. When an accessory is connected, authenticated, and ready to interact with your app, the framework sends an EAAccessoryDidConnectNotification notification. When an accessory is disconnected, it sends an EAAccessoryDidDisconnectNotification notification. You can register to receive these notifications using the default NSNotificationCenter, and both notifications include information about which accessory was affected. In addition to receiving notificationsthrough the default notification center, an app that is currently interacting with an accessory can assign a delegate to the corresponding EAAccessory object and be notified of changes. Delegate objects must conform to the EAAccessoryDelegateprotocol, which currently contains the optional accessoryDidDisconnect: method. You can use this method to receive disconnection notices without first setting up a notification observer. If your app is suspended in the background when an accessory notification arrives, that notification is put in a queue. When your app begins running again (either in the foreground or background), notifications in the queue are delivered to your app. Notifications are also coalesced and filtered wherever possible to eliminate any irrelevant events. For example, if an accessory was connected and subsequently disconnected while your app was suspended, your app would ultimately not receive any indication that such events took place. For more information about how to register to receive notifications, see Notification Programming Topics. 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 9 Monitoring Accessory-Related EventsThis table describes the changes to External Accessory Programming Topics. Date Notes 2012-02-24 Clarified that protocols must be declared in an app's Info.plist file. Corrected information about what happens when you connect to an existing session. 2011-09-22 2010-05-26 New document describing how to attach to external hardware devices. 2012-02-24 | © 2012 Apple Inc. All Rights Reserved. 10 Document Revision HistoryApple Inc. © 2012 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, iPad, iPhone, iPod, iPod touch, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries. App Store is a service mark of Apple Inc. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. Multimedia Programming GuideContents About Audio and Video 4 Organization of This Document 4 Using Audio 5 The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions 6 iOS Hardware and Software Audio Codecs 7 Audio Sessions 9 Playing Audio 11 Playing Audio Items with iPod Library Access 12 Playing UI Sound Effects or Invoking Vibration Using System Sound Services 12 Playing Sounds Easily with the AVAudioPlayer Class 15 Playing Sounds with Control Using Audio Queue Services 17 Playing Sounds with Positioning Using OpenAL 21 Recording Audio 21 Recording with the AVAudioRecorder Class 22 Recording with Audio Queue Services 24 Parsing Streamed Audio 25 Audio Unit Support in iOS 26 Best Practices for iOS Audio 27 Tips for Using Audio 27 Preferred Audio Formats in iOS 28 Using Video 30 Recording and Editing Video 30 Playing Video Files 31 Document Revision History 34 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 2Figures, Tables, and Listings Using Audio 5 Figure 1-1 Using iPod library access 12 Table 1-1 Audio playback formats and codecs 7 Table 1-2 Audio recording formats and codecs 8 Table 1-3 Features provided by the audio session APIs 9 Table 1-4 Handling audio interruptions 11 Table 1-5 System-supplied audio units 26 Table 1-6 Audio tips 27 Listing 1-1 Creating a sound ID object 13 Listing 1-2 Playing a system sound 14 Listing 1-3 Triggering vibration 14 Listing 1-4 Configuring an AVAudioPlayer object 15 Listing 1-5 Implementing an AVAudioPlayer delegate method 16 Listing 1-6 Controlling an AVAudioPlayer object 17 Listing 1-7 Creating an audio queue object 18 Listing 1-8 Setting the playback level directly 20 Listing 1-9 The AudioQueueLevelMeterState structure 21 Listing 1-10 Setting up the audio session and the sound file URL 22 Listing 1-11 A record/stop method using the AVAudioRecorder class 23 Using Video 30 Figure 2-1 Media player interface with transport controls 31 Listing 2-1 Playing full-screen movies 31 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 3Whether multimedia features are central or incidental to your application, iPhone, iPod touch, and iPad users expect high quality. When presenting video content, take advantage of the device’s high-resolution screen and high frame rates. When designing the audio portion of your application, keep in mind that compelling sound adds immeasurably to a user’s overall experience. You can take advantage of the iOS multimedia frameworks for adding features like: ● High-quality audio recording, playback, and streaming ● Immersive game sounds ● Live voice chat ● Playback of content from a user’s iPod library ● Video playback and recording on supported devices In iOS 4.0 and later, the AV Foundation framework gives you fine-grained control over inspecting, editing, and presenting audio-visual assets. Organization of This Document This document contains the following chapters: ● “Using Audio” (page 5) shows how to use the system’s audio technologies to play and record audio. ● “Using Video” (page 30) shows how to use the system’s video technologies to play and capture video. 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 4 About Audio and VideoImportant: This document contains information that used to be in iOS App Programming Guide . The information in this document has not been updated specifically for iOS 4.0. iOS offers a rich set of toolsfor working with sound in your application. These tools are arranged into frameworks according to the features they provide, as follows: ● Use the Media Player framework to play songs, audio books, or audio podcasts from a user’s iPod library. For details, see Media Player Framework Reference , iPod Library Access Programming Guide , and the AddMusic sample code project. ● Use the AV Foundation framework to play and record audio using a simple Objective-C interface. For details, see AV Foundation Framework Reference and the avTouch sample code project. ● Use the Audio Toolbox framework to play audio with synchronization capabilities, access packets of incoming audio, parse audio streams, convert audio formats, and record audio with access to individual packets. For details, see Audio Toolbox Framework Reference and the SpeakHere sample code project. ● Use the Audio Unit framework to connect to and use audio processing plug-ins. For details, see Audio Unit Hosting Guide for iOS . ● Use the OpenAL framework to provide positional audio playback in games and other applications. iOS supports OpenAL 1.1. For information on OpenAL, see the OpenAL website, OpenAL FAQ for iPhone OS , and the oalTouch sample code project. To allow your code to use the features of an audio framework, add that framework to your Xcode project, link against it in any relevant targets, and add an appropriate #import statement near the top of relevant source files. For example, to provide access to the AV Foundation framework in a source file, add a #import statement near the top of the file. For detailed information on how to add frameworks to your project, see “Files in Projects” in Xcode Project Management Guide . Important: To use the features of the Audio Unit framework, add the Audio Toolbox framework to your Xcode project and link against it in any relevant targets. Then add a #import statement near the top of relevant source files. This section on sound provides a quick introduction to implementing iOS audio features, as listed here: 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 5 Using Audio● To play songs, audio podcasts, and audio books from a user’s iPod library, see “Playing Audio Items with iPod Library Access” (page 12). ● To play and record audio in the fewest lines of code, use the AV Foundation framework. See “Playing Sounds Easily with the AVAudioPlayer Class” (page 15) and “Recording with the AVAudioRecorder Class” (page 22). ● To provide full-featured audio playback including stereo positioning, level control, and simultaneous sounds, use OpenAL. See “Playing Sounds with Positioning Using OpenAL” (page 21). ● To provide lowest latency audio, especially when doing simultaneous input and output (such as for a VoIP application), use the I/O unit or the Voice Processing I/O unit. See “Audio Unit Support in iOS” (page 26). ● To play sounds with the highest degree of control, including support forsynchronization, use Audio Queue Services. See “Playing Sounds with Control Using Audio Queue Services” (page 17). Audio Queue Services also supports recording and provides access to incoming audio packets, as described in “Recording with Audio Queue Services” (page 24). ● To parse audio streamed from a network connection, use Audio File Stream Services. See “Parsing Streamed Audio” (page 25). ● To play user-interface sound effects, or to invoke vibration on devicesthat provide that feature, use System Sound Services. See “Playing UI Sound Effects or Invoking Vibration Using System Sound Services” (page 12). Be sure to read the nextsection,“The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions” (page 6), for critical information on how audio works in iOS. Also read “Best Practices for iOS Audio” (page 27), which offers guidelines and liststhe audio and file formatsto use for best performance and best user experience. When you’re ready to dig deeper, the iOS Dev Center contains guides, reference books, sample code, and more. For tips on how to perform common audio tasks, see Audio & Video Coding How-To's. For in-depth explanations of audio development in iOS, see Core Audio Overview, Audio Session Programming Guide , Audio Queue Services Programming Guide , Audio Unit Hosting Guide for iOS , and iPod Library Access Programming Guide . The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions To get oriented toward iOS audio development, it’s important to understand a few critical things about the hardware and software architecture of iOS devices—described in this section. Using Audio The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 6iOS Hardware and Software Audio Codecs To ensure optimum performance and quality, you need to pick the right audio format and audio codec type. Starting in iOS 3.0, most audio formats can use software-based encoding (for recording) and decoding (for playback). Software codecs support simultaneous playback of multiple sounds, but may entail significant CPU overhead. Hardware-assisted decoding provides excellent performance—but does not support simultaneous playback of multiple sounds. If you need to maximize video frame rate in your application, minimize the CPU impact of your audio playback by using uncompressed audio or the IMA4 format, or use hardware-assisted decoding of your compressed audio assets. For best-practice advice on picking an audio format, see “Preferred Audio Formats in iOS” (page 28). Table 1-1 describes the playback audio codecs available on iOS devices. Table 1-1 Audio playback formats and codecs Hardware-assisted Software-based decoding decoding Audio decoder/playback format AAC (MPEG-4 Advanced Audio Coding) Yes Yes, starting in iOS 3.0 ALAC (Apple Lossless) Yes Yes, starting in iOS 3.0 HE-AAC (MPEG-4 High Efficiency AAC) Yes - iLBC (internet Low Bitrate Codec, another format - Yes for speech) IMA4 (IMA/ADPCM) - Yes Linear PCM (uncompressed, linear pulse-code - Yes modulation) MP3 (MPEG-1 audio layer 3) Yes Yes, starting in iOS 3.0 µ-law and a-law - Yes When using hardware-assisted decoding, the device can play only a single instance of one of the supported formats at a time. For example, if you are playing a stereo MP3 sound using the hardware codec, a second simultaneous MP3 sound will use software decoding. Similarly, you cannot simultaneously play an AAC and an ALAC sound using hardware. If the iPod application is playing an AAC or MP3 sound in the background, it has claimed the hardware codec; your application then plays AAC, ALAC, and MP3 audio using software decoding. Using Audio The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 7To play multiple sounds with best performance, or to efficiently play sounds while the iPod is playing in the background, use linear PCM (uncompressed) or IMA4 (compressed) audio. To learn how to check at runtime which hardware and software codecs are available on a device, read the discussion for the kAudioFormatProperty_HardwareCodecCapabilities constant in Audio Format Services Reference and read Technical Q&A QA1663, “Determining the availability of the AAC hardware encoder at runtime.” To summarize how iOS supports audio formats for single or multiple playback: ● Linear PCM and IMA4 (IMA/ADPCM) You can play multiple linear PCM or IMA4 sounds simultaneously in iOS without incurring CPU resource problems. The same is true for the iLBC speech-quality format, and for the µ-law and a-law compressed formats. When using compressed formats, check the sound quality to ensure it meets your needs. ● AAC, HE-AAC, MP3, and ALAC (Apple Lossless) Playback for AAC, HE-AAC, MP3, and ALAC sounds can use efficient hardware-assisted decoding on iOS devices, but these codecs all share a single hardware path. The device can play only a single instance of one of these formats at a time using hardware-assisted decoding. The single hardware path for AAC, HE-AAC, MP3, and ALAC playback has implications for “play along” style applications, such as a virtual piano. If the user is playing a song in one of these three formats in the iPod application, then your application—to play along over that audio—will employ software decoding. Table 1-2 describes the recording audio codecs available on iOS devices. Table 1-2 Audio recording formats and codecs Audio encoder/recording format Hardware-assisted encoding Software-based encoding Yes, starting in iOS 4.0 for iPhone 3GS and iPod touch (2nd generation) Yes, starting in iOS 3.1 for iPhone 3GS and iPod touch (2nd generation) Yes, starting in iOS 3.2 for iPad AAC (MPEG-4 Advanced Audio Coding) ALAC (Apple Lossless) - Yes iLBC (internet Low Bitrate Codec, for - Yes speech) IMA4 (IMA/ADPCM) - Yes Linear PCM (uncompressed, linear - Yes pulse-code modulation) Using Audio The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 8Audio encoder/recording format Hardware-assisted encoding Software-based encoding µ-law and a-law - Yes Audio Sessions The iOS audio session APIs let you define your application’s general audio behavior and design it to work well within the larger audio context of the device it’s running on. These APIs are described in Audio Session Services Reference and AVAudioSession Class Reference . Using these APIs, you can specify such behaviors as: ● Whether or not your audio should be silenced by the Silent switch (on iPhone, this is called the Ring/Silent switch ) ● Whether or not your audio should stop upon screen lock ● Whether other audio, such as from the iPod, should continue playing or be silenced when your audio starts The audio session APIs also let you respond to user actions, such as the plugging in or unplugging of headsets, and to events that use the device’s sound hardware, such as Clock and Calendar alarms and incoming phone calls. The audio session APIs provide three programmatic features, described in Table 1-3. Table 1-3 Features provided by the audio session APIs Audio session feature Description A category is a key that identifies a set of audio behaviorsfor your application. By setting a category, you indicate your audio intentions to iOS, such as whether your audio should continue when the screen locks. There are six categories, described in “Audio Session Categories”. You can fine-tune the behavior of some categories, as explained in “Fine-Tuning the Category” in Audio Session Programming Guide . Setting categories Your audio session posts messages when your audio is interrupted, when an interruption ends, and when the hardware audio route changes. These messages let you respond gracefully to changes in the larger audio environment—such as an interruption due to an incoming phone call. For details, see “Handling Audio Hardware Route Changes” and “Handling Audio Interruptions”. Handling interruptions and route changes You can query the audio session to discover characteristics of the device your application isrunning on,such as hardware sample rate, number of hardware channels, and whether audio input is available. For details, see “Optimizing for Device Hardware”. Optimizing for hardware characteristics Using Audio The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 9There are two interfaces for working with the audio session: ● A streamlined, objective-C interface that gives you accessto the core audio session features and is described in AVAudioSession Class Reference and AVAudioSessionDelegate Protocol Reference . ● A C-based interface that provides comprehensive access to all basic and advanced audio session features and is described in Audio Session Services Reference . You can mix and match audio session code from AV Foundation and Audio Session Services—the interfaces are completely compatible. An audio session comes with some default behavior that you can use to get started in development. However, except for certain special cases, the default behavior is unsuitable for a shipping application that uses audio. For example, when using the default audio session, audio in your application stops when the Auto-Lock period times out and the screen locks. If you want to ensure that playback continues with the screen locked, include the following lines in your application’s initialization code: NSError *setCategoryErr = nil; NSError *activationErr = nil; [[AVAudioSession sharedInstance] setCategory: AVAudioSessionCategoryPlayback error: &setCategoryErr]; [[AVAudioSession sharedInstance] setActive: YES error: &activationErr]; The AVAudioSessionCategoryPlayback category ensures that playback continues when the screen locks. Activating the audio session puts the specified category into effect. How you handle the interruption caused by an incoming phone call or Clock or Calendar alarm depends on the audio technology you are using, as shown in Table 1-4. Using Audio The Basics: Audio Codecs, Supported Audio Formats, and Audio Sessions 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 10Table 1-4 Handling audio interruptions Audio technology How interruptions work The AVAudioPlayer and AVAudioRecorder classes provide delegate methods for interruption start and end. Implement these methods to update your user interface and optionally, after interruption ends, to resume paused playback. The system automatically pauses playback or recording upon interruption, and reactivates your audio session when you resume playback or recording. If you want to save and restore playback position between application launches,save playback position on interruption as well as on application quit. AV Foundation framework These technologies put your application in control of handling interruptions. You are responsible for saving playback or recording position and reactivating your audio session after interruption ends. Implement the AVAudioSession interruption delegate methods or write an interruption listener callback function. Audio Queue Services, I/O audio unit When using OpenAL for playback, implement the AVAudioSession interruption delegate methods or write an interruption listener callback function—as when using Audio Queue Services. However, the delegate or callback must additionally manage the OpenAL context. OpenAL Sounds played using System Sound Services go silent when an interruption starts. They can automatically be used again if the interruption ends. Applications cannotinfluence the interruption behavior for sounds that use this playback technology. System Sound Services Every iOS application—with rare exception—should actively manage its audio session. For a complete explanation of how to do this,read Audio Session ProgrammingGuide . To ensure that your application conforms to Apple recommendations for audio session behavior, read “Sound” in iOS Human Interface Guidelinesin iOS Human Interface Guidelines. Playing Audio This section introduces you to playing sounds in iOS using iPod library access, System Sound Services, Audio Queue Services, the AV Foundation framework, and OpenAL. Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 11Playing Audio Items with iPod Library Access Starting in iOS 3.0, iPod library accesslets your application play a user’ssongs, audio books, and audio podcasts. The API design makes basic playback very simple while also supporting advanced searching and playback control. As shown in Figure 1-1, your application has two ways to retrieve media items. The media item picker, shown on the left, is an easy-to-use, pre-packaged view controller that behaves like the built-in iPod application’s music selection interface. For many applications, this is sufficient. If the picker doesn’t provide the specialized access control you want, the media query interface will. Itsupports predicate-based specification of itemsfrom the iPod library. Figure 1-1 Using iPod library access iPod Library Music Player Media Picker Media Query Your application As depicted in the figure to the right of your application, you then play the retrieved media items using the music player provided by this API. For a complete explanation of how to add media item playback to your application, see iPod Library Access Programming Guide . For a code example, see the AddMusic sample code project. Playing UI Sound Effects or Invoking Vibration Using System Sound Services To play user-interface sound effects (such as button clicks), or to invoke vibration on devices that support it, use System Sound Services. This compact interface is described in System Sound Services Reference . You can find sample code in the Audio UI Sounds (SysSound) sample in the iOS Dev Center. Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 12Note: Sounds played with System Sound Services are not subject to configuration using your audio session. As a result, you cannot keep the behavior of System Sound Services audio in line with other audio behavior in your application. This is the most important reason to avoid using System Sound Services for any audio apart from its intended uses. The AudioServicesPlaySystemSound function lets you very simply play short sound files. The simplicity carries with it a few restrictions. Your sound files must be: ● No longer than 30 seconds in duration ● In linear PCM or IMA4 (IMA/ADPCM) format ● Packaged in a .caf, .aif, or .wav file In addition, when you use the AudioServicesPlaySystemSound function: ● Sounds play at the current system audio volume, with no programmatic volume control available ● Sounds play immediately ● Looping and stereo positioning are unavailable ● Simultaneous playback is unavailable: You can play only one sound at a time The similar AudioServicesPlayAlertSound function plays a shortsound as an alert. If a user has configured their device to vibrate in Ring Settings, calling this function invokes vibration in addition to playing the sound file. Note: System-supplied alert sounds and system-supplied user-interface sound effects are not available to your application. For example, using the kSystemSoundID_UserPreferredAlert constant as a parameter to the AudioServicesPlayAlertSound function will not play anything. To play a sound with the AudioServicesPlaySystemSound or AudioServicesPlayAlertSound function, first create a sound ID object, as shown in Listing 1-1. Listing 1-1 Creating a sound ID object // Get the main bundle for the app CFBundleRef mainBundle = CFBundleGetMainBundle (); // Get the URL to the sound file to play. The file in this case // is "tap.aif" Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 13soundFileURLRef = CFBundleCopyResourceURL ( mainBundle, CFSTR ("tap"), CFSTR ("aif"), NULL ); // Create a system sound object representing the sound file AudioServicesCreateSystemSoundID ( soundFileURLRef, &soundFileObject ); Then play the sound, as shown in Listing 1-2. Listing 1-2 Playing a system sound - (IBAction) playSystemSound { AudioServicesPlaySystemSound (self.soundFileObject); } In typical use, which includes playing a sound occasionally or repeatedly, retain the sound ID object until your application quits. If you know that you will use a sound only once—for example, in the case of a startup sound—you can destroy the sound ID object immediately after playing the sound, freeing memory. Applicationsrunning on iOS devicesthatsupport vibration can trigger that feature using System Sound Services. You specify the vibrate option with the kSystemSoundID_Vibrate identifier. To trigger it, use the AudioServicesPlaySystemSound function, as shown in Listing 1-3. Listing 1-3 Triggering vibration #import #import - (void) vibratePhone { AudioServicesPlaySystemSound (kSystemSoundID_Vibrate); } Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 14If your application is running on an iPod touch, this code does nothing. Playing Sounds Easily with the AVAudioPlayer Class The AVAudioPlayer class provides a simple Objective-C interface for playing sounds. If your application does not require stereo positioning or precise synchronization, and if you are not playing audio captured from a network stream, Apple recommends that you use this class for playback. Using an audio player you can: ● Play sounds of any duration ● Play sounds from files or memory buffers ● Loop sounds ● Play multiple sounds simultaneously (although not with precise synchronization) ● Control relative playback level for each sound you are playing ● Seek to a particular point in a sound file, which supports application features such as fast forward and rewind ● Obtain audio power data that you can use for audio level metering The AVAudioPlayer class lets you play sound in any audio format available in iOS, as described in Table 1-1 (page 7). For a complete description of this class’s interface, see AVAudioPlayer Class Reference . To configure an audio player: 1. Assign a sound file to the audio player. 2. Prepare the audio player for playback, which acquires the hardware resources it needs. 3. Designate an audio player delegate object, which handlesinterruptions as well asthe playback-completed event. The code in Listing 1-4 illustratesthese steps. It would typically go into an initialization method of the controller class for your application. (In production code, you’d include appropriate error handling.) Listing 1-4 Configuring an AVAudioPlayer object // in the corresponding .h file: // @property (nonatomic, retain) AVAudioPlayer *player; // in the .m file: @synthesize player; // the player object Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 15NSString *soundFilePath = [[NSBundle mainBundle] pathForResource: @"sound" ofType: @"wav"]; NSURL *fileURL = [[NSURL alloc] initFileURLWithPath: soundFilePath]; AVAudioPlayer *newPlayer = [[AVAudioPlayer alloc] initWithContentsOfURL: fileURL error: nil]; [fileURL release]; self.player = newPlayer; [newPlayer release]; [player prepareToPlay]; [player setDelegate: self]; The delegate (which can be your controller object) handles interruptions and updates the user interface when a sound has finished playing. The delegate methods for the AVAudioPlayer class are described in AVAudioPlayerDelegate Protocol Reference . Listing 1-5 shows a simple implementation of one delegatemethod. This code updates the title of a Play/Pause toggle button when a sound has finished playing. Listing 1-5 Implementing an AVAudioPlayer delegate method - (void) audioPlayerDidFinishPlaying: (AVAudioPlayer *) player successfully: (BOOL) completed { if (completed == YES) { [self.button setTitle: @"Play" forState: UIControlStateNormal]; } } To play, pause, or stop an AVAudioPlayer object, call one of its playback control methods. You can test whether or not playback is in progress by using the playing property. Listing 1-6 shows a basic play/pause toggle method that controls playback and updates the title of a UIButton object. Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 16Listing 1-6 Controlling an AVAudioPlayer object - (IBAction) playOrPause: (id) sender { // if already playing, then pause if (self.player.playing) { [self.button setTitle: @"Play" forState: UIControlStateHighlighted]; [self.button setTitle: @"Play" forState: UIControlStateNormal]; [self.player pause]; // if stopped or paused, start playing } else { [self.button setTitle: @"Pause" forState: UIControlStateHighlighted]; [self.button setTitle: @"Pause" forState: UIControlStateNormal]; [self.player play]; } } The AVAudioPlayer class uses the Objective-C declared properties feature for managing information about a sound—such as the playback point within the sound’s timeline, and for accessing playback options—such as volume and looping. For example, you can set the playback volume for an audio player as shown here: [self.player setVolume: 1.0]; // available range is 0.0 through 1.0 For more information on the AVAudioPlayer class, see AVAudioPlayer Class Reference . Playing Sounds with Control Using Audio Queue Services Audio Queue Services adds playback capabilities beyond those available with the AVAudioPlayer class. Using Audio Queue Services for playback lets you: ● Precisely schedule when a sound plays, allowing synchronization ● Precisely control volume on a buffer-by-buffer basis ● Play audio that you have captured from a stream using Audio File Stream Services Audio Queue Services lets you play sound in any audio format available in iOS, as described in Table 1-1 (page 7). You can also use this technology for recording, as explained in “Recording Audio” (page 21). Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 17For detailed information on using this technology, see Audio Queue Services Programming Guide and Audio Queue Services Reference . For sample code, see the SpeakHere sample. Creating an Audio Queue Object To create an audio queue object for playback, perform these three steps: 1. Create a data structure to manage information needed by the audio queue, such as the audio format for the data you want to play. 2. Define a callback function for managing audio queue buffers. The callback uses Audio File Servicesto read the file you want to play. (In iOS 2.1 and later, you can also use Extended Audio File Services to read the file.) 3. Instantiate the playback audio queue using the AudioQueueNewOutput function. Listing 1-7 illustrates these steps using ANSI C. (In production code, you’d include appropriate error handling.) The SpeakHere sample project shows these same steps in the context of a C++ program. Listing 1-7 Creating an audio queue object static const int kNumberBuffers = 3; // Create a data structure to manage information needed by the audio queue struct myAQStruct { AudioFileID mAudioFile; CAStreamBasicDescription mDataFormat; AudioQueueRef mQueue; AudioQueueBufferRef mBuffers[kNumberBuffers]; SInt64 mCurrentPacket; UInt32 mNumPacketsToRead; AudioStreamPacketDescription *mPacketDescs; bool mDone; }; // Define a playback audio queue callback function static void AQTestBufferCallback( void *inUserData, AudioQueueRef inAQ, AudioQueueBufferRef inCompleteAQBuffer ) { Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 18myAQStruct *myInfo = (myAQStruct *)inUserData; if (myInfo->mDone) return; UInt32 numBytes; UInt32 nPackets = myInfo->mNumPacketsToRead; AudioFileReadPackets ( myInfo->mAudioFile, false, &numBytes, myInfo->mPacketDescs, myInfo->mCurrentPacket, &nPackets, inCompleteAQBuffer->mAudioData ); if (nPackets > 0) { inCompleteAQBuffer->mAudioDataByteSize = numBytes; AudioQueueEnqueueBuffer ( inAQ, inCompleteAQBuffer, (myInfo->mPacketDescs ? nPackets : 0), myInfo->mPacketDescs ); myInfo->mCurrentPacket += nPackets; } else { AudioQueueStop ( myInfo->mQueue, false ); myInfo->mDone = true; } } // Instantiate an audio queue object AudioQueueNewOutput ( &myInfo.mDataFormat, AQTestBufferCallback, Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 19&myInfo, CFRunLoopGetCurrent(), kCFRunLoopCommonModes, 0, &myInfo.mQueue ); Controlling the Playback Level Audio queue objects give you two ways to control playback level. To set playback level directly, use the AudioQueueSetParameter function with the kAudioQueueParam_Volume parameter, as shown in Listing 1-8. Level change takes effect immediately. Listing 1-8 Setting the playback level directly Float32 volume = 1; // linear scale, range from 0.0 through 1.0 AudioQueueSetParameter ( myAQstruct.audioQueueObject, kAudioQueueParam_Volume, volume ); You can also set playback level for an audio queue buffer by using the AudioQueueEnqueueBufferWithParameters function. This lets you assign audio queue settings that are, in effect, carried by an audio queue buffer as you enqueue it. Such changes take effect when the buffer begins playing. In both cases, level changes for an audio queue remain in effect until you change them again. Indicating Playback Level You can obtain the current playback level from an audio queue object by: 1. Enabling metering for the audio queue object by setting its kAudioQueueProperty_EnableLevelMetering property to true 2. Querying the audio queue object’s kAudioQueueProperty_CurrentLevelMeter property Using Audio Playing Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 20The value of this property is an array of AudioQueueLevelMeterState structures, one per channel. Listing 1-9 shows this structure: Listing 1-9 The AudioQueueLevelMeterState structure typedef struct AudioQueueLevelMeterState { Float32 mAveragePower; Float32 mPeakPower; }; AudioQueueLevelMeterState; Playing Multiple Sounds Simultaneously To play multiple sounds simultaneously, create one playback audio queue object for each sound. For each audio queue, schedule the first buffer of audio to start at the same time using the AudioQueueEnqueueBufferWithParameters function. Starting in iOS 3.0, nearly all supported audio formats can be used for simultaneous playback—namely, all those that can be played using software decoding, as described in Table 1-1 (page 7). For the most processor-efficient multiple playback, use linear PCM (uncompressed) or IMA4 (compressed) audio. Playing Sounds with Positioning Using OpenAL The open-sourced OpenAL audio API, available in iOS in the OpenAL framework, provides an interface optimized for positioning sounds in a stereo field during playback. Playing, positioning, and moving sounds works just asit does on other platforms. OpenAL also lets you mix sounds. OpenAL usesthe I/O unit for playback, resulting in the lowest latency. For all of these reasons, OpenAL is your best choice for playing sounds in game applications on iOS-based devices. However, OpenAL is also a good choice for general iOS application audio playback needs. OpenAL 1.1 support in iOS is built on top of Core Audio. For more information, see OpenAL FAQ for iPhone OS . For OpenAL documentation, see the OpenAL website at http://openal.org. For sample code, see oalTouch . Recording Audio iOS supports audio recording using the AVAudioRecorder class and Audio Queue Services. These interfaces do the work of connecting to the audio hardware, managing memory, and employing codecs as needed. You can record audio in any of the formats listed in Table 1-2 (page 8). Using Audio Recording Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 21Recording takes place at a system-defined input level in iOS. The system takes input from the audio source that the user has chosen—the built-in microphone or, if connected, the headset microphone or other input source. Recording with the AVAudioRecorder Class The easiest way to record sound in iOS is with the AVAudioRecorder class, described in AVAudioRecorder Class Reference . This class provides a highly-streamlined, Objective-C interface that makes it easy to provide sophisticated features like pausing/resuming recording and handling audio interruptions. At the same time, you retain complete control over recording format. To prepare for recording using an audio recorder: 1. Specify a sound file URL. 2. Set up the audio session. 3. Configure the audio recorder’s initial state. Application launch is a good time to do this part of the setup, as shown in Listing 1-10. Variables such as soundFileURL and recording in this example are declared in the class interface. (In production code, you would include appropriate error handling.) Listing 1-10 Setting up the audio session and the sound file URL - (void) viewDidLoad { [super viewDidLoad]; NSString *tempDir = NSTemporaryDirectory (); NSString *soundFilePath = [tempDir stringByAppendingString: @"sound.caf"]; NSURL *newURL = [[NSURL alloc] initFileURLWithPath: soundFilePath]; self.soundFileURL = newURL; [newURL release]; AVAudioSession *audioSession = [AVAudioSession sharedInstance]; audioSession.delegate = self; [audioSession setActive: YES error: nil]; Using Audio Recording Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 22recording = NO; playing = NO; } To handle interruptions and the completion of recording, add the AVAudioSessionDelegate and AVAudioRecorderDelegate protocol names to the interface declaration for your implementation. If your application also does playback, also adopt the AVAudioPlayerDelegate Protocol Reference protocol. To implement a record method, you can use code such as that shown in Listing 1-11. (In production code, you would include appropriate error handling.) Listing 1-11 A record/stop method using the AVAudioRecorder class - (IBAction) recordOrStop: (id) sender { if (recording) { [soundRecorder stop]; recording = NO; self.soundRecorder = nil; [recordOrStopButton setTitle: @"Record" forState: UIControlStateNormal]; [recordOrStopButton setTitle: @"Record" forState: UIControlStateHighlighted]; [[AVAudioSession sharedInstance] setActive: NO error: nil]; } else { [[AVAudioSession sharedInstance] setCategory: AVAudioSessionCategoryRecord error: nil]; NSDictionary *recordSettings = [[NSDictionary alloc] initWithObjectsAndKeys: Using Audio Recording Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 23[NSNumber numberWithFloat: 44100.0], AVSampleRateKey, [NSNumber numberWithInt: kAudioFormatAppleLossless], AVFormatIDKey, [NSNumber numberWithInt: 1], AVNumberOfChannelsKey, [NSNumber numberWithInt: AVAudioQualityMax], AVEncoderAudioQualityKey, nil]; AVAudioRecorder *newRecorder = [[AVAudioRecorder alloc] initWithURL: soundFileURL settings: recordSettings error: nil]; [recordSettings release]; self.soundRecorder = newRecorder; [newRecorder release]; soundRecorder.delegate = self; [soundRecorder prepareToRecord]; [soundRecorder record]; [recordOrStopButton setTitle: @"Stop" forState: UIControlStateNormal]; [recordOrStopButton setTitle: @"Stop" forState: UIControlStateHighlighted]; recording = YES; } } For more information on the AVAudioRecorder class, see AVAudioRecorder Class Reference . Recording with Audio Queue Services To set up for recording with audio with Audio Queue Services, your application instantiates a recording audio queue object and provides a callback function. The callback storesincoming audio data in memory for immediate use or writes it to a file for long-term storage. Just as with playback, you can obtain the current recording audio level from an audio queue object by querying its kAudioQueueProperty_CurrentLevelMeter property, as described in “Indicating Playback Level” (page 20). Using Audio Recording Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 24For detailed examples of how to use Audio Queue Services to record audio, see “Recording Audio” in Audio Queue Services Programming Guide . For sample code, see the SpeakHere sample. Parsing Streamed Audio To play streamed audio content, such as from a network connection, use Audio File Stream Services in concert with Audio Queue Services. Audio File Stream Services parses audio packets and metadata from common audio file container formats in a network bitstream. You can also use it to parse packets and metadata from on-disk files. In iOS, you can parse the same audio file and bitstream formats that you can in Mac OS X, as follows: ● MPEG-1 Audio Layer 3, used for .mp3 files ● MPEG-2 ADTS, used for the .aac audio data format ● AIFC ● AIFF ● CAF ● MPEG-4, used for .m4a, .mp4, and .3gp files ● NeXT ● WAVE Having retrieved audio packets, you can play back the recovered sound in any of the formats supported in iOS, as listed in Table 1-1 (page 7). For best performance, network streaming applications should use data from Wi-Fi connections. iOS lets you determine which networks are reachable and available through its System Configuration framework and its SCNetworkReachabilityRef opaque type, described in SCNetworkReachability Reference . Forsample code, see the Reachability sample in the iOS Dev Center. To connect to a network stream, use interfaces from Core Foundation, such as the one described in CFHTTPMessage Reference . Parse the network packetsto recover audio packets using Audio File StreamServices. Then buffer the audio packets and send them to a playback audio queue object. Audio File Stream Services relies on interfaces from Audio File Services, such as the AudioFramePacketTranslation structure and the AudioFilePacketTableInfo structure. These are described in Audio File Services Reference . For more information on using streams, refer to Audio File Stream Services Reference . Using Audio Parsing Streamed Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 25Audio Unit Support in iOS iOS provides a set of audio processing plug-ins, known as audio units, that you can use in any application. The interfaces in the Audio Unit framework let you open, connect, and use these audio units. To use the features of the Audio Unit framework, add the Audio Toolbox framework to your Xcode project and link against it in any relevant targets. Then add a #import statement near the top of relevant source files. For detailed information on how to add frameworks to your project, see “Files in Projects” in Xcode Project Management Guide . Table 1-5 lists the audio units provided in iOS. Table 1-5 System-supplied audio units Audio unit Description The iPod EQ unit, of type kAudioUnitSubType_AUiPodEQ, provides a simple, preset-based equalizer you can use in your application. For a demonstration of how to use this audio unit,see the sample code project iPhoneMixerEQGraphTest. iPod Equalizer unit The 3DMixer unit, oftype kAudioUnitSubType_AU3DMixerEmbedded, lets you mix multiple audio streams, specify stereo output panning, manipulate playback rate, and more. OpenAL is built on top of this audio unit and provides a higher-level API well suited for game apps. 3D Mixer unit The Multichannel Mixer unit, of type kAudioUnitSubType_- MultiChannelMixer, lets you mix multiple mono or stereo audio streams to a single stereo stream. It also supports left/right panning for each input. For a demonstration of how to use this audio unit, see the sample code project Audio Mixer (MixerHost). Multichannel Mixer unit The Remote I/Ounit, oftype kAudioUnitSubType_RemoteIO, connects to audio input and output hardware and supports realtime I/O. For demonstrations of how to use this audio unit,see the sample code project aurioTouch . Remote I/O unit The Voice Processing I/O unit, of type kAudioUnitSubType_- VoiceProcessingIO, has the characteristics of the I/O unit and adds echo suppression and other features for two-way communication. Voice Processing I/O unit The Generic Output unit, of type kAudioUnitSubType_- GenericOutput, supports converting to and from linear PCM format; can be used to start and stop a graph. Generic Output unit Using Audio Audio Unit Support in iOS 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 26Audio unit Description The Converter unit, of type kAudioUnitSubType_AUConverter, lets you convert audio data from one format to another. You typically obtain the features of this audio unit by using the Remote I/O unit, which incorporates a Converter unit. Converter unit For more information on using system audio units, see Audio Unit Hosting Guide for iOS . For reference documentation, see Audio Unit Framework Reference and Audio Unit Processing Graph Services Reference . The iOS Dev Center provides two sample-code projects that demonstrate use of system audio units: aurioTouch and iPhoneMultichannelMixerTest. Best Practices for iOS Audio This section lists some important tips for using audio in iOS and describes the best audio data formats for various uses. Tips for Using Audio Table 1-6 lists some important tips to keep in mind when using audio in iOS. Table 1-6 Audio tips Tip Action For AAC, MP3, and ALAC (Apple Lossless) audio, decoding can take place using hardware-assisted codecs. While efficient, this is limited to one audio stream at a time. If you need to play multiple sounds simultaneously, store those sounds using the IMA4 (compressed) or linear PCM (uncompressed) format. Use compressed audio appropriately The afconvert tool in Mac OS X lets you convert to a wide range of audio data formats and file types. See “Preferred Audio Formats in iOS” (page 28) and the afconvert man page. Convert to the data format and file format you need When playing sound with Audio Queue Services, you write a callback that sends short segments of audio data to audio queue buffers. In some cases, loading an entire sound file to memory for playback, which minimizes disk access, is best. In other cases, loading just enough data at a time to keep the buffers full is best. Test and evaluate which strategy works best for your application. Evaluate audiomemory issues Using Audio Best Practices for iOS Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 27Tip Action Sample rate and the number of bits per sample have a direct impact on the size of your audio files. If you need to play many such sounds, or long-duration sounds, consider reducing these valuesto reduce the memory footprint of the audio data. For example, rather than use 44.1 kHz sampling rate for sound effects, you could use a 32 kHz (or possibly lower) sample rate and still provide reasonable quality. Using monophonic (single-channel) audio instead of stereo (two channel) reduces file size. For each sound asset, consider whether mono could suit your needs. Reduce audio file sizes by limiting sample rates, bit depths, and channels Use OpenAL when you want a convenient, high-level interface for positioning sounds in a stereo field or when you need low latency playback. To parse audio packetsfrom a file or a network stream, use Audio File Stream Services. For simple playback of single or multiple sounds, use the AVAudioPlayer class. For recording to a file, use the AVAudioRecorder class. For audio chat, use the Voice Processing I/O unit. To play audio resources synced from a user’s iTunes library, use iPod Library Access. When your sole audio need is to play alerts and user-interface sound effects, use Core Audio’s System Sound Services. For other audio applications, including playback ofstreamed audio, precise synchronization, and access to packets of incoming audio, use Audio Queue Services. Pick the appropriate technology For the lowest possible playback latency, use OpenAL or use the I/O unit directly. Code for low latency Preferred Audio Formats in iOS For uncompressed (highest quality) audio, use 16-bit, little endian, linear PCM audio data packaged in a CAF file. You can convert an audio file to this format in Mac OS X using the afconvert command-line tool, as shown here: /usr/bin/afconvert -f caff -d LEI16 {INPUT} {OUTPUT} The afconvert tool lets you convert to a wide range of audio data formats and file types. See the afconvert man page, and enter afconvert -h at a shell prompt, for more information. For compressed audio when playing one sound at a time, and when you don’t need to play audio simultaneously with the iPod application, use the AAC format packaged in a CAF or m4a file. Using Audio Best Practices for iOS Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 28For less memory usage when you need to play multiple sounds simultaneously, use IMA4 (IMA/ADPCM) compression. This reduces file size but entails minimal CPU impact during decompression. As with linear PCM data, package IMA4 data in a CAF file. Using Audio Best Practices for iOS Audio 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 29Important: This document contains information that used to be in iOS App Programming Guide . The information in this document has not been updated specifically for iOS 4.0. Recording and Editing Video Starting in iOS 3.0, you can record video, with included audio, on supported devices. To display the video recording interface, create and push a UIImagePickerController object, just asfor displaying the still-camera interface. To record video, you must first check that the camera source type (UIImagePickerControllerSourceTypeCamera) is available and that the movie media type (kUTTypeMovie) is available for the camera. Depending on the media types you assign to the mediaTypes property, the picker can directly display the still camera or the video camera, or a selection interface that lets the user choose. Using the UIImagePickerControllerDelegate protocol, register as a delegate of the image picker. Your delegate object receives a completed video recording by way of the imagePickerController:didFinishPickingMediaWithInfo: method. On supported devices, you can also pick previously-recorded videos from a user’s photo library. For more information on using the image picker class, see UIImagePickerController Class Reference . For information on trimming recorded videos, see UIVideoEditorController Class Reference and UIVideoEditorControllerDelegate Protocol Reference . In iOS 4.0 and later, you can record from a device’s camera and display the incoming data live on screen. You use AVCaptureSession to manage data flow from inputs represented by AVCaptureInput objects (which mediate input from an AVCaptureDevice) to outputs represented by AVCaptureOutput. In iOS 4.0 and later, you can edit, assemble, and compose video using existing assets or with new raw materials. Assets are represented by AVAsset, which you can inspect asynchronously for better performance. You use AVMutableComposition to compose media from one or more sources, then AVAssetExportSession to encode output of a composition for delivery. 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 30 Using VideoPlaying Video Files Important: The information in this section currently reflects the usage of the Media Player framework in iOS 3.1 and earlier. Please see the headers for information about changes to this framework in iOS 4.0. iOS supportsthe ability to play back video files directly from your application using the Media Player framework, described in Media Player Framework Reference . Video playback is supported in full screen mode only and can be used by game developers who want to play short animations or by any developers who want to play media files. When you start a video from your application, the media player interface takes over, fading the screen to black and then fading in the video content. You can play a video with or without user controls for adjusting playback. Enabling some or all of these controls (shown in Figure 2-1) gives the user the ability to change the volume, change the playback point, or start and stop the video. If you disable all of these controls, the video plays until completion. Figure 2-1 Media player interface with transport controls To initiate video playback, you must know the URL of the file you want to play. For files your application provides, this would typically be a pointer to a file in your application’s bundle; however, it can also be a pointer to a file on a remote server. Use this URL to instantiate a new instance of the MPMoviePlayerController class. This class presides over the playback of your video file and manages user interactions, such as user taps in the transport controls (if shown). To start playback, call the play method described in MPMediaPlayback Protocol Reference . Listing 2-1 shows a sample method that plays back the video at a specified URL. The play method is an asynchronous call that returns control to the caller while the movie plays. The movie controller loadsthe movie in a full-screen view, and animates the movie into place on top of the application’s existing content. When playback is finished, the movie controller sends a notification received by the application controller object, which releases the movie controller now that it is no longer needed. Listing 2-1 Playing full-screen movies -(void) playMovieAtURL: (NSURL*) theURL { Using Video Playing Video Files 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 31MPMoviePlayerController* theMovie = [[MPMoviePlayerController alloc] initWithContentURL: theURL]; theMovie.scalingMode = MPMovieScalingModeAspectFill; theMovie.movieControlMode = MPMovieControlModeHidden; // Register for the playback finished notification [[NSNotificationCenter defaultCenter] addObserver: self selector: @selector(myMovieFinishedCallback:) name: MPMoviePlayerPlaybackDidFinishNotification object: theMovie]; // Movie playback is asynchronous, so this method returns immediately. [theMovie play]; } // When the movie is done, release the controller. -(void) myMovieFinishedCallback: (NSNotification*) aNotification { MPMoviePlayerController* theMovie = [aNotification object]; [[NSNotificationCenter defaultCenter] removeObserver: self name: MPMoviePlayerPlaybackDidFinishNotification object: theMovie]; // Release the movie instance created in playMovieAtURL: [theMovie release]; } For a list of supported video formats, see iOS Technology Overview. Using Video Playing Video Files 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 32In iOS 4.0 and later, you can play video using AVPlayer in conjunction with an AVPlayerLayer or an AVSynchronizedLayer object. You can use AVAudioMix and AVVideoComposition to customize the audio and video parts of playback respectively. You can also use AVCaptureVideoPreviewLayer to display video as it is being captured by an input device. Using Video Playing Video Files 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 33This table describes the changes to Multimedia Programming Guide . Date Notes Clarified usage of AudioServicesPlaySystemSound function in “Playing UI Sound Effects or Invoking Vibration Using System Sound Services” (page 12). 2010-09-01 Updated Table 1-2 (page 8) for iOS 4.0 by clarifying support for AAC encoding. 2010-05-27 2010-09-01 | © 2010 Apple Inc. All Rights Reserved. 34 Document Revision HistoryApple Inc. © 2010 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple’s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-labeled computers. Apple Inc. 1 Infinite Loop Cupertino, CA 95014 408-996-1010 Apple, the Apple logo, iPad, iPhone, iPod, iPod touch, iTunes, Mac, Mac OS, Objective-C, OS X, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries. NeXT is a trademark of NeXT Software, Inc., registered in the U.S. and other countries. iOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Even though Apple has reviewed this document, APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.ASARESULT, THISDOCUMENT IS PROVIDED “AS IS,” AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. THE WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS, ORAL OR WRITTEN, EXPRESS OR IMPLIED. No Apple dealer, agent, or employee is authorized to make any modification, extension, or addition to this warranty. Some states do not allow the exclusion or limitation of implied warranties or liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you. This warranty gives you specific legal rights, and you may also have other rights which vary from state to state. Concepts in Objective-C ProgrammingContents About the Basic Programming Concepts for Cocoa and Cocoa Touch 7 At a Glance 7 How to Use This Document 7 Prerequisites 8 See Also 8 Class Clusters 9 Without Class Clusters: Simple Concept but Complex Interface 9 With Class Clusters: Simple Concept and Simple Interface 10 Creating Instances 10 Class Clusters with Multiple Public Superclasses 11 Creating Subclasses Within a Class Cluster 12 A True Subclass 12 True Subclasses: An Example 14 A Composite Object 16 A Composite Object: An Example 16 Class Factory Methods 20 Delegates and Data Sources 22 How Delegation Works 22 The Form of Delegation Messages 24 Delegation and the Application Frameworks 25 Becoming the Delegate of a Framework Class 26 Locating Objects Through the delegate Property 27 Data Sources 27 Implementing a Delegate for a Custom Class 27 Introspection 29 Evaluating Inheritance Relationships 29 Method Implementation and Protocol Conformance 30 Object Comparison 31 Object Allocation 34 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 2Object Initialization 35 The Form of Initializers 35 Issues with Initializers 36 Implementing an Initializer 38 Multiple Initializers and the Designated Initializer 40 Model-View-Controller 43 Roles and Relationships of MVC Objects 43 Model Objects Encapsulate Data and Basic Behaviors 43 View Objects Present Information to the User 44 Controller Objects Tie the Model to the View 44 Combining Roles 45 Types of Cocoa Controller Objects 45 MVC as a Compound Design Pattern 47 Design Guidelines for MVC Applications 50 Model-View-Controller in Cocoa (OS X) 52 Object Modeling 53 Entities 53 Attributes 54 Relationships 55 Relationship Cardinality and Ownership 56 Accessing Properties 57 Keys 57 Values 57 Key Paths 58 Object Mutability 60 Why Mutable and Immutable Object Variants? 60 Programming with Mutable Objects 62 Creating and Converting Mutable Objects 62 Storing and Returning Mutable Instance Variables 63 Receiving Mutable Objects 64 Mutable Objects in Collections 66 Outlets 67 Receptionist Pattern 68 The Receptionist Design Pattern in Practice 68 When to Use the Receptionist Pattern 71 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 3 ContentsTarget-Action 73 The Target 73 The Action 74 Target-Action in the AppKit Framework 75 Controls, Cells, and Menu Items 75 Setting the Target and Action 77 Actions Defined by AppKit 78 Target-Action in UIKit 78 Toll-Free Bridging 80 Document Revision History 83 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 4 ContentsFigures, Tables, and Listings Class Clusters 9 Figure 1-1 A simple hierarchy for number classes 9 Figure 1-2 A more complete number class hierarchy 10 Figure 1-3 Class cluster architecture applied to number classes 10 Figure 1-4 An object that embeds a cluster object 16 Table 1-1 Class clusters and their public superclasses 11 Table 1-2 Derived methods and their possible implementations 13 Delegates and Data Sources 22 Figure 3-1 The mechanism of delegation 23 Figure 3-2 A more realistic sequence involving a delegate 23 Listing 3-1 Sample delegation methods with return values 24 Listing 3-2 Sample delegation methods returning void 24 Introspection 29 Listing 4-1 Using the class and superclass methods 29 Listing 4-2 Using isKindOfClass: 30 Listing 4-3 Using respondsToSelector: 31 Listing 4-4 Using conformsToProtocol: 31 Listing 4-5 Using isEqual: 32 Listing 4-6 Overriding isEqual: 32 Object Initialization 35 Figure 6-1 Initialization up the inheritance chain 39 Figure 6-2 Interactions of secondary and designated initializers 41 Model-View-Controller 43 Figure 7-1 Traditional version of MVC as a compound pattern 48 Figure 7-2 Cocoa version of MVC as a compound design pattern 48 Figure 7-3 Coordinating controller as the owner of a nib file 50 Object Modeling 53 Figure 8-1 Employee management application object diagram 54 Figure 8-2 Employees table view 55 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 5Figure 8-3 Relationships in the employee management application 56 Figure 8-4 Relationship cardinality 56 Figure 8-5 Object graph for the employee management application 58 Figure 8-6 Employees table view showing department name 59 Object Mutability 60 Listing 9-1 Returning an immutable copy of a mutable instance variable 63 Listing 9-2 Making a snapshot of a potentially mutable object 65 Receptionist Pattern 68 Figure 11-1 Bouncing KVO updates to the main operation queue 69 Listing 11-1 Declaring the receptionist class 69 Listing 11-2 The class factory method for creating a receptionist object 70 Listing 11-3 Handling the KVO notification 71 Listing 11-4 Creating a receptionist object 71 Target-Action 73 Figure 12-1 How the target-action mechanism works in the control-cell architecture 76 Toll-Free Bridging 80 Table 13-1 Data types that can be used interchangeably between Core Foundation and Foundation 81 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 6 Figures, Tables, and ListingsMany of the programmatic interfaces of the Cocoa and Cocoa Touch frameworks only make sense only if you are aware of the concepts on which they are based. These concepts express the rationale for many of the core designs of the frameworks. Knowledge of these concepts will illuminate your software-development practices. Model layer View layer Controller layer Application delegate System frameworks At a Glance This document contains articles that explain central concepts, design patterns, and mechanisms of the Cocoa and Cocoa Touch frameworks. The articles are arranged in alphabetical order. How to Use This Document If you read this document cover-to-cover, you learn important information about Cocoa and Cocoa Touch application development. However, most readers come to the articles in this document in one of two ways: ● Other documents—especially those that are intended for novice iOS and OS X developers—which link to these articles. ● In-line mini-articles (which appear when you click a dash-underlined word or phrase) that have a link to an article as a “Definitive Discussion.” 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 7 About the Basic Programming Concepts for Cocoa and Cocoa TouchPrerequisites Prior programming experience, especially with object-oriented languages, is recommended. See Also The Objective-C Programming Language offers further discussion of many of the language-related concepts covered in this document. About the Basic Programming Concepts for Cocoa and Cocoa Touch Prerequisites 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 8Class clusters are a design pattern that the Foundation framework makes extensive use of. Class clusters group a number of private concrete subclasses under a public abstract superclass. The grouping of classes in this way simplifies the publicly visible architecture of an object-oriented framework without reducing its functional richness. Class clusters are based on the Abstract Factory design pattern. Without Class Clusters: Simple Concept but Complex Interface To illustrate the class cluster architecture and its benefits, consider the problem of constructing a class hierarchy that defines objects to store numbers of different types (char, int, float, double). Because numbers of different types have many features in common (they can be converted from one type to another and can be represented as strings, for example), they could be represented by a single class. However, their storage requirements differ,so it’sinefficient to represent them all by the same class. Taking thisfact into consideration, one could design the class architecture depicted in Figure 1-1 to solve the problem. Figure 1-1 A simple hierarchy for number classes Number is the abstract superclass that declares in its methods the operations common to its subclasses. However, it doesn’t declare an instance variable to store a number. The subclasses declare such instance variables and share in the programmatic interface declared by Number. 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 9 Class ClustersSo far, this design is relatively simple. However, if the commonly used modifications of these basic C types are taken into account, the class hierarchy diagram looks more like Figure 1-2. Figure 1-2 A more complete number class hierarchy The simple concept—creating a class to hold number values—can easily burgeon to over a dozen classes. The class cluster architecture presents a design that reflects the simplicity of the concept. With Class Clusters: Simple Concept and Simple Interface Applying the class cluster design pattern to this problem yieldsthe class hierarchy in Figure 1-3 (private classes are in gray). Figure 1-3 Class cluster architecture applied to number classes Users of this hierarchy see only one public class, Number, so how is it possible to allocate instances of the proper subclass? The answer is in the way the abstract superclass handles instantiation. Creating Instances The abstract superclass in a class cluster must declare methods for creating instances of its private subclasses. It’s the superclass’s responsibility to dispense an object of the proper subclass based on the creation method that you invoke—you don’t, and can’t, choose the class of the instance. In the Foundation framework, you generally create an object by invoking a +className... method or the alloc... and init... methods. Taking the Foundation framework’s NSNumber class as an example, you could send these messages to create number objects: Class Clusters With Class Clusters: Simple Concept and Simple Interface 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 10NSNumber *aChar = [NSNumber numberWithChar:’a’]; NSNumber *anInt = [NSNumber numberWithInt:1]; NSNumber *aFloat = [NSNumber numberWithFloat:1.0]; NSNumber *aDouble = [NSNumber numberWithDouble:1.0]; You are not responsible for releasing the objects returned from factory methods. Many classes also provide the standard alloc... and init... methodsto create objectsthat require you to manage their deallocation. Each object returned—aChar, anInt, aFloat, and aDouble—may belong to a different private subclass (and in fact does). Although each object’s class membership is hidden, itsinterface is public, being the interface declared by the abstract superclass, NSNumber. Although it is not precisely correct, it’s convenient to consider the aChar, anInt, aFloat, and aDouble objects to be instances of the NSNumber class, because they’re created by NSNumber class methods and accessed through instance methods declared by NSNumber. Class Clusters with Multiple Public Superclasses In the example above, one abstract public class declares the interface for multiple private subclasses. This is a class cluster in the purest sense. It’s also possible, and often desirable, to have two (or possibly more) abstract public classes that declare the interface for the cluster. This is evident in the Foundation framework, which includes the clusters listed in Table 1-1. Table 1-1 Class clusters and their public superclasses Class cluster Public superclasses NSData NSData NSMutableData NSArray NSArray NSMutableArray NSDictionary NSDictionary NSMutableDictionary NSString NSString NSMutableString Class Clusters Class Clusters with Multiple Public Superclasses 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 11Other clusters of this type also exist, but these clearly illustrate how two abstract nodes cooperate in declaring the programmatic interface to a class cluster. In each of these clusters, one public node declares methods that all cluster objects can respond to, and the other node declares methods that are only appropriate for cluster objects that allow their contents to be modified. This factoring of the cluster’s interface helps make an object-oriented framework’s programmatic interface more expressive. For example, imagine an object representing a book that declares this method: - (NSString *)title; The book object could return its own instance variable or create a new string object and return that—it doesn’t matter. It’s clear from this declaration that the returned string can’t be modified. Any attempt to modify the returned object will elicit a compiler warning. Creating Subclasses Within a Class Cluster The class cluster architecture involves a trade-off between simplicity and extensibility: Having a few public classes stand in for a multitude of private ones makes it easier to learn and use the classes in a framework but somewhat harder to create subclasses within any of the clusters. However, if it’s rarely necessary to create a subclass, then the cluster architecture is clearly beneficial. Clusters are used in the Foundation framework in just these situations. If you find that a cluster doesn’t provide the functionality your program needs, then a subclass may be in order. For example, imagine that you want to create an array object whose storage is file-based rather than memory-based, as in the NSArray class cluster. Because you are changing the underlying storage mechanism of the class, you’d have to create a subclass. On the other hand, in some cases it might be sufficient (and easier) to define a class that embeds within it an object from the cluster. Let’s say that your program needs to be alerted whenever some data is modified. In this case, creating a simple class that wraps a data object that the Foundation framework defines may be the best approach. An object of this class could intervene in messages that modify the data, intercepting the messages, acting on them, and then forwarding them to the embedded data object. In summary, if you need to manage your object’sstorage, create a true subclass. Otherwise, create a composite object, one that embeds a standard Foundation framework object in an object of your own design. The following sections give more detail on these two approaches. A True Subclass A new class that you create within a class cluster must: Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 12● Be a subclass of the cluster’s abstract superclass ● Declare its own storage ● Override all initializer methods of the superclass ● Override the superclass’s primitive methods (described below) Because the cluster’s abstract superclass is the only publicly visible node in the cluster’s hierarchy, the first point is obvious. This implies that the new subclass will inherit the cluster’s interface but no instance variables, because the abstract superclass declares none. Thus the second point: The subclass must declare any instance variables it needs. Finally, the subclass must override any method it inherits that directly accesses an object’s instance variables. Such methods are called primitive methods. A class’s primitive methodsform the basisfor itsinterface. For example, take the NSArray class, which declares the interface to objects that manage arrays of objects. In concept, an array stores a number of data items, each of which is accessible by index. NSArray expresses this abstract notion through its two primitive methods, count and objectAtIndex:. With these methods as a base, other methods—derived methods—can be implemented; Table 1-2 gives two examples of derived methods. Table 1-2 Derived methods and their possible implementations Derived Method Possible Implementation Find the last object by sending the array object this message: [self objectAtIndex: ([self count] –1)]. lastObject Find an object by repeatedly sending the array object an objectAtIndex: message, each time incrementing the index until all objects in the array have been tested. containsObject: The division of an interface between primitive and derived methods makes creating subclasses easier. Your subclass must override inherited primitives, but having done so can be sure that all derived methods that it inherits will operate properly. The primitive-derived distinction applies to the interface of a fully initialized object. The question of how init... methods should be handled in a subclass also needs to be addressed. In general, a cluster’s abstract superclass declares a number of init... and + className methods. As described in “Creating Instances” (page 10), the abstract class decides which concrete subclass to instantiate based your choice of init... or + className method. You can consider that the abstract class declares these methods for the convenience of the subclass. Since the abstract class has no instance variables, it has no need of initialization methods. Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 13Your subclass should declare its own init... (if it needs to initialize its instance variables) and possibly + className methods. It should not rely on any of those that it inherits. To maintain its link in the initialization chain, it should invoke its superclass’s designated initializer within its own designated initializer method. It should also override all other inherited initializer methods and implement them to behave in a reasonable manner. (See ““The Runtime System”“ in The Objective-C Programming Language for a discussion of designated initializers.) Within a class cluster, the designated initializer of the abstract superclass is always init. True Subclasses: An Example Let’s say that you want to create a subclass of NSArray, named MonthArray, that returns the name of a month given its index position. However, a MonthArray object won’t actually store the array of month names as an instance variable. Instead, the method that returns a name given an index position (objectAtIndex:) will return constantstrings. Thus, only twelve string objects will be allocated, no matter how many MonthArray objects exist in an application. The MonthArray class is declared as: #import @interface MonthArray : NSArray { } + monthArray; - (unsigned)count; - (id)objectAtIndex:(unsigned)index; @end Note that the MonthArray class doesn’t declare an init... method because it has no instance variables to initialize. The count and objectAtIndex: methodssimply cover the inherited primitive methods, as described above. The implementation of the MonthArray class looks like this: #import "MonthArray.h" @implementation MonthArray Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 14static MonthArray *sharedMonthArray = nil; static NSString *months[] = { @"January", @"February", @"March", @"April", @"May", @"June", @"July", @"August", @"September", @"October", @"November", @"December" }; + monthArray { if (!sharedMonthArray) { sharedMonthArray = [[MonthArray alloc] init]; } return sharedMonthArray; } - (unsigned)count { return 12; } - objectAtIndex:(unsigned)index { if (index >= [self count]) [NSException raise:NSRangeException format:@"***%s: index (%d) beyond bounds (%d)", sel_getName(_cmd), index, [self count] - 1]; else return months[index]; } @end Because MonthArray overridesthe inherited primitive methods, the derived methodsthat it inherits will work properly without being overridden. NSArray’s lastObject, containsObject:, sortedArrayUsingSelector:, objectEnumerator, and other methods work without problems for MonthArray objects. Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 15A Composite Object By embedding a private cluster object in an object of your own design, you create a composite object. This composite object can rely on the cluster object for its basic functionality, only intercepting messages that the composite object wants to handle in some particular way. This architecture reduces the amount of code you must write and lets you take advantage of the tested code provided by the Foundation Framework. Figure 1-4 depicts this architecture. Figure 1-4 An object that embeds a cluster object The composite object must declare itself to be a subclass of the cluster’s abstract superclass. As a subclass, it must override the superclass’s primitive methods. It can also override derived methods, but this isn’t necessary because the derived methods work through the primitive ones. The count method of the NSArray class is an example; the intervening object’s implementation of a method it overrides can be as simple as: - (unsigned)count { return [embeddedObject count]; } However, your object could put code for its own purposes in the implementation of any method it overrides. A Composite Object: An Example To illustrate the use of a composite object, imagine you want a mutable array object that tests changes against some validation criteria before allowing any modification to the array’s contents. The example that follows describes a class called ValidatingArray, which contains a standardmutable array object. ValidatingArray overrides all of the primitive methods declared in its superclasses, NSArray and NSMutableArray. It also declares the array, validatingArray, and init methods, which can be used to create and initialize an instance: #import Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 16@interface ValidatingArray : NSMutableArray { NSMutableArray *embeddedArray; } + validatingArray; - init; - (unsigned)count; - objectAtIndex:(unsigned)index; - (void)addObject:object; - (void)replaceObjectAtIndex:(unsigned)index withObject:object; - (void)removeLastObject; - (void)insertObject:object atIndex:(unsigned)index; - (void)removeObjectAtIndex:(unsigned)index; @end The implementation file shows how, in an init method of the ValidatingArrayclass, the embedded object is created and assigned to the embeddedArray variable. Messages that simply access the array but don’t modify its contents are relayed to the embedded object. Messagesthat could change the contents are scrutinized (here in pseudocode) and relayed only if they pass the hypothetical validation test. #import "ValidatingArray.h" @implementation ValidatingArray - init { self = [super init]; if (self) { embeddedArray = [[NSMutableArray allocWithZone:[self zone]] init]; } return self; } Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 17+ validatingArray { return [[[self alloc] init] autorelease]; } - (unsigned)count { return [embeddedArray count]; } - objectAtIndex:(unsigned)index { return [embeddedArray objectAtIndex:index]; } - (void)addObject:object { if (/* modification is valid */) { [embeddedArray addObject:object]; } } - (void)replaceObjectAtIndex:(unsigned)index withObject:object; { if (/* modification is valid */) { [embeddedArray replaceObjectAtIndex:index withObject:object]; } } - (void)removeLastObject; { if (/* modification is valid */) { [embeddedArray removeLastObject]; Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 18} } - (void)insertObject:object atIndex:(unsigned)index; { if (/* modification is valid */) { [embeddedArray insertObject:object atIndex:index]; } } - (void)removeObjectAtIndex:(unsigned)index; { if (/* modification is valid */) { [embeddedArray removeObjectAtIndex:index]; } } Class Clusters Creating Subclasses Within a Class Cluster 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 19Class factory methods are implemented by a class as a convenience for clients. They combine allocation and initialization in one step and return the created object. However, the client receiving this object does not own the object and thus (per the object-ownership policy) is not responsible for releasing it. These methods are of the form + (type)className... (where className excludes any prefix). Cocoa provides plenty of examples, especially among the “value” classes. NSDate includes the following class factory methods: + (id)dateWithTimeIntervalSinceNow:(NSTimeInterval)secs; + (id)dateWithTimeIntervalSinceReferenceDate:(NSTimeInterval)secs; + (id)dateWithTimeIntervalSince1970:(NSTimeInterval)secs; And NSData offers the following factory methods: + (id)dataWithBytes:(const void *)bytes length:(unsigned)length; + (id)dataWithBytesNoCopy:(void *)bytes length:(unsigned)length; + (id)dataWithBytesNoCopy:(void *)bytes length:(unsigned)length freeWhenDone:(BOOL)b; + (id)dataWithContentsOfFile:(NSString *)path; + (id)dataWithContentsOfURL:(NSURL *)url; + (id)dataWithContentsOfMappedFile:(NSString *)path; Factory methods can be more than a simple convenience. They can not only combine allocation and initialization, but the allocation can inform the initialization. As an example, let’s say you must initialize a collection object from a property-list file that encodes any number of elements for the collection (NSString objects, NSData objects, NSNumber objects, and so on). Before the factory method can know how much memory to allocate for the collection, it must read the file and parse the property list to determine how many elements there are and what object type these elements are. 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 20 Class Factory MethodsAnother purpose for a classfactory method isto ensure that a certain class(NSWorkspace, for example) vends a singleton instance. Although an init... method could verify that only one instance exists at any one time in a program, it would require the prior allocation of a “raw” instance and then, in memory-managed code, would have to release that instance. A factory method, on the other hand, gives you a way to avoid blindly allocating memory for an object that you might not use, as in the following example: static AccountManager *DefaultManager = nil; + (AccountManager *)defaultManager { if (!DefaultManager) DefaultManager = [[self allocWithZone:NULL] init]; return DefaultManager; } Class Factory Methods 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 21A delegate is an object that acts on behalf of, or in coordination with, another object when that object encounters an event in a program. The delegating object is often a responder object—that is, an object inheriting from NSResponder in AppKit or UIResponder in UIKit—that is responding to a user event. The delegate is an object that is delegated control of the user interface for that event, or is at least asked to interpret the event in an application-specific manner. To better appreciate the value of delegation, it helps to consider an off-the-shelf Cocoa object such as a text field (an instance of NSTextField or UITextField) or a table view (an instance of NSTableView or UITableView ). These objects are designed to fulfill a specific role in a generic fashion; a window object in the AppKit framework, for example, responds to mouse manipulations of its controls and handles such things as closing, resizing, and moving the physical window. This restricted and generic behavior necessarily limits what the object can know about how an event affects (or will affect) something elsewhere in the application, especially when the affected behavior isspecific to your application. Delegation provides a way for your custom object to communicate application-specific behavior to the off-the-shelf object. The programming mechanism of delegation gives objects a chance to coordinate their appearance and state with changes occurring elsewhere in a program, changes usually brought about by user actions. More importantly, delegation makes it possible for one object to alter the behavior of another object without the need to inherit from it. The delegate is almost always one of your custom objects, and by definition it incorporates application-specific logic that the generic and delegating object cannot possibly know itself. How Delegation Works The design of the delegation mechanism is simple—see Figure 3-1 (page 23). The delegating class has an outlet or property, usually one that is named delegate; if it is an outlet, it includes methods for setting and accessing the value of the outlet. It also declares, without implementing, one or more methods that constitute a formal protocol or an informal protocol. A formal protocol that uses optional methods—a feature ofObjective-C 2.0—is the preferred approach, but both kinds of protocols are used by the Cocoa frameworks for delegation. In the informal protocol approach, the delegating class declares methods on a category of NSObject, and the delegate implements only those methods in which it has an interest in coordinating itself with the delegating object or affecting that object’s default behavior. If the delegating class declares a formal protocol, the delegate may choose to implement those methods marked optional, but it must implement the required ones. 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 22 Delegates and Data SourcesDelegation follows a common design, illustrated by Figure 3-1. Figure 3-1 The mechanism of delegation User just clicked close button; should window close? No windowShouldClose: Don't close. The window has unsaved data. windowDelegate The methods of the protocol mark significant events handled or anticipated by the delegating object. This object wants either to communicate these events to the delegate or, for impending events, to request input or approval from the delegate. For example, when a user clicks the close button of a window in OS X, the window object sends the windowShouldClose: message to its delegate; this gives the delegate the opportunity to veto or defer the closing of the window if, for example, the window has associated data that must be saved (see Figure 3-2). Figure 3-2 A more realistic sequence involving a delegate Yes windowShouldClose: ➌ ➊ ➍ ➋ aWindow aDelegate The delegating object sends a message only if the delegate implements the method. It makes this discovery by invoking the NSObjectmethod respondsToSelector: in the delegate first. Delegates and Data Sources How Delegation Works 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 23The Form of Delegation Messages Delegation methods have a conventional form. They begin with the name of the AppKit or UIKit object doing the delegating—application, window, control, and so on; this name is in lower-case and without the “NS” or “UI” prefix. Usually (but not always) this object name is followed by an auxiliary verb indicative of the temporal status of the reported event. This verb, in other words, indicates whether the event is about to occur (“Should” or “Will”) or whether it has just occurred (“Did” or “Has”). This temporal distinction helps to categorize those messagesthat expect a return value and those that don’t. Listing 3-1 includes a few AppKit delegation methods that expect a return value. Listing 3-1 Sample delegation methods with return values - (BOOL)application:(NSApplication *)sender openFile:(NSString *)filename; // NSApplication - (BOOL)application:(UIApplication *)application handleOpenURL:(NSURL *)url; // UIApplicationDelegate - (UITableRowIndexSet *)tableView:(NSTableView *)tableView willSelectRows:(UITableRowIndexSet *)selection; // UITableViewDelegate - (NSRect)windowWillUseStandardFrame:(NSWindow *)window defaultFrame:(NSRect)newFrame; // NSWindow The delegate that implements these methods can block the impending event (by returning NO in the first two methods) or alter a suggested value (the index set and the frame rectangle in the last two methods). It can even defer an impending event; for example, the delegate implementing the applicationShouldTerminate:method can delay application termination by returning NSTerminateLater. Other delegation methods are invoked by messagesthat don’t expect a return value and so are typed to return void. These messages are purely informational, and the method names often contain “Did”, “Will”, or some other indication of a transpired or impending event. Listing 3-2 shows a few examples of these kinds of delegation method. Listing 3-2 Sample delegation methods returning void - (void) tableView:(NSTableView*)tableView mouseDownInHeaderOfTableColumn:(NSTableColumn *)tableColumn; // NSTableView - (void)windowDidMove:(NSNotification *)notification; // NSWindow - (void)application:(UIApplication *)application willChangeStatusBarFrame:(CGRect)newStatusBarFrame; // UIApplication Delegates and Data Sources The Form of Delegation Messages 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 24- (void)applicationWillBecomeActive:(NSNotification *)notification; // NSApplication There are a couple of things to note about this last group of methods. The first is that an auxiliary verb of “Will” (as in the third method) does not necessarily mean that a return value is expected. In this case, the event is imminent and cannot be blocked, but the message gives the delegate an opportunity to prepare the program for the event. The other point of interest concernsthe second and last method declarationsin Listing 3-2 . The sole parameter of each of these methods is an NSNotification object, which means that these methods are invoked as the result of the posting of a particular notification. For example, the windowDidMove: method is associated with the NSWindow notification NSWindowDidMoveNotification. It’s important to understand the relationship of notifications to delegation messages in AppKit. The delegating object automatically makes its delegate an observer of all notifications it posts. All the delegate needs to do is implement the associated method to get the notification. To make an instance of your custom class the delegate of an AppKit object, simply connect the instance to the delegate outlet or property in Interface Builder. Or you can set it programmatically through the delegating object’s setDelegate: method or delegate property, preferably early on, such as in the awakeFromNib or applicationDidFinishLaunching: method. Delegation and the Application Frameworks The delegating object in a Cocoa or Cocoa Touch application is often a responder object such as a UIApplication, NSWindow, or NSTableView object. The delegate object itself istypically, but not necessarily, an object, often a custom object, that controls some part of the application (that is, a coordinating controller object). The following AppKit classes define a delegate: ● NSApplication ● NSBrowser ● NSControl ● NSDrawer ● NSFontManager ● NSFontPanel ● NSMatrix ● NSOutlineView ● NSSplitView Delegates and Data Sources Delegation and the Application Frameworks 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 25● NSTableView ● NSTabView ● NSText ● NSTextField ● NSTextView ● NSWindow The UIKit framework also uses delegation extensively and always implements it using formal protocols. The application delegate is extremely important in an application running in iOS because it must respond to application-launch, application-quit, low-memory, and other messages from the application object. The application delegate must adopt the UIApplicationDelegate protocol. Delegating objects do not (and should not) retain their delegates. However, clients of delegating objects (applications, usually) are responsible for ensuring that their delegates are around to receive delegation messages. To do this, they may have to retain the delegate in memory-managed code. This precaution applies equally to data sources, notification observers, and targets of action messages. Note that in a garbage-collection environment, the reference to the delegate is strong because the retain-cycle problem does not apply. Some AppKit classes have a more restricted type of delegate called a modal delegate . Objects of these classes (NSOpenPanel, for example) run modal dialogs that invoke a handler method in the designated delegate when the user clicksthe dialog’s OK button. Modal delegates are limited in scope to the operation of the modal dialog. Becoming the Delegate of a Framework Class A framework class or any other classthat implements delegation declares a delegate property and a protocol (usually a formal protocol). The protocol liststhe required and optional methodsthat the delegate implements. For an instance of your class to function as the delegate of a framework object, it must do the following: ● Set your object asthe delegate (by assigning it to the delegate property). You can do this programmatically or through Interface Builder. ● If the protocol is formal, declare that your class adopts the protocol in the class definition. For example: @interface MyControllerClass : UIViewController { ● Implement all required methods of the protocol and any optional methods that you want to participate in. Delegates and Data Sources Delegation and the Application Frameworks 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 26Locating Objects Through the delegate Property The existence of delegates has other programmatic uses. For example, with delegates it is easy for two coordinating controllers in the same program to find and communicate with each other. For example, the object controlling the application overall can find the controller of the application’sinspector window (assuming it’s the current key window) using code similar to the following: id winController = [[NSApp keyWindow] delegate]; And your code can find the application-controller object—by definition, the delegate of the global application instance—by doing something similar to the following: id appController = [NSApp delegate]; Data Sources A data source is like a delegate except that, instead of being delegated control of the user interface, it is delegated control of data. A data source is an outlet held by NSView and UIView objects such as table views and outline views that require a source from which to populate their rows of visible data. The data source for a view is usually the same object that acts as its delegate, but it can be any object. As with the delegate, the data source must implement one or more methods of an informal protocol to supply the view with the data it needs and, in more advanced implementations, to handle data that users directly edit in such views. As with delegates, data sources are objectsthat must be present to receive messagesfrom the objectsrequesting data. The application that uses them must ensure their persistence, retaining them if necessary in memory-managed code. Data sources are responsible for the persistence of the objectsthey hand out to user-interface objects. In other words, they are responsible for the memory management of those objects. However, whenever a view object such as an outline view or table view accesses the data from a data source, it retains the objects as long as it uses the data. But it does not use the data for very long. Typically it holds on to the data only long enough to display it. Implementing a Delegate for a Custom Class To implement a delegate for your custom class, complete the following steps: ● Declare the delegate accessor methods in your class header file. Delegates and Data Sources Data Sources 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 27- (id)delegate; - (void)setDelegate:(id)newDelegate; ● Implement the accessor methods. In a memory-managed program, to avoid retain cycles, the setter method should not retain or copy your delegate. - (id)delegate { return delegate; } - (void)setDelegate:(id)newDelegate { delegate = newDelegate; } In a garbage-collected environment, where retain cycles are not a problem, you should not make the delegate a weak reference (by using the __weak type modifier). For more on retain cycles, see Advanced MemoryManagement ProgrammingGuide . Formore on weak referencesin garbage collection,see “Garbage Collection for Cocoa Essentials” in Garbage Collection Programming Guide . ● Declare a formal or informal protocol containing the programmatic interface for the delegate. Informal protocols are categories on the NSObject class. If you declare a formal protocol for your delegate, make sure you mark groups of optional methods with the @optional directive. “The Form of Delegation Messages” (page 24) gives advice for naming your own delegation methods. ● Before invoking a delegation method, make sure the delegate implements it by sending it a respondsToSelector: message. - (void)someMethod { if ( [delegate respondsToSelector:@selector(operationShouldProceed)] ) { if ( [delegate operationShouldProceed] ) { // do something appropriate } } } The precaution is necessary only for optional methods in a formal protocol or methods of an informal protocol. Delegates and Data Sources Implementing a Delegate for a Custom Class 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 28Introspection is a powerful feature of object-oriented languages and environments, and introspection in Objective-C and Cocoa is no exception. Introspection refersto the capability of objectsto divulge details about themselves as objects at runtime. Such details include an object’s place in the inheritance tree, whether it conforms to a specific protocol, and whether it responds to a certain message. The NSObject protocol and class define many introspection methodsthat you can use to query the runtime in order to characterize objects. Used judiciously, introspection makes an object-oriented program more efficient and robust. It can help you avoid message-dispatch errors, erroneous assumptions of object equality, and similar problems. The following sections show how you might effectively use the NSObject introspection methods in your code. Evaluating Inheritance Relationships Once you know the class an object belongs to, you probably know quite a bit about the object. You might know what its capabilities are, what attributes it represents, and what kinds of messages it can respond to. Even if after introspection you are unfamiliar with the classto which an object belongs, you now know enough to not send it certain messages. The NSObject protocol declares several methods for determining an object’s position in the class hierarchy. These methods operate at different granularities. The class and superclass instance methods, for example, return the Class objects representing the class and superclass, respectively, of the receiver. These methods require you to compare one Class object with another. Listing 4-1 gives a simple (one might say trivial) example of their use. Listing 4-1 Using the class and superclass methods // ... while ( id anObject = [objectEnumerator nextObject] ) { if ( [self class] == [anObject superclass] ) { // do something appropriate... } } 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 29 IntrospectionNote: Sometimes you use the class or superclass methods to obtain an appropriate receiver for a class message. More commonly, to check an object’s class affiliation, you would send it a isKindOfClass: or isMemberOfClass: message. The former method returns whether the receiver is an instance of a given class or an instance of any class that inherits from that class. A isMemberOfClass: message, on the other hand, tells you if the receiver is an instance of the specified class. The isKindOfClass: method is generally more useful because from it you can know at once the complete range of messages you can send to an object. Consider the code snippet in Listing 4-2. Listing 4-2 Using isKindOfClass: if ([item isKindOfClass:[NSData class]]) { const unsigned char *bytes = [item bytes]; unsigned int length = [item length]; // ... } By learning that the object item inheritsfrom the NSData class, this code knowsit can send it the NSDatabytes and lengthmessages. The difference between isKindOfClass: and isMemberOfClass: becomes apparent if you assume that item is an instance of NSMutableData. If you use isMemberOfClass: instead of isKindOfClass:, the code in the conditionalized block is never executed because item is not an instance of NSData but rather of NSMutableData, a subclass of NSData. Method Implementation and Protocol Conformance Two of the more powerful introspection methods of NSObject are respondsToSelector: and conformsToProtocol:. These methodstell you, respectively, whether an object implements a certain method and whether an object conforms to a specified formal protocol (that is, adopts the protocol, if necessary, and implements all the methods of the protocol). You use these methodsin a similarsituation in your code. They enable you to discover whethersome potentially anonymous object can respond appropriately to a particular message or set of messages before you send it any of those messages. By making this check before sending a message, you can avoid the risk of runtime exceptionsresulting from unrecognized selectors. The AppKit framework implementsinformal protocols—the basis of delegation—by checking whether delegates implement a delegation method (using respondsToSelector:) prior to invoking that method. Introspection Method Implementation and Protocol Conformance 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 30Listing 4-3 illustrates how you might use the respondsToSelector: method in your code. Listing 4-3 Using respondsToSelector: - (void)doCommandBySelector:(SEL)aSelector { if ([self respondsToSelector:aSelector]) { [self performSelector:aSelector withObject:nil]; } else { [_client doCommandBySelector:aSelector]; } } Listing 4-4 illustrates how you might use the conformsToProtocol: method in your code. Listing 4-4 Using conformsToProtocol: // ... if (!([((id)testObject) conformsToProtocol:@protocol(NSMenuItem)])) { NSLog(@"Custom MenuItem, '%@', not loaded; it must conform to the 'NSMenuItem' protocol.\n", [testObject class]); [testObject release]; testObject = nil; } Object Comparison Although they are not strictly introspection methods, the hash and isEqual: methods fulfill a similar role. They are indispensable runtime toolsfor identifying and comparing objects. But instead of querying the runtime for information about an object, they rely on class-specific comparison logic. The hash and isEqual: methods, both declared by the NSObject protocol, are closely related. The hash method must be implemented to return an integer that can be used as a table addressin a hash table structure. If two objects are equal (as determined by the isEqual: method), they must have the same hash value. If your object could be included in collections such as NSSet objects, you need to define hash and verify the invariant that if two objects are equal, they return the same hash value. The default NSObject implementation of isEqual: simply checks for pointer equality. Introspection Object Comparison 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 31Using the isEqual: method is straightforward; it compares the receiver against the object supplied as a parameter. Object comparison frequently informsruntime decisions about whatshould be done with an object. As Listing 4-5 illustrates, you can use isEqual: to decide whether to perform an action, in this case to save user preferences that have been modified. Listing 4-5 Using isEqual: - (void)saveDefaults { NSDictionary *prefs = [self preferences]; if (![origValues isEqual:prefs]) [Preferences savePreferencesToDefaults:prefs]; } If you are creating a subclass, you might need to override isEqual: to add further checksfor points of equality. The subclass might define an extra attribute that has to be the same value in two instances for them to be considered equal. For example, say you create a subclass of NSObject called MyWidget that contains two instance variables, name and data. Both of these must be the same value for two instances of MyWidget to be considered equal. Listing 4-6 illustrates how you might implement isEqual: for the MyWidget class. Listing 4-6 Overriding isEqual: - (BOOL)isEqual:(id)other { if (other == self) return YES; if (!other || ![other isKindOfClass:[self class]]) return NO; return [self isEqualToWidget:other]; } - (BOOL)isEqualToWidget:(MyWidget *)aWidget { if (self == aWidget) return YES; if (![(id)[self name] isEqual:[aWidget name]]) return NO; if (![[self data] isEqualToData:[aWidget data]]) return NO; return YES; Introspection Object Comparison 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 32} This isEqual: method first checks for pointer equality, then class equality, and finally invokes an object comparator whose name indicates the class of object involved in the comparison. This type of comparator, which forcestype checking of the object passed in, is a common convention in Cocoa; the isEqualToString: method of the NSString class and the isEqualToTimeZone: method of the NSTimeZone class are but two examples. The class-specific comparator—isEqualToWidget: in this case—performs the checks for name and data equality. In all isEqualToType: methods of the Cocoa frameworks, nil is not a valid parameter and implementations of these methods may raise an exception upon receiving a nil. However, for backward compatibility, isEqual: methods of the Cocoa frameworks do accept nil, returning NO. Introspection Object Comparison 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 33When you allocate an object, part of what happens is what you might expect, given the term. Cocoa allocates enough memory for the object from a region of application virtual memory. To calculate how much memory to allocate, it takes the object’s instance variables into account—including their types and order—as specified by the object’s class. To allocate an object, you send the message alloc or allocWithZone: to the object’s class. In return, you get a “raw” (uninitialized) instance of the class. The alloc variant of the method uses the application’s default zone. A zone is a page-aligned area of memory for holding related objects and data allocated by an application. See Advanced Memory Management Programming Guide for more information on zones. An allocation message does other important things besides allocating memory: ● It sets the object’s retain count to one. ● It initializes the object’s isainstance variable to point to the object’s class, a runtime object in its own right that is compiled from the class definition. ● It initializes all other instance variables to zero (or to the equivalent type for zero, such as nil, NULL, and 0.0). An object’s isa instance variable is inherited from NSObject, so it is common to all Cocoa objects. After allocation sets isa to the object’s class, the object is integrated into the runtime’s view of the inheritance hierarchy and the current network of objects (class and instance) that constitute a program. Consequently an object can find whatever information it needs at runtime, such as another object’s place in the inheritance hierarchy, the protocols that other objects conform to, and the location of the method implementations it can perform in response to messages. In summary, allocation not only allocates memory for an object but initializes two small but very important attributes of any object: its isa instance variable and itsretain count. It also sets all remaining instance variables to zero. But the resulting object is not yet usable. Initializing methods such as init must yet initialize objects with their particular characteristics and return a functional object. 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 34 Object AllocationInitialization sets the instance variables of an object to reasonable and useful initial values. It can also allocate and prepare other global resources needed by the object, loading them if necessary from an external source such as a file. Every object that declares instance variables should implement an initializing method—unless the defaultset-everything-to-zero initialization issufficient. If an object does not implement an initializer, Cocoa invokes the initializer of the nearest ancestor instead. The Form of Initializers NSObject declares the init prototype for initializers; it is an instance method typed to return an object of type id. Overriding init is fine for subclasses that require no additional data to initialize their objects. But often initialization depends on external data to set an object to a reasonable initial state. For example, say you have an Account class; to initialize an Account object appropriately requires a unique account number, and this must be supplied to the initializer. Thus initializers can take one or more parameters; the only requirement is that the initializing method begins with the letters “init”. (The stylistic convention init... is sometimes used to refer to initializers.) Note: Instead of implementing an initializer with parameters, a subclass can implement only a simple init method and then use “set” accessor methods immediately after initialization to set the object to a useful initial state. (Accessor methods enforce encapsulation of object data by setting and getting the values of instance variables.) Or, if the subclass uses properties and the related access syntax, it may assign values to the properties immediately after initialization. Cocoa has plenty of examples of initializers with parameters. Here are a few (with the defining class in parentheses): - (id)initWithArray:(NSArray *)array; (from NSSet) - (id)initWithTimeInterval:(NSTimeInterval)secsToBeAdded sinceDate:(NSDate *)anotherDate; (from NSDate) - (id)initWithContentRect:(NSRect)contentRect styleMask:(unsigned int)aStyle backing:(NSBackingStoreType)bufferingType defer:(BOOL)flag; (from NSWindow) - (id)initWithFrame:(NSRect)frameRect; (from NSControl and NSView) 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 35 Object InitializationThese initializers are instance methods that begin with “init” and return an object of the dynamic type id. Other than that, they follow the Cocoa conventions for multiparameter methods, often using WithType: or FromSource: before the first and most important parameter. Issues with Initializers Although init... methods are required by their method signature to return an object, that object is not necessarily the one that was most recently allocated—the receiver of the init... message. In other words, the object you get back from an initializer might not be the one you thought was being initialized. Two conditions prompt the return of something other than the just-allocated object. The first involves two related situations: when there must be a singleton instance or when the defining attribute of an object must be unique. Some Cocoa classes—NSWorkspace, for instance—allow only one instance in a program; a class in such a case must ensure (in an initializer or, more likely, in a class factory method) that only one instance is created, returning this instance if there is any further request for a new one. A similar situation arises when an object is required to have an attribute that makes it unique. Recall the hypothetical Account class mentioned earlier. An account of any sort must have a unique identifier. If the initializer for this class—say, initWithAccountID:—is passed an identifier that has already been associated with an object, it must do two things: ● Release the newly allocated object (in memory-managed code) ● Return the Account object previously initialized with this unique identifier By doing this, the initializer ensures the uniqueness of the identifier while providing what was asked for: an Account instance with the requested identifier. Sometimes an init... method cannot perform the initialization requested. For example, an initFromFile: method expects to initialize an object from the contents of a file, the path to which is passed as a parameter. But if no file exists at that location, the object cannot be initialized. A similar problem happens if an initWithArray: initializer is passed an NSDictionary object instead of an NSArray object. When an init... method cannot initialize an object, it should: ● Release the newly allocated object (in memory-managed code) ● Return nil Returning nil from an initializer indicates that the requested object cannot be created. When you create an object, you should generally check whether the returned value is nil before proceeding: Object Initialization Issues with Initializers 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 36id anObject = [[MyClass alloc] init]; if (anObject) { [anObject doSomething]; // more messages... } else { // handle error } Because an init... method might return nil or an object other than the one explicitly allocated, it is dangerous to use the instance returned by alloc or allocWithZone: instead of the one returned by the initializer. Consider the following code: id myObject = [MyClass alloc]; [myObject init]; [myObject doSomething]; The init method in this example could have returned nil or could have substituted a different object. Because you can send a message to nil without raising an exception, nothing would happen in the former case except (perhaps) a debugging headache. But you should always rely on the initialized instance instead of the “raw” just-allocated one. Therefore, you should nest the allocation message inside the initialization message and test the object returned from the initializer before proceeding. id myObject = [[MyClass alloc] init]; if ( myObject ) { [myObject doSomething]; } else { // error recovery... } Once an object is initialized, you should not initialize it again. If you attempt a reinitialization, the framework class of the instantiated object often raises an exception. For example, the second initialization in this example would result in NSInvalidArgumentException being raised. NSString *aStr = [[NSString alloc] initWithString:@"Foo"]; aStr = [aStr initWithString:@"Bar"]; Object Initialization Issues with Initializers 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 37Implementing an Initializer There are several critical rules to follow when implementing an init... method that serves as a class’s sole initializer or, if there are multiple initializers, its designated initializer (described in “Multiple Initializers and the Designated Initializer” (page 40)): ● Always invoke the superclass (super) initializer first. ● Check the object returned by the superclass. If it is nil, then initialization cannot proceed; return nil to the receiver. ● When initializing instance variables that are references to objects, retain or copy the object as necessary (in memory-managed code). ● After setting instance variables to valid initial values, return self unless: ● It was necessary to return a substituted object, in which case release the freshly allocated object first (in memory-managed code). ● A problem prevented initialization from succeeding, in which case return nil. - (id)initWithAccountID:(NSString *)identifier { if ( self = [super init] ) { Account *ac = [accountDictionary objectForKey:identifier]; if (ac) { // object with that ID already exists [self release]; return [ac retain]; } if (identifier) { accountID = [identifier copy]; // accountID is instance variable [accountDictionary setObject:self forKey:identifier]; return self; } else { [self release]; return nil; } } else return nil; } Object Initialization Implementing an Initializer 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 38Note: Although, for the sake of simplicity, this example returns nil if the parameter is nil, the better Cocoa practice is to raise an exception. It isn’t necessary to initialize all instance variables of an object explicitly, just those that are necessary to make the object functional. The default set-to-zero initialization performed on an instance variable during allocation is often sufficient. Make sure that you retain or copy instance variables, as required for memory management. The requirement to invoke the superclass’s initializer as the first action is important. Recall that an object encapsulates not only the instance variables defined by its class but the instance variables defined by all of its ancestor classes. By invoking the initializer of super first, you help to ensure that the instance variables defined by classes up the inheritance chain are initialized first. The immediate superclass, in its initializer, invokes the initializer of its superclass, which invokes the main init... method of its superclass, and so on (see Figure 6-1). The proper order of initialization is critical because the later initializations of subclasses may depend on superclass-defined instance variables being initialized to reasonable values. Figure 6-1 Initialization up the inheritance chain super super Class A Class B Class C inherits from inherits from self - (id)initWithName:birthday: - (id)initWithName: Instance variables: NSString *name: Instance variables: NSString *name: NSDate *dob: - (id)initWithName:birthday:ssn: Instance variables: NSString *name: NSDate *dob: NSNumber *ssn: sets sets sets Object Initialization Implementing an Initializer 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 39Inherited initializers are a concern when you create a subclass. Sometimes a superclass init... method sufficiently initializes instances of your class. But because it is more likely it won’t, you should override the superclass’s initializer. If you don’t, the superclass’s implementation is invoked, and because the superclass knows nothing about your class, your instances may not be correctly initialized. Multiple Initializers and the Designated Initializer A class can define more than one initializer. Sometimes multiple initializers let clients of the class provide the input for the same initialization in different forms. The NSSet class, for example, offers clientsseveral initializers that accept the same data in different forms; one takes an NSArray object, another a counted list of elements, and another a nil-terminated list of elements: - (id)initWithArray:(NSArray *)array; - (id)initWithObjects:(id *)objects count:(unsigned)count; - (id)initWithObjects:(id)firstObj, ...; Some subclasses provide convenience initializers that supply default values to an initializer that takes the full complement of initialization parameters. Thisinitializer is usually the designated initializer, the most important initializer of a class. For example, assume there is a Task class and it declares a designated initializer with this signature: - (id)initWithTitle:(NSString *)aTitle date:(NSDate *)aDate; The Task class might include secondary, or convenience, initializersthatsimply invoke the designated initializer, passing it default values for those parameters the secondary initializer doesn’t explicitly request. This example shows a designated initializer and a secondary initializer. - (id)initWithTitle:(NSString *)aTitle { return [self initWithTitle:aTitle date:[NSDate date]]; } - (id)init { return [self initWithTitle:@"Task"]; } Object Initialization Multiple Initializers and the Designated Initializer 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 40The designated initializer plays an important role for a class. It ensures that inherited instance variables are initialized by invoking the designated initializer of the superclass. It is typically the init... method that has the most parameters and that does most of the initialization work, and it is the initializer that secondary initializers of the class invoke with messages to self. When you define a subclass, you must be able to identify the designated initializer of the superclass and invoke it in your subclass’s designated initializer through a message to super. You must also make sure that inherited initializers are covered in some way. And you may provide as many convenience initializers as you deem necessary. When designing the initializers of your class, keep in mind that designated initializers are chained to each other through messages to super; whereas other initializers are chained to the designated initializer of their class through messages to self. An example will make this clearer. Let’s say there are three classes, A, B, and C; class B inherits from class A, and class C inherits from class B. Each subclass adds an attribute as an instance variable and implements an init... method—the designated initializer—to initialize this instance variable. They also define secondary initializers and ensure that inherited initializers are overridden, if necessary. Figure 6-2 illustrates the initializers of all three classes and their relationships. Figure 6-2 Interactions of secondary and designated initializers - (id)init super super Class A Class B Class C inherits from inherits from self - (id)init - (id)initWithTitle: self - (id)initWithTitle: - (id)initWithTitle:date: Object Initialization Multiple Initializers and the Designated Initializer 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 41The designated initializer for each class is the initializer with the most coverage; it is the method that initializes the attribute added by the subclass. The designated initializer is also the init... method that invokes the designated initializer of the superclass in a message to super. In this example, the designated initializer of class C, initWithTitle:date:, invokesthe designated initializer of itssuperclass, initWithTitle:, which in turn invokes the init method of class A. When creating a subclass, it’s always important to know the designated initializer of the superclass. Although designated initializers are thus connected up the inheritance chain through messages to super, secondary initializers are connected to their class’s designated initializer through messagesto self. Secondary initializers (as in this example) are frequently overridden versions of inherited initializers. Class C overrides initWithTitle: to invoke its designated initializer, passing it a default date. This designated initializer, in turn, invokes the designated initializer of class B, which is the overridden method, initWithTitle:. If you sent an initWithTitle: message to objects of class B and class C, you’d be invoking different method implementations. On the other hand, if class C did not override initWithTitle: and you sent the message to an instance of class C, the class B implementation would be invoked. Consequently, the C instance would be incompletely initialized (since it would lack a date). When creating a subclass, it’s important to make sure that all inherited initializers are adequately covered. Sometimes the designated initializer of a superclass may be sufficient for the subclass, and so there is no need for the subclass to implement its own designated initializer. Other times, a class’s designated initializer may be an overridden version of its superclass's designated initializer. This is frequently the case when the subclass needs to supplement the work performed by the superclass’s designated initializer, even though the subclass does not add any instance variables of its own (or the instance variables it does add don’t require explicit initialization). Object Initialization Multiple Initializers and the Designated Initializer 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 42The Model-View-Controller design pattern (MVC) is quite old. Variations of it have been around at least since the early days of Smalltalk. It is a high-level pattern in that it concerns itself with the global architecture of an application and classifies objects according to the general rolesthey play in an application. It is also a compound pattern in that it comprises several, more elemental patterns. Object-oriented programs benefit in several ways by adapting the MVC design pattern for their designs. Many objectsin these programstend to be more reusable and their interfacestend to be better defined. The programs overall are more adaptable to changing requirements—in other words, they are more easily extensible than programs that are not based on MVC. Moreover, many technologies and architectures in Cocoa—such as bindings, the document architecture, and scriptability—are based on MVC and require that your custom objects play one of the roles defined by MVC. Roles and Relationships of MVC Objects The MVC design pattern considersthere to be three types of objects: model objects, view objects, and controller objects. The MVC pattern defines the roles that these types of objects play in the application and their lines of communication. When designing an application, a major step is choosing—or creating custom classes for—objects that fall into one of these three groups. Each of the three types of objects is separated from the others by abstract boundaries and communicates with objects of the other types across those boundaries. Model Objects Encapsulate Data and Basic Behaviors Model objects represent special knowledge and expertise. They hold an application’s data and define the logic that manipulates that data. A well-designed MVC application has all its important data encapsulated in model objects. Any data that is part of the persistent state of the application (whether that persistent state is stored in files or databases) should reside in the model objects once the data is loaded into the application. Because they represent knowledge and expertise related to a specific problem domain, they tend to be reusable. Ideally, a model object has no explicit connection to the user interface used to present and edit it. For example, if you have a model object that represents a person (say you are writing an address book), you might want to store a birthdate. That’s a good thing to store in your Person model object. However, storing a date format string or other information on how that date is to be presented is probably better off somewhere else. 2012-01-09 | © 2012 Apple Inc. All Rights Reserved. 43 Model-View-ControllerIn practice, thisseparation is not alwaysthe best thing, and there issome room for flexibility here, but in general a model object should not be concerned with interface and presentation issues. One example where a bit of an exception isreasonable is a drawing application that has model objectsthat represent the graphics displayed. It makes sense for the graphic objects to know how to draw themselves because the main reason for their existence is to define a visual thing. But even in this case, the graphic objects should not rely on living in a particular view or any view at all, and they should not be in charge of knowing when to draw themselves. They should be asked to draw themselves by the view object that wants to present them. View Objects Present Information to the User A view object knows how to display, and might allow users to edit, the data from the application’s model. The view should not be responsible forstoring the data it is displaying. (This does not mean the view never actually stores data it’s displaying, of course. A view can cache data or do similar tricks for performance reasons). A view object can be in charge of displaying just one part of a model object, or a whole model object, or even many different model objects. Views come in many different varieties. View objects tend to be reusable and configurable, and they provide consistency between applications. In Cocoa, the AppKit framework defines a large number of view objects and provides many of them in the Interface Builder library. By reusing the AppKit’s view objects, such as NSButton objects, you guarantee that buttons in your application behave just like buttonsin any other Cocoa application, assuring a high level of consistency in appearance and behavior across applications. A view should ensure it is displaying the model correctly. Consequently, it usually needsto know about changes to the model. Because model objects should not be tied to specific view objects, they need a generic way of indicating that they have changed. Controller Objects Tie the Model to the View A controller object acts as the intermediary between the application's view objects and its model objects. Controllers are often in charge of making sure the views have access to the model objects they need to display and act as the conduit through which views learn about changes to the model. Controller objects can also perform set-up and coordinating tasks for an application and manage the life cycles of other objects. In a typical Cocoa MVC de