Installshield Script Custom Dialog

Here is the process I take for handling Custom Dialog boxes
1. Switch to Dialog under User Interface
2. Create your custom dialog box with all its controls the way you need it.
3. Swtich to direct editor and select the Dialog table.
4. When you created your dialog you could specify the name you want for the dialog. That name should appear in this table.
5. Find that name and then scroll all the way to the right. The very last column contains the ID.
6. It seems that the default for this ID is 0 (unless I am doing something wrong). Change it to a number that does not conflict with existing dialogs. I usually use numbers in the 13000 range. What value you specify here you have to now define in your script.

Here is how I handle defining my dialogs.
1. Create a CustomDialogs.h file (the name does not matter).
2. In this .h file create prototypes and defines for each dialog and their controls.

prototype License();
prototype AbbrevLicense();
prototype NUMBER FrameWorkRegDlg();
prototype NUMBER PhoneRegDlg();
...
//define dialog IDs
#define RES_DIALOG_ID 13051
#define RES_DIALOG_PRODUCT_UNINSTALL 13071
#define REGISTRATION_DLG_ID 13034
#define LICENSE_DIALOG_ID 13035

//define controls for dialogs
//Control IDs
#define RADIO1 1000
#define RADIO2 1005
#define RADIO3 1010

#define CHECKBOX1 1100
#define CHECKBOX2 1105
#define CHECKBOX3 1110

#define TEXT1 3000
#define TEXT2 3005
#define TEXT3 3010

#define COMBO1 7000
#define COMBO2 7005

3. Create a CustomDialogs.rul file (name does not matter) and define your functions.

///////////////////////////////////////////////////////////////////////////////
//
// Function: DriverInstall
// Purpose: Custom Dialog requiring user to choose from 2 radio buttons. Large
// text description boxes above and below radio buttons.
//
///////////////////////////////////////////////////////////////////////////////
function DriverInstall(sMsg, sRadioOne, sRadioTwo)
STRING szDialogName;
HWND hDlg;
BOOL bBasicOnly;
NUMBER nReturn;
begin

// Specify a name to identify the custom dialog in this setup.
szDialogName = "CustomDialog";

// Define the dialog. Pass a null string in the second parameter
// to get the dialog from _isuser.dll or _isres.dll. Pass a null
// string in the third parameter because the dialog is identified
// by its ID in the fourth parameter.
nResult = EzDefineDialog(szDialogName, "", "", TEXT2_RADIO2);

if (nResult < 0) then
// Report an error; then terminate.
MessageBox ("Error in defining dialog", SEVERE);
abort;
endif;

// Initialize the indicator used to control the while loop.
bDone = FALSE;

// Loop until done.
repeat

// Display the dialog and return the next dialog event.
nCmdValue = WaitOnDialog(szDialogName);

// Respond to the event.
switch (nCmdValue)

case 2, DLG_CLOSE:
Do(EXIT); // The user clicked the window's close button.

case DLG_ERR:
MessageBox ("Dialog failed.", SEVERE);
bDone = TRUE;

case DLG_INIT: ;
// No initialization is required for this example.
CtrlSetState(szDialogName, RADIO1, BUTTON_CHECKED);
CtrlSetText(szDialogName, TEXT1, sMsg);
CtrlSetText(szDialogName, TEXT2, ""); //Second text box is not used, so set it to empty string.
CtrlSetText(szDialogName, RADIO1, sRadioOne);
CtrlSetText(szDialogName, RADIO2, sRadioTwo);

hDlg = CmdGetHwndDlg(szDialogName);
SetWindowText(hDlg, "Driver Option"); // Set the window title. Defined in Winsub.rul.

case SD_PBUT_CONTINUE:
if (CtrlGetState(szDialogName, RADIO1) = BUTTON_CHECKED) then
bInstDriver = FALSE;
elseif (CtrlGetState(szDialogName, RADIO2) = BUTTON_CHECKED) then
bInstDriver = TRUE;
endif;

bDone = TRUE;

case SD_PBUT_EXITSETUP: //Cancel button
Do(EXIT);
bDone=TRUE;

case SD_PBUT_BACK:
EndDialog(szDialogName); // Close the dialog box.
ReleaseDialog(szDialogName); // Free the dialog box from memory.
return BACK;

case SD_PBUT_CANCEL: //Esc key
Do(EXIT);

endswitch;

until bDone;


// Close the dialog box.
EndDialog(szDialogName);
// Free the dialog box from memory.
ReleaseDialog(szDialogName);

end;

Source : http://community.flexerasoftware.com/archive/index.php?t-144126.html