Common Features Test Specification for Generic VCS Module
Author: Jiri Kovalsky
Version: 1.0
Last update: Tue July 2 16:05:17 CET 2002
Table of Contents
1. Introduction:
Since Generic VCS module was designed to be as generic as possible and therefore open for a lot
of customization it is necessary to test especially its core features. This is exactly what is described in the following
sections and should be tested very carefully.
Legend:
- this icon represents a test case that is part of validation suite and is used during Q-build
testing.
- this icon represents a test case that was automated and is run daily in test runtime.
2. General availability
Purpose: This suite tests whether there are all features available in the IDE. It means to verify the presence of
all mounting points, Versioning menu, outputs in Runtime tab etc. Without these pieces whole module would become completely unusable.
Setup: Create new directory with similar contents like it is shown at these pictures.
| Test case # |
Description |
2.1 Versioning menu
|
1. Invoke Versioning | Mount Version Control from main menu.
2. Make sure there is an item Generic VCS. Select it.
3. Profile page of mounting wizard must appear. Close it without any further testing.
|
| RESULT: It must be possible to invoke mounting wizard using mentioned menu item. |
2.2 Popup menu
|
1. Right click "Filesystem" node in the explorer and choose Mount | Version Control
| Generic VCS item from the popup menu.
2. Setup "Working Directory" to the directory you prepared e.g. D:\Tests\Empty\Work1 and push "OK" button.
3. Select "Empty" profile from the list and push "Finish" button.
4. Right click "Empty D:\Tests\Empty\Work1" node in explorer and verify that its popup menu contains these actions:
Versioning Explorer and Refresh,
Refresh Recursively, Check in,
Check out, Lock,
Unlock, Add and
Remove in Empty submenu.
|
RESULT: There must be "Empty D:\Tests\Empty\Work1" filesystem mounted into explorer.
Its popup menu action must have all of mentioned actions. |
2.3 Runtime tab
|
1. Execute Empty | Lock on the root of your filesystem.
2. Status line of IDE must contain "Command Lock finished." message.
3. Switch to "Runtime" tab of explorer and expand "VCS Commands | Empty D:\Tests\Empty\Work1" nodes.
4. Right click "Lock" node and choose View Output action.
5. Verify that "Standard Output" tab contains "put your LOCK command here" message and close the window.
6. Verify that all properties have correct values especially "Processed Files" is "." and "Status"
is "Finished".
7. Click "Empty D:\Tests\Empty\Work1" node and set its "Number of Finished Commands To Keep" property to 1.
8. Get back to "Filesystem" tab and invoke Empty | Add popup menu action.
|
RESULT: Commands history in "Runtime" tab must work. Only one command should be stored
there for Empty filesystem after this test case. |
2.4 Search service
|
1. Right click the filesystem and choose Find... from popup menu.
2. Switch to "Status" tab, select "Local" option and push "Search" button.
3. Verify that only [Local] files were found. Click "Modify Search" button.
4. Switch to "Status" tab again and check "Match Only Files That Do Not Meet the Selected Criteria" checkbox.
5. Push "Search" button and make sure the complement to previous query is displayed now. Close the window.
|
| RESULT: Search service must be available and appropriate nodes should be found according to selected
statuses. These must be offered: Dead, Ignored, Local, Locally Modified, Not in Synch,
Unknown. |
2.5 Toolbar actions
|
1. Right click toolbars area and check "Versioning" checkbox.
2. Right click the filesystem and choose Customize.
3. Switch to "Advanced" tab and push "Edit Commands..." button.
4. Expand "Empty" folder and select "Add" node. Switch to "Expert" tab.
5. Set "General Command Action Name" property to "org.netbeans.modules.vcscore.actions.AddCommandAction"
and close editor and customizer.
6. Select the filesystem and click the third versioning toolbar button.
|
RESULT: Add command must be executed and status line of IDE must say: "Command Add finished." |
3. Mounting wizard
Purpose: This suite tests functionality of mounting wizard which means to verify that it is possible to mount
command-line filesystem according to given specifications except profiles management and commands/variables editors.
Setup: No preparation required.
| Test case # |
Description |
3.1 Profile selector
|
1. Invoke Versioning | Mount Version Control | Generic VCS from main menu.
2. From the upper left list choose CVS profile. The wizard must be reconfigured without any question.
3. Select another profile from the list. You must be asked to confirm the change. Answer "Yes" and make sure the dialog was rebuilt.
4. Select another one but answer "No" this time. Nothing must change.
|
| RESULT: Profile selector must always reconfigure the dialog. Except the first change user must be asked to confirm the operation. |
3.2 Buttons working
|
1. Uncheck "Only Show Profiles Compatible With Current Operating System" checkbox.
2. Select "CVS (Windows 95/98/ME)" profile from the list and confirm your decision.
3. Click each "Browse..." and "Select..." buttons and try to choose some file or directory with it.
|
| RESULT: All buttons must allow user to fill in the textfield next to them. |
3.3 Additional profiles link
|
1. Click the "http://vcsgeneric.netbeans.org/profiles/index.html" hypertext link.
|
| RESULT: External browser must get opened with the mentioned profiles page. |
3.4 Checkboxes working
|
1. Push "Next >" button to get to "Advanced" page of the wizard.
2. Make sure that "Call EDIT ..." checkbox enables its inner checkbox together with message textfield.
3. Push "< Back" button, choose PVCS profile for your system, confirm and go "Next >".
4. Verify the same behaviour with "Lock files ..." checkbox and its subcomponents.
|
| RESULT: Checkboxes must enable its subcomponents. CVS must have auto-editing feature enabled
and turned on unlike PVCS with auto-locking feature enabled but turned off by default. |
3.5 OS compatibility
|
1. Verify that your current operating system was detected correctly.
2. Verify that compatible and incompatible systems fields are right according to your platform.
3. Push "< Back" and uncheck "Only Show Profiles Compatible With Current Operating System" checkbox.
4. Select some profile for different platform and make sure that the same checkbox got disabled.
5. Select the previously selected profile finally and confirm the operation.
|
| RESULT: The mentioned checkbox must be enabled again and the wizard must be reconfigured successfully. |
3.6 Environment setup
|
1. Push "Next >" button that will shift you to the last page of wizard.
2. Push "Insert" button, type "MYNAME" as "Variable Name" and push "OK".
3. Delete this variable using "Delete" button and then repeat step 2.
4. New "MYNAME" record must appear in the table. Click its value area and type "Jirka" for example.
5. Uncheck arbitrary two existing environment variables and push "< Back" button.
|
| RESULT: It must be possible to create new and delete existing environment variables and disable system variables. |
3.7 Mount filesystem
|
1. Check finally all advanced checkboxes and push "< Back" button again.
2. Setup "Working Directory" to the one from previous test suite.
3. Push "Finish" button and verify that "PVCS D:\Tests\Empty\Work1" filesystem appeared in explorer.
4. Right click this filesystem and choose Customize action.
5. Verify that all settings in all tabs are set as before filesystem was mounted.
|
| RESULT: New PVCS filesystem must be mounted into explorer and all settings must be applied. |
3.8 Settings active
|
1. Right click the filesystem and choose PVCS | Lock... item.
2. You must be prompted with "Lock -" dialog. Push "Cancel" button.
3. There should be two messages in the standard output window of IDE: "Preprocessing ..." and
"Command Lock was cancelled".
4. Turn off "Advanced Options" and "Print Command Output" properties of the filesystem.
5. Invoke PVCS | Lock on the filesystem and repeat step 2. No messages
must be printed now.
|
| RESULT: Both advanced and debugging modes must be taken into account and available to turn off through
filesystem properties. |
4. Variables
Purpose: This suite tests possibility to create, customize and delete profile variables.
Setup: No preparation required.
| Test case # |
Description |
4.1 Variable editor
|
1. Invoke Customize on the filesystem.
2. Switch to "Advanced" tab of the customizer.
3. Push "Edit Variables..." button.
|
RESULT: "Variable Editor" must show up. Left pane must contain variable tree with
two nodes "Basic" and "Accessory" and right pane is property sheet. |
4.2 Accessory variable
|
1. Expand "Accessory" node and Delete variable
"CONFIG_INPUT_DESCRIPTOR".
2. Right click "Accessory" node and choose Add Variable action.
3. Setup "Name" as its name and push "OK" button.
4. Select "NAME" variable and set its "Value" property to "Jirka".
|
RESULT: "Basic" node must not be disabled any more and "Accessory" node must contain
"NAME" variable underneath. |
4.3 Basic variable
|
1. Expand "Basic" node and Delete variable
"PVCS Project Name".
2. Right click "Basic" node and choose Add Variable action.
3. Setup "MyFile" as its name and push "OK" button.
|
RESULT: "Basic" node must contain "MyFile" variable underneath. |
4.4 Customize basic variable
|
1. Select "MyFile" variable, set its "Value" property to "File.txt" and turn on
"Variable Is a Local File" property.
2. Push "OK" button to close the editor and switch to "Profile" tab of customizer.
3. There must be "MyFile" textfield set to "File.txt" and "Browse..." button
allowing to choose a file.
4. Move Up "MyFile" node's position, turn on "Variable
Is a Local Folder" and check both.
|
RESULT: "MyFile" variable must be displayed on "Profile" tab of customizer
in proper position with preset value and appropriate file/folder chooser. |
5. Commands
Purpose: This suite tests possibility to create, customize and delete profile commands. It also checks functionality
of the most important command properties.
Setup: No preparation required.
| Test case # |
Description |
5.1 Command editor
|
1. Switch to "Advanced" tab of the customizer again.
2. Push "Edit Commands..." button.
|
RESULT: "Command Editor" must show up. Left pane must contain
command tree and right pane is a property sheet. |
5.2 Popup devider
|
1. Right click root node and choose Add | Separator.
|
RESULT: "(separator)" node must appear as the last leaf of popup tree. |
5.3 Popup submenu
|
1. Right click root node and choose Add | Folder.
2. Type "My Submenu" as "Folder Label" and push "OK" button.
|
RESULT: "My Submenu" folder node must appear as the last leaf of popup tree. |
5.4 Popup action
|
1. Right click "My Submenu" node and choose Add | Command.
2. Type "Test" as "Command Name" and push "OK" button.
3. Expand "My Submenu" node.
|
RESULT: There must be "Test" node under "My Submenu" node. |
5.5 Customize menu
|
1. Right click "(separator)" node and choose Move Down.
2. Verify that order of devider and "My Submenu" has changed and the same action is disabled on that node.
3. Cut "(separator)" node and Paste it on "My Submenu" node.
4. Right click "VERIFY_ACTION" node and choose Delete action.
|
RESULT: Separator had to be temporarily below new submenu before its move under "My Submenu".
There must be no "VERIFY_ACTION" node any more. |
5.6 Setup command
|
1. Select "My Submenu | Test" node in the tree.
2. Set "Exec" property to cmd /x /c "echo Ahoj ${NAME}! && notepad ${MYFILE}" for Windows NT/2000
or sh -c "echo Ahoj ${NAME}!; xedit ${MYFILE}" for all the other operating systems.
3. Turn on "Display Output" property and push "OK" button.
|
RESULT: "My Submenu | Test" command must have both properties set as needed. |
5.7 Invoke command
|
1. Switch to "Profile" tab of the customizer.
2. Use "Browse..." button to choose some text file like C:\autoexec.bat or similar on Unix systems.
3. Push "Close" button on the customizer.
4. Right click the filesystem and choose My Submenu | Test action.
5. Make sure that status bar displays "Command Test is running ..." and "Output of VCS Command [Test]"
window contains "Kill" button.
6. Make sure that Notepad or Xedit was opened with the selected file and there is "Ahoj Jirka!" message in the "Standard
Output" tab of mentioned window.
7. Close the external editor and verify that status bar contains "Command Test finished." message.
|
RESULT: Button in output window must change to "Close" and the status must be "Finished". |
5.8 Confirmation messages
|
1. Invoke "Command Editor" and use test case "5.6 Setup command" for next three steps.
2. Setup "Confirmation Message Before Execution" property to "Really ?" text.
3. Setup "Notification Message After Fail" property to "Bad luck ! :-(" text.
4. Close the editor using "OK" button and "Close" the customizer too.
5. Invoke My Submenu | Test command on the root.
6. Answer "Yes" to "Really ?" question dialog.
7. Push "Kill" button on the output window.
|
RESULT: Information window saying "Bad luck ! :-(" must appear. Close it. |
| 5.9 Regular expressions |
1. Set "Exec" property of "Test" command to cmd /x /c "echo Ahoj ${NAME}! && echo Cau Martine!"
for Windows NT/2000 and sh -c "echo Ahoj ${NAME}! && echo Cau Martine!" for other systems.
2. Setup "Data Regex" expert property to ^Ahoj (.*)! value.
3. Setup "Confirmation Message Before Execution" property to default value.
4. Invoke My Submenu | Test command on the root.
|
RESULT: "Data Output" tab of output window must contain only "[Jirka]" line. |
| 5.10 Visibility restrictions |
1. Turn off "Visible on Folders" property of the same command.
2. Right click "PVCS D:\Tests\Empty\Work1 | test [Local]" node.
|
| RESULT: There must be no Test item in My Submenu menu. |
| 5.11 Enable restrictions |
1. Setup "Disabled on Statuses" expert property to "Local" value.
2. Right click the root node of your filesystem.
3. Verify that My Submenu | Test action is visible and enabled.
4. Right click "PVCS D:\Tests\Empty\Work1 | A_File [Local]" node.
|
| RESULT: My Submenu | Test command must be visible but disabled. |
| 5.12 Input descriptor |
1. Setup "Input Descriptor" property to LABEL("Hello!") PROMPT_FOR(NAME, "What's your name?") value.
2. Invoke My Submenu | Test action on the root node of your filesystem.
3. "Hello! -" dialog must show up with "What's your name?" textfield.
4. Type there "Maruska" and push "OK" button.
|
RESULT: There must be these two lines in "Standard Output" tab: "Hello Maruska!" and
"Cau Martine!". |
| 5.13 Multifile recognition |
1. Setup "Disabled on Statuses" property to default value.
2. Turn on "Run on Multiple Files" property.
3. Turn on "Process All Files" expert property.
4. Invoke Compile on "A_File [Local]" node.
5. Invoke My Submenu | Test command on multiselection of "A_File [Local]"
and "C_File [Local]" nodes.
6. Check "Prompt for Input on Each Successive File" checkbox and push "OK" button three times.
|
RESULT: There must be three "Test" tabs with "Ahoj !" and "Cau Martine!"
lines in "Standard Output" tabs of the output window. |
| 5.14 Failure notification |
1. Turn on "Do Not Warn of Failure" expert property.
2. Setup "Exec" property to cmd /x /c "ICQ #103430977" on Windows NT/2000 or sh -c "ICQ #103430977"
on other systems.
3. Turn off "Display Output" property.
4. Invoke My Submenu | Test command on "C_File [Local]" node.
5. Push "OK" button once "Hello! -" dialog appears.
|
RESULT: Status bar says: "Command Test finished.", no notification message appears, nothing gets printed
into standard output window of IDE. |
| 5.15 Status refreshing |
1. Setup "Exec" property to cmd /x /c "echo ${FILE}, Aktualni" on Windows NT/2000 or sh -c "echo ${FILE},
Aktualni" on other systems.
2. Setup "Data Regex" expert property to ^(.*), (.*) value.
3. Setup "File Index" refresh property to 0.
4. Setup "Status Index" refresh property to 1.
5. Invoke My Submenu | Test command on "C_File [Local]" node and push
"OK" button in "Hello! -" dialog.
|
RESULT: Status of C_File must change to "C_File [Aktualni]". |
| 5.16 Settings history |
1. Invoke My Submenu | Test command on "C_File [Aktualni]" node.
2. Type Jirka into the textfield, push "Set As Default" and then "OK" button.
3. Invoke that action again. Make sure that "Next" button is disabled and the textfield contains Jirka.
4. Type Maruska into the textfield and push "Get Default Values" button. Jirka must replace it.
5. Type Maruska into the textfield and push "Previous" button.
|
RESULT: Maruska must be replaced by Jirka again and "Next" button must get
enabled. Push "Cancel" button. |
6. Profiles management
Purpose: This suite tests possibility to create, customize and delete own profile.
Setup: No preparation required.
| Test case # |
Description |
6.1 Profile creation
|
1. Invoke Customize on your filesystem.
2. Push "Save As..." button.
3. Setup "myProfile" as "File Name".
4. Setup "My Profile" as "Profile Label" and push "Save" button.
|
RESULT: "My Profile" must be selected from the list. The checkbox below must be disabled.
Close the customizer. |
6.2 Profile usage
|
1. Invoke Versioning | Mount Version Control | Generic VCS item from main menu.
2. Select "My Profile" profile from the combo box.
|
RESULT: There must be "MyFile" textfield preset to defined value. |
6.3 Profile deletion
|
1. Push "Delete" button from the wizard.
2. Confirm your decision by clicking "Yes" button.
|
RESULT: The wizard must be reconfigured according to the first available profile and there must be no
"My Profile" profile in the list any more. |
7. Common settings
Purpose: This suite tests functionality of general options and filesystem properties.
Setup: No preparation required.
| Test case # |
Description |
7.1 Auto refresh
|
1. Invoke Tools | Options from the main menu.
2. Select "Source Creation and Management | Version Control Settings" node.
3. Setup "Auto Refresh" global option to "No Refresh" value.
4. Mount "D:\Tests\Empty\Work1" directory using PVCS profile.
5. Once the filesystem is mounted, expand it completely.
|
| RESULT: All nodes underneath stay [Local]. No "Refresh" must be executed and stored in the "VCS Commands" history. |
7.2 Home directory
|
1. Create "Home" command with execution string cmd /x /c "echo %HOMEDRIVE%%HOMEPATH%" for Windows or
sh -c "echo $HOME" for Unix systems.
2. Turn on "Display Output" property of that command.
3. Invoke new Home command on your filesystem.
|
RESULT: Value of "Home Directory" global option must be printed into "Standard Output"
tab of opened window. |
| 7.3 Offline mode |
1. Turn on "Offline Mode" global option.
2. Setup "Auto Refresh" global option to "Recursive on Mounting" value.
3. Mount "D:\Tests\Empty\Work1" directory using PVCS profile and expand it completely.
4. All nodes must stay [Local]. Invoke PVCS | Refresh action on your filesystem.
|
RESULT: You must be informed about active offline mode. Push "No" button. |
| 7.4 Override filesystem settings |
1. Turn off "Offline Mode" global option.
2. Make sure that the same property of filesystem mounted in previous test case is also switched off.
3. Turn off "Override filesystem settings" global option.
4. Turn on "Offline Mode" global option.
|
RESULT: "Offline Mode" property of used filesystem remains turned off. |
7.5 Annotation pattern
|
1. Setup "Annotation Pattern" property to ${fileName)$[?status][ >Status YES<][ >Status NO<] value.
2. Verify that all nodes must be in ">Status YES<" status.
3. Click the "..." button of "Annotation Pattern" filesystem property.
4. Select "If Defined: status | True | Text: >Status YES<" node.
5. Set its "Text" property to " [Up-to-date]" and push "Apply Changes" button.
|
| RESULT: All nodes must become [Up-to-date]. Close the editor. |
7.6 Command notification
|
1. Create "Print" command and set its "Notification Message After Success" property to "Great !".
2. Turn off "Command Notification" property of the filesystem.
3. Invoke your Print command on the filesystem.
|
RESULT: Status bar must say "Command Print finished." and no "Great !" message must show up. |
7.7 Ignored files
|
1. Setup "Ignored Files" expert property of the filesystem to A_File|B_File value.
|
RESULT: Both "A_File [Up-to-date]" and "B_File [Up-to-date]" nodes must disappear. |
7.8 Backup properties
|
1. Open "C_File [Up-to-date]" file, modify it and save it.
2. Turn off "Filter Backup Files" expert property of the filesystem.
3. "C_File.java~ [Up-to-date]" node must appear under "test [Up-to-date]" folder.
4. Turn off "Create Backup Files" expert property of the filesystem.
5. Repeat step 1 with "D_File [Up-to-date]" file.
|
RESULT: No additional file like "D_File.java~ [Up-to-date]" must appear under "another
[Up-to-date]" folder. |
| 7.9 Remember password |
1. Setup "Exec" property of "Print" command to cmd /x /c "echo ${PASSWORD}" for Windows NT/2000
and sh -c "echo ${PASSWORD}" for other systems.
2. Turn on "Remember Password" expert property of the filesystem.
3. Invoke Print action on the filesystem node.
4. Write some password and push "OK" button.
5. Restart IDE and repeat step 3.
|
| RESULT: No dialog requiring the password again must appear. |
| 7.10 Shorten file statuses |
1. Setup "POSSIBLE_FILE_STATUSES" accessory variable to "Current", "Missing", "Locally Modified", "Local".
2. Setup "POSSIBLE_FILE_STATUSES_LOCALIZED_SHORT" accessory variable to "Curr", "Miss", "LMod", "Loc".
3. Turn on "Shorten File Statuses" property of the filesystem.
|
RESULT: All visible nodes must get "[Loc]" status. |
7.11 Hide shadow files
|
1. Setup "Exec" property of "Print" command to cmd /x /c "echo F_File.java" for Windows NT/2000
and sh -c "echo F_File.java" for other systems.
2. Setup "Data Regex" expert property of the same command to ^(.*).
3. Setup "File Index" refresh property of the same command to 0.
4. Invoke Print command on the filesystem.
5. After a while "F_File.java [Miss]" must appear in the filesystem.
6. Turn on "Hide Shadow Files" expert property of the filesystem.
|
RESULT: The "F_File.java [Miss]" node must disappear from the filesystem after a while. |
7.12 Process all files
|
1. Compile "C_File [Up-to-date]" node.
2. Turn on "Process All Files" property of the filesystem.
3. Invoke PVCS | Lock command on "C_File [Up-to-date]" node.
4. Switch to "Runtime" tab of explorer and expand "VCS Commands" and "PVCS D:\Tests\Empty\Work1" filesystem nodes.
|
RESULT: There must be two "Lock" commands. Check both, one of them must have "Processed
Files" property set to "C_File.class". |
Example of tested directory structure:
 |
 |
 |
| Pic.1: Initial testing structure. |
Pic.2: Version controlled testing structure. |
Pic.3: Revisions history dialog. |
Any comments and suggestions please send to
Jiri Kovalsky.
