Friday, August 7, 2009

Software Testing

What is Software Testing?

In general, testing is finding out how well something works. In terms of human beings, testing tells what level of knowledge or skill has been acquired. In computer hardware and software development, testing is used at key checkpoints in the overall process to determine whether objectives are being met. For example, in software development, product objectives are sometimes tested by product user representatives. When the design is complete, coding follows and the finished code is then tested at the unit or module level by each programmer; at the component level by the group of programmers involved; and at the system level when all components are combined together. At early or late stages, a product or service may also be tested for usability. 

At the system level, the manufacturer or independent reviewer may subject a product or service to one or more performance tests, possibly using one or more benchmarks. Whether viewed as a product or a service or both, a Web site can also be tested in various ways - by observing user experiences, by asking questions of users, by timing the flow through specific usage scenarios, and by comparing it with other sites.

The answer to the primary role of software testing is two fold:

  • Determine whether the system meets specifications (Producer View), and

  • Determine whether the system meets business and user needs (Customer View)

Testing encompasses three concepts:

  1. The demonstration of the validity of the software at each stage in the system development life cycle.

  2. Determination of the validity of the final system with respect to user needs and requirements.

  3. Examination of the behavior of a system by executing the system on sample test data.




Software Metrics

METRICS

Although general set of standards has been agreed upon, the appropriate metrics to test how well software meets those standards are still poorly defined. Publications by IEEE, have presented numerous potential metrics that can be used to test each attribute. Different metrics may give different rank ordering of the same attribute, making comparisons across products difficult and uncertain.

List of Metrics available

Fault Density

Software purity level

Defect density

Estimated number of faults remaining (by seeding)

Cumulative failure profile

Requirements compliance

Fault days number

Test coverage

Functional or modular test coverage

Data or information flow complexity

Cause and effect graphing

Reliability growth function

Requirements traceability

Residual fault count

Defect indices

Failure analysis elapsed time

Error distribution (s)

Testing sufficiently

Software maturity index

Mean time to failure

Person-hours per major defect detected

Failure rate

Number of conflicting requirements

Software documentation and source listing

Software science measures

Software release readiness

Graph-theoretic complexity

Completeness

Cyclomatic Complexity

Test accuracy

Minimal unit test case determination

System performance reliability

Run reliability

Independent process reliability

Design structure

Combined hardware and software (system) availability

Mean time to discover the next K-faults

 

What makes good Metrics?

This is a very interesting question. Many people say that anything we do should be quantitative. Yes, that's absolutely correct. Working in a project, there should be certain measurement criteria for measuring the skills and productivity of every individual in the team. Metrics play an important role in giving a good analysis. Every organization should make a list of metrics applicable in the organization and each employee should be aware of it.

Now let us look at few attributes, which make up good metrics:

  • The metric should be simple and computable so that its easy and straight forward.

  • The metric should be measuring the correct attribute.

  • The metric results should be reproducible.

  • The metric should be consistent in units or dimensions.

  • The metric should be independent of programming language.

  • The metric should provide good and meaningful analysis.

what can be measured with Metrics?

Metrics can measure anything which is quantifiable, like:

  • Effort, time and expenditure in each stage of the project.

  • Number of functionalities implemented.

  • Number of errors reported.

  • Types of errors reported.

  • Project schedules.

  • Benchmark important milestones in the project.




Monday, July 20, 2009

Wednesday, July 15, 2009

Defect Tracking Template

Test Data For Time Field

  • 00:00:00

  • 00:00:01

  • 01:01:01

  • 11:59:59

  • 12:00:00

  • 12:01:01

  • 23:59:59

  • (now)

  • (1 second ago)

  • (1 second from now)

  • (1 minute ago)

  • (1 mnute from now)

  • (1 hour ago)

  • (1 hour from now)

  • 25:01:01

  • 01:60:01

  • 01:60:01

  • 01:01:60

  • 01:01:61

  • 01::01

  • 01::

  • 01:01:xx

  • 01:xx:01

  • xx:01:01

Test Data For Date Fields

  • 0/0/0000

  • (yesterday)

  • (today)

  • (today with all leading zeros, like 01/01/2006) 

  • (tomorrow)

  • (same day last week)

  • (same day next week)

  • (same day last month)

  • (same day next month)

  • (same day last year)

  • (same day next year)

  • (Jan 1 this year)

  • (Dec 31 this year) 

  • (Jan 1 last year)

  • (Dec 31 last year) 

  • (Jan 1 next year)

  • (Dec 31 next year) 

  • 1/1/1999

  • 12/31/1999

  • 1/1/2000

  • 2/28/2000

  • 2/29/2000

  • 2/30/2000

  • 2/31/2000 

  • 3/1/2000

  • 12/31/2000

  • 1/1/2001

  • 1/1/202 

  • 2/28/2004

  • 2/29/2004

  • 3/1/2004

  • 12/31/2004

  • 9/9/9999

  • 2/29/2001 

  • 4/31/2001

  • 9/31/2001 

  • 2/29/2100

  • 13/1/2006

  • 00/1/2006

  • 01/32/2006

  • 01/00/2006

  •  June 5, 2001
  • 06/05/2001

  • 6/5/2001

  • 06/05/01

  • 6/5/01

  • 06-05-01

  • 06-05-2001

  • 6-5-01

  • 6-5-2001

Test Steps for Check Box Field

  • Checked

  • Unchecked

Test Steps for Radio Button Field

  • On

  • Off

  • All buttons in a group Off

  • One button in a group On

  • More than one button in a group On

  • One button in a group Off

  • All buttons in a group On

Test Steps for List Box

  • None selected

  • Top item selected

  • Second item selected

  • Middle item selected

  • Next to bottom item selected

  • Bottom item selected

  • Top and bottom items selected

  • Several adjoining items selected

  • Several non-adjoining items selected

  • All items selected

  • All items except the top item selected

  • All items except the bottom item selected

  • All items except one middle item selected

Test Input Data for an Integer Field

  • Nothing

  • Empty field (clear any defaults)

  • More than the maximum number of digits or characters

  • 0

  • Any valid value

  • Minimum - 1

  • Minimum

  • Maximum

  • Maximum + 1

  • Much less than the Minimum

  • Much more than the Maximum

  • Minimum number of digits or characters

  • Minimum - 1 number of digits or characters

  • Maximum number of digits or characters

  • Maximum + 1 number of digits or characters

  • Much more than the Maximum number of digits or characters

  • A negative value

  • Nondigits, expecially / (ASCII character 47) and : (ASCII character 58)

  • Wrong data type (e.g., decimal into integer)

  • Expressions

  • A single leading space

  • Many leading spaces

  • A single trailing space

  • Many trailing spaces

  • Lead and trailing spaces

  • Leading 0

  • Many leading zeros

  • Leading + sign

  • Many leading + signs

  • Nonprinting character (e.g., Ctrl+char)

  • Operating system filename reserved characters (e.g., "*.:")

  • Language-specific reserved characters

  • Upper ASCII (128-254) (a.k.a. ANSI) characters

  • ASCII 255 (often interpreted as end of file)

  • Uppercase characters

  • Lowercase characters

  • Modifiers (e.g., Ctrl, Alt, Shift-Ctrl, and so on)

  • Function key (F2, F3, F5, and so on)

  • 254

  • 255 (1-byte, tinyint)

  • 256

  • 32766

  • 32767 (2-byte, smallint)

  • 32768

  • 65534

  • 65535 (2-byte unsigned, smallint)

  • 65536

  • 2147483646

  • 2147483647 (4 byte, int)

  • 2147483648

  • 4294967294

  • 4294967295 (4-byte unsigned, int)

  • 4294967296

  • 9223372036854775806

  • 9223372036854775807 (8 byte, bigint)

  • 9223372036854775808

  • 18446744073709551614

  • 18446744073709551615 (8 byte unsigned, bigint)

  • 18446744073709551616

The First Bug Report

Test Data – For Unit validation

Alphabhatic Characters

abcdefghijklmnopqrstuvwxyz

ABCDEFGHIJKLMNOPQRSTUVWXYZ

Special Characters

~!@#$%^&*()_+{}|:"<>?`-=[];',./

10 Characters
1234567890

123456789+

1234567.89

12345.6789

$123456.78

$12,345.67

12,345,678

123,456.78

abcdefghi+

abcdefghiX

ABCDEFGHI+

ABCDEFGHIx

WWWWWWWWWZ

~!@#$%^&*X

100 Characters
123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789+

1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890

123456789+123456789+123456789+123456789+123456789+123456789+123456789+123456789+123456789+123456789+

abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstu+

abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuZ

ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEF

GHIJKLMNOPQRSTU+

ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEF

GHIJKLMNOPQRSTUz

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW

WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWZ

 ~!@#$%^&*()_+{}|:"<>?`-=[];',./~!@#$%^&*()_+{}|:"<>?`-=[];',./~!@#$%^&*()_+{}|:"<>?`-=[];',./~!@#$%X

 

Thursday, July 9, 2009

Test Data Country Codes


Country names 2 Digit Code Country names 2 Digit Code
A   M  
AFGHANISTAN AF MACAO MO
ÅLAND ISLANDS AX MACEDONIA, THE FORMER YUGOSLAV REPUBLIC OF MK
ALBANIA AL MADAGASCAR MG
ALGERIA DZ MALAWI MW
AMERICAN SAMOA AS MALAYSIA MY
ANDORRA AD MALDIVES MV
ANGOLA AO MALI ML
ANGUILLA AI MALTA MT
ANTARCTICA AQ MARSHALL ISLANDS MH
ANTIGUA AND BARBUDA AG MARTINIQUE MQ
ARGENTINA AR MAURITANIA MR
ARMENIA AM MAURITIUS MU
ARUBA AW MAYOTTE YT
AUSTRALIA AU MEXICO MX
AUSTRIA AT MICRONESIA, FEDERATED STATES OF FM
AZERBAIJAN AZ MOLDOVA, REPUBLIC OF MD
B   MONACO MC
BAHAMAS BS MONGOLIA MN
BAHRAIN BH MONTENEGRO ME
BANGLADESH BD MONTSERRAT MS
BARBADOS BB MOROCCO MA
BELARUS BY MOZAMBIQUE MZ
BELGIUM BE MYANMAR MM
BELIZE BZ N  
BENIN BJ NAMIBIA NA
BERMUDA BM NAURU NR
BHUTAN BT NEPAL NP
BOLIVIA, PLURINATIONAL STATE OF BO NETHERLANDS NL
BOSNIA AND HERZEGOVINA BA NETHERLANDS ANTILLES AN
BOTSWANA BW NEW CALEDONIA NC
BOUVET ISLAND BV NEW ZEALAND NZ
BRAZIL BR NICARAGUA NI
BRITISH INDIAN OCEAN TERRITORY IO NIGER NE
BRUNEI DARUSSALAM BN NIGERIA NG
BULGARIA BG NIUE NU
BURKINA FASO BF NORFOLK ISLAND NF
BURUNDI BI NORTHERN MARIANA ISLANDS MP
C   NORWAY NO
CAMBODIA KH O  
CAMEROON CM OMAN OM
CANADA CA P  
CAPE VERDE CV PAKISTAN PK
CAYMAN ISLANDS KY PALAU PW
CENTRAL AFRICAN REPUBLIC CF PALESTINIAN TERRITORY, OCCUPIED PS
CHAD TD PANAMA PA
CHILE CL PAPUA NEW GUINEA PG
CHINA CN PARAGUAY PY
CHRISTMAS ISLAND CX PERU PE
COCOS (KEELING) ISLANDS CC PHILIPPINES PH
COLOMBIA CO PITCAIRN PN
COMOROS KM POLAND PL
CONGO CG PORTUGAL PT
CONGO, THE DEMOCRATIC REPUBLIC OF THE CD PUERTO RICO PR
COOK ISLANDS CK Q  
COSTA RICA CR QATAR QA
CÔTE D'IVOIRE CI R
CROATIA HR RÉUNION RE
CUBA CU ROMANIA RO
CYPRUS CY RUSSIAN FEDERATION RU
CZECH REPUBLIC CZ RWANDA RW
D   S  
DENMARK DK SAINT BARTHÉLEMY BL
DJIBOUTI DJ SAINT HELENA SH
DOMINICA DM SAINT KITTS AND NEVIS KN
DOMINICAN REPUBLIC DO SAINT LUCIA LC
E   SAINT MARTIN MF
ECUADOR EC SAINT PIERRE AND MIQUELON PM
EGYPT EG SAINT VINCENT AND THE GRENADINES VC
EL SALVADOR SV SAMOA WS
EQUATORIAL GUINEA GQ SAN MARINO SM
ERITREA ER SAO TOME AND PRINCIPE ST
ESTONIA EE SAUDI ARABIA SA
ETHIOPIA ET SENEGAL SN
F   SERBIA RS
FALKLAND ISLANDS (MALVINAS) FK SEYCHELLES SC
FAROE ISLANDS FO SIERRA LEONE SL
FIJI FJ SINGAPORE SG
FINLAND FI SLOVAKIA SK
FRANCE FR SLOVENIA SI
FRENCH GUIANA GF SOLOMON ISLANDS SB
FRENCH POLYNESIA PF SOMALIA SO
FRENCH SOUTHERN TERRITORIES TF SOUTH AFRICA ZA
G   SOUTH GEORGIA AND THE SOUTH SANDWICH ISLANDS GS
GABON GA SPAIN ES
GAMBIA GM SRI LANKA LK
GEORGIA GE SUDAN SD
GERMANY DE SURINAME SR
GHANA GH SVALBARD AND JAN MAYEN SJ
GIBRALTAR GI SWAZILAND SZ
GREECE GR SWEDEN SE
GREENLAND GL SWITZERLAND CH
GRENADA GD SYRIAN ARAB REPUBLIC SY
GUADELOUPE GP T  
GUAM GU TAIWAN, PROVINCE OF CHINA TW
GUATEMALA GT TAJIKISTAN TJ
GUERNSEY GG TANZANIA, UNITED REPUBLIC OF TZ
GUINEA GN THAILAND TH
GUINEA-BISSAU GW TIMOR-LESTE TL
GUYANA GY TOGO TG
H   TOKELAU TK
HAITI HT TONGA TO
HEARD ISLAND AND MCDONALD ISLANDS HM TRINIDAD AND TOBAGO TT
HOLY SEE (VATICAN CITY STATE) VA TUNISIA TN
HONDURAS HN TURKEY TR
HONG KONG HK TURKMENISTAN TM
HUNGARY HU TURKS AND CAICOS ISLANDS TC
I   TUVALU TV
ICELAND IS U  
INDIA IN UGANDA UG
INDONESIA ID UKRAINE UA
IRAN, ISLAMIC REPUBLIC OF IR UNITED ARAB EMIRATES AE
IRAQ IQ UNITED KINGDOM GB
IRELAND IE UNITED STATES US
ISLE OF MAN IM UNITED STATES MINOR OUTLYING ISLANDS UM
ISRAEL IL URUGUAY UY
ITALY IT UZBEKISTAN UZ
J   V  
JAMAICA JM VANUATU VU
JAPAN JP VATICAN CITY STATE VA
JERSEY JE VENEZUELA, BOLIVARIAN REPUBLIC OF VE
JORDAN JO VIET NAM VN
K   VIRGIN ISLANDS, BRITISH VG
KAZAKHSTAN KZ VIRGIN ISLANDS, U.S. VI
KENYA KE W  
KIRIBATI KI WALLIS AND FUTUNA WF
KOREA, DEMOCRATIC PEOPLE'S REPUBLIC OF KP WESTERN SAHARA EH
KOREA, REPUBLIC OF KR Y  
KUWAIT KW YEMEN YE
KYRGYZSTAN KG Z  
L   ZAMBIA ZM
LAO PEOPLE'S DEMOCRATIC REPUBLIC LA ZIMBABWE ZW
LATVIA LV

LEBANON LB

LESOTHO LS

LIBERIA LR

LIBYAN ARAB JAMAHIRIYA LY

LIECHTENSTEIN LI

LITHUANIA LT

LUXEMBOURG LU