When it comes to telephones, everybody has certain expectations.

Nobody wants a phone that fails occasionally. And most service providers have a model of phone that they trust; nobody wants to try out a new SIP phone and learn that it fails in an obscure, but important feature.

For example: Have you seen the new Yealinks? Or what about the improved Grandstreams?   How do you ensure the phone is going to do everything you need it to do -- like those unusual things a receptionist does, that a technician or engineer never does?

Below  you can find a set of 152 Test Criteria (TC's) I've used to test new models of SIP phones. If you develop test procedures to confirm each of these criteria are met, you'll have a good idea that the SIP phone is going to do all the PBX functions for a SIP phone.

Using these criteria requires you to have a general understanding of Service Provider VoIP. To develop a test procedure to confirm TC1, "Proper Registration via SBC." The value of this list is to help you remember things you might have forgotten otherwise, and expedite testing procedures.

Mark's 152 Test Criteria for SIP Phones

TC1 : Proper Registration via SBC

TC2 : Symmetric signaling (SIP to and from the same UDP port) and media (RTP to and from the same port)

TC3 : PSTN to SIP call, calling party disconnects first

TC4 : PSTN to SIP call, called party disconnects first

TC5 : SIP to PSTN call, calling party disconnects first

TC6 : PSTN to SIP call, called party disconnects first

TC7 : SIP to SIP call for each supported model or software version (e.g., Aastra 57i/2.2.0 calling to PolyCom SPIP 601)

TC8 : Calling-Party Name and Number display for "on-net" VoIP-CPE-to-VoIP-CPE calls

TC9 : Calling-Party Name and Number display for calls received from each PSTN carrier

TC96 : Called Party name displays on intra-group calls and calls to group-call-park

TC29 : Auto-answer for click-to-dial calls (i.e., supports SIP INVITE with alert to make calling phone automatically pick up)

TC39 : Calls sent to the DUT but not answered

TC40 : Calls placed by DUT but not answered

TC41 : DUT installed behind firewall with NAT

TC42 : DUT installed behind firewall with no NAT

TC43 : SIP over TCP (for devices that support it)

TC103 : Call Forward Always using key on phone and signaling on phone

TC51 : Call Forward Always using key on phone, synchronized with BroadWorks/Metaswitch/Feature Server settings

TC104 : Do-Not-Disturb using key on phone and signaling on phone

TC52 : Do-Not-Disturb using key on phone, synchronized with BroadWorks/Metaswitch/Feature Server settings

TC53 : DUT can be configured match Service Provider's standard digit map (aka "dial plan")

TC106 : DUT can dial calls the same whether the phone is off hook or on hook

TC84 : DUT re-registers a 5-15 seconds before the old registration expires.

TC10 : Call hold and retrieve using hold button or soft key on DUT

TC11 : Remote-party call hold and retrieve (i.e., the remote end of the conversation puts the call on hold, not the DUT)

TC12 : Three-way calling

TC13 : Three-way call disposition when DUT disconnects from call

TC14 : Blind transfer

TC15 : Consultative transfer

TC16 : Shared Call Appearance (SCA)

TC17 : Busy Lamp Field (BLF)

TC18 : Multiple lines configured on one telephone

TC19 : Proper NTP synchronization of clock display

TC20 : Long-call-duration calls

TC21 : Call Forward Always using key on phone

TC22 : Do-Not-Disturb using key on phone

TC23 : Multiple buttons configured for the same line

TC24 : DTMF transfer to Media Server, Conferencing Server, Auto Attendant server, etc.

TC25 : DTMF transfer to PSTN via PSTN gateway(s)

TC57 : PLAR / Automatic Ring-Down: DUT can be configured to automatically dial a number when the handset is lifted

TC67 : DUT should support up to 6 inbound calls stacked up (ringing inbound to phone) and function properly

TC68 : When you have lots of inbound calls stacked up, you can put a call on hold and park the most-recently-active call

TC69 : When call parked by the DUT reverts to the DUT phone, the DUT phone can retrieve the call properly

TC111 : When DUT transfers a call, and the Call Transfer Recall timer reverts the call back to DUT, it can retrieve the call properly

TC70 : DUT can call into a 10-line paging group and automatically announce to those lines

TC71 : DUT can be a member of a 10-line paging group, so that when the group is called it automatically goes on speaker and announces what the caller says

TC72 : (Push-to-Talk) Hoot-and-Holler call outbound: DUT can call to another phone and have it go off hook on speakerphone

TC73 : (Push-to-Talk) Hoot-and-Holler call inbound: DUT can receive an inbound call and have it go off hook on speakerphone

TC110 : Auto-answer "Push-to-talk" calls and sets up two-way conversation

TC85 : DUT supports Broadworks/Metaswitch N-Way conferencing (Low-priority test).

TC86 : DUT can put a call on hold and retrieve it 5 times, and have two-way audio each time the call is un-held.

TC87 : DUT can be on a call where the remote party puts the call on hold and retrieves is 5 times, and have two-way audio each time the call is un-held.

TC112 : DUT works properly with BroadWorks/Metaswitch Group Call Park

TC88 : DUT operates as any party in any call transfer scenario

TC88.1 : Blind transfer, original recipient is facilitator

TC88.1.a : Blind transfer, original recipient is facilitator, DUT is originator

TC88.1.b : Blind transfer, original recipient is facilitator, DUT is original recipient

TC88.1.c : Blind transfer, original recipient is facilitator, DUT is final recipient

TC88.2 : Blind transfer, originator is facilitator

TC88.2.a : Blind transfer, originator is facilitator, DUT is originator

TC88.2.b : Blind transfer, originator is facilitator, DUT is original recipient

TC88.2.c : Blind transfer, originator is facilitator, DUT is final recipient

TC88.3 : Blind transfer before call answer, original recipient is facilitator

TC88.3.a : Blind transfer before call answer, original recipient is facilitator, DUT is originator

TC88.3.b : Blind transfer before call answer, original recipient is facilitator, DUT is original recipient

TC88.3.c : Blind transfer before call answer, original recipient is facilitator, DUT is final recipient

TC88.4 : Blind transfer before call answer, originator is facilitator

TC88.4.a : Blind transfer before call answer, originator is facilitator, DUT is originator

TC88.4.b : Blind transfer before call answer, originator is facilitator, DUT is original recipient

TC88.4.c : Blind transfer before call answer, originator is facilitator, DUT is final recipient

TC88.5 : Attended transfer, original recipient is facilitator

TC88.5.a : Attended transfer, original recipient is facilitator, DUT is originator

TC88.5.b : Attended transfer, original recipient is facilitator, DUT is original recipient

TC88.5.c : Attended transfer, original recipient is facilitator, DUT is final recipient

TC88.6 : Attended transfer, originator is facilitator

TC88.6.a : Attended transfer, originator is facilitator, DUT is originator

TC88.6.b : Attended transfer, originator is facilitator, DUT is original recipient

TC88.6.c : Attended transfer, originator is facilitator, DUT is final recipient

TC88.7 : Busy-Failed Blind transfer, original recipient is facilitator

TC88.7.a : Busy-Failed Blind transfer, original recipient is facilitator, DUT is originator

TC88.7.b : Busy-Failed Blind transfer, original recipient is facilitator, DUT is original recipient

TC88.7.c : Busy-Failed Blind transfer, original recipient is facilitator, DUT is final recipient

TC88.8 : Busy-Failed Blind transfer, originator is facilitator

TC88.8.a : Busy-Failed Blind transfer, originator is facilitator, DUT is originator

TC88.8.b : Busy-Failed Blind transfer, originator is facilitator, DUT is original recipient

TC88.8.c : Busy-Failed Blind transfer, originator is facilitator, DUT is final recipient

TC88.9 : No-Answer-Failed Blind transfer, original recipient is facilitator

TC88.9.a : No-Answer-Failed Blind transfer, original recipient is facilitator, DUT is originator

TC88.9.b : No-Answer-Failed Blind transfer, original recipient is facilitator, DUT is original recipient

TC88.9.c : No-Answer-Failed Blind transfer, original recipient is facilitator, DUT is final recipient

TC88.10 : No-Answer-Failed Blind transfer, originator is facilitator, DUT is originator

TC88.10.a : No-Answer-Failed Blind transfer, originator is facilitator, DUT is originator

TC88.10.b : No-Answer-Failed Blind transfer, originator is facilitator, DUT is original recipient

TC88.10.c : No-Answer-Failed Blind transfer, originator is facilitator, DUT is final recipient

TC44 : DUT downloads via FTP/TFTP/HTTP/HTTPS and ability to detect and download new configurations

TC49 : Request IP address via DHCP upon phone startup

TC50 : Request IP address after lease expiration or after loss of Ethernet connectivity without power-cycling phone

TC46 : Detect voice and VLAN tags automatically (e.g., using CDP or 802.1AB/LLDP)

TC47 : 802.1q VLAN tagging on "LAN" port

TC48 : Function as Ethernet switch or hub for PC port. The DUT should switch data-VLAN traffic to the PC port, removing the VLAN tag. It should also switch frames received on the PC port to the LAN port, adding the appropriate VLAN tag.

TC54 : When using PoE, DUT should request a reasonable amount of power

TC56 : The switch in the phone should pass through untagged frame (i.e., any frame with no 802.1q vlan tag) from between the PC port and the LAN port to support the PC/access VLAN being untagged while the voice VLAN is tagged.

TC59 : DUT can locate the SIP server using SRV records.

TC60 : Locate the SIP server using A-record lookups for an outbound proxy.

TC64 : When using a wireless handset, it should work in the presence of moderate or heavy 802.11 traffic

TC74 : DUT can be rebooted remotely from the Reset function in BroadWorks/Metaswitch

TC75 : DUT creates logs of call activities and events

TC76 : DUT logs call statistics (packet loss, jitter, codec, etc.)

TC77 : DUT can upload its log to a server using FTP

TC78 : DUT with built-in switch can support wire-rate 100 Mbps for a PC plugged in behind the phone

TC79 : DUT with built-in switch works OK if device behind PC sends wake-on-LAN packets when it goes to sleep or wakes from sleep

TC80 : DUT works when connected to a network with 10% broadcast or multicast traffic loads

TC81 : DUT does not have a significantly-increased CPU load level in presence of 10% broadcast or multicast traffic loads (low-priority test: do this last)

TC82 : DUT works when connected to a standard customer-premise network switch configuration

TC89 : DUT can automatically detect when new firmware is available (by polling the server to check for new software), then selectively download and upgrade itself only when new software is available.

TC90 : DUT downloads new firmware using authenticated FTP.

TC93 : DUT can download new configurations and configurations via FTP using the username and password specified with DHCP response Option 66

TC94 : DUT can download new configurations via HTTPS using the username and password specified with DHCP response Option 66

TC105 : A brand-new-out-of-the-box DUT can be upgraded automatically, without manually logging into the DUT and triggering the upgrade

TC108 : DUT should work with slow FTP servers

TC107 : If DUT is unable to connect to the Configuration Server (FTP/HTTP/etc. server), it should use the configuration it last downloaded.

TC26 : RFC2833 DTMF send

TC27 : G.729ab and G.711u codec support

TC28 : 20 ms packetization time

TC30 : Ringback on outbound calls to PSTN

TC31 : Ringback on outbound calls to all other supported SIP devices

TC32 : Early media playback for calls to PSTN (e.g., for calls sent to announcement by PSTN but never connected)

TC33 : Early media playback for local calls (e.g., for calls to Sequential Ring in BroadWorks or Custom Ringback in other platforms)

TC34 : Proper handset, speakerphone, and external-headset operation

TC62 : TOS or DSCP value can be configured for signaling and for RTP.

TC63 : DUT generates RTCP for all RTP sessions.

TC95 : After SDP offer/answer, DUT uses its own preferred codec rather than the most-preferred codec in the SDP answer.

TC97 : DUT that supports G.729 can be configured to disable silence suppression

TC109 : DUT should support Multicast Paging that can be used with BroadWorks/Metaswitch

TC91 : DUT audio quality in both directions is OK in the presence of moderate to extreme electromagnetic interference (EMI)

TC35 : Acoustic echo cancellation or mitigation on handset and speakerphone

TC36 : Gain control for handset and speakerphone (to reduce talker loudness)

TC37 : Volume control to change loudness of remote party

TC38 : Proper and Working mute and un-mute operation

TC102 : DUT will cancel out far-end echo so the user of the DUT doesn't hear himself.

TC45 : Change call to use external headset

TC55 : If there are more softkey options available than there are softkeys, then DUT should show a "More..." button to get to the other softkey options.

TC58 : DUT can access and use vendor's demo XML applications via the Internet when the URL is specified as a hostname and not an IP address.

TC61 : Use one default Services Application on phones for which no device-specific configuration has been generated, and a different default Services Application on phones for which a device configuration has been generated.

TC65 : If DUT supports a sidecar, the buttons can be assigned to new lines or programmable as softkey actions, including dialing BroadWorks/Metaswitch feature codes

TC66 : If DUT supports a sidecar, the buttons behave like buttons built-in to the phone.

TC83 : If DUT works with two sidecars, any button on any sidecar should be assignable to line registrations or programmable hotkeys on the sidecars.

TC92 : Softkeys should be programmable to access a URL, speed dial a number or Broadworks/Metaswitch feature code.

TC96 : DUT should be configurable to strip leading digits in the dial plan.

TC99 : DUT can dial feature codes that begin with a pound character; e.g., "#58"

TC100 : DUT can configure BroadWorks/Metaswitch call park and call-retrieve to softkeys

TC101 : DUT can be configured to disable the Do-Not-Disturb button on the phone

Beyond the 152

Of course, some platforms are going to offer more features. So be on the lookout for:

• Video Conferencing
• Video Calling and  Video Voicemail
• Call Center Login/Logout
• User-Activated Centralized Call Recording (e.g., SIPREC activation)
• Phone-Based Call Recording