WiMax Core Integration Team Testing Procedures for Golden Markets

Prepared By: Engineers: Edgar Cabotega and Elsa Lam Project Manager: Marsha Kippes Version Date: July 2011

Table of Contents Introduction RF Parameters Golden Markets Golden Devices Sample Report Test Cases For USB (test cases 1-8) Smartphone (test case 9) Hotspots (test cases 10-13)

Introduction The objective of this document is to provide a test plan for WiMax Golden Market s. Golden markets refer to predefined sites where WiMax tests will be conducted. Each site represents a different hardware configuration of Base Station and ASN /Gateway, i.e. Motorola, Samsung, Huawei and WiChorus. The results of these test s will provide general measures of the WiMax Network Health in terms of Throughp ut, Network delay and efficiency and Handoff Latency. The document will provide test procedures using defined golden devices that cove r multiple types, tools required to conduct tests, and spreadsheet for record ke eping of test results.

RF Environment Parameters WiMax Parameters RF Environment RSSI CINR Good -65dBm 22dB Fair -65dBm to -75dBm 22dB to Poor -75dBm to -85dBm 16dB to CDMA Parameters RF Environment RSSI CINR Good -75dBm 10dB Fair -75dBm to -85dBm 10dB to Poor -85dBm to -90dBm 2dB

16dB 10dB

2dB

Golden Markets RAN Core/GW Market Name Market HA Address Huawei WiChorus Cleveland, Ohio 66.1.32.196 Motorola Motorola Richmond. VA 68.28.241.92 Samsung WiChorus Pittsburg, PA 68.28.113.92 Samsung Samsung (ACR) Washington DC area 68.28.121.92 Golden Devices to be used for testing USB U250 U300 U600 U602 Smartphones Epic EVO 3D Nexus S

GED Laptop Hotspots Andros 11z Netbook W802S Overdrive Pro (Owl) Indian MiFi M2M Digi or InMotion Fixed Modems BWX360 ODU (Cisco) Greenpacket

Sample Output/Checklist Report

Test Case 1 using USB Device Test Case: 1 using USB Title: Network Entry (SSV) SSV Device:

Description: Device will connect to the 4G WiMAX network in 10 15 seconds from h itting the connect button to connected state. Test output is captured for reportin g. Iteration: Perform this test procedure 5 times at each of RF condition: Good, Fair, Poor Tool: Use Microsoft clock under Date and Time Properties Measurement: Time (Second) Collect the Required Data: Collect in Excel format under different RF condition s (Good, Fair, Poor) every time the device successfully connects to WiMAX upon n etwork entry. Procedure: 1. Launch Windows clock utility a. Double-click on the window toolbar clock b. Ensure the second counter is visible 2. Click-on the WiMAX icon on the SSV window to initiate network connectivi ty. Wait for connected message to appear. 3. Note the time, in seconds, between the click connect button and the time the Connected establish message appears on the SSV window. 4. Verify using the web browser, establish HTTP connection to www.google.co m.

Test Case 2 using USB Device Test Case: 2 using USB Title: Device to HA Round-trip Delay SSV Device:

Description: Device connected to the 4G WiMAX network, measure the latency betwe en the laptop test device and Home Agent locations. Test output is captured for reporting. Iteration: Perform this test procedure at each of RF condition: Good, Fair, Po or Tool: Use Microsoft Command prompt or Utopia tools Measurement: Time (Millisecond) Collect the Required Data: Collect in excel format under different RF condition s (Good, Fair, Poor) when the device successfully ping the HA (Home Agent) IP Ad

dress as shown on the procedure. Capture the Utopia test results if using Utopi a tool. Procedure: 1. Click-on the WiMAX icon on the SSV window to initiate network connection a. Wait for connection to be established. 2. Verify the connection has been established using the web browser, establ ish HTTP connection to www.google.com. 3. Launch the Windows command prompt a. Select start, run from the windows main toolbar b. Enter cmd in the dialog box and click ok 4. C:\ tracert www.sprint.com. 5. From Tracert result, record the <ip address> on line #2 and #3. 6. In the windows command line UI, ping and capture output a. Enter: ping n 100 l 200 <IP Address#2> > <ping#2.txt> b. Enter: ping n 100 l 200 <IP Address#3> > <ping#3.txt> 7. Use the Utopia tool to run a Ping to HA IP address.

Test Case 3 using USB Device Test Case: 3 using USB Title: Round Trip Delay Correlation on Home Agent vs. Mu ltiple Sites SSV Device: Description: Device connected to the 4G WiMAX network, measure the latency betwe en the laptop test device and multiple sites. Test output is captured for report ing. Capture IP Config and trace route Iteration: Perform this test procedure at each of RF condition: Good, Fair, Po or Tool: Use SolarWinds Toolset, Enhanced Ping, Utopia tool or Command Prompt Measurement: Time (Millisecond) Collect the Required Data: Collect in excel format under different RF condition s (Good, Fair, Poor) when the device successfully ping the HA (Home Agent) IP Ad dress and external URLs simultaneously. Capture the multiple URLs Ping using Utop ia tool or either capture using SolarWinds, Enhanced Ping. Capture the Trace Route from Windows command prompt tracert www.sprint.com and save to a file including t he ipconfig /all. Procedure: 1. Click-on the WiMAX icon on the SSV window to initiate network connection a. Wait for connection to be established. 2. Verify the connection using the web browser, establish HTTP connection t o www.google.com. 3. Perform simultaneous Ping to Multiple URLs using Utopia. 4. If you preferred SolarWinds, Enhanced Ping perform simultaneous Ping to mu ltiple URLs including the HA IP address. 5. The objective is to ping multiple URLs at the same time and correlate th e results in Time (Delay) vs Time (Duration of test).

Example of Multi Ping test result for test case 3

Test Case 4 using USB Device Test Case: 4 using USB Title: Hand-Off (Inter Base Station) SSV Device:

Description: Verify the ability to hand-off between base stations and latency du ring hand-off in pre-selected route. Use the Microsoft Street maps to plot your driving directions over the targeted towers to perform the handoff. Test output is captured for reporting. Iteration: Perform this test procedure once through selected drive route. Tool: Use XCAL tool to capture Handover latencies and Microsoft command prompt t o run simultaneous Ping. Use Microsoft Streets & Trips tool to plot in maps the dr iving directions over the targeted towers to perform the inter handoff. Use Solar Winds, Enhanced Ping to capture multiple pings. Measurement: Time (Millisecond) Collect the Required Data: Capture the Microsoft Street & Trips map snapshot of d riving directions to perform inter handoff test. Using XCAL tool capture the Thr oughput behavior during handoff and correlate the results vs. CINR and RSSI. Cap ture device ping to HA (Home Agent) IP Address and external URLs simultaneously w hile performing the inter Base Station handover Procedure: 1. Locate the selected route. At the Start point of route, park your vehic le. 2. Launch XCAL. Select tab to provide CINR, RSSI, DownLink and UpLink chart in example below. Start logging and save filename Inter BS_Market Name_Route. 3. Click on the WiMAX icon on the SSV window to initiate network connection . 4. Verify web browser, establish HTTP connection to www.google.com. 5. Run Windows Command prompt: C: > ping t www.google.com > InterHO_Ping.txt . 6. Skip Step 5, if you have SolarWinds, Enhanced Ping, perform simultaneous P ing to multiple URLs 7. Run multiple FTP DL and UL 8. Start drive through selected route. 9. Stop logging at the end of the route. 10. Stop FTP DL and UL scripts. 11. Repeat for multiple ping (use Test Case 3 procedure) and drive through s elected route.

Example of Microsoft Streets and Trips for test case 4.

Example of XCAL Results for test case 4.

Example of Multi Inter BS Handover Results for test case 4.

Test Case 5 using USB Device Test Case: 5 using USB Title: Hand-Off (Intra Base Station) SSV Device:

Description: Verify the ability to hand-off between sectors within the same base stations that are homed to the same ASN-GateWay while maintaining an active dat a session. Use the Microsoft Street maps to plot your driving directions over th e targeted towers to perform the handoff. Iteration: Perform this test procedure once through selected drive route. If di sconnected repeat. Tool: Use XCAL tool to capture Handover latencies and Microsoft command prompt t o run simultaneous Ping. Use Microsoft Streets & Trips tool to plot in maps the dr iving directions over the targeted towers to perform the inter handoff. Use Solar Winds, Enhanced Ping to capture multiple pings. Measurement: Time (Millisecond)

Collect the Required Data: Capture the Microsoft Street & Trips map snapshot of d riving directions to perform inter handoff test. Using XCAL tool capture the Thr oughput behavior during handoff and correlate the results vs. CINR and RSSI. Cap ture device ping to HA (Home Agent) IP Address and external URLs simultaneously w hile performing the inter Base Station handover Procedure: Same procedure as Test Case 4 except for selected intra Base Station handoff route. 1. Locate the selected route. At the Start point of route, park your vehic le. 2. Launch XCAL. Select tab to provide CINR, RSSI, DownLink and UpLink chart in example below. Start logging and save filename Inter BS_Market Name_Route. 3. Click on the WiMAX icon on the SSV window to initiate network connection . 4. Verify web browser, establish HTTP connection to www.google.com. 5. Run Windows Command prompt: C: > ping t www.google.com > InterHO_Ping.txt . 6. Skip Step 5, if you have SolarWinds, Enhanced Ping, performs simultaneous Ping to multiple URLs. 7. Run multiple FTP DL and UL. 8. Start drive through selected route. 9. Stop logging at the end of the route. 10. Stop FTP DL and UL scripts. 11. Repeat for multiple ping (use Test Case 3 procedure) and drive through s elected route.

Example of Microsoft Streets and Trips For test case 5.

Test Case 6 using USB Device Test Case: 6 using USB Title: Device Data Rate Down Link Iperf TCP ce: SSV Devi

Description: For Production testing only in Washington DC market dependent on Te leworX Iperf server availability. Measure the network speed peak rate Down Link (DL) TCP. Test output is captured for reporting. Iteration: This test is performed for varying window sizes and repeated for eac h of Poor, Fair, Good RF conditions. Tool: TeleworX Iperf Server Measurement: Collect the Required Data: Procedure:

1. Launch the Windows command prompt a. Select start, run from the windows main toolbar b. Enter cmd in the dialog box and click ok 2. Click-on the WiMAX icon on the SSV window to initiate network connectivi ty a. Select 4G and click connect b. Wait for connection to be established 3. Verify the connection has been established using the Kannon application log a. Service Flow Added b. Network Entry Complete 4. Using the web browser, establish HTTP connection to a www.google.com a. Network connectivity verification purposes 5. Start iperf as server on personal laptop via the Windows command prompt (in the desktop directory) a. Enter: iperf s f k p 15000 i10 t60 w64k P4 6. Launch Windows command prompt window: a. Use the following command to start laptop as client b. Enter: iperf c <iperf IP address> -f k p 15000 -i10 -t60 -w64k P4 7. Read bandwidth from Windows command window 8. Repeat steps 6 to 8 for different window size a. Window Size = 64 KB b. Window Size = 256 KB c. Window Size = 375 KB

Test Case 7 using USB Device Test Case: 7 using USB Title: Device Data Rate Up Link Iperf TCP ce: SSV Devi

Description: For Production testing only in Washington DC market dependent on Te leworX Iperf server availability. Measure the network speed peak rate Up Link (U L) TCP. Test output is captured for reporting. Iteration: This test is performed for varying window sizes and repeated for eac h of Poor, Fair, Good RF conditions. Tool: TeleworX Iperf Server Measurement: Collect the Required Data: Procedure: 1. Launch the Windows command prompt a. Select start, run from the windows main toolbar b. Enter cmd in the dialog box and click ok 2. Click-on the WiMAX icon on the SSV window to initiate network connectivi ty a. Select 4G and click connect b. Wait for connection to be established 3. Verify the connection has been established using the Kannon application log a. Service Flow Added b. Network Entry Complete 4. Using the web browser, establish HTTP connection to a www.google.com a. Network connectivity verification purposes 5. At Windows command prompt, enter ipconfig /all to get laptop IP address 6. Start laptop as server a. Open windows command prompt window b. Use the following command to start iperf as server c. Enter: iperf s f k p 15000 i10 t60 w64k P4

7. At Windows command prompt, enter ipconfig /all to get laptop IP address 8. Start iperf as client on personal laptop via the Windows command prompt (in the desktop directory) a. Open Putty and connect to iperf server b. Start iperf as client c. Enter: Enter: iperf c <laptop IP address> -f k p 15000 -i10 -t60 -w64k P4 9. Read bandwidth from Windows command window 10. Repeat steps 6 to 8 for different window size a. Window Size = 64 KB b. Window Size = 256 KB c. Window Size = 375 KB Test Case 8 using USB Device Test Case: 8 using USB Title: Idle Mode Entry (USB Type Device) ce: SSV Devi

Description: Device is goes into IDLE Mode less than 6 minutes if no activity or background tasks are performed. This feature is verified at each new network r elease and confirmation of this will help preserve battery life or energy used f or devices in general. Test output is captured for reporting. Iteration: Perform this test procedure one completion per device. Tool: Kannon or XCal Tool Measurement: Time (Seconds) Collect the Required Data: Collect the IDLE Mode Entry using devices and USB dev ices. Collect and save in text the ping results when the device is at IDLE Mode State. Collect and save in text the ping results when the device is at Wake Up State. Procedure: Must terminate all processes that try to access the internet and ente rprise network 1. Using USB devices, click-on the WiMAX icon on the SSV window to initiate network connectivity a. Wait for connection to be established 2. Verify the connection has been established by opening a web browser and start a HTTP session. Then close the web browser. 3. Launch the windows task manager a. Hold the Ctrl-Alt-Delete buttons simultaneously b. Select the Task Manager to bring up the Windows Task Manager c. Look for processes that may try to access the network i. Example: Google desktop, windows update manager, virus updater, etc. ii. Do NOT remove any .exe files d. Highlight the process and click the end process button to terminate 4. Launch Kannon. 5. Keep the network connection up and wait for 3 to 6 minutes (from SSV Tim er) a. Do not attempt to open a web browser or any application b. Kannon application log should show Link Notification in IDLE MODE after tw o minutes c. Take a picture by pressing fn+prt sc at the same time. d. Save this to a .jpg file. 6. For Smartphones such as EVO, collect the XCal data and observed when the device changes their state from IDLE Mode to Wake-up State and vice versa. 7. Perform the ping from the Windows Command Prompt when the device at IDLE Mode State: a. Enter: ping n 10 l 64 <HA IP Address > IdleModeState_ping.txt 8. Perform the ping from the Windows Command Prompt when the device at Wake Up State: a. Enter: ping n 10 l 64 <HA IP Address > WakeUpState_ping.txt Test Case 9 using Smartphone

Test Case: 9 using Smartphone Device:

Title: Idle Mode Entry (Smartphone Device)

Description: Device is goes into IDLE Mode less than 10 seconds if no activity o r background tasks are performed. This feature is verified at each new network release and confirmation of this will help preserve battery life or energy used for devices in general. Test output is captured for reporting. Iteration: Perform this test procedure one completion per device. Tool: Kannon or XCal Tool Measurement: Time (Seconds) Collect the Required Data: Collect the IDLE Mode Entry using Smartphone devices and USB devices. Collect and save in text the ping results when the device is at IDLE Mode State. Collect and save in text the ping results when the device is at Wake Up State. Procedure: Must terminate all processes that try to access the internet and ente rprise network 1. From the Smartphone device, click the Market icon then search for Ping & DN S software and then install to smartphone device. 2. After installation is completed, open the Ping & DNS application and choos e the ping action and enter any URL address. (e.g. www.yahoo.com) then hit the Go. 3. e. 4. splay, 5. 6. ory as Step 2: allows the mobile to change from idle mode state to wake up stag Connect the USB cable from the Smartphone to the PC and on Smartphone di choose the HTC Sync. Then hit Done From PC, open the command window: (Start -> Run) then OPEN: type: cmd For HTC EVO, make sure you have the file listed below on your c:> direct shown below:

C:\htc> dir Directory of C:\htc 2,032,485 adb.exe 106,496 AdbWinApi.dll 45,674 AdbWinApi.zip 309,788 apph 217 apph.xml 633,508 tcpdump 7. Change the directory as shown below:

c: > cd htc C:\htc

Test Case 9 using Smartphone (page 2) Procedure continued: Enter the following commands below that are in bold letters: C:\htc> adb push apph /data/local 1138 KB/s (0 bytes in 309788.000s) C:\htc>adb push apph.xml /data/local 13 KB/s (0 bytes in 217.000s) C:\htc> adb shell

$ cd /data/local cd /data/local $ chmod 777 apph chmod 777 apph $ apph apph 15:15 5.698 >| 15:15 5.700 >| -> 15:15 5.704 15:15 5.704 >| Connect !showss !showphystats 8. If using an XCal tool, open the Windows command prompt then enter the fo llowing to allow the EVO to forward all TCP socket port:7771 into PC and run XCa l from the PC to collect XCal log. C:\> cd htc C:\htc> adb forward tcp:7771 tcp:7771 9. Open the XCal software and set the Port Setting Chipset = Sequans, Phone model = SQN FW 4.6.2 10. For Smartphones such as EVO, collect the XCal data and observed when the device change from their state from IDLE Mode to Wake-up State and vice versa. 11. When the Smartphone (EVO) shows in idle mode from XCal or from the adb c ommand above; ping the Smartphone device from other PC that are connected on sam e Base Station. 12. When the smartphone shows in idle mode from XCal or from the command abo ve, run the ping from the Smartphone device by using the Ping & DNS application wh ich shown on Step 2. 13. Ping the Smartphone when its on Wakeup state from another PC that are on same Base Station. Test Case 10 using Hotspot Test Case: 10 using Hotspot Title: Network Entry Device: MiFi

APPH/SPY APPH/SPY >| DCX/SPY DCX/SPY

info >| info >| info info >|

Initialized... Started... >| Started... Try to connect to the DCS...

Description: Device will connect to the 4G WiMAX network in less than 30 seconds from powering on device to 4G led, blue light, is flashing or solid. Iteration: Perform this test procedure 1 time at each of RF condition: Good, F air, Poor Tool: Use Microsoft clock under Date and Time Properties Measurement: Time (Second) Collect the Required Data: Collect in Excel format under different RF condition s (Good, Fair, Poor) every time the device led shows blue flashing/solid light. Procedure: 1. Launch Windows clock utility a. Double-click on the window toolbar clock b. Ensure the second counter is visible 2. Make sure MiFi is off and no USB cable is attached. 3. Power on by press down and hold the power button a few seconds, start cl ock counter. Release button and wait for correct sequence of led light: amber, green, and then blue. Device is connected to WiMax when led changes to blue. 4. Note the time, in seconds, between pressing power button and the time th e led changes to blue. 5. Verify launching WiFi on your laptop and connect to network provided by

your MiFi. Launch the internet explorer or any web browser and establish an htt p session.

Test Case 11 using Hotspot Test Case: 11 using Hotspot MiFi Title: Idle Mode Entry (Hotspot device) Device:

Description: Device is goes into IDLE Mode less than 6 minutes if no activity or background tasks are performed. This feature is verified at each new network r elease and confirmation of this will help preserve battery life or energy used f or devices in general. Test output is captured for reporting. Iteration: Perform this test procedure one completion per device. Tool: XCal Tool Measurement: MAC Message DREG and Time (ms) Collect the Required Data: Verify DREG from MAC Layer Message to confirm Device en tered the IDLE Mode/State. Collect and save in text the ping results when the device is at IDLE Mode State. Collect and save in text the ping results when the device is at Wake Up State. Procedure: Must terminate all processes that try to access the internet and ente rprise network 1. Power on MiFi, establish WiFi connection from laptop to MiFi device prov ided network. 2. Verify the connection has been established by opening a web browser and start a HTTP session. Then close the web browser 3. Open a web browser and type in http://192.168.1.1/ as URL a. Login as Administrator b. Click on WWAN Tab and status c. Write down the IP Address for use on Step 3. d. Close web browser. 4. Launch XCAL a. Ensure port setting is using Ethernet interface and IP Address obtained f rom previous step. b. Open MAC Layer Message Window c. Click on Filter from MAC Layer Message Window, uncheck all options except D REG, and click OK. The Message is split in half and the bottom window is blank at this time. 5. Launch the windows task manager a. Hold the Ctrl-Alt-Delete buttons simultaneously b. Select the Task Manager to bring up the Windows Task Manager c. Look for processes that may try to access the network i. Example: Google desktop, windows update manager, virus updater, etc. ii. Do NOT remove any .exe files d. Highlight the process and click the end process button to terminate. Test Case 11 using Hotspot (page 2) Procedure continued: 6. Keep the network connection up and wait for 3 to 6 minutes

a. Do not attempt to open a web browser or any application b. Make XCAL window is visible and you will see DREG message(s) appear on the bottom window when Idle Mode/State is reached c. Note: XCAL MAC Message State Box will still show NORMAL STATE, this is ra ther different from other Devices. d. Take a picture by pressing fn+prt sc at the same time. e. Save this to a .jpg file. 7. Perform the ping from the Windows Command Prompt when the device at IDLE Mode State: a. Enter: ping n 10 l 64 <HA IP Address> > IdleModeState_ping.txt 8. Perform the ping from the Windows Command Prompt when the device at Wake Up State: b. Enter: ping n 10 l 64 <HA IP Address > WakeUpState_ping.txt Test Case 12 TCP DownLink Throughput using Hotspot Test Case: 12 using Hotspot MiFi Title: TCP Throughput - DownLink Device:

Description: Verify the MiFi (untethered) Down Link TCP Throughput at various RF signal levels: Poor, Fair, and Good. Iteration: Perform this test procedure once per RF Condition. Tool: Use choice of tools (Utopia, ATS, BWMonitor, or XCAL) to capture the Maxim um Average TCP DL Throughput. Measurement: Speed (Mbps) Collect the Required Data: Collect the Maximum Average TCP DL Throughput over a five minutes period against RF condition (CINR and RSSI). Procedure: 1. Locate the selected Site for RF Condition, park your vehicle. 2. Power on MiFi, establish WiFi connection from laptop to MiFi provided ne twork. 3. Verify connection using web browser, establish HTTP connection to www.go ogle.com. 4. Run multiple FTP DL. 5. Launch BWMonitor a. Move mouse to top of graph where show an area color grey b. Right click mouse to involk menu c. Choose Rate Recorder d. Choose to monitor Interface WiMax Beceem e. Give a filename and save file 6. Continue to record for 5 minutes. 7. Stop logging by repeating Step 5 a d, and uncheck box next to Rate Recor der. 8. Stop FTP DL scripts. 9. Repeat for different RF Conditions.

Test Case 13 TCP Throughput UpLink using Hotspot Test Case: 13 using Hotspot Title: TCP Throughput - UpLink Device: MiFi

Description: Verify the MiFi (untethered) UpLink TCP Throughput at various RF si

gnal levels: Poor, Fair, and Good. Iteration: Perform this test procedure once per RF Condition. Tool: Use choice of tools (Utopia, ATS, BWMonitor, or XCAL) to capture the Maxim um Average TCP UL Throughput. Measurement: Speed (Mbps) Collect the Required Data: Collect the Maximum Average TCP UL Throughput over a five minutes period against RF condition (CINR and RSSI). Procedure: 1. Locate the selected Site for RF Condition, park your vehicle. 2. Power on MiFi, establish WiFi connection from laptop to MiFi provided ne twork. 3. Verify connection using web browser, establish HTTP connection to www.go ogle.com. 4. Run multiple FTP UL. 5. Launch BWMonitor a. Move mouse to top of graph where show an area color grey b. Right click mouse to involk menu c. Choose Rate Recorder d. Choose to monitor Interface WiMax Beceem e. Give a filename and save file 6. Continue to record for 5 minutes. 7. Stop logging by repeating Step 5 a d, and uncheck box next to Rate Recor der. 8. Stop FTP UL scripts. 9. Repeat for different RF Conditions. 10. Stop FTP UL scripts. 11. Repeat for multiple ping (use Test Case 3 procedure) and drive through s elected route.